homelab-configs/litellm/litellm-setup-guide.md

# LiteLLM Centralized API Gateway Setup

## ✅ What's Done

Your **LiteLLM API gateway** is now running and ready to use:

- **LiteLLM Service**: Running in Docker on `siklos` (192.168.88.27:4000)
- **Configuration**: Stored locally at `/opt/litellm/config/`
- **Git Repository**: Ready to push to Gitea (already initialized)

## 📋 Current Configuration

LiteLLM is configured to proxy these models:

```yaml
Models Available:
  ✓ gpt-4 (OpenAI)
  ✓ gpt-3.5-turbo (OpenAI)
  ✓ claude-3-sonnet (Anthropic)
  
Config Location: /opt/litellm/config/litellm_config.yaml
API Endpoint: http://192.168.88.27:4000
Master Key: litellm-local-key-change-in-production (⚠️ CHANGE THIS in production)
```

---

## 🔑 Next Steps (Required)

### Step 1: Create Gitea Repository (30 seconds)

**Why?** This centralizes your API key management in version control. Updates to one place affect all VMs.

1. Open: **http://192.168.88.27:3002** (Your Gitea instance)
2. Login with your account (jgitta)
3. Click **"+"** icon (top right) → **"New Repository"**
4. Fill in:
   - **Repository name**: `litellm-config`
   - **Description**: "Centralized LiteLLM API configuration"
   - Leave other settings as defaults
5. Click **"Create Repository"**

### Step 2: Push Config to Gitea

Once the repo is created, run:

```bash
ssh jgitta@192.168.88.27 "cd /opt/litellm && bash .git-setup.sh"
```

This will push your local config to Gitea and set up the git remote.

### Step 3: Add Your API Keys

1. Edit the config file:
   ```bash
   ssh jgitta@192.168.88.27 "nano /opt/litellm/config/litellm_config.yaml"
   ```

2. Replace placeholder API keys:
   - Find: `${OPENAI_API_KEY}` → Add your OpenAI key
   - Find: `${ANTHROPIC_API_KEY}` → Add your Anthropic key

3. Save and push to Gitea:
   ```bash
   ssh jgitta@192.168.88.27 << 'EOF'
   cd /opt/litellm
   git add config/litellm_config.yaml
   git commit -m "Add API keys"
   git push origin master
   EOF
   ```

4. LiteLLM will automatically reload (check logs in ~10 seconds)

---

## 🔌 How VMs Use LiteLLM

Instead of calling OpenAI/Anthropic directly, VMs call LiteLLM:

### From Any VM - Example Usage

```bash
# Test the gateway is working
curl -X POST http://192.168.88.27:4000/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer litellm-local-key-change-in-production" \
  -d '{
    "model": "gpt-4",
    "messages": [
      {"role": "user", "content": "Hello, what is 2+2?"}
    ]
  }'
```

### In Python (from any VM)

```python
import requests

response = requests.post(
    "http://192.168.88.27:4000/chat/completions",
    headers={
        "Authorization": "Bearer litellm-local-key-change-in-production",
        "Content-Type": "application/json"
    },
    json={
        "model": "gpt-4",
        "messages": [{"role": "user", "content": "Hello!"}]
    }
)
print(response.json())
```

### In Node.js (from any VM)

```javascript
const response = await fetch('http://192.168.88.27:4000/chat/completions', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer litellm-local-key-change-in-production',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    model: 'gpt-4',
    messages: [{ role: 'user', content: 'Hello!' }]
  })
});
const data = await response.json();
console.log(data);
```

---

## 📂 File Locations

For Linux newbies - here's where everything is:

```
docker-server (siklos @ 192.168.88.27)
└── /opt/litellm/
    ├── docker-compose.yml          # Docker configuration
    ├── .git/                       # Git repository
    ├── .git-setup.sh               # Script to push to Gitea
    └── config/
        ├── litellm_config.yaml     # Model definitions & API keys
        └── README.md               # Config documentation

Gitea (gitea @ 192.168.88.27:3002)
└── jgitta/litellm-config/
    └── (same files as above)
```

---

## 🛠️ Common Tasks

### Add a New Model

Edit `/opt/litellm/config/litellm_config.yaml` and add:

```yaml
- model_name: my-new-model
  litellm_params:
    model: openai/gpt-3.5-turbo
    api_key: ${OPENAI_API_KEY}
```

Then commit and push to Gitea.

### Change the Master Key (Recommended for Production)

1. Edit: `/opt/litellm/config/litellm_config.yaml`
2. Find: `master_key: litellm-local-key-change-in-production`
3. Change to a secure key
4. Restart container: `docker restart litellm`

### View Logs

```bash
ssh jgitta@192.168.88.27 "docker logs litellm -f"
```

### Stop/Start LiteLLM

```bash
ssh jgitta@192.168.88.27 "cd /opt/litellm && docker-compose down"
ssh jgitta@192.168.88.27 "cd /opt/litellm && docker-compose up -d"
```

---

## 🔒 Security Notes

- **Master Key**: Currently set to a placeholder. Change this before using in production.
- **API Keys**: Store in environment variables (not in config file directly) via `.env` file or Docker secrets.
- **Network**: LiteLLM listens on `192.168.88.27:4000` - accessible from all VMs on your network. Restrict with firewall if needed.

---

## ❓ Troubleshooting

**"LiteLLM not responding"**
- Check if container is running: `docker ps | grep litellm`
- View logs: `docker logs litellm`
- Wait 30-60 seconds after startup

**"API Key not working"**
- Verify key is in `litellm_config.yaml`
- Check logs for error messages
- Ensure you're using correct model name

**"Config not updating"**
- After git push, wait 10-15 seconds for LiteLLM to reload
- Check logs: `docker logs litellm | tail -20`

---

## 📌 Summary

You now have:
✅ **Centralized API Gateway** - Single endpoint for all models  
✅ **Git-Versioned Config** - Changes tracked in Gitea  
✅ **Easy Updates** - Update config in one place, affects all VMs  
✅ **Simple API** - Drop-in replacement for direct OpenAI/Anthropic calls  

Next: Create the Gitea repo and push your config!