Add complete homelab documentation and configurations
Documentation: - OpenCode + LiteLLM integration guide - LiteLLM complete setup with 16 models - Gitea centralized configuration guide - Testing procedures and verification - API keys setup instructions Configurations: - OpenCode config pointing to LiteLLM - Updated LiteLLM config with all models - Nextcloud docker-compose template Scripts: - Gitea setup automation - OpenCode testing script Infrastructure: - Gitea: 192.168.88.200:3000 - LiteLLM: 192.168.88.27:4000 - Nextcloud: 192.168.88.62
This commit is contained in:
@@ -0,0 +1,228 @@
|
||||
# LiteLLM Centralized API Gateway Setup
|
||||
|
||||
## ✅ What's Done
|
||||
|
||||
Your **LiteLLM API gateway** is now running and ready to use:
|
||||
|
||||
- **LiteLLM Service**: Running in Docker on `siklos` (192.168.88.27:4000)
|
||||
- **Configuration**: Stored locally at `/opt/litellm/config/`
|
||||
- **Git Repository**: Ready to push to Gitea (already initialized)
|
||||
|
||||
## 📋 Current Configuration
|
||||
|
||||
LiteLLM is configured to proxy these models:
|
||||
|
||||
```yaml
|
||||
Models Available:
|
||||
✓ gpt-4 (OpenAI)
|
||||
✓ gpt-3.5-turbo (OpenAI)
|
||||
✓ claude-3-sonnet (Anthropic)
|
||||
|
||||
Config Location: /opt/litellm/config/litellm_config.yaml
|
||||
API Endpoint: http://192.168.88.27:4000
|
||||
Master Key: litellm-local-key-change-in-production (⚠️ CHANGE THIS in production)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔑 Next Steps (Required)
|
||||
|
||||
### Step 1: Create Gitea Repository (30 seconds)
|
||||
|
||||
**Why?** This centralizes your API key management in version control. Updates to one place affect all VMs.
|
||||
|
||||
1. Open: **http://192.168.88.200:3000** (Your Gitea instance)
|
||||
2. Login with your account (jgitta)
|
||||
3. Click **"+"** icon (top right) → **"New Repository"**
|
||||
4. Fill in:
|
||||
- **Repository name**: `litellm-config`
|
||||
- **Description**: "Centralized LiteLLM API configuration"
|
||||
- Leave other settings as defaults
|
||||
5. Click **"Create Repository"**
|
||||
|
||||
### Step 2: Push Config to Gitea
|
||||
|
||||
Once the repo is created, run:
|
||||
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27 "cd /opt/litellm && bash .git-setup.sh"
|
||||
```
|
||||
|
||||
This will push your local config to Gitea and set up the git remote.
|
||||
|
||||
### Step 3: Add Your API Keys
|
||||
|
||||
1. Edit the config file:
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27 "nano /opt/litellm/config/litellm_config.yaml"
|
||||
```
|
||||
|
||||
2. Replace placeholder API keys:
|
||||
- Find: `${OPENAI_API_KEY}` → Add your OpenAI key
|
||||
- Find: `${ANTHROPIC_API_KEY}` → Add your Anthropic key
|
||||
|
||||
3. Save and push to Gitea:
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27 << 'EOF'
|
||||
cd /opt/litellm
|
||||
git add config/litellm_config.yaml
|
||||
git commit -m "Add API keys"
|
||||
git push origin master
|
||||
EOF
|
||||
```
|
||||
|
||||
4. LiteLLM will automatically reload (check logs in ~10 seconds)
|
||||
|
||||
---
|
||||
|
||||
## 🔌 How VMs Use LiteLLM
|
||||
|
||||
Instead of calling OpenAI/Anthropic directly, VMs call LiteLLM:
|
||||
|
||||
### From Any VM - Example Usage
|
||||
|
||||
```bash
|
||||
# Test the gateway is working
|
||||
curl -X POST http://192.168.88.27:4000/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer litellm-local-key-change-in-production" \
|
||||
-d '{
|
||||
"model": "gpt-4",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Hello, what is 2+2?"}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
### In Python (from any VM)
|
||||
|
||||
```python
|
||||
import requests
|
||||
|
||||
response = requests.post(
|
||||
"http://192.168.88.27:4000/chat/completions",
|
||||
headers={
|
||||
"Authorization": "Bearer litellm-local-key-change-in-production",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
json={
|
||||
"model": "gpt-4",
|
||||
"messages": [{"role": "user", "content": "Hello!"}]
|
||||
}
|
||||
)
|
||||
print(response.json())
|
||||
```
|
||||
|
||||
### In Node.js (from any VM)
|
||||
|
||||
```javascript
|
||||
const response = await fetch('http://192.168.88.27:4000/chat/completions', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': 'Bearer litellm-local-key-change-in-production',
|
||||
'Content-Type': 'application/json'
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: 'gpt-4',
|
||||
messages: [{ role: 'user', content: 'Hello!' }]
|
||||
})
|
||||
});
|
||||
const data = await response.json();
|
||||
console.log(data);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📂 File Locations
|
||||
|
||||
For Linux newbies - here's where everything is:
|
||||
|
||||
```
|
||||
docker-server (siklos @ 192.168.88.27)
|
||||
└── /opt/litellm/
|
||||
├── docker-compose.yml # Docker configuration
|
||||
├── .git/ # Git repository
|
||||
├── .git-setup.sh # Script to push to Gitea
|
||||
└── config/
|
||||
├── litellm_config.yaml # Model definitions & API keys
|
||||
└── README.md # Config documentation
|
||||
|
||||
Gitea (gitea @ 192.168.88.200:3000)
|
||||
└── jgitta/litellm-config/
|
||||
└── (same files as above)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Common Tasks
|
||||
|
||||
### Add a New Model
|
||||
|
||||
Edit `/opt/litellm/config/litellm_config.yaml` and add:
|
||||
|
||||
```yaml
|
||||
- model_name: my-new-model
|
||||
litellm_params:
|
||||
model: openai/gpt-3.5-turbo
|
||||
api_key: ${OPENAI_API_KEY}
|
||||
```
|
||||
|
||||
Then commit and push to Gitea.
|
||||
|
||||
### Change the Master Key (Recommended for Production)
|
||||
|
||||
1. Edit: `/opt/litellm/config/litellm_config.yaml`
|
||||
2. Find: `master_key: litellm-local-key-change-in-production`
|
||||
3. Change to a secure key
|
||||
4. Restart container: `docker restart litellm`
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27 "docker logs litellm -f"
|
||||
```
|
||||
|
||||
### Stop/Start LiteLLM
|
||||
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27 "cd /opt/litellm && docker-compose down"
|
||||
ssh jgitta@192.168.88.27 "cd /opt/litellm && docker-compose up -d"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Security Notes
|
||||
|
||||
- **Master Key**: Currently set to a placeholder. Change this before using in production.
|
||||
- **API Keys**: Store in environment variables (not in config file directly) via `.env` file or Docker secrets.
|
||||
- **Network**: LiteLLM listens on `192.168.88.27:4000` - accessible from all VMs on your network. Restrict with firewall if needed.
|
||||
|
||||
---
|
||||
|
||||
## ❓ Troubleshooting
|
||||
|
||||
**"LiteLLM not responding"**
|
||||
- Check if container is running: `docker ps | grep litellm`
|
||||
- View logs: `docker logs litellm`
|
||||
- Wait 30-60 seconds after startup
|
||||
|
||||
**"API Key not working"**
|
||||
- Verify key is in `litellm_config.yaml`
|
||||
- Check logs for error messages
|
||||
- Ensure you're using correct model name
|
||||
|
||||
**"Config not updating"**
|
||||
- After git push, wait 10-15 seconds for LiteLLM to reload
|
||||
- Check logs: `docker logs litellm | tail -20`
|
||||
|
||||
---
|
||||
|
||||
## 📌 Summary
|
||||
|
||||
You now have:
|
||||
✅ **Centralized API Gateway** - Single endpoint for all models
|
||||
✅ **Git-Versioned Config** - Changes tracked in Gitea
|
||||
✅ **Easy Updates** - Update config in one place, affects all VMs
|
||||
✅ **Simple API** - Drop-in replacement for direct OpenAI/Anthropic calls
|
||||
|
||||
Next: Create the Gitea repo and push your config!
|
||||
Reference in New Issue
Block a user