Add complete homelab documentation and configurations
Documentation: - OpenCode + LiteLLM integration guide - LiteLLM complete setup with 16 models - Gitea centralized configuration guide - Testing procedures and verification - API keys setup instructions Configurations: - OpenCode config pointing to LiteLLM - Updated LiteLLM config with all models - Nextcloud docker-compose template Scripts: - Gitea setup automation - OpenCode testing script Infrastructure: - Gitea: 192.168.88.200:3000 - LiteLLM: 192.168.88.27:4000 - Nextcloud: 192.168.88.62
This commit is contained in:
@@ -0,0 +1,261 @@
|
||||
# LiteLLM API Keys Setup - Claude & Gemini Pro Added
|
||||
|
||||
## ✅ What's Been Updated
|
||||
|
||||
Your LiteLLM gateway now supports **3 major AI platforms**:
|
||||
|
||||
### OpenAI (3 models)
|
||||
- `gpt-4` - Most capable
|
||||
- `gpt-4-turbo` - Faster
|
||||
- `gpt-3.5-turbo` - Budget-friendly
|
||||
|
||||
### Anthropic Claude (4 models)
|
||||
- `claude-3-opus` - Most capable Claude
|
||||
- `claude-3-sonnet` - Best balance
|
||||
- `claude-3-haiku` - Lightweight
|
||||
- `claude-2.1` - Previous generation
|
||||
|
||||
### Google Gemini (3 models)
|
||||
- `gemini-pro` - Main model
|
||||
- `gemini-pro-vision` - With vision
|
||||
- `gemini-1.5-pro` - Latest
|
||||
|
||||
---
|
||||
|
||||
## 🔑 Getting Your API Keys
|
||||
|
||||
### 1. OpenAI API Key
|
||||
```
|
||||
1. Go to: https://platform.openai.com/api-keys
|
||||
2. Click "Create new secret key"
|
||||
3. Copy the key (starts with sk-...)
|
||||
```
|
||||
|
||||
### 2. Claude (Anthropic) API Key
|
||||
```
|
||||
1. Go to: https://console.anthropic.com/
|
||||
2. Navigate to "API Keys"
|
||||
3. Create new key
|
||||
4. Copy it (starts with sk-ant-...)
|
||||
```
|
||||
|
||||
### 3. Gemini (Google) API Key
|
||||
```
|
||||
1. Go to: https://ai.google.dev/
|
||||
2. Click "Get API Key"
|
||||
3. Create new key in Google Cloud
|
||||
4. Copy it
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 Add Keys to Your Gateway
|
||||
|
||||
SSH into docker-server and edit the `.env` file:
|
||||
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27
|
||||
cd /opt/litellm
|
||||
nano .env
|
||||
```
|
||||
|
||||
Replace the placeholders with your actual keys:
|
||||
|
||||
```bash
|
||||
# OpenAI API Keys
|
||||
OPENAI_API_KEY=sk-your-actual-openai-key-here
|
||||
|
||||
# Anthropic Claude API Key
|
||||
ANTHROPIC_API_KEY=sk-ant-your-actual-claude-key-here
|
||||
|
||||
# Google Gemini API Key
|
||||
GOOGLE_API_KEY=your-actual-google-gemini-api-key-here
|
||||
|
||||
# Master authentication key (change this in production)
|
||||
LITELLM_MASTER_KEY=litellm-local-key-change-in-production
|
||||
```
|
||||
|
||||
**To save and exit nano:**
|
||||
```
|
||||
Ctrl+X → Y → Enter
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Restart LiteLLM with New Keys
|
||||
|
||||
After updating `.env`:
|
||||
|
||||
```bash
|
||||
cd /opt/litellm
|
||||
docker-compose down
|
||||
docker-compose up -d
|
||||
sleep 10
|
||||
docker logs litellm | tail -20
|
||||
```
|
||||
|
||||
If you see models loading without errors, you're good!
|
||||
|
||||
---
|
||||
|
||||
## ✅ Test Each Provider
|
||||
|
||||
### Test OpenAI (GPT-4)
|
||||
```bash
|
||||
curl -X POST http://192.168.88.27:4000/chat/completions \
|
||||
-H "Authorization: Bearer litellm-local-key-change-in-production" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "gpt-4",
|
||||
"messages": [{"role": "user", "content": "Say hello OpenAI!"}]
|
||||
}' | jq '.choices[0].message'
|
||||
```
|
||||
|
||||
### Test Claude (Anthropic)
|
||||
```bash
|
||||
curl -X POST http://192.168.88.27:4000/chat/completions \
|
||||
-H "Authorization: Bearer litellm-local-key-change-in-production" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "claude-3-opus",
|
||||
"messages": [{"role": "user", "content": "Say hello Claude!"}]
|
||||
}' | jq '.choices[0].message'
|
||||
```
|
||||
|
||||
### Test Gemini (Google)
|
||||
```bash
|
||||
curl -X POST http://192.168.88.27:4000/chat/completions \
|
||||
-H "Authorization: Bearer litellm-local-key-change-in-production" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "gemini-pro",
|
||||
"messages": [{"role": "user", "content": "Say hello Gemini!"}]
|
||||
}' | jq '.choices[0].message'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 💻 Use from Your VMs
|
||||
|
||||
Now any VM (jellyfin, next, photos, haos, etc.) can use any model:
|
||||
|
||||
### Python Example - Switch Between Models
|
||||
```python
|
||||
import requests
|
||||
import json
|
||||
|
||||
ENDPOINT = "http://192.168.88.27:4000/chat/completions"
|
||||
AUTH = "Bearer litellm-local-key-change-in-production"
|
||||
|
||||
def ask_model(model, question):
|
||||
response = requests.post(
|
||||
ENDPOINT,
|
||||
headers={"Authorization": AUTH, "Content-Type": "application/json"},
|
||||
json={
|
||||
"model": model,
|
||||
"messages": [{"role": "user", "content": question}]
|
||||
}
|
||||
)
|
||||
return response.json()['choices'][0]['message']['content']
|
||||
|
||||
# Use any model:
|
||||
print("GPT-4:", ask_model("gpt-4", "What is AI?"))
|
||||
print("Claude:", ask_model("claude-3-opus", "What is AI?"))
|
||||
print("Gemini:", ask_model("gemini-pro", "What is AI?"))
|
||||
```
|
||||
|
||||
### Bash Example - List All Available Models
|
||||
```bash
|
||||
curl http://192.168.88.27:4000/models \
|
||||
-H "Authorization: Bearer litellm-local-key-change-in-production" | jq '.data[].id'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Configuration Files Updated
|
||||
|
||||
All files are in `/opt/litellm/` on docker-server:
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `.env` | Your API keys (never commit to git) |
|
||||
| `config/litellm_config.yaml` | Model definitions |
|
||||
| `config/README.md` | Usage documentation |
|
||||
| `docker-compose.yml` | Docker service config |
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Security Reminders
|
||||
|
||||
⚠️ **IMPORTANT:**
|
||||
- ✅ API keys are in `.env` (not in git) - Good practice!
|
||||
- ✅ Each provider's key is separate - Can rotate independently
|
||||
- ✅ Master key is configurable - Change from default in production
|
||||
- 🔄 Git is initialized - Ready for version control when needed
|
||||
|
||||
### Never Do This:
|
||||
```bash
|
||||
# ❌ DON'T commit .env to git
|
||||
git add .env
|
||||
git commit -m "my keys"
|
||||
|
||||
# ❌ DON'T put API keys in litellm_config.yaml
|
||||
# Use ${ENVIRONMENT_VAR} references instead
|
||||
|
||||
# ❌ DON'T expose port 4000 to the internet without HTTPS
|
||||
# Use Caddy (already on your network) as reverse proxy
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What's Ready
|
||||
|
||||
✅ **LiteLLM Gateway**: Running on docker-server (192.168.88.27:4000)
|
||||
✅ **Configuration**: All 3 providers configured
|
||||
✅ **API Key Placeholders**: Ready for your actual keys
|
||||
✅ **Documentation**: Complete usage examples
|
||||
✅ **Git**: Initialized and ready for tracking changes
|
||||
✅ **Docker**: Compose file handles everything
|
||||
|
||||
---
|
||||
|
||||
## 📞 Troubleshooting
|
||||
|
||||
**"Model not available"**
|
||||
```bash
|
||||
# Check what models are actually loaded:
|
||||
curl http://192.168.88.27:4000/models \
|
||||
-H "Authorization: Bearer litellm-local-key-change-in-production" | jq
|
||||
```
|
||||
|
||||
**"Invalid API key"**
|
||||
- Verify key format matches provider (sk-* for OpenAI, sk-ant-* for Claude, etc.)
|
||||
- Check `.env` file saved correctly
|
||||
- Restart: `docker-compose down && docker-compose up -d`
|
||||
|
||||
**"Connection refused"**
|
||||
- Ensure container is running: `docker ps | grep litellm`
|
||||
- Check logs: `docker logs litellm`
|
||||
- Verify port 4000 is accessible: `curl http://localhost:4000/models`
|
||||
|
||||
---
|
||||
|
||||
## 📚 Quick Links
|
||||
|
||||
- **OpenAI API**: https://platform.openai.com/api-keys
|
||||
- **Anthropic Console**: https://console.anthropic.com/
|
||||
- **Google AI Studio**: https://ai.google.dev/
|
||||
- **LiteLLM Docs**: https://docs.litellm.ai/
|
||||
- **Your Complete Guide**: See `litellm-complete-setup.md`
|
||||
|
||||
---
|
||||
|
||||
## ✨ Next Steps
|
||||
|
||||
1. **Get API keys** from the three providers
|
||||
2. **Add keys to `.env`** on docker-server
|
||||
3. **Restart LiteLLM**
|
||||
4. **Test each model** with the curl examples above
|
||||
5. **Update your VMs** to use the gateway endpoint
|
||||
|
||||
That's it! All three providers are now available through a single unified endpoint. 🚀
|
||||
@@ -0,0 +1,464 @@
|
||||
# LiteLLM Centralized API Gateway - Complete Setup Guide
|
||||
|
||||
## ✅ Infrastructure Status
|
||||
|
||||
**All components installed and running:**
|
||||
|
||||
- ✅ **LiteLLM Container**: Running on docker-server (siklos, 192.168.88.27:4000)
|
||||
- ✅ **Docker Compose**: Configured and ready at `/opt/litellm/`
|
||||
- ✅ **Configuration**: Git initialized with config files in `/opt/litellm/config/`
|
||||
- ✅ **Gitea**: Ready at http://192.168.88.200:3000 for repository management
|
||||
- ✅ **All VMs**: Network access confirmed to docker-server
|
||||
|
||||
---
|
||||
|
||||
## 📁 Current Setup Directory Structure
|
||||
|
||||
```
|
||||
docker-server (siklos @ 192.168.88.27)
|
||||
└── /opt/litellm/
|
||||
├── docker-compose.yml # Docker service configuration
|
||||
├── .env # Environment variables (API keys)
|
||||
├── .git/ # Git version control
|
||||
├── .git-setup.sh # Helper script for Gitea push
|
||||
├── logs/ # Service logs
|
||||
└── config/
|
||||
├── README.md # Configuration guide
|
||||
└── litellm_config.yaml # Model definitions
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Next Steps: Configure API Keys
|
||||
|
||||
### Step 1: Add Your Real API Keys
|
||||
|
||||
Edit the `.env` file with your actual API credentials:
|
||||
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27 "nano /opt/litellm/.env"
|
||||
```
|
||||
|
||||
Update these lines with your real keys:
|
||||
```bash
|
||||
OPENAI_API_KEY=sk-your-real-openai-key-here
|
||||
ANTHROPIC_API_KEY=sk-ant-your-real-anthropic-key-here
|
||||
```
|
||||
|
||||
**⚠️ Security Note:** Never commit API keys to git. The `.env` file should never be pushed to Gitea.
|
||||
|
||||
### Step 2: Restart LiteLLM with Real Keys
|
||||
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27 << 'EOF'
|
||||
cd /opt/litellm
|
||||
docker-compose down
|
||||
docker-compose up -d
|
||||
sleep 10
|
||||
docker logs litellm | tail -20
|
||||
EOF
|
||||
```
|
||||
|
||||
### Step 3: Test the Gateway
|
||||
|
||||
Once running with real keys, test from any VM:
|
||||
|
||||
```bash
|
||||
curl -X POST http://192.168.88.27:4000/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer litellm-local-key-change-in-production" \
|
||||
-d '{
|
||||
"model": "gpt-4",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Hello! What is 2+2?"}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎯 How It Works: Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Your VMs │
|
||||
│ (jellyfin, next, photos, haos, etc.) │
|
||||
│ ↓ │
|
||||
│ Single API Endpoint: │
|
||||
│ http://192.168.88.27:4000 │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ LiteLLM API Gateway (Docker Container) │
|
||||
│ Running on docker-server (siklos) │
|
||||
│ │
|
||||
│ Routes requests to appropriate backend: │
|
||||
│ - gpt-4 → OpenAI API │
|
||||
│ - claude-3 → Anthropic API │
|
||||
│ - local-* → Local models (if configured) │
|
||||
│ │
|
||||
│ Advantages: │
|
||||
│ ✓ Single authentication point │
|
||||
│ ✓ Centralized API key management │
|
||||
│ ✓ Easy to add/remove models │
|
||||
│ ✓ Request logging & monitoring │
|
||||
│ ✓ Load balancing & fallbacks │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌──────┴──────┐
|
||||
↓ ↓
|
||||
OpenAI API Anthropic API
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 Configuration Files
|
||||
|
||||
### Docker Compose (`docker-compose.yml`)
|
||||
|
||||
The service is configured to:
|
||||
- **Listen on**: `0.0.0.0:4000` (accessible from all VMs)
|
||||
- **Load env from**: `.env` file
|
||||
- **Mount volumes**:
|
||||
- `./config/` → `/app/config/` (read-only)
|
||||
- `./logs/` → `/app/logs/` (read-write)
|
||||
- **Auto-restart**: Unless stopped manually
|
||||
|
||||
### Environment File (`.env`)
|
||||
|
||||
Contains your API keys. Format:
|
||||
```
|
||||
OPENAI_API_KEY=sk-xxx...
|
||||
ANTHROPIC_API_KEY=sk-ant-xxx...
|
||||
LITELLM_MASTER_KEY=your-master-auth-key
|
||||
```
|
||||
|
||||
### Configuration YAML (`config/litellm_config.yaml`)
|
||||
|
||||
Defines which models are available and how to route them. Current models:
|
||||
|
||||
```yaml
|
||||
Models Configured:
|
||||
✓ gpt-4 (OpenAI)
|
||||
✓ gpt-3.5-turbo (OpenAI)
|
||||
✓ claude-3-sonnet (Anthropic)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔑 API Key Management Strategy
|
||||
|
||||
### Local Storage (Current Setup)
|
||||
- API keys stored in `.env` file on docker-server
|
||||
- Not committed to git (security best practice)
|
||||
- Restart required after changes
|
||||
|
||||
### Optional: Gitea-Based Config (Future)
|
||||
Once tested, you can:
|
||||
1. Create a secure Gitea repository for non-sensitive config
|
||||
2. Store API keys as Docker secrets or environment file separately
|
||||
3. Pull config updates automatically
|
||||
|
||||
```bash
|
||||
# Create litellm-config repo in Gitea, then:
|
||||
cd /opt/litellm
|
||||
git remote add gitea http://192.168.88.200:3000/jgitta/litellm-config.git
|
||||
git pull gitea master
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing & Monitoring
|
||||
|
||||
### Check Service Status
|
||||
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27 << 'EOF'
|
||||
cd /opt/litellm
|
||||
docker-compose ps
|
||||
docker logs litellm -f # Follow logs in real-time
|
||||
EOF
|
||||
```
|
||||
|
||||
### View Active Models
|
||||
|
||||
```bash
|
||||
curl http://192.168.88.27:4000/models \
|
||||
-H "Authorization: Bearer litellm-local-key-change-in-production"
|
||||
```
|
||||
|
||||
### Monitor Logs Real-Time
|
||||
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27 "docker logs litellm -f"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 💻 Usage Examples from Your VMs
|
||||
|
||||
### Python Example
|
||||
|
||||
```python
|
||||
import requests
|
||||
import json
|
||||
|
||||
LITELLM_URL = "http://192.168.88.27:4000"
|
||||
MASTER_KEY = "litellm-local-key-change-in-production"
|
||||
|
||||
response = requests.post(
|
||||
f"{LITELLM_URL}/chat/completions",
|
||||
headers={
|
||||
"Authorization": f"Bearer {MASTER_KEY}",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
json={
|
||||
"model": "gpt-4",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Explain this to me like I'm 5"}
|
||||
]
|
||||
}
|
||||
)
|
||||
|
||||
print(response.json())
|
||||
```
|
||||
|
||||
### Bash Example
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
LITELLM_URL="http://192.168.88.27:4000"
|
||||
MASTER_KEY="litellm-local-key-change-in-production"
|
||||
MODEL="gpt-4"
|
||||
|
||||
curl -s -X POST "$LITELLM_URL/chat/completions" \
|
||||
-H "Authorization: Bearer $MASTER_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "'$MODEL'",
|
||||
"messages": [
|
||||
{"role": "user", "content": "What is the capital of France?"}
|
||||
]
|
||||
}' | jq '.choices[0].message.content'
|
||||
```
|
||||
|
||||
### Node.js Example
|
||||
|
||||
```javascript
|
||||
const fetch = require('node-fetch');
|
||||
|
||||
const LITELLM_URL = 'http://192.168.88.27:4000';
|
||||
const MASTER_KEY = 'litellm-local-key-change-in-production';
|
||||
|
||||
async function askLiteLLM(question, model = 'gpt-4') {
|
||||
const response = await fetch(`${LITELLM_URL}/chat/completions`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${MASTER_KEY}`,
|
||||
'Content-Type': 'application/json'
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: model,
|
||||
messages: [
|
||||
{ role: 'user', content: question }
|
||||
]
|
||||
})
|
||||
});
|
||||
|
||||
const data = await response.json();
|
||||
return data.choices[0].message.content;
|
||||
}
|
||||
|
||||
// Usage:
|
||||
askLiteLLM('Hello, who are you?').then(response => console.log(response));
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Security Considerations
|
||||
|
||||
### Current (Development) Setup
|
||||
- Master key is a placeholder: `litellm-local-key-change-in-production`
|
||||
- API keys stored in plain `.env` file
|
||||
- Service accessible from all VMs on network
|
||||
|
||||
### Production Recommendations
|
||||
1. **Change Master Key**
|
||||
```bash
|
||||
# Edit .env and set a strong key
|
||||
LITELLM_MASTER_KEY=your-random-secure-32-char-key
|
||||
docker-compose restart
|
||||
```
|
||||
|
||||
2. **Use Docker Secrets** (for Swarm/Kubernetes)
|
||||
```bash
|
||||
echo "your-real-api-key" | docker secret create openai_key -
|
||||
```
|
||||
|
||||
3. **Restrict Network Access**
|
||||
- Use firewall rules to limit which VMs can access port 4000
|
||||
- Only allow specific source IPs if possible
|
||||
|
||||
4. **SSL/TLS Encryption**
|
||||
- Run LiteLLM behind Caddy (your existing reverse proxy)
|
||||
- Use HTTPS instead of HTTP
|
||||
|
||||
5. **API Key Rotation**
|
||||
- Periodically update API keys in `.env`
|
||||
- Restart container after changes
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Advanced Customization
|
||||
|
||||
### Add More Models
|
||||
|
||||
Edit `config/litellm_config.yaml`:
|
||||
|
||||
```yaml
|
||||
model_list:
|
||||
- model_name: my-custom-model
|
||||
litellm_params:
|
||||
model: openai/gpt-3.5-turbo
|
||||
api_key: ${OPENAI_API_KEY}
|
||||
```
|
||||
|
||||
Then restart:
|
||||
```bash
|
||||
cd /opt/litellm
|
||||
docker-compose restart
|
||||
```
|
||||
|
||||
### Enable Request Logging
|
||||
|
||||
Edit `docker-compose.yml` and add:
|
||||
```yaml
|
||||
environment:
|
||||
LOG_LEVEL: DEBUG
|
||||
LITELLM_LOG_REQUESTS: "true"
|
||||
```
|
||||
|
||||
### Set Up Load Balancing
|
||||
|
||||
Multiple backend API keys for the same model:
|
||||
```yaml
|
||||
model_list:
|
||||
- model_name: gpt-4
|
||||
litellm_params:
|
||||
model: gpt-4-backup # Primary
|
||||
- model_name: gpt-4
|
||||
litellm_params:
|
||||
model: gpt-4-fallback # Fallback
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📞 Troubleshooting
|
||||
|
||||
### Container Won't Start
|
||||
|
||||
```bash
|
||||
# Check logs
|
||||
docker logs litellm 2>&1 | tail -50
|
||||
|
||||
# Most common: Missing API keys
|
||||
# Solution: Update .env with real keys
|
||||
|
||||
# Restart with fresh pull
|
||||
docker-compose down
|
||||
docker image pull ghcr.io/berriai/litellm:main
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
### "Connection Refused" from VM
|
||||
|
||||
```bash
|
||||
# Check container is running
|
||||
docker ps | grep litellm
|
||||
|
||||
# Check port is listening
|
||||
docker exec litellm netstat -tlnp | grep 4000
|
||||
|
||||
# Try from docker-server first
|
||||
docker exec litellm curl http://localhost:4000/health
|
||||
```
|
||||
|
||||
### "Unauthorized" Error
|
||||
|
||||
```bash
|
||||
# Verify master key in request matches .env
|
||||
# Default: litellm-local-key-change-in-production
|
||||
|
||||
# Test with correct key:
|
||||
curl -H "Authorization: Bearer litellm-local-key-change-in-production" \
|
||||
http://192.168.88.27:4000/models
|
||||
```
|
||||
|
||||
### Logs Show "Missing Credentials"
|
||||
|
||||
```bash
|
||||
# Edit .env with real API keys
|
||||
nano /opt/litellm/.env
|
||||
|
||||
# Restart the service
|
||||
docker-compose down && docker-compose up -d
|
||||
sleep 10
|
||||
docker logs litellm | tail -20
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📚 References & Documentation
|
||||
|
||||
- **LiteLLM Docs**: https://docs.litellm.ai/
|
||||
- **LiteLLM Proxy Server**: https://docs.litellm.ai/docs/proxy_server
|
||||
- **OpenAI API**: https://platform.openai.com/docs/
|
||||
- **Anthropic Claude**: https://docs.anthropic.com/
|
||||
|
||||
---
|
||||
|
||||
## 🎓 Quick Commands Reference
|
||||
|
||||
```bash
|
||||
# SSH to docker-server
|
||||
ssh jgitta@192.168.88.27
|
||||
|
||||
# Navigate to litellm directory
|
||||
cd /opt/litellm
|
||||
|
||||
# View status
|
||||
docker-compose ps
|
||||
docker logs litellm -f
|
||||
|
||||
# Restart service
|
||||
docker-compose restart
|
||||
|
||||
# Edit API keys
|
||||
nano .env
|
||||
docker-compose down && docker-compose up -d
|
||||
|
||||
# View config
|
||||
cat config/litellm_config.yaml
|
||||
|
||||
# Test from VM
|
||||
curl -X POST http://192.168.88.27:4000/chat/completions \
|
||||
-H "Authorization: Bearer litellm-local-key-change-in-production" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model":"gpt-4","messages":[{"role":"user","content":"test"}]}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✨ What's Next
|
||||
|
||||
1. **Add Real API Keys** to `.env`
|
||||
2. **Test the Gateway** from a VM
|
||||
3. **Monitor Logs** to ensure everything works
|
||||
4. **Document Your Models** in `config/README.md`
|
||||
5. **(Optional) Set up Gitea** for config version control
|
||||
6. **(Optional) Add Caddy** in front for HTTPS/SSL
|
||||
|
||||
---
|
||||
|
||||
**Your centralized API gateway is ready to go! All VMs now have a single endpoint to access multiple AI models. 🚀**
|
||||
@@ -0,0 +1,228 @@
|
||||
# LiteLLM Centralized API Gateway Setup
|
||||
|
||||
## ✅ What's Done
|
||||
|
||||
Your **LiteLLM API gateway** is now running and ready to use:
|
||||
|
||||
- **LiteLLM Service**: Running in Docker on `siklos` (192.168.88.27:4000)
|
||||
- **Configuration**: Stored locally at `/opt/litellm/config/`
|
||||
- **Git Repository**: Ready to push to Gitea (already initialized)
|
||||
|
||||
## 📋 Current Configuration
|
||||
|
||||
LiteLLM is configured to proxy these models:
|
||||
|
||||
```yaml
|
||||
Models Available:
|
||||
✓ gpt-4 (OpenAI)
|
||||
✓ gpt-3.5-turbo (OpenAI)
|
||||
✓ claude-3-sonnet (Anthropic)
|
||||
|
||||
Config Location: /opt/litellm/config/litellm_config.yaml
|
||||
API Endpoint: http://192.168.88.27:4000
|
||||
Master Key: litellm-local-key-change-in-production (⚠️ CHANGE THIS in production)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔑 Next Steps (Required)
|
||||
|
||||
### Step 1: Create Gitea Repository (30 seconds)
|
||||
|
||||
**Why?** This centralizes your API key management in version control. Updates to one place affect all VMs.
|
||||
|
||||
1. Open: **http://192.168.88.200:3000** (Your Gitea instance)
|
||||
2. Login with your account (jgitta)
|
||||
3. Click **"+"** icon (top right) → **"New Repository"**
|
||||
4. Fill in:
|
||||
- **Repository name**: `litellm-config`
|
||||
- **Description**: "Centralized LiteLLM API configuration"
|
||||
- Leave other settings as defaults
|
||||
5. Click **"Create Repository"**
|
||||
|
||||
### Step 2: Push Config to Gitea
|
||||
|
||||
Once the repo is created, run:
|
||||
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27 "cd /opt/litellm && bash .git-setup.sh"
|
||||
```
|
||||
|
||||
This will push your local config to Gitea and set up the git remote.
|
||||
|
||||
### Step 3: Add Your API Keys
|
||||
|
||||
1. Edit the config file:
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27 "nano /opt/litellm/config/litellm_config.yaml"
|
||||
```
|
||||
|
||||
2. Replace placeholder API keys:
|
||||
- Find: `${OPENAI_API_KEY}` → Add your OpenAI key
|
||||
- Find: `${ANTHROPIC_API_KEY}` → Add your Anthropic key
|
||||
|
||||
3. Save and push to Gitea:
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27 << 'EOF'
|
||||
cd /opt/litellm
|
||||
git add config/litellm_config.yaml
|
||||
git commit -m "Add API keys"
|
||||
git push origin master
|
||||
EOF
|
||||
```
|
||||
|
||||
4. LiteLLM will automatically reload (check logs in ~10 seconds)
|
||||
|
||||
---
|
||||
|
||||
## 🔌 How VMs Use LiteLLM
|
||||
|
||||
Instead of calling OpenAI/Anthropic directly, VMs call LiteLLM:
|
||||
|
||||
### From Any VM - Example Usage
|
||||
|
||||
```bash
|
||||
# Test the gateway is working
|
||||
curl -X POST http://192.168.88.27:4000/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer litellm-local-key-change-in-production" \
|
||||
-d '{
|
||||
"model": "gpt-4",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Hello, what is 2+2?"}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
### In Python (from any VM)
|
||||
|
||||
```python
|
||||
import requests
|
||||
|
||||
response = requests.post(
|
||||
"http://192.168.88.27:4000/chat/completions",
|
||||
headers={
|
||||
"Authorization": "Bearer litellm-local-key-change-in-production",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
json={
|
||||
"model": "gpt-4",
|
||||
"messages": [{"role": "user", "content": "Hello!"}]
|
||||
}
|
||||
)
|
||||
print(response.json())
|
||||
```
|
||||
|
||||
### In Node.js (from any VM)
|
||||
|
||||
```javascript
|
||||
const response = await fetch('http://192.168.88.27:4000/chat/completions', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': 'Bearer litellm-local-key-change-in-production',
|
||||
'Content-Type': 'application/json'
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: 'gpt-4',
|
||||
messages: [{ role: 'user', content: 'Hello!' }]
|
||||
})
|
||||
});
|
||||
const data = await response.json();
|
||||
console.log(data);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📂 File Locations
|
||||
|
||||
For Linux newbies - here's where everything is:
|
||||
|
||||
```
|
||||
docker-server (siklos @ 192.168.88.27)
|
||||
└── /opt/litellm/
|
||||
├── docker-compose.yml # Docker configuration
|
||||
├── .git/ # Git repository
|
||||
├── .git-setup.sh # Script to push to Gitea
|
||||
└── config/
|
||||
├── litellm_config.yaml # Model definitions & API keys
|
||||
└── README.md # Config documentation
|
||||
|
||||
Gitea (gitea @ 192.168.88.200:3000)
|
||||
└── jgitta/litellm-config/
|
||||
└── (same files as above)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Common Tasks
|
||||
|
||||
### Add a New Model
|
||||
|
||||
Edit `/opt/litellm/config/litellm_config.yaml` and add:
|
||||
|
||||
```yaml
|
||||
- model_name: my-new-model
|
||||
litellm_params:
|
||||
model: openai/gpt-3.5-turbo
|
||||
api_key: ${OPENAI_API_KEY}
|
||||
```
|
||||
|
||||
Then commit and push to Gitea.
|
||||
|
||||
### Change the Master Key (Recommended for Production)
|
||||
|
||||
1. Edit: `/opt/litellm/config/litellm_config.yaml`
|
||||
2. Find: `master_key: litellm-local-key-change-in-production`
|
||||
3. Change to a secure key
|
||||
4. Restart container: `docker restart litellm`
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27 "docker logs litellm -f"
|
||||
```
|
||||
|
||||
### Stop/Start LiteLLM
|
||||
|
||||
```bash
|
||||
ssh jgitta@192.168.88.27 "cd /opt/litellm && docker-compose down"
|
||||
ssh jgitta@192.168.88.27 "cd /opt/litellm && docker-compose up -d"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Security Notes
|
||||
|
||||
- **Master Key**: Currently set to a placeholder. Change this before using in production.
|
||||
- **API Keys**: Store in environment variables (not in config file directly) via `.env` file or Docker secrets.
|
||||
- **Network**: LiteLLM listens on `192.168.88.27:4000` - accessible from all VMs on your network. Restrict with firewall if needed.
|
||||
|
||||
---
|
||||
|
||||
## ❓ Troubleshooting
|
||||
|
||||
**"LiteLLM not responding"**
|
||||
- Check if container is running: `docker ps | grep litellm`
|
||||
- View logs: `docker logs litellm`
|
||||
- Wait 30-60 seconds after startup
|
||||
|
||||
**"API Key not working"**
|
||||
- Verify key is in `litellm_config.yaml`
|
||||
- Check logs for error messages
|
||||
- Ensure you're using correct model name
|
||||
|
||||
**"Config not updating"**
|
||||
- After git push, wait 10-15 seconds for LiteLLM to reload
|
||||
- Check logs: `docker logs litellm | tail -20`
|
||||
|
||||
---
|
||||
|
||||
## 📌 Summary
|
||||
|
||||
You now have:
|
||||
✅ **Centralized API Gateway** - Single endpoint for all models
|
||||
✅ **Git-Versioned Config** - Changes tracked in Gitea
|
||||
✅ **Easy Updates** - Update config in one place, affects all VMs
|
||||
✅ **Simple API** - Drop-in replacement for direct OpenAI/Anthropic calls
|
||||
|
||||
Next: Create the Gitea repo and push your config!
|
||||
@@ -0,0 +1,87 @@
|
||||
# Nextcloud VM (192.168.88.62) - Updated docker-compose.yml
|
||||
#
|
||||
# This configuration pulls OpenCode config from centralized Gitea repository
|
||||
# instead of managing per-VM configuration files
|
||||
#
|
||||
# Updated for centralized config management via http://192.168.88.200:3000
|
||||
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
# Keep existing Nextcloud service if present
|
||||
# nextcloud:
|
||||
# image: nextcloud:latest
|
||||
# ... (existing config)
|
||||
|
||||
opencode:
|
||||
image: opencode:latest
|
||||
container_name: opencode
|
||||
|
||||
# Volumes for persistent configuration and workspace
|
||||
volumes:
|
||||
- opencode-config:/root/.opencode
|
||||
- opencode-workspace:/workspace
|
||||
|
||||
# Expose port (adjust if needed)
|
||||
ports:
|
||||
- "8000:8000"
|
||||
|
||||
# Environment variables
|
||||
environment:
|
||||
HOME: /root
|
||||
# LiteLLM Gateway endpoint is in the config.json file
|
||||
# pulled from Gitea
|
||||
|
||||
# The key: pull config from Gitea at startup
|
||||
entrypoint: /bin/sh
|
||||
command: |
|
||||
-c "
|
||||
echo '🔄 Fetching configuration from Gitea...'
|
||||
mkdir -p /tmp/homelab-configs
|
||||
|
||||
# Clone the central configs repo
|
||||
if git clone http://192.168.88.200:3000/jgitta/homelab-configs.git /tmp/homelab-configs; then
|
||||
echo '✅ Configuration fetched successfully'
|
||||
else
|
||||
echo '⚠️ Could not fetch configs from Gitea, using defaults'
|
||||
fi
|
||||
|
||||
# Set up OpenCode config directory
|
||||
mkdir -p /root/.opencode
|
||||
|
||||
# Copy OpenCode config if available
|
||||
if [ -f /tmp/homelab-configs/opencode/config.json ]; then
|
||||
cp /tmp/homelab-configs/opencode/config.json /root/.opencode/config.json
|
||||
echo '✅ OpenCode config loaded from Gitea'
|
||||
else
|
||||
echo '⚠️ Using default OpenCode configuration'
|
||||
fi
|
||||
|
||||
# Start OpenCode
|
||||
echo '🚀 Starting OpenCode...'
|
||||
exec opencode
|
||||
"
|
||||
|
||||
# Restart policy
|
||||
restart: unless-stopped
|
||||
|
||||
# Resource limits (adjust as needed)
|
||||
# deploy:
|
||||
# resources:
|
||||
# limits:
|
||||
# cpus: '2'
|
||||
# memory: 2G
|
||||
# reservations:
|
||||
# cpus: '1'
|
||||
# memory: 1G
|
||||
|
||||
volumes:
|
||||
opencode-config:
|
||||
driver: local
|
||||
|
||||
opencode-workspace:
|
||||
driver: local
|
||||
|
||||
networks:
|
||||
default:
|
||||
driver: bridge
|
||||
Reference in New Issue
Block a user