jgitta/homelab-configs
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Add complete homelab documentation and configurations

Browse Source 
Documentation:
- OpenCode + LiteLLM integration guide
- LiteLLM complete setup with 16 models
- Gitea centralized configuration guide
- Testing procedures and verification
- API keys setup instructions

Configurations:
- OpenCode config pointing to LiteLLM
- Updated LiteLLM config with all models
- Nextcloud docker-compose template

Scripts:
- Gitea setup automation
- OpenCode testing script

Infrastructure:
- Gitea: 192.168.88.200:3000
- LiteLLM: 192.168.88.27:4000
- Nextcloud: 192.168.88.62
This commit is contained in:
Jozsef Gtta
2026-04-25 10:56:21 -05:00
parent eba13efd34
commit 290aedf82d
36 changed files with 8074 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
litellm-complete-setup.md
+464
View File
@@ -0,0 +1,464 @@
# LiteLLM Centralized API Gateway - Complete Setup Guide
## ✅ Infrastructure Status
**All components installed and running:**
- ✅ **LiteLLM Container**: Running on docker-server (siklos, 192.168.88.27:4000)
- ✅ **Docker Compose**: Configured and ready at `/opt/litellm/`
- ✅ **Configuration**: Git initialized with config files in `/opt/litellm/config/`
- ✅ **Gitea**: Ready at http://192.168.88.200:3000 for repository management
- ✅ **All VMs**: Network access confirmed to docker-server
---
## 📁 Current Setup Directory Structure
```
docker-server (siklos @ 192.168.88.27)
└── /opt/litellm/
├── docker-compose.yml # Docker service configuration
├── .env # Environment variables (API keys)
├── .git/ # Git version control
├── .git-setup.sh # Helper script for Gitea push
├── logs/ # Service logs
└── config/
├── README.md # Configuration guide
└── litellm_config.yaml # Model definitions
```
---
## 🔧 Next Steps: Configure API Keys
### Step 1: Add Your Real API Keys
Edit the `.env` file with your actual API credentials:
```bash
ssh jgitta@192.168.88.27 "nano /opt/litellm/.env"
```
Update these lines with your real keys:
```bash
OPENAI_API_KEY=sk-your-real-openai-key-here
ANTHROPIC_API_KEY=sk-ant-your-real-anthropic-key-here
```
**⚠️ Security Note:** Never commit API keys to git. The `.env` file should never be pushed to Gitea.
### Step 2: Restart LiteLLM with Real Keys
```bash
ssh jgitta@192.168.88.27 << 'EOF'
cd /opt/litellm
docker-compose down
docker-compose up -d
sleep 10
docker logs litellm | tail -20
EOF
```
### Step 3: Test the Gateway
Once running with real keys, test from any VM:
```bash
curl -X POST http://192.168.88.27:4000/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer litellm-local-key-change-in-production" \
-d '{
"model": "gpt-4",
"messages": [
{"role": "user", "content": "Hello! What is 2+2?"}
]
}'
```
---
## 🎯 How It Works: Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ Your VMs │
│ (jellyfin, next, photos, haos, etc.) │
│ ↓ │
│ Single API Endpoint: │
│ http://192.168.88.27:4000 │
└─────────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ LiteLLM API Gateway (Docker Container) │
│ Running on docker-server (siklos) │
│ │
│ Routes requests to appropriate backend: │
│ - gpt-4 → OpenAI API │
│ - claude-3 → Anthropic API │
│ - local-* → Local models (if configured) │
│ │
│ Advantages: │
│ ✓ Single authentication point │
│ ✓ Centralized API key management │
│ ✓ Easy to add/remove models │
│ ✓ Request logging & monitoring │
│ ✓ Load balancing & fallbacks │
└─────────────────────────────────────────────────────────────────┘
↓
┌──────┴──────┐
↓ ↓
OpenAI API Anthropic API
```
---
## 📝 Configuration Files
### Docker Compose (`docker-compose.yml`)
The service is configured to:
- **Listen on**: `0.0.0.0:4000` (accessible from all VMs)
- **Load env from**: `.env` file
- **Mount volumes**:
- `./config/` → `/app/config/` (read-only)
- `./logs/` → `/app/logs/` (read-write)
- **Auto-restart**: Unless stopped manually
### Environment File (`.env`)
Contains your API keys. Format:
```
OPENAI_API_KEY=sk-xxx...
ANTHROPIC_API_KEY=sk-ant-xxx...
LITELLM_MASTER_KEY=your-master-auth-key
```
### Configuration YAML (`config/litellm_config.yaml`)
Defines which models are available and how to route them. Current models:
```yaml
Models Configured:
✓ gpt-4 (OpenAI)
✓ gpt-3.5-turbo (OpenAI)
✓ claude-3-sonnet (Anthropic)
```
---
## 🔑 API Key Management Strategy
### Local Storage (Current Setup)
- API keys stored in `.env` file on docker-server
- Not committed to git (security best practice)
- Restart required after changes
### Optional: Gitea-Based Config (Future)
Once tested, you can:
1. Create a secure Gitea repository for non-sensitive config
2. Store API keys as Docker secrets or environment file separately
3. Pull config updates automatically
```bash
# Create litellm-config repo in Gitea, then:
cd /opt/litellm
git remote add gitea http://192.168.88.200:3000/jgitta/litellm-config.git
git pull gitea master
```
---
## 🧪 Testing & Monitoring
### Check Service Status
```bash
ssh jgitta@192.168.88.27 << 'EOF'
cd /opt/litellm
docker-compose ps
docker logs litellm -f # Follow logs in real-time
EOF
```
### View Active Models
```bash
curl http://192.168.88.27:4000/models \
-H "Authorization: Bearer litellm-local-key-change-in-production"
```
### Monitor Logs Real-Time
```bash
ssh jgitta@192.168.88.27 "docker logs litellm -f"
```
---
## 💻 Usage Examples from Your VMs
### Python Example
```python
import requests
import json
LITELLM_URL = "http://192.168.88.27:4000"
MASTER_KEY = "litellm-local-key-change-in-production"
response = requests.post(
f"{LITELLM_URL}/chat/completions",
headers={
"Authorization": f"Bearer {MASTER_KEY}",
"Content-Type": "application/json"
},
json={
"model": "gpt-4",
"messages": [
{"role": "user", "content": "Explain this to me like I'm 5"}
]
}
)
print(response.json())
```
### Bash Example
```bash
#!/bin/bash
LITELLM_URL="http://192.168.88.27:4000"
MASTER_KEY="litellm-local-key-change-in-production"
MODEL="gpt-4"
curl -s -X POST "$LITELLM_URL/chat/completions" \
-H "Authorization: Bearer $MASTER_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "'$MODEL'",
"messages": [
{"role": "user", "content": "What is the capital of France?"}
]
}' | jq '.choices[0].message.content'
```
### Node.js Example
```javascript
const fetch = require('node-fetch');
const LITELLM_URL = 'http://192.168.88.27:4000';
const MASTER_KEY = 'litellm-local-key-change-in-production';
async function askLiteLLM(question, model = 'gpt-4') {
const response = await fetch(`${LITELLM_URL}/chat/completions`, {
method: 'POST',
headers: {
'Authorization': `Bearer ${MASTER_KEY}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({
model: model,
messages: [
{ role: 'user', content: question }
]
})
});
const data = await response.json();
return data.choices[0].message.content;
}
// Usage:
askLiteLLM('Hello, who are you?').then(response => console.log(response));
```
---
## 🔒 Security Considerations
### Current (Development) Setup
- Master key is a placeholder: `litellm-local-key-change-in-production`
- API keys stored in plain `.env` file
- Service accessible from all VMs on network
### Production Recommendations
1. **Change Master Key**
```bash
# Edit .env and set a strong key
LITELLM_MASTER_KEY=your-random-secure-32-char-key
docker-compose restart
```
2. **Use Docker Secrets** (for Swarm/Kubernetes)
```bash
echo "your-real-api-key" | docker secret create openai_key -
```
3. **Restrict Network Access**
- Use firewall rules to limit which VMs can access port 4000
- Only allow specific source IPs if possible
4. **SSL/TLS Encryption**
- Run LiteLLM behind Caddy (your existing reverse proxy)
- Use HTTPS instead of HTTP
5. **API Key Rotation**
- Periodically update API keys in `.env`
- Restart container after changes
---
## 🚀 Advanced Customization
### Add More Models
Edit `config/litellm_config.yaml`:
```yaml
model_list:
- model_name: my-custom-model
litellm_params:
model: openai/gpt-3.5-turbo
api_key: ${OPENAI_API_KEY}
```
Then restart:
```bash
cd /opt/litellm
docker-compose restart
```
### Enable Request Logging
Edit `docker-compose.yml` and add:
```yaml
environment:
LOG_LEVEL: DEBUG
LITELLM_LOG_REQUESTS: "true"
```
### Set Up Load Balancing
Multiple backend API keys for the same model:
```yaml
model_list:
- model_name: gpt-4
litellm_params:
model: gpt-4-backup # Primary
- model_name: gpt-4
litellm_params:
model: gpt-4-fallback # Fallback
```
---
## 📞 Troubleshooting
### Container Won't Start
```bash
# Check logs
docker logs litellm 2>&1 | tail -50
# Most common: Missing API keys
# Solution: Update .env with real keys
# Restart with fresh pull
docker-compose down
docker image pull ghcr.io/berriai/litellm:main
docker-compose up -d
```
### "Connection Refused" from VM
```bash
# Check container is running
docker ps | grep litellm
# Check port is listening
docker exec litellm netstat -tlnp | grep 4000
# Try from docker-server first
docker exec litellm curl http://localhost:4000/health
```
### "Unauthorized" Error
```bash
# Verify master key in request matches .env
# Default: litellm-local-key-change-in-production
# Test with correct key:
curl -H "Authorization: Bearer litellm-local-key-change-in-production" \
http://192.168.88.27:4000/models
```
### Logs Show "Missing Credentials"
```bash
# Edit .env with real API keys
nano /opt/litellm/.env
# Restart the service
docker-compose down && docker-compose up -d
sleep 10
docker logs litellm | tail -20
```
---
## 📚 References & Documentation
- **LiteLLM Docs**: https://docs.litellm.ai/
- **LiteLLM Proxy Server**: https://docs.litellm.ai/docs/proxy_server
- **OpenAI API**: https://platform.openai.com/docs/
- **Anthropic Claude**: https://docs.anthropic.com/
---
## 🎓 Quick Commands Reference
```bash
# SSH to docker-server
ssh jgitta@192.168.88.27
# Navigate to litellm directory
cd /opt/litellm
# View status
docker-compose ps
docker logs litellm -f
# Restart service
docker-compose restart
# Edit API keys
nano .env
docker-compose down && docker-compose up -d
# View config
cat config/litellm_config.yaml
# Test from VM
curl -X POST http://192.168.88.27:4000/chat/completions \
-H "Authorization: Bearer litellm-local-key-change-in-production" \
-H "Content-Type: application/json" \
-d '{"model":"gpt-4","messages":[{"role":"user","content":"test"}]}'
```
---
## ✨ What's Next
1. **Add Real API Keys** to `.env`
2. **Test the Gateway** from a VM
3. **Monitor Logs** to ensure everything works
4. **Document Your Models** in `config/README.md`
5. **(Optional) Set up Gitea** for config version control
6. **(Optional) Add Caddy** in front for HTTPS/SSL
---
**Your centralized API gateway is ready to go! All VMs now have a single endpoint to access multiple AI models. 🚀**