homelab-configs/INFRASTRUCTURE-COMPLETE.md

Homelab Infrastructure - Complete Documentation

Last Updated: April 27, 2026
Status: ✅ Operational and Documented

---

📍 Network Overview

DNS & Routing

• MikroTik Router: 192.168.88.1 (service user)
• Primary Domain: jgitta.com
• DNS Records Configured:
  • kopia.jgitta.com → 192.168.88.27 (Kopia backup server)
  • gitea.jgitta.com → 192.168.88.27 (Gitea repository)
  • next.jgitta.com → 192.168.88.62 (Nextcloud)

Core Infrastructure

Host	IP	Role	Services
MikroTik	192.168.88.1	Router/DNS	DHCP, DNS, Firewall
jg-hud (Proxmox)	192.168.88.25	Hypervisor	VM/Container management
siklos (Docker)	192.168.88.27	Docker Host	Gitea, LiteLLM, Kopia
Nextcloud	192.168.88.62	Web Server	Nextcloud, MariaDB, Apache

---

🗄️ Storage Architecture

Nextcloud VM (192.168.88.62)

Storage Devices

/dev/sda (128GB) - Root Filesystem
├─ /dev/sda1: 117.3GB (/root system)
├─ /dev/sda2: 1KB (partition)
└─ /dev/sda5: 1.9GB (SWAP)

/dev/sdb (1TB) - Warm Storage (LVM)
└─ warm-storage-vg
   └─ warm-storage (980GB logical volume)
      └─ /mnt/warm-storage (ext4, 964GB available)

/dev/sdc (2TB) - Nextcloud Data
└─ /dev/sdc1 (2TB ext4)
   └─ /mnt/nextcloud-data (1.4TB used, 486GB free)

Mount Points & Usage

Mount Point	Size	Used	Available	Purpose
/	116GB	22GB	89GB	Root filesystem
/mnt/nextcloud-data	2TB	1.4TB	486GB	Nextcloud database & files
/mnt/warm-storage	964GB	13GB	915GB	Backblaze backup staging
/mnt/media	11TB	6.4TB	3.9TB	Media files (NFS from TrueNAS)

Warm Storage Configuration

What It Is: Temporary staging area for Backblaze backups

Technical Details:

• Type: Local LVM (Logical Volume Manager)
• Components:
  • PV1: /dev/sdb (1000GB physical disk)
  • PV2: /dev/loop4 (500GB sparse file on /mnt/nextcloud-data)
  • VG: warm-storage-vg (1.38TB total)
  • LV: warm-storage (1.37TB allocated, 1.4TB displayed via df)

Why 1.4TB is Correct:

• Nextcloud data: 1.4TB
• Warm storage needed: 1.4TB (for full backup staging)
• Available: 915GB (62% of data size = safe buffer)

Backup Workflow:

Nextcloud Data (1.4TB)
    ↓
Warm Storage (1.4TB local SSD)
    ↓
Kopia Backup (incremental)
    ↓
Backblaze B2 (encrypted cloud)

---

🔐 Backup Infrastructure

Kopia Backup Server

Location: siklos (192.168.88.27)

Docker Configuration

Container: kopia/kopia:latest
Port: 51515 (Web UI)
Config: /srv/docker/kopia/config/
Cache: /srv/docker/kopia/cache/
Mounts: /mnt/nextcloud → /data (read-only)

Repository Settings

Provider:        Backblaze B2
Bucket:          jg-kopia-nextcloud
Bucket ID:       42c2db92c477c1fe95df0019
Key ID:          00522b2471e5f090000000003
Encryption:      AES256-GCM-HMAC-SHA256
Hashing:         BLAKE2B-256-128
Key Derivation:  scrypt-65536-8-1
Repository PW:   kopia123 (change recommended)

Backup Sources

• Path: /data (mounted from /mnt/nextcloud on siklos)
• Includes:
  • Nextcloud user data
  • Database backups
  • Configuration files
  • Media and uploads

Retention Policy (Default)

• Annual: 3 snapshots
• Monthly: 24 snapshots
• Weekly: 4 snapshots
• Daily: 7 snapshots
• Hourly: 48 snapshots
• Latest: 10 snapshots

Maintenance

• Quick maintenance: Hourly
• Full maintenance: Daily
• Automatic deduplication
• Incremental backups

Web Access

Kopia Web UI (via Caddy Reverse Proxy):

• HTTPS URL: https://kopia.jgitta.com (recommended)
• Direct URL: http://192.168.88.27:51515 (internal access)
• Caddy Reverse Proxy: Configured on siklos port 443
• Credentials: admin / YourSecurePassword (update in docker-compose.yml)
• Features:
  • Create snapshots
  • Browse backup history
  • Restore files
  • View statistics
  • WebSocket support for real-time updates

Caddy Configuration:

• Serves HTTPS on kopia.jgitta.com
• Automatic TLS (self-signed for internal use)
• Security headers (HSTS, X-Frame-Options, etc.)
• Proxy to Kopia Web UI on localhost:51515
• See caddy-kopia-reverse-proxy.md for full configuration

Database Backup

Location: /mnt/nextcloud-data/nextcloud-db-backup.sql

Configuration:

• Database Type: MariaDB
• Server: 192.168.88.62
• Database: nextcloud
• Backup Size: ~1.1GB

Backup Methods:

1. File-based (current): SQL dump included in Kopia snapshots
2. Pre-snapshot hook (optional): Auto-dump before each snapshot

---

🔧 Services & APIs

Caddy Reverse Proxy

Location: CT 202 (Proxmox Container)

Configuration:

• Service: Running on CT 202
• Port: 443 (HTTPS), 80 (HTTP redirect)
• Config: Caddyfile (location varies by CT 202 setup)

Routes:

• kopia.jgitta.com → http://192.168.88.27:51515 (Kopia Web UI on siklos)

Features:

• Automatic HTTPS/TLS (self-signed for internal, Let's Encrypt ready)
• Security headers (HSTS, X-Frame-Options, CSP)
• WebSocket support
• Zero-downtime reloads
• Comprehensive logging

See Also:

• caddy-kopia-integration.md for CT 202 integration instructions
• caddy-kopia-reverse-proxy.md for detailed configuration reference

Gitea Repository

Location: siklos (192.168.88.27:3002)

URL: http://192.168.88.27:3002 (or http://gitea.jgitta.com)

Repository: homelab-configs

Contents:

• Infrastructure documentation
• Kopia setup guides
• OpenCode configuration
• LiteLLM models
• Backup scripts

LiteLLM API Gateway

Location: siklos (192.168.88.27:4000)

Features:

• Centralized API key management
• Multi-model support (Claude, GPT, Gemini)
• 16 models available
• Authentication via API keys

Nextcloud

Location: 192.168.88.62 (or next.jgitta.com)

Services:

• Web UI: Apache2
• Database: MariaDB
• Data Storage: /mnt/nextcloud-data (2TB)
• Backup Staging: /mnt/warm-storage (1.4TB)

---

📚 Complete File Structure

homelab-infrastructure/
├── INFRASTRUCTURE-COMPLETE.md          ← THIS FILE
├── README.md
├── AGENTS.md
│
├── Nextcloud & Backups
│   ├── nextcloud-warm-storage-final-1.4tb.md
│   ├── nextcloud-warm-storage-expansion.md
│   ├── nextcloud-warm-storage-expansion-completed.md
│   └── kopia-nextcloud-backblaze-setup.md
│
├── Reverse Proxy & HTTPS
│   ├── caddy-kopia-reverse-proxy.md
│   └── caddy/
│       ├── Caddyfile (to be deployed on siklos)
│       └── docker-compose.yml (to be deployed on siklos)
│
├── LiteLLM Setup
│   ├── litellm-complete-setup.md
│   ├── litellm-setup-guide.md
│   ├── litellm-api-keys-setup.md
│   ├── litellm-models-update.md
│   ├── litellm-testing-verification.md
│   ├── litellm_config_updated.yaml
│   └── litellm/
│
├── OpenCode Configuration
│   ├── opencode-litellm-ready.md
│   └── opencode/
│       └── config.json
│
├── Gitea Integration
│   ├── gitea-centralized-implementation.md
│   └── gitea-centralized-setup.sh
│
├── Testing & Validation
│   ├── opencode-testing-guide.md
│   ├── dedup-strategy-jgitta.md
│   ├── dedup-dryrun-report-cgitta.md
│   └── opencode/config.json
│
└── Docker Services
    ├── siklos-docker-services.md
    └── nextcloud-docker-compose-update.yml

---

🚀 Operational Procedures

Daily Operations

Monitor Backup Status:

# Check Kopia container
docker ps | grep kopia

# View recent logs
docker logs -n 50 kopia

# List latest snapshots
docker exec -e KOPIA_PASSWORD='kopia123' kopia \
  kopia snapshot list /data

Check Storage Usage:

# SSH to Nextcloud VM
ssh jgitta@next

# View disk usage
df -h /mnt/warm-storage /mnt/nextcloud-data

# Monitor warm storage
du -sh /mnt/warm-storage/*

Weekly Tasks

• Review backup success rate
• Verify Backblaze B2 uploads progressing
• Check storage trends
• Monitor system logs for errors

Monthly Tasks

• Test file restoration
• Review backup costs on B2
• Audit retention policies
• Verify database backup integrity

Quarterly Tasks

• Full test restore to alternate location
• Review and update encryption keys if needed
• Audit access logs
• Plan capacity expansion if needed

---

🔐 Security Configuration

Encryption at Rest

• Kopia Backups: AES256-GCM-HMAC-SHA256
• Database: MariaDB native (can be encrypted)
• Storage: LVM (unencrypted, but in private network)

Encryption in Transit

• B2 Connection: HTTPS/TLS
• SFTP to Nextcloud: SSH key-based
• Internal Network: Private LAN (192.168.88.0/24)

Access Control

Service	Access	Auth	Location
Kopia Web UI	51515	Username/Password	siklos
Gitea	3002	SSH key / Password	siklos
LiteLLM API	4000	API Key	siklos
Nextcloud	80/443	User accounts	next
MariaDB	3306	Local/socket	next

---

📊 Capacity Planning

Current Usage

System	Capacity	Used	Available	% Used
Root FS	116GB	22GB	89GB	19%
Nextcloud Data	2TB	1.4TB	486GB	70%
Warm Storage	1.4TB	13GB	915GB	1%
Media (NFS)	11TB	6.4TB	3.9TB	58%

Backup Capacity (B2)

• Initial upload: ~630GB
• Monthly incremental: ~10-50GB (varies)
• Estimated monthly cost: $2-4 USD
• Retention: 24+ monthly snapshots

Growth Projections (Annual)

• Nextcloud data: +10-20%
• Warm storage: Stable (staging only)
• Backups: +5-10% annually
• Total annual cost estimate: $30-50

---

🔗 Integration Points

External Systems

• Backblaze B2: Cloud backup storage
• MikroTik Router: Network and DNS
• TrueNAS: Media storage (11TB via NFS)

Internal Services

• Kopia → Backblaze B2 (encrypted backups)
• Nextcloud → Warm Storage (staging)
• Gitea → Central config repository
• LiteLLM → API gateway for models
• OpenCode → IDE with LiteLLM integration

---

📞 Troubleshooting Quick Reference

Warm Storage Issues

# Check mount
mount | grep warm

# Check LVM status
lvs warm-storage-vg

# Remount if needed
sudo umount /mnt/warm-storage
sudo mount /dev/warm-storage-vg/warm-storage /mnt/warm-storage

Kopia Issues

# Check container
docker ps | grep kopia

# View logs
docker logs -f kopia

# Restart
docker-compose -f /srv/docker/kopia/docker-compose.yml restart

Database Issues

# SSH to Nextcloud VM
ssh jgitta@next

# Check MariaDB
sudo systemctl status mariadb

# Manual backup
mysqldump -u nextcloud -p nextcloud > backup.sql

B2 Connectivity

# From Kopia container
docker exec kopia kopia repository validate-provider

# Check network
curl https://api.backblazeb2.com

---

📝 Important Notes

Passwords & Credentials

• Kopia repository password: kopia123 (CHANGE THIS)
• Backblaze credentials: Stored in Kopia config
• Database backup: Included in snapshots
• Update docker-compose.yml credentials regularly

Known Limitations

• Warm storage is non-redundant (single drive)
• Database backups are file-based (manual pre-backup setup recommended)
• LVM loopback device needs remounting after reboot (can be automated)
• Kopia Web UI uses HTTP (use HTTPS proxy in production)

Future Improvements

• Automate database dumps before backups
• Set up Nginx reverse proxy for HTTPS
• Add RAID to warm storage for redundancy
• Configure scheduled automated snapshots
• Implement backup verification tests
• Add monitoring/alerting for backup failures

---

✅ Deployment Checklist

• ✅ Nextcloud VM with 2TB data storage
• ✅ Warm storage expanded to 1.4TB
• ✅ LVM configured for flexibility
• ✅ Kopia installed and configured
• ✅ Backblaze B2 backend active
• ✅ DNS records created (kopia.jgitta.com, gitea.jgitta.com)
• ✅ Initial backups in progress
• ⏳ Automated snapshot schedule (TODO)
• ⏳ HTTPS/reverse proxy (TODO)
• ⏳ Monitoring and alerting (TODO)

---

📖 Related Documentation

Detailed Guides

• Nextcloud Warm Storage Expansion
• Kopia Backup Setup
• Caddy Reverse Proxy Integration (CT 202)
• Caddy Configuration Reference
• LiteLLM Configuration
• Gitea Integration

Configuration Files

• Kopia: /srv/docker/kopia/docker-compose.yml
• Nextcloud: /etc/apache2/sites-available/nextcloud.conf
• LiteLLM: litellm_config_updated.yaml
• OpenCode: opencode/config.json

Repository

• Location: http://192.168.88.27:3002/jgitta/homelab-configs
• Branch: main
• Push Command: git push origin main

---

Infrastructure Status: ✅ Fully Operational
Last Updated: April 27, 2026
Maintained By: jgitta@jgitta.com
Backup Status: Active (Kopia → Backblaze B2)