Add Caddy reverse proxy configuration and Kopia backup documentation

- Create INFRASTRUCTURE-COMPLETE.md: Comprehensive infrastructure overview * Network architecture and DNS configuration * Storage architecture for Nextcloud (1.4TB warm storage via LVM) * Kopia backup setup with Backblaze B2 integration * Caddy reverse proxy configuration (CT 202) * Security configuration and capacity planning - Add Caddy reverse proxy configurations: * caddy-kopia-integration.md: Integration steps for CT 202 * caddy-kopia-reverse-proxy.md: Complete configuration reference * caddy/Caddyfile: Production-ready configuration * caddy/docker-compose.yml: Docker deployment spec - Add Kopia backup documentation: * kopia-nextcloud-backblaze-setup.md: Complete Kopia setup guide * Backup strategy, retention policies, and restore procedures - Update related documentation with consolidated references Status: Infrastructure documented and ready for deployment
2026-04-27 11:46:47 -05:00
parent b792d5bf41
commit 821f954200
31 changed files with 2373 additions and 71 deletions
@@ -0,0 +1,211 @@
+# Nextcloud Warm Storage Expansion Plan
+**Target**: Expand from 900GB to 1.5TB for Backblaze backup staging
+
+---
+
+## 📊 Current Storage Status
+
+| Component | Current | Status |
+|-----------|---------|--------|
+| **Warm Storage** | 900GB | 🔴 **FULL (100%)** |
+| Mount Point | `/mnt/warm-storage` | Nextcloud VM (192.168.88.62) |
+| NFS Path | `/nvme5-1tb/nextcloud-data` | TrueNAS (192.168.88.25) |
+| Local Data | 2.0TB | 74% used (1.4TB) |
+
+---
+
+## ⚠️ Critical Issue
+
+The warm storage is **completely full (900GB/900GB)**. This will block:
+- Backblaze backups
+- Nextcloud uploads
+- Data synchronization
+
+**Action Required**: Expand to 1.5TB ASAP
+
+---
+
+## 🔧 Expansion Steps (When TrueNAS is Online)
+
+### Step 1: SSH into TrueNAS (192.168.88.25)
+
+```bash
+ssh root@192.168.88.25
+# Or use your TrueNAS credentials
+```
+
+### Step 2: Check Current Dataset Configuration
+
+```bash
+# List datasets on nvme5-1tb pool
+zfs list -r nvme5-1tb
+
+# Check the current nextcloud-data dataset
+zfs get quota,available,used nvme5-1tb/nextcloud-data
+
+# Check the pool status
+zpool status nvme5-1tb
+```
+
+### Step 3: Expand the Dataset (Two Options)
+
+#### **OPTION A: Expand Existing Dataset** (Recommended if pool has space)
+
+If the pool has 600GB+ available space, expand the quota:
+
+```bash
+# Increase quota from 900GB to 1.5TB
+zfs set quota=1500G nvme5-1tb/nextcloud-data
+
+# Verify the change
+zfs get quota nvme5-1tb/nextcloud-data
+```
+
+#### **OPTION B: Create New Dataset** (If pool is full)
+
+If the pool doesn't have enough space:
+
+```bash
+# Create a new larger dataset
+zfs create -o quota=1500G nvme5-1tb/nextcloud-data-new
+
+# Copy data from old to new
+rsync -avh /nvme5-1tb/nextcloud-data/ /nvme5-1tb/nextcloud-data-new/
+
+# Once verified, delete old dataset
+zfs destroy nvme5-1tb/nextcloud-data
+
+# Rename new dataset
+zfs rename nvme5-1tb/nextcloud-data-new nvme5-1tb/nextcloud-data
+```
+
+### Step 4: Verify NFS Export Configuration
+
+```bash
+# Check current NFS exports
+cat /etc/exports
+
+# The export should look like:
+# /nvme5-1tb/nextcloud-data 192.168.88.62(rw,async,no_subtree_check)
+
+# If needed, reload NFS
+exportfs -ra
+```
+
+### Step 5: Remount on Nextcloud VM
+
+On Nextcloud VM (192.168.88.62):
+
+```bash
+# Unmount warm storage (ensure no active connections)
+sudo umount /mnt/warm-storage
+
+# Remount to refresh the size
+sudo mount -t nfs4 192.168.88.25:/nvme5-1tb/nextcloud-data /mnt/warm-storage
+
+# Verify new size
+df -h /mnt/warm-storage
+```
+
+### Step 6: Verify Expansion
+
+```bash
+# From Nextcloud VM
+df -h /mnt/warm-storage
+# Should show 1.5T available
+
+# From TrueNAS (optional confirmation)
+zfs get quota,used nvme5-1tb/nextcloud-data
+```
+
+---
+
+## 📋 Pre-Expansion Checklist
+
+- [ ] TrueNAS is online and accessible via SSH
+- [ ] Check available space in nvme5-1tb pool
+- [ ] Verify backup of critical Nextcloud data
+- [ ] Schedule during low-traffic period if possible
+- [ ] Ensure no active Backblaze uploads
+
+---
+
+## ⏱️ Expected Downtime
+
+- **Unmount/Remount**: ~5-10 seconds
+- **Option A (Quota Expansion)**: 30 seconds total
+- **Option B (New Dataset + rsync)**: 1-4 hours (depending on data size)
+
+---
+
+## 🔄 Troubleshooting
+
+### If NFS mount won't remount
+
+```bash
+# Check for open connections
+lsof /mnt/warm-storage
+
+# Kill processes if necessary
+sudo fuser -k /mnt/warm-storage
+
+# Try remounting again
+sudo mount -t nfs4 192.168.88.25:/nvme5-1tb/nextcloud-data /mnt/warm-storage
+```
+
+### If dataset expansion fails
+
+```bash
+# Check pool space
+zpool list nvme5-1tb
+
+# If pool is full, need to expand the underlying storage pool
+# This requires physical disk expansion on TrueNAS
+```
+
+### If Backblaze can't write after expansion
+
+1. Restart Nextcloud on the VM:
+   ```bash
+   docker restart nextcloud  # or appropriate restart command
+   ```
+
+2. Check Nextcloud logs:
+   ```bash
+   docker logs nextcloud | tail -50
+   ```
+
+---
+
+## 📝 Commands Summary (Copy-Paste Ready)
+
+**On TrueNAS:**
+```bash
+zfs set quota=1500G nvme5-1tb/nextcloud-data
+zfs get quota nvme5-1tb/nextcloud-data
+```
+
+**On Nextcloud VM:**
+```bash
+sudo umount /mnt/warm-storage
+sudo mount -t nfs4 192.168.88.25:/nvme5-1tb/nextcloud-data /mnt/warm-storage
+df -h /mnt/warm-storage
+```
+
+---
+
+## 📞 Post-Expansion Verification
+
+After expansion is complete:
+
+1. ✅ Verify size with `df -h /mnt/warm-storage` (should show ~1.5T)
+2. ✅ Check available space: `df -h | grep warm`
+3. ✅ Test Nextcloud upload functionality
+4. ✅ Verify Backblaze can write to warm storage
+5. ✅ Monitor storage usage for 24 hours
+
+---
+
+**Last Updated**: April 27, 2026  
+**Status**: Pending TrueNAS availability  
+**Priority**: 🔴 HIGH - Warm storage is full