jgitta
290aedf82d
Documentation: - OpenCode + LiteLLM integration guide - LiteLLM complete setup with 16 models - Gitea centralized configuration guide - Testing procedures and verification - API keys setup instructions Configurations: - OpenCode config pointing to LiteLLM - Updated LiteLLM config with all models - Nextcloud docker-compose template Scripts: - Gitea setup automation - OpenCode testing script Infrastructure: - Gitea: 192.168.88.200:3000 - LiteLLM: 192.168.88.27:4000 - Nextcloud: 192.168.88.62
77 lines
2.7 KiB
Markdown
77 lines
2.7 KiB
Markdown
# Strategy: Dedup & Merge `cold-storage/jgitta` into Nextcloud jgitta
|
||
|
||
**Goal:** Take everything in `/mnt/cold-storage/jgitta`, compare it to jgitta's primary Nextcloud files, delete duplicates from cold-storage, and move unique files into jgitta's account.
|
||
|
||
**Scope:** jgitta only (separate from cgitta project, which is now complete).
|
||
|
||
---
|
||
|
||
## The Situation
|
||
|
||
| Location | Size | Notes |
|
||
|---|---|---|
|
||
| `/mnt/cold-storage/jgitta` | **UNKNOWN** (very large) | NFS-mounted from TrueNAS. ~25 subfolders: phone backups, Google Takeout, OneDrive dumps, etc. |
|
||
| `/mnt/nextcloud-data/jgitta/files` | **239 GB** | Local 2 TB SSD on VM103. jgitta's primary Nextcloud files. |
|
||
| Free space on `/mnt/nextcloud-data` | **488 GB** | This is the constraint. If cold-storage/jgitta > 488 GB of *unique* files, we'll run out of space. |
|
||
|
||
**⚠️ Critical:** jgitta's cold-storage is likely multi-TB. If unique files exceed 488 GB, we need to expand the primary disk or use warm-storage for overflow.
|
||
|
||
---
|
||
|
||
## Phase 0: Hash Scan (Read-only inventory)
|
||
|
||
**Step 1:** Start hash scans (may take 2-4 hours given size)
|
||
|
||
```bash
|
||
cd /var/tmp/dedup-jgitta
|
||
nohup /var/tmp/dedup-jgitta/scan.sh /mnt/cold-storage/jgitta /var/tmp/dedup-jgitta/cold > /dev/null 2>&1 &
|
||
nohup /var/tmp/dedup-jgitta/scan.sh /mnt/nextcloud-data/jgitta/files /var/tmp/dedup-jgitta/primary > /dev/null 2>&1 &
|
||
```
|
||
|
||
**Step 2:** When scans finish, build report (same as cgitta):
|
||
|
||
```bash
|
||
cd /var/tmp/dedup-jgitta
|
||
awk '{print $1}' cold.hashes | sort -u > cold.hashset
|
||
awk '{print $1}' primary.hashes | sort -u > primary.hashset
|
||
comm -12 cold.hashset primary.hashset > dup.hashset
|
||
comm -23 cold.hashset primary.hashset > cold-only.hashset
|
||
grep -Ff dup.hashset cold.hashes > dup.files
|
||
grep -Ff cold-only.hashset cold.hashes > unique.files
|
||
```
|
||
|
||
**Step 3:** Review numbers for capacity planning.
|
||
|
||
---
|
||
|
||
## ⚠️ CAPACITY DECISION POINT
|
||
|
||
Before proceeding to Phase 2, check:
|
||
|
||
1. **Size of unique files:** `awk '{print $1}' unique.files | wc -l` and estimate total GB
|
||
2. **Available space:** Currently 488 GB. If unique > 488 GB:
|
||
- **Option A:** Expand primary disk (add storage to `/mnt/nextcloud-data`)
|
||
- **Option B:** Move jgitta's cold-storage overflow to warm-storage tier (if available)
|
||
- **Option C:** Dedup in two passes (move largest categories first, verify space)
|
||
|
||
---
|
||
|
||
## Phases 2–5 (same as cgitta, if space is available)
|
||
|
||
Once capacity is confirmed:
|
||
|
||
- **Phase 2:** Delete duplicates from cold-storage (with ZFS snapshot backup)
|
||
- **Phase 3:** Move unique files to primary
|
||
- **Phase 4:** Fix ownership + Nextcloud rescan
|
||
- **Phase 5:** Verify + cleanup
|
||
|
||
---
|
||
|
||
## Next Step
|
||
|
||
1. Start Phase 0 hash scans
|
||
2. Wait for completion (may take several hours)
|
||
3. Report numbers
|
||
4. Decide on capacity strategy before Phase 2
|
||
|