jgitta/homelab-configs
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Add: Nextcloud Backblaze B2 backup solution - automated daily backups

Browse Source 
- Complete backup script for native Nextcloud (MySQL + files)
- Systemd service and timer for daily automated backups at 2 AM
- Backblaze B2 integration with secure credential storage
- Incremental backup support with 30-day retention
- zstd compression for 40-70% storage reduction
- Comprehensive documentation and deployment guides
- Successfully tested with first backup: 203MB DB + 1.3GB files
- Cost-optimized: ~/bin/bash.01-0.02/month for 1.5GB data

Deployment Details:
- Installed B2 CLI via pipx
- Created backup directories at /opt/nextcloud-backup/
- Configured systemd timer for daily execution at 2 AM
- First backup verified in B2 bucket: jg-nextcloud
- Automated scheduling enabled and tested

Documentation includes:
- Deployment report with complete setup details
- Quick start guide for future deployments
- Comprehensive technical reference
- Troubleshooting procedures
- File manifest and index
This commit is contained in:
Jozsef Gtta
2026-04-25 18:00:07 -05:00
commit 2625cf36d2
14 changed files with 3090 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
NEXTCLOUD_BACKBLAZE_SETUP.md
+445
View File
@@ -0,0 +1,445 @@
# Nextcloud to Backblaze B2 Automated Backups
Complete guide for setting up automated periodic backups of your Nextcloud instance to Backblaze B2.
## Overview
This solution provides:
- **Incremental & full backups** - Smart backup strategy to minimize data transfer
- **Database + files** - Complete Nextcloud backup including database and data
- **Automated scheduling** - systemd timer runs daily at 2 AM
- **Retention management** - Automatic cleanup of backups older than 30 days
- **Compression** - zstd compression to reduce storage costs
- **Monitoring** - Status checks and error reporting
## Prerequisites
### System Requirements
- Linux host with systemd
- Docker & Docker Compose (for Nextcloud)
- B2 CLI tool
- 10+ GB free disk space for temporary backups
### Backblaze B2 Setup
1. **Create a Backblaze B2 Account** (if not already done)
- Sign up at https://www.backblaze.com/b2/cloud-storage/
2. **Create a Bucket**
- In B2 console: Create private bucket named `nextcloud-backups` (or your preference)
- Note the bucket name
3. **Generate API Credentials**
- Go to Account → App Keys → Create Application Key
- Recommended settings:
- **File names** with prefix: `nextcloud-backups/`
- **Access**: Read & Write
- **Valid for**: No expiration (or your preference)
- Save the `Account ID` and `Application Key`
### Nextcloud Setup
You need to identify:
- **Nextcloud data path** (usually `/srv/docker/nextcloud/data` or similar)
- **Database details**:
- Type (MariaDB or PostgreSQL)
- Database name (usually `nextcloud`)
- Database user (usually `nextcloud`)
- Database password
## Installation
### Step 1: Install B2 CLI
```bash
# Install B2 command-line tool
pip3 install b2
# Verify installation
b2 version
```
### Step 2: Create Backup Directory
```bash
sudo mkdir -p /opt/nextcloud-backup
sudo mkdir -p /etc/nextcloud-backup
sudo chmod 700 /opt/nextcloud-backup
sudo chmod 700 /etc/nextcloud-backup
```
### Step 3: Deploy Backup Scripts
```bash
# Copy main backup script
sudo cp nextcloud-backup-to-backblaze.sh /opt/nextcloud-backup/
sudo chmod +x /opt/nextcloud-backup/nextcloud-backup-to-backblaze.sh
# Copy systemd files
sudo cp nextcloud-backup.service /etc/systemd/system/
sudo cp nextcloud-backup.timer /etc/systemd/system/
sudo chmod 644 /etc/systemd/system/nextcloud-backup.*
# Reload systemd
sudo systemctl daemon-reload
```
### Step 4: Configure Environment
Create the configuration files:
**File: `/opt/nextcloud-backup/.env`**
```bash
# Nextcloud configuration
NEXTCLOUD_COMPOSE_DIR=/path/to/nextcloud/docker-compose
NEXTCLOUD_DATA_PATH=/srv/docker/nextcloud/data
NEXTCLOUD_DB_NAME=nextcloud
NEXTCLOUD_DB_USER=nextcloud
NEXTCLOUD_DB_PASSWORD=your_db_password_here
NEXTCLOUD_CONTAINER=nextcloud
# Backup configuration
BACKUP_RETENTION_DAYS=30
COMPRESSION=zstd
BACKUP_DIR=/opt/nextcloud-backup/.backups
```
**File: `/etc/nextcloud-backup/b2-credentials.env`** (root-only, no world-readable)
```bash
# Backblaze B2 credentials
B2_ACCOUNT_ID=your_account_id_here
B2_APPLICATION_KEY=your_application_key_here
B2_BUCKET_NAME=nextcloud-backups
B2_REGION=us-west-002
```
Set secure permissions:
```bash
sudo chmod 600 /etc/nextcloud-backup/b2-credentials.env
sudo chown root:root /etc/nextcloud-backup/b2-credentials.env
```
### Step 5: Test the Backup
Run a test backup manually:
```bash
# Test with dry-run first
sudo DRY_RUN=true /opt/nextcloud-backup/nextcloud-backup-to-backblaze.sh
# Run actual incremental backup
sudo /opt/nextcloud-backup/nextcloud-backup-to-backblaze.sh incremental
# Or full backup
sudo /opt/nextcloud-backup/nextcloud-backup-to-backblaze.sh full
# Check backup status
sudo /opt/nextcloud-backup/nextcloud-backup-to-backblaze.sh status
```
### Step 6: Enable Automated Backups
```bash
# Enable and start the timer
sudo systemctl enable nextcloud-backup.timer
sudo systemctl start nextcloud-backup.timer
# Verify it's active
sudo systemctl status nextcloud-backup.timer
# See next scheduled run
sudo systemctl list-timers nextcloud-backup.timer
```
## Usage
### Manual Backup Commands
```bash
# Dry-run (shows what would happen, no actual upload)
DRY_RUN=true sudo /opt/nextcloud-backup/nextcloud-backup-to-backblaze.sh
# Incremental backup (only changed files)
sudo /opt/nextcloud-backup/nextcloud-backup-to-backblaze.sh incremental
# Full backup (all files)
sudo /opt/nextcloud-backup/nextcloud-backup-to-backblaze.sh full
# Check status
sudo /opt/nextcloud-backup/nextcloud-backup-to-backblaze.sh status
```
### View Backup Logs
```bash
# Recent backups
sudo tail -f /opt/nextcloud-backup/.backups/logs/backup_*.log
# Latest backup status
sudo ls -lh /opt/nextcloud-backup/.backups/logs/ | tail -5
```
### View Systemd Timer Status
```bash
# Timer status
sudo systemctl status nextcloud-backup.timer
# Last run results
sudo journalctl -u nextcloud-backup.service -n 50
# Live monitoring
sudo journalctl -u nextcloud-backup.service -f
```
## Backup Strategy
### Incremental vs Full
- **Incremental** (default, runs daily):
- Only backs up files changed since last backup
- Faster, saves bandwidth and storage
- Requires full backup to restore
- **Full** (recommended weekly):
- Backs up all files regardless of changes
- Slower but complete snapshot
- Standalone restore capability
### Recommended Schedule
```
Monday-Saturday: Incremental (02:00 AM)
Sunday: Full backup (02:00 AM)
```
To implement this, modify the `.timer` file's `OnCalendar` line:
```ini
OnCalendar=Mon-Sat *-*-* 02:00:00
OnCalendar=Sun *-*-* 02:00:00
```
## Monitoring and Alerts
### Check Backup Status
```bash
# List recent backups
sudo b2 list-file-names nextcloud-backups nextcloud-backups/
# Get total backup size
sudo b2 list-file-names nextcloud-backups nextcloud-backups/ | \
jq '[.[] | .contentLength] | add | . / 1024 / 1024 / 1024' \
echo "Total: $size GB"
```
### Monitor via Systemd
```bash
# Check last 10 backup runs
sudo journalctl -u nextcloud-backup.service --reverse | head -50
# Count successful vs failed backups
sudo journalctl -u nextcloud-backup.service | grep "SUCCESS\|ERROR" | wc -l
```
### Set up Email Alerts (Optional)
Install and configure `postfix` or `ssmtp`, then add to the backup script:
```bash
# On backup failure, send email
if ! run_backup; then
echo "Nextcloud backup failed at $(date)" | \
mail -s "Nextcloud Backup Failed" admin@example.com
fi
```
## Restoring from Backup
### List Available Backups
```bash
b2 list-file-names nextcloud-backups nextcloud-backups/ | \
jq -r '.fileName + " (" + (.contentLength | . / 1024 / 1024 | floor | tostring) + " MB)"'
```
### Restore Database
```bash
# Download database backup
b2 download-file-by-id nextcloud-backups \
<file-id> ./nextcloud-db-backup.sql.zst
# Restore to database
zstd -d nextcloud-db-backup.sql.zst -c | \
mysql -u root -p nextcloud
# Or for PostgreSQL:
zstd -d nextcloud-db-backup.sql.zst -c | \
psql -U postgres nextcloud
```
### Restore Files
```bash
# Download file archive
b2 download-file-by-id nextcloud-backups \
<file-id> ./nextcloud-files-backup.tar.zst
# Extract to temporary location
mkdir -p /tmp/nextcloud-restore
tar --zstd -xf nextcloud-files-backup.tar.zst -C /tmp/nextcloud-restore
# Review and copy back
cp -r /tmp/nextcloud-restore/srv/docker/nextcloud/data/* \
/srv/docker/nextcloud/data/
```
## Troubleshooting
### B2 Authentication Errors
```bash
# Clear cached auth
rm ~/.b2_account_info
# Re-authenticate
b2 authorize-account YOUR_ACCOUNT_ID YOUR_APP_KEY
# Test connection
b2 account-info
```
### Backup Size Issues
If backups are too large:
1. Check for duplicate/cache files in Nextcloud data
2. Enable deduplication in B2 console
3. Increase `BACKUP_RETENTION_DAYS` to allow fewer concurrent backups
### Permission Denied Errors
Ensure the backup script runs as root and has access to:
- Docker socket (for database dumps)
- Nextcloud data directory
- Temporary directory
### Database Connection Issues
Verify database credentials and that the container is running:
```bash
# Check if database container is running
docker-compose -f /path/to/nextcloud/docker-compose.yml ps
# Test database connection
docker-compose -f /path/to/nextcloud/docker-compose.yml exec -T mariadb \
mysql -u nextcloud -p${NEXTCLOUD_DB_PASSWORD} -e "SELECT 1"
```
## Cost Optimization
### Backblaze B2 Pricing
As of 2024:
- Storage: $0.006/GB/month
- Downloads: $0.003/GB (first 1GB free daily)
- API calls: $0.0004 per 1000 calls
### Optimize Costs
1. **Compression**: zstd reduces size by 40-70%
2. **Retention**: Keep only 30 days of backups
3. **Incremental**: Daily incremental + weekly full = smaller total size
4. **Deduplication**: Enable in B2 for identical blocks
Example: 100GB Nextcloud → ~30-40GB compressed → $0.18-0.24/month storage
## Maintenance
### Monthly Checks
```bash
# Verify backup integrity
sudo /opt/nextcloud-backup/nextcloud-backup-to-backblaze.sh status
# Check logs for errors
sudo grep "ERROR" /opt/nextcloud-backup/.backups/logs/*.log
# Verify B2 credentials still valid
sudo b2 account-info
```
### Update Schedule
- Check for B2 CLI updates monthly
- Review Backblaze pricing changes quarterly
- Test restore procedure every 6 months
## Advanced Configuration
### Custom Backup Schedule
Edit `/etc/systemd/system/nextcloud-backup.timer`:
```ini
# Run at 2 AM and 2 PM daily
OnCalendar=*-*-* 02:00:00
OnCalendar=*-*-* 14:00:00
# Or more complex: Mon-Fri 2 AM, weekends 4 AM
OnCalendar=Mon-Fri *-*-* 02:00:00
OnCalendar=Sat,Sun *-*-* 04:00:00
```
Then reload:
```bash
sudo systemctl daemon-reload
sudo systemctl restart nextcloud-backup.timer
```
### Exclude Directories
Modify the `get_changed_files()` function in the script:
```bash
find "${NEXTCLOUD_DATA_PATH}" -type f \
! -path '*/.*' \
! -path '*/cache/*' \
! -path '*/preview/*' \
> "$manifest_file"
```
### Pre/Post Backup Hooks
Add to the script before/after backup:
```bash
# Before backup
if [ -x /opt/nextcloud-backup/pre-backup-hook.sh ]; then
/opt/nextcloud-backup/pre-backup-hook.sh
fi
# ... backup logic ...
# After backup
if [ -x /opt/nextcloud-backup/post-backup-hook.sh ]; then
/opt/nextcloud-backup/post-backup-hook.sh
fi
```
## References
- [Backblaze B2 Documentation](https://www.backblaze.com/b2/docs/)
- [B2 CLI Reference](https://github.com/Backblaze/B2_Command_Line_Tool)
- [Nextcloud Backup Guide](https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/occ/maintenance.html)
- [systemd Timer Documentation](https://www.freedesktop.org/software/systemd/man/systemd.timer.html)
## Support
For issues:
1. Check logs: `journalctl -u nextcloud-backup.service -n 100`
2. Verify configuration: `cat /opt/nextcloud-backup/.env`
3. Test manually: `sudo /opt/nextcloud-backup/nextcloud-backup-to-backblaze.sh status`
4. Check B2 credentials: `b2 account-info`