feat: Add backup and security hardening
This commit is contained in:
568
Backup-Setup.md
Normal file
568
Backup-Setup.md
Normal file
@@ -0,0 +1,568 @@
|
||||
# Backup Setup Guide
|
||||
|
||||
Complete guide for the automated backup system on Gentoo workstation.
|
||||
|
||||
## Overview
|
||||
|
||||
This backup system provides automated, network-triggered backups to a NAS via SSH/rsync with multiple safeguards against backup spam and comprehensive logging.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Features](#features)
|
||||
- [Architecture](#architecture)
|
||||
- [Installation](#installation)
|
||||
- [Configuration](#configuration)
|
||||
- [Usage](#usage)
|
||||
- [Backup Types](#backup-types)
|
||||
- [Network Trigger](#network-trigger)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
- [Advanced Configuration](#advanced-configuration)
|
||||
|
||||
## Features
|
||||
|
||||
✅ **Four backup types**: Full system, home directory, incremental, configs
|
||||
✅ **Network-triggered backups**: Auto-backup when NAS detected
|
||||
✅ **Multiple safeguards**: Rate limiting, cooldown, lock files
|
||||
✅ **Interactive management**: Easy-to-use CLI interface
|
||||
✅ **Comprehensive logging**: Track all backup operations
|
||||
✅ **ZSH completion**: Tab completion for all commands
|
||||
✅ **User-space operation**: No sudo required for backups
|
||||
|
||||
## Architecture
|
||||
|
||||
### Components
|
||||
|
||||
```
|
||||
/usr/local/bin/
|
||||
├── backup-setup # Interactive management script
|
||||
├── backup-full # Full system backup
|
||||
├── backup-home # Home directory backup
|
||||
├── backup-incremental # Incremental backup
|
||||
├── backup-configs # Configuration files backup
|
||||
└── backup-trigger # Network trigger daemon
|
||||
|
||||
/etc/
|
||||
├── backup.conf # Configuration file
|
||||
└── init.d/
|
||||
└── backup-monitor # OpenRC service
|
||||
|
||||
~/.local/var/
|
||||
├── log/
|
||||
│ ├── backup.log # Backup operations log
|
||||
│ └── backup-monitor.log # Monitor daemon log
|
||||
└── backup/
|
||||
├── last-backup # Last backup timestamp
|
||||
├── last-check # Last NAS check timestamp
|
||||
└── backup-in-progress # Active backup marker
|
||||
```
|
||||
|
||||
### Backup Flow
|
||||
|
||||
```
|
||||
1. Network trigger detects NAS
|
||||
↓
|
||||
2. Check safeguards (cooldown, rate limit, lock files)
|
||||
↓
|
||||
3. Trigger backup in background
|
||||
↓
|
||||
4. Backup runs (rsync to NAS)
|
||||
↓
|
||||
5. Update timestamps and markers
|
||||
↓
|
||||
6. Wait for next check (5 minutes)
|
||||
```
|
||||
|
||||
## Installation
|
||||
|
||||
### Prerequisites
|
||||
|
||||
```bash
|
||||
# Required packages
|
||||
emerge -av net-misc/rsync net-misc/openssh
|
||||
|
||||
# Verify NAS is accessible
|
||||
ping 192.168.2.171
|
||||
ssh alexander@192.168.2.171
|
||||
```
|
||||
|
||||
### Install Scripts
|
||||
|
||||
```bash
|
||||
cd ~/repository/git.hinrichs.dev/alexander/claude/lenovo-gentoo/scripts/backup-setup
|
||||
sudo ./INSTALL.sh
|
||||
```
|
||||
|
||||
This will:
|
||||
1. Copy all backup scripts to `/usr/local/bin/`
|
||||
2. Set execute permissions
|
||||
3. Create log and state directories
|
||||
4. Show next steps
|
||||
|
||||
### Configure Backup
|
||||
|
||||
```bash
|
||||
# Copy example config
|
||||
sudo cp /usr/local/share/backup-setup/backup.conf.example /etc/backup.conf
|
||||
|
||||
# Edit configuration
|
||||
sudo nvim /etc/backup.conf
|
||||
```
|
||||
|
||||
### Install Network Trigger (Optional)
|
||||
|
||||
```bash
|
||||
# Install ZSH completion
|
||||
sudo cp _backup-setup /usr/local/share/zsh/site-functions/
|
||||
|
||||
# Install network trigger service
|
||||
sudo cp backup-trigger /usr/local/bin/
|
||||
sudo chmod +x /usr/local/bin/backup-trigger
|
||||
sudo cp backup-monitor /etc/init.d/
|
||||
sudo chmod +x /etc/init.d/backup-monitor
|
||||
|
||||
# Enable at boot
|
||||
sudo rc-update add backup-monitor default
|
||||
|
||||
# Start service
|
||||
sudo rc-service backup-monitor start
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Basic Settings
|
||||
|
||||
Edit `/etc/backup.conf`:
|
||||
|
||||
```bash
|
||||
# NAS Connection
|
||||
NAS_HOST="192.168.2.171"
|
||||
NAS_PORT="22"
|
||||
NAS_USER="alexander"
|
||||
NAS_PATH="/DATA/Backup/lenovo-gentoo-machine"
|
||||
|
||||
# Backup Settings
|
||||
RETENTION_DAYS="30"
|
||||
RETENTION_COUNT="10"
|
||||
BACKUP_COOLDOWN="3600" # 1 hour between backups
|
||||
```
|
||||
|
||||
### Exclusion Patterns
|
||||
|
||||
Add paths to exclude from backups:
|
||||
|
||||
```bash
|
||||
EXCLUDE_PATTERNS=(
|
||||
"/dev/*"
|
||||
"/proc/*"
|
||||
"/sys/*"
|
||||
"/tmp/*"
|
||||
"/run/*"
|
||||
"/mnt/*"
|
||||
"/media/*"
|
||||
"/var/tmp/portage/*"
|
||||
"/var/cache/*"
|
||||
"/.cache/*"
|
||||
"/home/*/.cache/*"
|
||||
"/home/*/Downloads/*.iso"
|
||||
)
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Interactive Management
|
||||
|
||||
```bash
|
||||
# Show backup status
|
||||
backup-setup status
|
||||
|
||||
# Trigger manual backup
|
||||
backup-setup backup [TYPE]
|
||||
|
||||
# List backups on NAS
|
||||
backup-setup list
|
||||
|
||||
# View recent logs
|
||||
backup-setup logs
|
||||
|
||||
# Test NAS connection
|
||||
backup-setup test
|
||||
```
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# Check backup status
|
||||
$ backup-setup
|
||||
========================================
|
||||
Backup Status
|
||||
========================================
|
||||
|
||||
Configuration:
|
||||
NAS: alexander@192.168.2.171:22
|
||||
Path: /DATA/Backup/lenovo-gentoo-machine
|
||||
|
||||
Last backup: 2025-11-07 20:48:32 (1h ago)
|
||||
|
||||
✓ NAS is reachable at alexander@192.168.2.171:22
|
||||
✓ Available space on NAS: 2.1T
|
||||
|
||||
# Run config backup
|
||||
$ backup-setup backup configs
|
||||
ℹ Starting configs backup...
|
||||
✓ Config backup completed successfully
|
||||
Backup size: 26MB
|
||||
Duration: 3s
|
||||
|
||||
# View logs
|
||||
$ backup-setup logs
|
||||
[2025-11-07 20:48:29] Starting config backup...
|
||||
[2025-11-07 20:48:32] Backup completed: 26MB in 3s
|
||||
```
|
||||
|
||||
## Backup Types
|
||||
|
||||
### 1. Full System Backup
|
||||
|
||||
**What it backs up**: Entire system excluding caches and temporary files
|
||||
|
||||
**Size**: Large (depends on system, typically 20-50GB)
|
||||
|
||||
**Duration**: Long (20-60 minutes)
|
||||
|
||||
**Use case**: Complete system restore, major changes
|
||||
|
||||
```bash
|
||||
backup-setup backup full
|
||||
```
|
||||
|
||||
**Exclusions**:
|
||||
- `/dev`, `/proc`, `/sys`, `/tmp`, `/run`
|
||||
- `/mnt`, `/media`
|
||||
- `/var/tmp/portage/*` (Portage build directory)
|
||||
- `/var/cache/*`
|
||||
- All `.cache` directories
|
||||
|
||||
### 2. Home Directory Backup
|
||||
|
||||
**What it backs up**: `/home/alexander` only
|
||||
|
||||
**Size**: Medium (depends on data, typically 10-50GB)
|
||||
|
||||
**Duration**: Medium (10-30 minutes)
|
||||
|
||||
**Use case**: Personal files, documents, projects
|
||||
|
||||
```bash
|
||||
backup-setup backup home
|
||||
```
|
||||
|
||||
**Exclusions**:
|
||||
- `.cache` directories
|
||||
- Browser caches
|
||||
- `Downloads/*.iso`, `Downloads/*.mp4`, `Downloads/*.mkv`
|
||||
|
||||
### 3. Incremental Backup
|
||||
|
||||
**What it backs up**: Only changed files since last backup
|
||||
|
||||
**Size**: Small (only changes)
|
||||
|
||||
**Duration**: Fast (1-10 minutes)
|
||||
|
||||
**Use case**: Daily backups, minimal bandwidth
|
||||
|
||||
```bash
|
||||
backup-setup backup incremental
|
||||
```
|
||||
|
||||
**How it works**: Uses rsync's `--link-dest` to hardlink unchanged files,
|
||||
saving space and time.
|
||||
|
||||
### 4. Configuration Backup
|
||||
|
||||
**What it backs up**: System configs, dotfiles, scripts
|
||||
|
||||
**Size**: Tiny (typically <100MB)
|
||||
|
||||
**Duration**: Very fast (<1 minute)
|
||||
|
||||
**Use case**: Quick config snapshots, testing changes
|
||||
|
||||
```bash
|
||||
backup-setup backup configs
|
||||
```
|
||||
|
||||
**Includes**:
|
||||
- `/etc` (system configuration)
|
||||
- `/home/alexander/.config` (user configs)
|
||||
- `/home/alexander/.local` (local data)
|
||||
- `/etc/portage` (Gentoo package config)
|
||||
- `/usr/local/bin` (custom scripts)
|
||||
- `/home/alexander/.zshrc`, `.bashrc`, etc.
|
||||
|
||||
## Network Trigger
|
||||
|
||||
### How It Works
|
||||
|
||||
The network trigger monitors for NAS availability and automatically triggers
|
||||
backups when the NAS is detected. It has multiple safeguards to prevent
|
||||
backup spam:
|
||||
|
||||
1. **Rate Limiting**: Checks for NAS only every 5 minutes
|
||||
2. **Cooldown Period**: Enforces minimum 1 hour between backups
|
||||
3. **Lock File Detection**: Skips if another backup is running
|
||||
4. **In-Progress Marker**: Tracks active backups
|
||||
5. **Stale Marker Cleanup**: Removes markers older than 6 hours
|
||||
|
||||
### Monitor Status
|
||||
|
||||
```bash
|
||||
# Check service status
|
||||
sudo rc-service backup-monitor status
|
||||
|
||||
# View monitor logs
|
||||
tail -f ~/.local/var/log/backup-monitor.log
|
||||
|
||||
# View backup logs
|
||||
tail -f ~/.local/var/log/backup.log
|
||||
```
|
||||
|
||||
### Example Log Output
|
||||
|
||||
```
|
||||
[2025-11-07 21:01:43] ==========================================
|
||||
[2025-11-07 21:01:43] Backup monitor started
|
||||
[2025-11-07 21:01:43] Monitoring for NAS: 192.168.2.171
|
||||
[2025-11-07 21:01:43] Check interval: 5 minutes
|
||||
[2025-11-07 21:01:43] Backup cooldown: 3600s (1h)
|
||||
[2025-11-07 21:01:43] ==========================================
|
||||
[2025-11-07 21:06:00] NAS is available at 192.168.2.171
|
||||
[2025-11-07 21:06:00] Cooldown passed, backup allowed
|
||||
[2025-11-07 21:06:00] NAS detected, triggering incremental backup...
|
||||
[2025-11-07 21:06:00] Backup started in background (PID: 12345)
|
||||
```
|
||||
|
||||
### Service Management
|
||||
|
||||
```bash
|
||||
# Start monitor
|
||||
sudo rc-service backup-monitor start
|
||||
|
||||
# Stop monitor
|
||||
sudo rc-service backup-monitor stop
|
||||
|
||||
# Restart monitor
|
||||
sudo rc-service backup-monitor restart
|
||||
|
||||
# Enable at boot
|
||||
sudo rc-update add backup-monitor default
|
||||
|
||||
# Disable at boot
|
||||
sudo rc-update del backup-monitor default
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Backup Fails with Permission Denied
|
||||
|
||||
**Problem**: Cannot write to log files or state files
|
||||
|
||||
**Solution**: Ensure directories exist and are writable:
|
||||
|
||||
```bash
|
||||
mkdir -p ~/.local/var/log
|
||||
mkdir -p ~/.local/var/backup
|
||||
chmod 755 ~/.local/var/log
|
||||
chmod 755 ~/.local/var/backup
|
||||
```
|
||||
|
||||
### Lock File Prevents Backup
|
||||
|
||||
**Problem**: Backup shows "already running" but no backup is active
|
||||
|
||||
**Solution**: Remove stale lock file:
|
||||
|
||||
```bash
|
||||
rm /tmp/backup-*.lock
|
||||
```
|
||||
|
||||
### NAS Not Detected
|
||||
|
||||
**Problem**: Network trigger doesn't detect NAS
|
||||
|
||||
**Check connection**:
|
||||
```bash
|
||||
# Test ping
|
||||
ping -c 3 192.168.2.171
|
||||
|
||||
# Test SSH
|
||||
ssh alexander@192.168.2.171 "echo test"
|
||||
|
||||
# Check monitor logs
|
||||
tail -20 ~/.local/var/log/backup-monitor.log
|
||||
```
|
||||
|
||||
**Verify config**:
|
||||
```bash
|
||||
cat /etc/backup.conf
|
||||
```
|
||||
|
||||
### Backup Too Slow
|
||||
|
||||
**Problem**: Backup takes too long
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Use incremental backup instead of full:
|
||||
```bash
|
||||
backup-setup backup incremental
|
||||
```
|
||||
|
||||
2. Add more exclusions to `/etc/backup.conf`:
|
||||
```bash
|
||||
EXCLUDE_PATTERNS+=(
|
||||
"/home/*/Videos/*"
|
||||
"/home/*/Music/*"
|
||||
"/home/*/.local/share/Steam/*"
|
||||
)
|
||||
```
|
||||
|
||||
3. Check network speed:
|
||||
```bash
|
||||
# Test transfer speed
|
||||
dd if=/dev/zero bs=1M count=100 | \
|
||||
ssh alexander@192.168.2.171 "cat > /dev/null"
|
||||
```
|
||||
|
||||
### Cooldown Too Short/Long
|
||||
|
||||
**Problem**: Backups happen too frequently or not frequently enough
|
||||
|
||||
**Solution**: Adjust `BACKUP_COOLDOWN` in `/etc/backup.conf`:
|
||||
|
||||
```bash
|
||||
# 30 minutes
|
||||
BACKUP_COOLDOWN="1800"
|
||||
|
||||
# 2 hours
|
||||
BACKUP_COOLDOWN="7200"
|
||||
|
||||
# 6 hours
|
||||
BACKUP_COOLDOWN="21600"
|
||||
```
|
||||
|
||||
Then restart the monitor:
|
||||
```bash
|
||||
sudo rc-service backup-monitor restart
|
||||
```
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### Custom Backup Schedules
|
||||
|
||||
Instead of using the network trigger, use cron for scheduled backups:
|
||||
|
||||
```bash
|
||||
# Edit crontab
|
||||
crontab -e
|
||||
|
||||
# Daily incremental at 2 AM
|
||||
0 2 * * * /usr/local/bin/backup-incremental
|
||||
|
||||
# Weekly full backup on Sunday at 3 AM
|
||||
0 3 * * 0 /usr/local/bin/backup-full
|
||||
|
||||
# Hourly config backup
|
||||
0 * * * * /usr/local/bin/backup-configs
|
||||
```
|
||||
|
||||
### Multiple NAS Destinations
|
||||
|
||||
Create separate configs for different NAS hosts:
|
||||
|
||||
```bash
|
||||
# Create configs
|
||||
sudo cp /etc/backup.conf /etc/backup-nas1.conf
|
||||
sudo cp /etc/backup.conf /etc/backup-nas2.conf
|
||||
|
||||
# Edit each config with different NAS_HOST and NAS_PATH
|
||||
|
||||
# Create wrapper scripts
|
||||
cat > ~/.local/bin/backup-nas1 <<'EOF'
|
||||
#!/bin/sh
|
||||
CONFIG_FILE="/etc/backup-nas1.conf" /usr/local/bin/backup-incremental "$@"
|
||||
EOF
|
||||
|
||||
cat > ~/.local/bin/backup-nas2 <<'EOF'
|
||||
#!/bin/sh
|
||||
CONFIG_FILE="/etc/backup-nas2.conf" /usr/local/bin/backup-incremental "$@"
|
||||
EOF
|
||||
|
||||
chmod +x ~/.local/bin/backup-nas*
|
||||
```
|
||||
|
||||
### Backup Verification
|
||||
|
||||
Add verification to backup scripts by editing them:
|
||||
|
||||
```bash
|
||||
# After rsync in any backup script, add:
|
||||
|
||||
# Verify backup
|
||||
info "Verifying backup..."
|
||||
if ssh -p "$NAS_PORT" "${NAS_USER}@${NAS_HOST}" \
|
||||
"test -d '$BACKUP_DEST' && du -sh '$BACKUP_DEST'"; then
|
||||
success "Backup verified"
|
||||
else
|
||||
error "Backup verification failed"
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
### Email Notifications
|
||||
|
||||
Install and configure email notifications:
|
||||
|
||||
```bash
|
||||
# Install mail client
|
||||
emerge -av mail-client/mailx
|
||||
|
||||
# Add to backup scripts after completion:
|
||||
echo "Backup completed at $(date)" | \
|
||||
mail -s "Backup: $HOSTNAME" your.email@example.com
|
||||
```
|
||||
|
||||
### Encryption
|
||||
|
||||
Encrypt backups using gpg before sending to NAS:
|
||||
|
||||
```bash
|
||||
# Install gpg
|
||||
emerge -av app-crypt/gnupg
|
||||
|
||||
# Generate key
|
||||
gpg --gen-key
|
||||
|
||||
# Modify backup scripts to encrypt:
|
||||
# Replace rsync command with:
|
||||
tar czf - / --exclude-from=<(printf '%s\n' "${EXCLUDE_PATTERNS[@]}") | \
|
||||
gpg --encrypt --recipient your@email.com | \
|
||||
ssh -p "$NAS_PORT" "${NAS_USER}@${NAS_HOST}" \
|
||||
"cat > '$BACKUP_DEST/backup.tar.gz.gpg'"
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Test Restores**: Regularly test restoring from backups
|
||||
2. **Monitor Logs**: Check backup logs weekly
|
||||
3. **Verify Space**: Ensure NAS has adequate free space
|
||||
4. **Update Exclusions**: Review and update exclusion patterns
|
||||
5. **Document Changes**: Keep notes on backup configuration changes
|
||||
6. **3-2-1 Rule**: Keep 3 copies, on 2 media types, 1 offsite
|
||||
|
||||
## See Also
|
||||
|
||||
- [USER-GUIDE.md](USER-GUIDE.md) - Quick reference guide
|
||||
- [CLAUDE.md](CLAUDE.md) - System overview
|
||||
- `/etc/backup.conf` - Backup configuration
|
||||
- `~/.local/var/log/backup.log` - Backup logs
|
||||
Reference in New Issue
Block a user