All checks were successful
Build and Release / build-and-release (push) Successful in 2m34s
Replace ZMQ-based service start/stop commands with SSH execution in tmux popups. This provides better user feedback with real-time systemctl output while eliminating blocking operations from the main message processing loop. Changes: - Service start/stop now use SSH with progress display - Added backup functionality with 'B' key - Preserved transitional icons (↑/↓) for immediate visual feedback - Removed all ZMQ service control commands and handlers - Updated configuration to include backup_alias setting - All operations (rebuild, backup, services) now use consistent SSH interface This ensures stable heartbeat processing while providing superior user experience with live command output and service status feedback.
190 lines
6.4 KiB
Markdown
190 lines
6.4 KiB
Markdown
# CM Dashboard - Infrastructure Monitoring TUI
|
|
|
|
## Overview
|
|
|
|
A high-performance Rust-based TUI dashboard for monitoring CMTEC infrastructure. Built with ZMQ-based metric collection and individual metrics architecture.
|
|
|
|
## Current Features
|
|
|
|
### Core Functionality
|
|
- **Real-time Monitoring**: CPU, RAM, Storage, and Service status
|
|
- **Service Management**: Start/stop services with user-stopped tracking
|
|
- **Multi-host Support**: Monitor multiple servers from single dashboard
|
|
- **NixOS Integration**: System rebuild via SSH + tmux popup
|
|
- **Backup Monitoring**: Borgbackup status and scheduling
|
|
|
|
### User-Stopped Service Tracking
|
|
- Services stopped via dashboard are marked as "user-stopped"
|
|
- User-stopped services report Status::OK instead of Warning
|
|
- Prevents false alerts during intentional maintenance
|
|
- Persistent storage survives agent restarts
|
|
- Automatic flag clearing when services are restarted via dashboard
|
|
|
|
### Custom Service Logs
|
|
- Configure service-specific log file paths per host in dashboard config
|
|
- Press `L` on any service to view custom log files via `tail -f`
|
|
- Configuration format in dashboard config:
|
|
```toml
|
|
[service_logs]
|
|
hostname1 = [
|
|
{ service_name = "nginx", log_file_path = "/var/log/nginx/access.log" },
|
|
{ service_name = "app", log_file_path = "/var/log/myapp/app.log" }
|
|
]
|
|
hostname2 = [
|
|
{ service_name = "database", log_file_path = "/var/log/postgres/postgres.log" }
|
|
]
|
|
```
|
|
|
|
### Service Management
|
|
- **Direct Control**: Arrow keys (↑↓) or vim keys (j/k) navigate services
|
|
- **Service Actions**:
|
|
- `s` - Start service (sends UserStart command)
|
|
- `S` - Stop service (sends UserStop command)
|
|
- `J` - Show service logs (journalctl in tmux popup)
|
|
- `L` - Show custom log files (tail -f custom paths in tmux popup)
|
|
- `R` - Rebuild current host
|
|
- **Visual Status**: Green ● (active), Yellow ◐ (inactive), Red ◯ (failed)
|
|
- **Transitional Icons**: Blue arrows during operations
|
|
|
|
### Navigation
|
|
- **Tab**: Switch between hosts
|
|
- **↑↓ or j/k**: Select services
|
|
- **s**: Start selected service (UserStart)
|
|
- **S**: Stop selected service (UserStop)
|
|
- **J**: Show service logs (journalctl)
|
|
- **L**: Show custom log files
|
|
- **R**: Rebuild current host
|
|
- **B**: Run backup on current host
|
|
- **q**: Quit dashboard
|
|
|
|
## Core Architecture Principles
|
|
|
|
### Individual Metrics Philosophy
|
|
- Agent collects individual metrics, dashboard composes widgets
|
|
- Each metric collected, transmitted, and stored individually
|
|
- Agent calculates status for each metric using thresholds
|
|
- Dashboard aggregates individual metric statuses for widget status
|
|
|
|
### Maintenance Mode
|
|
- Agent checks for `/tmp/cm-maintenance` file before sending notifications
|
|
- File presence suppresses all email notifications while continuing monitoring
|
|
- Dashboard continues to show real status, only notifications are blocked
|
|
|
|
Usage:
|
|
```bash
|
|
# Enable maintenance mode
|
|
touch /tmp/cm-maintenance
|
|
|
|
# Run maintenance tasks
|
|
systemctl stop service
|
|
# ... maintenance work ...
|
|
systemctl start service
|
|
|
|
# Disable maintenance mode
|
|
rm /tmp/cm-maintenance
|
|
```
|
|
|
|
## Development and Deployment Architecture
|
|
|
|
### Development Path
|
|
- **Location:** `~/projects/cm-dashboard`
|
|
- **Purpose:** Development workflow only - for committing new code
|
|
- **Access:** Only for developers to commit changes
|
|
|
|
### Deployment Path
|
|
- **Location:** `/var/lib/cm-dashboard/nixos-config`
|
|
- **Purpose:** Production deployment only - agent clones/pulls from git
|
|
- **Workflow:** git pull → `/var/lib/cm-dashboard/nixos-config` → nixos-rebuild
|
|
|
|
### Git Flow
|
|
```
|
|
Development: ~/projects/cm-dashboard → git commit → git push
|
|
Deployment: git pull → /var/lib/cm-dashboard/nixos-config → rebuild
|
|
```
|
|
|
|
## Automated Binary Release System
|
|
|
|
CM Dashboard uses automated binary releases instead of source builds.
|
|
|
|
### Creating New Releases
|
|
```bash
|
|
cd ~/projects/cm-dashboard
|
|
git tag v0.1.X
|
|
git push origin v0.1.X
|
|
```
|
|
|
|
This automatically:
|
|
- Builds static binaries with `RUSTFLAGS="-C target-feature=+crt-static"`
|
|
- Creates GitHub-style release with tarball
|
|
- Uploads binaries via Gitea API
|
|
|
|
### NixOS Configuration Updates
|
|
Edit `~/projects/nixosbox/hosts/services/cm-dashboard.nix`:
|
|
|
|
```nix
|
|
version = "v0.1.X";
|
|
src = pkgs.fetchurl {
|
|
url = "https://gitea.cmtec.se/cm/cm-dashboard/releases/download/${version}/cm-dashboard-linux-x86_64.tar.gz";
|
|
sha256 = "sha256-NEW_HASH_HERE";
|
|
};
|
|
```
|
|
|
|
### Get Release Hash
|
|
```bash
|
|
cd ~/projects/nixosbox
|
|
nix-build --no-out-link -E 'with import <nixpkgs> {}; fetchurl {
|
|
url = "https://gitea.cmtec.se/cm/cm-dashboard/releases/download/v0.1.X/cm-dashboard-linux-x86_64.tar.gz";
|
|
sha256 = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
|
|
}' 2>&1 | grep "got:"
|
|
```
|
|
|
|
### Building
|
|
|
|
**Testing & Building:**
|
|
- **Workspace builds**: `nix-shell -p openssl pkg-config --run "cargo build --workspace"`
|
|
- **Clean compilation**: Remove `target/` between major changes
|
|
|
|
## Important Communication Guidelines
|
|
|
|
Keep responses concise and focused. Avoid extensive implementation summaries unless requested.
|
|
|
|
## Commit Message Guidelines
|
|
|
|
**NEVER mention:**
|
|
- Claude or any AI assistant names
|
|
- Automation or AI-generated content
|
|
- Any reference to automated code generation
|
|
|
|
**ALWAYS:**
|
|
- Focus purely on technical changes and their purpose
|
|
- Use standard software development commit message format
|
|
- Describe what was changed and why, not how it was created
|
|
- Write from the perspective of a human developer
|
|
|
|
**Examples:**
|
|
- ❌ "Generated with Claude Code"
|
|
- ❌ "AI-assisted implementation"
|
|
- ❌ "Automated refactoring"
|
|
- ✅ "Implement maintenance mode for backup operations"
|
|
- ✅ "Restructure storage widget with improved layout"
|
|
- ✅ "Update CPU thresholds to production values"
|
|
|
|
## Implementation Rules
|
|
|
|
1. **Individual Metrics**: Each metric is collected, transmitted, and stored individually
|
|
2. **Agent Status Authority**: Agent calculates status for each metric using thresholds
|
|
3. **Dashboard Composition**: Dashboard widgets subscribe to specific metrics by name
|
|
4. **Status Aggregation**: Dashboard aggregates individual metric statuses for widget status
|
|
|
|
**NEVER:**
|
|
- Copy/paste ANY code from legacy implementations
|
|
- Calculate status in dashboard widgets
|
|
- Hardcode metric names in widgets (use const arrays)
|
|
- Create files unless absolutely necessary for achieving goals
|
|
- Create documentation files unless explicitly requested
|
|
|
|
**ALWAYS:**
|
|
- Prefer editing existing files to creating new ones
|
|
- Follow existing code conventions and patterns
|
|
- Use existing libraries and utilities
|
|
- Follow security best practices |