This implements the core functionality for executing remote commands through the dashboard and providing real-time visual feedback to users. Key Features: - Remote service control (start/stop/restart) via existing keyboard shortcuts - System rebuild command with maintenance mode integration - Real-time visual feedback with service status transitions - ZMQ command protocol extension for service and system operations Implementation Details: - Extended AgentCommand enum with ServiceControl and SystemRebuild variants - Added agent-side handlers for systemctl and nixos-rebuild execution - Implemented command status tracking system for visual feedback - Enhanced services widget to show progress states (⏳ restarting) - Integrated command execution with existing keyboard navigation Keyboard Controls: - Services Panel: Space (start/stop), R (restart) - System Panel: R (nixos-rebuild switch) - Backup Panel: B (trigger backup) Technical Architecture: - Command flow: UI → Dashboard → ZMQ → Agent → systemctl/nixos-rebuild - Status tracking: InProgress/Success/Failed states with visual indicators - Maintenance mode: Automatic /tmp/cm-maintenance file management - Service feedback: Icon transitions (● → ⏳ → ● with status text)
287 lines
10 KiB
Markdown
287 lines
10 KiB
Markdown
# CM Dashboard - Infrastructure Monitoring TUI
|
|
|
|
## Overview
|
|
|
|
A high-performance Rust-based TUI dashboard for monitoring CMTEC infrastructure. Built to replace Glance with a custom solution tailored for our specific monitoring needs and ZMQ-based metric collection.
|
|
|
|
## Implementation Strategy
|
|
|
|
### Current Implementation Status
|
|
|
|
**System Panel Enhancement - COMPLETED** ✅
|
|
|
|
All system panel features successfully implemented:
|
|
- ✅ **NixOS Collector**: Created collector for version and active users
|
|
- ✅ **System Widget**: Unified widget combining NixOS, CPU, RAM, and Storage
|
|
- ✅ **Build Display**: Shows NixOS build information without codename
|
|
- ✅ **Active Users**: Displays currently logged in users
|
|
- ✅ **Tmpfs Monitoring**: Added /tmp usage to RAM section
|
|
- ✅ **Agent Deployment**: NixOS collector working in production
|
|
|
|
**Keyboard Navigation and Service Management - COMPLETED** ✅
|
|
|
|
All keyboard navigation and service selection features successfully implemented:
|
|
- ✅ **Panel Navigation**: Shift+Tab cycles through visible panels only (System → Services → Backup)
|
|
- ✅ **Service Selection**: Up/Down arrows navigate through parent services with visual cursor
|
|
- ✅ **Focus Management**: Selection highlighting only visible when Services panel focused
|
|
- ✅ **Status Preservation**: Service health colors maintained during selection (green/red icons)
|
|
- ✅ **Smart Panel Switching**: Only cycles through panels with data (backup panel conditional)
|
|
- ✅ **Scroll Support**: All panels support content scrolling with proper overflow indicators
|
|
|
|
**Current Status - October 23, 2025:**
|
|
- All keyboard navigation features working correctly ✅
|
|
- Service selection cursor implemented with focus-aware highlighting ✅
|
|
- Panel scrolling fixed for System, Services, and Backup panels ✅
|
|
- Build display working: "Build: 25.05.20251004.3bcc93c" ✅
|
|
|
|
**Layout Achieved:**
|
|
```
|
|
NixOS:
|
|
Build: 25.05.20251004.3bcc93c # Target (currently shows "unknown")
|
|
Active users: cm, simon
|
|
CPU:
|
|
● Load: 0.02 0.31 0.86 • 3000MHz
|
|
RAM:
|
|
● Usage: 33% 2.6GB/7.6GB
|
|
● /tmp: 0% 0B/2.0GB
|
|
Storage:
|
|
● root (Single):
|
|
├─ ● nvme0n1 W: 1%
|
|
└─ ● 18% 167.4GB/928.2GB
|
|
```
|
|
|
|
**Current Status - October 23, 2025:**
|
|
- System panel layout fully implemented with blue tree symbols ✅
|
|
- Backup panel layout restructured per specification ✅
|
|
- Tree symbols now use consistent blue theming across all panels ✅
|
|
- Overflow handling restored for all widgets ("... and X more") ✅
|
|
- Agent hash display working correctly ✅
|
|
|
|
### Current Keyboard Navigation Implementation
|
|
|
|
**Navigation Controls:**
|
|
- **Tab**: Switch between hosts (cmbox, srv01, srv02, steambox, etc.)
|
|
- **Shift+Tab**: Cycle through visible panels (System → Services → Backup → System)
|
|
- **Up/Down (System/Backup)**: Scroll through panel content
|
|
- **Up/Down (Services)**: Move service selection cursor between parent services
|
|
- **q**: Quit dashboard
|
|
|
|
**Panel-Specific Features:**
|
|
- **System Panel**: Scrollable content with CPU, RAM, Storage details
|
|
- **Services Panel**: Service selection cursor for parent services only (docker, nginx, postgresql, etc.)
|
|
- **Backup Panel**: Scrollable repository list with proper overflow handling
|
|
|
|
**Visual Feedback:**
|
|
- **Focused Panel**: Blue border and title highlighting
|
|
- **Service Selection**: Blue background with preserved status icon colors (green ● for active, red ● for failed)
|
|
- **Focus-Aware Selection**: Selection highlighting only visible when Services panel focused
|
|
- **Dynamic Statusbar**: Context-aware shortcuts based on focused panel
|
|
|
|
### Current Priority: Visual Feedback Implementation
|
|
|
|
**Remote Command Execution - COMPLETED** ✅
|
|
|
|
All core remote command functionality implemented:
|
|
- ✅ **ZMQ Command Protocol**: Extended with ServiceControl and SystemRebuild variants
|
|
- ✅ **Agent Handlers**: systemctl and nixos-rebuild execution with maintenance mode
|
|
- ✅ **Dashboard Integration**: Existing keyboard shortcuts now execute commands
|
|
- ✅ **Command Flow**: UI → Dashboard → ZMQ → Agent → systemctl/nixos-rebuild
|
|
|
|
**Keyboard Controls Working:**
|
|
- **Services Panel**: Space (start/stop), R (restart)
|
|
- **System Panel**: R (nixos-rebuild)
|
|
- **Backup Panel**: B (trigger backup)
|
|
|
|
**Visual Feedback Implementation - IN PROGRESS:**
|
|
|
|
Context-appropriate progress indicators for each panel:
|
|
|
|
**Services Panel** (Service status transitions):
|
|
```
|
|
● nginx active → ⏳ nginx restarting → ● nginx active
|
|
● docker active → ⏳ docker stopping → ● docker inactive
|
|
```
|
|
|
|
**System Panel** (Build progress in NixOS section):
|
|
```
|
|
NixOS:
|
|
Build: 25.05.20251004.3bcc93c → Build: [████████████ ] 65%
|
|
Active users: cm, simon Active users: cm, simon
|
|
```
|
|
|
|
**Backup Panel** (OnGoing status with progress):
|
|
```
|
|
Latest backup: → Latest backup:
|
|
● 2024-10-23 14:32:15 ● OnGoing
|
|
└─ Duration: 1.3m └─ [██████ ] 60%
|
|
```
|
|
|
|
**Remaining Tasks:**
|
|
- Implement visual feedback system for command execution status
|
|
- Add confirmation dialogs for destructive actions
|
|
- Test complete command execution scenarios
|
|
|
|
**Future Enhanced Navigation:**
|
|
- Add Page Up/Down for faster scrolling through long service lists
|
|
- Implement search/filter functionality for services
|
|
- Add jump-to-service shortcuts (first letter navigation)
|
|
|
|
**Future Advanced Features:**
|
|
- Service dependency visualization
|
|
- Historical service status tracking
|
|
- Backup trigger functionality with progress indication
|
|
|
|
## Core Architecture Principles - CRITICAL
|
|
|
|
### Individual Metrics Philosophy
|
|
|
|
**NEW ARCHITECTURE**: Agent collects individual metrics, dashboard composes widgets from those metrics.
|
|
|
|
### Maintenance Mode
|
|
|
|
**Purpose:**
|
|
|
|
- Suppress email notifications during planned maintenance or backups
|
|
- Prevents false alerts when services are intentionally stopped
|
|
|
|
**Implementation:**
|
|
|
|
- Agent checks for `/tmp/cm-maintenance` file before sending notifications
|
|
- File presence suppresses all email notifications while continuing monitoring
|
|
- Dashboard continues to show real status, only notifications are blocked
|
|
|
|
**Usage:**
|
|
|
|
```bash
|
|
# Enable maintenance mode
|
|
touch /tmp/cm-maintenance
|
|
|
|
# Run maintenance tasks (backups, service restarts, etc.)
|
|
systemctl stop service
|
|
# ... maintenance work ...
|
|
systemctl start service
|
|
|
|
# Disable maintenance mode
|
|
rm /tmp/cm-maintenance
|
|
```
|
|
|
|
**NixOS Integration:**
|
|
|
|
- Borgbackup script automatically creates/removes maintenance file
|
|
- Automatic cleanup via trap ensures maintenance mode doesn't stick
|
|
- All cinfiguration are shall be done from nixos config
|
|
|
|
**ARCHITECTURE ENFORCEMENT**:
|
|
|
|
- **ZERO legacy code reuse** - Fresh implementation following ARCHITECT.md exactly
|
|
- **Individual metrics only** - NO grouped metric structures
|
|
- **Reference-only legacy** - Study old functionality, implement new architecture
|
|
- **Clean slate mindset** - Build as if legacy codebase never existed
|
|
|
|
**Implementation Rules**:
|
|
|
|
1. **Individual Metrics**: Each metric is collected, transmitted, and stored individually
|
|
2. **Agent Status Authority**: Agent calculates status for each metric using thresholds
|
|
3. **Dashboard Composition**: Dashboard widgets subscribe to specific metrics by name
|
|
4. **Status Aggregation**: Dashboard aggregates individual metric statuses for widget status
|
|
**Testing & Building**:
|
|
|
|
- **Workspace builds**: `cargo build --workspace` for all testing
|
|
- **Clean compilation**: Remove `target/` between architecture changes
|
|
- **ZMQ testing**: Test agent-dashboard communication independently
|
|
- **Widget testing**: Verify UI layout matches legacy appearance exactly
|
|
|
|
**NEVER in New Implementation**:
|
|
|
|
- Copy/paste ANY code from legacy backup
|
|
- Calculate status in dashboard widgets
|
|
- Hardcode metric names in widgets (use const arrays)
|
|
|
|
# Important Communication Guidelines
|
|
|
|
NEVER write that you have "successfully implemented" something or generate extensive summary text without first verifying with the user that the implementation is correct. This wastes tokens. Keep responses concise.
|
|
|
|
NEVER implement code without first getting explicit user agreement on the approach. Always ask for confirmation before proceeding with implementation.
|
|
|
|
## Commit Message Guidelines
|
|
|
|
**NEVER mention:**
|
|
|
|
- Claude or any AI assistant names
|
|
- Automation or AI-generated content
|
|
- Any reference to automated code generation
|
|
|
|
**ALWAYS:**
|
|
|
|
- Focus purely on technical changes and their purpose
|
|
- Use standard software development commit message format
|
|
- Describe what was changed and why, not how it was created
|
|
- Write from the perspective of a human developer
|
|
|
|
**Examples:**
|
|
|
|
- ❌ "Generated with Claude Code"
|
|
- ❌ "AI-assisted implementation"
|
|
- ❌ "Automated refactoring"
|
|
- ✅ "Implement maintenance mode for backup operations"
|
|
- ✅ "Restructure storage widget with improved layout"
|
|
- ✅ "Update CPU thresholds to production values"
|
|
|
|
## NixOS Configuration Updates
|
|
|
|
When code changes are made to cm-dashboard, the NixOS configuration at `~/nixosbox` must be updated to deploy the changes.
|
|
|
|
### Update Process
|
|
|
|
1. **Get Latest Commit Hash**
|
|
|
|
```bash
|
|
git log -1 --format="%H"
|
|
```
|
|
|
|
2. **Update NixOS Configuration**
|
|
Edit `~/nixosbox/hosts/common/cm-dashboard.nix`:
|
|
|
|
```nix
|
|
src = pkgs.fetchgit {
|
|
url = "https://gitea.cmtec.se/cm/cm-dashboard.git";
|
|
rev = "NEW_COMMIT_HASH_HERE";
|
|
sha256 = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="; # Placeholder
|
|
};
|
|
```
|
|
|
|
3. **Get Correct Source Hash**
|
|
Build with placeholder hash to get the actual hash:
|
|
|
|
```bash
|
|
cd ~/nixosbox
|
|
nix-build --no-out-link -E 'with import <nixpkgs> {}; fetchgit {
|
|
url = "https://gitea.cmtec.se/cm/cm-dashboard.git";
|
|
rev = "NEW_COMMIT_HASH";
|
|
sha256 = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
|
|
}' 2>&1 | grep "got:"
|
|
```
|
|
|
|
Example output:
|
|
|
|
```
|
|
error: hash mismatch in fixed-output derivation '/nix/store/...':
|
|
specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
|
|
got: sha256-x8crxNusOUYRrkP9mYEOG+Ga3JCPIdJLkEAc5P1ZxdQ=
|
|
```
|
|
|
|
4. **Update Configuration with Correct Hash**
|
|
Replace the placeholder with the hash from the error message (the "got:" line).
|
|
|
|
5. **Commit NixOS Configuration**
|
|
|
|
```bash
|
|
cd ~/nixosbox
|
|
git add hosts/common/cm-dashboard.nix
|
|
git commit -m "Update cm-dashboard to latest version (SHORT_HASH)"
|
|
git push
|
|
```
|
|
|
|
6. **Rebuild System**
|
|
The user handles the system rebuild step - this cannot be automated.
|