Christoffer Martinsson
39fc9cd22f
- Create NixOS collector for version and active users detection - Add SystemWidget combining all system information in TODO.md layout - Replace separate CPU/Memory widgets with unified system display - Add tree structure for storage with drive temperature/wear info - Support NixOS version, active users, load averages, memory usage - Follow exact decimal formatting from specification
222 lines
6.8 KiB
Markdown
222 lines
6.8 KiB
Markdown
# CM Dashboard - Infrastructure Monitoring TUI
|
|
|
|
## Overview
|
|
|
|
A high-performance Rust-based TUI dashboard for monitoring CMTEC infrastructure. Built to replace Glance with a custom solution tailored for our specific monitoring needs and ZMQ-based metric collection.
|
|
|
|
## Implementation Strategy
|
|
|
|
### Current Implementation Status
|
|
|
|
**Systemd Collector Optimization - COMPLETED** ✅
|
|
|
|
All phases successfully implemented:
|
|
- ✅ **Phase 1**: Exact name filtering implemented
|
|
- ✅ **Phase 2**: User service collection removed
|
|
- ✅ **Phase 3**: Wildcard pattern support added
|
|
- ✅ **Phase 4**: Status caching and systemctl call optimization completed
|
|
- ❌ **Phase 5**: Skipped (would increase systemctl calls vs current caching)
|
|
|
|
**Performance Results:**
|
|
- Reduced from ~21 systemctl calls to 1 call every 10 seconds (configurable)
|
|
- Fixed RwLock deadlock issues
|
|
- Removed hardcoded discovery intervals
|
|
|
|
### Next Priority: System Panel Enhancement (Based on TODO.md)
|
|
|
|
**Target Layout:**
|
|
```
|
|
NixOS:
|
|
Version: xxxxxxxxxx
|
|
Active users: cm, simon
|
|
CPU:
|
|
● Load: 0.02 0.31 0.86 • 3000 MHz
|
|
RAM:
|
|
● Usage: 33% 2.6GB/7.6GB
|
|
● /tmp: 0% 0B/2.0GB
|
|
Storage:
|
|
● root (Single):
|
|
├─ ● nvme0n1 Temp: 40C Wear: 4%
|
|
└─ ● 8% 75.0GB/906.2GB
|
|
```
|
|
|
|
**Implementation Tasks:**
|
|
1. **NixOS Version Display**
|
|
- Collect system version information
|
|
- Show timestamp/version for latest nixos rebuild
|
|
|
|
2. **Active Users Display**
|
|
- Implement user session detection
|
|
- Show currently logged in/active users
|
|
|
|
3. **System Widget Layout Update**
|
|
- Update dashboard to match new layout specification
|
|
- Integrate NixOS version and user information
|
|
|
|
### Future Priorities
|
|
|
|
**Keyboard Navigation (Dashboard):**
|
|
- Change host switching to "Shift-Tab"
|
|
- Add panel navigation with "Tab"
|
|
- Add scrolling support for overflow content
|
|
|
|
**Remote Execution (Agent/Dashboard):**
|
|
- Dynamic statusbar with context shortcuts
|
|
- Remote nixos rebuild commands
|
|
- Service start/stop/restart controls
|
|
- Backup trigger functionality
|
|
|
|
## Core Architecture Principles - CRITICAL
|
|
|
|
### Individual Metrics Philosophy
|
|
|
|
**NEW ARCHITECTURE**: Agent collects individual metrics, dashboard composes widgets from those metrics.
|
|
|
|
### Maintenance Mode
|
|
|
|
**Purpose:**
|
|
|
|
- Suppress email notifications during planned maintenance or backups
|
|
- Prevents false alerts when services are intentionally stopped
|
|
|
|
**Implementation:**
|
|
|
|
- Agent checks for `/tmp/cm-maintenance` file before sending notifications
|
|
- File presence suppresses all email notifications while continuing monitoring
|
|
- Dashboard continues to show real status, only notifications are blocked
|
|
|
|
**Usage:**
|
|
|
|
```bash
|
|
# Enable maintenance mode
|
|
touch /tmp/cm-maintenance
|
|
|
|
# Run maintenance tasks (backups, service restarts, etc.)
|
|
systemctl stop service
|
|
# ... maintenance work ...
|
|
systemctl start service
|
|
|
|
# Disable maintenance mode
|
|
rm /tmp/cm-maintenance
|
|
```
|
|
|
|
**NixOS Integration:**
|
|
|
|
- Borgbackup script automatically creates/removes maintenance file
|
|
- Automatic cleanup via trap ensures maintenance mode doesn't stick
|
|
- All cinfiguration are shall be done from nixos config
|
|
|
|
**ARCHITECTURE ENFORCEMENT**:
|
|
|
|
- **ZERO legacy code reuse** - Fresh implementation following ARCHITECT.md exactly
|
|
- **Individual metrics only** - NO grouped metric structures
|
|
- **Reference-only legacy** - Study old functionality, implement new architecture
|
|
- **Clean slate mindset** - Build as if legacy codebase never existed
|
|
|
|
**Implementation Rules**:
|
|
|
|
1. **Individual Metrics**: Each metric is collected, transmitted, and stored individually
|
|
2. **Agent Status Authority**: Agent calculates status for each metric using thresholds
|
|
3. **Dashboard Composition**: Dashboard widgets subscribe to specific metrics by name
|
|
4. **Status Aggregation**: Dashboard aggregates individual metric statuses for widget status
|
|
**Testing & Building**:
|
|
|
|
- **Workspace builds**: `cargo build --workspace` for all testing
|
|
- **Clean compilation**: Remove `target/` between architecture changes
|
|
- **ZMQ testing**: Test agent-dashboard communication independently
|
|
- **Widget testing**: Verify UI layout matches legacy appearance exactly
|
|
|
|
**NEVER in New Implementation**:
|
|
|
|
- Copy/paste ANY code from legacy backup
|
|
- Calculate status in dashboard widgets
|
|
- Hardcode metric names in widgets (use const arrays)
|
|
|
|
# Important Communication Guidelines
|
|
|
|
NEVER write that you have "successfully implemented" something or generate extensive summary text without first verifying with the user that the implementation is correct. This wastes tokens. Keep responses concise.
|
|
|
|
NEVER implement code without first getting explicit user agreement on the approach. Always ask for confirmation before proceeding with implementation.
|
|
|
|
## Commit Message Guidelines
|
|
|
|
**NEVER mention:**
|
|
|
|
- Claude or any AI assistant names
|
|
- Automation or AI-generated content
|
|
- Any reference to automated code generation
|
|
|
|
**ALWAYS:**
|
|
|
|
- Focus purely on technical changes and their purpose
|
|
- Use standard software development commit message format
|
|
- Describe what was changed and why, not how it was created
|
|
- Write from the perspective of a human developer
|
|
|
|
**Examples:**
|
|
|
|
- ❌ "Generated with Claude Code"
|
|
- ❌ "AI-assisted implementation"
|
|
- ❌ "Automated refactoring"
|
|
- ✅ "Implement maintenance mode for backup operations"
|
|
- ✅ "Restructure storage widget with improved layout"
|
|
- ✅ "Update CPU thresholds to production values"
|
|
|
|
## NixOS Configuration Updates
|
|
|
|
When code changes are made to cm-dashboard, the NixOS configuration at `~/nixosbox` must be updated to deploy the changes.
|
|
|
|
### Update Process
|
|
|
|
1. **Get Latest Commit Hash**
|
|
|
|
```bash
|
|
git log -1 --format="%H"
|
|
```
|
|
|
|
2. **Update NixOS Configuration**
|
|
Edit `~/nixosbox/hosts/common/cm-dashboard.nix`:
|
|
|
|
```nix
|
|
src = pkgs.fetchgit {
|
|
url = "https://gitea.cmtec.se/cm/cm-dashboard.git";
|
|
rev = "NEW_COMMIT_HASH_HERE";
|
|
sha256 = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="; # Placeholder
|
|
};
|
|
```
|
|
|
|
3. **Get Correct Source Hash**
|
|
Build with placeholder hash to get the actual hash:
|
|
|
|
```bash
|
|
cd ~/nixosbox
|
|
nix-build --no-out-link -E 'with import <nixpkgs> {}; fetchgit {
|
|
url = "https://gitea.cmtec.se/cm/cm-dashboard.git";
|
|
rev = "NEW_COMMIT_HASH";
|
|
sha256 = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
|
|
}' 2>&1 | grep "got:"
|
|
```
|
|
|
|
Example output:
|
|
|
|
```
|
|
error: hash mismatch in fixed-output derivation '/nix/store/...':
|
|
specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
|
|
got: sha256-x8crxNusOUYRrkP9mYEOG+Ga3JCPIdJLkEAc5P1ZxdQ=
|
|
```
|
|
|
|
4. **Update Configuration with Correct Hash**
|
|
Replace the placeholder with the hash from the error message (the "got:" line).
|
|
|
|
5. **Commit NixOS Configuration**
|
|
|
|
```bash
|
|
cd ~/nixosbox
|
|
git add hosts/common/cm-dashboard.nix
|
|
git commit -m "Update cm-dashboard to latest version (SHORT_HASH)"
|
|
git push
|
|
```
|
|
|
|
6. **Rebuild System**
|
|
The user handles the system rebuild step - this cannot be automated.
|