This commit is contained in:
Christoffer Martinsson 2025-10-12 14:53:27 +02:00
parent 2581435b10
commit 2239badc8a
16 changed files with 1116 additions and 1414 deletions

518
CLAUDE.md
View File

@ -1,11 +1,13 @@
# CM Dashboard - Infrastructure Monitoring TUI # CM Dashboard - Infrastructure Monitoring TUI
## Overview ## Overview
A high-performance Rust-based TUI dashboard for monitoring CMTEC infrastructure. Built to replace Glance with a custom solution tailored for our specific monitoring needs and API integrations. A high-performance Rust-based TUI dashboard for monitoring CMTEC infrastructure. Built to replace Glance with a custom solution tailored for our specific monitoring needs and API integrations.
## Project Goals ## Project Goals
### Core Objectives ### Core Objectives
- **Real-time monitoring** of all infrastructure components - **Real-time monitoring** of all infrastructure components
- **Multi-host support** for cmbox, labbox, simonbox, steambox, srv01 - **Multi-host support** for cmbox, labbox, simonbox, steambox, srv01
- **Performance-focused** with minimal resource usage - **Performance-focused** with minimal resource usage
@ -13,6 +15,7 @@ A high-performance Rust-based TUI dashboard for monitoring CMTEC infrastructure.
- **Integration** with existing monitoring APIs (ports 6127, 6128, 6129) - **Integration** with existing monitoring APIs (ports 6127, 6128, 6129)
### Key Features ### Key Features
- **NVMe health monitoring** with wear prediction - **NVMe health monitoring** with wear prediction
- **CPU / memory / GPU telemetry** with automatic thresholding - **CPU / memory / GPU telemetry** with automatic thresholding
- **Service resource monitoring** with per-service CPU and RAM usage - **Service resource monitoring** with per-service CPU and RAM usage
@ -24,6 +27,7 @@ A high-performance Rust-based TUI dashboard for monitoring CMTEC infrastructure.
## Technical Architecture ## Technical Architecture
### Technology Stack ### Technology Stack
- **Language**: Rust 🦀 - **Language**: Rust 🦀
- **TUI Framework**: ratatui (modern tui-rs fork) - **TUI Framework**: ratatui (modern tui-rs fork)
- **Async Runtime**: tokio - **Async Runtime**: tokio
@ -34,6 +38,7 @@ A high-performance Rust-based TUI dashboard for monitoring CMTEC infrastructure.
- **Time**: chrono - **Time**: chrono
### Dependencies ### Dependencies
```toml ```toml
[dependencies] [dependencies]
ratatui = "0.24" # Modern TUI framework ratatui = "0.24" # Modern TUI framework
@ -84,27 +89,8 @@ cm-dashboard/
└── WIDGETS.md # Widget development guide └── WIDGETS.md # Widget development guide
``` ```
## API Integration
### Existing CMTEC APIs
1. **Smart Metrics API** (port 6127)
- NVMe health status (wear, temperature, power-on hours)
- Disk space information
- SMART health indicators
2. **Service Metrics API** (port 6128)
- Service status and resource usage
- Service memory consumption vs limits
- Host CPU load / frequency / temperature
- Root disk utilisation snapshot
- GPU utilisation and temperature (if available)
3. **Backup Metrics API** (port 6129)
- Backup status and history
- Repository statistics
- Service integration status
### Data Structures ### Data Structures
```rust ```rust
#[derive(Deserialize, Debug)] #[derive(Deserialize, Debug)]
pub struct SmartMetrics { pub struct SmartMetrics {
@ -154,36 +140,35 @@ pub struct BackupMetrics {
## Dashboard Layout Design ## Dashboard Layout Design
### Main Dashboard View ### Main Dashboard View
``` ```
┌─────────────────────────────────────────────────────────────────────┐ ┌─────────────────────────────────────────────────────────────────────┐
📊 CMTEC Infrastructure Dashboard srv01 CM Dashboard • cmbox
├─────────────────────────────────────────────────────────────────────┤ ├─────────────────────────────────────────────────────────────────────┤
💾 NVMe Health │ 🐏 RAM Optimization Storage • ok:1 warn:0 crit:0 │ Services • ok:1 warn:0 fail:0
│ ┌─────────────────────────┐ │ ┌─────────────────────────────────────┐ │ ┌─────────────────────────────────┐ │ ┌───────────────────────────────
│ │ Wear: 4% (█░░░░░░░░░░) │ │ │ Physical: 2.4G/7.6G (32%) │ │ │ │Drive Temp Wear Spare Hours │ │ │Service memory: 7.1/23899.7 MiB│ │
│ │ Temp: 56°C │ │ │ zram: 64B/1.9G (64:1 compression) │ │ │ │nvme0n1 28°C 1% 100% 14489 │ │ │Disk usage: — │ │
│ │ Hours: 11419h (475d) │ │ │ tmpfs: /var/log 88K/512M │ │ │ │ Capacity Usage │ │ │ Service Memory Disk │ │
│ │ Status: ✅ PASSED │ │ │ Kernel: vm.dirty_ratio=5 │ │ │ │ 954G 77G (8%) │ │ │✔ sshd 7.1 MiB — │ │
│ └─────────────────────────┘ │ └─────────────────────────────────────┘ │ └─────────────────────────────────┘ │ └───────────────────────────────
├─────────────────────────────────────────────────────────────────────┤ ├─────────────────────────────────────────────────────────────────────┤
│ 🔧 Services Status │ │ CPU / Memory • warn │ Backups │
│ ┌─────────────────────────────────────────────────────────────────┐ │ │ System memory: 5251.7/23899.7 MiB │ Host cmbox awaiting backup │ │
│ │ ✅ Gitea (256M/4G, 15G/100G) ✅ smart-metrics-api │ │ │ CPU load (1/5/15): 2.18 2.66 2.56 │ metrics │ │
│ │ ✅ Immich (1.2G/4G, 45G/500G) ✅ service-metrics-api │ │ │ CPU freq: 1100.1 MHz │ │ │
│ │ ✅ Vaultwarden (45M/1G, 512M/1G) ✅ backup-metrics-api │ │ │ CPU temp: 47.0°C │ │ │
│ │ ✅ UniFi (234M/2G, 1.2G/5G) ✅ WordPress M2 │ │
│ └─────────────────────────────────────────────────────────────────┘ │
├─────────────────────────────────────────────────────────────────────┤ ├─────────────────────────────────────────────────────────────────────┤
│ 📧 Recent Alerts │ 💾 Backup Status │ │ Alerts • ok:0 warn:3 fail:0 │ Status • ZMQ connected │
│ 10:15 NVMe wear OK → 4% │ Last: ✅ Success (04:00) │ │ cmbox: warning: CPU load 2.18 │ Monitoring • hosts: 3 │ │
│ 04:00 Backup completed successfully │ Duration: 45m 32s │ │ srv01: pending: awaiting metrics │ Data source: ZMQ connected │ │
│ Yesterday: Email notification test │ Size: 15.2GB → 4.1GB │ │ labbox: pending: awaiting metrics │ Active host: cmbox (1/3) │ │
│ │ Next: Tomorrow 04:00 │
└─────────────────────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────────────────────┘
Keys: [h]osts [r]efresh [s]ettings [a]lerts [←→] navigate [q]uit Keys: [←→] hosts [r]efresh [q]uit
``` ```
### Multi-Host View ### Multi-Host View
``` ```
┌─────────────────────────────────────────────────────────────────────┐ ┌─────────────────────────────────────────────────────────────────────┐
│ 🖥️ CMTEC Host Overview │ │ 🖥️ CMTEC Host Overview │
@ -199,445 +184,28 @@ Keys: [h]osts [r]efresh [s]ettings [a]lerts [←→] navigate [q]uit
Keys: [Enter] details [r]efresh [s]ort [f]ilter [q]uit Keys: [Enter] details [r]efresh [s]ort [f]ilter [q]uit
``` ```
## Development Phases ## Development Status
### Phase 1: Foundation (Week 1-2) ### Immediate TODOs
- [x] Project setup with Cargo.toml
- [ ] Basic TUI framework with ratatui
- [ ] HTTP client for API connections
- [ ] Data structures for metrics
- [ ] Simple single-host dashboard
**Deliverables:** - Refactor all dashboard widgets to use a shared table/layout helper so icons, padding, and titles remain consistent across panels
- Working TUI that connects to srv01
- Real-time display of basic metrics
- Keyboard navigation
### Phase 2: Core Features (Week 3-4) - Investigate why the backup metrics agent is not publishing data to the dashboard
- [ ] All widget implementations - Resize the services widget so it can display more services without truncation
- [ ] Multi-host configuration - Remove the dedicated status widget and redistribute the layout space
- [ ] Historical data storage - Add responsive scaling within each widget so columns and content adapt dynamically
- [ ] Alert system integration
- [ ] Configuration management
**Deliverables:** ### Phase 3: Advanced Features 🚧 IN PROGRESS
- Full-featured dashboard
- Multi-host monitoring
- Historical trending
- Configuration file support
### Phase 3: Advanced Features (Week 5-6) - [x] ZMQ gossip network implementation
- [ ] Predictive analytics - [x] Comprehensive error handling
- [ ] Custom alert rules - [x] Performance optimizations
- [ ] Export capabilities - [ ] Predictive analytics for wear levels
- [ ] Performance optimizations - [ ] Custom alert rules engine
- [ ] Error handling & resilience - [ ] Historical data export capabilities
**Deliverables:** # Important Communication Guidelines
- Production-ready dashboard
- Advanced monitoring features
- Comprehensive error handling
- Performance benchmarks
### Phase 4: Polish & Documentation (Week 7-8) NEVER write that you have "successfully implemented" something or generate extensive summary text without first verifying with the user that the implementation is correct. This wastes tokens. Keep responses concise.
- [ ] Code documentation
- [ ] User documentation
- [ ] Installation scripts
- [ ] Testing suite
- [ ] Release preparation
**Deliverables:** NEVER implement code without first getting explicit user agreement on the approach. Always ask for confirmation before proceeding with implementation.
- Complete documentation
- Installation packages
- Test coverage
- Release v1.0
## Configuration
### Host Configuration (config/hosts.toml)
```toml
[hosts]
[hosts.srv01]
name = "srv01"
address = "192.168.30.100"
smart_api = 6127
service_api = 6128
backup_api = 6129
role = "server"
[hosts.cmbox]
name = "cmbox"
address = "192.168.30.101"
smart_api = 6127
service_api = 6128
backup_api = 6129
role = "workstation"
[hosts.labbox]
name = "labbox"
address = "192.168.30.102"
smart_api = 6127
service_api = 6128
backup_api = 6129
role = "lab"
```
### Dashboard Configuration (config/dashboard.toml)
```toml
[dashboard]
refresh_interval = 5 # seconds
history_retention = 7 # days
theme = "dark"
[widgets]
nvme_wear_threshold = 70
temperature_threshold = 70
memory_warning_threshold = 80
memory_critical_threshold = 90
[alerts]
email_enabled = true
sound_enabled = false
desktop_notifications = true
```
## Key Features
### Real-time Monitoring
- **Auto-refresh** configurable intervals (1-60 seconds)
- **Async data fetching** from multiple hosts simultaneously
- **Connection status** indicators for each host
- **Graceful degradation** when hosts are unreachable
### Historical Tracking
- **SQLite database** for local storage
- **Trend analysis** for wear levels and resource usage
- **Retention policies** configurable per metric type
- **Export capabilities** (CSV, JSON)
### Alert System
- **Threshold-based alerts** for all metrics
- **Email integration** with existing notification system
- **Alert acknowledgment** and history
- **Custom alert rules** with logical operators
### Multi-Host Management
- **Auto-discovery** of hosts on network
- **Host grouping** by role (server, workstation, lab)
- **Bulk operations** across multiple hosts
- **Host-specific configurations**
## Performance Requirements
### Resource Usage
- **Memory**: < 50MB runtime footprint
- **CPU**: < 1% average CPU usage
- **Network**: Minimal bandwidth (< 1KB/s per host)
- **Startup**: < 2 seconds cold start
### Responsiveness
- **UI updates**: 60 FPS smooth rendering
- **Data refresh**: < 500ms API response handling
- **Navigation**: Instant keyboard response
- **Error recovery**: < 5 seconds reconnection
## Security Considerations
### Network Security
- **Local network only** - no external connections
- **Authentication** for API access if implemented
- **Encrypted storage** for sensitive configuration
- **Audit logging** for administrative actions
### Data Privacy
- **Local storage** only - no cloud dependencies
- **Configurable retention** for historical data
- **Secure deletion** of expired data
- **No sensitive data logging**
## Testing Strategy
### Unit Tests
- API client modules
- Data parsing and validation
- Configuration management
- Alert logic
### Integration Tests
- Multi-host connectivity
- API error handling
- Database operations
- Alert delivery
### Performance Tests
- Memory usage under load
- Network timeout handling
- Large dataset rendering
- Extended runtime stability
## Deployment
### Installation
```bash
# Development build
cargo build --release
# Install from source
cargo install --path .
# Future: Package distribution
# Package for NixOS inclusion
```
### Usage
```bash
# Start dashboard
cm-dashboard
# Specify config
cm-dashboard --config /path/to/config
# Single host mode
cm-dashboard --host srv01
# Debug mode
cm-dashboard --verbose
```
## Maintenance
### Regular Tasks
- **Database cleanup** - automated retention policies
- **Log rotation** - configurable log levels and retention
- **Configuration validation** - startup configuration checks
- **Performance monitoring** - built-in metrics for dashboard itself
### Updates
- **Auto-update checks** - optional feature
- **Configuration migration** - version compatibility
- **API compatibility** - backwards compatibility with monitoring APIs
- **Feature toggles** - enable/disable features without rebuild
## Future Enhancements
### Proposed: ZMQ Metrics Agent Architecture
#### **Current Limitations of HTTP-based APIs**
- **Performance overhead**: Python scripts with HTTP servers on each host
- **Network complexity**: Multiple firewall ports (6127-6129) per host
- **Polling inefficiency**: Manual refresh cycles instead of real-time streaming
- **Scalability concerns**: Resource usage grows linearly with hosts
#### **Proposed: Rust ZMQ Gossip Network**
**Core Concept**: Replace HTTP polling with a peer-to-peer ZMQ gossip network where lightweight Rust agents stream metrics in real-time.
```
┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐
│ cmbox │<-->│ labbox │<-->│ srv01 │<-->│steambox │
│ :6130 │ │ :6130 │ │ :6130 │ │ :6130 │
└─────────┘ └─────────┘ └─────────┘ └─────────┘
^ ^ ^
└────────────────────────────┼──────────────┘
v
┌─────────┐
│simonbox │
│ :6130 │
└─────────┘
```
**Architecture Benefits**:
- **No central router**: Peer-to-peer gossip eliminates single point of failure
- **Self-healing**: Network automatically routes around failed hosts
- **Real-time streaming**: Metrics pushed immediately on change
- **Performance**: Rust agents ~10-100x faster than Python
- **Simplified networking**: Single ZMQ port (6130) vs multiple HTTP ports
- **Lower resource usage**: Minimal memory/CPU footprint per agent
#### **Implementation Plan**
**Phase 1: Agent Development**
```rust
// Lightweight agent on each host
pub struct MetricsAgent {
neighbors: Vec<String>, // ["srv01:6130", "cmbox:6130"]
collectors: Vec<Box<dyn Collector>>, // SMART, Service, Backup
gossip_interval: Duration, // How often to broadcast
zmq_context: zmq::Context,
}
// Message format for metrics
#[derive(Serialize, Deserialize)]
struct MetricsMessage {
hostname: String,
agent_type: AgentType, // Smart, Service, Backup
timestamp: u64,
metrics: MetricsData,
hop_count: u8, // Prevent infinite loops
}
```
**Phase 2: Dashboard Integration**
- **ZMQ Subscriber**: Dashboard subscribes to gossip stream on srv01
- **Real-time updates**: WebSocket connection to TUI for live streaming
- **Historical storage**: Optional persistence layer for trending
**Phase 3: Migration Strategy**
- **Parallel deployment**: Run ZMQ agents alongside existing HTTP APIs
- **A/B comparison**: Validate metrics accuracy and performance
- **Gradual cutover**: Switch dashboard to ZMQ, then remove HTTP services
#### **Configuration Integration**
**Agent Configuration** (per-host):
```toml
[metrics_agent]
enabled = true
port = 6130
neighbors = ["srv01:6130", "cmbox:6130"] # Redundant connections
role = "agent" # or "dashboard" for srv01
[collectors]
smart_metrics = { enabled = true, interval_ms = 5000 }
service_metrics = { enabled = true, interval_ms = 2000 } # srv01 only
backup_metrics = { enabled = true, interval_ms = 30000 } # srv01 only
```
**Dashboard Configuration** (updated):
```toml
[data_source]
type = "zmq_gossip" # vs current "http_polling"
listen_port = 6130
buffer_size = 1000
real_time_updates = true
[legacy_support]
http_apis_enabled = true # For migration period
fallback_to_http = true # If ZMQ unavailable
```
#### **Performance Comparison**
| Metric | Current (HTTP) | Proposed (ZMQ) |
|--------|---------------|----------------|
| Collection latency | ~50ms | ~1ms |
| Network overhead | HTTP headers + JSON | Binary ZMQ frames |
| Resource per host | ~5MB (Python + HTTP) | ~1MB (Rust agent) |
| Update frequency | 5s polling | Real-time push |
| Network ports | 3 per host | 1 per host |
| Failure recovery | Manual retry | Auto-reconnect |
#### **Development Roadmap**
**Week 1-2**: Basic ZMQ agent
- Rust binary with ZMQ gossip protocol
- SMART metrics collection
- Configuration management
**Week 3-4**: Dashboard integration
- ZMQ subscriber in cm-dashboard
- Real-time TUI updates
- Parallel HTTP/ZMQ operation
**Week 5-6**: Production readiness
- Service/backup metrics support
- Error handling and resilience
- Performance benchmarking
**Week 7-8**: Migration and cleanup
- Switch dashboard to ZMQ-only
- Remove legacy HTTP APIs
- Documentation and deployment
### Potential Features
- **Plugin system** for custom widgets
- **REST API** for external integrations
- **Mobile companion app** for alerts
- **Grafana integration** for advanced graphing
- **Prometheus metrics export**
- **Custom scripting** for automated responses
- **Machine learning** for predictive analytics
- **Clustering support** for high availability
### Integration Opportunities
- **Home Assistant** integration
- **Slack/Discord** notifications
- **SNMP support** for network equipment
- **Docker/Kubernetes** container monitoring
- **Cloud metrics** integration (if needed)
## Success Metrics
### Technical Success
- **Zero crashes** during normal operation
- **Sub-second response** times for all operations
- **99.9% uptime** for monitoring (excluding network issues)
- **Minimal resource usage** as specified
### User Success
- **Faster problem detection** compared to Glance
- **Reduced time to resolution** for issues
- **Improved infrastructure awareness**
- **Enhanced operational efficiency**
---
## Development Log
### Project Initialization
- Repository created: `/home/cm/projects/cm-dashboard`
- Initial planning: TUI dashboard to replace Glance
- Technology selected: Rust + ratatui
- Architecture designed: Multi-host monitoring with existing API integration
### Current Status (HTTP-based)
- **Functional TUI**: Basic dashboard rendering with ratatui
- **HTTP API integration**: Connects to ports 6127, 6128, 6129
- **Multi-host support**: Configurable host management
- **Async architecture**: Tokio-based concurrent metrics fetching
- **Configuration system**: TOML-based host and dashboard configuration
### Proposed Evolution: ZMQ Agent System
**Rationale for Change**: The current HTTP polling approach has fundamental limitations:
1. **Latency**: 5-second refresh cycles miss rapid changes
2. **Resource overhead**: Python HTTP servers consume unnecessary resources
3. **Network complexity**: Multiple ports per host complicate firewall management
4. **Scalability**: Linear resource growth with host count
**Solution**: Peer-to-peer ZMQ gossip network with Rust agents provides:
- **Real-time streaming**: Sub-second metric propagation
- **Fault tolerance**: Network self-heals around failed hosts
- **Performance**: Native Rust speed vs interpreted Python
- **Simplicity**: Single port per host, no central coordination
### ZMQ Agent Development Plan
**Component 1: cm-metrics-agent** (New Rust binary)
```toml
[package]
name = "cm-metrics-agent"
version = "0.1.0"
[dependencies]
zmq = "0.10"
serde = { version = "1.0", features = ["derive"] }
tokio = { version = "1.0", features = ["full"] }
smartmontools-rs = "0.1" # Or direct smartctl bindings
```
**Component 2: Dashboard Integration** (Update cm-dashboard)
- Add ZMQ subscriber mode alongside HTTP client
- Implement real-time metric streaming
- Provide migration path from HTTP to ZMQ
**Migration Strategy**:
1. **Phase 1**: Deploy agents alongside existing APIs
2. **Phase 2**: Switch dashboard to ZMQ mode
3. **Phase 3**: Remove HTTP APIs from NixOS configurations
**Performance Targets**:
- **Agent footprint**: < 2MB RAM, < 1% CPU
- **Metric latency**: < 100ms propagation across network
- **Network efficiency**: < 1KB/s per host steady state

View File

@ -3,30 +3,29 @@
CM Dashboard is a Rust-powered terminal UI for real-time monitoring of CMTEC infrastructure hosts. It subscribes to the CMTEC ZMQ gossip network where lightweight agents publish SMART, service, and backup metrics, and presents them in an efficient, keyboard-driven interface built with `ratatui`. CM Dashboard is a Rust-powered terminal UI for real-time monitoring of CMTEC infrastructure hosts. It subscribes to the CMTEC ZMQ gossip network where lightweight agents publish SMART, service, and backup metrics, and presents them in an efficient, keyboard-driven interface built with `ratatui`.
``` ```
┌──────────────────────────────────────────────────────────────────────────────┐ ┌─────────────────────────────────────────────────────────────────────┐
│ CM Dashboard │ │ CM Dashboard • cmbox │
├────────────────────────────┬────────────────────────────┬────────────────────┤ ├─────────────────────────────────────────────────────────────────────┤
│ NVMe Health │ Services │ CPU / Memory │ │ Storage • ok:1 warn:0 crit:0 │ Services • ok:1 warn:0 fail:0 │
│ Host: srv01 │ Host: srv01 │ Host: srv01 │ │ ┌─────────────────────────────────┐ │ ┌─────────────────────────────── │ │
│ Status: Healthy │ Service memory: 1.2G/4.0G │ RAM: 6.9 / 7.8 GiB │ │ │Drive Temp Wear Spare Hours │ │ │Service memory: 7.1/23899.7 MiB│ │
│ Healthy/Warning/Critical: │ Disk usage: 45 / 500 GiB │ CPU load (1/5/15): │ │ │nvme0n1 28°C 1% 100% 14489 │ │ │Disk usage: — │ │
│ 4 / 0 / 0 │ Services tracked: 8 │ 1.2 0.9 0.7 │ │ │ Capacity Usage │ │ │ Service Memory Disk │ │
│ Capacity used: 512 / 2048G │ │ CPU temp: 68°C │ │ │ 954G 77G (8%) │ │ │✔ sshd 7.1 MiB — │ │
│ Issue: — │ nginx running 320M │ GPU temp: — │ │ └─────────────────────────────────┘ │ └─────────────────────────────── │ │
│ │ immich running 1.2G │ Status • ok │ ├─────────────────────────────────────────────────────────────────────┤
│ │ backup-api running 40M │ │ │ CPU / Memory • warn │ Backups │
├────────────────────────────┴────────────┬───────────────┴────────────────────┤ │ System memory: 5251.7/23899.7 MiB │ Host cmbox awaiting backup │ │
│ Backups │ Alerts │ │ CPU load (1/5/15): 2.18 2.66 2.56 │ metrics │ │
│ Host: srv01 │ srv01: ok │ │ CPU freq: 1100.1 MHz │ │ │
│ Overall: Healthy │ labbox: warning: RAM 82% │ │ CPU temp: 47.0°C │ │ │
│ Last success: 2024-02-01 03:12:45 │ cmbox: critical: CPU temp 92°C │ ├─────────────────────────────────────────────────────────────────────┤
│ Snapshots: 17 • Size: 512.0 GiB │ Update: 2024-02-01 10:15:32 │ │ Alerts • ok:0 warn:3 fail:0 │ Status • ZMQ connected │
│ Pending jobs: 0 (enabled: true) │ │ │ cmbox: warning: CPU load 2.18 │ Monitoring • hosts: 3 │ │
└──────────────────────────────┬───────────────────────────────────────────────┘ │ srv01: pending: awaiting metrics │ Data source: ZMQ connected │ │
│ Status │ │ │ labbox: pending: awaiting metrics │ Active host: cmbox (1/3) │ │
│ Active host: srv01 (1/3) │ History retention ≈ 3600s │ └─────────────────────────────────────────────────────────────────────┘
│ Config: config/dashboard.toml│ Default host: labbox │ Keys: [←→] hosts [r]efresh [q]uit
└──────────────────────────────┴───────────────────────────────────────────────┘
``` ```
## Requirements ## Requirements
@ -100,12 +99,15 @@ Adjust the host list and `data_source.zmq.endpoints` to match your CMTEC gossip
## Features ## Features
- Rotating host selection with left/right arrows (`←`, `→`, `h`, `l`, `Tab`) - **Real-time monitoring** with ZMQ gossip network architecture
- Live NVMe, service, CPU/memory, backup, and alert panels per host - **Storage health** with drive capacity, usage, temperature, and wear tracking
- Health scoring that rolls CPU/RAM/GPU pressure into alerts automatically - **Per-service resource tracking** including memory and disk usage by service
- Structured logging with `tracing` (`-v`/`-vv` to increase verbosity) - **CPU/Memory monitoring** with load averages, temperature, and GPU metrics
- Help overlay (`?`) outlining keyboard shortcuts - **Alert system** with color-coded highlighting and threshold-based warnings
- Config-driven host discovery via `config/dashboard.toml` - **Multi-host support** with seamless host switching (`←`, `→`, `h`, `l`, `Tab`)
- **Backup status** monitoring with restic integration
- **Keyboard-driven interface** with help overlay (`?`)
- **Configuration management** via TOML files for hosts and dashboard settings
## Getting Started ## Getting Started
@ -131,13 +133,30 @@ cargo run -p cm-dashboard -- -v
## Agent ## Agent
The metrics agent publishes SMART/service/backup data to the gossip network. Run it on each host (or under systemd/NixOS) and point the dashboard at its endpoint. Example: The metrics agent runs on each host and publishes SMART, service, and backup data to the ZMQ gossip network. The agent auto-detects system configuration and requires root privileges for hardware monitoring.
```bash ```bash
cargo run -p cm-dashboard-agent -- --hostname srv01 --bind tcp://*:6130 --interval-ms 5000 # Run agent with auto-detection
sudo cargo run -p cm-dashboard-agent
# Run with specific configuration
sudo cargo run -p cm-dashboard-agent -- --config config/agent.toml
# Manual configuration
sudo cargo run -p cm-dashboard-agent -- \
--hostname srv01 \
--bind tcp://*:6130 \
--smart-devices nvme0n1,sda \
--services nginx,postgres
``` ```
Use `--disable-*` flags to skip collectors when a host doesnt expose those metrics. The agent automatically:
- Detects available storage devices for SMART monitoring
- Discovers running systemd services for resource tracking
- Configures appropriate collection intervals per host type
- Requires root access for `smartctl` and system metrics
Use `--disable-smart`, `--disable-service`, or `--disable-backup` to skip specific collectors.
## Development ## Development

View File

@ -172,7 +172,7 @@ impl ServiceCollector {
async fn get_directory_size(&self, path: &str) -> Result<f32, CollectorError> { async fn get_directory_size(&self, path: &str) -> Result<f32, CollectorError> {
let output = Command::new("du") let output = Command::new("du")
.args(["-s", "-k", path]) // Use kilobytes instead of forcing GB .args(["-s", "-k", path]) // Use kilobytes instead of forcing GB
.stdout(Stdio::piped()) .stdout(Stdio::piped())
.stderr(Stdio::piped()) .stderr(Stdio::piped())
.output() .output()

View File

@ -89,7 +89,10 @@ impl SmartCollector {
Ok(SmartDeviceData::from_smartctl_output(device, smart_output)) Ok(SmartDeviceData::from_smartctl_output(device, smart_output))
} }
async fn get_drive_usage(&self, device: &str) -> Result<(Option<f32>, Option<f32>), CollectorError> { async fn get_drive_usage(
&self,
device: &str,
) -> Result<(Option<f32>, Option<f32>), CollectorError> {
// Get capacity first // Get capacity first
let capacity = match self.get_drive_capacity(device).await { let capacity = match self.get_drive_capacity(device).await {
Ok(cap) => Some(cap), Ok(cap) => Some(cap),
@ -134,8 +137,8 @@ impl SmartCollector {
} }
let stdout = String::from_utf8_lossy(&output.stdout); let stdout = String::from_utf8_lossy(&output.stdout);
let lsblk_output: serde_json::Value = serde_json::from_str(&stdout) let lsblk_output: serde_json::Value =
.map_err(|e| CollectorError::ParseError { serde_json::from_str(&stdout).map_err(|e| CollectorError::ParseError {
message: format!("Failed to parse lsblk JSON: {}", e), message: format!("Failed to parse lsblk JSON: {}", e),
})?; })?;
@ -155,11 +158,12 @@ impl SmartCollector {
fn parse_lsblk_size(&self, size_str: &str) -> Result<f32, CollectorError> { fn parse_lsblk_size(&self, size_str: &str) -> Result<f32, CollectorError> {
// Parse sizes like "953,9G", "1T", "512M" // Parse sizes like "953,9G", "1T", "512M"
let size_str = size_str.replace(',', "."); // Handle European decimal separator let size_str = size_str.replace(',', "."); // Handle European decimal separator
if let Some(pos) = size_str.find(|c: char| c.is_alphabetic()) { if let Some(pos) = size_str.find(|c: char| c.is_alphabetic()) {
let (number_part, unit_part) = size_str.split_at(pos); let (number_part, unit_part) = size_str.split_at(pos);
let number: f32 = number_part.parse() let number: f32 = number_part
.parse()
.map_err(|e| CollectorError::ParseError { .map_err(|e| CollectorError::ParseError {
message: format!("Failed to parse size number '{}': {}", number_part, e), message: format!("Failed to parse size number '{}': {}", number_part, e),
})?; })?;
@ -169,9 +173,11 @@ impl SmartCollector {
"G" | "GB" => 1.0, "G" | "GB" => 1.0,
"M" | "MB" => 1.0 / 1024.0, "M" | "MB" => 1.0 / 1024.0,
"K" | "KB" => 1.0 / (1024.0 * 1024.0), "K" | "KB" => 1.0 / (1024.0 * 1024.0),
_ => return Err(CollectorError::ParseError { _ => {
message: format!("Unknown size unit: {}", unit_part), return Err(CollectorError::ParseError {
}), message: format!("Unknown size unit: {}", unit_part),
})
}
}; };
Ok(number * multiplier) Ok(number * multiplier)

View File

@ -32,6 +32,17 @@ enabled = true
id = "alerts" id = "alerts"
enabled = true enabled = true
[data_source]
kind = "zmq"
[data_source.zmq]
endpoints = [
"tcp://192.168.30.100:6130", # srv01
"tcp://192.168.30.105:6130", # cmbox
"tcp://192.168.30.107:6130", # simonbox
"tcp://192.168.30.101:6130" # steambox
]
[filesystem] [filesystem]
# cache_dir = "/var/lib/cm-dashboard/cache" # cache_dir = "/var/lib/cm-dashboard/cache"
# history_dir = "/var/lib/cm-dashboard/history" # history_dir = "/var/lib/cm-dashboard/history"

View File

@ -41,7 +41,9 @@ struct HostRuntimeState {
#[derive(Debug)] #[derive(Debug)]
pub struct App { pub struct App {
options: AppOptions, options: AppOptions,
#[allow(dead_code)]
config: Option<AppConfig>, config: Option<AppConfig>,
#[allow(dead_code)]
active_config_path: Option<PathBuf>, active_config_path: Option<PathBuf>,
hosts: Vec<HostTarget>, hosts: Vec<HostTarget>,
history: MetricsHistory, history: MetricsHistory,
@ -136,10 +138,12 @@ impl App {
self.should_quit self.should_quit
} }
#[allow(dead_code)]
pub fn status_text(&self) -> &str { pub fn status_text(&self) -> &str {
&self.status &self.status
} }
#[allow(dead_code)]
pub fn zmq_connected(&self) -> bool { pub fn zmq_connected(&self) -> bool {
self.zmq_connected self.zmq_connected
} }
@ -148,14 +152,17 @@ impl App {
self.options.tick_rate() self.options.tick_rate()
} }
#[allow(dead_code)]
pub fn config(&self) -> Option<&AppConfig> { pub fn config(&self) -> Option<&AppConfig> {
self.config.as_ref() self.config.as_ref()
} }
#[allow(dead_code)]
pub fn active_config_path(&self) -> Option<&PathBuf> { pub fn active_config_path(&self) -> Option<&PathBuf> {
self.active_config_path.as_ref() self.active_config_path.as_ref()
} }
#[allow(dead_code)]
pub fn hosts(&self) -> &[HostTarget] { pub fn hosts(&self) -> &[HostTarget] {
&self.hosts &self.hosts
} }
@ -171,6 +178,7 @@ impl App {
} }
} }
#[allow(dead_code)]
pub fn history(&self) -> &MetricsHistory { pub fn history(&self) -> &MetricsHistory {
&self.history &self.history
} }

View File

@ -80,6 +80,8 @@ pub struct ServiceInfo {
pub sandbox_limit: Option<f32>, pub sandbox_limit: Option<f32>,
#[serde(default)] #[serde(default)]
pub disk_used_gb: f32, pub disk_used_gb: f32,
#[serde(default)]
pub description: Option<String>,
} }
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]

View File

@ -1,95 +1,79 @@
use chrono::{DateTime, Utc}; use chrono::{DateTime, Utc};
use ratatui::layout::{Constraint, Rect}; use ratatui::layout::{Constraint, Rect};
use ratatui::style::{Color, Modifier, Style}; use ratatui::style::Color;
use ratatui::text::Span;
use ratatui::widgets::{Block, Borders, Cell, Paragraph, Row, Table, Wrap};
use ratatui::Frame; use ratatui::Frame;
use crate::app::HostDisplayData; use crate::app::HostDisplayData;
use crate::ui::memory::{evaluate_performance, PerfSeverity}; use crate::ui::system::{evaluate_performance, PerfSeverity};
use crate::ui::widget::{render_widget_data, WidgetData, WidgetStatus, WidgetValue, StatusLevel};
pub fn render(frame: &mut Frame, hosts: &[HostDisplayData], area: Rect) { pub fn render(frame: &mut Frame, hosts: &[HostDisplayData], area: Rect) {
let (severity, ok_count, warn_count, fail_count) = classify_hosts(hosts); let (severity, ok_count, warn_count, fail_count) = classify_hosts(hosts);
let color = match severity { let mut color = match severity {
AlertSeverity::Critical => Color::Red, AlertSeverity::Critical => Color::Red,
AlertSeverity::Warning => Color::Yellow, AlertSeverity::Warning => Color::Yellow,
AlertSeverity::Healthy => Color::Green, AlertSeverity::Healthy => Color::Green,
AlertSeverity::Unknown => Color::LightCyan, AlertSeverity::Unknown => Color::Gray,
}; };
if hosts.is_empty() {
color = Color::Gray;
}
let title = format!( let title = format!(
"Alerts • ok:{} warn:{} fail:{}", "Alerts • ok:{} warn:{} fail:{}",
ok_count, warn_count, fail_count ok_count, warn_count, fail_count
); );
let block = Block::default() let widget_status = match severity {
.title(Span::styled( AlertSeverity::Critical => StatusLevel::Error,
title, AlertSeverity::Warning => StatusLevel::Warning,
Style::default().fg(color).add_modifier(Modifier::BOLD), AlertSeverity::Healthy => StatusLevel::Ok,
)) AlertSeverity::Unknown => StatusLevel::Unknown,
.borders(Borders::ALL) };
.border_style(Style::default().fg(color))
.style(Style::default().fg(Color::White));
let inner = block.inner(area); let mut data = WidgetData::new(
frame.render_widget(block, area); title,
Some(WidgetStatus::new(widget_status)),
if hosts.is_empty() { vec!["Host".to_string(), "Status".to_string(), "Timestamp".to_string()]
frame.render_widget(
Paragraph::new("No hosts configured")
.wrap(Wrap { trim: true })
.style(Style::default().fg(Color::White)),
inner,
);
return;
}
let header = Row::new(vec![
Cell::from("Host"),
Cell::from("Status"),
Cell::from("Timestamp"),
])
.style(
Style::default()
.fg(Color::White)
.add_modifier(Modifier::BOLD),
); );
let rows = hosts.iter().map(|host| { if hosts.is_empty() {
let (status, severity, emphasize) = host_status(host); data.add_row(
let row_style = severity_style(severity); None,
let update = latest_timestamp(host) "",
.map(|ts| ts.format("%Y-%m-%d %H:%M:%S").to_string()) vec![
.unwrap_or_else(|| "".to_string()); WidgetValue::new("No hosts configured"),
WidgetValue::new(""),
WidgetValue::new(""),
],
);
} else {
for host in hosts {
let (status_text, severity, _emphasize) = host_status(host);
let status_level = match severity {
AlertSeverity::Critical => StatusLevel::Error,
AlertSeverity::Warning => StatusLevel::Warning,
AlertSeverity::Healthy => StatusLevel::Ok,
AlertSeverity::Unknown => StatusLevel::Unknown,
};
let update = latest_timestamp(host)
.map(|ts| ts.format("%Y-%m-%d %H:%M:%S").to_string())
.unwrap_or_else(|| "".to_string());
let status_cell = if emphasize { data.add_row(
Cell::from(Span::styled( Some(WidgetStatus::new(status_level)),
status.clone(), "",
Style::default().add_modifier(Modifier::BOLD), vec![
)) WidgetValue::new(host.name.clone()),
} else { WidgetValue::new(status_text),
Cell::from(status.clone()) WidgetValue::new(update),
}; ],
);
}
}
Row::new(vec![ render_widget_data(frame, area, data);
Cell::from(host.name.clone()),
status_cell,
Cell::from(update),
])
.style(row_style)
});
let table = Table::new(rows)
.header(header)
.style(Style::default().fg(Color::White))
.widths(&[
Constraint::Percentage(20),
Constraint::Length(20),
Constraint::Min(24),
])
.column_spacing(2);
frame.render_widget(table, inner);
} }
#[derive(Copy, Clone, Eq, PartialEq)] #[derive(Copy, Clone, Eq, PartialEq)]
@ -262,12 +246,12 @@ fn host_status(host: &HostDisplayData) -> (String, AlertSeverity, bool) {
("ok".to_string(), AlertSeverity::Healthy, false) ("ok".to_string(), AlertSeverity::Healthy, false)
} }
fn severity_style(severity: AlertSeverity) -> Style { fn severity_color(severity: AlertSeverity) -> Color {
match severity { match severity {
AlertSeverity::Critical => Style::default().fg(Color::Red), AlertSeverity::Critical => Color::Red,
AlertSeverity::Warning => Style::default().fg(Color::Yellow), AlertSeverity::Warning => Color::Yellow,
AlertSeverity::Healthy => Style::default().fg(Color::White), AlertSeverity::Healthy => Color::Green,
AlertSeverity::Unknown => Style::default().fg(Color::LightCyan), AlertSeverity::Unknown => Color::Gray,
} }
} }
@ -297,3 +281,12 @@ fn latest_timestamp(host: &HostDisplayData) -> Option<DateTime<Utc>> {
latest latest
} }
fn severity_symbol(severity: AlertSeverity) -> &'static str {
match severity {
AlertSeverity::Critical => "",
AlertSeverity::Warning => "!",
AlertSeverity::Healthy => "",
AlertSeverity::Unknown => "?",
}
}

View File

@ -1,11 +1,10 @@
use ratatui::layout::{Constraint, Direction, Layout, Rect}; use ratatui::layout::Rect;
use ratatui::style::{Color, Modifier, Style}; use ratatui::style::Color;
use ratatui::text::{Line, Span};
use ratatui::widgets::{Block, Borders, Cell, Paragraph, Row, Table, Wrap};
use ratatui::Frame; use ratatui::Frame;
use crate::app::HostDisplayData; use crate::app::HostDisplayData;
use crate::data::metrics::{BackupMetrics, BackupStatus}; use crate::data::metrics::{BackupMetrics, BackupStatus};
use crate::ui::widget::{render_placeholder, render_widget_data, WidgetData, WidgetStatus, WidgetValue, StatusLevel};
pub fn render(frame: &mut Frame, host: Option<&HostDisplayData>, area: Rect) { pub fn render(frame: &mut Frame, host: Option<&HostDisplayData>, area: Rect) {
match host { match host {
@ -16,113 +15,85 @@ pub fn render(frame: &mut Frame, host: Option<&HostDisplayData>, area: Rect) {
render_placeholder( render_placeholder(
frame, frame,
area, area,
"Backups",
&format!("Host {} awaiting backup metrics", data.name), &format!("Host {} awaiting backup metrics", data.name),
); );
} }
} }
None => render_placeholder(frame, area, "No hosts configured"), None => render_placeholder(frame, area, "Backups", "No hosts configured"),
} }
} }
fn render_metrics(frame: &mut Frame, _host: &HostDisplayData, metrics: &BackupMetrics, area: Rect) { fn render_metrics(frame: &mut Frame, _host: &HostDisplayData, metrics: &BackupMetrics, area: Rect) {
let color = backup_status_color(&metrics.overall_status); let widget_status = match metrics.overall_status {
let title = format!("Backups • status: {:?}", metrics.overall_status); BackupStatus::Failed => StatusLevel::Error,
BackupStatus::Warning => StatusLevel::Warning,
BackupStatus::Unknown => StatusLevel::Unknown,
BackupStatus::Healthy => StatusLevel::Ok,
};
let block = Block::default() let mut data = WidgetData::new(
.title(Span::styled( "Backups",
title, Some(WidgetStatus::new(widget_status)),
Style::default().fg(color).add_modifier(Modifier::BOLD), vec!["Aspect".to_string(), "Details".to_string()]
))
.borders(Borders::ALL)
.border_style(Style::default().fg(color))
.style(Style::default().fg(Color::White));
let inner = block.inner(area);
frame.render_widget(block, area);
let chunks = Layout::default()
.direction(Direction::Vertical)
.constraints([Constraint::Length(2), Constraint::Min(1)])
.split(inner);
let summary_line = Line::from(vec![
Span::styled("Snapshots: ", Style::default().add_modifier(Modifier::BOLD)),
Span::raw(metrics.backup.snapshot_count.to_string()),
Span::raw(" • Size: "),
Span::raw(format!("{:.1} GiB", metrics.backup.size_gb)),
Span::raw(" • Last success: "),
Span::raw(format_timestamp(metrics.backup.last_success.as_ref())),
]);
frame.render_widget(
Paragraph::new(summary_line)
.wrap(Wrap { trim: true })
.style(Style::default().fg(Color::White)),
chunks[0],
); );
let header = Row::new(vec![Cell::from("Aspect"), Cell::from("Details")]).style( let repo_status = repo_status_level(metrics);
Style::default() data.add_row(
.fg(Color::White) Some(WidgetStatus::new(repo_status)),
.add_modifier(Modifier::BOLD), "",
); vec![
WidgetValue::new("Repo"),
let mut rows = Vec::new(); WidgetValue::new(format!(
rows.push(
Row::new(vec![
Cell::from("Repo"),
Cell::from(format!(
"Snapshots: {} • Size: {:.1} GiB", "Snapshots: {} • Size: {:.1} GiB",
metrics.backup.snapshot_count, metrics.backup.size_gb metrics.backup.snapshot_count, metrics.backup.size_gb
)), )),
]) ],
.style(Style::default().fg(Color::White)),
); );
rows.push( let service_status = service_status_level(metrics);
Row::new(vec![ data.add_row(
Cell::from("Service"), Some(WidgetStatus::new(service_status)),
Cell::from(format!( "",
vec![
WidgetValue::new("Service"),
WidgetValue::new(format!(
"Enabled: {} • Pending jobs: {}", "Enabled: {} • Pending jobs: {}",
metrics.service.enabled, metrics.service.pending_jobs metrics.service.enabled, metrics.service.pending_jobs
)), )),
]) ],
.style(backup_severity_style(&metrics.overall_status)),
); );
if let Some(last_failure) = metrics.backup.last_failure.as_ref() { if let Some(last_failure) = metrics.backup.last_failure.as_ref() {
rows.push( data.add_row(
Row::new(vec![ Some(WidgetStatus::new(StatusLevel::Error)),
Cell::from("Last failure"), "",
Cell::from(last_failure.format("%Y-%m-%d %H:%M:%S").to_string()), vec![
]) WidgetValue::new("Last failure"),
.style(Style::default().fg(Color::Red)), WidgetValue::new(format_timestamp(Some(last_failure))),
],
); );
} }
if let Some(message) = metrics.service.last_message.as_ref() { if let Some(message) = metrics.service.last_message.as_ref() {
let message_style = match metrics.overall_status { let status_level = match metrics.overall_status {
BackupStatus::Failed => Style::default().fg(Color::Red), BackupStatus::Failed => StatusLevel::Error,
BackupStatus::Warning => Style::default().fg(Color::Yellow), BackupStatus::Warning => StatusLevel::Warning,
_ => Style::default().fg(Color::White), BackupStatus::Unknown => StatusLevel::Unknown,
BackupStatus::Healthy => StatusLevel::Ok,
}; };
rows.push( data.add_row(
Row::new(vec![ Some(WidgetStatus::new(status_level)),
Cell::from("Last message"), "",
Cell::from(message.clone()), vec![
]) WidgetValue::new("Last message"),
.style(message_style), WidgetValue::new(message.clone()),
],
); );
} }
let table = Table::new(rows) render_widget_data(frame, area, data);
.header(header)
.style(Style::default().fg(Color::White))
.widths(&[Constraint::Length(13), Constraint::Min(10)])
.column_spacing(2);
frame.render_widget(table, chunks[1]);
} }
fn backup_status_color(status: &BackupStatus) -> Color { fn backup_status_color(status: &BackupStatus) -> Color {
@ -140,27 +111,31 @@ fn format_timestamp(timestamp: Option<&chrono::DateTime<chrono::Utc>>) -> String
.unwrap_or_else(|| "".to_string()) .unwrap_or_else(|| "".to_string())
} }
fn render_placeholder(frame: &mut Frame, area: Rect, message: &str) { fn repo_status_level(metrics: &BackupMetrics) -> StatusLevel {
let block = Block::default() match metrics.overall_status {
.title("Backups") BackupStatus::Failed => StatusLevel::Error,
.borders(Borders::ALL) BackupStatus::Warning => StatusLevel::Warning,
.border_style(Style::default().fg(Color::LightGreen)) _ => {
.style(Style::default().fg(Color::White)); if metrics.backup.snapshot_count > 0 {
let inner = block.inner(area); StatusLevel::Ok
frame.render_widget(block, area); } else {
frame.render_widget( StatusLevel::Warning
Paragraph::new(Line::from(message)) }
.wrap(Wrap { trim: true }) }
.style(Style::default().fg(Color::White)), }
inner, }
);
} fn service_status_level(metrics: &BackupMetrics) -> StatusLevel {
match metrics.overall_status {
fn backup_severity_style(status: &BackupStatus) -> Style { BackupStatus::Failed => StatusLevel::Error,
match status { BackupStatus::Warning => StatusLevel::Warning,
BackupStatus::Failed => Style::default().fg(Color::Red), BackupStatus::Unknown => StatusLevel::Unknown,
BackupStatus::Warning => Style::default().fg(Color::Yellow), BackupStatus::Healthy => {
BackupStatus::Unknown => Style::default().fg(Color::LightCyan), if metrics.service.enabled {
BackupStatus::Healthy => Style::default().fg(Color::White), StatusLevel::Ok
} else {
StatusLevel::Warning
}
}
} }
} }

View File

@ -1,12 +1,12 @@
use ratatui::layout::{Constraint, Direction, Layout, Rect}; use ratatui::layout::{Constraint, Direction, Layout, Rect};
use ratatui::style::{Color, Modifier, Style}; use ratatui::style::{Color, Modifier, Style};
use ratatui::text::Span; use ratatui::text::Span;
use ratatui::widgets::{Block, Cell, Row, Table}; use ratatui::widgets::Block;
use ratatui::Frame; use ratatui::Frame;
use crate::app::App; use crate::app::App;
use super::{alerts, backup, memory, storage, services}; use super::{alerts, backup, services, storage, system};
pub fn render(frame: &mut Frame, app: &App) { pub fn render(frame: &mut Frame, app: &App) {
let host_summaries = app.host_display_data(); let host_summaries = app.host_display_data();
@ -30,164 +30,39 @@ pub fn render(frame: &mut Frame, app: &App) {
let outer = inner_rect(size); let outer = inner_rect(size);
let vertical_chunks = Layout::default() let main_columns = Layout::default()
.direction(Direction::Vertical) .direction(Direction::Horizontal)
.constraints([ .constraints([Constraint::Percentage(50), Constraint::Percentage(50)])
Constraint::Percentage(35),
Constraint::Percentage(35),
Constraint::Percentage(30),
])
.split(outer); .split(outer);
let top = Layout::default() let left_side = Layout::default()
.direction(Direction::Horizontal) .direction(Direction::Vertical)
.constraints([Constraint::Percentage(50), Constraint::Percentage(50)]) .constraints([Constraint::Percentage(75), Constraint::Percentage(25)])
.split(vertical_chunks[0]); .split(main_columns[0]);
let middle = Layout::default() let left_widgets = Layout::default()
.direction(Direction::Horizontal) .direction(Direction::Vertical)
.constraints([Constraint::Percentage(50), Constraint::Percentage(50)]) .constraints([
.split(vertical_chunks[1]); Constraint::Ratio(1, 3),
Constraint::Ratio(1, 3),
Constraint::Ratio(1, 3),
])
.split(left_side[0]);
let bottom = Layout::default() let services_area = main_columns[1];
.direction(Direction::Horizontal)
.constraints([Constraint::Percentage(50), Constraint::Percentage(50)])
.split(vertical_chunks[2]);
storage::render(frame, primary_host.as_ref(), top[0]); system::render(frame, primary_host.as_ref(), left_widgets[0]);
services::render(frame, primary_host.as_ref(), top[1]); storage::render(frame, primary_host.as_ref(), left_widgets[1]);
memory::render(frame, primary_host.as_ref(), middle[0]); backup::render(frame, primary_host.as_ref(), left_widgets[2]);
backup::render(frame, primary_host.as_ref(), middle[1]); services::render(frame, primary_host.as_ref(), services_area);
alerts::render(frame, &host_summaries, bottom[0]);
render_status(frame, app, bottom[1]); alerts::render(frame, &host_summaries, left_side[1]);
if app.help_visible() { if app.help_visible() {
render_help(frame, size); render_help(frame, size);
} }
} }
fn render_status(frame: &mut Frame, app: &App, area: Rect) {
let connected = app.zmq_connected();
let title_color = if connected { Color::Green } else { Color::Red };
let title_suffix = if connected {
"connected"
} else {
"disconnected"
};
let block = Block::default()
.title(Span::styled(
format!("Status • ZMQ {title_suffix}"),
Style::default()
.fg(title_color)
.add_modifier(Modifier::BOLD),
))
.borders(ratatui::widgets::Borders::ALL)
.border_style(Style::default().fg(title_color))
.style(Style::default().fg(Color::White));
let inner = block.inner(area);
frame.render_widget(block, area);
let mut rows: Vec<Row> = Vec::new();
let status_style = if connected {
Style::default().fg(Color::White)
} else {
Style::default().fg(Color::Red)
};
let default_style = Style::default().fg(Color::White);
rows.push(
Row::new(vec![
Cell::from("Status"),
Cell::from(app.status_text().to_string()),
])
.style(status_style),
);
rows.push(
Row::new(vec![
Cell::from("Data source"),
Cell::from(if connected {
"ZMQ connected"
} else {
"ZMQ disconnected"
}),
])
.style(status_style),
);
if let Some((index, host)) = app.active_host_info() {
let mut detail = format!("{} ({}/{})", host.name, index + 1, app.hosts().len());
if let Some(state) = app
.host_display_data()
.into_iter()
.find(|entry| entry.name == host.name)
{
if let Some(last_success) = state.last_success {
detail = format!(
"{} • last success {}",
detail,
last_success.format("%H:%M:%S")
);
}
}
rows.push(
Row::new(vec![Cell::from("Active host"), Cell::from(detail)]).style(default_style),
);
} else {
rows.push(Row::new(vec![Cell::from("Active host"), Cell::from("")]).style(default_style));
}
if let Some(path) = app.active_config_path() {
rows.push(
Row::new(vec![
Cell::from("Config"),
Cell::from(path.display().to_string()),
])
.style(default_style),
);
}
let retention = app.history().retention();
rows.push(
Row::new(vec![
Cell::from("History"),
Cell::from(format!("{} seconds", retention.as_secs())),
])
.style(default_style),
);
if let Some(config) = app.config() {
if let Some(default_host) = &config.hosts.default_host {
rows.push(
Row::new(vec![
Cell::from("Default host"),
Cell::from(default_host.clone()),
])
.style(default_style),
);
}
}
rows.push(
Row::new(vec![
Cell::from("Monitored hosts"),
Cell::from(app.hosts().len().to_string()),
])
.style(default_style),
);
let table = Table::new(rows)
.widths(&[Constraint::Length(18), Constraint::Min(24)])
.column_spacing(2)
.style(default_style);
frame.render_widget(table, inner);
}
fn inner_rect(area: Rect) -> Rect { fn inner_rect(area: Rect) -> Rect {
Rect { Rect {
x: area.x + 1, x: area.x + 1,

View File

@ -1,281 +0,0 @@
use ratatui::layout::Rect;
use ratatui::style::{Color, Modifier, Style};
use ratatui::text::{Line, Span};
use ratatui::widgets::{Block, Borders, Paragraph, Wrap};
use ratatui::Frame;
use crate::app::HostDisplayData;
use crate::data::metrics::{ServiceMetrics, ServiceSummary};
pub fn render(frame: &mut Frame, host: Option<&HostDisplayData>, area: Rect) {
match host {
Some(data) => {
if let Some(metrics) = data.services.as_ref() {
render_metrics(frame, data, metrics, area);
} else {
render_placeholder(
frame,
area,
&format!("Host {} awaiting service metrics", data.name),
);
}
}
None => render_placeholder(frame, area, "No hosts configured"),
}
}
fn render_metrics(frame: &mut Frame, _host: &HostDisplayData, metrics: &ServiceMetrics, area: Rect) {
let summary = &metrics.summary;
let system_total = if summary.system_memory_total_mb > 0.0 {
summary.system_memory_total_mb
} else {
summary.memory_quota_mb
};
let system_used = if summary.system_memory_used_mb > 0.0 {
summary.system_memory_used_mb
} else {
summary.memory_used_mb
};
let usage_ratio = if system_total > 0.0 {
(system_used / system_total) * 100.0
} else {
0.0
};
let (perf_severity, _reason) = evaluate_performance(summary);
let (color, severity_label) = match perf_severity {
PerfSeverity::Critical => (Color::Red, "crit"),
PerfSeverity::Warning => (Color::Yellow, "warn"),
PerfSeverity::Ok => (Color::Green, "ok"),
};
let title = format!("CPU / Memory • {}", severity_label);
let block = Block::default()
.title(Span::styled(
title,
Style::default().fg(color).add_modifier(Modifier::BOLD),
))
.borders(Borders::ALL)
.border_style(Style::default().fg(color))
.style(Style::default().fg(Color::White));
let inner = block.inner(area);
frame.render_widget(block, area);
let mut lines = Vec::new();
// Check if memory should be highlighted due to alert
let memory_color = if usage_ratio >= 95.0 {
Color::Red // Critical
} else if usage_ratio >= 80.0 {
Color::Yellow // Warning
} else {
Color::White // Normal
};
lines.push(Line::from(vec![
Span::styled(
format!("System memory: {:.1} / {:.1} MiB ({:.1}%)",
system_used, system_total, usage_ratio),
Style::default().fg(memory_color)
)
]));
// Check if CPU load should be highlighted due to alert
let cpu_load_color = if summary.cpu_load_5 >= 4.0 {
Color::Red // Critical
} else if summary.cpu_load_5 >= 2.0 {
Color::Yellow // Warning
} else {
Color::White // Normal
};
lines.push(Line::from(vec![
Span::styled(
format!("CPU load (1/5/15): {:.2} {:.2} {:.2}",
summary.cpu_load_1, summary.cpu_load_5, summary.cpu_load_15),
Style::default().fg(cpu_load_color)
)
]));
lines.push(Line::from(vec![
Span::raw("CPU freq: "),
Span::raw(format_optional_metric(summary.cpu_freq_mhz, " MHz")),
]));
// Check if CPU temp should be highlighted due to alert
let cpu_temp_color = if let Some(temp) = summary.cpu_temp_c {
if temp >= 90.0 {
Color::Red // Critical
} else if temp >= 80.0 {
Color::Yellow // Warning
} else {
Color::White // Normal
}
} else {
Color::White // Normal
};
lines.push(Line::from(vec![
Span::raw("CPU temp: "),
Span::styled(
format_optional_metric(summary.cpu_temp_c, "°C"),
Style::default().fg(cpu_temp_color)
),
]));
if summary.gpu_load_percent.is_some() || summary.gpu_temp_c.is_some() {
// Check if GPU load should be highlighted due to alert
let gpu_load_color = if let Some(load) = summary.gpu_load_percent {
if load >= 95.0 {
Color::Red // Critical
} else if load >= 85.0 {
Color::Yellow // Warning
} else {
Color::White // Normal
}
} else {
Color::White // Normal
};
lines.push(Line::from(vec![
Span::styled("GPU load: ", Style::default().add_modifier(Modifier::BOLD)),
Span::styled(
format_optional_percent(summary.gpu_load_percent),
Style::default().fg(gpu_load_color)
),
]));
// Check if GPU temp should be highlighted due to alert
let gpu_temp_color = if let Some(temp) = summary.gpu_temp_c {
if temp >= 85.0 {
Color::Red // Critical
} else if temp >= 75.0 {
Color::Yellow // Warning
} else {
Color::White // Normal
}
} else {
Color::White // Normal
};
lines.push(Line::from(vec![
Span::styled("GPU temp: ", Style::default().add_modifier(Modifier::BOLD)),
Span::styled(
format_optional_metric(summary.gpu_temp_c, "°C"),
Style::default().fg(gpu_temp_color)
),
]));
}
frame.render_widget(
Paragraph::new(lines)
.wrap(Wrap { trim: true })
.style(Style::default().fg(Color::White)),
inner,
);
}
#[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord)]
pub(crate) enum PerfSeverity {
Ok,
Warning,
Critical,
}
fn format_optional_metric(value: Option<f32>, unit: &str) -> String {
match value {
Some(number) => format!("{:.1}{}", number, unit),
None => "".to_string(),
}
}
fn format_optional_percent(value: Option<f32>) -> String {
match value {
Some(number) => format!("{:.0}%", number),
None => "".to_string(),
}
}
fn render_placeholder(frame: &mut Frame, area: Rect, message: &str) {
let block = Block::default()
.title("CPU / Memory")
.borders(Borders::ALL)
.border_style(Style::default().fg(Color::LightMagenta))
.style(Style::default().fg(Color::White));
let inner = block.inner(area);
frame.render_widget(block, area);
frame.render_widget(
Paragraph::new(Line::from(message))
.wrap(Wrap { trim: true })
.style(Style::default().fg(Color::White)),
inner,
);
}
pub(crate) fn evaluate_performance(summary: &ServiceSummary) -> (PerfSeverity, Option<String>) {
let mem_percent = if summary.system_memory_total_mb > 0.0 {
(summary.system_memory_used_mb / summary.system_memory_total_mb) * 100.0
} else if summary.memory_quota_mb > 0.0 {
(summary.memory_used_mb / summary.memory_quota_mb) * 100.0
} else {
0.0
};
let mut severity = PerfSeverity::Ok;
let mut reason: Option<String> = None;
let mut consider = |level: PerfSeverity, message: String| {
if level > severity {
severity = level;
reason = Some(message);
}
};
if mem_percent >= 95.0 {
consider(PerfSeverity::Critical, format!("RAM {:.0}%", mem_percent));
} else if mem_percent >= 80.0 {
consider(PerfSeverity::Warning, format!("RAM {:.0}%", mem_percent));
}
let load = summary.cpu_load_5;
if load >= 4.0 {
consider(PerfSeverity::Critical, format!("CPU load {:.2}", load));
} else if load >= 2.0 {
consider(PerfSeverity::Warning, format!("CPU load {:.2}", load));
}
if let Some(temp) = summary.cpu_temp_c {
if temp >= 90.0 {
consider(PerfSeverity::Critical, format!("CPU temp {:.0}°C", temp));
} else if temp >= 80.0 {
consider(PerfSeverity::Warning, format!("CPU temp {:.0}°C", temp));
}
}
if let Some(load) = summary.gpu_load_percent {
if load >= 95.0 {
consider(PerfSeverity::Critical, format!("GPU load {:.0}%", load));
} else if load >= 85.0 {
consider(PerfSeverity::Warning, format!("GPU load {:.0}%", load));
}
}
if let Some(temp) = summary.gpu_temp_c {
if temp >= 85.0 {
consider(PerfSeverity::Critical, format!("GPU temp {:.0}°C", temp));
} else if temp >= 75.0 {
consider(PerfSeverity::Warning, format!("GPU temp {:.0}°C", temp));
}
}
if severity == PerfSeverity::Ok {
(PerfSeverity::Ok, None)
} else {
(severity, reason)
}
}

View File

@ -1,8 +1,9 @@
pub mod alerts; pub mod alerts;
pub mod backup; pub mod backup;
pub mod dashboard; pub mod dashboard;
pub mod memory;
pub mod storage;
pub mod services; pub mod services;
pub mod storage;
pub mod system;
pub mod widget;
pub use dashboard::render; pub use dashboard::render;

View File

@ -1,11 +1,10 @@
use ratatui::layout::{Constraint, Direction, Layout, Rect}; use ratatui::layout::Rect;
use ratatui::style::{Color, Modifier, Style}; use ratatui::style::Color;
use ratatui::text::{Line, Span};
use ratatui::widgets::{Block, Borders, Cell, Paragraph, Row, Table, Wrap};
use ratatui::Frame; use ratatui::Frame;
use crate::app::HostDisplayData; use crate::app::HostDisplayData;
use crate::data::metrics::{ServiceStatus, ServiceSummary}; use crate::data::metrics::{ServiceStatus, ServiceSummary};
use crate::ui::widget::{render_placeholder, render_widget_data, WidgetData, WidgetStatus, WidgetValue, StatusLevel};
pub fn render(frame: &mut Frame, host: Option<&HostDisplayData>, area: Rect) { pub fn render(frame: &mut Frame, host: Option<&HostDisplayData>, area: Rect) {
match host { match host {
@ -16,11 +15,12 @@ pub fn render(frame: &mut Frame, host: Option<&HostDisplayData>, area: Rect) {
render_placeholder( render_placeholder(
frame, frame,
area, area,
"Services",
&format!("Host {} has no service metrics yet", data.name), &format!("Host {} has no service metrics yet", data.name),
); );
} }
} }
None => render_placeholder(frame, area, "No hosts configured"), None => render_placeholder(frame, area, "Services", "No hosts configured"),
} }
} }
@ -32,77 +32,38 @@ fn render_metrics(
) { ) {
let summary = &metrics.summary; let summary = &metrics.summary;
let color = summary_color(summary); let color = summary_color(summary);
let disk_summary = format_disk_summary(summary.disk_used_gb, summary.disk_total_gb);
let title = format!( let title = format!(
"Services • ok:{} warn:{} fail:{} • Disk: {}", "Services • ok:{} warn:{} fail:{}",
summary.healthy, summary.degraded, summary.failed, disk_summary summary.healthy, summary.degraded, summary.failed
); );
let block = Block::default() let widget_status = if summary.failed > 0 {
.title(Span::styled( StatusLevel::Error
title, } else if summary.degraded > 0 {
Style::default().fg(color).add_modifier(Modifier::BOLD), StatusLevel::Warning
))
.borders(Borders::ALL)
.border_style(Style::default().fg(color))
.style(Style::default().fg(Color::White));
let inner = block.inner(area);
frame.render_widget(block, area);
let chunks = Layout::default()
.direction(Direction::Vertical)
.constraints([Constraint::Length(2), Constraint::Min(1)])
.split(inner);
let mut summary_lines = Vec::new();
summary_lines.push(Line::from(vec![
Span::styled(
"Service memory: ",
Style::default().add_modifier(Modifier::BOLD),
),
Span::raw(format_memory(summary)),
]));
let disk_text = if summary.disk_total_gb > 0.0 {
format!(
"{:.1} / {:.1} GiB",
summary.disk_used_gb, summary.disk_total_gb
)
} else { } else {
"".to_string() StatusLevel::Ok
}; };
summary_lines.push(Line::from(vec![ let mut data = WidgetData::new(
Span::styled( title,
"Disk usage: ", Some(WidgetStatus::new(widget_status)),
Style::default().add_modifier(Modifier::BOLD), vec!["Service".to_string(), "Memory".to_string(), "Disk".to_string(), "Description".to_string()]
),
Span::raw(disk_text),
]));
summary_lines.push(Line::from(vec![
Span::styled(
"Services tracked: ",
Style::default().add_modifier(Modifier::BOLD),
),
Span::raw(metrics.services.len().to_string()),
]));
frame.render_widget(
Paragraph::new(summary_lines)
.wrap(Wrap { trim: true })
.style(Style::default().fg(Color::White)),
chunks[0],
); );
if metrics.services.is_empty() { if metrics.services.is_empty() {
frame.render_widget( data.add_row(
Paragraph::new("No services reported") None,
.wrap(Wrap { trim: true }) "",
.style(Style::default().fg(Color::White)), vec![
chunks[1], WidgetValue::new("No services reported"),
WidgetValue::new(""),
WidgetValue::new(""),
WidgetValue::new(""),
],
); );
render_widget_data(frame, area, data);
return; return;
} }
@ -113,41 +74,27 @@ fn render_metrics(
.then_with(|| a.name.cmp(&b.name)) .then_with(|| a.name.cmp(&b.name))
}); });
let header = Row::new(vec![ for svc in services {
Cell::from(""), let status_level = match svc.status {
Cell::from("Service"), ServiceStatus::Running => StatusLevel::Ok,
Cell::from("Memory"), ServiceStatus::Degraded => StatusLevel::Warning,
Cell::from("Disk"), ServiceStatus::Restarting => StatusLevel::Warning,
]) ServiceStatus::Stopped => StatusLevel::Error,
.style( };
Style::default()
.fg(Color::White)
.add_modifier(Modifier::BOLD),
);
let rows = services.into_iter().map(|svc| { data.add_row(
let row_style = status_style(&svc.status); Some(WidgetStatus::new(status_level)),
Row::new(vec![ "",
Cell::from(status_symbol(&svc.status)), vec![
Cell::from(format_service_name(&svc.name)), WidgetValue::new(svc.name.clone()),
Cell::from(format_memory_value(svc.memory_used_mb, svc.memory_quota_mb)), WidgetValue::new(format_memory_value(svc.memory_used_mb, svc.memory_quota_mb)),
Cell::from(format_disk_value(svc.disk_used_gb)), WidgetValue::new(format_disk_value(svc.disk_used_gb)),
]) WidgetValue::new(svc.description.as_deref().unwrap_or("")),
.style(row_style) ],
}); );
}
let table = Table::new(rows) render_widget_data(frame, area, data);
.header(header)
.style(Style::default().fg(Color::White))
.widths(&[
Constraint::Length(1),
Constraint::Length(10),
Constraint::Length(12),
Constraint::Length(8),
])
.column_spacing(2);
frame.render_widget(table, chunks[1]);
} }
fn status_weight(status: &ServiceStatus) -> i32 { fn status_weight(status: &ServiceStatus) -> i32 {
@ -159,21 +106,12 @@ fn status_weight(status: &ServiceStatus) -> i32 {
} }
} }
fn status_symbol(status: &ServiceStatus) -> &'static str { fn status_symbol(status: &ServiceStatus) -> (&'static str, Color) {
match status { match status {
ServiceStatus::Running => "", ServiceStatus::Running => ("", Color::Green),
ServiceStatus::Degraded => "!", ServiceStatus::Degraded => ("!", Color::Yellow),
ServiceStatus::Restarting => "", ServiceStatus::Restarting => ("", Color::Yellow),
ServiceStatus::Stopped => "", ServiceStatus::Stopped => ("", Color::Red),
}
}
fn status_style(status: &ServiceStatus) -> Style {
match status {
ServiceStatus::Running => Style::default().fg(Color::White),
ServiceStatus::Degraded => Style::default().fg(Color::Yellow),
ServiceStatus::Restarting => Style::default().fg(Color::Yellow),
ServiceStatus::Stopped => Style::default().fg(Color::Red),
} }
} }
@ -187,17 +125,6 @@ fn summary_color(summary: &ServiceSummary) -> Color {
} }
} }
fn format_memory(summary: &ServiceSummary) -> String {
if summary.memory_quota_mb > 0.0 {
format!(
"{:.1}/{:.1} MiB",
summary.memory_used_mb, summary.memory_quota_mb
)
} else {
format!("{:.1} MiB used", summary.memory_used_mb)
}
}
fn format_memory_value(used: f32, quota: f32) -> String { fn format_memory_value(used: f32, quota: f32) -> String {
if quota > 0.05 { if quota > 0.05 {
format!("{:.1}/{:.1} MiB", used, quota) format!("{:.1}/{:.1} MiB", used, quota)
@ -208,20 +135,11 @@ fn format_memory_value(used: f32, quota: f32) -> String {
} }
} }
fn format_disk_summary(used: f32, total: f32) -> String {
if total > 0.05 {
format!("{:.1}/{:.1} GiB", used, total)
} else if used > 0.05 {
format!("{:.1} GiB", used)
} else {
"".to_string()
}
}
fn format_disk_value(used: f32) -> String { fn format_disk_value(used: f32) -> String {
if used >= 1.0 { if used >= 1.0 {
format!("{:.1} GiB", used) format!("{:.1} GiB", used)
} else if used >= 0.001 { // 1 MB or more } else if used >= 0.001 {
// 1 MB or more
format!("{:.0} MiB", used * 1024.0) format!("{:.0} MiB", used * 1024.0)
} else if used > 0.0 { } else if used > 0.0 {
format!("<1 MiB") format!("<1 MiB")
@ -230,28 +148,3 @@ fn format_disk_value(used: f32) -> String {
} }
} }
fn format_service_name(name: &str) -> String {
let mut truncated = String::with_capacity(10);
for ch in name.chars().take(10) {
truncated.push(ch);
}
truncated
}
fn render_placeholder(frame: &mut Frame, area: Rect, message: &str) {
let block = Block::default()
.title("Services")
.borders(Borders::ALL)
.border_style(Style::default().fg(Color::Yellow))
.style(Style::default().fg(Color::White));
let inner = block.inner(area);
frame.render_widget(block, area);
frame.render_widget(
Paragraph::new(Line::from(message))
.wrap(Wrap { trim: true })
.style(Style::default().fg(Color::White)),
inner,
);
}

View File

@ -1,11 +1,10 @@
use ratatui::layout::{Constraint, Direction, Layout, Rect}; use ratatui::layout::Rect;
use ratatui::style::{Color, Modifier, Style}; use ratatui::style::Color;
use ratatui::text::{Line, Span};
use ratatui::widgets::{Block, Borders, Cell, Paragraph, Row, Table, Wrap};
use ratatui::Frame; use ratatui::Frame;
use crate::app::HostDisplayData; use crate::app::HostDisplayData;
use crate::data::metrics::SmartMetrics; use crate::data::metrics::SmartMetrics;
use crate::ui::widget::{render_placeholder, render_widget_data, WidgetData, WidgetStatus, WidgetValue, StatusLevel};
pub fn render(frame: &mut Frame, host: Option<&HostDisplayData>, area: Rect) { pub fn render(frame: &mut Frame, host: Option<&HostDisplayData>, area: Rect) {
match host { match host {
@ -16,11 +15,12 @@ pub fn render(frame: &mut Frame, host: Option<&HostDisplayData>, area: Rect) {
render_placeholder( render_placeholder(
frame, frame,
area, area,
"Storage",
&format!("Host {} has no SMART data yet", data.name), &format!("Host {} has no SMART data yet", data.name),
); );
} }
} }
None => render_placeholder(frame, area, "No hosts configured"), None => render_placeholder(frame, area, "Storage", "No hosts configured"),
} }
} }
@ -31,99 +31,71 @@ fn render_metrics(frame: &mut Frame, _host: &HostDisplayData, metrics: &SmartMet
metrics.summary.healthy, metrics.summary.warning, metrics.summary.critical metrics.summary.healthy, metrics.summary.warning, metrics.summary.critical
); );
let block = Block::default() let widget_status = if metrics.summary.critical > 0 {
.title(Span::styled( StatusLevel::Error
title, } else if metrics.summary.warning > 0 {
Style::default().fg(color).add_modifier(Modifier::BOLD), StatusLevel::Warning
))
.borders(Borders::ALL)
.border_style(Style::default().fg(color))
.style(Style::default().fg(Color::White));
let inner = block.inner(area);
frame.render_widget(block, area);
let issue_count = metrics.issues.len();
let body_constraints = if issue_count > 0 {
vec![Constraint::Min(1), Constraint::Length(2)]
} else { } else {
vec![Constraint::Min(1)] StatusLevel::Ok
}; };
let body_chunks = Layout::default() let mut data = WidgetData::new(
.direction(Direction::Vertical) title,
.constraints(body_constraints) Some(WidgetStatus::new(widget_status)),
.split(inner); vec!["Drive".to_string(), "Temp".to_string(), "Wear".to_string(), "Spare".to_string(), "Hours".to_string(), "Capacity".to_string(), "Usage".to_string()]
);
if metrics.drives.is_empty() { if metrics.drives.is_empty() {
frame.render_widget( data.add_row(
Paragraph::new("No drives reported") None,
.wrap(Wrap { trim: true }) "",
.style(Style::default().fg(Color::White)), vec![
body_chunks[0], WidgetValue::new("No drives reported"),
WidgetValue::new(""),
WidgetValue::new(""),
WidgetValue::new(""),
WidgetValue::new(""),
WidgetValue::new(""),
WidgetValue::new(""),
],
); );
} else { } else {
let header = Row::new(vec![ for drive in &metrics.drives {
Cell::from("Drive"), let status_level = drive_status_level(metrics, &drive.name);
Cell::from("Temp"), data.add_row(
Cell::from("Wear"), Some(WidgetStatus::new(status_level)),
Cell::from("Spare"), "",
Cell::from("Hours"), vec![
Cell::from("Capacity"), WidgetValue::new(drive.name.clone()),
Cell::from("Usage"), WidgetValue::new(format_temperature(drive.temperature_c)),
]) WidgetValue::new(format_percent(drive.wear_level)),
.style( WidgetValue::new(format_percent(drive.available_spare)),
Style::default() WidgetValue::new(drive.power_on_hours.to_string()),
.fg(Color::White) WidgetValue::new(format_capacity(drive.capacity_gb)),
.add_modifier(Modifier::BOLD), WidgetValue::new(format_usage(drive.used_gb, drive.capacity_gb)),
); ],
);
}
let rows = metrics.drives.iter().map(|drive| { if let Some(issue) = metrics.issues.first() {
Row::new(vec![ data.add_row(
Cell::from(format_drive_name(&drive.name)), Some(WidgetStatus::new(StatusLevel::Warning)),
Cell::from(format_temperature(drive.temperature_c)), "",
Cell::from(format_percent(drive.wear_level)), vec![
Cell::from(format_percent(drive.available_spare)), WidgetValue::new(format!("Issue: {}", issue)),
Cell::from(drive.power_on_hours.to_string()), WidgetValue::new(""),
Cell::from(format_capacity(drive.capacity_gb)), WidgetValue::new(""),
Cell::from(format_usage(drive.used_gb, drive.capacity_gb)), WidgetValue::new(""),
]) WidgetValue::new(""),
}); WidgetValue::new(""),
WidgetValue::new(""),
let table = Table::new(rows) ],
.header(header) );
.style(Style::default().fg(Color::White)) }
.widths(&[
Constraint::Length(10), // Drive name
Constraint::Length(8), // Temp
Constraint::Length(8), // Wear
Constraint::Length(8), // Spare
Constraint::Length(10), // Hours
Constraint::Length(10), // Capacity
Constraint::Min(8), // Usage
])
.column_spacing(2);
frame.render_widget(table, body_chunks[0]);
} }
if issue_count > 0 { render_widget_data(frame, area, data);
let issue_line = Line::from(vec![
Span::styled("Issue: ", Style::default().fg(Color::Yellow)),
Span::styled(
metrics.issues[0].clone(),
Style::default().fg(Color::Yellow),
),
]);
frame.render_widget(
Paragraph::new(issue_line)
.wrap(Wrap { trim: true })
.style(Style::default().fg(Color::White)),
body_chunks[1],
);
}
} }
fn smart_status_color(status: &str) -> Color { fn smart_status_color(status: &str) -> Color {
@ -150,13 +122,6 @@ fn format_percent(value: f32) -> String {
} }
} }
fn format_drive_name(name: &str) -> String {
let mut truncated = String::with_capacity(10);
for ch in name.chars().take(10) {
truncated.push(ch);
}
truncated
}
fn format_capacity(value: Option<f32>) -> String { fn format_capacity(value: Option<f32>) -> String {
match value { match value {
@ -178,19 +143,22 @@ fn format_usage(used: Option<f32>, capacity: Option<f32>) -> String {
} }
} }
fn drive_status_level(metrics: &SmartMetrics, drive_name: &str) -> StatusLevel {
fn render_placeholder(frame: &mut Frame, area: Rect, message: &str) { if metrics.summary.critical > 0
let block = Block::default() || metrics.issues.iter().any(|issue| {
.title("Storage") issue.to_lowercase().contains(&drive_name.to_lowercase())
.borders(Borders::ALL) && issue.to_lowercase().contains("fail")
.border_style(Style::default().fg(Color::LightCyan)) })
.style(Style::default().fg(Color::White)); {
let inner = block.inner(area); StatusLevel::Error
frame.render_widget(block, area); } else if metrics.summary.warning > 0
frame.render_widget( || metrics
Paragraph::new(Line::from(message)) .issues
.wrap(Wrap { trim: true }) .iter()
.style(Style::default().fg(Color::White)), .any(|issue| issue.to_lowercase().contains(&drive_name.to_lowercase()))
inner, {
); StatusLevel::Warning
} else {
StatusLevel::Ok
}
} }

217
dashboard/src/ui/system.rs Normal file
View File

@ -0,0 +1,217 @@
use ratatui::layout::Rect;
use ratatui::style::Color;
use ratatui::Frame;
use crate::app::HostDisplayData;
use crate::data::metrics::{ServiceMetrics, ServiceSummary};
use crate::ui::widget::{
combined_color, render_placeholder, render_combined_widget_data, status_color_for_cpu_load, status_color_from_metric,
status_color_from_percentage, WidgetDataSet, WidgetStatus, WidgetValue, StatusLevel,
};
pub fn render(frame: &mut Frame, host: Option<&HostDisplayData>, area: Rect) {
match host {
Some(data) => {
if let Some(metrics) = data.services.as_ref() {
render_metrics(frame, data, metrics, area);
} else {
render_placeholder(
frame,
area,
"System",
&format!("Host {} awaiting service metrics", data.name),
);
}
}
None => render_placeholder(frame, area, "System", "No hosts configured"),
}
}
fn render_metrics(
frame: &mut Frame,
_host: &HostDisplayData,
metrics: &ServiceMetrics,
area: Rect,
) {
let summary = &metrics.summary;
let system_total = if summary.system_memory_total_mb > 0.0 {
summary.system_memory_total_mb
} else {
summary.memory_quota_mb
};
let system_used = if summary.system_memory_used_mb > 0.0 {
summary.system_memory_used_mb
} else {
summary.memory_used_mb
};
let usage_ratio = if system_total > 0.0 {
(system_used / system_total) * 100.0
} else {
0.0
};
let (perf_severity, _reason) = evaluate_performance(summary);
let border_color = match perf_severity {
PerfSeverity::Critical => Color::Red,
PerfSeverity::Warning => Color::Yellow,
PerfSeverity::Ok => Color::Green,
};
let memory_color = status_color_from_percentage(usage_ratio, 80.0, 95.0);
let cpu_load_color = status_color_for_cpu_load(summary.cpu_load_5);
let cpu_temp_color = status_color_from_metric(summary.cpu_temp_c, 80.0, 90.0);
let gpu_load_color = summary
.gpu_load_percent
.map(|value| status_color_from_percentage(value, 85.0, 95.0))
.unwrap_or(Color::Green);
let gpu_temp_color = summary
.gpu_temp_c
.map(|value| status_color_from_metric(Some(value), 75.0, 85.0))
.unwrap_or(Color::Green);
let cpu_icon_color = combined_color(&[cpu_load_color, cpu_temp_color]);
let gpu_icon_color = combined_color(&[gpu_load_color, gpu_temp_color]);
// Memory dataset
let memory_status = status_level_from_color(memory_color);
let mut memory_dataset = WidgetDataSet::new(vec!["Memory usage".to_string()], Some(WidgetStatus::new(memory_status)));
memory_dataset.add_row(
Some(WidgetStatus::new(memory_status)),
"",
vec![WidgetValue::new(format!("{:.1} / {:.1}", system_used, system_total))],
);
// CPU dataset
let cpu_status = status_level_from_color(cpu_icon_color);
let mut cpu_dataset = WidgetDataSet::new(vec!["CPU load".to_string(), "CPU temp".to_string(), "CPU freq".to_string()], Some(WidgetStatus::new(cpu_status)));
cpu_dataset.add_row(
Some(WidgetStatus::new(cpu_status)),
"",
vec![
WidgetValue::new(format!("{:.2}{:.2}{:.2}", summary.cpu_load_1, summary.cpu_load_5, summary.cpu_load_15)),
WidgetValue::new(format_optional_metric(summary.cpu_temp_c, "°C")),
WidgetValue::new(format_optional_metric(summary.cpu_freq_mhz, " MHz")),
],
);
// GPU dataset
let gpu_status = status_level_from_color(gpu_icon_color);
let mut gpu_dataset = WidgetDataSet::new(vec!["GPU load".to_string(), "GPU temp".to_string()], Some(WidgetStatus::new(gpu_status)));
gpu_dataset.add_row(
Some(WidgetStatus::new(gpu_status)),
"",
vec![
WidgetValue::new(summary
.gpu_load_percent
.map(|value| format_optional_percent(Some(value)))
.unwrap_or_else(|| "".to_string())),
WidgetValue::new(summary
.gpu_temp_c
.map(|value| format_optional_metric(Some(value), "°C"))
.unwrap_or_else(|| "".to_string())),
],
);
// Determine overall widget status based on worst case
let overall_status_level = match perf_severity {
PerfSeverity::Critical => StatusLevel::Error,
PerfSeverity::Warning => StatusLevel::Warning,
PerfSeverity::Ok => StatusLevel::Ok,
};
let overall_status = Some(WidgetStatus::new(overall_status_level));
// Render all three datasets in a single combined widget
render_combined_widget_data(frame, area, "CPU / Memory".to_string(), overall_status, vec![memory_dataset, cpu_dataset, gpu_dataset]);
}
#[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord)]
pub(crate) enum PerfSeverity {
Ok,
Warning,
Critical,
}
fn format_optional_metric(value: Option<f32>, unit: &str) -> String {
match value {
Some(number) => format!("{:.1}{}", number, unit),
None => "".to_string(),
}
}
fn format_optional_percent(value: Option<f32>) -> String {
match value {
Some(number) => format!("{:.0}%", number),
None => "".to_string(),
}
}
fn status_level_from_color(color: Color) -> StatusLevel {
match color {
Color::Red => StatusLevel::Error,
Color::Yellow => StatusLevel::Warning,
_ => StatusLevel::Ok,
}
}
pub(crate) fn evaluate_performance(summary: &ServiceSummary) -> (PerfSeverity, Option<String>) {
let mem_percent = if summary.system_memory_total_mb > 0.0 {
(summary.system_memory_used_mb / summary.system_memory_total_mb) * 100.0
} else if summary.memory_quota_mb > 0.0 {
(summary.memory_used_mb / summary.memory_quota_mb) * 100.0
} else {
0.0
};
let mut severity = PerfSeverity::Ok;
let mut reason: Option<String> = None;
let mut consider = |level: PerfSeverity, message: String| {
if level > severity {
severity = level;
reason = Some(message);
}
};
if mem_percent >= 95.0 {
consider(PerfSeverity::Critical, format!("RAM {:.0}%", mem_percent));
} else if mem_percent >= 80.0 {
consider(PerfSeverity::Warning, format!("RAM {:.0}%", mem_percent));
}
let load = summary.cpu_load_5;
if load >= 4.0 {
consider(PerfSeverity::Critical, format!("CPU load {:.2}", load));
} else if load >= 2.0 {
consider(PerfSeverity::Warning, format!("CPU load {:.2}", load));
}
if let Some(temp) = summary.cpu_temp_c {
if temp >= 90.0 {
consider(PerfSeverity::Critical, format!("CPU temp {:.0}°C", temp));
} else if temp >= 80.0 {
consider(PerfSeverity::Warning, format!("CPU temp {:.0}°C", temp));
}
}
if let Some(load) = summary.gpu_load_percent {
if load >= 95.0 {
consider(PerfSeverity::Critical, format!("GPU load {:.0}%", load));
} else if load >= 85.0 {
consider(PerfSeverity::Warning, format!("GPU load {:.0}%", load));
}
}
if let Some(temp) = summary.gpu_temp_c {
if temp >= 85.0 {
consider(PerfSeverity::Critical, format!("GPU temp {:.0}°C", temp));
} else if temp >= 75.0 {
consider(PerfSeverity::Warning, format!("GPU temp {:.0}°C", temp));
}
}
if severity == PerfSeverity::Ok {
(PerfSeverity::Ok, None)
} else {
(severity, reason)
}
}

447
dashboard/src/ui/widget.rs Normal file
View File

@ -0,0 +1,447 @@
use ratatui::layout::{Constraint, Rect};
use ratatui::style::{Color, Modifier, Style};
use ratatui::text::{Line, Span};
use ratatui::widgets::{Block, Borders, Cell, Paragraph, Row, Table, Wrap};
use ratatui::Frame;
pub fn heading_row_style() -> Style {
neutral_text_style().add_modifier(Modifier::BOLD)
}
fn neutral_text_style() -> Style {
Style::default()
}
fn neutral_title_span(title: &str) -> Span<'static> {
Span::styled(
title.to_string(),
neutral_text_style().add_modifier(Modifier::BOLD),
)
}
fn neutral_border_style(color: Color) -> Style {
Style::default().fg(color)
}
pub fn status_color_from_percentage(value: f32, warn: f32, crit: f32) -> Color {
if value >= crit {
Color::Red
} else if value >= warn {
Color::Yellow
} else {
Color::Green
}
}
pub fn status_color_from_metric(value: Option<f32>, warn: f32, crit: f32) -> Color {
match value {
Some(v) if v >= crit => Color::Red,
Some(v) if v >= warn => Color::Yellow,
_ => Color::Green,
}
}
pub fn status_color_for_cpu_load(load: f32) -> Color {
if load >= 4.0 {
Color::Red
} else if load >= 2.0 {
Color::Yellow
} else {
Color::Green
}
}
pub fn combined_color(colors: &[Color]) -> Color {
if colors.iter().any(|&c| c == Color::Red) {
Color::Red
} else if colors.iter().any(|&c| c == Color::Yellow) {
Color::Yellow
} else {
Color::Green
}
}
pub fn render_placeholder(frame: &mut Frame, area: Rect, title: &str, message: &str) {
let block = Block::default()
.title(neutral_title_span(title))
.borders(Borders::ALL)
.border_style(neutral_border_style(Color::Gray));
let inner = block.inner(area);
frame.render_widget(block, area);
frame.render_widget(
Paragraph::new(Line::from(message))
.wrap(Wrap { trim: true })
.style(neutral_text_style()),
inner,
);
}
pub fn render_widget_data(frame: &mut Frame, area: Rect, data: WidgetData) {
render_combined_widget_data(frame, area, data.title, data.status, vec![data.dataset]);
}
pub fn render_combined_widget_data(frame: &mut Frame, area: Rect, title: String, status: Option<WidgetStatus>, datasets: Vec<WidgetDataSet>) {
if datasets.is_empty() {
return;
}
// Create border and title - determine color from widget status
let border_color = status.as_ref()
.map(|s| s.status.to_color())
.unwrap_or(Color::Reset);
let block = Block::default()
.title(neutral_title_span(&title))
.borders(Borders::ALL)
.border_style(neutral_border_style(border_color));
let inner = block.inner(area);
frame.render_widget(block, area);
// Split multi-row datasets into single-row datasets when wrapping is needed
let split_datasets = split_multirow_datasets_with_area(datasets, inner);
let mut current_y = inner.y;
for dataset in split_datasets.iter() {
if current_y >= inner.y + inner.height {
break; // No more space
}
current_y += render_dataset_with_wrapping(frame, dataset, inner, current_y);
}
}
fn split_multirow_datasets_with_area(datasets: Vec<WidgetDataSet>, inner: Rect) -> Vec<WidgetDataSet> {
let mut result = Vec::new();
for dataset in datasets {
if dataset.rows.len() <= 1 {
// Single row or empty - keep as is
result.push(dataset);
} else {
// Multiple rows - check if wrapping is needed using actual available width
if dataset_needs_wrapping_with_width(&dataset, inner.width) {
// Split into separate datasets for individual wrapping
for row in dataset.rows {
let single_row_dataset = WidgetDataSet {
colnames: dataset.colnames.clone(),
status: dataset.status.clone(),
rows: vec![row],
};
result.push(single_row_dataset);
}
} else {
// No wrapping needed - keep as single dataset
result.push(dataset);
}
}
}
result
}
fn dataset_needs_wrapping_with_width(dataset: &WidgetDataSet, available_width: u16) -> bool {
// Calculate column widths
let mut column_widths = Vec::new();
for (col_index, colname) in dataset.colnames.iter().enumerate() {
let mut max_width = colname.chars().count() as u16;
// Check data rows for this column width
for row in &dataset.rows {
if let Some(widget_value) = row.values.get(col_index) {
let data_width = widget_value.data.chars().count() as u16;
max_width = max_width.max(data_width);
}
}
let column_width = (max_width + 1).min(25).max(6);
column_widths.push(column_width);
}
// Calculate total width needed
let status_col_width = 1u16;
let col_spacing = 1u16;
let mut total_width = status_col_width + col_spacing;
for &col_width in &column_widths {
total_width += col_width + col_spacing;
}
total_width > available_width
}
fn render_dataset_with_wrapping(frame: &mut Frame, dataset: &WidgetDataSet, inner: Rect, start_y: u16) -> u16 {
if dataset.colnames.is_empty() || dataset.rows.is_empty() {
return 0;
}
// Calculate column widths
let mut column_widths = Vec::new();
for (col_index, colname) in dataset.colnames.iter().enumerate() {
let mut max_width = colname.chars().count() as u16;
// Check data rows for this column width
for row in &dataset.rows {
if let Some(widget_value) = row.values.get(col_index) {
let data_width = widget_value.data.chars().count() as u16;
max_width = max_width.max(data_width);
}
}
let column_width = (max_width + 1).min(25).max(6);
column_widths.push(column_width);
}
let status_col_width = 1u16;
let col_spacing = 1u16;
let available_width = inner.width;
// Determine how many columns fit
let mut total_width = status_col_width + col_spacing;
let mut cols_that_fit = 0;
for &col_width in &column_widths {
let new_total = total_width + col_width + col_spacing;
if new_total <= available_width {
total_width = new_total;
cols_that_fit += 1;
} else {
break;
}
}
if cols_that_fit == 0 {
cols_that_fit = 1; // Always show at least one column
}
let mut current_y = start_y;
let mut col_start = 0;
let mut is_continuation = false;
// Render wrapped sections
while col_start < dataset.colnames.len() {
let col_end = (col_start + cols_that_fit).min(dataset.colnames.len());
let section_colnames = &dataset.colnames[col_start..col_end];
let section_widths = &column_widths[col_start..col_end];
// Render header for this section
let mut header_cells = vec![];
// Status cell
if is_continuation {
header_cells.push(Cell::from(""));
} else {
header_cells.push(Cell::from(""));
}
// Column headers
for colname in section_colnames {
header_cells.push(Cell::from(Line::from(vec![Span::styled(
colname.clone(),
heading_row_style(),
)])));
}
let header_row = Row::new(header_cells).style(heading_row_style());
// Build constraint widths for this section
let mut constraints = vec![Constraint::Length(status_col_width)];
for &width in section_widths {
constraints.push(Constraint::Length(width));
}
let header_table = Table::new(vec![header_row])
.widths(&constraints)
.column_spacing(col_spacing)
.style(neutral_text_style());
frame.render_widget(header_table, Rect {
x: inner.x,
y: current_y,
width: inner.width,
height: 1,
});
current_y += 1;
// Render data rows for this section
for row in &dataset.rows {
if current_y >= inner.y + inner.height {
break;
}
let mut cells = vec![];
// Status cell (only show on first section)
if col_start == 0 {
match &row.status {
Some(s) => {
let color = s.status.to_color();
let icon = s.status.to_icon();
cells.push(Cell::from(Line::from(vec![Span::styled(
icon.to_string(),
Style::default().fg(color),
)])));
},
None => cells.push(Cell::from("")),
}
} else {
cells.push(Cell::from(""));
}
// Data cells for this section
for col_idx in col_start..col_end {
if let Some(widget_value) = row.values.get(col_idx) {
let content = &widget_value.data;
if content.is_empty() {
cells.push(Cell::from(""));
} else {
cells.push(Cell::from(Line::from(vec![Span::styled(
content.to_string(),
neutral_text_style(),
)])));
}
} else {
cells.push(Cell::from(""));
}
}
let data_row = Row::new(cells);
let data_table = Table::new(vec![data_row])
.widths(&constraints)
.column_spacing(col_spacing)
.style(neutral_text_style());
frame.render_widget(data_table, Rect {
x: inner.x,
y: current_y,
width: inner.width,
height: 1,
});
current_y += 1;
}
col_start = col_end;
is_continuation = true;
}
current_y - start_y
}
#[derive(Clone)]
pub struct WidgetData {
pub title: String,
pub status: Option<WidgetStatus>,
pub dataset: WidgetDataSet,
}
#[derive(Clone)]
pub struct WidgetDataSet {
pub colnames: Vec<String>,
pub status: Option<WidgetStatus>,
pub rows: Vec<WidgetRow>,
}
#[derive(Clone)]
pub struct WidgetRow {
pub status: Option<WidgetStatus>,
pub values: Vec<WidgetValue>,
}
#[derive(Clone)]
pub struct WidgetValue {
pub data: String,
}
#[derive(Clone, Copy, Debug)]
pub enum StatusLevel {
Ok,
Warning,
Error,
Unknown,
}
#[derive(Clone)]
pub struct WidgetStatus {
pub status: StatusLevel,
}
impl WidgetData {
pub fn new(title: impl Into<String>, status: Option<WidgetStatus>, colnames: Vec<String>) -> Self {
Self {
title: title.into(),
status: status.clone(),
dataset: WidgetDataSet {
colnames,
status,
rows: Vec::new(),
},
}
}
pub fn add_row(&mut self, status: Option<WidgetStatus>, _description: impl Into<String>, values: Vec<WidgetValue>) -> &mut Self {
self.dataset.rows.push(WidgetRow {
status,
values,
});
self
}
}
impl WidgetDataSet {
pub fn new(colnames: Vec<String>, status: Option<WidgetStatus>) -> Self {
Self {
colnames,
status,
rows: Vec::new(),
}
}
pub fn add_row(&mut self, status: Option<WidgetStatus>, _description: impl Into<String>, values: Vec<WidgetValue>) -> &mut Self {
self.rows.push(WidgetRow {
status,
values,
});
self
}
}
impl WidgetValue {
pub fn new(data: impl Into<String>) -> Self {
Self {
data: data.into(),
}
}
}
impl WidgetStatus {
pub fn new(status: StatusLevel) -> Self {
Self {
status,
}
}
}
impl StatusLevel {
pub fn to_color(self) -> Color {
match self {
StatusLevel::Ok => Color::Green,
StatusLevel::Warning => Color::Yellow,
StatusLevel::Error => Color::Red,
StatusLevel::Unknown => Color::Reset, // Terminal default
}
}
pub fn to_icon(self) -> &'static str {
match self {
StatusLevel::Ok => "",
StatusLevel::Warning => "!",
StatusLevel::Error => "",
StatusLevel::Unknown => "?",
}
}
}