Implement systemd service approach for nixos-rebuild operations

- Add cm-rebuild systemd service for process isolation - Add sudo permissions for service control and journal access - Remove verbose flag for cleaner output - Ensures reliable rebuild operations without agent crashes
Update Cargo.toml versions to 0.1.11
2025-10-26 23:18:09 +01:00 · 2025-10-26 14:12:03 +01:00 · 2025-10-26 13:34:56 +01:00 · 2025-10-26 13:00:53 +01:00 · 2025-10-26 11:39:03 +01:00 · 2025-10-26 10:42:26 +01:00
36 changed files with 1031 additions and 1615 deletions
--- a/.gitea/workflows/release.yml
+++ b/.gitea/workflows/release.yml
@@ -0,0 +1,128 @@
+name: Build and Release
+
+on:
+  push:
+    tags:
+      - 'v*'
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Version to release (e.g., v0.1.0)'
+        required: true
+        default: 'v0.1.0'
+
+jobs:
+  build-and-release:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Rust
+        uses: actions-rs/toolchain@v1
+        with:
+          toolchain: stable
+          profile: minimal
+          override: true
+
+      - name: Install system dependencies
+        run: |
+          apt-get update
+          apt-get install -y pkg-config libssl-dev libzmq3-dev
+
+      - name: Build workspace (static)
+        run: |
+          export RUSTFLAGS="-C target-feature=+crt-static"
+          cargo build --release --workspace --target x86_64-unknown-linux-gnu
+
+      - name: Create release directory
+        run: |
+          mkdir -p release
+          cp target/x86_64-unknown-linux-gnu/release/cm-dashboard release/cm-dashboard-linux-x86_64
+          cp target/x86_64-unknown-linux-gnu/release/cm-dashboard-agent release/cm-dashboard-agent-linux-x86_64
+
+      - name: Create tarball
+        run: |
+          cd release
+          tar -czf cm-dashboard-linux-x86_64.tar.gz cm-dashboard-linux-x86_64 cm-dashboard-agent-linux-x86_64
+
+      - name: Set version variable
+        id: version
+        run: |
+          if [ "${{ gitea.event_name }}" == "workflow_dispatch" ]; then
+            echo "VERSION=${{ gitea.event.inputs.version }}" >> $GITHUB_OUTPUT
+          else
+            echo "VERSION=${GITHUB_REF#refs/tags/}" >> $GITHUB_OUTPUT
+          fi
+
+      - name: Create Release with curl
+        env:
+          GITEA_TOKEN: ${{ secrets.GITEATOKEN }}
+        run: |
+          VERSION="${{ steps.version.outputs.VERSION }}"
+          
+          # Create release
+          curl -X POST \
+            -H "Authorization: token $GITEA_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d '{
+              "tag_name": "'$VERSION'",
+              "name": "cm-dashboard '$VERSION'",
+              "body": "## cm-dashboard '$VERSION'\n\nPre-built binaries for Linux x86_64:\n- cm-dashboard-linux-x86_64 - Dashboard TUI binary\n- cm-dashboard-agent-linux-x86_64 - Agent daemon binary\n- cm-dashboard-linux-x86_64.tar.gz - Combined tarball"
+            }' \
+            "https://gitea.cmtec.se/api/v1/repos/cm/cm-dashboard/releases"
+          
+          # Get release ID
+          RELEASE_ID=$(curl -s -H "Authorization: token $GITEA_TOKEN" \
+            "https://gitea.cmtec.se/api/v1/repos/cm/cm-dashboard/releases/tags/$VERSION" | \
+            grep -o '"id":[0-9]*' | head -1 | cut -d':' -f2)
+          
+          # Upload binaries
+          curl -X POST \
+            -H "Authorization: token $GITEA_TOKEN" \
+            -F "attachment=@release/cm-dashboard-linux-x86_64" \
+            "https://gitea.cmtec.se/api/v1/repos/cm/cm-dashboard/releases/$RELEASE_ID/assets?name=cm-dashboard-linux-x86_64"
+          
+          curl -X POST \
+            -H "Authorization: token $GITEA_TOKEN" \
+            -F "attachment=@release/cm-dashboard-agent-linux-x86_64" \
+            "https://gitea.cmtec.se/api/v1/repos/cm/cm-dashboard/releases/$RELEASE_ID/assets?name=cm-dashboard-agent-linux-x86_64"
+          
+          curl -X POST \
+            -H "Authorization: token $GITEA_TOKEN" \
+            -F "attachment=@release/cm-dashboard-linux-x86_64.tar.gz" \
+            "https://gitea.cmtec.se/api/v1/repos/cm/cm-dashboard/releases/$RELEASE_ID/assets?name=cm-dashboard-linux-x86_64.tar.gz"
+
+      - name: Update NixOS Configuration
+        env:
+          GITEA_TOKEN: ${{ secrets.GITEATOKEN }}
+        run: |
+          VERSION="${{ steps.version.outputs.VERSION }}"
+          
+          # Clone nixosbox repository
+          git clone https://$GITEA_TOKEN@gitea.cmtec.se/cm/nixosbox.git nixosbox-update
+          cd nixosbox-update
+          
+          # Get hash for the new release tarball
+          TARBALL_URL="https://gitea.cmtec.se/cm/cm-dashboard/releases/download/$VERSION/cm-dashboard-linux-x86_64.tar.gz"
+          
+          # Download tarball to get correct hash
+          curl -L -o cm-dashboard.tar.gz "$TARBALL_URL"
+          # Convert sha256 hex to base64 for Nix hash format using Python
+          NEW_HASH=$(sha256sum cm-dashboard.tar.gz | cut -d' ' -f1)
+          NIX_HASH="sha256-$(python3 -c "import base64, binascii; print(base64.b64encode(binascii.unhexlify('$NEW_HASH')).decode())")"
+          
+          # Update the NixOS configuration
+          sed -i "s|version = \"v[^\"]*\"|version = \"$VERSION\"|" hosts/common/cm-dashboard.nix
+          sed -i "s|sha256 = \"sha256-[^\"]*\"|sha256 = \"$NIX_HASH\"|" hosts/common/cm-dashboard.nix
+          
+          # Commit and push changes
+          git config user.name "Gitea Actions"
+          git config user.email "actions@gitea.cmtec.se"
+          git add hosts/common/cm-dashboard.nix
+          git commit -m "Auto-update cm-dashboard to $VERSION
+
+          - Update version to $VERSION with automated release
+          - Update tarball hash for new static binaries
+          - Automated update from cm-dashboard release workflow"
+          git push
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -28,18 +28,21 @@ All keyboard navigation and service selection features successfully implemented:
 - ✅ **Smart Panel Switching**: Only cycles through panels with data (backup panel conditional)
 - ✅ **Scroll Support**: All panels support content scrolling with proper overflow indicators

-**Current Status - October 24, 2025:**
+**Current Status - October 26, 2025:**
 - All keyboard navigation features working correctly ✅
 - Service selection cursor implemented with focus-aware highlighting ✅
 - Panel scrolling fixed for System, Services, and Backup panels ✅
 - Build display working: "Build: 25.05.20251004.3bcc93c" ✅
- Configuration hash display implemented: "Config: d16f0d0" ✅
+- Agent version display working: "Agent: 3kvc03nd" ✅
+- Cross-host version comparison implemented ✅
+- Automated binary release system working ✅
+- SMART data consolidated into disk collector ✅

-**Layout Achieved:**
+**Current Layout:**
 ```
 NixOS:
 Build: 25.05.20251004.3bcc93c
-Config: d16f0d0  # Shows actual nixosbox config hash
+Agent: 3kvc03nd  # Shows agent version (nix store hash)
 Active users: cm, simon
 CPU:
 ● Load: 0.02 0.31 0.86 • 3000MHz
@@ -55,7 +58,8 @@ Storage:
 **System panel layout fully implemented with blue tree symbols ✅**
 **Tree symbols now use consistent blue theming across all panels ✅**
 **Overflow handling restored for all widgets ("... and X more") ✅**
-**Agent hash display working correctly ✅**
+**Agent version display working correctly ✅**
+**Cross-host version comparison logging warnings ✅**

 ### Current Keyboard Navigation Implementation

@@ -88,6 +92,56 @@ Storage:
 - ✅ **Git Clone Approach**: Implemented for nixos-rebuild to avoid directory permissions
 - ✅ **Visual Feedback**: Directional arrows for service status (↑ starting, ↓ stopping, ↻ restarting)

+### Terminal Popup for Real-time Output - IMPLEMENTED ✅
+
+**Status (as of 2025-10-26):**
+- ✅ **Terminal Popup UI**: 80% screen coverage with terminal styling and color-coded output
+- ✅ **ZMQ Streaming Protocol**: CommandOutputMessage for real-time output transmission
+- ✅ **Keyboard Controls**: ESC/Q to close, ↑↓ to scroll, manual close (no auto-close)
+- ✅ **Real-time Display**: Live streaming of command output as it happens
+- ✅ **Version-based Agent Reporting**: Shows "Agent: v0.1.13" instead of nix store hash
+
+**Current Implementation Issues:**
+- ❌ **Agent Process Crashes**: Agent dies during nixos-rebuild execution
+- ❌ **Inconsistent Output**: Different outputs each time 'R' is pressed
+- ❌ **Limited Output Visibility**: Not capturing all nixos-rebuild progress
+
+**PLANNED SOLUTION - Systemd Service Approach:**
+
+**Problem**: Direct nixos-rebuild execution in agent causes process crashes and inconsistent output.
+
+**Solution**: Create dedicated systemd service for rebuild operations.
+
+**Implementation Plan:**
+1. **NixOS Systemd Service**:
+   ```nix
+   systemd.services.cm-rebuild = {
+     description = "CM Dashboard NixOS Rebuild";
+     serviceConfig = {
+       Type = "oneshot";
+       ExecStart = "${pkgs.nixos-rebuild}/bin/nixos-rebuild switch --flake . --option sandbox false";
+       WorkingDirectory = "/var/lib/cm-dashboard/nixos-config";
+       User = "root";
+       StandardOutput = "journal";
+       StandardError = "journal";
+     };
+   };
+   ```
+
+2. **Agent Modification**:
+   - Replace direct nixos-rebuild execution with: `systemctl start cm-rebuild`
+   - Stream output via: `journalctl -u cm-rebuild -f --no-pager`
+   - Monitor service status for completion detection
+
+3. **Benefits**:
+   - **Process Isolation**: Service runs independently, won't crash agent
+   - **Consistent Output**: Always same deterministic rebuild process
+   - **Proper Logging**: systemd journal handles all output management
+   - **Resource Management**: systemd manages cleanup and resource limits
+   - **Status Tracking**: Can query service status (running/failed/success)
+
+**Next Priority**: Implement systemd service approach for reliable rebuild operations.
+
 **Keyboard Controls Status:**
 - **Services Panel**: 
  - R (restart) ✅ Working
@@ -120,15 +174,44 @@ Latest backup:              →    Latest backup:
 └─ Duration: 1.3m                 └─ [██████       ] 60%
 ```

+**Critical Configuration Hash Fix - HIGH PRIORITY:**
+
+**Problem:** Configuration hash currently shows git commit hash instead of actual deployed system hash.
+
+**Current (incorrect):** 
+- Shows git hash: `db11f82` (source repository commit)
+- Not accurate - doesn't reflect what's actually deployed
+
+**Target (correct):**
+- Show nix store hash: `d8ivwiar` (first 8 chars from deployed system)  
+- Source: `/nix/store/d8ivwiarhwhgqzskj6q2482r58z46qjf-nixos-system-cmbox-25.05.20251004.3bcc93c`
+- Pattern: Extract hash from `/nix/store/HASH-nixos-system-HOSTNAME-VERSION`
+
+**Benefits:**
+1. **Deployment Verification:** Confirms rebuild actually succeeded
+2. **Accurate Status:** Shows what's truly running, not just source
+3. **Rebuild Completion Detection:** Hash change = rebuild completed
+4. **Rollback Tracking:** Each deployment has unique identifier
+
+**Implementation Required:**
+1. Agent extracts nix store hash from `ls -la /run/current-system` 
+2. Reports this as `system_config_hash` metric instead of git hash
+3. Dashboard displays first 8 characters: `Config: d8ivwiar`
+
 **Next Session Priority Tasks:**

 **Remaining Features:**
-1. **Command Response Protocol**:
+1. **Fix Configuration Hash Display (CRITICAL)**:
+   - Use nix store hash instead of git commit hash
+   - Extract from `/run/current-system` -> `/nix/store/HASH-nixos-system-*`
+   - Enables proper rebuild completion detection
+
+2. **Command Response Protocol**:
   - Agent sends command completion/failure back to dashboard via ZMQ
   - Dashboard updates UI status from ⏳ to ● when commands complete
   - Clear success/failure status after timeout

-2. **Backup Panel Features**:
+3. **Backup Panel Features**:
   - Implement backup trigger functionality (B key)
   - Complete visual feedback for backup operations
   - Add backup progress indicators
@@ -244,60 +327,83 @@ NEVER implement code without first getting explicit user agreement on the approa
 - ✅ "Restructure storage widget with improved layout"
 - ✅ "Update CPU thresholds to production values"

-## NixOS Configuration Updates
+## Development and Deployment Architecture

-When code changes are made to cm-dashboard, the NixOS configuration at `~/nixosbox` must be updated to deploy the changes.
+**CRITICAL:** Development and deployment paths are completely separate:

-### Update Process
+### Development Path
+- **Location:** `~/projects/nixosbox` 
+- **Purpose:** Development workflow only - for committing new cm-dashboard code
+- **Access:** Only for developers to commit changes
+- **Code Access:** Running cm-dashboard code shall NEVER access this path

-1. **Get Latest Commit Hash**
+### Deployment Path  
+- **Location:** `/var/lib/cm-dashboard/nixos-config`
+- **Purpose:** Production deployment only - agent clones/pulls from git
+- **Access:** Only cm-dashboard agent for deployment operations
+- **Workflow:** git pull → `/var/lib/cm-dashboard/nixos-config` → nixos-rebuild

+### Git Flow
+```
+Development: ~/projects/nixosbox → git commit → git push
+Deployment:  git pull → /var/lib/cm-dashboard/nixos-config → rebuild
+```
+
+## Automated Binary Release System
+
+**IMPLEMENTED:** cm-dashboard now uses automated binary releases instead of source builds.
+
+### Release Workflow
+
+1. **Automated Release Creation**
+   - Gitea Actions workflow builds static binaries on tag push
+   - Creates release with `cm-dashboard-linux-x86_64.tar.gz` tarball
+   - No manual intervention required for binary generation
+
+2. **Creating New Releases**
   ```bash
-   git log -1 --format="%H"
+   cd ~/projects/cm-dashboard
+   git tag v0.1.X
+   git push origin v0.1.X
   ```
+   
+   This automatically:
+   - Builds static binaries with `RUSTFLAGS="-C target-feature=+crt-static"`
+   - Creates GitHub-style release with tarball
+   - Uploads binaries via Gitea API

-2. **Update NixOS Configuration**
-   Edit `~/nixosbox/hosts/common/cm-dashboard.nix`:
+3. **NixOS Configuration Updates**
+   Edit `~/projects/nixosbox/hosts/common/cm-dashboard.nix`:

   ```nix
-   src = pkgs.fetchgit {
-     url = "https://gitea.cmtec.se/cm/cm-dashboard.git";
-     rev = "NEW_COMMIT_HASH_HERE";
-     sha256 = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="; # Placeholder
+   version = "v0.1.X";
+   src = pkgs.fetchurl {
+     url = "https://gitea.cmtec.se/cm/cm-dashboard/releases/download/${version}/cm-dashboard-linux-x86_64.tar.gz";
+     sha256 = "sha256-NEW_HASH_HERE";
   };
   ```

-3. **Get Correct Source Hash**
-   Build with placeholder hash to get the actual hash:
-
+4. **Get Release Hash**
   ```bash
-   cd ~/nixosbox
-   nix-build --no-out-link -E 'with import <nixpkgs> {}; fetchgit {
-     url = "https://gitea.cmtec.se/cm/cm-dashboard.git";
-     rev = "NEW_COMMIT_HASH";
+   cd ~/projects/nixosbox
+   nix-build --no-out-link -E 'with import <nixpkgs> {}; fetchurl {
+     url = "https://gitea.cmtec.se/cm/cm-dashboard/releases/download/v0.1.X/cm-dashboard-linux-x86_64.tar.gz";
     sha256 = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
   }' 2>&1 | grep "got:"
   ```

-   Example output:
-
-   ```
-   error: hash mismatch in fixed-output derivation '/nix/store/...':
-            specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
-               got:    sha256-x8crxNusOUYRrkP9mYEOG+Ga3JCPIdJLkEAc5P1ZxdQ=
-   ```
-
-4. **Update Configuration with Correct Hash**
-   Replace the placeholder with the hash from the error message (the "got:" line).
-
-5. **Commit NixOS Configuration**
-
+5. **Commit and Deploy**
   ```bash
-   cd ~/nixosbox
+   cd ~/projects/nixosbox
   git add hosts/common/cm-dashboard.nix
-   git commit -m "Update cm-dashboard to latest version (SHORT_HASH)"
+   git commit -m "Update cm-dashboard to v0.1.X with static binaries"
   git push
   ```

-6. **Rebuild System**
-   The user handles the system rebuild step - this cannot be automated.
+### Benefits
+
+- **No compilation overhead** on each host
+- **Consistent static binaries** across all hosts
+- **Faster deployments** - download vs compile
+- **No library dependency issues** - static linking
+- **Automated pipeline** - tag push triggers everything
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -270,7 +270,7 @@ checksum = "a1d728cc89cf3aee9ff92b05e62b19ee65a02b5702cff7d5a377e32c6ae29d8d"

 [[package]]
 name = "cm-dashboard"
-version = "0.1.0"
+version = "0.1.13"
 dependencies = [
 "anyhow",
 "chrono",
@@ -291,7 +291,7 @@ dependencies = [

 [[package]]
 name = "cm-dashboard-agent"
-version = "0.1.0"
+version = "0.1.13"
 dependencies = [
 "anyhow",
 "async-trait",
@@ -314,7 +314,7 @@ dependencies = [

 [[package]]
 name = "cm-dashboard-shared"
-version = "0.1.0"
+version = "0.1.13"
 dependencies = [
 "chrono",
 "serde",
--- a/README.md
+++ b/README.md
@@ -152,10 +152,13 @@ interval_seconds = 10
 memory_warning_mb = 1000.0
 memory_critical_mb = 2000.0
 service_name_filters = [
-  "nginx", "postgresql", "redis", "docker", "sshd"
+  "nginx*", "postgresql*", "redis*", "docker*", "sshd*", 
+  "gitea*", "immich*", "haasp*", "mosquitto*", "mysql*", 
+  "unifi*", "vaultwarden*"
 ]
 excluded_services = [
-  "nginx-config-reload", "sshd-keygen"
+  "nginx-config-reload", "sshd-keygen", "systemd-", 
+  "getty@", "user@", "dbus-", "NetworkManager-"
 ]

 [notifications]
--- a/agent/Cargo.toml
+++ b/agent/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "cm-dashboard-agent"
-version = "0.1.0"
+version = "0.1.13"
 edition = "2021"

 [dependencies]
--- a/agent/src/agent.rs
+++ b/agent/src/agent.rs
@@ -9,7 +9,7 @@ use crate::config::AgentConfig;
 use crate::metrics::MetricCollectionManager;
 use crate::notifications::NotificationManager;
 use crate::status::HostStatusManager;
-use cm_dashboard_shared::{Metric, MetricMessage};
+use cm_dashboard_shared::{CommandOutputMessage, Metric, MetricMessage, MetricValue, Status};

 pub struct Agent {
    hostname: String,
@@ -86,9 +86,9 @@ impl Agent {
                    }
                }
                _ = transmission_interval.tick() => {
-                    // Send all cached metrics via ZMQ every 1 second
-                    if let Err(e) = self.broadcast_all_cached_metrics().await {
-                        error!("Failed to broadcast cached metrics: {}", e);
+                    // Send all metrics via ZMQ every 1 second
+                    if let Err(e) = self.broadcast_all_metrics().await {
+                        error!("Failed to broadcast metrics: {}", e);
                    }
                }
                _ = notification_interval.tick() => {
@@ -152,37 +152,59 @@ impl Agent {
        Ok(())
    }

-    async fn broadcast_all_cached_metrics(&mut self) -> Result<()> {
-        debug!("Broadcasting all cached metrics via ZMQ");
+    async fn broadcast_all_metrics(&mut self) -> Result<()> {
+        debug!("Broadcasting all metrics via ZMQ");

-        // Get all cached metrics from the metric manager
-        let mut cached_metrics = self.metric_manager.get_all_cached_metrics().await?;
+        // Get all current metrics from collectors
+        let mut metrics = self.metric_manager.collect_all_metrics().await?;

        // Add the host status summary metric from status manager
        let host_status_metric = self.host_status_manager.get_host_status_metric();
-        cached_metrics.push(host_status_metric);
+        metrics.push(host_status_metric);

-        if cached_metrics.is_empty() {
-            debug!("No cached metrics to broadcast");
+        // Add agent version metric for cross-host version comparison
+        let version_metric = self.get_agent_version_metric();
+        metrics.push(version_metric);
+
+        if metrics.is_empty() {
+            debug!("No metrics to broadcast");
            return Ok(());
        }

-        debug!("Broadcasting {} cached metrics (including host status summary)", cached_metrics.len());
+        debug!("Broadcasting {} metrics (including host status summary)", metrics.len());

-        // Create and send message with all cached data
-        let message = MetricMessage::new(self.hostname.clone(), cached_metrics);
+        // Create and send message with all current data
+        let message = MetricMessage::new(self.hostname.clone(), metrics);
        self.zmq_handler.publish_metrics(&message).await?;

-        debug!("Cached metrics broadcasted successfully");
+        debug!("Metrics broadcasted successfully");
        Ok(())
    }

    async fn process_metrics(&mut self, metrics: &[Metric]) {
        for metric in metrics {
-            self.host_status_manager.process_metric(metric, &mut self.notification_manager, self.metric_manager.get_cache_manager()).await;
+            self.host_status_manager.process_metric(metric, &mut self.notification_manager).await;
        }
    }

+    /// Create agent version metric for cross-host version comparison
+    fn get_agent_version_metric(&self) -> Metric {
+        // Get version from executable path (same logic as main.rs get_version)
+        let version = self.get_agent_version();
+        
+        Metric::new(
+            "agent_version".to_string(),
+            MetricValue::String(version),
+            Status::Ok,
+        )
+    }
+
+    /// Get agent version from Cargo package version
+    fn get_agent_version(&self) -> String {
+        // Use the version from Cargo.toml (e.g., "0.1.11")
+        format!("v{}", env!("CARGO_PKG_VERSION"))
+    }
+
    async fn handle_commands(&mut self) -> Result<()> {
        // Try to receive commands (non-blocking)
        match self.zmq_handler.try_receive_command() {
@@ -281,73 +303,209 @@ impl Agent {
        Ok(())
    }

-    /// Handle NixOS system rebuild commands with git clone approach
+    /// Handle NixOS system rebuild commands with real-time output streaming
    async fn handle_system_rebuild(&self, git_url: &str, git_branch: &str, working_dir: &str, api_key_file: Option<&str>) -> Result<()> {
        info!("Starting NixOS system rebuild: {} @ {} -> {}", git_url, git_branch, working_dir);

+        let command_id = format!("rebuild_{}", chrono::Utc::now().timestamp());
+        
+        // Send initial status
+        self.send_command_output(&command_id, "SystemRebuild", "Starting NixOS system rebuild...").await?;
+
        // Enable maintenance mode before rebuild
        let maintenance_file = "/tmp/cm-maintenance";
        if let Err(e) = tokio::fs::File::create(maintenance_file).await {
-            error!("Failed to create maintenance mode file: {}", e);
+            self.send_command_output(&command_id, "SystemRebuild", &format!("Warning: Failed to create maintenance mode file: {}", e)).await?;
        } else {
-            info!("Maintenance mode enabled");
+            self.send_command_output(&command_id, "SystemRebuild", "Maintenance mode enabled").await?;
        }

        // Clone or update repository
-        let git_result = self.ensure_git_repository(git_url, git_branch, working_dir, api_key_file).await;
+        self.send_command_output(&command_id, "SystemRebuild", "Cloning/updating git repository...").await?;
+        let git_result = self.ensure_git_repository_with_output(&command_id, git_url, git_branch, working_dir, api_key_file).await;
        
-        // Execute nixos-rebuild if git operation succeeded
-        let rebuild_result = if git_result.is_ok() {
-            info!("Git repository ready, executing nixos-rebuild");
-            tokio::process::Command::new("sudo")
-                .arg("/run/current-system/sw/bin/nixos-rebuild")
-                .arg("switch")
-                .arg("--option")
-                .arg("sandbox")
-                .arg("false")
-                .arg("--flake")
-                .arg(".")
-                .current_dir(working_dir)
-                .output()
-                .await
-        } else {
-            return git_result.and_then(|_| unreachable!());
-        };
+        if git_result.is_err() {
+            self.send_command_output(&command_id, "SystemRebuild", &format!("Git operation failed: {:?}", git_result)).await?;
+            self.send_command_output_complete(&command_id, "SystemRebuild").await?;
+            return git_result;
+        }
+
+        self.send_command_output(&command_id, "SystemRebuild", "Git repository ready, starting nixos-rebuild...").await?;
+        
+        // Execute nixos-rebuild with real-time output streaming
+        let rebuild_result = self.execute_nixos_rebuild_with_streaming(&command_id, working_dir).await;

        // Always try to remove maintenance mode file
        if let Err(e) = tokio::fs::remove_file(maintenance_file).await {
            if e.kind() != std::io::ErrorKind::NotFound {
-                error!("Failed to remove maintenance mode file: {}", e);
+                self.send_command_output(&command_id, "SystemRebuild", &format!("Warning: Failed to remove maintenance mode file: {}", e)).await?;
            }
        } else {
-            info!("Maintenance mode disabled");
+            self.send_command_output(&command_id, "SystemRebuild", "Maintenance mode disabled").await?;
        }

-        // Check rebuild result
+        // Handle rebuild result
        match rebuild_result {
-            Ok(output) => {
-                if output.status.success() {
-                    info!("NixOS rebuild completed successfully");
-                    if !output.stdout.is_empty() {
-                        debug!("rebuild stdout: {}", String::from_utf8_lossy(&output.stdout));
-                    }
-                } else {
-                    let stderr = String::from_utf8_lossy(&output.stderr);
-                    error!("NixOS rebuild failed: {}", stderr);
-                    return Err(anyhow::anyhow!("nixos-rebuild failed: {}", stderr));
-                }
+            Ok(()) => {
+                self.send_command_output(&command_id, "SystemRebuild", "✓ NixOS rebuild completed successfully!").await?;
            }
            Err(e) => {
-                error!("Failed to execute nixos-rebuild: {}", e);
-                return Err(anyhow::anyhow!("Failed to execute nixos-rebuild: {}", e));
+                self.send_command_output(&command_id, "SystemRebuild", &format!("✗ NixOS rebuild failed: {}", e)).await?;
            }
        }

-        info!("System rebuild completed, triggering metric refresh");
+        // Signal completion
+        self.send_command_output_complete(&command_id, "SystemRebuild").await?;
+        
+        info!("System rebuild streaming completed");
        Ok(())
    }

-    /// Ensure git repository is cloned and up to date
+    /// Send command output line to dashboard
+    async fn send_command_output(&self, command_id: &str, command_type: &str, output_line: &str) -> Result<()> {
+        let message = CommandOutputMessage::new(
+            self.hostname.clone(),
+            command_id.to_string(),
+            command_type.to_string(),
+            output_line.to_string(),
+            false,
+        );
+        self.zmq_handler.publish_command_output(&message).await
+    }
+
+    /// Send command completion signal to dashboard
+    async fn send_command_output_complete(&self, command_id: &str, command_type: &str) -> Result<()> {
+        let message = CommandOutputMessage::new(
+            self.hostname.clone(),
+            command_id.to_string(),
+            command_type.to_string(),
+            "Command completed".to_string(),
+            true,
+        );
+        self.zmq_handler.publish_command_output(&message).await
+    }
+
+    /// Execute nixos-rebuild via systemd service with journal streaming
+    async fn execute_nixos_rebuild_with_streaming(&self, command_id: &str, _working_dir: &str) -> Result<()> {
+        use tokio::io::{AsyncBufReadExt, BufReader};
+        use tokio::process::Command;
+
+        self.send_command_output(command_id, "SystemRebuild", "Starting nixos-rebuild via systemd service...").await?;
+
+        // Start the cm-rebuild systemd service
+        let start_result = Command::new("sudo")
+            .arg("systemctl")
+            .arg("start")
+            .arg("cm-rebuild")
+            .output()
+            .await?;
+
+        if !start_result.status.success() {
+            let error = String::from_utf8_lossy(&start_result.stderr);
+            return Err(anyhow::anyhow!("Failed to start cm-rebuild service: {}", error));
+        }
+
+        self.send_command_output(command_id, "SystemRebuild", "✓ Service started, streaming output...").await?;
+
+        // Stream journal output in real-time
+        let mut journal_child = Command::new("sudo")
+            .arg("journalctl")
+            .arg("-u")
+            .arg("cm-rebuild")
+            .arg("-f")
+            .arg("--no-pager")
+            .arg("--since")
+            .arg("now")
+            .stdout(std::process::Stdio::piped())
+            .stderr(std::process::Stdio::piped())
+            .spawn()?;
+
+        let stdout = journal_child.stdout.take().expect("Failed to get journalctl stdout");
+        let mut reader = BufReader::new(stdout);
+        let mut lines = reader.lines();
+
+        // Stream journal output and monitor service status
+        let mut service_completed = false;
+        let mut status_check_interval = tokio::time::interval(tokio::time::Duration::from_secs(2));
+
+        loop {
+            tokio::select! {
+                // Read journal output
+                line = lines.next_line() => {
+                    match line {
+                        Ok(Some(line)) => {
+                            // Clean up journal format (remove timestamp/service prefix if needed)
+                            let clean_line = self.clean_journal_line(&line);
+                            self.send_command_output(command_id, "SystemRebuild", &clean_line).await?;
+                        }
+                        Ok(None) => {
+                            // journalctl stream ended
+                            break;
+                        }
+                        Err(_) => {
+                            // Error reading journal
+                            break;
+                        }
+                    }
+                }
+                // Periodically check service status
+                _ = status_check_interval.tick() => {
+                    if let Ok(status_result) = Command::new("sudo")
+                        .arg("systemctl")
+                        .arg("is-active")
+                        .arg("cm-rebuild")
+                        .output()
+                        .await
+                    {
+                        let status = String::from_utf8_lossy(&status_result.stdout).trim().to_string();
+                        if status == "inactive" {
+                            service_completed = true;
+                            break;
+                        }
+                    }
+                }
+            }
+        }
+
+        // Kill journalctl process
+        let _ = journal_child.kill().await;
+
+        // Check final service result
+        let result = Command::new("sudo")
+            .arg("systemctl")
+            .arg("is-failed")
+            .arg("cm-rebuild")
+            .output()
+            .await?;
+
+        let is_failed = String::from_utf8_lossy(&result.stdout).trim();
+        if is_failed == "failed" {
+            return Err(anyhow::anyhow!("cm-rebuild service failed"));
+        }
+
+        Ok(())
+    }
+
+    /// Clean journal line to remove systemd metadata
+    fn clean_journal_line(&self, line: &str) -> String {
+        // Remove timestamp and service name prefix from journal entries
+        // Example: "Oct 26 10:30:15 cmbox cm-rebuild[1234]: actual output"
+        // Becomes: "actual output"
+        
+        if let Some(colon_pos) = line.rfind(": ") {
+            line[colon_pos + 2..].to_string()
+        } else {
+            line.to_string()
+        }
+    }
+
+    /// Ensure git repository with output streaming
+    async fn ensure_git_repository_with_output(&self, command_id: &str, git_url: &str, git_branch: &str, working_dir: &str, api_key_file: Option<&str>) -> Result<()> {
+        // This is a simplified version - we can enhance this later with git output streaming
+        self.ensure_git_repository(git_url, git_branch, working_dir, api_key_file).await
+    }
+
+    /// Ensure git repository is cloned and up to date with force clone approach
    async fn ensure_git_repository(&self, git_url: &str, git_branch: &str, working_dir: &str, api_key_file: Option<&str>) -> Result<()> {
        use std::path::Path;
        
@@ -379,49 +537,37 @@ impl Agent {
            git_url.to_string()
        };
        
-        let git_dir = Path::new(working_dir).join(".git");
-        
-        if git_dir.exists() {
-            info!("Git repository exists, updating to latest {}", git_branch);
-            
-            // Pull latest changes
-            let output = tokio::process::Command::new("git")
-                .arg("pull")
-                .arg("origin")
-                .arg(git_branch)
-                .current_dir(working_dir)
-                .output()
-                .await?;
-                
-            if !output.status.success() {
-                let stderr = String::from_utf8_lossy(&output.stderr);
-                error!("Git pull failed: {}", stderr);
-                return Err(anyhow::anyhow!("Git pull failed: {}", stderr));
+        // Always remove existing directory and do fresh clone for consistent state
+        let working_path = Path::new(working_dir);
+        if working_path.exists() {
+            info!("Removing existing repository directory: {}", working_dir);
+            if let Err(e) = tokio::fs::remove_dir_all(working_path).await {
+                error!("Failed to remove existing directory: {}", e);
+                return Err(anyhow::anyhow!("Failed to remove existing directory: {}", e));
            }
-            
-            info!("Git repository updated successfully");
-        } else {
-            info!("Cloning git repository from {} (branch: {})", git_url, git_branch);
-            
-            // Clone repository with authentication if available
-            let output = tokio::process::Command::new("git")
-                .arg("clone")
-                .arg("--branch")
-                .arg(git_branch)
-                .arg(&auth_url)  // Use authenticated URL
-                .arg(working_dir)
-                .output()
-                .await?;
-                
-            if !output.status.success() {
-                let stderr = String::from_utf8_lossy(&output.stderr);
-                error!("Git clone failed: {}", stderr);
-                return Err(anyhow::anyhow!("Git clone failed: {}", stderr));
-            }
-            
-            info!("Git repository cloned successfully");
        }
        
+        info!("Force cloning git repository from {} (branch: {})", git_url, git_branch);
+        
+        // Force clone with depth 1 for efficiency (no history needed for deployment)
+        let output = tokio::process::Command::new("git")
+            .arg("clone")
+            .arg("--depth")
+            .arg("1")
+            .arg("--branch")
+            .arg(git_branch)
+            .arg(&auth_url)
+            .arg(working_dir)
+            .output()
+            .await?;
+            
+        if !output.status.success() {
+            let stderr = String::from_utf8_lossy(&output.stderr);
+            error!("Git clone failed: {}", stderr);
+            return Err(anyhow::anyhow!("Git clone failed: {}", stderr));
+        }
+        
+        info!("Git repository cloned successfully with latest state");
        Ok(())
    }
 }
--- a/agent/src/cache/cached_metric.rs
+++ b/agent/src/cache/cached_metric.rs
@@ -1,10 +0,0 @@
-use cm_dashboard_shared::Metric;
-use std::time::Instant;
-
-/// A cached metric with metadata
-#[derive(Debug, Clone)]
-pub struct CachedMetric {
-    pub metric: Metric,
-    pub collected_at: Instant,
-    pub access_count: u64,
-}
--- a/agent/src/cache/manager.rs
+++ b/agent/src/cache/manager.rs
@@ -1,33 +0,0 @@
-use super::ConfigurableCache;
-use cm_dashboard_shared::{CacheConfig, Metric};
-use std::sync::Arc;
-use tracing::info;
-
-/// Manages metric caching with background tasks
-pub struct MetricCacheManager {
-    cache: Arc<ConfigurableCache>,
-}
-
-impl MetricCacheManager {
-    pub fn new(config: CacheConfig) -> Self {
-        let cache = Arc::new(ConfigurableCache::new(config.clone()));
-
-        Self { cache }
-    }
-
-    /// Start background cache management tasks
-    pub async fn start_background_tasks(&self) {
-        // Temporarily disabled to isolate CPU usage issue
-        info!("Cache manager background tasks disabled for debugging");
-    }
-
-    /// Store metric in cache
-    pub async fn cache_metric(&self, metric: Metric) {
-        self.cache.store_metric(metric).await;
-    }
-
-    /// Get all cached metrics (including expired ones) for broadcasting
-    pub async fn get_all_cached_metrics(&self) -> Vec<Metric> {
-        self.cache.get_all_cached_metrics().await
-    }
-}
--- a/agent/src/cache/mod.rs
+++ b/agent/src/cache/mod.rs
@@ -1,129 +0,0 @@
-use cm_dashboard_shared::{CacheConfig, Metric};
-use std::collections::HashMap;
-use std::fs;
-use std::path::Path;
-use std::sync::Arc;
-use tokio::sync::RwLock;
-use tracing::{info, warn, error};
-
-/// Simple persistent cache for metrics
-pub struct SimpleCache {
-    metrics: RwLock<HashMap<String, Metric>>,
-    persist_path: String,
-}
-
-impl SimpleCache {
-    pub fn new(config: CacheConfig) -> Self {
-        let cache = Self {
-            metrics: RwLock::new(HashMap::new()),
-            persist_path: config.persist_path,
-        };
-        
-        // Clear cache file on startup to ensure fresh data
-        cache.clear_cache_file();
-        cache
-    }
-
-    /// Store metric in cache
-    pub async fn store_metric(&self, metric: Metric) {
-        let mut metrics = self.metrics.write().await;
-        metrics.insert(metric.name.clone(), metric);
-    }
-
-    /// Get all cached metrics
-    pub async fn get_all_cached_metrics(&self) -> Vec<Metric> {
-        let metrics = self.metrics.read().await;
-        metrics.values().cloned().collect()
-    }
-
-    /// Save cache to disk
-    pub async fn save_to_disk(&self) {
-        let metrics = self.metrics.read().await;
-        
-        // Create directory if needed
-        if let Some(parent) = Path::new(&self.persist_path).parent() {
-            if let Err(e) = fs::create_dir_all(parent) {
-                warn!("Failed to create cache directory {}: {}", parent.display(), e);
-                return;
-            }
-        }
-
-        // Serialize and save
-        match serde_json::to_string_pretty(&*metrics) {
-            Ok(json) => {
-                if let Err(e) = fs::write(&self.persist_path, json) {
-                    error!("Failed to save cache to {}: {}", self.persist_path, e);
-                }
-            }
-            Err(e) => {
-                error!("Failed to serialize cache: {}", e);
-            }
-        }
-    }
-
-    /// Load cache from disk
-    fn load_from_disk(&self) {
-        match fs::read_to_string(&self.persist_path) {
-            Ok(content) => {
-                match serde_json::from_str::<HashMap<String, Metric>>(&content) {
-                    Ok(loaded_metrics) => {
-                        if let Ok(mut metrics) = self.metrics.try_write() {
-                            *metrics = loaded_metrics;
-                            info!("Loaded {} metrics from cache", metrics.len());
-                        }
-                    }
-                    Err(e) => {
-                        warn!("Failed to parse cache file {}: {}", self.persist_path, e);
-                    }
-                }
-            }
-            Err(_) => {
-                info!("No cache file found at {}, starting fresh", self.persist_path);
-            }
-        }
-    }
-
-    /// Clear cache file on startup to ensure fresh data
-    fn clear_cache_file(&self) {
-        if Path::new(&self.persist_path).exists() {
-            match fs::remove_file(&self.persist_path) {
-                Ok(_) => info!("Cleared cache file {} on startup", self.persist_path),
-                Err(e) => warn!("Failed to clear cache file {}: {}", self.persist_path, e),
-            }
-        }
-    }
-}
-
-
-#[derive(Clone)]
-pub struct MetricCacheManager {
-    cache: Arc<SimpleCache>,
-}
-
-impl MetricCacheManager {
-    pub fn new(config: CacheConfig) -> Self {
-        Self {
-            cache: Arc::new(SimpleCache::new(config)),
-        }
-    }
-
-    pub async fn store_metric(&self, metric: Metric) {
-        self.cache.store_metric(metric).await;
-    }
-
-    pub async fn cache_metric(&self, metric: Metric) {
-        self.store_metric(metric).await;
-    }
-
-    pub async fn start_background_tasks(&self) {
-        // No background tasks needed for simple cache
-    }
-
-    pub async fn get_all_cached_metrics(&self) -> Result<Vec<Metric>, anyhow::Error> {
-        Ok(self.cache.get_all_cached_metrics().await)
-    }
-
-    pub async fn save_to_disk(&self) {
-        self.cache.save_to_disk().await;
-    }
-}
--- a/agent/src/collectors/backup.rs
+++ b/agent/src/collectors/backup.rs
@@ -107,9 +107,6 @@ impl BackupCollector {

 #[async_trait]
 impl Collector for BackupCollector {
-    fn name(&self) -> &str {
-        "backup"
-    }

    async fn collect(&self, _status_tracker: &mut StatusTracker) -> Result<Vec<Metric>, CollectorError> {
        let backup_status_option = self.read_backup_status().await?;
--- a/agent/src/collectors/cpu.rs
+++ b/agent/src/collectors/cpu.rs
@@ -15,7 +15,6 @@ use crate::config::CpuConfig;
 /// - No process spawning
 /// - <0.1ms collection time target
 pub struct CpuCollector {
-    name: String,
    load_thresholds: HysteresisThresholds,
    temperature_thresholds: HysteresisThresholds,
 }
@@ -34,7 +33,6 @@ impl CpuCollector {
        );
        
        Self {
-            name: "cpu".to_string(),
            load_thresholds,
            temperature_thresholds,
        }
@@ -197,9 +195,6 @@ impl CpuCollector {

 #[async_trait]
 impl Collector for CpuCollector {
-    fn name(&self) -> &str {
-        &self.name
-    }

    async fn collect(&self, status_tracker: &mut StatusTracker) -> Result<Vec<Metric>, CollectorError> {
        debug!("Collecting CPU metrics");
--- a/agent/src/collectors/disk.rs
+++ b/agent/src/collectors/disk.rs
@@ -41,11 +41,11 @@ pub struct DiskCollector {

 impl DiskCollector {
    pub fn new(config: DiskConfig) -> Self {
-        // Create hysteresis thresholds for disk temperature
+        // Create hysteresis thresholds for disk temperature from config
        let temperature_thresholds = HysteresisThresholds::with_custom_gaps(
-            60.0, // warning at 60°C
+            config.temperature_warning_celsius,
            5.0,  // 5°C gap for recovery
-            70.0, // critical at 70°C  
+            config.temperature_critical_celsius,
            5.0,  // 5°C gap for recovery
        );
        
@@ -219,18 +219,12 @@ impl DiskCollector {
    }

    /// Parse wear level from SMART output (SSD wear leveling)
+    /// Supports both NVMe and SATA SSD wear indicators
    fn parse_wear_level_from_smart(&self, smart_output: &str) -> Option<f32> {
        for line in smart_output.lines() {
-            // Look for wear leveling indicators
-            if line.contains("Wear_Leveling_Count") || line.contains("Media_Wearout_Indicator") {
-                let parts: Vec<&str> = line.split_whitespace().collect();
-                if parts.len() >= 10 {
-                    if let Ok(wear) = parts[9].parse::<f32>() {
-                        return Some(100.0 - wear); // Convert to percentage used
-                    }
-                }
-            }
-            // NVMe drives might show percentage used directly
+            let line = line.trim();
+            
+            // NVMe drives - direct percentage used
            if line.contains("Percentage Used:") {
                if let Some(wear_part) = line.split("Percentage Used:").nth(1) {
                    if let Some(wear_str) = wear_part.split('%').next() {
@@ -240,6 +234,38 @@ impl DiskCollector {
                    }
                }
            }
+            
+            // SATA SSD attributes - parse SMART table format
+            // Format: ID ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
+            let parts: Vec<&str> = line.split_whitespace().collect();
+            if parts.len() >= 10 {
+                // SSD Life Left / Percent Lifetime Remaining (higher = less wear)
+                if line.contains("SSD_Life_Left") || line.contains("Percent_Lifetime_Remain") {
+                    if let Ok(remaining) = parts[3].parse::<f32>() { // VALUE column
+                        return Some(100.0 - remaining); // Convert remaining to used
+                    }
+                }
+                
+                // Media Wearout Indicator (lower = more wear, normalize to 0-100)
+                if line.contains("Media_Wearout_Indicator") {
+                    if let Ok(remaining) = parts[3].parse::<f32>() { // VALUE column
+                        return Some(100.0 - remaining); // Convert remaining to used
+                    }
+                }
+                
+                // Wear Leveling Count (higher = less wear, but varies by manufacturer)
+                if line.contains("Wear_Leveling_Count") {
+                    if let Ok(wear_count) = parts[3].parse::<f32>() { // VALUE column
+                        // Most SSDs: 100 = new, decreases with wear
+                        if wear_count <= 100.0 {
+                            return Some(100.0 - wear_count);
+                        }
+                    }
+                }
+                
+                // Total LBAs Written - calculate against typical endurance if available
+                // This is more complex and manufacturer-specific, so we skip for now
+            }
        }
        None
    }
@@ -325,33 +351,6 @@ impl DiskCollector {
        Some(device_name.to_string())
    }

-    /// Get directory size using du command (efficient for single directory)
-    fn get_directory_size(&self, path: &str) -> Result<u64> {
-        let output = Command::new("du")
-            .arg("-s")
-            .arg("--block-size=1")
-            .arg(path)
-            .output()?;
-
-        // du returns success even with permission denied warnings in stderr
-        // We only care if the command completely failed or produced no stdout
-        let output_str = String::from_utf8(output.stdout)?;
-
-        if output_str.trim().is_empty() {
-            return Err(anyhow::anyhow!(
-                "du command produced no output for {}",
-                path
-            ));
-        }
-
-        let size_str = output_str
-            .split_whitespace()
-            .next()
-            .ok_or_else(|| anyhow::anyhow!("Failed to parse du output"))?;
-
-        let size_bytes = size_str.parse::<u64>()?;
-        Ok(size_bytes)
-    }

    /// Get filesystem info using df command
    fn get_filesystem_info(&self, path: &str) -> Result<(u64, u64)> {
@@ -382,23 +381,6 @@ impl DiskCollector {
        Ok((total_bytes, used_bytes))
    }

-    /// Calculate status based on usage percentage
-    fn calculate_usage_status(&self, used_bytes: u64, total_bytes: u64) -> Status {
-        if total_bytes == 0 {
-            return Status::Unknown;
-        }
-
-        let usage_percent = (used_bytes as f64 / total_bytes as f64) * 100.0;
-
-        // Thresholds for disk usage
-        if usage_percent >= 95.0 {
-            Status::Critical
-        } else if usage_percent >= 85.0 {
-            Status::Warning
-        } else {
-            Status::Ok
-        }
-    }

    /// Parse size string (e.g., "120G", "45M") to GB value
    fn parse_size_to_gb(&self, size_str: &str) -> f32 {
@@ -435,9 +417,6 @@ impl DiskCollector {

 #[async_trait]
 impl Collector for DiskCollector {
-    fn name(&self) -> &str {
-        "disk"
-    }

    async fn collect(&self, status_tracker: &mut StatusTracker) -> Result<Vec<Metric>, CollectorError> {
        let start_time = Instant::now();
--- a/agent/src/collectors/memory.rs
+++ b/agent/src/collectors/memory.rs
@@ -15,7 +15,6 @@ use crate::config::MemoryConfig;
 /// - No regex or complex parsing
 /// - <0.1ms collection time target
 pub struct MemoryCollector {
-    name: String,
    usage_thresholds: HysteresisThresholds,
 }

@@ -42,7 +41,6 @@ impl MemoryCollector {
        );
        
        Self {
-            name: "memory".to_string(),
            usage_thresholds,
        }
    }
@@ -284,9 +282,6 @@ impl MemoryCollector {

 #[async_trait]
 impl Collector for MemoryCollector {
-    fn name(&self) -> &str {
-        &self.name
-    }

    async fn collect(&self, status_tracker: &mut StatusTracker) -> Result<Vec<Metric>, CollectorError> {
        debug!("Collecting memory metrics");
--- a/agent/src/collectors/mod.rs
+++ b/agent/src/collectors/mod.rs
@@ -16,9 +16,6 @@ pub use error::CollectorError;
 /// Base trait for all collectors with extreme efficiency requirements
 #[async_trait]
 pub trait Collector: Send + Sync {
-    /// Name of this collector
-    fn name(&self) -> &str;
-
    /// Collect all metrics this collector provides
    async fn collect(&self, status_tracker: &mut StatusTracker) -> Result<Vec<Metric>, CollectorError>;

--- a/agent/src/collectors/nixos.rs
+++ b/agent/src/collectors/nixos.rs
@@ -12,12 +12,11 @@ use crate::config::NixOSConfig;
 /// - NixOS version and build information
 /// - Currently active/logged in users
 pub struct NixOSCollector {
-    config: NixOSConfig,
 }

 impl NixOSCollector {
-    pub fn new(config: NixOSConfig) -> Self {
-        Self { config }
+    pub fn new(_config: NixOSConfig) -> Self {
+        Self {}
    }

    /// Get NixOS build information
@@ -116,9 +115,6 @@ impl NixOSCollector {

 #[async_trait]
 impl Collector for NixOSCollector {
-    fn name(&self) -> &str {
-        "nixos"
-    }

    async fn collect(&self, _status_tracker: &mut StatusTracker) -> Result<Vec<Metric>, CollectorError> {
        debug!("Collecting NixOS system information");
--- a/agent/src/collectors/systemd.rs
+++ b/agent/src/collectors/systemd.rs
@@ -42,7 +42,6 @@ struct ServiceStatusInfo {
    load_state: String,
    active_state: String,
    sub_state: String,
-    description: String,
 }

 impl SystemdCollector {
@@ -170,18 +169,12 @@ impl SystemdCollector {
                let load_state = fields.get(1).unwrap_or(&"unknown").to_string();
                let active_state = fields.get(2).unwrap_or(&"unknown").to_string();
                let sub_state = fields.get(3).unwrap_or(&"unknown").to_string();
-                let description = if fields.len() > 4 {
-                    fields[4..].join(" ")
-                } else {
-                    "".to_string()
-                };

                // Cache the status information
                status_cache.insert(service_name.to_string(), ServiceStatusInfo {
                    load_state: load_state.clone(),
                    active_state: active_state.clone(),
                    sub_state: sub_state.clone(),
-                    description,
                });

                all_service_names.insert(service_name.to_string());
@@ -432,9 +425,6 @@ impl SystemdCollector {

 #[async_trait]
 impl Collector for SystemdCollector {
-    fn name(&self) -> &str {
-        "systemd"
-    }

    async fn collect(&self, _status_tracker: &mut StatusTracker) -> Result<Vec<Metric>, CollectorError> {
        let start_time = Instant::now();
--- a/agent/src/communication/mod.rs
+++ b/agent/src/communication/mod.rs
@@ -1,5 +1,5 @@
 use anyhow::Result;
-use cm_dashboard_shared::{MessageEnvelope, MetricMessage};
+use cm_dashboard_shared::{CommandOutputMessage, MessageEnvelope, MetricMessage};
 use tracing::{debug, info};
 use zmq::{Context, Socket, SocketType};

@@ -65,6 +65,24 @@ impl ZmqHandler {
        Ok(())
    }

+    /// Publish command output message via ZMQ
+    pub async fn publish_command_output(&self, message: &CommandOutputMessage) -> Result<()> {
+        debug!(
+            "Publishing command output for host {} (command: {}): {}",
+            message.hostname,
+            message.command_type,
+            message.output_line
+        );
+
+        let envelope = MessageEnvelope::command_output(message.clone())?;
+        let serialized = serde_json::to_vec(&envelope)?;
+        
+        self.publisher.send(&serialized, 0)?;
+        
+        debug!("Command output published successfully");
+        Ok(())
+    }
+
    /// Send heartbeat (placeholder for future use)

    /// Try to receive a command (non-blocking)
--- a/agent/src/config/mod.rs
+++ b/agent/src/config/mod.rs
@@ -36,7 +36,6 @@ pub struct CollectorConfig {
    pub memory: MemoryConfig,
    pub disk: DiskConfig,
    pub systemd: SystemdConfig,
-    pub smart: SmartConfig,
    pub backup: BackupConfig,
    pub network: NetworkConfig,
    pub nixos: NixOSConfig,
@@ -75,6 +74,11 @@ pub struct DiskConfig {
    pub usage_critical_percent: f32,
    /// Filesystem configurations
    pub filesystems: Vec<FilesystemConfig>,
+    /// SMART monitoring thresholds
+    pub temperature_warning_celsius: f32,
+    pub temperature_critical_celsius: f32,
+    pub wear_warning_percent: f32,
+    pub wear_critical_percent: f32,
 }

 /// Filesystem configuration entry
@@ -102,16 +106,6 @@ pub struct SystemdConfig {
    pub host_user_mapping: String,
 }

-/// SMART collector configuration
-#[derive(Debug, Clone, Serialize, Deserialize)]
-pub struct SmartConfig {
-    pub enabled: bool,
-    pub interval_seconds: u64,
-    pub temperature_warning_celsius: f32,
-    pub temperature_critical_celsius: f32,
-    pub wear_warning_percent: f32,
-    pub wear_critical_percent: f32,
-}

 /// NixOS collector configuration
 #[derive(Debug, Clone, Serialize, Deserialize)]
--- a/agent/src/main.rs
+++ b/agent/src/main.rs
@@ -4,7 +4,6 @@ use tracing::{error, info};
 use tracing_subscriber::EnvFilter;

 mod agent;
-mod cache;
 mod collectors;
 mod communication;
 mod config;
@@ -14,10 +13,26 @@ mod status;

 use agent::Agent;

+/// Get version showing cm-dashboard-agent package hash for easy deployment verification
+fn get_version() -> &'static str {
+    // Get the path of the current executable
+    let exe_path = std::env::current_exe().expect("Failed to get executable path");
+    let exe_str = exe_path.to_string_lossy();
+    
+    // Extract Nix store hash from path like /nix/store/HASH-cm-dashboard-v0.1.8/bin/cm-dashboard-agent
+    let hash_part = exe_str.strip_prefix("/nix/store/").expect("Not a nix store path");
+    let hash = hash_part.split('-').next().expect("Invalid nix store path format");
+    assert!(hash.len() >= 8, "Hash too short");
+    
+    // Return first 8 characters of nix store hash
+    let short_hash = hash[..8].to_string();
+    Box::leak(short_hash.into_boxed_str())
+}
+
 #[derive(Parser)]
 #[command(name = "cm-dashboard-agent")]
 #[command(about = "CM Dashboard metrics agent with individual metric collection")]
-#[command(version)]
+#[command(version = get_version())]
 struct Cli {
    /// Increase logging verbosity (-v, -vv)
    #[arg(short, long, action = clap::ArgAction::Count)]
--- a/agent/src/metrics/mod.rs
+++ b/agent/src/metrics/mod.rs
@@ -1,26 +1,21 @@
 use anyhow::Result;
 use cm_dashboard_shared::{Metric, StatusTracker};
-use std::collections::HashMap;
-use std::time::Instant;
-use tracing::{debug, error, info};
+use tracing::{error, info};

-use crate::cache::MetricCacheManager;
 use crate::collectors::{
    backup::BackupCollector, cpu::CpuCollector, disk::DiskCollector, memory::MemoryCollector,
    nixos::NixOSCollector, systemd::SystemdCollector, Collector,
 };
 use crate::config::{AgentConfig, CollectorConfig};

-/// Manages all metric collectors with intelligent caching
+/// Manages all metric collectors
 pub struct MetricCollectionManager {
    collectors: Vec<Box<dyn Collector>>,
-    cache_manager: MetricCacheManager,
-    last_collection_times: HashMap<String, Instant>,
    status_tracker: StatusTracker,
 }

 impl MetricCollectionManager {
-    pub async fn new(config: &CollectorConfig, agent_config: &AgentConfig) -> Result<Self> {
+    pub async fn new(config: &CollectorConfig, _agent_config: &AgentConfig) -> Result<Self> {
        let mut collectors: Vec<Box<dyn Collector>> = Vec::new();

        // Benchmark mode - only enable specific collector based on env var
@@ -106,156 +101,41 @@ impl MetricCollectionManager {
                    collectors.push(Box::new(nixos_collector));
                    info!("NixOS collector initialized");
                }
+
            }
        }

-        // Initialize cache manager with configuration
-        let cache_manager = MetricCacheManager::new(agent_config.cache.clone());
-
-        // Start background cache tasks
-        cache_manager.start_background_tasks().await;
-
        info!(
-            "Metric collection manager initialized with {} collectors and caching enabled",
+            "Metric collection manager initialized with {} collectors",
            collectors.len()
        );

        Ok(Self {
            collectors,
-            cache_manager,
-            last_collection_times: HashMap::new(),
            status_tracker: StatusTracker::new(),
        })
    }

    /// Force collection from ALL collectors immediately (used at startup)
    pub async fn collect_all_metrics_force(&mut self) -> Result<Vec<Metric>> {
-        let mut all_metrics = Vec::new();
-        let now = Instant::now();
-
-        info!(
-            "Force collecting from ALL {} collectors for startup",
-            self.collectors.len()
-        );
-
-        // Force collection from every collector regardless of intervals
-        for collector in &self.collectors {
-            let collector_name = collector.name();
-
-            match collector.collect(&mut self.status_tracker).await {
-                Ok(metrics) => {
-                    info!(
-                        "Force collected {} metrics from {} collector",
-                        metrics.len(),
-                        collector_name
-                    );
-
-                    // Cache all new metrics
-                    for metric in &metrics {
-                        self.cache_manager.cache_metric(metric.clone()).await;
-                    }
-
-                    all_metrics.extend(metrics);
-                    self.last_collection_times
-                        .insert(collector_name.to_string(), now);
-                }
-                Err(e) => {
-                    error!(
-                        "Collector '{}' failed during force collection: {}",
-                        collector_name, e
-                    );
-                    // Continue with other collectors even if one fails
-                }
-            }
-        }
-
-        info!(
-            "Force collection completed: {} total metrics cached",
-            all_metrics.len()
-        );
-        Ok(all_metrics)
+        self.collect_all_metrics().await
    }

-    /// Collect metrics from all collectors with intelligent caching
+    /// Collect metrics from all collectors
    pub async fn collect_all_metrics(&mut self) -> Result<Vec<Metric>> {
        let mut all_metrics = Vec::new();
-        let now = Instant::now();

-        // Collecting metrics from collectors (debug logging disabled for performance)
-
-        // Keep track of which collector types we're collecting fresh data from
-        let mut collecting_fresh = std::collections::HashSet::new();
-
-        // For each collector, check if we need to collect based on time intervals
        for collector in &self.collectors {
-            let collector_name = collector.name();
-
-            // Determine cache interval for this collector type based on data volatility
-            let cache_interval_secs = match collector_name {
-                "cpu" | "memory" => 5,    // Fast updates for volatile metrics
-                "systemd" => 30,          // Service status changes less frequently
-                "disk" => 300,            // SMART data changes very slowly (5 minutes)
-                "backup" => 600,          // Backup status changes rarely (10 minutes)
-                _ => 30,                  // Default: moderate frequency
-            };
-
-            let should_collect =
-                if let Some(last_time) = self.last_collection_times.get(collector_name) {
-                    now.duration_since(*last_time).as_secs() >= cache_interval_secs
-                } else {
-                    true // First collection
-                };
-
-            if should_collect {
-                collecting_fresh.insert(collector_name.to_string());
-                match collector.collect(&mut self.status_tracker).await {
-                    Ok(metrics) => {
-                        // Collector returned fresh metrics (debug logging disabled for performance)
-
-                        // Cache all new metrics
-                        for metric in &metrics {
-                            self.cache_manager.cache_metric(metric.clone()).await;
-                        }
-
-                        all_metrics.extend(metrics);
-                        self.last_collection_times
-                            .insert(collector_name.to_string(), now);
-                    }
-                    Err(e) => {
-                        error!("Collector '{}' failed: {}", collector_name, e);
-                        // Continue with other collectors even if one fails
-                    }
+            match collector.collect(&mut self.status_tracker).await {
+                Ok(metrics) => {
+                    all_metrics.extend(metrics);
+                }
+                Err(e) => {
+                    error!("Collector failed: {}", e);
                }
-            } else {
-                let _elapsed = self
-                    .last_collection_times
-                    .get(collector_name)
-                    .map(|t| now.duration_since(*t).as_secs())
-                    .unwrap_or(0);
-                // Collector skipped (debug logging disabled for performance)
            }
        }
-
-        // For 2-second intervals, skip cached metrics to avoid duplicates
-        // (Cache system disabled for realtime updates)
-
-        // Collected metrics total (debug logging disabled for performance)
        Ok(all_metrics)
    }

-
-    /// Get all cached metrics from the cache manager
-    pub async fn get_all_cached_metrics(&self) -> Result<Vec<Metric>> {
-        let cached_metrics = self.cache_manager.get_all_cached_metrics().await?;
-        debug!(
-            "Retrieved {} cached metrics for broadcast",
-            cached_metrics.len()
-        );
-        Ok(cached_metrics)
-    }
-
-    pub fn get_cache_manager(&self) -> &MetricCacheManager {
-        &self.cache_manager
-    }
-
 }
--- a/agent/src/status/mod.rs
+++ b/agent/src/status/mod.rs
@@ -70,7 +70,7 @@ impl HostStatusManager {

    /// Update the status of a specific service and recalculate host status
    /// Updates real-time status and buffers changes for email notifications
-    pub fn update_service_status(&mut self, service: String, status: Status, cache_manager: Option<&crate::cache::MetricCacheManager>) {
+    pub fn update_service_status(&mut self, service: String, status: Status) {
        if !self.config.enabled {
            return;
        }
@@ -82,14 +82,6 @@ impl HostStatusManager {
            return;
        }

-        // Save cache when status changes (clone cache manager reference for async)
-        if let Some(cache) = cache_manager {
-            let cache = cache.clone();
-            tokio::spawn(async move {
-                cache.save_to_disk().await;
-            });
-        }
-
        // Initialize batch if this is the first change
        if self.batch_start_time.is_none() {
            self.batch_start_time = Some(Instant::now());
@@ -169,9 +161,9 @@ impl HostStatusManager {


    /// Process a metric - updates status (notifications handled separately via batching)
-    pub async fn process_metric(&mut self, metric: &Metric, _notification_manager: &mut crate::notifications::NotificationManager, cache_manager: &crate::cache::MetricCacheManager) {
+    pub async fn process_metric(&mut self, metric: &Metric, _notification_manager: &mut crate::notifications::NotificationManager) {
        // Just update status - notifications are handled by process_pending_notifications
-        self.update_service_status(metric.name.clone(), metric.status, Some(cache_manager));
+        self.update_service_status(metric.name.clone(), metric.status);
    }

    /// Process pending notifications - call this at notification intervals
--- a/dashboard/Cargo.toml
+++ b/dashboard/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "cm-dashboard"
-version = "0.1.0"
+version = "0.1.13"
 edition = "2021"

 [dependencies]
--- a/dashboard/src/app.rs
+++ b/dashboard/src/app.rs
@@ -236,6 +236,13 @@ impl Dashboard {
                    self.metric_store
                        .update_metrics(&metric_message.hostname, metric_message.metrics);

+                    // Check for agent version mismatches across hosts
+                    if let Some((current_version, outdated_hosts)) = self.metric_store.get_version_mismatches() {
+                        for outdated_host in &outdated_hosts {
+                            warn!("Host {} has outdated agent version (current: {})", outdated_host, current_version);
+                        }
+                    }
+
                    // Update TUI with new hosts and metrics (only if not headless)
                    if let Some(ref mut tui_app) = self.tui_app {
                        let mut connected_hosts = self
@@ -261,6 +268,23 @@ impl Dashboard {
                        tui_app.update_metrics(&self.metric_store);
                    }
                }
+                
+                // Also check for command output messages
+                if let Ok(Some(cmd_output)) = self.zmq_consumer.receive_command_output().await {
+                    debug!(
+                        "Received command output from {}: {}",
+                        cmd_output.hostname,
+                        cmd_output.output_line
+                    );
+
+                    // Forward to TUI if not headless
+                    if let Some(ref mut tui_app) = self.tui_app {
+                        tui_app.add_terminal_output(&cmd_output.hostname, cmd_output.output_line);
+                        
+                        // Note: Popup stays open for manual review - close with ESC/Q
+                    }
+                }
+                
                last_metrics_check = Instant::now();
            }

@@ -332,21 +356,6 @@ impl Dashboard {
        Ok(())
    }

-    /// Get current service status from metrics to determine start/stop action
-    fn get_service_status(&self, hostname: &str, service_name: &str) -> Option<String> {
-        let metrics = self.metric_store.get_metrics_for_host(hostname);
-        
-        // Look for systemd service status metric
-        for metric in metrics {
-            if metric.name == format!("systemd_{}_status", service_name) {
-                if let cm_dashboard_shared::MetricValue::String(status) = &metric.value {
-                    return Some(status.clone());
-                }
-            }
-        }
-        
-        None
-    }
 }

 impl Drop for Dashboard {
--- a/dashboard/src/communication/mod.rs
+++ b/dashboard/src/communication/mod.rs
@@ -1,5 +1,5 @@
 use anyhow::Result;
-use cm_dashboard_shared::{MessageEnvelope, MessageType, MetricMessage};
+use cm_dashboard_shared::{CommandOutputMessage, MessageEnvelope, MessageType, MetricMessage};
 use tracing::{debug, error, info, warn};
 use zmq::{Context, Socket, SocketType};

@@ -103,6 +103,43 @@ impl ZmqConsumer {
        Ok(())
    }

+    /// Receive command output from any connected agent (non-blocking)  
+    pub async fn receive_command_output(&mut self) -> Result<Option<CommandOutputMessage>> {
+        match self.subscriber.recv_bytes(zmq::DONTWAIT) {
+            Ok(data) => {
+                // Deserialize envelope
+                let envelope: MessageEnvelope = serde_json::from_slice(&data)
+                    .map_err(|e| anyhow::anyhow!("Failed to deserialize envelope: {}", e))?;
+
+                // Check message type
+                match envelope.message_type {
+                    MessageType::CommandOutput => {
+                        let cmd_output = envelope
+                            .decode_command_output()
+                            .map_err(|e| anyhow::anyhow!("Failed to decode command output: {}", e))?;
+
+                        debug!(
+                            "Received command output from {}: {}",
+                            cmd_output.hostname,
+                            cmd_output.output_line
+                        );
+
+                        Ok(Some(cmd_output))
+                    }
+                    _ => Ok(None), // Not a command output message
+                }
+            }
+            Err(zmq::Error::EAGAIN) => {
+                // No message available (non-blocking mode)
+                Ok(None)
+            }
+            Err(e) => {
+                error!("ZMQ receive error: {}", e);
+                Err(anyhow::anyhow!("ZMQ receive error: {}", e))
+            }
+        }
+    }
+
    /// Receive metrics from any connected agent (non-blocking)
    pub async fn receive_metrics(&mut self) -> Result<Option<MetricMessage>> {
        match self.subscriber.recv_bytes(zmq::DONTWAIT) {
@@ -132,6 +169,10 @@ impl ZmqConsumer {
                        debug!("Received heartbeat");
                        Ok(None) // Don't return heartbeats as metrics
                    }
+                    MessageType::CommandOutput => {
+                        debug!("Received command output (will be handled by receive_command_output)");
+                        Ok(None) // Command output handled by separate method
+                    }
                    _ => {
                        debug!("Received non-metrics message: {:?}", envelope.message_type);
                        Ok(None)
--- a/dashboard/src/main.rs
+++ b/dashboard/src/main.rs
@@ -11,10 +11,26 @@ mod ui;

 use app::Dashboard;

+/// Get version showing cm-dashboard package hash for easy rebuild verification
+fn get_version() -> &'static str {
+    // Get the path of the current executable
+    let exe_path = std::env::current_exe().expect("Failed to get executable path");
+    let exe_str = exe_path.to_string_lossy();
+    
+    // Extract Nix store hash from path like /nix/store/HASH-cm-dashboard-0.1.0/bin/cm-dashboard
+    let hash_part = exe_str.strip_prefix("/nix/store/").expect("Not a nix store path");
+    let hash = hash_part.split('-').next().expect("Invalid nix store path format");
+    assert!(hash.len() >= 8, "Hash too short");
+    
+    // Return first 8 characters of nix store hash
+    let short_hash = hash[..8].to_string();
+    Box::leak(short_hash.into_boxed_str())
+}
+
 #[derive(Parser)]
 #[command(name = "cm-dashboard")]
 #[command(about = "CM Dashboard TUI with individual metric consumption")]
-#[command(version)]
+#[command(version = get_version())]
 struct Cli {
    /// Increase logging verbosity (-v, -vv)
    #[arg(short, long, action = clap::ArgAction::Count)]
--- a/dashboard/src/metrics/store.rs
+++ b/dashboard/src/metrics/store.rs
@@ -124,4 +124,52 @@ impl MetricStore {
            }
        }
    }
+
+    /// Get agent versions from all hosts for cross-host comparison
+    pub fn get_agent_versions(&self) -> HashMap<String, String> {
+        let mut versions = HashMap::new();
+        
+        for (hostname, metrics) in &self.current_metrics {
+            if let Some(version_metric) = metrics.get("agent_version") {
+                if let cm_dashboard_shared::MetricValue::String(version) = &version_metric.value {
+                    versions.insert(hostname.clone(), version.clone());
+                }
+            }
+        }
+        
+        versions
+    }
+
+    /// Check for agent version mismatches across hosts
+    pub fn get_version_mismatches(&self) -> Option<(String, Vec<String>)> {
+        let versions = self.get_agent_versions();
+        
+        if versions.len() < 2 {
+            return None; // Need at least 2 hosts to compare
+        }
+        
+        // Find the most common version (assume it's the "current" version)
+        let mut version_counts = HashMap::new();
+        for version in versions.values() {
+            *version_counts.entry(version.clone()).or_insert(0) += 1;
+        }
+        
+        let most_common_version = version_counts
+            .iter()
+            .max_by_key(|(_, count)| *count)
+            .map(|(version, _)| version.clone())?;
+        
+        // Find hosts with different versions
+        let outdated_hosts: Vec<String> = versions
+            .iter()
+            .filter(|(_, version)| *version != &most_common_version)
+            .map(|(hostname, _)| hostname.clone())
+            .collect();
+        
+        if outdated_hosts.is_empty() {
+            None
+        } else {
+            Some((most_common_version, outdated_hosts))
+        }
+    }
 }
--- a/dashboard/src/ui/mod.rs
+++ b/dashboard/src/ui/mod.rs
@@ -15,7 +15,7 @@ pub mod widgets;

 use crate::metrics::MetricStore;
 use cm_dashboard_shared::{Metric, Status};
-use theme::{Components, Layout as ThemeLayout, StatusIcons, Theme, Typography};
+use theme::{Components, Layout as ThemeLayout, Theme, Typography};
 use widgets::{BackupWidget, ServicesWidget, SystemWidget, Widget};

 /// Commands that can be triggered from the UI
@@ -34,9 +34,7 @@ pub enum CommandStatus {
    /// Command is executing
    InProgress { command_type: CommandType, target: String, start_time: std::time::Instant },
    /// Command completed successfully
-    Success { command_type: CommandType, target: String, duration: std::time::Duration, completed_at: std::time::Instant },
-    /// Command failed
-    Failed { command_type: CommandType, target: String, error: String, failed_at: std::time::Instant },
+    Success { command_type: CommandType, completed_at: std::time::Instant },
 }

 /// Types of commands for status tracking
@@ -58,28 +56,6 @@ pub enum PanelType {
 }

 impl PanelType {
-    /// Get all panel types in order
-    pub fn all() -> [PanelType; 3] {
-        [PanelType::System, PanelType::Services, PanelType::Backup]
-    }
-    
-    /// Get the next panel in cycle (System → Services → Backup → System)
-    pub fn next(self) -> PanelType {
-        match self {
-            PanelType::System => PanelType::Services,
-            PanelType::Services => PanelType::Backup,
-            PanelType::Backup => PanelType::System,
-        }
-    }
-    
-    /// Get the previous panel in cycle (System ← Services ← Backup ← System)
-    pub fn previous(self) -> PanelType {
-        match self {
-            PanelType::System => PanelType::Backup,
-            PanelType::Services => PanelType::System,
-            PanelType::Backup => PanelType::Services,
-        }
-    }
 }

 /// Widget states for a specific host
@@ -116,6 +92,51 @@ impl HostWidgets {
    }
 }

+/// Terminal popup for streaming command output
+#[derive(Clone)]
+pub struct TerminalPopup {
+    /// Is the popup currently visible
+    pub visible: bool,
+    /// Command being executed
+    pub command_type: CommandType,
+    /// Target hostname
+    pub hostname: String,
+    /// Target service/operation name
+    pub target: String,
+    /// Output lines collected so far
+    pub output_lines: Vec<String>,
+    /// Scroll offset for the output
+    pub scroll_offset: usize,
+    /// Start time of the operation
+    pub start_time: Instant,
+}
+
+impl TerminalPopup {
+    pub fn new(command_type: CommandType, hostname: String, target: String) -> Self {
+        Self {
+            visible: true,
+            command_type,
+            hostname,
+            target,
+            output_lines: Vec::new(),
+            scroll_offset: 0,
+            start_time: Instant::now(),
+        }
+    }
+
+    pub fn add_output_line(&mut self, line: String) {
+        self.output_lines.push(line);
+        // Auto-scroll to bottom when new content arrives
+        if self.output_lines.len() > 20 {
+            self.scroll_offset = self.output_lines.len().saturating_sub(20);
+        }
+    }
+
+    pub fn close(&mut self) {
+        self.visible = false;
+    }
+}
+
 /// Main TUI application
 pub struct TuiApp {
    /// Widget states per host (hostname -> HostWidgets)
@@ -132,6 +153,8 @@ pub struct TuiApp {
    should_quit: bool,
    /// Track if user manually navigated away from localhost
    user_navigated_away: bool,
+    /// Terminal popup for streaming command output
+    terminal_popup: Option<TerminalPopup>,
 }

 impl TuiApp {
@@ -144,6 +167,7 @@ impl TuiApp {
            focused_panel: PanelType::System, // Start with System panel focused
            should_quit: false,
            user_navigated_away: false,
+            terminal_popup: None,
        }
    }

@@ -202,7 +226,7 @@ impl TuiApp {
                // Add NixOS metrics - using exact matching for build display fix
                let nixos_metrics: Vec<&Metric> = all_metrics
                    .iter()
-                    .filter(|m| m.name == "system_nixos_build" || m.name == "system_active_users" || m.name == "system_agent_hash")
+                    .filter(|m| m.name == "system_nixos_build" || m.name == "system_active_users" || m.name == "agent_version")
                    .copied()
                    .collect();
                system_metrics.extend(nixos_metrics);
@@ -274,6 +298,38 @@ impl TuiApp {
    /// Handle keyboard input
    pub fn handle_input(&mut self, event: Event) -> Result<Option<UiCommand>> {
        if let Event::Key(key) = event {
+            // If terminal popup is visible, handle popup-specific keys first
+            if let Some(ref mut popup) = self.terminal_popup {
+                if popup.visible {
+                    match key.code {
+                        KeyCode::Esc => {
+                            popup.close();
+                            self.terminal_popup = None;
+                            return Ok(None);
+                        }
+                        KeyCode::Up => {
+                            popup.scroll_offset = popup.scroll_offset.saturating_sub(1);
+                            return Ok(None);
+                        }
+                        KeyCode::Down => {
+                            let max_scroll = if popup.output_lines.len() > 20 {
+                                popup.output_lines.len() - 20
+                            } else {
+                                0
+                            };
+                            popup.scroll_offset = (popup.scroll_offset + 1).min(max_scroll);
+                            return Ok(None);
+                        }
+                        KeyCode::Char('q') => {
+                            popup.close();
+                            self.terminal_popup = None;
+                            return Ok(None);
+                        }
+                        _ => return Ok(None), // Consume all other keys when popup is open
+                    }
+                }
+            }
+
            match key.code {
                KeyCode::Char('q') => {
                    self.should_quit = true;
@@ -290,6 +346,12 @@ impl TuiApp {
                            // System rebuild command
                            if let Some(hostname) = self.current_host.clone() {
                                self.start_command(&hostname, CommandType::SystemRebuild, hostname.clone());
+                                // Open terminal popup for real-time output
+                                self.terminal_popup = Some(TerminalPopup::new(
+                                    CommandType::SystemRebuild,
+                                    hostname.clone(),
+                                    "NixOS Rebuild".to_string()
+                                ));
                                return Ok(Some(UiCommand::SystemRebuild { hostname }));
                            }
                        }
@@ -423,34 +485,7 @@ impl TuiApp {
        info!("Switched to panel: {:?}", self.focused_panel);
    }

-    /// Switch to previous panel (Shift+Tab in reverse) - only cycles through visible panels
-    pub fn previous_panel(&mut self) {
-        let visible_panels = self.get_visible_panels();
-        if visible_panels.len() <= 1 {
-            return; // Can't switch if only one or no panels visible
-        }
-        
-        // Find current panel index in visible panels
-        if let Some(current_index) = visible_panels.iter().position(|&p| p == self.focused_panel) {
-            // Move to previous visible panel
-            let prev_index = if current_index == 0 {
-                visible_panels.len() - 1
-            } else {
-                current_index - 1
-            };
-            self.focused_panel = visible_panels[prev_index];
-        } else {
-            // Current panel not visible, switch to last visible panel
-            self.focused_panel = visible_panels[visible_panels.len() - 1];
-        }
-        
-        info!("Switched to panel: {:?}", self.focused_panel);
-    }

-    /// Get the currently focused panel
-    pub fn get_focused_panel(&self) -> PanelType {
-        self.focused_panel
-    }

    /// Get the currently selected service name from the services widget
    fn get_selected_service(&self) -> Option<String> {
@@ -462,15 +497,6 @@ impl TuiApp {
        None
    }

-    /// Get command status for current host
-    pub fn get_command_status(&self) -> Option<&CommandStatus> {
-        if let Some(hostname) = &self.current_host {
-            if let Some(host_widgets) = self.host_widgets.get(hostname) {
-                return host_widgets.command_status.as_ref();
-            }
-        }
-        None
-    }

    /// Should quit application
    pub fn should_quit(&self) -> bool {
@@ -491,31 +517,15 @@ impl TuiApp {
    /// Mark command as completed successfully
    pub fn complete_command(&mut self, hostname: &str) {
        if let Some(host_widgets) = self.host_widgets.get_mut(hostname) {
-            if let Some(CommandStatus::InProgress { command_type, target, start_time }) = &host_widgets.command_status {
-                let duration = start_time.elapsed();
+            if let Some(CommandStatus::InProgress { command_type, .. }) = &host_widgets.command_status {
                host_widgets.command_status = Some(CommandStatus::Success {
                    command_type: command_type.clone(),
-                    target: target.clone(),
-                    duration,
                    completed_at: Instant::now(),
                });
            }
        }
    }

-    /// Mark command as failed
-    pub fn fail_command(&mut self, hostname: &str, error: String) {
-        if let Some(host_widgets) = self.host_widgets.get_mut(hostname) {
-            if let Some(CommandStatus::InProgress { command_type, target, .. }) = &host_widgets.command_status {
-                host_widgets.command_status = Some(CommandStatus::Failed {
-                    command_type: command_type.clone(),
-                    target: target.clone(),
-                    error,
-                    failed_at: Instant::now(),
-                });
-            }
-        }
-    }

    /// Check for command timeouts and automatically clear them
    pub fn check_command_timeouts(&mut self) {
@@ -539,11 +549,6 @@ impl TuiApp {
                    hosts_to_clear.push(hostname.clone());
                }
            }
-            else if let Some(CommandStatus::Failed { failed_at, .. }) = &host_widgets.command_status {
-                if now.duration_since(*failed_at) > Duration::from_secs(5) {
-                    hosts_to_clear.push(hostname.clone());
-                }
-            }
        }
        
        // Clear timed out commands
@@ -554,6 +559,25 @@ impl TuiApp {
        }
    }

+    /// Add output line to terminal popup
+    pub fn add_terminal_output(&mut self, hostname: &str, line: String) {
+        if let Some(ref mut popup) = self.terminal_popup {
+            if popup.hostname == hostname && popup.visible {
+                popup.add_output_line(line);
+            }
+        }
+    }
+
+    /// Close terminal popup for a specific hostname
+    pub fn close_terminal_popup(&mut self, hostname: &str) {
+        if let Some(ref mut popup) = self.terminal_popup {
+            if popup.hostname == hostname {
+                popup.close();
+                self.terminal_popup = None;
+            }
+        }
+    }
+
    /// Check for rebuild completion by detecting agent hash changes
    pub fn check_rebuild_completion(&mut self, metric_store: &MetricStore) {
        let mut hosts_to_complete = Vec::new();
@@ -620,14 +644,6 @@ impl TuiApp {
        }
    }

-    /// Get total count of services for bounds checking
-    fn get_total_services_count(&self, hostname: &str) -> usize {
-        if let Some(host_widgets) = self.host_widgets.get(hostname) {
-            host_widgets.services_widget.get_total_services_count()
-        } else {
-            0
-        }
-    }

    /// Get list of currently visible panels
    fn get_visible_panels(&self) -> Vec<PanelType> {
@@ -725,6 +741,13 @@ impl TuiApp {

        // Render statusbar at the bottom
        self.render_statusbar(frame, main_chunks[2]); // main_chunks[2] is the statusbar area
+
+        // Render terminal popup on top of everything else
+        if let Some(ref popup) = self.terminal_popup {
+            if popup.visible {
+                self.render_terminal_popup(frame, size, popup);
+            }
+        }
    }

    /// Render btop-style minimal title with host status colors
@@ -759,10 +782,6 @@ impl TuiApp {
                        // Show green checkmark for successful rebuild
                        ("✓", Theme::success())
                    }
-                    Some(CommandStatus::Failed { command_type: CommandType::SystemRebuild, .. }) => {
-                        // Show red X for failed rebuild
-                        ("✗", Theme::error())
-                    }
                    _ => {
                        // Normal status icon based on metrics
                        let host_status = self.calculate_host_status(host, metric_store);
@@ -928,297 +947,112 @@ impl TuiApp {
        }
    }

-    fn render_storage_section(&self, frame: &mut Frame, area: Rect, metric_store: &MetricStore) {
-        if area.height < 2 {
-            return;
+    /// Render terminal popup with streaming output
+    fn render_terminal_popup(&self, frame: &mut Frame, area: Rect, popup: &TerminalPopup) {
+        use ratatui::{
+            style::{Color, Modifier, Style},
+            text::{Line, Span},
+            widgets::{Block, Borders, Clear, Paragraph, Wrap},
+        };
+
+        // Calculate popup size (80% of screen, centered)
+        let popup_width = area.width * 80 / 100;
+        let popup_height = area.height * 80 / 100;
+        let popup_x = (area.width - popup_width) / 2;
+        let popup_y = (area.height - popup_height) / 2;
+        
+        let popup_area = Rect {
+            x: popup_x,
+            y: popup_y,
+            width: popup_width,
+            height: popup_height,
+        };
+
+        // Clear background
+        frame.render_widget(Clear, popup_area);
+
+        // Create terminal-style block
+        let title = format!(" {} → {} ({:.1}s) ", 
+            popup.hostname, 
+            popup.target,
+            popup.start_time.elapsed().as_secs_f32()
+        );
+        
+        let block = Block::default()
+            .title(title)
+            .borders(Borders::ALL)
+            .border_style(Style::default().fg(Color::Cyan))
+            .style(Style::default().bg(Color::Black));
+
+        let inner_area = block.inner(popup_area);
+        frame.render_widget(block, popup_area);
+
+        // Render output content
+        let available_height = inner_area.height as usize;
+        let total_lines = popup.output_lines.len();
+        
+        // Calculate which lines to show based on scroll offset
+        let start_line = popup.scroll_offset;
+        let end_line = (start_line + available_height).min(total_lines);
+        
+        let visible_lines: Vec<Line> = popup.output_lines[start_line..end_line]
+            .iter()
+            .map(|line| {
+                // Style output lines with terminal colors
+                if line.contains("error") || line.contains("Error") || line.contains("failed") {
+                    Line::from(Span::styled(line.clone(), Style::default().fg(Color::Red)))
+                } else if line.contains("warning") || line.contains("Warning") {
+                    Line::from(Span::styled(line.clone(), Style::default().fg(Color::Yellow)))
+                } else if line.contains("building") || line.contains("Building") {
+                    Line::from(Span::styled(line.clone(), Style::default().fg(Color::Blue)))
+                } else if line.contains("✓") || line.contains("success") || line.contains("completed") {
+                    Line::from(Span::styled(line.clone(), Style::default().fg(Color::Green)))
+                } else {
+                    Line::from(Span::styled(line.clone(), Style::default().fg(Color::White)))
+                }
+            })
+            .collect();
+
+        let content = Paragraph::new(visible_lines)
+            .wrap(Wrap { trim: false })
+            .style(Style::default().bg(Color::Black));
+
+        frame.render_widget(content, inner_area);
+
+        // Render scroll indicator if needed
+        if total_lines > available_height {
+            let scroll_info = format!(" {}% ", 
+                if total_lines > 0 { 
+                    (end_line * 100) / total_lines 
+                } else { 
+                    100 
+                }
+            );
+            
+            let scroll_area = Rect {
+                x: popup_area.x + popup_area.width - scroll_info.len() as u16 - 1,
+                y: popup_area.y + popup_area.height - 1,
+                width: scroll_info.len() as u16,
+                height: 1,
+            };
+            
+            let scroll_widget = Paragraph::new(scroll_info)
+                .style(Style::default().fg(Color::Cyan).bg(Color::Black));
+            frame.render_widget(scroll_widget, scroll_area);
        }

-        if let Some(ref hostname) = self.current_host {
-            // Discover storage pools from metrics (look for disk_{pool}_usage_percent patterns)
-            let mut storage_pools: std::collections::HashMap<String, Vec<String>> = 
-                std::collections::HashMap::new();
-            
-            let all_metrics = metric_store.get_metrics_for_host(hostname);
-            
-            // Find storage pools by looking for usage metrics
-            for metric in &all_metrics {
-                if metric.name.starts_with("disk_") && metric.name.ends_with("_usage_percent") {
-                    let pool_name = metric.name
-                        .strip_prefix("disk_")
-                        .and_then(|s| s.strip_suffix("_usage_percent"))
-                        .unwrap_or_default()
-                        .to_string();
-                    
-                    if !pool_name.is_empty() && pool_name != "tmp" {
-                        storage_pools.entry(pool_name.clone()).or_insert_with(Vec::new);
-                    }
-                }
-            }
-            
-            // Find individual drives for each pool
-            for metric in &all_metrics {
-                if metric.name.starts_with("disk_") && metric.name.contains("_") && metric.name.ends_with("_health") {
-                    // Parse disk_{pool}_{drive}_health format
-                    let parts: Vec<&str> = metric.name.split('_').collect();
-                    if parts.len() >= 4 && parts[0] == "disk" && parts[parts.len()-1] == "health" {
-                        // Extract pool name (everything between "disk_" and "_{drive}_health")
-                        let drive_name = parts[parts.len()-2].to_string();
-                        let pool_part_end = parts.len() - 2;
-                        let pool_name = parts[1..pool_part_end].join("_");
-                        
-                        if let Some(drives) = storage_pools.get_mut(&pool_name) {
-                            if !drives.contains(&drive_name) {
-                                drives.push(drive_name);
-                            }
-                        }
-                    }
-                }
-            }
-
-            // Check if we found any storage pools
-            if storage_pools.is_empty() {
-                // No storage pools found - show error/waiting message
-                let content_chunks = ratatui::layout::Layout::default()
-                    .direction(Direction::Vertical)
-                    .constraints([Constraint::Length(1), Constraint::Min(0)])
-                    .split(area);
-
-                let storage_title = Paragraph::new("Storage:").style(Typography::widget_title());
-                frame.render_widget(storage_title, content_chunks[0]);
-
-                let no_storage_spans =
-                    StatusIcons::create_status_spans(Status::Unknown, "No storage pools detected");
-                let no_storage_para = Paragraph::new(ratatui::text::Line::from(no_storage_spans));
-                frame.render_widget(no_storage_para, content_chunks[1]);
-                return;
-            }
-
-            let available_lines = area.height as usize;
-            let mut constraints = Vec::new();
-            let mut pools_to_show = Vec::new();
-            let mut current_line = 0;
-
-            // Sort storage pools by name for consistent ordering
-            let mut sorted_pools: Vec<_> = storage_pools.iter().collect();
-            sorted_pools.sort_by_key(|(pool_name, _)| pool_name.as_str());
-            
-            // Add section title if we have pools
-            let mut title_added = false;
-            
-            for (pool_name, drives) in sorted_pools {
-                // Calculate lines needed: pool header + drives + usage line (+ section title if first)
-                let section_title_lines = if !title_added { 1 } else { 0 };
-                let lines_for_this_pool = section_title_lines + 1 + drives.len() + 1;
-                
-                if current_line + lines_for_this_pool <= available_lines {
-                    pools_to_show.push((pool_name.clone(), drives.clone()));
-
-                    // Add section title constraint if this is the first pool
-                    if !title_added {
-                        constraints.push(Constraint::Length(1)); // "Storage:" section title
-                        title_added = true;
-                    }
-
-                    // Add constraints for this pool
-                    constraints.push(Constraint::Length(1)); // Pool header with status
-                    for _ in 0..drives.len() {
-                        constraints.push(Constraint::Length(1)); // Drive line with tree symbol
-                    }
-                    constraints.push(Constraint::Length(1)); // Usage line with end tree symbol
-
-                    current_line += lines_for_this_pool;
-                } else {
-                    break; // Can't fit more pools
-                }
-            }
-
-            // Add remaining space if any
-            if constraints.len() < available_lines {
-                constraints.push(Constraint::Min(0));
-            }
-
-            let content_chunks = ratatui::layout::Layout::default()
-                .direction(Direction::Vertical)
-                .constraints(constraints)
-                .split(area);
-
-            let mut chunk_index = 0;
-
-            // Render "Storage:" section title if we have pools
-            if !pools_to_show.is_empty() {
-                let storage_title = Paragraph::new("Storage:").style(Typography::widget_title());
-                frame.render_widget(storage_title, content_chunks[chunk_index]);
-                chunk_index += 1;
-            }
-
-            // Display each storage pool with tree structure
-            for (pool_name, drives) in &pools_to_show {
-                // Pool header with status icon and type
-                let pool_display_name = if pool_name == "root" {
-                    "root".to_string()
-                } else {
-                    pool_name.clone()
-                };
-                
-                let pool_type = if drives.len() > 1 { "multi-drive" } else { "Single" };
-                
-                // Get pool status from usage metric
-                let pool_status = metric_store
-                    .get_metric(hostname, &format!("disk_{}_usage_percent", pool_name))
-                    .map(|m| m.status)
-                    .unwrap_or(Status::Unknown);
-                
-                // Create pool header with status icon
-                let pool_status_icon = StatusIcons::get_icon(pool_status);
-                let pool_status_color = Theme::status_color(pool_status);
-                let pool_header_text = format!("{} ({}):", pool_display_name, pool_type);
-                
-                let pool_header_spans = vec![
-                    ratatui::text::Span::styled(
-                        format!("{} ", pool_status_icon),
-                        Style::default().fg(pool_status_color),
-                    ),
-                    ratatui::text::Span::styled(
-                        pool_header_text,
-                        Style::default().fg(Theme::primary_text()),
-                    ),
-                ];
-                let pool_header_para = Paragraph::new(ratatui::text::Line::from(pool_header_spans));
-                frame.render_widget(pool_header_para, content_chunks[chunk_index]);
-                chunk_index += 1;
-
-                // Individual drive lines with tree symbols
-                let mut sorted_drives = drives.clone();
-                sorted_drives.sort();
-                for (_drive_idx, drive_name) in sorted_drives.iter().enumerate() {
-                    // Get drive health status
-                    let drive_health_metric = metric_store
-                        .get_metric(hostname, &format!("disk_{}_{}_health", pool_name, drive_name));
-                    let drive_status = drive_health_metric
-                        .map(|m| m.status)
-                        .unwrap_or(Status::Unknown);
-                    
-                    // Get drive temperature
-                    let temp_text = metric_store
-                        .get_metric(hostname, &format!("disk_{}_{}_temperature", pool_name, drive_name))
-                        .and_then(|m| m.value.as_f32())
-                        .map(|temp| format!(" T:{:.0}°C", temp))
-                        .unwrap_or_default();
-
-                    // Get drive wear level (SSDs)
-                    let wear_text = metric_store
-                        .get_metric(hostname, &format!("disk_{}_{}_wear_percent", pool_name, drive_name))
-                        .and_then(|m| m.value.as_f32())
-                        .map(|wear| format!(" W:{:.0}%", wear))
-                        .unwrap_or_default();
-
-                    // Build drive line with tree symbol
-                    let tree_symbol = "├─";
-                    let drive_status_icon = StatusIcons::get_icon(drive_status);
-                    let drive_status_color = Theme::status_color(drive_status);
-                    let drive_text = format!("{}{}{}", drive_name, temp_text, wear_text);
-                    
-                    let drive_spans = vec![
-                        ratatui::text::Span::styled("  ", Style::default()), // 2-space indentation
-                        ratatui::text::Span::styled(
-                            format!("{} ", tree_symbol),
-                            Style::default().fg(Theme::muted_text()),
-                        ),
-                        ratatui::text::Span::styled(
-                            format!("{} ", drive_status_icon),
-                            Style::default().fg(drive_status_color),
-                        ),
-                        ratatui::text::Span::styled(
-                            drive_text,
-                            Style::default().fg(Theme::primary_text()),
-                        ),
-                    ];
-                    let drive_para = Paragraph::new(ratatui::text::Line::from(drive_spans));
-                    frame.render_widget(drive_para, content_chunks[chunk_index]);
-                    chunk_index += 1;
-                }
-
-                // Usage line with end tree symbol and status icon
-                let usage_percent = metric_store
-                    .get_metric(hostname, &format!("disk_{}_usage_percent", pool_name))
-                    .and_then(|m| m.value.as_f32())
-                    .unwrap_or(0.0);
-
-                let used_gb = metric_store
-                    .get_metric(hostname, &format!("disk_{}_used_gb", pool_name))
-                    .and_then(|m| m.value.as_f32())
-                    .unwrap_or(0.0);
-
-                let total_gb = metric_store
-                    .get_metric(hostname, &format!("disk_{}_total_gb", pool_name))
-                    .and_then(|m| m.value.as_f32())
-                    .unwrap_or(0.0);
-
-                let usage_status = metric_store
-                    .get_metric(hostname, &format!("disk_{}_usage_percent", pool_name))
-                    .map(|m| m.status)
-                    .unwrap_or(Status::Unknown);
-
-                // Format usage with proper units
-                let (used_display, total_display, unit) = if total_gb < 1.0 {
-                    (used_gb * 1024.0, total_gb * 1024.0, "MB")
-                } else {
-                    (used_gb, total_gb, "GB")
-                };
-
-                let end_tree_symbol = "└─";
-                let usage_status_icon = StatusIcons::get_icon(usage_status);
-                let usage_status_color = Theme::status_color(usage_status);
-                let usage_text = format!("{:.1}% {:.1}{}/{:.1}{}", 
-                    usage_percent, used_display, unit, total_display, unit);
-                
-                let usage_spans = vec![
-                    ratatui::text::Span::styled("  ", Style::default()), // 2-space indentation
-                    ratatui::text::Span::styled(
-                        format!("{} ", end_tree_symbol),
-                        Style::default().fg(Theme::muted_text()),
-                    ),
-                    ratatui::text::Span::styled(
-                        format!("{} ", usage_status_icon),
-                        Style::default().fg(usage_status_color),
-                    ),
-                    ratatui::text::Span::styled(
-                        usage_text,
-                        Style::default().fg(Theme::primary_text()),
-                    ),
-                ];
-                let usage_para = Paragraph::new(ratatui::text::Line::from(usage_spans));
-                frame.render_widget(usage_para, content_chunks[chunk_index]);
-                chunk_index += 1;
-            }
-
-            // Show truncation indicator if we couldn't display all pools
-            if pools_to_show.len() < storage_pools.len() {
-                if let Some(last_chunk) = content_chunks.last() {
-                    let truncated_count = storage_pools.len() - pools_to_show.len();
-                    let truncated_text = format!(
-                        "... and {} more pool{}",
-                        truncated_count,
-                        if truncated_count == 1 { "" } else { "s" }
-                    );
-                    let truncated_para = Paragraph::new(truncated_text).style(Typography::muted());
-                    frame.render_widget(truncated_para, *last_chunk);
-                }
-            }
-        } else {
-            // No host connected
-            let content_chunks = ratatui::layout::Layout::default()
-                .direction(Direction::Vertical)
-                .constraints([Constraint::Length(1), Constraint::Min(0)])
-                .split(area);
-
-            let storage_title = Paragraph::new("Storage:").style(Typography::widget_title());
-            frame.render_widget(storage_title, content_chunks[0]);
-
-            let no_host_spans =
-                StatusIcons::create_status_spans(Status::Unknown, "No host connected");
-            let no_host_para = Paragraph::new(ratatui::text::Line::from(no_host_spans));
-            frame.render_widget(no_host_para, content_chunks[1]);
-        }
+        // Instructions at bottom
+        let instructions = " ESC/Q: Close • ↑↓: Scroll ";
+        let instructions_area = Rect {
+            x: popup_area.x + 1,
+            y: popup_area.y + popup_area.height - 1,
+            width: instructions.len() as u16,
+            height: 1,
+        };
+        
+        let instructions_widget = Paragraph::new(instructions)
+            .style(Style::default().fg(Color::Gray).bg(Color::Black));
+        frame.render_widget(instructions_widget, instructions_area);
    }
+
 }
--- a/dashboard/src/ui/theme.rs
+++ b/dashboard/src/ui/theme.rs
@@ -226,10 +226,6 @@ impl Layout {
    /// System vs backup split (equal)
    pub const SYSTEM_PANEL_HEIGHT: u16 = 50;
    pub const BACKUP_PANEL_HEIGHT: u16 = 50;
-    /// System panel CPU section height
-    pub const CPU_SECTION_HEIGHT: u16 = 2;
-    /// System panel memory section height  
-    pub const MEMORY_SECTION_HEIGHT: u16 = 3;
 }

 /// Typography system
--- a/dashboard/src/ui/widgets/backup.rs
+++ b/dashboard/src/ui/widgets/backup.rs
@@ -81,38 +81,7 @@ impl BackupWidget {



-    /// Format timestamp for display
-    fn format_last_run(&self) -> String {
-        match self.last_run_timestamp {
-            Some(timestamp) => {
-                let duration = chrono::Utc::now().timestamp() - timestamp;
-                if duration < 3600 {
-                    format!("{}m ago", duration / 60)
-                } else if duration < 86400 {
-                    format!("{}h ago", duration / 3600)
-                } else {
-                    format!("{}d ago", duration / 86400)
-                }
-            }
-            None => "—".to_string(),
-        }
-    }

-    /// Format disk usage in format "usedGB/totalGB"
-    fn format_repo_size(&self) -> String {
-        match (self.backup_disk_used_gb, self.backup_disk_total_gb) {
-            (Some(used_gb), Some(total_gb)) => {
-                let used_str = Self::format_size_with_proper_units(used_gb);
-                let total_str = Self::format_size_with_proper_units(total_gb);
-                format!("{}/{}", used_str, total_str)
-            }
-            (Some(used_gb), None) => {
-                // Fallback to just used size if total not available
-                Self::format_size_with_proper_units(used_gb)
-            }
-            _ => "—".to_string(),
-        }
-    }

    /// Format size with proper units (xxxkB/MB/GB/TB)
    fn format_size_with_proper_units(size_gb: f32) -> String {
@@ -137,23 +106,7 @@ impl BackupWidget {
        }
    }

-    /// Format product name display
-    fn format_product_name(&self) -> String {
-        if let Some(ref product_name) = self.backup_disk_product_name {
-            format!("P/N: {}", product_name)
-        } else {
-            "P/N: Unknown".to_string()
-        }
-    }

-    /// Format serial number display
-    fn format_serial_number(&self) -> String {
-        if let Some(ref serial) = self.backup_disk_serial_number {
-            format!("S/N: {}", serial)
-        } else {
-            "S/N: Unknown".to_string()
-        }
-    }

    /// Extract service name from metric name (e.g., "backup_service_gitea_status" -> "gitea")
    fn extract_service_name(metric_name: &str) -> Option<String> {
@@ -324,9 +277,6 @@ impl Widget for BackupWidget {
        }
    }

-    fn render(&mut self, frame: &mut Frame, area: Rect) {
-        self.render_with_scroll(frame, area, 0);
-    }
 }

 impl BackupWidget {
--- a/dashboard/src/ui/widgets/cpu.rs
+++ b/dashboard/src/ui/widgets/cpu.rs
@@ -1,139 +1 @@
-use cm_dashboard_shared::{Metric, Status};
-use ratatui::{
-    layout::{Constraint, Direction, Layout, Rect},
-    widgets::Paragraph,
-    Frame,
-};
-use tracing::debug;
-
-use super::Widget;
-use crate::ui::theme::{StatusIcons, Typography};
-
-/// CPU widget displaying load, temperature, and frequency
-#[derive(Clone)]
-pub struct CpuWidget {
-    /// CPU load averages (1, 5, 15 minutes)
-    load_1min: Option<f32>,
-    load_5min: Option<f32>,
-    load_15min: Option<f32>,
-    /// CPU temperature in Celsius
-    temperature: Option<f32>,
-    /// CPU frequency in MHz
-    frequency: Option<f32>,
-    /// Aggregated status
-    status: Status,
-    /// Last update indicator
-    has_data: bool,
-}
-
-impl CpuWidget {
-    pub fn new() -> Self {
-        Self {
-            load_1min: None,
-            load_5min: None,
-            load_15min: None,
-            temperature: None,
-            frequency: None,
-            status: Status::Unknown,
-            has_data: false,
-        }
-    }
-
-    /// Format load average for display
-    fn format_load(&self) -> String {
-        match (self.load_1min, self.load_5min, self.load_15min) {
-            (Some(l1), Some(l5), Some(l15)) => {
-                format!("{:.2} {:.2} {:.2}", l1, l5, l15)
-            }
-            _ => "— — —".to_string(),
-        }
-    }
-
-    /// Format frequency for display
-    fn format_frequency(&self) -> String {
-        match self.frequency {
-            Some(freq) => format!("{:.1} MHz", freq),
-            None => "— MHz".to_string(),
-        }
-    }
-}
-
-impl Widget for CpuWidget {
-    fn update_from_metrics(&mut self, metrics: &[&Metric]) {
-        debug!("CPU widget updating with {} metrics", metrics.len());
-
-        // Reset status aggregation
-        let mut statuses = Vec::new();
-
-        for metric in metrics {
-            match metric.name.as_str() {
-                "cpu_load_1min" => {
-                    if let Some(value) = metric.value.as_f32() {
-                        self.load_1min = Some(value);
-                        statuses.push(metric.status);
-                    }
-                }
-                "cpu_load_5min" => {
-                    if let Some(value) = metric.value.as_f32() {
-                        self.load_5min = Some(value);
-                        statuses.push(metric.status);
-                    }
-                }
-                "cpu_load_15min" => {
-                    if let Some(value) = metric.value.as_f32() {
-                        self.load_15min = Some(value);
-                        statuses.push(metric.status);
-                    }
-                }
-                "cpu_temperature_celsius" => {
-                    if let Some(value) = metric.value.as_f32() {
-                        self.temperature = Some(value);
-                        statuses.push(metric.status);
-                    }
-                }
-                "cpu_frequency_mhz" => {
-                    if let Some(value) = metric.value.as_f32() {
-                        self.frequency = Some(value);
-                        statuses.push(metric.status);
-                    }
-                }
-                _ => {}
-            }
-        }
-
-        // Aggregate status
-        self.status = if statuses.is_empty() {
-            Status::Unknown
-        } else {
-            Status::aggregate(&statuses)
-        };
-
-        self.has_data = !metrics.is_empty();
-
-        debug!(
-            "CPU widget updated: load={:?}, temp={:?}, freq={:?}, status={:?}",
-            self.load_1min, self.temperature, self.frequency, self.status
-        );
-    }
-
-    fn render(&mut self, frame: &mut Frame, area: Rect) {
-        let content_chunks = Layout::default()
-            .direction(Direction::Vertical)
-            .constraints([Constraint::Length(1), Constraint::Length(1)])
-            .split(area);
-        let cpu_title = Paragraph::new("CPU:").style(Typography::widget_title());
-        frame.render_widget(cpu_title, content_chunks[0]);
-        let load_freq_spans = StatusIcons::create_status_spans(
-            self.status,
-            &format!("Load: {} • {}", self.format_load(), self.format_frequency()),
-        );
-        let load_freq_para = Paragraph::new(ratatui::text::Line::from(load_freq_spans));
-        frame.render_widget(load_freq_para, content_chunks[1]);
-    }
-}
-
-impl Default for CpuWidget {
-    fn default() -> Self {
-        Self::new()
-    }
-}
+// This file is intentionally left minimal - CPU functionality is handled by the SystemWidget
--- a/dashboard/src/ui/widgets/memory.rs
+++ b/dashboard/src/ui/widgets/memory.rs
@@ -1,253 +1 @@
-use cm_dashboard_shared::{Metric, Status};
-use ratatui::{
-    layout::{Constraint, Direction, Layout, Rect},
-    widgets::Paragraph,
-    Frame,
-};
-use tracing::debug;
-
-use super::Widget;
-use crate::ui::theme::{StatusIcons, Typography};
-
-/// Memory widget displaying usage, totals, and swap information
-#[derive(Clone)]
-pub struct MemoryWidget {
-    /// Memory usage percentage
-    usage_percent: Option<f32>,
-    /// Total memory in GB
-    total_gb: Option<f32>,
-    /// Used memory in GB
-    used_gb: Option<f32>,
-    /// Available memory in GB
-    available_gb: Option<f32>,
-    /// Total swap in GB
-    swap_total_gb: Option<f32>,
-    /// Used swap in GB
-    swap_used_gb: Option<f32>,
-    /// /tmp directory size in MB
-    tmp_size_mb: Option<f32>,
-    /// /tmp total size in MB
-    tmp_total_mb: Option<f32>,
-    /// /tmp usage percentage
-    tmp_usage_percent: Option<f32>,
-    /// Aggregated status
-    status: Status,
-    /// Last update indicator
-    has_data: bool,
-}
-
-impl MemoryWidget {
-    pub fn new() -> Self {
-        Self {
-            usage_percent: None,
-            total_gb: None,
-            used_gb: None,
-            available_gb: None,
-            swap_total_gb: None,
-            swap_used_gb: None,
-            tmp_size_mb: None,
-            tmp_total_mb: None,
-            tmp_usage_percent: None,
-            status: Status::Unknown,
-            has_data: false,
-        }
-    }
-
-    /// Get memory usage percentage for gauge
-    fn get_memory_percentage(&self) -> u16 {
-        match self.usage_percent {
-            Some(percent) => percent.min(100.0).max(0.0) as u16,
-            None => {
-                // Calculate from used/total if percentage not available
-                match (self.used_gb, self.total_gb) {
-                    (Some(used), Some(total)) if total > 0.0 => {
-                        let percent = (used / total * 100.0).min(100.0).max(0.0);
-                        percent as u16
-                    }
-                    _ => 0,
-                }
-            }
-        }
-    }
-
-    /// Format size with proper units (kB/MB/GB)
-    fn format_size_units(size_mb: f32) -> String {
-        if size_mb >= 1024.0 {
-            // Convert to GB
-            let size_gb = size_mb / 1024.0;
-            format!("{:.1}GB", size_gb)
-        } else if size_mb >= 1.0 {
-            // Show as MB
-            format!("{:.0}MB", size_mb)
-        } else if size_mb >= 0.001 {
-            // Convert to kB
-            let size_kb = size_mb * 1024.0;
-            format!("{:.0}kB", size_kb)
-        } else {
-            // Show very small sizes in bytes
-            let size_bytes = size_mb * 1024.0 * 1024.0;
-            format!("{:.0}B", size_bytes)
-        }
-    }
-
-    /// Format /tmp usage as "xx% yyykB/MB/GB/zzzGB"
-    fn format_tmp_usage(&self) -> String {
-        match (self.tmp_usage_percent, self.tmp_size_mb, self.tmp_total_mb) {
-            (Some(percent), Some(used_mb), Some(total_mb)) => {
-                let used_str = Self::format_size_units(used_mb);
-                let total_str = Self::format_size_units(total_mb);
-                format!("{:.1}% {}/{}", percent, used_str, total_str)
-            }
-            (Some(percent), Some(used_mb), None) => {
-                let used_str = Self::format_size_units(used_mb);
-                format!("{:.1}% {}", percent, used_str)
-            }
-            (None, Some(used_mb), Some(total_mb)) => {
-                let used_str = Self::format_size_units(used_mb);
-                let total_str = Self::format_size_units(total_mb);
-                format!("{}/{}", used_str, total_str)
-            }
-            (None, Some(used_mb), None) => Self::format_size_units(used_mb),
-            _ => "—".to_string(),
-        }
-    }
-
-    /// Get tmp status based on usage percentage
-    fn get_tmp_status(&self) -> Status {
-        if let Some(tmp_percent) = self.tmp_usage_percent {
-            if tmp_percent >= 90.0 {
-                Status::Critical
-            } else if tmp_percent >= 70.0 {
-                Status::Warning
-            } else {
-                Status::Ok
-            }
-        } else {
-            Status::Unknown
-        }
-    }
-}
-
-impl Widget for MemoryWidget {
-    fn update_from_metrics(&mut self, metrics: &[&Metric]) {
-        debug!("Memory widget updating with {} metrics", metrics.len());
-
-        // Reset status aggregation
-        let mut statuses = Vec::new();
-
-        for metric in metrics {
-            match metric.name.as_str() {
-                "memory_usage_percent" => {
-                    if let Some(value) = metric.value.as_f32() {
-                        self.usage_percent = Some(value);
-                        statuses.push(metric.status);
-                    }
-                }
-                "memory_total_gb" => {
-                    if let Some(value) = metric.value.as_f32() {
-                        self.total_gb = Some(value);
-                        statuses.push(metric.status);
-                    }
-                }
-                "memory_used_gb" => {
-                    if let Some(value) = metric.value.as_f32() {
-                        self.used_gb = Some(value);
-                        statuses.push(metric.status);
-                    }
-                }
-                "memory_available_gb" => {
-                    if let Some(value) = metric.value.as_f32() {
-                        self.available_gb = Some(value);
-                        statuses.push(metric.status);
-                    }
-                }
-                "memory_swap_total_gb" => {
-                    if let Some(value) = metric.value.as_f32() {
-                        self.swap_total_gb = Some(value);
-                        statuses.push(metric.status);
-                    }
-                }
-                "memory_swap_used_gb" => {
-                    if let Some(value) = metric.value.as_f32() {
-                        self.swap_used_gb = Some(value);
-                        statuses.push(metric.status);
-                    }
-                }
-                "disk_tmp_size_mb" => {
-                    if let Some(value) = metric.value.as_f32() {
-                        self.tmp_size_mb = Some(value);
-                        statuses.push(metric.status);
-                    }
-                }
-                "disk_tmp_total_mb" => {
-                    if let Some(value) = metric.value.as_f32() {
-                        self.tmp_total_mb = Some(value);
-                        statuses.push(metric.status);
-                    }
-                }
-                "disk_tmp_usage_percent" => {
-                    if let Some(value) = metric.value.as_f32() {
-                        self.tmp_usage_percent = Some(value);
-                        statuses.push(metric.status);
-                    }
-                }
-                _ => {}
-            }
-        }
-
-        // Aggregate status
-        self.status = if statuses.is_empty() {
-            Status::Unknown
-        } else {
-            Status::aggregate(&statuses)
-        };
-
-        self.has_data = !metrics.is_empty();
-
-        debug!("Memory widget updated: usage={:?}%, total={:?}GB, swap_total={:?}GB, tmp={:?}/{:?}MB, status={:?}",
-               self.usage_percent, self.total_gb, self.swap_total_gb, self.tmp_size_mb, self.tmp_total_mb, self.status);
-    }
-
-    fn render(&mut self, frame: &mut Frame, area: Rect) {
-        let content_chunks = Layout::default()
-            .direction(Direction::Vertical)
-            .constraints([
-                Constraint::Length(1),
-                Constraint::Length(1),
-                Constraint::Length(1),
-            ])
-            .split(area);
-        let mem_title = Paragraph::new("RAM:").style(Typography::widget_title());
-        frame.render_widget(mem_title, content_chunks[0]);
-
-        // Format used and total memory with smart units, percentage, and status icon
-        let used_str = self
-            .used_gb
-            .map_or("—".to_string(), |v| Self::format_size_units(v * 1024.0)); // Convert GB to MB for formatting
-        let total_str = self
-            .total_gb
-            .map_or("—".to_string(), |v| Self::format_size_units(v * 1024.0)); // Convert GB to MB for formatting
-        let percentage = self.get_memory_percentage();
-        let mem_details_spans = StatusIcons::create_status_spans(
-            self.status,
-            &format!("Used: {}% {}/{}", percentage, used_str, total_str),
-        );
-        let mem_details_para = Paragraph::new(ratatui::text::Line::from(mem_details_spans));
-        frame.render_widget(mem_details_para, content_chunks[1]);
-
-        // /tmp usage line with status icon
-        let tmp_status = self.get_tmp_status();
-        let tmp_spans = StatusIcons::create_status_spans(
-            tmp_status,
-            &format!("tmp: {}", self.format_tmp_usage()),
-        );
-        let tmp_para = Paragraph::new(ratatui::text::Line::from(tmp_spans));
-        frame.render_widget(tmp_para, content_chunks[2]);
-    }
-}
-
-impl Default for MemoryWidget {
-    fn default() -> Self {
-        Self::new()
-    }
-}
+// This file is intentionally left minimal - Memory functionality is handled by the SystemWidget
--- a/dashboard/src/ui/widgets/mod.rs
+++ b/dashboard/src/ui/widgets/mod.rs
@@ -1,5 +1,4 @@
 use cm_dashboard_shared::Metric;
-use ratatui::{layout::Rect, Frame};

 pub mod backup;
 pub mod cpu;
@@ -16,6 +15,4 @@ pub trait Widget {
    /// Update widget with new metrics data
    fn update_from_metrics(&mut self, metrics: &[&Metric]);

-    /// Render the widget to a terminal frame
-    fn render(&mut self, frame: &mut Frame, area: Rect);
 }
--- a/dashboard/src/ui/widgets/services.rs
+++ b/dashboard/src/ui/widgets/services.rs
@@ -163,15 +163,6 @@ impl ServicesWidget {
        (icon.to_string(), info.status.clone(), status_color)
    }

-    /// Create spans for sub-service with icon next to name
-    fn create_sub_service_spans(
-        &self,
-        name: &str,
-        info: &ServiceInfo,
-        is_last: bool,
-    ) -> Vec<ratatui::text::Span<'static>> {
-        self.create_sub_service_spans_with_status(name, info, is_last, None)
-    }

    /// Create spans for sub-service with icon next to name, considering command status
    fn create_sub_service_spans_with_status(
@@ -432,16 +423,9 @@ impl Widget for ServicesWidget {
        );
    }

-    fn render(&mut self, frame: &mut Frame, area: Rect) {
-        self.render_with_focus(frame, area, false);
-    }
 }

 impl ServicesWidget {
-    /// Render with optional focus indicator and scroll support
-    pub fn render_with_focus(&mut self, frame: &mut Frame, area: Rect, is_focused: bool) {
-        self.render_with_focus_and_scroll(frame, area, is_focused, 0);
-    }

    /// Render with focus, scroll, and command status for visual feedback
    pub fn render_with_command_status(&mut self, frame: &mut Frame, area: Rect, is_focused: bool, scroll_offset: usize, command_status: Option<&CommandStatus>) {
@@ -635,167 +619,6 @@ impl ServicesWidget {
            }
        }
    }
-
-    /// Render with focus indicator and scroll offset
-    pub fn render_with_focus_and_scroll(&mut self, frame: &mut Frame, area: Rect, is_focused: bool, scroll_offset: usize) {
-        let services_block = if is_focused {
-            Components::focused_widget_block("services")
-        } else {
-            Components::widget_block("services")
-        };
-        let inner_area = services_block.inner(area);
-        frame.render_widget(services_block, area);
-
-        let content_chunks = Layout::default()
-            .direction(Direction::Vertical)
-            .constraints([Constraint::Length(1), Constraint::Min(0)])
-            .split(inner_area);
-
-        // Header
-        let header = format!(
-            "{:<25} {:<10} {:<8} {:<8}",
-            "Service:", "Status:", "RAM:", "Disk:"
-        );
-        let header_para = Paragraph::new(header).style(Typography::muted());
-        frame.render_widget(header_para, content_chunks[0]);
-
-        // Check if we have any services to display
-        if self.parent_services.is_empty() && self.sub_services.is_empty() {
-            let empty_text = Paragraph::new("No process data").style(Typography::muted());
-            frame.render_widget(empty_text, content_chunks[1]);
-            return;
-        }
-
-        // Build hierarchical service list for display
-        let mut display_lines: Vec<(String, Status, bool, Option<(ServiceInfo, bool)>)> = Vec::new();
-
-        // Sort parent services alphabetically for consistent order
-        let mut parent_services: Vec<_> = self.parent_services.iter().collect();
-        parent_services.sort_by(|(a, _), (b, _)| a.cmp(b));
-
-        for (parent_name, parent_info) in parent_services {
-            // Add parent service line
-            let parent_line = self.format_parent_service_line(parent_name, parent_info);
-            display_lines.push((parent_line, parent_info.widget_status, false, None)); // false = not sub-service
-
-            // Add sub-services for this parent (if any)
-            if let Some(sub_list) = self.sub_services.get(parent_name) {
-                // Sort sub-services by name for consistent display
-                let mut sorted_subs = sub_list.clone();
-                sorted_subs.sort_by(|(a, _), (b, _)| a.cmp(b));
-
-                for (i, (sub_name, sub_info)) in sorted_subs.iter().enumerate() {
-                    let is_last_sub = i == sorted_subs.len() - 1;
-                    // Store sub-service info for custom span rendering
-                    display_lines.push((
-                        sub_name.clone(),
-                        sub_info.widget_status,
-                        true,
-                        Some((sub_info.clone(), is_last_sub)),
-                    )); // true = sub-service, with is_last info
-                }
-            }
-        }
-
-        // Apply scroll offset and render visible lines
-        let available_lines = content_chunks[1].height as usize;
-        let total_lines = display_lines.len();
-        
-        // Calculate scroll boundaries
-        let max_scroll = if total_lines > available_lines {
-            total_lines - available_lines
-        } else {
-            total_lines.saturating_sub(1)
-        };
-        let effective_scroll = scroll_offset.min(max_scroll);
-        
-        // Get visible lines after scrolling
-        let visible_lines: Vec<_> = display_lines
-            .iter()
-            .skip(effective_scroll)
-            .take(available_lines)
-            .collect();
-        
-        let lines_to_show = visible_lines.len();
-
-        if lines_to_show > 0 {
-            let service_chunks = Layout::default()
-                .direction(Direction::Vertical)
-                .constraints(vec![Constraint::Length(1); lines_to_show])
-                .split(content_chunks[1]);
-
-            for (i, (line_text, line_status, is_sub, sub_info)) in visible_lines.iter().enumerate()
-            {
-                let actual_index = effective_scroll + i; // Real index in the full list
-                
-                // Only parent services can be selected - calculate parent service index
-                let is_selected = if !*is_sub {
-                    // This is a parent service - count how many parent services came before this one
-                    let parent_index = self.calculate_parent_service_index(&actual_index);
-                    parent_index == self.selected_index
-                } else {
-                    false // Sub-services are never selected
-                };
-                
-                let mut spans = if *is_sub && sub_info.is_some() {
-                    // Use custom sub-service span creation
-                    let (service_info, is_last) = sub_info.as_ref().unwrap();
-                    self.create_sub_service_spans(line_text, service_info, *is_last)
-                } else {
-                    // Use regular status spans for parent services
-                    StatusIcons::create_status_spans(*line_status, line_text)
-                };
-                
-                // Apply selection highlighting to parent services only, preserving status icon color
-                // Only show selection when Services panel is focused
-                if is_selected && !*is_sub && is_focused {
-                    for (i, span) in spans.iter_mut().enumerate() {
-                        if i == 0 {
-                            // First span is the status icon - preserve its color
-                            span.style = span.style.bg(Theme::highlight());
-                        } else {
-                            // Other spans (text) get full selection highlighting
-                            span.style = span.style
-                                .bg(Theme::highlight())
-                                .fg(Theme::background());
-                        }
-                    }
-                }
-                
-                let service_para = Paragraph::new(ratatui::text::Line::from(spans));
-                
-                frame.render_widget(service_para, service_chunks[i]);
-            }
-        }
-
-        // Show scroll indicator if there are more services than we can display
-        if total_lines > available_lines {
-            let hidden_above = effective_scroll;
-            let hidden_below = total_lines.saturating_sub(effective_scroll + available_lines);
-            
-            if hidden_above > 0 || hidden_below > 0 {
-                let scroll_text = if hidden_above > 0 && hidden_below > 0 {
-                    format!("... {} above, {} below", hidden_above, hidden_below)
-                } else if hidden_above > 0 {
-                    format!("... {} more above", hidden_above)
-                } else {
-                    format!("... {} more below", hidden_below)
-                };
-                
-                if available_lines > 0 && lines_to_show > 0 {
-                    let last_line_area = Rect {
-                        x: content_chunks[1].x,
-                        y: content_chunks[1].y + (lines_to_show - 1) as u16,
-                        width: content_chunks[1].width,
-                        height: 1,
-                    };
-                    
-                    let scroll_para = Paragraph::new(scroll_text).style(Typography::muted());
-                    frame.render_widget(scroll_para, last_line_area);
-                }
-            }
-        }
-    }
 }

 impl Default for ServicesWidget {
--- a/dashboard/src/ui/widgets/system.rs
+++ b/dashboard/src/ui/widgets/system.rs
@@ -249,7 +249,7 @@ impl SystemWidget {
    }

    /// Render storage section with tree structure
-    fn render_storage(&self) -> Vec<Line> {
+    fn render_storage(&self) -> Vec<Line<'_>> {
        let mut lines = Vec::new();

        for pool in &self.storage_pools {
@@ -339,9 +339,9 @@ impl Widget for SystemWidget {
                        self.active_users = Some(users.clone());
                    }
                }
-                "system_agent_hash" => {
-                    if let MetricValue::String(hash) = &metric.value {
-                        self.agent_hash = Some(hash.clone());
+                "agent_version" => {
+                    if let MetricValue::String(version) = &metric.value {
+                        self.agent_hash = Some(version.clone());
                    }
                }
                
@@ -410,9 +410,6 @@ impl Widget for SystemWidget {
        self.update_storage_from_metrics(metrics);
    }

-    fn render(&mut self, frame: &mut Frame, area: Rect) {
-        self.render_with_scroll(frame, area, 0);
-    }
 }

 impl SystemWidget {
@@ -429,20 +426,15 @@ impl SystemWidget {
        lines.push(Line::from(vec![
            Span::styled(format!("Build: {}", build_text), Typography::secondary())
        ]));
-
-        let config_text = self.config_hash.as_deref().unwrap_or("unknown");
+        
+        let agent_version_text = self.agent_hash.as_deref().unwrap_or("unknown");
        lines.push(Line::from(vec![
-            Span::styled(format!("Config: {}", config_text), Typography::secondary())
+            Span::styled(format!("Agent: {}", agent_version_text), Typography::secondary())
        ]));
-
-        let agent_hash_text = self.agent_hash.as_deref().unwrap_or("unknown");
-        let short_hash = if agent_hash_text.len() > 8 && agent_hash_text != "unknown" {
-            &agent_hash_text[..8]
-        } else {
-            agent_hash_text
-        };
+        
+        let users_text = self.active_users.as_deref().unwrap_or("unknown");
        lines.push(Line::from(vec![
-            Span::styled(format!("Agent: {}", short_hash), Typography::secondary())
+            Span::styled(format!("Active users: {}", users_text), Typography::secondary())
        ]));

        // CPU section
--- a/shared/Cargo.toml
+++ b/shared/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "cm-dashboard-shared"
-version = "0.1.0"
+version = "0.1.13"
 edition = "2021"

 [dependencies]
--- a/shared/src/protocol.rs
+++ b/shared/src/protocol.rs
@@ -9,6 +9,17 @@ pub struct MetricMessage {
    pub metrics: Vec<Metric>,
 }

+/// Command output streaming message
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct CommandOutputMessage {
+    pub hostname: String,
+    pub command_id: String,
+    pub command_type: String,
+    pub output_line: String,
+    pub is_complete: bool,
+    pub timestamp: u64,
+}
+
 impl MetricMessage {
    pub fn new(hostname: String, metrics: Vec<Metric>) -> Self {
        Self {
@@ -19,6 +30,19 @@ impl MetricMessage {
    }
 }

+impl CommandOutputMessage {
+    pub fn new(hostname: String, command_id: String, command_type: String, output_line: String, is_complete: bool) -> Self {
+        Self {
+            hostname,
+            command_id,
+            command_type,
+            output_line,
+            is_complete,
+            timestamp: chrono::Utc::now().timestamp() as u64,
+        }
+    }
+}
+
 /// Commands that can be sent from dashboard to agent
 #[derive(Debug, Serialize, Deserialize)]
 pub enum Command {
@@ -55,6 +79,7 @@ pub enum MessageType {
    Metrics,
    Command,
    CommandResponse,
+    CommandOutput,
    Heartbeat,
 }

@@ -80,6 +105,13 @@ impl MessageEnvelope {
        })
    }

+    pub fn command_output(message: CommandOutputMessage) -> Result<Self, crate::SharedError> {
+        Ok(Self {
+            message_type: MessageType::CommandOutput,
+            payload: serde_json::to_vec(&message)?,
+        })
+    }
+
    pub fn heartbeat() -> Result<Self, crate::SharedError> {
        Ok(Self {
            message_type: MessageType::Heartbeat,
@@ -113,4 +145,13 @@ impl MessageEnvelope {
            }),
        }
    }
+
+    pub fn decode_command_output(&self) -> Result<CommandOutputMessage, crate::SharedError> {
+        match self.message_type {
+            MessageType::CommandOutput => Ok(serde_json::from_slice(&self.payload)?),
+            _ => Err(crate::SharedError::Protocol {
+                message: "Expected command output message".to_string(),
+            }),
+        }
+    }
 }
Author	SHA1	Message	Date
Christoffer Martinsson	69892a2d84	Implement systemd service approach for nixos-rebuild operations Some checks failed Build and Release / build-and-release (push) Failing after 1m58s Details - Add cm-rebuild systemd service for process isolation - Add sudo permissions for service control and journal access - Remove verbose flag for cleaner output - Ensures reliable rebuild operations without agent crashes	2025-10-26 23:18:09 +01:00
Christoffer Martinsson	a928d73134	Update Cargo.toml versions to 0.1.11 All checks were successful Build and Release / build-and-release (push) Successful in 3m4s Details - Update agent, dashboard, and shared package versions from 0.1.0 to 0.1.11 - Ensures agent version reporting shows correct v0.1.11 instead of v0.1.0 - Synchronize package versions with git tag for consistent version tracking	2025-10-26 14:12:03 +01:00
Christoffer Martinsson	af52d49194	Fix system panel layout and switch to version-based agent reporting All checks were successful Build and Release / build-and-release (push) Successful in 2m6s Details - Remove auto-close behavior from terminal popup for manual review - Fix system panel to show correct NixOS section layout - Add missing Active users line after Agent version - Switch agent version from nix store hash to actual version number (v0.1.11) - Display full version string without truncation for clear version tracking	2025-10-26 13:34:56 +01:00
Christoffer Martinsson	bc94f75328	Enable real-time output streaming for nixos-rebuild command All checks were successful Build and Release / build-and-release (push) Successful in 1m24s Details - Replace simulated progress messages with actual stdout/stderr capture - Stream all nixos-rebuild output line-by-line to terminal popup - Show transparent build process including downloads, compilation, and activation - Maintain real-time visibility into complete rebuild process	2025-10-26 13:00:53 +01:00
Christoffer Martinsson	b6da71b7e7	Implement real-time terminal popup for system rebuild operations All checks were successful Build and Release / build-and-release (push) Successful in 1m21s Details - Add terminal popup UI component with 80% screen coverage and terminal styling - Extend ZMQ protocol with CommandOutputMessage for streaming output - Implement real-time output streaming in agent system rebuild handler - Add keyboard controls (ESC/Q to close, ↑↓ to scroll) for popup interaction - Fix system panel Build display to show actual NixOS build instead of config hash - Update service filters in README with wildcard patterns for better matching - Add periodic progress updates during nixos-rebuild execution - Integrate command output handling in dashboard main loop	2025-10-26 11:39:03 +01:00
Christoffer Martinsson	aaf7edfbce	Implement cross-host agent version comparison - MetricStore tracks agent versions from all hosts - Detects version mismatches using most common version as reference - Dashboard logs warnings for hosts with outdated agents - Foundation for visual version mismatch indicators in UI - Helps identify deployment inconsistencies across infrastructure	2025-10-26 10:42:26 +01:00
Christoffer Martinsson	bb72c42726	Add agent version reporting and display - Agent reports version via agent_version metric using nix store hash - Dashboard displays agent version in system widget - Foundation for cross-host version comparison - Both agent -V and dashboard show versions	2025-10-26 10:38:20 +01:00
Christoffer Martinsson	af5f96ce2f	Fix sed command in automated NixOS update workflow All checks were successful Build and Release / build-and-release (push) Successful in 1m23s Details - Use pipe delimiter instead of forward slash to avoid conflicts - Should fix 'number option to s command may not be zero' error - More robust regex pattern matching	2025-10-26 01:13:58 +02:00
Christoffer Martinsson	8dffe18a23	Improve SATA SSD wear level calculation Some checks failed Build and Release / build-and-release (push) Failing after 1m24s Details - Support multiple SATA SSD wear attributes (SSD_Life_Left, Media_Wearout_Indicator, etc.) - Handle manufacturer differences in wear reporting - Proper parsing of SMART table format with VALUE column - Covers Samsung, Intel, Crucial and other common SSD types - NVMe Percentage Used support maintained	2025-10-25 22:32:09 +02:00
Christoffer Martinsson	0c544753f9	Move SMART configuration into disk config - Consolidate SMART thresholds into DiskConfig structure - Remove separate SmartConfig - disk collector handles all drive data - Update NixOS configuration to use disk.temperature_* settings - Remove hardcoded temperature thresholds in disk collector - Logical grouping: disk collector owns all disk/drive configuration	2025-10-25 22:29:26 +02:00
Christoffer Martinsson	c8e26b9bac	Remove redundant smart collector - consolidate SMART into disk collector - Remove separate smart collector implementation - Disk collector already handles SMART data for drives - Eliminates duplicate smartctl calls causing performance issues - SMART functionality remains in logical place with disk monitoring - Fixes infinite smartctl loop issue	2025-10-25 22:25:22 +02:00
Christoffer Martinsson	60ef712fac	Fix hash conversion in NixOS update workflow All checks were successful Build and Release / build-and-release (push) Successful in 2m38s Details - Replace xxd with Python for hex to base64 conversion - Use standard tools available in GitHub Actions runners - Should fix hash conversion error in automated workflow	2025-10-25 17:24:37 +02:00
Christoffer Martinsson	1ed4666dfd	Add automated NixOS configuration updates to release workflow Some checks failed Build and Release / build-and-release (push) Failing after 1m22s Details - Clone nixosbox repository after creating release - Download and hash new tarball automatically - Update version and hash in cm-dashboard.nix - Commit and push changes with automated message - Eliminates manual NixOS config update step	2025-10-25 17:21:52 +02:00
Christoffer Martinsson	59d260680e	Integrate smart collector into metrics manager All checks were successful Build and Release / build-and-release (push) Successful in 1m54s Details - Add SmartCollector import and initialization - Enable in both normal and benchmark modes - Fixes infinite smartctl loop issue by properly managing collector - Smart collector now active when config.smart.enabled = true	2025-10-25 17:14:54 +02:00
Christoffer Martinsson	9160fac80b	Fix smart collector compilation errors - Update to match current Metric structure - Use correct Status enum and collector interface - Fix MetricValue types and constructor usage - Builds successfully with warnings only	2025-10-25 17:13:04 +02:00
Christoffer Martinsson	83cb43bcf1	Restore missing smart collector implementation Some checks failed Build and Release / build-and-release (push) Failing after 1m24s Details - Rewrite smart collector to match current architecture - Add back to mod.rs exports - Fixes infinite smartctl loop issue - Uses simple health and temperature monitoring	2025-10-25 16:59:09 +02:00
Christoffer Martinsson	b310206f1f	Document automated binary release system - Replace source build instructions with release workflow - Document tag-based release process with Gitea Actions - Include NixOS config update process for releases - Highlight benefits of static binary approach	2025-10-25 16:36:07 +02:00
Christoffer Martinsson	f9bf3ce610	Fix environment variable references for Gitea Actions All checks were successful Build and Release / build-and-release (push) Successful in 2m6s Details - Use GITHUB_OUTPUT instead of GITEA_OUTPUT - Use GITHUB_REF instead of GITEA_REF - Should fix TagName required error	2025-10-25 16:21:27 +02:00
Christoffer Martinsson	5f8c933844	Build static binaries to avoid library dependency issues Some checks failed Build and Release / build-and-release (push) Failing after 2m6s Details - Add RUSTFLAGS for static linking - Use explicit x86_64-unknown-linux-gnu target - Update binary paths to match target directory	2025-10-25 16:18:34 +02:00
Christoffer Martinsson	e61fd7fd76	Remove sudo from workflow commands Some checks failed Build and Release / build-and-release (push) Failing after 1m18s Details Gitea Actions runner doesn't have sudo available	2025-10-25 16:06:56 +02:00
Christoffer Martinsson	64ceed6236	Add Gitea Actions workflow for automated binary releases - Build cm-dashboard and cm-dashboard-agent binaries on tag push - Upload binaries as release assets via Gitea API - Use curl-based approach instead of external actions - Support manual workflow dispatch for testing	2025-10-25 16:04:31 +02:00
Christoffer Martinsson	09dcd53da5	Fix workflow to use GITEATOKEN secret name	2025-10-25 15:58:09 +02:00
Christoffer Martinsson	43196af70c	Add Gitea Actions workflow for automated binary releases Create workflow to build and release pre-built binaries: - Triggers on git tags (v*) or manual dispatch - Builds cm-dashboard and cm-dashboard-agent for Linux x86_64 - Creates Gitea release with attached binary files - Provides tarball for easy distribution This enables switching from source builds to pre-built binaries in NixOS configuration for faster rebuilds.	2025-10-25 15:51:23 +02:00
Christoffer Martinsson	1b3f8671c0	Add rebuild output logging for debugging Redirect nixos-rebuild stdout/stderr to /var/log/cm-dashboard/nixos-rebuild.log while keeping the process detached. This allows monitoring rebuild progress and debugging why cargo builds in /tmp aren't visible when agent runs. Use: tail -f /var/log/cm-dashboard/nixos-rebuild.log to monitor progress.	2025-10-25 15:23:20 +02:00
Christoffer Martinsson	16ea853f5b	Fix agent self-update issue by running nixos-rebuild detached Run nixos-rebuild with nohup in background to prevent the agent from killing itself during system rebuild. The rebuild process now runs independently, allowing the agent to return success immediately and avoid crashes during binary updates. This fixes the issue where agent would crash during rebuild and restart with the old binary due to missing daemon-reload.	2025-10-25 15:09:17 +02:00
Christoffer Martinsson	d463272cf2	Remove Config field and fix Build/Agent hash display - Remove Config field completely from NixOS section - Build: now shows NixOS system hash (from /run/current-system) - Agent: shows cm-dashboard package hash (first 8 chars) Build and Agent now display different hashes as intended.	2025-10-25 14:57:40 +02:00
Christoffer Martinsson	17b5921d8d	Fix dashboard -V to show cm-dashboard package hash not system hash Make dashboard -V show the same hash as the agent by extracting the hash from the dashboard binary's nix store path instead of the system configuration path. Now both will show identical hashes since they're from the same cm-dashboard package.	2025-10-25 14:45:26 +02:00
Christoffer Martinsson	3d187c9220	Make dashboard -V show actual config hash for rebuild verification Replace hardcoded version with first 8 characters of current system's nix store hash. This makes it easy to verify when rebuilds complete as the hash changes with each deployment. No fallback - fails hard if config hash cannot be determined.	2025-10-25 14:31:20 +02:00
Christoffer Martinsson	4b54a59e35	Remove unused code and eliminate compiler warnings - Remove unused fields from CommandStatus variants - Clean up unused methods and unused collector fields - Fix lifetime syntax warning in SystemWidget - Delete unused cache module completely - Remove redundant render methods from widgets All agent and dashboard warnings eliminated while preserving panel switching and scrolling functionality.	2025-10-25 14:15:52 +02:00
Christoffer Martinsson	8dd943e8f1	Fix config hash to use nix store hash and disable cache persistence	2025-10-25 12:57:47 +02:00