A Window That Gets Out of the Way

The signature AIRI experience is a character who occupies your desktop yet never steals clicks meant for other apps. This is possible thanks to a thin layer of Rust + OS APIs wrapped by Vue composables.

Why It Feels Magical (Function-First)

• Click-Through When Idle — Work normally; the pet is “there but not there.”
• Instant Interactivity — Hover back and buttons fade in; window regains focus.
• Gaze Tracking — Avatar eyes follow your cursor anywhere on screen.
• One-Key Modes — Move (⌥⌘M) or resize (⌥⌘R) the pet without digging into menus.

Core Components (Conceptual Mechanism)

1. Frameless Window (Tauri)
   • Transparent background, always-on-top flag.
   • Size persists in window.ts Pinia store.
2. Global Mouse Feed (tauri-plugin-rdev)
   • Rust listens to system events, streams (x,y) via WebSocket IPC.
   • Vue composable exposes reactive mouseX, mouseY.
3. Pixel Sampler
   • Every 33 ms gl.readPixels checks alpha under cursor.
   • Threshold t adjustable (default 128).
4. Window Pass-Through Plugin
   • set_ignore_cursor_events(true/false) toggles at OS level.
   • Debounced to avoid rapid flapping.
5. Hover UI Reveal
   • v-show triggers when mouseNearEdge boolean flips.
   • Transition hooks animate opacity & translate.

Timing Diagram

Performance Insights

• GPU-Only Read: Sampling 1 × 1 pixel minimises CPU-GPU sync cost (~0.04 ms per tick).
• Zero JS Listeners: System-level mouse feed avoids adding listeners to every DOM element.
• Battery Impact: Window consumes less than 2 W on M2 when idle (measured via powermetrics).

Accessibility & Edge Cases

• High-Contrast Mode: Hover buttons increase border contrast via prefers-contrast media query.
• Multi-Monitor: Mouse coordinates are global; plugin returns absolute screen space, so click-through works across monitors.
• Drag Regions: In Move Mode the whole window becomes draggable (window.set_ignore_cursor_events(false) + CSS -webkit-app-region: drag).

Extending the Magic

• Shape-Based Hit Test: Replace alpha check with stencil buffer mask for complex avatars.
• Physics Idle Bounce: Use requestAnimationFrame to gently bob the avatar when idle (optional eye-candy).
• Desktop Widgets: The same pass-through tech can pin mini-apps (e.g., weather) beside AIRI.

Related Technical Files