Why a Desktop Companion?
Many AI chatbots live in browsers or messaging apps. AIRI is different: it lives with you on the desktop, visible and reactive like a friendly pet. This creates a constant sense of presence without interrupting your workflow.
What You Experience
- Always-On Presence: A transparent, frameless window shows your 2D/3D avatar sitting anywhere on the screen.
- Click-Through Magic: Move the cursor away and the window lets clicks pass to apps beneath. Hover back and AIRI wakes up, ready to interact.
- Gaze Tracking: The character’s eyes follow your mouse, making the AI feel attentive.
- Quick Actions on Hover: Subtle buttons appear near the avatar edge—open chat, toggle settings, or drag to move the pet.
- Voice & Text Chat: Talk naturally—voice detection captures your speech only when you speak. Alternatively, pop open the chat window for typing.
Result: An AI friend that is present yet never blocks your work, turning everyday computer use into a lightly gamified experience.
How It Works
- Frameless Window: The desktop app launches a border-less window that can be transparent. The avatar is rendered in WebGL (Live2D or Three.js VRM).
- Pixel Sampling for Transparency: Every ~33 ms the app checks the RGBA value of the pixel under the cursor. If the avatar isn’t under the pointer (alpha < threshold), a tiny Rust plugin tells the OS to ignore mouse events in that area.
- Global Mouse Feed: A companion plugin streams global mouse coordinates to the frontend so gaze tracking and click-through logic stay perfectly in sync.
- Reactive UI Reveal: Vue reactivity toggles the visibility of hover buttons and status overlays. Utility-first CSS keeps visual noise low.
- Voice Loop Integration: When you speak, local VAD/STT plugs into the same UI, displaying waveform or loading bars on a small “Resource Status Island.”
Diagram:
Power Features for Power Users
| Feature | How to Enable | Why You Might Want It |
|---|---|---|
| Move Mode | Press Ctrl + Cmd + M (macOS) or Ctrl + Alt + M (Windows) | Drag the avatar to a new spot |
| Resize Mode | Ctrl + Cmd + R | Make the pet larger or smaller |
| Dynamic Hue Theme | Toggle in Settings → Appearance | Avatar background subtly changes colour over time |
| Click-Through Threshold | Adjust slider in Settings → Interaction | Fine-tune transparency sensitivity for different model edges |
Accessibility & Performance Notes
- The app uses hardware-accelerated rendering; CPU impact is minimal (less than 3% on typical laptops).
- All voice processing can run locally; no conversation audio leaves your device unless you choose a cloud STT provider.
- High-contrast mode and reduced-motion settings are respected via OS accessibility prefs.
Related Technical Files
| Functional Role | Code File | Description |
|---|---|---|
| Main Companion Page | apps/stage-tamagotchi/src/pages/index.vue | Hosts avatar canvas, gaze tracking, and hover UI |
| Click-Through Window Plugin (Rust) | crates/tauri-plugin-window-pass-through-on-hover/src/lib.rs | Enables / disables OS-level mouse event ignoring |
| Global Mouse Plugin (Rust) | crates/tauri-plugin-rdev/src/lib.rs | Streams system-wide mouse events to Vue |
| Gaze Tracking Store | packages/stage-ui/src/stores/live2d.ts, packages/stage-ui/src/stores/vrm.ts | Updates model eye look-at based on mouse |
| Hover Button Component | apps/stage-tamagotchi/src/components/HoverButtons.vue | Shows chat/settings buttons only on hover |