Local · On-device · v1

SENTINEL

The assistant that watches you work — and obeys your hands.

It sees your posture and focus through the webcam, coaches you in real time, and lets you command your screen with bare-handed gestures. Every frame stays on your machine. No cloud. No account. Free.

See how it runs → The gestures
Presence
LOCKED
Posture
UPRIGHT
Focus block
00:38:12
Last gesture
OPEN_PALM
// readout above is a simulated demo of the on-device HUD
01 / Perception

It sees you the way a good coach would.

OpenCV + MediaPipe on your own webcam. No footage is recorded or sent anywhere — the frame is read and discarded.
👁

Presence

Knows when you sit down, step away, and return. Greets the work, not the empty chair.

🧍

Posture

Calibrates your upright baseline, then catches the slow slouch before your back does.

⏱

Stillness

Tracks how long you've been frozen in the chair and tells you to stand before the hour is gone.

🪟

Screen thrash

Counts how fast you're flipping between windows and calls it when you're flailing, not working.

02 / Command

Eleven controllers — run it all with your bare hands.

Hold a pose ~0.6s and it fires once; swipe your hand to navigate. A passing hand can't spam your OS, and window actions hit the app you're working in — never Sentinel.
Hold — poses
✋
CLEAR / BACK
open palm · toggle all down ↔ up
✊
MAXIMIZE
fist · fill the screen
✌️
SHRINK
victory · small + centered
👍
VOLUME +
thumbs up
👎
VOLUME −
thumbs down
🤟
MUTE
I-love-you
☝️
PLAY / PAUSE
point up · media
Swipe — navigation
👉
NEXT DESKTOP
swipe right
👈
PREV DESKTOP
swipe left
👆
TASK VIEW
swipe up
👇
SHOW DESKTOP
swipe down
Two-way voice — say "Jarvis"
Just say “Jarvis…” — or hold F9 — then talk: “volume up”, “next desktop”, “be ruthless”, “what's running”. It hears you with on-device Whisper and answers out loud, in Tony-Stark's-Jarvis register. F9 always works as a backup.
03 / The mentor

One line. The right amount of pressure.

Event-triggered with a hard 90-second floor, so it disciplines without nagging. Pick the voltage.
Sit up. Straighten that spine.
04 / On your machine

The whole thing runs on the laptop in front of you.

No servers. No subscription. The most private an AI assistant can be — because nothing leaves.
0
bytes of video leave your device. The camera frame is read in memory and dropped.
$0
to run. Free, open tooling — Python, OpenCV, MediaPipe, Whisper. Pennies a day only if you turn on the Claude mentor brain.
100%
local. Works on a plane. Your posture, your focus, your face — yours alone.
05 / Research

Why these features — grounded in the science.

Not gimmicks. Each nudge maps to known research on screens, bodies and attention. Credited, never invented.
🪑

Sedentary risk

The WHO flags prolonged sitting as a real health risk; the remedy is frequent movement. → our sitting timer + stand-up nudges.

👁

20-20-20

American Academy of Ophthalmology: every 20 min, look ~20 ft away for 20 s to ease screen eye-strain. → our eye-rest reminder.

💧

Computer Vision Syndrome

American Optometric Association: screens sharply cut your blink rate and dry the eyes. → we surface your blinks/min.

🦴

“Tech neck”

Cervical-load research: a forward-tilted head multiplies the strain on your neck. → posture calibration + slouch catch.

🔀

Attention residue

Sophie Leroy's work: switching tasks leaves part of your mind stuck on the last one. → window-thrash detection.

🫀

Camera vitals (rPPG)

Poh, McDuff & Picard, MIT Media Lab: pulse and breathing from ordinary webcam video. → roadmap; research-grade, not in v1.

06 / Where this goes

A real assistant — for the betterment of the human using it.

Sentinel is the seed. The arc: it learns you — posture, voice, rhythm, goals — and helps you reach your ideal self.
Body
Head, neck, eyes and posture across the day; blink and break reminders; trained on you over time. A coach that knows your baseline.
Care
Assistive + remote monitoring: medication-time prompts, and control by voice or even a blink for people who can't use their hands. Wellness, not diagnosis — not a medical device.
Sport
Form, fatigue and posture cues for athletes and anyone training — the same eyes, pointed at performance.
Out of scope, on purpose: Sentinel never analyses your body uncovered — posture and presence read fine fully clothed.
07 / The stack

Built on open source — and what plugs in next.

A living map: what's live now, what's ready to add, and the wearables/IoT we can wire in. Honestly tagged — nothing faked.
👁

Vision

Live: MediaPipe Pose / Face / Gesture + OpenCV — body skeleton, blink, hands, on-device. Next: YOLO-Pose, py-feat expressions, rPPG vitals (research-grade).

🎙

Voice

Live: faster-whisper (hears) + pyttsx3 (speaks). Next: openWakeWord (a real “hey jarvis” model), Piper neural TTS, Silero VAD.

🧠

Brain & control

Live: Claude (Jarvis) + pywin32 (windows, keys, cursor). Next: Ollama local LLM ($0, private), Playwright for precise in-app selection.

⌚

Wearables & IoT

Documented, not yet wired: Fitbit / Withings / Garmin APIs; ESP32 sensors (MAX30102 heart-rate, MPU-6050 posture); Home Assistant for TV / lights (“change the channel by blink”).

Full map lives in the repo's STACK.md — pick a row, wire it, it goes live. The project grows in the open (to me).

Two lines and it's watching.

Built as a personal tool — sibling to my FlowLite dictation app. Pure Python, no installer, no telemetry. The ~8MB gesture model downloads itself on first run.

← Back to sriram.site
$ cd sentinel
$ pip install -r requirements.txt
$ python app.py
› SENTINEL is watching. Raise an open palm.