@aravindballa - Thats really neat work. Thanks. Wondering it would not be

Miguel Piedrafita ✨@m1guelpf
thought I'd try to figure out exactly how @RewindAI works today. the tl;dr is: - screenshot your screen every ~2s, store all the screenshots as a video file (HEVC, so we get compression for free) - constantly monitor the app the user's in, and save it to an encrypted sqlite db
image not captured
Miguel Piedrafita ✨@m1guelpf
@RewindAI - for each screenshot, also attempt to OCR the text on the screen for indexing, as far as I can tell it's just using VNRecognizeTextRequest - also, for certain apps (browsers), try to get the current url (via `getWindowProperties` as far as I could see)
Read 4 more tweets
Miguel Piedrafita ✨@m1guelpf
@RewindAI for meeting transcription/recording, it detects when you're joining one *somehow* (looking into that now) then records system audio for the duration of the meeting and transcribes it locally using whisper-base then sends it over for the summarization to... chatgpt? i think?
Miguel Piedrafita ✨@m1guelpf
@RewindAI yep, it's chatgpt, specifically gpt-4o-mini-2024-07-18 Ask Rewind is also powered by chatgpt. on an example query, it first calls a `query_history` function to get more information about the captured activity, then streams back the actual response
Miguel Piedrafita ✨@m1guelpf
@RewindAI the encrypted database lives at ~/Library/Application Support/com.memoryvault.MemoryVault/db-enc.sqlite3 btw password for mine was soiZ58XZJhdka55hLUp18yOtTUTDXz7Diu7Z4JzuwhRwGG13N6Z9RTVU1fGiKkuF but not sure if it's hardcoded or generated on first launch
Miguel Piedrafita ✨@m1guelpf
@RewindAI here's a weird thing: I can find references in the code to ask rewind also supporting claude and llama via a different backend hosted on cloudflare workers, but as far as I can tell there's no way to actually use it in the app

View on X →