Reply Pilot Gmail

Tato stranka popisuje modul reply-pilot-gmail/.

Role modulu

  • vlastni Google OAuth a Gmail API integraci
  • vystavuje interni HTTP API pro reply-pilot-be
  • vytvari mailbox snapshot payloady a zapisuje Gmail-owned mailbox cache
  • vlastni emailovou runtime cache pro inbox read model, attachment bytes a weekly-count reporty pouzivane Jira Reports
  • umi inkrementalni sync pres Gmail historyId pro vychozi inbox/sent cache
  • umi udrzovat Gmail watch stav a prijmout Pub/Sub notifikaci o zmene mailboxu
  • vytvari Gmail reply drafty

Runtime

  • bezi jako samostatny Java 21 Spring Boot/Maven kontejner
  • zapisuje do reply-pilot-gmail/data a reply-pilot-gmail/logs
  • conf/ zustava read-only mount
  • na interni siti se hlasi jako reply-pilot-gmail
  • reply-pilot-be ho vola pres http://reply-pilot-gmail:5000
  • reply-pilot-jira-reports ho vola pro email weekly-count report pres http://reply-pilot-gmail:5000/api/reports/email-weekly-counts
  • Gmail token se drzi v data/gmail_token.json, aby refresh flow mohl zapisovat
  • watch stav se drzi v data/gmail-watch.json, aby se dal bezpecne obnovovat mezi restarty
  • Gmail mailbox cache se drzi pod data/accounts/<GMAIL_CACHE_ACCOUNT_ID>/emails/; vychozi account id je default po safe path sanitizaci
  • reply-pilot-gmail/data/ je runtime-only adresar a nesmi byt verzovany v gitu

Endpointy

  • GET /healthz
  • kontroluje jen lokalni runtime modulu a pritomnost/citelnost Gmail token souboru
  • neprovadi zadny primy call do Gmail API; dostupnost Gmailu se projevi az pri syncu nebo tvorbe draftu
  • GET /api/meta
  • GET /api/rate-limit/status
  • GET /api/gmail/rate-limit/status
  • GET /api/watch
  • POST /api/watch/ensure
  • POST /api/watch/stop
  • POST /api/watch/notify
  • POST /api/watch/pull
  • POST /api/watch/ack
  • POST /api/mailbox/snapshot
  • POST /api/mailbox/history
  • GET /api/reports/email-weekly-counts
  • GET /api/cache/status
  • GET /api/cache/inbox
  • GET /api/cache/emails
  • GET /api/cache/emails/<email_id>
  • GET /api/cache/emails/<email_id>/attachments
  • GET /api/cache/emails/<email_id>/attachments/<filename>
  • GET /api/emails/<email_id>
  • GET /api/emails/<email_id>/attachments
  • POST /api/emails/<email_id>/drafts/reply
  • POST /api/emails/drafts/outbound
  • POST /api/emails/<email_id>/send/reply
  • POST /api/emails/send/outbound
  • POST /api/emails/<email_id>/send/forward

Reply draft/send endpointy a outbound send endpoint podporuji JSON bez priloh a multipart/form-data s textovymi poli recipient, subject, body_text, volitelne reply_to_message_id a opakovanym file polem attachments. Limit je 10 souboru, 10 MB na soubor a 20 MB celkem.

Gmail-Owned Mailbox Cache

reply-pilot-gmail je jediny modul, ktery smi drzet Gmail OAuth material, Google client dependencies, Gmail watch/rate-limit stav a fyzicke mailbox cache soubory. reply-pilot-be pouziva jen interni HTTP endpointy Gmail modulu.

Fyzicky layout:

  • /app/data/accounts/<GMAIL_CACHE_ACCOUNT_ID>/emails/index.json
  • /app/data/accounts/<GMAIL_CACHE_ACCOUNT_ID>/emails/threads/*.json
  • /app/data/accounts/<GMAIL_CACHE_ACCOUNT_ID>/emails/attachments/<thread>/<file>

GMAIL_CACHE_ACCOUNT_ID defaultuje na default a runtime jej sanitizuje pro bezpecnou filesystem komponentu. Attachment relative_path v JSON payloadu a v DB metadata zustava logicka hodnota jako emails/attachments/<thread>/<file>, aby nebyla potreba DB migrace.

Jednorazovy presun stare BE cache:

cd /Users/jan/projects/reply-pilot/reply-pilot-gmail
scripts/gmail-migrate-cache.sh --account-id default

Script kopiruje vychozi ../reply-pilot-be/data/emails do ${HOST_DATA_DIR}/accounts/<account-id>/emails a odmita ne-prazdny target, pokud neni explicitne zadane --force.

Watch workflow

  • POST /api/watch/ensure vola Gmail users.watch jen kdyz existujici watch nema dostatecnou rezervu do expirace
  • vychozi watch labels jsou INBOX,SENT, aby incremental sync zachytil prijata vlakna i sent-only odchozi vlakna
  • worker muze watch pravidelne obnovovat pres WORKER_GMAIL_WATCH_* promenne bez dalsiho full scanu mailboxu
  • POST /api/watch/pull cte Gmail Pub/Sub Pull subscription a vraci dekodovane notifikace vcetne ack_id
  • POST /api/watch/notify ulozi metadata prijate Pub/Sub notifikace a vraci, zda je potreba spustit navazujici import workflow
  • pull flow drzi ack_id v Gmail-owned gmail-watch.json a potvrdi je az po uspesnem zapisu mailbox-wide full snapshotu nebo incremental history cursoru; cilene backfill snapshoty pending watch notifikace nepotvrzuji
  • POST /api/watch/ack zustava explicitni admin operace; pri sync/cache chybe automaticky flow ack neposila, aby doslo k redelivery
  • doporuceny provoz je zapnout watch renewal a workerovy gmail_synchronizer, ktery spojuje watch pull i mailbox import do jedne pipeline

Gmail OAuth Token Lifecycle

Gmail OAuth token je uzivatelsky token pro Gmail API a je odlisny od Pub/Sub service account JSON. Runtime soubor je:

  • host: ${HOST_DATA_DIR}/gmail_token.json
  • kontejner: /app/data/gmail_token.json
  • env: GMAIL_TOKEN_FILE=/app/data/gmail_token.json

Sifrovany source of truth:

  • secrets/local/reply-pilot-gmail-token.json pro lokalni vyvojovy mailbox
  • secrets/prod/reply-pilot-gmail-token.json pro produkcni mailbox

Start/deploy workflow materializuje token do reply-pilot-gmail/data/, pokud odpovidajici SOPS secret existuje. Token se nema kopirovat do Docker image a reply-pilot-gmail/data/ se necommituje.

Vygenerovani tokenu:

cd /Users/jan/projects/reply-pilot/reply-pilot-gmail
scripts/gmail-generate-token.sh \
  --token-file /tmp/reply-pilot-gmail-token.json \
  --client-id '...apps.googleusercontent.com' \
  --client-secret 'GOCSPX-...'

Zasifrovani lokalniho tokenu:

sops encrypt \
  --input-type json \
  --output-type json \
  --filename-override ../secrets/local/reply-pilot-gmail-token.json \
  /tmp/reply-pilot-gmail-token.json \
  > ../secrets/local/reply-pilot-gmail-token.json
rm -f /tmp/reply-pilot-gmail-token.json

Pro produkci pouzij ../secrets/prod/reply-pilot-gmail-token.json. Lokalni vyvojovy token ma patrit osobnimu nebo testovacimu mailboxu; produkcni token se lokalne nepouziva kvuli Gmail API kvotam a riziku zasahu do produkcni schranky.

Pole expiry v JSONu je expirace kratkodobeho access tokenu. Dlouhodobe opravneni drzi refresh_token; aplikace pri expiraci vola Google token endpoint a ziska novy access token.

Pub/Sub Runtime Config

  • GMAIL_WATCH_TOPIC_NAME je plne jmeno Gmail watch topicu, napriklad projects/<project-id>/topics/gmail-watch
  • GMAIL_WATCH_SUBSCRIPTION_NAME je plne jmeno environment-specific Pull subscription, napriklad projects/<project-id>/subscriptions/reply-pilot-dev-sub
  • GMAIL_PUBSUB_CREDENTIALS_FILE muze ukazovat na service account JSON pod /app/conf/...; kdyz chybi, modul zkusi Application Default Credentials
  • doporuceny sifrovany source of truth pro Pub/Sub service account JSON je secrets/<environment>/reply-pilot-gmail-pubsub.json; start/deploy workflow ho materializuje do reply-pilot-gmail/conf/reply-pilot-pubsub.json
  • lokalni a produkcni instance maji mit vlastni subscription; obe ale mohou cilit na stejny topic
  • lokalni a produkcni instance nemaji sdilet stejnou Pull subscription, pokud maji bezet soucasne
  • pokud lokalne netestujes Pub/Sub, vypni workerove WORKER_GMAIL_WATCH_* a WORKER_GMAIL_WATCH_PULL_* prepinace