Ali FarooqBook a call

Case study · Document AI · 2026

Screenshot-to-CRM pipeline

A vision-LLM pipeline that reads screenshots of workflow submissions and performance reports, drafts structured records, and publishes them to a production CRM once a person signs off.

Hand-drawn diagram of the six steps. A screenshot is classified, extracted and verified, then a person reviews each record before it is written to the production CRM.
Every document takes the same six steps. Nothing reaches the CRM until a person approves the draft.
Client
Entertainment-industry software company
Engagement
Independent consulting
Timeline
Jul – Aug 2026
My role
Sole engineer
Stack
TypeScript · Next.js · BullMQ · Postgres
Outcome
Full work days cut to a few hours
On this page
  1. 01Challenge
  2. 02Constraints
  3. 03What I owned
  4. 04Approach
  5. 05Results
  6. 06Skills this shows
  7. 07For your project

We’re really happy with how V0 turned out. Everything has been handed off, and it’s been wonderful working with Ali.

Founder & CEOentertainment-industry software company

Challenge

Workflow submissions and performance reports arrive as screenshots. Getting them into the client’s CRM as structured projects, roles and people records was manual work that took the team full work days. The two document types carry the same information in inverted layouts, either one project with many people or one person across many projects.

Constraints

  • A wrong contact or a wrong merge is far worse than a paraphrased title, and hard to undo once it reaches production.
  • Screenshots are noisy, rotated and full of small text. Generic vision-to-JSON invents fields.
  • The CRM’s pages are built from frames, so a normal browser screenshot captures a blank shell.
  • Model confidence isn’t a calibrated probability.

What I owned

I was the only engineer, from the first commit to production.

  • Vision pipeline. Classify, extract, verify and enrich as queued jobs, over a closed Zod schema of entities and relationships.
  • Review interface. Two review forms for the two document types, with versioned drafts and conflict detection.
  • Saving to the CRM. Projects, roles and team links saved back into the client’s CRM from the server. The browser never holds the CRM token.
  • Evaluation. A multi-model harness (Claude, Gemini, GPT) scoring classification and extraction against screenshots with checked, correct answers, with per-stage token and latency numbers. Every model ran through OpenRouter, so swapping one for another was a config change, not a rewrite.
  • Capture extension. A Chrome MV3 side panel that captures full framed pages over the DevTools protocol and sends them for processing.

Approach

The model reads the image and emits typed records in one call, validated against a closed vocabulary. There is no separate OCR step. Classification comes first. If a document can’t be identified with confidence, it stops there, before extraction or any database write.

Trust thresholds follow the value at stake in an error. Contact details and merges need more confidence than titles, and anything uncertain goes to review instead of the CRM. A lighter model classifies and a stronger one extracts, chosen after measuring that extraction drove most of the token usage and latency.

Fuzzy cross-document merging was built, then taken out of the production flow once it proved to be the riskiest kind of mistake.

Results

  • Work that took full work days of manual entry now takes a few hours.
  • Getting screenshots in is now quick and orderly, and the structured records now power other services downstream.
  • Structured records published to the client’s production CRM, each approved by a person first.
  • Models chosen by testing them on screenshots with checked, correct answers, on accuracy, value and latency, not reputation.
  • Full-page capture of the client’s framed pages, which standard browser capture couldn’t do.

Skills this shows

If your role needs Evidence in this project
Document AI and extraction Schema-constrained vision extraction. A document that can’t be identified with confidence stops before extraction.
LLM evaluation A multi-model harness tested on screenshots with checked, correct answers, with per-stage token and latency numbers.
Human-in-the-loop design A review UI with versioned drafts; nothing reaches production unreviewed.
Integrating with existing systems Saving records back into the client’s CRM from the server, with clear errors when part of a save fails.
Browser tooling An MV3 extension capturing framed pages over CDP.

What this means for your project

  • Have documents or screenshots people retype? I’ll get them into your system as structured, reviewed records, and prove their accuracy and value before launch.
  • Worried about AI writing bad data? I design where automation stops and a person approves, weighing the value at stake in each kind of decision.
  • Need one engineer to own it end to end? This was a solo build, from browser capture to saving records back into the client’s CRM.