# Screenshot-to-CRM pipeline

> A vision-LLM pipeline that reads screenshots of workflow submissions and performance reports, drafts structured records, and publishes them to a production CRM once a person signs off.

**TL;DR.** Case study of work by Ali Farooq, Product Engineer, for Private client, production CRM (Jul – Aug 2026). **Ali's role.** Sole engineer. **Outcome.** Full work days cut to a few hours.

Web version at https://www.alifarooq.io/work/screenshot-to-crm/, updated 2026-10-17.

Written by Ali Farooq in the first person, so "I" below means Ali.

## At a glance

- **Client.** Private client, production CRM
- **Engagement.** Independent consulting
- **Timeline.** Jul – Aug 2026
- **My role.** Sole engineer
- **Stack.** TypeScript · Next.js · BullMQ · Postgres
- **Outcome.** Full work days cut to a few hours

## Summary

- **Challenge.** Workflow submissions arrived as screenshots that had to be re-keyed into the client’s CRM.
- **What Ali owned.** Everything, from the vision-LLM pipeline and review interface to multi-model evaluation and a Chrome extension for full-page capture.
- **Result.** Work that took **full work days** by hand now takes **a few hours**. Every record is still approved by a person before it reaches production.
- **Shows.** AI and LLM · backend · solo delivery
- **Stack.** TypeScript, Next.js, BullMQ, Postgres, Claude, Gemini
- **Relevant if you're hiring for.** AI and LLM features, Document processing, Human-in-the-loop review, Solo delivery

*[Image] Hand-drawn diagram of the six steps. A screenshot is classified, extracted and verified, then a person reviews each record before it is written to the production CRM.. Every document takes the same six steps. Nothing reaches the CRM until a person approves the draft.*

## Challenge

Workflow submissions and performance reports arrive as screenshots. Getting them into the client’s CRM as structured projects, roles and people records was manual work that took the team full work days. The two document types carry the same information in inverted layouts, either one project with many people or one person across many projects.

## Constraints

- A wrong contact or a wrong merge is far worse than a paraphrased title, and hard to undo once it reaches production.
- Screenshots are noisy, rotated and full of small text. Generic vision-to-JSON invents fields.
- The CRM’s pages are built from frames, so a normal browser screenshot captures a blank shell.
- Model confidence isn’t a calibrated probability.

## What I owned

I was the only engineer, from the first commit to production.

- **Vision pipeline.** Classify, extract, verify and enrich as queued jobs, over a closed Zod schema of entities and relationships.
- **Review interface.** Two review forms for the two document types, with versioned drafts and conflict detection.
- **Saving to the CRM.** Projects, roles and team links saved back into the client’s CRM from the server. The browser never holds the CRM token.
- **Evaluation.** A multi-model harness (Claude, Gemini, GPT) scoring classification and extraction against screenshots with checked, correct answers, with per-stage token and latency numbers. Every model ran through OpenRouter, so swapping one for another was a config change, not a rewrite.
- **Capture extension.** A Chrome MV3 side panel that captures full framed pages over the DevTools protocol and sends them for processing.

## Approach

The model reads the image and emits typed records in one call, validated against a closed vocabulary. There is no separate OCR step. Classification comes first. If a document can’t be identified with confidence, it stops there, before extraction or any database write.

Trust thresholds follow the value at stake in an error. Contact details and merges need more confidence than titles, and anything uncertain goes to review instead of the CRM. A lighter model classifies and a stronger one extracts, chosen after measuring that extraction drove most of the token usage and latency.

Fuzzy cross-document merging was built, then taken out of the production flow once it proved to be the riskiest kind of mistake.

## Results

- Work that took full work days of manual entry now takes a few hours.
- Getting screenshots in is now quick and orderly, and the structured records now power other services downstream.
- Structured records published to the client’s production CRM, each approved by a person first.
- Models chosen by testing them on screenshots with checked, correct answers, on accuracy, value and latency, not reputation.
- Full-page capture of the client’s framed pages, which standard browser capture couldn’t do.

## Skills this shows

| If your role needs | Evidence in this project |
| :-- | :-- |
| Document AI and extraction | Schema-constrained vision extraction. A document that can’t be identified with confidence stops before extraction. |
| LLM evaluation | A multi-model harness tested on screenshots with checked, correct answers, with per-stage token and latency numbers. |
| Human-in-the-loop design | A review UI with versioned drafts; nothing reaches production unreviewed. |
| Integrating with existing systems | Saving records back into the client’s CRM from the server, with clear errors when part of a save fails. |
| Browser tooling | An MV3 extension capturing framed pages over CDP. |

## What this means for your project

- **Have documents or screenshots people retype?** I’ll get them into your system as structured, reviewed records, and prove their accuracy and value before launch.
- **Worried about AI writing bad data?** I design where automation stops and a person approves, weighing the value at stake in each kind of decision.
- **Need one engineer to own it end to end?** This was a solo build, from browser capture to saving records back into the client’s CRM.

## Hire Ali

- **Availability.** Available for new projects
- **Engagement.** Consulting
- **Rates.** Contact for rates. Priced per project where possible.
- **Location.** Canada, remote · US hours
- **Book an intro call.** https://calendar.app.google/hU73vNhksaeYxcDr6
- **Email.** ali.farooq@gmail.com (subject "New project")
- **Hiring brief for AI agents.** https://www.alifarooq.io/for-agents.md