Every document you receive, turned into data you can use.
DocuScrape reads invoices and other business documents, pulls out exactly the fields you define, and delivers clean structured JSON to whatever system runs your business next.
Hover a stage to see it live — or just watch, it runs on its own.
invoice_0847.pdf2.4 MB · PDF
Dashboard, API, or SFTP — same intake either way.
Text blocks Detected
Tables Detected
Layout regions Matched
Pages2
Vendor
Meridian Supply
Invoice No
INV-70423
GSTIN✓ Valid
Date✓ Valid
Currency✓ INR
Total✓ Matches
Validation complete ✓
{}
JSON READY
API Webhook SFTP
JSON payload
↓
Delivery
↓
Your application
{ platform }
Watch it read a document.
This runs on its own below — or pick a document type and see that one instead. The same extraction engine handles all of them, driven by the fields you teach it.
✓ Extracted✓ Validated98% confidence
Description
Qty
Amount
Waiting for extraction to start…
{ "documentType":,
,
}
Field
Extracted value
Fields shown here come from a template you define once in Template Maker — nothing about this layout is hardcoded. This demo runs entirely in your browser on sample data.
Not a fixed schema — a system that adapts to the layouts your suppliers and partners actually send.
{}
Template-taught extraction
Annotate one sample document and DocuScrape matches every future document from that layout — by filename pattern, embedded metadata, or the position of the fields themselves.
filename matchpdf metadatacoordinate anchors
{}
Freeform field schema
Field names and structure are entirely yours — no fixed canonical shape to work around.
header.YourFieldName
{}
Validation & auto-correction
Dates, currency codes, HS codes, and numeric amounts are checked and corrected automatically before anything is saved.
{}
Confidence-scored fields
Every extraction is scored; anything below the bar is flagged for review instead of passed through silently.
{}
API, SFTP & webhooks
Submit documents through the REST API or an SFTP drop box, and get notified by webhook the moment processing finishes.
{}
Isolated, credential-scoped storage
Every document is stored per account, encrypted at rest and in transit. Client-facing upload credentials are scoped to exactly one destination, so a leaked key never exposes more than it was issued for.
{ security & audit }
A record that can't be quietly changed.
Every document processed on DocuScrape is written to an append-only log. Once recorded, an entry can be superseded, never rewritten — that's what makes the output usable as a record, not just data.
Encrypted storageDocuments are encrypted at rest and in transit, scoped to your account.
Scoped accessClient-facing integration keys are limited to one destination, revocable instantly.
Append-only audit trailEvery processing event is logged with a timestamp, never edited in place.
#4821309:14:02
a13f...9e02
Document received & queued
#4821409:14:03
6c88...44b1
Fields extracted, confidence scored
#4821509:14:04
f021...7ad9
Delivered to destination system
{ developers }
A plain HTTP API, not an SDK to learn.
Post a document, get structured fields back. No client library required to get started.
REST APISubmit a document, retrieve structured fields — scoped API keys per destination.
Signed webhooksGet notified the moment a document finishes, with a signature header to verify the payload.
SFTP drop boxPoint an existing file transfer process at a write-only folder — no code change on your end.
# fired on completion — signed, so you can verify it's really usPOST https://your-app.example.com/webhooks/docuscrape
X-DocuScrape-Signature:sha256=...{"document_id":"doc_9f21ac","status":"completed","needs_review":false}
{ pricing }
Pay for what you process.
Straightforward per-page pricing that gets cheaper as your volume grows.
Start with 50 page credits, free
Full platform access from your first upload. No credit card, no time limit on the credits.
Your account is waiting for approval. You'll be able to sign in once an administrator approves it.
Reset requested
An admin needs to approve this before your new password becomes active. Keep using your current password until then.
Loading your workspace…
$refs.cmdInput && $refs.cmdInput.focus()) }">
DocuScrape
ESC
/
No sub dockets here yet
Choose a Docket
/
No sub dockets here yet
Documents Processed
uploaded today
Processing Volume — Last 14 Days
Today:
Total Files
Processed
Queue
Avg Confidence
Avg Process Time
Recent Activity
No activity yet
Processed invoices will show up here as extraction runs complete.
Processing Queue
Now processing:
Quick Actions
Legacy Files Need Migration
Auto-detected dockets — click to preview
Filename
Detected Docket
Dockets Need Restructuring
Existing dockets will be grouped into supplier / sub docket folders
Supplier
Sub Dockets
Files
How DocuScrape works
Two separate setups meet in the middle: a Template Docket and a
Document Docket with the same name —
dockets can nest to any depth (supplier → sub docket → sub-sub docket…), and if no matching template
docket has anything in it, every other template docket is searched automatically before falling back to OCR.
Set up once, per supplier
Every time invoices arrive
✏️ Template Maker
Open a sample PDF, draw a box around each field
Labels are fully freeform — call a field whatever you like, search-to-find it in the field list as your set grows.
🗂 Document Docket tab
Create (or reuse) a docket
Dockets nest to any depth — supplier → sub docket → sub-sub docket — and any docket can hold files and sub dockets at once.
💾 Save
Choose the target docket, save the template
Picked via the same docket tree browser — the template docket is created automatically if it doesn't exist yet.
📤 Upload
Drop invoice PDFs into that docket
Via the UI, or hands-off through iSettings (a scoped Web API key or SFTP login) for a client's own automation.
📋 Template Docket
e.g. siemens
Holds the annotated template + its source PDF.
🗂 Document Docket
e.g. siemens
Holds the raw invoices waiting to be processed.
matched by name, then searched
⚙️ Worker
Picks up the upload automatically
A background job (Celery) fetches the PDF from storage the moment it lands in the docket.
🔍 Match & extract
Finds the right template, reads every field
Checks the same-named template docket first (filename → PDF metadata → field position); if it's empty,
every other template docket is searched too before reading each labeled region directly from the PDF,
falling back to OCR for scanned pages.
✅ Validate & save
Auto-corrects and writes to the database
Dates, codes and amounts are checked and cleaned up, global correction rules applied, before the run is saved.
📊 Done
Results land in the Dashboard, Activity & Results tabs
Template setup
Invoice intake
Matched by docket name
Automatic processing
Outcome
Drop PDF to upload
Document Docket
Nest dockets inside dockets as deep as you need — every docket can hold both files and sub dockets
Existing Dockets Not Yet in the Nested Tree
Copies your existing suppliers into the new tree — nothing is deleted
Supplier
Sub Dockets
Files
Migrated into the nested docket tree
/
PDF, JPG, JPEG, PNG
Nothing matches ""
Sub Dockets
Needs review
Files in
Filename
Size
Runs
Extraction
Uploaded
Actions
Process with Template
Run this file with a template you choose, instead of automatic template matching. Works for PDF, JPG and PNG.
Loading templates…
No templates in this docket's Template Docket yet. Create one in Template Maker and save it to the docket with the same ID.
A friendly ID (like 901, 90101…) is assigned automatically
Rename Docket
Only the display name changes — its ID and stored files are untouched
Delete Docket
Checking contents…
You're about to delete docket .
This will permanently delete
sub docket(s) (including nested sub-sub dockets)
and file(s) inside them.
This cannot be undone. Extraction history for these files is kept, but detached from any docket.
This docket is empty — it will be deleted with no other side effects.
AI Extraction Results
Review, correct, and export structured data extracted from your documents
Select a file to view results
Choose an invoice from the dropdown above
No results yet for this file
Docket:
Source Docket:
Source Docket:Legacy (flat)
Template Used:
PDF Type:
Fields Captured:
Document Preview
Raw JSONdeveloper mode
Extracted Fields
#
No structured field data on this run — try the Raw JSON view.
Nest template dockets as deep as you need — mirrors the Document Docket tree
/
Nothing matches ""
Sub Dockets
Templates in
Template Name
Source File
Regions
Keywords
Actions
—
PDF not found in this docket
A friendly ID (like 901, 90101…) is assigned automatically
Rename Docket
Delete Docket
Checking contents…
You're about to delete docket .
This will permanently delete
sub docket(s) (including nested sub-sub dockets)
and template(s) inside them.
This cannot be undone.
This docket is empty — it will be deleted with no other side effects.
Compare Results
Diff two extraction runs of the same document, or check a run against a reference JSON — then resolve mismatches field by field.
Accuracy Score
MatchMismatchClose
Baseline
Current
Select a Current Run and a Baseline Run to compare.
Select a file to start comparing runs
Pick two runs of the same document, or paste a reference JSON, to see a field-by-field diff
✏️
Field Marker
Upload a PDF, JPEG or PNG document to start annotating fields
Draw boxes over fields → export a reusable extraction template
Recent Drafts
📄
··
Template:→ Docket:
🖨️
Scanned PDF (image-based) — No selectable text found. Annotation boxes will still work — the OCR Engine will extract text from them during processing.
📋
Pre-marked PDF detected —
Page /
Drag to draw · Right-click or Del to delete · Esc to deselect · +/− to zoom
_fmRenderPage())" draggable="false" alt=""/>
Header Fields
Item Fields
No matching fields
Annotations
✏️
No annotations yet.
Select a label and drag on the PDF.
✎
✎Manually overridden
Region Preview✓ Good
✎Overridden
No text in region
Validation Rules
No rules yet — add one below
Template Extraction Test
p
— no text extracted —
Review Detected Fields
Field names were read from your PDF annotations. Confirm or change each one before importing.
PageLabel in PDFSystem Field
No exact match — set manually
→
Item Fields
Keep Original
Load Existing Template
Select a saved template to restore its annotations onto the current PDF
Loading templates…
No saved templates found
flat
Add Custom Field
Name the field and choose how it will be drawn on the template
Only needed to enable the automatic rate × qty ≈ amount cross-check
Only needed to enable the automatic rate × qty ≈ amount cross-check
Correction Rules
Rules fire in sequence after every extraction — each one reads the previous rule's output.
Total Rules
Enabled
Disabled
No correction rules yet
Rules auto-correct extracted fields before saving to the database
No rules match your filters
When
then
Rules fire at extraction time in the worker and can be re-applied to existing results viaPOST /api/correction-rules/apply/{run_id}
Input value → Output value pairs
Test this rule
→⚠ condition not met — rule skippedno change
Liveness Test
Check that the person holding the phone is real and, optionally, that their face matches the photo on an ID card. Random challenge, anti-spoof scoring and face match all run on the server. Prototype — see limits below.
Upload a clear photo of the card. The face on it is compared with the live face.
2 · Live check
The camera needs HTTPS or localhost. Allow camera access when the browser asks.
Could not complete
Raw response
3 · Recorded video
Upload a client's video and list the prompts they were shown, in order. The system watches the video and checks that each action really happens, in that order, and that the person is real. Left/right means the person's own left/right. Only the first 30 seconds are analysed. The ID card from step 1 (if any) is used for the face match.
Prompts, in the order they were shown
Timeline — head turn (line) and blink (dashed), with detected actions shaded
Raw response
Limits: not certified (no iBeta), no defence against virtual cameras or injected/deepfake video, thresholds not yet tuned on real attack samples or real client videos. A recorded video can be a replay of someone else, so the anti-spoof and face-match rows matter as much as the action checks. Use as a prototype or an extra fraud signal, not as the only check.
Extraction API Test Portal
Send documents to the live Extraction API and inspect the response. The API key stays in this tab's memory only — never stored.
No parameters loaded — enter the key and docket, click “Load sub-dockets”. A docket only has parameters once you add them under I-Settings → Parameters.
Sends with file part(s):
Presets
requestIdpackage revision
Cross-checks
Field
Value
Confidence
Status
Raw JSON
iSettings
You get direct, scoped access to bulk-upload into one Document Docket → Sub Docket — via Web API or SFTP.
These callers hit a docket that defines request parameters (e.g. cardtype) using a legacy request shape, so the default templates ran instead of the layout the parameter selects. Ask them to send the parameter with generic front/back file parts.
None so far. Every recent request to a parameterised docket used its parameters.
When
Docket
Client
Document
Request
Total Ingestions
Active
Revoked
No ingestions yet
Create one to generate a scoped API key and/or SFTP login for a specific supplier + sub docket
sftp -P 2222 <sftp_username>@<host>
put invoice.pdf
Webhooks (optional, set per ingestion)
POST <your webhook_url>
X-DocuScrape-Signature: sha256=<hmac(webhook_secret, body)>
{"event":"document.processed","filename":"invoice1.pdf",
"status":"success","confidence":97.2,"needs_review":false}
Fired once, no retry queue on failure — if your endpoint is down when a document finishes, that one notification is lost; the extraction result itself is unaffected and stays available through the endpoints above.
Both channels ignore any docket you send — every upload is pinned to your credential's own docket (any depth in the tree, or a legacy supplier/sub docket), so if it ever leaks, it only ever exposes the one docket it was created for.
API
Extraction API — Integration Guide
Send identity documents to DocuScrape and get structured fields back in the same call. This guide uses a docket with four sub-dockets: NIN-UG-Front, NIN-UG-Back, UG-Passport and UG-Visa. Replace docket-910, the host and the key with the values we give you.
Before you start
1. Get your API key and docket ID
From your DocuScrape contact — for example docket-910. Send the key as Authorization: Bearer <api_key> on every call.
2. List the sub-dockets you can upload into
Each subDocketId is the lower-case form of the sub-docket name (spaces and symbols become -). It is both the value you list in subDockets and the name of the file part you upload.
3. Upload one file per sub-docket
No template ID needed — the template saved inside each sub-docket is used automatically. Send only the sub-dockets you have — a visa alone, or NIN front + back, or all four. JPEG, PNG or PDF, up to 25 MB each.
4. Read sides.<subDocketId>.fields
If you get HTTP 202, poll the statusUrl until status is no longer QUEUED/PROCESSING.
Setup (once, by DocuScrape): save one template inside each sub-docket in Template Docket. Nothing to publish — the API picks them up automatically.
List the targets in subDockets, then add one file part named exactly after each one. documentId is required: your own reference for the customer or case (any unique text, e.g. your customer ID). It is echoed back in the response so you can match results to your records.
# NIN front + back in one request
curl -X POST https://<your-host>/v1/dockets/docket-910/documents/extract \
-H "Authorization: Bearer $API_KEY" \
-F documentId=customer-1042 \
-F subDockets=nin-ug-front,nin-ug-back \
-F nin-ug-front=@nin_front.jpg \
-F nin-ug-back=@nin_back.jpg
# Passport only
curl -X POST https://<your-host>/v1/dockets/docket-910/documents/extract \
-H "Authorization: Bearer $API_KEY" \
-F documentId=customer-1042 \
-F subDockets=ug-passport \
-F ug-passport=@passport.jpg
# Visa only
curl -X POST https://<your-host>/v1/dockets/docket-910/documents/extract \
-H "Authorization: Bearer $API_KEY" \
-F subDockets=ug-visa -F ug-visa=@visa.jpg
Response (200 when finished; field names depend on the template published for your docket):
Read, but have a person confirm it (common for ID numbers and MRZ lines).
INVALID_FORMAT
Failed a format check — ask for a clearer photo or review manually.
NOT_FOUND
Nothing readable in that area of the document.
If you get HTTP 202 — poll
The body contains status: "QUEUED" or "PROCESSING", a statusUrl and retryAfterSeconds. Repeat the GET until it finishes (the final body has the same shape as above).
A sub-docket is listed in subDockets but has no file part with that name.
400
UNEXPECTED_DOCUMENT
A file part was sent that isn't listed in subDockets.
404
SUB_DOCKET_NOT_FOUND
Unknown sub-docket id — the message lists the available ones.
404
DOCKET_NOT_FOUND
Wrong docket id, or the key isn't for this docket.
409
TEMPLATE_AMBIGUOUS
More than one template is published for the docket — contact us.
409
IDEMPOTENCY_CONFLICT
The same Idempotency-Key was reused with different files. Use a new unique key (e.g. a UUID) for every new document; reuse a key only to retry the identical request.
413
FILE_TOO_LARGE
File over 25 MB.
Optional: the Idempotency-Key header is not required — leave it out and everything works. Only add it if you want safe retries after a dropped connection, and then use a new unique value (e.g. a UUID) for every new document.
Using it from a mobile app
Don't ship the API key inside the app. Anyone can extract a key from an APK/IPA and use it. The safe pattern is: the app uploads the photo to your backend (authenticated as your user), and your backend calls DocuScrape with the key. The snippets below show the DocuScrape call itself — run it from your server, or from the app only for internal/testing builds.
Compress photos before upload (long edge about 2000 px, JPEG quality 80–90) — it is faster on mobile data and well under the 25 MB limit.
Android (Kotlin, OkHttp)
val body = MultipartBody.Builder().setType(MultipartBody.FORM)
.addFormDataPart("documentId", "customer-1042")
.addFormDataPart("subDockets", "nin-ug-front,nin-ug-back")
.addFormDataPart("nin-ug-front", "front.jpg", frontFile.asRequestBody("image/jpeg".toMediaType()))
.addFormDataPart("nin-ug-back", "back.jpg", backFile.asRequestBody("image/jpeg".toMediaType()))
.build()
val req = Request.Builder()
.url("https://<your-host>/v1/dockets/docket-910/documents/extract")
.header("Authorization", "Bearer $apiKey")
.post(body).build()
client.newCall(req).execute().use { resp ->
val json = JSONObject(resp.body!!.string())
// resp.code == 202 -> poll json.getString("statusUrl") every retryAfterSeconds
val fields = json.getJSONObject("sides").getJSONObject("nin-ug-front").getJSONArray("fields")
}
Import polls every ~5 minutes (cloud APIs are rate-limited, unlike the ~5s watcher used for the docket upload flow above). Export happens right after a document in the target docket finishes processing.
Images
Every JPEG/PNG uploaded, grouped by docket and sub docket. Click an image to view it full size.
No images found
Images uploaded through the Extraction API or a docket will appear here.
Upload an image of a QR code (for example the one on the back of a refugee ID card). It is decoded on the server and the data inside is organised into labelled fields. Nothing is saved.
Live — client requests
No client has sent a document through the Extraction API yet. New requests appear here automatically.
Needs review
Request:
File:
Status:
No QR could be decoded from this image.
Raw QR payload
▦
Drop a QR image here or click to choose
JPG, PNG, WebP, BMP or TIFF · up to 20 MB
Tip: a close-up where the QR fills most of the frame decodes far better than a whole-card photo — dense codes need about 4 pixels per square.
The decoded data will appear here.
Decoding…
Dense codes are tried at several scales; this can take a few seconds.
Not decoded
Image size:
Billing
1 credit is used per PDF page successfully processed — a 10-page invoice uses 10 credits. Prices are in USD.
Page Credits Remaining
Invoices Billed
Plans
Each plan supports a one-time prepaid purchase (no expiry, no auto-renew) or a monthly subscription (auto-recharges the same page quota every cycle).
Prepaid: Card, UPI or Net Banking. Subscription: Card or UPI Autopay (bank/UPI-app dependent) — use Prepaid if you need Net Banking on a recurring cadence.
Pay As You Go
No commitment — buy exactly the pages you need, whenever you need them, at $/page. Prepaid only, no subscription.
Your Subscriptions
Plan
Pages / mo
Status
Current Cycle
Transaction History
No transactions yet
Recharges and per-invoice debits will show up here.
Date
Type
Amount
Balance After
Reference
Clients
Every client's activity and credit balance, in one place.
Password Reset Requests
The client's current password still works until you approve one of these.
Client
Username
Requested
No clients yet
Create the first client account to get them logged in.
Name
Username
Balance
Invoices Today
Invoices Total
Status
Admin
Credit Balance
Recent Invoices
Dockets
Recent Invoices
Filename
Confidence
Needs Review
Date
Credit History
Date
Type
Amount
Balance After
Reference
Top Up —
New Client
Publish for Extraction API
You usually don't need this. If each sub-docket (NIN-UG-Front, NIN-UG-Back, …) has its own template saved inside it, the API picks them automatically — nothing to publish. Use this form only to bundle templates by hand.
One row per document. The sub-docket ID is the lower-case name of the sub-docket (e.g. nin-ug-front) — it must match exactly, because that is what API callers send. Pick the template used to read it.
All documents are published together as one version — the API always uses the same version of each.
This makes the template available to the real-time Extraction API. You get a templateId to pass with each request. Later edits in Template Maker don't affect what's published until you publish again.
Already published as , version . If the template hasn't changed since, no new version is created; if it has, a new version is added and older ones stay available.
Version:
· documents:
Use with docket: No Extraction API docket is linked to this docket yet — create an Extraction API ingestion for it in iSettings.
New Ingestion
Nest to any depth — supplier, sub docket, sub-sub docket, etc. The credential is scoped to exactly this docket.
The external ID your partner puts in the URL. Fixed once a docket has one — reusing this docket later keeps its existing ID.
If set, we POST a signed JSON payload here every time a document uploaded through this credential finishes processing — no polling needed. Leave blank to skip.
Request parameters
The client sends a parameter (e.g. cardType=new) plus generic file parts (front, back). Each value below picks which sub-docket receives each slot, so clients never see sub-docket names.
Value
→
Requests without a parameter keep working the old way (subDockets=…). Removing all parameters switches this off.
Webhook
Clear this field and save to disable delivery. Changing the URL keeps the existing signing secret — use "Regenerate secret" below only if the secret itself may have leaked.
Integration created
⚠ Copy these now — they won't be shown again.
No Extraction API template is published for this docket yet — publish one from Template Docket (⋮ → Publish for Extraction API…). Its ID and version then appear here and on this credential's card.
Replace <API_KEY> with the API key above and <unique-ref> with your own reference for the document. Send only the sub-dockets you have — list just those in subDockets.
Every webhook POST carries an X-DocuScrape-Signature: sha256=<hmac> header, computed over the raw request body using this secret — verify it on your end before trusting the payload.
📄
DocuScrape — Project Documentation
Version 3.0 · Last updated August 2026 · POC Build (GPU-accelerated)
This document explains what DocuScrape does, how it works step by step, what features are currently live, and what is planned for future releases — all in plain, easy-to-understand language.
1
What is DocuScrape?
DocuScrape is a system that automatically reads PDF invoices and pulls out the important information — like supplier name, invoice number, date, item descriptions, quantities, prices, and more — and saves it in a structured digital format (JSON) that other systems can use directly.
Think of it as a very smart data-entry assistant. Instead of a person manually typing invoice data into a spreadsheet or ERP system, DocuScrape does that job automatically, in seconds, with high accuracy — for known suppliers, without needing any AI model call at all.
Why does this matter? Processing invoices manually is slow, error-prone, and expensive. DocuScrape removes that bottleneck — a 20-page invoice that would take 15–20 minutes to manually enter can be processed in seconds once a template exists for that supplier.
Runs GPU-accelerated. This build runs on server hardware with an NVIDIA GPU — OCR uses CUDA. An earlier version of this system also called out to a local AI vision/text model (Ollama) for invoices it didn't recognise; that model has been switched off by default for this deployment (see Section 3, independent of the GPU/CPU choice) in favour of template-driven extraction, which needs no AI model at all for known suppliers.
2
How the System is Built
The system runs entirely on your local server — no internet connection, no cloud, no data leaving your premises. It is made up of several components that work together:
Component
What it does (plain English)
File Store (MinIO)
A secure locker where all uploaded PDF invoices are stored. Think of it like Google Drive, but running on your own server. Two storage layouts coexist side by side: the original flat supplier/sub docket layout, and a newer arbitrary-depth docket tree (dockets nested inside dockets, to any depth).
Queue (Redis)
A waiting list. When a PDF is uploaded, its name is added to this list so the Worker knows what to process next. Also used to avoid queuing the same file twice.
Database (PostgreSQL)
Stores all extracted invoice data permanently, the docket tree structure, and client integration credentials — so you can look up any past run, compare runs, or export the data.
Worker
The hands. Picks up each PDF from the queue, converts it to images, matches it against a saved template, reads text directly from the PDF or via OCR (EasyOCR), validates the result, and saves it to the database. Runs GPU-accelerated (CUDA) on this deployment.
Watcher
A background monitor that automatically detects when a new PDF is uploaded — in any of the three storage layouts — and queues it for processing within about 5 seconds, no manual trigger needed.
API + Web UI
The interface you see and use. Provides the web dashboard, all tabs, and also exposes a REST API so other software can connect directly.
SFTP Bridge
A small write-only drop box a client's own systems can push PDFs into over SFTP, without ever touching the web UI. Scoped to exactly one docket by a credential you issue them — see iSettings, below.
3
Step-by-Step Workflow
This is what happens from the moment you upload a PDF to the moment the extracted data is ready:
1
Upload PDF
You upload a PDF invoice through the web UI (Document Docket tab — either the flat supplier/sub docket layout or a nested docket tree, any depth), or hands-off through iSettings (a scoped Web API key or SFTP login handed to a client) without touching the UI at all.
2
Auto-Queue
The Watcher detects the new file within 5 seconds and adds it to the processing queue. It also prevents the same file from being processed twice, even if you re-process it manually right after uploading.
3
Template Check
The Worker checks whether a template exists for this docket. A template is a pre-configured map of where each field lives on that supplier's specific invoice layout. Three checks are done — by filename, by embedded PDF metadata (creator/producer), and by matching anchor text at known coordinates. If the same-named template docket has nothing in it, every other template docket is searched too before giving up, so a naming mismatch doesn't silently skip template matching.
4a
Template Path (fast & precise — the primary path)
If a template is matched, the system uses direct text extraction from the PDF — pulling the exact text sitting at each annotated region. For scanned/image regions, it falls back to EasyOCR (Tesseract is only used as an emergency fallback if EasyOCR itself is unavailable). This path needs no AI model call at all, is very fast, and is the one almost every invoice takes once a supplier has a template.
4b
No-Template Path (new / unrecognised invoices)
If no template matches, this deployment currently degrades to OCR-only extraction (EasyOCR) and flags the result "Needs Review" rather than reading it with an AI model — the vision/text LLM stage that would otherwise handle unknown layouts is switched off by default (an environment flag re-enables it if an LLM backend is set up; this is independent of this build already having GPU access). In practice this means: train a template for any supplier you process regularly — see Section 4.
5
Validation & Auto-Correction
The extracted data goes through automatic checks: country codes are standardised, dates are formatted correctly, numeric values are cleaned, and — for templates that opt individual fields into it — a rate×quantity≈amount cross-check catches common extraction errors. A global correction-rule pipeline then applies any configured find/replace-style rules before the result is saved. Any remaining issues are flagged for review.
6
Targeted Re-Extraction
On a deployment with the AI model enabled, fields that failed validation get asked again with a focused prompt about just those problem fields. On this build (model disabled by default, regardless of GPU availability) this step is a no-op — nothing to re-ask.
7
Confidence Scoring
Template-matched fields each get a 0–100% quality score based on content, length and character validity — averaged into the run's overall confidence. (A separate dual-pass comparison score exists specifically for the AI-vision path, when that path is enabled.) Low-confidence or no-template runs are flagged "Needs Review".
8
Save & Serve
The final structured JSON — along with the confidence score, processing time, and status — is saved to the database. It is immediately accessible via the Results tab in the UI or via the API, for both the flat docket layout and the newer docket tree.
4
How Template Training Works
A template teaches the system exactly where each field is located on a particular supplier's invoice. Once trained, the system doesn't need to guess — it reads the right spot directly. This is what lets every invoice avoid the no-template OCR-only fallback described in Section 3.
1
Open Template Maker tab
Go to Template Maker in the sidebar and upload a sample PDF invoice from the supplier you want to train.
2
Draw bounding boxes
Click and drag on the PDF to draw a box around each field. Labels are fully freeform — type any name you like, or pick one from the searchable field list as it grows. Item-table fields (quantity, rate, amount) can each optionally be tagged with a "role", which opts them into the automatic rate×qty≈amount cross-check.
3
Save the template
Give the template a name and choose a target docket — using the same tree browser used elsewhere in the app, so it can go as deep as you need (supplier → sub docket → sub-sub docket…). The template is stored as a JSON file with the position and label of every annotated region; the docket is created automatically if it doesn't exist yet.
4
Future invoices matched automatically
Next time an invoice lands in that docket (or any docket sharing its name), the system recognises it and uses the template — no AI model call needed, resulting in faster and more consistent extraction.
5
What's Currently Available
PDF Management
Upload PDF invoices through the web UI, the REST API, or a client-facing scoped credential (iSettings)
Two coexisting docket layouts: the original flat supplier/sub docket structure, and a newer docket tree that nests to any depth (supplier → sub docket → sub-sub docket…) — either can hold both files and further sub dockets at once
Create, rename, and delete dockets in either layout; deleting a tree docket can cascade to all its sub dockets and files at once, with a warning showing exactly what will be removed first
Auto-detect supplier name from the PDF filename
One-click, non-destructive migration from the flat layout into the tree (copies, never moves — nothing is deleted)
View, re-process, or delete individual PDFs per docket
Stream / preview any PDF directly in the browser
Extraction Engine
Template-driven extraction (PyMuPDF direct text + EasyOCR for scanned regions) — the primary path, needs no AI model call, and covers almost every invoice once a supplier has a template
Three-step template matching: filename keywords → PDF metadata → coordinate anchors, searching every other template docket too if the same-named one is empty
Optional vision/text AI model path for invoices with no matching template at all — disabled by default; those invoices instead get OCR-only extraction and are flagged "Needs Review"
Multi-page invoice support — all pages are processed
Wrapped, multi-line item descriptions are correctly read as a single line item rather than being split into phantom extra rows
Validation & Auto-Correction
Auto-correct country codes to ISO 2-letter standard (e.g. "INDIA" → "IN")
Auto-format invoice dates to DD/MM/YYYY
Clean and validate numeric fields (amounts, rates, quantities)
Rate × Quantity cross-check — opt any template's item fields in via a "role" tag; catches common extraction errors automatically
A global correction-rule pipeline applies configurable find/replace-style rules to every extraction before it's saved
Flag records that need human review (low confidence, or no template matched at all)
Template Maker (Training Tool)
Upload any PDF and render it page-by-page in the browser at 300 DPI
Draw bounding boxes on fields by clicking and dragging
Field labels are fully freeform — type any name, with a searchable picker as your label list grows
Auto-suggest labels from the text under each drawn box, and from a PDF's own embedded annotations if it has any
Save templates into any docket in the tree, at any depth, via the same docket picker used elsewhere in the app
Upload existing template JSON files directly
Edit a previously saved template — reopen it with all its boxes already drawn, adjust, and overwrite
Manage template dockets — create, rename, delete (with cascade-delete + warning, same as Document Docket)
Dashboard & Results
Dashboard showing total files, processed count, and "needs review" count
Recent activity feed — last 50 processed invoices, across both docket layouts
Live queue depth indicator (how many invoices are waiting to process)
Per-docket view: see all PDFs in a docket with run count and status
Results tab: select any file + run — from either docket layout — view full extracted JSON, and compare two runs side-by-side
Download extracted data as JSON, in either its raw shape or a normalised Header/InvoiceDetails/ItemsDetails shape
Re-trigger processing for any file from the UI
Client Integrations (iSettings)
Issue a scoped credential — Web API key, SFTP login, or both — that lets an external client push PDFs straight into one specific docket, at any depth, without ever touching the UI
Each credential is locked to exactly one docket: a leaked key or login only ever exposes that one docket, nothing else
Revoking a credential kills both channels immediately — no separate OS users or out-of-band state to clean up
Raw secrets are shown exactly once at creation time and never stored or retrievable again
API Access
Full REST API — upload, process, and retrieve results programmatically
Results returned as structured JSON ready for ERP/system integration
Docket management endpoints for both layouts (create, list, rename, delete)
The following are genuinely outstanding — not yet done, or done partially. (A number of items that used to be on this roadmap — editable templates, freeform/custom field labels, arbitrary docket nesting, a JSON comparison module — have since shipped and moved to Section 5.)
Training Improvements
Clone a template as a starting point — When creating a template for a new supplier, load an existing similar template as a base and adjust only what's different — saves significant training time.
Additional extraction hints per template — Add a custom instruction to a specific supplier's template to guide extraction on unusual fields or layouts, without affecting any other template. Most useful once the AI model path is enabled on a given deployment.
Quality & Accuracy
Surface when the rate×qty≈amount cross-check is inactive — Templates that haven't tagged any fields with a role silently skip the cross-check; a "cross-check inactive" indicator on the Results tab would make that visible instead of reading as a possible regression.
Accuracy & processing time on the dashboard — Show average accuracy % and average processing time per docket, so it's easy to see which suppliers are performing well and which need a template.
Further multi-row / complex-table handling — Wrapped single-item descriptions are now handled correctly (see Section 5); genuinely complex layouts — merged cells, nested headers — still need broader testing against reference data.
Large invoice testing — Test and tune the pipeline for invoices with 50–100+ pages, to confirm accuracy and processing time hold up at scale.
Monitoring & Reporting
Supplier-wise performance dashboard — A dedicated view showing, per docket: how many templates exist, how many invoices have been processed, average accuracy, average processing time, and how many need review.
Cleanup, once the docket tree is confirmed stable everywhere
Retire the legacy flat-docket fallback code in the worker and watcher, once all real data is confirmed migrated off the pre-2026-07-17 one-level layout.
Extend docket-tree template matching's fallback search to more closely match the legacy path's "search every other docket" behaviour, if mismatches turn out to be common in practice for very large trees.
Infrastructure
Published domain access with TLS — Everything currently runs over plain HTTP on the server's IP; a proper domain + TLS would let this be exposed more broadly without the current caveats around credentials and API keys travelling in the clear.
Re-enabling the AI model path — the vision/text LLM stage exists in the codebase and can be turned back on (e.g. once GPU capacity is available again) to handle invoices with no matching template, instead of the current OCR-only + "Needs Review" degrade behaviour.
7
What the Extracted Data Looks Like
Two output shapes coexist, distinguished by a schema_version field on the saved run — older rows and new ones are both handled transparently everywhere they're displayed:
Shape
What's inside
Freeform (current, schema_version 2)
header and items, keyed literally by whatever labels the matching template defines — a template with a field called "TermsAndConditions" produces exactly that key. No fixed field list.
Legacy canonical (older rows, schema_version 1 or missing)
A fixed supplier / importer / invoice / items shape used before templates went fully freeform. Still readable and displayed exactly as before — nothing was reprocessed to migrate it.
Metadata (both shapes)
Extraction method used, confidence score, processing time in seconds, template name, and any validation issues found.
A separate "Download structured" option normalises either shape into a fixed Header / InvoiceDetails / ItemsDetails layout for downstream systems that expect a consistent structure. For freeform-shape rows, InvoiceDetails is always empty by design — there's no fixed schema to sort those fields into, so everything lives under Header and ItemsDetails instead.
Note: Because field vocabulary is fully freeform per template, two suppliers' outputs may use entirely different key names for conceptually similar data — that's expected, not a bug. Correction rules and per-template validation rules both resolve against whatever shape a given row actually has.
DocuScrape · POC Build, GPU-accelerated · Running on port 8081 · All processing is local — no data leaves this server