Building an OCR-to-JSON Workflow: Extract Invoice Data from Scanned Documents

Scanned invoices, receipts and forms are only useful once their contents are in your database. Plain OCR gets you halfway: you end up with a block of text, in reading order if you are lucky, with no idea which line is the vendor and which is the tax number. This guide shows how to build a production OCR workflow that ends in clean, validated JSON: image preparation, text recognition, schema-based extraction, validation rules and human review.

The pipeline at a glance

A reliable document-ingestion workflow has four stages. Each one has a single job, which makes problems easy to find when a document goes wrong:

1. OCR: image or PDF in, raw text out.
2. Extraction: raw text plus a schema in, typed fields out (JSON Extractor).
3. Validation: your code checks the fields against business rules.
4. Routing: clean documents are stored automatically; doubtful ones go to a human queue.

Prepare the image before OCR

The biggest accuracy gains come before any text recognition happens. OCR engines read clean, high-contrast, upright images well and struggle with everything else. A few cheap preprocessing steps make a noticeable difference:

• Resolution: scan or render at 300 DPI. Phone photos usually need no upscaling, but thumbnails do.
• Orientation: rotate and straighten (deskew) pages so lines of text are horizontal.
• Contrast: convert to grayscale and sharpen or threshold faint scans.
• Cropping: remove borders, shadows and backgrounds from photographed documents.
• PDFs: if a PDF already contains selectable text, extract it directly and skip OCR. Only scanned pages need recognition.

Step 1: Get the text with an OCR engine

For a self-hosted option, Tesseract is the standard open-source engine. In Node.js, the tesseract.js package runs it without any system install:

import Tesseract from 'tesseract.js';

/**
 * Returns { text, confidence } for an image path or URL.
 * confidence is the engine's average score from 0 to 100.
 */
export async function readDocument(image) {
  const { data } = await Tesseract.recognize(image, 'eng');
  return { text: data.text.trim(), confidence: data.confidence };
}

In Python the equivalent is pytesseract together with Pillow. Cloud OCR services such as Google Cloud Vision, AWS Textract or Azure Document Intelligence are good choices when you need handwriting, complex tables or very high volume. Whichever you choose, keep the engine behind one small function so you can swap it later without touching the rest of the pipeline.

Step 2: Turn the text into structured fields

The JSON Extractor takes the raw text and a schema that describes the fields you want. Add custom_instructions for formatting rules such as date formats. The standard text limit is 20,000 characters, which covers most single documents:

const SCHEMA = {
  vendor_name: 'string',
  invoice_number: 'string',
  invoice_date: 'string',
  currency: 'string',
  subtotal: 'number',
  tax: 'number',
  total: 'number',
  line_items: [{ description: 'string', quantity: 'number', unit_price: 'number' }],
};

export async function extractInvoice(ocrText) {
  const res = await fetch('https://rsflowhub.com/api/v1/ai/json-extractor', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json', 'X-API-Key': process.env.RSFLOWHUB_KEY },
    body: JSON.stringify({
      text: ocrText,
      schema: SCHEMA,
      custom_instructions: 'Format dates as YYYY-MM-DD. Use null for any field not present in the text.',
    }),
    signal: AbortSignal.timeout(60_000),
  });

  if (!res.ok) throw new Error(`Extraction failed: HTTP ${res.status}`);
  return (await res.json()).data.result; // { extracted_data, schema_conforms, parsed, raw_content }
}

The response contains extracted_data with your fields, plus two useful flags: parsed tells you the model's output was valid JSON, and schema_conforms tells you it matched your schema. Treat either one being false as a failed extraction.

{
  "success": true,
  "data": {
    "result": {
      "extracted_data": {
        "vendor_name": "Northwind Traders",
        "invoice_number": "INV-2041",
        "invoice_date": "2026-09-14",
        "currency": "USD",
        "subtotal": 1190.48,
        "tax": 59.52,
        "total": 1250.0,
        "line_items": [
          { "description": "Standing desk frame", "quantity": 2, "unit_price": 595.24 }
        ]
      },
      "schema_conforms": true,
      "parsed": true
    }
  }
}

Step 3: Validate before you trust it

An extractor can only report what the OCR text contains. If OCR misread a digit, the JSON will be perfectly formatted and still wrong. Validation catches the mistakes that matter by checking relationships the model cannot guarantee:

export function validateInvoice(result, ocrConfidence) {
  const problems = [];
  const d = result.extracted_data ?? {};

  if (!result.parsed || !result.schema_conforms) problems.push('extraction did not match schema');
  if (!d.invoice_number) problems.push('missing invoice number');
  if (!/^\d{4}-\d{2}-\d{2}$/.test(d.invoice_date ?? '')) problems.push('invalid date');
  if (!(d.total > 0)) problems.push('total must be positive');

  // The most valuable check: do the numbers add up?
  const itemsTotal = (d.line_items ?? []).reduce((sum, i) => sum + i.quantity * i.unit_price, 0);
  if (d.subtotal != null && Math.abs(itemsTotal - d.subtotal) > 0.05) problems.push('line items do not add up to subtotal');
  if (d.subtotal != null && d.tax != null && Math.abs(d.subtotal + d.tax - d.total) > 0.05) problems.push('subtotal + tax does not equal total');

  if (ocrConfidence < 80) problems.push(`low OCR confidence (${ocrConfidence.toFixed(0)})`);

  return { ok: problems.length === 0, problems };
}

Arithmetic checks are the best defence against OCR errors: a misread digit almost always breaks the relationship between line items, subtotal, tax and total. Add checks that fit your domain too, such as "vendor is in our approved list" or "invoice date is not in the future".

Step 4: Route doubtful documents to a human

Put it all together and decide what happens next. Clean documents flow straight into your accounting system. Everything else lands in a review queue together with the original image and the list of problems, so the reviewer fixes one field instead of retyping the whole document:

import { readDocument } from './lib/ocr.js';
import { extractInvoice } from './lib/extract-invoice.js';
import { validateInvoice } from './lib/validate-invoice.js';

export async function processDocument(imagePath) {
  const { text, confidence } = await readDocument(imagePath);
  const result = await extractInvoice(text);
  const check = validateInvoice(result, confidence);

  if (check.ok) {
    await saveInvoice(result.extracted_data);
    return { status: 'processed' };
  }

  await addToReviewQueue({ imagePath, text, extracted: result.extracted_data, problems: check.problems });
  return { status: 'needs_review', problems: check.problems };
}

Track the share of documents that need review. In the first weeks it will be high; every time a reviewer corrects a recurring error you can improve the preprocessing, the schema or the instructions, and the rate falls.

Real-world use cases

• Accounts payable: invoice to fields to accounting system, with arithmetic checks and a review queue for exceptions.
• Expense reports: receipt photos to merchant, date, amount and category.
• Logistics: shipping labels and delivery notes to tracking number, recipient and address.
• Onboarding: forms and application documents to the fields an operations team would otherwise retype.
• Archive digitisation: paper records to searchable text, with extraction for the fields you want to filter on.

If the documents contain personal data, mask it before storing or sharing the text. The PII redaction tutorial shows how to add that step.

Accuracy and known limits

• Print beats handwriting: clean printed text is read reliably; handwriting needs a stronger OCR engine and more review.
• Resolution matters: blurry or low-resolution images degrade recognition. Aim for 300 DPI.
• Tables and columns: basic OCR loses layout, so columns can be read out of order. Services built for forms and tables preserve structure better.
• Text size: the extractor accepts up to 20,000 characters per request by default. Split multi-document files before extracting.
• Missing data: instruct the extractor to return null rather than guess, and treat nulls in required fields as review cases.

Common mistakes to avoid

• Skipping validation. Valid JSON is not correct JSON. Check the arithmetic.
• Running OCR on text PDFs. If the PDF has a text layer, read it directly; it is faster and exact.
• Processing multi-page files as one blob. Extract per document, or per page where the layout repeats.
• No audit trail. Store the original image, the OCR text and the extracted JSON so any record can be traced and re-processed.
• Hard-coding a single vendor layout. The advantage of schema-based extraction is that new layouts need no new code. Do not rebuild templates by hand.
• Calling from the browser. Keep the API key on your server.

curl -X POST "https://rsflowhub.com/api/v1/ai/json-extractor" \
  -H "X-API-Key: YOUR_FLOWHUB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text": "Invoice INV-2041 ... Total 1,250.00 USD", "schema": {"invoice_number": "string", "total": "number"}}'

Frequently Asked Questions