Extract budget data from PDFs: a reviewable workflow

Begin a budget extraction project with one document, one table and a written definition of the rows you need. Keep the original PDF alongside the extracted data so a reviewer can trace an amount back to its source. The workflow below is a proposed implementation plan, not a claim that a particular government report mandates these tools or that an extraction has already been tested.

Inspect the document before choosing a route

Try selecting text on representative pages, then compare the extracted characters with the visible headings and amounts. Check more than the cover page. Record which tables have usable text, which need optical character recognition, and which need manual transcription or a different source file.

The pdfplumber documentation says the library works best with machine-generated PDFs. It does not provide OCR and does not offer strong support for table extraction from OCR-processed documents. For scanned pages, therefore, plan a separate OCR or transcription step and a review of its output before attempting the same extraction procedure.

Extract one table and inspect its boundaries

For a suitable text-based page, pdfplumber provides table extraction and visual debugging tools. Its table documentation distinguishes extract_table(), which selects the largest table, from extract_tables(), which returns all detected tables on a page.

Choose deliberately when a page contains several tables. Inspect the detected boundaries, headings and row alignment against the PDF. Save the selected page area, extraction settings and library version with your working files. Repeat this check on a page with a different layout before applying the settings to the rest of the document.

Define a reviewable budget row

As a starting design, keep these fields in your working dataset:

• Source document reference, page number and original row label.
• Reporting organization, budget period and program or account code.
• Measure, such as proposed budget, revised budget or reported spending.
• Original amount text, parsed amount, currency and stated unit scale.
• Review status and a note for unresolved interpretation.

Adapt that list to the document. Do not assign a category solely from a neighboring row. If a heading spans several columns, write down how you apply it to each extracted value. Preserve the original text when interpreting separators, negative values or amounts stated in thousands.

Validate structure without inventing values

The Frictionless Table Schema specification describes field names, types and validation constraints. It also provides a missingValues setting for strings treated as null. A primary key can consist of one field or a combination of fields.

Use an explicit schema to check the dataset you intend to publish. Choose a row identifier that distinguishes the measure and period as well as the account. Document the meaning of an empty cell or dash before mapping it to null; do not silently turn it into zero. Put ambiguous rows in a review file with their source references.

Reconcile against the source

For each reviewed table, compare the extracted rows with its printed subtotal and total. First confirm that you are comparing the same period, unit scale and measure. Keep subtotal rows separate from detail rows when calculating a check, and record any exclusions.

Treat a mismatch as a reason to inspect the mapping. Do not change a value just to force agreement. Likewise, treat a matching total as one check in the review, alongside row labels, missing lines and column alignment. Define any rounding tolerance before using it and retain the difference in the validation report.

Publish the data with its review record

Release the reviewed CSV or JSON together with its schema, source references, extraction instructions and known limitations. Record the source edition and the date you retrieved it. When a new edition arrives, preserve the previous dataset and review the changes before replacing the published version. Make unresolved rows visible in the accompanying notes rather than presenting them as verified amounts.