Python and PDF: A Review of Existing Tools

by Johannes Filter, Apr 11, 2020 (last update: Aug 5, 2020)

The Portable Document Format (PDF) was invented in the early 1990s and it’s still thriving. But PDFs are mainly for humans – not machines. So it’s often hard to automatically extract information out of PDFs. Besides, more and more functionalities were put into PDF. The complexity of the format makes it hard to get started. But there are existing, well-established software tools. Some have been around for decades. In this blog post, I review some existing tools with the focus on Python. Python is the default language for data processing right now.

PDF Utilities / System Libraries

There are some PDF libraries that power PDF Viewers but can also be used to work on PDFs from the command line. There are general purpose tools that can be used for several tasks.

  • ghostscript: PDF started with ghostscript so this is the mother of all PDF tools.
  • poppler: Used for the vast majority of Linux-based PDF viewers, Libre Office, etc. ‘poppler-utils’ is a collection of tools built upon poppler, e.g. ‘pdftotext’.
  • mutool: Another alternative, lightweight PDF library (there is also a PDF viewer).
  • qpdf: Not a fully-fledged PDF library, mainly to manipulate PDFs.

PDF Toolkits / Python Bindings

  • pymupdf: wrapper around ‘mutool’ (but also extends it in some cases (e.g. PDF EmbeddedFiles)). General purpose tool with a lot of example scripts.
  • pikepdf (recommended): wrapper around ‘qpdf’.
  • pdflib: wrapper around ‘poppler’. This library replaces the need for the specialized packages pdftotext and pdf2image that also use ‘poppler’ underneath. However, it’s hard to setup.
  • pdftk: PDF manipulation. (unmaintained, a maintained Java re-implementation)

Creating & Reading PDFs

Getting Information out of PDFs

  • parsr (recommended): tries to transform PDF into structured data, internally uses ‘pdfminer.six’, ‘camelot’ and more.
  • pdfminer.six: a maintained fork of pdfminer
  • pdfplumber (recommended): works best on digital PDFs, built upon ‘pdfminer.six’
  • py-pdf-parser: another tool built upon ‘pdfminer.six’, includes a simple tool to visualize elements of an PDF document
  • pdfreader: pure Python

Tables

Invoices

  • invoice2data: extract content from invoices with with help of pre-defined templates

General Text Extraction of Files

OCR

Preprocessing Scans

  • Scan Tailor: GUI for post-processing scanned pages (unmaintained).
  • pdfCropMargins: cropping the margins from a PDF
  • unpaper: oldschool image preprocessing, no Python bindings
  • pypillowfight: new fork of unpaper (incl. Python bindings)

Miscellaneous

  • mat2: metadata removal
  • DangerZone: GUI to create safe PDFs, 1. transform PDF into images, 2. transform images into a PDF, 3. OCR (& compress) PDF
  • pdfc: compression
  • pdf-redactor: redaction
  • pdf-scripts: a collection of scripts to work on PDF (I’m the author)

Further Resources

Please write me an email if you think a tool is missing.