How to copy text from a photo or screenshot with OCR

OCR turns a picture of text into text you can copy and edit. The result can be near perfect or a mess, depending on the image. This guide explains what makes the difference and how to check the output before you rely on it.

Updated

How OCR reads an image

An OCR engine such as Tesseract, which the swift.id tool uses, first finds the lines of text in the image. A neural network trained on thousands of printed typefaces then reads each line as a sequence of characters, with a confidence score. The language data you choose helps decide which letters and words are likely. So two things matter most: the quality of the image, and whether the language setting matches what is in it.

What affects accuracy

ProblemWhat you seeFix
Text too smallSwapped lettersShoot closer
Page at an angleLines mergeShoot straight
Uneven lightParts missingEven light
Light text on darkMany misreadsLight mode
Photo of a screenBroken lettersScreenshot
HandwritingMostly wrongType it

The Tesseract documentation says it works best on images of at least 300 DPI, that line finding gets much worse when a page is too skewed, and that a background of uneven darkness gives poorer results. For Tesseract 4 and later it recommends dark text on a light background, so retake a dark-mode screenshot after switching the app to light mode.

The swift.id tool enlarges images under 1 megapixel by up to 3 times before reading them, because very small letters are hard to separate. Enlarging only goes so far: letters that are blurred in the original don't become sharp.

Photographing a document for OCR

  1. Put the page on a flat surface in even light, near a window for instance. Turn the flash off; its glare wipes out the letters in the middle of the page.
  2. Hold the phone parallel to the paper, so all four edges of the page look straight and the page nearly fills the screen. A 12-megapixel photo filled by a Letter or A4 page works out to roughly 340 to 360 DPI; if the page fills only half the frame's width, that drops to about 170 DPI.
  3. Tap the text on screen so the camera focuses there, and wait until the image is sharp.
  4. If your phone has a document scanner, use it: Scan Documents in the iPhone Notes app or Scan in Google Drive straighten the perspective and crop to the page.
  5. For text on a computer or phone screen, don't photograph the screen. Take a screenshot instead; it has no glare and no stripes from the screen's pixel grid.

Languages

The tool reads English, Indonesian, Spanish, Portuguese, French and German, and can use two of them at once. On the English page it starts with English only, with Second language set to None. Choose the languages that are actually in the image; a second language you don't need makes reading slower. If you change the language after an image has been read, a Read again in … button appears to re-read everything. Non-Latin scripts such as Arabic, Chinese or Cyrillic are not available.

Handwriting and tables

The Tesseract FAQ is blunt: you can use it for handwriting, but it won't work very well, because it is designed for printed text. Very neat notes are sometimes partly readable; the rest is quicker to type.

Tables are hard too. The Tesseract documentation acknowledges a known problem with reading data from tables, and the tool's output is plain text, so cells from different columns can end up on one line. For a table, crop the image (or take a screenshot) one column at a time, read each, and paste them into the matching spreadsheet columns. That also makes the numbers easier to check.

Proofreading the result

OCR output always needs a second look, especially numbers. The pairs most often swapped are rn and m, the letters l or I and the digit 1, the letter O and the digit 0, and S and 5. Words split by a hyphen at the end of a line, such as "infor-mation", need joining back up.

  • Look at the confidence figure. Every image gets a confidence percentage. Below 60%, the tool adds a warning to check the result carefully.
  • Compare two readings. Photograph the same page twice from slightly different positions, read both, and paste the results into the Diff Checker. The lines that differ are where the OCR was unsure, so those are the only lines you need to check against the image.
  • Count the length. Paste an account number into the Word & Character Counter: a UK IBAN, for example, is always 22 characters. Word counts per page also reveal a paragraph that went missing.
  • Run a spell checker. The result box deliberately has the browser's spell check off. Paste the text into a word processor and unknown words get flagged.

Extracting text on swift.id

  1. Open Image to Text (OCR). Choose or drop one or more images, or paste a screenshot with Ctrl+V.
  2. Check the Text language and Second language. The first use downloads the OCR engine and language data; your images are not uploaded.
  3. Wait for the percentage under the file name to finish. If an image is sideways or upside down, press the Rotate 90° button and it is read again.
  4. Correct the text in the result box, then press Copy or Download .txt. For several pages, Copy all joins the results in list order with a blank line between them.

The tool doesn't accept PDFs yet. For a scanned PDF, take a screenshot of each page. If you can select the text in your PDF reader, copy it from there instead; that is always more accurate than OCR.

Questions

Are my photos uploaded?
No. Only the OCR engine and its language data are downloaded from swift.id; the images are read on your device. Still check every number in the result before using it on an official form.
Why is the text from my dark-mode screenshot garbled?
The Tesseract documentation recommends dark text on a light background for version 4 and later. Switch the app to light mode, take the screenshot again, and read it once more.
Can it read iPhone photos (HEIC)?
It depends on the browser. Chrome can't open HEIC, and the tool will tell you so. Convert the photo with HEIC to JPG first, then add the JPG.
How accurate is it?
For a clean screenshot of printed text, usually almost error-free. For photos of creased paper, small print or tables, misreads are far more common. A confidence figure below 60% is the signal to check every line.