Main test sets (used on this site)

English Wikipedia as html files

This set consists of all articles in the English Wikipedia marked as ether featured or good articles, converted into standalone html files.

set

Enron files

With a total of 43 426 files, and a good mix of typical enterprise files like .pdf, .word, .xls, images etc, the Enron file set is a good resource to simulate a file server.

This data set contains files send as email attachments from about 150 users, mostly senior management of Enron. This data was originally made public, and posted to the web, by the US Federal Energy Regulatory Commission during its Enron investigation. The data set was created by extracting all email attachments from the original EDRM set.

set

Additional test sets

Full English Wikipedia as html files

This set consists of all articles in English Wikipedia converted into standalone html files.

Enron email

This data set contains 852 088 email from about 150 users, mostly senior management of Enron. This data was originally made public, and posted to the web, by the US Federal Energy Regulatory Commission during its Enron investigation. This data set was originally released by EDRM .

set

File samples

I am trying to collect as many different file formats as possible to use for testing text extraction.  It is currently a work in progress. If you have files to contribute, please contact me.

Lipsum

Example text in several different languages to test language compatibility. Thanks to lorem-ipsum.info for text.