Statistics

Some basic statistics:

Total of 1631 LOI’s were uploaded.
Of those, 1574 are unique. This was tested by looking for a very very high degree of matching between documents.
Some LOI’s seem to be very close to others - perhaps updates. If you exclude those, then we have 1560 unique documents.
This batch of LOI’s were scanned from the main site on September 16, 2020.

Calculating similarity

A word-vector for each document is made. From that, simple vector distance between documents is calculated. The closer the vector distance, the more common words are used. When documents are identical, of course, their vector distance is one. If you calculate the 2x2 matrix for the similarity between all documents (and ignore the diagonal), you get the following:

Document Similarity

Everything to the right of the duplicates line is classified as being identical below. And everything to the right of the updates line is classified as being an update.

Duplicates

These LOI’s are duplicates of each other.

Updates

These LOI’s might be updates of each other