Survey of ZIP APPNOTE documents

[2019-09-21: I’ve edited this document to include some new discoveries.]

I already wrote an article about the version history of the PKZIP software, so now it’s time for one about the version history of the ZIP format. That sounds like it should be easy, but… not so much.

I think the format versions are more important than the software versions. Many ZIP files that use the old format versions are still around, while nobody uses the old software versions.

This research is not as complete as it could be, but I’ve spent too much time on it already, so I’m writing up what I know.

Preliminary notes

The ZIP format documentation is usually found in a file named APPNOTE.TXT. For convenience, I’ll use “APPNOTE” to refer to it, regardless of the actual filename. The APPNOTE.TXT filename is not actually specific to ZIP, as it was also used by the predecessor PKARC/PKPAK software (for a different format).

These are plain text files, and I encountered many of the typical problems that can make it difficult to compare two text files: different line ending codes, different amounts of indentation, different word wrapping, etc.

The version number landscape

Very generally, there are at least three different kinds of versions that might be relevant:

  1. The version of the file format.
  2. The version of the software associated with the format.
  3. The version of the specifications document that defines the format.

With ZIP/PKZIP, these concepts are conflated in confusing ways.

Inside a ZIP file, every member file has two version numbers stored with it: A “version made by” number, and a “version needed to extract” number. These version numbers have a strict format: an integer major version number, and a minor version number that can only be 0 through 9. Later versions of APPNOTE have three-component version numbers like “6.3.5”, though there is no standard way to encode the “.5” part in a ZIP file.

These in-file version numbers were originally documented as being software version numbers. Later documentation (as of APPNOTE-6.2.0) says they are specification version numbers.

APPNOTE versions from about 6.1.0 on include a nice table of the “version needed to extract” for various features.

Only APPNOTES from about v4.0 on tell us their specification version number. Two older APPNOTEs were retroactively assigned version numbers: 1.0 and 2.0.

Note that a version number like “4.5.0” is the same as “4.5”. The final “.0” may or may not appear.

APPNOTE eras and categories

Era 1

Each version of the PKZIP software through 1.93 includes documentation of the file format, usually in a file named APPNOTE.TXT. These documents do not have any dates or meaningful version numbers in them.

Basically, through v1.10, there is no distinction between the format version, the software version, and the APPNOTE version. (v1.93 writes format version 2.0.)

Era 2

With the release of PKZIP 2.04c in late 1992, we enter a murky era in which APPNOTEs were de-coupled from the software, but still lack a version number. They can generally be assigned a release date, but it’s hard to be sure you’ve got it right.

There could be any number of APPNOTEs from this era whose existence I’m not aware of.

It’s not clear to me how these APPNOTEs were distributed. It wouldn’t have just been via a website, because the web was a new thing at the beginning of this era.

This era apparently lasted until the release of APPNOTE v4.0, dated 2000-11-01.

Era 3

Starting with APPNOTE v4.0, APPNOTEs have a clear version number and a date.

Starting with v6.2.0, they even include a change log (yay!), listing the previous versions back to 5.2. So we can be pretty sure we know all the official releases back to 5.2.

At the time of this writing, some, but not all, of these APPNOTEs are available at the PKWARE website.

Info-Zip APPNOTEs

To add to the confusion, the makers of the independent Info-ZIP software released a series of unofficial APPNOTEs of their own, usually in a file named “appnote.iz”. These are based on the official PKWARE documents, edited with corrections and extensions.

There are eight Info-ZIP APPNOTEs that I know of. As of this writing, all are available from the Info-ZIP FTP site.

Drafts

I’m including in my list the not-really-official documents from PKZIP 0.8.0-beta, and 1.9.3-alpha.

I know that some other publicly-visible draft versions have existed, but I’m not very interested in such documents, and anyway it would probably be next to impossible to research them.

Known APPNOTE versions

Since there’s no consistent naming or numbering convention for APPNOTE versions, I’m inventing some of my own. An “AN-” prefix indicates a PKWARE version number or release date. “IZ-” is for Info-ZIP versions. Filenames of the form “appnote-YYYYMMDD-XX.zip” were invented by Info-ZIP, though some contain a PKWARE APPNOTE.

I’m listing the file size, in bytes, of most APPNOTEs, which may help to figure out which one is which. Keep in mind that any amount of reformatting or re-saving is likely to change the file size.

AN-1989-01-10, PKZIP 0.80-beta, size=7395+5382. The information was divided into two files: FORMAT.DOC and EXTRACT.DOC.

AN-1989-02-10, PKZIP 0.90, PKZIP 0.92, size=13715. First ZIP use of the APPNOTE.TXT filename. The APPNOTE from PKZIP 0.90 was unchanged in 0.92.

AN-1989-07-21, PKZIP 1.01, PKZIP 1.02, size=25662. The APPNOTE from PKZIP 1.01 was unchanged in 1.02.

AN-1990-03-15, PKZIP 1.10, APPNOTE-1.0.txt, AN-1.0, size=25811 or 25863. The APPNOTE-1.0.txt variant has a copyright message added, but no other changes.

AN-1991-10-15, PKZIP 1.93-alpha, size=32227. This is an unpolished draft. It might be the first ever documentation of the Deflate compression scheme, which it calls “method 8”.

AN-1993-01-25, APPNOTE-2.0.txt, AN-2.0, size=37084 or 37141. Sometimes found inside a file named APPNOTE.ZIP. Has the same release date as PKZIP 2.04e. The date 1993-02-01 is also claimed for this APPNOTE. One variant changes the capitalization of “huffman”. The APPNOTE-2.0.txt variant also has a copyright message added.

AN-1996-02-15, size=39585(?). All Info-ZIP APPNOTEs claim to be ultimately based on this one, but I have not found a copy of it. But, if IZ-1996-08-01 is to be trusted, I can reconstruct it by reversing its patch. It turns out to be only a little different from AN-1996-09-15.

IZ-1996-08-01, appnote-19960801-iz.zip, size=38709(LF) or 39773(CRLF). The distribution file contains files named README and appnote.txt. The README file has a patch to convert AN-1996-02-15 to this one.

AN-1996-09-15, appnote-19960915-pk.zip, size=42321.

IZ-1997-03-11, appnote-19970311-iz.zip, size=58769(LF) or 60405(CRLF). This and later Info-ZIP APPNOTEs use the filename “appnote.iz”.

AN-1997-05-01, appnote-19970531-pk.zip, size=43503. The “31” part of the filename assigned by Info-ZIP is possibly a clerical error.

IZ-1998-04-27, appnote-19980427-iz.zip, size=72103(LF) or 73816(CRLF). There are discrepancies as to the exact date; it could also be 04-21 or 04-26. But I think it’s all the same version. Based in part on AN-1997-05-01.

AN-1998-09-01. IZ-1998-11-19 claims to be based in part on this version, but I could not locate a copy of it.

IZ-1998-11-15, appnote-19981115-iz.zip, size=79342(LF) or 81202(CRLF). Based in part on AN-1997-05-01.

IZ-1998-11-19, appnote-19981119-iz.zip, size=81733(LF) or 83657(CRLF). Based in part on AN-1998-09-01.

AN-2000-11-01, AN-4.0.0, size=55179.

AN-2001-11-01, APPNOTE-4.5.0.txt, AN-4.5.0, size=59859.

IZ-2001-12-03, appnote-20011203-iz.zip, size=104575(LF) or 107045(CRLF). Based in part on AN-4.5.0.

AN-2003-06-02, AN-5.2.0, size=74993. There are two “version 5.2” APPNOTEs, which I consider to be different enough to list separately.

AN-2003-07-16, APPNOTE-5.2.0.txt, AN-5.2.0, size=75111. Technically identical to AN-2003-06-02, but has an additional notice about the encryption being patent-pending. Several of the later APPNOTEs come in two varieties, differing only in the date they report for v5.2 (either “06/02/2003” or “07/16/2003”).

AN-2004-01-20, AN-6.1.0. The only copy I could find of this version is from the Wayback Machine, and it’s embedded in an HTML document, so it’s not pristine.

AN-2004-04-26, APPNOTE-6.2.0.txt, AN-6.2.0, size=117686.

IZ-2004-05-28, appnote-20040528-iz.zip, size=159819(LF) or 163445(CRLF). Based in part on AN-6.2.0.

IZ-2004-09-08, appnote-20040908-iz.zip, size=166297(CRLF). Based in part on AN-6.2.0. Due to an apparent clerical error, claims to be “20040528”, the previous version.

AN-2005-04-01, APPNOTE-6.2.1.TXT, AN-6.2.1, size=109785.

AN-2006-01-06, APPNOTE-6.2.2.TXT, AN-6.2.2, size=112901.

AN-2006-09-29, APPNOTE-6.3.0.txt, AN-6.3.0, size=136852.

AN-2007-04-11, APPNOTE-6.3.1.TXT, AN-6.3.1, size=136949.

AN-2007-09-28, APPNOTE-6.3.2.TXT, AN-6.3.2, size=145310.

AN-2012-09-01, APPNOTE-6.3.3.txt, AN-6.3.3, size=161412.

AN-2014-10-01, APPNOTE-6.3.4.txt, AN-6.3.4, size=161480.

AN-2018-11-31, APPNOTE-6.3.5.txt, AN-6.3.5, size=171222.

AN-2019-04-26, APPNOTE-6.3.6.txt, AN-6.3.6, size=171326.

Of course, there could be more versions after 6.3.6 by the time you read this.

References

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s