MPQ Archives

Name breaking

What is name breaking ?

Jump directly to listfile table

In general, MPQ archives don't contain names of archived files. The files are stored by their name's hash value. The algorithm of calculating hash value is one-way only, so it's impossible to get back the original archived file name from the hash value. Some files can be found by monitoring calls of SFileOpenFileEx from the Storm.dll library and logging of file names. But most games contain files never used by the game itself. Names of these files cannot be found everywhere (except of hacking developer's server in Blizzard :-). So how to obtain the complete listfile, which is the one way how to extract all files from an MPQ archive ?

Name breaking is a way, using a brute-force attack method for searching names of files store in MPQs. It generates all possible combinations of a name and tests whether such file does exist in the MPQ archive.

When is the name breaking necessary ?

Since the game of StarCraft, programmers of Blizzard began to include listfiles in the MPQ archives. The listfile's name is "(listfile)". It seems that the battle is won, what is better than listfile obtained from the authors ? If you try to use this internal listfile, you'll notice that the listfile contains less than 10% of all archived files. What is such listfile good for ? If you explore MPQ archives from Diablo II, you'll find that the listfiles are complete. Warcraft III contains complete listfiles too. In the games of Diablo, StarCraft, Brood War and Warcraft II BNE the listfiles are missing or are incomplete. The lists can be downloaded from internet, they they are incomplete too.

Name breaking problems (Bad news)

A name of archived file usually includes directory, filename, (at the most 8 characters) and an extension. Let's think about a case when we know the directory and the extension and look the name only. It can consist of digits (10 combinations), uppercase letters (26 combinations) and the "underscore" character. If the file name length were one character, there is 10+26+1 = 37 total combinations of the file name. At two character length, there can be 37^2 (37*37) = 1369 of combinations. If we think about maximum name length 8 characters, we get 37^8 + 37^7 + 37^6 + 37^5 + 37^4 + 37^3 + 37^2 + 37 = 3,61 * 10^12 (over three trillions) of combinations. It is clear that if the test of one combination take 1 microsecond, the total estimated time to check all possible combinations is more that 1000 hours (41 days), not speaking about different extensions and different directories. It is not possible to ease the work using a dictionary, because the file names ave very often abbreviated. Fortunately, most file names in MPQ archives (where name breaking is necessary) are maximum 8 characters. Newer games (Warcraft III) have longer file names, often over 20-30 characters. Is it impossible to find such file names using brute force attack in geologically short time at all.

Name breaking optimizations (Good news)

There are some optimizations, which ease and speed-up the name breaking

1. It is good to sort the names in the archive by their ordinal numbers in the archive. Because the archives are most often built from locally stored files in one batch, archived files from the same directory and the same file types mostly follow in a line.
2. Some file names differ only by their index (flame1.cel, flame2.cel, ...). This can be used for quick guess of file names ("try to win")
3. File size can tell something too. It is possible to get the size without decrypting the file.
4. MPQ encryption contains a weakness, which can be used for quick detection of decryption key of compressed files. Most of archived files are compressed, except for the files that are compressed from nature, like SMKs or MP3s. So it is possible to extract the file and explore its structure. This allows us to quess the file type and thus, the extension.
5. Like disk folders, MPQ directories often collect files of the same type. For example if all known files within a directory are WAVes, the unknowns will be most probably WAVes too.
6. When checking names, it is possible to pre-calculate hash values for a given directory and then finish the calculation with plain name only. This greatly speeds up the computing of hash values and thus speeds up the complete name breaking process
7. If a certain group of files seem to have the same prefix (e.g. 2 characters), it is possible to include this prefix into the searched directory and search only the remaining 6 characters of the name. This will shorten the necessary time for testing all names to minutes.

Name breaking implementation - The Name Breaker

The Name Breaker is a part of Ladik's MPQ Editor. It allows file viewing before name breaking, saving of current work and supports auto-detection of unknown file types, using weakness in MPQ encryption. Name breaking runs on the background with lower priority, so you can work or play games while name breaker will use the remaining CPU time. For quick access to the name breaker, run MPQ editor with the following parameters:

MPQEditor.exe /namebreaker mpqfile listfile

How much time the name breaking takes

The time necessary to search the whole range of the names depends on the name length. Here are aproximate values, which I measured on machine with CPU Intel Pentium IV, 2.4 GHz with 533 MHz FSB and DDRAM 333 MHz (512 MB):

• Searching 5-character name takes 8 seconds
• Searching 6-character name takes 5 minutes
• Searching 7-character name takes 3 hours
• Searching 8-character name takes 114 hours (4 days 18 hours)

"MPQ@Home"

The aim of this page is to create a project of name breaking, similar to the famous "seti@home". In this page, I will publish the latest MPQ filelists, for the games using MPQ archives. If you want to join me in name breaking, mail me of download the MPQ Editor and some listfiles and try to break some names. Visit the pages of StarcraftFreak, he deals with MPQ listfiles too.

Download the last version of MPQ listfiles

Listfiles

The following table contains overview of number of unknown files in MPQ archives. You can download the most recent version of the listfiles in the Download section. Thanks to people who participated on building listfiles:

• Thalick (Special thanks for very hard work !!!)
• StarcraftFreak
• Uzume
• Christopher Ott
• Mystery
• Valery Anisimovsky
• DW and Ojan Pojan

Copyright (c) Ladislav Zezula 2003 - 2013