For security purposes, you sometimes want a dictionary with possible passwords. Perhaps you want to assess password strength in an in-house built application, or you want to test the security of the WPA wireless network.
There are plenty of dictionaries around to use to warn (or exclude) users from creating an account with such a password, or for the latter purpose, to brute force the WPA key.
After the standard dictionaries, a good next step is to create a password list for specific circumstances. For example; you are building an app for a company and want to make sure that easy passwords are excluded. You then need to build a list of passwords that any cracker would try.
There are good commercial products that can do this kind of stuff automatically, but for the poor man, it's easy enough to just get the stuff from the company website, or the Wikipedia article on them. With standard Unix/Linux tools, you're good to go.
I assume we create a directory in your home directory called "crack". First, use wget to get the source of words that shouldn't be a password:
$ mkdir ~/crack $ cd ~/crack $ wget "http://nl.wikipedia.org/wiki/Rotterdam" -O wordlist.html
Then use sed to strip out the HTML:
$ cat wordlist.html | sed -e :a -e 's/<[^>]*>//g;/</N;//ba' > wordlist.txt
Then edit the list with vi, and manually remove some JavaScript cruft at the bottom if necessary, then clean it up a bit.
First put each word on a separate line. The ^M indicates the Enter key. Create it by pressing CTRL+V, then pressing enter.
:%s/ */^M/g
Remove extra whitespace:
:%s/ *//g
Remove empty lines:
:g/^$/d
Done. Now we're going to expand the wordlist with all sorts of word play that people might think of, like using years, or numbers, or using leet speak. John the Ripper by default has a nice set of mangling rules, but it's very limited.
Matt Weir has greatly enhanced the default set of mangling rules, and that's the one we'll be using.
Make sure you have a recent version of John the Ripper, because Matt's rules need it. Your Linux distribution will probably have an older version, so download and compile version 1.7.8 or something later. I'm assuming you'll put John in your home directory in opt, then add it to your path:
$ mkdir ~/opt $ cd ~/opt $ wget http://www.openwall.com/john/g/john-1.7.8.tar.gz $ tar xfz john-1.7.8.tar.gz $ cd john-1.7.8/src $ make
(make now lists a number of targets, choose the best for your architecture and re-run make)
$ make linux-x86-64
When finished, the executable and configuration file reside in $HOME/opt/john-1.7.8/run. Download Matt Weir's configuration file to this directory, but back up the original configuration first:
$ cd ~/opt/john-1.7.8/run $ mv john.conf john.conf.bak $ wget http://sites.google.com/site/reusablesec/Home/john-the-ripper-files/john-the-ripper-sample-configs-1/john.conf?attredirects=0&d=1 -O john.conf
Since John the Ripper is used by us in WordList mode, we need to edit the configuration file so it uses the correct rules. Edit john.conf, and search for the line
[List.rules:Wordlist]
(around line 335) and disable it by adding a few characters, for example
[List.rules:WordlistXYZ]
Then find the following line:
[List.rules:Modified_Single]
(around line 21), and change it to:
[List.rules:Wordlist]
Now add John the Ripper to your path, by adding the following line to your ~/.bashrc:
export PATH=$HOME/opt/john-1.7.8/run
Logout, and login again. Typing 'john' should print the version. Now mangle your newly tested word list. To make sure the new mangling rules are used, create a file called test.txt with only one line, for example 'rotterdam'. Then run john. It should look something like this:
$ cd ~/crack $ vi test.txt $ john -w=test.txt --stdout --rules rotterdam rotter Rotterdam rotterdamrotterdam ROTTERDAM rotte rotterdam1 rotterdam2 rotterdam3 rotterdam4
(skipping a whole lot of lines)
ROTtERDAm ROTTerdAm ROTTerDAm ROTTeRdAm ROTTeRDAm ROTTErdAm ROTTErDAm ROTTERdAm ROTTERDAm RoTTeRDaM Tpyyrtfs, Eirrwesan words: 4745 time: 0:00:00:00 100% w/s: 39541 current: Eirrwesan $
As you can see, Matt's rules generate all sorts of permutations which you can then use in your software or brute forcing efforts. From one word, 4745 mutations have been created! If you see less (for example between 20 and 40), then the default rules have been used and there is something wrong with the configuration file. Perhaps john couldn't find it?
If this went okay, then re-run john on your custom wordlist:
$ john -w=wordlist.txt --stdout --rules > wordlist_mangled.txt
Voilà, this resulting word list can now be used in your password strength assessment, or brute forcing efforts.