It all started with this cartoon:

This cartoon basically started a hype about how XKCD was getting “it”. Jason posted a blog post stating that he did not agree with XKCD since:

• While four words in theory have 44 bits of entropy (2⁴⁴), it is actually 250,000 to the power of 4 (250,000⁴) since English only has 4about 250,000 words
• Most people actually would use three words, giving 15,625,000,000,000,000 combinations
• Most people know even less then 250,000 words

So what is my take on this? The key to “it” is at the bottom of the cartoon:

“Through 20 years of effort, we’ve successfully trained everyone to use passwords that are hard for humans to remember, but easy for computers to guess”

This is really the “it” XKCD does get.

So why do we use password policies in the first place? What problem are we trying to tackle?

First of all we are trying to tackle the problem that users are very bad a picking good password without guidance. This tweet illustrates that:

If you don’t give users guidance they will often pick from a set of very well known passwords. But more recent research shows that since the average person has over 50 passwords, some with and some without password policy on it, most people need a coping strategy to deal with this.

In my talk “The Road to Hell is paved with best practices” I give this example of likely passwords for a certain password policy:

• 7 characters: welcome
• 7 characters + 1 capital: Welcome
• 7 characters + 1 capital + 1 numeral: W3lc0m3
• 7 characters + 1 capital + 1 numeral + 1 special: W3lc0m3!
• 10 characters + 1 capital + 1 numeral + 1 special: W3lc0m3!!!
• 10 characters + 1 capital + 1 numeral + 1 special, 30 days max, cannot reuse last 12: Welcome01!, Welcome02!, Welcome03!, etc

As security people we need to understand that each security measure will alter peoples behaviour and sometimes not for the good.

Studies have shown that even if password policies are used, probabilistic techniques can be used to aid in password cracking attacks, that password expiry is only of limited use, that password expiry policies do not meet their goal.

Experiments with an online windows password cracker showed that “hard” passwords do not take longer to crack that “easy” passwords when rainbow tables are used:

• Empty password – 2 seconds
• 72@Fee4S@mura! – 5 seconds
• (689!!!<>”QTHp – 8 seconds
• *mZ?9%^jS743:! – 5 seconds
• T&p/E$v-O6,1@} – 11 seconds

So what is my opinion?

Security policies have driven people to the top of their ability to remember passwords and as users have got increasing amounts of passwords the behavior it induced did not improve matters. We need to tune some of these measures down and replace them with education.

Passwords should be:

• Relatively long
• Not guessable (correcthorsebatterystaple is not o.k. anymore thanks to XKCD)
• Your system should block guessing attempts or really slow them down

If hackers have you password hashes you are toast…

Passwords are like Pants... a Creative Commons Attribution, Noncommercial, No Derivative Works image from Richard Parmiter's Flickr fotostream

One of the DefCon contests that most sparked my imagination was the “Crack me if you can” password cracking contest, organized by KoreLogic. The goal of the contest is to crack as many of the password hashes provided as possible. The rules of the contest allow the use of off-site and on-site computer equipment of any kind, but in order to be eligible for any prize money at least one team member had to be physically present at the DefCon conference.

The competition is interesting in more than one way. First of all the contest is educational in setup. Even though the amount of computer power a team can come up with is important in getting to good results, it is not the determining factor in winning or losing the contest. Key to winning or doing well in the contest was understanding human behavior. KoreLogic generated a set of passwords they feel is representative of what they actually encounter in the field. Most corporate environments rely on a common set of rules that are used to enforce user to pick “strong” passwords and force them to change them regularly. While the goal of the rules is actually commendable, KoreLogic’s experience learns them that the human behavior triggered by these rules cause passwords to be very predictable. “If you force employees to change their passwords four times a year, they will select something that naturally changes four times in most cities (except Las Vegas)”, typical passwords we find are things like Winter2010. Once you understand this pattern, you can actually reliably predict what this password will in say 9 months or a year. Teams that actually saw this pattern and used it to make smarter password guesses did better in the competition.

The key to making hard to guess passwords is to break with this predicable behavior. If people have to put a special character in their passwords they usually put them in the beginning or at the end of their password, e.g. Summer1969! We had a number of passwords that actually had a password in the middle of it and these passwords where significantly harder to crack.

There is a significant difference between the success rates of cracking certain password hashes. E.g. windows password hashes have proven at be extremely easy to crack. All the teams together cracked 94% of all the windows password hashes provided to them. These contain some LM hashes, but mostly NTLM and NTLM2 hashes. A stupid 20 character long Windows administrator password (2345678901234567890) was guessed by all teams, even though there are no rainbow tables available for passwords of this length . Operating systems like FreeBSD do much better, less than ten of these hashes where cracked and BCrypt hashes achieved an even better success rate, only a few hashes where cracked. Absolute winner where the Oracle password hashes, none of these where cracked.

While this was a serious competition and the first prize of $600 was won by team HashCat, the competition was mostly educational in its setup. Only teams that published their methods for cracking are eligible to win and all results and methods used will be published online later this week (@@@@). The contestants used an interesting array of computer equipment. Graphics Cards based systems, clustered Amazon EC2 instances and a university super computer cluster with 1TB of memory where all used as well as plain simple desktop computers.
Hopefully this competition will not only learn us how to better crack passwords, but also how to pick better passwords and thus make us all a little bit more secure.

Update: Matt’s blog, Slide deck, Sebastien Raveau’s word list (1, 2)

There are basically two types of password cracking, Online by trying usernames and passwords directly in the login screen. This only gives you a few tries since the system and its countermeasures is still opertional.
Offline, by trying to match passwords against password hashes, mostly for forensic reasons.

Basic cracking process:
1.    Generate a lot of password guesses
2.    Generate hashes for these passwords
3.    Compare the hashes against the target hashes
4.    If you don’t have enough, redo from 1

Matt at first did most of his research work from home, not going into the lab, but after his first power bill arrived which was increased by 75% he used a 3 year old Dell in his lab.

If you want to investigate how human generate passwords, you have to have a list of user generated passwords, but where can you get them? (Un)fortunately hackers helped us out and there is such a list available.
The hacker who compromised Phpbb.com website published a list of 259k unsalted md5’s and 83k salted hashes. Because of time limitations, Matt only attacked the unsalted MD5 hashes. The hacker himself submitted 117k hashes to an online password cracker. 24% of the passwords were cracked (28,635).

He used an online cracker called hashkiller.com. They do a good job and track how efficient they are and how efficient other crackers are. There is also md5-utils which is a site that submits hashes to other sites as well. However if you are going to use such a system and think that the owners are not going to keep a copy of the found passwords you are too naïve to work in security.

There are also crackers you run on your own machine. The best one is John the ripper. It is free and open source. It is actively maintained and has an active community. If you can think of a problem that might occur, it is usually already covered in John the Ripper. As an added bonus John the Ripper take its password guesses from standard input.

Usign John the Ripper Matt was able to get the following results:
4 hours – 38% cracked.
1 week  – 62% cracked.
1 month and 1 week – 89% cracked.
Currently – 95% cracked.

Some quick password statistics:
Average length of a password: 7.2 characters long
Only 6% of the passwords contained an upper case character.
Only 1% of the passwords contained a special character.
51% of the passwords consist only of lower case letters.

So where are you going to take your passwords from? There are good word files out there. Large word lists are good if the system does not enforce any password policy.

Sabastien Raveau has created an excellent word list by getting all words from all Wikipedia and related projects articles. But you can also used John the Rippers generator which is based on linguistic probability.

If a system enforces password policies, you are better of with smaller more specific word lists preferably one which is based on previously cracked hashes. Unfortunately Matt cannot share his results due to privacy implications.

There are ways to speed up the brute force process, by using Probabilistic Cracking
Certain Words more often used, e.g. password, monkey and football are very common. Also certain mangling principles are more popular then others: appending 123, 007or  $$$, capitalizing the first character, replacing the o by 0, etc.

Matt program takes words and mangling rules and assigns a weight to them. Then it starts with the most likely combinations.

Matt then shows a demonstration which clearly shows that weak passwords get cracked first.

Matt strongly believes that forcing frequent password changes does more harm then good. Humans are clearly not good at generating truly random passwords and if you let them do it often you only decrease anthropy.

Matt also indicated that salting passwords (adding a random string to the password before hasing) greatly increases the amount of effort required to brute force password hashes. It means that every hash has to be tried with every salt. But, salting only works if multiple passwords are decoded, it does not make a single hash more secure.

I think we have all seen them, the annoying pop-up messages or scrolling text before you log onto a system telling you that it is an offence to log on unless you are authorized, etc, etc. If you do not know what I am talking about, here is the MS knowledge base article on how to set it up on a windows box.

The debate was around the question if such disclaimers actually add any security to the system. In order to answer that question we need to understand the origin of the disclaimer a little better. Apparently there has been a a court case in which an hacker who was changed with breaking into a computer system by guessing the administrator password successfully defend himself by stating that when he first opened the system he was asked to “Please enter your username and password”. When he entered his username and password he got a message stating “invalid username or password, please try again”. So he was not trying to break into the system, but just doing what he was requested.

This little story makes me wonder, would this excuse for hacking still fly today. As I am not a lawyer (or have any intentions of becoming one) I am in not a position to give an authoritive answer, but I am going to make a guess based on what I do know about Dutch law. In order for someone to be found guilty of trespassing (either IRL or on a computer) you must prove that the person entered an “area” that he was not allowed to enter and that he knew was restricted. In other words if you just happen to wonder into an restricted area, but you were unable to know that you should not go there it is not trespassing. However if you did jump a fence in the progress, you would be hard put to state that you were not aware. In the case of a computer it would be my opinion that having to enter a username and password should be sufficient reason for you to know that the system is restricted.

In my opinion this measure is one of those measures that we take out of sheer inertia, or to keep up with the Joneses, just like changing your password every month or putting a disclaimer on the bottom of an email.

Like the disclaimer, the monthly password change has a historical origin. Rumor has it that the “industry standard” monthly password change is derived from a calculation of how long it would take to perform brute force password attack on an old mainframe. Based on the outcome of this calculation (two months) changing  your password every month very effectively reduces the risk of your password being cracked. However, the basic assumptions on which this habit is based have changed dramatically. For example due to Moore’s law and Rainbow tables.

Does sticking to these out-dated practices hurt? On the one hand these measures are cheap to implement. It only takes some changes to the registry, group policy or a text file. On the other hand they can be counterproductive. The disclaimer can cause annoyance when you have to click it away multiple times a day and will certainly not be read every time it is displayed.

The once a month password change is worse, because it encourages bad password practices like writing passwords down or using numbered increments. (Password03, Password04, etc)

Better alternatives like awareness trainings and dual factor authentication are available.

I would like to hear your thoughts on the matter fbreedijk (at) schubergphilis (dot) com