Common pitfalls when following a responsible disclosure policyAndreas Thienemann
While we try to fix these things every time, sometimes things slip through.
We want to tackle security issues on our infrastructure, but we might miss things ourselves. It’s important to have more eyes looking for issues, which is why we have implemented a responsible disclosure policy. If a security researcher finds a vulnerability in our systems, we’d like to hear about these things and fix them. But we’d also like to give credit to the security researcher(s) and present them with a small token of our appreciation for the time spent on helping us to improve our infrastructure.
So far, our security bounty program has a been a great success: we’ve received a lot of security reports from a number of smart security researchers.
After having run the program for a while, however, we have noticed that there are some reports about potential security vulnerabilities coming in which we cannot consider valid security findings.
In this article, we’d like to share the most common cases of security reports we’re considering invalid reports. We are doing this both to save security researchers from spending time on invalid findings, but also to share some basic rules and discoveries we’ve made running our own responsible disclosure program.
Not all login screens are created equalA simple approach to judge the security of a web-application is to have a look and see if its administrative interface is accessible and possibly even some default credentials are enabled. On our infrastructure, we’ve gone through the effort of identifying these attack vectors and have either disabled them or we have added a two-factor authentication system to it, increasing the complexity of a potential attack by a significant margin. Despite these precautions, we have received reports claiming that we have a vulnerable Wordpress administration interface available at http://www.schubergphilis.com/admin/.
The generic WordPress authentication screen
The Schuberg Philis Two-Factor Authentication Portal
Comparing the design of the generic WordPress login screen with the one visible on our infrastructure shows that a stronger authentication than just a username and a password is required. At this point, it should be clear that a general report about a WordPress Admin screen being available to the public is not going to be considered a valid security report.
Absence of proof is not proof of absenceA strong authentication system is great at protecting some underlying resources, but said protection might not amount to much if passwords can easily be brute-forced or circumvented through simple and guessable password hints or password-recovery questions. To prevent an adversary from succeeding at attacking the authentication system itself, features such as captchas, slowdowns and lockouts are often added by developers.
The absence of appropriate feedback confirming these features is not a proof of their absence. An account might very well be locked out temporarily after three incorrect passwords have been tried without the user receiving feedback. In our case, both regular passwords as well as two factor authentication portals only allow a certain number of incorrect attempts before locking down the account. As such, any report about authentication vulnerabilities should explain in detail why it is applicable beyond a lack of instantly visible lockout, etc. security features.
The early bird catches the wormWaiting and seeing what others do can be useful. It allows learning from the mistakes others made and one can improve on their results.
But sometimes, it’s more important to be first. From time to time we do see duplicate reports for the same vulnerability. This is especially very common after a new 0day exploit has been published. We’ll only be able to consider the first report for a vulnerability. If a researcher has a question about whether a vulnerability has already been submitted, we’ll let you know when you file your report.
Secure configuration is hard, but not all configuration mistakes are security relevantAmongst the relevant security findings we have received, there were a few reports about obvious configuration mistakes. Some of these mistakes had a definite security impact and needed fixing. In some cases, however, the impact wasn't that clear. If a configuration problem or a software bug requires a specially malformed request to be triggered, we are very much interested in hearing about it and fixing the issue in order to improve the resiliency of our infrastructure. But even then we might not always be in the position to actually consider this a valid security finding. For this, it needs to be actually exploitable.
Nevertheless if a researcher is unsure about the impact of a potential vulnerability or cannot provide a working Proof of Concept, we’re still very happy to hear about the issue at hand. Even a preliminary report is welcome and we’ll have a close look to see if it is a security finding or not.
Words are silver, a working PoC is goldHaving found an issue the next step is to tell us about it.
For simple issues such as a trivial cross site scripting vulnerability a screenshot showing the URL might be enough. But we’ve found that a good explanation of the issue is very important as a picture does not always speak a thousand words. An explanation contains information about the test environment, possible dependencies or other assumptions which need to be fulfilled. This is especially true with complex security vulnerabilities as it is very hard otherwise to understand, reproduce and then validate a finding.
Even with a solid explanation, a working Proof of Concept exploit is best. That way any doubts about the exploitability of a potential issue are cleared up and the impact can quickly and objectively be judged.
Not all information disclosures disclose anything of valueInformation disclosure is a common occurrence in the industry. By default, every system and every server will happily tell a visitor what software it is running. This can be an invaluable tool for a system engineer when debugging a problem. But it can also explain possible attack vectors to an attacker by making it obvious that a system is running outdated software with known security holes.
On the other hand it can also happen that even though a system discloses some information to the user that said disclosed information is useless or not considered of any value.
Our general rule of thumb with information disclosure reports is to have a look at the system and decide if the information is giving an attacker a clear advantage choosing his point of attack or not.
If an exact minor release version of a server with known vulnerabilities is leaked, we absolutely do consider this a valid security finding. The fact that the webserver is running Apache 2.x or that the webserver behind the load balancer is using a RFC1918 address however is not necessarily considered a valid security finding.
The only secure system is a powered-off systemWhile the security of a system is important, it is equally important to not forget about usability. If a system is locked down so tight that nobody is able to use it, it might very well be secure but it is also useless at the same time. The Internet is a dangerous place and every few days there’s a report about some website having their user database (passwords included) stolen. Browsers have been including password stores for some time now which allowed users to stop sharing passwords between sites and actually use a different password per website. Without a doubt, this has done a lot for security on the World Wide Web.
Caching passwords on the client however incurs a certain risk: The password store might not be secure or a user might accidentally store their password on a public system. We have decided for our infrastructure that this is an acceptable risk. We prefer this behavior, as it encourages users to not share passwords across different sites.
In the same vein, caching of two factor authentication tokens is not an issue either as they are extremely limited in validity and expire very quickly.
In the end, it’s always sensible to have a look at the overall picture and then judge the security impact of an issue. Sometimes it is well worth it to accept the lesser evil.
We compiled the above list based on the reports we've seen in the past, hoping that by offering these examples of incorrect reports we can save some time for other security researchers and allow them to concentrate on real security findings.
Having said that, if your spidey-sense is tingling about an issue we do not consider having a security impact: Please do send us your explanation why we're wrong. We are always interested in learning new tricks.