Steve Sheng, Brad Wardman, Gary Warner, Lorrie Cranor, Jason Hong, and Chengshan Zhang

Conference on Email and Spam (CEAS)

January 2009

In this paper, we study the e*ectiveness of phishing blacklists. We used 191 fresh phish that were less than 30 minutes old to conduct two tests on eight anti-phishing toolbars. We found that 63% of the phishing campaigns in our dataset lasted less than two hours. Blacklists were ine*ective when protecting users initially, as most of them caught less than 20% of phish at hour zero. We also found that blacklists were updated at di*erent speeds, and varied in coverage, as 47% - 83% of phish appeared on blacklists 12 hours from the initial test. We found that two tools using heuristics to complement blacklists caught signi*cantly more phish initially than those using only blacklists. However, it took a long time for phish detected by heuristics to appear on blacklists. Finally, we tested the toolbars on a set of 15,345 legitimate URLs for false positives, and did not *nd any instance of mislabeling for either blacklists or heuristics. We present these fi*ndings and discuss ways in which anti-phishing tools can be improved.

phishing, usable privacy and security