Caching and Processing 2TB Mozilla Crash Reports

Mozilla processes TBs of Firefox crash reports daily using HBase, Hadoop, Python and Thrift protocol. The project is called Socorro, a system for collecting, processing, and displaying crash reports from clients. Today the Socorro application stores about 2.6 million crash reports per day. During peak traffic, it receives about 2.5K crashes per minute.

via Caching and Processing 2TB Mozilla Crash Reports in memory with Hazelcast

A peak of 40 crash reports per second and an average around 30 per second! I wonder what the distribution looks like for installs sending in crash reports. Are the majority of the reports coming in from a small portion of users or is it more spread out across the entire install base?

2 thoughts on “Caching and Processing 2TB Mozilla Crash Reports”

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>