It is no secret that in the last two months GlobalSign was affected by outages at relating to our use of CloudFlare. I won’t go into the specifics behind those outages because the CloudFlare team does a great job of documenting their outages as well as working to make sure the mistakes of the past do not reoccur. With that said we have been working closely with CloudFlare to ensure that our services are better isolated from their other customers and to optimize their network for the traffic our services generate.
I should add that I have a ton of faith in the CloudFlare team, these guys are knowledgeable, incredibly hard working and very self critical — I consider them great partners.
When looking at these events it is important to look at them holistically; for example one of the outages was a result of mitigating what has been called the largest publically announced DDOS in the history of the Internet.
While no downtime is acceptable and I am embarrassed we have had any downtime it’s also important to look at the positives that come from these events, for one we have had an opportunity to test our mitigations for such events and improve them so that in the future we can withstand even larger such attacks.
Additionally it’s also useful to look the actual uptime these services have had and to give those numbers some context look at them next to one of our peers. Thankfully I have this data as a result of the revocation report which tracks performance and uptime from 21 different network worldwide perspectives every minute.
For 05/2012-12/2012 we see:
Service | Uptime(%) | Avg(ms) | |
GlobalSign/AlphaSSL | OCSP | 100.00 | 101.29 |
VeriSign/Symantec/Thawte/GeoTrust/Trustcenter | OCSP | 99.92 | 319.40 |
GlobalSign/AlphaSSL | CRL | 100 | 96.86 |
VeriSign/Symantec/Thawte/GeoTrust/Trustcenter | CRL | 99.97 | 311.42 |
For 01/2013 to 04/2013 we see:
Service | Uptime(%) | Avg(ms) | |
GlobalSign/AlphaSSL | OCSP | 99.98 | 76.44 |
VeriSign/Symantec/Thawte/GeoTrust/Trustcenter | OCSP | 99.85 | 302.88 |
GlobalSign/AlphaSSL | CRL | 99.98 | 76.44 |
VeriSign/Symantec/Thawte/GeoTrust/Trustcenter | CRL | 99.22 | 296.97 |
NOTE: Symantec operates several different infrastructures – which one you hit is dependent on which brand you buy from and some cases which product you buy. We operate only two brands which share the same infrastructure. I averaged the results for each of their brands together to create these two tables. If you want to see the independent numbers see the Excel document linked to this post.
As you can see no one is perfect; I don’t share this to say our downtime is acceptable because it is not, but instead I want to make it clear this is data we track and use to improve our services and to make it clear what the impact really was.
By the way if you want to see the data I used in the above computation you can download these spreadsheets.