Category Archives: Thoughts

Using Monitis to monitor OCSP and CRL performance

Earlier I did a post on the performance of revocation repositories, in that I used the Monitis do some basic bench-marking for repository performance.

They have a cool feature that allows me to create a public monitoring portal on one of my own domains, I have created two:

  1. OCSP Performance
  2. CRL Performance

It also looks like CloudFlare (one of our CDN partners), the ones we are working with to get these great numbers have also done a blog post on what we are doing here – cool stuff!

Ryan

 

A revised look at the new Windows Update SSL configuration

The other day I did a quick post about how SSL was configure for Windows Update, I thought I would double back and see if they have made any changes and it looks like they have.

From looking at the SSL Labs results I notice a few changes:

  1. The servers are now indicating a cipher suite preference.
  2. The servers are now putting TLS suites above the SSL suites.
  3. The servers no longer support TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA.
Since they made some changes since the post we can assume the goal was to improve things, I think they have but it doesn’t entirely make sense to me the choices they have made, for example:
  1. Why continue to support SSL 2.0 when no client that only supports SSL 2.0 has the WU client software on it.
  2. Why include more than one SSL cipher suite, if the goal is to enable the use of SSL 3.0 and the only clients to this server are based on SCHANNEL and CryptoAPI a single suite would be enough.
  3. If your going to support the RC4 cipher suites, why not prioritize them above the other suites so that they are resistant to BEAST.
  4. Why remove TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, I am guessing its performance but that is surprising since its not that much more expensive.

We also see a few changes when browsing to https://www.update.microsoft.com, unlike before when we browse from a client that doesn’t have a WU agent available on it (like XP /w IE6 and no SP) we no longer get instructions on how to get patched we now get a blank page.

Anyhow thats what I see today.

Ryan

Setting HTTP headers with OpenSSL and the OCSP test client

So I have been doing some OCSP performance bench-marking the last few weeks, one of the things we noticed while doing this is that OpenSSL does not set the HOST header when sending requests.

This is a problem when interacting with a OCSP responder that is on a shared host (a virtual host), without it the web-server doesn’t know which instance to send the request to. You can work around this by using the “-header” command, for example:

openssl ocsp -noverify -no_nonce -respout ocspglobalsignca.resp -reqout ocspglobalsignca.req -issuer globalsigng2.cer -cert globalsign.com.cer -url “http://ocsp2.globalsign.com/gsextendvalg2” -header “HOST'” “ocsp2.globalsign.com”

You need to ensure the HOST header is set consistently with the host contained in the -url switch but it works.

Ryan

Measuring OCSP Responder Performance with Powershell

The other day I did a post about testing if OCSP stapling was working, but what about telling how fast your OCSP responder is?

Well that is what this post is about, on Unix based systems with curl you would do something along these lines:

time curl –verbose –url http://ocsp2.globalsign.com/gsextendvalg2/MFMwUTBPME0wSzAJBgUrDgMCGgUA

BBSgcg6ganxiAlTyqPWd0nuk87cvpAQUsLBK/Rx1KPgcYaoT9vrBkD1rFqMCEhEh

D0Xjo/V7lgq3ziGoWG69rA==

But there is no time on Windows so how can you do it there?

PowerShell.exe
Measure-Command {Start-Process -wait “curl” “-verbose “”http://ocsp2.globalsign.com/gsextendvalg2/MFMwUTBPME0wSzAJBgUrDgMCGgUA

BBSgcg6ganxiAlTyqPWd0nuk87cvpAQUsLBK/Rx1KPgcYaoT9vrBkD1rFqMCEhEh

D0Xjo/V7lgq3ziGoWG69rA==”””}

 

Days : 0
Hours : 0
Minutes : 0
Seconds : 0
Milliseconds : 53
Ticks : 531200
TotalDays : 6.14814814814815E-07
TotalHours : 1.47555555555556E-05
TotalMinutes : 0.000885333333333333
TotalSeconds : 0.05312
TotalMilliseconds : 53.12

 

Here you see that it took 53 milliseconds to run the command, its important to note that there different overhead buried each time measurement approach (time vs Measure-Command) that make it inappropriate to compare the results from two approaches. With that said its still a good way to measure performance over time when using a single measurement approach.

Also if you wanted to do the same measurement for POSTs the command would look like one of these :

Measure-Command {Start-Process -wait “openssl” “ocsp -noverify -no_nonce -respout ocspglobalsignca.resp -reqout ocspglobalsignca.req

-issuer globalsigng2.cer -cert globalsign.com.cer -url “”http://ocsp2.globalsign.com/gsextendvalg2″” -header “”HOST”” “”ocsp2.globalsign.com”””}

 

Measure-Command {Start-Process -wait “curl” “-verbose -data-binary  @ocspglobalsignca.req -H “”Content-Type:application/ocsp-request””

http://ocsp2.globalsign.com/gsextendvalg2″}

Pulse data publicly availible

SSL Pulse is an initiative ran by Qualys to monitor the overall health of the SSL deployments on the Internet. It is based on the SSLLABS work Ivan Ristik has done; he has recently published the data used to derive these reports.

There are some interesting findings in buried in the raw data, for example:

  • Most of the certificates (85%?) are expired.
  • Most of the certificates are self-signed or from internal PKIs.
  • Those 5 “shorter than 1024bit” keys in the Pulse Dashboard are down to 3 (based on manual verification) are time valid certificates from public CAs, two expire this year the last in 2014.
Host Issuance Date Expiration Date Key Size Issuer
www.pysoft.com 01/25/2009 02/24/2014 512 Equifax Secure Global eBusiness CA-1
www.comlink.com.br 10/13/2009 11/07/2012 512 Thawte Premium Server CA
www.rtp.com 04/13/2009 06/04/2012 512 Thawte Premium Server CA

 

  • There are 2,472 RSAwithMD5 certificates in the 215,607 sample-set or around .01% of the hosts.

This last point gives us some context to some numbers Venafi published recently, they indicated 17.4% of the certificates they see are signed using MD5; this is a rate significantly higher than what we see here. Its probably safe to assume the difference is that their sample is primarily derived from intranets where private CAs are commonly are set up and forgotten.

For me the largest conclusion we can take from this data appears to be that there is a large number of organizations who set-up SSL (and PKIs) and simply forget about them – this is of course not a surprise but it’s neat to see it backed up with numbers.

Ryan

 

A look at the new Windows Update SSL certificates

This morning I noticed a tweet by Mikko about the Windows Update certificate chain looking odd so I decided to take a look myself.

I started with the webserver configuration using SSLLABS, unfortunately it did not fare well:

Looking a little closer we see a few things of interest:

  • SSLLABS is unable to validate the certificate
  • The server is using weak ciphers
  • The server is vulnerable to the BEAST attack
  • The server is not using an Extended Validation  (EV) Certificate
  • The server is supporting SSL 2.0

To understand the specifics here we needed to look a little deeper, the OpenSSL s_client is a great tool for this:

openssl s_client –showcerts -status –connect www.update.microsoft.com:443

Loading ‘screen’ into random state – done

CONNECTED(0000017C)

OCSP response: no response sent

depth=1 C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, CN = Microsoft Update Secure Server CA 1

verify error:num=20:unable to get local issuer certificate

verify return:0

Certificate chain

0 s:/C=US/ST=Washington/L=Redmond/O=Microsoft/OU=WUPDS/CN=www.update.microsoft.com

i:/C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/CN=Microsoft Update Secure Server CA 1

1 s:/C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/CN=Microsoft Update Secure Server CA 1

i:/DC=com/DC=microsoft/CN=Microsoft Root Certificate Authority

Server certificate

—–BEGIN CERTIFICATE—–

MIIF4TCCA8mgAwIBAgITMwAAAAPxs7enAjT5gQAAAAAAAzANBgkqhkiG9w0BAQUF

—–END CERTIFICATE—–

1 s:/C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/CN=Microsoft Update S

ecure Server CA 1

i:/DC=com/DC=microsoft/CN=Microsoft Root Certificate Authority

—–BEGIN CERTIFICATE—–

MIIGwDCCBKigAwIBAgITMwAAADTNCXaXRxx1YwAAAAAANDANBgkqhkiG9w0BAQUF

—–END CERTIFICATE—–

subject=/C=US/ST=Washington/L=Redmond/O=Microsoft/OU=WUPDS/CN=www.update.microsoft.com issuer=/C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/CN=Microsoft Update

Secure Server CA 1

No client certificate CA names sent

SSL handshake has read 3403 bytes and written 536 bytes

New, TLSv1/SSLv3, Cipher is AES128-SHA

Server public key is 2048 bit

Secure Renegotiation IS supported

Compression: NONE

Expansion: NONE

SSL-Session:

Protocol  : TLSv1

Cipher    : AES128-SHA

Session-ID: 33240000580DB2DE3D476EDAF84BEF7B357988A66A05249F71F4B7C90AB62986

 

Session-ID-ctx:

Master-Key: BD56664815654CA31DF75E7D6C35BD43D03186A2BDA4071CE188DF3AA296B1F9674BE721C90109179749AF2D7F1F6EE5

Key-Arg   : None

PSK identity: None

PSK identity hint: None

Start Time: 1339954151

Timeout   : 300 (sec)

Verify return code: 20 (unable to get local issuer certificate)

read:errno=10054

 

With this detail we can also look at the certificates with the Windows Certificate viewer, we just extract the server certificate Base64 and put it into a text file with a .cer extension and open it with Explorer:

   
   
   

 

From these we see a few additional things:

  • OCSP Stapling is not enabled on the server
  • The issuing CA was created on 5/30/2012 at 8:49pm
  • The issuing CA was issued by the 2001 SHA1 “Microsoft Root Authority”

So with this extra information let’s tackle each of these observations and see what conclusions we come to.

 

SSLLABS is unable to validate the certificate; there are two possible reasons:

a. The server isn’t including the intermediate certificates (it is) and SSLLABS doesn’t chase intermediates specified in the AIA:IssuerCert extension (doubt it does) or that extension isn’t present (it is).

b. The Root CA isn’t trusted by SSLLABS (which appears to be the case here).

My guess based on this is that Ivan only included the certificates in the “Third-Party Root Certification Authorities” store and did not include those in the “Trusted Root Certification Authorities” which are required for Windows to work.

Basically he never expected these Roots to be used to authenticate a public website.

[2:00 PM 6/18/2012] Ivan has confirmed he currently only checks the Mozilla trusted roots, therefor this root wouldn’t be trusted by SSLLABS.

Microsoft’s decision to use this roots means that any browser that doesn’t use the CryptoAPI certificate validation functions (Safari, Opera, Chrome on non-Windows platforms, Firefox, etc.) will fail to validate this certificate.

This was probably done to allow them to do pinning using the “Microsoft” policy in CertVerifyCertificateChainPolicy.

I believe this was not the right approach since I think it’s probably legitimate to use another browser to download patches.

[2:00 PM 6/18/2012] The assumption in this statement (and it may turn out I am wrong) is that it is possible for someone to reach a path where from a browser they can download patches; its my understanding this is an experience that XP machines using a different browser have when visiting this URL I — I have not verified this.

[3:00 PM 6/18/2012] Harry says that you have not been able to download from these URLs without IE ever, so this would be a non-issue if that is the case.

To address this Microsoft would need to either:

  • Have their PKI operate in accordance with the requirements that other CAs have to meet and be audited and be found to meet the requirements of each of the root programs that are out there.
  • Have two separate URLs and certificate chains one for the website anchored under a publicly trusted CA and another under this private “Product” root. The manifests would be downloaded from the “Product” root backed host and the web experience would be from the “Public” root backed host.
  • Cross certifying the issuing CA “Microsoft Update Secure Server CA 1” under a public CA also (cross certification), for example under their IT root that is publically trusted and include that intermediate in the web server configuration also. Then have a CertVerifyCertificateChainPolicy implementation that checks for that CA instead of the “Product” roots.

 

The server is using weak ciphers; the server is using several weak ciphers:


I see no reason to support the MD5 based ciphers as I find it hard to believe that there are any clients that can communicate with this site that do not support their SHA1 equivalents.

 

[2:00 PM 6/18/2012] I have been told I am too critical by calling these MD5 based ciphers as weak in that they are used as HMAC, it is true that when used with a key as is the case with HMAC the current attacks are not relevant. With that said any client that supports these suites will also support their SHA1 counterpart and there is no reason to support the weaker suites that use MD5.

 

The server is vulnerable to the BEAST attack; and SSLLABS isn’t able to tell if the server is specifying a cipher suite preference, this means it probably is not.

It is the cipher suite ordering issue that is actually resulting in the warning about the BEAST attack though. It is addressed by putting RC4 cipher suites at the top of the cipher suite order list.

[2:00 PM 6/18/2012] It’s been argued the BEAST attack isn’t relevant here because the client is normally not a browser, these pages that are returned do contain JS and there are cases where users visit it via the browser — otherwise there would not be HTML and JS in them. As such the attacker could use the attack to influence you to install malicious content as if it came from Microsoft. Maybe its not a leakage of personal information initially but its an issue.

 

It is not using an Extended Validation (EV) Certificate; this is an odd one, is an EV certificates necessary when someone is attesting to their own identity? Technically I would argue no, however no one can reasonably expect a user to go and look at a certificate chain and be knowledgeable enough to that this is what is going on.

The only mechanism to communicate the identity to the user in as clear a way is to make the certificate be an EV certificate.

Microsoft really should re-issue this certificate as an EV certificate – if there was ever a case to be sure who you are talking to it would certainly include when you are installing kernel mode drivers.

 

The server is supporting SSL 2.0; this also has to be an oversight in the servers configuration of SSL 2.0 has been known to have numerous security issues for some time.

They need to disable this weak version of SSL.

 

OCSP Stapling is not enabled on the server; OCSP stapling allows a webserver to send its own revocations status along with its certificate improving performance, reliability and privacy for revocation checking. According to Netcraft Windows Update is running on IIS 7 which supports it by default.

This means Microsoft is either not allowing these web servers to make outbound connections or they have explicitly disabled this feature (login.live.com has it enabled and working). While it is not a security issue per-se enabling it certainly is a best practice and since it’s on by default it seems they are intentionally not doing it for some reason.

 

The issuing CA was created on 5/30/2012 at 8:49pm; this isn’t a security issue but it’s interesting that the issuing CA was created four days before the Flame Security advisory. It was a late night for the folks operating the CA.

 

That’s it for now,

 

Ryan

What is your organizations policy on SSL?

In other posts I discussed how to redirect the initial request to a website from the HTTP version to the HTTPS  (for Apache and IIS).

By following those steps your website no longer will serve HTTP content but users will still be able to get to your site without having to know to type the HTTPS:\\ before they browse to your site.

This is an important part of making your site reachable by users over SSL because:

  1. Most users do not type a URL moniker at all when entering an URL.
  2. Since 99% of the traffic on the Internet is not available over HTTPS so browsers default to HTTP.
  3. Existing HTTP URLs that have been indexed by search engines, embedded in documents, passed off in emails can continue to work.

This approach isn’t perfect, for example:

  1. An attacker can perform a Man-In-The-Middle on the initial request and bypass the SSL protection (see sslstrip).
  2. It has the potential to “train users” to not navigate to the HTTPs version of your site initially.

The problem is of course that the alternative of returning an error when a user requests the HTTP version of the website (say a 403.4 – SSL required) or simply not having a server listening on the HTTP port is almost the same as saying your site isn’t accessible to the mainstream users.

So how can you manage these problems? There are a few things you can do:

For sensitive services like those for login and those that collect personal information or credit cards actually use 403.4 errors. This tells the user in no uncertain terms that SSL is required for that task but since the browsing experience does not typically “start here” you do break the user experience for your users.

You may also want to consider hosting the most sensitive content like login and account details on a separate virtual host that does not have a HTTP listener (for example login.example.com or accounts.example.com).

Next you should communicate your policy on SSL to the web browsers so they can do the right thing for the users, there are several ways for you to do this:

  1. Set the HTTP Strict-Transport-Security (HSTS) header for your pages this will tell the browsers to require SSL on your site.
  2. Request that your site be added to the HTTPS Everywhere Rule list.
  3. Request that your site be added to the Preloaded HSTS list in Google Chrome.
  4. If you are a larger site you can also request that Google “pin” your web server’s public keys to your domain.

These things will not eliminate these risks but it does help, especially for those using browsers that support HSTS (Chrome and Firefox as of today) or those that are using plugins like HTTPS Everywhere and NoScript.

As for the last two, they are clearly Chrome specific but it represents about 32% of the browser market today and as such is worth paying attention to.

Ryan

Additional Resources

How to Deploy HTTPS Correctly

 

Was the Flame WSUS attack caused just because of the use of MD5?

This morning I saw a number of posts on Twitter about Flame and the attacks use of a collision attack against MD5.

This flurry of posts was brought on about by Venafi, they have good tools for enterprises for assessing what certificates they have in their environments, what algorithms are used, when the certificates expire, etc. These tools are also part of their suite used for certificate management.

They published some statistics on the usage of MD5, specifically they say they see MD5 in 17.4% of the certificates seen by their assessment tools. Their assessment tool can be thought of as a combination of nmap and sslyze with a reporting module.

Based on this we can assume the certificates they found are limited to SSL certificates, this by itself is interesting but not indicative of being vulnerable to the same attack that was used by Flame in this case.

 

 

Don’t get me wrong Microsoft absolutely should not have been issuing certificates signed using MD5 but the collision was not caused (at least exclusively) by their use of MD5; it was a union of:

  1. Use of non-random serial numbers
  2. Use of 512 bit RSA keys
  3. Use of MD5 as a hashing algorithm
  4. Poorly thought out certificate profiles

If any one of these things changed the attack would have become more difficult, additionally if they had their PKI thought out well the only thing at risk would have been their license revenue.

If it was strictly about MD5 Microsoft’s announcement the other day about blocking RSA keys smaller than 1024 bit would have also included MD5 – but it did not.

So what does this mean for you? Well you shouldn’t be using MD5 and if you are you should stop and question your vendors who are sending you down the path of doing so but you also need to take a holistic look at your use of PKI and make sure you are actually using best practices (key length, serial numbers, etc). With that said the sky is not falling, walk don’t run to the nearest fire escape.

Ryan

Flame and Certificate Revocation

Microsoft has published patches that insert the CAs directly associated with the Terminal Services Licensing PKI into the “Untrusted” certificate store, this has the same effect as revoking the certificates for those that:

  1. Apply the patch
  2. Use Microsoft’s chain validation engine

But what about those that do not apply the patch? Or those that simply import the roots from the Microsoft store to use in another engine (like NSS or OpenSSLs, one example of such an application would be Adobe X)?

Well today those people will still trust certificates issued by those certificates, they potentially would not though if Microsoft published revocation information for those CAs.

As it stands most of the certificates in question do contain revocation pointers (a CRLdp extension in this case) but the URLs they point to are invalid, for example:

  1. “Microsoft Enforced Licensing Intermediate PCA” – No revocation information
  2. “Microsoft Enforced Licensing Registration Authority CA” – http://crl.microsoft.com/pki/crl/products/MicEnfLicPCA_12-10-09.crl
  3. “Microsoft LSRA PA” –  http://crl.microsoft.com/pki/crl/products/MicEnfLicRegAutCA_2009-12-10.crl
  4. “Terminal Services LS” – http://tk5paxprdsrv09.partners.extranet.microsoft.com/CertEnroll/Microsoft%20LSRA%20PA.crl

This may not be beneficial for the attack vector used in Flame (I do not know if it even does revocation checking – though it very probably does) but it would certainly help other cases.

Today the invalid URLs in these certificates would result in a “Unknown or Offline” response which would likely be ignored by applications due to Microsoft’s “soft-fail revocation checking” defaults.

Even with those defaults though if they published these CRLs clients would get some benefit, it would:

  1. Reduce risk until the patch could get applied.
  2. Help others who mistakenly trusted the Microsoft roots when they “imported” them from the “Trusted Root Store”.

It’s my hope Microsoft decides to publish these CRLs for relying party applications to rely on.

[3:22 PM 6/19/2012] FYI as of 6/19 the MicEnfLicPCA_12-10-09.crl is published (good), it appears to have been created June 14th several days after this post. The other two are still invalid, it’s likely they decided against publishing them since a well behaved client would fail with just one of the CAs being bad. It’s not a unreasonable decision, I am guessing the URL choices for the CRLs made it difficult to do all of the CRLs.

Ryan

Flame was just one use of the Terminal Services Licensing PKI

I wanted to do a post on how the Terminal Services Licensing PKI could be used in another attack; though the variants are endless I have one concrete example of how this was used in in an attack in 2002 in this certificate:

   
   

 

What we see in this certificate is that the attacker took the License Certificate (Terminal Services LS) and issued himself a certificate underneath it. This only worked because of another defect in the Windows chain engine, this one relating to how it handled Basic Constraints (see MS 02-050) – that was fixed in 2003.

But why would he have done this? I see two possible reasons:

  1. Bypassing the terminal services licensing mechanism.
  2. Bypassing some client authentication scheme based on X.509.

We won’t explore #1 as I am not interested in the attacks on license revenue for Microsoft but how this infrastructure designed to protect that license revenue was used to attack customers and as such I will focus on #2.

Now, before I go much further I have to be clear – I am just guessing here, I don’t have enough data to be 100% sure of what this was used for but it seems pretty likely I am right.

What I see here is the X.509 serial number is different than the Serial Number RDN the attacker put into the Subject’s DN.

This certificate also has a value in the CN of the Subject DN that looks like it is a user name (PORTATILE).

Neither of these should be necessary to attack the licensing mechanism as legitimate license certificates don’t have these properties or anything that looks roughly like them.

My theory is that the attacker found some Windows Server configured to do client authentication (possibly for LDAPS or HTTPS) and wanted to trick the server into mapping this certificate into the principal represented by that CN and Serial Number in the DN.

Why would this have worked? Well as I said there was the Basic Constraints handling defect, but that wouldn’t have  been enough the server would have had to had blindly trusted all the root certificates in the root store for “all usages”.

This was a fairly common practices back then, web servers and other server applications would be based on other libraries (for example SSLEAY/OpenSSL) and just import all the roots in the CryptoAPI store and use them with this other library.

This is actually still done by some applications today, for example Adobe Reader X:

 

The problem with this behavior is that these applications do not necessarily (read almost never) import the properties associated with these certificates, nor do they support the same enforcement constraints (for example Nested Extended Key usages).

The net effect of which is that when they import the roots they end up trusting them for more than the underlying Windows subsystem does.

The attackers in this case were likely taking advantage of this bad design of this client authentication solution as well as the bad design of the Terminal Services Licensing PKI.

The combination of which would have allowed the attacker to impersonate the user PORTATILE and perform any actions as that user they liked.

Even if I am wrong in this particular case, this sort of thing was definitely possible.

What lessons should we take away from this?

For one always do a formal security review of your solutions as you build them as well as before you deploy them; pay special attention to your dependencies – sometimes they are configuration as well as code.

Also if an issue as fundamental as the Terminal Services Licensing PKI can fly under the radar of a world class security program like Microsoft’s you probably have some sleepers in your own infrastructure too – go take a look once more for the assumptions of the past before they bite you too.

Ryan