The first web-server and the meaning of the SSL lock

The first web-server was developed by CERN in 1990 — that is twenty-four years ago!

Back then the web was a much simpler place. For the most part web pages were static files hosted by a single server owned and operated by the same entity that managed core network infrastructure, and DNS. In many cases they even owned the building where the systems were located.

As the web became more popular the architecture of these systems needed to evolve. At first that was done by bolting on basic search capabilities.  The database-backing search was simply another process running in background indexing the documents.

1

Around 1994 SSL came about. It was used almost exclusively in e-commerce scenarios.

These systems e-commerce systems were really the beginning of the complex n-tier deployments we have today. With that said they were still very simple by today’s standards. These new systems were essentially made up of a cluster of web servers sitting in front of a shared database one-network port away.

The processes of getting a SSL certificate back then was also quite onerous even when compared to what we do for Extended Validation today. To get a certificate in many cases you had to actually visit a public notary with documentation that proved your affiliation with the business you were getting a certificate for; the notary would then attest that they saw the originals of your identification as well as those documents. I even know of cases where company’s executive staff were required to visit the CA in person.

This complexity was because businesses identity was inherently part of what that certificate was about. As a consumer if you were dealing with a online business you knew they had a clue about technology (relatively) and because this online presence was an extension of their brick-and-mortar business you already knew — they were a known quantity and knowing it was them  gave you confidence they would be applying the same diligence and practices from their online business to their online transactions.

At this point the certificates used in SSL cost as much as $1,500 each and while this slowed the adoption of SSL it also gave a signal to visitors of that the that sites that had them were not some fly-by-night operation as they were willing to spend “real” money to ensure people knew who they were.

Above and beyond that when users saw the “SSL” lock users knew their users their sessions were encrypted end-to-end and as a result their data was not going to be stolen in transit.

2

Given the kinds of organizations that would operate these sites (at the time this was banks and large e-commerce businesses) there was also an element of “these guys get security” – after all they knew how to do all of the above and had their existing brick-and-mortar reputations they were building on.

Over the next decade those small server clusters that ran these websites became more and more complicated. For these site to scale what used to run on one or two boxes got moved across many. At the edge dedicated systems were used to terminate SSL and forward clear text to back-end systems that were sometimes owned and operated by different entities and often spanned multiple networks.

3

The mega-sites like those run by Google and Microsoft still are designed in this way because it is the only way to cost effectively scale and be agile enough to meet market needs for systems of this size.
For the rest of the Internet this model just isn’t used that much anymore – its just not cost effective for small sites and most organizations don’t have access to the skills or resources to deploy the kind of networks and systems that these larger sites do. For this reason most sites have moved from deploying onto hardware and networks they own to those owned and operated by other people.

It is now the norm and not the exception to have numerous service providers embedded in a single website, the physical hardware being used by these the site and the service providers are almost always multi-tenant, even the databases backing a them are likely shared.

Expectations of users about how the web performs has changed as well, for this reason an entire industry developed to provide yet another shared service — high-end networking services that logically sit in front of these machines to ensure timely delivery wherever the user is at (AKA CDNs).

To keep pace with the demand for SSL the way certificates were priced and are validated changed as well. Today around 70% of all SSL certificates are Domain Validated (DV) and in many cases they can be had for free.

For entrepreneurs this means they can build an online business more quickly and cost-effectively than ever before. For users it means that these online businesses are fast and more professional looking but it also makes it harder to understand the security assumption behind the operational practices of the site.

The “site” as the user sees it may literally be operated by a half dozen an entities such as the network provider, CDN, DNS, hosting provider, analytics, data providers and the site it amongst others.

You don’t know what agreements the site has with these providers, how any of the entities store your data, which they share it with, or if they attempt to use good security practices in the development and operations of the services.
These things were broadly speaking inferable in the 1990’s given how few sites were on the network and the kind of investment necessary to even get online. Today some college kid in a garage could be operating your favorite site, he is motivated not by protecting his current business but by getting to market quick enough to grow his new one.

To me this means it is more important than ever before to understand whom it is your dealing with and what their security practices are. This isn’t a change that happened over night but something that has happened slowly over the last twenty-five years.

This is why its great sites publish their security and privacy practices, even if we must take them at their word. This is why it is also important to understand whom it is you are doing business with, without this how can you make an informed decision on the credibility of their word?

5

In a perfect world these things would not be items to be concerned with but as my father always told me we have to see the world the way it is and not the way we want it to be if we ever want to change it.

What are some upsides of googles’s SHA1 deprecation plan?

NOTE: Google has since adopted a more gradual plan for migration which will addresses the potential false sense of urgency the prior plan represented. Personally I think the new plan is a good one. The upsides in this post are still accurate and it is my hope people switch to SHA256 based certificates as quickly as possible.

The Internet is about to embark on another Heartbleed-esq certificate migration. This time there is no immediate danger (which was certainly not the case with Heartbleed) and there is a proposed twelve weeks to plan and respond.

During this time (unless that plan changes) a large majority of the SSL secured Internet will need to swap out their SSL certificates or the users of these sites will see a little scarier user experience. To be fair some of these certificates will be expiring regardless and need to be replaced anyway but this still represents a large number of additional sites that will need to replace certificates sooner than they had planned.

That said there are upsides, for example given how many of the top sites now use SSL the users of these sites will need to move to modern browsers not dependent on platform crypto or update to a newer version of Windows in the process gaining access  to modern web technologies and security fixes.

Another benefit is that CAs that are not active participant in the CABFORUM and who do not follow the root program requirements closely will be sure to stop their use of SHA1 based signatures as soon as they see the user experience impacted.

The same thing will be true of device companies and enterprises who do not as of today have the option to participate in the CABFORUM and even if they did are frankly unlikely to. That is when they see their support calls go up they will change their products and/or processes so that such certificates are not used.

The net of which is by the end of 2017 we will most likely see the complete end-of-life of SHA1 as part of signature suites and we may see an above average increase in modern browser adoption.

Ryan

 

 

Why might you have a certificate with a SHA1 based signature in its chain that is valid beyond 2016/1/1?

NOTEGoogle has updated the plan they will be using to deprecate SHA1 based certificates. The content in this post is still mostly accurate but for dates please see the thread. Personally I think the new plan is a good one. The upsides in this post are still accurate and it is my hope people switch to SHA256 based certificates as quickly as possible.

So there is a plan under discussion to “degrade” the user experience for SSL sessions protected with certificates (or chains) that contain a SHA1 based signature that are valid beyond 2016/1/1.

This 2016/1/1 date was apparently discussed at a CAB Forum meeting six months ago, prior to that the “sunset date” for SHA2 was considered to be 2017/1/1.

Given Chrome represents such a large percentage of the browser ecosystem and they appear to be unwaveringly marching towards this new date I think its fair to refer to this date as the “new sunset date”.

There have been lots of conversations about this topic from the perspective of a CA and that of a browser but not so much from a perspective of a certificate holder.

There are a few cases why you might have such a certificate:

  1. Your certificate was issued before the new sunset date was specified.
  2. When the new sunset date was specified your certificate authority did not update their system to restrict use of that algorithm to expire by that new date.
  3. Your certificate authority gave you the option of choosing which signature suite (and hash algorithm) and expiration dates to use and you chose SHA1.

Some might ask why CAs did not simply stop issuing certificates that utilize SHA1 based signatures all together when Microsoft issued their goal to deprecate by 2017. The answer to this is simple; there is a large number of XP machines out there (15% of the Internet and over 35% of browsers in China) and its unclear how many of them have Service Pack 3 which is necessary to support certificates with SHA2. There are also concerns about the number of mobile and embedded devices that also do not support SHA2.

So how big of a risk is the interoperability impact? It’s hard to say; some numbers i have seen suggest it is less than 1% of traffic but honestly it doesn’t appear possible to measure  the number of XP machines without SP3 and if it were it still wouldn’t take into consideration the devices that do not support SHA2 and we know such devices were shipping as recently as two years ago.

So that takes me to the main reason for this post; it’s my guess that the primary reason you have a certificate that will be effected by this change is that the CAs honestly did not realize google was moving the sunset date forward and were adopting migration plans that they felt balanced interoperability, usability and security.

With that said I believe google sincerely feels this change is in the best interest of the internet and that the user interface changes they are proposing are subtle enough that it wont be noticed by most (see : A Large-Scale Field Study of Browser Security Warning Effectiveness [pdf]).

Unfortunately this leaves you the server administrator stuck somewhat in the middle. You will have to choose to give up views and revenue from these clients that do not support SHA2 or all of your users who use Chrome will see a degraded user experience.

What will Chrome’s SHA1 early warning look like?

NOTEGoogle has since revised its plan to enable a more gradual migration to SHA256, this post is no longer accurate.

For the last few weeks there has been an ongoing discussion on the Chromium security-dev mailing list on how Google intends to implement a user interface change to warn users that a SHA1 certificate is in use.

I wont talk to the reasoning behind this change or to the current and future security properties of SHA1 in this post but I thought some folks might be interested in what this might ultimately look like. I say might because right now there is only a mail thread and who knows how things will evolve and what the copy would be in such user interfaces.

With that said the thread does describe what affordances they intend to use when a site has a certificate where it or the corresponding certificate chain has SHA1 based signature in it (excluding the root) that expires after 2016/1/1 the user interface may be “degraded” for these sessions.

At this time it seems the “red x” that is used for mixed content will be used; if so this will look something like this:

 1

 

 

 

 

For the SHA1 certificates that expire after 2017/1/1 if that page contains active content such as JavaScript and CSS that is served over a SSL session with such a certificate they will not be loaded unless the user explicitly chooses to approve their execution, this would look something like this:

2

 

 

 

 

 

Again for SHA1 certificates that expire after 2017/1/1 if the page contains passive content (such as images) that is served over a SSL session with such a certificate it will not be loaded unless the user chooses to do so and the lock will get a yellow arrow, which will look something like this:

3

 

 

 

 

 

 

 

 

Which combinations of these things one will see would be dependent on the specific combination of conditions but this will give you some idea on what these changes may look like.

Ryan

Capital One Venture – the card NOT to travel with

Those of you who know me know that I do a fair bit of travel; in the last year and a half I flew at least 250,000 miles through Asia and Europe for work.

As part of being a regular traveler I wanted a credit card with travel benefits. After doing some research I settled on the Venture Card from Capital One primarily because of its decent interest rate and its competitive points system. With that said I still cannot recommend this as a travel card.

Why you ask? Well there is more to a travel credit card than the interest rate and points. To explain let me tell you about my trip to Russia.

First to be honest I did not explicitly tell any of my cards that I was going to Russia. That said when I know I am going to be a heavy traveler like I was in this time  of my life I will notify any cards (which I did in this case) I intend to use them while traveling that I will essentially be living on the road.

Though it is a good practice to notify your card companies every time you travel internationally calling them every 3 weeks isn’t a reasonable thing to do — and in my defense until this event it was never a problem.

The first part of the trip was in Belarus where almost no one took cards and I ended up paying for everything with cash — even Internet access. The second part of this trip we went to Moscow and the first time I tried to use the Venture card it was denied.

This is the same card I had previously used throughout the rest of Europe and Asia with no problem. Assuming it was an attempt to “protect” me from card fraud I calmly called support reaching what was apparently a Philippines call center where I was instructed that my card had been flagged as stolen by someone in Russia.

I explained this was the first use of the card in Russia and the suspected fraud was me. The agent informed me that despite this fact in the name of my best interests she would be canceling my credit card.

I of course protested; I was after all in another country for another month and had planed to use the points I earned to cover some of the costs of the trip and more importantly I had left my backup travel cards in Belarus. Without this card I was in essence dependent on the limited amount of cash I had left.

I explained my situation to the agent and was told not to worry that she would have a card to me at my home within 24 hours. I explained again that I was in Russia and that sending card to the states wouldn’t be of any use.

The agent then offered to mail me the card in Russia but couldn’t guarantee when it would arrive. I explained that this could take weeks — when I ship items via the fastest choice to Russia they typically get to the country within two days but don’t get delivered for three or more weeks. The agent responded that that this was the best they could offer but after some pushing I managed to get escalated to someone in the US where I hoped I might get a better answer.

It turned out that the US office was closed at that time but a few days later I did get a call back — unfortunately though it was clear this office at least understood the situation (the agent in Philippines office was very poorly trained) I was informed that since the other agent had already canceled my card there was nothing else they could do other than send me a replacement to my home in Seattle.

This is the core of why I wouldn’t recommend Capital One for a travel card — at least to an international traveler; when your traveling your credit card is your safety net, it is how you handle currency conversions, make sure you can feed yourself, have a place to stay and can handle the surprises you may encounter. More than the points, more than the interest rate this is what a travel card is. American Express built its reputation on being that card and when I have had issues in the past they have been there to help – Capital One on the other hand left me stranded.

Anyway I was so dissatisfied with Capital One’s handling of this when I got home I paid off the balance and did not activate the new card they sent.

Fast forward to over 6 months later and I get an email saying they have charged me the renewal fee for this card that in my mind was closed. I was a little disgusted by them charging me a renewal fee for an account they in-essence took from me when I needed it most but I was going to open a card anyway and decided to activate the card they had sent previously and pay the fee.

When I activated the card the automated system told me the card was ready for use but when I tried to use the card the first time it was denied. Frustrated I set the card aside until I had enough time to mess with their support again.

When I called to resolve this I was treated like someone who was avoiding paying a long standing balance and not someone who was trying to just resolve them miss-handling an issue so I just canceled the card.

Long story short — a good travel card has to have good customer service, they have to be your partner and look out for you and Capital One just doesn’t do that.

Though in my new role I don’t do much if any international travel I do a ton of domestic and have been using the Barclay Arrival card. I have had the occasion to talk to their customer service several times, each time they were professional and helpful. While I have not had a similar situation happen while using them as my primary travel card I suspect based on these experiences they would handle things differently.

Smart cards, PC/SC and Chrome

Smart cards have been around since 1974 and as a technology while they have expanded their capabilities they still work in very much the same way they did back then.

These cards expose a protocol represented in Application Data Units (APDUs), the devices themselves are typically connected to computers via smart card readers (either embedded or external) that communicate via PC/SC.

Shortly after PC/SC was defined a class interface for USB PC/SC devices was defined called CCID with devices that conform to this specification one does not need vendor specific drivers to interact with the PC/SC device.

Since Chrome 26 Google has supported an interface that allows plug-ins to interact with USB devices. While I have not looked at this interface in detail I do know that the Google Gnubby (aka FIDO U2F) uses this interface to interact with their devices. I also understand that the U2F devices are in simple JCOP cards with a Gnubby applet on them.

Based on the above it seems rational to believe a third-party (aka someone other than Google) should also be able to create a Chrome plug-in (which is nothing more than Javascript) that allows a web-page to interact with smart-cards.

This would when paired with a reasonable card-edge that supports secp256k1 enable Multi-signature Bitcoin transactions leveraging smart cards without the need for a “fat” client.

Smart cards, PIV, Bitcoin and secp256k1

I am thrilled to see Multi-sig (P2SH) getting adopted across Bitcoin industry it has the potential to significantly reduce the risks involve with storing Bitcoin online. With that said it is still dependent on software keys, which can be trivially stolen via malware and other attack vectors.

One way to address this risk is to move the keys off of the host and into a isolated computing environment like a smart card.

Hardware devices like the Trezor do this by creating a Bitcoin specific computing environment, which has a many benefits (like being able to enforce policy on the card and get trusted implementations of the whole Bitcoin stack) but they turn into single use devices as a result of this approach also.

An alternate or really complimentary approach would be to have a smart card (or USB token) that supports the same cryptography used in Bitcoin as well as other more commonly used algorithms.

The thing to understand about smart cards is that for the most part every one you see is a proprietary non-interoperable mess. This is a function card industry attempting fend off the race to the bottom by differentiating at the card protocol layer which resulted in devices that are based on “standards” yet are totally non-interoperable.

Where they do “interoperate” it is because middleware has been written to mask these cross-vendor idiosyncrasies. The largest case where this has not happened is in the PIV card-edge, which was defined by the US government as their standard for logical and physical access control.

This card-edge explicitly supports only two ECC curves ansip256r1 and ansip384r1. That said the mechanism the caller has to specify which curve to use is via numbers in IDs specified in SP800-78 that map to the algorithm to be used (see table 6-2 for those algorithms) one could squat on un-used IDs and have a card that could also support secp256k1. This means it is possible to extend a standard PIV applet to support storing and protecting Bitcoin keys also.

PIV has other limitations that make it not ideal for these scenarios, specifically the default ACL set for the cards are such that users can not create keys themselves. GoldKey a smart card vendor who’s product uses the PIV card-edge works around this by adjusting those ACLs and embedding an administrative key within their “driver” that enables key generation to happen when their driver is used.

The net of all of this is that one can reasonably create a smart card that supported all of the rich capabilities that are available to users of a PIV device and also support protecting Bitcoin wallet keys.

Two Factor Authentication with BitGo and Coinbase

Online wallets such as BitGo and Coinbase make Bitcoin much more approachable. That said neither seem to prioritize enabling two-factor authentication. This is surely an artifact of them trying to minimize account setup friction.

The net of this decision is that users have to know setting up two-factor authentication is a good idea and go spelunking for the setting in the account settings.

For technical users this maybe fine but for the less technical often times they either don’t know such an option even exists or don’t have the patience to find where this is done.

It might seem like an unfair criticism to suggest this is a bad approach since most banks and e-commerce sites don’t go make this experience much better but I think Bitcoin companies can and should do more.

If LinkedIn, Facebook and Twitter can remind us to improve our social profiles these high-tech financial institutions can remind us to improve our account safety.

UPDATE 07/28/14: Mike Belshe of BitGo points out that it may not be directly obvious but BitGo does actually require two factor authentication once you add a wallet to the account but since I did not attempt to create a wallet until the account was adequately secured I never observed this enforcement. This approach represents a decent trade-off for reduction of account sign-up friction and account security.

UPDATE 07/29/14: I should also point out that Coinbase does require you to use multi-factor if you use their vault feature. I personally think that they should still be encouraging non-vault users to use multi-factor though.

I would add that while these new wallet services are much easier to use than their predecessors I think there is still plenty of room for improvement and I am looking forward to seeing what they and the newer entrants to this space will bring to the table for users.

If you’re not familiar with the user experiences these two services check out these presentations that show you how to setup accounts with them:

Cupid and Understanding Your Exposure

In a past life I was responsible for a number of Windows Enterprise Networking technologies including the EAP implementation and was a contributor to EAP-TLS.

As a result when I saw the Heartbleed vulnerability announced I was painfully aware of what an attacker could do if they encountered a EAP-TLS implementation based on a vulnerable version of OpenSSL.

I have read a few articles this morning discussing the implications to those who use EAP-TLS and many of them get it wrong; the core issue being they don’t seem to understand the various actors in a EAP-TLS negotiation.

To understand how EAP works you need to understand the following terms:

Peer; this is the client to the wireless network. Be it your desktop, laptop or mobile phone (though EAPs use is not limited to these types of devices). Sometimes this will be called the supplicant.

Authenticator; this is your wireless router, when it comes to authenticator this entity delegates that responsibility to the authentication server (typically over RADIUS).

Authentication Server; this is the entity who is responsible for deciding if the peer can have access or not, it is also typically the EAP-Server but it can delegate this to another entity as in the case of complex wireless federation systems like.

EAP Server; this is the entity who actually implements the EAP protocol.

When a client connects to a router the router is configured to forward all requests to an Authentication Server. This results in a tunnel through the router to the authentication server.

In the most common case that authentication server implements the EAP protocol along with all of the various protocols (methods) that it supports.

In a Windows environment that server is called the Network Policy Server (NPS),  CISCO’s is called Cisco Secure Access Control Server (ACS), Juniper’s is called Steel Belted Radius (SBR) and the two most common Open Source distributions are OpenRADIUS and FreeRadius both of which use OpenSSL ([1], [2]).

The reason I wrote this is most of these articles talking about CUPID do so in such a way that it suggests the wireless router itself is the issue; while this is technically possible because there are routers out that also contain the authentication server and EAP Server these are not commonly used – especially with a TLS based EAP method. The main reason for this is that this quickly doesn’t scale, one of the largest reasons being that environments mature enough to take on the management overhead of a TLS based solution also probably have to provide service to a space larger than a single access point could cover.

There is another class of wireless access solution that leverages a “wireless controller” that manages multiple radios for this very reason. These solutions also often have built in RADIUS and EAP servers (and in my experience they are based on one of the above OpenSource solutions) but again this capability doesn’t appear to be used often with TLS based EAP methods since organizations often already have existing authentication infrastructure (Active Directory and NPS for example) that they leverage instead.

Long story short your probably only at risk of this if your using one of the Open Source Radius servers but since some vendors just repackage this code there are deployment models where you may be exposed its just not super likely. Here is a quick decision tree to help you understand your exposure:

Cupid Exposure