The Goals of HTTPS and TLS
The aim of Secure HTTP without HTTPS is to be a viable alternative to HTTPS, but what would an alternative look like? What does HTTPS do? In order to review an alternative, we need to have some understanding of HTTPS and TLS.
Wikipedia describes HTTPS as “…communication over Hypertext Transfer Protocol (HTTP) within a connection encrypted by Transport Layer Security, or its predecessor, Secure Sockets Layer. The main motivation for HTTPS is authentication of the visited website and protection of the privacy and integrity of the exchanged data”. The page actually goes on to give an eloquent comparison of HTTP and HTTPS; “HTTP is not encrypted and is vulnerable to man-in-the-middle and eavesdropping attacks, which can let attackers gain access to website accounts and sensitive information, and modify webpages to inject malware or advertisements. HTTPS is designed to withstand such attacks and is considered secure against them…”.
Now that we have some better understanding of HTTPS, then what is TLS? What does it do?
Once again, back on Wikipedia, there is a lengthy, but excellent description of the primary aims of TLS:
“When secured by TLS, connections between a client (e.g., a web browser) and a server (e.g., wikipedia.org) have one or more of the following properties:
- The connection is private (or secure) because symmetric cryptography is used to encrypt the data transmitted. The keys for this symmetric encryption are generated uniquely for each connection and are based on a shared secret negotiated at the start of the session (see TLS handshake protocol). The server and client negotiate the details of which encryption algorithm and cryptographic keys to use before the first byte of data is transmitted (see Algorithm below). The negotiation of a shared secret is both secure (the negotiated secret is unavailable to eavesdroppers and cannot be obtained, even by an attacker who places themselves in the middle of the connection) and reliable (no attacker can modify the communications during the negotiation without being detected).
- The identity of the communicating parties can be authenticated using public-key cryptography. This authentication can be made optional, but is generally required for at least one of the parties (typically the server).
- The connection ensures integrity because each message transmitted includes a message integrity check using a message authentication code to prevent undetected loss or alteration of the data during transmission.”
Reducing all of this down, HTTPS with TLS, provide us with the following:
- We know that information transmitted over the connection is private, and others cannot determine the contents. This is achieved by:
- The use of unique symmetric keys for each connection.
- Negotiating these keys in a secure (private) and reliable manner.
- We can verify who the other party is, if we wish.
- We know that the information transmitted has not been modified, either maliciously or through data loss.
Now that we have a clearer understanding of HTTPS, we can start to look at how we want to assess Secure HTTP without HTTPS. I felt that the best evaluation criteria would be to assess how effectively it achieves the same outcomes as HTTPS. The code base should also be reviewed to ensure that it is at a satisfactory level, free from defects, and free of backdoors.
My final criteria for the review:
- Data transmitted is kept private, others cannot determine the contents.
- Data cannot be modified without changes being detected by the receiving party.
- Parties can verify who each other are.
- Sensitive material is handled in an appropriate manner.
- The code is free from obvious defects.
- There are no backdoors.
I opted to just do an offline code review, I didn't setup a running environment.
Now that I have the code, and the evaluation criteria, my next step was to put in a professional development request with my employer, Readify. One of the great reasons that I enjoy working at Readify is that we get some amazing opportunities to learn and develop our skills.
Once the time was approved, I got to work.
How does Secure HTTP without HTTPS work?
Secure HTTP without HTTPS, or SMAES as it refers to itself, has two components, a client component, which is a Unity Asset written in C#, and a server component written in PHP.
Implementing the client components of SMAES is easy and only a minimal amount of changes is required to your code. You start by importing the required classes, add the initialization code, specify a shared key, and then ask it to encrypt (or decrypt) information as required.
The server implementation expects that you are using PHP. Once again integration is simple, include the files as required, add some initialization code, specify the same shared key as the client and once again encrypt and decrypt information as required. Optionally, you may want to setup and configure a SQL table on either MySQL or PostgreSQL for storing client keys.
So, what does an “encrypted” conversation look like? On the client side, we perform the following steps:
- Assemble the HTTP request, with parameters. For example: [http://myserver.com/smaes/mypage.php?password=MyPasswordIsSecure&username=Kieran]
- Call SMUtil.encryptURL(URL). Under the covers, the function performs the following steps:
- The Unity variable SystemInfo.deviceUniqueIdentifier is reversed and stored as device ID. We will discuss this Unity variable later.
- AES class is created, with the device ID as the initialization vector and the shared key as the AES encryption key.
- For each parameter in the URL, the value is encrypted with AES, and hashed and encrypted as a check value (PC). The request URL is modified to include: the device ID, each parameter and its encrypted value, each parameter’s encrypted md5 hash as name_PC. The resulting string is returned: http://myserver.com/smaes/mypage.php?password=encryptedstring&password_pc=encryptedmd5hash&username=encryptedstring&username_pc=encryptedmd5hash
- Now you perform the request as you normally would in Unity.
- Decrypt the server’s response using SMAES.decryptTLF(string) as required.
Use of a shared key
In TLS, transmitted data is kept private by encrypting it with symmetric cryptography. They keys used for the symmetric encryption are generated for each connection and are based upon a shared secret negotiated at the start of the session. In TLS, only the client and the server can read the data transmitted, no one else.
With Secure HTTP without HTTPS, data is still kept private through the use of symmetric cryptography, however the key used is fixed for all clients and the server. Whilst the data is private from a casual observer, the data sent from one client to the server isn’t private to the rest of the clients. The issue is, if more than 2 people can read the private data sent, is it really that private?
SMAES also provides functionality for the client and server to synchronize to new, random encryption keys. This is performed via the SMKey.syncKey() and syncKey.php on the client and server respectfully. During this process, the server will generate a new encryption key and send it back to the client. The new key will be the md5 hash of a timestamp and the clients IP address. Unfortunately, there is a critical issue in this functionality, as discussed in Device ID as Encryption Key.
Device ID as Initialization Vector
Initialization Vectors (IV) are a fixed-sized random input into a cryptographic function. The reason these values are random is to ensure that encryption functions achieve semantic security, that is, where repeated usage of the same key does not allow an attacker to infer relationships between parts of an encrypted message. In an algorithm like AES, the key protects our data but the IV ensures that if two identical pieces of text are encrypted, they do not produce the same encrypted result; as such the IV needs to be random and unique every time we go to encrypt something. The IV doesn’t protect the data, so we can transmit it as plaintext with the data; an eavesdropper will gain nothing from learning the IV. If you want to know more about initialization vectors, I recommend this CryptoFails post: Crypto Noobs #1: Initialization Vectors.
During my review, I discovered that a fixed value was being used for the IV. SMAES uses the Unity SystemInfo.deviceUniqueIdentifier as the IV for all encrypted transmissions, something that whilst unique per client, is not going to be random and unique on each use. It Is my belief that they did this as a workaround for the fact they are using a shared key on all of the clients, no matter, there is no justifiable reason to implement a fixed, non-random IV as it dramatically weakens the cryptographic protection of AES.
For those of you interested, according to the Unity Documentation, SystemInfo.deviceUniqueIdentifier is a unique device identify that Unity will determine differently, depending upon the client’s platform:
- For IOS:
- Prior to iOS 7: hash of MAC address (the WIFI adapter).
- After iOS 7: UIDevice identifierForVendor or ASIdentifierManager advertisingIdentifier.
- Windows Store Apps: AdvertisingManager::AdvertisingId or HardwareIdentification::GetPackageSpecificToken().Id.
- Windows Apps: A hash of the following identifiers combined:
- Android: I don’t see any documentation, however on the forums I found that the logic has changed a number of times. It currently appears that it could be: IMEI/MEID/ESN or ANDROID_ID (a random string that is created when you first setup an Android device) or MAC address.
- Unsupported Systems: SystemInfo.unsupportedIdentifier.
If you are not convinced about the risks of a poorly selected IV, consider the 802.11 encryption algorithm WEP (Wired Equivalent Privacy). In WEP, a short 24 bit IV was used, this wasn’t enough, leading to IVs being reused with the same key, and led to the key being easily cracked.
Device ID as Encryption Key
Now we get to the most shocking, and unbelievably foolish issue that I discovered during my review.
To improve security, the developers included a mechanism for the client and server to synchronize a new encryption key, as I previous said, to alleviate the issues of having a single shared key. Unfortunately, in their attempt to do so, they introduced what is one of the most significant vulnerabilities in the product, one that makes man-in-the-middle attacks trivially easy.
The developer’s goal was to have a function which a client could call, and have the server provide a new key for the AES encryption, except a new vulnerability is introduced. When the Client calls the function syncKey() (shown in the image below), AES is reconfigured with the device ID being used for BOTH THE IV AND AS THE KEY. The client sends the device ID and the original shared key to the server, just as any other call, and the server generates a new key sending it back to the client with the device ID as the encryption key. Yes, we are going to send our encryption keys in plain text now.