In the previous article of the Certificates and Security series, we spoke about initiatives the industry has taken to make the web a safer place. In this installment of the series, we will explore how public datasets and open-source tools developed over the past two decades have played a significant role in making the web a more secure place.
Harnessing the Power of Public Datasets
Public datasets have proven to be valuable resources for researchers, private companies, and universities worldwide. These datasets, particularly those related to TLS (Transport Layer Security) certificates, have allowed experts to delve deep into the intricacies of web security.
Here’s a look at a few of the notable projects that have made a positive impact on certificate security:
While some of the projects are relatively new and still active, others are not being maintained actively, or have morphed into commercial offerings.
- SSL Pulse –This project scans all the websites listed in Alexa’s top website index for quality/support of SSL. It publishes monthly reports, aiding researchers and security professionals in monitoring the state of SSL security.
- Censys – This started as a research project at Michigan university and has transformed into a commercial entity offering several paid services. However, it continues to share its dataset of 5 billion certificates with researchers for free to support ongoing research efforts.
- Scans.io : An initiative from Stanford, this is a public archive of research datasets including Censys dataset mentioned above.
- Certificate Transparency logs: These can be a rich source for getting interesting insights and early warning of emerging threats. The simplest way to access Certificate logs is by using websocat.
If you have/use websocat, running the following command will get you the logs.
websocat -t - autoreconnect:wss://certstream.calidog.io/ Another interesting site to explore CT is cloudFlare’s MerkelTown
- https://crt.sh/: This is a certificate search engine that allows you to search/filter on various fields like
- EFF SSL observatory: While not actively maintained in recent years, in its prime days, this project focused on unearthing issues with certificate authorities.
SSL/TLS Tools
Regularly testing your SSL/TLS setup is important to identify and address potential vulnerabilities and weaknesses. There are several open-source/free tools available in the public domain to help you ensure the proper configuration and security of SSL/TLS certificates and servers. Here are a few popular ones:
- SSL Server Test: This is one of the most widely used tools for testing SSL/TLS certificates and server configurations. It provides a comprehensive assessment of your server’s SSL/TLS setup, including protocol support, key exchange, cipher strength, and vulnerabilities.
- Mozilla Observatory : Mozilla Observatory is another web-based tool that checks your website’s security headers, SSL configuration, and other security-related settings.
- TestSSL: A free command line tool, it performs various SSL/TLS checks, including cipher suite evaluation, certificate expiration, and security vulnerabilities. Several tools like Burp suite package TestSSL as part of their offering.
- SSL Scan: SSL Scan is a command-line tool that tests SSL/TLS-enabled services for supported protocols and ciphers, as well as vulnerabilities like Heartbleed.
- OpenSSL: OpenSSL is a comprehensive library related to SSL providing functionality to generate CSRs, create private keys, install certificates. It also has options to test SSL/TLS connections, certificates, and ciphers. For example, you can use the s_client option to initiate a connection and inspect the certificate chain.
- Nmap: It is an open-source tool for network exploration and security auditing. It was designed to rapidly scan large networks, although it works fine against single hosts. While Nmap is primarily known as a network scanner, it can be used to scan for SSL/TLS-related information. The ssl-enum-ciphers script can help identify supported SSL/TLS versions and cipher suites.
- zmap: is a collection of open-source projects focused on building tools for doing large scale studies of services and hosts that are part of the public internet. Zcertificate which is part of the zmap project is a command line utility for parsing SSL/TLS certificates. Another tool zLint is developed to check consistencies of certificates with standards (e.g. RFC 5280) and other relevant PKI requirements (e.g. CA/Browser Forum Baseline Requirements).
- Wireshark : Wireshark is a powerful network protocol analyzer. While it doesn’t directly decrypt SSL/TLS traffic without the private key, it can still capture and analyze SSL/TLS handshake data and certificates.
- Bettercap : Bettercap is a Swiss Army knife for network attacks and monitoring. Often used by Red Teams, it can perform SSL/TLS stripping attacks and analyze encrypted traffic.
- Ettercap : Ettercap is a comprehensive suite for man-in-the-middle attacks. It can intercept SSL/TLS traffic and provide red teams with insights into encrypted communication.
Wrapping Up
The web’s security landscape has been greatly fortified by the availability of public datasets and open-source tools. It’s thanks to them that researchers, security professionals, and organizations worldwide today have the resources they need to understand, monitor, and enhance SSL/TLS security.
The breadth and depth of these datasets and open-source tools are an important reminder of how much it takes to make – and keep – the web secure for all of us.