Solarwinds Source Code Breach - How to Protect Your Source Code Management Platform

If you’re a security professional, by now, you've already heard about the epic Solorigate or Sunburst breach.

The massive hack was exposed in mid-December 2020 (ah 2020; the “gift” that just keeps on giving…) and compromised numerous high-profile companies and government organizations. Security giant FireEye first discovered the widespread breach which resulted in obtaining code developed by the company’s Red Team to simulate cyber attacks, as well as breaching and exfiltrating data from many other organizations.

While this sophisticated, multi-stage breach is still under investigation, federal institutions along with the international cybersecurity community already have a pretty good idea about how this breach occurred; in a nutshell, the adversary (allegedly a nation-state actor) managed to first submit malicious source code into Solarwinds Orion product suite, one of the most prevalent IT monitoring solutions. The malicious code created a backdoor inside that product and once installed in a customer network, the compromised server contacted its command and control center in order to receive instructions. These instructions were capable of privilege escalation, downloading and executing payloads, moving laterally throughout the network, and compromising other assets.

Securing Your Source Code is Essential

The topics of how exactly Sunburst/Solorigate spread across networks and how organizations can detect it have already been covered in many great articles (here’s Microsoft analysis and recommendations). In this article, we are going to touch upon the root-cause for Solorigate to provide infosec and corporate security teams some practical recommendations on how they can better secure their source code management platforms.

Traditionally, source code management platforms are owned and managed day-to-day by development teams and the reality is that security aspects and controls are, at times, deprioritized. This breach serves as a painful eye-opener -- corporate security teams must take a stand and emphasize the critical nature of security with their respective counterparts--and then, make sure they do everything possible to harden and secure their source code and version control platforms.

Practical Tips to Secure Source Code Platforms

Version control platforms have greatly matured in recent years and now natively offer many controls which can be easily implemented and don’t require any additional tools. For the purpose of this article, we’ve used controls available in GitHub and its respective terminology, since this is one of the most prevalent source code platforms. Nevertheless, most of these controls are also available in some shape or form in other products.

Platform-specific security controls

• Sign your commits using GPG keys - Committing code to a repository can be spoofed quite easily using the command line.
• Restrict the number of users who can create repositories and projects.
• Review your repositories and make sure none of them are anonymously accessible (public).
• Disable forking from your private repositories (should be all of them).
• Use SSH certificates to push code - SSH keys authenticate trusted computers, without involving passwords.
• Periodically rotate personal access tokens and SSH keys to minimize impact of leaked out keys.
• Activate automatic security scanning and get alerted when a new vulnerability is found in one of your dependencies.
• Review any third-party apps that have access to your source code platform. You can use the following set of questions to categorize such apps:

1. What access level/s does it have? (for example, is it limited to low sensitivity data / read-only, etc.) - you can collect these based on the OAuth scopes that each app has.
2. Which automation can such apps initiate?
3. Who is the user who approved them and for what reason? Are they still needed?
4. If a third party is compromised it puts you at risk as well, so verify the authenticity of the App author.

• Review your activity audits login logs to find sensitive activities such as:

1. repo.create
2. repo.add_member
3. integration_installation.create
4. repository_vulnerability_alerts.enable
5. repository_dependency_graph.enable

Secure Code

• Remove any sensitive data such as credentials, secrets, configuration variables, and any other breadcrumbs that would help an attacker. Use secret management tools like vault or git-secrets.
  In case you discover sensitive data, removing it is not enough as git saves all of the repository histories, utilize 'purge file' from your repository history.
  git filter-branch --index-filter "git rm -rf --cached --ignore-unmatch path_to_file" HEAD
• If you wish to contribute an open source project to the community, make sure to completely separate it from your organization's projects and repositories.
  Many organizations have accidentally exposed legitimate credentials in publicly accessible repositories.
• Be very cautious with open source packages - We all copy and paste, importing and cloning code from external sources - be sure to perform full audits on the imported source code.

General Controls:

• Implement Single Sign-on - SSO provides organization owners with a way to control and secure access to GitHub resources like repositories, issues, and pull requests. Additionally, connecting user deprovisioning that’s initiated by your IAM solution ensures that when an employee leaves your company, you can rest assured that they'll be de-provisioned from your source code as well.
• Enforce MFA for your organization admins - usually, admins can bypass SSO with user/pass credentials for resilience reasons -  prevent easily accomplished user credential theft and access to your projects.
• Make sure no users outside your organization have admin permissions.
• Set strong password policies.
• Identify and remove inactive users to reduce your attack surface.
• Review elevated privileges (e.g., delete projects, change repository's visibility to ‘public’) and constantly and adopt least privilege policy. Give contributors access only to the data they need to perform their work.

• Restrict who can invite and approve new contributors.
• Monitor newly granted admins and privileged users.

• Ensure you’ve implemented zero trust access mechanisms for accessing your source code platform; for instance, not only user authentication but also ensure that the network and device that’s connecting at any given moment are validated and trusted.

In conclusion

Aside from the immediate impact Solorigate has had on many organizations, as well as the tremendous efforts that are being made to identify and recover from this breach, we truly believe that this a learning opportunity; as Winston Churchill said, “Never let a good crisis go to waste.”

While most organizations face far less sophisticated attacks than nation-state backed ones, attacks are always growing more complex and advanced. This is why it’s critical to get back to the basics, such as applying preventive security measures, hardening all sensitive platforms, and continuously monitoring them, to reduce the chances of getting breached. And while we’ve only examined one facet of the Solorigate breach and a single attack vector out of many, we should all take this opportunity to do what we can to contain the blast radius of such breaches, if and when they occur.

Check out how Adaptive Shield can help you protect your SaaS apps using continuous monitoring of their configurations.