Yes, Clearly AI has our SOC 2. But that is not the purpose of this blog post 🙂

At Amazon, AppSec engineers are generally insulated from other components of security. I knew that central enterprise security, GRC, and SOC (Security Operations Center, yes… another SOC) existed, but I never really had to interact with them. So, my first experience with SOC 2 happened at Moveworks, where I was on a 7-person security team responsible for the full security, privacy, and compliance posture of an enterprise AI product. 

At Moveworks, we had an awesome GRC lead. She managed 9 different certifications, including the incredibly intense FedRAMP program. SOC 2 was just one piece of the puzzle, and many of the SOC 2 specific requirements involved robust policies especially around HR (e.g., clear job descriptions and performance reviews). Naturally you can imagine my surprise when I get to Y Combinator and everyone is talking about getting their SOC 2 certification.

I thought to myself: Why does a two-person startup need performance reviews or background checks? Or policies for pull requests and change management when there’s often only one person contributing to the codebase?  

As a security engineer I remember telling my batchmates it makes no sense for them to spend the time to get a SOC 2 during YC. Instead they should be thinking about building their product secure by default! Set up a WAF! Change your default database credentials! Use a password manager and MFA! Why are we putting the cart before the horse?

For early-stage startups not yet ready for formal compliance, foundational security practices are key. Prioritize implementing strong access controls, securing your development environment, regularly scanning code for vulnerabilities and ensuring all sensitive data is encrypted. These steps build a secure foundation that can be scaled later.

I even chatted with other security and GRC experts and they all agreed! When your company has just two co-founders and an MVP, a library of policies isn’t really important. So I followed the advice of one expert who eventually became our SOC 2 auditor at Clearly AI: hold off and right-size your controls and just be a good security engineer. Start adding in pre-launch testing and code scanning now! 

Yet, as a startup founder you need a SOC 2 to successfully sell to mid-market companies and above. There are tons of companies selling “SOC 2 Readiness,” and even more that require a SOC 2 just to enter the procurement process. Teams rush to get SOC 2 to close deals in time for Demo Day.

The advice from my security community (“WAIT!”) alongside the startup community (“RIGHT NOW!”) broke my brain. What’s the gap here? This blog will explore the nuances of SOC 2, its impact on vendor risk assessments, and how we can help bridge this divide to foster a more effective security ecosystem.

What is SOC 2?

Let’s take a step back. What is SOC 2, and what does this compliance framework entail? 

SOC stands for System and Organization Controls and is a framework from the AICPA; the American Institute of Certified Public Accountants, a national professional organization for Certified Public Accountants (CPAs). The three reports (SOC 1, SOC 2, and SOC 3) each focus on different types of controls. SOC 2 relies on 5 Trust Services Criteria: Security, Availability, Processing Integrity, Confidentiality, and Privacy. Generally, when folks colloquially say “I have a SOC 2” they mean they were evaluated on the Security controls either at a single point in time (Type 1) or over a trailing window (Type 2).

SOC 2 is an incredibly flexible framework, which works wonders when used properly, but is frustrating to GRC professionals when abused. Generally, there’s a set of criteria, and organizations can set any reasonable controls to meet those criteria. Therefore, what SOC 2 auditors are assessing is the effectiveness of the stated controls. As an organization, that means building custom policies that align with how you work, and then providing evidence that you are following your own policies. The final audit report has a detailed description of your system, your controls, and an auditor opinion that calls out any gaps.

In the best case scenario, your SOC 2 looks a lot like ours.

We performed a gap assessment with our auditor to discuss where we had missing policies or insufficient controls (or couldn’t provide evidence that we actually met those controls), and then right-sized our controls to our company profile, the customer data we handle, and the customer population we serve. 

In the worst case scenario, you buy a SOC 2 readiness platform, and then spend countless engineering and founder cycles “turning the checkmarks green” without really knowing why you need each control. You could then get “annoying” auditors that don’t trust the output of the readiness platform and still ask you to go collect screenshots as evidence. And your final report has an exception because you missed something! You can be left feeling that all of that money and resource planning were for naught.

The commoditization of SOC 2 has led to a mindless “one size fits all” approach to a framework that was designed not to be that way. The worst part of this is the “SOC 2 infection” where an organization puts as a control for Trust Services Criteria CC9.2: “The entity assesses and manages risks associated with vendors and business partners,” that they review the SOC 2 of each of their vendors. In this case, the organization can get stuck. They must require all of their vendors to be SOC 2 certified. Now this organization has painted themselves into a corner. A SOC 2 might not be the right fit for each of their vendors. Would you prefer your new startup vendor spend 30+ hours on security policies and processes to “check the box” or prioritize security controls that fit how you are using the vendor in your environment?

How SOC 2 Impacts Vendor Risk Assessments

Trust Services Criteria CC9.2: “The entity assesses and manages risks associated with vendors and business partners”, is a good one to demonstrate impact. 30% of security breaches occur due to third party or vendor compromise. The overall criteria 9.0 “Common Criteria Related to Risk Mitigation” demonstrates that the goal of CC9.2 is to mitigate risk. Performing a Vendor Risk Assessment is crucial to understanding the risk posture of a third party vendor, and effectively mitigating that risk appropriately.

A key aspect of risk mitigation is appropriately evaluating the risk of a given vendor, and assessing them accordingly. The depth of assessment should be directly related to the risk the vendor possesses to your organization based on their access to your network, data, and product availability. 

One overwhelmed TPRM leader complained to me “Why do I need to assess the security of our caterer?!” The answer is – you shouldn’t. So, what’s the impact if your caterer’s network is compromised? Maybe credit card fraud if they stored PCI inappropriately, but presents minimal risk. Instead of spending time conducting due diligence on the caterer’s IT infrastructure, why not create a virtual credit card with a set limit that can only be spent at the caterer? Threat mitigated.

The caterer sounds like a toy example. It’s not. TPRM teams are drowning in security reviews of vendors that, based on the risk model, shouldn’t require security reviews. A good SOC 2 auditor / GRC leader at your organization can and should prevent that with appropriate risk-based policies.

What about a startup that handles some internal sensitive information that could have moderate impact if leaked? Probably worth evaluating their security posture. If they have a SOC 2 report, great! The TPRM team can use the SOC 2 to determine alignment with the controls their organization cares about. Without a SOC 2 report, what information does the TPRM team have to determine the security posture of that startup?

Enter the dreaded security questionnaire.

How SOC 2 and Security Questionnaires Go Together

We all hate security questionnaires, but let’s give the inventors of security questionnaires the benefit of the doubt for a moment… The ideal security questionnaire asks all the questions that are important to a TPRM team to determine the overall risk posture of a new vendor. Questions are calibrated based on the type of data the vendor handles, the integrations into the buyer’s environment, and the buyer’s risk tolerance. Armed with these answers, the TPRM team should be able to make astute judgments around what additional controls may need to be implemented on their side to mitigate risk sufficiently.

We believe in the original goals of the security questionnaire, generally. It’s important to have a risk-calibrated runbook of the information needed to make an informed decision. But, many organizations have taken this too far by:

1. Asking the same questions of every vendor, regardless of risk profile
2. Repetitive, confusing questions that use GRC language instead of technical language, and
3. Enforcing the same requirements and standards of every vendor, regardless of whether they interact with the organization's sensitive data or not.

To me, the ideal value of a SOC 2 is to avoid scenario #2. Instead of having to answer questions that are often aligned closely with a Common Criteria, let me just send you the SOC 2. What’s even better? A third party auditor actually verified that I’m doing what I say I’m doing. Huge bonus over the self-attestation of a vendor-completed questionnaire. As an aside, Clearly AI now categorizes evidence by trust level, as a way to highlight any conflicts between vendor-provided and auditor-prepared information.

The intended goal of Trust Centers was to avoid the above scenarios by providing all the information to the GRC team in one place and saying “pull the information you care about.” Yet, that puts more work on the plate of the GRC team to sift through the piles of information to find the answers to the things they care about. Especially when they need to “request permission” to view the SOC 2 or other related documentation. By the time the permission is granted? They’ve moved on to another ticket in their queue…  So often, teams just stick with the 200+ questions instead.

We designed Clearly AI to get back to the original mission of a security questionnaire. 

The TPRM team tells us what they care about. It can be 200 questions or only 10. They can use one of our out-of-the-box templates, their own custom list, or an industry standard (e.g., CAIQ). We can ask different questions depending on the vendor risk profile, integrations, and product.

Then, we ask the vendor for all their content. If they have a SOC 2, great! If they don’t, they give us their policies, whitepapers, and onboarding documents. Give us any information about your platform, and even better if it’s verified or verifiable. If there are gaps, we’ll ask the vendor for more information. Our goal: To give teams the answers they need to appropriately assess risk. When vendors only get a few, targeted questions they answer them more thoughtfully. If they’re getting the same questions from multiple buyers? Probably a good idea to document the answers in a policy or procedure. 

Yes, there are plenty of “GRC” tools out there with the goal to automate the security questionnaire process. But they’re missing the forest for the trees. By auto-filling the huge questionnaire using AI, they are acting as a sales enablement tool, not a GRC tool. Going through the question-answering process can, and should, be a way for startups to think more deeply about their security posture, but many GRC tools are turning the process into a checkbox exercise to make the sale; and not a thoughtful alignment with your organization's controls.

This “race to the bottom,” results in a system where compliance becomes a superficial hurdle rather than a genuine measure of security, ultimately leading to less secure vendor relationships and a false sense of security for buyers. 

GRC is Governance, Risk, AND Compliance, not JUST Compliance

To sum it up, modern GRC has been trending towards only thinking about compliance instead of risk. The best companies align their internal governance processes towards a risk-based approach. Because SOC 2 is all about how well you execute on your own policies and controls, the SOC 2 compliance of these companies is calibrated towards risk as well.

When startups begin to think about GRC as an important part of building a strong company, and not just a blocker in the sales process, magical things happen. They see each new customer’s third party risk evaluation as a way to pressure test the risk-based policies they’ve put in place.  

For example, Clearly AI went through the TPRM process with a Fortune 500 Fintech before we finished our SOC 2 (this was about 10 months ago). We had most of our policies and procedures in place, but were missing a background check policy. At the time, Clearly AI was just the two of us co-founders. And… we’re married. I’ve known Joe for almost a decade. As a SecEng, when thinking about the security risk of Clearly AI, insider threat was literally the last thing on my list. But, the TPRM team explained it to me: They don’t know us. They sent us their template for a background check policy, and asked for us to run background checks on ourselves aligned with their policy. Instead of the exchange being adversarial, it was a great learning opportunity and a way for Clearly AI to improve our own posture. Now, when I complete background checks on new hires, I know we’re completing all of the appropriate due diligence.

The only way for us to build a better TPRM ecosystem is for enough buyers to change what they ask of their vendors, especially startups. More TPRM teams need to ask focused risk-calibrated questions and use SOC 2s not as a “must-have check-box” but as an easy way to answer most questions about a company’s posture.

Buyers should actively engage with vendors to understand their unique security context, leverage SOC 2 reports as a starting point for deeper discussions, and prioritize a vendor’s actual security practices over mere certifications. 

At Clearly AI, our goal is to help you thoughtfully review your vendors at scale.  Want to learn more? Book a call with us.

Thank you to Brooke Bowman for reviewing a draft of this blog!

‍