Tag: #cybersecurity

Are We Peak CISO?

Let’s be honest…the CISO role is weird right now. It is going through a transformative phase and the industry is at an inflection point similar to what other C-Level roles (like the CFO) have gone through in the past. What makes the role weird? The CISO community and any company that has a CISO is facing unprecedented regulatory pressure, the economy and interest rates have people on edge, layoffs in the tech sector have shaken employee confidence (to the applause of investors) and technology innovation via AI is causing additional disruption and risk across all sectors.

In additional to these external pressures the past few years have seen the proliferation of CISO title sprawl and confusion from companies about how to best employ and utilize a CISO (hint, we aren’t your scapegoats). Despite all of this turmoil, change is also a time for opportunity and there are a few things I think will help clarify and mature the CISO role.

CISO Title Sprawl

I’ve been tracking job titles and job postings on LinkedIn for the past year or so and I’ve noticed a phenomenon I’ll call title sprawl. A quick search for titles shows there are vCISOs, Advisory CISOs, Fractional CISOs, CISOs In Residence and Field CISOs. On top of this, add in Chief Security Officers, Chief Trust Officers and Heads of Security. Do we need all of these titles? Maybe, but I think this title sprawl is more indicative of three things 1) People with CISO titles are in high demand and people want to retain the title once they get it and 2) Companies are still uncertain about how to title and employ someone to lead their security function. 3) Title sprawl is a result of the political power struggle occurring between the CISO role and other C-Level roles (more on that below).

From the titles above there are really only four functions for a current or former CISO – board member (in some capacity), executive management (officer of the company), consultant and sales. There is similar title sprawl and variance with CTO titles, but not to the extent of the CISO title (yet). Time will tell if other C-Level roles start to follow suit, but for now, let’s break down the functional CISO role buckets.

Board Member – These are current or former CISOs who sit on a board either as a technical advisor, business advisor or some combination thereof.

Executive Management – Individuals employed by a company to lead the information security program. May also manage other parts of IT such as identity, privacy, data, etc. Titles may be CISO, CSO, CISO in Residence (for Venture Capital), Chief Trust Officer and Head of Security.

Consultant – These are individuals who are providing their expertise as a current or former CISO to other companies to help them establish, transition or manage a security program. Often the companies employing these individuals claim they can’t afford a full time CISO, but they seem to be able to afford other full time C-Suite titles (hmm…)? Titles may include Virtual CISO (vCISO), Fractional CISO, CISO in Residence and Consulting CISO. (CISO in Residence again because they can “consult” to their VC holding companies about the state of their security programs).

Sales – These are people who are experts in the field of security, may hold one or more certifications and may be past CISOs. Their job is to help the company they work for drive sales. Typically the title they use is Field CISO or Advisory CISO.

Standardize The Reporting Structure

Moving on from title sprawl, companies are also confused about where the CISO title should sit. Some companies advertise it as a Director level role reporting into the VP of some function. Other’s title it as a VP level role reporting into a Senior VP or some other executive. Still other companies have the CISO reporting to the CEO, CIO, CTO or General Counsel. It is even possible this person is an individual contributor. Companies are clearly confused about whether the CISO is a technologist, regulatory compliance specialist or true C-Suite executive. While reporting structure may be a direct reflection on company culture, it is also a public example of the battle for equivalency that is playing out between the CISO and other C-Level roles. Often, CISOs are hired by other C-Levels (not the CEO) and until it becomes more common for CISOs to report to the CEO as an accepted peer to other C-Levels, this confusion and variance will persist. That being said, if you are considering a CISO title and the company isn’t willing to add you to the D&O liability policy then you may be better off taking a lower level title to eliminate personal risk.

Bolster Security Management Certifications

Security certifications from popular organizations talk a lot about regulations, risk and different security concepts (technical or not), but few, if any, offer a comprehensive certification on what it truly takes to be a CISO. Any CISO level certification should include potential career paths that lead to the CISO role, career paths post CISO role, difference in the CISO role based on company size, exposure to business topics in addition to security topics, SEC reporting, interfacing with law enforcement and lastly discussion of how to maximize success based on where the role sits – e.g. reporting to the CEO, CTO or CIO and how that may change your lens as a CISO. This begs the question if there should be a true professional level CISO certification similar to a professional engineer, accountant or lawyer, but let’s save that discussion for a future blog post.

Embrace Increased Regulation

Given the recent increase in regulation, particularly from the SEC, bolstering CISO certifications to include more business acumen may soon be table stakes instead of a nice to have. Recent regulations forcing companies to disclose material cybersecurity events in their 8k filings are starting to accelerate the maturity of the CISO role at publicly traded companies. Companies can no longer fail to invest in security or report breaches (unless they want steep penalties). In particular, this is forcing the CISO role into the board room or at least on par with other C-Level roles because they have to help these companies navigate the decision to report material events in their filings. Existing and future CISOs can embrace this increase in regulation to backstop their authority at companies who are struggling to fully embrace the CISO role as a C-Level executive. While it may not elevate the current role with a promotion, it should at least open the door to the board room and provide a seat at the table for discussion.

While CISO reporting structure may be a direct reflection on company culture, it is also a public example of the battle for equivalency that is playing out between the CISO and other C-Level roles.

The last point I’ll make about regulation is – while the SEC watered down the requirements for cybersecurity expertise on boards, I predict this expertise will still be required and in demand as companies start to navigate the new SEC reporting requirements. In particular, companies may be penalized and eventually required to demonstrate cybersecurity board expertise (via experience or certifications) if they are found to have a material security breach and can’t demonstrate appropriate security governance at the board level.

What’s The End Result?

It is clear the security industry and the CISO role are in a state of confusion as a result of the tight job market, uncertain economy, increased regulation and pace of technology innovation. The net effect of title sprawl and the struggle for equivalency is – it confuses customers, investors, partners, recruiters and job candidates. Title sprawl artificially increases competition for jobs and causes a wide variance in how the CISO role is employed. However, I think this state of confusion is a good thing because it is forcing conversations and causing people to stop and think. The CISO role is the newest member of the C-Suite and it is growing up and trading in the hoodie for a collared shirt. We are starting to claim our seat at the board level and are able to hold our own or make other C-Level roles redundant. As the CISO role evolves from a “nice to have” to a “must have” in the C-Suite, we will see this confusion fade away and the CISO role will truly reach its peak.

What’s The Relationship Between Security Governance and Organizational Maturity?

Organizational and security governance is touted as a key component of any successful security program. However, I’ve been thinking about governance lately and how it relates to the overall maturity of an organization. This has prompted some questions such as: what happens if you have too much governance? and What’s the relationship between security governance and organizational maturity?

What Is Governance?

First, let’s talk about what governance is.

Governance is the process by which an organization defines, implements and controls the business.

Let’s unpack what this means for a security organization. The process of defining security for the business is done through policies, standards and guidelines. Security policies are requirements the business must meet based on laws, regulations or best practices adopted by the business. These policies align to business objectives. Implementation is done through security controls that are put in place to meet a specific policy or to manage a risk. Lastly, controlling the business is done via audits and compliance checks. The security org follows up on how well the business is following policies, implementing controls and managing risk. Control can also include enforcement, which can involve gating processes, such as requiring approval for business critical and high risk activities, or recommending additional security requirements for the business to manage a risk.

Why Do We Need Governance At All?

In an ideal world we wouldn’t. Imagine a business that is created entirely of clones of yourself. There would be implicit and explicit trust between you and your other selves to do what is best for the business. Communication would be simple and you would already be aligned. In this case you don’t need a lot (or any) governance because you can trust yourself to do the things. However, unless you are Michael Keaton in Multiplicity, this just isn’t a reality.

Governance achieves a few things for a business. First, it communicates what is required of its employees and aligns those employees to common objectives. Second, it helps employees prioritize activities. None of this would be needed if human’s weren’t so complex with diverse backgrounds, experiences, perspectives, education, etc. In an ideal world we wouldn’t need any governance at all. The reality is, we do need governance, but it needs to be balanced so it doesn’t unnecessarily impede the business.

How Does This Relate To Organizational Maturity?

Organizational maturity refers to how your employees are able to execute their tasks to achieve the objectives of the business. This relates to things like the quality of code, how quickly teams resolve operational issues or how efficiently they perform a series of tasks. It can be loosely thought of as efficiency, but I actually think it combines efficiency with professionalism and integrity. Maturity is knowing what good is and being able to execute efficiently to get there. There is a fantastic book about this topic called Accelerate: The Science of Lean Software and DevOps: Building High Performing Technology Organizations by Nicole Forsgren PhD.

Which brings us to the relationship of governance and maturity…

There is an inverse relationship between organizational maturity and organizational governance. In simple terms:

The less mature an organization, the more governance is needed.

For example, if your organization struggles to apply patches in a timely manner, continually introduces new code vulnerabilities into production or repeatedly demonstrates behavior that places the business at risk, then your organizational maturity is low. When organizational maturity is low, the business needs to put processes and controls in place to align employees and direct behavior to achieve the desired outcomes. In the examples above, increased governance is an attempt to manage risk because your employees are behaving in a way that lacks maturity and is placing the business at risk.

What causes low organizational maturity?

Organizational maturity is a reflection of employee behavior, skillset, knowledge, education and alignment. In other words, organizational maturity is a reflection of your organizational culture. In practical terms your employees may simply not know how to do something. They may not have experience with working for your type of business or in the industry you operate in. Perhaps they had a really bad boss at a past job and learned bad behavior. Whatever the reason, low organizational maturity is linked to lots of sub-optimal outcomes in business.

How To Improve Organizational Maturity?

If governance and maturity are inversely linked, the question becomes how can we increase organizational maturity so we need less governance? There are a lot of ways to increase organizational maturity. One that is fairly obvious is to start with a mature organization and maintain it over time. However, this is easier said than done and is why some organizations are fanatical about culture. This relates to everything from hiring to talent management and requires strong leadership at all levels of the company.

“The less mature an organization,
the more governance is needed.”

Other ways to improve organizational maturity are through training and education. This is why security awareness and training programs are so critical to a successful security program. Security awareness and training programs are literally attempting to improve organizational maturity through education.

One last way to improve maturity is via process. The security organization can establish a new process that all teams must follow. As teams go through this process you can educate them and reward teams that exhibit the ideal behavior by relaxing the process for them. You can also help teams educate themselves by publishing the requirements and making the process transparent. The challenge with imposing a new process is having the discipline to modify or remove the process when needed, which comes back to governance.

What’s the right level of governance?

The optimal level of governance is going to be based on your organizational maturity and desired business outcomes. In order to determine if you have too much or too little governance you need to measure organizational maturity and the effectiveness of existing organizational governance. There are industry standard processes for measuring organizational maturity, like the Capability Maturity Model Integration (CMMI) and Six Sigma, or you can create your own metrics. Some ways to measure governance effectiveness are:

  • Ask For Feedback On Security Processes – Are the processes effective? Do teams view them as an impediment or are they viewed favorably? Are the processes easy to navigate and objective or are they opaque and subjective?
  • Measure Effectiveness Of Security Controls – Are your security controls effective? If you ask a team to do work to implement a security control you should have clear metrics that determine if that control is effective. If you implement a control, but that control hasn’t changed the outcome, then the control is ineffective. This can indicate your governance is ineffective or your organizational maturity needs to improve.
  • Assess and Update Policy – Security policies should be living documents. They shouldn’t be set in stone. Security policies need to map back to laws and regulations they support and the business requirements needed to be successful. Laws, regulations and business requirements all change over time and so should your security policies. By having up to date and relevant security policies you can ensure your organizational governance matches the maturity of the business.

What Are Typical Scenarios For Governance And Maturity?

There are four scenarios related to governance and maturity:

A mature organization with too much governance – your organization is mature, but you are overly controlling with process and requirements. The net effect will be to slow down and impede the business unnecessarily. You are effectively lowering the organizational maturity due to too much governance.

An immature organization with too little governance – this is a recipe for disaster. If your organization is immature and you fail to govern the organization you will open the business up to unnecessary risk. You will get out maneuvered by your competitors, you will miss opportunities, you will fail to comply with laws and regulations and generally will have a lot of activity without any result. Your employees will lack coordination and as a result your business will suffer.

A mature organization with too little governance – This isn’t a bad scenario to be in. A mature organization implies they are doing the right things and don’t need a lot of guidance. A laissez faire attitude may be the right thing to allow employees flexibility and freedom, but it does come with inherent risk of not being compliant with laws and regulations. It may also mean there is duplication of effort or multiple ways of doing things, which could be optimized.

Governance and maturity are balanced – obviously this is the ideal scenario where your organizational governance is balanced to the level of maturity of the organization. Easy to think about in practice, difficult to achieve in reality.

Wrapping Up

Organizational governance and maturity are inversely related and need to be balanced in order for the business to operate effectively. There are ways to measure organizational maturity and governance effectiveness and by having a continual feedback loop you can optimally align both for success.

The Dichotomy Of Security

If you have ever read Extreme Ownership or The Dichotomy of Leadership by Jocko Willink, then you will be familiar with the concept of dichotomy and how opposing forces of a skill set can compliment each other. Mastering both sides can allow flexibility and increase the effectiveness of that skill set when dynamically applied to a given situation. This is true in the security space, where fundamental opposing forces need to be balanced in order to manage risk and achieve success. Let’s take a look at a few examples.

Security Extremes

The easiest example of the dichotomy of security is to look at the extremes. Security professionals jokingly say the most secure company is one that is not connected to the internet. While this may be true, it will also prevent the company from conducting business effectively and so the company will cease to exist and security will no longer be needed.

On the other end of the spectrum there is the extreme of a business that has zero security and so there are no impediments to conducting business. While this may sound great to some, the reality is the company will be unable to effectively conduct business because of the real threats that exist on the internet. In the situation the company will also cease to exist because they will be hacked into oblivion.

It is obvious there is a dichotomy between no security and no connectivity and these forces need to be appropriately balanced for a security program to be effective, while allowing the business to operate.

Manual vs Automated Security

Another example of dichotomy is between manual security tasks and automation. While every CISO I know is striving to increase automation of security tasks, the reality is humans are still going to be needed in any security program for the foreseeable future.

Manual tasks are ideal for situations where humans need to demonstrate creativity, intuition or make complex decisions based on subtle context. Security functions like penetration testing, threat hunting, red teaming and offensive security require high amounts of skill and experience that automation, like AI, hasn’t been able to replicate. Additionally, soft skills such as reporting to the board, shifting culture, building alliances and making prioritization decisions are all extremely complex and unlikely candidates for automation. However, while manual activities benefit activities that require a high degree of creativity, they are inherently slow and can impede the normal flow of business.

Recently, the advances in automation and artificial intelligence have exponentially increased their usefulness. Automation is extremely useful for offloading repeatable tasks that lend themselves to being programmatically defined. For example, attack simulation products have made huge strides in offloading repetitive tasks of reconnaissance, enumeration, vulnerability assessment and remedial exploitation. We are seeing additional advances in automation related to incident response where events can be correlated and specific activities in an IR playbook can be completed to offload analysts and help focus their attention. AI has also helped to offload lower level operational activities like call centers and help desk inquiries.

While automation may accelerate parts of the business and offload humans from repeatable tasks, it does introduce complexity, which can be difficult to troubleshoot or can cause outright failures. Automation is also rigid because it is only as good as the parameters of the process it is following. This means it can’t think outside of the box or demonstrate creativity. There is also the risk of introducing bias into your processes if your underlying model is flawed.

As you can see manual security processes and automated security processes are opposing forces that need to be balanced based on the skill of your security team and the needs of the business.

The Human Problem

The last dichotomy I want to discuss is the human problem in security. Humans are necessary because of their creativity, diversity and capacity for adapting to an infinite number of situations. However, the flexibility in human nature also presents one of the fundamental security problems – how to you protect against human nature?

The reality is humans are flawed, but in a good way. Threat actors can try to take advantage of these flaws, whether they are logical (like firewall rules) or physical (like human psychology). Humans are essential to every aspect of a business and so we have to figure out how to protect them. The most difficult balance in security is developing a program that is comprehensive enough to protect against human nature without stifling it.

The Security Ideal

The ideal security program will recognize the dichotomy of the security challenges it faces and balance them accordingly. The ideal security program balances security with flexibility. We are seeing this balance manifest in mature security programs via concepts like security guard rails and the paved path. The paved path and guard rails attempt to allow a certain amount of latitude for acceptable behavior, while being rigid enough to protect users and the business accordingly.

Application In Other Domains

The concept of dichotomy is universal across any domain. In fact, this is an area of extensive research in disciplines like mathematics, computer science, military strategy, and economics. Specifically, in the space of network and graph theory there is a concept call max flow, min cut. These are counter principles that are opposite, yet complimentary. If you think of any network (road, supply chain, computer network, etc.) the point of maximum flow across that network is also the point where maximum disruption (minimum cut) can occur. From a military or security stand point you will want to protect the max flow/min cut, but from an attacker stand point, the max flow / min cut, is the area that will require the least amount of effort for maximum damage. Pretty neat!

Wrapping Up

An effective security program will balance the needs of security with the needs business with the ultimate goal of effectively managing risk. A critical skill for any security practitioner is to be flexible and adaptive. Specifically, by recognizing that security issues have two sides to them, security practitioners can demonstrate empathy towards the business and find an appropriate balance that can protect without impeding the business.

Using Exceptions As A Discovery Tool

Security exceptions should be used sparingly and should be truly exceptional circumstances that are granted after the business accepts a risk. In mature security programs the security exceptions process is well defined and has clear criteria for what will and will not meet the exception criteria. In mature programs exceptions should be the exception, not the norm. However, in newer security programs exceptions can be a useful tool that provides discovery as well as risk acceptance.

Maturing A Security Program

One of the first things a new CISO will need to do is understand the business and how it functions. As part of this process the CISO will need to take an inventory of the current state of things so he or she can begin to form a strategy on how to best manage risk. As a new CISO your security program may not have well defined security policies and standards. As you begin to define your program and roll out these policies, the exception process can be a valuable tool that gives the perception of a choice, while allowing the security team to uncover areas of the business that need security improvement. Over time, as the business and security program mature, the CISO can gradually deny any requests to renew or extend these exceptions.

Rolling Out A New Security Process

Another area that is useful to have an exceptions process is when rolling out a new security process. For example, if you are rolling out a new process that will require teams to perform SAST and DAST scanning of their code and fix vulnerabilities before going into production, then allowing security exceptions during the initial rollout of the process can be useful to allow teams more time to adapt their development processes to incorporate the new security process. Allowing exceptions can foster good will with the development team and allow the security function visibility into the behavior and culture of the rest of the business. This can allow the security function and development team the opportunity to collaborate together with the ultimate goal of removing any exceptions and following the process to reduce risk to the business.

Tackling Security Tech Debt or Shadow IT

A common maturity evolution for companies is the elimination of shadow IT. The security function can assist with the elimination of shadow IT by creating an exception process and allowing an amnesty period where the business is allowed to continue to operate their shadow IT as long as it is declared. In reality you are giving the business the perception that they will be granted an exception when they are really giving the security function visibility into things they wouldn’t otherwise know about. This can be a useful tool to discover and eliminate policy exceptions as long as it is used sparingly and with good intent (not punitively).

Documentation Is Key

No matter how you choose to use exceptions within your security program there are a few best practices to follow.

  1. Exceptions should be truly exceptional. If you do grant one for discovery purposes make sure there is a plan to close the exception. Exceptions shouldn’t be the rule and they shouldn’t be expected. Sometimes the rest of the business just needs someone to tell them no.
  2. Time box the exception. Don’t just grant an exception without some sort of end date. The business needs to know an exception is temporary and there should be a well defined plan to make improvements and close the exception. The security team should grant a reasonable amount of time to execute that plan, but it shouldn’t be a never ending story.
  3. Review often. Security exceptions should be reviewed often. Part of your security program should review the open exceptions, which ones are ending, if there are patterns where there are lots of similar exceptions and if there are teams who request a high volume of exceptions. Reviewing exceptions gives you insight into how well security processes and controls are working. It also gives you insight into which parts of the business need help.
  4. Require the business owner to sign off. The reality of a well run security program is the business ultimately owns the decision if they want to accept a risk or not. The CISO makes a recommendation, but they don’t own the business systems or processes. As a result, the security exception process should require the business owner to sign off on any exception. This will ensure there is documentation that they were made aware of the risk, but this can also act as a visibility tool for the business owner into their own teams. I’ve often found a business leader is not always aware of what their teams are doing at the tactical level and the exceptions process can provide them the opportunity to check their team and correct behavior before it gets to the CISO.

Wrapping Up

The exception process can be a valuable tool for discovery of hidden risk throughout the business. By offering an amnesty period and giving the perception of flexibility, the security team can foster good will with the business while gaining valuable visibility into areas that may be hidden. The exception process also is a valuable tool for the security program to document risk acceptance by the applicable business owner, but can also provide business owners visibility into how well their team is meeting security requirements. Lastly, as the security program matures, the security team can gradually require the business to close down the exceptions by improving their security posture.

What happens post incident?

One of the most exciting, stressful and true tests of a CSOs ability to lead during crisis is during a security incident. Unfortunately, it is inevitable that CSOs will experience several security incidents during their tenure. This could be something as small as a configuration error, or as large as a full on public data breach. I also think CSOs should assume they are operating in some state of compromise such as a malware infection or an attacker with complete remote access. Given this volatility of enterprise environments and the inevitability of some sort of security incident, the question becomes what happens afterwards? Just because you are no longer under active attack doesn’t mean your work is done. As a CSO, here are some things you should consider after an incident:

Retro / Post Mortem

First, I highly recommend conducting a retro or post mortem on the incident. This is a blameless session to discuss what happened, why it happened and most importantly what the team learned from the incident and what they are collectively going to do differently. During this exercise the CSO should plan to ask questions and listen a lot. I find being the note taker is exceptionally helpful because it is difficult for me to talk while taking notes. I want to hear opinions and ideas from the team on what happened and how we are going to improve. The result of this retro will likely kick of follow on activities such as increased training, requests for investment, development of new processes, creation of new detections or even additional automation. The point is it is a time to learn and rebuild.

Investment

Never let an incident go to waste. Instead of viewing the incident as a failure, I choose to look it as an opportunity for growth. It is easy to point the finger after the fact, but no environment is 100% perfect and secure. Therefore, after the retro is completed it is important to capitalize on the incident to ask for additional investment to respond to the improvement areas raised in the retro. This could mean asking for additional personnel to respond to events, it could mean additional training to fill knowledge gaps, it could mean a new tool or technology to improve detection and response or new processes to improve responses. As the CSO you need to distill down the retro suggestions into an actionable plan that focuses on the risks along with areas for improvement and investment.

Increased Regulatory Requirements

Unfortunately, there are consequences for having a security incident. For publicly traded companies this can come in the form of increased regulatory requirements such as being required to have an independent 3rd party audit your security practice on a periodic basis. It could also mean fines or reporting the incident as material in your upcoming SEC filings. You may even get inquiries from various regulatory bodies asking what happened and what you are doing to prevent it in the future.

Legal Ramifications

Similar to the regulatory requirements your company may face increased legal pressure as a result of the security incident. This could come from a number of different sources such as: lawsuits from outside entities (such as a consumer group or class action), increased contractual obligations from customers who are now concerned about your security practices, law enforcement investigations and costs for outside counsel who have expertise in your specific business area and are helping your company limit the damage.

Financial Implications

This is probably the biggest area for a company to navigate after an incident. Financial implications can manifest in a number of ways. Here are a few:

Increased Cyber Insurance Costs

As a result of the incident, your company may face an increase in cyber insurance premiums. These premiums may even become so expensive that your company can no longer afford to pay them, or your company could be deemed uninsurable.

Customer Impact

Depending on your business, this could be something simple like contracts not getting renewed and loss of new business or it could be something more material like having to pay to notify all of your consumers, paying for credit monitoring and having to compensate your loyal customers in some way to retain their business.

Market Impact

This is a broad area, but post incident your company could face a decrease in stock price. It may even be more difficult for your company to secure lines of credit because of the business risk. The severity and financial impact of the incident could potentially put your company at risk for M&A takeover or may require the business to declare bankruptcy and look for a buyer who has enough capital to weather the storm.

Budget Impact

Again, this is a broad area, but whenever I have an incident I try to keep track of the number of people involved, number of hours it took to deal with it and the opportunity cost of dealing with this versus doing something else I had planned. As an example, the opportunity cost could be something like “as a result of this incident we had to delay project x for 3 months while we dealt with the incident.” All of this will have an impact to timing, budgets and manpower.

Can You Calculate The True Cost Of An Incident?

Unless your company keeps track of everyone’s time and is able to create a time code entry for each specific incident, I find it unlikely that we will ever know the true cost of an incident. There are many direct costs, but there are so many indirect costs that are hard to estimate. The ramifications of an incident may stretch on for years and may change hands from one CSO to the next. However, I do think a CSO should have enough grounding in business principles to be able to estimate the cost of an incident. This can be useful when gaining support from your other C-Suite peers, when presenting to the board or when making investment requests to the CFO. Having context for all of the ramifications of an incident along with potential areas of growth and improvement can be a valuable story to tell.

Compliance Corner Series: Building A Successful Security & Compliance Program

This blog post is part of the Compliance Corner Series developed in partnership with Milan Patel. This series includes a variety of discussion topics around the intersection of security and compliance. The series includes blog posts, live web streams (with Q&A) and podcasts.

What are the primary factors, considerations, and challenges that a security leader needs to consider to build a successful security and compliance program?

  1. If you were starting a security and compliance program from scratch where would you start?

Lee: I’ve written about starting security programs from scratch in past blog posts, but in my opinion a security leader needs to first select what security controls framework they want to use to inform their strategy, initial assessment and planning. It is rare that a security leader will enter an environment that has zero security and so my suggestion for any leader is to conduct an initial baseline assessment for how well the organization is meeting the security controls within the framework they select. Two of the most popular security control frameworks in the United States are the CIS Top 18 Critical Security Controls and the NIST Cybersecurity Framework. Both of these are excellent frameworks that cover everything a comprehensive security program needs to have along with a recommended prioritized order. It will be difficult for a security leader to start a compliance program without baselining and mapping their environment to a security controls framework because this is the equivalent of never practicing a sport, yet going out and trying to play in a game. Security is the practice and discipline that sets the foundation for successful compliance programs i.e. the sport you are playing. Your security strategy will inform the what skillsets you need to hire, what tooling you need to build or buy and what processes you need to establish to make your security program successful.

Once you have selected a security control framework and have a strategy in place, my recommendation is to then evaluate your business for the markets it addresses, the geographic regions it operates in and the customers or partners it serves. This will inform your compliance team what compliance frameworks and audits they need to consider to make the business successful. For example, if you are a consumer based business and process credit card payments, then you will need to plan to comply with and pass a PCI-DSS audit or if you plan to serve the U.S. Federal Government you may need to achieve some level of FedRAMP. Understanding the business drivers, customer demand and regulatory environment is extremely important to setting up a successful compliance program that enables the business.

Milan: I definitely agree that you must start with a baseline assessment of where you are, but also with what your business goals are.  Are you going into financial spaces and require SOC1 Type 2? Do you want to baseline your core expectations with SOC2? What about healthcare, HIPAA anyone?  We won’t even get into FedRAMP if you want to sell to government agencies, so I suggest that you need to really understand, and baseline, your business goals in terms of verticals, industries, even governments/regions you want to sell into.  The compliance leader will need to balance out these requirements (including privacy, accessibility, etc), to ensure that for whatever region/country/industry you want to go into, you can cover the core baseline controls.

Once you have that, you need a way to track and ensure you can meet those controls.  Like we talked about before, and as I talk about now, integrating this with engineering and the appropriate metrics are key.  This includes how to “pre-audit” your services for new frameworks/certifications so you can minimize issues during an actual audit.  If you don’t do this from the start, retro-fitting it in after the fact is difficult and costly.  

You also then need personnel that understand these controls and requirements, and importantly how to translate and negotiate these with engineering.  If the people you hire cannot do that, then you fall into the same old challenge of friction between engineering and compliance that introduces risk. Having the right personnel that can translate and bring harmony among the pillars required for compliance (compliance, engineering, security, and operations) will be key to standing up a successful compliance team.

2. What if you inherit a security and compliance program that has been established by your predecessor? What advice do you have for leaders when adopting and continuing a program established by someone else?

Lee: Congratulations if you have been promoted or landed a new role as the leader of a security or compliance program! In my opinion the fundamentals don’t change whether you are starting a program from scratch or inheriting an existing program. I think it is critically important for a leader to baseline and understand what is already in place, what gaps exist and then build their strategic plan to address those gaps. The big difference when you inherit a program is it may take longer for the business to shift priorities towards the new plan due to organizational momentum and budget cycles. Technologically and practically, it may not actually take that long to do the things on your plan, but psychologically the organization will need to adjust and adapt. This means you will need to spend more time building alliances, getting teams on board, changing processes and shifting the culture to support your new vision. Keep in mind that closing the gaps in security control frameworks or compliance activities may take a significant amount of time because the organization may need to procure new tools, hire new talent or create new processes to satisfy a specific control. This has a knock on effect to your compliance program because most compliance audits have both readiness audits and the actual audit themselves. Both of these can take months to complete and so expectations need to be set for timelines, critical paths and when objectives will be met. If the business is expecting to get FedRAMp or HyTrust overnight, that is unrealistic because there are robust processes and timelines that need to be followed to pass those compliance activities.

Milan: I still agree that it doesn’t really change much from standing up a new compliance program, other than you may have a bit more work to change the culture from bad practices to new ones.  The key is still to align your compliance strategy with engineering goals/metrics.  Inheriting a program that doesn’t align to engineering requires immediate attention, as until you can align it, it will be difficult to help the cultural and behavioral changes needed to more closely align priorities and drive the actual execution.

As you say, it will likely still take time to close gaps, hire individuals, build tooling, but if the actual metrics and success criteria isn’t aligned with what engineering will need to have to actually change priorities, then most of that work may become “wasted” as while it may get you through a single audit, it won’t be efficient or at scale.  I would still suggest that new compliance org leaders take a deep look at how the compliance team articulates and aligns success metrics first, before doing anything else, and change/align them to engineering first otherwise they will continue to have friction, and risk, with executing the compliance programs that require engineering (or even operations and security) changes.

3. What challenges can a leader expect to face when building or maturing a security and compliance program and what advice do you have to overcome these challenges?

Lee: There are a few challenges that I have run into over the course of my career. The first challenge is mis-alignment of expectations between stakeholders. This could be a mismatch from the board, from other C-Suite executives or from the rest of the business (like sales). In these cases, it is imperative for the security leader to work closely with these stakeholders to align expectations. You are the expert in this field and you need to help these other leaders and teams understand what good looks like and when they can expect to achieve it. 

Expectation mis-alignment is also closely related to cultural mis-alignment. For example, if the technical culture is extremely permissive and allows teams to do whatever they want, then the security leader will need to spend time explaining the advantages of increased security control and how that can actually accelerate the business, while reducing risk. Culture mis-alignment is often the biggest challenge I’ve run into over the course of my career and it can be very time consuming to fix. Security leaders are often brought in to improve a security program, when in reality the culture is at issue. This often means the security team becomes a cultural change agent and they cannot change the culture alone. Security leaders will burn out or leave extremely quickly if the culture is not aligned and they don’t have the support of other executive leaders to achieve the security goals specified in the security strategy.

Finally, one of the other issues I’ve run into is budgetary. This can come in two forms – first, the security team struggles to get appropriate funding and second, the business insists the security or compliance team performs activities that may not actually make sense financially (like achieving a specific compliance audit). The first budgetary issue – not getting appropriate funding can be a real problem for the security team if the business arbitrarily cuts budgets instead of choosing to fund the security team based on risk. This funding issue is closely related to a mismatch of stakeholder expectations and a cultural misalignment. The second budgetary issue is often an issue between sales teams and the security and compliance team. Often, the sales teams insist a specific security control is necessary or a certain compliance framework is required in order to exist in a certain market or achieve a sale. While this can often be the case, it is important the security leader has the support to be able to push back on these requests where it makes sense. If a security and compliance activity is insisted on by the sales team, I always require a minimum threshold of business to cover the cost of the control and compliance activity. This should be written into the sales teams number, tied to their comp and ultimately they should be held accountable for justifying the need.

Milan:  I still think the biggest challenge for compliance leaders is building credibility.  I talk about how leaders look at all companies as “making money, saving money, or costing money”, and compliance has always been put into “costing money”.  So how does a compliance leader build their individual and team credibility with the engineering teams (the main control owners), that they are just not a pain in their collective engineering sides “costing money”?

Compliance leaders must start from the beginning showing engineering how they are “helping them”, which mainly to me is “saving money” (engineering/operations/security time).  Time is the key metric that is tracked by most engineering leaders (sprint and engineering time), so it’s incumbent on compliance leaders to show them how they are efficient, or even saving time based on their previous compliance experience.  Compliance leaders must be able to show, on paper with metrics, that the compliance work is efficient (I use first time resolution as a critical metric), and how we continuously drive for the ultimate metric, of removing engineering completely from the evidence collection flow via automation, etc.

If you cannot build that trust, then you will always just be a leader that doesn’t understand the engineering priorities, cost them money (i.e. time) by being inefficient, and having no credibility and influence as they have no real reason to engage with you.  It all starts with the ability to develop and agree on metrics that actually help engineering minimize the time it takes to do auditing/etc, and providing that value back to the actual engineers.

You get risk mitigation by default with the right metrics, as the evidence is clear and concise, and once first time resolution is high, you can now automate it. We have seen that the risk (and the deviations) have gone way down as we negotiate and approve with external auditors the evidence that will pass that QA for review, we have less “drift” with bad evidence, which reduces overall risk (and actually increases predictability and timeliness in audits).

I also believe that the requirements “accepting” of new frameworks requires a much more detailed review and acceptance than I’ve generally seen.  There is no point gaining new certifications if sales doesn’t have a solid plan to get the ROI with customers.  It’s costly to go through the certifications, and without appropriate sales to support the cost, it can become wasted work.  If the cost/benefit numbers and plans for sales are good, then it becomes much easier to justify the compliance work.  Also, if the success metrics and tracking of workload for compliance is also detailed, then justifying resourcing for new projects becomes easier.  Compliance teams must continuously run compliance “like a business” being able to show metrics that detail out the work, down to the team and individual level, otherwise justifying new resources becomes challenging.

The Most Powerful Word A CSO Can Say Is No

A Tale Of Two Extremes

A few weeks ago I was sitting across the dinner table from a CIO and a CISO who were discussing how security works within their specific businesses. The CIO was complaining that the security team had unreasonable processes that slowed down his ability to deliver new technology projects within his org and as a result he ignored a lot of their requests. This resulted in an engaging discussion about the best way to balance security requirements against the needs of the business. I found it interesting that some of my fellow CISOs were adamant about having teams meet security requirements without exception, regardless of the impact to the business.

After this discussion I spent some time thinking about own my stance on this issue and how I have tried to balance security requirements against the needs of the business over the course of my career. I pride myself on finding ways to partner with and support the business, instead of just telling them no all the time. However, I have also found that sometimes the most powerful word in my vocabulary is NO. Saying no is particularly useful during the rollout of new processes or security controls where you are changing behavior more than you are implementing a new technology.

Tug Of War

Security requirements and business needs can often be in a perpetual tug of war. This isn’t necessarily a bad thing when you consider the purpose of a CISO organization is to help protect the business not only from attackers, but often from itself. However, it can be difficult when the tug of war is biased towards one side or the other. If the business simply steam rolls and ignores all security requirements then clearly the CISO isn’t valued and the business isn’t interested in managing risk. However, if the CISO says no to everything, then this can be a significant and costly drag on the business in terms of people, processes, technology and time. Lost productivity, lost revenue, inability to deliver a product to market quickly can be difficult to measure, but have real impact to the business. Worse, the business may just ignore you. It is important for CISOs to find an appropriate balance to allowing the business to function, while meeting the desired security objectives to protect it. I firmly believe when done correctly, CISOs can avoid being a drag on the business and can actually enable it.

Just Say No

Despite my general inclination to find ways to keep the business moving forward, I’ve also found saying no to certain things can be extremely useful. For some things, when teams want an exception I usually say no as my initial response. I have found often teams just need to hear an exception isn’t an option and this unblocks them to pursue another alternative to the problem. As a result the teams improve their security while also delivering their business objectives.

Sometimes the teams will ask for an exception a second time. In these cases, I usually reconsider, but often still tell them no. My expectation after telling them no a second time is to either get them to fix the issue or if the issue can’t be fixed to present a plan with different options along with their recommendation. When teams come back for the third time it ends up being an actual business risk discussion instead of an exception discussion. The outcome usually ends up being some sort of compromise on both sides, which is exactly what you want. Taking a balanced approach develops an appropriate level of partnership between security and the rest of the business where one side isn’t always sacrificing their objectives for the other side.

Seriously, Just Say No

Next time a team comes to you with an exception request try saying no and see if they respond differently. You may or may not be surprised when they find an alternate solution that doesn’t require an exception. For me, it has become a powerful tool to nudge teams towards achieving my security goals, while still delivering on their business objectives.

Compliance Corner Series: Compliance Landscape 2023

This blog post is part of the Compliance Corner Series developed in partnership with Milan Patel. This series will include a variety of discussion topics around the intersection of security and compliance. The series will include blog posts, live web streams (with Q&A) and video blogs.

“How has globalization, increasing regulatory requirements from governments and industries change how Security and Compliance must operate internally”

  1. Growing Regulatory Requirements – Impact To The Whole Company vs Single Business Unit

Lee – From a security perspective, it is difficult to standardize security controls and baselines across the whole company vs. a single business. Business lines have different requirements, customers and industries, which are unique to their product lines and customer base. Mandating the entire company meet a global regulatory requirement (like PCI-DSS, FedRAMP or HIPAA) can be costly for those lines of businesses that don’t have customers in those industries where these regulations apply. While the security program may be effectively managing risk for their particular line of business, the difference in regulatory requirements can lead to gaps in customer expectations across product lines or lines of business. It can also be costly to maintain compliance programs since you typically have to audit and produce evidence for each product. This puts a burden on development teams and can take time away from improving the security of their products, vs. generating sufficient evidence to pass audits. For smaller lines of businesses this can be difficult to justify the cost to keep up with global regulations, certify audits and maintain levels on par with the rest of the company.

It is possible to centralize compliance activities across the company, but it also requires every part of the company to standardize their security controls, risk management practices and reporting principles. This may work well for small to medium sized companies that have grown organically or who have done a good job of homogenizing their organization. For larger organizations that have grown via acquisition and have dozens of lines of businesses and product lines, centralizing compliance and security activities may be impossible.

Milan – For compliance, the issue is that many of these new requirements have no “corporate” baseline expectations.  They have not been translated (and sometimes agreed to), by individual lines of business.  Without a standard “bar” to execute against, it’s very difficult for individual compliance teams to attest that it’s being met.  In many cases, these items have no tests, or have not even been implemented.  It becomes a real issue at scale, as these are yet another new compliance item that engineering will have to meet, which will continue to compete with their feature development time requirements.

My experience is that it’s challenging for many of the central compliance/legal teams (that try to set these requirements for engineering teams), from both a positional authority, as well as a credibility perspective.  There are few engineers on these central compliance teams that are experienced with translating these regulations to actual engineering practices.  Also, there is a lack of organizational authority to enforce it uniformly across all business groups.  This just increases the challenge as it adds more time to negotiate and drive priority, especially with the speed of these incoming regulations.

2. Security and Compliance Practices – What Has Changed?

Lee – For security, there has been an increasing trend in transparency as well as a reduction in time to fix security vulnerabilities and report security incidents. Often these regulations or new compliance practices have good intent, but will be so costly to implement that they are impractical or are technologically impossible. These problems are further compounded by modern software development practices like DevOps that move incredibly quickly and can amplify deployment of internet scale products across cloud providers. Software supply chain security and reporting for compliance reasons is particularly challenging due to the complexity of modern software libraries that have multiple interlinking dependencies. Third party libraries and open source software is even more difficult because you don’t always know if you can trust the source code of that library. Providing an accurate software bill of materials (SBOM) can be a significant burden on the business to develop and maintain for compliance reporting purposes. Additionally, the priority of the business can sometimes take precedence over other activities like solving tech debt or modernizing technology stacks. Technical debt can not only hold back the business from optimizing and becoming more efficient, but can also make security and compliance activities challenging.

Milan – I’ve been doing this for a long time, on the engineering side with a large multi-billion dollar software product, and more recently on the compliance side across many products.  What I can say is that while engineering has changed… DevSecOps, updating features/software monthly, instead of yearly… or even every 5 years when hardware went “end of life” (yes, I’m totally aging myself here), compliance practices basically have not changed internally.  

Even major companies are still trying to “throw money” at the problem.  The internal compliance teams know that the SLT has to pay for compliance… but I believe the days of the compliance “bat” and “blank checkbook” are gone.  It doesn’t scale.  I will never get more bodies to manually collect and review evidence for large amounts of controls and services (and not regulatory requirements) that we will have to test and account for. Spreadsheets and email won’t work anymore.  Compliance needs to change, and be run more like a business. Even the third party auditors didn’t have incentive to be more efficient… remember, they mostly bill hours… So the more inefficient and longer it takes for a company to do an audit, the more money they make.  We’ve changed that quite a bit at Oracle, but it’s still early in the journey.

3. Looking Ahead: How Must Security and Compliance Practices Change?

Lee – For security, teams need to shift to a continuous compliance model. Instead of periodically assessing the business once a year (or some other interval) teams need to implement security controls and reporting mechanisms that can automate and continuously evaluate how the security controls meet the various regulatory requirements required for the business. By moving to a continuous compliance model it will force the business to modernize their Software Development Life Cycle (SDLC), modernize their technology stacks and resolve technical debt, which will ultimately benefit security programs. A continuous compliance program will allow security teams to tackle security at multiple points across the technology stack. First, it will evaluate, report and control how software is written. Second, it will evaluate, report and control how infrastructure is deployed and third, it will evaluate, report and control how security posture changes over time so teams can react to security drift.

Security and compliance practices must also change by holding the rest of the business accountable for their decisions. It is impossible for a security or compliance team to be successful without support from the rest of the business because the security and compliance teams don’t “own” the infrastructure and products the business uses to make money. Therefore, there needs to be support at the highest levels of executive leadership to hold the lines of business accountable and tie performance and incentives to how these businesses satisfy their security and compliance requirements.

Milan – Well, I think it must happen on two levels.  First level is central oversight and governance.  

Without a strong central compliance function that can translate, and negotiate requirements, enforce and govern equally with the engineering leaders, it won’t scale effectively.  The main challenge now is that the “fox is guarding the hen house” so to speak.  When the engineering team compliance is solely under engineering leaders in the product group, there is the ability (or even just the appearance) that decisions are made without compliance “equality” of decision making.  If a compliance item isn’t resolved, who questions whether that amount of risk is acceptable to the senior product leader?  Even if accepting the risk is “OK”, any external party can question whether the right level of compliance leadership made that decision, and not just an engineering leader who may prioritize compliance lower than say… a feature request.  

This mostly only becomes an issue after the fact.  A compliance issue happens, then the questions of “Who made that decision… why was that compliance issue not escalated… etc?  Escalate to who?  While it’s a great idea that a compliance leader (buried usually multiple levels down in an organization), will be “politically unencumbered” and rise up and strongly disagree with an issue with their senior leader (who by the way signs their checks). My experience is that it can be difficult, as being seen as an antagonist is not great for one’s career in this space.   I’ve noticed that compliance positions that have high turnover, this has been one of the main indicators of how effective the compliance leader is in the organization feels they are.  If they feel they cannot speak up, or they do and are dismissed “for higher priorities”, they feel they are not effective, or will become the scapegoat if there is an issue.  

The second level is that compliance needs to change their success metrics, and run like a business.

Most senior leaders I know, including myself, put everything into three buckets…  You’re making money… saving money… or costing money…  Compliance has always been seen as “overhead” and costing money.  Even today, I’ve had to battle that, until we made some major changes in how we operate, and what metrics we track for success.  So the question is, if I’m always seen as “costing money”, how do I change that, and what do I change it to?  I realized that while compliance does “make money” (new frameworks open new markets), I don’t really get to claim that.  I’m not out there hustling deals… So I realized years ago I could “save money”, but not in the traditional sense.  I had to change my compliance principles and metrics.  First, everyone accepts that compliance is important, but why did they hate it so much?  As an engineer doing this on a large project, the compliance team never understood that my metric was “completing the audit” or “collecting XXX number of evidence pieces”.

My engineering metric of success was TIME.

As a development operations, release manager, and development compliance owner for the large project, I only had so many engineering hours, so I needed to minimize time I spent on each engineering function, including compliance.

So in compliance, I changed the success metric to First Time Resolution (FTR) for audit evidence. Everyone knows you have to collect at least one, but we had so many “kickbacks” for many reasons.. Either the engineering team didn’t provide what was asked for, or the evidence instructions were incomplete or unclear (remember what I said earlier for “billable hours” from the auditors)? I put at least 30 min per kickback.  I used to own 1000 pieces of evidence, sometimes I could resolve them in 5 min, sometimes it took days.  By raising the FTR up to a goal of 95% (makes it through the first time), I can show, on paper… how much money I’m saving the engineering team (in time).  I was able to show this year, saving millions of dollars in audit costs (by going with an efficient auditor, that didn’t bill me for all those hours), and saving Tens of Thousands of engineering hours by eliminating the “unplanned work” of kickbacks.  

By changing the metrics to what the engineers care about, it built trust and transparency, and increased the overall value of compliance, plus their willingness to look for more efficiencies. 

Needless to say there are a lot more details to talk about for the journey to get here, but the engineering leaders saw, with hard metrics, that compliance “saved money” (via time) for them to use on other features, bug fixing, etc.  And more importantly, it built trust with them that the compliance team was a good steward of their currency, which is time.

4. Where do we go from here?

Lee – Milan, it’s been great discussing how we evolved to this state, and some great ideas and positions on what has to change in our respective security and compliance areas.  There is so much to discuss here I’m looking forward to continuing this series to dive into topics around the intersection of compliance and security.
Milan – Yes Lee, it’s been great discussing these topics.  For me, I feel like this is the beginning of another Industrial Revolution for Compliance, we have seen the change in the engineering and globalization, and we’re on the verge of revolutionizing the Security and Compliance space.  We didn’t even really get into the overall governance and continuous security/compliance versus point in time.  Maybe that’s the next discussion!

Can Risk Truly Be Measured?

As a CSO, everything you do needs to be evaluated in terms of risk to the business. When you build a security program, prioritize objectives or respond to incidents all of your decisions will take into consideration risk and how to effectively manage it. Which begs the question: Can Risk Truly Be Measured?

The Problem With Risk

The problem with risk is that it is subjective. Everyone views risk, and the behaviors that contribute to risk, differently. Trying to measure a subjective concept can lead to inconsistencies, fluctuations or even conscious and subconscious influencing of the outcome. Even worse, as you engage in behavior that involves risk, your risk tolerance will adapt over time to accept the risk as normal. For the security space, this is the equivalent of failing to implement a specific security control and yet avoiding a breach or incident. As a result the business may try to rationalize that you don’t need to implement the control because you haven’t suffered the consequences of a breach. How do you consistently and objectively measure risk for yourself, your team or your business if it is subjective?

Objectivity Leads to Consistency

Risk can be measured, but it requires you to apply an objective and consistent process to gain consensus and get consistent results. There are several methods for obtaining consistent results when dealing with a subjective concept. In Operations Research there is a process known as the Analytical Hierarchy Process (AHP), which attempts to achieve a mathematical decision for a subjective concept (like deciding which house to buy). Similarly, when attempting to decide between different subjective concepts you can use simple mechanisms like pairwise comparisons or a triangle distribution to express the decision mathematically.

Here are two examples:

Pairwise comparison: This comparison is useful for comparing two things to determine which one is more important. Example: Patching our publicly facing systems with critical vulnerabilities is ten times more important than patching our systems with critical vulnerabilities that are only internally accessible.

Triangle Distribution: This distribution is useful when attempting to distill a decision down to a value when you may only know two data points, such as the extremes. Or, if two people offer different opinions for the value of something you can use the triangle distribution to find a compromise in the middle. Example: Manager 1 – The impact of this event is a 2 out of 5 (5 being highest). Manager 2 – The impact of this event is a 4 out of 5. Using a triangle distribution we decide the impact of this event is a 3.

Either of these methods can be used to determine the likelihood and impact for a specific security control in an industry standard security framework like NIST or CIS.

There are other objective methods for assessing and assigning risk. I’ve discussed the CIS Risk Assessment Method (RAM) in detail in my blog post about Building A Strategic Plan. The advantage of this method is you can apply a consistent process to assign and measure risk for every single CIS control and sub-control. This allows you to measure how you well you are managing risk over the lifetime of your security program in a repeatable way.

When Things Go Wrong

Following an objective process to measure risk requires you to take the time to methodically apply the process to each decision or control you need to asses. This can be time consuming at first, but once the initial baseline is set it is easy to update and maintain. However, it is easy to mess this process up. Here are a few things I don’t recommend:

  1. Don’t make up your own security control framework – There are industry standard security frameworks for a reason. The standard frameworks are comprehensive and have clear criteria for how to apply and asses the effectiveness of a control. The worst “frameworks” I have seen are typically made up by someone in the security team that things they know better. These “frameworks” are subjective, inconsistent, contradictory, redundant or unclear about how to apply specific controls. On top of that these “frameworks” require someone to update them, maintain them and communicate them to the rest of the org. Why duplicate work when you don’t have to?
  2. Failing to apply a consistent process – Failing to apply a consistent process for measuring and assessing risk means you will fail to get a consistent and objective result. This means you will have an inaccurate measurement of risk or the risk will fluctuate with each measurement, which will render it useless. Also in this category is allowing different teams within your company to use different processes for assessing risk. This means you will not be able to consistently compare the risk of different security controls across the business, which can lead to gaps or overconfidence.
  3. Failing to validate the result – This may be obvious, but each of your measurements should be backed up by evidence. Don’t assume your inventory is correct. Don’t assume your IDS is working and don’t assuming your DDoS protection will work properly. Test these things, audit them and spot check them periodically. Risk doesn’t always have to move in one direction. Controls can regress, postures can change and threats are constantly evolving. Validate your results on a regular basis to make sure you are getting the most accurate picture possible.

So Can You Really Measure Risk?

I believe you can measure risk and develop an accurate risk profile for your company. Despite the subjective nature of risk, you can reach an objective result by using simply mathematical principles and by using an consistent objective process. When applying these methods to an industry standard control framework, you can obtain an objective result that will accurately measure your risk for a single decision, or an entire control framework. Once you have an accurate risk profile you can use the risk to make prioritized decisions to manage risk for your business.

Security Vendor Questionnaires: Too Much or Not Enough?

Over the past few years there has been an increasing trend for customers and partners to request security teams to fill out lengthy security questionnaires seeking specific details about the state of their security program. These requests often come as part of routine audits, regulatory requirements or contract negotiations. As someone who has both sent out questionnaires and been a recipient of questionnaires I’m wondering if the industry has gone too far down this trend or if it hasn’t gone far enough? Let me explain…

As a CSO, I want to discover and manage as much risk as possible. This includes conducting business with partners, customers and other companies. I want to understand my supply chain and limit my exposure to any of their security weaknesses that could be used to attack my company. However, I also want to limit the amount of information I disclose about my security program because once I disclose it, I no longer control that information and it could eventually make its way to an adversary if the company I disclosed it to has a breach.

How do we balance these differing requirements and are security questionnaires really the best mechanism for understanding your supply chain?

How Did We Get Here?

Let’s take a step back and consider how we collectively arrived at the need for security questionnaires. There have been several high profile breaches that have set us down this path. The first was the Target breach in 2013 where Target had their point of sale systems compromised as a result of a third party HVAC vendor. The magnitude of this breach along with the realization that Target was compromised via a third party placed a spotlight on supply chain security for the entire industry.

The second high profile breach was the Solar Winds attack in 2020. This attack infiltrated the software supply chain of Solar Winds and placed a backdoor in the product. Given that Solar Winds was used by a huge number of companies this effectively compromised the software supply chain of those companies as well. This attack increased the scrutiny on the supply chain with additional emphasis on software supply chain and even leading to some sectors (like the government) requiring disclosure of a Software Bill of Materials (SBOM).

Increased Regulatory Pressure

These notable attacks (among others) have lead to an increase in regulations that force companies to disclose details around security breaches, but also to invest appropriately in security programs. Despite these investments and disclosures, companies can still face steep fines and costly lawsuits for security breaches. New regulations such as the SEC Cyber Risk Management rules and recent White House Executive Orders on Improving Cybersecurity and establishing a National Cybersecurity Strategy have elevated awareness and focus on supply chain security to the national stage.

Cyber Insurance Isn’t Helping

As Cybersecurity insurance premiums become more and more expensive, companies will continue to look for ways to decrease the cost, while still maintaining coverage. One of the most effective ways to do this is to establish and document a mature security program that you review in detail with your insurer to explain your risks and how you are mitigating them with appropriate controls. Questionnaires are one part of a security program that can demonstrate how you are evaluating and managing supply chain risk and hopefully drive down your premiums (for now). The problem is this creates an incentive where it is everyone for themselves in an attempt to lower their own premiums.

Transparency Is Lacking

The biggest issue security questionnaires are attempting to address is lack of transparency into the details of the security programs. In general, large publicly traded companies (particular cloud companies) and security product companies tend to be more transparent about security because it is built into their brand as a selling point to attract customers. However, details about technologies, program structure, response times, etc. generally lack specificity (for good reason) and the security questionnaire is an attempt to uncover those details to understand what risks exist when entering into a relationship with another company.

You might argue that companies should simply be more transparent with details about their security program, but this is not the solution. Companies should cover high level details with some specificity to demonstrate they have a security program and how it is structured. However, giving specific details about processes, response times, technologies, etc. will reveal details that can be used by an adversary for an attack. Additionally, do we really know what is happening with all this data from security questionnaires? It may be protected under Non-Disclosure Agreements (NDAs) and confidentiality agreements, but that doesn’t prevent the data from being leaked via an unprotected S3 bucket. It is extremely difficult to change a security program quickly and so it may be in the best interest of the responding company to refuse to answer the questionnaire and instead have an undocumented conversation (depending on your level of paranoia).

Yet More Audit and Regulatory Pressure

On top of all these issues, there are still very real requirements to respond to audits, questions from regulators or to provide these responses to your customers and partners that operate in heavily regulated industries (finance, healthcare, government, etc.). Responding to the questionnaires still takes time and places a burden on your security team and still comes with the risk that the information could be involuntarily disclosed to an adversary.

What’s Really Going On Here?

Responding to regulatory and audit requirements aren’t new requirements for our industry and so answering security questionnaires has been the norm for quite some time. However, I think the use of the security questionnaire has been hijacked by the industry as a catch-all way to accomplish a few things with respect to security:

  1. Assert your security requirements over another company (may work for small companies if they can fund it, but generally doesn’t work for large companies with mature programs).
  2. Minimize risk of doing business with and potentially pass on liability to the recipient company. I.e. “We asked them if they did this thing, but we got breached because of them so they clearly didn’t do that thing so it is their fault.”
  3. Create negotiating points as part of contract negotiations for concessions.

The problem with these is they are attempting to impose a solution or liability on a program they don’t control. As a CSO I can’t agree to these things because being contractually obligated to a specific security solution or SLAs removes my decision making power for how to best manage that risk within my security program. Security programs change and are constantly adapting to stay ahead of threats and risks to the business. Being boxed into a solution contractually can actually create a risk, where there wouldn’t normally be one.

Fatigue Is Real

I genuinely struggle with this problem because security questionnaires have their uses, but are causing real fatigue across the industry. The questions fall into one of two categories: either they are largely the same across customers or they are completely bonkers and don’t justify a response. As both a sender and recipient of questionnaires I can definitely understand both sides of the issue. I want as much information about my supply chain customers, but want to minimize the specifics that I share outside of my control. I want lower cyber insurance premiums and I want to pass all my audits and regulatory inquiries. However, I think the industry has deviated from the original intent of the security questionnaire due to the real fear of being held liable for a failure in their security program, which includes their supply chain.