#compliance – 370 Security Blog

If Data Is Our Most Valuable Asset, Why Aren’t We Treating It That Way?

There have been several high profile data breaches and ransomware attacks in the news lately and the common theme between all of them has been the disclosure (or threat of disclosure) of customer data. The after effects of a data breach or ransomware attack are far reaching and typically include loss of customer trust, refunds or credits to customer accounts, class action lawsuits, increased cyber insurance premiums, loss of cyber insurance coverage, increased regulatory oversight and fines. The total cost of these after effects far outweigh the cost of implementing proactive security controls like proper business continuity planning, disaster recovery (BCP/DR) and data governance, which begs the question – if data is our most valuable asset, why aren’t we treating it that way?

The Landscape Has Shifted

Over two decades ago, the rise of free consumer cloud services, like the ones provided by Google and Microsoft, ushered in the era of mass data collection in exchange for free services. Fast forward to today, the volume of data growth and the value of that data has skyrocketed as companies have shifted to become digital first or mine that data for advertising purposes and other business insights. The proliferation of AI has also ushered in a new data gold rush as companies strive to train their LLMs on bigger and bigger data sets. While the value of data has increased for companies, it has also become a lucrative attack vector for threat actors in the form of data breaches or ransomware attacks.

The biggest problem with business models that monetize data is: security controls and data governance haven’t kept pace with the value of the data. If your company has been around for more than a few years chances are you have a lot of data, but data governance and data security has been an afterthought. The biggest problem with bolting on security controls and data governance after the fact is it is hard to reign in pandoras box. This is also compounded by the fact that it is hard to put a quantitative value on data, and re-architecting data flows is seen as a sunk cost to the business. The rest of the business may find it difficult to understand the need to rearchitect their entire business IT operations since there isn’t an immediate and tangible business benefit.

Finally, increased global regulation is changing how data can be collected and governed. Data collection is shifting from requiring consumers to opt-out to requiring them to explicitly opt-in. This means consumers and users (an their associated data) will no longer be the presumptive product of these free services without their explicit consent. Typically, increased regulation also comes with specific requirements for data security, data governance and even data sovereignty. Companies that don’t have robust data security and data governance are already behind the curve.

False Sense Of Security

In addition to increased regulation and a shifting business landscape, the technology for protecting data really hasn’t changed in the past three decades. However, few companies implement effective security controls on their data (as we continue to see in data breach notifications and ransomware attacks). A common technology used to protect data is encryption at rest and encryption in transit (TLS), but these technologies are insufficient to protect data from anything except physical theft and network snooping (MITM). Both provide a false sense of security related to data protection.

Furthermore, common regulatory compliance audits don’t sufficiently specify protection of data throughout the data lifecycle beyond encryption at rest, encryption in transit and access controls. Passing these compliance audits can give a company a false sense of security that they are sufficiently protecting their data, when the opposite is true.

Just because you passed your compliance audit, doesn’t mean you are good to go from a data security and governance perspective.

Embrace Best Practices

Businesses can get ahead of this problem to make data breaches and ransomware attacks a non-event by implementing effective data security controls and data governance, including BCP/DR. Here are some of my recommendations for protecting your most valuable asset:

Stop Storing and Working On Plain Text Data

Sounds simple, but this will require significant changes to business processes and technology. The premise is the second data hits your control it should be encrypted and never, ever, unencrypted. This means data will be protected even if an attacker accesses the data store, but it also will mean the business will need to figure out how to modify their operations to work on encrypted data. Recent technologies such as homomorphic encryption have been introduced to solve these challenges, but even simpler activities like tokenizing the data can be an effective solution. Businesses can go one step further and create a unique cryptographic key for every “unique” customer. This would allow for simpler data governance, such as deletion of data.

Be Ruthless With Data Governance

Storage is cheap and it is easy to collect data. As a result companies are becoming digital data hoarders. However, to truly protect your business you need to ruthlessly govern your data. Data governance policies need to be established and technically implemented before any production data touches the business. These policies need to be reviewed regularly and data should be purged the second it is no longer needed. A comprehensive data inventory should be a fundamental part of your security and privacy program so you know where the data is, who owns it and where the data is in the data lifecycle.

The biggest problem with business models that monetize data is: security controls and data governance haven’t kept pace with the value of the data.

Ruthlessly governing data can have a number of benefits to the business. First, it will help control data storage costs. Second, it will minimize the impact of a data breach or ransomware attack to the explicit time period you have kept data. Lastly, it can protect the business from liability and lawsuits by demonstrating the data is properly protected, governed and/or deleted. (You can’t disclose what doesn’t exist).

Implement An Effective BCP/DR and BIA Program

Conducting a proper Business Impact Analysis (BIA) of your data should be table stakes for every business. Your BIA should include what data you have, where it is and most importantly, what would happen if this data wasn’t available? Building on top of the BIA should be a comprehensive BCP/DR plan that appropriately tiers and backs up data to support your uptime objectives. However, it seems like companies are still relying on untested BCP/DR plans or worse solely relying on single cloud regions for data availability.

Every BCP/DR plan should include a write once, read many (WORM) backup of critical data that is encrypted at the object or data layer. Create WORM backups to support your RTO and RPO and manage the backups according to your data governance plan. Having a WORM backup will prevent ransomware attacks from being able to encrypt the data and if there is a data breach it will be meaningless because the data is encrypted. BCP / DR plans should be regularly tested (up to full business failover) and security teams need to be involved in the creation of BCP/DR plans to make sure the data will have the confidentiality, integrity and availability when needed.

Don’t Rely On Regulatory Compliance Activities As Your Sole Benchmark

My last recommendation for any business is – just because you passed your compliance audit, doesn’t mean you are good to go from a data security and governance perspective. Compliance audits exist as standards for specific industries to establish a minimum bar for security. Compliance standards can be watered down due to industry feedback, lobbying or legal challenges and a well designed security program should be more comprehensive than any compliance audit. Furthermore, compliance audits are typically tailored to specific products and services, have specific scopes and limited time frames. If you design your security program to properly manage the risks to the business, including data security and data governance, you should have no issues passing a compliance audit that assesses these aspects.

Wrapping Up

Every business needs to have proper data security and data governance as part of a comprehensive security program. Data should never be stored in plain text and it should be ruthlessly governed so it is deleted the second it is no longer needed. BCP/DR plans should be regularly tested to simulate data loss, ransomware attacks or other impacts to data and, while compliance audits are necessary, they should not be the sole benchmark for how you measure the effectiveness of your security program. Proper data protection and governance will make ransomware and data breaches a thing of the past, but this will only happen if businesses stop treating data as a commodity and start treating it as their most valuable asset.

Security Considerations For M&A and Divestitures

I’ve been speaking to security startups over the last few weeks and some of the discussions made me think about the non-technical aspects of security that CISOs need to worry about. Specifically, things like mergers, acquisitions and divestitures and the different risks you will run into when executing these activities. There are a number of security issues that can materialize when combining businesses or separating businesses and in this post I’ll share some of the things you need to think about from a security perspective that may not be obvious at first glance.

What’s Going On Here?

There are a number of reasons for mergers & acquisitions (M&A) or divestitures. For the past two decades, the tech industry has used M&A to acquire smaller startup companies as a way to collect intellectual property, acquire specific talent or gain a competitive advantage. Divestitures may be the result of changing business priorities, separating business functions for regulatory reasons, eliminating redundancies or a way to sell a part of the business to cover costs. Mergers, acquisitions and divestitures are similar because you will want to review the same things from a security perspective, but it is probably easiest to think of divestitures as the reverse of an M&A – you are separating a business instead of combining a business. Divestitures are definitely less common than M&A in the tech space, but they aren’t unheard of. There are also differences in terms of the security risks you need to think about depending on if you are acquiring a business or separating a business. My best advice is to work with the legal and finance teams performing the due diligence and have a set process (that you have contributed to) so you don’t forget anything. With that, let’s dive into a few different areas.

Physical Security

Physical security is something you will need to think about for both M&A and divestitures. For M&A you will want to perform a physical security assessment on the facilities you are acquiring to make sure they meet or exceed your standards. Reviewing physical security controls like badging systems, fencing, bollards, cameras, fire suppression, emergency lighting, tempest controls (if required), safes and door locks will all help make sure your new facilities are up to standard. If you aren’t sure how to perform this, hire a company that specializes in physical security assessments or physical red teaming.

While physical security for M&A may seems straight forward, there are a few gotchas when performing divestitures. The biggest gotcha is understanding and reviewing the existing access of the people that are part of the divestiture because you will now need to consider them outsiders. All of your standard off-boarding processes will apply here such as terminating accesses to make sure someone doesn’t retain access to a system they are no longer authorized to access (like HR, Finance, etc.).

Things can get complicated if parts of the business are divesting, but not fully. Some examples of this are when the business divests a smaller part, but allows the smaller part to co-locate in their existing facilities. This may complicate physical security requirements such as how to schedule or access common areas, how to schedule conference rooms, how to separate wifi and network access, etc. In the above example, the larger company may act like a service provider to the divested part of the business, but there still needs to be effective security controls in place between the two parts.

Personnel Security

I touched on this a bit already, but personnel security is something to consider when performing M&A or divestitures. With M&A the biggest issue will be how to smash the two IAM systems and HR systems together without punching huge holes in your network. Typically what happens is the two parts operate separately for a while and then consolidate to a single system and the employees of the acquired business get new accounts and access.

For divestitures, particularly if they don’t result in a clean split, you will need to focus heavily on access control and insider threats. Think about how you will separate access to things like source code, financial systems, HR systems, etc. If the smaller company has physical access to your space then you need to build in proper physical and logical controls to limit what each business can do, particularly for confidentiality and competitive reasons.

What’s an example of where this can go wrong? Let’s say business A is going to divest a small part of its business (business B). The complete divestiture is going to take a while to finalize so company A agrees to allow company B to continue to access their existing office space, including conference rooms. However, the legal team didn’t realize the conference rooms are tied to company A’s SSO and calendaring system so company B has no way to schedule the conference rooms without retaining access to company A’s IAM system creating a major security risk. Whoops!

The biggest gotcha is understanding and reviewing the existing access of the people that are part of the divestiture because you will now need to consider them outsiders.

Contracts

Contracts may not seem like a typical security issue, but they should be part of your review, particularly when performing M&A. Why? You are acquiring a business that is worth something and that business will have existing contracts with customers. The contractual terms with those customers may not match the contractual terms of the acquiring company, which can cause a risk if there is a significant difference in contract terms. Smaller companies are more agile, but they also usually have less negotiating power compared to large companies and as a result are more likely to agree to non-standard contract terms. What are some terms you need to think about?

Vulnerability Remediation Times – How quickly did the new company promise to fix vulnerabilities for their customers?
Incident & Breach Disclosure Time Frames – How quickly did the new company promise to notify customers of a breach or incident? I have seen very small time frames suggested in contracts, which are impossible to meet, so I definitely recommend reviewing these.
Disclosure of Security Postures – Does the new company have contractual terms promising to provide SBOMs or other security posture assessments to their customers on a regular basis?
Compliance Requirements – Has the new company agreed to be contractually obligated to maintain compliance certifications such as PCI-DSS, SOC 2, ISO27001, etc.
Penetration Testing & Audits – Has the new company contractually agreed to have their products or services penetration tested or have their security program audited? Have they agreed to provide these reports to their customers on a regular basis?
Privacy & Data Governance Terms – Is the new company required to comply with privacy regulations such as allowing customers have their data deleted, or mandating certain data governance requirements like DLP, encryption, data deletion, etc?
BCP/DR and SLAs – Are there contractual uptime SLAs or response times and does the existing BCP/DR plan support these SLAs?

My advice is to set a timeline post acquisition to review and standardize all of your contracts to a single set of standard clauses covering the above topics. This is usually part of a security addendum that the legal team can help you create. The biggest challenge with contracts will be to “re-paper” all of your customers to hopefully get them on the same standardized contract terms so your security program doesn’t have a bunch of different requirements they have to try to meet.

Accuracy Of M&A’s

One of the biggest risk of performing M&A’s is trying to get an accurate picture of the existing security posture of the company being acquired. Why is this so difficult? The company being acquired is trying to look as good as possible so they get top dollar. They can’t hide things, but they aren’t going to tell you where all the skeletons are buried either. The acquiring company usually doesn’t get a full picture of the existing security posture until after the deal is done and you start trying to integrate the two parts of the business. If you have a chance to interview the existing security team before the M&A closes definitely ask to see their latest audit reports, compliance certifications, penetration testing reports, etc. Consider working with legal to set conditions for how old these reports can be (e.g. no older than 6 months) to hopefully give you a more accurate picture or require the acquired company to update them before the deal closes. Interview key members of the staff to ask how processes work, what are their biggest pain points, etc. Consider hiring an outside company to perform an assessment, or you can even consider talking to one of their largest customers to get their external view point (if possible).

Wrapping Up

M&A and divestitures can be exiting and stressful at the same time. It is important for the security team to be integrated into both processes and to have documented steps to make sure risks are being assessed and addressed. I’ve listed a few key focus areas above, but most importantly standardizing your M&A security review can help avoid “buyers remorse” or creating unnecessary risk to the acquiring business. Finally, having a documented divestiture process and reviewing the divestiture with legal can help avoid security risks after the fact.

Security Theater Is The Worst

We have all been there…we’ve had moments in our life where we have had to “comply” or “just do it” to meet a security requirement that doesn’t make sense. We see this throughout our lives when we travel, in our communities and in our every day jobs. While some people may think security theater has merit because it “checks a box” or provides a deterrent, in my opinion security theater does more harm than good and should be eradicated from security programs.

What Is Security Theater?

Security theater was first coined by Bruce Schneier and refers to the practice of implementing security measures in the form of people, processes or technologies that give the illusion of improved security. In practical terms, this means there is something happening, but what that something is and how it actually provides any protection is questionable at best.

Examples Of Security Theater

Real life examples of security theater can be seen all over the place, particularly when we travel. The biggest travel security theater is related to liquids. TSA has a requirement that you can’t bring liquids through security unless they are 3 ounces or smaller. However, you can bring a bottle of water through if it is fully frozen…what? Why does being frozen matter? What happens if I bring 100, 3 ounce shampoo bottles through security? I still end up with the same volume of liquid and security has done nothing to prevent me from bringing the liquid through. As for water, the only thing that makes sense for why they haven’t relaxed this requirements is to prop up the businesses in the terminal that want to sell overpriced bottles of water to passengers. Complete theater.

“Security theater is the practice of implementing security measures that give the illusion of improved security.”

Corporate security programs also have examples of security theater. This can come up if you have an auditor that is evaluating your security program against an audit requirement and they don’t understand the purpose of the requirement. For example, and auditor may insist you install antivirus on your systems to prevent viruses and malware, when your business model is to provide Software as a Service (SaaS). With SaaS your users are consuming software in a way that nothing is installed on their end user workstations and so there is little to no risk of malware spreading from your SaaS product to their workstations. Complete theater.

Another example of security theater is asking for attestation a team is meeting a security requirement instead of designing a process or security control that actually achieves the desired outcome. In this example, the attestation is nothing more than a facade designed to pass accountability from the security team, that should be designing and implementing effective controls, to the business team. It is masking ineffective process and technologies. Complete theater.

Lastly, a classic example of security theater is security by obscurity. Otherwise known as hiding in plain sight. If your security program is relying on the hope that attackers won’t find something in your environment then prepare to be disappointed. Reconnaissance tools are highly effective and with enough time threat actors will find anything you are trying to hide. Hope is not a strategy. Complete theater.

What Is The Impact Of Security Theater?

Tangible And Intangible Costs

Everything we do in life has a cost and this is certainly true with security theater. In the examples above there is a real cost in terms of time and money. People who travel are advised to get to the airport at least two hours early. This cost results in lost productivity, lost time with family and decreased self care.

In addition to tangible costs like those above, there are also intangible costs. If people don’t understand the “why” for your security control, they won’t be philosophically aligned to support it. The end result is security theater will erode confidence and trust in your organization, which will undermine your authority. This is never a place you want to be as a CISO.

Some people may argue that security theater is a deterrent because the show of doing “security things” will deter bad people from doing bad things. This sounds more like a hope than reality. People are smart. They understand when things make sense and if you are implementing controls that don’t make sense they will find ways around them or worse, ignore you when something important comes up.

With any effective security program the cost of a security control should never outweigh the cost of the risk, but security theater does exactly that.

Real Risks

The biggest problem with security theater is it can give a false sense of security to the organization that implements it. The mere act of doing “all the things” can make the security team think they are mitigating a risk when in reality they are creating the perfect scenario for a false negative.

How To Avoid Security Theater?

The easiest way to avoid security theater is to have security controls that are grounded in sound requirements and establish metrics to evaluate their effectiveness. Part of your evaluation should evaluate the cost of the control versus the cost of the risk. If your control costs more than the risk then it doesn’t make sense and you shouldn’t do it.

The other way to avoid security theater is to exercise integrity. Don’t just “check the box” and don’t ask the business you support to check the box either. Take the time to understand requirements from laws, regulations and auditors to determine what the real risk is. Figure out what an effective control will be to manage that risk and document your reasoning and decision.

The biggest way to avoid security theater is to explain the “why” behind a particular security control. If you can’t link it back to a risk or business objective and explain it in a way people will understand then it is security theater.

Can we stop with all the theater?

Using Exceptions As A Discovery Tool

Security exceptions should be used sparingly and should be truly exceptional circumstances that are granted after the business accepts a risk. In mature security programs the security exceptions process is well defined and has clear criteria for what will and will not meet the exception criteria. In mature programs exceptions should be the exception, not the norm. However, in newer security programs exceptions can be a useful tool that provides discovery as well as risk acceptance.

Maturing A Security Program

One of the first things a new CISO will need to do is understand the business and how it functions. As part of this process the CISO will need to take an inventory of the current state of things so he or she can begin to form a strategy on how to best manage risk. As a new CISO your security program may not have well defined security policies and standards. As you begin to define your program and roll out these policies, the exception process can be a valuable tool that gives the perception of a choice, while allowing the security team to uncover areas of the business that need security improvement. Over time, as the business and security program mature, the CISO can gradually deny any requests to renew or extend these exceptions.

Rolling Out A New Security Process

Another area that is useful to have an exceptions process is when rolling out a new security process. For example, if you are rolling out a new process that will require teams to perform SAST and DAST scanning of their code and fix vulnerabilities before going into production, then allowing security exceptions during the initial rollout of the process can be useful to allow teams more time to adapt their development processes to incorporate the new security process. Allowing exceptions can foster good will with the development team and allow the security function visibility into the behavior and culture of the rest of the business. This can allow the security function and development team the opportunity to collaborate together with the ultimate goal of removing any exceptions and following the process to reduce risk to the business.

Tackling Security Tech Debt or Shadow IT

A common maturity evolution for companies is the elimination of shadow IT. The security function can assist with the elimination of shadow IT by creating an exception process and allowing an amnesty period where the business is allowed to continue to operate their shadow IT as long as it is declared. In reality you are giving the business the perception that they will be granted an exception when they are really giving the security function visibility into things they wouldn’t otherwise know about. This can be a useful tool to discover and eliminate policy exceptions as long as it is used sparingly and with good intent (not punitively).

Documentation Is Key

No matter how you choose to use exceptions within your security program there are a few best practices to follow.

Exceptions should be truly exceptional. If you do grant one for discovery purposes make sure there is a plan to close the exception. Exceptions shouldn’t be the rule and they shouldn’t be expected. Sometimes the rest of the business just needs someone to tell them no.
Time box the exception. Don’t just grant an exception without some sort of end date. The business needs to know an exception is temporary and there should be a well defined plan to make improvements and close the exception. The security team should grant a reasonable amount of time to execute that plan, but it shouldn’t be a never ending story.
Review often. Security exceptions should be reviewed often. Part of your security program should review the open exceptions, which ones are ending, if there are patterns where there are lots of similar exceptions and if there are teams who request a high volume of exceptions. Reviewing exceptions gives you insight into how well security processes and controls are working. It also gives you insight into which parts of the business need help.
Require the business owner to sign off. The reality of a well run security program is the business ultimately owns the decision if they want to accept a risk or not. The CISO makes a recommendation, but they don’t own the business systems or processes. As a result, the security exception process should require the business owner to sign off on any exception. This will ensure there is documentation that they were made aware of the risk, but this can also act as a visibility tool for the business owner into their own teams. I’ve often found a business leader is not always aware of what their teams are doing at the tactical level and the exceptions process can provide them the opportunity to check their team and correct behavior before it gets to the CISO.

Wrapping Up

The exception process can be a valuable tool for discovery of hidden risk throughout the business. By offering an amnesty period and giving the perception of flexibility, the security team can foster good will with the business while gaining valuable visibility into areas that may be hidden. The exception process also is a valuable tool for the security program to document risk acceptance by the applicable business owner, but can also provide business owners visibility into how well their team is meeting security requirements. Lastly, as the security program matures, the security team can gradually require the business to close down the exceptions by improving their security posture.

Compliance Corner Series: Building A Successful Security & Compliance Program

This blog post is part of the Compliance Corner Series developed in partnership with Milan Patel. This series includes a variety of discussion topics around the intersection of security and compliance. The series includes blog posts, live web streams (with Q&A) and podcasts.

What are the primary factors, considerations, and challenges that a security leader needs to consider to build a successful security and compliance program?

If you were starting a security and compliance program from scratch where would you start?

Lee: I’ve written about starting security programs from scratch in past blog posts, but in my opinion a security leader needs to first select what security controls framework they want to use to inform their strategy, initial assessment and planning. It is rare that a security leader will enter an environment that has zero security and so my suggestion for any leader is to conduct an initial baseline assessment for how well the organization is meeting the security controls within the framework they select. Two of the most popular security control frameworks in the United States are the CIS Top 18 Critical Security Controls and the NIST Cybersecurity Framework. Both of these are excellent frameworks that cover everything a comprehensive security program needs to have along with a recommended prioritized order. It will be difficult for a security leader to start a compliance program without baselining and mapping their environment to a security controls framework because this is the equivalent of never practicing a sport, yet going out and trying to play in a game. Security is the practice and discipline that sets the foundation for successful compliance programs i.e. the sport you are playing. Your security strategy will inform the what skillsets you need to hire, what tooling you need to build or buy and what processes you need to establish to make your security program successful.

Once you have selected a security control framework and have a strategy in place, my recommendation is to then evaluate your business for the markets it addresses, the geographic regions it operates in and the customers or partners it serves. This will inform your compliance team what compliance frameworks and audits they need to consider to make the business successful. For example, if you are a consumer based business and process credit card payments, then you will need to plan to comply with and pass a PCI-DSS audit or if you plan to serve the U.S. Federal Government you may need to achieve some level of FedRAMP. Understanding the business drivers, customer demand and regulatory environment is extremely important to setting up a successful compliance program that enables the business.

Milan: I definitely agree that you must start with a baseline assessment of where you are, but also with what your business goals are. Are you going into financial spaces and require SOC1 Type 2? Do you want to baseline your core expectations with SOC2? What about healthcare, HIPAA anyone? We won’t even get into FedRAMP if you want to sell to government agencies, so I suggest that you need to really understand, and baseline, your business goals in terms of verticals, industries, even governments/regions you want to sell into. The compliance leader will need to balance out these requirements (including privacy, accessibility, etc), to ensure that for whatever region/country/industry you want to go into, you can cover the core baseline controls.

Once you have that, you need a way to track and ensure you can meet those controls. Like we talked about before, and as I talk about now, integrating this with engineering and the appropriate metrics are key. This includes how to “pre-audit” your services for new frameworks/certifications so you can minimize issues during an actual audit. If you don’t do this from the start, retro-fitting it in after the fact is difficult and costly.

You also then need personnel that understand these controls and requirements, and importantly how to translate and negotiate these with engineering. If the people you hire cannot do that, then you fall into the same old challenge of friction between engineering and compliance that introduces risk. Having the right personnel that can translate and bring harmony among the pillars required for compliance (compliance, engineering, security, and operations) will be key to standing up a successful compliance team.

2. What if you inherit a security and compliance program that has been established by your predecessor? What advice do you have for leaders when adopting and continuing a program established by someone else?

Lee: Congratulations if you have been promoted or landed a new role as the leader of a security or compliance program! In my opinion the fundamentals don’t change whether you are starting a program from scratch or inheriting an existing program. I think it is critically important for a leader to baseline and understand what is already in place, what gaps exist and then build their strategic plan to address those gaps. The big difference when you inherit a program is it may take longer for the business to shift priorities towards the new plan due to organizational momentum and budget cycles. Technologically and practically, it may not actually take that long to do the things on your plan, but psychologically the organization will need to adjust and adapt. This means you will need to spend more time building alliances, getting teams on board, changing processes and shifting the culture to support your new vision. Keep in mind that closing the gaps in security control frameworks or compliance activities may take a significant amount of time because the organization may need to procure new tools, hire new talent or create new processes to satisfy a specific control. This has a knock on effect to your compliance program because most compliance audits have both readiness audits and the actual audit themselves. Both of these can take months to complete and so expectations need to be set for timelines, critical paths and when objectives will be met. If the business is expecting to get FedRAMp or HyTrust overnight, that is unrealistic because there are robust processes and timelines that need to be followed to pass those compliance activities.

Milan: I still agree that it doesn’t really change much from standing up a new compliance program, other than you may have a bit more work to change the culture from bad practices to new ones. The key is still to align your compliance strategy with engineering goals/metrics. Inheriting a program that doesn’t align to engineering requires immediate attention, as until you can align it, it will be difficult to help the cultural and behavioral changes needed to more closely align priorities and drive the actual execution.

As you say, it will likely still take time to close gaps, hire individuals, build tooling, but if the actual metrics and success criteria isn’t aligned with what engineering will need to have to actually change priorities, then most of that work may become “wasted” as while it may get you through a single audit, it won’t be efficient or at scale. I would still suggest that new compliance org leaders take a deep look at how the compliance team articulates and aligns success metrics first, before doing anything else, and change/align them to engineering first otherwise they will continue to have friction, and risk, with executing the compliance programs that require engineering (or even operations and security) changes.

3. What challenges can a leader expect to face when building or maturing a security and compliance program and what advice do you have to overcome these challenges?

Lee: There are a few challenges that I have run into over the course of my career. The first challenge is mis-alignment of expectations between stakeholders. This could be a mismatch from the board, from other C-Suite executives or from the rest of the business (like sales). In these cases, it is imperative for the security leader to work closely with these stakeholders to align expectations. You are the expert in this field and you need to help these other leaders and teams understand what good looks like and when they can expect to achieve it.

Expectation mis-alignment is also closely related to cultural mis-alignment. For example, if the technical culture is extremely permissive and allows teams to do whatever they want, then the security leader will need to spend time explaining the advantages of increased security control and how that can actually accelerate the business, while reducing risk. Culture mis-alignment is often the biggest challenge I’ve run into over the course of my career and it can be very time consuming to fix. Security leaders are often brought in to improve a security program, when in reality the culture is at issue. This often means the security team becomes a cultural change agent and they cannot change the culture alone. Security leaders will burn out or leave extremely quickly if the culture is not aligned and they don’t have the support of other executive leaders to achieve the security goals specified in the security strategy.

Finally, one of the other issues I’ve run into is budgetary. This can come in two forms – first, the security team struggles to get appropriate funding and second, the business insists the security or compliance team performs activities that may not actually make sense financially (like achieving a specific compliance audit). The first budgetary issue – not getting appropriate funding can be a real problem for the security team if the business arbitrarily cuts budgets instead of choosing to fund the security team based on risk. This funding issue is closely related to a mismatch of stakeholder expectations and a cultural misalignment. The second budgetary issue is often an issue between sales teams and the security and compliance team. Often, the sales teams insist a specific security control is necessary or a certain compliance framework is required in order to exist in a certain market or achieve a sale. While this can often be the case, it is important the security leader has the support to be able to push back on these requests where it makes sense. If a security and compliance activity is insisted on by the sales team, I always require a minimum threshold of business to cover the cost of the control and compliance activity. This should be written into the sales teams number, tied to their comp and ultimately they should be held accountable for justifying the need.

Milan: I still think the biggest challenge for compliance leaders is building credibility. I talk about how leaders look at all companies as “making money, saving money, or costing money”, and compliance has always been put into “costing money”. So how does a compliance leader build their individual and team credibility with the engineering teams (the main control owners), that they are just not a pain in their collective engineering sides “costing money”?

Compliance leaders must start from the beginning showing engineering how they are “helping them”, which mainly to me is “saving money” (engineering/operations/security time). Time is the key metric that is tracked by most engineering leaders (sprint and engineering time), so it’s incumbent on compliance leaders to show them how they are efficient, or even saving time based on their previous compliance experience. Compliance leaders must be able to show, on paper with metrics, that the compliance work is efficient (I use first time resolution as a critical metric), and how we continuously drive for the ultimate metric, of removing engineering completely from the evidence collection flow via automation, etc.

If you cannot build that trust, then you will always just be a leader that doesn’t understand the engineering priorities, cost them money (i.e. time) by being inefficient, and having no credibility and influence as they have no real reason to engage with you. It all starts with the ability to develop and agree on metrics that actually help engineering minimize the time it takes to do auditing/etc, and providing that value back to the actual engineers.

You get risk mitigation by default with the right metrics, as the evidence is clear and concise, and once first time resolution is high, you can now automate it. We have seen that the risk (and the deviations) have gone way down as we negotiate and approve with external auditors the evidence that will pass that QA for review, we have less “drift” with bad evidence, which reduces overall risk (and actually increases predictability and timeliness in audits).

I also believe that the requirements “accepting” of new frameworks requires a much more detailed review and acceptance than I’ve generally seen. There is no point gaining new certifications if sales doesn’t have a solid plan to get the ROI with customers. It’s costly to go through the certifications, and without appropriate sales to support the cost, it can become wasted work. If the cost/benefit numbers and plans for sales are good, then it becomes much easier to justify the compliance work. Also, if the success metrics and tracking of workload for compliance is also detailed, then justifying resourcing for new projects becomes easier. Compliance teams must continuously run compliance “like a business” being able to show metrics that detail out the work, down to the team and individual level, otherwise justifying new resources becomes challenging.

Compliance Corner Series: Compliance Landscape 2023

This blog post is part of the Compliance Corner Series developed in partnership with Milan Patel. This series will include a variety of discussion topics around the intersection of security and compliance. The series will include blog posts, live web streams (with Q&A) and video blogs.

“How has globalization, increasing regulatory requirements from governments and industries change how Security and Compliance must operate internally”

Growing Regulatory Requirements – Impact To The Whole Company vs Single Business Unit

Lee – From a security perspective, it is difficult to standardize security controls and baselines across the whole company vs. a single business. Business lines have different requirements, customers and industries, which are unique to their product lines and customer base. Mandating the entire company meet a global regulatory requirement (like PCI-DSS, FedRAMP or HIPAA) can be costly for those lines of businesses that don’t have customers in those industries where these regulations apply. While the security program may be effectively managing risk for their particular line of business, the difference in regulatory requirements can lead to gaps in customer expectations across product lines or lines of business. It can also be costly to maintain compliance programs since you typically have to audit and produce evidence for each product. This puts a burden on development teams and can take time away from improving the security of their products, vs. generating sufficient evidence to pass audits. For smaller lines of businesses this can be difficult to justify the cost to keep up with global regulations, certify audits and maintain levels on par with the rest of the company.

It is possible to centralize compliance activities across the company, but it also requires every part of the company to standardize their security controls, risk management practices and reporting principles. This may work well for small to medium sized companies that have grown organically or who have done a good job of homogenizing their organization. For larger organizations that have grown via acquisition and have dozens of lines of businesses and product lines, centralizing compliance and security activities may be impossible.

Milan – For compliance, the issue is that many of these new requirements have no “corporate” baseline expectations. They have not been translated (and sometimes agreed to), by individual lines of business. Without a standard “bar” to execute against, it’s very difficult for individual compliance teams to attest that it’s being met. In many cases, these items have no tests, or have not even been implemented. It becomes a real issue at scale, as these are yet another new compliance item that engineering will have to meet, which will continue to compete with their feature development time requirements.

My experience is that it’s challenging for many of the central compliance/legal teams (that try to set these requirements for engineering teams), from both a positional authority, as well as a credibility perspective. There are few engineers on these central compliance teams that are experienced with translating these regulations to actual engineering practices. Also, there is a lack of organizational authority to enforce it uniformly across all business groups. This just increases the challenge as it adds more time to negotiate and drive priority, especially with the speed of these incoming regulations.

2. Security and Compliance Practices – What Has Changed?

Lee – For security, there has been an increasing trend in transparency as well as a reduction in time to fix security vulnerabilities and report security incidents. Often these regulations or new compliance practices have good intent, but will be so costly to implement that they are impractical or are technologically impossible. These problems are further compounded by modern software development practices like DevOps that move incredibly quickly and can amplify deployment of internet scale products across cloud providers. Software supply chain security and reporting for compliance reasons is particularly challenging due to the complexity of modern software libraries that have multiple interlinking dependencies. Third party libraries and open source software is even more difficult because you don’t always know if you can trust the source code of that library. Providing an accurate software bill of materials (SBOM) can be a significant burden on the business to develop and maintain for compliance reporting purposes. Additionally, the priority of the business can sometimes take precedence over other activities like solving tech debt or modernizing technology stacks. Technical debt can not only hold back the business from optimizing and becoming more efficient, but can also make security and compliance activities challenging.

Milan – I’ve been doing this for a long time, on the engineering side with a large multi-billion dollar software product, and more recently on the compliance side across many products. What I can say is that while engineering has changed… DevSecOps, updating features/software monthly, instead of yearly… or even every 5 years when hardware went “end of life” (yes, I’m totally aging myself here), compliance practices basically have not changed internally.

Even major companies are still trying to “throw money” at the problem. The internal compliance teams know that the SLT has to pay for compliance… but I believe the days of the compliance “bat” and “blank checkbook” are gone. It doesn’t scale. I will never get more bodies to manually collect and review evidence for large amounts of controls and services (and not regulatory requirements) that we will have to test and account for. Spreadsheets and email won’t work anymore. Compliance needs to change, and be run more like a business. Even the third party auditors didn’t have incentive to be more efficient… remember, they mostly bill hours… So the more inefficient and longer it takes for a company to do an audit, the more money they make. We’ve changed that quite a bit at Oracle, but it’s still early in the journey.

3. Looking Ahead: How Must Security and Compliance Practices Change?

Lee – For security, teams need to shift to a continuous compliance model. Instead of periodically assessing the business once a year (or some other interval) teams need to implement security controls and reporting mechanisms that can automate and continuously evaluate how the security controls meet the various regulatory requirements required for the business. By moving to a continuous compliance model it will force the business to modernize their Software Development Life Cycle (SDLC), modernize their technology stacks and resolve technical debt, which will ultimately benefit security programs. A continuous compliance program will allow security teams to tackle security at multiple points across the technology stack. First, it will evaluate, report and control how software is written. Second, it will evaluate, report and control how infrastructure is deployed and third, it will evaluate, report and control how security posture changes over time so teams can react to security drift.

Security and compliance practices must also change by holding the rest of the business accountable for their decisions. It is impossible for a security or compliance team to be successful without support from the rest of the business because the security and compliance teams don’t “own” the infrastructure and products the business uses to make money. Therefore, there needs to be support at the highest levels of executive leadership to hold the lines of business accountable and tie performance and incentives to how these businesses satisfy their security and compliance requirements.

Milan – Well, I think it must happen on two levels. First level is central oversight and governance.

Without a strong central compliance function that can translate, and negotiate requirements, enforce and govern equally with the engineering leaders, it won’t scale effectively. The main challenge now is that the “fox is guarding the hen house” so to speak. When the engineering team compliance is solely under engineering leaders in the product group, there is the ability (or even just the appearance) that decisions are made without compliance “equality” of decision making. If a compliance item isn’t resolved, who questions whether that amount of risk is acceptable to the senior product leader? Even if accepting the risk is “OK”, any external party can question whether the right level of compliance leadership made that decision, and not just an engineering leader who may prioritize compliance lower than say… a feature request.

This mostly only becomes an issue after the fact. A compliance issue happens, then the questions of “Who made that decision… why was that compliance issue not escalated… etc? Escalate to who? While it’s a great idea that a compliance leader (buried usually multiple levels down in an organization), will be “politically unencumbered” and rise up and strongly disagree with an issue with their senior leader (who by the way signs their checks). My experience is that it can be difficult, as being seen as an antagonist is not great for one’s career in this space. I’ve noticed that compliance positions that have high turnover, this has been one of the main indicators of how effective the compliance leader is in the organization feels they are. If they feel they cannot speak up, or they do and are dismissed “for higher priorities”, they feel they are not effective, or will become the scapegoat if there is an issue.

The second level is that compliance needs to change their success metrics, and run like a business.

Most senior leaders I know, including myself, put everything into three buckets… You’re making money… saving money… or costing money… Compliance has always been seen as “overhead” and costing money. Even today, I’ve had to battle that, until we made some major changes in how we operate, and what metrics we track for success. So the question is, if I’m always seen as “costing money”, how do I change that, and what do I change it to? I realized that while compliance does “make money” (new frameworks open new markets), I don’t really get to claim that. I’m not out there hustling deals… So I realized years ago I could “save money”, but not in the traditional sense. I had to change my compliance principles and metrics. First, everyone accepts that compliance is important, but why did they hate it so much? As an engineer doing this on a large project, the compliance team never understood that my metric was “completing the audit” or “collecting XXX number of evidence pieces”.

My engineering metric of success was TIME.

As a development operations, release manager, and development compliance owner for the large project, I only had so many engineering hours, so I needed to minimize time I spent on each engineering function, including compliance.

So in compliance, I changed the success metric to First Time Resolution (FTR) for audit evidence. Everyone knows you have to collect at least one, but we had so many “kickbacks” for many reasons.. Either the engineering team didn’t provide what was asked for, or the evidence instructions were incomplete or unclear (remember what I said earlier for “billable hours” from the auditors)? I put at least 30 min per kickback. I used to own 1000 pieces of evidence, sometimes I could resolve them in 5 min, sometimes it took days. By raising the FTR up to a goal of 95% (makes it through the first time), I can show, on paper… how much money I’m saving the engineering team (in time). I was able to show this year, saving millions of dollars in audit costs (by going with an efficient auditor, that didn’t bill me for all those hours), and saving Tens of Thousands of engineering hours by eliminating the “unplanned work” of kickbacks.

By changing the metrics to what the engineers care about, it built trust and transparency, and increased the overall value of compliance, plus their willingness to look for more efficiencies.

Needless to say there are a lot more details to talk about for the journey to get here, but the engineering leaders saw, with hard metrics, that compliance “saved money” (via time) for them to use on other features, bug fixing, etc. And more importantly, it built trust with them that the compliance team was a good steward of their currency, which is time.

4. Where do we go from here?

Lee – Milan, it’s been great discussing how we evolved to this state, and some great ideas and positions on what has to change in our respective security and compliance areas. There is so much to discuss here I’m looking forward to continuing this series to dive into topics around the intersection of compliance and security.
Milan – Yes Lee, it’s been great discussing these topics. For me, I feel like this is the beginning of another Industrial Revolution for Compliance, we have seen the change in the engineering and globalization, and we’re on the verge of revolutionizing the Security and Compliance space. We didn’t even really get into the overall governance and continuous security/compliance versus point in time. Maybe that’s the next discussion!