If Data Is Our Most Valuable Asset, Why Aren’t We Treating It That Way?

There have been several high profile data breaches and ransomware attacks in the news lately and the common theme between all of them has been the disclosure (or threat of disclosure) of customer data. The after effects of a data breach or ransomware attack are far reaching and typically include loss of customer trust, refunds or credits to customer accounts, class action lawsuits, increased cyber insurance premiums, loss of cyber insurance coverage, increased regulatory oversight and fines. The total cost of these after effects far outweigh the cost of implementing proactive security controls like proper business continuity planning, disaster recovery (BCP/DR) and data governance, which begs the question – if data is our most valuable asset, why aren’t we treating it that way?

The Landscape Has Shifted

Over two decades ago, the rise of free consumer cloud services, like the ones provided by Google and Microsoft, ushered in the era of mass data collection in exchange for free services. Fast forward to today, the volume of data growth and the value of that data has skyrocketed as companies have shifted to become digital first or mine that data for advertising purposes and other business insights. The proliferation of AI has also ushered in a new data gold rush as companies strive to train their LLMs on bigger and bigger data sets. While the value of data has increased for companies, it has also become a lucrative attack vector for threat actors in the form of data breaches or ransomware attacks.

The biggest problem with business models that monetize data is: security controls and data governance haven’t kept pace with the value of the data. If your company has been around for more than a few years chances are you have a lot of data, but data governance and data security has been an afterthought. The biggest problem with bolting on security controls and data governance after the fact is it is hard to reign in pandoras box. This is also compounded by the fact that it is hard to put a quantitative value on data, and re-architecting data flows is seen as a sunk cost to the business. The rest of the business may find it difficult to understand the need to rearchitect their entire business IT operations since there isn’t an immediate and tangible business benefit.

Finally, increased global regulation is changing how data can be collected and governed. Data collection is shifting from requiring consumers to opt-out to requiring them to explicitly opt-in. This means consumers and users (an their associated data) will no longer be the presumptive product of these free services without their explicit consent. Typically, increased regulation also comes with specific requirements for data security, data governance and even data sovereignty. Companies that don’t have robust data security and data governance are already behind the curve.

False Sense Of Security

In addition to increased regulation and a shifting business landscape, the technology for protecting data really hasn’t changed in the past three decades. However, few companies implement effective security controls on their data (as we continue to see in data breach notifications and ransomware attacks). A common technology used to protect data is encryption at rest and encryption in transit (TLS), but these technologies are insufficient to protect data from anything except physical theft and network snooping (MITM). Both provide a false sense of security related to data protection.

Furthermore, common regulatory compliance audits don’t sufficiently specify protection of data throughout the data lifecycle beyond encryption at rest, encryption in transit and access controls. Passing these compliance audits can give a company a false sense of security that they are sufficiently protecting their data, when the opposite is true.

Just because you passed your compliance audit, doesn’t mean you are good to go from a data security and governance perspective.

Embrace Best Practices

Businesses can get ahead of this problem to make data breaches and ransomware attacks a non-event by implementing effective data security controls and data governance, including BCP/DR. Here are some of my recommendations for protecting your most valuable asset:

Stop Storing and Working On Plain Text Data

Sounds simple, but this will require significant changes to business processes and technology. The premise is the second data hits your control it should be encrypted and never, ever, unencrypted. This means data will be protected even if an attacker accesses the data store, but it also will mean the business will need to figure out how to modify their operations to work on encrypted data. Recent technologies such as homomorphic encryption have been introduced to solve these challenges, but even simpler activities like tokenizing the data can be an effective solution. Businesses can go one step further and create a unique cryptographic key for every “unique” customer. This would allow for simpler data governance, such as deletion of data.

Be Ruthless With Data Governance

Storage is cheap and it is easy to collect data. As a result companies are becoming digital data hoarders. However, to truly protect your business you need to ruthlessly govern your data. Data governance policies need to be established and technically implemented before any production data touches the business. These policies need to be reviewed regularly and data should be purged the second it is no longer needed. A comprehensive data inventory should be a fundamental part of your security and privacy program so you know where the data is, who owns it and where the data is in the data lifecycle.

The biggest problem with business models that monetize data is: security controls and data governance haven’t kept pace with the value of the data.

Ruthlessly governing data can have a number of benefits to the business. First, it will help control data storage costs. Second, it will minimize the impact of a data breach or ransomware attack to the explicit time period you have kept data. Lastly, it can protect the business from liability and lawsuits by demonstrating the data is properly protected, governed and/or deleted. (You can’t disclose what doesn’t exist).

Implement An Effective BCP/DR and BIA Program

Conducting a proper Business Impact Analysis (BIA) of your data should be table stakes for every business. Your BIA should include what data you have, where it is and most importantly, what would happen if this data wasn’t available? Building on top of the BIA should be a comprehensive BCP/DR plan that appropriately tiers and backs up data to support your uptime objectives. However, it seems like companies are still relying on untested BCP/DR plans or worse solely relying on single cloud regions for data availability.

Every BCP/DR plan should include a write once, read many (WORM) backup of critical data that is encrypted at the object or data layer. Create WORM backups to support your RTO and RPO and manage the backups according to your data governance plan. Having a WORM backup will prevent ransomware attacks from being able to encrypt the data and if there is a data breach it will be meaningless because the data is encrypted. BCP / DR plans should be regularly tested (up to full business failover) and security teams need to be involved in the creation of BCP/DR plans to make sure the data will have the confidentiality, integrity and availability when needed.

Don’t Rely On Regulatory Compliance Activities As Your Sole Benchmark

My last recommendation for any business is – just because you passed your compliance audit, doesn’t mean you are good to go from a data security and governance perspective. Compliance audits exist as standards for specific industries to establish a minimum bar for security. Compliance standards can be watered down due to industry feedback, lobbying or legal challenges and a well designed security program should be more comprehensive than any compliance audit. Furthermore, compliance audits are typically tailored to specific products and services, have specific scopes and limited time frames. If you design your security program to properly manage the risks to the business, including data security and data governance, you should have no issues passing a compliance audit that assesses these aspects.

Wrapping Up

Every business needs to have proper data security and data governance as part of a comprehensive security program. Data should never be stored in plain text and it should be ruthlessly governed so it is deleted the second it is no longer needed. BCP/DR plans should be regularly tested to simulate data loss, ransomware attacks or other impacts to data and, while compliance audits are necessary, they should not be the sole benchmark for how you measure the effectiveness of your security program. Proper data protection and governance will make ransomware and data breaches a thing of the past, but this will only happen if businesses stop treating data as a commodity and start treating it as their most valuable asset.

Security Considerations For M&A and Divestitures

I’ve been speaking to security startups over the last few weeks and some of the discussions made me think about the non-technical aspects of security that CISOs need to worry about. Specifically, things like mergers, acquisitions and divestitures and the different risks you will run into when executing these activities. There are a number of security issues that can materialize when combining businesses or separating businesses and in this post I’ll share some of the things you need to think about from a security perspective that may not be obvious at first glance.

What’s Going On Here?

There are a number of reasons for mergers & acquisitions (M&A) or divestitures. For the past two decades, the tech industry has used M&A to acquire smaller startup companies as a way to collect intellectual property, acquire specific talent or gain a competitive advantage. Divestitures may be the result of changing business priorities, separating business functions for regulatory reasons, eliminating redundancies or a way to sell a part of the business to cover costs. Mergers, acquisitions and divestitures are similar because you will want to review the same things from a security perspective, but it is probably easiest to think of divestitures as the reverse of an M&A – you are separating a business instead of combining a business. Divestitures are definitely less common than M&A in the tech space, but they aren’t unheard of. There are also differences in terms of the security risks you need to think about depending on if you are acquiring a business or separating a business. My best advice is to work with the legal and finance teams performing the due diligence and have a set process (that you have contributed to) so you don’t forget anything. With that, let’s dive into a few different areas.

Physical Security

Physical security is something you will need to think about for both M&A and divestitures. For M&A you will want to perform a physical security assessment on the facilities you are acquiring to make sure they meet or exceed your standards. Reviewing physical security controls like badging systems, fencing, bollards, cameras, fire suppression, emergency lighting, tempest controls (if required), safes and door locks will all help make sure your new facilities are up to standard. If you aren’t sure how to perform this, hire a company that specializes in physical security assessments or physical red teaming.

While physical security for M&A may seems straight forward, there are a few gotchas when performing divestitures. The biggest gotcha is understanding and reviewing the existing access of the people that are part of the divestiture because you will now need to consider them outsiders. All of your standard off-boarding processes will apply here such as terminating accesses to make sure someone doesn’t retain access to a system they are no longer authorized to access (like HR, Finance, etc.).

Things can get complicated if parts of the business are divesting, but not fully. Some examples of this are when the business divests a smaller part, but allows the smaller part to co-locate in their existing facilities. This may complicate physical security requirements such as how to schedule or access common areas, how to schedule conference rooms, how to separate wifi and network access, etc. In the above example, the larger company may act like a service provider to the divested part of the business, but there still needs to be effective security controls in place between the two parts.

Personnel Security

I touched on this a bit already, but personnel security is something to consider when performing M&A or divestitures. With M&A the biggest issue will be how to smash the two IAM systems and HR systems together without punching huge holes in your network. Typically what happens is the two parts operate separately for a while and then consolidate to a single system and the employees of the acquired business get new accounts and access.

For divestitures, particularly if they don’t result in a clean split, you will need to focus heavily on access control and insider threats. Think about how you will separate access to things like source code, financial systems, HR systems, etc. If the smaller company has physical access to your space then you need to build in proper physical and logical controls to limit what each business can do, particularly for confidentiality and competitive reasons.

What’s an example of where this can go wrong? Let’s say business A is going to divest a small part of its business (business B). The complete divestiture is going to take a while to finalize so company A agrees to allow company B to continue to access their existing office space, including conference rooms. However, the legal team didn’t realize the conference rooms are tied to company A’s SSO and calendaring system so company B has no way to schedule the conference rooms without retaining access to company A’s IAM system creating a major security risk. Whoops!

The biggest gotcha is understanding and reviewing the existing access of the people that are part of the divestiture because you will now need to consider them outsiders.

Contracts

Contracts may not seem like a typical security issue, but they should be part of your review, particularly when performing M&A. Why? You are acquiring a business that is worth something and that business will have existing contracts with customers. The contractual terms with those customers may not match the contractual terms of the acquiring company, which can cause a risk if there is a significant difference in contract terms. Smaller companies are more agile, but they also usually have less negotiating power compared to large companies and as a result are more likely to agree to non-standard contract terms. What are some terms you need to think about?

  • Vulnerability Remediation Times – How quickly did the new company promise to fix vulnerabilities for their customers?
  • Incident & Breach Disclosure Time Frames – How quickly did the new company promise to notify customers of a breach or incident? I have seen very small time frames suggested in contracts, which are impossible to meet, so I definitely recommend reviewing these.
  • Disclosure of Security Postures – Does the new company have contractual terms promising to provide SBOMs or other security posture assessments to their customers on a regular basis?
  • Compliance Requirements – Has the new company agreed to be contractually obligated to maintain compliance certifications such as PCI-DSS, SOC 2, ISO27001, etc.
  • Penetration Testing & Audits – Has the new company contractually agreed to have their products or services penetration tested or have their security program audited? Have they agreed to provide these reports to their customers on a regular basis?
  • Privacy & Data Governance Terms – Is the new company required to comply with privacy regulations such as allowing customers have their data deleted, or mandating certain data governance requirements like DLP, encryption, data deletion, etc?
  • BCP/DR and SLAs – Are there contractual uptime SLAs or response times and does the existing BCP/DR plan support these SLAs?

My advice is to set a timeline post acquisition to review and standardize all of your contracts to a single set of standard clauses covering the above topics. This is usually part of a security addendum that the legal team can help you create. The biggest challenge with contracts will be to “re-paper” all of your customers to hopefully get them on the same standardized contract terms so your security program doesn’t have a bunch of different requirements they have to try to meet.

Accuracy Of M&A’s

One of the biggest risk of performing M&A’s is trying to get an accurate picture of the existing security posture of the company being acquired. Why is this so difficult? The company being acquired is trying to look as good as possible so they get top dollar. They can’t hide things, but they aren’t going to tell you where all the skeletons are buried either. The acquiring company usually doesn’t get a full picture of the existing security posture until after the deal is done and you start trying to integrate the two parts of the business. If you have a chance to interview the existing security team before the M&A closes definitely ask to see their latest audit reports, compliance certifications, penetration testing reports, etc. Consider working with legal to set conditions for how old these reports can be (e.g. no older than 6 months) to hopefully give you a more accurate picture or require the acquired company to update them before the deal closes. Interview key members of the staff to ask how processes work, what are their biggest pain points, etc. Consider hiring an outside company to perform an assessment, or you can even consider talking to one of their largest customers to get their external view point (if possible).

Wrapping Up

M&A and divestitures can be exiting and stressful at the same time. It is important for the security team to be integrated into both processes and to have documented steps to make sure risks are being assessed and addressed. I’ve listed a few key focus areas above, but most importantly standardizing your M&A security review can help avoid “buyers remorse” or creating unnecessary risk to the acquiring business. Finally, having a documented divestiture process and reviewing the divestiture with legal can help avoid security risks after the fact.

Should There Be A Professional CISO Certification and Organization?

I’ve been thinking a lot about the CISO role and how it is rapidly maturing from a technology and compliance role to a more generalized business executive role that specializes in security and risk. The primary catalyst for this evolution is the recent release of the SEC rules requiring companies to report material incidents on their 8K forms. It also requires companies to disclose their process for governing security issues (via committees or other processes) and their process for determining materiality (via their annual 10k filing). All of this is having a similar effect on the CISO role that Sarbanes-Oxley had on the CEO and CFO role after it was passed in 2002. The end result is public companies are now being expected to demonstrate investment and expertise in governing security issues, which is elevating the CISO role to become a true executive officer and is ushering the role into the board room.

Why Did The SEC Establish The New Requirements?

Security reporting and disclosures by public companies has been lacking. There has been zero incentive or accountability for companies to report these events other than via lawsuits, stock price corrections or brand and reputation impact These disclosures often happen as a result of a news report published months or years after the actual incident. The company then issues a generic statement downplaying the event and emphasizing how serious they take security. The SEC has determined this pattern of behavior is insufficient for investors to accurately make decisions about the health of the company.

Why Do Professional Certifications Exist?

Professional certifications exist for a number of reasons. Doctors, accountants. professional engineers and lawyers all must demonstrate a minimum level of knowledge to get licensed in their chosen profession. They must also agree to conduct themselves according to a specific code of conduct. This allows the practitioners to wield specific credentials demonstrating proficiency and credibility in that field. Displaying professional credentials attests these professionals bear the responsibility to protect life, prevent fraud or protect assets.

Additionally, professional credentials afford the practitioners a number of benefits such as knowledge sharing, continual career development, job placement and act as a back stop if someone’s conduct is called into question. Certifying organizations can testify on someone’s behalf if they believe they have upheld the requirements of the profession, or they can self regulate and strip someone of their credentials for fraud or gross negligence.

A short list of fields with professional certifications are as follows:

  • Lawyers – Bar
  • Doctors – Medical license, National Board of Medical Examiners (NBME), State level licenses, American Board of Medical Specialities (ABMS)
  • Accountants – Financial Accounting Standards Board (FASB), Government Accounting Standards Board (GASB), Generally Accepted Accounting Principles (GAAP), Certified Public Accountant (CPA)
  • Engineers – Certified Professional Engineer (CPE)
  • Privacy Professionals – International Association of Privacy Professionals (IAPP)

Existing Security Certifications And Organization Are Lacking

There are already a number of certifications security professionals can choose from on their path to becoming a CISO. A short list of common certifications listed on CISO job postings or LinkedIn profiles is as follows:

  • C|CISO
  • CISSP
  • CISM
  • CISA
  • CRISC

Of these certifications, only the C|CISO certification comes close to offering a specific certification for CISOs. The rest serve either as generalized security certifications or specific offshoots of the security profession. These certifications are often bundled together by professionals to demonstrate breadth of knowledge in the security field.

While existing certifications are good, they are all lacking in what is needed for someone to serve as a CISO at a publicly traded company. They are more generalized about how to serve as a CISO at any company (small to large), but publicly traded companies have specific requirements and demands. Specifically, most of the certifications above are extremely heavy on a breadth of technical aspects and popular industry frameworks. Some of them do cover how to create and manage a security program. Some even cover basic board level conversations (although these are usually technical discussions, which are unrealistic). Where I find these certifications lacking is as follows:

  • Realistic board level conversations about risk and tradeoffs including building effective presentations
  • Board and legal conversations about materiality for security incidents
  • Common board committees and what to expect as a CISO serving on a board level committee for your company
  • Testifying or providing legal evidence post incident
  • Legal conversations about how to best notify customers of breaches including drafting communications
  • Legal conversations with security researchers and navigating vulnerability disclosures
  • How to establish and manage a bug bounty program
  • Navigating conversations with law enforcement or national security issues
  • How to effectively change or strengthen security culture
  • How to have conversations with other C-Suite executives about security
  • Navigating customer and industry requests for disclosure of security program information
  • Managing the budget / P&L for a security function including tooling, licenses, services, travel, expenses, equipment, certifications, etc.
  • Common security team structures and how to design a security org that add maximum value for the business
  • Personnel management, skillsets expected for different roles, matching training and certifications to job function, etc.
  • Negotiating with vendors and cyber insurance companies
  • Contract review and negotiation with customers (including common security and privacy clauses)
  • Creating RFPs, RFIs and RFQs
  • Talking to customers about security at your company or hot button security issues
  • Establishing requirements, conducting trade-off analyses and performing build vs buy analysis
  • How to effectively network with peers
  • Industry resources such as ISACs, Infraguard, etc.
  • Top recruiting agencies for placing CISOs at publicly traded companies
  • Career development post operational CISO (boards, consulting, etc.)
  • Properly documenting your security program
  • How to navigate achieving common compliance certifications such as SOC1, SOC2, FedRAMP, ISO27001, HIPAA, PCI-DSS. Typical costs, consulting companies that can help with these processes and what to expect during the process.
  • When to outsource your security program to an MSP
  • When to bring in an outside consulting or incident response firm
  • Successfully passing an external audit
  • Negotiating for a job including severance, D&O liability, assessing the role, etc.
  • Differences in the CISO role depending on who it reports to (General Counsel, CTO, CIO, CEO, CFO)
  • How to navigate common security related political and moral hazards at public companies

As you can see, there is a big difference between what certifications offer and the real demands of a public company CISO. Additionally, there are a number of professional security organizations such as the Information Systems Security Certification Consortium (ISC2), Information Systems Audit and Control Association (ISACA) and The Council of E-Commerce Consultants (EC-Council). Each has their own certification track, terminology and code of conduct. Each is good in their own right, but there is still a lack of a single certifying body for public company CISOs similar to a CPA. Arguably, ISACA comes closest to being an international organization that can back CISOs, but they lack a CISO specific certification covering the majority of the topics above.

While existing certifications are good, they are all lacking in what is needed to prepare someone to serve as a CISO at a publicly traded company.

Why There Should Be A Professional CISO Certification

The SEC requirements are forcing public companies to govern security to the same standard forced by Sarbanes-Oxley 20 years ago. The SEC considers security to be a material concern to investors and public companies need to treat the issue accordingly. As a result CISOs are getting elevated to the board room and CISOs need to be prepared to navigate the issues they will encounter while serving at a public company.

The advantages of a professional CISO certification and accompanying organization are as follows:

  • Standard of ethics and conduct – CISOs face a difficult job and often walk into roles that aren’t properly supported or properly funded. Yet, CISOs are asked to bear the responsibility and accountability for the security health of the organization. A standard of ethics and conduct, similar to a CPA, will backstop the authority of the CISO and serve as guidelines for how to navigate common issues at publicly traded companies.
  • Standard credential for publicly traded companies – Large companies face a difficult job sorting through the credentials and titles of job applicants. Most public companies hire executive recruiting firms to help navigate the sea of candidates to find ones that are truly qualified for the role. However, a single professional CISO certification would distinguish individuals who have met the standard to be a CISO at a publicly traded company and distinguish these credential holders from other individuals with discretionary CISO titles.
  • Shelter the role from (some) liability – One advantage of a professional certification like the ones for doctors, engineers, lawyers and public accountants is it provides a standard of conduct. These professionals can fall back on this standard of conduct if their professionalism is called into question and they can even have the certifying organization offer testimony on their behalf. As CISO take on more liability, a professional CISO organization can be useful to help support CISOs, testify on their behalf, offer recommendations for liability insurance policies or even provide low cost liability insurance through the organization. They can even help review employment contract terms to evaluate liability policies, severance, legal coverage, etc.
  • Board Level Expertise – One of the primary roles of public company CISOs is to present to the board and help the company navigate regulatory and compliance requirements such as SEC filings, breach notifications, etc. A professional CISO certification offer individuals this experience and it can give them the confidence to speak to the board on how to navigate topics of risk. By certifying individuals are qualified to operate in the board room the board will gain another voice to balance the other C-Suite executives who aren’t grounded in technology and security issues.
  • Consulting and auditing – One final advantage of a professional CISO certification is for the “big 4” consulting firms or other agencies who are contracted by investment companies to audit and certify the filings and reports of public companies. In this case, a certified CISO can represent shareholders and investors for the accuracy of security filings around governance processes, representation in board committees, recommendations for appropriate investment in security governance and generally offering advice on industry best practices for security governance at publicly traded companies.

Wrapping Up

I’m bullish on the CISO role long term because I think it is the ultimate C-Suite executive. Public company CISOs touch all aspects of the business, they need to have strong technical chops, need to understand business topics and need to have the political chops to build alliances and navigate big company politics. Existing security certifications are good, but none of them offer a comprehensive breadth of topics to prepare individuals to become a CISO at a publicly traded company. As CISOs establish their role and credibility in the board room, it will become critical for these individuals to have credentials that back their experience, offer support and can elevate the CISO role on par with other C-Level execs, similar to what Sarbanes-Oxley did for CFOs after 2002.

Are We Peak CISO?

Let’s be honest…the CISO role is weird right now. It is going through a transformative phase and the industry is at an inflection point similar to what other C-Level roles (like the CFO) have gone through in the past. What makes the role weird? The CISO community and any company that has a CISO is facing unprecedented regulatory pressure, the economy and interest rates have people on edge, layoffs in the tech sector have shaken employee confidence (to the applause of investors) and technology innovation via AI is causing additional disruption and risk across all sectors.

In additional to these external pressures the past few years have seen the proliferation of CISO title sprawl and confusion from companies about how to best employ and utilize a CISO (hint, we aren’t your scapegoats). Despite all of this turmoil, change is also a time for opportunity and there are a few things I think will help clarify and mature the CISO role.

CISO Title Sprawl

I’ve been tracking job titles and job postings on LinkedIn for the past year or so and I’ve noticed a phenomenon I’ll call title sprawl. A quick search for titles shows there are vCISOs, Advisory CISOs, Fractional CISOs, CISOs In Residence and Field CISOs. On top of this, add in Chief Security Officers, Chief Trust Officers and Heads of Security. Do we need all of these titles? Maybe, but I think this title sprawl is more indicative of three things 1) People with CISO titles are in high demand and people want to retain the title once they get it and 2) Companies are still uncertain about how to title and employ someone to lead their security function. 3) Title sprawl is a result of the political power struggle occurring between the CISO role and other C-Level roles (more on that below).

From the titles above there are really only four functions for a current or former CISO – board member (in some capacity), executive management (officer of the company), consultant and sales. There is similar title sprawl and variance with CTO titles, but not to the extent of the CISO title (yet). Time will tell if other C-Level roles start to follow suit, but for now, let’s break down the functional CISO role buckets.

Board Member – These are current or former CISOs who sit on a board either as a technical advisor, business advisor or some combination thereof.

Executive Management – Individuals employed by a company to lead the information security program. May also manage other parts of IT such as identity, privacy, data, etc. Titles may be CISO, CSO, CISO in Residence (for Venture Capital), Chief Trust Officer and Head of Security.

Consultant – These are individuals who are providing their expertise as a current or former CISO to other companies to help them establish, transition or manage a security program. Often the companies employing these individuals claim they can’t afford a full time CISO, but they seem to be able to afford other full time C-Suite titles (hmm…)? Titles may include Virtual CISO (vCISO), Fractional CISO, CISO in Residence and Consulting CISO. (CISO in Residence again because they can “consult” to their VC holding companies about the state of their security programs).

Sales – These are people who are experts in the field of security, may hold one or more certifications and may be past CISOs. Their job is to help the company they work for drive sales. Typically the title they use is Field CISO or Advisory CISO.

Standardize The Reporting Structure

Moving on from title sprawl, companies are also confused about where the CISO title should sit. Some companies advertise it as a Director level role reporting into the VP of some function. Other’s title it as a VP level role reporting into a Senior VP or some other executive. Still other companies have the CISO reporting to the CEO, CIO, CTO or General Counsel. It is even possible this person is an individual contributor. Companies are clearly confused about whether the CISO is a technologist, regulatory compliance specialist or true C-Suite executive. While reporting structure may be a direct reflection on company culture, it is also a public example of the battle for equivalency that is playing out between the CISO and other C-Level roles. Often, CISOs are hired by other C-Levels (not the CEO) and until it becomes more common for CISOs to report to the CEO as an accepted peer to other C-Levels, this confusion and variance will persist. That being said, if you are considering a CISO title and the company isn’t willing to add you to the D&O liability policy then you may be better off taking a lower level title to eliminate personal risk.

Bolster Security Management Certifications

Security certifications from popular organizations talk a lot about regulations, risk and different security concepts (technical or not), but few, if any, offer a comprehensive certification on what it truly takes to be a CISO. Any CISO level certification should include potential career paths that lead to the CISO role, career paths post CISO role, difference in the CISO role based on company size, exposure to business topics in addition to security topics, SEC reporting, interfacing with law enforcement and lastly discussion of how to maximize success based on where the role sits – e.g. reporting to the CEO, CTO or CIO and how that may change your lens as a CISO. This begs the question if there should be a true professional level CISO certification similar to a professional engineer, accountant or lawyer, but let’s save that discussion for a future blog post.

Embrace Increased Regulation

Given the recent increase in regulation, particularly from the SEC, bolstering CISO certifications to include more business acumen may soon be table stakes instead of a nice to have. Recent regulations forcing companies to disclose material cybersecurity events in their 8k filings are starting to accelerate the maturity of the CISO role at publicly traded companies. Companies can no longer fail to invest in security or report breaches (unless they want steep penalties). In particular, this is forcing the CISO role into the board room or at least on par with other C-Level roles because they have to help these companies navigate the decision to report material events in their filings. Existing and future CISOs can embrace this increase in regulation to backstop their authority at companies who are struggling to fully embrace the CISO role as a C-Level executive. While it may not elevate the current role with a promotion, it should at least open the door to the board room and provide a seat at the table for discussion.

While CISO reporting structure may be a direct reflection on company culture, it is also a public example of the battle for equivalency that is playing out between the CISO and other C-Level roles.

The last point I’ll make about regulation is – while the SEC watered down the requirements for cybersecurity expertise on boards, I predict this expertise will still be required and in demand as companies start to navigate the new SEC reporting requirements. In particular, companies may be penalized and eventually required to demonstrate cybersecurity board expertise (via experience or certifications) if they are found to have a material security breach and can’t demonstrate appropriate security governance at the board level.

What’s The End Result?

It is clear the security industry and the CISO role are in a state of confusion as a result of the tight job market, uncertain economy, increased regulation and pace of technology innovation. The net effect of title sprawl and the struggle for equivalency is – it confuses customers, investors, partners, recruiters and job candidates. Title sprawl artificially increases competition for jobs and causes a wide variance in how the CISO role is employed. However, I think this state of confusion is a good thing because it is forcing conversations and causing people to stop and think. The CISO role is the newest member of the C-Suite and it is growing up and trading in the hoodie for a collared shirt. We are starting to claim our seat at the board level and are able to hold our own or make other C-Level roles redundant. As the CISO role evolves from a “nice to have” to a “must have” in the C-Suite, we will see this confusion fade away and the CISO role will truly reach its peak.

Security Theater Is The Worst

We have all been there…we’ve had moments in our life where we have had to “comply” or “just do it” to meet a security requirement that doesn’t make sense. We see this throughout our lives when we travel, in our communities and in our every day jobs. While some people may think security theater has merit because it “checks a box” or provides a deterrent, in my opinion security theater does more harm than good and should be eradicated from security programs.

What Is Security Theater?

Security theater was first coined by Bruce Schneier and refers to the practice of implementing security measures in the form of people, processes or technologies that give the illusion of improved security. In practical terms, this means there is something happening, but what that something is and how it actually provides any protection is questionable at best.

Examples Of Security Theater

Real life examples of security theater can be seen all over the place, particularly when we travel. The biggest travel security theater is related to liquids. TSA has a requirement that you can’t bring liquids through security unless they are 3 ounces or smaller. However, you can bring a bottle of water through if it is fully frozen…what? Why does being frozen matter? What happens if I bring 100, 3 ounce shampoo bottles through security? I still end up with the same volume of liquid and security has done nothing to prevent me from bringing the liquid through. As for water, the only thing that makes sense for why they haven’t relaxed this requirements is to prop up the businesses in the terminal that want to sell overpriced bottles of water to passengers. Complete theater.

“Security theater is the practice of implementing security measures that give the illusion of improved security.”

Corporate security programs also have examples of security theater. This can come up if you have an auditor that is evaluating your security program against an audit requirement and they don’t understand the purpose of the requirement. For example, and auditor may insist you install antivirus on your systems to prevent viruses and malware, when your business model is to provide Software as a Service (SaaS). With SaaS your users are consuming software in a way that nothing is installed on their end user workstations and so there is little to no risk of malware spreading from your SaaS product to their workstations. Complete theater.

Another example of security theater is asking for attestation a team is meeting a security requirement instead of designing a process or security control that actually achieves the desired outcome. In this example, the attestation is nothing more than a facade designed to pass accountability from the security team, that should be designing and implementing effective controls, to the business team. It is masking ineffective process and technologies. Complete theater.

Lastly, a classic example of security theater is security by obscurity. Otherwise known as hiding in plain sight. If your security program is relying on the hope that attackers won’t find something in your environment then prepare to be disappointed. Reconnaissance tools are highly effective and with enough time threat actors will find anything you are trying to hide. Hope is not a strategy. Complete theater.

What Is The Impact Of Security Theater?

Tangible And Intangible Costs

Everything we do in life has a cost and this is certainly true with security theater. In the examples above there is a real cost in terms of time and money. People who travel are advised to get to the airport at least two hours early. This cost results in lost productivity, lost time with family and decreased self care.

In addition to tangible costs like those above, there are also intangible costs. If people don’t understand the “why” for your security control, they won’t be philosophically aligned to support it. The end result is security theater will erode confidence and trust in your organization, which will undermine your authority. This is never a place you want to be as a CISO.

Some people may argue that security theater is a deterrent because the show of doing “security things” will deter bad people from doing bad things. This sounds more like a hope than reality. People are smart. They understand when things make sense and if you are implementing controls that don’t make sense they will find ways around them or worse, ignore you when something important comes up.

With any effective security program the cost of a security control should never outweigh the cost of the risk, but security theater does exactly that.

Real Risks

The biggest problem with security theater is it can give a false sense of security to the organization that implements it. The mere act of doing “all the things” can make the security team think they are mitigating a risk when in reality they are creating the perfect scenario for a false negative.

How To Avoid Security Theater?

The easiest way to avoid security theater is to have security controls that are grounded in sound requirements and establish metrics to evaluate their effectiveness. Part of your evaluation should evaluate the cost of the control versus the cost of the risk. If your control costs more than the risk then it doesn’t make sense and you shouldn’t do it.

The other way to avoid security theater is to exercise integrity. Don’t just “check the box” and don’t ask the business you support to check the box either. Take the time to understand requirements from laws, regulations and auditors to determine what the real risk is. Figure out what an effective control will be to manage that risk and document your reasoning and decision.

The biggest way to avoid security theater is to explain the “why” behind a particular security control. If you can’t link it back to a risk or business objective and explain it in a way people will understand then it is security theater.

Can we stop with all the theater?

What’s The Relationship Between Security Governance and Organizational Maturity?

Organizational and security governance is touted as a key component of any successful security program. However, I’ve been thinking about governance lately and how it relates to the overall maturity of an organization. This has prompted some questions such as: what happens if you have too much governance? and What’s the relationship between security governance and organizational maturity?

What Is Governance?

First, let’s talk about what governance is.

Governance is the process by which an organization defines, implements and controls the business.

Let’s unpack what this means for a security organization. The process of defining security for the business is done through policies, standards and guidelines. Security policies are requirements the business must meet based on laws, regulations or best practices adopted by the business. These policies align to business objectives. Implementation is done through security controls that are put in place to meet a specific policy or to manage a risk. Lastly, controlling the business is done via audits and compliance checks. The security org follows up on how well the business is following policies, implementing controls and managing risk. Control can also include enforcement, which can involve gating processes, such as requiring approval for business critical and high risk activities, or recommending additional security requirements for the business to manage a risk.

Why Do We Need Governance At All?

In an ideal world we wouldn’t. Imagine a business that is created entirely of clones of yourself. There would be implicit and explicit trust between you and your other selves to do what is best for the business. Communication would be simple and you would already be aligned. In this case you don’t need a lot (or any) governance because you can trust yourself to do the things. However, unless you are Michael Keaton in Multiplicity, this just isn’t a reality.

Governance achieves a few things for a business. First, it communicates what is required of its employees and aligns those employees to common objectives. Second, it helps employees prioritize activities. None of this would be needed if human’s weren’t so complex with diverse backgrounds, experiences, perspectives, education, etc. In an ideal world we wouldn’t need any governance at all. The reality is, we do need governance, but it needs to be balanced so it doesn’t unnecessarily impede the business.

How Does This Relate To Organizational Maturity?

Organizational maturity refers to how your employees are able to execute their tasks to achieve the objectives of the business. This relates to things like the quality of code, how quickly teams resolve operational issues or how efficiently they perform a series of tasks. It can be loosely thought of as efficiency, but I actually think it combines efficiency with professionalism and integrity. Maturity is knowing what good is and being able to execute efficiently to get there. There is a fantastic book about this topic called Accelerate: The Science of Lean Software and DevOps: Building High Performing Technology Organizations by Nicole Forsgren PhD.

Which brings us to the relationship of governance and maturity…

There is an inverse relationship between organizational maturity and organizational governance. In simple terms:

The less mature an organization, the more governance is needed.

For example, if your organization struggles to apply patches in a timely manner, continually introduces new code vulnerabilities into production or repeatedly demonstrates behavior that places the business at risk, then your organizational maturity is low. When organizational maturity is low, the business needs to put processes and controls in place to align employees and direct behavior to achieve the desired outcomes. In the examples above, increased governance is an attempt to manage risk because your employees are behaving in a way that lacks maturity and is placing the business at risk.

What causes low organizational maturity?

Organizational maturity is a reflection of employee behavior, skillset, knowledge, education and alignment. In other words, organizational maturity is a reflection of your organizational culture. In practical terms your employees may simply not know how to do something. They may not have experience with working for your type of business or in the industry you operate in. Perhaps they had a really bad boss at a past job and learned bad behavior. Whatever the reason, low organizational maturity is linked to lots of sub-optimal outcomes in business.

How To Improve Organizational Maturity?

If governance and maturity are inversely linked, the question becomes how can we increase organizational maturity so we need less governance? There are a lot of ways to increase organizational maturity. One that is fairly obvious is to start with a mature organization and maintain it over time. However, this is easier said than done and is why some organizations are fanatical about culture. This relates to everything from hiring to talent management and requires strong leadership at all levels of the company.

“The less mature an organization,
the more governance is needed.”

Other ways to improve organizational maturity are through training and education. This is why security awareness and training programs are so critical to a successful security program. Security awareness and training programs are literally attempting to improve organizational maturity through education.

One last way to improve maturity is via process. The security organization can establish a new process that all teams must follow. As teams go through this process you can educate them and reward teams that exhibit the ideal behavior by relaxing the process for them. You can also help teams educate themselves by publishing the requirements and making the process transparent. The challenge with imposing a new process is having the discipline to modify or remove the process when needed, which comes back to governance.

What’s the right level of governance?

The optimal level of governance is going to be based on your organizational maturity and desired business outcomes. In order to determine if you have too much or too little governance you need to measure organizational maturity and the effectiveness of existing organizational governance. There are industry standard processes for measuring organizational maturity, like the Capability Maturity Model Integration (CMMI) and Six Sigma, or you can create your own metrics. Some ways to measure governance effectiveness are:

  • Ask For Feedback On Security Processes – Are the processes effective? Do teams view them as an impediment or are they viewed favorably? Are the processes easy to navigate and objective or are they opaque and subjective?
  • Measure Effectiveness Of Security Controls – Are your security controls effective? If you ask a team to do work to implement a security control you should have clear metrics that determine if that control is effective. If you implement a control, but that control hasn’t changed the outcome, then the control is ineffective. This can indicate your governance is ineffective or your organizational maturity needs to improve.
  • Assess and Update Policy – Security policies should be living documents. They shouldn’t be set in stone. Security policies need to map back to laws and regulations they support and the business requirements needed to be successful. Laws, regulations and business requirements all change over time and so should your security policies. By having up to date and relevant security policies you can ensure your organizational governance matches the maturity of the business.

What Are Typical Scenarios For Governance And Maturity?

There are four scenarios related to governance and maturity:

A mature organization with too much governance – your organization is mature, but you are overly controlling with process and requirements. The net effect will be to slow down and impede the business unnecessarily. You are effectively lowering the organizational maturity due to too much governance.

An immature organization with too little governance – this is a recipe for disaster. If your organization is immature and you fail to govern the organization you will open the business up to unnecessary risk. You will get out maneuvered by your competitors, you will miss opportunities, you will fail to comply with laws and regulations and generally will have a lot of activity without any result. Your employees will lack coordination and as a result your business will suffer.

A mature organization with too little governance – This isn’t a bad scenario to be in. A mature organization implies they are doing the right things and don’t need a lot of guidance. A laissez faire attitude may be the right thing to allow employees flexibility and freedom, but it does come with inherent risk of not being compliant with laws and regulations. It may also mean there is duplication of effort or multiple ways of doing things, which could be optimized.

Governance and maturity are balanced – obviously this is the ideal scenario where your organizational governance is balanced to the level of maturity of the organization. Easy to think about in practice, difficult to achieve in reality.

Wrapping Up

Organizational governance and maturity are inversely related and need to be balanced in order for the business to operate effectively. There are ways to measure organizational maturity and governance effectiveness and by having a continual feedback loop you can optimally align both for success.

Are Traditional IT Roles Still Relevant In Today’s Modern Security Org?

As more and more businesses shift to the cloud and micro-services, the scope of responsibility for security and operations gets pushed up the stack. As a result of this scope compression, teams no longer need to worry about maintaining physical infrastructure like deploying servers, provisioning storage systems or managing network devices. As this scope falls off, the question becomes – are traditional IT roles still relevant in today’s modern security org?

Cloud Service Models

First, let’s talk about cloud service models most companies will consume because this is going to determine what roles you will need within your security organization. This post is also assuming you are not working at a hyper-scale cloud organization like AWS, Azure, Google Cloud or Oracle because those companies still deploy hardware as part of the services they consume internally and provide to their customers.

Infrastructure as a Service (IaaS)

Infrastructure as a Service (IaaS) is what you typically think of when you consume resources from a Cloud Service Provider (CSP). In IaaS, the CSP provides and manages the underlying infrastructure of network, storage and compute. The customer is responsible for managing how they consume these resources and any application that are built on top of the underlying IaaS.

Platform as a Service (PaaS)

In Platform as a Service (PaaS), the cloud service provider manages the underlying infrastructure and provides a platform for customers to develop applications. All the customer needs to do is write and deploy an application onto the platform.

Software as a Service (SaaS)

With Software as a Service (SaaS) customers consume software provided by the cloud service provider. All the customer needs to worry about is bringing their own data or figuring out how to apply the SaaS to their business.

IaaS, PaaS & SaaS Cloud Service Provider Logical Model

As you can see from the above model, organizations that adopt cloud services will only have to manage security at certain layers in the stack (there is some nuance to this, but let’s keep it simple for now).

What Are Some Traditional IT Roles?

There are a variety of traditional information technology (IT) roles that will exist when an organization manages their own hardware, network connections and data centers. Some or all of these roles will no longer apply as companies shift to the cloud. Here is a short list of those roles:

  • Hardware Engineer – Server and hardware selection, provisioning, maintenance and management (racking and stacking)
  • Data Center Engineer – Experience designing and managing data centers and physical facilities (heating, cooling, cabling, power)
  • Virtualization Administrator – Experience with hypervisors and virtualization technologies*
  • Storage Engineer – Experience designing, deploying and provisioning physical storage
  • Network Engineer – Experience with a variety of network technologies at OSI layer 2 and layer 3 such as BGP, OSPF, routing and switching

*May still be needed if organizations choose to deploy virtualization technologies on top of IaaS

Who Performs Traditional IT Roles In The Cloud?

Why don’t organizations need these traditional IT roles anymore? This is because of the shared service model that exists in the cloud. As a customer of a cloud service provider you are paying that CSP to make it easy for you to consume these resources. As a result you don’t have to worry about the capital expenditure of purchasing hardware or the financial accounting jujitsu needed to amortize or depreciate those assets.

In a shared service model the CSP is responsible for maintaining everything in the stack for the model you are consuming. For example, in the IaaS model, the CSP will provide you with the network, storage and compute resources you have requested. Behind the scenes they will make sure all these things are up to date, patched, properly cooled, properly powered, accessible and reliable. As a CSP IaaS customer, you are responsible for maintaining anything you deploy into the cloud. This means you need to maintain and update the OS, platform, services and applications that you install or create on top of IaaS as part of your business model.

Everything Is Code

One advantage of moving to the cloud is everything becomes “code”. In an IaaS model this means requesting storage, networking, compute, deploying the OS and building your application are all code. The end result of everything is code means you no longer need dedicated roles to provision or configure the underlying IaaS. Now, single teams of developers can provision infrastructure and deploy applications on demand. This skillset shift resulted in an organizational shift that spawned the terms developer operations (DevOps) and continuous integration / continuous delivery (CI/CD). Now you have whole teams deploying and operating in a continuous model.

Shift From Dedicated Roles To Breadth Of Skills

Ok, but don’t we still need traditional IT skills in security? Yes, yes you do. You need the skills, but not a dedicated role.

Imagine a model where everyone at your company works remotely from home and your business model is cloud native, using PaaS to deploy your custom application. As the CISO of this organization, what roles do you need in your security team?

From a business standpoint, you still need to worry about data and how it flows, you need to worry about how your applications are used and can be abused, but your team will primarily be focused on making sure the code your business uses to deploy resources and applications in the cloud is secure. You also need to make sure your business is following appropriate laws and regulations. However, you will no longer need dedicated people managing firewalls, routers or hardening servers.

What you will need is people with an understanding of technologies like identity, networking, storage and operating systems. These skills will be necessary so your security team can validate resources are being consumed securely. You will also need a lot of people who understand application security and you will need compliance folks to make sure the services you are consuming are following best practices (like SOC 2 and SOC 3 reports).

What Do You Recommend For People Who Want To Get Into Security Or Are Deciding On A Career Path?

I want to wrap up this post by talking about skills I think people need to get into security. Security is a wonderful field because there are so many different specialization areas. Anyone with enough time and motivation can learn about the different areas of security. In fact, the U.S. Government is kind enough to publish a ton of frameworks and documents talking about all aspects of security if you have the time and motivation to read them. That being said, if I was just starting out in security I would advise people to first pick something that interests them.

  • Are you motivated by building things? Learn how to be a security engineer or application security engineer. Learn how to script, write code and be familiar with a variety of technologies.
  • Are you motivated by breaking things? Learn how to be a penetration tester, threat hunter or offensive security engineer.
  • Do you like legal topics, regulations and following the rules? Look into becoming an auditor or compliance specialist.
  • Do you like detective work, investigating problems and periodic excitement? Learn how to be an incident response or security operations analyst.

Ask Questions For Understanding

The above questions and recommendations are just the tip of the iceberg for security. My biggest piece of advice is once you find an area that interests you start asking a lot of questions. Don’t take it for granted that your CSP magically provides you with whatever resources you ask for. Figure out how that works. Don’t blindly accept a new regulation. Dissect it and understand the motivation behind it. Don’t blindly follow an incident response playbook. Understand why the steps exist and make suggestions to improve it. If a new vulnerability is released that impacts your product, understand how and why it is vulnerable. The point is, as a security professional the more understanding you have of why things exist, how they work and what options you have for managing them, the more skills you will add to your resume and the more successful you will be in your career, especially as your security org collapses roles as a result of moving to the cloud.

The Dichotomy Of Security

If you have ever read Extreme Ownership or The Dichotomy of Leadership by Jocko Willink, then you will be familiar with the concept of dichotomy and how opposing forces of a skill set can compliment each other. Mastering both sides can allow flexibility and increase the effectiveness of that skill set when dynamically applied to a given situation. This is true in the security space, where fundamental opposing forces need to be balanced in order to manage risk and achieve success. Let’s take a look at a few examples.

Security Extremes

The easiest example of the dichotomy of security is to look at the extremes. Security professionals jokingly say the most secure company is one that is not connected to the internet. While this may be true, it will also prevent the company from conducting business effectively and so the company will cease to exist and security will no longer be needed.

On the other end of the spectrum there is the extreme of a business that has zero security and so there are no impediments to conducting business. While this may sound great to some, the reality is the company will be unable to effectively conduct business because of the real threats that exist on the internet. In the situation the company will also cease to exist because they will be hacked into oblivion.

It is obvious there is a dichotomy between no security and no connectivity and these forces need to be appropriately balanced for a security program to be effective, while allowing the business to operate.

Manual vs Automated Security

Another example of dichotomy is between manual security tasks and automation. While every CISO I know is striving to increase automation of security tasks, the reality is humans are still going to be needed in any security program for the foreseeable future.

Manual tasks are ideal for situations where humans need to demonstrate creativity, intuition or make complex decisions based on subtle context. Security functions like penetration testing, threat hunting, red teaming and offensive security require high amounts of skill and experience that automation, like AI, hasn’t been able to replicate. Additionally, soft skills such as reporting to the board, shifting culture, building alliances and making prioritization decisions are all extremely complex and unlikely candidates for automation. However, while manual activities benefit activities that require a high degree of creativity, they are inherently slow and can impede the normal flow of business.

Recently, the advances in automation and artificial intelligence have exponentially increased their usefulness. Automation is extremely useful for offloading repeatable tasks that lend themselves to being programmatically defined. For example, attack simulation products have made huge strides in offloading repetitive tasks of reconnaissance, enumeration, vulnerability assessment and remedial exploitation. We are seeing additional advances in automation related to incident response where events can be correlated and specific activities in an IR playbook can be completed to offload analysts and help focus their attention. AI has also helped to offload lower level operational activities like call centers and help desk inquiries.

While automation may accelerate parts of the business and offload humans from repeatable tasks, it does introduce complexity, which can be difficult to troubleshoot or can cause outright failures. Automation is also rigid because it is only as good as the parameters of the process it is following. This means it can’t think outside of the box or demonstrate creativity. There is also the risk of introducing bias into your processes if your underlying model is flawed.

As you can see manual security processes and automated security processes are opposing forces that need to be balanced based on the skill of your security team and the needs of the business.

The Human Problem

The last dichotomy I want to discuss is the human problem in security. Humans are necessary because of their creativity, diversity and capacity for adapting to an infinite number of situations. However, the flexibility in human nature also presents one of the fundamental security problems – how to you protect against human nature?

The reality is humans are flawed, but in a good way. Threat actors can try to take advantage of these flaws, whether they are logical (like firewall rules) or physical (like human psychology). Humans are essential to every aspect of a business and so we have to figure out how to protect them. The most difficult balance in security is developing a program that is comprehensive enough to protect against human nature without stifling it.

The Security Ideal

The ideal security program will recognize the dichotomy of the security challenges it faces and balance them accordingly. The ideal security program balances security with flexibility. We are seeing this balance manifest in mature security programs via concepts like security guard rails and the paved path. The paved path and guard rails attempt to allow a certain amount of latitude for acceptable behavior, while being rigid enough to protect users and the business accordingly.

Application In Other Domains

The concept of dichotomy is universal across any domain. In fact, this is an area of extensive research in disciplines like mathematics, computer science, military strategy, and economics. Specifically, in the space of network and graph theory there is a concept call max flow, min cut. These are counter principles that are opposite, yet complimentary. If you think of any network (road, supply chain, computer network, etc.) the point of maximum flow across that network is also the point where maximum disruption (minimum cut) can occur. From a military or security stand point you will want to protect the max flow/min cut, but from an attacker stand point, the max flow / min cut, is the area that will require the least amount of effort for maximum damage. Pretty neat!

Wrapping Up

An effective security program will balance the needs of security with the needs business with the ultimate goal of effectively managing risk. A critical skill for any security practitioner is to be flexible and adaptive. Specifically, by recognizing that security issues have two sides to them, security practitioners can demonstrate empathy towards the business and find an appropriate balance that can protect without impeding the business.

We Are Drowning In Patches (and what to do about it)

Last week I had an interesting discussion with some friends about how to prioritize patches using criticality and a risk based approach. After the discussion I starting thinking about how nice it would be if we could all just automatically patch everything and not have to worry about prioritization and the never ending backlog of patches, but unfortunately this isn’t a reality for the majority of organizations.

Whats the problem?

There are several issues that create a huge backlog of patches for organizations.

First, let’s talk about the patching landscape organizations need to deal with. This is largely spit into two different areas. The first area is operating system (OS) and service patches. These are patches that are released periodically for the operating systems used by the business to run applications or products. Common operating systems for production workloads will be either Windows or Linux and will have stability, security or new feature patches released periodically.

Second, there are patches for software libraries that are included in the software and applications developed by your business. Typically these are lumped into the category of 3rd party libraries, which means your organization didn’t write these libraries, but they are included in your software. 3rd party library security vulnerabilities have become a huge issue over the last decade (but thats a blog post for another day).

These two patch types, OS and 3rd party library patches, require different approaches to discover, manage and remediate, which is the first challenge for auto patching. When combined with the volume of new vulnerabilities being discovered, large heterogeneous environments and the need to keep business critical applications available, keeping your assets patched and up to date becomes a real challenge.

Why isn’t auto patching a thing?

Well it is, but…

There are a few challenges to overcome before you can auto-patch.

Stability and Functionality

First, both operating system and 3rd party library patches need to be tested for stability and functionality. Usually, patches fix some sort of issue or introduce new features, but this can cause issues in other areas such as stability or functionality. It can be a complex process to roll back patches and restore business critical applications to a stable version, which is why most businesses test their patches in a staging environment before rolling them out to production. Cash is king and businesses want to minimize any disruption to cash flow.

Investment and Maturity

It is possible to automate testing for stability and functionality, but this requires a level of maturity and investment that most organizations haven’t achieved. For example, assuming your staging environment is a mirror image of your production environment (it is right?), you could auto apply the patches in staging, automatically check for stability and functionality over a set period of time and then roll those updates to production with minimal interaction. However, if your environment requires reboots or you have limited resources, patching may require down time, which could impact making money.

In order to have an environment that can support multiple versions, seamless cut over, proper load balancing, caching, etc. requires significant investment. Typically this investment is useful for keeping your products functioning and making money even if something goes wrong, but this investment can also be used to buffer maintenance activities such as patching without disruption.

Software Development Lifecycle

The last section assumes a level of software development maturity such as adoption of Agile development processes and CI/CD (continuous integration / continuous delivery). However, if your engineering group uses a different development process such as Incremental or Waterfall, then patching may become even more difficult because you are now competing with additional constraints and priorities.

What are some strategies to prioritize patching and reduce volume?

If your business runs products that aren’t mission critical, or you simply can’t justify the investment to operate an environment without down time, then auto patching probably isn’t a reality for you unless you are Austin Powers and like to live dangerously. For most organizations, you will need to come up with a strategy to prioritize patching and reduce the volume down to a manageable level.

Interestingly, this problem space has had a bunch of brain power dedicated to it over the years because it resembles a knapsack problem, which is a common problem space in mathematics, computer science and economics. Knapsack problems are problems where you have a finite amount of a resource (space, time, etc.) and you want to optimize the use of that resource to maximize some sort of requirement (like value). In the case of patching, this would mean applying the largest volume of the highest severity patches in a fixed time period to realize the maximum risk reduction possible.

Critical Assets First

Staying in the knapsack problem space, one strategy is to start with your most critical assets and apply the highest severity patches until you reach your threshold for risk tolerance. This requires your organization to have an up to date asset inventory and have categorized your assets based on business criticality and risk. For example, let’s say you have two applications at your business. One is a mission critical application for customers and generates 80% of your annual revenue. The other application provides non-mission critical functionality and accounts for the other 20% of revenue. Your risk tolerance based on your company policies is to apply all critical and high patches within 72 hours of release. In this example you would apply all critical and high patches to the mission critical application as quickly as possible (assuming other requirements are met like availability, etc.).

Guard Rails and Gates

Another strategy for reducing volume is to have guard rails or gates as part of your software development lifecycle. This means your engineering teams will be required to pass through these gates at different stages before being allowed to go to production. For example, your organization may have a policy that no critical vulnerabilities are allowed in production applications. The security organization creates a gate that scans for OS and 3rd party library vulnerabilities whenever an engineering team attempts to make changes to the production environment (like pushing new features). This way the engineering team needs to satisfy any vulnerability findings and apply patches at regular intervals coinciding with changes to production.

Wrapping Up

With the proliferation of open source software, the speed of development and the maturity of researchers and attackers to find new vulnerabilities, patching has become an overwhelming problem for a lot of organizations. In fact, it is such a big problem CISA and the Executive Order On Improving The Nation’s Cybersecurity list software patches and vulnerabilities as a key national security issue. I’ve outlined a few strategies to prioritize and reduce the volume of patches if your organization can’t afford the investment to absorb downtime without disruption. However, no matter what strategy you choose, all of them require strong fundamentals in asset inventory, asset categorization and defined risk tolerance. While these investments may seem tedious at first, the more disciplined you are about enforcing the security fundamentals (and engineering maturity), the less you will drown in patches and the closer your organization will come to the reality of auto-patching.

Are Phishing Campaigns Worth It?

Phishing campaigns are often touted as a complementary exercise to security training as a way to measure training effectiveness. The thought is, if your training is effective, users will be less likely to fall for and click on phishing emails, which will correlate to a decrease in the number of phishing incidents at your company. This sounds great in theory, but phishing campaigns have a lot of downsides that need to be considered before you hit the send button.

What Is Phishing?

The Cybersecurity and Infrastructure Security Agency (CISA) defines phishing as:

“a form of social engineering where malicious actors lure victims (typically via email) to visit a malicious site or deceive them into providing login credentials.”

What does this mean for a CISO in practical terms? It means your employees will constantly receive emails that look legitimate, but are actually scams. They are trying to get your employees to click on links in the email so they can steal credentials, install malware, get access to sensitive data, or steal money. Phishing campaigns are often one of the first methods attempted in a more targeted attack that can use the phished credentials to allow the attacker to gain a foothold into your environment.

What Are The Common Defenses Against Phishing?

User Awareness Training

One of the most effective ways to counter the threat of phishing attacks is to educate your users. Regular user awareness training on how to recognize and take action against phishing emails has proven to be highly effective. Why? Phishing is trying to trick your users into performing an action they wouldn’t normally perform. This is a form of phycological or social engineering and the best way to instill the proper mindset in someone is through regular training. This training should test for understanding and the ability for users to recognize and report phishing emails. When in doubt, report it and delete it.

DMARC, SPF and DKIM

Domain-based Message Authentication, Reporting and Conformance (DMARC), Sender Policy Framework (SPF) and Domain Keys Identified Mail (DKIM) are technologies that can be implemented by businesses to verify the sender of incoming email and authenticate that incoming messages are valid (not spoofed). Technologies such as SPF allow domains to publish lists of IPs and servers that will send emails, and DKIM allows domain owners to digitally sign emails coming from their domain so recipients can cryptographically validate the messages. SPF and DKIM are forms of authentication in the email world and help prevent spammers from sending mail on domains they don’t own.

DMARC is the enforcement arm in the email world. It takes the output from SPF and DKIM and takes action. This action can be configured based on your organizational preferences, but typical actions for messages that fail SPF or DKIM checks are to deliver the message, mark as SPAM or reject the message entirely. When configured properly, all three of these technologies will help filter and reduce potential phishing emails that make their way into your user’s inboxes.

MFA

Another technology that is critically important to protect against phishing attacks is to enable Multi-Factor Authentication (MFA). This is another form of defense that will protect your user accounts if a phishing email makes it through the filters and your user clicks on the phishing link in the email.

For example, a typical phishing email may impersonate a legitimate business website that requires authentication. The formatting, graphics and appearance may all look exactly the same. The only way to tell the email is a phishing email is by looking at the sender domain or email headers to detect subtle variances in spelling or formatting. If a user falls for this phishing email, clicks on it and enters their username and password, MFA will help prevent their credentials from being fully compromised. Yes, the user will need to have their password changed, but MFA such as one time passwords, tokens or passkeys, will prevent the attackers from using the phished credentials.

What Are Phishing Campaigns?

Phishing campaigns are controlled email campaigns sent by your own organization or a contracted third party to send fake phishing emails to your users to test how many open and click on the phishing links. Phishing campaigns allow organizations to directly test how well their user awareness training is working to recognize and avoid phishing attacks. Phishing campaigns can be stand alone events or they can be tied into other security testing like penetration tests.

What Are the Downsides To Phishing Campaigns?

Phishing campaigns, while popular, have questionable morality and effectiveness for a few reasons.

  1. The primary method of business communication is email. Phishing campaigns are teaching users to mistrust and in some cases stop using email for business purposes. Security organizations should find ways to support and protect the business without unnecessarily impeding it and for this reason I believe phishing campaigns are counter to the mission of an effective security organization.
  2. The top businesses have cultures that support and encourage psychological safety. Being able to respectfully speak your mind, have support from your colleagues and feel valued are all important aspects for job satisfaction and effectiveness. Phishing campaigns go against the idea of psychological safety. They attempt to trick your users into clicking on emails with questionable tactics such as promising bonuses, legitimate business purposes or even funny cat videos.
  3. One large problem with phishing campaigns is they tend to have punitive outcomes. Anyone that falls for the phishing email gets sent to remedial training or may be given a reduced set of permissions for a period of time. These punitive actions punish users for using their primary method of communication, destroy the concept of psychological safety and discourage productivity.
  4. Speaking of productivity, I see a lot of metrics about the percentage of users that clicked on phishing campaign emails along with targets to reduce those numbers after sending people to remedial training. What I don’t see are metrics on the impact the campaign has to productivity. How much longer will it take the person on finance to do their job now that she doesn’t trust anything in her email? How much longer will it take IT support to resolve the help desk ticket when they have been scolded repeatedly for falling for phishing emails? These metrics unfortunately are overlooked or not even captured. Security programs should ground their activities against the overall business strategy and make sure their programs are generating true value for the business that is measurable in the form of reduced risk as a tradeoff to other areas of the business.

A More Effective Solution

Whenever someone asks me for my thoughts about phishing campaigns I tell them honestly that I am not a fan. I’ve been on penetration testing teams that have crafted emails as part of phishing campaigns and I’ve seen the effect it has on users. I think there is a better way.

A lot of this post has gathered inspiration from various sources, but one of the main sources is the Cybersecurity and Infrastructure Agency (CISA). In October 2023, CISA the FBI and the NSA published a joint article on guidance for stopping phishing attacks. You can read their excellent recommendations here. Their article supports my sentiments here because one thing that is not in their recommendations is conducting a phishing campaign against your own users.

What are my recommendations?

  1. Conduct proactive training that tests not only comprehension, but the ability to accurately recognize phishing emails. Give employees the skills to look at email headers and give them the latitude to report suspicious emails or delete them altogether. Accept that email will be a slower and less trusted form of communication and even prevent the use of email for critical business functions (like contracting or financial activities). This training should have hands on practicals that gives the security function and senior leaders confidence they have trained their users to the best of their abilities to minimize the risk.
  2. Put controls in place to protect your employees. DMARC, DKIM, SPF and MFA can protect your users. Endpoint protection, monitoring and logging, ingress and egress filtering, etc. can all provide defense in depth to stop phishing attacks from being successful. The point here is, a comprehensive and well executed security program is one of your best defenses against phishing attacks.
  3. Employees that fall for real world phishing emails should be given a second chance. Assume good intent here. Most employees will recognize when they have done something bad and will feel guilty about it. They will punish themselves so the organization should support them, offer them additional training and help them get back to doing their job. Having proper security controls in place can help minimize the impact of your employees clicking on phishing emails.
  4. As a last resort, I will recommend some sort of punitive action, but this should not be the default and should be used sparingly. For users that just don’t get it and are repeat offenders they should face disciplinary action such as termination or reduced job responsibilities. This ties into organizations that evaluate how well employees support and uphold the security objectives of the organization. Repeated violations of Acceptable Use Policies (AUP) should fall under an HR/Legal action that minimizes the risk of the employee to the business.

Wrapping Up

Phishing is a form of social engineering and is a real business risk. It can lead to credential compromise, malware infections or Business Email Compromise (BEC) resulting in real business loss. A well rounded and comprehensive security program will help counter the threat of phishing attacks through comprehensive security controls and processes. Most importantly, I recommend security teams remove phishing campaigns from their tool chest and instead use proactive techniques to educate users, while protecting the business with a defense in depth approach.