Tag: #cso

Navigating The First 90-180 Days In A New CISO Role

Late one Friday afternoon a call comes in and you find out you landed your next CISO role. All the interview prep, research, networking and public speaking has paid off! Then it dawns on you that you could be walking into a very difficult situation over the next few months. Even though the interview answered a lot of questions, you won’t know the reality of the situation until you start. How will your expectations differ from reality? What can you do to minimize risk as you come up to speed? How should you navigate these first 90-180 days in your new role?

Prior To Starting

Let’s assume you have some time to wind down your current position and you are also going to take some time off before starting the new role. During this transition period I highly advise you reach out to your peers in the new role and start asking questions to get more detail about the top challenges and risks you need to address. Start with the rest of the C-Suite, but also get time with board members and other senior business leaders to get their perspectives. Focus on building rapport, but also gather information to build on what you learned during the interview process so you can hit the ground running.

You can also use this time to reach out to your CISO peers in your network who are in the same industry, vertical or company type to get their perspective on what they did when they first joined their company. Learn from their experience and try to accelerate your journey once you start. Keep the lines of communication open so if you run into a situation you are unsure of you can ask for advice.

Once You Start

Build Relationships

First and foremost, start building relationships as quickly as possible. Target senior leadership first, such as board members, the C-Suite and other senior leaders. Work your way down by identifying key influencers and decision makers throughout the org. Play the “new person card” and ask questions about anything and everything. Gain an understanding of the “operational tempo” of the business such as when key meetings take place (like board meetings). Understand the historical reasons why certain challenges exist. Understand the political reasons why challenges persist. Understand the OKRs, KPIs and other business objectives carried by your peers. Learn the near and long term strategy for the business. Start building out a picture of what the true situation is and how you want to begin prioritizing.

Understand the historical reasons why certain challenges exist. Understand the political reasons why challenges persist.

Plan For The Worst

Don’t be surprised if you take a new role and are immediately thrown into an incident or other significant situation. You may not have had time to review playbooks or processes, but you can still fall back on your prior experience to guide the team through this event and learn from it. Most importantly, you can use this experience to identify key talent and let them lead, while you observe and take notes. You can also use your observation of the incident to take notes on things that need to be improved such as interaction with non-security groups, when to inform the board, how to communicate with customers or how to improve coordination among your team.

Act With Urgency

Your first few months in the role are extremely vulnerable periods for both you and the company. During this period you won’t have a full picture of the risks to the business and you may not have fully developed your long term plan. Despite these challenges, you still need to act with urgency to gain an understanding of the business and the risk landscape as quickly as possible. Build on the existing program (if any) to document your assumptions, discoveries, controls and risks so you can begin to litigation proof your org. Map the maturity of security controls to an industry framework to help inform your view of the current state of risk at the company. Begin building out templates for communicating your findings, asks, etc. to both the board and your peers. Most importantly, the company will benefit from your fresh perspective so be candid about your findings and initial recommendations.

Evaluate The Security Org

In addition to the recommendations above, one of the first things I like to do is evaluate the org I have inherited. I try to talk to everyone and answer a few questions:

  1. Is the current org structure best positioned to support the rest of the business?
  2. How does the rest of the business perceive the security org?
  3. Where do we have talent gaps in the org?
  4. What improvements do we need to make to culture, diversity, processes, etc. to optimize the existing talent of the org?

Answering these questions may require you to work with your HR business partner to build out new role definitions and career paths for your org. You may also need to start a diversity campaign or a culture improvement campaign within the security org. Most importantly, evaluate the people in your org to see if you have the right people in the right places with the right skillsets.

A Plan Takes Shape

As you glide past the 90 day mark and start establishing your position as a trusted business partner, you should arrive at a point where a clear vision and strategy is starting to take shape. Use the information you have gathered from your peers, your program documentation and your observations to start building a comprehensive plan and strategy. I’ve documented this process in detail here. In addition to building your program plan you can also begin to more accurately communicate the state of your security program to senior leaders and the board. Show how much the existing program addresses business risk and where additional investment is needed. I’ve documented a suggested process here. Somewhere between your 90 and 180 day mark you should have a formalized plan for where you are over invested, under invested or need to make changes to optimize existing investment. This could include restructuring your org, buying a new technology, adjusting contractual terms or purchasing short term cyber insurance. It could even include outsourcing key functions of the security org for the short term, until you can get the rest of your program up to a certain standard. Most importantly, document how you arrived at key decisions and priorities.

Take Care Of Yourself

Lastly, on a personal note, make sure to take care of yourself. Starting a new role is hectic and exciting, but it is also a time where you can quickly overwork yourself. Remember building and leading a successful security program is a marathon not a sprint. The work is never done. Get your program to a comfortable position as quickly as possible by addressing key gaps so you can avoid burning yourself out. Try to establish a routine to allow for physical and mental health and communicate your goals to your business partners so they can support you.

During this time (or the first year) you may also want to minimize external commitments like dinners, conferences and speaking engagements. When you start a new role everyone will want your time and attention, but be cautious and protective of your time. While it is nice to get a free meal, these dinners can often take up a lot of time for little value on your end (you are the product after all). Most companies have an active marketing department that will ask you to engage with customers and the industry. Build a good relationship with your marketing peers to interweave customer commitments with industry events so you are appropriately balancing your time and attending the events that will be most impactful for the company, your network and your career.

Wrapping Up

Landing your next CISO role is exciting and definitely worth celebrating. However, the first 90-180 days are critical to gain an understanding of the business, key stakeholders and how you want to start prioritizing activities. Most importantly, build relationships, act with urgency and document everything so you can minimize the window of exposure as you are coming up to speed in your new role.

Following SnowFlake, Cloud Providers Need To Shift To Secure By Default

In May 2024, SnowFlake experienced a data breach as a result of exposed credentials that allowed a threat actor to access customer accounts that weren’t secured with MFA. The fallout from this data breach ultimately impacted large SnowFlake customers like Ticketmaster, AutoZone, Santander Bank and AT&T. Following the announcement of the breach, SnowFlake implemented refined security measures to avoid similar incidents in the future. However, the question remains why aren’t publicly accessible cloud companies secure by default?

A Pervasive Stigma Against Security

Before we can answer the question about why companies aren’t secure by default, we need to look at the underlying psychology and motivation for companies and in particular the arguments that are made against implementing security.

Startup Mentality

One of the most pervasive (and quite frankly horrible) arguments against building in security by default is the “move fast and break things” mentality that is pervasive at startups. Startup life is a tough one and a good metaphor is you are building your parachute as you are falling. Either you succeed and live, or you burn in and cease to exist. The problem with startup mentality is when you succeed and live, most startups fail to to shift from survival mode to maturity mode as the company grows and matures.

In maturity mode, companies need to resolve all of the debt they incurred just to survive. This can be operational debt, technical debt or security debt. Unfortunately, if the survival mentality persists, this debt continues to accrue and can kill the company because the cost to continue to operate exceeds the incoming revenue.

Security Is Bad For Productivity

Another argument that frequently pops up against implementing security is the perception that security is bad for productivity. I find this argument particularly ironic since employees seem willing to tolerate bad processes, bad experiences and other examples of bad friction, yet they complain the loudest about new security controls (like being required to change their password periodically). My own opinion about this perception is employees are largely indifferent to security (or in general they think it is a good thing). However, security often results in very visible changes to processes and ways of working and it is the change that employees don’t like. They associate security with change and since change is bad, security is bad.

This is similar to the argument that security increases friction and the assumption that all friction is bad. While this assumption is not only false, it also leads to the thought process that any friction in the customer experience will lead to lost customers and sales. The reality is some friction is good and acts as safeguards to steer people towards a desired (secure) outcome.

Security As An Upsell

One last reason for failing to implement security by default is when companies choose to profit from security as an upsell (I’m looking at you Microsoft). By charging extra for the most useful or best features these companies are implicitly and explicitly placing a cost on adding in security, which is perpetuating the stigma that security is bad.

The reality is some friction is good and acts as safeguards to steer people towards the desired (secure) outcome.

Changing Perception

Leading research for high performing cultures indicates teams that are able to effectively prioritize and execute on all of their demands are the highest performing teams. In particular, teams that were able to incorporate security into their processes actually went faster and performed better, than teams who struggled with or ignored security altogether. If you want to read more on this you can check out Accelerate by Nicole Forsgren, PhD.

One other thing we can do to change this negative perception of security is to stop allowing members of the security function introduce bad friction. We have all experienced bad friction in the form of time wasters, security theater and the dreaded “no”. This behavior doesn’t help the mission of security and perpetuates the stigma against our profession.

Default Opted-In

Assuming companies can overcome the startup mentality, successfully incorporate security into their development processes and overcome the stigma of security as being bad, what should they be doing to make their products and services secure by default?

The first thing companies can do is discard the notion that increased security will inhibit sales or drive customers away. Instead, companies should use security as a selling point and configure their services to be secure by default, which means customers will need to go through some sort of initial security setup when they purchase the product or service. Customers that don’t want to do this will need to explicitly opt-out or seek alternate providers, firmly placing the liability for not meeting security best practices on their shoulders.

Enforce Security Best Practices

What security functionality should companies offer by default to their customers? Here is a short list:

Multi-Factor Authentication – including the option for OTP, secure tokens and passkeys.

Encryption – all data and transport protocols should be encrypted by default with the latest versions available.

Access Control and Detection – default deny for access to resources and make customers explicitly allow access. This includes making resources non-public by default until a customer specifies otherwise. Detect changes in the state of resources and notify customer contacts of abnormalities.

Easy Button For Fundamentals – make it easy for customers to pull a comprehensive asset inventory, control their instance or tenancy with a master account and offer simple reports for ways they can improve their security posture.

Wrapping Up

There a lots of reasons why security becomes an afterthought for companies. Often, it is because they fail to shift from survival mode to maturity mode. Other times, their culture persists the notion that security has a bad stigma and inhibits the business. Some companies even upsell customers on security functionality, which limits the adoption of security controls. The reality is companies that practice secure by design and incorporate security into development cultures move faster and outpace their competition. Companies that offer publicly available software and services need to shift their mentality to make security a default setting that is turned on at the onset of the relationship, like any other core product feature. Until companies start making security default opt-in, we will continue to experience massive data breaches like the one from SnowFlake.

Navigating The CISO Job Market

I had an interesting conversation with a friend over coffee last week and we were discussing how weird the CISO job market is right now. Even though the unemployment rates are favorable, the tech sector has actually seen slightly negative employment growth rates, which is not normal. This is largely due to a hangover effect from record hiring during COVID, but there are also other issues in the market right now that is making it challenging. The following is a review of all the things I am seeing in the tech job market right now, particularly with respect to hiring for CISO positions.

Macro Tech Environment

Let’s take a step back and look at the overall economy to understand some of the higher level factors influencing the CISO job market. First, let’s look at one end of the tech market starting with large companies. Over hiring and high compensation packages from COVID have made existing employees stay in place and so natural turnover at public companies is below average. In addition to this, fears of a recession and high interest rates have made large companies cautious about hiring new employees. When the cost to borrow money is higher, it slows growth and ultimately impacts hiring. As a result, companies are trying to get back to growth through layoffs and attrition. They are trying to artificially increase attrition by withholding bonuses, pay raises and promotions, or requiring new job requirements like return to office 4 or 5 days a week.

Second, at the other end of the market, higher interest rates impact Venture Capital (VC) and Private Equity (PE), which ultimately impacts funding for startups and subsequent job creation. With the smaller end of the market being squeezed (VC / PE) and the larger end of the market also being squeezed there aren’t a lot of options for candidates to go. Compound this with record tech layoffs over the past year and an influx of new college grads to the job market and you create a highly competitive market.

Too Much Noise

The highly competitive job market is making job candidates seeking employment and existing CISOs seeking career growth (or a change) compete with each other. The competition is causing candidates to get desperate and apply to any job that sounds sounds remotely interesting, regardless of whether or not they are qualified for the role. This is also compounded by unrealistic career expectations from past promotions, boot camps and college campuses that make people think they can qualify for the top spots, despite lacking meaningful experience. Add in how easy LinkedIn and other jobs sites have made it to apply for jobs and the net effect is to create tons of noise for recruiters and drown out qualified candidates.

I spoke to a recruiter a few weeks ago who had a job posting up for 24 hours and received thousands of applicants, of which only a handful were qualified and advanced to the interview process. Due to the volume of unqualified applicants, recruiters are only pushing through the first handful of qualified candidates and are passing on the rest of the backlog. Of all these applicants the only candidates who are getting to the first round interview phase are direct referrals.

In addition to too much applicant noise, recruiters are also finding a high number of candidates that are mis-representing themselves. Recruiters and hiring managers aren’t stupid. They can read between the lines of your career history and discern what you were really doing. If you claim to be a CISO, yet have never held more than a manager level job, then you are mis-representing yourself. The reality is, recruiters want to get paid on placing the top candidates. They are unwilling to put someone forward for a top spot that can’t back up their resume. Top candidates can not only defend their experience, but have lots of direct and indirect network connections that can vouch for them as referrals, if needed. The CISO community is a small one and people know who is the real deal and who is faking it. The sad reality is, people who mis-represent themselves are only hurting themselves by artificially placing themselves in a higher, more competitive tier than they are qualified for and as a result will never land that top spot.

Companies Are Being More Strict

High interest rates, tight budgets and a noisy applicant process mean companies are being more strict with their job requirements. More top CISO positions are requiring candidates to be on site at the corporate headquarters location at least 4 days a week. Companies are also searching globally, but hiring locally by giving preference to local candidates they don’t have to relocate and also preference to internal candidates that cost less than a retained search. CISO salaries have also slowed or stagnated with only the top spots paying top salaries. The rest are paying mid-range or low balling candidates in an attempt to get a qualified applicant at a lower price. On top of this, companies are also being more strict with degree requirements (usually a Masters for CISOs), years of experience and certifications. They are also filtering out candidates with lots of job hopping and short career stints because even though you may have carried the CISO title, it is highly unlikely you accomplished anything meaningful if you were there for less than 18 months.

The only candidates who are getting to the first round interview phase are direct referrals.

Be Cautions

Lastly, there are a few other issues that are disrupting the job market. The first is fake job postings. There are more and more reports of fake job postings that entice applicants, but are really out to steal their personal information. Be cautious and use your network to validate the postings if you are interested in applying for a CISO role (this comes back to direct referrals also).

Second, companies are leaving zombie positions out there to give the impression they have open roles, when they really don’t. They are doing this for a few reasons – they want the market and their employees to think they are hiring and growing even when budgets are tight and companies are trying to cut headcount. If you see a job posting out there for more than a few days, it is highly likely it is a zombie posting.

The last issue I want to highlight is how job sites mis-represent numbers to entice companies to spend money with them, while hurting applicants. I’m specifically referring to how LinkedIn and other job sites show metrics on “number of applicants” for job postings, when in reality these are only the number of people that have viewed the posting, not applied. I mention this because I have seen a number of posts from people who have expressed interest in a role, but have been discouraged by the “number of applicants” and as a result didn’t apply.

Maximizing Your Opportunity

Now that you understand what is going on with the job market, let’s discuss what you can do to maximize the likelihood you will land that interview and get the job.

  1. Invest in yourself – take this time to get certifications, degrees, etc. that make you competitive and demonstrate constant learning and knowledge. Invest in yourself while looking for a new role.
  2. Invest in your network – do a deep dive on your network. LinkedIn makes it easy to download your list of connections and sort them my company, degree of connection, etc. Use this analysis to understand where you have connections and where you don’t. Look for people that can connect you to individuals that hire for positions you want at your targeted companies. Find ways to meet with these people. Do the same for recruiters. Build these connections before you need them because it is always better to be a live person than a random InMail on LinkedIn.
  3. Update your resume and LinkedIn – Seriously, if you don’t know how then ask someone or pay someone. First impressions matter.
  4. Practice interview questions – Write down key accomplishments and the details for how you achieved them. Think of your weaknesses and how you turn those into strengths. Ask your network for recent interview questions and develop answers. Preparation matters and will pay off during the interview process.
  5. Stop blasting your resume into the ether – If you see a role you want to apply for, poll your network to see if you know anyone at the company or if your network knows someone at the company. Get your resume directly into the hands of the recruiter or hiring manager. Direct referrals are the only reliable way to get an interview.
  6. Get focused – Have you been attending a lot of networking events lately in the hope of meeting someone who is hiring? Consider the value of all the “networking” activities you are doing. As a single person you can’t scale to attend every event that is out there so you need to be targeted. Consider the audience of who is attending and consider the value of the event. If you are attending events that are also attended by all of your competition then you probably aren’t going to land your next job there. Instead, consider all the events and networking groups in your area, which one’s have the most likelihood of putting you in front of people that hire for your role and focus on maximizing the potential of those events.
  7. Stop directly asking people for jobs – there is no faster way to end a conversation or relationship than asking someone for a job they don’t have. Instead, if you have the opportunity to make an ask of someone, ask them to connect you with someone they know may be looking for someone with your background. Take the pressure off of them, keep the connection alive and expand your network at the same time.
  8. Consider staying put – the tech sector seems to lag what the overall economy is doing by a few years. If the tech sector is contracting it will eventually expand and get back positive employment rates. This can also give you time to build your credentials, while looking for the ideal next step.

Should Companies Be Held Liable For Software Flaws?

Following the CrowdStrike event two weeks ago, there has been an interesting exchange between Delta Airlines and CrowdStrike. In particular, Delta has threatened to sue CrowdStrike to pursue compensation for the estimated $500M of losses allegedly incurred during the outage. CrowdStrike has recently hit back at Delta claiming the airline’s recovery efforts took far longer than their peers and other companies impacted by the outage. This entire exchange prompts some interesting questions about whether a technology company should be held liable for flaws in their software and where the liability should start and end.

Strategic Technology Trends

Software quality, including defects that lead to vulnerabilities, has been identified as a strategic imperative according to CISA and the Whitehouse in the 2023 National Cybersecurity Strategy. Specifically, the United States wants to “shift liability for software products and services to promote secure development practices” and it would seem the CrowdStrike event falls into this category of liability and secure software development practices.

In addition to strategic directives, I am also seeing companies prioritize speed to market over quality (and even security). In some respects it makes sense to prioritize speed, particularly when pushing updates for new detections. However, there is clearly a conflict in priorities when a company optimizes for speed over quality for a critical detection update that causes an impact larger than if the detection update had not been pushed at all. Modern cloud infrastructure and software development practices prioritize speed to market over all else. Hyperscale cloud providers have made a giant easy button that allows developers to consume storage, network and compute resources without consideration for the down stream consequences. Attempts by the rest of the business to introduce friction, gates or restrictions on these development processes are met with derision and usually follow accusations of slowing down the business or impeding sales. Security often falls in this category of “bad friction” because they are seen as the “department of no”, but as the CrowdStrike event clearly shows, there needs to be a balance between speed and quality in order to effectively manage risk to the business.

One last trend is the reliance on “the cloud” as the only BCP / DR plan. While cloud companies certainly market themselves as globally available services, they are not without their own issues. Cloud environments still need to follow IT operations best practices by completing a business impact analysis and implementing a BCP / DR plan. At the very least, cloud environments should have a rollback option in order to revert to the last known good state.

…as the CrowdStrike event clearly shows, there needs to be a balance between speed and quality in order to effectively manage risk to the business.

What Can Companies Do Differently?

Companies that push software updates, new services or new products to their customers need to adopt best practices for quality control and quality assurance. This means rigorously testing your products before they hit production to make sure they are as free of defects as possible. CrowdStrike clearly failed to properly test their update due to a claimed flaw in their testing platform. While it is nice to know why the defect made it into production, CrowdStrike still has a responsibility to make sure their products are free from defects and should have had additional testing and observability in place.

Second, for critical updates (like detections), there is an imperative by companies to push the update globally as quickly as possible. Instead, companies like CrowdStrike should prioritize customers in terms of industry risk. They should then create a phased rollout plan that stages their updates with a ramping schedule. By starting small, monitoring changes and then ramping up the rollout, CrowdStrike could have minimized the impact to a handful of customers and avoided a global event.

Lastly, companies need to implement better monitoring and BCP / DR for their business. In the case of CrowdStrike, they should have had monitoring in place that immediately detected their products going offline and they should have had the ability to roll back or revert to the last known good state. Going a step further they could even change the behavior of their software where instead of causing a kernel panic that crashes the system, the OS recovers gracefully and automatically rolls back to the last known good state. However, the reality is sophisticated logic like this costs money to develop and it is difficult for development teams to justify this investment unless the company has felt a financial penalty for their failures.

Cloud environments still need to follow IT operations best practices by completing a business impact analysis and implementing a BCP / DR plan.

Contracts & Liability

Speaking of financial penalties, the big question is whether or not CrowdStrike can be held liable for the global outage. My guess is this will depend on what it says in their contracts. Most contracts have a clause that limits liability for both sides and so CrowdStrike could certainly face damages within those limits (probably only a few million at most). It is more likely CrowdStrike will face losses for new customers and existing customers that are up for contract renewal. Some customers will terminate their contracts. Others will negotiate better terms or expect larger discounts on renewal to make up for the outage. At most this will hit CrowdStrike for the next 3 to 5 years (depending on contract length) and then the pricing and terms will bounce back. It will be difficult for customers to exit CrowdStrike en masse because it is already a sunk cost and companies wont want to spend the time or energy to deploy a new technology. Some of the largest customers may have the best terms and ability to extract concessions from CrowdStrike, but overall I don’t think this will impact them for very long and I don’t think they will be held legally liable in any material sense.

Delta Lags Industry Standard

If CrowdStrike isn’t going to be held legally liable, what happens to Delta and their claimed lost $500M? Let’s look at some facts. First, as CrowdStrike has rightfully pointed out, Delta lagged the world for recovering from this event. They took about 20 times longer to get back to normal operations than other airlines and large companies. This points to clear underinvestment in identifying critical points of failure (their crew scheduling application) and developing sufficient plans to backup and recover if critical parts of their operation failed.

Second, Delta clearly hasn’t designed their operations for ease of management or resiliency. They have also failed to perform an adequate Business Impact Analysis (BIA) or properly test their BCP / DR plans. I don’t know any specifics about their underlying IT operations, but a few recommendations come to mind such as implementing active / active instances for critical services and moving to thin clients or PXE boot for airport kiosks and terminals. Remove the need for a human to touch any of these systems physically, and instead implement processes to remotely identify, manage and recover these systems from a variety of different failure scenarios. Clearly Delta has a big gap in their IT Operations processes and their customers suffered as a result.

Wrapping Up

What the CrowdStrike event highlights is the need for companies to prioritize quality, resiliency and stability over speed to market. The National Cybersecurity Strategy has identified software defects as a strategic imperative because they lead to vulnerabilities, supply chain compromise and global outages. Companies with the size and reach of CrowdStrike can no longer afford to prioritize speed over all else and instead need to shift to a more mature and higher quality SDLC. In addition, companies that use popular software need to consider diversifying their supply chain, implementing IT operations best practices (like SRE) and implementing a mature BCP and DR plan on par with industry standards.

What the CrowdStrike event highlights is the need for companies to prioritize quality, resiliency and stability over speed to market.

When it comes to holding companies liable for global outages, like the one two weeks ago, I think it will be difficult for this to play out in the courts without resorting to a legal tit-for-tat that no one wins. Instead, the market and customers need to weigh in and hold these companies accountable through share prices, contractual negotiation or even switching to a competitor. Given the complexity of modern software, I don’t think companies should be held liable for software flaws because it is impossible to eliminate all flaws. Additionally, modern SDLCs and CI/CD pipelines are exceptionally complex and this complexity can often result in failure. This is why BCP/DR and SRE is so important, so you can recover quickly if needed. Yes, CrowdStrike could have done better, but clearly Delta wasn’t even meeting industry standards. Instead of questioning whether companies should be held liable for software flaws, a better question is: At what point does a company become so essential that they by default become critical infrastructure?

A CISO’s Analysis Of the CrowdStrike Global Outage

Overnight from July 18 to July 19, 2024, Windows systems running CrowdStrike ceased functioning and displayed the blue screen of death (BSOD). As people woke up on the morning of July 19th they discovered a wide reaching global outage of the consumer services they rely on for their daily lives, such as healthcare, travel, fast food and even emergency services. The ramifications of this event will continue to be felt for at least the next week as businesses recover from the outage and investors react to the realization that global businesses are extremely fragile when it comes to technology and business operations.

Technical Details

An update by CrowdStrike (CS) to the C-00000291*.sys file dated 0409UTC was pushed to all customers running CS Falcon agents. This file was corrupt (reports indicate a null byte header issue) and when Windows attempted to load this file it crashed. Rebooting the impacted systems does not resolve the issue because of the way CS Falcon works. CS Falcon has access to the inner workings of the operating system (kernel) such as memory access, drivers, and registry entries that allow CS to detect malicious software and activity. The CS Falcon agent is designed to receive updates automatically in order to keep the agent up to date with the latest detections. In this case, the update file was not properly tested and somehow made it through Quality Assurance and Quality Control, before being pushed globally to all CS customers. Additionally, CrowdStrike customers are clearly running CS Falcon on production systems and do not have processes in place to stage updates to CS Falcon in order to minimize the impact of failed updates (more on this below).

Global Impact

This truly is a global outage and the list of industries is far reaching attesting to the success of CS, but also the risks that can impact your software supply chain. As of Monday, Delta airlines is still experiencing flight cancellations and delays as a result of impacts to their pilot scheduling system. The list of impacted companies can be found here, here and here, but I’ll provide a short list as follows:

Travel – United, Delta, American, major airports

Banking and Trading – VISA, stock exchanges

Emergency & Security Services – Some 911 services and ADT

Cloud Providers – AWS, Azure

Consumer – Starbucks, McDonalds, FedEx

Once the immediate global impact subsides, there will be plenty of finger pointing at CrowdStrike for failing to properly test an update, but what this event clearly shows is a lack of investment by some major global companies in site reliability engineering (SRE), business continuity planning (BCP), disaster recovery (DR), business impact analysis (BIA) and proper change control. If companies were truly investing in SRE, BCP, DR and BIA beyond a simple checkbox exercise, this failed update would have been a non-event. Businesses would have simply executed their BCP / DR plan and failed over, or immediately recovered their critical services to get back up and running (which some did). Or, if they are running proper change control along immutable infrastructure they could have immediately rolled back to the last good version with minimal impact. Clearly, more work needs to be done by all of these companies to improve their plans, processes and execution when a disruptive event occurs.

Are global companies really allowing live updates to mission critical software in production without going through proper testing? Or even better, production systems should be immutable, preventing any change to production without being updated in the CI/CD pipeline and then re-deployed. Failed updates became an issue almost two decades ago when Microsoft began patch Tuesday. Companies quickly figured out they couldn’t trust the quality of the patches and instead would test the patches in staging, which runs a duplicate environment to production. While this may have created a short window of vulnerability, it came with the advantages of stability and uninterrupted business operations.

Modern day IT Operations (called Platform Engineering or Site Reliability Engineering) now design production environments to be immutable and somewhat self healing. All changes need to be updated in code and then re-pushed through dev , test and staging environments to make sure proper QA and QC is followed. This minimizes impact from failed code pushes and will also minimize disruption from failed patches and updates like this one. SRE also closely monitors production environments for latency thresholds, availability targets and other operational metrics. If the environment exceeds a specific threshold then it throws alerts and will attempt to self heal by allocating more resources, or by rolling back to the previous known good image.

Ramifications

Materiality

Setting aside maturity of business and IT operations, there are some clear ramifications for this event. First, this had a global impact to a wide variety of businesses and services. Some of the biggest impacts were felt by publicly traded companies and as a result these companies will need to make an 8K filing with the SEC to report a material event to their business. Even though this wasn’t a cybersecurity attack, it was still an event that disrupted business operations and so companies will need to report the expected impact and loss accordingly. CrowdStrike in particular will need to make an 8K filling, not only for loss of stock value, but for expected loss of revenue through lost customers, contractual concessions and other tangible impacts to their business. When I started this post Friday of the even, CS stock was down over 10% and by Monday morning they were down almost 20%. The stock has started to recover, but that is clearly a material event to investors.

Greater Investment In BCP / DR & BIA

Recent events, such as this one and the UHC Change Healthcare ransomware attack, have clearly shown that some business are not investing properly in BCP / DR. They may have plans on paper, but plans still need to be fully tested including rapidly identifying service degradation and implementing recovery operations as quickly as possible. The reality is this should have been a non-event and any business that was impacted longer than a few hours needs to consider additional investment in their BCP / DR plan to minimize the impact of future events. CISOs need to work with the rest of the C-Suite to review existing BCP / DR plans and update them accordingly based on the risk tolerance of the business and desired RTO and RPO.

Boards Need To Step Up

During an event like this one boards need to take a step back and remember their primary purpose is to represent and protect investors. In this case, the sub-committees that govern technology, cybersecurity and risk should be asking hard questions about how to minimize the impact of future events like this and consider if the existing investment in BCP / DR technology and processes is sufficient to offset a projected loss of business. This may include more frequent reports on when the last time BCP / DR plans were properly tested and if those plans are properly accounting for all of the possible scenarios that could impact the business such as ransomware, supply chain disruption or global events like this one. The board may also push the executive staff to accelerate plans to invest in and modernize IT operations to eliminate tech debt and adopt industry best practices such as immutable infra or SRE. The board may also insist on a detailed analysis of the risks of the supply chain, including plans to minimize single points of failure, while limiting the blast radius of future events.

Negative Outcomes

Unfortunately, this event is likely to cause a negative perception of cybersecurity in the short term for a few different reasons. First, the obvious business disruption is one people will be questioning. How, is it a global cybersecurity company is able to disrupt so much with a single update? Could this same process act as an attack vector for attackers? Reports are already indicating that malicious domains have been set up to look like the fix for this event, but instead push malware. There are also malicious domains that have been created for phishing purposes and the reality is any company impacted by this event may also be vulnerable to ransomware attacks, social engineering and other follow on attacks.

Second, this event may cause a negative perception of automatic updates within the IT operations groups. I personally believe this is the wrong reaction, but the reality is some businesses will turn off the auto-updates, which will leave them more vulnerable to malware and other attacks.

The reality is this should have been a non-event and any business that was impacted longer than a few hours needs to consider additional investment in their BCP / DR plan to minimize the impact of future events.

What CISOs Should Do

With all this in mind, what should CISOs do to help the board, the C-Suite and the rest of the business navigate this event? Here are my suggestions:

First, review your contractual terms with 3rd party providers to understand contractually defined SLAs, liability, restitution and other clauses that can help protect your business due to an event caused by a third party. This should also include a risk analysis of your entire supply chain to determine single points of failure and how to protect your business appropriately.

Second, insist on increased investment in your BIA, BCP and DR plans including designing for site reliability and random events (chaos monkey) to proactively identify and recover from disruption, including review of RTO and RPO. If your BCP / DR plan is not where it needs to be, it may require investment in a multi-year technology transformation plan including resolving legacy systems and tech debt. It may also require modernizing your SDLC to shift to CI/CD including dev, test, staging and prod environments that are tightly controlled. The ultimate goal will be to move to immutable infrastructure and IT operations best practices that allow your services to operate and recover without disruption. I’ve captured my thoughts on some of the best practices here.

Third, resist the temptation to over react. The C-Suite and investors are going to ask some hard questions about your business and they will suggest a wide range of solutions such as turning off auto-patches, ripping out CS or even building your own solution. All of these suggestions have a clear tradeoff in terms of risk and operational investment. Making a poor, reactive, decision immediately after this event can harm the business more than it can help.

Finally, for mission critical services consider shifting to a heterogeneous environment that statistically minimizes the impact of any one vendor. The concept is simple, if you need an security technology to protect your systems consider purchasing multiple vendors that all have similar capabilities, but will minimize the impact of your business operations if one of them has an issue. This obviously raises the complexity and operational cost of your environment and should only be used for mission critical or highly sensitive services that need to absolutely minimize any risk to operations. However, this event does highlight the risks of consolidating to a single vendor and you should conduct a risk analysis to determine the best course of action for your business and supply chain.

Wrapping Up

For some companies this was a non-event. Once they realized there was an outage they simply executed their recovery plans and were back online relatively quickly. For other companies, this event highlighted lack of investment in IT operations fundamentals like BCP / DR or supply chain risk management. On the positive side, this wasn’t a ransomware or other cybersecurity attack and so recovery is relatively straightforward for most businesses. On the negative side, this event can have negative consequences if businesses over react and make poor decisions. As a CISO, I highly recommend you take advantage of this event to learn from your weaknesses and make plans to shore up aspects of your operations that were sub-standard.

Should CISOs Be Technical?

Don’t want to read this? Watch a video short of the topic here.

There are a lot of different paths to becoming a CISO and everyone’s journey is different, however two of the most common paths are coming up through the technical ranks or transitioning over from the compliance function. Coming up through the technical ranks is common because cybersecurity is a technically heavy field, particularly when attempting to understand the complexities of how exploits work and the best way to defend against attackers. Coming up through the compliance ranks is also common because companies are often focused on getting a particular compliance certification in order for them to conduct business and interact with the customers. Each of these paths offers advantages and disadvantages, but I will argue being technical is more challenging than some of the softer cybersecurity disciplines like compliance, which leads to a common question – do CISOs need to be technical?

Yes, but…

If you don’t want to read any further the short answer is yes, CISOs need to be technical. The longer answer is, being technical is a necessary, but insufficient characteristic of a well rounded CISO. The reason being technical is insufficient is because for the past few years the CISO role at public companies has been transforming from a technical role to a business savvy executive role. CISOs are expected to report to the board, which requires speaking the language of business, risk and finance. I have seen CISOs quickly lose their audience in board meetings when they start talking about tooling, vulnerabilities and detailed technical aspects of their security program. CISOs need to be able to translate their security program into the language of risk and they need to be savvy enough to weave in financial and business terminology that the board and other C-Suite executives will understand.

Obtain (and maintain) A Technical Grounding

Even though being technical is no longer sufficient for a well rounded CISO it is important for a CISO to obtain or maintain a technical grounding. A technical grounding will help the CISO translate technical concepts (like vulnerabilities and exploits) into higher level business language like strategy, risk or profit and loss (P&L). It is also important for a CISO to understand technical concepts so they can dig in when needed to make sure their program is on track or controls are operating effectively. Lastly, it is important to maintain technical credibility with other technical C-Suite stakeholders like the CTO and CIO. Speaking their language will help align these powerful C-Suite members with your security program, who can then lend critical support when making asks for the rest of the C-Suite or board.

What other skills does a CISO need?

In addition to a technical grounding, there are a number of skills CISOs need to master in order to be effective in their role. The following is a short list of skills CISOs need to have in order to be successful at a public company:

  • Executive presence and public speaking skills with the ability to translate security concepts into business risk that resonates with senior executives and the board
  • Ability to lead and communicate during a crisis
  • Politically savvy, with ability to partner with and build alliances with other parts of the business
  • Ability to understand the core parts of the business, how they operate and what their strategy is
  • Ability to explain the “value” of your security program in business and financial terms
  • Strong understanding of financial concepts such as CAPEX, OPEX, P&L, budgeting and ability to understand balance sheets, earning results and SEC filings
  • Understand and navigate legal concepts (such as privilege), regulations and compliance activities with the ability to map these concepts back to your security program or testify in court (if needed)
  • Ability to interact with auditors (when needed) to satisfy compliance asks or guide responses
  • Ability to interact with customers to either reassure them about the maturity of your security program or act as an extension of the sales team to help acquire new customers
  • Interact with law enforcement and other government agencies, depending on the nature of the business

If this seems like a long list that doesn’t fit your concept of what a CISO does, then you may have some weaknesses you need to work on. This list also reflects the evolving nature of the CISO role, particularly with respect to board interaction and leadership at public companies. More importantly, a lot of these concepts are not covered in popular security certifications and you definitely won’t get all of this experience from start ups or non-public companies. That is ok, because recognizing and acknowledging your weaknesses is the first step to becoming a better CISO.

Are Security Certifications Worth Renewing?

Almost weekly I see someone post a question on social media asking: “Is renewing my security certification worth it?” This is a valid question since security certifications are often expensive, time consuming and hard won. Maintaining your security certification may be required to land a new job, but not required to continue in the role. At the end of the day the question people are really asking is: “Is the continued expense of this certification worth it after I’ve landed the role I’m after or achieved my career objectives?” In this post I’ll explore the pros and cons of renewing a security certification and wrap up with my specific recommendation for those of you looking for guidance.

Getting Certifications

There are a number of popular security certifications that can demonstrate general or specific expertise. Some of the most popular are:

  • CISSP
  • CISM
  • CEH
  • Security+
  • CISA
  • CCSP

Along with the variety of certifications there are different ways to earn a certification. The least expensive and most time consuming is to purchase the course material, self study and then sit for the exam. The most expensive and least time consuming is to attend a boot camp and then test for the exam on the last day. If you are lucky your employer will pay for or reimburse the expense of the certification. No matter which way you go, there is a material cost in terms of dollars and time. This cost can make people reluctant to let certifications expire because the certifications have a high barrier to entry, but a (relatively) low maintenance cost.

Pros To Renewing

One of the main reasons to continue to maintain your certification is because they are required by some job roles in order to be hired and perform the role. One example of this is in the U.S. Government Department of Defense (DoD) 8570 Approved Baseline Certifications. The 8570 specifies “Personnel performing IA functions must obtain one of the certifications required for their position, category/specialty and level to fulfill the IA baseline certification requirement.” So if you want the job, and want to keep the job, then you need the certification.

In addition to job requirements, maintaining an active certification gives the impression of having expertise in a particular area. Demonstrating expertise is useful when speaking, consulting, providing legal testimony or simply cementing your position as an expert in the field of security. Expertise is useful when trying to land a new role or get a promotion. This can also be useful to limit personal liability if you can demonstrate you followed the best practices indicated by the certification. Maintaining a certification and this expertise is arguably a low cost, low effort activity with a lot of upside and not a lot of downside. Even though there is a dollar cost for renewal, this is a minor amount compared to the overall expense or time invested in getting the certification in the first place.

One final reason to maintain your security certifications is because it demonstrates you have a baseline level of knowledge as indicated by the certification. When you were studying and testing for the certification you were learning new concepts and confirming mastery of other concepts. This can be useful to validate your expertise, but also to demonstrate to others that you have mastered these concepts and can operate at the same level as other individuals that have the certification. This goes beyond demonstrating expertise in that is establishes a baseline of knowledge for security practitioners in the field and this is why employers often list specific certifications on job descriptions.

Cons To Renewing

Even though there are a number of benefits to maintaining a certification, there are also a lot of cons.

First, there is the obvious annual cost for renewing the certification. Not only is there a dollar cost, but there is also usually a time cost in the form of continuing education credits that have to be earned and submitted throughout the year. The idea is to drive engagement in the security community by requiring these continuing education credits, but in my opinion this has had mixed results. For anyone on the fence about renewing a certification the time and dollar cost can be the breaking point where it no longer makes sense to continue to invest in something that isn’t demonstrating continued value.

Speaking of continued value, what are you really getting by spending time on continuing education and paying the renewal fee? You get the privilege to list the certification on your resume, but you’ve already gained the knowledge and passed the test. Renewing doesn’t typically require another test so is there really continued value (assuming you aren’t required to maintain it for your job)? The value is questionable if it isn’t required and so it can be difficult to justify maintaining.

Another downside to maintaining the certification is the continuing education credits. There are a number of low cost or free ways to earn credits, but it can be difficult or almost impossible to get to the required number without spending money. This is a con in my opinion because renewing the certification is perpetuating additional expenses such as more certifications, attending more conferences or other expenses just to earn enough credits. This means even though there is a low renewal cost, there can be a really high dollar or time cost to earn enough continuing education credits to maintain the certification.

In the pros section I listed the DoD 8750, which requires certain certifications to obtain and perform certain roles. However, requiring certifications for a job can also have a downside by eroding the exclusivity of the certification. This happened to the MCSE in the late 90’s and early 2000’s when everyone wanted an MCSE because it paid really well. However, soon everyone had it even if they weren’t doing the job and the MCSE became useless. It was no longer a good barometer for demonstrating expertise because so many non experts had it. Some security certifications are the same way and the DoD 8570 (or other employers) can contribute to this erosion of exclusivity if the people earning the certification are simply getting it to fill the role instead of becoming experts in the field.

One last con for renewing certifications is you may no longer be doing the type of job that requires the certification. In the past I held the GCIH, GREM and GPEN certifications, but I no longer do those hands on activities so it doesn’t make sense for me to maintain those certifications. If your career has taken you on a different path, then you no longer need to maintain the cert. Also, I will argue your job title can be more useful to demonstrate expertise than a certification. This isn’t always the case and this can sometimes be difficult to tease out with discretionary titles, but generally if you have carried the CISO or CSO title in some capacity do you really need to maintain an active certification? I’ve seen several individuals list their expired certifications on their resume, which continues to demonstrate the expertise, but without the added expense.

My Recommendation

If you are on the fence about whether or not to renew your security certification here is a simplistic flow chart for helping you with the decision. Feel free to recreate and add your own additional criteria as necessary.

My particular recommendation is as follows: if you want to maintain the credibility, demonstrate expertise, are still doing the job and can afford the renewal cost (both time and dollar), then renewing is typically not too expensive and worth it. I am also seeing a lot of job descriptions require active certifications so if you are about to job hunt or at risk of getting laid off then maintaining your certifications is a good idea. If you are no long doing the job, don’t need the credibility or expertise and the certification isn’t required by your job then I suggest no longer renewing and focusing on other areas. In my case, I have dropped most of the specialist certifications, while maintaining the generalist certifications in line with my role.

Navigating Hardware Supply Chain Security

Lately, I’ve been thinking a lot about hardware supply chain security and how the risks and controls differ from software supply chain security. As a CSO, one of your responsibilities is to ensure your supply chain is secure, yet the distributed nature of our global supply chain makes this a challenging endeavor. In this post I’ll explore how a CSO should think about the risks of hardware supply chain security, how they should think about governing this problem and some techniques for implementing security assurance within your hardware supply chain.

What Is Hardware Supply Chain?

Hardware supply chain relates to the manufacturing, assembly, distribution and logistics of physical systems. This includes the physical components and the underlying software that comes together to make a functioning system. A real world example could be something as complex as an entire server or something as simple as a USB drive. Your company can be at the start of the supply chain by sourcing and producing raw materials like copper and silicon, at the middle of the supply chain producing individual components like microchips, or at the end of the supply chain assembling and integrating components into an end product for customers.

What Are The Risks?

There are a lot of risks when it comes to the security of hardware supply chains. Hardware typically has longer lead times and longer shelf life than software. This means compromises can be harder to detect (due to all the stops along the way) and can persist for a long time (e.g. decades in cases like industrial control systems). It can be extremely difficult or impossible to mitigate a compromise in hardware without replacing the entire system (or requiring downtime), which is costly to a business or deadly to a mission critical system.

The risk of physical or logical compromise can happen in two ways – interdiction and seeding. Both involve physically tampering with a hardware device, but occur at different points in the supply chain. Seeding occurs during the physical manufacture of components and involves someone inserting something malicious (like a backdoor) into a design or component. Insertion early in the process means the compromise can persist for a long period of time if it is not detected before final assembly.

Interdiction happens later in the supply chain when the finished product is being shipped from the manufacturer to the end customer. During interdiction the product is intercepted en route, opened, altered and then sent to the end customer in an altered or compromised state. The hope is the recipient won’t detect the slight shipping delay or the compromised product, which will allow anything from GPS location data to full remote access.

Governance

CSOs should take a comprehensive approach to manage the risks associated with hardware supply chain security that includes policies, processes, contractual language and technology.

Policies

CSOs should establish and maintain policies specifying the security requirements at every step of the hardware supply chain. This starts at the requirements gathering phase and includes design, sourcing, manufacturing, assembly and shipping. These policies should align to the objectives and risks of the overall business with careful consideration for how to control risk at each step. An example policy could be your business requires independent validation and verification of your hardware design specification to make sure it doesn’t include malicious components or logic. Or, another example policy can require all personnel who physically manufacture components in your supply chain receive periodic background checks.

Processes

Designing and implementing secure processes can help manage the risks in your supply chain and CSOs should be involved in the design and review these processes. Processes can help detect compromises in your supply chain and can create or reduce friction where needed (depending on risk). For example, if your company is involved in national security programs you may establish processes that perform verification and validation of components prior to assembly. You also may want to establish robust processes and security controls related to intellectual property (IP) and research and development (R&D). Controlling access to and dissemination of IP and R&D can make it more difficult to seed or interdict hardware components later on.

Contractual Language

An avenue CSOs should regularly review with their legal department are the contractual clauses used by your company for the companies and suppliers in your supply chain. Contractual language can extend your security requirements to these third parties and even allow your security team to audit and review their manufacturing processes to make sure they are secure.

Technology

The last piece of governance CSOs should invest in is technology. These are the specific technology controls to ensure physical and logical security of the manufacturing and assembly facilities that your company operates. Technology can include badging systems, cameras, RFID tracking, GPS tracking, anti-tamper controls and even technology to help assess the security assurance of components and products. The technologies a CSO selects should complement and augment their entire security program in addition to normal security controls like physical security, network security, insider threat, RBAC, etc.

Detecting Compromises

One aspect of hardware supply chain that is arguably more challenging than software supply chain is detection of compromise. With the proliferation of open source software and technologies like sandboxing, it is possible to review and understand how a software program behaves. Yet, it is much more difficult to do this at the hardware layer. There are some techniques that I have discovered while thinking about and researching this problem and they all relate back to how to detect if a hardware component has been compromised or is not performing as expected.

Basic Techniques

Some of the more simple techniques for detecting if hardware has been modified is via imaging. After the design and prototype is complete you can image the finished product and then compare all products produced against this image. This can tell you if the product has had any unauthorized components added or removed, but it won’t tell you if the internal logic has been compromised.

Another technique for detecting compromised components is similar to unit testing in software and is known as functional verification. In functional verification, individual components have their logic and sub-logic tested against known inputs and outputs to verify they are functioning properly. This may be impractical to do with every component if they are manufactured at scale so statistical sampling may be needed to probabilistically ensure all of the components in a batch are good. The assumption here is if all of your components pass functional verification or statistic sampling then the overall system has the appropriate level of integrity.

To detect interdiction or logistics compromises companies can implement logistics tracking such as unique serial numbers (down to the component level), tamper evident seals, anti-tamper technology that renders the system inoperable if tampered with or makes it difficult to tamper with something without destroying it and even shipping thresholds to detect shipping delay abnormalities.

Advanced Techniques

More advanced detection techniques for detecting compromise can include destructive testing. Similar to statistical sampling, destructive testing involves physically breaking apart a component to make sure nothing malicious has been inserted. Destructive testing makes sure the component was physically manufactured and assembled properly.

In addition to destructive testing, companies can create hardware signatures that include expected patterns of behavior for how a system should physically behave. This is a more advanced method of functional testing where multiple components or even finished products are analyzed together for known patterns of behavior to make sure they are functioning as designed and not compromised. Some hardware components that can assist with this validation are technologies like Trusted Platform Modules (TPM).

Continuing with functional operation, a more advanced method of security assurance for hardware components is function masking and isolation. Function masking attempts to mask a function so it is more difficult to reverse engineer the component. Isolation limits how components can behave with other components and usually has to be done at the design level, which effectively begins to sandbox components at the hardware level. Isolation could rely on TPM to limit functionality of components until the integrity of the system can be verified, or it could just limit functionality of one component with another.

Lastly, one of the most advanced techniques for detecting compromise is called 2nd order analysis and validation. 2nd order analysis looks at the byproduct of the component when it is operating by looking at things like power consumption, thermal signatures, electromagnetic emissions, acoustic properties and photonic (light) emissions. These 2nd order emissions can be analyzed to see if they are within expected limits and if not it could indicate the component is compromised.

Wrapping Up

Hardware supply chain security is a complex space given the distributed nature of hardware supply chains and the variety of attack vectors spanning physical and logical realms. A comprehensive security program needs to weigh the risks of supply chain compromise against the risks and objectives of the business. For companies that operate in highly secure environments, investing in advanced techniques ranging from individual component testing to logistics security is absolutely critical and can help ensure your security program is effectively managing the risks to your supply chain.

References:

Guarding Against Supply Chain Attacks Part 2 (Microsoft)

Long-Term Strategy for DoD Trusted Foundry Needs (ITEA)

Whats The Difference Between A CSO and CISO?

Like Arnold Schwarzenegger to Danny DeVito in the movie Twins, the Chief Security Officer (CSO) role is the big brother to the Chief Information Security Officer (CISO) role. What is the difference between these two roles and what skills does a CISO need to focus on if they aspire to become a CSO? In this post I’ll explore the role of the Chief Security Officer (CSO) and what additional responsibilities the role covers when compared to the CISO role.

Big Brother

Lately, there has been a lot of focus on the Chief Information Security Officer (CISO) role following the new SEC guidelines, recent ransomware attacks and supply chain security vulnerabilities (XZ). There can be a lot of different titles for the top security executive at a public company, but the two most common titles for a public company are Chief Information Security Officer (CISO) and Chief Security Officer (CSO). The Twins movie is a good analogy to describe the relationship between the CSO and CISO because in the movie Arnold protects Danny DeVito by helping him avoid trouble, while Danny is super scrappy and shows Arnold how the real world works. They complement each other, protect each other and help each other. One is the overall leader and one has a great hustle.

What the Twins analogy highlights is the main difference between a CSO and CISO is scope. A CSO typically has a bigger scope than a CISO. A CISO will have responsibility for all of the information and technology assets of a company, but a CSO will have this responsibility and additional responsibilities for physical security, executive protection, corporate investigations and other non-information technology based security domains. In fact, for public companies that have an established CSO role, it is typical for the CISO role and function to report to the CSO as one overall security function. Let’s dig into some of the additional functions of a CSO.

Like Arnold Schwarzenegger to Danny DeVito in the movie Twins, the Chief Security Officer (CSO) role is the big brother to the Chief Information Security Officer (CISO) role.

Physical Security

One of the biggest responsibilities for a CSO is physical security. Physical security includes site security for offices and the physical security of the personnel working at the facilities were the company operates. This can include things like cameras and video monitoring, badging systems, security and fire alarm systems, safes, locks, lighting, parking and loading docks, contractor access, mail and package security, bollards and traffic control, security guards and gates, fencing, fire suppression and other physical environment aspects. Depending on the nature of your business, this could also involve supply chain security of manufacturing facilities and components, or even critical infrastructure. It can also include tempest and RF control, including design and management of classified spaces.

One interesting aspect of physical security is to work with construction companies or physical security consulting firms to design and assess the security controls of your facilities. Books like Red Cell by Richard Marcinko offer an interesting historical perspective of how the military physically tests the security of their military installations and public companies should similarly consider an annual or periodic review of their physical security for weaknesses and risks.

If your company is involved in manufacturing, another interesting aspect of physical security is supply chain security and logistics. This is ensuring your products are manufactured securely and aren’t tampered with during the manufacturing process. It can also include assessing the security of component manufacturers, assembly plants and even shipping and logistics companies to make sure your products arrive to your customers and are functioning securely.

Lastly, another aspect of the CSO’s physical security responsibilities is interfacing with local and federal law enforcement for trends, threats and dealing with physical disruptions at your places of business like the recent examples of protests at Google offices.

Executive & Travel Protection

Another responsibility of the CSO, which is related to physical security, is executive and travel protection. Executive and travel protection covers how to physically protect your top executives from threats when they are in public, traveling, at their offices or at their homes. This can include arranging trusted transportation, route planning, on site security surveys, sending advanced teams ahead of the execs, kidnap and ransom insurance, medical support and even online reputation management. You may even arrange training for your execs such as mock kidnapping situations or how to deal with other emergency situations (like riots, terrorist attacks, wars or coups).

Executive and travel protection can include interfacing with local embassies, law enforcement or emergency services depending on the threat level of the country your senior execs are visiting. This is in addition to the existing CISO responsibilities of interfacing with law enforcement for security breaches, APTs, ransomware attacks, digital fraud, etc. Exec and travel protection can also include arranging for security companies to beef up the security of their home(s) and arranging to have their home security monitored by a private security company (if this is part of their perquisites).

Lastly, one very important aspect of executive and travel protection is digital device security. This responsibility may get delegated to the CISO, but the CSO still needs to understand and include digital security as comprehensive part of their executive protection strategy. Certain countries are known to be digitally hostile by attempting to siphon information from or compromise the devices of executives at top companies. This can be attempts at industrial espionage, theft of military and defense information, gaining business advantages, disrupting business, leveraging the exec as an attack vector into the broader company, trade advantages or potential blackmail. The CSO should consider these risks based on the destination country and provide appropriate controls to executive devices such as providing burner phones and laptops for specific country use that are sterile and won’t impact the company or personal reputation of the executive if compromised.

Executive and travel protection is important to ensure your top execs are safe and secure when traveling, but also, if your business is controversial or your top execs like to make controversial statements, this function can ensure they are safe and protected no matter what situation they are in.

Corporate Investigations

One final area of responsibility for the CSO is corporate security investigations beyond the normal technology investigations handled by the CISO. Corporate security investigations can include theft, financial crimes, waste, abuse, vandalism, misconduct, bribery and supply chain control (for ITAR or other export / import laws). You may work closely with law enforcement at the state or federal level depending on the nature and scope of the investigation and the CSO function is critical to coordinating the investigation and representing the business appropriately. Corporate investigations can also involve acting as an expert witness or providing testimony in court on behalf of your company.

One important aspect to remember is, CSOs need to have clear processes and policies defined for how and when to involve law enforcement. The decision to involve law enforcement may be based on legal requirements or may be based on other decisions, but involving law enforcement gives up control of the investigation, which could result in property being confiscated as evidence. If the evidence is a critical business asset like IT equipment, the CSO needs to ensure there are redundancies in place so the business is not disrupted or left without that capability while supporting the investigation.

Wrapping Up

The CSO role is an interesting top security executive role and offers a broader scope than the CISO role. CISOs looking to expand their remit should consider establishing credibility in the areas I’ve described above, but should also remember that most professional security certifications like the CISSP cover aspects of physical security as one of the knowledge based domains. If you don’t have a military or law enforcement background, two interesting certifications that can establish physical security credibility for CISOs are ISMA and ASIS. Lastly, CSOs will typically have responsibility for the CISO function (with the CISO reporting to them), but will also have additional remit in areas of physical security, executive protection, travel protection and corporate investigations. In my experience, the CSO role is more interesting because you get involved in all aspects of security for a company allowing you to channel your inner Arnold Schwarzenegger from Twins, while still retaining the option to flex your Danny DeVito (CISO) roots.

When Evaluating A New CISO Role Don’t Forget The SEC 10-K And Other Governance Forms

When evaluating a new CISO role it is common to do research on the company, industry, product line, etc., but an area that is often overlooked are SEC filings like the SEC Form 10-K and board committee charters. SEC filings and committee charters can offer a wealth of information about how a company views and governs key issues like cybersecurity and risk. In this post I’ll cover where to find key information, red flags to watch out for and other useful information that can be discussion topics during the interview process.

Finding The Right Forms

If you are new to reviewing SEC filings and corporate governance documents there are a number of places to find documents about corporate governance and how the company strategically views cybersecurity and risk. These documents will provide insight into who you may need to influence in order to execute a successful security program and it will also give you an implicit understanding of the priority the company assigns to cybersecurity issues. The two best places to find relevant forms are on SEC.gov (Edgar) or on the company’s own investor relations website.

SEC Forms

The most common SEC forms you will want to review when preparing for a new CISO role are the SEC Forms 10-K, 10-Q and 8-K.

  • 10-K: The SEC Form 10-K is a comprehensive annual report filed by public companies. It has a wealth of information such as their financials, how they view the market, executive compensation and more. When considering a new CISO role definitely check out section 1 and 1A. Section 1 covers an overview of the business and section 1A covers macro risk factors (you may be asked to help mitigate these risks). Section 1 may also specifically call out cybersecurity governance and have details on the reporting structure, responsibilities, experience and methods for governing cybersecurity at the company. Also check out section 7, which will detail how management describes the company and can also have details on recent acquisitions or restructuring activities, which could continue to present a risk to the business.
  • 10-Q: The SEC Form 10-Q is a comprehensive quarterly report filed by the public company. This will detail their quarterly results and will also provide any updates or changes to the sections I listed above – mainly section 1, 1A and section 7. Most of the time there won’t be any updates to these sections and they will refer back to the 10K, but it is still good to review the latest 10-Q available.
  • 8-K: The SEC Form 8-K is a form companies must file to notify investors of major events. The biggest thing CISO candidates will want to review is if the company has had any material cybersecurity or operational incidents. However, if the company deems an event isn’t material it may not be in the 8-K and so it is a good idea to do a web search of the company as well.
  • Committee Charter Docs: The last set of documents to review are the committee charter documents. This will tell you how the board is structured, which can give you insights into what to expect if you take the role and give periodic updates to the board. The committee charter documents will also outline how they govern cybersecurity, risk and technology and the committee charter documents can give you implicit insight into how the company views the role of the CSO / CISO and cybersecurity.

How Should Cybersecurity Be Governed?

When reviewing the governance and committee documents of a public company, you may find cybersecurity discussed in different places. You should review these documents and also consider discussing cybersecurity governance during the interview process.

Audit committee

The audit committee is the most common committee to govern cybersecurity and risk at a public company. The challenge with placing cybersecurity and risk in the audit committee is the primary function of that committee is financial accuracy and integrity. Cybersecurity and risk are typically listed as “other functions”, which runs the risk of it not having the same priority as financial activities and the committee members may not have the right expertise to govern these functions. The typical executive experience of an audit committee member can be CEO, CFO or COO and these individuals typically aren’t experts in cybersecurity or risk. It isn’t the end of the world, but as a CISO candidate you should review the backgrounds of the audit committee board members and ask how they interact with existing C-Level executives when discussing cybersecurity, technology and risk. You may even want to ask to interview with one of the committee members before taking the job. The main goal is to make sure you are going to get the consideration, prioritization and support you need.

Tech and cyber committee

Aside from the audit committee, the other committee that governs cybersecurity and risk is the technology and cyber committee. However, the existence of this committee is currently non-standard at public companies even though it is considered best practice for corporate governance. If the company you are interviewing has a technology and cybersecurity committee consider yourself fortunate, but you should still do your own due diligence by researching the existing committee members and their backgrounds. Consider requesting an interview with one of these committee members (if it isn’t part of the interview process) to get their perspective on cybersecurity governance and issues at the company.

The challenge with placing cybersecurity and risk in the audit committee is the primary function of that committee is financial accuracy and integrity.

Other Cybersecurity Governance Aspects To Consider

There are a few other aspects to consider when reviewing corporate governance documents. These other areas can give you valuable insight into what is expected of you if and when you assume the role of CISO at the company. First, I recommend covering materiality during the interview process. Ask if the company has a process and if possible discuss their criteria for determining materiality of a security incident. Second, review and assess how often the board committee responsible for cybersecurity meets. This can give you an idea of how often you will be expected to present to the board and may even give you an idea of the topics that are discussed.

Red Flags

The whole point of reviewing these documents is to help you make an informed decision about what you are walking into if you take the role. There are few red flags you should look out for in these documents that should definitely be discussed during the interview to make sure you are clear on your role and expectations. These red flags may also help you when negotiating for things like severance, inclusion in the D&O liability policy or other concessions.

10-K & 10-Q

Remember, the 10-K and 10-Q will have a section on risks and the company may specifically call out cybersecurity risk as a macro issue they are concerned about. However, one red flag I would bring up for discussion is does the company address how they plan to manage these risks? Something as simple as “we plan to discuss and manage these risks inline with business priorities and expectations to minimize their impact” indicates they have at least given it some thought. Even better, if the company has a detailed section on risk and risk management that addresses how they plan to govern the company to address these risks. If the 10-K and 10-Q just list the risks, it may be an indication the company is paying lip service to cybersecurity or it could mean they are waiting for the right candidate to come in and develop a plan.

Experience Of Committee Board Members

Another potential red flag is the background and experience of the board members for the committee that governs cybersecurity and technology risk. Review their background, how long they have been serving on the board and when they are up for re-election. If the committee members have a strong technology or cybersecurity background you can expect to find an ally in the board room. If the committee members haven’t been technology executives you may find you have to change your message or do some education when reporting to the board. The SEC has indicated cybersecurity experience is necessary for the board to effectively govern risks, so if there isn’t clear experience, it is something to bring up in the interview for how and when the company is planning to address the experience gap.

Cybersecurity As Part Of The Audit Committee

I previously mentioned most public companies have cybersecurity listed as an additional function of the audit committee. This can be a red flag if the board doesn’t have committee members with technology experience, but can also be a red flag if the company views the CISO role and security program as more of a compliance function. The view of the board will be directly related to how much funding and support you are able to get from the rest of the company like the CEO and CFO.

Having cybersecurity and risk as part of the audit committee can also lead to a disconnect from the main security program. For example, if the audit committee treats security more as a compliance function, they may request a group that reports directly to them that audits the effectiveness of the corporate security program. This can lead to duplication of effort, cross purposes and mixed messaging at the board level. It can also undermine the authority of the CISO if the board is independently dictating security actions to the company outside of the main security program. However, having cybersecurity as part of the audit committee isn’t the end of the world and can actually lead to support from the board, but it will require additional effort and relationship management to make sure the board is supporting your program effectively. These are all topics you will want to explore during your interview.

Other SEC Filings

There are a few other areas you should review when conducting research for a new CISO position. I highly recommend reviewing recent 8-K filings and conducting internet searches to see if the company has reported any recent security incidents or breaches. If they have, you may be walking into a situation where they need immediate help to get back to a good state, but that support may wane after the urgency of the situation dies down. If you are considering taking a role that is walking into a post incident situation, be really clear on expectations and success criteria and try to build those into your employment contract.

The other area I recommend reviewing is recent or ongoing M&A activity. This will be listed in the 10-K or 10-Q filings for the company and it can give you some insight into what you may be walking into as a CSO / CISO. M&A activity is notorious for “closing the deal” and then sorting everything out later. As a CISO this means you could be inheriting a heterogenous security program or you may have to spend a significant amount of time up-leveling the acquisition to meet the standards of the rest of the company. There may even be extensive integration, standardization, etc. that needs to be completed. All of these are risks that you should be aware of when walking into a new CISO role.

Wrapping Up

When evaluating a new CISO role for a public company I recommend thoroughly researching the company as part of your evaluation process for the role. Familiarize yourself with their business model, the latest news articles, key members of the executive staff, board members and financial statements. If you have a strong CISO network I recommend reaching out to them and getting their perspective on the position. However, overlooked areas of research are the public company filings with the SEC and other investor relations documents that can give you more perspective on the company. It is particularly important to review these documents to get an idea of how the company governs cybersecurity and risk. These documents will also highlight potential red flags and discussion topics to explore during your interview. Thoroughly researching the company and the role will not only help prepare you for the interview process, but can also give you insight into how other public companies govern these issues so you can compare with your current position and make the best decision possible for your career.

Resources

SEC Search

DDN Discussion Of Cybersecurity Governance