The Problem With Vulnerability Scanners

Vulnerability scanners are table stakes for any security program and they help security teams proactively identify and report on the security posture of assets, but unless you tune them properly they can lead to more problems than they fix. Here are a few things you need to take into consideration when selecting and using a vulnerability scanner as part of your security tooling.

Scanning Technique

There are two primary scanning techniques used by vulnerability scanners. The first is an unauthenticated scan, which is essentially an external scan that attempts to enumerate the ports and services running on the system or device and then match those up to known vulnerabilities. The advantage of an unauthenticated scan is it is usually easier to implement because you don’t have to load any agents onto systems. You simply turn on the service at a centralized location, tell it the IP ranges or subnet to scan and then have it dump the output somewhere you can review. However, this convenience comes with a tradeoff in terms of accuracy and coverage. By coverage I mean the ability for the scanner to fully scan everything that could be potentially be vulnerable on the system it is scanning.

The second type of scan is an authenticated scan. Authenticated scans typically require an agent to be installed on the system or for the scanner to somehow log into the device so it can scan the running services and applications for known vulnerabilities. An authenticated scan is much more accurate because the scanning agents are running on the same OS as the services and so it can eliminate false positives and provide a more comprehensive scan. However, authenticated scans also come with a tradeoff, which is you are getting much higher accuracy and coverage, but that increase in volume doesn’t factor in where that system is in your environment (i.e. internally vs. externally facing). As a result your reporting may not accurately measure the true risk to the business and you could end up having engineering teams spend time fixing vulnerabilities that may not really be a risk.

How Good Is Your Inventory?

No matter which type of scanning you choose, the scanners are only as good as your asset inventory. You can’t report on and fix what you don’t know about and so the ability to identify and scan assets is critically important. This is where an authenticated scan via agents can present a false picture to a security team. It is easy for the team to assume they are scanning the full environment, but that may not be the case if the agents aren’t installed on everything or if the scanner isn’t scanning all of your devices. Vulnerability scanners shouldn’t just scan your operating systems (compute), but should also scan your network, storage, IoT devices and anything else with a network address to present a complete picture of your enterprise.

Agents

Agents are great for increasing your scan accuracy and coverage, but they present their own challenges. First, you need to deploy all those agents onto your systems and make sure they don’t cause performance issues. This can be a time consuming process to tune the compute, storage and memory for your workloads. Second, it is easy to run into agent creep and have systems with dozens of agents on them that each do something different. These agents can conflict with each other for resources and can be difficult to manage for operational support teams.

Scan Remediation Gap

Vulnerability scanners have an inherent problem when identifying new vulnerabilities. It is often the case that a vulnerability scanner can get updated with a new vulnerability, but a fix or patch may not yet exist to remediate the vulnerability. This can present a problem for businesses that are trying to react to a critical issue quickly, but have to wait for a vendor or developer to provide a fix or implement a more complex compensating control instead. A good question for your vulnerability scan vendor is how quickly they can update their scanner and do they provide additional information about whether a fix is available or not.

CVSS Scores Aren’t Very Useful

CVSS scores have their value, but they aren’t particularly useful to prioritize risk. A truly effective vulnerability management program will take into consideration a lot of other data along with the vulnerability score to make a true determination of risk and prioritize remediation efforts. Some things I recommend adding to your vulnerability reporting to help you prioritize are the following:

  • Business criticality of the system (can it go down or is it revenue generating?)
  • How much revenue does this system make?
  • Is the system publicly facing or is it only accessible internally?
  • Is an exploit available?
  • Is a patch available?
  • What is the complexity of the exploit?
  • Are there other compensating controls already in place (like a WAF) or can you put them in place quickly?

Wrapping Up

Vulnerability scanners are an essential tool in any security program, but they can give security teams false confidence or worse, create a lot of noise for engineering teams. Understanding what type of scan you are using, the tradeoffs for that type of scan and linking the results of your scan to business risk can help any security team accurately identify and prioritize vulnerability remediation.

The Different States Of A Security Program

It may be obvious, but every company that has a security program is in a different state of maturity. As a CSO, it is important to recognize and understand what these different states mean in terms of where your energy will be applied. If you are interviewing or hiring into a company, it is critically important to understand what state the security program is in so you can determine if the opportunity is right for you and to ultimately maximize your impact in the role.

The Different States

In general, a security program can be in one of three different states:

  • New / Building
  • Existing / Incremental
  • Shrinking / Decline

New / Building

A security program that is new typically comes along with new companies, startups or possibly new business units that are acquired via acquisition. However, a company may also be establishing a new program if they are found deficient during an audit or if they suffered a security breach. In this state the CSO (or security leader) needs to establish a program from scratch, which will include mapping risks, developing a budget and establishing funding, recommending tools, evangelizing security best practices and hiring a team. There will be a lot of focus on foundational aspects of security like asset inventory, reporting and initial risk baselines for the organization. Your team will also go after initial program certifications like ISO27001, SOC or other compliance activities. You may even need to establish new processes and ways of working.

Here are some good questions to ask to determine if a program is in the new / building state:

  • Who is performing the function of security today?
  • What goals does the organization have in the first year and three years from now?
  • What is the expected annual budget?
  • How many headcount do you expect for the security team in the first year?
  • Where does your company operate and do you expect to have security resources in those geographic regions?
  • What security tooling is in place today (if any)?
  • Does the company have any existing compliance certifications (like SOC, ISO, etc.)?
  • Why is the company focusing on hiring a security leader and building a security program? Did this come about due to a security incident or other security event like a failed audit?
  • What industries does the company do business in? E.g. finance, government, healthcare, etc.

In my experience, establishing a new security program from scratch is a rare opportunity, but if you get the chance it is truly exciting and offers the opportunity for giant leaps forward in terms of security maturity for the company.

Existing / Incremental

The next state of maturity is existing or incremental and most companies will be in this state. In this state a security program has already been established and has the foundations in place in terms of people, processes and technology. Tooling has already been purchased and implemented, an annual budget has been established and a team exists with different functions like security engineering, security operations and security compliance.

An existing security program usually has smaller goals or incremental annual objectives designed to address some specific area of risk that has been outstanding, or to address a new risk area based on business growth. For example, perhaps the organization has an existing Identity and Access Management (IAM) program, but needs to roll out 2-Factor Authentication (2FA) to further secure access. Or, maybe the business is expanding into the financial industry and needs to become PCI-DSS compliant. These are incremental improvements to the security program and will require increases or reallocation of people and budgets.

A CSO or security leader in charge of an existing security program will generally keep things running smoothly, make sure the company doesn’t regress with respect to security maturity and will continually be evaluating the business for new or existing risks that need to be managed.

Here are some questions you can ask if you are interviewing for a new role that will lead an existing security program:

  • What is the annual budget for the security program?
  • What security tools are in place?
  • How is the team structured?
  • What are the security objectives for this year? For three years?
  • What security compliance certifications does the company maintain (e.g. SOC, ISO, etc.)?
  • How many people are in the security team?
  • What functions does the security team perform? (I.e. security engineering, compliance, risk, product security, security architecture, security operations and incident response, etc.)
  • Why are you looking for hire for this role or who am I replacing if I am hired?
  • How do you expect the business to perform over the next year?

Shrinking / Decline

It is an unfortunate reality that not all programs are in the building or existing states. Sometimes security programs shrink or slip into decline. This can be for a number of reasons such as poor leadership or a declining business. A shrinking security program can also be a temporary state that matches normal expansion / contraction of a mature business and the economy. Whatever the reason, leading a declining security program has significant challenges. First, the security leader will need to over communicate the existing risks to the business and make sure budget and headcount reductions match the reduction of risk as the business shrinks. A CSO can run into real trouble if the reductions are arbitrary and leave the business exposed.

Second, you can expect to have to do more with less. As the business contracts your team will still need to perform, but there may not be additional perks such as training, travel, new tooling, etc. You may also need to consider shrinking budgets and reductions in license counts or other tooling.

Another reason for a shrinking / declining security program is during mergers and acquisitions. Depending on how the deal is structured and the capabilities of the acquiring business, your security team may be redundant or parts of your team may no longer be needed.

A shrinking / declining security program isn’t the end of the world, but it does require careful leadership to make sure the risks are managed appropriately and morale doesn’t completely decline and impact the performance of the remaining team.

Not Everyone Is Good In All States

Not everyone will admit it, but the reality is not everyone is good in all states. This shouldn’t be surprising. Startup founders routinely find they can’t scale a company past a certain point and require additional help. Similarly, I have personally experienced that security programs require different leadership depending on the state of the program and the skills of the individual. Some people just can’t scale a program past the building phase and into the incremental phase. Some people don’t know how to handle decline. Leadership skills aside, some people just have a specific preference for what they like to do.

No matter where you are in your professional career or whatever state your security program is in, I hope this post will help you identify and navigate the type of security program you enjoy leading or are looking to lead one day.

Should Compensation Be Tied To Security Performance?

Lately, I’ve been thinking about how to incentivize security performance across an organization that struggles with the discipline for good security. When done correctly, security can actually help accelerate development lifecycles and is strongly tied to increased organization performance. However, for organizations that struggle, I wonder if they can reward good security behavior with some type of compensation?

CISO compensation is already tied to the security performance of the organization. The success of the security organization to deliver on security goals are already tied to annual KPIs or other performance metrics that tie back to how the CISO is reviewed and ultimately compensated. However, these goals become more risky and less achievable when the CISO is held accountable for security goals across the entire org. The reason for this is the CISO typically doesn’t own the products, systems, applications, etc. that run the underlying business. Instead, the CISO needs to manage the risk for these things and it may often be the case that the CISO or the business will need to make tradeoffs that could be sub-optimal. This could result in the CISO failing to achieve security goals across the org if the rest of the org isn’t held equally accountable.

In an ideal scenario, the rest of the C-Suite will also carry some sort of annual security goal(s) as part of their KPIs. This will effectively tie the performance and compensation of these leaders (CEO, CTO, CFO, CIO, etc.) to how well they deliver on the security goals that are set in agreement with the CISO. If the organization uses cascading goals or KPIs this means the entire org will have some part of their performance compensation tied to how well they execute their security objectives. I can guarantee an engineering team will never skip a security patch again if they are told they won’t get their annual bonus because they missed their annual security goal by shipping a product with a critical security vulnerability.

I also think organizations can gamify and incentivize compensation for security performance even further than just annual performance and compensation. Establishing an internal bug bounty program that rewards employees who find legitimate security issues or rewards teams who never deploy with a critical vulnerability can really accelerate a security program. The challenge for this is it costs money and needs to be balanced with normal business operations. However, I argue paying the people in your org to accelerate security performance will ultimately cost less than the cost of a security breach.

I personally would like to see an organization take security serious enough where they hold the other C-Suite executives accountable for security by tying their compensation to the security performance of their orgs. By bringing this issue to the forefront people will immediately see the real effects of security performance in their paychecks and they won’t be able to ignore the conversation any longer.

Here are the things I think should be part of an organization wide security performance program:

  • Meeting established security Service Level Objectives (SLOs) for patching
  • Meeting incident recovery or remediation SLOs
  • Deploying any type of infrastructure (OS, network, storage, etc.) without critical or high vulnerabilities
  • Deploying or shipping products and applications without critical or high vulnerabilities
  • Meeting SLOs for resolution of critical security findings from security researchers or external bug bounty programs
  • Resolving security risks discovered and documented during mergers and acquisitions within a set time frame (e.g. 1 year or less)
  • Requiring other C-Suite executives to carry a security performance goal for their organization that is tied to their compensation (same with their org)
  • Establishing and compensating employees via an internal bug bounty / security issue disclosure program
  • Closing security exceptions on time or before the due date
  • Achieving all security audit requirements (e.g. FedRAMP, SOC, ISO, etc.)
  • Having the entire organization go a set time frame without a phishing incident or BEC

This isn’t an exhaustive list, but I think you get the idea. Organizations should start structuring performance and compensation goals to help the security org and ultimately hold the rest of the business accountable for the security performance of the things they own. This can help remove the adversarial relationship that often develops between security and other groups and push security into the forefront of the decision making process for the rest of the business.

Vendors: If You Want To Reach CISOs Stop All The Noise

I had an interesting conversation with a bunch of CISO friends last week about the current problem with vendor communications and particularly how there is a high volume of noise in the industry right now. The current problem is predicated on the assumption that more volume of communication will lead to sales leads, which will lead to an actual deal. I’ve been both on the sales side and the corporate side and I can tell you that vendors view it as a numbers game. However, I’m here to tell you the numbers game isn’t helping you achieve your goals because you are annoying the very people you are trying to connect with.

Bad Behavior

Most CISOs and security professionals I know don’t start out having a contentious relationship with vendors, but once you achieve a specific level and title, the calls, texts and SPAM inevitably start. If you are a vendor, here are a few things guaranteed to get you blacklisted from the people you are trying to reach:

  • Claiming you were referred by someone else in the company (when you weren’t. Yes we check.)
  • Sending repeated SPAM (it is unsolicited for a reason and we have no obligation to respond so stop it.)
  • Calling our personal cell phones (at any time)
  • Reaching out to our boss and saying we aren’t responding to you (you’re kidding right?)
  • Sending unsolicited texts
  • Sending unsolicited gifts

I know the behavior above is referring to a small minority of vendors, but the problem is the volume of noise from these vendors drowns out any legitimate outreach from anyone else.

What Can CISOs Do To Cut Down On The Noise?

If you have a public profile on any social media platform it is inevitable vendors will find you. My suggestion is to use a custom email address for each platform so you know if the social media platform sells or leaks your information. You can do the same thing for conferences or if you fill in your email for webinars and other vendor activities. If you don’t have the ability to create custom email addresses you can also use throw away or temporary free email services. If the vendor won’t accept these temp email services then move on…trust me that white paper isn’t worth it.

Controlling your email is not the only way to cut down on the noise. If vendors, conferences or other activities require you to input your work address you can use the primary address field or secondary address field to list the name of the vendor. This way if they send you junk mail or gifts you know who it came from.

Aside from controlling your information, I know several CISOs that set specific rules for how to engage with them. These are typically rules like only going through a certain Value Added Reseller (VAR) and only coming to the office and presenting on certain days (like the last Friday of every month). These specific days are good ways for security groups and CISOs to learn about a lot of different technologies at once.

What Can Vendors Do To Reach CISOs?

The biggest thing vendors can do to reach CISOs is respect boundaries. Treat CISOs and their security teams the same way you want to be treated. Most importantly, no one wants to feel like a transaction. The security industry is small and if you truly want results, cultivate lasting relationships that can follow you wherever you go. If you abuse people’s time and attention you will get blacklisted. We all talk and word will get around quickly.

The most effective way for vendors to connect with CISOs is to go to vendor days or partner with VARs and get on their docket for innovation days where the VAR shows you off along with a list of other companies that are of interest to the CISO.

Wrapping Up

Being a CISO is stressful enough without getting harassed constantly by vendors. CISOs need to set rules for how vendors can engage with them so vendors know not to resort to bad behavior to get their attention. On the other hand, vendors need to respect boundaries and cultivate lasting relationships independent of the technology company they represent. If you resort to bad behavior you will be blacklisted and word will get around not to do business with you.

A CISO Primer On Navigating Build vs Buy Decisions

Every year CISOs propose and are allocated annual budgets to accomplish their goals for the upcoming year. Within these budgets are allocations for purchasing tooling or hiring new headcount. As part of this exercise CISOs and their respective security teams are asking: should we build this thing ourselves or should we just buy it? It may be tempting to simply buy a tool or to build it yourselves, but both options have advantages and disadvantages. Here are my thoughts on how CISOs should think about this classic business problem.

Strategic Considerations

The first question I ask myself and my team is – will building this thing ourselves become a strategic capability or differentiator for our business? If we build it can we use it ourselves and sell it to customers? Are we building a capability unique to the industry that could lead to patents or a competitive advantage? Most importantly, do we have the resources to develop, maintain and support this capability for the indefinite future? If the answer to these questions is yes, then you should consider building the capability yourself, but this also comes with a cost in terms of people resources.

Use of People resources

While building a capability can look attractive at first, it generally has long term costs that can easily add up to be more than the cost of just purchasing a tool or capability. This is because CISOs will need to staff engineers or developers to build the thing. This means they will need to hire (or borrow) resources who will need coding skills, database skills, AI/Big Data Skills and a bunch of other skills that aren’t typically key skills in a traditional security team.

Let’s say you will need to hire or borrow people to build your new thing. These people have salaries, benefits, bonuses, equipment costs, facilities costs and other expenses that can easily cost as much as (if not more than) the annual cost of purchasing a tool. Additionally, if you hired people, they can’t just move on once the thing is built. They will need to support it, maintain it, etc. If you borrowed resources then you will need to figure out who is going to handle ongoing operations and maintenance of the tool and you need to consider the opportunity cost of using these borrowed resources to build something for you instead of doing something else for the business that could have value.

The point is people aren’t cheap and they tend to be the most valuable resources for a business. Using these resources wisely and in a cost effective way is an important consideration for every CISO.

Financial allocation (CAPEX vs OPEX)

One other consideration for Build vs Buy is how your company allocates financial costs towards either CAPEX or OPEX. The reason this is something to consider is it may be easier to get OPEX budget than CAPEX (or vice versa). This can influence your decision to buy something over building it depending on how finance wants you to allocate the cost (or how easy it is to get budget in one of these buckets).

Time to deploy

Another consideration for Build vs Buy is – when do you need the capability and how long will it take to build it vs how long it will take to buy something and deploy it? If you need the capability immediately it may make sense to buy the tool and deploy it rather than trying to hire resources, onboard them, build the thing, support it, etc.

Integration Costs

Similarly, integration costs can be a huge factor towards whether the capability is truly effective or not. For example, if you can stand up the new thing relatively quickly, but it takes six months or a year to integrate it into your existing tools then that could throw your overall timeline off and may sway your decision towards building it yourselves instead.

Security Considerations of SaaS / Cloud Products

Lastly, and most important, CISOs need to think about the security considerations of buying a product vs building it in house. Software supply chain security is a top security risk for businesses and CISOs need to evaluate new tooling to see if they are adhering to the security requirements required by the CISO. If the product is a SaaS or Cloud Product then CISOs need to think about the risk of sending their data, source code or other information to a third party environment they don’t directly control. Similarly, if the CISO chooses to build the capability in house then they will need to make sure the team is making the new capability as secure as possible so the business and their customers aren’t exposed to unnecessary risk.

Wrapping Up

Choosing to build or buy a new capability isn’t an easy decision. Both decisions have explicit and hidden costs that can be difficult to navigate. Like any decision the CISO should weigh the risk of the decision and ultimately choose the option that supports the strategic direction of the business, meets financial and budgeting requirements and is sustainable by the security organization for the life of the capability.

Why Veterans Make Great Security Team Members

Every year the United States honors its fallen service members during Memorial Day. As a Navy Veteran, I spent this past memorial day reflecting on my time in service, the memories I’ve taken away and most importantly remembering the people I served with who made the ultimate sacrifice.

I also thought about the incredible number of people that work for me and with me who are veterans. In general, the veterans I have led, worked next to or served under tended to be the best employees, peers or leaders over the course of my career. Here is why I think veterans make great security team members.

Candor

Anyone who has served in the military or had a military family member knows people who have served tell it like it is. This is a carry over from giving and receiving orders in times of stress that need to be clear and concise. It is also a firm belief that life is too short and at some point you need to stop talking and take action.

Veterans aren’t afraid speak up in times of uncertainty because when we were in the military confusion could lead to loss of life. It is better to ask the question and be really clear than to keep quiet and risk disaster.

This candor is particularly important in a security team. Is there a weakness the business doesn’t know about? Are you seeing something anomalous that other people have dismissed? Do you have a new idea that could improve a process or reduce risk to the business? Veterans aren’t afraid to speak up when they have something to say.

Perseverence

No matter what branch of service you come from, all veteran’s made it through some level of training that was more difficult than the civilian life they left behind. Sleep deprivation, physical hardship and generally being uncomfortable are table stakes in the military. This means veterans are hardened against failure and generally hate to lose. They will persevere through difficult tasks and can be relied upon when things become chaotic and difficult. They also seek out training to better themselves and add new skills to their repertoire because they may come in handy in the future.

This perseverance is particularly useful in all aspects of security. Attempting to change a culture to a security first mindset requires incredible perseverance. Similarly, implementing new controls, resolving an incident or passing an audit also requires perseverance. I’ve found the veterans on my team take these events in stride and enter them with the confidence they will accomplish their task.

Perspective

Veterans also possess a unique perspective. This perspective comes from the hardship they endured during the military and carries over to civilian life. No matter how bad the situation gets every veteran thinks back to a time that was worse in the military and says “hey, this isn’t that bad!” Civilian life can be stressful and I’ve certainly had my share of burnout, breakdowns and disillusionment, but every time I think back to my time in the Navy and am thankful I’m not deployed away from my family, I’m not getting shot at and I’m not being asked to do things that could put me in harms way.

This perspective is useful during security incidents, but can also be useful during every day routine engagements with the rest of the business. Security isn’t always going to go perfectly and sometimes this perspective can help you see the big picture, keep calm and work towards a solution.

Willing To Take Risks

It shouldn’t be surprising that veterans are willing to take risks. Everyone who has served took a huge risk by leaving their civilian safety net behind. We deployed to dangerous parts of the world in order to protect our country. Additionally, veterans will tell you they served because of the camaraderie of the people who sat to their left and right. We are willing to take huge personal risk to protect our fellow service members.

This risk taking attitude is useful in the security space because it lets us try new things. We aren’t afraid to fail because we know we will learn from the experience and can try again. We are also willing to put ourselves out there if we know it will result in a better security posture or reduce risk to the business.

Security Mindset

I’ll generalize here, but I think veterans inherently possess a security mindset. We are evaluating strengths and weaknesses of attackers. We are looking at the physical security of spaces. We are considering if a control is good enough to manage the risk or if we need to push harder to secure something. Serving in the military means serving in an organization whose sole purpose is to ensure the security of the nation it protects. This mindset exists at all levels and is readily transferable to the civilian sector.

This shouldn’t be surprising since a large number of veterans often pursue a post military career in law enforcement, the government sector or private security. However, I also find tons of veterans in the IT sector and particularly in the security space. We have a common mentality and it is usually very easy to spot someone else who has served.

Wrapping Up

If you find yourself lucky enough to lead or work with veterans, like I do, then I encourage you to take some time to explore their background and what they did in the military. I’ve often found swapping stories with another veteran is a quick way to build rapport. Their candor, perseverance, perspective and security mindset can be huge assets to your security team and your business.

Centralized vs. De-Centralized Security Team?

Whether you are building a security team from scratch, expanding your team or re-allocating resources, you may be wondering what is more effective – a centralized or decentralized security team? Both have their pros and cons and I’ll discuss them and my experience with each in this blog post.

Centralized Security Team

This is probably the most common structure for a security team. In most organizations it makes sense to group all people doing the same thing into a single org. Sales people, IT, Finance, HR, etc. all get grouped into a single org with an executive leader at the top. For the security team it has some distinct advantages.

First, the CISO has direct control over the resources in their org. The reality is, whoever is responsible for the performance reviews and paycheck for the resource, is the one who actually controls that resource. This may sound obvious, but I have seen a lot of weird matrixed, resource sharing organization structures that quite frankly don’t work. There can only be one leader and centralizing the security resources under a single security org provides direct control of how those resources will be used.

Second, it provides a single point of contact or “front door concept” for the rest of the business. If there is an incident, security question, customer inquiry, etc. everyone knows who to reach out to and who the leader is for the security group. This can allow the CISO to more easily track metrics, measure risk and dynamically adjust priorities based on the needs of the business.

However, the downside of a centralized security organization is it often gives the impression that the rest of the business is absolved of their responsibility for security. I have heard the following from various parts of the rest of the business:

Why isn’t security doing that?

What is security doing if I have to do it?

What are you doing with all those resources?

A centralized security team can exacerbate the confusion about who is ultimately responsible and accountable for security within the organization. Or, the security team is held accountable for the security failings of the rest of the business even though they aren’t responsible for doing the things that will make the business more secure. These shortcomings can be overcome with a strong security first culture and when the CISO has strong relationships with the other business leaders in the org.

De-Centralized Security Team

A de-centralized security team can improve on some of the short comings of a centralized security team, but it also has disadvantages.

First, a de-centralized security team allows the business to place resources close to and often within the team that is actually responsible for doing the thing. Think about fixing software vulnerabilities. If the development team building the software product has security expertise on their team, that resource can help prioritize and even fix some of the issues as part of an embedded team member. They can raise the security performance of the whole team. This can be an efficient way to deploy resources on a limited budget.

A de-centralized security team can also spread the cost of security around the org in an equitable way. If each function is required to embed a few security resources then those resources (and headcount) are allocated to that business function.

The downside of a de-centralized team is loss of control. The CISO may still be held accountable for the security of the business, but they may not control the headcount budget for these embedded resources. If the CISO is able to hold onto the headcount budget, that is great, but it doesn’t prevent another issue – having the resources go native.

In my experience, de-centralized teams can often go native. This means the resource fails to prioritize the security asks of the team, fail to hold the team accountable or simply start doing non-security work when asked to do so by the rest of the team. If the CISO doesn’t control the headcount then this is effectively a lost (or non) security resource. Even if they do control the headcount, they may have to constantly battle and remind the embedded resources to prioritize security work. This is a particularly glaring problem when there is a weak security culture within the rest of the business.

What Should I Choose?

There really is no right answer here, but if I had to choose one over the other I would choose to centralize the security team and then spend a large amount of time with the rest of the org to articulate their responsibility for security. In an ideal world, that has a large enough headcount budget, I would choose both. Keep a core centralized team like incident response and GRC, but de-centralize application security engineers and architects within the teams that do development work. The structure of a centralized team and even a de-centralized team will be highly dependent on the needs of the business and who is ultimately responsible security.

However, the reality is your organization probably grew organically with the rest of the company and at some point you may be wondering if your organization structure is best to support the rest of the business. Shifting from centralized to de-centralized (or vice versa) is not impossible, but will require careful thought on how to deploy and control the resources so they can be effective. My suggestion is to start small, experiment and see what works for your org.

Leadership During An Incident

At some point in your CSO career you are going to have to deal with and lead through an incident. Here are some things I have found helpful.

Know Your Role

Unless you work at a very small company, I argue your role is not to be hands on keyboard during an incident. You shouldn’t be looking up hashes, checking logs, etc. Your role is to coordinate resources, focus efforts and cognitively offload your team from key decisions. You need to lead people during this chaotic event.

Declaring An Actual Incident

This may vary depending on company size and type, but in general the CSO should not be the one to declare a security incident. The CSO (and their representatives) can certainly advise and recommend, but declaring an incident carries legal, regulatory and business ramifications that should be made by a combination of the Chief Legal Counsel and some representation of C-Suite members (CEO, CTO, etc.). Once an incident is declared, your company will most likely need to disclose it on SEC forms and customers may need to be notified. All of this could impact your company’s reputation, stock price and customer goodwill.

Use A War Room

A war room is simply a place where everyone can gather for updates, questions, etc. It is a place that is dedicated to this function. If you are physically in the office, it is a dedicated conference room that has privacy from onlookers. If you have a virtual team it is a Zoom, Teams, WebEx, etc. that gets created and shared with people that need to know.

The CSO’s role in the war room is to keep the war room active and focused. Once the war room is created and the right people join, everyone should discuss what happened, what is impacted and what the course of action should be. Document this somewhere and pin it to the appropriate channels. If people join and start asking basic questions, send them away to read the existing documentation first. If people want to have a detailed technical discussion then send them to a breakout room. The point is to keep the main room clear for making decisions and directing resources.

Bridge The Gap

Your role during an incident is two fold – 1) Communicate to other leaders within the company about what happened so you can get the appropriate support to resolve the incident and 2) Direct the appropriate resources to focus on resolving the incident quickly, while following appropriate chain of evidence, legal requirements, customer notifications, etc.

Communicating To Executive Leadership and the Board
 

Keep it short and sweet so they can respond as needed. The purpose of this email is to inform them so they can give you the support you and your team need. Make sure to invoke legal privilege and keep the audience small (I discuss this in my post about Legal Privilege).

I use the following email template when communicating about an incident.

Subject: PRIVILEGED – Security Incident In [Product/Service X]

A security incident was detected at [Date / Time] in [product x] resulting in [data breach/ransomware/etc.] At this time the cause of the incident is suspected to be [x]. Customer impact is [low/medium/high/critical].

The security team and impacted product team are actively working to resolve the incident by [doing x]. This resolution is expected [at date / time x].

For any questions please reach out to me directly or join the war room [here].

Next update to this audience in [x time period].

Communicating To Responders
 

Your job here is to get the team any resources they need, offload them from decisions and then get out of their way. It is also important that you buffer them from any distractions and protect them from burnout by enforcing handoffs and reminding people to take breaks. It is easy for your team to get caught up in the excitement and sacrifice their personal well being. Learn to recognize the signs of fatigue and have resource contingency plans in place so you can shift resources as needed to keep the overall investigation and response on track.

Designate someone to help coordinate logistics like meeting times, capturing notes, etc. Capture action items, who owns the action item and when the next update or expected completion time will be.

Have A Backup Plan An Practice Using Them

Hope for the best and prepare for the worst. Can your incident response team still function if your messaging service is down? What if your paging program doesn’t work or you can’t stand up a virtual war room? Part of your incident response playbooks should include fallback plans for out of band communications in the event of a total disruption of productivity services at your company. Practice using these during table top exercises so everyone knows the protocols for when to fall back on them if needed.

Wrapping Up

Incidents are both exciting and stressful. It is up to the CSO to lead from the front and provide guidance to their team, executive leadership and the rest of the organization. CSO’s need to buffer their teams to allow them to focus on the task at hand, while protecting them from burnout. CSO’s also need to remember the conduct and response of the organization could be recalled in court some day so following appropriate evidence collection, notification guidelines and legal best practices are a must.

Do You Need A Degree To Work In Cyber?

In the timeless debate of What qualifications are needed to work in security? (or even the broader IT sector), I want to first start off by saying there are no hard rules. I am not going to gate keep people from the industry by stating you have to have a degree or specific certifications. On the contrary, I think anyone who is sufficiently motivated is welcome to pursue whatever career gives them personal satisfaction. I have seen plenty of individuals who are self taught, without a degree that are amazing. I have also seen plenty of people with degrees that are absolute garbage and so a degree is not a guarantee of quality or suitability for a role. That being said, if I had to choose between two equally qualified candidates in terms of years of experience, qualifications for the job and culture fit, I would choose the candidate with a degree every time and the rest of this post will explain why.

Follow Your Destiny

I want to start by re-iterating that a degree is NOT required to work in cyber or really anywhere in the information technology sector. With the right motivation, curiosity and ambition, anyone can achieve a meaningful career of their choice. There are plenty of online courses, books, certifications, local meetups and professional groups that can offer support to individuals seeking the right knowledge. I think this really comes down to financial opportunity and motivation. If you are unable to afford a four year degree program, are unwilling to take on student loans or are the type of individual that knows without a doubt they want a career in security, then a degree will simply delay you from your destiny.

Setting aside socio-economic, financial and other considerations, I do think degrees offer candidates a number of distinct advantages to individuals in the field of security.

Trade vs Profession

Some of the oldest jobs in the world have made distinctions between trades and professions. Trades like plumbing, electricians and general contracting can offer lifelong job prospects, but don’t offer a lot of flexibility to move between them without re-training. Trades also aren’t typically designing things, establishing standards or inspecting completed work. Contrast this with engineers who are designing the components, establishing standards, certifying designs and inspecting completed projects. The difference is an engineer requires a minimum standard of education to make sure the designs, plans and inspections aren’t going to cause loss of life. Simply put an electrician installs the circuit breaker, but an engineer designs it.

This can be true in the security industry as well. It is certainly easier to gain knowledge and grow in your security career without a degree, than it is in physical trades like plumbing. However, without a degree you are committing yourself to that specific field and assuming a certain amount of personal risk if that field declines or gets oversaturated with candidates. Having a degree offers the flexibility to switch careers or blend disciplines based on the company, economy or personal interest. A degree allows you to diversify your knowledge and specialization outside of your specific job and therefore offers advantages over non-degree holders.

Depth and Perspective

A standard four year college degree also provides depth of education. Degrees introduce students to topics of learning they most likely would never explore or discover on their own. Degrees also broaden perspectives by introducing students to new cultures via languages, travel or exchange programs. In my case, after performing horribly in math for my entire high school career, college helped me discover I was not only good at math, but excelled in a specific field of math called Operations Research.

Degrees also provide a standard of education that require students to master basic subjects like finance, public speaking, communication and writing. These skills are invaluable within the technology sector, which is typically dominated by a technical meritocracy at the expense of softer people skills. They are even more important within the management ranks to help explain and lead initiatives at all levels. It fundamentally doesn’t matter how technically proficient you are if you can’t communicate that knowledge and purpose to others in an effective way.

Perseverance and Commitment

Another benefit of a degree is it provides basic insight into the character of an individual. Degrees demonstrate several key traits that are important for a candidate. First, a college degree conveys an individual is able to take on a long term endeavor and complete it. It shows an ability to commit to and persevere when faced with a challenge. Second, a degree demonstrates willingness to learn and flexibility of mind. You are daring yourself to confront new ideas and grow stronger as a result. Third, a degree demonstrates a basic appetite for risk and a willingness to learn from failure. Students are launching themselves into unknown experiences and confronting failure on a daily basis in order to learn and grow as part of their degree program. Lastly, a degree demonstrates the ability to exist and function within a larger community. Existing, functioning and participating in a group setting is a basic life skill that is essential at all career levels.

Officer vs Enlisted

The military is a good example of why degrees are useful. A four year college degree is a minimum requirement to become an officer in the United States military. Officers have a breadth of knowledge along with some specialization in a specific field that provides an inherent advantage for leadership. General education skills like writing and communication are table stakes for military officers because they help explain mission purpose, gain support from senior leadership, develop tactical and strategic plans, or prioritize courses of action that can snatch victory from the jaws of defeat.

A degree affords the same advantages to management and leadership within the security industry as it does to the military. The ability to understand a variety of topics, think critically, communicate effectively and lead people to desired outcomes is increased when you have a college degree.

Final Thoughts

Degrees are NOT necessary to have a successful career in security. Choosing to pursue a degree should not be compulsory for any role in security and is a highly personalized choice. Information technology fields like security have demonstrated that the barrier between a trade and a profession can be torn down with the right motivation and support. However, I do think degrees provide distinct advantages particularly if you are interested in moving into management or simply becoming more effective in your career. A quality degree in any subject will teach you to think for yourself and demonstrate basic character traits that are valuable in any career field, particularly security.

Techniques For Influencing & Changing Security Culture

Throughout my career I’ve participated in varying degrees of organizational maturity with respect to security. This has involved moving from the datacenter to the cloud, moving between different cloud providers, moving to a ZeroTrust architecture, creating a security program from scratch and maturing existing security programs. During each of these experiences I learned valuable lessons on how to influence the organization to achieve my objectives and ultimately improve security. Below I share four different techniques that you can apply in your organization to get the buy in you need.

Jedi Mind Trick

First up is what I like to call the Jedi Mind Trick and this is one of the most effective techniques for shifting organizational culture. This is my go to technique for philosophically aligning major parts of the organization behind the scenes to get ground swell for an idea. Here’s how it works:

First, identify who the key decision maker is for what you are trying to achieve. Alternatively, you can identify people who are in key positions to block or impede the objective. Next, identify the people who influence these key stakeholders. This can be their direct reports, their peers or even their boss. Begin having regular conversations with these influencers about your idea, why it will benefit the business, how to achieve it, etc. The goal here is to get these people to philosophically align with your objective. Spend most of your energy with these influencers, but don’t neglect the key stakeholders. You still need to have conversations with the key stakeholders and discuss your idea, but you aren’t trying to convince them you are simply trying to make them familiar with the idea. At some point (it could be weeks or months) the key stakeholder(s) will begin to repeat your idea back to you and seek your opinion. All of your hard work has paid off because the influencers have finally done the hard work for you and convinced the key stakeholder(s) to pick up the torch for your objective. The key stakeholder will most likely think this is a unique idea or objective that they identified on their own. This is the moment you have been waiting for. Offer support, discuss what success looks like and then move on to your next objective, confident in the knowledge that your Jedi Mind Trick was successful!

Summary of Jedi Mind Trick Steps

  1. Identify key stakeholders
  2. Identify people who influence key stakeholders
  3. Spend majority of time philosophically aligning influencers. The influencers will do the hard work for you by convincing the key stakeholders
  4. Don’t neglect the key stakeholders. They need to be familiar with the idea, but you aren’t trying to convince them
  5. Once the key stakeholders begin parroting your idea back to you, the Jedi Mind Trick has been successful. Sit back and offer advice and support!

Switcheroo

Next up is a technique I like to call the switcheroo. This technique was actually discovered by one of my Lead Security Architects when we were trying to implement ZeroTrust. During this project we found a number of people who were resistant to the idea because their processes, roles and even self identity were anchored in the status quo. We found the switcheroo to be extremely effective in getting hold outs and naysayers to jump sides and support the objective. Here is how it works:

First, identify people with influence or in critical positions that can derail your project. This may take some time because it won’t be immediately apparent. People don’t usually just say no to something outright. They instead resist change through inaction or by countering your arguments. There is no easy formula for identifying these people. You need to have a strong network throughout the organization and approach your objective in your normal way. Eventually, conversations with stakeholders, influencers, etc. will identify these people as holdouts. Begin spending time with these hold outs to explain the why of your project, how it will benefit the business, etc. Give this person room to voice their opinions, counter arguments, etc. Eventually, it will become obvious that his person is entrenched in their way of thinking and it is now time to break them out of it. During your next meeting continue to explain the objectives, the why and how it will benefit the business, but this time when they voice their objections ask them this simple question:

“I understand your objections for why this won’t work, can you give me a few reasons why this will work?”

Sometimes all you need to do is shift someone’s perspective and I have found the switcheroo to be very effective in doing that. What ends up happening is the person actually convinces themselves for why something will work and in effect you use their own psychology against them. Next time you are up against a hold out that doesn’t want to get on board, try shifting their perspective with the switcheroo.

Summary of Switcheroo Steps

  1. Identify key stakeholders
  2. Identify key holdouts
  3. Spend time with key holdouts to explain the why behind your idea and allow them to express their objections
  4. After a few times listening to the objections of key holdouts, ask them to give a few reasons why your idea will work.

The Noise Breakthrough

The Noise Breakthrough is a similar technique to the Jedi Mind Trick, but it is more direct. The Noise Breakthrough is most useful when you have regular conversations with someone and are trying to convince them to support a particular objective. Regular interactions with key stakeholders across the business are essential for a CISO to be successful, but this can also have diminishing returns. This regular interaction makes it difficult for your stakeholders to parse signal from noise or, said another way, this means your stakeholders are unable to discern when you are saying something that is really important vs. the normal business as usual.

The inability to discern signal from noise isn’t a new phenomenon and it isn’t unique to the business world. Consider your parents growing up and how they would nag you to clean your room or do some other chore. Eventually, you learn to filter them out. The same with a spouse, partner or best friend who is regularly on your case about something. The constant feedback for the same thing has diminishing returns until eventually it won’t even register as something that is important. How can we break through the noise and get back to a signal?

Enter the Noise Breakthrough and here’s how it works. Let’s say you are trying to get the CTO to resolve a security problem, which has been an issue for several months. You’ve been discussing this with the CTO and they philosophically agree it needs to be fixed, but the problem remains. Like other influencing techniques you need to identify who are the key influencers for the CTO. This could be their lead architect, their chief of staff or one of their direct reports. Spend time with this person and get them to align with you. Then ask this person to spend time with the CTO to convince them to take action towards your objective. Sometimes someone just needs to hear something explained in a different way from a different person. Usually, this is enough to break through the noise and get your project back on track.

Summary of Noise Breakthrough Steps

  1. Identify key influencers for your stakeholder
  2. Spend time with influencer to get them aligned to your objective
  3. Ask key influencer to spend time with key stakeholder to help align them to your objective

Compliment Sandwich

The last technique I have found successful is the Compliment Sandwich. The Compliment Sandwich is most useful when you have to deliver constructive criticism or feedback when something is not going as planned. The compliment sandwich allows you to disarm the recipient by first paying them a compliment. The person is then primed to receive additional feedback and this is where you give them the constructive criticism. Finally, you end with something positive such as another compliment or a positive affirmation that the situation will get resolved. Let’s use an example to see how this works:

“Hey Alice, I really liked your lunch and learn last week. It was really informative. However, I couldn’t help noticing you didn’t ground the objective of the presentation in an industry standard control. As a result, your audience failed to grasp “why” your topic was important. It is important to explain “the why” and the priority of what you want people to do so they can prioritize accordingly. Next time let’s work on this together so your message is more impactful. This is a really important concept for your career development and I know you’ll master it after we work on it together.”

Summary of Compliment Sandwich Steps

  1. Give a compliment, praise or positive feedback
  2. Give constructive criticism
  3. End with a positive affirmation or positive statement

Wrapping Up

Security organizations often find themselves at the tip of the spear for technological and organizational change. As the CISO you need to apply different techniques to effect change so you can improve security and manage risk. The techniques above are simple and effective methods for winning over key stakeholders or breaking through barriers that are preventing you from achieving your security objectives.