Knowing how to build an appropriate security budget is foundational to helping an organization do security and manage risk.
We can't do the right security work if we aren't spending the time and money in the right places. We can't prioritize without thinking about money.
Many people who are running IT at mid-sized companies or CTO's at companies with little to no security team, and even seasoned CISO's are basically guessing when they build their budgets. The best at it balance the real risks by investing in the right places.
But what is the real risk?
And what are the real places?
I was once at a US history museum with a friend and our kids and he said the following to our kids:
If you want to understand what really happened, when in doubt follow the money.History Buff
Though we at Jemurai are strongly rooted in community, volunteering (OWASP), open source and optimistic approaches to security, we do believe that there is a truth to my friend’s expression: the budget tells a true story. Unfortunately, it's not always a good story.
This article is all about making sure you don't look bad when that story gets told.
One of the good things about thinking about budgets is that they don't lie.
If you funded a certain activity, there’s a strong possibility it was done.
If you didn't, it probably wasn't.
Budgets represent the ground truth around not just the size of a security program but its focus. For example, a glance at the budget can immediately show if you have invested in appropriate end-user training.
One of the most obvious red flags is that the budget is way too small or way too large for the organization. This points to the fact that the planning process wasn't legitimate. One reason this happens is that the person assembling the budget is bringing their knowledge from a past company into a new place.
We see people fail with their thinking about security budgets for a lot of reasons. A budget that hasn't been socialized with other executive leadership usually has little chance of getting properly funded.
Security budgets are often tool heavy. It isn't unusual to see obvious vendor direction in the budget discussion. We need to make sure we think about the budget and priorities for ourselves.
Sometimes an organization doesn't have a sense of risk. Other times teams don't get input from other people because ... well, it's money so not everyone is comfortable talking about it. The obvious problem with budgeting in a vacuum or without taking broader organizational awareness into account is that you won't be planning for the correct scenarios.
Simple Percentage Models
A common model for sizing security budgets is to say that the overall spend or size should be a function of the top line ... something like:
- 10% of IT Spend
- 3% of Revenue
- 1 security person per 100 engineers
- 5% more than last year's security budget
You can find some published metrics and suggestions to quantify your security budget. Deloitte Insights research on “Reshaping the Cybersecurity Landscape” benchmarked security budgets in a few different ways. The research focused on financial services and institutions, organizations that typically have high risk profiles. Deloitte quantified the average security budgets for these companies as follows:
- 0.48% of overall revenue
- 10.9% of all IT spending
- $2,691 per full-time employee
Accepting simple models like these, we're saying that if some budget calculation is good enough for XYZ, then it should be good enough for us.
These types of broad brush estimates are interesting but we see a few problems with them in reality.
- Usually they are so far off of historical measures that they don't resonate
- They feel arbitrary (because they are)
- It is very possible XYZ company didn't put much thought into their budget either
- Last year's security budget wasn't any good either
We like to incorporate this type of number as part of a discussion, but only as a reference.
As an engineering leader, I built plans to develop dozens of systems and then executed on those plans and watched how well the plans worked and whether the estimates were correct.
Deep dark secret: they didn't always work.
Top Down Estimates
A Top Down estimate is where we look at the big picture and all the moving parts, then we make educated guesses about how much work something will take at a macro level.
We might quickly allocate a week for setting up a system, or even a few days per page in the application we build, but we're not actually thinking about what the details behind the pages actually are when we do that. Meaning, we're looking at the work but without getting into the dangerous details.
In my experience, Top Down estimates generally work out to be smaller than how engineering projects play out in real life.
When you hear executives complain about projects being late, it is often because they are working off of the top down estimates, because looking only at the high level tasks means you don’t consider the underlying complexities.
A quick caveat: we often needed top down estimates to make certain types of planning decisions. We couldn't get the whole team together to do detailed estimates of every possible feature before figuring out what to actually go do. But when we did this rough planning, we had to stay vigilant with other parts of the organization so that they understood these estimates were not fully baked.
So in a security context, you might t-shirt size different initiatives and then map them out over several years based on priority and bandwidth.
Bottom Up Estimates
In Bottom Up estimation, we enumerate the likely fine grain tasks in great detail. Usually this needs to be done by the teams that are going to do the actual work, as they have sufficient understanding of what needs to be done.
You might think this would be the most accurate estimating technique, but I have found it tends to blows up the estimates. Why? Because every task estimate becomes a worst-case scenario, and there is no accounting for economies of scale or improved efficiency from doing the thing over and over.
So detailed Bottom Up estimates are typically much higher than Top Down estimates. However, we need this to balance the optimism and oversimplification of the Top Down estimate.
Now here, I'm not talking about security risks. I'm still in the engineering project planning mindset so I'm talking about risks to the project succeeding or being delivered on time.
We faced all kinds of risks. Attrition. Product roadmap changes. Technology changes. Poorly understood requirements. Areas where the approach was not yet well understood.
Based on the amount of risk, we would put what we called "contingencies" on certain line items in an estimate. The "contingency" would be higher. You could imagine that a line item in a plan might take 3 weeks (+/- a week) if we mostly understand it. It might take 3 weeks +/- 2 weeks if we don't know some part of the solution yet.
A great real-world example of a risk scenario that bridges security and engineering is integrating into a new identity provider. On some level, we know it can be done, it is just SAML, right? But on another level, we don't know just how complex the setup might be. More importantly, we don't know that we can control the 3rd party IDP provider or the users that need to be migrated!!!
Technical Debt and Budgets
Another concept that translates really well from the engineering planning world to the security planning world is technical debt.
I think of technical debt as being "deferred work that accumulates as we cut corners or take easier paths along the way."
In a programming project, it might be skipping unit tests. In a security project, it might be underfunding a part of the program, either intentionally or accidentallly.
If you drop into a new place and you are tasked with building a security budget, you really need to know what was done in the past! For example, was the cloud environment built with a "lift and shift" model? If so, you can bet that the cloud security posture is old fashioned, which means there is additional accumulated technical debt. You can't expect to reach a high bar of security from this position of deficit without spending more money to get there. Having a way to think about technical debt and bring it into the security discussion is a really good idea.
Bringing It All Together
Once I have numbers for a top down estimate, a bottom up estimate, I've weighed the risks and technical debt and I've matched that up against the simple percentage based approach, I can start to bring them together to formulate what the budget numbers and structure should look like.
Suppose the top down estimate was 3 months to complete and the bottom up estimate was 6 months to complete. We might end up pitching a 4 month timeframe with a 1 month + or -.
If there is a lot of technical debt or risk, then the estimate might change a lot - it might double or more!
The idea here is that you are taking several inputs into account and weighing them in a transparent way so that you can get to an informed decision.
Tools represent one of the biggest challenges with building budgets. A CISO can spend all of their time vetting vendors and doing POC's for different security tools. You can certainly spend all your money on tools. I've seen a lot of tools sit on the shelf too.
So a program is far more than having some appropriate tools. A program has to address RISK.
Therefore, a security budget should include appropriate tools. Their adoption should be prioritized and tied to real world risks. For every line item in your tool budget, you should know whether it is a "basic needs" type of tool or an advanced "ahead of the curve" tool.
You should know if you're buying "best in class" or "barely passing".
You should also know which tools lock you in over a long time period.
One general comment about tools is that some of the most important ones are consolidating with key platform vendors.
I'm cynical about tools for a variety of reasons. I see a lot of commercial tools that are building on open source solutions. I see tools that oversell what they can do. I see vendors talk about unified platforms that do things together. I see green checkboxes and storybook endings. It feels like the promise of most tools is far greater than the actual benefit. That is not to say there aren't tools out there that are great ideas that should be used more widely than they are. It's just really hard to know which are which!
In researching this topic, I asked some people what they think a good security budget should look like and the awesome Dan Cornell (@DanielCornell) said:
- Tools [50% of budget]
- Tools to make sense of the other tools [25% of budget]
- Consultants [25% of budget]
- Staff development and enablement [$0]Dan Cornell (@danielcornell) - Twitter
In case it isn't obvious, this was said with tongue in cheek but is funny because it's so often true.
So as a general guideline, my recommendation is to build your budget around people, risks and goals. A core set of tools will emerge but to make absolutely sure you are funding people to run and tune the tools.
Refresh Your Point of View
Wherever you start, it's not where you want to end.
To advance, we need to adapt and change. Sometimes, that means we need to rethink a whole approach to securing something. Consider the shift from prevent to detect?
Are firewalls a key technology for most startups? They're just included in the cloud! You don't need to even think about them. The same could be said for logging, intrusion detection, and backup technology.
Things we assume based on what was important at some formative part of our career inherently change and become more or less important, better or worse at their job, something we want to go out of our way to get or something we're willing to live without.
There are so many examples: Firewalls, Antivirus, Hashicorp Vault . Hunt team?
Make Risk Something You Can Talk About
As security leaders, our job is to manage risk. We need to take certain actions to reduce the likelihood that certain bad things happen.
If we eliminate a particular risk, but the likelihood of it happening was already very low, that is probably not the most effective use of resources. Instead, we should focus on the most important risks and build the program to address those.
In many cases, we have good data about our risks. We know about past fraud or phishing or ATO. We should use that data. If we account for those—not as isolated events or issues but as part of a suite of things the team is tasked to minimize—then we can mitigate proven risks.
A key exception: the risks where the likelihood is low but the impact is very, very high. Existential threats to the company may not play out often, but they still need to be prevented.
Just remember, there is rarely just one way to mitigate a risk, and a given tool is almost never a silver bullet!!!
Perhaps the most important thing is to socialize the idea of risk management so that both executive sponsors and teams have a shared working understanding of the purpose of the program.
No matter how you come to your budget numbers today, you’ll find it a challenge because what you are recommending is going to be different from past budgets—usually with a sharp increase.
I can't tell you how many times I've had the conversation with consulting companies about adding an option to their proposals that includes "enhanced security" and the response is always:
But if we make them pay more for security, what will they think about what we have done in the past? What if they realize we weren't giving them the "enhanced security" option before?Common Client Statement
That is one reason course corrections around the scope of a security program for people in an existing role can be really hard. You have to be able to convince your funders (eg. your CEO) that the change is worth it.
It can be helpful to realize that the whole landscape has changed continuously, and it isn't common that organizations have kept up. In other words, I don't think you should feel guilty about saying that we've learned and we're going to improve.
But that only works when you have a reasoned position that you can stand by under inspection.
IT Versus Security Budgets
Something that complicates security budgeting is that a lot of things security wants to do bleed into other areas of the organization, most often IT. For example, I would expect IT to have a plan for how to patch devices. That is partly a security consideration, but it is part of IT's responsibility. This is also true for classic things like having antivirus / EDR and having secure ways to send email and share files.
Other parts of the security budget tie to development - for example using appropriate services in cloud infrastructure. Consider that things like turning on file system encryption and logging are not often baked into a base cloud solution without security guidance. But the costs show up in regular cloud billing.
So part of figuring out how to do security budgets is, and I mean this is a constructive way, figuring out how and when to spend security budget in classic business functions.
My general thought is that security should help make best practices and figure out how to do those things securely but that if IT or DevOps is running the tools, it should come from their budget.
You can account for that with feedback loops and in some cases it can be helpful to use a special reserved security budget to take care of something (like say Identity with a common vendor and SSO process) with security so that it is effective across other teams. On the other hand, in some cases this is a recipe for failure because the teams that didn't pay for the solution aren't as committed to getting it done!
Ultimately, like a lot of other things, it comes down to context and risk calculations. If the single best thing I can do to reduce the company's risk is to implement better phishing training and IT and HR want to run this, but need security to fund it, it could very easily be the best thing to fund it.
Without wanting to be political, it is important to follow through here though. If you funded some IT project for security reasons, you have a right to updates and status so that future investments can reflect the learning from the earlier one.
Security is in a tricky spot in this sense, because we can't just delegate risk to IT or DevOps or other departments. We need to learn the details about what is being done to really manage risk.
Conflict and Compromise
It is inevitable that the budget won't get approved exactly the way you want it.
An important part of providing leadership is to talk about this dispassionately. When there are disagreements, we must be able to talk about risk in a real way and be ready to compromise to work through getting the most important things.
That being said, sometimes there are situations where you just won't get a realistic budget and you may not have any chance of success. In that kind of case, it can be helpful to the organization and yourself to discuss the prospect of going different ways.
Similarly, if you are taking on a security role where you have responsibility for the budget, you should ask about the budget before you take the job so that you know whether it is a realistic fit.
The following pulls all of this together in a relatively simplistic but hopefully illustrative model.
The model depends on a number of factors:
- Cloud spend
- Number of engineers
- Number of employees
- Whether audits are planned or required
- Whether the company is regulated
- Whether security is a priority
We have a calculator that lets you plug different values into these variables and it estimates what your security budget should be. You can check it out at the end.
We think it makes sense for a minimum of something like 5% of cloud spend to go to security. That means turning on cloud security features you already have access to. This is totally arbitrary but aligns to trends we see coming with things like Google's Security Command Center.
Number of Engineers
Typically engineers are a good indicator of effort building your own technology. That can be a reason to add additional training, have staff to support the engineering efforts and tools integrated into your SDLC.
Number of Employees
The number of employees is a good proxy for revenue, stability and target maturity. As firms get over 100 people, they start to think about building out their own security teams. When there are more than 1,000 the needs change. 10,000 they change further. There are costs associated with each individual person, but we're not really trying to measure those so much as use this as an indication of maturity that would trigger other activities or the scope of other activities.
Some companies don't have revenue yet. That doesn't mean that they shouldn't do security, but it certainly changes the calculus about what makes sense. Other companies have large revenue figures but fewer employees or products. In those cases, revenue is a good signal that something more needs to be done. Sometimes higher revenue is associated with higher risk. We need to identify that risk even if the other measures are more moderated.
Some activities need to be done on every product. A startup might have one product, while a large company might have dozens. An example is products that should be pen tested. That is an incremental cost per major product.
If a company is planning to do a SOC 2 audit, or an ISO 27001 audit or FedRamp or HITRUST for that matter, there are significant financial implications. That includes the cost of the audit itself of course, but also includes the cost of preparation, evidence gathering and usually some general overhead to adhere to operational processes that will be the subject of the audit.
For companies that handle PHI or other types of regulated data, security will be a question with partners, investors and customers. Often companies can self identify that they are regulated but they can't necessarily tell you what they need to do from a security perspective.
This sounds idealistic, but some companies truly do spend more money on security than they necessarily have to because it is a priority. It is part of the culture. Part of the identity of the founders or executive team or something like that.
This leads to more proactive and thorough security programs, which of course, cost more money to implement. We applaud organizations that are able to do this - and we aspire to be one ourselves - but not everyone can do this.
If you are in blockchain, security should be a priority.
If you are healthcare tech or legal tech or fin tech, security should be a priority. If you are a security company, security should be a priority.
But if you are publishing a newsletter or a website to support a consulting business, you may not need to worry so much about security. Famous last words, I know (*yes, update wordpress!), but really the investment has to be commensurate with the risk and the situation and it doesn't necessarily make sense for security to be a top priority for every company.
The estimator pulls these different factors together, assigns weights to them, drops common security costs into an algorithm and spits out a suggested security budget.
We tested the estimator on companies we have information on to see if it matched reality and so far, it seems to fit. We will keep updating the model and the estimator as time passes and we learn more about different budget considerations.
You can try the estimator here: https://jemurai.github.io/security-budget-estimator-ui/.
Our Building a Security Program book includes a chapter around building security budgets. In addition to the content here, we talk about some more specific line items we typically see in budgets.
Your security budget offers an unflinching view into your investment in security. If you don't get it close to right, you're probably not doing a great job. Of course, there is no universal right answer, so use a variety of approaches and blend them to get to something that you can feel good about.