editione1.0.0
Updated October 9, 2023🚀 As explained by Laura
While almost all organizations rely on software in some way or another to get the job done, not all organizations need to build their own software. If you do, though, this chapter is for you.
This includes companies that sell their software (perhaps as a SaaS model) or those companies that need to use custom software internally but do not sell this software as part of their product or service offering (for example, a service organization that has built custom software to manage their scheduling, workflows, and billing). Some organizations build their own software, while others pay external companies to build software on their behalf. Whichever approach your company has taken, security needs to be front of mind throughout the process.
Throughout this chapter, we’ll take a high-level look at the factors you need to consider when building software and how to ensure security is part of your overall software development process. Specifically:
The risks involved with building software, whether that be internal or using an external software development partner.
How to validate or verify a potential software development partner is taking a mature approach to application security.
How to work with specialist security assurance or consulting practices to validate and assess your applications and approaches.
Note that this chapter is aimed at someone in a more operational or managerial level, rather than a deeply technical person on your engineering team.
Deciding whether you should build a team internally to develop your software or work with an external company to build the software on your behalf is complex and involves consideration of much more than just security—including cost, market, complexity, and time requirements. For the purposes of this chapter, we will focus on the security elements, including reviewing what we are protecting in this decision, the risks that you will face, and your options in each scenario.
For most organizations, there are two primary security concerns when building software:
protection of your intellectual property (IP)
protection of the data stored within your systems once they launch.
Software companies invest in their technology to produce a unique solution to a problem. This solution, including the software, algorithms, or functions involved, represent IP for the company and this is counted as a significant and valuable asset. When valuing an organization, protection of IP is often high on the list of factors considered. If a company is failing to protect its IP, its value often significantly suffers and the company may lose defensibility.
There is a lot of jargon going in there, so to unpack it and put it in more plain language: if you don’t protect the software that makes your product special, you may lose the ability to compete and your company will become less valuable.
Choosing how your software is built (and, specifically, where) is a significant part of protecting this IP and your competitive advantage. When we outsource our software development, we are placing our trust in this external organization to protect our IP. This can be challenging and requires planning and process to get right.
If developing software internally, you can protect your IP by ensuring the integrity of your team and setting up controls to protect the systems they work on. If using an outsourced development team, you don’t have that kind of control.
Let’s first look at steps you can take to secure an internal development team.
The following security measures should be considered when building an internal software development team:
Hire the right people. Perform background checks and reference checks during recruitment processes to ensure that team members meet the expected standards of behavior and have no prior history of criminal or high-risk behaviors.
Principle of least privilege. Use the principle of least privilege to ensure that access to code bases, designs, or other sources of intellectual property are restricted to only those roles with need and only for the period that need exists.
Monitor your environment. Review access logs for sensitive locations frequently to ensure that access patterns are as expected and any anomalies can be identified and responded to quickly.
Store your IP appropriately. Use appropriate tools for the storage of sensitive information, such as implementing version control for code bases. Use file stores for non-code items, including designs, plans, recipes, and formulas.
Protect the devices you use. Control and protect end-user computing devices, such as laptops, to ensure they are free from malicious software, unauthorized access, or other mechanisms for harvesting and exfiltrating data.
Control the usage of production data. Your production data (the data of real customers using your system) is very valuable and often contains commercially sensitive data or personally identifiable information. While it can be tempting to use copies of this data in your development and test environments, this poses a high risk of data loss and should be avoided. Where real data must be used, it should be anonymized or sanitized such that customers are protected.
Lock down your accounts. Build strong foundation practices, such as enforcing good-quality passwords and multi-factor authentication.
Weave security throughout your software development life cycle. Use guidance such as that produced by OWASP and NIST to find steps to weave security throughout your software development process.
Now let’s look at steps you should take if you’re using an outsourced development team.
The following security measures should be considered when using an external software development team or agency to build software on your behalf:
Retain control of your source code. A scarily high number of organizations that use outsourced software providers never store a copy of the software they have commissioned, entrusting the development, deployment, and maintenance of this code entirely with the software vendor. This creates a number of risks including but not limited to:
What happens if your software vendor shuts down?
What happens if the vendor is acquired or merges with another organization?
What if you choose to change software development companies?
What if your software vendor suffers a breach or incident that compromises or destroys their systems?
Do your research and check references. Good marketing doesn’t mean good security. Remember that just like any significant investment your company makes, you need to ensure that your software development partner shares your expectations around security and that you can validate this behavior with current or former customers. Choosing vendors with internationally recognized security accreditation such as SOC2 can make this easier.
Verify your vendor’s environments. In security, we often say “trust, but verify.” When choosing a vendor, listen to their promises and ask them about their practices—but retain the right to audit them in your contract. Use that right to conduct security reviews if you feel it’s justified. Often you would employ a specialist security assurance company to conduct this review or similar assurance activities like penetration testing. Alternatively, if you choose to review this yourself, you may wish to use something like Google’s Vendor Application Security questionnaires (VASQ), which provide a range of open-source questionnaires to assess the security of your vendor, their security program, and any associated applications.
Provide suitable environments. If you cannot assess your software partner’s development environment, provide the environment yourself. This choice often removes risks from development teams sharing their development spaces between multiple partners or customers. However, it can also increase the chance of IP reuse or leakage across customers without your knowledge. That risk is just lower than using an environment you aren’t able to vet.
Explicitly manage IP in all contracts. Your contracts and agreements with software development companies and vendors should explicitly outline the ownership, storage, sharing, and controls around intellectual property. IP already built by your software partner may need specific exemptions to ensure that the line between what your organization owns and what your provider retains is clear.
Have a plan for incidents. If there is a security incident in your software partner, you will want to know about it and any repercussions for your company. Make an incident response playbook for this scenario and collaborate with your partner to ensure it will work. Conducting a test of this playbook involving both companies is highly recommended. You can read more about this in Part IV.
In these early stages, whether you build your own software or engage others to build it on your behalf, there are some steps you can take to make sure you understand the risks in your software and weave security through the software development process. The key is to consistently take these steps even as you grow.
To make this consistency easier, let’s look at a few final tips we’ve gathered along the years to help keep your software, its users, and your data safe, including how and when you should get help and what to look out for when working with security specialists.
Most early-stage companies don’t have a dedicated security person, let alone someone who specializes in application or software security. It’s common for security to be part of another role or a shared responsibility in those first years, and while that’s not the end of the world, it doesn’t necessarily mean your team has the right specialist skills to help you secure your applications.
In this section, we will take a look at the sorts of external help you can use to support your software development lifecycle and how to get the most out of this process.
| Service | Aim | Typical Outcomes |
|---|---|---|
| Design and Architecture Reviews | Review the design or proposed architecture of your software before it is built or before significant changes are made. | Identifies potential vulnerabilities before the software is built to allow you to plan design changes or monitoring approaches. |
| Vulnerability Assessment | Use automated tools to frequently review your built and deployed software to identify “low-hanging fruit,” or common, simple-to-exploit vulnerabilities. | A list of potential vulnerabilities in your software that can be investigated and addressed. |
| Penetration Testing | The use of a specialist training team or professional to simulate the process taken by an attacker and identify vulnerabilities in your application. | A report documenting specific, confirmed vulnerabilities identified in your software, how they were found and recommendations for their remediation. |
| Bug Bounty Programs | The provision of a managed program for security researchers. This program will incentivize researchers to investigate and find vulnerabilities in your software in return for cash or other rewards. | Documented vulnerability submissions from a global community of security researchers. |
| Development Lifecycle Consultancy | Reviewing your software development process to identify changes or additions that can be made to increase the presence of security and increase the likelihood that vulnerabilities are identified before release. | Reports or findings that document proposed improvements to your software development lifecycle. In some cases, engineers may be available to implement the suggested changes alongside your team. |
Much has been written about each of these service types and their advantages and disadvantages. Use the above guide as a starting point and work with external security assurance companies or consultants to explore how their offerings work and their proposed costs and benefits.
When engaging with an external security service provider, remember you need to shop around and make sure the provider is the right fit for your team, maturity, and needs.
Here are some questions you may want to ask before you engage:
What services do they offer and how do they differ from each other?
What standards and frameworks do they follow?
Do they have reference clients you can speak to who are in a similar position, maturity level, sector, or size?
How much do their services cost and how long will they take (in terms of days of effort)?
Does their assessment include the ability to have your improvements or remediation efforts checked (often known as remediation testing)?
Where is the team located? Do they do the work themselves or do they outsource?
What will your organization need to have in place before the work starts?
Can you see a sample or anonymized report from a previous engagement of this type?
Shop around, there are many providers and each has a different style. There are hundreds of different companies providing security services ranging from big name consultancies to boutique specialists. Shop around and find an organization that understands your company’s age and stage, and whose communications and culture compliments your own. Disconnected experiences and culture will result in findings and recommendations that won’t work for your context, making them hard to implement or ineffective.
Don’t buy based on price alone. Like any service industry, there are a range of prices to choose from. While you may be budget conscious, don’t choose based purely on price. As many of our parents once taught us, sometimes you get what you pay for. If choosing based on price, it is doubly important you check their references thoroughly before engaging.
Ask them to integrate with your workflows. Most service providers will give you a beautifully formatted PDF document of your findings, which will appeal to boards and auditors. However if your team is more Jira than Adobe Acrobat, you might waste a lot of time importing findings between systems. Ask for your results in formats that can be easily uploaded into your tool chains, such as CSV formats. If the provider doesn’t understand why this would matter, see point 1 above, they probably aren’t a good cultural fit.
Be open and forthcoming. Remember that most engagements are time-bound and if your system is complex, that can be a lot to cover. Giving your service provider a guide to sensitive areas or areas of concern can help you chase down high-value issues and focus the effort. The same can be said for sharing vulnerabilities you already know about; don’t pay someone to confirm what you already know, let them know in advance and help them explore other areas.
Have a documented contract and ensure they are insured. I know, you already have too much to do, and drawing up and reviewing contracts feels like more effort. In this case however, it is worth it. Contracts with external security providers document who is responsible for vulnerabilities if they are missed in testing and where liability is covered by insurance. Take the time, check the contract, and check they are registered and insured with appropriate levels of professional and technical insurances.
Check their references. Get on the phone, do a video chat, or go for a coffee. Ask about their experiences (good and bad), whether they felt they had good value, and whether they would recommend the service again. Where possible, also double-check in your technical communities for any impartial recommendations that have not come from your vendor themselves.
Rotate providers regularly to keep things fresh. Sadly, external security services are often something we need on a regular basis. Whether that’s because our platform is evolving or because we are covered by a compliance scheme with a frequent audit requirement, this is not a one-off affair. While it’s great to have a trusted provider, remember that reviewing and testing things you are overly familiar with is very hard. Get fresh eyes on your company and systems by rotating providers at regular intervals to avoid over familiarity.
Remember than an empty report/no findings is not necessarily a good thing. Sorry, but if they don’t find anything, this is cause to be cautious and skeptical rather than proud and excited. Don’t hope for an empty report—hope for a short report with a few very context-specific bugs that your external specialist had to work hard to find. That’s the sign of a good relationship and a high-quality engagement.
Ask questions from your provider and encourage them to be coaches. There is no magic in security. Nothing we do is “secret sauce” or “too sensitive to share.” Find a partner who is transparent and shares their approaches. This gives you confidence in their frameworks and standards, and offers your team the chance to learn from the engagement.
With security, as with most things, once our software has been delivered and we are happily serving our customers, our job has only just begun. Security is important for the life of the application or system, which (we hope) is for many years to come.
All internet-exposed systems are subject to security activity, and it’s important that you spend some time thinking about how you and your team will identify when such activities are taking place. The sooner you know, the sooner you and your team can respond and protect your systems and data from harm.
While we won’t cover how to set up monitoring and alerting systems in this book, we will give you some idea of the things to consider when doing this crucial work.
confusion Remember, these guidelines apply whether your team is responsible for monitoring and maintaining your software, or you have outsourced this to a support team. These requirements are either intended for you to follow or for you to communicate to your supplier as part of your expectations on them.
Monitoring is all about anomaly detection and correlation. The more data and events you log, the more you can correlate and use for anomaly detection. You can’t get back data if you didn’t log in the event of bad things happening.
Store logs away from your software and make them immutable. If there is an issue with your system, you want your logs to be safe from harm and stored away from the system they relate to. Keep them safe, keep them for a long time, and don’t let anyone edit them.
A log you don’t check can’t help you. It’s all very well to log everything and implement monitoring systems but if you and your team never check them, they are doing you no good. Make it easy to check your logs.
Embrace interrupt-driven response. Configure your monitoring system to alert you when unusual situations occur. This alert will draw your attention and prompt you to investigate. If you are hosting your logs in a cloud provider, check out their platform offerings in this space, as they may provide anomaly detection as part of their service.
Centralize your alerts. Many of us are guilty of filing emails and alerts into a folder and never checking them. Bring your alerts to a central location that your team can monitor together. This reduces the risk of missing something important.
Have a plan for when things go wrong. If you spot something suspicious or your team sees an alert that’s alarming, they should all know what to do. Define a plan for these situations and practice it. The more prepared you are for this sort of event, the easier it will be to get your business back on track if something goes wrong.
resourcesSecure software development is a big topic, and impossible to cover fully in one chapter. These resources are aimed at a technical audience, and will most likely suit people in engineering leadership roles:
Microsoft Secure Development Lifecycle, an open guide to Microsoft’s approach to secure development, featuring well-documented guidance on processes such as threat modeling.
Agile Application Security by Bell, Smith, Bird, and Brunton-Spall, a complete guide to weaving security through fast-paced software development life cycles.
The Open Web Application Security Project (OWASP), home of the de facto standard guide to common vulnerabilities, the OWASP Top 10, and hundreds of other resources including tools and frameworks. Looking to get started? Check out the Software Assurance Maturity Model (SAMM), the Application Security Verification Standard (ASVS), or any of their free-to-download ebooks.
🚀 As explained by Laura
We build our companies, including our own software, on top of other software. That makes sense—nobody has the time or resources to build every component of their business from scratch. That would be a real waste and would stop us from focusing on what matters most. As a result, our software is much like Russian dolls: bigger things containing much smaller things underneath it all.