When specifying cybersecurity testing for your organisation you’ll come across various approaches. Penetration testing and bug bounty programmes are two likely options.
But is this an either-or situation? We highlight the main differences between bug bounty and penetration testing and explain why they actually complement each other, keeping your organisation as safe as possible, 365 days a year.
Differences in starting knowledge
Bug bounty: With no prior knowledge about your organisation, this approach can be more realistic.
Can you find vulnerabilities by simply looking from the outside in? This is where a cyber attacker starts. They have access to the internet and all the time in the world to look, watch and learn before attacking.
In fact, bug bounty programmes are ideal for public-facing systems such as websites and customer portals.Penetration testing: With a fixed timeframe, you usually share knowledge with us to achieve the most during the programme. Targeting testing on specific systems, pre-testing knowledge such as code and configuration reviews help to focus on the right issues quickly.
But this method is not as realistic as a live attack. We generally recommend penetration testing for internal systems, those under development, or for situations where point in time assurance is required.
Differences in testing timeframe
Bug bounty: Set to run over a longer period – typically months or years – bug bounty programmes are a continuous testing tool. Organisations typically run a penetration test annually, yet new vulnerabilities can emerge the day after they’re completed. This might be from an internal action, or externally for off-the-shelf software when vendors complete an update. The timeframe is not an issue for bug bounties. You can run your programme for as long as you want.
Penetration testing: Usually structured to run for one to two weeks, penetration tests always have a fixed and limited timeframe. The difference is you’ll know exactly when this is and exactly what’s being tested.
A specifically scoped and timed penetration test can also be useful when you’ve made significant changes to a system. It’s a short and intense method with specific objectives.
Differences in depth or breadth
Depth and breadth are both important for cybersecurity testing. Yet, it’s hard to achieve both using a single testing method.
Bug bounty: This approach delivers deep technical testing. With no fixed timeframe a bounty hunter can spend days, even weeks, delving into the same rabbit hole. They can tease out a vulnerability that could otherwise lie hidden for years to come, always leaving you at risk from real attackers.
Bug bounty programmes help keep you on the front foot. Given you can only control what you know exists, bounty hunters can discover unknown vulnerabilities due to poor employee practice or dark web breaches, for example.
Penetration testing: Delivering a pre-determined balance of breadth and depth within a fixed period, penetration tests will only get so deep. It would be irresponsible to excessively test in one area to the detriment of another. So, using this testing method alone could leave you at risk of attack from deeply hidden vulnerabilities yet to be uncovered.
Differences in pool of expertise
Bug bounty: We open your bug bounty programme to a large pool of background-checked bug hunters. They’re drawn from various offensive security teams across our business. Bringing diverse knowledge and capabilities, each will have different approaches to your testing.
This fresh thinking ensures a realistic ‘outside world’ approach. Whilst your system security is paramount, they challenge it in ways no other testing method can.
Penetration testing: When you hire us for your penetration testing, we handpick one or two specialists with ample knowledge and experience to suit your scope.
With full access to your specialists during your programme, they’ll exclusively review and test your system using their deep insight. Once complete, they’ll report to you and provide clear guidance on the actions you should take to reduce any risk.
Differences in reporting
Bug bounty: This method offers a huge benefit – real-time reporting. With direct access to our platform 24/7, you can learn about each vulnerability as soon as we do. In fact, you can set up alerts – by text message or email – at a frequency that suits you.
With secure direct access to your diagnostics, drill into the detail and directly question the hunter who discovered your vulnerability.
Penetration testing: At the end of your fixed testing period, we’ll deliver a full report of findings. You can discuss them with your specialists and receive guidance on the next steps to take. We’ll support you as much as we can.
Differences in testing costs
Bug bounty: You only pay for outcomes. That’s the main principle behind this testing method (using a budget ceiling you control). You’ll have a fee for each vulnerability found, based on its severity.
We transparently use CVSS version 3 (Common Vulnerability Scoring System). A critical vulnerability would risk catastrophic damage to your business and would therefore have a higher finding fee.
Importantly, the fee also covers our advice on resolving the vulnerability, answering unlimited questions, and unlimited retesting. We’re with you until the bug is fixed.
Penetration testing: In contrast, penetration tests have a fixed project fee based on our time and materials used. Regardless of what our testing uncovers – many or few vulnerabilities - the fee remains the same. And yet, there’s value for you in both outcomes.
Should I use bug bounty or penetration testing?
There are clearly large differences between bug bounty and pentesting. Yet, they both have a complementary role when maintaining your cybersecurity. To use one without the other leaves you open to risk from attackers.
Your ongoing security is greater when you combine both approaches.
Many organisations welcome blending the two different costing approaches too. Incorporating bug bounties to run between penetration tests only adds to your budget on the discovery of important vulnerabilities. It’s easy to quantify.
In this way, you enjoy the best of both worlds and heighten your defence against real-life cyber attackers and the damage that ensues.