What a good external security assessment gives you
What an external assessment covers, what it does not, and what should be in the report.
If you are buying a security assessment, here is what to expect, and what to ask for.
Written authorization comes first
Testing a system you do not own, without permission, is a crime in most countries. Examples: the Computer Fraud and Abuse Act in the US, the Computer Misuse Act 1990 in the UK, section 252 of the Crimes Act 1961 in New Zealand. A serious tester will not start before you sign a document that names the systems, the dates and the rules. If the system is hosted by a third party, they may need to approve too. Be wary of anyone who will test “anything” without paperwork.
Scope is a list
The scope says which domains, apps and APIs are in, which are out, and what is off limits (for example denial of service, or real customer data). Systems found along the way are reported, not tested, unless the scope allows it.
What “external” means
The tester looks from the internet, as an attacker would. That covers what is exposed, the web app and API, and cloud settings. It does not say anything about your internal network or your laptops. A good report says what it did not cover.
What you should get back
- A summary a non-technical person can read.
- Each finding with: what happened, what should have happened, impact, steps to reproduce, evidence, how to fix it.
- A severity and a confidence for each. “Confirmed” and “suspected” are different things.
- A list of what was tested and what was skipped, with the reason.
- A list of test accounts and data created so you can remove them.
A list of scanner output with no explanation is not a report.
Fix and retest
Your engineers can fix the findings, or the tester can. Either way, the tester should retest the original steps after the fix and say, for each finding, fixed, not fixed, or could not verify. Ask whether retest is included.
No findings is not safe
A clean report means the tester did not find issues in the areas they checked in the time they had. It does not prove there are none. That is why coverage is listed.
Questions to ask a tester
- What exactly is in the scope and what is not?
- Do I get a written authorization to sign?
- Will you tell me straight away about something serious?
- What does the report contain? Can I see a sample?
- Is retest included?
- How long do you keep my data?