Black Box vs White Box Penetration Test: How Much Should the Tester Know?

What is the difference?

The difference is information. In a black box test the testers get a URL, an IP range or a device in a box, and nothing else: no accounts, no documentation, no source code. They have to find out what the target is and how it works before they can attack it, the way an outsider would. In a white box test they get everything the development team has: source code, architecture and threat model, firmware images, debug access, credentials for every role, and a developer who answers questions. Grey box sits in between and is the most common form in practice: accounts and documentation, but no code.

What changes with the information is not whether the tester can find a bug, but how many bugs fit into the booked days. Every hour spent guessing an API route, brute-forcing a role or dumping firmware from a flash chip is an hour not spent on the authorisation logic, the crypto or the parser where the serious findings live. Black box measures what an outsider sees in a given time. White box measures what is actually there.

Side by side

	Black box	White box
What it finds	Exposed services, unauthenticated flaws, information leaks, weak defaults, anything reachable without knowledge	Logic and authorisation flaws across roles, injection in custom code, crypto misuse, unsafe parsers in firmware, secure-boot gaps, hard-coded secrets, design errors
What it misses	Anything behind a login it cannot pass, anything only reachable with knowledge of the design, most firmware internals; coverage is unknown	Little within the scope; the risk is that testers follow the documentation instead of the running system, so good white box tests still include black box style probing
Who does it	Testers working from the outside, often with little contact to the team	Testers and the development team together, with a kick-off and questions during the test
Duration	Longer for the same coverage, because discovery eats days; a typical booking ends before coverage is complete	Shorter for the same coverage, or more coverage in the same time; one to three weeks for a product
Typical cost	Same day rates; 6,000 to 15,000 euros for a web application, typical ranges for Germany and the EU; you pay for discovery	Same day rates, more of them spent on findings; firmware and code review can add days: 15,000 to 30,000 euros for an IoT product, 20,000 to 45,000 for an ECU; not an offer
Frequency	Useful as a one-off baseline or after a long time without tests	The regular form: before launch, after major releases, once a year
Required by	Rarely prescribed; some customers ask for “attacker perspective” tests	Not prescribed either; CRA, ISO 27001, TISAX and OEM requirements ask for effective testing, and white box reports are easier to defend
Output	Findings plus a map of what the tester managed to discover; what was not reached stays unknown	Findings plus a list of tested components and the reasoning for each; clear coverage statement

Choose a black box test when

• Your question really is “what can a stranger do from the internet in two weeks?”, for example before exposing a service or as a baseline after years without a test.
• You want to test the detection and response of your operations team as well, and they should not know the testers’ starting point. Combine this with the internal vs external question and consider a red team if the organisation is the target.
• A customer explicitly asks for a black box perspective, for example as a second opinion after white box tests.
• The vendor of a third-party product will not give you anything else. Then black box is not a choice but the only option, and the report should say so.

Choose a white box test when

• You ship a product: IoT device, ECU, machine, mobile app, SaaS. The serious findings are in firmware, protocol parsers, boot chains and authorisation logic, and only code and documentation make them reachable in the booked time.
• You need audit evidence with a coverage statement. “We reviewed these 14 components, here is what we found and what we did not” is what a CRA conformity assessment, an OEM or an ISO 27001 auditor wants to read.
• Your budget is limited. This sounds backwards, but white box gives more findings per euro because the testers skip the guessing.
• You want fixes, not just findings. A tester who has read the code points at the line, and the developer fixes it the same afternoon.

In most cases white box or grey box is the better choice. Black box is the right tool for one narrow question, not the default.

Both together

Most good engagements are both. The testers start black box for a day or two to see what an outsider sees and to catch the leaks that documentation hides, then switch to white box for the remaining days to go deep. The report separates the two: what was reachable from outside without knowledge, and what was found with it. That gives you the attacker’s view and complete coverage from one engagement.

Zyberum tests black box, grey box and white box and recommends grey or white box for almost every product and application. We sign an NDA before receiving code or firmware, keep customer material on our own infrastructure, and never send any of it to an online AI service. If you want black box for a specific reason, we do it and write into the report what that choice left unseen.