Black Box, Grey Box and White Box Testing

What are black box, grey box and white box testing?

The three terms describe the information base of a security test. In a black box test the tester knows only what an outside attacker would: a URL, an IP range, a device in a box. In a white box test the tester has everything the engineering team has: source code, firmware, architecture documents, schematics, credentials for every role. Grey box sits in between and is the most common form: user accounts, an architecture overview, API documentation and a contact for questions, but no code.

The terms come from software testing, where white box means testing with knowledge of the internal structure and black box means testing against the external behaviour only. In penetration testing they describe scope and starting conditions, not technique: the same tester uses the same tools and reaches the same depth of exploitation, just from a different starting point.

Where is it defined?

There is no single normative definition. The BSI study on penetration testing classifies tests along several criteria, the information base among them, and distinguishes black box from white box tests. NIST SP 800-115 uses the related distinction between overt testing, done with the knowledge and cooperation of the IT staff, and covert testing, done without their knowledge, and between external and internal viewpoints. The OWASP Web Security Testing Guide describes black box and grey box testing of web applications and argues for giving testers as much information as possible. The Penetration Testing Execution Standard treats the question in its pre-engagement phase, where the information provided is agreed together with scope and rules of engagement.

What it means in practice

Our default recommendation is grey box, moving towards white box wherever the material exists. The reason is efficiency: a pentest is bought by the day, and every day spent reverse engineering an undocumented protocol that the client could have documented in an afternoon is a day not spent finding vulnerabilities.

For embedded and automotive targets the difference is largest. On a power-electronics ECU with firmware images and a diagnostic database we can enumerate every UDS service, data identifier and routine on the first day and spend the rest on the SecurityAccess implementation and the parsers behind it. Without them, the first week goes into extracting and reverse engineering the firmware, which is interesting but tells the client less per euro. Black box has its place. A short black box phase at the start of an engagement shows what an attacker finds without help and tests how visible the product’s attack surface is. A full black box test is the right choice when the question is specifically about exposure, for example an external perimeter test, or when the client cannot share internals for contractual reasons.

The information base also changes the report. A black box report says what was reachable in ten days from outside. A white box report can say that a vulnerability class is absent from the code, which is the statement a product owner needs before a release or a compliance audit.

Common misunderstandings

Black box is not more realistic. Real attackers have unlimited time, buy devices, download firmware updates and read patents; a ten-day black box test simulates none of that. White box is not a code review: the tester still attacks the running system and proves exploitability, the code merely tells them where to look. And grey box is not a compromise for undecided clients. It is the form most tests should take.