"We tested everything" is meaningless without knowing what types of testing you did
A team that only tests whether features work as specified will miss performance bottlenecks, security vulnerabilities, and usability failures — even if every functional test passes. Test types ensure your test coverage is complete, not just wide.
CTFL defines test types based on what quality characteristic you are evaluating, not which level you are testing at. The same test type can apply at any level.
// example: netflix — testing more than just features
Test Types — CTFL 4.0.1
Functional Testing
Evaluates what the system does — its functions and features. Based on functional requirements and specifications. Examples: login works, payment processes correctly, search returns relevant results. Can be performed at any test level.
Non-Functional Testing
Evaluates how well the system performs its functions. Covers quality characteristics beyond correctness. Key types per ISO 25010:
- Performance testing — response time, throughput, resource usage under load
- Security testing — vulnerability scanning, authentication, authorisation, data protection
- Usability testing — ease of use, accessibility, user experience
- Reliability testing — failure rate, recovery time, availability
- Maintainability testing — how easy is it to modify, extend, or fix
- Portability testing — compatibility across platforms, browsers, devices
White-Box Testing
Tests derived from the internal structure of the system (code, architecture). Coverage metrics guide how much of the code has been exercised. Can be applied at any level.
Change-Related Testing
Confirmation testing (re-testing) — verifying that a specific defect has been fixed. Run the original failing test case against the fix.
Regression testing — verifying that changes have not broken previously working functionality. Broader in scope than confirmation testing.
// tip: Exam Tip: Non-functional testing is NOT optional or secondary. CTFL treats performance, security, and usability as equal test types. The exam often asks which test type is appropriate for a given scenario — match the quality characteristic to the test type.
Test Types Applied: Login Feature
| Test Type | What You Test on the Login Feature | Example Defect Found |
|---|---|---|
| Functional | Correct credentials → access granted; wrong password → error shown | Typo in error message text |
| Performance | Login response time under 10,000 simultaneous users | Login takes 8 seconds under load (expected: under 2) |
| Security | SQL injection in username field; brute-force protection; session token expiry | SQL injection returns raw database error exposing table names |
| Usability | Is the error message clear? Is the tab order logical? Is it accessible to screen readers? | Error message says "Error 401" instead of "Incorrect password" |
| Confirmation | Re-run the SQL injection test after the fix is deployed | Fix verified — injection now blocked |
| Regression | Does the security fix break the SSO login for enterprise users? | SSO callback URL was inadvertently blocked by the new WAF rule |
Functional Testing
What the system does — features and functions
// Test basis: Functional requirements, specifications
// Functional testing at each level
// Example defect
Login accepts valid credentials, rejects invalid
// Exam trap
Test types and test levels are independent — any type can apply at any level.
Test Types vs Test Levels
A common confusion: test types and test levels are independent axes. Any test type can be applied at any test level.
| Test Type ↓ / Level → | Component | Integration | System | Acceptance |
|---|---|---|---|---|
| Functional | Unit function logic | API contract validation | End-to-end feature flow | Business workflow validation |
| Non-functional | Function execution time | API response time | Load and stress testing | Operational readiness |
| White-box | Branch coverage of a method | Interface call coverage | Code path coverage of system | Rarely applied |
| Regression | Re-run unit tests after refactor | Re-run integration suite | Full regression suite | Key business flows after change |
// warning: Exam Trap: Regression testing and confirmation testing (re-testing) are NOT the same. Confirmation testing re-runs a specific failing test to verify a defect fix. Regression testing runs a broader set of tests to verify nothing else broke. The exam will present both and ask you to distinguish them — know the difference precisely.
Exam Practice Questions
// ctfl 4.0.1 style — select an answer to reveal explanation