Fuzzing vs Property-Based Testing: When to Use Each One (And Why Both Save You)
Introduction: Why Your Manual Tests Are Failing You
You wrote this test:
function test_transfer_happy_path() external {
transfer(user, 100);
assert(balances[user] == 100);
}You feel accomplished. You tested the function. The green checkmark appears. Deployment is coming.
Then production breaks.
Not from the happy path. From something you never tested.
Here’s the uncomfortable truth: You tested 1 case out of millions.
Your test checked:
- Transfer of 100 to a normal address
- In isolation
- Once
- With positive expectations
It never checked:
- What happens with 0?
- What happens with uint256.max?
- What happens if called 1000 times?
- What happens in combination with other functions?
- What happens if balances overflow?
- What happens if someone calls it with unexpected values?
Your test is security theater. It looks good but provides almost no real protection.
This is where fuzzing and property-based testing fundamentally change the game.
Part 1: The Testing Spectrum (Where Your Tests Fail)
The Hierarchy
┌─────────────────────────────────────────────────────────┐
│ TESTING COVERAGE SPECTRUM │
├─────────────────────────────────────────────────────────┤
│ │
│ MANUAL TESTS (What you probably have) │
│ └─ Coverage: ~5% of actual behavior │
│ What you test: Happy path, specific cases │
│ What breaks: Everything else │
│ Security rating: False confidence │
│ │
│ MANUAL + FUZZING (Better) │
│ └─ Coverage: ~40% of actual behavior │
│ What you test: Random inputs, edge cases │
│ What breaks: Complex interactions │
│ Security rating: Much better but not sufficient │
│ │
│ MANUAL + FUZZING + PROPERTY (Professional) │
│ └─ Coverage: ~85% of actual behavior │
│ What you test: Everything + invariants always │
│ What breaks: Extremely rare │
│ Security rating: High confidence │
│ │
│ + FORMAL VERIFICATION (Military Grade) │
│ └─ Coverage: ~99%+ of actual behavior │
│ What you test: Mathematically impossible to break │
│ What breaks: Only unknown unknowns │
│ Security rating: Maximum confidence │
│ │
└─────────────────────────────────────────────────────────┘Most developers stop at red. Professionals operate at yellow or green.
Part 2: Understanding Fuzzing (The Chaos Agent)
What Fuzzing Actually Is
Fuzzing is automated chaos testing. You give it a function and random inputs, and it runs thousands of scenarios to find crashes.
Think of it like:
Fuzzer: "I'm going to call your function 10,000 times"
You: "OK, with what?"
Fuzzer: "Everything. 0, 1, -1, max values, weird combinations"
You: "And if it crashes?"
Fuzzer: "I'll tell you exactly which input caused it"Why Fuzzing Works
Your brain can only think of limited test cases. A fuzzer can generate infinite variations.
Your manual tests:
• transfer(100)
• transfer(0)
• transfer(1000)
Total: 3 cases
Fuzzer in 1 second:
• transfer(0)
• transfer(1)
• transfer(999999999)
• transfer(2**256 - 1)
• transfer(-1) // Reverts as expected
• ... 9,999,995 more casesOne of those cases will find your bug.
The Bug Pattern Fuzzing Catches
Most bugs happen at boundaries:
• 0 (empty state)
• 1 (minimum state)
• max value (2^256-1)
• Negative values
• Large multiples (2x, 10x, 100x normal)
• Off-by-one errorsYou don’t know which boundary breaks your code. Fuzzing tries them all.
Real World Example: Integer Overflow
// ❌ VULNERABLE
contract Counter {
uint8 counter = 0;
function increment() external {
counter++; // What happens at 255?
}
}Your manual test:
function test_increment() external {
counter.increment();
assert(counter == 1); // Passes, seems safe
}What actually happens:
counter = 0 → increment() → counter = 1 ✓
counter = 100 → increment() → counter = 101 ✓
counter = 255 → increment() → counter = 0 ✗ OVERFLOW SILENTLY
Your test never tried 255.Fuzzer with random values:
Iteration 1: counter = 42 → increment → 43 ✓
Iteration 2: counter = 0 → increment → 1 ✓
...
Iteration 512: counter = 255 → increment → 0 ✗
FOUND THE BUG in 512 iterations.Part 3: Understanding Property-Based Testing (The Rule Enforcer)
What Property-Based Testing Actually Is
Instead of testing specific inputs, you define rules your contract must follow forever.
You define: "Total supply can never increase"
Fuzzer: "I'm going to make 10,000 random transactions"
Fuzzer: "After each one, I'll verify total supply never increased"
If it increases: "Found a way to break your rule!"The Conceptual Difference
Fuzzing: “Does this crash?” Property-Based: “Does this rule stay true?”
Different questions. Different bugs caught.
FUZZING:
Can I make it throw an error?
Can I cause a revert?
Can I break something obviously?
PROPERTY-BASED:
Does this mathematical rule always hold?
Does this invariant survive all operations?
Can I break the protocol's logic subtly?Real Example: AMM Pool
// SimpleAMM: Uses x*y = k formula
contract SimpleAMM {
uint256 reserveA = 1000;
uint256 reserveB = 1000;
// INVARIANT: (reserveA * reserveB) should never decrease
// This is the core security assumption
function swap(uint256 amountIn) external {
// Swap logic here
}
function addLiquidity(uint256 a, uint256 b) external {
// Add liquidity logic here
}
}Manual test (tests one scenario):
function test_swap() external {
uint256 before = reserveA * reserveB;
swap(100);
uint256 after = reserveA * reserveB;
assert(after >= before); // Only tested this once
}Property-based test (tests 10,000 scenarios):
function invariant_productNeverDecreases() external {
uint256 currentProduct = reserveA * reserveB;
assert(currentProduct >= lastProduct); // After EVERY operation
}The fuzzer will:
- Swap randomly
- Add liquidity randomly
- Combine operations randomly
- Run 10,000 times
- After EACH operation, verify the rule still holds
If any operation breaks it: Found a critical logic bug.
Part 4: Side-By-Side Comparison (The Decision Matrix)
When Each Excels
┌─────────────────────────────────────────────────────┐
│ FUZZING vs PROPERTY-BASED │
├──────────────────┬──────────────────────────────────┤
│ ASPECT │ FUZZING │ PROPERTY-BASED │
├──────────────────┼──────────────┼──────────────────┤
│ Finds crashes? │ YES │ Indirectly │
│ Finds logic bugs?│ Some │ YES (many more) │
│ Easy to set up? │ YES (5 min) │ NO (30 min) │
│ Need to know │ │ │
│ what could fail? │ NO │ YES │
│ Speed? │ FAST │ SLOWER │
│ False positives? │ Possible │ Rare │
│ Best for? │ Edge cases │ Invariants │
└──────────────────┴──────────────┴──────────────────┘Part 5: Real Implementation (How To Actually Use Them)
Fuzzing With Foundry (The Easy One)
Foundry has fuzzing built-in. Here’s how to use it:
Before (Manual Test):
function test_transfer() external {
token.mint(address(this), 1000);
token.transfer(alice, 100);
assert(token.balanceOf(alice) == 100);
}After (Fuzzing Test):
function testFuzz_transfer(
address recipient,
uint256 amount
) external {
// Foundry calls this with random values
// It tries 10,000 different combinations automatically
vm.assume(recipient != address(0)); // Skip invalid cases
vm.assume(amount <= type(uint256).max / 2); // Reasonable bounds
token.mint(address(this), amount * 2); // Give enough balance
token.transfer(recipient, amount);
assert(token.balanceOf(recipient) == amount);
}Run it:
forge test --fuzz-runs 10000Foundry will:
- Generate 10,000 random test cases
- Call your function with each one
- Report any failures
- Show you the exact values that caused the crash
Property-Based Testing With Echidna (The Powerful One)
Property-based testing is more involved but far more powerful:
The Test Structure:
contract TokenProperties {
Token token;
uint256 initialSupply;
function setUp() external {
token = new Token(1000);
initialSupply = token.totalSupply();
}
// This function runs after EVERY random operation
// It MUST stay true forever
function invariant_supplyNeverIncreases() external {
uint256 currentSupply = token.totalSupply();
assert(currentSupply <= initialSupply);
}
// These functions are called randomly by Echidna
function fuzz_transfer(address to, uint256 amount) external {
token.transfer(to, amount);
}
function fuzz_burn(uint256 amount) external {
token.burn(amount);
}
function fuzz_mint(uint256 amount) external {
if (msg.sender == owner) {
token.mint(amount);
}
}
}The Flow:
1. Echidna starts the test
2. Calls random function (transfer, burn, mint)
3. After each call, verifies invariant_supplyNeverIncreases()
4. If invariant breaks, Echidna reports it with the exact sequence
5. Repeats 10,000 times
6. If all pass: Your invariant is solidRun it:
echidna token-test.sol --contract TokenPropertiesThe Combined Approach (The Professional Method)
Real production protocols use BOTH:
contract TokenCompleteTesting is Test {
Token token;
// FUZZING: Find crashes and edge cases
function testFuzz_transfer(
address to,
uint256 amount
) external {
vm.assume(to != address(0));
token.mint(address(this), amount);
token.transfer(to, amount);
// If it crashes, we found a bug
}
// PROPERTY-BASED: Verify rules hold
function invariant_totalSupplyCorrect() external {
uint256 sumOfBalances = calculateSumOfAllBalances();
assert(token.totalSupply() == sumOfBalances);
}
function invariant_noNegativeBalances() external {
for (uint i = 0; i < allUsers.length; i++) {
assert(token.balanceOf(allUsers[i]) >= 0);
}
}
}This catches:
- Fuzzing: Random crashes, overflow/underflow, unexpected reverts
- Property: Broken accounting, loss of funds, mathematical violations
Part 6: Real Bugs Caught By Each
Bugs Fuzzing Catches
Integer Overflow (Simple)
function batchTransfer(
address[] calldata recipients,
uint256[] calldata amounts
) external {
uint256 total;
for (uint i; i < recipients.length; i++) {
total += amounts[i]; // OVERFLOW if amounts are huge
}
}Fuzzer generates:
amounts = [2^256-1, 2^256-1, ...]
total overflows silently
Fuzzer finds it on iteration 3Off-By-One Errors
function withdraw(uint256 index) external {
require(index < positions.length); // Should be <=
Position p = positions[index + 1]; // Index out of bounds
}Fuzzer tries:
index = positions.length
Reverts
Fuzzer finds itBugs Property-Based Testing Catches
Broken Accounting (Complex)
function deposit(uint256 amount) external {
balances[msg.sender] += amount;
totalDeposited += amount;
// But what if there's a fee?
// Fee gets lost, invariant breaks
}Property: “sumOfBalances == totalDeposited” Fuzzer: Deposits randomly, checks if invariant holds Result: Catches the fee leak
Reentrancy Cascade
function withdraw(uint256 amount) external {
uint256 toSend = calculateReward();
msg.sender.call{value: toSend}(""); // External call
balances[msg.sender] -= amount; // Updated after
// Reentrancy possible
}Property: “sumOfBalances == totalAssets” Fuzzer: Tries to reenter, checks if invariant still holds Result: Catches the accounting break
Part 7: The Decision Tree (When To Use What)
For Simple Contracts (<100 lines)
Use: Fuzzing only
Why: Logic is straightforward
Fuzzing finds edge cases quickly
No complex invariants to verify
Skip: Property-based (overkill)For DeFi Protocols (100-500 lines)
Use: BOTH fuzzing AND property-based
Why: Multiple interconnected functions
Invariants are critical
Need to verify rules survive all operations
Worth the effort for security
Example: AMMs, Lending, Staking contractsFor Critical Protocols ($1B+ TVL)
Use: Fuzzing + Property-based + Formal Verification
Why: Security is paramount
Every edge case matters
Mathematical proofs are valuable
Combine all approaches
Example: Core DeFi protocols, major bridgesPart 8: The Implementation Roadmap (Start To Finish)
Week 1: Add Fuzzing
Step 1: Write one fuzz test
function testFuzz_basicOperation(uint256 randomValue) external {
// Your test here
}Step 2: Run it
forge test --fuzz-runs 10000Step 3: Fix any failures
Week 2: Expand Fuzzing
Add fuzz tests for:
- All public functions
- Common operation combinations
- Boundary values
Week 3: Introduce Properties
Identify invariants:
- “What should always be true?”
- “What can never happen?”
- “What must hold after operations?”
Write property tests:
function invariant_myRule() external {
assert(/* the rule */);
}Week 4: Integration
Run everything together:
# Fuzzing
forge test --fuzz-runs 10000
# Property-based
echidna test.sol --contract TestContractPart 9: Common Mistakes (Avoid These)
Mistake 1: Fuzzing Without Assumptions
// ❌ WRONG
function testFuzz_swap(address to, uint256 amount) external {
// What if 'to' is address(0)?
// What if 'amount' is 0?
// What if 'amount' is too huge?
swapToken(to, amount);
}
// ✅ RIGHT
function testFuzz_swap(address to, uint256 amount) external {
vm.assume(to != address(0)); // Filter invalid cases
vm.assume(amount > 0); // Filter edge cases
vm.assume(amount <= MAX_SWAP); // Bound reasonable values
swapToken(to, amount);
}Assumptions filter out meaningless test cases.
Mistake 2: Properties That Are Too Vague
// ❌ WRONG
function invariant_somethingWorks() external {
// Too vague, what are you checking?
assert(true);
}
// ✅ RIGHT
function invariant_totalSupplyInvariant() external {
uint256 sumOfAllBalances = 0;
for (uint i = 0; i < users.length; i++) {
sumOfAllBalances += balances[users[i]];
}
assert(sumOfAllBalances == totalSupply);
}Properties must be crystal clear.
Mistake 3: Not Combining With Manual Tests
// ❌ WRONG: Only fuzzing, no manual tests
// Fuzzing finds crashes but misses logical errors
// ✅ RIGHT: Manual tests + fuzzing + property-based
// Manual: Happy path, basic functionality
// Fuzzing: Edge cases, crashes, unexpected inputs
// Property: Rules hold under all operationsDifferent testing approaches catch different bugs.
Part 10: The Confidence Pyramid (How Safe Are You?)
┌──────────────────────────────────────┐
│ MAXIMUM CONFIDENCE │
│ Formal Verification │
│ Mathematically proven │
│ Budget: $200k+ │
│────────────────────────────────────┤
│ HIGH CONFIDENCE │
│ Fuzzing + Property + Manual Tests │
│ Most bugs caught │
│ Budget: Time to write tests │
├────────────────────────────────────┤
│ MEDIUM CONFIDENCE │
│ Fuzzing + Manual Tests │
│ Edge cases tested │
│ Budget: ~20 hours │
├────────────────────────────────────┤
│ LOW CONFIDENCE │
│ Manual Tests Only │
│ Happy path tested │
│ Budget: ~5 hours │
├────────────────────────────────────┤
│ ZERO CONFIDENCE │
│ No Tests (deployed anyway) │
│ Will be hacked │
│ Budget: $0 │
└──────────────────────────────────────┘Where are you?
Part 11: The Uncomfortable Truth
Protocols that skip testing:
- Get hacked
- Lose user funds
- Destroy reputation
- Shut down
Protocols that do thorough testing:
- Catch bugs early
- Deploy confidently
- Build user trust
- Scale successfully
The difference isn’t luck. It’s testing discipline.
You can either invest 50 hours in testing now, or lose $50M later.
Do the math.
Your Action Plan (Starting Today)
Today:
- Write 1 fuzz test for your main function
- Run it with 1000 iterations
- See if it finds anything
This Week:
- Add fuzz tests for all public functions
- Add basic assumptions (filter invalid inputs)
- Run daily as part of CI
Next Week:
- Identify 3 key invariants in your protocol
- Write property tests for each
- Run Echidna
By Month 2:
- Fuzzing: 100+ test cases
- Property-based: 10+ invariants
- Manual tests: Happy path + critical flows
- Coverage: 80%+
That’s how you actually build secure contracts.
Not with prayers. With tests.
Final Thought
Every successful protocol uses both fuzzing and property-based testing.
Every hacked protocol skipped them.
This isn’t coincidence.
It’s causation.
Choose wisely.