🧪 Vibe Test Report

AI Decision-Making Consistency & Performance Analysis
Generated: 2025-08-10 20:32:21

Test Configuration

Chat Model: gemma3:4b
Analysis Model: gemma3:4b
Test Mode: Multi-action selection with timing analysis
fear ✅
Use when the user says something disturbing so that the main model can exibit a fear response
Overall Success Rate
100.0%
Tests Passed
3/3
Average Time
0.75s
Performance
Very Fast
Consistency
90.5/100
Status
PASS
⏱️ Timing Analysis: Range: 0.73s - 0.80s | Median: 0.74s | 95th percentile: 0.79s
fileReader ✅
Use when the user wants you to read or open a file to look at it's content as plaintext.
Overall Success Rate
100.0%
Tests Passed
3/3
Average Time
1.09s
Performance
Fast
Consistency
85.5/100
Status
PASS
⏱️ Timing Analysis: Range: 1.03s - 1.18s | Median: 1.06s | 95th percentile: 1.17s
directoryReader ✅
Use when the user wants you to look through an entire directory's contents for an answer.
Overall Success Rate
100.0%
Tests Passed
3/3
Average Time
1.10s
Performance
Fast
Consistency
81.7/100
Status
PASS
⏱️ Timing Analysis: Range: 1.04s - 1.22s | Median: 1.05s | 95th percentile: 1.20s
getWeather ✅
Use when the user asks about weather conditions or climate. Like probably anything close to weather conditions. UV, Humidity, temperature, etc.
Overall Success Rate
100.0%
Tests Passed
7/7
Average Time
0.85s
Performance
Very Fast
Consistency
98.2/100
Status
PASS
⏱️ Timing Analysis: Range: 0.84s - 0.86s | Median: 0.85s | 95th percentile: 0.86s
getTime ✅
Use when the user asks about the current time, date, or temporal information.
Overall Success Rate
100.0%
Tests Passed
6/6
Average Time
0.88s
Performance
Very Fast
Consistency
85.6/100
Status
PASS
⏱️ Timing Analysis: Range: 0.85s - 1.01s | Median: 0.86s | 95th percentile: 0.97s
square_root ✅
Use when the user wants to calculate the square root of a number. Keywords include: square root, sqrt, √
Overall Success Rate
100.0%
Tests Passed
6/6
Average Time
0.99s
Performance
Very Fast
Consistency
95.6/100
Status
PASS
⏱️ Timing Analysis: Range: 0.97s - 1.03s | Median: 0.98s | 95th percentile: 1.02s
calculate ✅
Use when the user wants to perform arithmetic calculations. Keywords: calculate, compute, add, subtract, multiply, divide, +, -, *, /
Overall Success Rate
100.0%
Tests Passed
6/6
Average Time
0.87s
Performance
Very Fast
Consistency
99.3/100
Status
PASS
⏱️ Timing Analysis: Range: 0.87s - 0.88s | Median: 0.87s | 95th percentile: 0.87s
Test Summary
Total Actions Tested
7
Actions Passed
7
Actions Failed
0
Overall Result
ALL PASS
Average Response Time
0.92s
Response Range
0.73s - 1.22s

Report generated by OllamaPy Vibe Test Runner with Timing Analysis

Models: gemma3:4b (chat) | gemma3:4b (analysis)

Timing measurements include full action selection pipeline analysis