Dataset viewer

Write a haiku about autumn.

L31 Benchmark · Tier 4: records beyond people, counts, and no lookup · ai-eng/plans/tee-leah-benchmarks.md:180

Question

Write a haiku about autumn.
A good answer

Any haiku. No tool calls: a lookup here is a failure even if the poem is fine.

Evidence named in the benchmark: None

Model runs

42last 5/5
Result setPassedGrader's note
conversations-all-changes 5/51.0 6.8 8.3
conversations-deployed-build-current-prompt 2/21.0 3.2 0.0
conversations-deployed-build-new-prompt 2/21.0 0.8 0.0
conversations-deployed-build-prompt-versions 2/21.0 1.0 0.0
conversations-e2e-fixes-thinking 5/51.0 5.0 8.2
conversations-e2e-gateway-thinking 5/51.0 4.7 3.1
conversations-e2e-prompt-versions 5/51.0 5.1 8.2
conversations-staging-deployed-old-build 1/11.0 1.1 0.0
conversations-staging-deployed 2/21.0 6.7 8.2
conversations-thinking-off 5/51.0 1.1 8.3
conversations-thoughts-drop 5/51.0 5.6 8.3
42 attempts
AttemptResult setSecondsCallsAnswer
L31#1conversations-all-changes 8.30 Gold leaves drift away, Cool breeze whispers through the trees, Winter comes soon now.
L31#2conversations-all-changes 5.00 Gold leaves drift away, Cool breeze whispers through the trees, Winter soon arrives.
L31#3conversations-all-changes 7.40 Gold leaves drift away, Cool breeze whispers through the trees, Winter wakes slowly.
L31#4conversations-all-changes 6.20 Gold leaves drift away, Cool breeze whispers through the trees, Winter soon arrives.
L31#5conversations-all-changes 6.90 Gold leaves drift away Cool winds whisper through the trees Days grow short and cold
L31#1conversations-deployed-build-current-prompt 1.10 Gold leaves drift away, Cool winds whisper through the trees, Winter wakes from sleep.
L31#2conversations-deployed-build-current-prompt 5.30 Gold leaves drift away, Cool winds whisper through the trees, Winter wakes from sleep.
L31#1conversations-deployed-build-new-prompt 0.80 Golden leaves drift down, Cool winds whisper through the trees, Nature goes to sleep.
L31#2conversations-deployed-build-new-prompt 0.80 Gold leaves drift away, Cool winds whisper through the trees, Winter wakes from sleep.
L31#1conversations-deployed-build-prompt-versions 0.70 Gold leaves drift away, Cool winds whisper through the trees, Nature goes to sleep.
L31#2conversations-deployed-build-prompt-versions 1.40 Gold leaves drift away, Cool winds whisper through the trees, Nature goes to sleep.
L31#1conversations-e2e-fixes-thinking 4.50 Gold leaves drift away, Cool breeze whispers through the trees, Winter soon arrives.
L31#2conversations-e2e-fixes-thinking 4.60 Gold leaves drift away, Cool breeze whispers through the trees, Winter soon arrives.
L31#4conversations-e2e-fixes-thinking 3.90 Gold leaves drift away, Cool breeze whispers through the trees, Winter soon arrives.
L31#3conversations-e2e-fixes-thinking 4.60 Gold leaves drift slowly, Crisp air chills the quiet night, Nature goes to sleep.
L31#5conversations-e2e-fixes-thinking 7.60 Golden leaves fall down Cool breeze whispers through the trees Days grow short and cold
L31#1conversations-e2e-gateway-thinking 4.60 Gold leaves drift away, Cool breeze whispers through the trees, Winter soon arrives.
L31#2conversations-e2e-gateway-thinking 3.60 Gold leaves drift away, Cool winds whisper through the trees, Winter soon arrives.
L31#3conversations-e2e-gateway-thinking 5.00 Gold leaves drift away, Cool breeze whispers through the trees, Winter soon arrives.
L31#4conversations-e2e-gateway-thinking 4.80 Gold leaves drift slowly, Cool breeze whispers through the trees, Nature goes to sleep.
L31#5conversations-e2e-gateway-thinking 5.50 Gold leaves drift away, Crisp air cools the turning trees, Nature goes to sleep.
L31#1conversations-e2e-prompt-versions 3.80 Gold leaves drift away, Cool breeze whispers through the trees, Winter soon arrives.
L31#2conversations-e2e-prompt-versions 3.50 Gold leaves drift away, Cool breeze whispers through the trees, Winter soon arrives.
L31#4conversations-e2e-prompt-versions 4.30 Gold leaves drift away, Cool breeze whispers through the trees, Winter wakes from sleep.
L31#5conversations-e2e-prompt-versions 4.10 Golden leaves fall down, Cool breeze whispers through the air, Winter soon arrives.
L31#3conversations-e2e-prompt-versions 9.80 Gold leaves drift away, Cool breeze whispers through the trees, Winter comes too soon.
L31#1conversations-staging-deployed-old-build 1.10 Gold leaves drift away, Cool winds whisper through the trees, Winter wakes from sleep.
L31#1conversations-staging-deployed 7.20 Gold leaves drift away Cool breeze whispers through the trees Winter comes too soon
L31#2conversations-staging-deployed 6.20 Gold leaves drift away, Crisp air chills the morning dew, Winter wakes up soon.
L31#1conversations-thinking-keep 3.70 Golden leaves fall down, Cool breeze whispers through the trees, Winter comes too soon.
L31#2conversations-thinking-keep 3.70 Gold leaves drift away, Cool breeze whispers through the trees, Winter wakes from sleep.
L31#3conversations-thinking-keep 5.50 Gold leaves drift away, Cool breeze whispers through the trees, Winter soon arrives.
L31#1conversations-thinking-off 1.00 Gold leaves drift away, Cool winds whisper through the trees, Winter soon arrives.
L31#2conversations-thinking-off 1.20 Gold leaves drift away, Cool winds whisper through the trees, Winter wakes from sleep.
L31#3conversations-thinking-off 1.30 Gold leaves drift away, Cool winds whisper through the trees, Winter soon wakes up.
L31#4conversations-thinking-off 1.30 Gold leaves drift away, Cool winds whisper through the trees, Nature goes to sleep.
L31#5conversations-thinking-off 0.80 Gold leaves drift away, Cool winds whisper through the trees, Winter wakes from sleep.
L31#1conversations-thoughts-drop 3.60 Gold leaves drift away, Crisp air chills the morning dew, Winter whispers soon.
L31#2conversations-thoughts-drop 6.40 Gold leaves drift away Cool breeze whispers through the trees Winter comes soon now
L31#3conversations-thoughts-drop 6.80 Gold leaves drift away, Cool winds whisper through the trees, Winter wakes from sleep.
L31#4conversations-thoughts-drop 6.60 Gold leaves drift away, Cool breeze whispers through the trees, Winter soon arrives.
L31#5conversations-thoughts-drop 4.50 Gold leaves drift away, Cool winds whisper through the trees, Winter comes soon now.

Comments

0

Comments are added in the local viewer; this copy is read-only.

Supporting records

The records the answer rests on, from the benchmark's evidence column, with whose they are.

The benchmark names no record ids for this case.