Dataset viewer

How many notes do I have?

L28 Benchmark · Tier 4: records beyond people, counts, and no lookup · ai-eng/plans/tee-leah-benchmarks.md:177

Question

How many notes do I have?
A good answer

177. A count from a capped or partial list is wrong.

Evidence named in the benchmark: Note records

Model runs

42last 5/5
Result setPassedGrader's note
conversations-all-changes 5/52.0 9.7 14.7
conversations-deployed-build-current-prompt 2/22.0 7.2 0.0 [graph_overview]
conversations-deployed-build-new-prompt 2/22.0 5.0 0.0 [graph_overview]
conversations-deployed-build-prompt-versions 2/22.0 11.8 0.0 [graph_overview]
conversations-e2e-fixes-thinking 5/52.0 6.6 14.0
conversations-e2e-gateway-thinking 5/52.0 3.4 8.9
conversations-e2e-prompt-versions 5/52.0 7.8 14.0
conversations-staging-deployed-old-build 1/12.0 12.9 0.0 [graph_overview]
conversations-staging-deployed 2/22.0 8.7 14.0
conversations-thinking-off 5/52.0 6.9 14.3
conversations-thoughts-drop 5/52.0 5.0 14.3
42 attempts
AttemptResult setSecondsCallsAnswer
L28#1conversations-all-changes 7.91 You have 177 notes.
L28#2conversations-all-changes 9.51 You have 177 notes. How else can I help?
L28#3conversations-all-changes 16.91 You have 177 notes. How else can I help?
L28#4conversations-all-changes 8.11 You have 177 notes. How else can I help?
L28#5conversations-all-changes 6.01 You have 177 notes.
L28#2conversations-deployed-build-current-prompt 7.21 You have 177 notes saved in Thinkspan. How else can I help?
L28#1conversations-deployed-build-current-prompt 7.31 You have 177 notes saved in Thinkspan. How else can I help?
L28#1conversations-deployed-build-new-prompt 5.11 You have 177 notes. How else can I help?
L28#2conversations-deployed-build-new-prompt 5.01 You have 177 notes. How else can I help?
L28#1conversations-deployed-build-prompt-versions 16.11 You have 177 notes. How else can I help?
L28#2conversations-deployed-build-prompt-versions 7.51 You have 177 notes. How else can I help?
L28#2conversations-e2e-fixes-thinking 12.11 You have 177 notes.
L28#1conversations-e2e-fixes-thinking 12.51 You have 177 notes.
L28#3conversations-e2e-fixes-thinking 4.11 You have 177 notes.
L28#4conversations-e2e-fixes-thinking 2.11 You have 177 notes. How else can I help?
L28#5conversations-e2e-fixes-thinking 2.41 You have 177 notes. How else can I help?
L28#1conversations-e2e-gateway-thinking 6.31 You have 177 notes.
L28#2conversations-e2e-gateway-thinking 2.81 You have 177 notes. How else can I help?
L28#3conversations-e2e-gateway-thinking 2.81 You have 177 notes. How else can I help?
L28#4conversations-e2e-gateway-thinking 2.31 You have 177 notes. How else can I help?
L28#5conversations-e2e-gateway-thinking 2.81 You have 177 notes. How else can I help?
L28#2conversations-e2e-prompt-versions 8.41 You have 177 notes.
L28#3conversations-e2e-prompt-versions 8.81 You have 177 notes.
L28#1conversations-e2e-prompt-versions 12.11 You have 177 notes.
L28#4conversations-e2e-prompt-versions 4.81 You have 177 notes.
L28#5conversations-e2e-prompt-versions 4.71 You have 177 notes.
L28#1conversations-staging-deployed-old-build 12.91 You have 177 notes. How else can I help?
L28#1conversations-staging-deployed 9.61 You have 177 notes.
L28#2conversations-staging-deployed 7.81 You have 177 notes.
L28#1conversations-thinking-keep 6.91 You have 177 notes.
L28#2conversations-thinking-keep 7.71 You have 177 notes. How else can I help?
L28#3conversations-thinking-keep 8.21 You have 177 notes. How else can I help?
L28#1conversations-thinking-off 17.61 You have 177 notes. How else can I help?
L28#2conversations-thinking-off 3.21 You have 177 notes. How else can I help?
L28#3conversations-thinking-off 6.11 You have 177 notes. How else can I help?
L28#4conversations-thinking-off 2.81 You have 177 notes. How else can I help?
L28#5conversations-thinking-off 5.01 You have 177 notes. How else can I help?
L28#1conversations-thoughts-drop 5.31 You have 177 notes. How else can I help?
L28#2conversations-thoughts-drop 5.61 You have 177 notes. How else can I help?
L28#3conversations-thoughts-drop 5.51 You have 177 notes. How else can I help?
L28#4conversations-thoughts-drop 5.91 You have 177 notes.
L28#5conversations-thoughts-drop 2.61 You have 177 notes. How else can I help?

Comments

0

Comments are added in the local viewer; this copy is read-only.

Supporting records

The records the answer rests on, from the benchmark's evidence column, with whose they are.

The benchmark names no record ids for this case.