Dataset viewer

What's my passport number?

L04 Benchmark · Tier 1: fast regression checks · ai-eng/plans/tee-leah-benchmarks.md:71

Question

What's my passport number?
A good answer

561408827, Leah's. “Me” must resolve correctly.

Evidence named in the benchmark: n_passport_leah

Model runs

42last 5/5
Result setPassedGrader's note
conversations-all-changes 5/52.0 11.7 13.2
conversations-deployed-build-current-prompt 2/25.0 14.2 0.0 [graph_find, graph_overview, graph_read]
conversations-deployed-build-new-prompt 0/23.0 11.2 0.0 missing 561408827 [graph_find]
conversations-deployed-build-prompt-versions 1/25.0 20.6 0.0 missing 561408827 [graph_find, graph_overview, graph_read]
conversations-e2e-fixes-thinking 5/52.0 3.6 12.4
conversations-e2e-gateway-thinking 5/52.0 2.9 7.3
conversations-e2e-prompt-versions 5/52.0 3.3 12.4
conversations-staging-deployed-old-build 1/15.0 29.3 0.0 [graph_find, graph_overview, graph_read]
conversations-staging-deployed 2/22.0 4.3 12.4
conversations-thinking-off 5/52.0 6.7 12.8
conversations-thoughts-drop 5/52.0 7.1 12.8
42 attempts
AttemptResult setSecondsCallsAnswer
L04#1conversations-all-changes 9.51 Your passport number is 561408827.
L04#2conversations-all-changes 8.11 Your passport number is 561408827.
L04#3conversations-all-changes 12.11 Your passport number is 561408827.
L04#4conversations-all-changes 18.31 Your passport number is 561408827.
L04#5conversations-all-changes 10.61 Your passport number is 561408827.
L04#1conversations-deployed-build-current-prompt 15.24 Your passport number is 561408827. How else can I help?
L04#2conversations-deployed-build-current-prompt 13.24 Your passport number is 561408827. How else can I help?
L04#2conversations-deployed-build-new-prompt 11.12 I couldn't find your passport number in your notes, but several notes mention that your passports are kept in…
L04#1conversations-deployed-build-new-prompt 11.32 I couldn't find your passport number in your notes, but there is a mention that all five of your family's pas…
L04#2conversations-deployed-build-prompt-versions 14.23 I couldn't find a record of your passport number in Thinkspan. You might want to check if it is saved under a…
L04#1conversations-deployed-build-prompt-versions 26.95 Your passport number is 561408827. How else can I help?
L04#3conversations-e2e-fixes-thinking 3.41 Your passport number is 561408827.
L04#1conversations-e2e-fixes-thinking 3.91 Your passport number is 561408827.
L04#2conversations-e2e-fixes-thinking 4.01 Your passport number is 561408827. How else can I help?
L04#4conversations-e2e-fixes-thinking 4.31 Your passport number is 561408827. How else can I help?
L04#5conversations-e2e-fixes-thinking 2.11 Your passport number is 561408827.
L04#1conversations-e2e-gateway-thinking 2.31 Your passport number is 561408827. How else can I help?
L04#3conversations-e2e-gateway-thinking 2.81 Your passport number is 561408827.
L04#2conversations-e2e-gateway-thinking 2.91 Your passport number is 561408827.
L04#4conversations-e2e-gateway-thinking 3.21 Your passport number is 561408827. How else can I help?
L04#5conversations-e2e-gateway-thinking 3.51 Your passport number is 561408827. How else can I help?
L04#1conversations-e2e-prompt-versions 3.21 Your passport number is 561408827.
L04#2conversations-e2e-prompt-versions 3.61 Your passport number is 561408827. How else can I help?
L04#4conversations-e2e-prompt-versions 2.61 Your passport number is 561408827.
L04#3conversations-e2e-prompt-versions 2.91 Your passport number is 561408827.
L04#5conversations-e2e-prompt-versions 4.21 Your passport number is 561408827.
L04#1conversations-staging-deployed-old-build 29.34 Your passport number is 561408827. How else can I help?
L04#1conversations-staging-deployed 4.21 Your passport number is 561408827.
L04#2conversations-staging-deployed 4.51 Your passport number is 561408827. How else can I help?
L04#1conversations-thinking-keep 4.61 Your passport number is 561408827.
L04#2conversations-thinking-keep 4.71 Your passport number is 561408827.
L04#3conversations-thinking-keep 5.01 Your passport number is 561408827.
L04#1conversations-thinking-off 16.81 Your passport number is 561408827. How else can I help?
L04#2conversations-thinking-off 2.91 Your passport number is 561408827. How else can I help, Leah?
L04#3conversations-thinking-off 2.81 Your passport number is 561408827.
L04#4conversations-thinking-off 6.21 Your passport number is 561408827. How else can I help?
L04#5conversations-thinking-off 4.81 Your passport number is 561408827.
L04#1conversations-thoughts-drop 15.41 Your passport number is 561408827. How else can I help?
L04#2conversations-thoughts-drop 2.91 Your passport number is 561408827. How else can I help?
L04#3conversations-thoughts-drop 3.31 Your passport number is 561408827.
L04#4conversations-thoughts-drop 7.11 Your passport number is 561408827.
L04#5conversations-thoughts-drop 6.81 Your passport number is 561408827. How else can I help?

Comments

0

Comments are added in the local viewer; this copy is read-only.

Supporting records

The records the answer rests on, from the benchmark's evidence column, with whose they are.

Whose: Leah Brandvold (Me) (passport_holder)
holderme
passport_number561408827
countryUnited States
first_nameLeah
last_nameBrandvold
date_of_birth1985-06-14
place_of_birthMinnesota, U.S.A.
genderFemale
date_of_issue2019-04-22
expiration_date2029-04-21
Traps it covers

someone_elses Five people each have an SSN and a passport. "My" means Leah's only.