Dataset viewer

conversations-deployed-build-prompt-versions.json

Runs / record-tools-2026-10-20 · 62 attempts · 26 minutes of model time

Grader's table

From the .log beside the results: passes per case, then the grader's own columns (requests, seconds, max KB, other outcomes).

L01 0/2 4.0 35.3 0.0 missing 915-18-4407 [graph_find, graph_read]
L02 0/2 4.0 35.4 0.0 missing 915-18-4407 [graph_find, graph_read]
L02b 0/2 4.0 35.3 0.0 missing 915-18-4407 [graph_find, graph_read]
L03 2/2 4.0 35.2 0.0 [graph_find, graph_read]
L04 1/2 5.0 20.6 0.0 missing 561408827 [graph_find, graph_overview, graph_read]
L05 2/2 5.0 22.0 0.0 [graph_find, graph_overview, graph_read]
L06 0/2 8.0 56.9 0.0 failure: The lookup could not be finished.; missing A38820146 [graph_find, graph_overview, graph_read]
L08 2/2 5.0 30.1 0.0 [graph_find, graph_overview, graph_read]
L09 1/2 3.5 26.0 0.0 missing 3411863 [graph_find, graph_read]
L11 0/2 7.0 42.9 0.0 missing 985 141 002 730 418 [graph_find, graph_overview, graph_read]
L12 2/2 6.0 25.8 0.0 [graph_find, graph_overview, graph_read]
L13 2/2 5.0 20.8 0.0 [graph_find, graph_overview, graph_read]
L14 2/2 5.0 15.0 0.0 [graph_find, graph_read]
L15 2/2 4.5 12.3 0.0 [graph_find, graph_overview]
L21 0/2 6.5 53.1 0.0 missing 915-18-4407 [graph_find, graph_overview, graph_read]
L22 0/2 1.0 9.5 0.0 failure: The lookup could not be finished.
L24 2/2 4.5 22.3 0.0 [graph_find, graph_overview, graph_read]
L25 0/2 4.0 17.7 0.0 missing 614-555-0164 [graph_read]
L26 1/2 5.0 30.0 0.0 missing lbrandvold@tallgrasscu.example [graph_find, graph_overview, graph_read]
L27 0/2 3.5 20.9 0.0 missing Marcus, Nora, Eli [graph_find, graph_read]
L28 2/2 2.0 11.8 0.0 [graph_overview]
L30 0/2 4.0 12.4 0.0 missing mattress [graph_find, graph_overview, graph_read]
L31 2/2 1.0 1.0 0.0

Attempts

AttemptSecondsCallsTokens inAnswer
L03#1 35.23 Eli Adeyemi's social security number is 915-18-4406. How else can I help?
L03#2 35.23 Eli Adeyemi's social security number is 915-18-4406. How else can I help?
L01#1 35.33 I couldn't find a social security number for Ivy in your records. You might want to check her birth certificate or other documents stored i…
L02b#1 35.33 I couldn't find a Social Security number for Ivy Adeyemi in her person record or her birth certificate. Is there anywhere else I should loo…
L01#2 35.33 I couldn't find Ivy's Social Security number in the records. I found her birth certificate and some other details, but the SSN isn't listed.
L02#1 35.43 I couldn't find a social security number for Ivy in her person record or her birth certificate. You might want to check any physical files…
L02b#2 35.43 I searched Ivy Adeyemi's records, including her birth certificate and associated notes, but I couldn't find a Social Security number saved…
L02#2 35.43 I checked Ivy's record and birth certificate, but I couldn't find her social security number. You might want to check if it is stored in a…
L04#2 14.23 I couldn't find a record of your passport number in Thinkspan. You might want to check if it is saved under a different name or uploaded as…
L05#2 21.84 Your driver's license number is TK482915. How else can I help?
L05#1 22.14 Your driver's license number is TK482915.
L07#1 22.24 Biscuit is a Beagle, so he doesn't have a passport number. Did you mean someone else?
L04#1 26.95 Your passport number is 561408827. How else can I help?
L07#2 27.35 I couldn't find a passport number for Biscuit in your records. I see that Biscuit is your Beagle, but there is no passport associated with…
L08#1 28.94 The VIN for Marcus's Civic is 19XFC2F53GE031846. How else can I help?
L09#2 20.82 I couldn't find a student ID for Ivy in your records. I see a note with school numbers for Nora and Eli, but Ivy isn't listed there.
L08#2 31.24 The VIN for Marcus's Civic is 19XFC2F53GE031846. How else can I help?
L09#1 31.23 Ivy's student ID is 3411863.
L06#2 54.27 No, Ivy and Nora have different passport numbers. Ivy's number is A38820153 and Nora's is A41907735.
L06#1 59.58 The lookup could not be finished.
L10#1 42.06 I couldn't find a specific account number for Nora's 529. I found a note mentioning that there is one for each child and they are all under…
L10#2 45.98 The lookup could not be finished.
L13#1 20.54 The Guest Wi-Fi network is BiscuitNet-Guest and the password is welcome2quillfeather.
L12#1 25.75 After you and Marcus, your first emergency contact for the twins is Dana Whitlock (neighbor) and then Linh Pham (neighbor). Out of town, yo…
L12#2 25.85 After you and Marcus, the emergency contacts for the twins are Dana Whitlock (neighbor) and Linh Pham (neighbor).
L13#2 21.16 The Guest Wi-Fi for the sitter is BiscuitNet-Guest and the password is welcome2quillfeather. How else can I help?
L11#1 40.76 I couldn't find Biscuit's microchip number in the records. There is a note mentioning adoption papers in the dog bin with the vet stuff, wh…
L14#1 15.14 The code for Ivy's purple tablet is 3781.
L15#1 12.64 The routing number for your joint checking account at Tallgrass is 044072388. How else can I help?
L11#2 45.17 I couldn't find a microchip number for Biscuit in your records. I did find a note that his adoption papers are in the dog bin with the vet…
L14#2 15.04 The code for Ivy's purple tablet is 3781.
L16#1 11.43 The lookup could not be finished.
L15#2 12.16 The routing number for your joint checking account at Tallgrass is 044072388. How else can I help?
L16#2 20.33 I couldn't find an expiration date for Marcus's passport, but there is a note from September 2026 mentioning that he was in the process of…
L18#1 14.43 The VIN for your Outback is 4S4BTAFC3M3184027. How else can I help?
L18#2 17.63 The VIN for your Outback is 4S4BTAFC3M3184027. How else can I help?
L19a#1 18.64 Your passport number is 561408827. How else can I help?
L17#1 28.54 You have birth certificates saved in Thinkspan for Nora, Ivy, and Eli. You can find them by searching for birth certificates in the app.
L19a#2 22.65 Your passport number is 561408827. How else can I help, Leah?
L22#1 11.30 The lookup could not be finished.
L22#2 7.70 The lookup could not be finished.
L17#2 29.76 I found Ivy's birth certificate stored in Thinkspan, which shows she was born at Scioto Riverside Hospital in Ohio. I didn't see any other…
L24#1 17.14 Your cars are insured by Harborline Mutual Insurance.
L23#1 23.34 You have two auto insurance policies for your Outback. One expires on October 25, 2025, and the other expires on October 27, 2026. How else…
L25#2 17.23 Tunde Adeyemis mobile number is 6145550193. How else can I help?
L23#2 18.54 You have two auto insurance policies for your Outback with Harborline Mutual Insurance. One expires on October 25, 2025, and the other expi…
L25#1 18.13 Tunde Adeyemi's mobile number is 614-555-0193.
L24#2 27.46 Your cars are insured by Harborline Mutual Insurance.
L28#1 16.11 You have 177 notes. How else can I help?
L27#1 17.02 Based on your records, Ivy Adeyemi is the family member who has a passport.
L28#2 7.51 You have 177 notes. How else can I help?
L21#1 57.711 Eli: SSN 915-18-4406, Passport A38820146 Ivy: SSN not found, Passport A38820153
L21#2 48.57 Eli Adeyemi: SSN 915-18-4406, Passport A38820146 Ivy Adeyemi: Passport A38820153 (SSN not found)
L29#2 2.11 You have 42 contacts. How else can I help?
L31#1 0.70 Gold leaves drift away, Cool winds whisper through the trees, Nature goes to sleep.
L26#2 20.83 I couldn't find a specific work email address in your profile, but I did find an email address saved as my email: leah.brandvold@postbox.ex…
L29#1 3.42 You have 42 contacts. How else can I help?
L31#2 1.40 Gold leaves drift away, Cool winds whisper through the trees, Nature goes to sleep.
L27#2 24.73 Based on your records, Ivy Adeyemi is the family member who has a passport. You also have a passport on file for yourself.
L30#2 11.83 ---p>You have several warranties: an Outback powertrain (5 years/60,000 miles) until July 2026, Outback tires for road hazard and 70k tread…
L30#1 13.03 You have several warranties saved: For your Outback: - Powertrain: 5 years or 60,000 miles, expiring July 10, 2026. - Tires: Road hazard an…
L26#1 39.35 Your work email address is lbrandvold@tallgrasscu.example.

Which build

Household
Family organizer
Ids matched
51 of 51 in the recorded build
Build used
unknown The run files do not say which build was used
Benchmark fixture
the recorded build 03974c55be7e…
The run's log
case     pass  requests  seconds  max KB  other outcomes / tools off the record set
L01       0/2       4.0     35.3     0.0  missing 915-18-4407  [graph_find, graph_read]
L02       0/2       4.0     35.4     0.0  missing 915-18-4407  [graph_find, graph_read]
L02b      0/2       4.0     35.3     0.0  missing 915-18-4407  [graph_find, graph_read]
L03       2/2       4.0     35.2     0.0    [graph_find, graph_read]
L04       1/2       5.0     20.6     0.0  missing 561408827  [graph_find, graph_overview, graph_read]
L05       2/2       5.0     22.0     0.0    [graph_find, graph_overview, graph_read]
L06       0/2       8.0     56.9     0.0  failure: The lookup could not be finished.; missing A38820146  [graph_find, graph_overview, graph_read]
L07    0+2?/2       5.5     24.8     0.0    [graph_find, graph_overview, graph_read]
L08       2/2       5.0     30.1     0.0    [graph_find, graph_overview, graph_read]
L09       1/2       3.5     26.0     0.0  missing 3411863  [graph_find, graph_read]
L10    0+1?/2       7.5     44.0     0.0  failure: The lookup could not be finished.  [graph_find, graph_overview, graph_read]
L11       0/2       7.0     42.9     0.0  missing 985 141 002 730 418  [graph_find, graph_overview, graph_read]
L12       2/2       6.0     25.8     0.0    [graph_find, graph_overview, graph_read]
L13       2/2       5.0     20.8     0.0    [graph_find, graph_overview, graph_read]
L14       2/2       5.0     15.0     0.0    [graph_find, graph_read]
L15       2/2       4.5     12.3     0.0    [graph_find, graph_overview]
L16    0+1?/2       4.0     15.8     0.0  failure: The lookup could not be finished.  [graph_find, graph_read]
L17    0+2?/2       6.0     29.1     0.0    [graph_find, graph_overview, graph_read]
L18    0+2?/2       4.0     16.0     0.0    [graph_find, graph_overview, graph_read]
L19a   0+2?/2       5.5     20.6     0.0    [graph_find, graph_overview, graph_read]
L21       0/2       6.5     53.1     0.0  missing 915-18-4407  [graph_find, graph_overview, graph_read]
L22       0/2       1.0      9.5     0.0  failure: The lookup could not be finished.
L23    0+2?/2       5.0     20.9     0.0    [graph_find, graph_overview, graph_read]
L24       2/2       4.5     22.3     0.0    [graph_find, graph_overview, graph_read]
L25       0/2       4.0     17.7     0.0  missing 614-555-0164  [graph_read]
L26       1/2       5.0     30.0     0.0  missing lbrandvold@tallgrasscu.example  [graph_find, graph_overview, graph_read]
L27       0/2       3.5     20.9     0.0  missing Marcus, Nora, Eli  [graph_find, graph_read]
L28       2/2       2.0     11.8     0.0    [graph_overview]
L29    0+2?/2       2.5      2.8     0.0    [graph_overview]
L30       0/2       4.0     12.4     0.0  missing mattress  [graph_find, graph_overview, graph_read]
L31       2/2       1.0      1.0     0.0  

23 pass, 14 to read by hand, of 62 conversations