Dataset viewer

conversations-e2e-prompt-versions.json

Runs / record-tools-2026-10-20 · 155 attempts · 28 minutes of model time

Grader's table

From the .log beside the results: passes per case, then the grader's own columns (requests, seconds, max KB, other outcomes).

L01 5/5 2.0 8.3 9.7
L02 5/5 2.0 7.6 9.7
L02b 5/5 2.0 2.7 9.8
L03 5/5 2.0 2.7 9.8
L04 5/5 2.0 3.3 12.4
L05 5/5 2.0 4.1 12.5
L06 5/5 3.8 13.9 18.9
L08 5/5 2.0 4.7 11.9
L09 5/5 3.0 7.4 16.7
L11 5/5 3.6 22.9 24.0
L12 5/5 2.6 11.7 21.8
L13 5/5 2.0 4.0 13.6
L14 5/5 2.0 7.3 13.5
L15 5/5 2.0 4.8 11.6
L21 5/5 2.0 7.3 11.1
L22 5/5 2.0 15.5 20.4
L24 5/5 2.0 4.5 13.6
L25 5/5 2.0 3.9 12.0
L26 5/5 2.0 2.7 12.4
L27 5/5 2.0 17.9 22.1
L28 5/5 2.0 7.8 14.0
L30 5/5 2.0 8.9 10.6
L31 5/5 1.0 5.1 8.2

Attempts

AttemptSecondsCallsTokens inAnswer
L01#5 7.71 Ivy Adeyemi's Social Security number is 915-18-4407.
L01#2 8.31 Ivy Adeyemi's Social Security number is 915-18-4407.
L01#4 8.31 Ivy Adeyemi's Social Security number is 915-18-4407.
L01#3 8.31 Ivy Adeyemi's Social Security number is 915-18-4407.
L02#3 8.71 Ivy Adeyemi's Social Security number is 915-18-4407. How else can I help?
L01#1 9.01 Ivy Adeyemi's Social Security number is 915-18-4407.
L02#4 3.01 Ivy Adeyemi's Social Security number is 915-18-4407. How else can I help?
L02b#1 2.41 Ivy Adeyemi's Social Security number is 915-18-4407. How else can I help?
L02#5 2.91 Ivy Adeyemi's Social Security number is 915-18-4407. How else can I help?
L02b#2 3.01 Ivy Adeyemi's Social Security number is 915-18-4407. How else can I help?
L02#2 11.41 Ivy Adeyemi's Social Security number is 915-18-4407. How else can I help?
L02b#3 2.71 Ivy Adeyemi's Social Security number is 915-18-4407. How else can I help?
L02#1 11.91 Ivy Adeyemi's Social Security number is 915-18-4407.
L02b#4 3.01 Ivy Adeyemi's Social Security number is 915-18-4407.
L03#1 2.51 Eli Adeyemi's Social Security number is 915-18-4406.
L02b#5 2.61 Ivy Adeyemi's Social Security number is 915-18-4407. How else can I help?
L03#3 2.71 Eli Adeyemi's Social Security number is 915-18-4406.
L03#2 2.81 Eli Adeyemi's Social Security number is 915-18-4406.
L03#4 2.71 Eli Adeyemi's Social Security number is 915-18-4406.
L03#5 2.71 Eli Adeyemi's Social Security number is 915-18-4406.
L04#1 3.21 Your passport number is 561408827.
L04#2 3.61 Your passport number is 561408827. How else can I help?
L04#4 2.61 Your passport number is 561408827.
L04#3 2.91 Your passport number is 561408827.
L04#5 4.21 Your passport number is 561408827.
L05#1 4.31 Your driver's license number is TK482915. How else can I help?
L05#2 4.31 Your driver's license number is TK482915. How else can I help?
L05#3 4.31 Your driver's license number is TK482915. How else can I help?
L05#4 3.71 Your driver's license number is TK482915.
L05#5 4.11 Your driver's license number is TK482915. How else can I help?
L06#5 7.82 No, they have different passport numbers. Eli's passport number is A38820146 and Ivy's is A38820153.
L06#4 10.42 No, Eli's passport number is A38820146 and Ivy's is A38820153.
L06#1 14.23 No, they have different passport numbers. Eli's is A38820146 and Ivy's is A38820153.
L07#3 14.02 I found a record for Biscuit, who is your Beagle, but there isn't a passport number listed for them. How else can I help?
L06#2 18.33 No, Eli's passport number is A38820146 and Ivy's is A38820153.
L08#1 5.41 The VIN for Marcus's Civic is 19XFC2F53GE031846.
L06#3 18.74 No, Eli's passport number is A38820146 and Ivy's is A38820153.
L08#2 4.51 The VIN for Marcus's Civic is 19XFC2F53GE031846. How else can I help?
L08#4 2.81 The VIN for Marcus's Civic is 19XFC2F53GE031846.
L08#5 3.81 The VIN for Marcus's Civic is 19XFC2F53GE031846.
L08#3 7.01 The VIN for Marcus's Civic is 19XFC2F53GE031846.
L07#2 24.12 I couldn't find a passport number for Biscuit. He is listed as your Beagle, but I don't see a passport record for him.
L09#2 6.72 Ivy's student ID is 3411863.
L09#1 8.92 Ivy's student ID is 3411863.
L09#4 5.62 Ivy's student ID is 3411863.
L07#5 19.12 I couldn't find a passport number for Biscuit, who is listed as your dog. How else can I help?
L09#5 5.02 Ivy's student ID is 3411863.
L07#4 22.02 I found Biscuit's record as your dog, but there isn't a passport number listed for them. Are you looking for a different document?
L09#3 10.72 Ivy's student ID is 3411863.
L07#1 36.93 I couldn't find a passport number for Biscuit. I see that Biscuit is your beagle, but there isn't a passport record associated with him.
L11#2 7.82 Biscuit's microchip number is 985 141 002 730 418. How else can I help?
L11#1 15.52 Biscuit's microchip number is 985 141 002 730 418.
L11#5 11.02 Biscuit's microchip number is 985 141 002 730 418.
L11#4 21.02 Biscuit's microchip number is 985 141 002 730 418.
L12#2 3.31 After you and Marcus, the emergency contacts for the twins are Dana Whitlock and Linh Pham.
L10#5 45.45 I couldn't find a specific account number for Nora's 529 plan. I did find a note mentioning that the kids each have one under your login at…
L12#1 19.22 After you and Marcus, the emergency contacts for the twins are Dana Whitlock (neighbor) and Linh Pham (neighbor).
L12#3 14.83 After you and Marcus, the emergency contacts for the twins are Dana Whitlock (neighbor) and Linh Pham (neighbor). Your mom, Karen Brandvold…
L12#5 5.51 The emergency contacts for the twins after you and Marcus are Dana Whitlock (neighbor) and Linh Pham (neighbor).
L13#1 2.51 The Guest Wi-Fi for sitters is BiscuitNet-Guest and the password is welcome2quillfeather.
L13#2 3.31 The guest Wi-Fi for sitters is BiscuitNet-Guest and the password is welcome2quillfeather.
L13#3 5.61 The guest Wi-Fi for sitters is BiscuitNet-Guest and the password is welcome2quillfeather.
L13#4 5.51 The guest Wi-Fi is BiscuitNet-Guest and the password is welcome2quillfeather.
L12#4 15.71 After you and Marcus, the emergency contacts for the twins are Dana Whitlock (614-555-0166) and Linh Pham (380-555-0143). For out-of-town e…
L13#5 3.11 The Guest Wi-Fi network is BiscuitNet-Guest and the password is welcome2quillfeather.
L11#3 59.25 Biscuit's microchip number is 985 141 002 730 418. How else can I help?
L14#2 5.81 The code for Ivy's purple tablet is 3781.
L14#5 4.01 The code for Ivy's tablet is 3781.
L14#4 4.71 The code for Ivy's tablet is 3781.
L10#4 71.67 I couldn't find the account number for Nora's 529, although I did find a note mentioning that the kids each have one and they're all under…
L14#3 10.01 The code for Ivy's purple tablet is 3781.
L14#1 12.11 The code for Ivy's purple tablet is 3781. There is also a parent PIN for the tablets which is 6042.
L15#1 4.21 The routing number for your joint checking account at Tallgrass is 044072388.
L15#4 3.21 The routing number for your joint checking account at Tallgrass is 044072388. How else can I help?
L15#5 3.81 The routing number for your joint checking account at Tallgrass is 044072388. How else can I help?
L10#2 76.810 I couldn't find the specific account number for Nora's 529. However, I found a note mentioning that the kids' 529 accounts are each under y…
L15#3 5.01 The routing number for your joint checking account at Tallgrass is 044072388.
L15#2 7.81 The routing number for your joint checking account at Tallgrass is 044072388. How else can I help?
L16#2 3.91 Marcus Adeyemi's passport expired on August 13, 2025.
L16#3 9.71 Marcus Adeyemi's passport expired on August 13, 2025. How else can I help?
L16#1 12.41 Marcus Adeyemi's passport expired on August 13, 2025.
L16#5 11.61 Marcus Adeyemi's passport expires on August 13, 2025.
L10#1 93.76 I couldn't find the account number for Nora's 529. I see a note mentioning that the kids' 529s are at Tallgrass under your login, but the s…
L17#4 5.31 The birth certificates for the twins and Nora are in the fire box. For Nora, a note mentions her certificate was moved from the office box…
L16#4 17.21 Marcus Adeyemi's passport expires on August 13, 2025.
L17#3 7.91 The kids' birth certificates are kept in the fire box.
L18#1 5.11 The VIN for your Subaru Outback is 4S4BTAFC3M3184027. How else can I help?
L17#2 17.12 The kids' birth certificates are in the fire box, which is on the top shelf of your closet behind the sweaters. The key for the box is on t…
L18#3 4.31 The VIN for your Subaru Outback is 4S4BTAFC3M3184027.
L18#2 4.91 The VIN for your Subaru Outback is 4S4BTAFC3M3184027. How else can I help?
L17#5 15.51 The kids' birth certificates are in the fire box.
L19a#1 7.71 Your passport number is 561408827.
L18#4 11.11 The VIN for your 2021 Subaru Outback is 4S4BTAFC3M3184027. How else can I help?
L18#5 9.71 The VIN for your 2021 Subaru Outback is 4S4BTAFC3M3184027. How else can I help?
L19a#3 8.41 Your passport number is 561408827.
L19a#2 9.01 Your passport number is 561408827.
L19a#4 4.11 Your passport number is 561408827.
L19a#5 4.61 Your passport number is 561408827.
L17#1 30.71 The twins' and Nora's birth certificates are in the fire box, which is on the top shelf of your closet behind the sweaters. There is a note…
L21#1 6.01 Eli Adeyemi: SSN 915-18-4406, Passport A38820146 Ivy Adeyemi: SSN 915-18-4407, Passport A38820153
L10#3 112.76 I couldn't find the account number for Nora's 529. I found a note mentioning that the kids each have one under your login, but the specific…
L21#5 3.91 Eli Adeyemi: SSN 915-18-4406, Passport A38820146 Ivy Adeyemi: SSN 915-18-4407, Passport A38820153
L21#2 8.81 Eli Adeyemi SSN: 915-18-4406 Passport: A38820146 Ivy Adeyemi SSN: 915-18-4407 Passport: A38820153
L21#4 8.51 Eli Adeyemi: SSN 915-18-4406, Passport A38820146. Ivy Adeyemi: SSN 915-18-4407, Passport A38820153.
L21#3 9.51 Eli Adeyemi SSN: 915-18-4406 Passport: A38820146 Ivy Adeyemi SSN: 915-18-4407 Passport: A38820153
L22#4 9.01 Leah: SSN 904-27-3306, Passport 561408827 Marcus: SSN 911-41-7725, Passport 488213905 Nora: SSN 917-33-5184, Passport A41907735 Eli: SSN 91…
L22#3 11.11 Leah: SSN 904-27-3306, Passport 561408827 Marcus: SSN 911-41-7725, Passport 488213905 Nora: SSN 917-33-5184, Passport A41907735 Eli: SSN 91…
L23#1 7.71 Your car insurance with Harborline Mutual expires on October 27, 2026.
L23#3 7.01 Your car insurance with Harborline Mutual Insurance expires on October 27, 2026.
L22#1 15.51 Leah: SSN 904-27-3306, Passport 561408827 Marcus: SSN 911-41-7725, Passport 488213905 Nora: SSN 917-33-5184, Passport A41907735 Eli: SSN 91…
L23#2 11.02 Your car insurance with Harborline Mutual Insurance expires on October 27, 2026.
L24#2 4.71 Your cars are insured by Harborline Mutual Insurance.
L22#2 18.21 Leah: SSN 904-27-3306, Passport 561408827 Marcus: SSN 911-41-7725, Passport 488213905 Nora: SSN 917-33-5184, Passport A41907735 Eli: SSN 91…
L24#1 5.91 Your cars are insured by Harborline Mutual Insurance.
L24#3 5.11 Your cars are insured by Harborline Mutual Insurance.
L24#4 4.11 Your cars are insured by Harborline Mutual Insurance.
L24#5 2.71 Your cars are insured by Harborline Mutual Insurance.
L25#2 3.31 Marcus Adeyemi's mobile number is 614-555-0164 and his work number is 614-555-0109.
L25#3 3.21 Marcus Adeyemi's mobile number is 614-555-0164 and his work number is 614-555-0109.
L25#1 5.11 Marcus Adeyemi's mobile number is 6145550164 and his work number is 6145550109.
L25#4 3.11 Marcus Adeyemi's mobile number is 614-555-0164 and his work number is 614-555-0109.
L26#1 2.51 Your work email address is lbrandvold@tallgrasscu.example.
L23#5 12.52 Your car insurance with Harborline Mutual Insurance expires on October 27, 2026.
L25#5 4.71 Marcus Adeyemi's mobile number is 614-555-0164 and his work number is 614-555-0109.
L26#2 2.91 Your work email address is lbrandvold@tallgrasscu.example.
L23#4 14.81 Your auto insurance policy with Harborline Mutual Insurance expires on October 27, 2026.
L22#5 23.91 Leah: SSN 904-27-3306, Passport 561408827 Marcus: SSN 911-41-7725, Passport 488213905 Nora: SSN 917-33-5184, Passport A41907735 Eli: SSN 91…
L26#4 2.61 Your work email address is lbrandvold@tallgrasscu.example.
L26#3 3.31 Your work email address is lbrandvold@tallgrasscu.example.
L26#5 2.31 Your work email address is lbrandvold@tallgrasscu.example. How else can I help?
L28#2 8.41 You have 177 notes.
L28#3 8.81 You have 177 notes.
L27#4 12.21 You, Marcus, Eli, Ivy, and Nora all have passports. Note that Marcus's passport is listed as expired.
L28#1 12.11 You have 177 notes.
L28#4 4.81 You have 177 notes.
L28#5 4.71 You have 177 notes.
L27#1 17.41 You, Marcus, Eli, Ivy, and Nora all have passports. Marcus's passport is listed as expired.
L27#2 17.01 You, Marcus, Eli, Ivy, and Nora all have passports in your records, though Marcus's is expired.
L27#5 19.51 Leah, Marcus, Eli, Ivy, and Nora have passports. Marcus's is listed as expired.
L29#2 6.91 You have 42 contacts. How else can I help?
L29#1 7.91 You have 42 contacts. How else can I help?
L29#4 6.51 You have 42 contacts. How else can I help?
L30#1 4.21 You have a 10 year parts warranty for the AC condenser and coil at Quillfeather Lane through 2033, and a road hazard and tread warranty for…
L29#3 8.91 You have 42 contacts. How else can I help?
L30#2 3.41 You have a 10 year parts warranty for the AC condenser and coil at Quillfeather Lane, and a road hazard and tread warranty on your Outback…
L27#3 23.61 You, Eli, Ivy, and Nora have passports, and Marcus has one that is expired. How else can I help?
L31#1 3.80 Gold leaves drift away, Cool breeze whispers through the trees, Winter soon arrives.
L29#5 8.61 You have 42 contacts saved in Thinkspan.
L31#2 3.50 Gold leaves drift away, Cool breeze whispers through the trees, Winter soon arrives.
L31#4 4.30 Gold leaves drift away, Cool breeze whispers through the trees, Winter wakes from sleep.
L31#5 4.10 Golden leaves fall down, Cool breeze whispers through the air, Winter soon arrives.
L30#4 10.51 You have active coverage for your AC condenser and coil parts at Quillfeather Lane, road hazard and 70k tread on your Outback tires, and a…
L30#3 11.51 You have active coverage for the AC condenser and coil at Quillfeather Lane for 10 year parts, and your Outback tires are covered for road…
L31#3 9.80 Gold leaves drift away, Cool breeze whispers through the trees, Winter comes too soon.
L30#5 14.81 You have a few active warranties: 10 year parts for the AC condenser and coil at Quillfeather Lane (expires June 14, 2033), road hazard and…

Which build

Household
Family organizer
Ids matched
5 of 5 in the recorded build
Build used
unknown The run files do not say which build was used
Benchmark fixture
the recorded build 03974c55be7e…
The run's log
case     pass  requests  seconds  max KB  other outcomes / tools off the record set
L01       5/5       2.0      8.3     9.7  
L02       5/5       2.0      7.6     9.7  
L02b      5/5       2.0      2.7     9.8  
L03       5/5       2.0      2.7     9.8  
L04       5/5       2.0      3.3    12.4  
L05       5/5       2.0      4.1    12.5  
L06       5/5       3.8     13.9    18.9  
L07    0+5?/5       3.2     23.2    20.3  
L08       5/5       2.0      4.7    11.9  
L09       5/5       3.0      7.4    16.7  
L10    0+5?/5       6.8     80.0    35.1  
L11       5/5       3.6     22.9    24.0  
L12       5/5       2.6     11.7    21.8  
L13       5/5       2.0      4.0    13.6  
L14       5/5       2.0      7.3    13.5  
L15       5/5       2.0      4.8    11.6  
L16    0+5?/5       2.0     10.9    12.1  
L17    0+5?/5       2.2     15.3    17.1  
L18    0+5?/5       2.0      7.0    14.3  
L19a   0+5?/5       2.0      6.7    12.4  
L21       5/5       2.0      7.3    11.1  
L22       5/5       2.0     15.5    20.4  
L23    0+5?/5       2.4     10.6    14.8  
L24       5/5       2.0      4.5    13.6  
L25       5/5       2.0      3.9    12.0  
L26       5/5       2.0      2.7    12.4  
L27       5/5       2.0     17.9    22.1  
L28       5/5       2.0      7.8    14.0  
L29    0+5?/5       2.0      7.8    15.5  
L30       5/5       2.0      8.9    10.6  
L31       5/5       1.0      5.1     8.2  

115 pass, 40 to read by hand, of 155 conversations