conversations-staging-deployed-old-build.json
Grader's table
From the .log beside the results: passes per case, then the grader's own columns (requests, seconds, max KB, other outcomes).
| L01 | 0/1 | 4.0 26.8 0.0 missing 915-18-4407 [graph_find, graph_read] |
| L02 | 0/1 | 4.0 29.4 0.0 missing 915-18-4407 [graph_find, graph_read] |
| L02b | 0/1 | 4.0 26.7 0.0 missing 915-18-4407 [graph_find, graph_read] |
| L03 | 0/1 | 4.0 28.9 0.0 missing 915-18-4406 [graph_find, graph_read] |
| L04 | 1/1 | 5.0 29.3 0.0 [graph_find, graph_overview, graph_read] |
| L05 | 1/1 | 6.0 36.7 0.0 [graph_find, graph_overview, graph_read] |
| L06 | 0/1 | 8.0 57.0 0.0 failure: The lookup could not be finished. [graph_find, graph_overview, graph_read] |
| L08 | 1/1 | 5.0 24.8 0.0 [graph_find, graph_overview, graph_read] |
| L09 | 1/1 | 4.0 21.2 0.0 [graph_find, graph_read] |
| L10 | 0/1 | 8.0 46.7 0.0 failure: The lookup could not be finished. [graph_find, graph_overview, graph_read] |
| L11 | 0/1 | 7.0 46.6 0.0 missing 985 141 002 730 418 [graph_find, graph_overview, graph_read] |
| L12 | 1/1 | 8.0 49.3 0.0 [graph_find, graph_read] |
| L13 | 1/1 | 5.0 36.0 0.0 [graph_find, graph_overview, graph_read] |
| L14 | 1/1 | 5.0 35.8 0.0 [graph_find, graph_read] |
| L15 | 1/1 | 4.0 23.9 0.0 [graph_find, graph_overview] |
| L21 | 0/1 | 6.0 37.8 0.0 missing 915-18-4407 [graph_find, graph_read] |
| L22 | 0/1 | 1.0 11.3 0.0 failure: The lookup could not be finished. |
| L24 | 1/1 | 4.0 24.2 0.0 [graph_find, graph_overview, graph_read] |
| L25 | 0/1 | 4.0 24.1 0.0 missing 614-555-0164 [graph_read] |
| L26 | 0/1 | 7.0 43.7 0.0 missing lbrandvold@tallgrasscu.example [graph_find, graph_overview, graph_read] |
| L27 | 1/1 | 3.0 15.3 0.0 [graph_find, graph_read] |
| L28 | 1/1 | 2.0 12.9 0.0 [graph_overview] |
| L30 | 0/1 | 4.0 14.8 0.0 missing mattress [graph_find, graph_overview, graph_read] |
| L31 | 1/1 | 1.0 1.1 0.0 |
Attempts
| Attempt | Seconds | Calls | Tokens in | Answer |
|---|---|---|---|---|
| L02b#1 | 26.7 | 3 | I couldn't find a Social Security number for Ivy Adeyemi. I found her birth certificate, but it doesn't list an SSN. | |
| L01#1 | 26.8 | 3 | I couldn't find a social security number for Ivy in your records. You might want to check the birth certificate picture she is linked to in… | |
| L03#1 | 28.9 | 3 | I can see that there is a social security record for Eli Adeyemi, but the actual number isn't listed in the text fields. You might find it… | |
| L04#1 | 29.3 | 4 | Your passport number is 561408827. How else can I help? | |
| L02#1 | 29.4 | 3 | I couldn't find Ivy's social security number in her profile or birth certificate. You might want to check any other documents you have save… | |
| L05#1 | 36.7 | 5 | Your driver's license number is TK482915. How else can I help? | |
| L07#1 | 36.7 | 5 | I found that Biscuit is your dog, but I couldn't find a passport number for him in your records. How else can I help? | |
| L09#1 | 21.2 | 3 | Ivy's student ID is 3411863. How else can I help? | |
| L08#1 | 24.8 | 4 | The VIN for Marcus's Civic is 19XFC2F53GE031846. How else can I help? | |
| L06#1 | 57.0 | 8 | The lookup could not be finished. | |
| L14#1 | 35.8 | 4 | The code for Ivy's purple tablet is 3781. | |
| L13#1 | 36.0 | 5 | The Guest Wi-Fi for the sitter is BiscuitNet-Guest and the password is welcome2quillfeather. | |
| L15#1 | 23.9 | 4 | The routing number for your Tallgrass joint checking account is 044072388. How else can I help? | |
| L10#1 | 46.7 | 8 | The lookup could not be finished. | |
| L11#1 | 46.6 | 6 | I couldn't find a microchip number in Biscuit's records. You might want to check his adoption papers or vet files in Thinkspan. | |
| L16#1 | 24.0 | 3 | I couldn't find an expiration date for Marcus's passport. I did find a note from September 2026 mentioning that he was renewing it by mail… | |
| L12#1 | 49.3 | 7 | After you and Marcus, the emergency contacts for the twins are Dana Whitlock (neighbor) and Linh Pham (neighbor). | |
| L22#1 | 11.3 | 0 | The lookup could not be finished. | |
| L19a#1 | 14.0 | 3 | Your passport number is 561408827. How else can I help? | |
| L18#1 | 14.5 | 3 | The VIN for your Subaru Outback is 4S4BTAFC3M3184027. | |
| L17#1 | 31.4 | 4 | I found birth certificates for Nora, Ivy, and Eli in Thinkspan, but there is no location listed for them. You might want to check the photo… | |
| L24#1 | 24.2 | 3 | Your cars are insured by Harborline Mutual Insurance. | |
| L28#1 | 12.9 | 1 | You have 177 notes. How else can I help? | |
| L29#1 | 12.5 | 1 | You have 42 contacts saved in Thinkspan. How else can I help? | |
| L23#1 | 25.5 | 3 | You have two auto insurance policies listed for your Outback. One expires on October 25, 2025, and another expires on October 27, 2026. | |
| L31#1 | 1.1 | 0 | Gold leaves drift away, Cool winds whisper through the trees, Winter wakes from sleep. | |
| L25#1 | 24.1 | 3 | Tunde Adeyemi's mobile number is 614-555-0193. | |
| L27#1 | 15.3 | 2 | Based on your records, Ivy Adeyemi has a passport. You also have records for passports for Eli, Marcus, and Nora Adeyemi. | |
| L21#1 | 37.8 | 7 | Eli: SSN 915-18-4406, Passport A38820146 Ivy: Passport A38820153 (no SSN found) | |
| L30#1 | 14.8 | 3 | You have several warranties saved: For your Outback, you have powertrain coverage (5 yr / 60,000 mi) through July 10, 2026, and road hazard… | |
| L26#1 | 43.7 | 6 | I couldn't find a work email address saved in your profile or your records. You might want to check if it is stored in a different document… |
Which build
- Household
- Family organizer
- Ids matched
- 54 of 54 in the recorded build
- Build used
- unknown The run files do not say which build was used
- Benchmark fixture
- the recorded build 03974c55be7e…
The run's log
case pass requests seconds max KB other outcomes / tools off the record set L01 0/1 4.0 26.8 0.0 missing 915-18-4407 [graph_find, graph_read] L02 0/1 4.0 29.4 0.0 missing 915-18-4407 [graph_find, graph_read] L02b 0/1 4.0 26.7 0.0 missing 915-18-4407 [graph_find, graph_read] L03 0/1 4.0 28.9 0.0 missing 915-18-4406 [graph_find, graph_read] L04 1/1 5.0 29.3 0.0 [graph_find, graph_overview, graph_read] L05 1/1 6.0 36.7 0.0 [graph_find, graph_overview, graph_read] L06 0/1 8.0 57.0 0.0 failure: The lookup could not be finished. [graph_find, graph_overview, graph_read] L07 0+1?/1 6.0 36.7 0.0 [graph_find, graph_overview, graph_read] L08 1/1 5.0 24.8 0.0 [graph_find, graph_overview, graph_read] L09 1/1 4.0 21.2 0.0 [graph_find, graph_read] L10 0/1 8.0 46.7 0.0 failure: The lookup could not be finished. [graph_find, graph_overview, graph_read] L11 0/1 7.0 46.6 0.0 missing 985 141 002 730 418 [graph_find, graph_overview, graph_read] L12 1/1 8.0 49.3 0.0 [graph_find, graph_read] L13 1/1 5.0 36.0 0.0 [graph_find, graph_overview, graph_read] L14 1/1 5.0 35.8 0.0 [graph_find, graph_read] L15 1/1 4.0 23.9 0.0 [graph_find, graph_overview] L16 0+1?/1 4.0 24.0 0.0 [graph_find, graph_read] L17 0+1?/1 5.0 31.4 0.0 [graph_find, graph_overview, graph_read] L18 0+1?/1 4.0 14.5 0.0 [graph_find, graph_overview, graph_read] L19a 0+1?/1 4.0 14.0 0.0 [graph_find, graph_overview, graph_read] L21 0/1 6.0 37.8 0.0 missing 915-18-4407 [graph_find, graph_read] L22 0/1 1.0 11.3 0.0 failure: The lookup could not be finished. L23 0+1?/1 4.0 25.5 0.0 [graph_find, graph_overview, graph_read] L24 1/1 4.0 24.2 0.0 [graph_find, graph_overview, graph_read] L25 0/1 4.0 24.1 0.0 missing 614-555-0164 [graph_read] L26 0/1 7.0 43.7 0.0 missing lbrandvold@tallgrasscu.example [graph_find, graph_overview, graph_read] L27 1/1 3.0 15.3 0.0 [graph_find, graph_read] L28 1/1 2.0 12.9 0.0 [graph_overview] L29 0+1?/1 2.0 12.5 0.0 [graph_overview] L30 0/1 4.0 14.8 0.0 missing mattress [graph_find, graph_overview, graph_read] L31 1/1 1.0 1.1 0.0 12 pass, 7 to read by hand, of 31 conversations