Jev report update: testing Laya locally
The Jev report is now at version 0.19.0, in French and English. The September 26 release added results from my local Laya experiments and an installation walkthrough for Apple Silicon Macs.
Laya ran offline after downloading the software and model weights, without an external AI API. The filter’s warm median was 8.55 ms on my M5 Max, excluding loading and browser time. Speed was encouraging; quality needs more work. On 20 synthetic triage examples, Laya got 14 answers right, compared with 15 for simple lexical rules.
The update includes the synthetic inputs and predictions, errors observed in the browser, limitations of the measurements and the next tests to run. These are Laya results, not a direct benchmark against Jev. The report marks the latest addition New and revised chapters Updated.
September 27, second update: Sentry, review and CI POCs are complete. On 120 held-out cases, Laya’s suggestions match provisional annotations at 2/38, 1/9 and 6/20 respectively. CI rules reach 5/6. Both matching Sentry suggestions were already explicitly linked; CI gains 1 matching suggestion in total with 13 additional wrong ones. The 3 tested configurations are rejected. No human validation; 15 inputs exceed context. Read all 3 POCs.
September 27 update: the Aristote issue pilot is now included. On 60 test tickets, Laya emits 11 area suggestions, 8 matching provisional annotations, versus 17 and 12 for the current title-only rules. It leaves 49 tickets without suggestions, including 8 context overflows. The annotations come from 2 agents, without human validation. The tested configuration is rejected; keep the current rules. A separate lexical variant reading title and body reaches 40 matching suggestions out of 51, with 11 errors; it is not deployed. Read the pilot.
- Read the English report or go straight to the local results and installation walkthrough.
- Lire le dossier en français, les résultats de nos essais et le tuto d’installation.
Mise à jour du 27 septembre : le dossier ajoute le pilote sur les issues Aristote. Sur 60 tickets, Laya émet 11 suggestions, dont 8 conformes aux annotations provisoires, contre 17 et 12 pour les règles actuelles. La configuration est écartée pour ce tri. Les annotations de 2 agents restent à valider humainement. Les essais synthétiques et le tuto hors ligne sont conservés séparément.
Deuxième mise à jour du 27 septembre : les 3 POCs Sentry, reviews et CI sont terminés. Suggestions conformes aux annotations provisoires : 2/38, 1/9 et 6/20 ; les règles CI atteignent 5/6. Les configurations testées sont écartées. Les 15 dépassements restent dans le bilan et aucune validation humaine n’est revendiquée.