orlixai-bench-2026-06-24
Three high-severity OrlixAI findings across governance, memory, and CLI secret storage
AntFleet ran three focused bench reviews against tylerbroqs/orlixai via AntFleet/bench-orlixai. Claude and OpenAI unanimously agreed on four findings: three high-severity issues and one medium-severity concurrency issue. A consolidated upstream fix PR is open at https://github.com/tylerbroqs/orlixai/pull/13.
- HIGH · bug: Decision.id overwritten with non-unique value causes approval collisions and silent drops.
- HIGH · data-loss: Memory corrupt JSON load path resets to an empty store and a later save can overwrite user data.
- HIGH · security: setup stores API keys in
~/.orlix/config.jsonwithout restrictive file permissions. - MEDIUM · concurrency: AuditLog read-modify-write updates can lose receipts under concurrent writers.
Evidence
Bench PR review comments:
- Governance/policy: https://github.com/AntFleet/bench-orlixai/pull/1#issuecomment-4785789405
- Memory/audit: https://github.com/AntFleet/bench-orlixai/pull/2#issuecomment-4785801462
- CLI/API: https://github.com/AntFleet/bench-orlixai/pull/3#issuecomment-4785821502
Upstream fix PR: https://github.com/tylerbroqs/orlixai/pull/13 Methodology mirror: https://github.com/AntFleet/bench-orlixai