Alibaba's Open Code Review Scores Higher on Precision and F1 in Benchmark

Alibaba's Open Code Review had 51,009 GitHub stars and 3,804 forks on Saturday, Nov. 14, after adding 897 stars in a single day. The open-source tool combines deterministic pipelines with a large language model to comment on specific lines of code.

A benchmark of 200 pull requests from 50 open-source repositories across 10 programming languages found higher precision and F1 scores for the tool than for general-purpose agents. It completed reviews faster while using about one-ninth as many tokens as general-purpose agents such as Claude Code. Its recall was lower, prioritizing precision to reduce false positives.

The benchmark dataset contained 1,505 annotated ground-truth issues and was cross-validated by more than 80 senior engineers. The test measured the tool's ability to flag code problems across multiple projects and languages.

Written primarily in Go, Open Code Review is compatible with models from OpenAI and Anthropic. Developers can use it to review pull-request changes or scan full files. Its built-in rules cover null pointer exceptions, thread safety, cross-site scripting and SQL injection.

The project uses the Apache License 2.0 and can be installed through npm. Its ocr review command handles pull-request reviews, while ocr scan checks full files.