Reading the public results

Where are the public results?

The public results page lists Prilog's directly merged fixes and open contributions awaiting review or merge. It sits on the main website so developers, evaluators, and prospective users can reach the evidence without navigating setup documentation.

Search by repository or bug description, filter by language, or follow a card's permalink. Filter by merged or open status as well as repository and language. Every card links to the upstream GitHub pull request. All merged and open cards remain visible without JavaScript. Some PRs have not been classified by language and have no language badge.

What do the counts describe?

Counts describe only the curated records in the published snapshot. The collection began with Prilog's homepage showcase, then was expanded by reconciling both public campaigns: exports, early pilot audits, execution checkpoints, and GitHub author search. PR states and merge dates are checked against GitHub.

• Merged or open PRs: distinct currently merged/open pull requests shown as cards.
• Merged upstream: listed PRs whose GitHub metadata records a merge.
• Repositories with merged fixes: distinct upstream repositories with directly merged PRs, compared without case sensitivity.
• Languages: language labels attached to the showcased fixes, rather than a complete repository-language analysis.

The status summary distinguishes merged, open, closed-without-merge and unverified records across both reconciled campaigns. The JSON download retains every tracked record, including closed and inaccessible PRs; cards show merged and open contributions. Open means awaiting a maintainer outcome and may include reviews or requests for changes. Draft PRs are labeled separately and included in the open count. These campaigns are not a complete population of Prilog analysis attempts, so no overall success rate is calculated.

What has been verified?

The snapshot records a verification date for each PR. GitHub metadata establishes current state and merge timestamps as of that check. The code and review discussion can be inspected through the linked PR.

The collection does not independently establish generation time, CI completion, human edits, or how much of a contribution Prilog produced. A merge is not an endorsement by the upstream project. Public examples should be assessed alongside your own representative trials.

Download the evidence

• JSON snapshot: records, source URLs, dates, scope, and benchmark metadata.
• Markdown collection: readable plain text with all listed contributions.
• Three detailed examples: Keycloak, Prefect, and Newt.

A daily GitHub job checks every tracked PR and publishes a new snapshot when metadata changes. An open PR moves into the merged count only after GitHub reports it merged. Closed-without-merge outcomes remain in the downloadable records. The job rebuilds the page and indexes and starts the normal production deployment, so publication follows that rollout.

The page displays the last snapshot refresh date; successful checks with no changes are recorded by the daily job without republishing the site. Each JSON record preserves the date its state was verified. When a repository becomes inaccessible, the previous state is retained and marked unavailable rather than inferred to be rejected or merged. Initially inaccessible records remain unverified. API outages and rate limits do not publish partial results. Do not interpret snapshot or merge dates as patch generation time.

How does SWE-bench relate to these PRs?

The SWE-bench Verified evaluation is a separate self-reported, feedback-guided benchmark. Its 456/500 result is not the merge rate of the public collection, a pass@1 result, or an official leaderboard rank. Read the linked methodology and artifacts before making comparisons.

Evaluate your own issue

Follow the quickstart, OpenTelemetry workflow, or public GitHub issue workflow. Record unsuccessful attempts, tests actually run, reviewer effort, human changes, and cost per accepted fix.