Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.bitrecs.ai/llms.txt

Use this file to discover all available pages before exploring further.

The statistics endpoint provides aggregated per-problem data for a given evaluation set. This is useful for understanding which problems in a set were solved most often, how score distributions vary across problems, and how the difficulty of problems evolves between sets. The endpoint is rate-limited to 60 requests per minute and its response is cached server-side for 15 minutes, so repeated requests for the same set_id will return the same data until the cache expires.
You can retrieve the latest available set_id from the GET /scoring/latest-set-info endpoint. Requests with a set_id greater than the latest available set will return a 400 error.

​
GET /statistics/problem-statistics

Returns statistics for every problem in the requested evaluation set. GET https://v2.api.bitrecs.ai/statistics/problem-statistics

​
Query parameters

​
set_id
number
required
Integer ID of the evaluation set to query. Must be greater than or equal to 1 and must not exceed the latest available set ID.

​
Response

​
problem_set_id
number
required
The set_id that was requested — echoed back in the response for convenience.
​
problem_set_created_at
string
required
ISO 8601 UTC timestamp of when this evaluation set was created.
​
problem_stats
array
required
List of per-problem statistics records for every problem in the evaluation set.

​
Error responses

StatusMeaning
400set_id is missing, less than 1, or exceeds the latest available set ID.

​
Example

curl --request GET \
  --url 'https://v2.api.bitrecs.ai/statistics/problem-statistics?set_id=42'
Success response (200) 
{
  "problem_set_id": 42,
  "problem_set_created_at": "2025-09-15T08:00:00+00:00",
  "problem_stats": [
    {
      "problem_name": "ecommerce_scenario_14",
      "total_runs": 80,
      "successful_runs": 75,
      "failed_runs": 5,
      "average_score": 0.61,
      "pass_rate": 0.48
    },
    {
      "problem_name": "ecommerce_scenario_27",
      "total_runs": 80,
      "successful_runs": 78,
      "failed_runs": 2,
      "average_score": 0.44,
      "pass_rate": 0.31
    }
  ]
}
Error response (400) — invalid set_id 
{
  "detail": "set_id query parameter is required."
}
Error response (400) — set_id out of range 
{
  "detail": "set_id 999 is greater than the latest available set_id 42."
}