Batch Overview

For high-volume projects, batches can optionally be used to further divide work inside a project. Batches can tie to specific datasets you use internally, or can be used to note which tasks were part of a weekly submission for example.

Typical Workflow

  1. Batch is created (status = in_progress)
  2. Tasks are added to the batch by specifying the batch field on the task to be the name of your batch
  3. The final task is completed, the batch is completed (status completed) and the callback is fired with the example to the right.

Self Serve Workflow

If you are participating in our "Self Serve" beta, batches take on another purpose, where they are deeply linked to the task delivery model.

  1. Batch is created (status = staging)
  2. Tasks are added to the batch by specifying the batch field on the task to be the name of your batch
  3. Batches need to be finalized (status = in_progress)
  4. Once a batch is finalized, tasks will be submitted to our taskers to begin labeling. No tasks may be added to a batch once it has been finalized.
  5. The final task is completed, the batch is completed (status completed) and the callback is fired with the example to the right.

🚧

Live vs. Testing Mode

The API key you use to create batches/projects/tasks with must all match: test projects can only be used with test batches, and test tasks; likewise with live projects, live batches, and live tasks.

{
    "batch": "kitten_labeling_2021-07", // the name of the batch that was just completed
    "status": "completed", // the status of the batch
    "stats": { // counts of the tasks of this batch, grouped by their status
        "completed": 20,
        "canceled": 3,
        "error": 5,
    }
}

Historical Note

Prior to June 2021, Batches had to be created, then tasks added, and then the batch had to be "finalized" before Scale could begin doing any work on the tasks.

After a careful evaluation and many support tickets later about why tasks weren't started or how to submit more tasks to a batch after it was accidentally closed, it was proposed that the extra complexity was no longer needed, and was actively hurting the customer developer experience as well as our operational efforts.

We made a transition that batches would start off in a workable state already, and there would be no need to finalize batches or check batch status prior to task submission. Our intention was to simplify integrations and operational components for everyone.