Write Instructions

This is an optional step. If your labelers are already well familiarized with the task at hand, feel free to hit the Save button and skip this step.

If you are teaching a new team of labelers how to complete a task, writing these instructions will help your labelers understand how they should be labeling the tasks. Thorough instructions will result in quality data because everything will be laid out clearly for the taskers to follow.

Instructions should be thorough and be clear enough for taskers to follow

Good instructions will include:

• The overall goal of the task
• Definitions of all labels and attributes
• Sufficient examples that are representative of the overall dataset
• Key concepts the tasker should understand
• Any nuances or special cases labelers might encounter

How to Set Up Instructions

You can either write your instructions directly in the instructions editor, or if you have pre-written instructions that exist elsewhere, you can upload a PDF (“Upload an image” or "Upload a PDF") or a separate URL (“Embed a link”). Note that instruction writing is an iterative process. We’ve built in a feedback system (via Calibration batches) for your annotators to give you input on your task instructions.

Writing Instructions Natively: If you choose to use the default instructions editor, it will be pre-populated with a template based on the taxonomy you've created as well as any templating you've selected.

Using the native editor, you can even specify very tactical examples of what "good" or "bad" labels look like. This is available for "Label", "Attribute", and "Rule" blocks.

Embedding Pre-Written Instructions: If you have pre-written instructions that exist elsewhere, you can upload a PDF (“image or PDF”) or a google doc (“Embed a link").