DocumentationAPI ReferenceCreate an Account
Connect with Sales

Task Audits

When a task has been completed, it can be audited. The outcome of an audit can be that the task is one of accepted, rejected, or fixed. Fixing tasks is only available for certain task types and plans.

We have information about the auditing workflow inside the web application in our documentation.

Audit Example

{
  "task_id": "576ba74eec471ff9b01557cc",
  "created_at": "2021-06-22T09:09:34.752Z",
  "completed_at": "2021-06-23T09:10:02.798Z",  
  "type": "imageannotation",
  "status": "completed",
  "params": {
    ...
  },
  "response": {
    "category": "big"
  },
  "customer_review_status": "fixed",
  "customer_review_comments": "Was small, actually is big",
  "prior_responses": [
    {
      "category": "small"
    }
  ],
  "audits": [
    {
      "audited_by": "[email protected]",
      "audited_at": "2021-06-24T15:32:03.585Z",
      "audit_time_secs": 120,
      "audit_result": "accepted",
      "audit_source": "customer"
    },
    {
      "audited_by": "[email protected]",
      "audited_at": "2021-06-23T10:01:02.352Z",
      "audit_time_secs": 511,
      "audit_result": "fixed",
      "audit_source": "customer"
    }
  ],
  ...
}

What happens after a Task Audit

When an audit is submitted, several new fields will be added to the task object:

Property

Type

Description

customer_review_status

string

Reflects the result of the audit as accepted, rejected or fixed

customer_review_comments

string

Reflects any comments saved during the audit

prior_responses

array

If a task is fixed, this will contain all prior, original responses.

If a task is fixed through multiple audits:
response: will be updated with the most recent audit
customer_review_status: will reflect the most recent audit
customer_review_comments: will append the comment to the array
prior_responses: will append previous responses to the array

The audits array

The audits field is an array of audit records. It is only available for tasks that have been completed and audited. Each record has the following properties:

Property

Type

Description

audited_by

string (email)

The person who performed the audit.

audited_at

timestamp

The time an audit is performed.

audit_time_secs

number

The total time, in seconds, the audit tab is in focus in a browser.

audit_active_time_secs

number

The total time, in seconds, when the audit tab is up in a browser, and there hasn't been any mouse or keyboard movements for over 60 seconds.

audit_result

string

The result of the audit. It is one of accepted, fixed, or rejected.

audit_source

string

The origin of the audit. It is either customer, or scale.

audit_per_annotation_errors

array, optional

If per annotation error auditing is enabled, the error array. More details in the next section.

The audit records are sorted by audited_at in descending order, in other words, the most recent audit is the first one in the array.

In most cases, the audit_time_secs and 'audit_active_time_secs' values should match, but in other cases, the audit_active_time_secs can be more precise, depending on one's workflows and how one wants to measure the audit time.

For audit_active_time_secs, after the 60 seconds threshold, the audit time would stop accumulating.

Per Annotation Errors

The audit_per_annotation_errors is an array of per annotation error records. It is only available for audits that have per annotation errors enabled. Each record has the following properties:

Property

Type

Description

uuid

string

The UUID of the annotation associated with the error.

severity

string

The severity of the error. It is either default or critical.

error_type

string

The type of the error. It is one of class, geometry, attribute, extraneous, missing, or other.

comment

string

The comment added to the error.

attribute

string, optional

For attribute errors, the erroneous attribute.

is_fixed

boolean, optional

True if the annotation is fixed with the audit.

original_label

string, optional

For class errors, the original label.

audit_label

string, optional

For class errors, the fixed audit label.

frame_num

number, optional

For video audits, the frame that the error appears on.

tagged_reasons

array, optional

The reasons tagged by the auditor.

original_box

Box, optional

For geometry errors, the original annotation's box. Fields include left, top, width, height, and rotation.

audit_box

Box, optional

For geometry errors, the fixed audit annotation's box. Fields include left, top, width, height, and rotation.

label_index

number, optional

For semantic segmentation audits, the index of the label.

pixel_location

array, optional

For semantic segmentation audits, the locations of the erroneous pixels.

is_without_annotation

boolean, optional

True if the error was tagged without drawing an annotation, so uuid doesn't correspond to an actual annotation.