Callback Format

The response field, which is part of the callback POST request and permanently stored as part of the task object, will contain a link to a file containing a list of cuboid objects per frame. A Cuboid is defined by the center point of the cuboid, world-normalized, the dimensions of the cuboid, and the yaw / z-rotation of the cuboid indicating the front of the labeled object.

Visualization Examples


A visual representation of a cuboid


Scale follows the ENU (East, North, Up) world coordinate system, where 0 yaw is along the vector <0, 1, 0> and rotates in a counter-clockwise direction. See the yaw field for more details.

Enabling Pitch and Roll

To enable pitch and roll, specify the following in the JSON body used to create tasks:
"pitch_and_roll_enabled": true

This is a top-level field, at the same level of attachments and labels.

Definition: Cuboid

Each cuboid will have these properties:

[
  {
    "cuboids": [
      {
        "uuid": "4DBCC561-D6E1-4F1E-B862-054189939075",
        "camera_used": 1,
        "label": "Car",
        "position": {
          "x": 87.65,
          "y": 934.81,
          "z": 75.49
        },
        "dimensions": {
          "x": 175.85,
          "y": 399.34,
          "z": 150.99
        },
        "yaw": 1.63,
        "distance_to_device": 17.23,
        "numberOfPoints": 64,
        "stationary": false,
        "attributes": {
          "heading": "front"
        }
      },
      {
        "uuid": "0F81F763-A35C-4B11-9791-019F8CDA6524",
        "camera_used": 2,
        "label": "Car",
        "position": {
          "x": -532.97,
          "y": 920.32,
          "z": 69.83
        },
        "dimensions": {
          "x": 169.11,
          "y": 381.33,
          "z": 139.67
        },
        "yaw": 1.59,
        "distance_to_device": 14.68,
        "numberOfPoints": 61,
        "stationary": false,
        "attributes": {
          "heading": "back"
        }
      }
    ]
  },
  {
    "cuboids": [
      {
        "uuid": "4DBCC561-D6E1-4F1E-B862-054189939075",
        "camera_used": 1,
        "label": "Car",
        "position": {
          "x": -49.72,
          "y": 937.14,
          "z": 74.29
        },
        "dimensions": {
          "x": 175.85,
          "y": 399.34,
          "z": 150.99
        },
        "yaw": 1.63,
        "distance_to_device": 17.89,
        "numberOfPoints": 59,
        "stationary": false,
        "attributes": {
          "heading": "front"
        }
      },
      {
        "uuid": "0F81F763-A35C-4B11-9791-019F8CDA6524",
        "camera_used": 2,
        "label": "Car",
        "position": {
          "x": -678.94,
          "y": 929.31,
          "z": 70.10
        },
        "dimensions": {
          "x": 169.11,
          "y": 381.33,
          "z": 139.67
        },
        "yaw": 1.59,
        "distance_to_device": 15.21,
        "numberOfPoints": 57,
        "stationary": false,
        "attributes": {
          "heading": "back"
        }
      }
    ]
  }
]
FieldTypeDescription
uuidstringA computer-generated identifier for this cuboid, and will be used to correlate this object across different frames in the same series.
labelstringThe label passed in the request (e.g. car, pedestrian) that matches this cuboid.
camera_usedintegerThe index of the CameraImage that was used to annotate the cuboid at that frame, or null if the cuboid doesn't appear in any. Particularly in cases where the point cloud is sparse, the CameraImage is used to produce an accurate cuboid.
positionVector3The center of the cuboid.
dimensionsVector3The dimensions of the cuboid based on the world dimensions.
x: float, width of the cuboid from left to right
y: float, length of the cuboid from front to back
z: float, height of the cuboid from top to bottom
yawfloatradians from which the cuboid is rotated along the z-axis. 0 radians is equivalent to the direction of the vector <0, 1, 0>. The vector points at the length-side. Rotation happens counter-clockwise, i.e. PI/2 is pointing in the same direction as the vector <-1, 0, 0>.
distance_to_devicefloatThe 3d distance between the ego device and the centroid of the cuboid. The unit is the same unit of measure for the point cloud.
stationarybooleanwhether or not the cuboid is stationary
attributesobjectSee the Annotation Attributes section for more details about the attributes response field.

The annotations will be stored on S3 and be a JSON encoded list of cuboid objects per frame. The index of each cuboid list will correspond to the frame defined in the original response.

The order of the cuboid lists will correspond directly to the order of the frame attachments that you sent in your original request. That is, the first cuboid list will correspond to the first frame in the attachments array, etc.

Once the JSON file is reviewed and uploaded, we will send a response to your callback URL.

Definition: Polygon

[
  {
    "id": "6a1b4e87-4c12-9bc1-c185-112c24bece51",
    "label": "No Label Area",
    "points": [
      { "x": 1146.168562424232, "y": 8195.758645258815 },
      { "x": 1146.1051907749688, "y": 8195.40186608039 },
      { "x": 1145.776783493437, "y": 8195.47382667585 },
      { "x": 1145.8425839483823, "y": 8195.805531115831 },
      { "x": 1146.168562424232, "y": 8195.758645258815 }
    ],
    "visible_ranges": [
      { "start": 2, "end": 15 }
    ]
  }
]
ParameterTypeDescription
idstringA computer-generated identifier for this polygon, and will be used to correlate this object across different frames in the same series.
labelstringThe label passed in the request the matches this polygon.
pointsarrayAn array of objects that describe the points of the polygon. The first and last points have equal values (closed polygon).
visible_rangesarrayAn array of objects that describe the start and end frames when the polygon is visible. The start is inclusive and end is exclusive [start, end)

The polygon_annotations will be stored on S3 and be a JSON encoded list of polygon objects.

Polygon objects are stationary. Their positions are static, but visibility can be toggled on and off.