Introduction to 3D LiDAR

📘
LIDAR annotation is available to Enterprise customers only.
If you’re interested in using Studio’s LIDAR annotation capabilities, please contact the team at [email protected].

Introduction

In this task, labelers view outputs from a series of LIDAR frames, along with optional radar and camera data, and annotate where different objects exist in the 3D space with 3D cuboids.

In this document, we’ll introduce you to the following concepts:

Tooling and Common Shortcuts Overview
Common Labeling Workflow

API documentation: https://docs.scale.com/reference/lidar-annotation-annotation

Tooling and Common Shortcuts Overview

You can find a list of all shortcuts with their definitions in the bottom-left of the tasking screen. Below, we explain the most commonly used shortcuts.

Tool / Term	Definition
WASD	If you have not selected a cuboid, these keys will move the camera: - W - Forward - A - Left - S - Back - D - Right If you have selected a cuboid, these keys will move the cuboid. Note, if you select shift + WASD, you'll notice that cuboid will move in smaller increments.
Arrow keys	If you have not selected a cuboid, the arrow keys will pan you around the scene. If you have selected a cuboid, these keys will change the heading of the cuboid.
plus (+) / minus (-)	These keys will move you forward (+) and backward (-) across frames.
T	Enter top down view. We generally suggest that labelers draw cuboids starting from top down view.
C	Toggle between camera view and LiDAR view. Note, you can also click on any of the image frames at the bottom of the screen to review a specific frame.
N	Cycle through cuboids when reviewing a task.
Cuboid Views	Select a cuboid to see different views (top, side, back etc.) This is particularly helpful when making minor sizing and positioning changes.
Point Density (aggregated point cloud, or APC)	Aggregate points from multiple frames into one frame. You can also increase point density: - Larger numbers: Greater density - Smaller numbers: Lower density This tool is helpful for: - Finding stationary (non-moving) objects - Finding dynamic tracks / moving objects - Correctly sizing and placing cuboids for stationary objects
Color by	Color by different features of the LiDAR scene, such as: - Cuboid ID - Label - Bottom Z - Attributes
Points >> Size	Increase the point size.
Points >> Brightness	Increase the brightness.
Points >> Intensity	Increase the point intensity.
Keyframes	These are the frames where you change one or many features of a cuboid. Keyframes mark the start and end of interpolated (predicted frames.)
Interpolation	This is a key tool to use for annotating moving objects since it will help you annotate them quickly and accurately via prediction. Note that the first keyframe you draw should be in the frame where you can see the object as clearly as possible - not necessarily the first or last frame. It is best to draw a keyframe every few frames, and to check the predicted frames to see that the cuboid follows the object well (see example below.)
Directional Heading	The direction that the object / cuboid is facing. To ensure your directional heading is correct, make sure the front of the object faces the top of the screen. The colored side should always correspond to the front of the object. You can adjust the directional heading in any of the following ways: 1. Drag and drop the heading in the Overhead LiDAR View in the cuboid detail panel 2. Use the Rotate 90 degree or Rotate 180 degrees buttons under the LiDAR Views in the cuboid detail panel 3. Use SHIFT with the Left or Right arrow keys 4. Click a cuboid and press the Left or Right arrow keys
Bottom Z Index or Bottom Z Coordinate	The location / height where the cuboid is placed. More important in some projects than others.

Please see visual examples below:

1626 — Use the WASD keys to navigate through the scene.

1324 — Press the + and - keys to move forward and backward across frames.

1621 — Press "T" to toggle topdown view. Press "C" to toggle between camera and lidar view.

3328 — Click on an image frame at the bottom of the screen to view a specific frame.

376 — Click a cuboid and review different angles to make minor sizing and positioning changes.

1625 — Change the "Cuboids" tool to color cuboids by different labels, attributes, z index etc.

1619 — Change the "points" to increase point size and density.

692 — Turn on APC to identify dynamic vs stationary objects. In this example, the purple cars and stationary whereas the green cars are dynamic (you can see the LiDAR points change position but the object's shape remains intact).

988 — Interpolation: the first step to use interpolation is to create a keyframe. To create a keyframe, drop a cuboid in topdown view (keyboard shortcut: "T.") Click on the cuboid, add the label and attributes. Finally, adjust the size and positioning in the side view.

988 — Interpolation: You'll notice that as you move to subsequent frames, the cuboid does not continue to follow the vehicle. You'll need to use interpolation to have the cuboid path follow the object of interest.

988 — Interpolation: Select "Interpolation" under "Status." Move ahead a few frames. Adjust the cuboid so that it is correctly sized / positioned around the object in future frames.

988 — Interpolation: If you need to refine the interpolation, create additional keyframes until you achieve a smooth path. Ensure you also have keyframes for the first and last frames. The dark green bars at the bottom represent the keyframes. The light green bars represent interpolated frames.

1474 — Directional Heading: Ensure the top of the cuboid aligns with the top of the object.

Common LiDAR Labeling Workflow

Carefully read the project instructions
Look at all camera views to better understand the scene
Review LiDAR view to gain additional scene context
Work on one object at a time, dropping cuboids in topdown view. In order to drop a cuboid, you'll need to click the mouse and then drag and drop over the object. Make sure that the heading of the cuboid aligns with the top of your screen.
Carefully set label and attributes (if applicable) in each frame. Size and position the cuboid accordingly.

Remember that by clicking on each cuboid, you can see different views (e.g. side, overhead) and adjust the cuboid(s) accordingly.
If you need to adjust the cuboid, you can use the WASD / Shift + WASD keys, arrow keys and Shift + arrow keys.
6. Use APC and move back and forth across the scene to check for overall quality. Make adjustments as needed.

847 — Go to topdown view to drop cuboids. Ensure the heading of the object corresponds to the top of your screen.

399 — Select the labels and attributes for each object.

409 — Adjust the cuboid size and position by reviewing the side viewer. After you've reviewed the task for overall quality, you are ready to submit it!