LiDAR · SLAM · Point-cloud reconstruction

Rebuilding the T1 full-density point cloud from raw sensor returns

The dense cloud didn't come out of the scanner as a file. It was reconstructed — every raw laser return re-projected through the scanner's own motion path. Here's every concept and every step, start to finish.

68.3M
valid returns re-projected into 3D
~12×
denser than the device's own export
1
quaternion bug that ate the night

Result is live at t1-potree.apps.sandytulloch.com

01 The instrument

FJDynamics "Trion Scan" S2 — a handheld SLAM laser scanner

Everything starts with the sensor. The S2 is a walk-around scanner: you carry it through a space and it builds a 3D model as you move. Four parts matter for the story.

motor spins the head 16 laser beams, tilted fan IMU (500 Hz) RTK GNSS (separate track) handle / pole
The tilted 16-beam Hesai XT16 rides on a motor that can rotate the whole head. An IMU measures rotation/acceleration; an RTK GNSS antenna logs a satellite position (kept as a separate track — it never touches the cloud).

02 What one laser return actually is

Polar measurement → a point in the sensor's own frame

A LiDAR doesn't measure points — it measures range along a direction. Each firing gives you one number, a distance r, for one beam. The direction is known from two angles: which of the 16 channels fired (its fixed elevation θ) and where the head was pointing at that instant (azimuth α). Range + two angles = spherical coordinates, which convert to XYZ.

sensor origin z (up) horizontal range r θ elevation = which of 16 beams the 3D point
One return: distance r along a beam whose elevation θ is fixed per channel and azimuth α is the head angle at that instant. Trigonometry turns it into an XYZ point in the sensor's coordinate frame.
# sensor-frame position of one return (Hesai convention)
x = r · cos(θ) · sin(α)
y = r · cos(θ) · cos(α)
z = r · sin(θ)

The 16 fixed elevation angles (roughly +15.1°…−14.9°) live in a calibration table shipped inside the data (hesai_correction). Do this for all 68 million returns and you have a cloud — but only in the sensor's own frame, which is glued to the scanner and moves with every step you take. That's the problem the rest of the pipeline solves.

03 The puck: R1 vs T1

Why one scan was far easier to reconstruct than its twin

Two scans, R1 and T1, walked the same ~540 m route back-to-back. The only difference: whether the motor was spinning the head. We proved which was which straight from the motor_raw_data log — the histogram of head-angle over the scan is unmistakable.

T1 — head parked
Every sample sits near 0° (it only jitters across the 360°↔0° wrap). The head is fixed.
R1 — head rotating
Angle is spread evenly across all 12 bins — the signature of a head spinning steadily (~40 rpm).

The naming even lines up: R1 = Rotating. And it decided which scan to reconstruct first: with T1's head parked at ~0°, there's no motor rotation to model — one fewer moving part in the geometry. So T1 was the one we rebuilt into the full-density cloud.

04 Finding the raw returns

A scrape → encrypted database → a packet capture of the laser itself

The raw returns weren't lying around as a nice file. They were buried several layers deep in the device's project bundle. Unwrapping it was archaeology:

.fjdslams2 gzip bundle SQLite DB XOR-encrypted key dede5e5a… 8 blobs lidar · imu · motor… PCAP hesai_lidar_data 42.8 MB 68k packets gunzip decrypt read blob each packet = one PandarXT-16 frame → 8 blocks × 16 channels of (range, reflectivity) UDP port 2368 · 68,362 packets · GPS-timestamped
The bundle is gzip around an XOR-encrypted SQLite database (the repeating 8-byte key was visible in the ciphertext where the plaintext was zeros). Inside, 8 named blobs; the one called hesai_lidar_data is a plain PCAP — a capture of the LiDAR's own network packets. That's the raw returns.

The other blobs are exactly the extra ingredients the reconstruction needs: imu_raw, motor_raw_data (head angle over time), hesai_correction (the 16 beam angles), calib_params (how the sensor is mounted), and basetime (the clock). Everything required was in the box — it just had to be decoded.

05 SLAM and the trajectory

The single most important output: where the scanner was, moment to moment

SLAM = Simultaneous Localization and Mapping. As you walk, the device matches each new LiDAR sweep against the map it's building so far, and solves two things at once: the map, and its own pose. We don't need its map — we need its trajectory: the scanner's position and orientation at every instant.

start = origin (0,0,0) each tick = one pose: position (x,y,z) + orientation (quaternion), at a timestamp → heading↑ up
The trajectory is a list of poses over time — a position and an orientation at each timestamp. Crucially, it lives in a local frame: the origin is just wherever the scan started, axes gravity-aligned by the IMU. No map, no latitude/longitude.
The key idea The trajectory is the "puck location" over time. To place a laser return in the world, we don't need GPS — we need the pose at the exact instant that return was fired. The scanner already solved that for us.

06 Three coordinate frames

Sensor → head → world, chained by a rotation and an offset

A point measured by the laser is in the sensor frame. But the sensor is bolted onto the head at a fixed angle, and the head is wherever the scanner was standing. So a return has to be carried through two hops to reach the shared world frame where all 68 M points finally agree.

sensor raw x,y,z head / body after −20° yaw + mount world (local) shared frame fixed mount same for every point SLAM pose different every instant
Hop 1 is a fixed rotation (how the tilted sensor is mounted — a −20° mechanical yaw plus the calibrated extrinsic), identical for every point. Hop 2 is the SLAM pose — a rotation and a translation that's different at every instant. Get either wrong and the cloud won't line up.
# the whole transform for one return, at its firing time t
p_world = R_pose(t) · ( R_mount · p_sensor ) + t_pose(t)

07 The reconstruction recipe

Four steps, applied to every one of 68 million returns

Now it all composes. For each laser return in the PCAP:

1 · sensor XYZ from range r + beam θ + head azimuth α 2 · −20° yaw fixed mechanical mount offset 3 · rotate by pose quaternion at time t (the tricky one — §9) 4 · + position add pose x,y,z → world point The hinge: time sync each packet's timestamp shares the trajectory's clock (same Unix epoch), so we interpolate the exact pose for every firing — even mid-stride
Steps 1–4 turn a polar range measurement into a world-frame point. The quiet hero is time synchronization: because each packet and each pose share one clock, we can look up the scanner's pose at the precise microsecond each beam fired.

Run that loop over the whole capture and 68.3 million valid returns land in one consistent 3D model. No GPS, no manual alignment — just the scanner's own motion path, applied return by return.

08 Two clouds, made differently

The device's own export vs. the full-density reconstruction

The scanner does ship a point cloud — but a thinned one (5.76 M points). It was tempting to assume it threw away the dim returns. It doesn't. We measured the reflectivity distribution on both sides: if a brightness threshold were culling points, the low bins would be emptied. They aren't.

Reflectivity distribution — raw returns vs the device's LAS
Raw returns (68.3 M) Device LAS (5.76 M)
Both distributions are dominated by the dimmest bin. Weak returns are kept, not filtered — so the ~12:1 reduction is spatial de-duplication (merging repeated hits on the same surface), not a brightness cutoff.
CloudPointsHow it's madeFrame
Reduced5.76 Mdevice voxel de-dup → Potree octreelocal
Original68.3 Mour reconstruction from raw PCAPlocal

The reduced cloud was easy — the device already ships it as a ready-to-serve octree. The original full-density cloud is the one that required all of §2–§7.

09 The bug that ate the night

A quaternion stored in an order I didn't expect

Step 3 rotates each point by the pose quaternion — four numbers (w, x, y, z) that encode an orientation. The trap: which slot is which. I assumed [x, y, z, w]. The device stores [w, x, y, z]w first.

Why it was so sneaky A wrong quaternion order is invisible while the scanner is standing still — when the orientation isn't changing, the error is a single constant rotation that the fitting step silently absorbs. The moment the scanner turns, that "constant" starts changing and the cloud smears apart. It registered beautifully on the static start, then fell to pieces in motion.
wrong order — smears in motion correct order — stays sharp a wall becomes a fog a wall stays a wall
Same data, same everything — only the quaternion element order differs. Left: mid-motion, a flat surface disperses into noise. Right: with [w,x,y,z], surfaces stay crisp across the whole run.

The lesson that went straight into memory: never validate a pose-based reconstruction on a stationary window alone. Only moving data can tell a right rotation from a wrong-but-conveniently-cancelling one.

10 Proving it's right

Overlaying the reconstruction on the device's own cloud

How do you know a rebuilt cloud is correct and not just plausible? Register it against a known-good reference — the device's own reduced cloud. We voxelized space and asked: what fraction of my reconstructed points fall in a voxel the device also filled? Higher = better alignment.

Overlay agreement with the device's cloud
Static / dwell window In-motion window
With the wrong quaternion order the motion window collapses (overfit to the static start). With [w,x,y,z] it holds across the run. The remaining static→motion gap isn't misalignment — it's the metric under-counting where the sparse reference cloud simply has fewer filled voxels to match against.

~95% on dwell, ~70% in motion, consistently across held-out windows with one clean set of parameters — that consistency is the proof. An overfit set looks great on one window and falls apart on the next; this didn't.

11 The one thing still open: GPS

Why the cloud can't sit on a map yet

Remember the RTK GNSS antenna from §1? It logged a genuine, centimetre-grade lat/lon track. But it was never fused into the cloud. The reconstruction lives entirely in SLAM's local frame — an arbitrary origin at the scan start, no projection. So there are two good datasets that don't yet speak the same language:

reconstructed cloud local frame · origin at start · no lat/lon RTK GPS track real lat/lon · but only a path, no cloud −33.7185, 151.0876 ?
Two coordinate systems, not yet linked. To put the cloud on a real map you'd fit a rigid transform (yaw + translation, or a full 7-parameter Helmert) between the SLAM trajectory and the RTK track — matching them by their shared timestamps. That step hasn't been done.

Everything before this point is finished and live. Georeferencing is the natural next chapter whenever you want the cloud to line up with the world.

Built from the reconstruction notes for scan T1_2026-03-02-12-25-26 (FJDynamics S2, Hesai XT16). Live viewer: t1-potree.apps.sandytulloch.com. Charts use a CVD-validated two-hue palette; hover any bar for its value.