The dense cloud didn't come out of the scanner as a file. It was reconstructed — every raw laser return re-projected through the scanner's own motion path. Here's every concept and every step, start to finish.
Result is live at t1-potree.apps.sandytulloch.com
Everything starts with the sensor. The S2 is a walk-around scanner: you carry it through a space and it builds a 3D model as you move. Four parts matter for the story.
A LiDAR doesn't measure points — it measures range along a direction. Each firing gives
you one number, a distance r, for one beam. The direction is known from two angles:
which of the 16 channels fired (its fixed elevation θ) and where the head was pointing at that
instant (azimuth α). Range + two angles = spherical coordinates, which convert to XYZ.
r along a beam whose elevation θ is fixed per channel
and azimuth α is the head angle at that instant. Trigonometry turns it into an XYZ point in the
sensor's coordinate frame.The 16 fixed elevation angles (roughly +15.1°…−14.9°) live in a calibration table shipped inside
the data (hesai_correction). Do this for all 68 million returns and you have a cloud — but only in
the sensor's own frame, which is glued to the scanner and moves with every step you take. That's the problem
the rest of the pipeline solves.
Two scans, R1 and T1, walked the same ~540 m route
back-to-back. The only difference: whether the motor was spinning the head. We proved which was which straight
from the motor_raw_data log — the histogram of head-angle over the scan is unmistakable.
The naming even lines up: R1 = Rotating. And it decided which scan to reconstruct first: with T1's head parked at ~0°, there's no motor rotation to model — one fewer moving part in the geometry. So T1 was the one we rebuilt into the full-density cloud.
The raw returns weren't lying around as a nice file. They were buried several layers deep in the device's project bundle. Unwrapping it was archaeology:
hesai_lidar_data is a plain PCAP — a capture of the LiDAR's own network packets.
That's the raw returns.The other blobs are exactly the extra ingredients the reconstruction needs: imu_raw,
motor_raw_data (head angle over time), hesai_correction (the 16 beam angles),
calib_params (how the sensor is mounted), and basetime (the clock). Everything
required was in the box — it just had to be decoded.
SLAM = Simultaneous Localization and Mapping. As you walk, the device matches each new LiDAR sweep against the map it's building so far, and solves two things at once: the map, and its own pose. We don't need its map — we need its trajectory: the scanner's position and orientation at every instant.
A point measured by the laser is in the sensor frame. But the sensor is bolted onto the head at a fixed angle, and the head is wherever the scanner was standing. So a return has to be carried through two hops to reach the shared world frame where all 68 M points finally agree.
Now it all composes. For each laser return in the PCAP:
Run that loop over the whole capture and 68.3 million valid returns land in one consistent 3D model. No GPS, no manual alignment — just the scanner's own motion path, applied return by return.
The scanner does ship a point cloud — but a thinned one (5.76 M points). It was tempting to assume it threw away the dim returns. It doesn't. We measured the reflectivity distribution on both sides: if a brightness threshold were culling points, the low bins would be emptied. They aren't.
| Cloud | Points | How it's made | Frame |
|---|---|---|---|
| Reduced | 5.76 M | device voxel de-dup → Potree octree | local |
| Original | 68.3 M | our reconstruction from raw PCAP | local |
The reduced cloud was easy — the device already ships it as a ready-to-serve octree. The original full-density cloud is the one that required all of §2–§7.
Step 3 rotates each point by the pose quaternion — four numbers (w, x, y, z) that
encode an orientation. The trap: which slot is which. I assumed [x, y, z, w]. The device
stores [w, x, y, z] — w first.
[w,x,y,z], surfaces stay crisp across the whole run.The lesson that went straight into memory: never validate a pose-based reconstruction on a stationary window alone. Only moving data can tell a right rotation from a wrong-but-conveniently-cancelling one.
How do you know a rebuilt cloud is correct and not just plausible? Register it against a known-good reference — the device's own reduced cloud. We voxelized space and asked: what fraction of my reconstructed points fall in a voxel the device also filled? Higher = better alignment.
[w,x,y,z] it holds across the run. The remaining static→motion gap isn't
misalignment — it's the metric under-counting where the sparse reference cloud simply has fewer filled
voxels to match against.~95% on dwell, ~70% in motion, consistently across held-out windows with one clean set of parameters — that consistency is the proof. An overfit set looks great on one window and falls apart on the next; this didn't.
Remember the RTK GNSS antenna from §1? It logged a genuine, centimetre-grade lat/lon track. But it was never fused into the cloud. The reconstruction lives entirely in SLAM's local frame — an arbitrary origin at the scan start, no projection. So there are two good datasets that don't yet speak the same language:
Everything before this point is finished and live. Georeferencing is the natural next chapter whenever you want the cloud to line up with the world.
Built from the reconstruction notes for scan T1_2026-03-02-12-25-26 (FJDynamics S2,
Hesai XT16). Live viewer: t1-potree.apps.sandytulloch.com.
Charts use a CVD-validated two-hue palette; hover any bar for its value.