We have conducted extensive tests with the Robosense RS-LiDAR16. The results are compared with Quanergy M8-1, Sick 551 and Rplidar A3.

We have conducted extensive tests with the Robosense RS-LiDAR16. The results are compared with other LiDAR (Quanergy M8-1, Sick 551 and RpLiDAR A3) to have a better understanding of its strengths and weaknesses.

The distance measurement obtained with a LiDAR can be influenced by internal (hardware components, mathematics computations, communication protocols, etc.) and external (sun light, temperature, humidity, environment size, materials and incidence angles obstacles, etc.) factors and sources.

In order to understand the influence of these parameters on the distance measurements accuracy, we have implemented and analysed different environnements. Here is a list of different assumption biais on LiDAR :

Biais	Tested	Details
Material obstacles	Yes	Ten different materials
Sunlight	Yes	Four environments (dark, artificial light, under shade and under sunlight)
Time drift	Yes	Two environments with 45 minutes of test each
Temperature	Yes	Two environments : under shade and under sun
Humidity	No
Incidence angle	No
Hardware components	No	Tests done on same LiDAR
Communication protocols	No	Ethernet and USB depending of specs
Mathematics computations	No	Pointcloud/laserscan raw value read
Beams Differences	Yes	With vertical obstacles
Scanning frequency influence	Yes	At 5, 10 and 20 Hz
Short/long distance differences	Yes	6 different distances
Difference between two angular ranges of same LiDAR	Yes	Phase at 0° and +90°
Continued/discontinued obstacles	Observed	Additional environments but not reported with same process
Circle/perpendicular shapes	Observed	Additional environments but not reported with same process

This is our first extensive test with the Robosense RS-LiDAR16. We tried to cover different biais, the results may not correspond to other tests that include repeatability. It will however give you a good idea of RS-LiDAR16 performances.

Several tests highlighted upper errors for RS-LiDAR16 than other LiDAR in general. However it is good to remind us that RoboSense has 16 beams and in mapping applications, moving around allows to correct those errors by iterations.

3D LiDAR: real-time, wide range, high precision

Nowadays, LiDAR, and especially 3D LiDAR are widely used in new technologies like autonomous driving and mobile robotics. Indeed, they are an essential part in the robot’s decision making process, in autonomous navigation applications.

The latests improvements in LiDAR technology allow to get every detail of an environment, segment and cluster obstacles, thanks to the accuracy.

Several companies are developing their own components and mapping software. They provide 3D information about their environment in real time, without any contact, with a wide range, and a high precision measurement.

There are two methods (direct and indirect) of Time-of-flight technology [2]:

The direct method is based on the measurement of time interval between an emitted and detected signals.
In the indirect method, the distance is obtained by using a phase shift of a modulated light with either a sinusoidal signal or a pulsed signal.

The applications require a high level of accuracy, distance range, the resolution and the accuracy of the sensor are the main focus parameters.

Robosense RS-LiDAR16 presentation

The RoboSense RS-LiDAR16 is an attractive cheaper model of 3D LiDAR . It has good specification for a price around 4 128,00€ (tax inc.). The RS-LiDAR16 has 16 laser beams. The system consists of 16 2D line scanners which can spin at a rate from 5Hz to 20Hz. The 16 beams are spread out over a 30◦vertical field of view (FOV) and the entire unit rotates to give a full 360° by 30° FOV.

An overview of the specifications for the RoboSense scanner is given below:

ROS compatible
Laser beams: 16
Wavelength: 905 nm
Laser class: 1
Accuracy: ±2 cm
Detection range: 20 cm to 150 m (20% reflectivity)
Data rate: 320,000 pts/s
Field of view (vertical): +15° to -15° (30°)
Angular resolution (vertical): 2°
Field of view (horizontal): 360°
Angular resolution (horizontal/azimuth): 0.09° to 0.36° (5–20 Hz)
Rotational velocity: 300 to 1,200 rpm (5–20 Hz)
Input voltage: 9–32 Vdc
Power consumption: 9W
Protection: IP67
Operating temperature: -10 to +60 °C
Data type: spatial coordinates/intensity
Dimensions: 82.7 mm (H) x 109 mm (φ)
Weight: 840 g (without cables)

To enable the use of the sensor, RoboSense provide a software to read values on Windows and Ubuntu and an SDK for using the LiDAR on ROS .

Robosense 16-beams LiDAR characterisation

As mentioned in the above table, measurement errors can have different sources. Here, we attempt to evaluate the errors due to the system design and the environment.

During the post-process, we will focus most of the time on central beams (8 & 16) :

Répartition des faisceaux laser du RS-LiDAR16 — *RS-LiDAR16 beams repartition*

1. Material obstacles

LiDAR performances were different depending on the material of the obstacle. In order to quantify those differences, we consider a static scene, the RS-LiDAR16 is set up in front of a different material obstacles, at a distance of approximately 2m ± 0.01m .

For each material, the LiDAR is launched in an indoor environment after at least 2 minutes of non-use and for a duration of 3 minutes. A 30 seconds bag is recorded for data post-process.

Ten materials have been tested: window, TV screen (glass), white board, door, foam, transport plastic, cardboard box, iron, wall, condensed wood.

2. Sunlight

Some LiDAR are not performing well in outdoor environments, due to the sunlight influence. Indeed, the sun is emitting light on a wide range of wavelengths, which are altering the LiDAR measurements.

Considering a static scene, the RS-LiDAR16 is set up in front of a planar wall ( condensed wood ), at a distance of approximately 2m ± 0.01m . For each environment, the LiDAR is launched after at least 2 minutes of non-use and for a duration of 3 minutes. A 30 seconds bag is recorded for data post-process.

Four environments have been tested: inside in total dark environment, inside with artificial lamps, outside in the shade and outside under the sun.

3. Time drift & Temperature

Some components’ accuracy is decreased on long period of runnin-time. This is especially true when the temperature is hotter.

Considering a static scene, the RS-LiDAR16 is set up in front of a planar wall ( condensed wood ), at a distance of approximately 2m ± 0.01m . For each environment, the LiDAR is launched after at least 10 minutes of non-use and for a duration of 45 minutes. Every 10 minutes a 1 minute bag is recorded for data post-process.

Two environments have been tested : Outside in the shade (ambient temperature ~ 25°C) and outside, in a sunny environnement (contact temperature ~ 32°C).

4. Beams differences & Scanning frequency influence & Short/long distance differences

3D LiDAR, have to take into account the physical behaviour of waves. Due to vertical angles differences laser will not have the same results.

With a scanning frequency of 20 Hz, the LiDAR motor is spinning at double speed of 10Hz. It is interesting to test if results are impacted.

Finally, long range LiDAR are not really good at short range. We want to quantify those errors differences.

First scene

Considering a static scene, the RS-LiDAR16, Quanergy M8, RPLIDAR A3 and the Sick 551 are positioning in front of a planar wall ( cardboard boxes ), at a distance of approximately 1.01m and 1.90m with ± 0.01m .

For each environment, the LiDAR is launched after at least 2 minutes of non-use and for a duration of 3 minutes. A 15 seconds bag is recorded for data post-process.

Second scene

Considering a static scene, the RS-LiDAR16 and Quanergy M8 are positioning in front of a planar wall ( condensed wood ), at a distance of approximately 5.00m, 9.98m, 19.97m and 27.81m with ± 0.01m .

For each environment, the LiDAR is launched after at least 2 minutes of non-use and for a duration of 3 minutes. A 30 seconds bag is recorded for data post-process.

5. Differences between two angular ranges

LiDAR may have lower performances on one part of the angular range.

Considering a static scene, the RS-LiDAR16 is positioning in front of a planar wall ( cardboard boxes ), at a distance of approximately 1.01m ± 0.01m.

For each environment, the LiDAR is launched after at least 2 minutes of non-use and for a duration of 3 minutes. A 15 seconds bag is recorded for data post-process.

Four environments have been tested: the environment does not change, the LiDAR rotates 90° between the two record, +180° and +270°.

Algorithme details

The algorithm used is not that complicated, it reads the raw data through the LiDAR topic which transmit pointcloud2 messages for the RS-LiDAR, Quanergy and Sick LiDAR and Laserscan for RpLiDAR.

For those experiments, only X and Y axis data are taken in account to get the average range, standard deviation and variance on bag data. We are on a restricted area (X +/- 0.1 and Y +/- 0.5), matching the obstacle known coordinates.

On the data set, we are counting how values the LiDAR detected to interpret the results differently.
We are calculating the values on all the bag since the environment is not changing to get more accurate results.

Experimental results

Now you know the tests conditions from part III, we can discuss and interpret the results. Our tests results are available here: Robosense RS-LIDAR16 tests results .

1. Material obstacles

The RS-LiDAR16 has good results (± 1cm) with TV screen, white board, door and foam but has relatively spread data with a ±2cm of standard deviation.

About the window obstacle, only 4 values returned was expected since transparent obstacles are the most difficult to be measured by LiDAR.

Finally, unexpectedly, cardboard boxes, solid wall, condensed wood and iron have poorer results with an accuracy of +4cm and +7cm with a low standard deviation (± 1.5cm).

This test shows that the difference between materials can be important and unexpected.

To be sure, new test should be done on materials which lead to poorer results.

For the other test, material chosen need to be kept in mind.

2. Sunlight

This test has been done on a condensed wood obstacle, RS-LiDAR seems to have better results (-3cm) indoor environment than in outdoor environment (+5cm in the shadow and +7cm under the sun).

With the same material, the difference between environments is quite important.

3. Time drift & Temperature

This test is performed with a condensed wood obstacle, in the shadow, the RS-LiDAR16 does not seem to have time drift but the opposite, it gets more accurate through time. Indeed first measurements have bigger error (+5cm at 0min against +1cm at 45min for beam 8).

Under the sun, the beams studied are the 6 & 7 because there is an offset on the horizontal plane (they correspond to the 8 & 16 of the previous scene), the behaviour is the same, but the error is converging around +5cm. It could be consequences of temperature influence. Indeed, the LiDAR was hotter than in the shadow.

Both under the shade and the sun, the beams 16 & 7, get upper errors (+3cm and +2cm).

4. Beams differences & Scanning frequency influence & Short/long distance differences

Firstly, indoor environment, for short distances (1,01m) 2D LiDAR have better results (rpLiDAR : 0cm and Sick : +1cm) thanks to their perfectly horizontal beam.

About 3D LiDAR, the Quanergy has +0cm and the RS-LiDAR16 has with less deviation +3cm of error in average on beams.

For 1.90m distance, 2D still have good results (rpLiDAR : 0cm and Sick : +2cm).

About 3D LiDAR, both Quanergy and RS-LiDAR16 have +1cm of error in average on beams.

Then, for outdoor environment, accuracy is better for 10Hz (-1cm of error than for 20Hz). Note that a scanning frequency of 5Hz is dangerous for mobile robots.

About 5.00m distance, accuracy is better on central beams. On those beams (8 & 16), RS-LiDAR16 has +1 cm of error and the quanergy +2cm.

For 9.98m distance, the Quanergy is more accurate with only +1cm of error for both frequencies and +3cm at 10Hz ; +5cm at 20Hz for the RS-LiDAR16.

For 19.97m, both have a +3cm of error.

Finally for a distance of 27.81m, both accurate with +1cm for RS-LiDAR and +2cm for the Quanergy.

Due to resolution, RS-LiDAR16 has few values at this range on a small obstacle (1,20m x 0.8m). At 20Hz, he detects no points.

5. Differences between two angular ranges

Results are quite similar but have some small differences (± 1cm of error and ±0.5 cm of deviation).

The test conditions do not allow us to conclude on a specific behaviour. Experiment needs to be done several times with different environments.

One point can be noted, measurements seems volatils, the error is random and not systematic.

6. Conclusion and future work

The aim of this article was to investigate through a set of tests the quality of the RS-LiDAR16 sensor. This sensor is a new LiDAR on the market. Our study was designed to assess the measurements provided by it in order to determine if it was possible to use it in the applications covered by the manufacturer.

The experiments should have been done several times to get more samples and in different environments (other obstacles form and distances) , but it is still giving a good insight and range of RoboSense RS-LiDAR16 performances.

Several tests highlighted upper errors for RS-LiDAR16 than other LiDAR in general. Concerning 2D LiDAR, it is logic to get higher accuracy because of the number of beams.
Against the Quanergy M8-1, it is good to remind us that RoboSense has 16 beams and in mapping applications, moving around allows to correct those errors by iterations .

Again, it is important to remember that the received sensor is one of the very first delivered. We hope that future versions will cor-rect errors encountered. Accordingly, the arrival of this sensor can be seen as an opening for wider distribution of this type of sensor.

Finally, this is our first real test, we tried to cover different biais, the results may not correspond to the ones find with repeatability but it gives you a good idea of RS-LiDAR16 performances.

Appendices and references

This article does not aim to be validated as a research paper Assessment inspired by Quanergy M8 assessment [1]

[1] Mittet, A., Nouira, H., Roynard, X., Goulette, F. and Deschaud, J-E. “EXPERIMENTAL ASSESSMENT OF THE QUANERGY M8 LIDAR SENSOR”, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLI-B5, 2016 XXIII ISPRS Congress, 12–19 July 2016, Prague, Czech Republic

[2] Kilpela, A., Pennala, R. and Kostamovaara, J. “Precise pulsed time-of-flight laser range finder for industrial distance measurements”, Review of Scientific Instruments72(4), pp. 2197–2202, 2001.