m3
Is the Mesotech M3
compliant with
IHO standards?


John E. Hughes Clarke
Ocean Mapping Group
Dept. Geodesy and Geomatics Engineering
University of New Brunswick
CANADA

January 2015
pole
test configuration, Feb 2014
pole mounted M3 pair: +45° and leve
l



Summary:

Based on field data collected in February 2014, bathymetry from  an M3 multibeam sonar in depths ranging from 15-40m were used to assess the compliance of that data with IHO vertical accuracy and target detection standards. The results  indicate the following:

As with any integrated swath sonar system in shallow water, the component external inputs that affect the vertical position of the transducer will tend to dominate the total vertical uncertainty. Thus the sonar contribution (bottom detection and resolution) will generally have only a secondary influence on whether IHO compliance is met. Compared to other significantly narrower beam sonars (results shown herein), the M3 sounding variance and target detection is  poorer. It is clear, however, that those compromises (a reflection of a 3-6x lower cost) do not inhibit the M3 from maintaining IHO compliance.


Project Aims:

Given that the M3 is a significantly cheaper and wider beam multibeam than most others on the market today, an obvious question is whether one can meet the IHO accuracy and target detection requirements with that sonar.

To answer that it is important to specify exactly what those criteria really are.

IHO Vertical Accuracy:

This is defined as 95% of the data falling within a range based on a combination of a fixed value ("a") and a depth scaling value ("b"). The combination is the root of the sum of those squared components.

The two orders of interest are Special and Order 1:

a
b
Special
0.25m
0.75%
Order 1
0.50m
1.30%

The static component was designed to reflect non depth-scaling parameters such as tide, draft, squat and heave uncertainty.  For a specific survey (a realization) these would normally be reflected in systematic biases that overprint the data. The depth scaling component, in contrast was designed to reflect the sonar and medium (sound speed) capability that generally does scale linearly with depth. Thus the relative contribution of the two components will evolve with depth.

The depths of water that most people would be using an M3 for IHO compliance include typical port and harbour depths that range from 5-40m with most concern in the <20m range. At those depths, the static component increasingly dominates as the water shoals so that, unless the operator has significant tide or draft issues, the sonar component of the error could even be worse than the criteria used herein.

To illustrated the dominance of the static component, the total allowable deviation is calculated for depths from 10 - 40m:

Allowable Special Order Allowable Order 1

+/-95% in metres
+/-95% in %Z
one sigma in %Z
+/-95% in metres +/-95% in %Z one sigma in %Z
10m
0.26m
2.61%
1.31%
0.52m
5.17%
2.58%
20m
0.29m
1.46%
0.73%
0.56m
2.81%
1.41%
30m
0.34m
1.21%
0.57%
0.63m 2.11%
1.06%
40m
0.39m
0.98%
0.48%
0.72m 1.80% 0.90%
As can be seen, even as deep as 40m, the  Special Order total allowable standard deviation is about +/-0.5% of depth. In more typical depths of 20m, this grows to ~0.75%. Thus, if the sonar vertical referencing is  well defined, a noisy bottom detection can get lost in the static allowable component.

As a result, for the purpose of this analysis, to assess specifically whether the M3 contribution can meet the IHO criteria, the standard deviation of the soundings around a biased mean is calculated separately. The bias itself, while extracted to assess its scale, is merely a reflection of the quality of the integration of the user.

As presented in the figures below, the standard deviation and bias in percentage of depth is shown across the swath.  For the purpose of this assessment, the standard deviation about the (potentially biased) mean is used to assess the M3 bottom detection contribution. As ~2 standard deviations reflect 95%, a one sigma level of 0.375% is used as the cutoff requirement for Special Order and 0.65% for Order 1A.

The figures below describe and illustrate the IHO standards and the changing influence of depth.
iho
iho
IHO S44 5th edition standards
expressed in graphical form


IHO Target Detection:
The Special Order requirement is defined as detecting a 1m cube and the Order 1A requirement is a 2m cube. Again, however, how those criteria vary with depth needs to be understood. The aim in both cases is to meet the needs of delineating hazards to surface shipping and thus depths much below likely keel depths are of declining interest. As originally conceived, these criteria were to be met using sidescans that were towed at a fixed altitude above the seabed and thus the resolution capability was not depth scaling. For the case of a surface-mounted multibeam however, clearly the resolution capability would be depth scaling.

As defined, the Order 1A requirement is to be maintained to 40m (at which point it is a 5%Z cube). Deeper than that the requirement relaxes to 10%Z (4m cube at the transition depth). The Special Order requirement, however has no depth limit specified. For a conservative interpretation, a 1m cube at 40m might then be considered (2.5%Z). That same cube, of course, would be 5% at 20m (equivalent to 1A) and 10% at 10m.

Herein, 1m cubes are assessed at either 20m or 40m. No 2m cubes were used as, frankly, with most modern sonars they are hard to miss. But resolving a 1m cube at 20m is equivalent to a 2m cube at 40m. Thus both the most demanding interpretations of Special and 1A may be tested, as well as the more realistic usage of Special (a 1m cube at 20m).



poleThe Hardware Configuration used for testing:

In February 2014, two M3 sonar heads were mounted on a pole on the CSL Heron, a 10m research vessel operated by the University of New Brunswick.  One of the heads was mounted vertically (+/-60°) and one tilted to port by 45° (thus -15° to +105° potentially). The pole was deployed daily and had a rigid lock-down mechanism.

The M3 was integrated with a POS/MV 320 for motion and position. The POS/MV was in turn fed RTCM corrections from a C-Nav 2050 receiver. Tidal corrections used were observed tides at Sidney BC. Sound speed information was obtained from an MVP-30 deployed once per test site. Although there was a surface sound speed probe installed on the vessel for other multibeams on board, it was not used to update the surface value used by the M3.

The M3 was operated as part of a multi-frequency experimental program involving the simultaneous use of an EM710 (70-100kHz) and EM2040C (200-400 kHz). The 710 was mounted on a gondola on the hull and the 2040C on a keel blister.

While both M3 heads could be operated together (alternate pinging) they were tested separately so that the ping rate was controlled by the maximum two-way-travel time utilized by a single head. For some of the experiments, the ping rate was controlled by a master synchronization from an EM710 (operating at +/-70°) and thus the ping rate was artificially slow. However, for later experiments, when it became apparent that the M3 neither interfered with, nor was interfered by, the simultaneous use of the EM710 and/or  EM2040C, it was allowed to run independently. This is an important factor for target detection as the requirement to get 3 strikes on a target along track is difficult enough already at typical survey speeds (6-8 knots).

Note that, although the issue of 3x3 strikes on a target is mentioned herein, it is not a formal IHO requirement. Nevertheless,  some agencies (LINZ, NOAA, SMA) do include various flavours of that requirement.




mapThe Test Targets

The target detection compliance was tested against a series of standard concrete 1m cubes deployed close to the CHS base in Saanich Inlet in British Columbia.  The Warrior Point Target Range was established in 2011 with the first EM2040S trials. All subsequent work has examined these targets.

As 2m cubes were so easy to see (IHO Order 1 requirement), the focus was on 1m and 0.5m cubes.

The targets are laid along the 20 and 40m contours. The idea is to interpret the IHO Special Order criterion (a 1m cube) at its most stringent by placing it at the depth that an Order 1 target standard is relaxed (40m - i.e. a 2.5% of Z object). However, it is recognized that most Special Order surveying occurs in depths of 20m or less. At that depth the target is a 5% of Z object, equivalent to the Order 1 standard of a 2m cube at 40m. Thus the line of targets at 20m is included.

Furthermore, as the 1m cubes are normally always detected, a second set of smaller 50cm cube targets are laid along the 20m contour to assess potential MLO detection capability.

For many of the targets an additional pole protrusion was added to try and see whether this affects the target detection and whether the pole itself could be imaged.



Vertical Accuracy Test Area:profile
mapA smooth section of the seabed slightly to the north of Sidney Marina (off Roberts Point) was used as  the test area to derive statistics for the two M3 configurations and the EM710 and the EM2040C.

The image to the left shows the area. Through that area a cross line was run in July 2014 using an EM2040D (1/2° by 1°) with only the +/-45° sector used. This line was specifically designed to cross over the most featureless part of that terrain.

 This cross line was used as the reference against which all main survey data from M3, 2040C and 710 were assessed against. The depth range along the cross line varied from 18 to 29m.

For each main N-S survey line, the mean bias and standard deviation about the bias was derived by comparison with the reference data. All the observations were sorted by incidence angle. All data is presented below in the same standard plot format with the results plotted  as a percentage of water depth and sorted by incidence angle, varying from -75° to + 75°.



Vertical Accuracy Testing:

Using  the reference line described above, each survey line of the M3 in either configuration was compared to the reference and the statistics compiled by incidence angle. For each plot shown below, there are three curves that are generated:
Rather than produce average statistics for all lines that cross the reference data, each line is presented separately so that, if the tidal or refraction errors change over time, they are seen in the local bias for that line rather than being averaged into the apparent sounding variance.

As well as results for the two M3 configurations (level and tilted receivers), examples from other sonars on exactly the same site (EM2040C and EM710) or close by (EM2040D) are shown to put the M3 results in perspective compared to other higher priced, narrower beam systems.

Interpretation of Biases:
The bias is primarily attributed to factors like tide and draft errors as well as residual refraction errors and roll biases. Any bias that is not just linear or follows a typical refraction shape may, however, reflect imperfect bottom detection biases.

Interpretation of Standard Deviations:
As mentioned above, to meet IHO Special Order, the sonar bottom detection noise about the (potentially biased) mean should be within 0.375% at a one sigma level.  To meet the Order 1 standard requires only 0.65% of depth at one sigma level. As can be seen, this is actually off the scale of the plot as I've never seen a multibeam bottom detection noise that bad since the 90's..


M3 head facing down (+/-60°)
M3 head tilted by 45°
looking down
looking to the side
COMMENTS:
biases - there are notable step like biases on the starboard side that show up as ship-track parallel ridges. While they are only ~0.05%Z in magnitude they appear to reflect limitations in the bottom detection algorithm.

standard deviation: while greater than the other sonars considered, the data is all within 0.375% out to 60° on either side. This is thus maintaining Special Order compliance, even though noisier than the narrower beam examples. This is to be expected with the lower source level and poorer directivity index of this fat beamed system.
COMMENTS:
biases - there appears to be a residual roll bias with a superimposed refraction (or surface sound speed) artefact.

standard deviation: Compared to the vertically mounted head, the bottom detection noise is notably improved as the narrowest unsteered beams are now pointing out at 45°. As a result, the 0.375% limit is not actually reached until > 70° on the port side.
Also, while the data is excellent on the side facing out, for reasons not understood, the bottom detection for the 15° of data facing the other way, degrades very rapidly.

EM2040C at 300 kHz

CSL Heron February 2014
(acquired at the exact same time as the vertical M3).

EM710 - 1
°x2°
CSL Heron February 2014
(acquired at the exact same time as the vertical M3).
stats stats
COMMENTS:
biases - clearly a roll bias, but no other issues.

standard deviation: remarkably low apparent sounding noise out to > 60°.  Interestingly achieving a lower level than seen with the EM2040D (plots below). This may be due to using a shorter pulse as the 2040C has only a single sector and therefore more available bandwidth as not spread over the 6 sectors used for the dual swath, dual receiver EM2040D.
COMMENTS:
biases - a notable residual refraction artefact. Superimposed on this are slight apparent sector boundary steps and perhaps small steps at the phase- amplitude transition. To be expected when using a 0.2ms pulse in such shallow water.

standard deviation: notably higher than the 300 kHz systems rising up to the edge of the amplitude detection zone and then dropping down again into phase detections.. Again related to the use of a relatively narrow band (long) pulse in such shallow water. Unavoidable as a EM710 is designed to operate to more than 1km in depth.

Dual EM2040D (0.5°x 1.0°) in 40m
July 2014

Dual EM2040D (0.5°x 1.0°) in 20m
July 2014
stats stats
COMMENTS:
biases - notably there is a slight refraction bias that changes slowly over the duration of these test lines (~ 20 minutes). A typical problem in summertime high stratification areas in tidal waters.

standard deviation: while the 2040D should have lower sounding noise than the 2040C, what one is seeing here is partly the effect of strong and variable stratification that was developed in this area in July which was absent for the February 2040C trials. And, as mentioned above, the receiver bandwidth has to be shared over 6 sectors.
COMMENTS:
biases - in this case, the refraction bias is much more pronounced but more stable during the duration of this test period.

standard deviation:  As the same pulse length is used as in the 40m example, as the water depths are now shallower, the sounding noise, as a function of percent of depth is larger.



Target Detection Testing:

Comparative examples are shown of the M3 bottom tracking on 1m cubes at 20m and 40m depth as well as the same targets being tracked by an EM710, an EM2040C and an EM2040D.

The 20m depth cubes:

The following are the targets in a line of 1m and 0.5m cubes with various protrusions along the 20m corridor from left to right.

1m CUBE AT 20m
(pseudo) IHO Order 1 Compliance (5%Z)
"typical" Special Order
0.5m CUBE AT 20m
Potential Mine-Hunting Compliance
photo photo photo photo photo photo
as built target - 1m_E as built target - 0.5m_Q as built target - 1m_F as built target - 0.5m_R as built target - 0.5m_P as built target - 0.5m_S


The 1m cube without protrusions (F) is the best equivalent test of a Order 1 detection (2m at 40m). Anomalies with an EM2040D are now seen out to 70°.


For all detections, three views are presented:
From these views one can see whether the target would be visible is a derived terrain model ,and also how many individual strikes were achieved on the target (across and along track) and how proud of the surface, those strikes showed up as.

EM2040D 0.5°x 1.0° - dual swath - +/- 72°
  July 2014

label label label label label label label label label

71
66
56
37
2
33
54
65
70

sun sun sun sun sun sun sun sun sun

across across across across across across across across across

across across across across across across across across across

EM2040C (single swath) +/-65°
CSL Heron February 2014

70
65
55
35
0
35
55
65
70



sun sun sun sun sun sun



across across across across across across



across across across across across across

EM710 1°x2°  dual swath  - +/-70° at 8 knots



41
38
2
24
51
59




sun sun sun sun sun sun



across across across across across across



across across across across across across

M3 vertical mount +/-60° - not roll stabilized
(tracked! - but note that the targets are smeared out along track due to the 3° Tx beam width)



59
38
nadir
39
53





sun sun sun sun sun




across across across across across




across across across across across



The 40m depth cubes:

The four 1m cubes at 40m depth are identical in basal size but differ in whether they have protrusions sticking out from the top. The cubes images are described in the following photos as they lie on the seabed from from west to east

1m CUBE AT 40m
(strictest) IHO Special Order Compliance (2.5%Z)
photo photo photo photo
as built target - 1m_A as built target - 1m_B as built target - 1m_C as built target - 1m_D

The following set of results are for the 1m cube without an inserted pole (D). As noted in the 2013 trials, the addition of the poles, while not detected separately, makes the bathymetric anomaly larger. The protrusion-free target, is thus the best test for the limit of Special Order Detection.

Again, as above, the same target is viewed by the M3 (bottom) as well as an EM710, and EM2040C and an EM2040D.

EM2040D 0.5°x 1.0° - dual swath - +/- 72°
  July 2014

label label label label label label label label label label label label label label

sun sun sun sun sun sun sun sun sun sun sun sun sun sun

across across across across across across across across across across across across across across

across across across across across across across across across across across across across across

EM2040C (single swath) +/-65°
CSL Heron February 2014







nadir
~20
~40
~50
~60










sun
sun sun sun sun









across across across across across









across across across across across



EM710 1°x2°  dual swath  - +/-70° at 8 knots
CSL Heron February 2014





incidence
angle
 nadir
28
45
56
63










sun sun sun sun sun









across across across across across









across across across across across



M3 vertical mount +/-60° - not roll stabilized
CSL Heron February 2014
(only very narrow swath achieved)





incidence
angle
nadir
28













sun sun












across across












across across








28m depth Natural Target Detection:

Although the concrete cubes provide a formal compliance test with IHO orders, in reality, the systems are far more likely to be mapping natural targets such as boulders or coral heads. These occur over a wide range of sizes and aspect ratios. To test out the performance of the sonars over a wider range of target types, a natural field of glacial erratic boulder that sit in the Miner Channel (east of Sidney Spit) were selected. This had been previously surveyed by the REMUS 600 using its Edgetech sidescan (see below).

From that imagery it is clear that there are fields of boulders of a wide range of sizes down to smaller than the sonar would be able to meet. These targets form a particularly useful and repeatable test range.

Inspection mode EM2040D (+/-45 degrees with 100% overlap).
compared to Edgetech sidescan imagery.
anim
The ultimate view of what's on the bottom
Edgetech sidescan from REMUS 600 collected in 2011


As viewed with the sonars considered in this report:
others
Comparing with other sonars:


Still of the examples presented in the animation above. They include the line through middle of the field of view as well as the line to the north and to the south that has overlapping coverage so that the outermost edge of the swath may be viewed.


EM2040D - SN6 - 300kHz +/-65 deg
July 2014

EM2040D - SN6 - 300kHz +/-75 deg
July 2014
targ
targ
targ
targ
targ
targ
note even sounding density on edge of swath
due to 3-sector yaw stabilization  (and dual swath)
note even sounding density on edge of swath
due to 3-sector yaw stabilization  (and dual swath)
- even with much lower ping rate for +/-75°

Examples of other sonar systems deployed in the AUV Boulder area. These data were collected in February of 2014 off the CSL Heron.

EM2040C single swath (+/-65°)

EM710 1°x2° dual swath (+/-65°)
sun
sun
sun
sun
sun sun
note variable sounding density on edge of swath
(not guaranteeing 3x3 solutions) due to
no yaw stabilization (single sector) and single swath
note even sounding density on edge of swath
due to 3-sector yaw stabilization  (and dual swath)

M3 - vertically mounted
(+/-60° - no roll stabilization)

sun

sun
sun
note variable sounding density on edge of swath
(not guaranteeing 3x3 solutions) due to
no yaw stabilization (single sector) and single swath







page created by JEHC in January 2015, based on Feb. 2014 data