Tuesday, June 16, 2020

Headless

Here's the problem I had to solve: my Differential Global Navigation Satellite System (GNSS) base station runs in "survey-in" mode for days in order to achieve a positioning resolution down to a few centimeters. This procedure takes so long that the Raspberry Pi 3B+, on which my gpstool software runs to process the output from the u-blox ZED-F9P GNSS module, is powered by an Uninterruptible Power Supply (UPS).
 (Click on any image to see a larger version.)
Untitled

gpstool has a useful, simple, text-based real-time output display. I'd like to keep an eye on the display as it runs. But the Raspberry Pi base station runs headless - sans display, keyboard, or mouse - tucked inside a narrow drawer near where its antenna is mounted in a skylight. To avoid disrupting this lengthy operation, gpstool runs as a daemon, carefully disassociating itself from any human interface device, and insulating itself from Linux/GNU software signals that might interfere with it. How do I track its progress?

The typical approach in development projects on which I've worked in the past is to log copious text messages to the system log, a service provided by Linux/GNU that saves such messages to a file or files in a protected system directory, which is managed by a privileged syslog process that itself is a daemon. gpstool makes use of this facility. But the rate at which the state of things change in the GNSS module and in gpstool is frequent enough to be a kind of firehose of data to the syslog. It would be a lot more user friendly to carefully ssh into the Raspberry Pi - an action which itself is not without some risk - and use some kind of command line tool to bring up the real-time display, then later discard it and log out, all without interfering with gpstool itself.

This seemed to me to be a common enough problem that instead of merely implementing some specialized solution in Hazer, the git repository home of gpstool and my other GNSS-related software, I should implement it in Diminuto, my git repository containing a general purpose C-based Linux/GNU systems programming library and toolkit. Diminuto underlies Hazer and many of my other projects. 

This article describes what I did and how it works.

Step 1: The Application Programming Interface (API)

The Diminuto observation API provides the following function calls for applications like gpstool. (I'll explain what these library functions do under the hood in a bit.)

FILE * diminuto_observation_create(const char * path, char ** tempp)

The application calls diminuto_observation_create, passing it the path name of an observation file to which it wants to write its real-time display. The library function returns a standard input/output file pointer that the application can use with standard C library calls like fprintf to write its display. The function also provides a pointer to a character string that the application is responsible for providing to subsequent API calls.

FILE * diminuto_observation_commit(FILE * fp, char ** tempp)

When the application is finished with the observation file (all the output for its current display iteration has been written to it), the application calls diminuto_observation_commit with the original file pointer and the original variable containing the pointer to the character string. Once the observation file is committed - and not before - it is visible in the file system to other software, and to humans via the ls command. The library function closes the file pointer and releases the storage associated with the character string, so the contents of the two arguments are no longer useful. The library function returns a null file pointer to indicate success.

FILE * diminuto_observation_discard(FILE * fp, char ** tempp)

Should the application want to discard the current observation file and its contents, it calls diminuto_observation_discard. The observation file is never visible in the file system, and any data contained in it is lost. The file pointer is closed, and the storage associated with the character string is released. The library function returns a null file pointer to indicate success.

FILE * diminuto_observation_checkpoint(FILE * fp, char ** tempp)

Should the application want to keep the current observation file before it commits or discards it - an action that might be stimulated by a human operator doing something like sending the application a SIGHUP or "hangup" software signal (a common idiom in the Linux/GNU world, and one used by gpstool) - the application calls diminuto_observation_checkpoint. A new file appears in the file system that has the name of the original observation file appended with a microsecond-resolution timestamp. This checkpoint file persists in the file system with whatever data was written to the file pointer between the time of the create and the commit or the discard, regardless of when the checkpoint function was called.

Step 2: The Application Task Loop

The Hazer gpstool command line utility implements a task loop in which it reads and processes data from the GNSS module; about once a second, it pauses to update its real-time display.

At the top of the task loop, gpstool calls diminuto_observation_create and gets a pointer to a standard I/O file object. As it processes information from the GNSS module, it writes to this file pointer.

Screen Shot 2020-06-16 at 11.32.54 AM

The contents of the file looks something like this once a complete display has been generated. When this file is complete, gpstool calls diminuto_observation_commit and the observation file containing this display is now visible in the file system. Then gpstool loops back to the top of the task loop, calling diminuto_observation_create again.

Should gpstool receive a SIGHUP software signal, it makes a note of this fact, and eventually calls diminuto_observation_checkpoint.

When gpstool exits, it calls diminuto_observation_discard to clean up any uncommitted temporary file that may have existed from its final, partial, iteration of the task loop.

Step 3: The Library Implementation

This is how the Diminuto observation module is implemented.

diminuto_observation_create:
  • Dynamically allocate a character string containing the observation file path name appended with the string "-XXXXXX". The use of the observation file path name is important, as it insures that this character string will name a file that is in the same directory as that of the observation file.
  • Use the standard mkstemp function to create a file using the character string as its name, but automatically replacing the "XXXXXX" with a randomly generated character sequence like "qx03ru" (actual example) that guarantees that the file is unique in the target directory. The standard function returns an open file descriptor (fd) for this new file.
  • Use the standard fdopen function to create an open standard I/O file pointer for this descriptor.
  • Store the pointer to the allocated character string in the provided variable.
  • Return the open file pointer to the temporary file for success
diminuto_observation_commit:
  • Recover the original observation file path name by truncating the added temporary file suffix from the character string in a second dynamically acquired character string.
  • Close the file pointer using the standard fclose function. This has the desirable side effect of flushing the standard I/O memory buffer to the temporary file.
  • Use the standard rename system call to rename the temporary file to be the observation file. Because the two files are in the same directory, this system call performs this action atomically: the temporary file disappears as if it were deleted, and the observation file appears with the full intact contents of the data written to the temporary file by the application. (rename performs its action atomically provided the source and destination are both in the same file system; being in the same directory is a simple way to insure this.) The rename system call replaces any existing file with the same name, so any prior observation file is deleted from the file system as a side effect. There is never a time when a partially written observation file is visible.
  • Free the character string that contains the temporary file name.
  • Free the observation file path name that was recovered in the first step.
  • Return a null file pointer for success
diminuto_observation_discard:
  • Close the file pointer using the standard fclose function.
  • Delete the temporary file using the standard unlink system call.
  • Free the character string containing the temporary file name.
  • Return a null file pointer for success.
diminuto_observation_checkpoint:
  • Read the system clock using the Diminuto function diminuto_time_zulu.
  • Create a new file name by truncating the mkstemp suffix from the character string that is the name of the temporary file, and append a new suffix that is a UTC timestamp like "-20200616T161740Z958048" (actual example), in another dynamically acquired character string. Note that this contains the year, month, day, hour, minute, second, and microseconds, in an order that will collate alphabetically in time order.
  • Use the standard function fflush to flush the standard I/O buffer for the temporary file out to the file system.
  • Use the standard link system call to create a hard link - a kind of alias in the file system - between the temporary file and the checkpoint file name. Because the two files are in the same directory, this action is also done atomically: the checkpoint file appears in the file system containing all the data that is in the temporary file. Because it is a hard link, as the application continues to write to the temporary file, the data will also appear in the checkpoint file (which are, in fact, the same file, now known under two names). When the temporary file is either committed or discarded, the checkpoint file and its contents will remain.
  • Free the checkpoint file path name.
  • Return the file pointer to the temporary file for success.
Step 4: The Observation Script

We now have a mechanism that gpstool, or any other application, can use to create a sequential series of output files. How to these files get displayed?

The Linux kernel has a facility called inotify, which can be used to monitor file system activity and report it to an application. Lots of existing tools use this facility, like the udev mechanism that supports the hot-plugging of peripherals and the automatic attachment of removable media like USB thumb drives. Most Linux distros have a package of user-space utilities, inotify-tools, that provide command line to this facility.

Diminuto has an observe shell script that calls the utility inotifywait in a loop with the appropriate parameters so that the script is told when a file with the name of the observation file appears in the observation file directory as a result of a move operation. The implementation of the diminuto_observation_commit function emulates what the mv command does, and so it triggers inotifywait to emit the name of the observation file whenever a commit operation is performed. The observe script captures this name and emits it itself to whomever is running it, then loops to call inotifywait again.

Note that observe doesn't actually display the observation file. It has no idea what the observation file contains, or how an application like gpstool wants it displayed, or to where. It just watches for the file to show up in the file system.

(By the way, the observe script has its own SIGHUP implementation. So while the Hazer gpstool uses diminuto_observation_checkpoint to checkpoint the observation file, the Diminuto observe script provides a similar function.)

Step 5: The Rendering Script

To actually display the observation file, Hazer has a peruse script that includes a lot of Hazer-specific context about which Diminuto knows nothing. The Hazer peruse script merely calls the Diminuto observe script with the path name of where gpstool will create the observation file, based on the gpstool -H (for headless) command line option. It pipes the output of observe into a pipeline that reads the file name when it appears, clears the terminal screen, does some minor pretty-printing post-processing of the contents off the observation file, and copies it to standard output. It does this every time a new observation file by that name is moved to the target directory (even though it replaces an existing observation file).

This separates the processing of the input from the GNSS module from the output of the real-time display. I can ssh into the Raspberry Pi running a long-term survey-in as the Differential GNSS base station, fire up the peruse script, check on its progress, and then control-C out of the peruse script, with no impact to gpstool.

Example

base.csv  base.out                         base.out-ee02oB
base.err  base.out-20200608T190630Z025023  base.pid

This is an actual directory listing from a long term base station survey that's running right now.
  • base.out is the latest committed observation file;
  • base.out-ee02oB the current temporary file being written that will replace it once committed;
  • base.out-20200608T190630Z025023 is a checkpointed observation file.
In addition, there are some other files generated by gpstool.
  • base.csv is a dataset of GNSS solutions in Comma Separated Values (CSV) format;
  • base.err is the file to which gpstool is redirecting its standard error output;
  • base.pid contains the process identifier of gpstool used to sent it a SIGHUP signal.
Remarks

The ability to checkpoint observation files is so useful that I use this mechanism even when I'm not doing a long-term survey. Just yesterday, Mrs. Overclock kindly served as my co-driver as we tested the u-blox NEO-M8U, another GNSS module which includes an Inertial Measurement Unit (IMU). The tiny board-mounted module's IMU contains a gyroscope and accelerometers implemented as a Micro-ElectroMechanical System (MEMS). This can be used to approximate the module's location even when the satellite signals cannot be received - like when we drove through a series of highway tunnels on route US6 west of where we live near Denver Colorado.

I wrote a script that combined gpstool using the Diminuto observation capability, with the peruse script, and another script, hups, that sends gpstool a SIGHUP signal any time any key was pressed on the laptop running my software. This made it easy for Mrs. Overclock to capture the real-time gpstool display in a series of timestamped files, for example as we entered a short tunnel about 215 meters in length, and moments later when we exited it.

Screen Shot 2020-06-15 at 1.44.39 PM

Here's a visualization from Google Earth, produced using the Positioning, Navigation, and Timing (PNT) data captured by gpstool about once per second, converted into a Keyhole Markup Language (KML) file by another Hazer script, then imported into Google Earth. (The red continuous visualized path is not a product of the observation and checkpointing mechanism; but that mechanism was used to identify the locations marked by Google Earth with the yellow push-pins.)

The IMU tracked our path from east (right) to west (left) as we went through the tunnel. (You can see remains of the old pre-tunnel road in the satellite imagery too.) As we left the tunnel and the GNSS signals were re-acquired, the NEO-M8U determined that the IMU had our location a little off and corrected it.

I assure you that we didn't do two sudden tire-smoking turns as we exited the tunnel. Although had we done so, my Subaru WRX would have been the vehicle in which to do it.

Monday, June 01, 2020

Location, Location, Location

That's the punch line for the old joke: "What's the three most important factors in determining the desirability of a piece of real estate?" It's also the answer to "What's the three most important factors that affect precision and accuracy when using Global Navigation Satellite Systems?" It turns out that antenna placement is critical, with antenna selection running a distant second.

The data I'm going to present are not the results of careful controlled experiments. They are derived from datasets I already had lying around that I collected while testing Tumbleweed, my Differential GNSS project. But the data exposes what really matters. Some of this data has already appeared in my prior article Negative Results Are Still Results. In Dilution of Precision I explain how a poor view of the sky both limits the number of satellites your receiver's algorithm can use in its iterative solution, and its flexibility in choosing amongst those satellites to find the ones with the widest orbital separation.  More is better.

For all of these examples, I'll be using two metrics: the diameter of the smallest circle that can drawn around all of the positioning solutions gathered over time (using my tools csvlimits and geodesic), and a visualization of the positioning solutions achieved by converting the CSV output of my software into a Keyhole Markup Language (KML) file and imported it into Google Earth (using my tool csv2kml).
(2020-06-03: This article has been edited to append an additional example at the end.)
Location: NGS KK1446

Untitled

This used a survey-grade multi-band GNSS antenna (the white saucer-shaped object on top of the tripod) purchased from Ardusimple with the u-blox ZED-F9P GNSS module with my software running on a Raspberry Pi 3B+. The site was an open field containing NGS survey marker KK1446. The view of the sky was excellent. This combination allowed the receiver to use as many as twenty-nine satellites from four different constellations for its solution.

Uncorrected: 0.6925 meters

Mobile Vagabundo

Corrected: 0.0423 meters

Benchmark Vagabundo

The combination of a survey-grade multi-band antenna with the nearly perfect view of the sky gives an excellent showing. The use of differential corrections reduces the diameter of the solution circle from about 0.7 meters to just about four and a quarter centimeters, a factor of more than sixteen improvement. The corrected view from Google Earth has red marks so small that you need to click on the image to see it in the larger version.

Location: Second Story Window

Untitled

This used a multi-band GNSS antenna (far left) purchased from Ardusimple with the u-blox ZED-F9P GNSS module with my software running on a Raspberry Pi 3B+. The antenna no longer appears in their catalog, and has no markings on it; but from its weight and dimensions, I believe it has a built-in ground plane (a good thing). The site is a south-facing second-story window in my home office. The view of the sky is fair: only the southern half of the sky is visible, but that part is mostly unobstructed. Even with this partial view, the receiver was able to see as many as twenty-six satellites from four different constellations.

Uncorrected: 5.1404 meters

mobile-gold

Corrected: 2.3099 meters

benchmark-gold

The use of differential corrections reduces the diameter of the solution circle from about five meters to about two and a quarter meters, a factor of about two. This illustrates how differential correction can only do just so much. The uncorrected diameter is very typical of consumer GNSS receivers with their integral patch antennas and a good view of the sky.

Location: Lab Bench

Zhejiang JC SY-301 Helical GNSS Antenna

This used a multi-band GNSS antenna purchased from Ardusimple with the u-blox ZED-F9P GNSS module with my software running on an Intel i7-class system. The antenna - the small black vertical cylinder attached directly to an Ardusimple SimpleRTK2B board in a 3D-printed case - is a helical antenna intended for applications like aerial drones, its principle advantages being its light weight and its relative insensitivity to orientation. In my application, the antenna picked up only two of the four GNSS constellations, GPS and GLONASS, during the test period, despite being advertised to pick up all four. The site is the lab bench that sits on the opposite side of my home office, a little over two meters from the same window as above. The view of the sky is poor. Typically only four or five satellites from the two constellations are visible, for example, three GPS, two GLONASS; had this not been a multi-band antenna, it would not have been able to make a position fix using the minimum of four satellites.

Uncorrected: 37.5433 meters

mobile-cadmium

Corrected: 68.2234 meters

benchmark-cadmium

You are not imagining this: the uncorrected test run was better than the corrected test run. (I was so skeptical myself that I triple checked that I hadn't gotten the datasets confused.) Neither were worth writing home about. Some of the data points from on the corrected test blew clean over my home to the lot to the east, and even to the lot east of that. This test setup is adequate for regression testing my software for basic functionality, but in no way yields actually useful positioning except at very coarse granularity.

Location: Second Story Window (added 2020-06-03)

Untitled

This is a later test in which I took the same helical multi-band GNSS antenna (on the left) and tested it in the same location as and alongside the prior setup (which used the antenna on the right) in the south-facing home office window. As before, it uses the Ardusimple SimpleRTK2B board with the u-blox ZED-F9P module, although in this case the Linux/GNU host running my software was an ancient H-P Mini 110 netbook with an Intel i686-class processor. (The u-blox module does all the heavy lifting; my software merely runs the real-time display and collects the data.)

Uncorrected: 3.4650 meters

Screen Shot 2020-06-03 at 9.19.49 AM

Merely moving the helical antenna a couple of meters or so from the lab bench to the window makes a huge difference. In fact, in this one instance the small light helical antenna slightly outperformed the larger antenna from the prior test. Moving the antenna to a better view of the sky also allowed it to use as many as twenty-five satellites from all four constellations, versus the barely minimally adequate five satellites from two constellations. The solution diameter of 3.465 meters - over eleven feet - still illustrates how coarse the resolution of uncorrected GNSS is. An aerial drone using this configuration and seeing this kind of resolution would find autonomous navigation problematic.

Location, Location, Location

Differential GNSS can make a big difference, and antenna selection is important. But unless you have proper antenna placement, neither a good antenna nor differential corrections are going to save you.

Thursday, May 28, 2020

Negative Results Are Still Results

Tumbleweed is the code name for my experiments with the u-blox ZED-F9P module and its ability to both generate and accept differential corrections to Positioning, Navigation, and Timing (PNT) solutions using Global Navigation Satellite Systems (GNSS) like the U.S. Global Position System (GPS). Tumbleweed leverages and expands the work I did in another project, Hazer, an open source C-based library and toolkit that I wrote to help me explore the world of satellite-based geolocation.

Hazer and Tumbleweed have challenged my capabilities as an embedded software developer. The artifacts that have come out of those projects include a Network Time Protocol (NTP) server with a cesium atomic clock, and a Differential GNSS (DGNSS) system that has the potential to achieve accuracies at the centimeter level or better from a technology - GPS -  that fundamentally is limited to accuracies of about five meters or so. At the beginning of both of these efforts I pondered how I was going to test them. Projects which, if they were successful, would result in devices that could measure time and space to a resolution far better than any measurement instrument I owned, could afford, or could borrow.

In this longish article I will try to describe the difficulties in testing the DGNSS system that came out of Tumbleweed, and how I tried - with varying amounts of success - to address them.
You can click on any image to see a larger version. A list of references and sources for this work can be found in the README file in the Hazer repository on GitHub.
Precision versus Accuracy

There are two basic and different aspects to measuring systems like Tumbleweed: precision and accuracy. The classic way in which the difference between precision and accuracy is explained is this:

An archer shoots arrows at a target. If an arrow lands right in the bullseye, the archer is said to be accurate. If all of the arrows land near the same spot on the target (but not necessarily in the bullseye), the archer is said to be precise. Accuracy is about correctness. Precision is about consistency. The archer wants to be both accurate and precise.

You can see how this applies to devices like clocks. A precise clock strikes noon at the exact same time every day. An accurate clock strikes noon when it really is noon. You'd like a clock to be both accurate and precise.

As we will see, measuring the precision of Tumbleweed is fairly straightforward. Trying to measure its accuracy dropped me down a rabbit hole from which I am still trying to extricate myself. It launched me into a self-study program into the science of geodesy:
Geodesy is the science of accurately measuring and understanding three fundamental properties of the Earth: its geometric shape, its orientation in space, and its gravity field— as well as the changes of these properties with time. [NOAA]
It also unexpectedly required that I learn a little bit about surveying:
Surveying or land surveying is the technique, profession, art and science of determining the terrestrial or three-dimensional positions of points and the distances and angles between them. A land surveying professional is called a land surveyor. These points are usually on the surface of the Earth, and they are often used to establish maps and boundaries for ownership, locations, such as building corners or the surface location of subsurface features, or other purposes required by government or civil law, such as property sales. [Wikipedia]
To fully appreciate how deep this rabbit hole is, a little context is in order.

Background

Positions on the Earth are traditionally measured in terms of latitude and longitude.

Latitude is the number of degrees north or south a location is from the Equator. The Equator is the circumferential waist of the Earth as defined by its axis of rotation. While there might be come variation about where exactly the Equator is - its position drifts about nine meters or thirty feet per year as the Earth wobbles - it is a natural reference point from which to measure latitude. The Equator is zero degrees (0°). and is a great circle: a circle whose plane passes through the center of the Earth (a position that is, however, not nearly so well defined as you might think). Locations north of the Equator are between 0° (the Equator) and 90° north (the North Pole), and locations south of the Equator are between 0° and 90° south (the South Pole). (Sometimes southern latitudes are represented as negative values.)

Longitude has no such natural reference. The arbitrary establishment of a Prime Meridian, the meridian or circumferential line (also a great circle) from the North Pole to the South Pole which defines zero degrees longitude, was, for a long time a matter of heated debate during the Age of Sail. And also, it must be said, of national pride amongst the seafaring superpowers that depended on accurate compasses, maps, and clocks in order to navigate with sufficient skill to loot the New World and enslave its inhabitants.

Untitled

In 1884, the Prime Meridian was established by international treaty as running through the Royal Observatory in Greenwich England. The Royal Observatory was one such place where astronomical observations were made with sufficient accuracy to set clocks that could be used to navigate at sea, using them to measure longitude comparing the time on the clock on board a ship with local noon, when the Sun was at its highest point in the sky, or with other astronomical objects like the North Star.

Untitled

This Prime Meridian was the standard for many decades, despite the disgruntled French who thought the Prime Meridian should run through Paris. Locations east of the Prime Meridian are said to be between 0° and 180° east, and locations west are between 0° and 180° west. (Sometimes western longitudes are represented as negative values, although in some contexts I have seen eastern longitudes represented negatively.)

GPS Meridian at the Royal Observatory

In 1973, the International Earth Rotation and Reference Systems Service (IERS), the same organization that inserts and potentially removes (although that has never happened) leap seconds into the accepted definition of Universal Coordinated Time (UTC) to keep clocks in sync with changes in the rotation of the Earth, established a new Prime Meridian. It based this new Prime Meridian on the latest scientific data at the time, much of it based in satellite observations, regarding the shape of the Earth and its center of gravity. This new Prime Meridian, which is also the GPS Meridian, is on a lawn about one hundred meters east of what is now referred to as the Greenwich Meridian, where I am standing in the photograph above, with the GPS Test Plus app on my iPhone. (Photo credit: Mrs. Overclock.)

Prime Meridian at the Royal Observatory

The Greenwich Meridian, where I took the screenshot above, is about six seconds longitude west of the GPS Meridian.

Latitude and longitude are not the only coordinate systems in broad use today. The U.S. military uses the Military Grid Reference System (MGRS). This system lays down a grid of equal sized squares, the standardized size of which depends on the resolution of the map in question. MGRS lacks the sometimes problematic effect that longitude lines get closer together as you approach either pole, so that the distance between degrees of longitude ranges from just over 111 kilometers at the Equator to zero at either pole. (Degree of latitude are always sixty nautical miles apart by definition.) Many GPS receivers can be switched to display using MGRS.

While latitude and longitude are commonly used in civilian applications, including land surveying and sea navigation, there are different systems used for measuring latitude and longitude, and these systems do not yield the same results. This is due to the fact that they use different models, or datums, of the shape of the Earth. Things would be far simpler if the Earth were perfect sphere. And indeed, for short-ish distances, such an assumption served ancient mariners well. But the Earth is an oblate spheroid or ellipsoid, bulging with middle-age spread at its Equator, and flattened at either pole with the geodesic equivalent of male pattern baldness.

In 1901, the geodetic center of the United States was defined by the U.S. National Geodetic Survey (NGS), an agency of the U.S. government established in 1807, to be on Meade's Ranch, Kansas, a location that is today on private property. Since then, all land surveying in the continental United States has been done relative to this marker. Every single surveyed location is traceable through a chain of surveyor measurements and established benchmarks that eventually lead to the marker at Meade's Ranch.

In time, the NGS defined the North American Datum of 1927 (NAD27). You still see references to this abstract model of the shape of the earth and its center of gravity in old surveying documents today.

In 1983, a new, more accurate datum, NAD83, was adopted by the NGS. It was based on far more accurate measurements, mostly done from satellite observations, and it became the standard for all surveying in the United States today. (In between NAD27 and NAD83 there were other datums, just as one day, perhaps relatively soon, NAD83 will be replaced with an even more accurate datum.)

NAD83 is based relative to the North American tectonic plate. Because the North American tectonic plate moves at a rate of over two centimeters per year (and the Pacific plate, on which part of California sits, as much as seven to eleven centimeters per year), measurements made using NAD83 do not change (much) as the tectonic plate takes its leisurely stroll over a thick layer of lubricating rock.

KK1446, Google Nexus 5, GPS Test +

"Much" because the NGS survey markers - typically inscribed cast metal disks embedded in concrete pillars or in sidewalks - are often embedded in ground which moves relative to local landmarks just due to smaller, local shifts in the ground. Or, as I like to say, "Shift happens".

Untitled

(Local municipalities will also establish their own survey markers, sometimes fiberglass embedded in concrete.)

The use of relatively local coordinate systems on which to base latitude and longitude is quite common, and especially important in locales in which there is a lot of continental drift due to tectonic movement. New Zealand resides on two different tectonic plates, the Australian plate (the North Island and the western part of the South Island) and the Pacific plate (the rest of the South Island). The Pacific plate rotates counter-clockwise compared to the relatively stable Australian plate, making surveying in New Zealand especially challenging.

In 1984, the U.S. National Geospatial-intelligence Agency (NGA) established the World Geodetic System (WGS84). This is the datum on which GPS is based. The NGA is responsible for, amongst other things, providing accurate maps to the U.S. Department of Defense. WGS84 creates yet another coordinate system, but unlike NAD83, this one one is not relative to a tectonic plate, but is instead a global coordinate system. Because WGS84 deliberately does not take plate tectonics into account, GPS coordinates of a particular location may change (slowly) over time as the continental plate drifts. But because GPS coordinates can be made so swiftly - in minutes or seconds instead of hours or days required for expensive manual surveying - this isn't generally an issue.

However, this means that the GPS coordinates - made using WGS84 - of an NGS marker - whose position was originally determined using NAD83 - cannot be directly compared. Worse yet, any conversion between WGS84 and NAD83 has to take the date the measurements were made into account in order to adjust for continental drift.

Now that we've covered all the stuff I had to learn and worry about regarding exactly what the results from measurements I did with Tumbleweed even mean, we're ready to talk about what metrics of quality I used and what tools I used to measure and analyze them.

Quality Metrics

The gpstool command line utility is the Swiss Army knife of Hazer. It parses the output of the ZED-F9P module using the functions in libhazer: National Marine Equipment Association (NMEA) 0183 sentences, the most typical output of GNSS receivers, are parsed by the Hazer functions; proprietary u-blox binary messages in UBX are parsed by the Yodel functions; packets conforming to the Radio Technical Commission for Maritime Services (RTCM) Special Committee (SC) 104 standard are parsed by the Tumbleweed functions.

When used with the ZED-F9P (and many other u-blox GNSS receivers), gpstool configures the module to emit the proprietary High Precision Geodetic Position Solution (UBX-NAV-HPPOSLLH) message, a UBX message with a higher precision latitude, longitude, and altitude than is typically available via NMEA. The message also includes a Horizontal Accuracy Estimate and a Vertical Accuracy Estimate in units of one-tenth of a millimeter. How these estimates are computed isn't documented (that I've found). But it is plausible that the ZED-F9P could base these metrics on how closely it was able to bring the range spheres using the iterative least squares algorithms before the range spheres began to move away.

HPP   39.821674271 -105.095174932 ±     0.0141m
HPA   1722.1183m MSL   1700.5934m WGS84 ±     0.0100m

The High Precision Position (in decimal degrees) and the High Precision Altitude (in meters), along with the accuracy estimates (in meters), are displayed in real-time by gpstool as the HPP and HPA output lines. The HPA line includes both the altitude above Mean Sea Level (MSL) and above the WGS84 ellipsoid (WGS84). The HPP line is formatted such that the coordinates can be cut and pasted directly into Google Maps. The example above is from an actual field test of a Tumbleweed Rover receiving differential corrections from a Tumbleweed Base.

The Horizontal and Vertical Accuracy Estimates provide a self-reported metric of quality for the ZED-F9P, although since the module doesn't know what the actual coordinates are, I'm not sure you can actually call these metrics a measure of "accuracy". But given that uncorrected GPS under the best circumstances is accurate to about five meters or about fifteen feet, and under poor circumstances a lot worse, small numbers for these metrics are a good thing. Note in the example above that the horizontal accuracy estimate is 1.41 centimeters (a little over half an inch), and the vertical accuracy estimate is 1.00 centimeters (less than half an inch).

Screen Shot 2020-05-28 at 7.25.51 AM

gpstool has the option (-T) of appending each instance of the high precision solution, along with the GPS Time, to a Comma Separated Value (CSV) file. The ZED-F9P is configured by gpstool to emit a high precision solution once a second (one hertz). The CSV file simplifies analysis with Excel or other tools, especially when looking at how the position solution changes over time.

The geodesic command line utility in Hazer computes the shortest distance between two sets of latitude and longitude coordinates as measured on the surface of the WGS84 ellipsoid. It is implemented using code borrowed from the GeographicLib library developed by Charles F. F. Karney and licensed under the MIT/X11 License. geodesic is useful for comparing coordinates generated with Tumbleweed with those from other sources like professional differential GNSS hardware and from NGS benchmarks, and can be used to assess both the precision and accuracy of the device.

The haversine command line utility in Hazer is similar to geodesic except that it uses the simpler Great Circle Route algorithm, from spherical trigonometry, that assumes that the Earth is a perfect sphere. For small distances it produces comparable results. For large distances, the output of the two utilities typically differ significantly. (I haven't used haversine since I wrote geodesic.)

The csvlimits command line utility in Hazer reads a CSV file from standard input and displays the minimum longitude value, the minimum latitude value (not necessarily from the same sample), the maximum longitude value, and the maximum latitude value (ditto). These values form the opposite corners of a square which bounds all of the latitude and longitude coordinates in the CSV file. It can also be thought of as a diameter defining a (slightly larger) circle. All of the positioning solutions will fit within the square/circle. Using the geodesic utility on these synthesized minimum and maximum coordinates gives a measurement of the precision of the module in different circumstances when it is stationary.

The csv2kml command line utility in Hazer reads a CSV file from standard input and converts it into a Keyhole Markup Language (KML) file which it emits to standard output. KML is an XML-based language that is understood by Google Earth. This allows you to take the CSV output from gpstool and visualize it graphically on top of actual satellite imagery of the Earth's surface. This can be used to assess both the precision and the accuracy (as it compares with Google Earth) of the device.

The NGS NGSDataExplorer is a web-based tool that gives you access to the NGS online database of documented survey markers. The tool provides a graphical map location of the marker and access to the official data sheets describing the marker, its location, and how it was surveyed. This can be used, with some conversion, to compare the NGS coordinates with that of the device.

The National Oceanographic and Atmospheric Administration (NOAA) Horizontal Time Dependent Positioning (HTDP) web-based tool converts between geodesic coordinate systems like NGS83 and WGS84, taking the time the measurements were made into account. This allows you to assess the accuracy to the device by comparing its output with the coordinates of, for example, an NGS survey marker.

The mapstool command line utility in Hazer coverts coordinates in a variety of formats, including those used in the NGS data sheets, into a decimal degrees form that can be used with geodesic, HTDP, Google Maps, and Google Earth.

The Tumbleweed Base is somewhat arbitrarily (by me) configured to survey-in to an accuracy of 2.5 centimeters or just less than an inch; it looks days of uninterrupted geolocation for the long term weighted average of the positioning solution to converge to that accuracy. The duration (or even success) of the Base survey depends greatly on antenna placement. A survey with marginal antenna placement, if successful at all, can take several days. (I'll talk more about that in a later article.) The Base surveyed into the following coordinates.

High Precision Position: 39.794275645, -105.153414843
Horizontal Accuracy Estimate: 0.0204m
High Precision Altitude: 1708.4968m MSL 1686.9969m WGS84
Vertical Accuracy Estimate: 0.0144m

Screen Shot 2020-05-28 at 9.43.55 AM

These are the coordinates of the skylight above my kitchen, near the peak of the roof, although Google Maps places it just to the upper left of the skylight. Whether this is a measurement error in the survey-in process or some slop in Google Maps is an open question.

Precision and Absolute Accuracy

Untitled

I measured precision and absolute accuracy using NGS marker KK1446 "Dover". "KK1446" is the marker's Persistent Identifier (PID) and "Dover" its designation (it's off a street named Dover Way). KK1446 is set in a small field next to a fenced off municipal water tank at the edge of a suburban housing development in Arvada Colorado. It was established in 1977 and resurveyed in 1993 at the following NAD83 coordinates (shown in the NGS hour minutes seconds data sheet format).
39 49 17.98157(N) 105 05 42.59638(W)
HTDP converts those NAD83 coordinates into following WGS84 coordinates (also in NGS data sheet format).
39 49 17.99968(N) 105 05 42.64410(W)
mapstool converts those coordinates from NGS data sheet format into decimal degrees.
39.821666578, -105.095178917

Untitled

The stationary Rover antenna was centered directly over the KK1446 and twenty minutes of data was collected, yielding over a thousand data points in the CSV file. Precision was determined using csvlimits and geodesic to compute the diameter of the enclosing circle (or square). Absolute accuracy was determined using geodesic to compute the distance between the coordinates of the marker (converted into WGS84) and the high precision position determined by the ZED-F9P.

Two tests were run: one using corrections from the Base, and one with no corrections. The CSV files from each test were converted using csv2kml and imported into to Google Earth. The corrected test run is precise enough that you'll need to click on the images to see a larger version in order to see the red plot made by Google Earth; blowing up the images any more blurs the detail.

Screen Shot 2020-05-14 at 1.49.20 PM

Date: 2020-05-14
Location: NGS KK1446
Configuration: Corrected Rover
High Precision Position: 39.821674271, -105.095174932
Horizontal Accuracy Estimate: 0.0141m
High Precision Altitude: 1722.1183m MSL 1700.5934m WGS84
Vertical Accuracy Estimate: 0.0100m
Precision: 0.0423m
Accuracy: 0.9198m

Screen Shot 2020-05-18 at 12.33.24 PM

Date: 2020-05-18
Location: NGS KK1446
Configuration: Uncorrected Rover
High Precision Position: 39.821671099, -105.095169044
Horizontal Accuracy Estimate: 0.4327m
High Precision Altitude: 1724.6724m MSL   1703.1475m WGS84
Vertical Accuracy Estimate: 0.6372m
Precision: 0.6925m
Accuracy: 0.9831m

The precision - the diameter of the circle in which all of the position solutions lie - of the corrected Rover test is a little over four centimeters. This is not as good as I had hoped (my original goal was to get within 2.5 centimeters or about an inch), but far better than the uncorrected precision which was almost seven-tenths of a meter or more than two and a quarter feet.

The accuracy - how far the computed WGS84 coordinates differ from the surveyed NAD83 coordinates converted to WGS84 - of the corrected Rover is not that great: nine-tenths of a meter, and not that much better than the uncorrected Rover.

Since I don't know exactly how the ZED-F9P computes its own accuracy estimates, I'm not sure how to compare them with my own quality metrics.

I would be tempted to blame the multi-step conversion process I used to get WGS84 coordinates of KK1446 that I could directly compare with my own results, had not a chance meeting occurred.

Comparative Accuracy

Untitled

During one of my field tests in October 2019 I discovered I wasn't the only person that liked using NGS KK1446 for testing their GNSS equipment: I had to share the spot with a professional surveyor who showed up to check his own equipment as part of a job. He very graciously shared the results from his high-zoot survey-quality Trimble GNSS with me. I confess to a certain amount of equipment envy.

Dover job properties

One of the advantages of professional equipment is his Trimble displayed its GNSS measurements using the NAD83 datum, standard in the surveying field, instead of WGS84, the standard for GPS.

Dover coordinates

Taking the Trimble's NAD83 coordinates, using the HTDP web tool to convert them to WGS84 coordinates, then using geodesic to compute the distance between those coordinates and the KK1446 coordinates converted to WGS84 (because geodesic assumes WGS84) yields a difference of 0.0307 meters or about three centimeters. That's very good, and far better than what I've been able to achieve.

Relative Accuracy

Untitled

I measured relative accuracy by building a test fixture. I used a sheet of plywood on which I marked off, as carefully as I could with a square and a meter tape, one square meter. I got the board approximately level using a torpedo level and some makeshift leveling blocks.

Untitled

I set my Rover in its sunshade and on its stand on the board (which wasn't really necessary for the results, but allowed me to reach all four corners of the meter square without using more coax cable). I collected five minutes of data at each corner, keeping the antenna as motionless and level (using a circular level built into the pole) as possible. I repeated the cycle of all four corners twice.

The geodesic measurement between successive corners ranged from 0.9810 meters to 1.010 meters with a mean of 0.9939 and a standard deviation of 0.0101 (computed using Excel). That's pretty good; much of the variation may be a result of my own inability to lay out a perfect square meter on a sheet of plywood and to hold the survey pole with the antenna steady while my neighbors watch me with their usual curiosity. My original goal was an accuracy of 2.5 centimeters or about an inch, and this falls well within that.

Negative Results Are Still Results

The title of this article came from a remark my thesis advisor Bob Dixon made thirty-seven years ago, when I was working on my master's thesis in computer science at Wright State University in Dayton Ohio.

My original goal for this project was to see if consumer grade differential GNSS technology was precise enough for applications like agriculture: self-navigating tractors, etc. The results indicate that the ZED-F9P is indeed capable of precise, self-consistent, repeatable geolocation, and that the use of differential corrections from a base station also equipped with a ZED-F9P makes a significant measurable difference.

I was disappointed in its accuracy results when trying to duplicate the coordinates of surveyed markers like KK1446. There are a lot of reasons why this lack of accuracy may have been the case, although the success of the Trimble in this regard makes me think better accuracy should be achievable (and that you get what you pay for). It's possible that I need a better antenna, or that there's a mistake in my own calculations. (While some of the tools I wrote for Tumbleweed use double precision floating point, gpstool and the functions it uses in libhazer that parse the output of GNSS devices use only integer arithmetic, even though the displayed values may appear to be decimal. I have also tried to keep track of the significant digits in various internal calculations.)

I keep alluding to the important of antenna selection and placement in achieving good results with GNSS, whether or not differential corrections are being used. This will be the topic of an upcoming article.

Acknowledgements

Thanks to Charles Karney, who wrote GeographicLib and open sourced it, and of which I used only a tiny part in my geodesic computations.

Thanks to Brad Gabbard of Flatirons, Inc., a surveying, engineering, and geomatics firm based in Boulder Colorado, who graciously put up with the old guy and his a tub of improvised equipment, and who so generously shared his results with me.