Showing posts with label Raspberry Pi. Show all posts
Showing posts with label Raspberry Pi. Show all posts

Monday, April 08, 2024

Ancient History

I bought a four terabyte (4TB) SSD the other day at the local big box store. A Samsung T7 Shield (which I think just means it comes with a rubberlike case around it). It was substantially discounted, probably because the new T9 model is out. Easily fits in my shirt pocket. Hooked it up via the included USB cable to our network-attached storage box, and I'm now using it to automatically back up two Mac laptops and a Mac desktop at the Palatial Overclock Estate.

Mind blown.

Because I am ancient almost beyond belief - it's a miracle I'm still alive, especially considering my hobbies - I remember thirty years ago helping to write a proposal to DARPA to build a one terabyte (1TB) hierarchical storage system that would have included rotating disks and a robotic tape library. It would have taken up an entire room. Can't blame them for not funding it. Someone smarter than me (which could have been just about anyone) probably saw this all coming.

That same organization, the National Center for Atmospheric Research (NCAR) in Boulder Colorado, had the only CRAY-3 supercomputer outside of Seymour Cray's Cray Computer Corporation. Today, your Raspberry Pi Single Board Computer (SBC) - and not even the latest Pi model 5 - has more horsepower than that CRAY-3. And the SBC would fit in your shirt pocket as well.

Don't bet against Moore's Law.

Although as I am always quick to point out, what exactly Moore's Law implies has changed over the past few years. Which is why I was tickled when someone who is using my Diminuto C systems programming library passed along a command line to build the software by running make across sixteen parallel threads of execution - taking advantage of the trend towards multicore processors, now that it's become difficult to make individual processors faster. Between much of the build process being I/O bound, and having four processor cores on the Raspberry Pi 5, this approach really speeds up the makefile.

CRAY-3

That's me, about thirty years ago, leaning against the world's most expensive aquarium; the CRAY-3 logic modules were visible under the transparent top, fully immersed in Fluorinert.

Wednesday, March 20, 2024

Converting GPIO from the legacy sysfs ABI to the ioctl ABI in Diminuto and Hazer

It could be that no one but me is using my "Hazer" GNSS library and toolkit, or the "Diminuto" C-based systems programming library on which it depends. But just in case: I'm close to finishing testing of the develop branch of both repos, both of which have some major changes to how General Purpose Input/Output (GPIO) - the generic term for controlling digital input and output pins in software - and merging develop back into the master branch.

This was all motivated by my being one of the lucky few to get a backordered Raspberry Pi 5, and putting the latest version of Raspberry Pi OS, based on Debian "bookworm" and Linux 6.1, on it, only to find when unit and functional testing my code that the deprecated sysfs-based GPIO ABI no long worked. This wasn't a big surprise - I had read at least two years ago that the old ABI was being phased out in favor of a new ioctl-based ABI. My code makes heavy use of GPIO for a lot of my stuff, e.g. interrupts from real-time sensors, the One Pulse Per Second (1PPS) signal from GNSS receivers, status LEDS, etc. So it was finally time to bite the bullet and replace all the places where I used the sysfs-based Diminuto Pin feature (diminuto_pin) with a new ABI using the ioctl. Hence, the Diminuto Line (borrowing a term from the new ABI) feature (diminuto_line).

Line is now used in place of Pin in all of the Diminuto functional tests that run on  hardware test fixtures I wired up many years ago just for this purpose, and all the functional tests work. The Hazer gpstool utility has similarly been converted to using Line instead of Pin and has been tested with an Ardusimple board using a u-blox UBX-NEO-F10T GNSS receiver.

IMG_5717

(That's a Pi4 on the left connected to my test fixture, and a Pi5 on the right connected to the GNSS receiver.)

Two complaints.

[1] The new ABI is woefully under documented. However, I found some code examples in the Linux kernel source repo under tools/gpio that were essential to my understanding. (I chose not to use the new-ish libgpiod for my work, for reasons, but that is a story for another time. I have no reason to believe that it's not perfectly fine to use.)

[2] The way the ioctl GPIO driver is implemented on older versus newer Raspberry Pi OS versions makes it difficult - I am tempted to say impossible, but maybe I'm just not that smart - to write code that easily runs on either platform using the new ABI. Specifically, the GPIO device drivers in the OSes use a different symbolic naming scheme, making it impossible for application code to select the correct GPIO device and line offset portably on the two platforms. But maybe I'm just missing something. (I hope so.)

I like the new ioctl ABI, and expect to use it exclusively moving forward, even though this will orphan Pis I have that might run older versions of the OS. (I think I have at least one example of every Pi version ever sitting around the Palatial Overclock Estate. Ten of them run 24x7 in roles such as a web server, an Open Street Map server, a Differential GNSS base station, and an NTP server with a cesium atomic clock for holdover.) I have tagged the last version of both repos that still use the sysfs ABI.

That's it.

Update (2024-03-23)

I merged the develop branch in to the master branch this morning. Both the Diminuto build and the Hazer build passed sanity and functional testing (and I'm currently running the long-running "geologic" unit test suite against Diminuto). I had tagged the old master branch in both repos with the name sysfsgpio in case I needed to build them, but I don't anticipate any further development of the old code.

Monday, July 03, 2023

Hazer with the U-blox NEO-F10T GNSS Receiver on the Ardusimple SimpleGNSS Board

It's been a while since I talked about my GPS/GNSS efforts. Some time ago I bought a SimpleGNSS board from Ardusimple to try out. The SimpleGNSS has the new U-blox NEO-F10T GNSS receiver. This was my first experience using a new U-blox generation 10 device. It is the first GNSS device of any kind I've used that includes features specific to the latest version 4.11 of the National Marine Electronics Association 0183 standard. And it's the first I've used that's capable of receiving the new L5 band signal from the latest Block III GPS satellites.

I was about ready to dismantle this experiment. But before I removed it from my workbench, I thought I'd update y'all on my latest version of the Hazer library and its gpstool utility.

Hazer is a Linux/GNU/C-based library that supports the processing not only of the usual NMEA 0183 output of GNSS receivers, but also proprietary binary output like UBX from U-blox devices, and CPO output from Garmin devices. It also handles input and output of RTCM messages in support of differential GNSS, yielding geolocation precision down to about a 1.5 centimeters. (I run a DGNSS fixed base and a stationary rover 24x7 at the Palatial Overclock Estate.)

gpstool is the Swiss Army Knife of Hazer. It's one of those old-school tools that has a nightmare of command line options. I use it to functionally test the library, and also as the main component of most of my GNSS efforts. You wouldn't want to use gpstool to navigate cross-country (although I have used it in conjunction with OpenStreetMap to generate a moving map display in real-time). But I find it really useful for testing and evaluating new GNSS devices and just generally futzing around with geolocation and precision timing.

(You may want to click on the short video clips to view them on YouTube instead of from this blog; the UI seems to crop the images, losing some information. You can also click on photographs to see a larger image.)

Here is a short video clip of a Raspberry Pi 4B running gpstool with the NEO-F10T connected via USB.

Besides processing the output of the U-blox device, gpstool is following the 1 Hz One Pulse Per Second (1PPS) digital output signal from the device, which is syntonized to GPS time, and strobes another digital line on the Raspberry PI, syntonized to 1PPS (subject to software latency), to which I've attached a red LED.

Here is a short screen capture that shows the output from gpstool. I've SSHed into the Raspberry Pi from my Mac desktop to run gpstool. The utility uses some simple ANSI screen controls to update the display in real-time. I'm viewing standard output directly from gpstool, but the utility also has the ability to run headless in the background as a daemon, and you can view this real-time display remotely at leisure. (This is how I run my DGNSS systems.)

If you have perused the output of gpstool in any of my prior articles, you may notice a new field in the SAT lines showing the frequency band used by the device to receive data from the indicated satellite, e.g. L1 C/A. Some GNSS devices (like this one) may receive data from the same satellite over more than one band. (I confess this "new feature" is a long delayed bug fix because I botched the handling of new fields in version 4.10 of the NMEA 0183 spec. I have no excuse.)

gpstool isn't just a passive monitor. You can use its command line options to send NMEA, UBX, and other message formats to the device to interrogate and configure it. I did this with the NEO-F10T. Here is a screen shot of a snippet of a script similar to the one I used to generate the output shown in this article.  It sends a batch of UBX messages, sequentially, waiting for an acknowledgement for most of them, to configure the device.

Screen Shot 2023-07-06 at 13.28.38

You can find this script in its entirety in the Hazer repository on GitHub.

Finally, in addition to the standard output, gpstool generates log output on standard error, which here I capture in a file. (If you run gpstool headless, log output can also be automatically directed to the system log without any need for a command line option or API call.)

Hazer gpstool Example Log File

In this example, I have the log output level set to INFO (informational) and above; setting it to the DBUG (debug) level and above generates more output than I typically need unless I am actually debugging new code. (The logging system is a feature of Diminuto, my Linux/GNU/C-based systems programming library on which Hazer and gpstool are built; Diminuto has its own ginormous feature set.)

That's a quick update of some of my latest poking around with GNSS. I'm always on the lookout for new (to me, anyway) GNSS devices to play with!

Wednesday, June 07, 2023

Bug Blast from the Past

I found a Day One bug in some unit test code today that I wrote in 2014, nine years ago. What caused the bug to show up now? I had botched where I had put a thread scheduling yield function call relative to a critical section in the unit test. Apparently this is the first time I have ported this multi-threaded C code to a single-core processor - in this instance, a Raspberry Pi Zero

This is remarkable. As someone said when I related this story to them, it's not uncommon to uncover bugs in code being ported from single core to multi-core processors, but the opposite transition happens seldom enough that this kind of event is rare.

For most of my career - which began before microprocessors even existed - all such computer chips were single core, that is: capable of executing only a single machine instruction at a time. I started writing this library, Diminuto, for a single core ARM4 processor in 2008. But it didn't take long to migrate to inexpensive ARM and Intel chips that could execute more than one instruction at a time. Today, I routinely use small development and test machines that have four or even eight processor cores. As a former colleague of mine, who I met when I worked in Boulder at the National Center for Atmospheric Research and he with Seymour Cray at Cray Computer Company in Colorado Springs, once remarked, your Raspberry Pi 4B single board computer can, by most metrics, outperform Seymour's last supercomputer, the Cray-3 a.k.a. "The Fish Bowl".

The evolution of multi-core microprocessors was largely motivated by, IMNSHO, the end of Moore's Law, at which point doing stuff in parallel was the only way to produce faster microprocessors... faster, at least, for those applications which could take advantage of them. See also: Amdahl's Law.

CRAY-3

Yes, that's a young(er) iteration of me leaning on a Cray-3 in NCAR's computer room at their Mesa Laboratory in Boulder Colorado. Compared to what we can do with computers today, those were the old days, but not necessarily the good old days.

Friday, August 19, 2022

Playing with a RISC-V SBC running Linux

Recently I began playing with a VisionFive RISC-V-based Single Board Computer (SBC). It was designed by StarFive. I purchased it from ameriDroid.

StarFive VisionFive 2-core RISC-V SBC "boron"

RISC-V ("risk five") is a relatively new Instruction Set Architecture (ISA), developed by a team at UC Berkeley, that is entirely open source, unlike Intel and ARM processor ISAs. This is the first RISC-V board I know of that is designed to run Linux (in this case, based on the Fedora distribution). It has two 64-bit 1GHz SiFive U74 CPU cores and eight gigabytes of RAM.

"boron" is my code name for this project. (Every project gets a code name.) I added the nylon feet to hold the board up off the anti-static mat on my workbench. I started out with a keyboard, mouse, and HDMI display attached to the board, but since it supported SSH right out of the box, I quickly reverted to using the command line to build and test my SW, my preferred approach. Also, it reduces the cable clutter on my workbench, which is supporting another project at the moment.

The two projects of mine that I use the most, Diminuto (a C systems-programming library) and Hazer (a GNSS toolkit), were unusually easy to clone, build, and run; unlike most other Linux distros, I didn't have to install any optional packages (much to my surprise). I have successfully completed running all the unit tests for both projects, and am now moving on to functional testing and practical use.

Since my work tends not to be very CPU intensive, I haven't run any performance tests. I'll probably leave that to someone else. I was more worried about SW functionality. But note that this board is more expensive, and and its processor slower, than the ARM-based Raspberry Pi 4 SBCs I use these days.

But the ARM processor design, and the BroadCom implementation of it, on the Pi is closed, and I appreciated the chance to try my stuff out on the open RISC-V, after having read so much about it over the past few years.

Update: 2022-08-20

Just to provide some evidence that I have non-trivial stuff running on the VisionFive, here's a screen snapshot of gpstool running on the board.

Screen Shot 2022-08-19 at 16.22.19

gpstool is the Swiss Army knife of the Hazer toolkit, which is built on top of Diminuto. Here, it's processing input in real-time from a GlobalSat BU-353W10 GNSS receiver. The receiver, a USB dongle, is based on the widely used U-blox UBX-M8030 module.

Update (2023-03-06)

I participated in crowd funding the StarFive VisionFive 2 SBC, which has four 1.5GHz RISC-V processor cores and 8GB of memory.

Calcium

I got the board a few weeks ago and just recently got around to booting up the Debian-based Linux image provided by StarFive. Next step is to get my own software working on it.

Thursday, March 31, 2022

Supply and Demand

I've been using the Raspberry Pi single board computers (SBC) ever since the Model 1 (see below) came out in 2012. At about US$40 for a board, it is often easier just to buy a new board for a new project, rather than taking apart an existing project and sacrificing its SBC. Today, I'm buying Raspberry Pi 4B boards, still around US$40 each - more if you want more memory, and who doesn't?

Raspberry Pi

For a long time now I've been having an intermittent problem with one of my projects, Obelisk, my WWVB Radio Clock, a desk clock/NTP server I built using a Raspberry Pi 3B and tiny AM receiver board that picks up the NIST WWVB digital time signals from the transmitter near Fort Collins Colorado. Obelisk drops off my home WiFi network, after which it becomes impossible to diagnose since I can't log into it. I power cycle the unit, and it works fine for another month or so until it fails again. Since the clock and display keep running (meaning my software on the 3B is still running), this looked to be a problem with the WiFi chip on the 3B. I have about a dozen Raspberry Pis around the Palatial Overclock Estate that run 24x7, and this is the only one that seems to have this problem.

Obelisk (O-3)

I finally decided to replace the Raspberry Pi in Obelisk. I figured I would just order another 3B board, only to be taught an important lesson about supply and demand: the cheapest 3B I could find online was US$175, and that was used; a new one was over US$250!

That isn't as crazy as it sounds. Aside from the current semiconductor chip shortage, one of the design changes made in the Raspberry Pi 4B was to reverse the placement of the Ethernet jack with the dual USB A jacks at one end of the board. This not only made all of the older Raspberry Pi cases useless with the new 4B, but made use of the 4B problematic to retrofit into a lot of applications that were originally made with older versions of the SBC. (The 4B also added a second micro HDMI port, so those old cases would have been obsolete anyway... which may explain why the Pi architects swapped the Ethernet and USB A jacks.)

Fortunately I have four plastic storage bins of Raspberry Pi paraphernalia in the basement, ranging from my original Raspberry Pi model 1 to, fortunately, several Raspberry Pi 3Bs. They were all in cases, labeled, and ready to be reused as part of whatever project for which they were originally purchased. But in this case, I figured the only reasonable thing to do was to open up one of the cases and extract the 3B. (So long to Tin, a Raspberry Pi 3B that took part in my Roundhouse IPv6 project.)

It only took me a few minutes, and accidentally breaking a nylon spacer, to swap the 3B in Obelisk with this other one. Will that fix the problem? Only time will tell. But for sure it had never occurred to me that the Little SBC That Could would become a collector's item.

Tuesday, January 25, 2022

Fun Facts to Know and Tell: RPi on an SSD, Microsoft Linux

As an aside from my main projects - mostly my GPS and LTE-M efforts - I've been working on a couple of things that I am a little surprised to have worked as well as they have (even though they may not be perfect).

Raspberry Pi Running From An SSD

The latest Raspberry Pi board - the 4B - introduced a firmware-based boot loader than can be easily configured to boot from something other than the resident micro SD card. It was pretty easy to configure it to boot from a USB-attached SSD.

Here's an image of one of my Raspberry Pi 4B SBCs sitting on top of a Samsung T5 that it booted from and is running off of. (The T5s have since been replaced by the newer Samsung T7.)

Untitled

I also found some instructions to enable the SATA TRIM command on the SSD so that the OS could get sectors on the SSD erased as a background activity.

It's not perfect: the SSD pulls enough power that I had to add an externally powered USB hub to run other stuff like GNSS receivers and LTE-M radios. But I'm running a full blown Ubuntu OS instead of Raspberry Pi OS (a.k.a. Raspbian). I run Ubuntu elsewhere on a Raspberry Pi using a micro SD card, but this is a lot more spacious, not to mention a lot faster.

Microsoft's Linux Distribution

Remarkably, Microsoft has created its own Linux distro for the x86_64 target: the Fedora-ish CBL-Mariner (for Common Base Linux). It's intended as an internal Linux distro for Microsoft's Azure cloud infrastructure. I got it running as a virtual machine under VMware Fusion on my old desktop Mac. It works pretty well; I got my Diminuto C library and my Hazer GPS project running on it with very little effort.

Screen Shot 2022-01-25 at 10.05.16

It's not perfect: weirdly, it can't display the Unicode degree symbol, which is used in a couple of places in my Hazer code (gotta believe this is just a Unicode font issue). And the IPv6 unit tests in Diminuto don't pass 100% (current thinking this has nothing to do with CBL-Meridian, but with a change I made in my company web site). But otherwise I'm processing input as usual from a GNSS USB dongle.

Update 2022-01-25: The degree symbol display was a day-one bug in my Hazer code, related to setting the locale, that didn't manifest under Debian-based OSes. (If I had followed my own documentation, this would not have occurred.) The IPv6 issues were breakage caused by my own changes in the Digital Aggregates web site to support HTTPS and were unrelated to either CBL-Meridian or support in Diminuto for IPv6.

Tuesday, March 23, 2021

Where the RF Meets the Road

My Hazer project (com-diag-hazer) - the basis for all my GPS/GNSS work that I've written about - depends mightily on my Diminuto project (com-diag-diminuto) - my C systems programming library and framework - for its infrastructure. Testing some of the Hazer features finally got complicated enough that I built a dedicated test fixture for that project, just as I had to build a little test fixture for  Diminuto that I described in Where the Silicon Means the Road.

Diminuto GPIO/Hazer 1PPS Test Fixture

While Hazer's gpstool utility supports a wide variety of GPS/GNSS receivers, the one I chose for the test fixture is a SparkFun board equipped with a u-blox NEO M9N device. Here is some gpstool output generated while exercising the M9N.

Hazer gpstool using a u-blox M8N

I chose the M9N because it can receive and process transmissions from the satellite constellations of the U.S. GPS (a.k.a. NAVSTAR) system, the Russian GLONASS system, the European Galileo system, and the Chinese BeiDou 2 (a.k.a. COMPASS) system simultaneously. This tests more code paths in the underlying Hazer library and in gpstool.

Diminuto GPIO/Hazer 1PPS Test Fixture

The M9N is powered by a USB C connection to a dedicated Raspberry Pi 4B running Raspberry Pi OS, the Debian-based version of GNU/Linux formerly known as Raspbian. The USB connection is also the serial communications channel between the Pi and the M9N. The Pi runs headless; when necessary, I ssh into it from my desktop Mac and use the command line interface (although in a pinch, I have connected up a display, keyboard, and mouse to the Pi, or even attached a USB-to-serial adapter to its console port pins).

Diminuto GPIO/Hazer 1PPS Test Fixture

You can see the SMA Radio Frequency (RF) connector emerging from the container, where it connects to the coaxial cable of the GNSS antenna used by the M9N. The small multi-band active patch antenna sits on the top shelf of my workbench within view of the window.

u-blox Multi-band Active GNSS Antenna

You can also see a green wire and a blue wire that run from the M9N container to the Raspberry Pi.

The green wire connects the 1PPS ("one pulse per second") signal from the M9N to a General Purpose Input/Output (GPIO) pin on the Raspberry Pi. 1PPS is a standard precision timing signal derived from the GNSS solution by the receiver (although not all of them export it, and when they do, the mechanism varies). I make use of 1PPS in all of my GNSS-disciplined Network Time Protocol (NTP) micro servers that I've written about previously, including the one that incorporates a chip-scale cesium atomic clock. When gpstool is configured to monitor the 1PPS signal, it uses Diminuto's GPIO feature to interrogate the input GPIO pin using the select(2) system call inside a dedicated POSIX thread (which is also a Diminuto feature).

The blue wire connects an second, output, GPIO pin on the Raspberry Pi to an LED in the little container where the M9N is mounted. When gpstool monitors 1PPS, it can be configured for the thread to strobe the second GPIO pin to follow the 1PPS signal. This is not just a test of Hazer and gpstool, but a test of Diminuto's GPIO feature as well.

Diminuto GPIO/Hazer 1PPS Test Fixture

My hardware engineer colleagues will confirm that I'm not a hardware guy by any stretch of the imagination. But hacking together these little hardware test fixtures gives me peace of mind. As long as I see that LED blinking at 1Hz, I know that, while my code is not perfect, it is at least sane.

Friday, February 19, 2021

A Moving Map Display Using OpenStreetMap

Google Maps is a treasure. Hardly a day goes by that I don't use it. If I see a news item that interests me that includes an address, or even just a neighborhood, chances are good I'll be looking it up. By the time Mrs. Overclock and I got to Helsinki Finland to attend the World Science Fiction Convention in 2017, I had already "walked" all around the neighborhood of our hotel using Street View.

But although using Google Maps this way for me is free, it is a costly commercial product to develop and support. The maps have to be continuously updated with corrections and changes. The Street View images are routinely laboriously updated and ground truth established by humans driving extensively instrumented vehicles. The Satellite View images are updated frequently enough that I am often surprised to find that the season or the landscaping has changed since the last time I looked at my own neighborhood. It can't be cheap. Google supports this "free" service through a combination of fees for businesses that choose to advertise on this platform, and by licensing its use for products that use this platform for their own mapping needs.

By now, anyone reading my blog has come to realize that I have had a long running, broad, and deep, personal effort at writing applications using the U.S. Global Positioning System (GPS), and more broadly, other Global Navigation Satellite Systems (GNSS). More recently I branched out to include Inertial Measurement Units (IMU). This is part of my Hazer project, the source code repository for which can be found on GitHub. I am understandably interested in mapping the results of my tests, either in real-time or as part of my post-test analysis. Google Maps would have been ideal for this, but the price tag was a little high for something that amounts to an intensive hobby.

In my article Better Never Than Late (2017), I tracked a mobile GPS unit in my car by transmitting its geolocation data back to my home office via an LTE modem, to be mapped in real-time using Google Earth, an application program that provides a programmatic way to use satellite imagery. This worked great; but sometime between 2017 and 2018 the capability to do this in real-time with Google Earth stopped working. I was still able to do the post-test mapping by importing the data into Google Earth in Keyhole Markup Language (KML) form, as illustrated in my article Dead Reckoning (2020). But I wanted a solution for a real-time moving map display.

Recently I became aware of OpenStreetMap (OSM), an open-data, community-driven, project to provide an alternative to Google Maps for exactly the kind of real-time mapping in which I was interested. Thus was born another project, Tesoro, also hosted on GitHub, with the goal of incorporating OSM with Hazer to create a real-time moving map display.

Demonstration

Before I get to the nuts and bolts of Tesoro and OSM, let's go right to the good stuff: a video of the moving map display that I implemented. This is a forty-five minute (45:47) video, so no one (not even me) is going to want to watch all of it. But it's fun to skip around and watch different parts. For best results, watch this video in a separate YouTube window, and if your user interface supports it (not all do), choose the High Definition option so you can read the labels on the map. Opening it up into Full Screen mode helps too. Or be prepared to do a lot of squinting. (If the video below doesn't show up for you, try this link to the video on YouTube.)


This test used a playback of geolocation data I collected during IMU testing conducted on 2020-09-17. The playback mechanism uses the same UDP-based networking to feed the Hazer data stream to Tesoro that I have used previously in the field, so I would expect it to work almost as well with a real-time feed, subject to the occasional data loss I witnessed using LTE modems. (In Better Never Than Late I make the case as to why UDP, and not TCP, is the correct internet protocol to use for this. Alas, an efficient and easy to use method of encrypting and authenticating UDP traffic - while maintaining the characteristics of UDP that I find important - is in my opinion an unsolved problem.)

Overview

Tesoro broadly consists of four components: the Rover, the Model, the View, and the Controller. Web developers will recognize the Model-View-Controller (MVC) nomenclature used to describe a design pattern for interactive applications. GNSS developers will recognize Rover as the mobile portion of a Differential GNSS (DGNSS) system. (As always, you can click on any image to see a larger version.)

Tesoro

Although most of my development of Tesoro was done by running all of these components on a single computer, that didn't reflect how I expect the system to be used in real life. So for the video above, I split the components onto four different computers.

The Rover

The Rover plays the role of the component in the field, running the Hazer gpstool application and forwarding the resulting geolocation data via UDP back to the home base. For this test, the Rover was one of my Raspberry Pi 3B+ development machines running a version of the Debian Linux distro called Raspberry Pi OS (formerly called Raspbian).

gpstool saves geolocation data in a Comma Separated Value (CSV) format so it can be easily imported into a spreadsheet. The Hazer csvmeter script reads the stored CSV data and meters it out at approximately the same rate - typically 1Hz - as it was originally generated by the GNSS receiver under test. The output of csvmeter is piped to the Hazer csv2dgm utility that trivially converts it into JavaScript Object Notation (JSON) - the standard data serialization format used by JavaScript and lots of other applications because it is far simpler than eXtensible Markup Language (XML) - and sends the resulting datagram on its way.

The Rover can be tested by using the Linux/GNU utility socat to receive the datagrams and display them on the console.

The Model

The Model (short for Data Model) provides the database of map data used by the moving map display. The Model is an OpenStreetMap tile server, a server-side web application and database that provides tiles - graphical map segments - via Hyper Text Transfer Protocol (HTTP). It is hosted on a Raspberry Pi 4B computer with a two terabyte (2TB) Solid State Disk (SSD) in addition to its own root file system on a microSDcard.

Untitled

This tiny Single Board Computer (SBC) runs, not the typical Raspbian, but a full blown version of Ubuntu, another Debian-based Linux distro. I chose Ubuntu over the usual Raspbian that I run on the other ten Raspberry Pis that run 24x7 at the Palatial Overclock Estate because the OSM stack was known to work under Ubuntu. Could it run under Raspbian? Got me. When I found out that there were instructions for installing OSM under Ubuntu 20.04, and there was a version of Ubuntu 20.04 that could be installed on the Raspberry Pi 4B, I looked no further. Installing Ubuntu, then the entire OSM stack, then the map data for North America, then building the map database, on the Raspberry Pi was a relatively simple but time-consuming process.

I was careful to keep the map dataset (which is very large) and the Postgres database (which is very active) on the SSD. Given that just today I saw an ad for a one terabyte (1TB) microSDcard, it might be possible to implement a tile server without the SSD.

The Model can be tested by issuing an appropriate HTTP request from a browser or the Linux/GNU command line utility wget and seeing what comes back.

The Controller

The Controller provides the data stream that steers the moving map. It is implemented by channel, a small server-side JavaScript program I wrote that stands up a microscopic HTTP server. The channel receives the JSON datagrams from the Rover via UDP and caches the most recent one. When channel receives an HTTP request of any kind (it's not picky) over TCP, it responds by replying with the latest JSON datagram.

The channel is merely the conduit through which the JSON datagrams from the Rover can be received by the client-side web browser a.k.a. the View. Non-web developers may be surprised (I was) at how difficult this simple requirement is to solve. Browser security is an ever tightening chain of rules and sandboxing designed to keep malicious web sites from stealing your identity or infecting your laptop with malware. A lot of solutions you might take for granted as being feasible are actually deliberately impossible.

JavaScript programs running on the server-side under the standard Node.js compiler/interpreter have no such restrictions, however. I ran the channel JavaScript program on one of my x86_64 Ubuntu development systems, but any Linux/GNU system running Node.js would do.

The Controller can be tested by running channel and then making a web request using a browser or wget.

The View

The View is the visualization engine of Tesoro. It consists of a lot of client-side JavaScript code running inside of the web browser of your choice, being steered by the JSON it periodically fetches via HTTP from the Controller, and then rendering the appropriate map segments it requests from the Model. The View is implemented by a combination of Hyper Text Markup Language (HTML) files that download and run my movingmap JavaScript program. movingmap is built on top of Leaflet (1.7.1), a widely used JavaScript library that provides a client-side API to OSM. It is Leaflet that does all of the heavy lifting with OSM.

The View client-side HTML and JavaScript code (including Leaflet) are hosted on the Model, the OSM tile server, as just another Apache web site application. They are downloaded and executed inside the web browser on whatever computer the browser is running on. You can probably run the View on any modern web browser that supports JavaScript. The video above was recorded with QuickTime using the Google Chrome browser running on my desktop Apple Mac mini running macOS 10 with an Apple Cinema Display. But I have also run the View with Firefox on an x86_64 Ubuntu system, and with both Firefox and Safari on the same Mac.

Tesoro includes several HTML web pages that can be used to incrementally test the View and the rest of the software stack, first displaying a static map at a fixed location, then a static map whose latitude and longitude are specified as a query in the Uniform Resource Locator (URL), then finally as dynamic moving map display being steered by a user-specified channel.

Details

This is an image of the Raspberry Pi that is the OpenStreetMap tile server (center). It resides in an old stereo rack alongside my web server (left), and a Real-Time Kinematic (RTK) update server (right) that is part of my DGNSS project.

IMG_4345

This is the console output of the Rover's csv2dgm utility, which is written in C and is part of Hazer. When debugging output is enabled, csv2dgm displays both the incoming CSV record and the outgoing JSON datagram.

Screen Shot 2021-02-17 at 1.41.37 PM

Although for this example I was playing back a dataset of CSV records previously recorded as part of an actual field test, the same csv2dgm utility is used in the script that runs on the Rover in the field; this output is an easy way to verify that the Rover is generating datagrams before deploying it.

The Controller's channel JavaScript program that runs under Node.js is pretty chatty as well. Every time it receives a datagram over UDP, it displays its JSON payload as well as the sender from which it was received. Every time the View (or anyone else) issues an HTTP request over TCP, the program displays the requestor and the JSON it was sent.

Screen Shot 2021-02-17 at 1.42.52 PM

This output is especially useful since it can be saved as a permanent record on the local computer on which channel runs, independent on to whether the CSV dataset saved on the Rover in the field is eventually recovered.

All three of the web browsers I tested with - Chrome, Firefox, and Safari - have very useful debugging tools built in. In this screen shot from the video above, you can see the moving map on the left, and an optional console log on the right. Tesoro's movingmap client-side JavaScript program makes use of this log to reveal what it is doing. All three browsers have a console log like this, variously called "JavaScript Console", "Web Console", or a "Console" tab on a "Developer Tools" window.

Screen Shot 2021-02-19 at 12.24.55 PM

All three browsers also have tools to examine the headers used in the HTTP protocol. This is useful for debugging all three components, Model, View, and Controller, since all use HTTP.

Screen Shot 2021-02-17 at 10.00.57 AM

The README file for the Tesoro repository on GitHub has links to the pages I used to learn how to install Ubuntu on the Raspberry Pi, to install the OSM software stack, to download the map data, and to build the Postgres database, all on the Model tile server.

In addition, the repository includes a text file of mostly unstructured notes that might be useful if you get stuck trying to reproduce my results. Be aware, though, that not all of the notes led to something I actually used or did; some were dead ends.

Conclusions

I'm a little surprised how well this project worked, mostly because I wasn't a web developer when I started it. I'm not a web developer now, either. JavaScript will not be appearing on my LinkedIn profile.

But thanks to having hacked other developer's JavaScript on various paying gigs that had a web component, and to the wealth of documentation I found online, I was able - as usual - to learn from people smarter than myself.

(Note: edited several times for corrections and clarity.)

Monday, September 21, 2020

Dead Reckoning

 U-blox, the Swiss manufacturer of GPS and GNSS receivers that I've used in a number of projects, has models that incorporate both a multi-band receiver and an Inertial Measurement Unit (IMU). The IMU uses accelerometers and gyroscopes to estimate the movement and rotation of the receiver (and whatever it is embedded in) even in the absence of a usable GNSS signal. This is referred to as dead reckoning. The u-blox NEO-M8U module, for example, uses both the ensemble GNSS fix (if it exists) and the IMU measurements to produce a continuous fusion fix that can be maintained even as the GNSS fix is temporarily lost and then reacquired.

How large is the IMU? It's inside the thumbnail-sized M8U module. Here's is an image of the M8U mounted on a tiny red printed circuit board from SparkFun Electronics, which I in turn mounted in a plastic box on a test fixture, along with a multi-band GNSS active antenna.

Note: Click on any image to see a larger version.

Untitled

How is this possible? The accelerometers and gyroscopes are examples of a MicroElectroMechanical System (MEMS): microscopic electro-mechanical devices with moving parts, manufactured using semiconductor chip fabrication technology.

The IMU isn't perfect. Because the MEMS devices in the module are fairly noisy, the IMU has to do a fair amount of filtering and debouncing of their output. This leads to a noticeable latency of several seconds for the IMU to respond to changes in roll, pitch (road grade), or yaw (vehicle heading). This latency might explain some of the results I present below.

The IMU requires a short period of calibration at start up for it to determine the vehicle reference frame. The automatic calibration requires that the vehicle initially make turns and travel at sufficient speeds, all with adequate GNSS reception. I had to account for this calibration period during my test drives.

I also found that I got the best results when the M8U module was leveled and securely mounted with respect to the vehicle. Ultimately, this led me to mount the module on a gimbaled camera mount (like you might use for a GoPro), and to incorporate a spirit level into the test fixture.

But it's kind of remarkable that it works at all. The controller in the device itself does all of the heavy lifting. My software, running on a laptop or small computer like a Raspberry Pi, merely configures the module using proprietary UBX commands, and then reads the continuous stream of data from the device in the form of standard NMEA sentences and UBX messages. The device generates a position fix once a second, and reports useful information about the fix, like whether GNSS, or the IMU, or (more typically) both, were used in the solution.

Test Fixture

Here is the test fixture mounted on the dashboard of my Subaru WRX during a field test.

Untitled

The module connects via a USB cable to a laptop running some flavor of Linux/GNU and my open source C-based Hazer software package. The USB cable provides both power to, and two-way communication with, the M8U module. As is typical, the active antenna is powered by the M8U via a voltage bias applied to its coaxial cable.

Here is the computer I used in later test runs: a GPD MicroPC industrial laptop running MATE, an Ubuntu-based version of Linux.

Untitled

You won't want to do any touch typing on this tiny, almost pocket sized, computer, but it was perfect for this application. (Earlier I used an ancient HP 110 netbook running Mint, another Ubuntu-based Linux. It worked fine too.)

Test Strategy

I had to do some pondering as to how to test this device. The most obviously way was to obscure the antenna with something that would block the signals from the various GNSS satellite constellations. I owned several RF-blocking bags - originally purchased when I was testing cellular data modems - that I could have used for this purpose. I did do some early testing using this technique. But I wanted a more dynamic, organic approach that was in line with the M8U's intended use case in road vehicles.

That's when I remembered I lived in a western suburb of Denver Colorado, just east of the Rocky Mountains. Mountains with lots of highway tunnels. Like the short tunnels built in the 1930s along the United States route 6 road just a few minutes west of where I live. Or the ginormous 2.7km (1.7 mile) Eisenhower and Johnson tunnels built in the 1970s, that carry the Interstate route 70 highway through the Continental Divide about an hour west of my home. This project clearly called for a road trip.

My software can be configured to save the current position fix - latitude, longitude, and altitude, along with a bunch of other data - once a second. The data is saved in human-readable form to a file in Comma Separated Value (CSV) format. This makes it easy to post-process using simple scripts or even Excel. I wrote scripts that converted the CSV data into various files in Keyhole Markup Language (KML), an XML-based data format used to annotate place marks and visualize two- and three-dimensional paths in "earth browsers" like Google Earth.

So far I have done four test drives with my test fixture, through either the US-6 tunnels, or the I-70 tunnels, or (as luck would have it) a route that went through both.

Note: All of the screen shots from Google Earth below follow the usual convention of west to the left, east to the right, north up, and south down. In the U.S. we drive on the right-hand side, so in an east-west road, the upper lanes would be west-bound and the lower lanes would be east-bound. The discussion below assumes that Google Earth is pretty accurate in its mapping of latitude/longitude coordinates, and that they use the same datum, or definition of latitude/longitude, as GPS: WGS84.

Here's is a screen shot of one of my test drives as visualized in Google Earth.

Screen Shot 2020-09-15 at 12.13.25 PM

The red line (a LineString in KML) marks my path, and the yellow push pins (Placemarks in KML) mark the important points like tunnels along the way (different post-processing scripts define the push pins differently). The label on each push pin is effectively elapsed seconds. (It's actually the position fix number, but there is a fix recorded every second.)

The total travel duration for this road trip was about two and a half hours. This route took me west through on I-70 through a short tunnel just east of Idaho Springs, then through the Eisenhower Tunnel, turning around at Silverthorne, then back east on I-70 through it's peer, the Johnson Tunnel, exiting onto US-6 through a series of smaller tunnels, and finally back home. I began each test drive with enough meandering around my neighborhood that the IMU should have been able to calibrate. (Whether more meandering would have yielded better results is an open question.)

Road Trips

Here is a satellite view of the eastern end of the Eisenhower/Johnson tunnels on I-70. The yellow push pins here mark where there were changes in the position fix - a change in the number of satellites in view, or a total loss of GNSS navigation with a fallback onto the IMU. I'm entering the west-bound Eisenhower Tunnel at the top, and later exiting the east-bound Johnson Tunnel at the bottom.

Screen Shot 2020-09-21 at 8.41.08 AM

When I entered the Eisenhower Tunnel at the top going west,  I quickly lost GNSS reception until it was completely gone and I was relying solely on the IMU at the last of the west-bound push pins. Later, on the return leg, when I exited the Johnson Tunnel at the bottom going east, having turned around, it took a few moments for the module to reacquire GNSS signals at the first east-bound push pin, but then it quickly acquired more satellites by the last east-bound push pin.

A similar story is found at the western end of the Eisenhower/Johnson tunnels, exiting the west-bound Eisenhower Tunnel at the top, entering the east-bound Johnson Tunnel at the bottom.

Screen Shot 2020-09-21 at 8.40.40 AM

It took a few moments to reacquire the fix upon exiting the tunnel when heading west. Going back east on my return leg, I lost the GNSS fix quickly upon entering the tunnel. 

I traversed the Eisenhower and Johnson tunnels on three different test drives, each with slightly different results.

Screen Shot 2020-06-22 at 1.19.49 PM

In screen shot above, at the western end of the tunnels, the purely IMU position fix of my west-bound track drifted so far left (down), it crossed the east-bound track and made it look like I was in the wrong tunnel (which is impossible). When the module reacquired GNSS, it quickly corrected the fix to put me in the correct lane.

Screen Shot 2020-09-03 at 12.54.01 PM

In the screen shot above, its hard to say whether the IMU position fix drifted since I don't know exactly the path of the two tunnels through the mountain range, but it seems likely. (There is a pronounced bend in the tunnels at the eastern end, which you can see from the orientation of the eastern entrance relative to the western entrance.)

Screen Shot 2020-09-15 at 12.15.02 PM

Once again, there is an obvious drift in the screen shot above that makes it look like the tunnels merge at the western end (they do not). It's corrected as soon as I get near enough to the tunnel mouth that GNSS is reacquired.

There were even more radical corrections on the far shorter tunnels along US-6. (Updated 2020-10-03) Note, however, that both of these tests shown below were performed before I came up with the more stable test fixture on which I mounted the GNSS module; results after that change were noticeably better. This illustrates the importance of having a secure mounting. (End Update)

Screen Shot 2020-06-15 at 1.44.39 PM

On the western end of the tunnel above, you can see the sudden and abrupt correction made as I exited the tunnel going west-bound. It looks like I was smoking the tires on the WRX as I made a pair of bootlegger turns.

Screen Shot 2020-09-15 at 12.19.16 PM

Above, on another tunnel along different section of US-6 going east bound, you can see a similar sudden correction.

But some of the test runs in some of the tunnels looked nearly perfect.

Screen Shot 2020-09-03 at 12.52.54 PM

This tunnel is far shorter than, and not as deeply buried as, the Eisenhower/Johnson tunnels. Going back and examining the original CSV file, I see in this particular case that a GNSS fix was maintained for about half way through the west-bound tunnel, and for about a third of the way through the east-bound tunnel. In both directions, the position fix was then made solely using the IMU until I exited the tunnel.

Parking Garage

Another idea I had to test the IMU was by driving around inside a four-level parking garage at a shopping mall just a few minutes away from my home. This is what that path looked like after I converted the CSV into KML and imported it into Google Earth.

Screen Shot 2020-09-17 at 10.14.11 AM

Even driving around in the lowest two levels, then parking at the lowest, darkest, level of the garage, where no direct sunlight was visible, I was able to get a GNSS fix with at least six satellites (the minimum necessary is four if you want to also solve for altitude). The M8U used both the GNSS and the IMU for a position fix, but in this case the IMU was continuously able to be checked and if necessary corrected using GNSS.

Screen Shot 2020-09-17 at 10.34.31 AM

Looking at a close up, tilted, view of the parking garage in Google Earth, the path deviates enough that it looks like my WRX flew out of the parking deck and (somehow) back in again in a couple of places. This could be a lack of precision in the position fix of the M8U (GNSS isn't perfect either), an error on the part of Google Earth, or maybe I should audition for the next Fast and Furious movie. If I were to fly out of the parking deck and back in again, the Subaru WRX would be the car I'd use to do it.

Since the iterative least-squares algorithms typically used by GNSS receivers to compute the overdetermined solution is more accurate the more satellites it uses, and in open sky the M8U is able to use many satellites from several constellations, it is possible that in some circumstances in the garage the IMU could actually be more accurate than the GNSS when only a very few satellites are visible and they are close enough in their orbits that Dilution of Precision is an issue.

Conclusions

The integrated Inertial Measurement Unit in the NEO-M8U added to the accuracy of the position fix when GNSS signals were completely unavailable, but only for a short amount of time. The longer the device went without GNSS, the more the IMU position fix drifted.

The IMU might be quite useful for its intended application with typical road vehicles. I wouldn't use it to navigate in any coal mines. But it works fine over short periods of no satellite reception. And it's remarkable that it works at all.

Monday, June 01, 2020

Location, Location, Location

That's the punch line for the old joke: "What's the three most important factors in determining the desirability of a piece of real estate?" It's also the answer to "What's the three most important factors that affect precision and accuracy when using Global Navigation Satellite Systems?" It turns out that antenna placement is critical, with antenna selection running a distant second.

The data I'm going to present are not the results of careful controlled experiments. They are derived from datasets I already had lying around that I collected while testing Tumbleweed, my Differential GNSS project. But the data exposes what really matters. Some of this data has already appeared in my prior article Negative Results Are Still Results. In Dilution of Precision I explain how a poor view of the sky both limits the number of satellites your receiver's algorithm can use in its iterative solution, and its flexibility in choosing amongst those satellites to find the ones with the widest orbital separation.  More is better.

For all of these examples, I'll be using two metrics: the diameter of the smallest circle that can drawn around all of the positioning solutions gathered over time (using my tools csvlimits and geodesic), and a visualization of the positioning solutions achieved by converting the CSV output of my software into a Keyhole Markup Language (KML) file and imported it into Google Earth (using my tool csv2kml).
(2020-06-03: This article has been edited to append an additional example at the end.)
Location: NGS KK1446

Untitled

This used a survey-grade multi-band GNSS antenna (the white saucer-shaped object on top of the tripod) purchased from Ardusimple with the u-blox ZED-F9P GNSS module with my software running on a Raspberry Pi 3B+. The site was an open field containing NGS survey marker KK1446. The view of the sky was excellent. This combination allowed the receiver to use as many as twenty-nine satellites from four different constellations for its solution.

Uncorrected: 0.6925 meters

Mobile Vagabundo

Corrected: 0.0423 meters

Benchmark Vagabundo

The combination of a survey-grade multi-band antenna with the nearly perfect view of the sky gives an excellent showing. The use of differential corrections reduces the diameter of the solution circle from about 0.7 meters to just about four and a quarter centimeters, a factor of more than sixteen improvement. The corrected view from Google Earth has red marks so small that you need to click on the image to see it in the larger version.

Location: Second Story Window

Untitled

This used a multi-band GNSS antenna (far left) purchased from Ardusimple with the u-blox ZED-F9P GNSS module with my software running on a Raspberry Pi 3B+. The antenna no longer appears in their catalog, and has no markings on it; but from its weight and dimensions, I believe it has a built-in ground plane (a good thing). The site is a south-facing second-story window in my home office. The view of the sky is fair: only the southern half of the sky is visible, but that part is mostly unobstructed. Even with this partial view, the receiver was able to see as many as twenty-six satellites from four different constellations.

Uncorrected: 5.1404 meters

mobile-gold

Corrected: 2.3099 meters

benchmark-gold

The use of differential corrections reduces the diameter of the solution circle from about five meters to about two and a quarter meters, a factor of about two. This illustrates how differential correction can only do just so much. The uncorrected diameter is very typical of consumer GNSS receivers with their integral patch antennas and a good view of the sky.

Location: Lab Bench

Zhejiang JC SY-301 Helical GNSS Antenna

This used a multi-band GNSS antenna purchased from Ardusimple with the u-blox ZED-F9P GNSS module with my software running on an Intel i7-class system. The antenna - the small black vertical cylinder attached directly to an Ardusimple SimpleRTK2B board in a 3D-printed case - is a helical antenna intended for applications like aerial drones, its principle advantages being its light weight and its relative insensitivity to orientation. In my application, the antenna picked up only two of the four GNSS constellations, GPS and GLONASS, during the test period, despite being advertised to pick up all four. The site is the lab bench that sits on the opposite side of my home office, a little over two meters from the same window as above. The view of the sky is poor. Typically only four or five satellites from the two constellations are visible, for example, three GPS, two GLONASS; had this not been a multi-band antenna, it would not have been able to make a position fix using the minimum of four satellites.

Uncorrected: 37.5433 meters

mobile-cadmium

Corrected: 68.2234 meters

benchmark-cadmium

You are not imagining this: the uncorrected test run was better than the corrected test run. (I was so skeptical myself that I triple checked that I hadn't gotten the datasets confused.) Neither were worth writing home about. Some of the data points from on the corrected test blew clean over my home to the lot to the east, and even to the lot east of that. This test setup is adequate for regression testing my software for basic functionality, but in no way yields actually useful positioning except at very coarse granularity.

Location: Second Story Window (added 2020-06-03)

Untitled

This is a later test in which I took the same helical multi-band GNSS antenna (on the left) and tested it in the same location as and alongside the prior setup (which used the antenna on the right) in the south-facing home office window. As before, it uses the Ardusimple SimpleRTK2B board with the u-blox ZED-F9P module, although in this case the Linux/GNU host running my software was an ancient H-P Mini 110 netbook with an Intel i686-class processor. (The u-blox module does all the heavy lifting; my software merely runs the real-time display and collects the data.)

Uncorrected: 3.4650 meters

Screen Shot 2020-06-03 at 9.19.49 AM

Merely moving the helical antenna a couple of meters or so from the lab bench to the window makes a huge difference. In fact, in this one instance the small light helical antenna slightly outperformed the larger antenna from the prior test. Moving the antenna to a better view of the sky also allowed it to use as many as twenty-five satellites from all four constellations, versus the barely minimally adequate five satellites from two constellations. The solution diameter of 3.465 meters - over eleven feet - still illustrates how coarse the resolution of uncorrected GNSS is. An aerial drone using this configuration and seeing this kind of resolution would find autonomous navigation problematic.

Location, Location, Location

Differential GNSS can make a big difference, and antenna selection is important. But unless you have proper antenna placement, neither a good antenna nor differential corrections are going to save you.