Wireless Remote Sensing with Arduino and Zigbee

In "Remote Sensing with Arduino", part of my Amigo project, I described how to access light and temperature sensors on an Arduino board over Ethernet via a web server. In this article I'll describe how I've taken the same basic sensor platform and am now accessing it wirelessly via XBee radios.

Xbee is a family of tiny radios made by Digi International and based on Zigbee, the IEEE 802.15.4 standard for personal area networks (PAN) that are characterized by low power (one milliwatt), short ranges (about one hundred feet), and a pretty sophisticated network stack. Zigbee is capable of implementing a wireless mesh network, but the radios and firmware I am using in this application do something much simpler, so simple in fact that it requires no configuration and works right out of the box: a 9600 baud wireless serial connection.

I did this using a SparkFun Xbee Wireless Kit. The kit includes two XBee 2.4 gigaHertz series 1 radio modules, an XBee Arduino shield (some assembly required, so I had to break out the soldering iron to attach the Arduino headers), and an XBee Explorer. The Explorer is a tiny board with a socket for the radio module and an FTDI chip that performs the serial to USB conversions. The Explorer implements the PC/Mac end of the wireless serial connection. The Explorer requires the FTDI Virtual COM Port (VCP) drivers be installed on your PC or Mac. (I've written of my affection for FTDI and the quality of their products before, in "Deconstructing the AR.drone: Part 5".)

Below you can see the new setup. The breadboard at the top has exactly the same sensor setup as before. The Ethernet shield has been replaced on the Arduino board with the red XBee shield. The blue XBee radio module is plugged into a socket on the top of the shield. Jumper wires for power, ground, and the two sensors are plugged into the headers on top of the shield just as they were on the Ethernet shield. The Explorer is to the left with its own blue XBee radio module and a USB cable connecting it to my Mac. Note that there is no USB connection from the Arduino to my Mac; that's just a power cable from a wall wart powering the Arduino board. You could also power the Arduino side with something as simple as a nine volt transistor radio battery.

Here's a side view where you can see the power cable coming into the Arduino, with the XBee shield plugged in on top of it similar to the Ethernet shield.

Here's a closeup of the XBee shield. Near the lower right on the shield you might be able to see a tiny silver surface-mounted slider switch, so small you need a something like a toothpick to throw it. The switch has two positions, labelled DLINE, in which you can load your Arduino board over its USB cable just as you normally would, and UART, in which the XBee hijacks the serial pins on the Atmel ATmega328 so that, for example, whatever your software displays over the microcontroller's serial port goes across the wireless XBee channel instead of the USB cable.

And here's a closeup of the XBee Explorer. It is powered completely off the USB cable connected to my Mac. Although the XBee modules can be configured to do different things, right out of the box they implement a wireless serial connection and automatically pair with one another. Digi refers to this as transparent mode. Under the hood however, the radios are implementing a peer-to-peer network based on a Carrier Sense Multiple Access/Collision Avoidance (CSMA/CA) model.

So I loaded the FTDI drivers onto my Mac and restarted it. I plugged the USB cable from the XBee Explorer into my Mac and verified that a new serial device, /dev/tty.usbserial-A800f3aA, showed up. I hooked up the Arduino to its USB cable and downloaded my program over its own serial device, /dev/tty.usbmodem26431. I brought up the serial monitor tool in the Arduino IDE and verified that I was printing sensor values. I plugged in the power cable to the Arduino and removed the USB cable. I flipped the tiny switch on the XBee shield from DLINE to UART. I brought up the Arduino serial monitor tool again, this time using the Serial Port option in the IDE's Tools pull down menu to specify the new serial device from the XBee Explorer. And this is what I saw.

No, it is not 30.55°F in my office. Unfortunately I haven't replaced the temperature sensor I fried in "Another Victim of Electrostatic Discharge", so we'll ignore the actual temperature readings for now. But the Arduino is reliably reporting sensor readings wirelessly via its serial port once per second. Here's the complete program, which is mostly the same as before with all the Ethernet and Webduino stuff stripped out of it.

byte lightLevel = 0;
float temperatureCentigrade = 0;
float temperatureFahrenheit = 0;
unsigned long then = 0;

static const int lightAnalog = 4, lightDigital = A4;
static const int temperatureAnalog = 5, temperatureDigital = A5;

static const int oversample = 10;
static const float reference = 4.66; // Measured; nominally 5v, or 3.3v external.
static const boolean internal = true;

void sense(unsigned long now) {
 int lightRaw = 0;
 for (int ii = 0; ii < oversample; ++ii) {
   lightRaw += analogRead(lightAnalog);
   delay(1);
 }
 lightRaw /= oversample;
 int temperatureRaw = 0;
 for (int ii = 0; ii < oversample; ++ii) {
   temperatureRaw += analogRead(temperatureAnalog);
   delay(1);
 }
 temperatureRaw /= oversample;
 float lightVolts = (lightRaw * reference) / 1023.0;
 lightLevel = 100 - ((100UL * lightRaw) / 1023);
 float temperatureVolts = (temperatureRaw * reference) / 1023.0;
 temperatureCentigrade = (temperatureVolts - 0.5) * 100.0;
 temperatureFahrenheit = (temperatureCentigrade * 1.8) + 32.0;
 Serial.print(lightRaw);
 Serial.print('=');
 Serial.print(lightVolts);
 Serial.print('v');
 Serial.print('=');
 Serial.print(lightLevel);
 Serial.print('%');
 Serial.print(' ');
 Serial.print(temperatureRaw);
 Serial.print('=');
 Serial.print(temperatureVolts);
 Serial.print('v');
 Serial.print('=');
 Serial.print(temperatureCentigrade);
 Serial.print('C');
 Serial.print('=');
 Serial.print(temperatureFahrenheit);
 Serial.println('F');
 then = now;
}

void sense() {
 sense(millis());
}

void setup() {
 Serial.begin(9600);
 pinMode(lightDigital, INPUT);
 digitalWrite(lightDigital, LOW);
 pinMode(temperatureDigital, INPUT);
 digitalWrite(temperatureDigital, LOW);
 if (!internal) {
   analogReference(EXTERNAL);
   delay(1);
 }
 sense();
}

void loop() {
 unsigned long now = millis();
 if ((now - then) > 1000) {
   sense(now);
 }
}

Some models of the Xbee radios do allow you to some configuration, in order to, for example, use them in other than transparent mode. Unfortunately, the tool to do this from Digi, X-CTU, is only available for Windows. But as James Sentman has pointed out, you can run X-CTU on the Mac, although to do so requires that you install MacPorts, a software package that makes it easier to port GNU-based open source applications to MacOS, and Wine, a software package that lets you run Windows applications on MacOS (and other OSes as well). Being a glutton for punishment, I of course did all of this, eventually running the X-CTU installer on the Mac under Wine, and then running X-CTU itself under Wine to talk to the XBee Explorer. (Update 2012-02-03: Wrong. I got X-CTU to run on the Mac but was mistaken thinking it was talking to the XBee. However, X-CTU on my Windows netbook talked to it just fine. More tinkering needed.) However, you don't need to do any of this to accomplish what I've described here.

I found some of these references handy (in no particular order):

"XBee Manual" from SparkFun;

"XBee Data Sheet" from SparkFun;

"Simple XBee Communication with Arduino" by Matt Fisher;

"XBee Introduction and Buying Guide" from SparkFun;

"Configuring XBees" from Indestructables;

"XBee radios - connecting, configuring, and upgrading" by Limor Fried.

But once again, you don't need any of these to actually get this wireless setup working. The hardest part was, for me, not being a hardware guy, soldering the headers onto the XBee shield, and even that worked the first time. I was, however, careful to wear an ESD strap while putting all of this together.

Another Tragic Victim of Electrostatic Discharge

If Chip spends more than a day trying to solve a software problem, it's a hardware problem. -- Mrs. Overclock's Law

This isn't always true, but it's been true often enough, going all the way back to 1976 when a recalcitrant disk controller on a PDP-11/05 caused my device driver to work in the morning but not in the afternoon after the lab had warmed up. I typically work with hardware engineers of such caliber that I am extremely reluctant to invoke this law. However, when it's my hands on the hardware, all bets are off.

After writing the article "Remote Sensing with Arduino" I anxiously awaited for Mrs. Overclock (a.k.a. Dr. Overclock, Medicine Woman) to return home from a hard day courageously saving lives so that I could demonstrate the latest part of my Amigo project on her own laptop. Here is what she saw.

Let me assure you that it is seldom if ever 38.77° Fahrenheit in my home office where the Arduino board was connected to the network. Convinced that I had somehow botched some recent software changes, or maybe knocked something loose on the breadboard, I spent several hours fiddling with the code, swapping jumper wires, peering at the Arduino source code that supports the analog pins, reading documentation about the analog-to-digital convertor in the Atmel ATmega328 microcontroller, and just generally wasting my time making all of this a lot harder than it needed to be. Everything pointed to the Analog Devices TMP36 temperature sensor as the problem.

Finally, first thing this morning, I invoked Mrs. Overclock's Law and found this excellent description of how to test the TMP36 by (as I was to find out only just now) none other than Limor Fried herself. Here's a close-up of the tiny TMP36 surrounded by probes to my multimeter and wires providing power right from the Arduino board.

Initially the device reported 140 millivolts; that's around -33°F. A few minutes later it was 350mV or 5°F. Right now it reports 624mV or 54°F. Since I have my sleeves rolled up, this seems unlikely. This is a semiconductor device that had been working just fine.

I have no one to blame but myself. While everything you see here is placed carefully on an anti-static mat, I am not always careful to ground myself using the anti-static wrist strap, to my woe. Especially in the dry winter months, it is not unusual for me to experience a spark here and there. I suspect I fried the part while putting my finger on it to warm it up during testing.

It's not really the semiconductor part that's the issue here, although if I had fried an OMAP microprocessor or some other expensive or hard to get component, it might have been. The TMP36 costs two bucks U.S. or less. It's the fact that my own carelessness cost me hours of my time. When you are self-employed and you bill by the hour, you are constantly aware of the value of your time in real dollars. Using a grounded wrist strap to protect against electrostatic discharge (ESD) would have saved me a lot of time and money.

I have a grounded ESD mat and wrist strap permanently wired up on my work table in my office.

I even carry a portable model in my tool bag.

Back in my Bell Labs days we were required to take an annual refresher course in ESD practices. Or, I can just learn the hard way. If you lay hands on bare hardware, get yourself an ESD mat and wrist strap. And have the discipline to use them.

Remote Sensing with Arduino

If you have met me in meatspace then you know that I am so old as to be on the verge of being a brain in a jar connected to wires and tubes. When I started doing real-time development, it was on PDP-11s with just four kilobytes of memory. This goes a long way towards explaining my affection for microcontrollers like the tiny eight-bit sixteen mega-Hertz Atmel ATmega328 used on the Arduino Uno board that I am using in my Amigo project. It has thirty-two kilobytes of non-volatile flash memory and a scant two kilobytes of volatile static random access memory (SRAM) all on-chip. (Update 2012-01-25: I got the SRAM size wrong in the original version of this article.) There is a skill, becoming rarer in these days of increasing software bloat, of writing code that will run on such a platform. Thanks to its resource constraints and powerful software toolkit, Arduino is an excellent platform with which to learn (or re-learn) this skill.

Why bother? Because it is microcontrollers just like this that are permeating our lives, whether we realize it or not: embedded in our appliances and toys and transportation and even computers, sometimes alongside much more powerful Intel or ARM processors that do the heavy lifting required by graphical user interfaces. Microcontrollers are the universal machines that are called upon to bridge the gap between the real world and the digital, between meatspace and cyberspace. It is a market that is growing even as the market for traditional high-end microprocessors is being impacted as the typical consumer finds less and less reason to own a desktop computer or even a laptop, finding their cyberneeds better met by a smartphone or tablet and more and more computation is moved into the cloud. Thanks to their relatively low price tags, microcontrollers are increasingly becoming replacements for more complex discrete circuitry for glueing other digital components together and for implementing complex behavior. Microcontrollers bring the added advantage of being able to have their firmware updated in the field; discrete electronics, not so much.

One application for tiny microcontrollers is remote sensing. Because of their small size and frugal power consumption, microcontrollers are useful for deploying digital control and sensing into the physical world, where they can gather intelligence and report back. Playing with remote sensing is easily done with Arduino with the addition of a Ethernet shield.

Shields in the world of Arduino are daughter boards that provide specialized hardware and peripherals. They are designed with interface pins protruding from their lower surfaces that mate with the connectors or headers on the Arduino board. Those digital connections that they need themselves they take, those that they don't or which can be shared with additional boards they connect to another set of headers on their top surfaces. The Ethernet shield adds an Ethernet RJ45 connector and supporting hardware to provide an Arduino board with a wired network connection. (There are shields which provide wireless connectivity as well; one step at a time.)

Here is a side view of my Arduino Uno board with the Ethernet shield plugged in on top of it. You can see the metal pins from the shield above plugged into the headers on the Uno below. A CAT5 Ethernet cable is connected to the shield on the left. Below that you can see USB and power cables connecting to the Uno below. Jumper wires are plugged into the headers on the shield.

The SparkFun Inventor's Kit for Arduino has all the hardware you need to do several projects that involve reading sensors. I chose to combine two of them, CIRC-09 ("Light") that uses a photoresister (a device whose resistance changes with the amount of light falling on it), and CIRC-10 ("Temperature") that uses a low voltage temperature sensor from Analog Devices. The TMP36 temperature sensor is a clever little device that is complex enough that its data sheet deserves a look.

Both of these devices can be hooked up to pins on the Arduino board and read using the analog-to-digital convertor built into the chip.

So what about the "remote" part? The Arduino software includes an Ethernet library that has a rudimentary internet protocol (IP) stack that, for example, responds to ping. The Ethernet library supports DHCP but I gave the board a static IP address on the local private network. The bulk of the IP stack is actually hard-wired firmware inside a Wiznet W5100 chip on the Ethernet shield, which is how an IP stack can be shoehorned onto a microcontroller with only two kilobytes of RAM. (Indeed, there are Arduino shields with processors more powerful than the Arduino microcontroller itself.)

On top of that I added the Webduino software that allows you to build a tiny web server that runs on the Arduino board. This web server, when interrogated by the browser on my Mac desktop, responds with a measure of the amount of light impinging on the photoresistor and the temperature in both Centigrade and Fahrenheit. Here's what it looks like, right from Safari.

I compute the light measurement to look like a percentage, but some calibration might be necessary for other applications. Still, it is easy for the software to tell the difference between night and day or whether room lights are on or off.

Here's a photograph of the entire fixture from above. The photoresistor is on the left and the temperature sensor is on the right on the breadboard, but they are so tiny that the jumper wires make them hard to see. Green, yellow and blue are sensor wires (the green one is merely reading the power bus), red are for power, and brown are for ground.

Here is the complete source code for my remote sensing project. It's not long. The underlying libraries do most of the work.

#include <SPI.h>
#include <Dhcp.h>
#include <Dns.h>
#include <Ethernet.h>
#include <EthernetClient.h>
#include <EthernetServer.h>
#include <EthernetUdp.h>
#include <WebServer.h>
#include <util.h>
#include <avr/pgmspace.h>

byte lightLevel = 0;
float temperatureCentigrade = 0;
float temperatureFarenheit = 0;
unsigned long then = 0;

void getIndexHtml(WebServer & server, WebServer::ConnectionType type, char * urlTail, bool tailComplete) {
 server.httpSuccess();
 if (type == WebServer::HEAD) { return; }
 server.print("<h1>");
 server.print(lightLevel); server.print('%');
 server.print(' ');
 server.print(temperatureCentigrade); server.print('C');
 server.print(' ');
 server.print(temperatureFarenheit); server.print('F');
 server.print("</h1>");
}

static const int referencePin = 3;
static const int lightPin = 4;
static const int temperaturePin = 5;

void sense(unsigned long now) {
 int referenceRaw = analogRead(referencePin);
 int lightRaw = analogRead(lightPin);
 lightLevel = 100 - ((100UL * lightRaw) / 1023);
 int temperatureRaw = analogRead(temperaturePin);
 float temperatureVolts = (temperatureRaw * 5.0) / 1023.0;
 temperatureCentigrade = (temperatureVolts - 0.5) * 100.0;
 temperatureFarenheit = (temperatureCentigrade * 1.8) + 32.0;
 Serial.print(referenceRaw);
 Serial.print(' ');
 Serial.print(lightRaw);
 Serial.print(' ');
 Serial.print(lightLevel);
 Serial.print(' ');
 Serial.print(temperatureRaw);
 Serial.print(' ');
 Serial.print(temperatureVolts);
 Serial.print(' ');
 Serial.print(temperatureCentigrade);
 Serial.print(' ');
 Serial.println(temperatureFarenheit);
 then = now;
 delay(1);
}

void sense() {
 sense(millis());
}

static const byte __attribute__((__progmem__)) myMacAddress[] = { 0x90, 0xa2, 0xda, 0x0d, 0x03, 0x4c };
static const byte __attribute__((__progmem__)) myIpAddress[] = { 192, 168, 1, 223 };
static const byte __attribute__((__progmem__)) myGatewayAddress[] = { 192, 168, 1, 1 };
static const byte __attribute__((__progmem__)) mySubnetMask[] = { 255, 255, 255, 0 };

WebServer webServer;

#define countof(_ARRAY_) (sizeof(_ARRAY_)/sizeof(_ARRAY_[0]))
#define flash2sram(_TO_, _FROM_, _COUNT_) byte _TO_[_COUNT_]; memcpy_P(_TO_, _FROM_, sizeof(_TO_))

void setup() {
 Serial.begin(9600);
 flash2sram(tempMacAddress, myMacAddress, countof(myMacAddress));
 flash2sram(tempIpAddress, myIpAddress, countof(myIpAddress));
 flash2sram(tempGatewayAddress, myGatewayAddress, countof(myGatewayAddress));
 flash2sram(tempSubnetMask, mySubnetMask, countof(mySubnetMask));
 Ethernet.begin(tempMacAddress, tempIpAddress, tempGatewayAddress, tempSubnetMask);
 webServer.setDefaultCommand(&getIndexHtml);
 webServer.addCommand("index.html", &getIndexHtml);
 webServer.begin();
 sense();
}

void loop() {
 unsigned long now = millis();
 if ((now - then) > 1000) {
  sense(now);
 }
 webServer.processConnection();
}

As is typical, I made this a little more difficult than it needed to be for pedagogical reasons.

To assist in debugging, I displayed a lot of the partial and final results over the serial port that is part of the ATmega328 microcontroller. This travels back over the USB cable to my desktop where it can be displayed using the serial monitor tool that is part of the Arduino development environment. Serial is a C++ reference to a pre-built object in the Arduino run-time system that provides an API to do this.

When all of your storage for volatile variables has to fit in two kilobytes of SRAM, it pays to put data that doesn't change somewhere else. You'll notice that, for example, the hard-coded IP addresses all have a mysterious __attribute__((__progmem__)) as part of their definitions. The __attribute__ keyword is a special directive to GNU C++ that can be used to communicate special implementation details to the compiler. The __progmem__ argument, specific to this family of microcontrollers, tells the compiler that the variable to which it is applied resides not in volatile SRAM but in non-volatile program memory, meaning the flash memory where executable code is normally stored persistently.

Access to program memory on this microcontroller has to be done through a different machine instruction, load program memory (lpm), than when accessing data memory (SRAM). If you are into this kind of thing, you will now realize that the ATmega328 is an example of a Harvard architecture, one in which data and instructions are stored and processed separately. The lpm instruction bridges these two worlds by allowing you to copy from program memory into temporary variables in data memory. (A von Neumann architecture is one in which data and instructions are stored in the same memory and may be accessed similarly to one another.)

The Arduino and underlying microcontroller software define C/C++ macros (for example, PROGMEM), data types (prog_char), and functions (memcpy_P), that simplify using this mechanism. But I coded the __attribute__((__progmem__)) explicitly because the use of the GCC __attribute__ directive is so common when developing close to bare metal, and for similar reasons, that I knew that many embedded developers reading this would immediately see something they recognize.

In the loop function, the main work loop, I only interrogate the sensors once a second. The call to processConnection looks to see if a web request has arrived, and if so processes it. The only request I've defined is an HTML GET for the web page index.html. Querying that web page on Arduino returns the latest sensed values.

It's pretty impressive what can be done within the resource footprint of a tiny microcontroller. This program takes a little more than half of the thirty-two kilobytes of flash, and a little less than a quarter of the two kilobytes of SRAM not including the stack. That's without much optimization on my part.

Small is beautiful.

How to Contribute to the Robert Dixon Scholarship Fund

I have already been contacted by several old friends and colleagues asking how to contribute to this cause. Write a check, specify that it is to go to the "Robert Dixon Scholarship Fund", and mail it to

Wright State University Foundation

3640 Colonel Glenn Hwy.

Dayton, OH 45435

The account is already established and I assure that that the Foundation will be delighted to expeditiously cash your check and send you an official acknowledgement for your generous tax deductible contribution.

You will also have my own undying gratitude and a promise that I will buy you an appropriate beverage of your choice the next time I see you.

Thanks!

The Robert Dixon Scholarship Fund

The Wright State University Foundation has a scholarship fund endowed in honor of Robert Dixon. Bob was the first faculty member hired by Wright State. His first office was in the farm house on the property while the campus was being built. He founded the Department of Mathematics and the Department of Computer Science. He served as Chair of both of those departments (some more than once) as well as Vice President of the Faculty and occasionally the Director of the Computer Center. Bob was also my, and many others, mentor and thesis advisor. I first had him as a professor in 1975.

Bob is still around, still writing software and doing math, which is his version of enjoying retirement. I had lunch with him back in November when we were both visiting Dayton Ohio, home of Wright State University. I have Bob to thank for much of what I am and have accomplished over the decades. I am pleased to have been able to contribute to the Robert Dixon Scholarship Fund. It would be great for others to do so too. It would also be great to see others honor their mentors in a similar fashion. Perhaps it's time to reflect on how lucky we have been and to whom we owe in part for our good fortune.

Thanks, Bob.

Update (2012-01-20)

I have already been contacted by several old friends and colleagues asking how to contribute to this cause. Write a check, specify that it is to go to the "Robert Dixon Scholarship Fund", and mail it to

Wright State University Foundation

3640 Colonel Glenn Hwy.

Dayton, OH 45435

The account is already established and I assure that that the Foundation will be delighted to expeditiously cash your check and send you an official acknowledgement for your generous tax deductible contribution.

You will also have my own undying gratitude and a promise that I will buy you an appropriate beverage of your choice the next time I see you.

Thanks!

Update (2012-03-20)

The Wright State University Libraries and the WSU Retires Association together are sponsoring an oral history project. Here's a link to an interview by Lewis Shupe of Bob Dixon about the start Wright State when it was nothing but a corn field and a farm house!

Lewis Shupe Interviews Bob Dixon

Update (2022-12-26)

Bob Dixon passed away on November 17th at age 86 at his home in Hot Springs, North Carolina.

It's hard to describe the enormous impact Bob Dixon had on my career and professional identity. But when I say that from time to time I ask myself "What would Bob do?", I'm not joking. That I've had a great career spanning almost fifty years is much due to Bob and his intellect, his insight, his wisdom, his humanity, his sense of humor, his friendship, and his sensitive bull shit detector.

Love you, Bob. Thanks for everything.

Robert Dan "Bob" Dixon's Obituary in the Yellow Springs News

Can what you don't know hurt you?

In my recent article Pulse-Width Modulation with Arduino for my Amigo project I showed a series of screen captures from a Saleae Logic to illustrate the kinds of square waves you could generate using Pulse-Width Modulation (PWM). The use of the Logic was critical to my understanding of what was going on under the hood in my software. The Logic is a logic analyzer: a tool that captures, interprets, and displays signals from digital circuits.

Logic analyzers have been a standard piece of kit for digital hardware designers for decades. They come in a broad range of prices, capabilities, and form factors, from units costing tens of thousands of dollars that support sub-microsecond measurements and require a dedicated equipment cart to wheel them around, to the latest generation of relatively inexpensive devices that connect to your laptop and fit in your shirt pocket. The Logic fits in the latter category. We'll talk more about it in a bit. I've become convinced that devices like the Logic need to become part of the everyday toolkit of software developers like me who routinely deal with real-time events and work close to bare metal.

If you've perused that article on PWM with Arduino, and if you have any experience with Arduino, you can be excused if you thought I had gone off the deep end with my lengthy and seemingly overly complex software to generate a PWM square wave. Under other circumstances, I would have thought so as well. I had a Logic sitting in one of my tool bags and I thought it would be a great idea (and less effort than the alternatives) to use it to create snapshots of the PWM square wave output from my program. My original program was only a dozen or so lines long and was dirt simple. What could possibly go wrong?

The first clue that something was seriously amiss was one of my first Logic traces taken when I was hard-coding the duty cycle, before I'd added the code to read the potentiometer using the analog/digital convertor. (You can click on any of these images to get access to larger sizes.)

There are several things not to like about this trace.

First, I had intended to generate a square wave with a 50% duty cycle, but in this trace the off phase is significantly longer than the on phase, by nearly 13%. The off part of the square wave had significantly more software latency: it was taking me 13% longer to get around to turning the GPIO pin back on. I couldn't account for that in my own code.

Second, the frequency was a lot lower than I thought it should be, given the tiny amount of code I thought I was executing and the real-time delays I had built into my code. I was using the Arduino digitalWrite function and it was taking a long time to turn the GPIO bit on and off.

It was time to take a deep dive into the open-source of the Arduino run-time system. I was reminded of the various bootloaders I've had to hack in times past: no OS, no multi-threading, everything running in the equivalent of kernel mode, software accessing hardware as directly as possible and routinely using privileged machine instructions. This means that delays can't be accounted for by the kinds of scheduling latency you see when running software under non-real-time multi-tasking operating systems like Linux. Delays mean that my software was doing something, even if that something was busy-waiting for some peripheral hardware to respond.

The first step was to see who called the loop function that is the equivalent to the main function in every Arduino program. A common design pattern in this kind of low level OS-less software is to have a work loop that is continuously executed. It's a loop because modern microprocessors don't have a halt instruction; the only mode the central processing unit knows is to continuously execute machine code, even if it's just a tight loop. Operating systems like Linux also have such a work loop, you just don't see it. It consists of the Linux kernel looking for a runnable task to execute. There is always at least one such a task, if nothing else a do-nothing task typically referred to as the idle task.

Here's the code in Arduino that calls the setup and the loop functions that are part of every Arduino application program. It is the main program in Arduino that is automatically invoked by the C++ run-time system once it has completed initialization, and it is the complete contents of the file main.cpp.

#include <arduino.h>

int main(void)

{

init();

#if defined(USBCON)

USB.attach();

#endif

setup();

for (;;) {

loop();

if (serialEventRun) serialEventRun();

}

return 0;

}

Pretty simple. It calls its own init function, it calls the application's setup function, and then it enters an infinite loop that calls the application's loop function followed by some code that checks for events on the serial port.

The function serialEventRun is declared elsewhere as a weak external reference: it's a function that may not be defined, and if it's not, its address will be set to zero (NULL) instead of causing an error when the Arduino is compiled and linked. This allows Arduino to be compiled for microcontroller chips that do not have a serial port, or in hardware speak, a Universal Asynchronous Receiver/Transmitter (UART), in which case the serialEventRun code is conditionally not included as part of the Arduino run-time system.

The ATmega328P microcontroller I'm using has a serial port, so this code was being executed. This explains why the timing of the off period in the square wave wasn't quite right: my original naive code turned the GPIO bit on, delayed for a while, turned the GPIO bit off, delayed for a while, and then the main program was doing some additional work. That's when I added the more complex mechanism that computed a variable delay.

I tackled the second problem by looking the implementation of the digitalWrite function.

void digitalWrite(uint8_t pin, uint8_t val)

{

uint8_t timer = digitalPinToTimer(pin);

uint8_t bit = digitalPinToBitMask(pin);

uint8_t port = digitalPinToPort(pin);

volatile uint8_t *out;

if (port == NOT_A_PIN) return;

// If the pin that support PWM output, we need to turn it off

// before doing a digital write.

if (timer != NOT_ON_TIMER) turnOffPWM(timer);

out = portOutputRegister(port);

uint8_t oldSREG = SREG;

cli();

if (val == LOW) {

*out &= ~bit;

} else {

*out |= bit;

}

SREG = oldSREG;

}

This function in Arduino is doing a lot of computation to derive a register address and bit mask from an pin number, and it does it every time. A pin number in Arduino is an abstraction that simplifies the use of the GPIO pins by mapping a number printed on the actual Arduino circuit board to a GPIO register address and bit mask. This mapping may not seem like much overhead, but Jean-Claude Wippler of JeeLabs has characterized the digitalWrite function as taking as long as fifty-eight times (update from Jean-Claude: twenty-nine times) more processor cycles than a direct register write. So I added the code to my own setup function to compute the register address and bit mask once, and then added code to do direct register reads and writes in my loop function. After that it was just some minor tweaks here and there to get the timing right.

Then I added the code using the Arduino analogRead function to sense the value of the potentiometer at the beginning of every square wave period, which was every one hundred iterations of my loop function. My next Logic trace looked like this.

You can see the bounce in the GPIO pin output at the beginning of every square wave. So the next step was to look at the Arduino analogRead function.

int analogRead(uint8_t pin)

{

uint8_t low, high;

#if defined(__AVR_ATmega1280__) || defined(__AVR_ATmega2560__)

if (pin >= 54) pin -= 54; // allow for channel or pin numbers

#elif defined(__AVR_ATmega32U4__)

if (pin >= 18) pin -= 18; // allow for channel or pin numbers

#else

if (pin >= 14) pin -= 14; // allow for channel or pin numbers

#endif

#if defined(__AVR_ATmega32U4__)

pin = analogPinToChannel(pin);

ADCSRB = (ADCSRB & ~(1 << MUX5)) | (((pin >> 3) & 0x01) << MUX5);

#elif defined(ADCSRB) && defined(MUX5)

// the MUX5 bit of ADCSRB selects whether we're reading from channels

// 0 to 7 (MUX5 low) or 8 to 15 (MUX5 high).

ADCSRB = (ADCSRB & ~(1 << MUX5)) | (((pin >> 3) & 0x01) << MUX5);

#endif

// set the analog reference (high two bits of ADMUX) and select the

// channel (low 4 bits). this also sets ADLAR (left-adjust result)

// to 0 (the default).

#if defined(ADMUX)

ADMUX = (analog_reference << 6) | (pin & 0x07);

#endif

// without a delay, we seem to read from the wrong channel

//delay(1);

#if defined(ADCSRA) && defined(ADCL)

// start the conversion

sbi(ADCSRA, ADSC);

// ADSC is cleared when the conversion finishes

while (bit_is_set(ADCSRA, ADSC));

// we have to read ADCL first; doing so locks both ADCL

// and ADCH until ADCH is read. reading ADCL second would

// cause the results of each conversion to be discarded,

// as ADCL and ADCH would be locked when it completed.

low = ADCL;

high = ADCH;

#else

// we dont have an ADC, return 0

low = 0;

high = 0;

#endif

// combine the two bytes

return (high << 8) | low;

}

There is again a lot of mapping from the abstract pin number to the actual hardware resources. But there isn't anything that obviously links this code to the square wave output pin, which is different from the input pin for the potentiometer. Next step was the 567 page Atmel reference manual for this family of microcontrollers. Also, the Arduino Uno circuit board schematic was worth a look.

567 pages is not long compared to other processor reference manuals. When you are working close to bare metal you quickly learn that more detail is better. You'll groan when you find the manufacturer's reference manual for the microcontroller you are using is only 100 pages long. You know it's going to be relatively useless. (Simpler devices may have perfectly serviceable data sheets of only a dozen pages.) You'll be tempted to send candy and flowers to vendors whose reference manuals have a good table of contents and index. A searchable PDF is also a very good thing indeed.

Even if you're not a hardware person, if you work close to bare metal you should get used to reading schematics. Regardless of your age and your eyesight, unless you have a printer that prints pages the size of movie posters, buy a magnifying glass and get used to using it. You will be amazed at what you learn when you start following circuit traces from a GPIO pin that is misbehaving and discover to what it is really connected. You might even earn some respect from your hardware colleagues. Maybe.

The Atmel documentation warns against making changes in digital output pins close in time to doing analog/digital conversions. Rather than hosing up my timing that I worked so hard to refine, I decided to limit the number of times I read the potentiometer to four times a second. I figured this would minimize the number of times I hosed up the square wave while still giving me a reasonable response time between turning the dial on the potentiometer and the electric motor speeding up or slowing down. That seemed to work okay.

So what's the point of all of this?

I wouldn't even have known of any of these issues had I not hooked up the Saleae Logic and looked in detail at what was happening on microsecond scales. In fact, even with my original naive software design, the electric motor sped up and slowed down appropriately when I turned the potentiometer. For most applications of Arduino, that's something we call success.

But that's not always the case, with Arduino or other real-time applications of microcontrollers and microprocessors. For many products on which I've worked, what happens at the microsecond scale matters. If it doesn't matter to me, it may matter to another piece of hardware that has a much higher frequency attention span than I do. It may be that you, like me, routinely work on real-time systems which are expected to have 99.999% uptime. Or on which lives depend. Or perhaps you just don't want to be humiliated when something you wrote goes pear-shaped in the field in a failure that's visible to hundreds of people.

That's why I think tools like logic analyzers in general, and for lower speed applications, PC and Mac based tools like the Saleae Logic in particular, need to be part of every real-time software developer's toolkit. The Saleae comes in eight and sixteen channel models at prices of around US$150 to US$300. The eight channel model samples at 24MHz and the sixteen channel model can sample at 100MHz. I have the less expensive model and it is likely to be all I will ever need for this kind of work. The Logic can also decode common serial communication and bus standards at signal levels of 5V and below such as asynchronous serial, I²C, and SPI. I used it to verify the pin-outs on the diagnostic serial port of the AR.drone.

The device has a tiny logic pod smaller than my penknife, barely big enough for a company property sticker, to which you connect a wiring harness with eight signal inputs plus ground. You can connect the wires on the harness directly to pins on a circuit board or use the detachable logic clips.

The pod is a rugged piece made of machined aluminum, which, let's admit, is kind of sexy. It all goes into a little zippered case that fits handily in my tool bag.

The pod connects via USB 2.0 to your PC or Mac on which the Logic software runs. I run Logic on both a Windows 7 netbook I use for field troubleshooting and on my Mac Mini desktop in my home office. Here's a photograph of me using the Logic with the netbook on the AR.drone.

Other similar tools provide comparable capabilities at competitive prices. There is really no excuse not to use one of these tools. Sometimes you really need to know what's going on under the hood. What you don't know might just hurt you.

Pulse-Width Modulation with Arduino

One of the most used features on microcontrollers and microprocessors that you find in embedded systems are the General Purpose I/O (GPIO) pins. These are pins directly on the chip that can be set to low (ground) or high (typically 3.3 or 5 volts) values, or whose digital voltages can be read and turned in to bits, under the control of software running on the chip.

System-on-a-chip (SoC) solutions are so-called because they incorporate one or more processor cores with on-chip peripheral controllers that handle I/O and other functions commonly used in embedded systems. Although many of these controllers can handle relatively sophisticated functions like Ethernet physical layer protocols or specialized communication busses like I²C, SoCs inevitably have at least a handful of pins that can be manipulated by an on-chip GPIO controller.

This capability gives the embedded developer a way to implement non-standard hardware protocols (or, sometimes, implement a protocol that the existing SoC controller didn't quite get right), to talk to application specific devices like a Field Programmable Gate Array (FPGA), to sense a button or switch on the front panel, or even to do something as mundane as blink a Light Emitting Diode (LED). This last example is exactly what was going on in my Amigo project article Getting to Hello World with Arduino: some simple software used the digitalWrite function to blink an LED. If you spend five minutes poking around in the Arduino run-time software you will see that ultimately this is implemented by just setting and clearing a bit at a memory location that represents a memory-mapped register in a GPIO controller built into the little eight-bit ATmega328P chip. (This is also exactly what was going on in my article Memory Mapped Devices on the BeagleBoard with Android except with a Texas Instruments DM3730 SoC with a powerful thirty-two bit ARM Cortex-A8 processor running Linux and Android.)

So let's suppose that we hook up a Radio Shack digital multimeter and a Saleae Logic analyzer to an Arduino and look at a GPIO pin. What might we see? (Ignore for a moment the explosion of wiring on the breadboard. We'll get to that in a moment.)

The multimeter is reading negative millivolts, which I think is just noise below the logic level of the GPIO pin. (You can click on any of these photographs to get access to larger sizes.)

Channel zero on the logic analyzer is reading a logic zero. So far, not that interesting. So let's write a little Arduino code to set the GPIO pin to a logic one value, just as we did to blink the LED in the previous article.

The multimeter reads about 4.5 volts. (That should nominally be five volts; the reason it's not I believe is that I'm actually driving four GPIO pins all at one time; it simplified the wiring.)

Channel zero on the logic analyzer reads a logic one. That's what my Digital Signal Processor (DSP) buddies would call a DC (for direct current) signal, by which they mean it's a constant value. They would use the term unmodulated.

So lets say we write a little more Arduino code to modulate the output of the GPIO pin, that is, turn it on and off. We'll turn it on for the same amount of time we turn it off. That means the pin will be on half, or 50%, of the time, and off the other half of the time. When it's on, it will output its full voltage, and when it's off it will be zero volts or ground. We'll refer to this by saying that the output of the GPIO pin has a 50% duty cycle.

You can see that the output of the GPIO pin is a square wave with a period of a smidge over two milliseconds. In a bit you will see that I was shooting for a delay of two milliseconds in the Arduino software. The actual duty cycle, measured by the analyzer, and thanks to slop in timing, is a little over 50%. Any guess as to what the multimeter will read?

The multimeter reads this square wave as a voltage of a little over 50% of the maximum. That's because the multimeter is expecting a constant DC input and it is reporting the average voltage over some time window. It turns out a lot of electrical devices work this way. So if we can generalize this, we can fool other devices to think they're getting different voltages the same way we fooled the multimeter, and do it all under software control. Let's try an 80% duty cycle.

So far so good. What does the multimeter say?

Yep, that's about right. What we've just done is reinvent Pulse-Width Modulation (PWM). PWM is a method of modulation that encodes a digital signal by altering the duty cycle of a square wave, that is, by changing the width of the pulse. PWM can be used as a way of encoding bits onto a wire, although not in any modern communications system of which I am aware. But its ability to fool electrical devices into thinking they are seeing different voltages is used in lots of applications.

Digital power supplies, ranging from the desktop unit you might see on a lab bench to the unit inside the wall wart that is powering your laptop, use PWM to change 120V from your wall outlet to whatever output voltage is desired. Digitally controlled PWM circuitry replaced old heavy transformers which were nothing more than heavy iron cores with a lot of copper wire wrapped around them, enabling wall warts to become pocket sized and to be much more efficient. The variable speed cooling fans in your computer are probably controlled by firmware that is reading a temperature sensor and then using PWM to apparently alter the voltage going to the electric motors in the fans. The popular AR.drone, a WiFi-controlled quadcopter toy that I've obsessively written about, uses PWM to individually control the speed of each of the four electric motors that drive its fan blades.

Indeed, PWM is so widely used that many SoC chips include a dedicated PWM controller that modulates the output of a GPIO pin at a frequency synchronized to a hardware timer. The ATmega328P chip used in Arduino is such an SoC. PWM requires no more work on the software developer's part than using the Arduino analogWrite function to set a duty cycle encoded as a number from 0 to 255.

But what would the fun be in that?

PWM has long been on my list of things with which to experiment. Naturally, I insisted on doing it the hard way: by writing a PWM algorithm completely in software without taking advantage of the PWM controller built into the chip. I combined two of the circuits included in the SparkFun Inventor's Kit for Arduino, CIRC-03 ("Transistors and Motors") and CIRC-08 ("Potentiometers"), to create an electric motor whose speed could be digitally controlled by turning the dial on a potentiometer. The voltage drop across the potentiometer, a kind of variable resister that can be tuned by turning its dial, is read periodically by the software using an analog-to-digital (A/D) convertor built into the SoC that returns a value in the range 0 to 1023, corresponding to a range of zero to five volts from the reference source. That value is used to compute a duty cycle in the range 0 to 100 for the PWM square wave. The square wave is then fed into a transistor, which is used to switch power on and off to the electric motor. In the photographs above, you can see the electric motor sitting just above the SparkFun fixture, and the blue potentiometer in the lower center with an arrow on its dial.

Here's the complete Arduino program. (As usual, the Blogger editor did violence to the indentation.) I broke up the square wave generator (function square), the timing generator (function timing), and the potentiometer sensing (function control) to simplify trying different things, like hard-coding the duty cycle instead of reading it from the potentiometer. The specific mechanism of generating the square wave is a dirt simple one known as time-proportioning: there is a counter that cycles in the range 0 to 99; the on portion of the square wave begins when the counter is greater than or equal to 100 minus the duty cycle. I also included function toggle to toggle a GPIO pin as quickly as possible to see what my upper limit on frequency was (best case about 500 Hertz), and function pwm to use the built-in PWM controller (which is a lot simpler and is what you should do in an actual PWM application).

int sensorPin = 0;
int motorPin = 9;
int meterPin = 10;
int logicPin = 11;
int ledPin = 13;

volatile uint8_t * pointer = 0;
uint8_t mask = 0;

int tick = 0;              /* 0..99 */
int dutyCycle = 0;         /* 0..100 */
unsigned int period = 20;  /* microseconds duration */
long sample = 0;           /* 0.5 seconds */
long iteration = 0;        /* 0..(sample - 1) */
boolean before = false;    /* true or false */
unsigned long then = 0;    /* microseconds elapsed */

void setup() {
 pinMode(motorPin, OUTPUT);
 digitalWrite(motorPin, LOW);
 pinMode(meterPin, OUTPUT);
 digitalWrite(meterPin, LOW);
 pinMode(logicPin, OUTPUT);
 digitalWrite(logicPin, LOW);
 pointer = portOutputRegister(digitalPinToPort(motorPin));
 mask = digitalPinToBitMask(motorPin) | digitalPinToBitMask(meterPin) | digitalPinToBitMask(logicPin) | digitalPinToBitMask(ledPin);
 sample = 1000000L / period / 4;
 then = micros();
}

void toggle() {
 for (long ii = 0; ii < 25000000; ++ii) {
  *pointer |= mask;
  *pointer &= ~mask;
 }
}

void square() {
 boolean state = (tick >= (100 - dutyCycle));
 if (state != before) {
  uint8_t oldSREG = SREG;
  cli();
  if (state) {
   *pointer |= mask;
  } else {
   *pointer &= ~mask;
  }
  SREG = oldSREG;
  before = state;
 }
 tick = (tick + 1) % 100;
}

void timing() {
 unsigned long now = micros();
 unsigned int duration = now - then;
 if (duration < period) {
  delayMicroseconds(period - duration);
 }
 square();
 then = now;
}

void control() {
 timing();
 if (iteration == 0) {
  long value = analogRead(sensorPin) / (1024 / 16);
  dutyCycle = (value * 100) / 15;
 }
 iteration = (iteration + 1) % sample;
}

void pwm() {
 dutyCycle = analogRead(sensorPin) / 4; /* 0..255 */
 analogWrite(motorPin, dutyCycle);
 analogWrite(meterPin, dutyCycle);
 analogWrite(logicPin, dutyCycle);
 delay(250);
}

void loop() {
 control();
}

I used the hard-coded duty cycle capability to do the test cases presented above so I could get as close to the desired values as possible. But lest you think the potentiometer solution doesn't work, here's a test run in which I lovingly hand dialed the potentiometer to get a duty cycle of about 66% and a voltage of about three volts.

Just for comparison, here's a test run using my pwm function that uses the built-in PWM generator, at about a 50% duty cycle dialed-in manually using the potentiometer, instead of my purely software-only version. The SoC PWM square wave generator has a higher frequency and its timing is more precise, hence it is able to generate more accurate voltages, and with a lot less effort on my part.

I don't recommend the do-it-yourself approach for an actual application using PWM. I had to do a lot of work to get the timing to be even as precise as you see here, and to deal with the odd side effect here and there like the A/D conversion causing bounce in the GPIO output pin. But along the way I did learn an awful lot about the Atmel hardware and the Arduino software, as I used the Saleae Logic analyzer to characterize the square wave I was generating. That, I do recommend.