Getting from Power Up to Running Main in Arduino

While trying to bring up FreeRTOS on a Freetronics EtherMega board (all the "Free" prefaces here being coincidental) as part of my Amigo project, I spent more hours than I care to admit figuring out how Arduino gets from power up to executing my software, a long and involved process whose explanation I found to be widely distributed and not easily forthcoming. This article details that journey.

I'll warn you right now: this is long and rambling, covers a lot of territory, and is fairly technical. Everything I'll be describing in this article relates to the EtherMega board and its ATmega2560 microcontroller, but is equally applicable, differing in only very minor details, to other Arduino boards and their eight-bit Atmel microcontrollers. Much of it applies even to non-Arduino uses of those microcontrollers. In fact, much of what I describe about the tool chain applies to any platform using GNU compilers, including Mac, Linux, and Windows.

The EtherMega Board

The EtherMega is a Freetronics clone of the standard Arduino Mega board, an Arduino board that use the ATmega2560 microcontroller. The ATmega2560 has significantly more resources than the ATmega328P used on the smaller, cheaper, and more common Arduino Uno board. The 2560 has 256 kilobytes (KB) of persistent flash memory compared to the thirty-two KB in the 328, eight KB of non-persistent static random access memory (SRAM) compared to two KB, and four KB of persistent electrically erasable read-only memory (EEPROM) compared to one KB. These additional resources make multitasking much more palatable since there is much more SRAM to hold the context, including the stack, for each task; although FreeRTOS has been ported to the 328, the restrictions on stack space, a resource that is especially critical if you are coding in C or C++, make it a little problematic for my purposes.

Here is a photograph of the EtherMega with a universal serial bus (USB) cable on the left connecting it to my desktop Mac, and a ribbon cable connecting it to an Atmel AVRISP mkII device that is off screen. More on that in a bit. In case the scale isn't apparent, this board is about four inches or ten centimeters in length. There's a lot packed in there. The ATmega2560 chip is the big one at the upper right.

Besides all the usual general purpose I/O (GPIO) headers found on the Arduino Mega, the EtherMega also has Ethernet, a microSD card slot, and the hardware surround for each. This makes it a combination of the Arduino Mega and the Arduino Ethernet shield all on one board.

The Ethernet is provided by a WIZnet W5100 chip, an application-specific integrated circuit (ASIC) with an RJ45 connector. The W5100 provides not just the Ethernet physical interface (PHY), and Ethernet controller, but a pretty complete internet protocol (IP) stack, all on-chip. (You have to wonder if there is a microcontroller core with firmware in there somewhere, but WIZnet says it's all silicon.)

This bundling of network protocol stacks or other advanced functionality inside a chip is very common on small embedded systems. It is not uncommon for an peripheral chip to have a more powerful microcontroller core embedded within than the external microcontroller for which you find yourself writing software. In fact, external microcontrollers, and the software we write for them, are frequently little more than glue used to integrate many other powerful components like ASICs and field programmable gate arrays (FPGAs) together and provide some small level of digital control to their management or their interface to the outside world.

This makes systems like the EtherMega conceptually like a asymmetric multiprocessing (AMP) system in that there is a lot of intelligence all operating independently and concurrently. But in fact, all recent Arduino boards really are AMP systems: they (and the EtherMega) have a second Atmel microcontroller (on the EtherMega it's an ATmega8u2) programmed to be a USB-to-serial interface that connects to one of the serial ports (it has four) on the ATmega2560. Welcome to embedded development. (Stuff like this is why I and others claim that the skill sets for developing for embedded systems, for distributed systems, and for high performance systems are basically the same; it's just a matter of scale.)

The ATmega2560 Microcontroller

The ATmega2560, like the rest of the Atmel ATmega family, and indeed most small microcontrollers, is a Harvard architecture: the machine instructions that it executes are stored in a completely separate memory from that of the data on which those machine instructions operate. The machine instructions reside in persistent flash memory, which Atmel refers to as program memory, and the data reside in non-persistent SRAM or data memory. This is in contrast to the larger microprocessors found in servers, your desktop, or even your smartphone, which are Von Neumann architecture: instructions and data reside in a common non-persistent memory, typically RAM, having been copied from some persistent store during the process of booting.

(Exceptions: I've helped develop fairly large embedded products which executed their machine instructions directly from persistent read-only memory or ROM based on a silicon technology called NOR-flash. This is referred to as Execute In Place or XIP. Such systems are typically not thought of as Harvard architectures because the same machine instructions are used to access either ROM or RAM; the only difference is the range of addresses used. This is becoming rarer because because NOR-flash, whose physical interface resembles that of RAM, is significantly more expensive than NAND-flash technology, whose physical interface resembles, or even emulates, that of a disk. For that reason, most of what we call flash today is NAND-flash. However, given how it's used, I'm assuming what the ATmega chips call flash is NOR-flash.)

This physical separation of program and data memory makes for a slightly different style of programming than you may be accustomed to. Fortunately, many of the details of dealing with this are handled for us by the tool chain and its library, which in our case is the GNU C and C++ compiler, the GNU assembler, the GNU linker, several GNU utilities, and the non-GNU and decidedly non-POSIX compatible AVR-specific C library.

Just to make things even harder, while data memory is byte addressed, program memory is word addressed, where a word is two bytes. There is no way for a machine instruction to address an individual byte in program memory, because that lowest order bit of an address that might indicate an odd or even byte within a word isn't even encoded in the machine instruction or in a pointer into program memory. All program memory fetches are done by the word, and every machine instruction fetched is one or two words. In this article I'll use byte addresses and lengths to refer to program memory just to be consistent in nomenclature when I talk about data memory. If you are familiar with the AVR architecture, this might make the some of the lengths or addresses for program memory look wrong. (It is also possible I will have botched something.)

In addition to flash (program) and SRAM (data) memory, the ATmega2560 used in the Mega, the Mega ADK, and the EtherMega, as well as the ATmega328P used in the Uno, have a second memory, EEPROM. EEPROM can be used to persistently store application-specific read-only parameters. Unlike program memory, instructions cannot be executed from EEPROM nor can data be read directly from it; it has to be accessed more like an I/O device.

Finally, all of the AVR microcontrollers have a few additional bytes of persistent read-only memory: several bytes of signature that identifies the specific model of chip, lock bits that control who can write where and when into EEPROM and flash, and several bytes of option bits that control how the chip itself operates, including what it does when power is first applied, bytes that Atmel refers to as fuses. The ATmega2560 has three fuse bytes: an extended fuse byte, a high fuse byte, and a low fuse byte.

Like the flash memory and the EEPROM, these fuses and lock bits (and, I'm assuming, even the signature) can be modified using the right tool. But fortunately for us they are not easily changed by software. As we shall soon see, this is how the microcontroller defends itself from what my Bell Labs colleagues referred to as death by download: the reduction of a device into little more than a space heater by loading software into it in which a bug breaks not just the device's function but the software download mechanism itself, a catastrophic event that is known as bricking the device.

The internet is full of tales of people bricking their smartphone while trying to to jailbreak it: loading software on it unencumbered by vendor-imposed restrictions. Once bricked, a device frequently requires a specialized tool, like an in-system programmer (ISP) in order to recover it. The ISP is typically more expensive (sometimes much more so) than the bricked device.

It is for this reason that embedded developers worry a lot about death by download. It's bad enough when the device rendered useless is a smartphone. It can be much worse when it is a core component in a multi-million dollar telecommunications system. Or a mission critical public utility, medical, or defense system. Or a space craft. My friend and occasional colleague Steve (a.k.a. The Space Cowboy) worked on the NASA Deep Impact comet probe and tells me stories of unmanned space probes launched with just the flight control avionics and a downloader. The remaining mission software was written back on Earth while the probe was in route. Talk about a hard deadline. You can be sure that that was the most thoroughly reviewed and tested downloader ever written.

The fuse bits in the ATmega2560 that are relevant to our interests are BOOTRST, BOOTSZ1, and BOOTSZ0 in the high fuse byte, and BLB01, BLB02, BLB11, and BLB12 that are lock bits. (All of this stuff is defined in detail in the Atmel reference manuals for each microcontroller model.)

BOOTRST determines at what address the microcontroller initially starts executing code when power is first applied. The unprogrammed value of this bit is the value one (1). Unprogrammed means right off the assembly line, because when they are erased or before they have been programmed for the very first time, flash and EEPROM memory devices revert to a state of all one bits; this isn't really a design choice, it is a consequence of how they work at the silicon level.

A BOOTRST value of one causes the microcontroller to start executing code at address zero (0x0) in the program memory. Location zero is known as the reset vector: typically what you program at this address is a jump machine instruction to the start of your program elsewhere in program memory. It's called a vector because it points somewhere else. It's done this way because the reset vector is just the first of many successive words that are vectors for other special conditions detected by the hardware, like interrupts or requests for service from hardware peripherals or even from devices outside of the microcontroller but connected to it via an external interrupt signal.

The BOOTRST value on the EtherMega is set to zero (0). This causes the microcontroller to instead start executing code at a location specified by the BOOTSZ1 and BOOTSZ0 bits. Two bits give you four different choices. On the EtherMega, BOOTSZ1 and BOOTSZ0 are both set to zero, which logically partitions the 256 KB of program memory into two pieces: 248 KB for the application, and eight KB for a boot loader starting at location 0x3e000. (This partition is smaller on the ATmega328P because of its smaller program memory and is hence at a different location.)

The Boot Loader

It is in this special partition that the Arduino boot loader software resides. The boot loaders for the Uno and the Mega boards are different, by the way, and implement a different protocol: the Mega boards use stk500v2 (emulating Atmel's boot loader for their STK500 development board) and the Uno board uses the smaller optiboot. But they both operate the same way. When you power up your Arduino, the microcontroller starts executing not at the reset vector, but instead at the location of the boot loader. The boot loader listens for a while to the serial port to see if a software download is in progress. The timeout differs depending on the boot loader and the microcontroller for which it was compiled, but on the ATmega2560 it's about a second. (The source code as well as the Intel hex binary images for both boot loaders are part of the Arduino distribution. Check it out.)

If it sees a boot load protocol message it recognizes on the serial port, the boot loader uses the flash memory hardware that is part of the microcontroller to write (also known as burn or flash) the binary application image coming down the pipe into the application partition of program memory. If the timeout expires, suggesting there is no download in progress, or once the download completes, the boot loader just jumps to the reset vector, into which it assumes as been stored a jump instruction to the beginning of the application. (I haven't checked, but I'm guessing maybe that in a brand-new Arduino right off the shelf the jump instruction at the reset vector just leads the microcontroller back to the boot loader.)

This is where the boot lock bits come in. To avoid death by download, the BLB01, BLB02, BLB11, and BLB12 bits in the EEPROM are coded to prevent the boot loader from writing into the boot loader partition of program memory. The boot loader is electrically prevented from writing over itself. In order to overwrite the bootloader, you need a special tool like the in-system programmer, a device that can write to program memory or even the EEPROM inside the microcontroller without running software on the microcontroller itself.

The Atmel AVRISP mkII is one such device, although there are others capable of doing the same functions, and sometimes other useful things like debugging. Here is a photograph of an AVRISP connected to the EtherMega board.

The ribbon cable from the AVRISP connects to a six-pin header on the EtherMega. (And on the Mega and Uno boards too. There is even a second six-pin header for programming the ATmega8u2 microcontroller; don't mix them up.) In the photograph below you can see the six-pin header on the EtherMega just below and right of the microSD card slot. If you squint you can barely see a tiny yellow "1" printed on the circuit board at the upper left of the header; this tells you how to orient the ribbon cable, which has a (barely visible) matching triangular arrow on its plastic connector near where the ribbon cable attaches to it.

If you decide that you want your Arduino board to always execute your application and you don't intend to ever load other software into it (or if you do so, you'll need use an ISP instead of the bootloader software), you can actually use an ISP to change the fuse and lock bytes, and overwrite the program memory that would otherwise be occupied by the bootloader. Most of us will never have a reason to do this, but if you are using an Arduino in a fixed production application, this is a quite reasonable thing to do. Every time power is applied, the microcontroller will jump through the reset vector into your application.

The AVRDUDE Utility

I mentioned a protocol that the boot loader software running on the microcontroller understands. What's using that protocol to send the binary application image to the microcontroller? For most of us, that would be the open source AVR Downloader UploaDEr utility, or avrdude, although other similar proprietary Atmel tools serve the same function. avrdude runs on your desktop and speaks to the Arduino boot loader. It's what the Arduino integrated development environment (IDE) uses to download your compiled sketch onto your board.

It turns out that avrdude can also speak to any number of ISP devices, including the AVRISP mkII. And there is some reason for you to want it to do so. It turns out that part of the boot load protocol involves commands for reading, and even modifying, the signature, lock, and fuse bytes in the EEPROM. Unfortunately, the Arduino boot loaders don't implement the complete boot load command set that avrdude can generate. Specifically, when you try to read EEPROM bytes using avrdude, the Arduino boot loaders lie to you, returning a hard-coded signature no matter what the actual underlying device is, and returning zeros for all the fuse bytes. This confused an old man for some time and made me even more cranky. Hence the AVRISP mkII in the photographs.

Below are two screen shots of me using avrdude to query both the EtherMega and an Uno for their fuse bytes using the ISP. (As usual you can click on these images to eventually see a larger size.)

The Reset Operation

Pressing the tiny reset button on the EtherMega (visible in the photograph above) or Uno boards is equivalent to a power cycle: the hardware is reset and the entire process begins again with the microcontroller beginning execution at the boot loader. Remarkably, the same thing happens when you connect to your Arduino board via its serial port using avrdude, the Arduino IDE's Serial Monitor, or even a tool like PuTTY on Windows or the screen utility on the Mac; that's because the ATmega8u2 microcontroller is programmed and wired up so that when a serial connection presents itself over the USB connection, it toggles the reset pin on the main microcontroller. (There are ways to defeat this if this isn't the behavior you want.)

Jumping to to the start of the boot loader code in program memory, or even to the reset vector for that matter, is not the same thing as resetting the system, because it doesn't reset the hardware back to its initial state. That's why Atmel recommends that if you need to reset the system from inside your code, you use the watchdog feature on the microcontroller itself. Watchdogs are hardware functions built into many processors that reset the hardware in the event the software goes insane. Insanity is usually detected by the software failing to periodically perform some action on the watchdog hardware, like set a bit in a special register. Your software can deliberately stimulate the watchdog to reset the both the hardware and the software in your system.

(Update 2012-05-08: Important Safety Tip: after playing with the hardware watchdog timer (WDT) built into the ATmega2560 microcontroller, I don't think there's a safe way to make use of it when you are using the standard Arduino boot loader. I'd be happy to be proven wrong. When the microcontroller takes a watchdog reset (WDR), the WDT remains enabled as the microcontroller emerges from reset. No matter what timeout you may have set the WDT to originally -- the maximum is eight seconds -- it emerges from WDR with a the shortest possible timeout value, which is about sixteen milliseconds. The boot loader has a timeout of about a second, depending on the Arduino model, before it gives up talking to avrdude and transfers control to your application. This means there is no way to disable the WDT before another WDR occurs. This results in rolling reboots. I wasn't able to get out of this mode even by power cycling the EtherMega board. I had to use the AVRISP mkII to reflash the microcontroller from scratch. If I hadn't have had the in-system programmer handy, I would have bricked my EtherMega board. Not good. I found some discussions about this issue on the web. The fix is to have the Arduino boot loader disable the WDT in its own initialization.)

The Tool Chain and Library

So now we have software running on our microcontroller. Ah, not so fast; what code does the reset vector actually jump to? main()? Fat chance.

If you are developing in a higher level language like C or C++, there is a lot of stuff that has to be done under the hood before your main program ever begins. Stuff that maybe you've been taking for granted. But nothing happens by magic. Somewhere there has to be machine instructions that do it. These instructions are either generated by the compiler and reside inside the object files that result when you compile your source file, or they are already compiled and reside inside AVR-specific C or C++ library routines that are part of the run-time system and which are incorporated into your application when you link all of your object files into a single binary image to be downloaded to your Arduino. (A similar process occurs whether you're developing an application in C or C++ for an ARM processor in your smartphone, an Intel processor on your desktop, etc.)

For C this includes setting the values of variables that are not automatic (that is, global or static variables, not temporarily allocated on the stack while executing inside a function) and for which you coded an initializer. Here's an example from Arduino's EthernetClient.cpp.

uint16_t EthernetClient::_srcport = 1024;

For C++ this also includes running constructor methods (and when your main program exits, destructor methods) for objects that are neither automatic nor allocated on the heap (that is, via the C++ new operator, which, by the way, the GNU AVR C++ library doesn't implement). Here's an example from Arduino's HardwareSerial.cpp.

HardwareSerial Serial(&rx_buffer, &tx_buffer, &UBRRH, &UBRRL, &UCSRA, &UCSRB, &UDR, RXEN, TXEN, RXCIE, UDRIE, U2X);

To understand how all this initialization gets done (and how the corresponding finalization gets done at program termination) we have to first look at what the GNU linker actually does.

Going back into the dark ages well before Arduino, the segments or collections of bytes inside an object file generated when the compiler compiles your source code can be broadly classified into three different categories: text, which are executable machine instructions, data, which are non-automatic variables to which you have given an initial (typically non-zero) value, and bss, which are non-automatic variables which do not have initial values. (The term bss stands for Block Started by Symbol and is an ancient term going all the way back to the 1960s.)

You can see why it would be especially useful to do this on a Harvard architecture: bytes in the text segments must be loaded into program memory; bytes in the bss segments must be reserved in data memory and zeroed out during initialization; and bytes in the data segments must be stored in program memory and then copied into bytes reserved in data memory during initialization. But the process is similar even on Von Neumann architectures since the text segments are typically placed in memory pages in RAM that are marked as read-only using a memory controller.

Conceptually, the linker concatenates all of the text segments, all of the data segments, and all of the bss segments, from all the object files into one big binary image. It then looks through the text and data segments in that image for any unresolved external references, that is, pointers to external variables or calls to external functions. It tries to find those variables and functions inside your image, and if it finds them, it fixes up the addresses to point to the right places. If it doesn't find them, it starts looking through all of the libraries you've told it about. If if finds them there, it appends the appropriate library routines to your binary image, and again fixes up the addresses to point to the right places. Because the library routines may themselves have unresolved references, it may have to do this several times. If it successfully resolves all of the external references by iteratively repeating this process, then it's done. It it doesn't, you get a link error.

(Arduino programs are statically linked. Programs for other platforms may be dynamically linked against shared libraries or what Windows calls dynamic link libraries or DLLs. This means a library actually resides in memory and that one copy may be shared by all running programs, executing the very same copy of every library function at the very same memory location. But ultimately the process of linking is similar.)

But of course it isn't that simple. Each individual object file, whether it is one of yours or from a library, has its own requirements for initialization and finalization. All of those individual snippets of initialization code has to be done before your main program begins, and all of those individual snippets of finalization code has to be done after your main program returns. This is dealt with by associating those code snippets with named sections, which is just a way for the software developer to further classify byte snippets in the object file that is orthogonal to their segment classification.

By convention, the AVR tool chain supports sections with names .init0 through .init9, and .fini9 through .fini0. (The order there isn't accidental.) There are both assembler directives (if you are writing code in AVR assembler) and compiler directives (for C and C++) that tells the compiler which, if any, of these sections you want a particular code snippet to be identified with. For example, the code below, from the stk500v2 boot loader used on the EtherMega board, specifies that the function __jumpMain is to be in section .init9. This is just an instruction to the linker regarding special handling for this particular function.

void __jumpMain (void) __attribute__ ((naked)) __attribute__ ((section (".init9")));

Remember way back when we talked about the reset vector at location zero in program memory? (It seems so long ago.) When you compile, link, and download your program into your Arduino, that reset vector ends up being a jump instruction to the entry point of a routine from the AVR libc.a library and which can be seen in the assembler source file gcrt1.S. The entry point name for that assembler routine is __init, and the reset vector contains nothing more than the instruction jmp __init.

This GNU C Run-Time routine #1 executes all of the initialization code in section .init0, then all the code in .init1, and so forth all the way through .init9. By convention, .init2 contains code to set up the stack; .init4 has code to initialize the data and bss segments; .init6 has calls to the C++ static and global constructors; and the code in .init9 is as simple as the instruction call main.

Similarly, when your main program returns, all the code in section .fini9 is executed, then .fini8, all the way down to .fini0. By convention, .fini9 contains the code implementing the C _exit function; .fini6 has the calls to the C++ static and global destructors; on most platforms, code in .fini0 would signal the operating system that the application is complete. Since Arduino has no operating system, .fini0 goes into an infinite loop.

So, I guess that __init routine is in .init0, right? Yep. And it just calls all that other initialization code, right? Nope. It doesn't have to. In fact, for the most part, __init doesn't even know about any of that initialization code. It's actually simpler than that.

Every time you link your application the GNU linker uses a linker script. This is a script written in the linker's own language that tells the linker how to put together a executable binary image for your particular processor. You can provide your own linker script, but if you don't, a default one specific to your processor architecture is used. These scripts live in a directory inside the GNU tool chain called something like /lib/ldscripts, depending on where your tool chain is installed.

The ATmega2560 microcontroller on the EtherMega is an AVR6 architecture, and it uses the script name avr6.x. The ATmega328P on the Uno is an AVR5 architecture, and it uses the script avr5.x. Perhaps you perceive a pattern. In the article Hidden Variables: Arduino and AVR Microcontrollers, I described how to display the implicit preprocessor symbols inserted into your source program by the GNU compiler. One of these symbols tells you what your microcontroller architecture is. You can find these scripts on your own system.

Here's a snippet right from the avr6.x script from the GNU tool chain on my desktop.

*(.init0) /* Start here after reset. */

KEEP (*(.init0))

*(.init1)

KEEP (*(.init1))

*(.init2) /* Clear __zero_reg__, set up stack pointer. */

KEEP (*(.init2))

*(.init3)

KEEP (*(.init3))

*(.init4) /* Initialize data and BSS. */

KEEP (*(.init4))

*(.init5)

KEEP (*(.init5))

*(.init6) /* C++ constructors. */

KEEP (*(.init6))

*(.init7)

KEEP (*(.init7))

*(.init8)

KEEP (*(.init8))

*(.init9) /* Call main(). */

KEEP (*(.init9))

*(.text)

. = ALIGN(2);

*(.text.*)

. = ALIGN(2);

*(.fini9) /* _exit() starts here. */

KEEP (*(.fini9))

*(.fini8)

KEEP (*(.fini8))

*(.fini7)

KEEP (*(.fini7))

*(.fini6) /* C++ destructors. */

KEEP (*(.fini6))

*(.fini5)

KEEP (*(.fini5))

*(.fini4)

KEEP (*(.fini4))

*(.fini3)

KEEP (*(.fini3))

*(.fini2)

KEEP (*(.fini2))

*(.fini1)

KEEP (*(.fini1))

*(.fini0) /* Infinite loop after program termination. */

KEEP (*(.fini0))

This snippet instructs the linker to append all of the code in .init0 to the application image, then all the code in .init1, all the way through .init9, then all of the code in .text segments that are not otherwise classified, then all of the code in .fini9, and all the way through .fini0. It merely concatenates all of these sections in the order specified by the script. So __init doesn't have to call the code in .init1 or even know about it. It merely falls through to the code in .init1.

When .init9 does the call main the very next machine instruction, the one that will be executed when main returns, is jmp exit. This is a little assembler routine in the C library that can be found in exit.S which does nothing more than disable all interrupts and execute the instruction jmp _exit. That's an entry point to a routine that can be found in the GNU libgcc.a library and which can be seen in the assembler source file libgcc.S. That routine is in section .fini9, from whence execution falls through all the way through .fini0.

The Arduino Sketch

If you write code for Arduino then you already know that, unlike most C and C++ based development platforms, you don't actually write a main program for your sketch. Instead, you just write setup() and loop() functions. That's because main() is already written for you. It resides in the file main.cpp. Here is the source for that file from the Arduino 1.0 distribution.

#include <Arduino.h>

int main(void)

{

init();

#if defined(USBCON)

USB.attach();

#endif

setup();

for (;;) {

loop();

if (serialEventRun) serialEventRun();

}

return 0;

}

You can see that main() calls its own init() function, then your setup() function, then calls your loop() function in an endless loop while checking for data on the serial console port.

The End

My adventures with Arduino continues. What I really like about Arduino, the GNU tool chain, and the AVR C and GNU GCC libraries, is that you can actually figure this stuff out. It is not opaque. It's complicated, but not too complicated. It's mysterious, but only for a while. For sure it's the best platform I've seen so far on which to teach embedded and real-time development.

(Thanks to Udo Klein who reviewed the early version of this document and suggested some corrections, which I have made. You can find his original remarks in the comments.)