Deconstructing the AR.drone: Part 2

When we last left our hero (that would be me) I was TELNETing into the AR.drone at the address its DHCP server publishes as the gateway (a.k.a. router) address. When I do so, I am delighted to see the standard greeting from the BusyBox built-in shell, and a root prompt. If you've been paying attention, you've seen BusyBox mentioned many times in this blog. It's the Swiss Army Knife of embedded Linux software: a single binary that implements dozens of the standard Linux utilities (and some not so standard ones). I've used BusyBox to implement a full-blown Linux embedded system with just three binary executables: the Linux kernel, the BusyBox binary, and an application binary. BusyBox has been mentioned here in the context of Diminto, Arroyo, and Contraption. (As usual, apologies for the poor formatting of the code throughout this article.)

BusyBox v1.14.0 (2010-12-02 15:13:17 CET) built-in shell (ash)
Enter 'help' for a list of built-in commands.




A quick check of the network interfaces reveals an ath0 interface at the gateway address and the usual loopback interface lo.

# ifconfig
ath0 Link encap:Ethernet HWaddr 00:26:7E:50:5D:A6
inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:9382 errors:0 dropped:0 overruns:0 frame:0
TX packets:45641 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1279047 (1.2 MiB) TX bytes:63893346 (60.9 MiB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

When we used the Nmap utility to scan for open ports on the AR.drone, we found four open TCP ports. Scanning for open UDP ports is more problematic. But now we can just ask the drone itself for the ports on which it is listening. We see the TCP ports we already knew about, plus several UDP ports: 5554, 5555, 5556, and 67. UDP port 67 is typically a BOOTP network boot server for diskless workstations. BOOTP is a service provided by many DHCP servers, so I'm guessing this port is just an artifact of the drone's use of DHCP.


# netstat -a
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:5551 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:21 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:5559 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:23 0.0.0.0:* LISTEN
tcp 0 0 192.168.1.1:5559 192.168.1.2:52707 ESTABLISHED
tcp 0 0 192.168.1.1:5551 192.168.1.2:52699 ESTABLISHED
tcp 0 549 192.168.1.1:23 192.168.1.3:1736 ESTABLISHED
udp 0 0 0.0.0.0:5554 0.0.0.0:*
udp 0 0 0.0.0.0:5555 0.0.0.0:*
udp 0 0 0.0.0.0:5556 0.0.0.0:*
udp 0 0 0.0.0.0:67 0.0.0.0:*
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags Type State I-Node Path
unix 3 [ ] DGRAM 809 /dev/log
unix 2 [ ] DGRAM 811


Next, a quick check of what's running. The wide disparity of the process identifiers suggests that this platform uses a lot of scripting which tends to spawn a lot of short lived subprocesses (although other explanations are of course possible). We can see the TELNET daemon telnetd running, our own shell and command, and the DHCP daemon udhcpd. More intriguing, there are kernel and system logging daemons klogd and syslogd running. Everything in square brackets is a kernel process, typically something like a device driver or kernel module that runs at least partially in a process context.

# ps
PID USER VSZ STAT COMMAND
1 root 2732 S init
2 root 0 SW< [kthreadd]

3 root 0 SW< [ksoftirqd/0]

4 root 0 SW< [watchdog/0]

5 root 0 SW< [events/0]

6 root 0 SW< [khelper]

75 root 0 SW< [kblockd/0]

87 root 0 SW< [khubd]

93 root 0 SW< [kmmcd]

116 root 0 SW [pdflush]

117 root 0 SW [pdflush]

118 root 0 SW< [kswapd0]

119 root 0 SW< [aio/0]

120 root 0 SW< [nfsiod]

764 root 0 SW< [ubi_bgt0d]

767 root 0 SW< [ubi_bgt1d]

771 root 0 SW< [ubi_bgt2d]

790 root 0 SW< [p6-spi.0]

803 root 0 SW< [rpciod/0]

812 root 0 SW< [ubifs_bgt1_0]

824 root 0 SW< [ubifs_bgt2_0]

826 root 0 SW< [ubifs_bgt2_1]

831 root 1632 S /bin/factory_reset_cb

892 root 0 SW< [ksdiorqd]

893 root 0 SW< [ar6000_io]

949 root 2736 S telnetd -l /bin/sh

951 root 2732 S udhcpd /tmp/udhcpd.conf

957 root 2812 S inetd

958 root 2736 S /bin/sh /bin/check_update.sh

959 root 11824 S /bin/program.elf

961 root 2732 S init

962 root 2732 S /sbin/syslogd -n -m 0

963 root 2732 S /sbin/klogd -n

989 root 2736 S /bin/sh

1026 root 2816 R ps

I appear to have landed in the root directory when I logged in. Next step is to peruse the /proc file system. This is where the kernel and device drivers expose internal state in the form of pseudo-files in the file system. I've found /proc invaluable when I've written my own device drivers to expose state and debugging information.

# pwd
/
# cd proc

First we check what version of the Linux kernel and GNU libraries the system is using, and what tool chain it was built with. Note it was built with my favorite ARM tool chain from CodeSourcery. A version of this same tool chain was used for Arroyo and Contraption.

# cat version
Linux version 2.6.27.47-parrot (aferran@Arrakis)
(gcc version 4.3.3 (Sourcery G++ Lite 2009q1-203) )
#1 PREEMPT Thu Dec 2 15:21:36 CET 2010

Looking at the CPU into shows we're running on an ARM926 core. It's probably part of a System On a Chip (SoC) that incorporates many embedded peripheral I/O controllers. (The ARM926 core is similar to the ARM920 core used in the Atmel AT91 processor that was part of Diminuto and Arroyo.) This ARM core include support for the Thumb (16-bit) and Java byte code instruction sets in addition to the usual 32-bit ARM instruction set.

# cat cpuinfo
Processor : ARM926EJ-S rev 5 (v5l)
BogoMIPS : 233.47
Features : swp half thumb fastmult edsp java
CPU implementer : 0x41
CPU architecture: 5TEJ
CPU variant : 0x0
CPU part : 0x926
CPU revision : 5
Cache type : write-back
Cache clean : cp15 c7 ops
Cache lockdown : format C
Cache format : Harvard
I size : 32768
I assoc : 4
I line length : 32
I sets : 256
D size : 16384
D assoc : 4
D line length : 32
D sets : 128

Hardware : Mykonos Parrot platform
Revision : 0904
Serial : 0000000000000000

Looking at the command line passed to the kernel by the boot loader tells us something about how the system is configured (if at all) as it boots up. The sizes of the flash partitions are specified, and we now know that terminal device ttyPA0 is the console terminal and it runs at 115200 baud. The AR.drone has an external USB port accessible on its underside using a non-standard connector. It's under a little rubber cover that has the USB symbol on it. The 115200 baud rate is very common for serial to USB converter devices, so we'll keep that in mind as we poke around.

# cat cmdline
parrotparts=nand0:256K(Pbootloader),8M(Pmain_boot),
8M(Pfactory),16M(Psystem),98048K(Pupdate)
console=ttyPA0,115200 loglevel=4 ubi.mtd=Pfactory,2048
ubi.mtd=Psystem,2048 ubi.mtd=Pupdate,2048
root=ubi1:system rootfstype=ubifs parrot5.low_latency=1

Checking the I/O memory tells me what segments of physical memory have been reserved by the kernel and device drivers. Much of these physical memory segments are likely to be control and status registers for memory-mapped I/O devices. Or particular interest are devices that unless they are grossly misnamed appear to be NAND flash devices (p6-nand), an On The Go USB controller (dwc_otg), an SD card interface (p6-sdhci), some Universal Asynchronous Receiver Transmitters a.k.a. serial interfaces (uart), some Serial Peripheral Interfaces a.k.a. SPI (p6-spi), and some Inter-Integrated Circuit a.k.a. I²C interfaces (parrot5-i2cm). SPI and I²C are serial bus standards used to talk to other chips such as (in my experience anyway) networking chips (for SPI) or sensors (for I²C). I'm assuming the p6_camif devices are the cameras. Not sure what the ba315 is, even after a web search (but later we'll see that it is some kind of NAND flash controller).

# cat iomem
40000000-47ffffff : System RAM
40025000-40347fff : Kernel text
40348000-403e3f63 : Kernel data
c0300000-c03fffff : p6-nand.0
c0400000-c04fffff : dwc_otg.0
c0600000-c06fffff : dma-pl08x.0
c0700100-c07001ff : p6-sdhci.1
c0700100-c07001ff : p6-sdhci
d0040000-d004ffff : p6_camif.0
d0050000-d005ffff : p6_camif.1
d0070000-d007ffff : parrot5-uart.0
d0070000-d0070fff : parrot5-uart
d0080000-d008ffff : parrot5-uart.1
d0080000-d0080fff : parrot5-uart
d0090000-d009ffff : parrot5-uart.2
d0090000-d0090fff : parrot5-uart
d00b0000-d00bffff : p6-spi.0
d00b0000-d00bffff : p6-spi
d00f0000-d00fffff : p6-nand.0
d00f0000-d00fffff : ba315
d0150000-d015ffff : parrot5-i2cm.0
d0150000-d015ffff : parrot5-i2cm
d0160000-d016ffff : parrot5-i2cm.1
d0160000-d016ffff : parrot5-i2cm

Device drivers are always good to peruse. Apart from what the I/O memory has already suggested, we see a flash driver (mtd for Memory Technology Device), a RAM disk, and a Multi-Media Card driver (mmc). Significantly, the latter two are block devices, meaning they are likely used by the file system layer. (I have to admit now that although I'm pretending to be seeing this all this for the first time, I've already been doing some hacking on the AR.drone and can verify that it does indeed implement a persistent read-write file system.)

# cat devices
Character devices:
1 mem
2 pty
3 ttyp
4 /dev/vc/0
4 tty
5 /dev/tty
5 /dev/console
5 /dev/ptmx
7 vcs
10 misc
13 input
81 video4linux
89 i2c
90 mtd
128 ptm
136 pts
180 usb
189 usb_device
204 ttyPA
204 ttyJ
251 usbmon
252 ubi2
253 ubi1
254 ubi0



Block devices:
1 ramdisk
179 mmc

Linux device drivers are identified by a major device number. Each driver can use a minor device number as essentially a parameter. The miscellaneous device driver (misc) is a generic driver that manages small device drivers that don't need their own major number but are instead identified by a minor device number. (The device driver I talked about for Contraption was such a driver.) Of interest among the miscellaneous drivers is a General Purpose I/O driver (gpio), and a watchdog (watchdog). GPIO is a simple hardware interface that allows a driver to set individual hardware lines to high or low states. It is sometimes used to implement standard controllers like SPI and I²C in software, or to control devices which do not conform to some other standard interface. A watchdog is typically a device that autonomously reboots the system if it is not petted (stimulated) periodically. It is a way to recover from the software getting seriously sideways like a tight uninterruptable loop.

# cat misc
56 network_throughput
57 network_latency
58 cpu_dma_latency
64 pwm
66 gpio
67 dmamem
59 ubi_ctrl
60 log_radio
61 log_events
62 log_main
63 binder
130 watchdog

Modules are loadable kernel drivers or other software that can be dynamically loaded and linked into the kernel at run-time instead of being compiled into the kernel statically. I see one such module, which I suspect is an SD card driver.

# cat modules
p6_sdhci 3588 0 - Live 0xbf000000

As embedded systems go, this one is pretty well endowed with RAM. It looks to be a 128 megabyte system of which about 105 megabytes are free when it is mostly idle.

# cat meminfo
MemTotal: 126072 kB
MemFree: 105700 kB
Buffers: 0 kB
Cached: 3572 kB
SwapCached: 0 kB
Active: 5392 kB
Inactive: 1924 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 3756 kB
Mapped: 2788 kB
Slab: 3636 kB
SReclaimable: 1460 kB
SUnreclaim: 2176 kB
PageTables: 168 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 113464 kB
Committed_AS: 5968 kB
VmallocTotal: 843776 kB
VmallocUsed: 5296 kB
VmallocChunk: 837948 kB

Unsorted Block Image File System (UBIFS) is a flash-based file system which I have never used; I've used the older Journalling Flash File System 2 (JFFS2). We see several mounted UBIFS file systems, including root /, /factory, /update, and /data. I'm guessing at least two of these are redundant file systems used during firmware updates or reflashing so that if the update fails there is a backup system to which to fall back to.

# cat mounts
rootfs / rootfs rw 0 0
ubi1:system / ubifs rw 0 0
tmp /tmp tmpfs rw 0 0
proc /proc proc rw 0 0
dev /dev tmpfs rw 0 0
devpts /dev/pts devpts rw,mode=600 0 0
sys /sys sysfs rw 0 0
ubi0:factory /factory ubifs ro 0 0
ubi2:update /update ubifs rw,sync 0 0
ubi2:data /data ubifs rw 0 0

The Memory Technology Device or mtd shows us how the flash memory is partitioned. I'm guessing these MTD partitions map to the UBIFS file systems and at least two are redundant. The names also suggest to me that this processor may have a two-stage boot loader. It is not uncommon to have a simple boot loader that loads a more complex boot loader like U-Boot, that then loads the Linux kernel.

# cat mtd
dev: size erasesize name
mtd0: 00040000 00020000 "Pbootloader"
mtd1: 00800000 00020000 "Pmain_boot"
mtd2: 00800000 00020000 "Pfactory"
mtd3: 01000000 00020000 "Psystem"
mtd4: 05fc0000 00020000 "Pupdate"

Finally, we look at the counts of how many times different device drivers have received interrupts from the hardware. This can be a good clue about how active different devices in the system are. For example, the high counts on some of the serial devices (parriot5-uart) suggests that they are some kind of sensors or controllers and not a currently unused console or debug port. Ditto for our guess about the camera devices (p6_camif).

# cat interrupts
CPU0
4: 696468 VIC mmc0
5: 513 VIC parrot5-uart
6: 354060 VIC parrot5-uart
7: 177177 VIC parrot5-uart
10: 6087 VIC BA315 NAND controller
14: 0 VIC dma-pl08x
19: 90212 VIC Parrot6 Timer Tick
21: 493317 VIC p6_camif.0
22: 477970 VIC p6_camif.1
28: 8600 VIC parrot5-i2cm, parrot5-i2cm
32: 0 p6-gpio p6_kbd_input
33: 0 p6-gpio p6_kbd_input
Err: 0


Next up: figuring out the boot-time sequence and how the AR.drone configures its network.

Deconstructing the AR.drone: Part 1

When I first became aware of the existence of the AR.drone, made by the Paris-based manufacturer of wireless gadgets Parrot, I knew I was going to own one. To some, the AR.drone is a radio-controlled quad-rotor helicopter. To others, it's a fragile $300 toy. To me, it's nothing less than a sophisticated Linux-based WiFi-enabled flying embedded system just begging for me to peek under its hood. The device has been so popular, I am walking down a road well traveled; but it is scenery I have never seen before.

The AR.drone is piloted by an iPod app, making it usable with any WiFi-enabled iDevice (although Android-based apps are starting to appear). The drone has two video cameras, one front facing and the other down facing, plus an accelerometer, two gyrometers, and an ultrasonic altimeter. It is all controlled by an ARM-based processor running Linux. Both the iOS-based app and the Linux-based embedded system are hackable; Parrot provides an SDK and encourages developers to use it as an application platform. I control my AR.drone with my iPad.

Unlike other RC helicopters of my personal experience, the AR.drone is relatively easy to fly, by virtue of its sophisticated embedded flight control software and sensors. The system can take-off and hover with stability with just the press of a virtual button in the app. A few minutes of practice and I was able to slowly maneuver it around a room in the Palatial Overclock Estate. Roll and pitch are controlled by tilting the iPad; yaw is controlled by a virtual thumbwheel in the app. The controls on the app are superimposed on a video image from the front facing camera. (The very first time I flew the device I was so focused on the controls that I failed to notice the screen-filling face of one of our house cats on the display just as I hit the take-off button. Wackiness ensued.)

As much fun as the AR.drone is to play with, it didn't take long for my child-like curiosity to overcome my child-like glee. I fired up one of my most useful embedded system tools - an HP Mini 110 netbook running Windows 7 and equipped with a raft of useful software - and started poking at the AR.drone.

My drone appeared as an ad hoc WiFi network whose service set identifier (SSID) was ardrone_040582. An ad hoc WiFi network is one which does not require any infrastructure like an access point or router. All members of an ad hoc WiFi network implement a packet radio network, treating their shared radio channel in a collision-sense multiple-access / collision detect (CSMA/CD) manner. This is just like the original unswitched baseband Ethernet architecture circa 1975, which borrowed from the even earlier ALOHAnet, which itself was a wireless packet radio network that first became operational in 1971. Pairing the iPad with the drone network is a simple manner of choosing the right network in the iPad WiFi control.

Notice that the iPad gets its internet address, mask, and default gateway (router) via Dynamic Host Configuration Protocol (DHCP). That means the AR.drone must be running a DHCP server. The upside is this makes pairing the iPad to the drone easy. The downside is this makes integrating the drone into an existing WiFi infrastructure difficult, since running two DHCP servers (one of which you don't control) on the same subnet is problematic. And who is the router? That must be the AR.drone as well.

Next, I used the netbook to scan for WiFi networks, and connected it to the same ad hoc network. (To make my neighbor's lives a little easier, I've obfuscated the other SSIDs in my neighborhood.)

I left the netbook set up for DHCP, so it also acquired its internet parameters from the DHCP server in the AR.drone.

I fired up IPScan on the netbook to see if there were any other hosts responding on the 192.168.1.0 subnet running on the drone's ad hoc WiFi network. 192.168.1.1 is the AR.drone. 192.168.1.2 is the iPad. 192.168.1.3 named gypsum is the netbook.

Next, I pointed Nmap at the AR.drone to scan for TCP ports that the device exposes. The tool can see ports 23 (typically the network terminal protocol TELNET), 21 (typically File Transfer Protocol or FTP), and two other ports: 5559 and 5551.

A service scan of the known ports reveals that port 21 can be accessed for anonymous FTP, making the AR.drone a flying file server which provides three files right out of the box: licenses.txt, repair, and sys_bootldr.bin. Although it's not shown in this screen snapshot, the tool also reveals that the TELNET port delivers a root prompt without requiring a password.

At this point I would be remiss if I didn't TELNET into the AR.drone using my favorite Windows-based virtual terminal tool, PuTTY. Configuring PuTTY to TELNET into the AR.drone is easy given we know its IP address and port number.

Connecting to the TELNET port yields a root shell prompt. Below is a photograph of, from left to right, the netbook TELNETed into the AR.drone, the AR.drone, and the iPad running the AR.drone app. These three devices constitute a tiny wireless IP subnet, one component of which could be flying around the room.

Next up: exploring the host software side of the AR.drone as a flying Linux system.

Getting to Hello World on Android in About Five Minutes

Back when I started getting paid real money to make computers do things in the 1970s, I was called a systems programmer. That job description has fallen by the wayside. Now I'm just known to my co-workers as that fat old guy.

But my compulsion to peak under the hood of things has served me well career-wise over the past three plus decades, most recently earning me a very good living doing embedded Linux development. Maybe that's why I spent most of my early time with Android figuring out what violence had been done to the underlying platform (a lot) and seeing if my skills were actually transferrable (they were).

But I figured it was finally time to actually try the traditional Hello World program using the Eclipse-based Android SDK. I'd been using Eclipse for years, both in the embedded domain and in the enterprise Java domain (where I was still a systems programmer). But I didn't expect to have the sample Android Hello World up and running, not only in the emulator that is part of the SDK, but on the actual Beagle Board, about five minutes after reading the first few pages of Ed Burnette's book Hello, Android from The Pragmatic Programmers. The hardest part was finding the tiny USB OTG port on the Beagle Board to connect it to my desktop.

I know there's going to be a lot more to it than this to accomplish what I want to do. But still, I can't help but be impressed with the tool chain and its level of integration. A lot of things have changed since the IBM 029 keypunch.

Those who don't study Vista are doomed to repeat it.

A few years ago I got a brand new IBM ThinkPad that shipped with Windows Vista. I chose this platform because I had so much luck with an older ThinkPad running XP. The new laptop was in hindsight the best thing that ever happened to me. Because Vista was such a disaster - slow, unstable, unfamiliar, unusable - that I quickly replaced the ThinkPad with an Apple MacBook Air. Which I loved so much, I replaced my Windows XP-hosted desktop development environment with a Mac Mini with a 27" cinema display. And then got an iPad. And then an iPod Nano. Back when wooly mammoths strode my neighborhood I had converted from using a Mac Classic to a Windows box. But now I'm back.

It's possible that Vista might have been an improvement over XP. But who could tell? It was slow, buggy, and unstable. It would hang so severely that I had to not only power cycle the laptop but pull the battery to recover. The interface changes required a lot of effort to learn, which was hampered by the system hanging, crashing, and just plain making it difficult to get stuff done. I really wanted to hang in there with Windows, because my financial and cognitive investment was so high. But the shockingly low quality of the product just made it impossible to do so.

It was for me a Vistapocalypse. I'm a one-man company that relies on good tools to get my work done. I felt betrayed by Microsoft, who had up to that point been a reliable de facto business partner. I sometimes wonder how many tiny shops went out of business because they couldn't turn on a dime like I had to and adopt alternative technology due to reasons of technical expertise, cash-flow, or timing. I was lucky. But the trust was gone.

I eventually did a completely clean install of Windows 7, when that was released much later, on the laptop to get it back into usable form. I turn it on maybe once a month for some business application or another that only runs on Windows. Otherwise, I'm an Apple fanboy now.

As a professional product developer, I think about this a lot. If any company in the world could write functioning software, you would think it would be the largest, most successful, software company in the world. That they botched this, and so badly, gives me reason to pause. There's got to be a whole college course's worth of case studies in this, for both software engineering and business.

Those who don't study Windows Vista are doomed to repeat it.

(From a comment I originally published on io9, the Gawker Media science fiction blog.)

The System & The System: Overlaying GNU on Android on the Beagle Board

One of the best novels I read last year was The City & The City by China Miéville. This police procedural set in the present day takes place in the fictional European city-states of Beszel and Ul Qoma. What places this novel squarely in the realm of weird fiction is that Beszel and Ul Qoma occupy the same geographical space. Citizens of one city must unsee the citizens and features of the other city even as they live among them. This seperation extends even to buildings, where some floors or even rooms are in Beszel while others are in Ul Qoma. Citizens of each city must ignore the other lest they fall prey to Breach, a secret police whose operatives are apparently the only ones who can cross from one city to the other at will. It is a tribute to Miéville's ability that this bizarre concept is carried off without calling upon the reader to assume any supernatural agency; it is an act of perception and conditioning on behalf of the inhabitants of both cities.

And so it is with Contraption, my project to overlay a GNU environment on top of the otherwise very un-GNU-like Android on the Beagle Board. Like the citizens of Beszel and Ul Qoma, the GNU and Android environments co-exist in close proximity, within the same geographic file system and random access memory, yet kept separate through the auspices of the Breach-like Linux kernel. You can interact with Android from one one screen while running GNU-based software from a bash shell inside an ssh session from another. For the most part the two systems simply unsee one another by virtue of using different PATH and LD_LIBRARY_PATH environmental variables and by being dynamically linked to different shared objects. The Linux kernel and the dynamic loader keep it all separate.

This isn't some kind of virtual machine magic. It's just a trick of engineering. GNU software lives in /usr/sbin, /usr/bin, and /bin. Android software lives in /sbin, /system/sbin, /system/bin, and /system/xbin. GNU shared objects live in /lib, necessary because some GNU software ignores LD_LIBRARY_PATH. Android shared objects live in /system/lib. Android doesn't use /etc at all, opting instead for /system/etc. While Android forgoes all the usual user-level authentication mechanisms like /etc/passwd, GNU has full access to all of it, enabling for example the dropbear ssh server and the busybox telnet daemon to run unmodified. GNU software dynamically links to the GNU-based shared objects from the Sourcery G++ Lite ARM tool chain from CodeSourcery. Android software dynamically links to the shared objects from the Android ARM tool chain (which is a modified version of the CodeSourcery toolchain) from the FroYo release of the Rowboat port of Android to the Beagle Board.

The only directory conflict was /sbin, which on Android contains only adbd, the Android debugging daemon that serves as a proxy on the target (the Beagle Board) for a development environment running on the host (my desktop). To allow Android to unsee the executables that would normally have been in GNU /sbin, I moved them to /usr/sbin where there were no conflicts with other GNU-based executables.

The utilities, shared object, and memory mapped device driver from my Diminuto project that I described in Memory Mapped Devices on the Beagle Board with Android live in /usr/local/bin, /usr/local/lib, and /lib/modules respectively. They work just as before, except the Android insmod and the busybox insmod have slightly different syntax!

The Contraption project consists almost solely of a single Makefile that configures and builds the necessary software, including the modified Linux kernel, the Android root file system, and the various GNU-based packages such as busybox, bash, strace, and dropbear, creates an SD card of the Android system that can be booted on the Beagle Board, then overlays Contraption on top of it.

Here is the cinema display showing an Eclipse IDE with the Contraption Makefile, a bash shell running via an ssh session on Beagle Board, a serial console to Beagle Board; a little green Android figure standing next to the Beagle Board; a second display showing a web browser running on Android on the Beagle Board.

Here's a screen snapshot of a of the Beagle Board serial console. I booted up Android and fired up the dropbear ssh server from the command line. (See below for some notes on how to automate this.)

Here's a screen snapshot of a bash session via ssh to the Beagle Board. I ran the busybox ps command and used grep to pick out some processes.

And here is a close up of the second display.

Here's a close up of the Beagle Board sitting between the two displays.

As I worked on the design of this, I began to realize the architects of Android must have wanted it to be possible for their system to be used this way, probably to make Android easier to integrate with existing Linux-based mobile phone platforms. Even though I've worked in the telecommunications field for most of the past fifteen years, my inspiration for this project wasn't smart phones. It was the student paper "Using Android in Industrial Automation" by Manuel Di Cerbo and Andreas Rudolf at the University of Applied Sciences at Northwestern Switzerland that impressed me with the possibilities of using Android in other embedded domains.

To make the Android platform really usable in this way I felt it had to be able to easily exploit the huge volume of GNU-based software. Even if one had to reside in Beszel and the other in Ul Qoma.

(You can find a link to a tar ball containing the Contraption distribution on the project web page at the Digital Aggregates Corporation web site.)

Update (2011-02-18)

I added some screen snapshots since I originally published this article.

Update (2011-02-25)

I modified the /init.rc boot script to automate some of the stuff described here.

In the on boot section I changed

hostname localhost

to

hostname contraption

and added

setprop net.dns1 8.8.8.8

setprop net.dns2 8.8.4.4

where 8.8.8.8 and 8.8.4.4 are Google's public DNS servers.

At the bottom I added

service dropbear /usr/sbin/dropbear

oneshot

to start the ssh server automatically. The oneshot option is important: the server is a daemon which appears to Android to exit, and you don't want Android to continuously restart it, otherwise wackiness will ensue.

Can't We All Just Get Along, C++ and C?

Several times in my career I've found myself doing product development in large legacy code bases. By large I mean on the order of many millions of lines of code, representing software shared by many products across a broad product line. Successful product lines inevitably evolve over time to use different hardware targets, operating system platforms, and even programming languages.

One of the most common evolutionary paths I see is the transition from C to C++. C++ is a complex language; some say too complex. To quote C++ inventor Bjarne Stroustrup: "In C++ it's harder to shoot yourself in the foot, but when you do, you blow off your whole leg." But with some care and discipline it can be effectively used even in embedded systems, which can exploit the advanced performance and code generation capabilities that C++ brings to the table, while retaining the ability of C to work close to bare metal.

This evolution results in code bases that are a mixture of C and C++. Making them play well together is one of the paths to success in such an evolution. Using C code from C++ is business as usual; C++ developers routinely do this. Using C++ code from C is rarer, but it is not only possible, it can be a big win, allowing you to incrementally refactor C code into C++ without having to do a forklift upgrade on your entire code base. Here's how I do it.

(There are examples of everything I write about here in my Desperado library that implements design patterns I have found useful in fifteen years of developing embedded systems in C++.)

First, the basics that you may already know.

C and C++ have different linkage conventions, that is, different mechanisms through which the object code produced by each compiler makes functions calls, and with which the linker recognizes function names. C for the most part only understands C linkage conventions. C++ however can be directed to use linkage conventions different from the default C++ conventions. It does this by extending the extern keyword to take an optional argument: the language name. This new operator can be applied to a single C++ function prototype or function implementation, or to a block of them. When applied to a prototype, the operator tells the C++ compiler what linkage convention to use to call the function. When applied to an implementation, it tells the C++ compiler what linkage convention to use to be called. This mechanism can be used to allow C++ programs to call C functions, or C programs to call C++ functions.

Here are some examples that you might see in a C++ header file.

extern "C" size_t dump_bytes(Dump * that, const void * data, size_t length);

extern "C" {

void * heap_malloc(Heap * that, size_t size);

void heap_free(Heap * that, void * ptr);

}

This allows these C++ functions to be called by either C or C++. The C standard header files, for example stdio.h, routinely use this mechanism to make the C functions they declare available to C++ code. What's less well known is that your C++ compiler is allowed provide linkages for other languages as well. You can even specify "C++" as the language.

There are some limitations. C doesn't understand this new argument to extern, it's part of C++. You can't declare a C++ class or instance method to have C linkage, because C has no understanding of C++ classes, so you can only use this in freestanding C++ functions. C also has no understanding of C++ namespaces, so it only works on C++ functions in the global namespace. If you specify "C" as the linkage, you have to conform to C linkage conventions, for example you have to specify an empty parameter list as (void) instead of () just as you do with C. Finally, when you mix C and C++, your main program must be C++, otherwise the C++ runtime system will not be properly set up. (Your first clue that you botched this will be that your C++ static variables will mysteriously not be initialized.) We'll get past many of these issues in just a bit.

A header file can tell whether it is being compiled by a C or C++ compiler. The C++ compiler will automatically define the preprocessor symbol __cplusplus. (This isn't just part of the GNU C++ compiler, it is part of the ISO C++ standard.) So here is a code snippet illustrating a common pattern for a header file declaring C++ functions that is to be included in either C++ or C.

#if defined(__cplusplus)

extern "C" {

#endif

extern void * heap_malloc(Heap * that, size_t size);

extern void heap_free(Heap * that, void * ptr);

#if defined(__cplusplus)

}

#endif

A C++ program including this header file will understand that the functions have C linkage. A C program doing the same will just see the prototypes.

Here is another approach: Desperado has a cxxcapi.h (for C++/C Application Programming Interface) header file that defines preprocessor macros that simplify inoperability between C and C++. It has constructs like this.

#if defined(__cplusplus)

#define CXXCAPI extern "C"

#else

#define CXXCAPI extern

#endif

This allows you to do stuff like this in a header file destined to be included from either C or C++.

#include "cxxcapi.h"

CXXCAPI size_t dump_bytes(Dump * that, const void * data, size_t length);

Okay, that's the basics. Now here's the part that may make your head explode.

All this is well and good. But what you really want is a way to use C++ objects in a C program, not just C++ functions. You can.

In the examples above the types Dump and Heap are actually C++ classes, and the variable that represents a pointer to such an object in the C code, just as the built-in variable this does in a C++ program. Here's the funny thing about C that makes this more than just crazy talk. In C (and for that matter, in C++) you can declare a pointer to a type without actually defining the type. This is a form of forward reference: you are telling the compiler "I'll tell you what this type is later, just hang in there". As long as you never dereference the pointer in the C code or do something that requires the compiler to know something about the type, that is you never write *that or that->field or sizeof(Heap), you never have to define the type. The compiler is totally okay with this. It is so trusting.

typedef struct Heap Heap;

Heap * that;

This C code snippet declares but does not define a new structure named Heap, and a new type also named Heap for this structure. Then it declares that to be a pointer to an object of type Heap without saying exactly what a Heap is. You can pass the variable that around in your C code just as you please. The structure named Heap doesn't have to be defined unless you plan on dereferencing a pointer to it. It's an opaque type, but one that is type safe; its use takes advantage of all the C and C++ compile time type checking.

So, let's put all this together in a code snippet that would be part of a header file that could be included from either C or C++. It describes a C++ class that can be used from either language.

#if defined(__cplusplus)

class Heap {

public:

Heap();

virtual ~Heap();

virtual void * malloc(size_t size);

virtual void free(void * ptr);

};

extern "C" Heap * heap_new(void);

extern "C" void heap_delete(Heap * that);

extern "C" void * heap_malloc(Heap * that, size_t size);

extern "C" void heap_free(Heap * that, void * ptr);

#else

typedef struct Heap Heap;

extern Heap * heap_new(void);

extern void heap_delete(Heap * that);

extern void * heap_malloc(Heap * that, size_t size);

extern void heap_free(Heap * that, void * ptr);

#endif

This one snippet declares two different interfaces to the same Heap object: a C++ class-based interface, and a C function-based interface. Note that the C function-based interface can actually be used from C++. (This is especially useful if you are using a C++-based unit testing framework like Google Test, of which I am quite the fan.) But generally, your C++ code will use the C++ interface, and your C code will use the C interface. It is all in one header file, so it can be easily maintained in parallel.

I won't bore you with what the C++ class actually does; your imagination can fill in the details. The C functions could be implemented in the same C++ source file as the C++ class implementation. (In Desperado, I actually keep them separate so statically linked embedded C++ applications don't need to carry the extra baggage of the thin C implementation layer that they don't need and won't use.) We don't need any __cplusplus magic since it's all C++ code.

#include "Heap.h"

extern "C" Heap * heap_new(void) {

return new Heap;

}

extern "C" void heap_delete(Heap * that) {

delete that;

}

extern "C" void * heap_malloc(Heap * that, size_t size) {

return that->malloc(size);

}

extern "C" void heap_free(Heap * that, void * ptr) {

that->free(ptr);

}

Note that since this is a C++ source file, its Heap refers to the actual C++ Heap class. This is totally transparent to the C code that may carry around the address in its own pointer variable of type Heap. This C to C++ adaptation layer can take advantage of virtual functions and namespaces. It is full fledged C++ code.

So here's a snippet of a C application, compiled with the C compiler, that uses the C++ class.

#include "Heap.h"

Heap * heap;

void * data;

heap = heap_new();

data = heap_malloc(heap, 1024);

heap_free(heap, data);

heap_delete(heap);

This is especially cool when using virtual functions. If you have another C++ class that derives from Heap, say TraceableHeap, but its C interface is the same, you can just substitute your traceableheap_new() for heap_new() in your C code and rely on the existing Heap C interface to do all the heavy lifting.

With very little effort and minimal ongoing additional maintenance cost, you can use this technique to gradually introduce C++ into C code bases, and take advantage of not just C functions in C++ programs, but C++ classes in C programs.

(A big Thank You to Marshall Cline and the C++ FAQ from which I developed this technique many years ago.)

Update (2011-02-21)

You might be wondering if, instead of using an opaque type, you can define a complete C structure definition whose fields shadow those of the C++ class definition. That would allow you to access the C++ fields from C. The answer is: maybe; but it's really dangerous. That's because whether it works or not depends on how your C++ compiler implements objects.

For example, if you have virtual functions, the compiler will insert an invisible field known as the virtual pointer somewhere in your object. The virtual pointer points to a table of function pointers that point to the virtual functions for the class. C won't know squat about this. Some compilers place this pointer at the beginning of the object; others at the end. The ISO standard makes this an implementation issue.

Even if it worked, C could have no respect for the protected and private qualifiers in C++. Generally speaking: I recommend against it, for no other reasons than code portability.

The fact that your C++ compiler may place invisible fields in your object as part of its implementation is the same reason why you cannot safely apply memory operations from the C standard library, like memset and memcpy, to C++ objects. Doing so writes over these invisible fields. Specifically, they write over the virtual pointer. The result is (on a good day) your program crashes, or (on a bad day) it now behaves really weirdly because you've just caused the virtual pointer in a derived class object to now point to the virtual table of a base class object. Wackiness ensues.