Monday, February 28, 2011

Deconstructing the AR.drone: Part 2

When we last left our hero (that would be me) I was TELNETing into the AR.drone at the address its DHCP server publishes as the gateway (a.k.a. router) address. When I do so, I am delighted to see the standard greeting from the BusyBox built-in shell, and a root prompt. If you've been paying attention, you've seen BusyBox mentioned many times in this blog. It's the Swiss Army Knife of embedded Linux software: a single binary that implements dozens of the standard Linux utilities (and some not so standard ones). I've used BusyBox to implement a full-blown Linux embedded system with just three binary executables: the Linux kernel, the BusyBox binary, and an application binary. BusyBox has been mentioned here in the context of Diminto, Arroyo, and Contraption. (As usual, apologies for the poor formatting of the code throughout this article.)

BusyBox v1.14.0 (2010-12-02 15:13:17 CET) built-in shell (ash)
Enter 'help' for a list of built-in commands.

A quick check of the network interfaces reveals an ath0 interface at the gateway address and the usual loopback interface lo.

# ifconfig
ath0 Link encap:Ethernet HWaddr 00:26:7E:50:5D:A6
inet addr: Bcast: Mask:
RX packets:9382 errors:0 dropped:0 overruns:0 frame:0
TX packets:45641 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1279047 (1.2 MiB) TX bytes:63893346 (60.9 MiB)

lo Link encap:Local Loopback
inet addr: Mask:
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

When we used the Nmap utility to scan for open ports on the AR.drone, we found four open TCP ports. Scanning for open UDP ports is more problematic. But now we can just ask the drone itself for the ports on which it is listening. We see the TCP ports we already knew about, plus several UDP ports: 5554, 5555, 5556, and 67. UDP port 67 is typically a BOOTP network boot server for diskless workstations. BOOTP is a service provided by many DHCP servers, so I'm guessing this port is just an artifact of the drone's use of DHCP.

# netstat -a
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0* LISTEN
tcp 0 0* LISTEN
tcp 0 0* LISTEN
tcp 0 0* LISTEN
udp 0 0*
udp 0 0*
udp 0 0*
udp 0 0*
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags Type State I-Node Path
unix 3 [ ] DGRAM 809 /dev/log
unix 2 [ ] DGRAM 811

Next, a quick check of what's running. The wide disparity of the process identifiers suggests that this platform uses a lot of scripting which tends to spawn a lot of short lived subprocesses (although other explanations are of course possible). We can see the TELNET daemon telnetd running, our own shell and command, and the DHCP daemon udhcpd. More intriguing, there are kernel and system logging daemons klogd and syslogd running. Everything in square brackets is a kernel process, typically something like a device driver or kernel module that runs at least partially in a process context.

# ps
1 root 2732 S init
2 root 0 SW< [kthreadd]
3 root 0 SW< [ksoftirqd/0]
4 root 0 SW< [watchdog/0]
5 root 0 SW< [events/0]
6 root 0 SW< [khelper]
75 root 0 SW< [kblockd/0]
87 root 0 SW< [khubd]
93 root 0 SW< [kmmcd]
116 root 0 SW [pdflush]
117 root 0 SW [pdflush]
118 root 0 SW< [kswapd0]
119 root 0 SW< [aio/0]
120 root 0 SW< [nfsiod]
764 root 0 SW< [ubi_bgt0d]
767 root 0 SW< [ubi_bgt1d]
771 root 0 SW< [ubi_bgt2d]
790 root 0 SW< [p6-spi.0]
803 root 0 SW< [rpciod/0]
812 root 0 SW< [ubifs_bgt1_0]
824 root 0 SW< [ubifs_bgt2_0]
826 root 0 SW< [ubifs_bgt2_1]
831 root 1632 S /bin/factory_reset_cb
892 root 0 SW< [ksdiorqd]
893 root 0 SW< [ar6000_io]
949 root 2736 S telnetd -l /bin/sh
951 root 2732 S udhcpd /tmp/udhcpd.conf
957 root 2812 S inetd
958 root 2736 S /bin/sh /bin/
959 root 11824 S /bin/program.elf
961 root 2732 S init
962 root 2732 S /sbin/syslogd -n -m 0
963 root 2732 S /sbin/klogd -n
989 root 2736 S /bin/sh
1026 root 2816 R ps

I appear to have landed in the root directory when I logged in. Next step is to peruse the /proc file system. This is where the kernel and device drivers expose internal state in the form of pseudo-files in the file system. I've found /proc invaluable when I've written my own device drivers to expose state and debugging information.

# pwd
# cd proc

First we check what version of the Linux kernel and GNU libraries the system is using, and what tool chain it was built with. Note it was built with my favorite ARM tool chain from CodeSourcery. A version of this same tool chain was used for Arroyo and Contraption.

# cat version
Linux version (aferran@Arrakis)
(gcc version 4.3.3 (Sourcery G++ Lite 2009q1-203) )
#1 PREEMPT Thu Dec 2 15:21:36 CET 2010

Looking at the CPU into shows we're running on an ARM926 core. It's probably part of a System On a Chip (SoC) that incorporates many embedded peripheral I/O controllers. (The ARM926 core is similar to the ARM920 core used in the Atmel AT91 processor that was part of Diminuto and Arroyo.) This ARM core include support for the Thumb (16-bit) and Java byte code instruction sets in addition to the usual 32-bit ARM instruction set.

# cat cpuinfo
Processor : ARM926EJ-S rev 5 (v5l)
BogoMIPS : 233.47
Features : swp half thumb fastmult edsp java
CPU implementer : 0x41
CPU architecture: 5TEJ
CPU variant : 0x0
CPU part : 0x926
CPU revision : 5
Cache type : write-back
Cache clean : cp15 c7 ops
Cache lockdown : format C
Cache format : Harvard
I size : 32768
I assoc : 4
I line length : 32
I sets : 256
D size : 16384
D assoc : 4
D line length : 32
D sets : 128

Hardware : Mykonos Parrot platform
Revision : 0904
Serial : 0000000000000000

Looking at the command line passed to the kernel by the boot loader tells us something about how the system is configured (if at all) as it boots up. The sizes of the flash partitions are specified, and we now know that terminal device ttyPA0 is the console terminal and it runs at 115200 baud. The AR.drone has an external USB port accessible on its underside using a non-standard connector. It's under a little rubber cover that has the USB symbol on it. The 115200 baud rate is very common for serial to USB converter devices, so we'll keep that in mind as we poke around.

# cat cmdline
console=ttyPA0,115200 loglevel=4 ubi.mtd=Pfactory,2048
ubi.mtd=Psystem,2048 ubi.mtd=Pupdate,2048
root=ubi1:system rootfstype=ubifs parrot5.low_latency=1

Checking the I/O memory tells me what segments of physical memory have been reserved by the kernel and device drivers. Much of these physical memory segments are likely to be control and status registers for memory-mapped I/O devices. Or particular interest are devices that unless they are grossly misnamed appear to be NAND flash devices (p6-nand), an On The Go USB controller (dwc_otg), an SD card interface (p6-sdhci), some Universal Asynchronous Receiver Transmitters a.k.a. serial interfaces (uart), some Serial Peripheral Interfaces a.k.a. SPI (p6-spi), and some Inter-Integrated Circuit a.k.a. I2C interfaces (parrot5-i2cm). SPI and I2C are serial bus standards used to talk to other chips such as (in my experience anyway) networking chips (for SPI) or sensors (for I2C). I'm assuming the p6_camif devices are the cameras. Not sure what the ba315 is, even after a web search (but later we'll see that it is some kind of NAND flash controller).

# cat iomem
40000000-47ffffff : System RAM
40025000-40347fff : Kernel text
40348000-403e3f63 : Kernel data
c0300000-c03fffff : p6-nand.0
c0400000-c04fffff : dwc_otg.0
c0600000-c06fffff : dma-pl08x.0
c0700100-c07001ff : p6-sdhci.1
c0700100-c07001ff : p6-sdhci
d0040000-d004ffff : p6_camif.0
d0050000-d005ffff : p6_camif.1
d0070000-d007ffff : parrot5-uart.0
d0070000-d0070fff : parrot5-uart
d0080000-d008ffff : parrot5-uart.1
d0080000-d0080fff : parrot5-uart
d0090000-d009ffff : parrot5-uart.2
d0090000-d0090fff : parrot5-uart
d00b0000-d00bffff : p6-spi.0
d00b0000-d00bffff : p6-spi
d00f0000-d00fffff : p6-nand.0
d00f0000-d00fffff : ba315
d0150000-d015ffff : parrot5-i2cm.0
d0150000-d015ffff : parrot5-i2cm
d0160000-d016ffff : parrot5-i2cm.1
d0160000-d016ffff : parrot5-i2cm

Device drivers are always good to peruse. Apart from what the I/O memory has already suggested, we see a flash driver (mtd for Memory Technology Device), a RAM disk, and a Multi-Media Card driver (mmc). Significantly, the latter two are block devices, meaning they are likely used by the file system layer. (I have to admit now that although I'm pretending to be seeing this all this for the first time, I've already been doing some hacking on the AR.drone and can verify that it does indeed implement a persistent read-write file system.)

# cat devices
Character devices:
1 mem
2 pty
3 ttyp
4 /dev/vc/0
4 tty
5 /dev/tty
5 /dev/console
5 /dev/ptmx
7 vcs
10 misc
13 input
81 video4linux
89 i2c
90 mtd
128 ptm
136 pts
180 usb
189 usb_device
204 ttyPA
204 ttyJ
251 usbmon
252 ubi2
253 ubi1
254 ubi0

Block devices:
1 ramdisk
179 mmc

Linux device drivers are identified by a major device number. Each driver can use a minor device number as essentially a parameter. The miscellaneous device driver (misc) is a generic driver that manages small device drivers that don't need their own major number but are instead identified by a minor device number. (The device driver I talked about for Contraption was such a driver.) Of interest among the miscellaneous drivers is a General Purpose I/O driver (gpio), and a watchdog (watchdog). GPIO is a simple hardware interface that allows a driver to set individual hardware lines to high or low states. It is sometimes used to implement standard controllers like SPI and I2C in software, or to control devices which do not conform to some other standard interface. A watchdog is typically a device that autonomously reboots the system if it is not petted (stimulated) periodically. It is a way to recover from the software getting seriously sideways like a tight uninterruptable loop.

# cat misc
56 network_throughput
57 network_latency
58 cpu_dma_latency
64 pwm
66 gpio
67 dmamem
59 ubi_ctrl
60 log_radio
61 log_events
62 log_main
63 binder
130 watchdog

Modules are loadable kernel drivers or other software that can be dynamically loaded and linked into the kernel at run-time instead of being compiled into the kernel statically. I see one such module, which I suspect is an SD card driver.

# cat modules
p6_sdhci 3588 0 - Live 0xbf000000

As embedded systems go, this one is pretty well endowed with RAM. It looks to be a 128 megabyte system of which about 105 megabytes are free when it is mostly idle.

# cat meminfo
MemTotal: 126072 kB
MemFree: 105700 kB
Buffers: 0 kB
Cached: 3572 kB
SwapCached: 0 kB
Active: 5392 kB
Inactive: 1924 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 3756 kB
Mapped: 2788 kB
Slab: 3636 kB
SReclaimable: 1460 kB
SUnreclaim: 2176 kB
PageTables: 168 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 113464 kB
Committed_AS: 5968 kB
VmallocTotal: 843776 kB
VmallocUsed: 5296 kB
VmallocChunk: 837948 kB

Unsorted Block Image File System (UBIFS) is a flash-based file system which I have never used; I've used the older Journalling Flash File System 2 (JFFS2). We see several mounted UBIFS file systems, including root /, /factory, /update, and /data. I'm guessing at least two of these are redundant file systems used during firmware updates or reflashing so that if the update fails there is a backup system to which to fall back to.

# cat mounts
rootfs / rootfs rw 0 0
ubi1:system / ubifs rw 0 0
tmp /tmp tmpfs rw 0 0
proc /proc proc rw 0 0
dev /dev tmpfs rw 0 0
devpts /dev/pts devpts rw,mode=600 0 0
sys /sys sysfs rw 0 0
ubi0:factory /factory ubifs ro 0 0
ubi2:update /update ubifs rw,sync 0 0
ubi2:data /data ubifs rw 0 0

The Memory Technology Device or mtd shows us how the flash memory is partitioned. I'm guessing these MTD partitions map to the UBIFS file systems and at least two are redundant. The names also suggest to me that this processor may have a two-stage boot loader. It is not uncommon to have a simple boot loader that loads a more complex boot loader like U-Boot, that then loads the Linux kernel.

# cat mtd
dev: size erasesize name
mtd0: 00040000 00020000 "Pbootloader"
mtd1: 00800000 00020000 "Pmain_boot"
mtd2: 00800000 00020000 "Pfactory"
mtd3: 01000000 00020000 "Psystem"
mtd4: 05fc0000 00020000 "Pupdate"

Finally, we look at the counts of how many times different device drivers have received interrupts from the hardware. This can be a good clue about how active different devices in the system are. For example, the high counts on some of the serial devices (parriot5-uart) suggests that they are some kind of sensors or controllers and not a currently unused console or debug port. Ditto for our guess about the camera devices (p6_camif).

# cat interrupts
4: 696468 VIC mmc0
5: 513 VIC parrot5-uart
6: 354060 VIC parrot5-uart
7: 177177 VIC parrot5-uart
10: 6087 VIC BA315 NAND controller
14: 0 VIC dma-pl08x
19: 90212 VIC Parrot6 Timer Tick
21: 493317 VIC p6_camif.0
22: 477970 VIC p6_camif.1
28: 8600 VIC parrot5-i2cm, parrot5-i2cm
32: 0 p6-gpio p6_kbd_input
33: 0 p6-gpio p6_kbd_input
Err: 0

Next up: figuring out the boot-time sequence and how the AR.drone configures its network.


Aaron Phelps said...

Hi Chip,

Great work on taking apart the ARDrone! Me and my colleagues were wondering if you've nailed down what you think might be the UART connections that give commands to the speed controllers on the motors. Also, if you could give me any hints as to how to write an embedded executable that would be able to give commands directly to the motors, I would be really grateful. This would be for research purposes, not commercial.


Aaron Phelps

Chip Overclock said...

(Edited to correct a typo.)

Thanks for the kind words! I confess I've moved on to another personal project. These two guys (URLS below) have gone a lot further than I have, including some work at how to talk to the navboard and the motors, although exactly how the motors are controlled still sounds like a mystery.

I do have a theory though: a colleague of mine has done cooling fan control using Pulse Width Modulation (PWM) to control the speed of the fans by controlling the duration of off versus on power intervals to the fans. I'm wondering if the serial output is somehow demultiplexed into separate PWM bit streams that control the fans. That would be an interesting way to control the speed of each individual fan, which I think is the only "knob" the "autopilot" has.

An inexpensive logic analyzer like the Saleae Logic that I wrote about in one of the AR.drone articles might tell you something. But I'd check the voltages to the motors first to make sure it was something the analyzer could handle, since you're looking for power on/off intervals to the motors. If I'm right, then the bits controlling the motors probably feed into some kind of relay that handles the actual power to the motors, and it's the bits you're interested in, not really the power.

The latter blog identifies ttyPA1 as the motor control port, so just looking at that will probably tell you something interesting. It says ttyPA2 is the navboard.

There's lot of good articles on the web on how to use PWM algorithms to control stuff like motor speed and blinky LEDs. It's actually pretty cool and pretty simple all at the same time.

Or I could be full of crap.

Good luck, and thanks for the comment!