Tux Droid, a Python-Fueled robot

• http://www.tuxisalive.com

Introduction

David Bourgeois FOSDEM 2007 Wireless robot for Linux Python API OSS and community

Hello,

I'm David Bourgeois, I'm very pleased to be here today to present you Tux Droid and how Python is related to it.

I work for C2ME, a small Belgian company which is doing electronics, gadgets and Toys and I'm currently spending most of my time developing the Tux Droid Open Source Robot of the Kysoh company.

So what is Tux Droid? This is Tux Droid. This is a wireless robot for your computer and as you can imagine it doesn't run very well under Windows. That means if you're the happy owner of a Linux box, you can use Tux Droid as a companion for your OS and applications. You can control it completely from your computer. For example you can have tux on top of your screen and each time you click with your mouse, tux can flash the blue eyes. Great function, I'm sure some of you won't be able to live without that anymore. It's even better if you add a sound. Imagine you put your Tux in the kitchen and each time you click on your mouse there's a flash of the eyes and a 'kwak' sound, I'm sure your wife will love it.

And as we used Python to do that, well, not exactly that but that kind of things, that's why I'm here today. We started to develop a Python API that can be used to take control of your tux quite easily.

As you can do a bit whatever you want with Tux Droid, there's not really a single software that goes with it. As most people will have different needs, we set up a community portal where everybody will be able to share it's own applications. We wanted an open source project, you'll find there all the software codes there but also the schematic and the firmware codes.

Outline

Functionalities Dissecting Tux Droid Limitations of the embedded processors Python to the rescue Software architecture Python API, interpreter and scripts Where the fun starts Getting the idea? I don't like Python

A quick overview of what I will present you.

At first we'll look at the functionalities of Tux Droid as a standalone robot, without the computer. We'll have a quick look on the inside so you should get an idea of what's possible to do with Tux Droid alone and what's not.

The second part will show you how we can extend those functionalities with the use of a computer or the other way around, how we can extend the functionalities of the computer by using Tux Doid.

There's no conclusions but I'd like to have a final word for those of you who don't like Python but are brave enough to join this python dev room.

Functionalities

Let's take a tux out of his box and I'll show you what it can do.

Tux has a set of actuators and sensors and can do some basic stuff with that.

Actuators: movement of the eyes, the mouth, the flippers, spinning. Blue led's can be lit separately. There's an IR emitter so you should be able to send some codes to your TV or HiFi. You heard the speaker already an with the internal sound flash it's possible to store around 2 minutes of local sounds.

Sensors include an IR receiver. We also provide a remote control with Tux Droid and you can use it in this standalone mode to control movements. The photosensor measures the ambient light so it's possible to react differently when it's dark or light. The microphone is mostly useful when the captured sound can be sent back to the computer but it can still be used to detect handclaps or if there's background noise or not. We have buttons on the head and the flippers.

The standalone behavior is used to give some animation to Tux when it's not connected to the computer. It uses the event manager to associate events, like the push of a button, to a small script. For example if I plug the charger, there's an event when the plug is inserted that turns on one blue led. Slightly after the charge will start and a small sequence with sounds is associated.

The wireless link at 2.4GHz is used to share all of this with the computer. We'll see that later on, now we'll have a quick look at the behavior and sound modules which make the main PCB of Tux Droid.

The core of Tux Droid is made of 2 small 8bits AVR of Atmel. Those microcontrollers are communicating together and handle all sensors and actuators, the sound flash and the communication with the RF module. Let's have a look at the real thing.

Dissecting Tux Droid

You can see the 2 AVR on the main PCB. At the bottom we have al the motor drivers, the charger IC, on the left hand side there are 6 switches to control the movements and 3 as buttons, and the connector for the head PCB. There you'll find the blue leds, the IR led and receiver and the phototransistor. The sound flash is at the top. At the right we have the power supply, the audio amplifier and the microphone preamplifier. The microphone is in the mouth.

Limitations of the embedded processors

CPU:Those small AVR processors are great to handle sensors and actuators and can deal perfectly with real time tasks like capturing an IR code or stopping the eyes motor exactly at the closed position. A little bit too late and they reopen.

But they have an 8kB program memory and 512 bytes ram. That's not much. You can forget about wifi or a TCP/IP stack of course. Audio processing, we could take the microphone input to compute a peak signal in real time or detect a handclap but that's about it. Forget about capturing the light measurement during the whole day.

Behavior: We can do some basic behavior like triggering a sequence when an event occurs. But you can't track all the events to build a full context and history. We could have tux saying 'hello' when someone passes in front and the light level changes suddenly, but we can't have tux saying 'I didn't see anybody for 2 hours, where were you?'.

Input: You won't be able to give life to your robot if all it knows is the state of a few buttons. With access to your computer status and the internet, you could have tux saying 'Hello, it's raining today', well depending on the weather of course but in Belgium it will probably be that.

Output: Again, you can't have tux stream a podcast from the internet and move the mouth in synchronisation with the sound, you don't have the media and you don't have the audio processing power.

Python to the rescue

Object-oriented Interpreted Interactive Modular Dynamic High-level Portable Extensible

I would love to tell you how great python is. There's a lot of reasons and if I was a real python developer, I could have explained you many things about the pros of Python. But I'm not so I won't do it. I'm just going to connect the dongle to the computer and launch a simple Python script.

Python to the rescue

#!/usr/bin/python import tux tux.connect_to_daemon() tux.getalive()

d: And you'll see that now Tux can talk a bit more

t: Yes I can talk

d: And that he can move the flippers

t: Like this? Hey Dave, what's up? what's those stupid things you ask me to do?

d: Well, that was just to show to the audience you're alive now

t: Off course I can talk and flap, look at me, do I look that unreal, hum? I'm alive!

d: Yeah I can see that, thanks. Can you calm down a bit so I can continue my speech?

t: You're boring us you know?

d: Tux?

t: Ok, ok

So to understand what's going on here we need to have a quick look at the daemon interfacing the dongle.

Software architecture

The daemon is the interface between the dongle and your application. There's a USB communication with the dongle, a TCP/IP communication with the application. Audio is not handled by the daemon as the dongle has a standard USB Audio Device interface so any audio application can directly use the device.

Let's have a look at what happens when I push on a button of the remote, as everybody noticed that I was using the remote to get the next sentence.

First the IR code is captured by the core CPU of Tux, this status is sent on the RF link to the dongle which places it on the USB buffer. The daemon has a USB thread that regularly polls the dongle to get new status. The IR status is now stored in a complete status structure in the daemon.

The application is connected to the daemon by TCP/IP and will be notified of this new status. Now we have associated this IR keystroke in the application with a sound and a movement. The application will send the sound to the USB audio device directly and that's it, the sound will get out of the speaker. The movement command is sent to the daemon which forwards it to the dongle, the RF and will be executed by the core CPU.

So we have a complete status structure in the daemon and our application can access to it and send commands.

Python API, interpreter and scripts

• You can use the API directly from the interpreter you can
  

>>> tux.cmd.leds_blink(200,10)
>>> tux.cmd.eyes_on_free(4)
>>> tux.tts.speak("Do pythons eat penguins?")
>>> if tux.event.wait_lwing_bt_pushed(60):
...     tux.tts.speak("Bye bye!")

• You can easily write a small script

#!/usr/bin/python
from tux import *

tux.tts.select_voice(3,130)
tux.cmd.leds_blink(200,10)
tux.cmd.mouth_on_free(22)
tux.tts.speak("The box said: 'Requires Windows 95 or better', so I installed LINUX")
tux.cmd.eyes_on_free(4)
tux.cmd.wings_on_free(4)
tux.tts.speak("ha ha ha")
if tux.event.wait_head_bt_pushed(60):
    tux.tts.speak("Ouch! That hurts!")

The Python API we developed handles the TCP/IP communication for you so you don't have to worry about that. Let's have a look at a simple script.

We can directly use the API from the interpreter. There's a cmd class which groups all the commands that can be sent to Tux. So the command tux.cmd.leds_blink(200,10) will blink the blue leds 200 times with a period of 10 time units which is 40ms.

The event class is used to monitor the status and generate an event when the condition is met. Here we just wait for the left wing button to be pushed.

The tts class of the API is all the text to speech functions. Just give your text to get it out loud.

Let's have a try with the interpreter.

Where the fun starts

Explain the event registration and show the clock application, use a sound that says "something"

Then show the remote control and most of it's functions.

Maybe a script on music to show Tux dancing?

Getting the idea?

We're at the beginning of developping applications for Tux Droid so we have more ideas than running applications. I'm going to share some of them with you so you'll get a better view of all we should be able to do with Tux.

I don't like Python

There are still a couple things you can do: Join the AFPY, read the "Python Bible" and get converted :-) Port the complete API to your favorite language Use Tux as a development kit to learn microcontrollers programming and electronics

You're not alone, many of you have already been converted. I suggest you start reading the python bible with the help of the AFPY guys and everything will be fine. In case you don't survive that step, you may port the AI to your favorite language of course. But you should consider a reconversion in electronics and use Tux as a development platform to learn microcontrollers programming, how to drive motors, acquire signals and sound, deal with interrupts, all kind of things. The GNU toolchain is available for the AVR so there's no reason not to step into that.

Tux Droid, a Python-Fueled robot