The assigned brief or mission is to control a BB8 Star wars droid using a Raspberry PI with Bluetooth and Speech recognition. All this should be very possible using just Python scripting and some open source libraries. 


“I find your lack of faith disturbing.” — Darth Vader

 

Steps

  1. Setting Up Linux Development Environment
  2. Bluetooth under Python: BluePy
  3. Speech Recognition: PocketSphinx
  4. Speech Recognition: Jasper
  5. Python Scripts for the BB8

The Story

My son has bought one of those Star Wars toy called a BB8 droid. I had heard vaguely somewhere it has a kind of API available for it. Thinking he could learn a bit of coding and how things work. There is of course a Javascript API  however I wanted to use Python for this concept.

“Never tell me the odds!” — Han Solo

To begin not all the libraries for Linux are compatible with Mac, or at least not so straight to install without some head scratching. My plan originally at first, I thought would be simple, run the API’s I need on my Macbook. However this turned out quite lengthy and not feasible. The concept even though seemed elementary was this,  get two libraries for Python to run and operate together. The first of these libraries would be of course the  bluetooth module known as BluePY. The BluePy  library is what needed to communicate over Bluetooth from computer to the BB8 droid within Python scripting. While the second library was the magic to allow some sort of speech recognition via a microphone. What should have taken half an hour became a bit of a struggle. I just want to tell the droid to move forward and back etc…  Nonetheless there is a solution.

The solution I started with was on Linux (as my Raspberry board I ordered hadn’t arrived yet).  The Pi 3 Raspberry has built in Bluetooth 4.0, so all that’s required should be code and a usb microphone!

Hardware Required

  • For the BB8 we need a Bluetooth 4.0 adapter.
  • Microphone
  • Raspberry Pi 3
  • Micro SD 4GB

Linux Development Environment

For our test environment I’m going to use Linux as all the library work well under this OS.  So let’s begin by downloading Virtual Box and an Ubuntu ISO image to setup a virtual machine on my Macbook Air.

If you don’t need a test environment and wish to run everything on the Pi skip to the next step 

The reason is I need to have ‘bluepy’ working. This is a library for Python to connect over Bluetooth. It seems to have a lot of difficult working under MacOS. I myself had major problems getting the ‘glib’ working too even when I had it installed by using Homebrew. (Brew by the between is a useful tool to install packages in MacOS similar to apt-get under Linux.) However the “brew” wasn’t cutting it! That’s how I came to have a go with the Virtual Box running on the Mac.

“Do. Or do not. There is no try.” — Yoda

Of course the same can be done for Windows too. You download the software its free and open source then follow the screenshots below.

Linux Virtual Box Setup

Linux Virtual Box Setup

Linux Virtual Box Setup

Linux Virtual Box Setup

Ubuntu Installation on Virtualbox

Ubuntu Installation on Virtualbox

Bluetooth under Python

Once your environment (Ubuntu or Raspbian) is up and running. It was time to get the BluePy alive and connecting to the droid device.

From the terminal, run the following commands:

sudo apt-get install python-pip libglib2.0-dev
sudo pip install bluepy
Installing modules on Ubuntu

Installing modules on Ubuntu


Verify that Python is installed by executing:

python --version

Next check if bluetooth is available. For the BB8 we need a Bluetooth 4.0 adapter.

sudo hcitool lescan

If ‘No such device’ is displayed enable the USB bluetooth adapter in the Virtual Box from the USB icon on the bottom of the window.

Test Bluetooth Python Module: BluePY

cd /usr/local/lib/python2.7/dist-packages/bluepy
python btle.py <MAC Address>

This should display some information about the device.

Now, test Python runs and sees the library.

python

then enter the following code

 from bluepy.btle import UUID, Peripheral
 if no error occur you are good to go.

if all goes well, no error message should be shown in the Python environment.

“What if I told you that the Republic was now under the control of a Dark Lord of the Sith?” — Count Dooku

Then exit the Python interpreter. Then proceed to the next step.

Speech Recognition: PocketSphinx

So the next step of the project is getting some speech recognition up and running. Pocketsphinx is an open source library which doesn’t require an internet connection when installed on an embedded device such as a Raspberry. So privacy advocates should be happy and including myself. The PocketSphinx library can be improved by use difference Models and Dictionaries. Both of these are available as free downloads too. It also supports many languages. So if you wish to speak German or Chinese to your Robot – “Kein Problem, 没问题” .

The project is available on Github : https://github.com/cmusphinx/pocketsphinx-python

sudo apt-get install -y python python-dev python-pip build-essential swig git libpulse-dev
sudo pip install pocketsphinx
sudo apt install pocketsphinx
sudo apt-get install pocketsphinx-utils

You’ll need PyAudio

sudo apt-get install python-pyaudio python3-pyaudio

To test use the following command

PocketSphinx_continous -inmic yes

Once you got things up and running,
If you need more accuracy then download and implement the difference models.
https://sourceforge.net/projects/cmusphinx/files/Acoustic%20and%20Language%20Models/US%20English/

Speech Recognition: Jasper

Jasper takes things further it’s also based on PocketSphinx too : http://jasperproject.github.io In part two we’ll install this on the Raspberry Pi for now here some video explaining the possibilities with Jasper.

 “There’s always a bigger fish.” — Qui-Gon Jinn

Python Scripts for the BB8

To combine command with PocketSphinx – https://github.com/slowrunner/Pi3RoadTest/blob/master/Sphinx%20On%20Pi3.txt

Following are Python scripts to control the BB8.
https://github.com/jjinking/SpheroBB8-python
https://github.com/jchadwhite/SpheroBB8-python
https://gist.github.com/ali1234/5e5758d9c591090291d6

 

In Part Two

Write the script for the Raspberry, get Jasper up and running on an Raspberry PI

 “I’m just a simple man trying to make my way in the universe.” — Jango Fett