The assigned brief or mission is to control a BB8 Star wars droid using a Raspberry PI with Bluetooth and Speech recognition. All this should be very possible using just Python scripting and some open source libraries.
“I find your lack of faith disturbing.” — Darth Vader
- Setting Up Linux Development Environment
- Bluetooth under Python: BluePy
- Speech Recognition: PocketSphinx
- Speech Recognition: Jasper
- Python Scripts for the BB8
“Never tell me the odds!” — Han Solo
To begin not all the libraries for Linux are compatible with Mac, or at least not so straight to install without some head scratching. My plan originally at first, I thought would be simple, run the API’s I need on my Macbook. However this turned out quite lengthy and not feasible. The concept even though seemed elementary was this, get two libraries for Python to run and operate together. The first of these libraries would be of course the bluetooth module known as BluePY. The BluePy library is what needed to communicate over Bluetooth from computer to the BB8 droid within Python scripting. While the second library was the magic to allow some sort of speech recognition via a microphone. What should have taken half an hour became a bit of a struggle. I just want to tell the droid to move forward and back etc… Nonetheless there is a solution.
The solution I started with was on Linux (as my Raspberry board I ordered hadn’t arrived yet). The Pi 3 Raspberry has built in Bluetooth 4.0, so all that’s required should be code and a usb microphone!
- For the BB8 we need a Bluetooth 4.0 adapter.
- Raspberry Pi 3
- Micro SD 4GB
For our test environment I’m going to use Linux as all the library work well under this OS. So let’s begin by downloading Virtual Box and an Ubuntu ISO image to setup a virtual machine on my Macbook Air.
If you don’t need a test environment and wish to run everything on the Pi skip to the next step
The reason is I need to have ‘bluepy’ working. This is a library for Python to connect over Bluetooth. It seems to have a lot of difficult working under MacOS. I myself had major problems getting the ‘glib’ working too even when I had it installed by using Homebrew. (Brew by the between is a useful tool to install packages in MacOS similar to apt-get under Linux.) However the “brew” wasn’t cutting it! That’s how I came to have a go with the Virtual Box running on the Mac.
“Do. Or do not. There is no try.” — Yoda
Of course the same can be done for Windows too. You download the software its free and open source then follow the screenshots below.
Once your environment (Ubuntu or Raspbian) is up and running. It was time to get the BluePy alive and connecting to the droid device.
From the terminal, run the following commands:
sudo apt-get install python-pip libglib2.0-dev sudo pip install bluepy
Verify that Python is installed by executing:
Next check if bluetooth is available. For the BB8 we need a Bluetooth 4.0 adapter.
sudo hcitool lescan
If ‘No such device’ is displayed enable the USB bluetooth adapter in the Virtual Box from the USB icon on the bottom of the window.
Test Bluetooth Python Module: BluePY
cd /usr/local/lib/python2.7/dist-packages/bluepy python btle.py <MAC Address>
This should display some information about the device.
Now, test Python runs and sees the library.
then enter the following code
from bluepy.btle import UUID, Peripheral if no error occur you are good to go.
if all goes well, no error message should be shown in the Python environment.
“What if I told you that the Republic was now under the control of a Dark Lord of the Sith?” — Count Dooku
Then exit the Python interpreter. Then proceed to the next step.
So the next step of the project is getting some speech recognition up and running. Pocketsphinx is an open source library which doesn’t require an internet connection when installed on an embedded device such as a Raspberry. So privacy advocates should be happy and including myself. The PocketSphinx library can be improved by use difference Models and Dictionaries. Both of these are available as free downloads too. It also supports many languages. So if you wish to speak German or Chinese to your Robot – “Kein Problem, 没问题” .
The project is available on Github : https://github.com/cmusphinx/pocketsphinx-python
sudo apt-get install -y python python-dev python-pip build-essential swig git libpulse-dev sudo pip install pocketsphinx sudo apt install pocketsphinx sudo apt-get install pocketsphinx-utils
You’ll need PyAudio
sudo apt-get install python-pyaudio python3-pyaudio
To test use the following command
PocketSphinx_continous -inmic yes
Once you got things up and running,
If you need more accuracy then download and implement the difference models.
Jasper takes things further it’s also based on PocketSphinx too : http://jasperproject.github.io In part two we’ll install this on the Raspberry Pi for now here some video explaining the possibilities with Jasper.
“There’s always a bigger fish.” — Qui-Gon Jinn
To combine command with PocketSphinx – https://github.com/slowrunner/Pi3RoadTest/blob/master/Sphinx%20On%20Pi3.txt
Following are Python scripts to control the BB8.
In Part Two
Write the script for the Raspberry, get Jasper up and running on an Raspberry PI
“I’m just a simple man trying to make my way in the universe.” — Jango Fett