Gesture Based TV Remote Control


 

Seeing my children mastering swiping, touching and tapping on the tablet, gave me the idea of making a gesture based TV remote control. Using the Raspberry Pi, the Skywriter HAT from Pimoroni and an IR LED, this was made possible rather easily. Here’s how I did it …

Raspberry Pi

Preparing the Raspberry Pi is the same as always. I flashed the latest Raspbian Jessie image onto a 8GB microSd card using “dd”.

For this project, I’m using the Raspberry Pi A+ as it has the exact same form factor as the Skywriter HAT.

Skywriter HAT

The Skywriter HAT is an add-on board by Pimoroni, capable of detecting gestures. The HAT fits on top of the Raspberry Pi via the 40-pin GPIO header, and the software can be installed with a single command. The command takes care of installing the software dependencies, enabling I2C and providing some example applications. The documentation can be found on the Pimoroni GitHub page: https://github.com/pimoroni/skywriter-hat
After connecting the HAT, I executed the oneliner to install the software:

And that’s all there is to it. Compliments to Pimoroni for making this install so easy!

After setting up the software, the Skywriter HAT can be tested. One of the example applications is called “test.py” and reports the detected gestures on screen. To run the example, execute following command:

Lirc

Lirc is the software in charge of the infrared (IR) signals. It is capable of recording signals from remote, as well as send them. This is why I have foreseen both an IR LED transmitter and and IR receiver.

Circuit

The circuit consists of two parts:

  • transmitter
  • receiver

For the transmitter part, an IR LED is controlled via a transistor and a GPIO pin. A resistor of about 50 ohms is put in series with the IR LED and the GPIO pin is connected to the base pin of the transistor using a 10k ohm resistor. The LED is powered using the Pi’s 3.3V GPIO pin.

For the receiver part, also powered at 3.3V, the data pin is connected to a GPIO pin via a 10k ohm resistor. Careful, depending on your type of IR receiver, the pinout could be different!

Screen Shot 2016-02-12 at 21.09.33IMG_0922

After prototyping the circuit and before soldering it onto a prototyping HAT, I verified basic functionality using an oscilloscope.

IMG_0920 IMG_0924

Software

First step is of course to install the lirc application itself. This can be done using “apt-get”.

Edit the module files and append following lines:

The default hardware config file won’t work. Here’s an example of one that will. I commented out the default values so you can see which ones changed.

Finally, one last change is required in the boot config.

Don’t forget to reboot to apply all the changes. After that, you should have a “lirc0” device available.

It is also possible to verify the correct modules (lirc_rpi & lirc_dev) have been loaded using the “lsmod” command.

If that’s the case, everything should be ready to use.

Record

When recording keys, the correct names need to be assigned to identify them. The full list of possible values can be obtained using the following command:

In order to start recording IR signals from the remotes, the lirc process needs to be stopped.

With lirc stopped, the recording process can start. Make sure you have the remote of the TV or decoder handy and follow the instructions.

The above was created using the remote control of the TV. But as we also have a decoder for digital TV, I repeated the process for the other remote. Except this time, I ned the output file “lirc-remote2.conf”. Both config files were then combined  (“lirc-remote1.conf” and “lirc-remote2.conf”) into a single file (“lirc-combined.conf”).

In the config file, it is also possible to giev the ramotes a proper name. I called mine “TV” and “Digibox”.

The finished file can then be copied to the proper location for the lirc application to use.

When all recording is done, the lirc process can be restarted.

Send

Before sensing any commands, it’s good to check what is available matches what was recorded. This can be achieved using the “irsend LIST” command, followed by the remote name and empty quotes.

Once the commands have been confirmed, it’s possible to test out the signals using the “irsend SEND_ONCE” command.

In my case, the TV and Digibox turned ON. Repeating the same commands turned them OFF.

Combined

Combining the Skywriter HAT’s functinality with Lirc is simply a matter fo taking the skywriter example code and expanding/adapting the functions with the desired IR send commands.

The code above does the following:

  • flick left and right to change the channel
  • flick up and down to change the volume
  • make a circular motion to turn TV and Digibox ON and OFF

Enclosure

For the enclosure, I milled two parts with the necessary slots for power, USB and the IR LEDs using the CNC. The two pieces are joined using four screws.

The enclosure’s finish is simple: a bit of sanding and oiling.

IMG_0934 IMG_0952
Let me know what you think in the comments! And don’t forget I’m giving away one of my Pi Zero to celebrate my first 1000 subscribers on Youtube and the fact that I’m going to exhibit at Maker Faire Paris: http://frederickvandenbosch.be/?p=1563

54 thoughts on “Gesture Based TV Remote Control”

  1. This is wonderful,tell me,how much would you charge me to make,let’s say,100 of these devises for re-sale?

      1. sure. But if someone would be interested in throwing money at you to get one, how much money would one need to throw at you? cos I am insanely interested in getting one.

    1. You could control any device via IR signals, as long as you can record them first, as explained in the post.

      With some modifications, you could use another mechanism than IR for control.

  2. Hi! Amazing project, I’m very interested to try it out. Quick question, where did you get all the IR related components? It’s completely DIY rather than a pre set module, right? Could you maybe provide me with some links? Much appreciated!

  3. Finally an unique way to control an RPi used for music, etc. in a car! No need for keypad or a touch screen, just wiggle your finger at the dash. Brilliant!!!

  4. Are you using Raspberry pi 3 or RPi2 because I am trying on the pi 3 and I am not getting any response when pressing buttons on a remote after entering the command. irrecord -d /dev/lirc0 lirc-remote1.conf. Would this be a software issue or do you think the hardware setup was incorrect

    1. Hi Jason,

      actually, I’m using a Raspberry Pi A+, though I don’t think that should matter.
      Verify your infrared receiver is properly wired, and if you have a scope, verify if there is an incoming signal on its data pin.

  5. Most impressive. Thank you for taking the time to be so detailed and thorough.
    Are you going to also post the steps and coding that made the Theremin music instrument? I’m most interested in seeing real time musical instrument Pi with an analog input device.
    Thanks again.

  6. Hi! congrat for the project!
    Can you describe the components what did you used? like the transistor type, Infrared kind etc. ? 🙂 I want to build it, but I don’t know what components need to buy ?

    1. Hi Szabi, the components used were provided as part of those basic electronics kits with buttons, LEDs, etc … so I don’t have an exact model number.

      I suspect if you make use of the most commonly available components, it should work out for you as well.

  7. Hello, Thank you very much for sharing your science.
    My final year project is about using Hybrid of gesture and voice commands to control TV, I been looking for ways to come up about it, Seeing this made me realise i can do it .. Pls can i have your email address, so i can send you a mail if i have any difficulty achieving my project using your science

    Thank you.

  8. One question is the control conected to the tv or what should i have on the tv, for having a good performance and also is it possible if i used IR with two pins instead of three as the image shows?

    1. Hi Josue,

      there is nothing connected to the TV, it just acts the same way the regular remote control does. The IR transmitter has two pins, the receiver has three. The receiver is used to record the signals from the original remote.

  9. Hello Frederick,

    Please can you help me with the circuit diagram for your circuit (receiver and transmitter) , am having difficulty designing mine

    Thank you

  10. hi i’m trying to repeat this project but im having troubles with the programming how do you program the tv i mean how the signal is sended or every tv has his own especifications?

    1. Hi Josue,

      you have to record the signals from the original remote on the Pi, with the “irrecord” command.
      The presses on the remote should be detected by the Pi, and you can then tell the Pi wich button that is. This info is then saved in a configuration file which you can use to play out the same signals again with the Pi.

      Check out the “Record” paragraph.

  11. do you have a video of how you configured the signals or how you use your tv control, or eve how do you program the tv or decoder for the signals in the lirc, i’m having troubles with that situation

  12. which pins yo use to solder in the raspberry pi and also you solder some extra pins in the raspberry pi for the power supply of the circuit

  13. do you know why the rapsberry shows the message that has failed to start load kernel modules when i try to edit the file of modules and also add the gpio 22 and 23 ? sorry for ask so much im trying to do your project for school purposes and also learning purposes

      1. systemd-modules-load.service – Load Kernel Modules
        Loaded: loaded (/usr/lib/systemd/system/systemd-modules-load.service; static)
        Active: failed (Result: exit-code) since Sat, 2017-04-27 12:29:18 CET; 2h 14min ago
        Docs: man:systemd-modules-load.service(8)
        man:modules-load.d(5)
        Process: 134 ExecStart=/usr/lib/systemd/systemd-modules-load (code=exited, status=1/FAILURE)
        CGroup: name=systemd:/system/systemd-modules-load.service

        also when i turn on the raspberry appears this message:
        Failed to start Load Kernel Modules.
        See ‘systemctl status systemd-modules-load.service’ for details.
        6.080707] systemd[1]: failed to start Load Kernel Modules.
        6.097303] systemd[1]: Unit systemd-modules-load.service entered failed state.

  14. Hi, great Project!!!
    My question is, in which distance does the module work? Can I sit on my sofa three meter away from the skywriter?

    Greetings
    Thomas

    1. Hi Thomas,

      the gestures need to be within centimeters of the skywriter HAT. The IR signals it generates do reach a couple of meters to the TV for example, just like a normal remote control.

Leave a Reply

Your email address will not be published. Required fields are marked *