While at Fab Academy, I began to think about ways to universally control physical objects and how to create a universal remote that could fluidly control objects, without a mass of unintuitive buttons to press.
Anime Puppet Masters
When thinking about how a new interface should feel, I often use artistic, literary or conceptual mental models as a way of translating electronic devices into extensions of the human body. This puppetmaster project grew out of a desire to be able to "pull the strings" using a gestural interface instead of pushing buttons.
The artistic inspiration that pushed me in the direction of using puppets and pulling strings to control objects as a mental model for physical interaction was Naruto (an anime / manga by Masashi Kishimoto - popular with the "young adult" demographic in both Japan and the US.) I won't get into the full story of these ninja clans with magical powers here. However, the concept that inspired this system lies in the relationship between the use of handsigns in combination their flow of chakra to enable martial arts technique.
Jutsu / Handsigns
In this fictional world, the ninja use jutsu by manipulating a form of energy known as chakra, or a combination of the mental physical and energies within the body.
These Jutsu are fall into three categories:
- genjutsu, illusion creating techniques;
- taijutsu, physical attacks that only the body's physical energy;
- ninjutsu, general techniques.
- Kekkei genkai, inherited traits
Series creator Masashi Kishimoto incorporated and expanded concept of jutsu to explain the supernatural feats ninja perform in the Naruto series. I found these designations very useful for the development of universal remote control. In terms of actual use of control systems, these metaphors may become a conceptual foundation for controlling and modifying physical objects. In addition, they are helpful starting points for both simple and complex gestural languages as well as a model for customization.
example hand signs - based on signs of the zodiac.
The series uses handsigns as a way to call a technique. In terms of an actual puppetmaster system and a gestural language, I see the handsigns (gestures) as a way of calling programs or systems(jutsu or the technique itself).
Instead of using the category types above - we could apply them to modern user needs.
|Jutsu Type||Technological Equivalent|
|genjutsu, illusion creating techniques;||
|calls entertainment / medial related functions|
|taijutsu, physical attacks that only the body's physical energy;||
|affects the physical world directly, moves objects, etc.|
|ninjutsu, general techniques.||
|common tasks that are preformed by most people using the system (simple gestures)|
|Kekkei genkai, inherited traits||
|these could be highly customized tasks that power users create for their own use|
The concepts detailed above are not fully executed in my
design, but this gestural language is something that I will be pursuing
far beyond the scope of Fab Academy. I intend to study ASL or American
sign language this fall and I hope to gain a better understanding of
gestural languages in daily use within an entire population (the deaf).
My final project for Fab Academy is a universal remote I am calling Puppet Master. The goal of this project is to create a gestural remote control device that uses infrared to communicate with a device.
Overall Project Goals / Design Considerations:
- Create a remote control that takes input from finger movements / gestures
- Remote is small and wearable (will likely look like a bracelet or a watch) with connections to the fingertips.
- Remote is comfortable enough for long-term wear, will help to relieve hand / joint fatigue
Initial Form Factor Mockup
First Steps to Board 1 - Modify Step Response Board
My first step in building this project was to start with the basics. I milled, stuffed and programmed the hello step response board (modified to use the FTDI header by Shawn Wallace). I have a Mac and I can't troubleshoot / develop at home with the serial header examples.
The hello step response has only one input - so I design my own board with the following modifications:
- 3 additional inputs for each of the four fingers.
- swapped out the attiny 45 (it does not have enough input / charging pins for 4 inputs)
- swapped in the attiny44.
- modified the code to run on the attiny44.
Code modification attiny 45 to attiny 44
- Attiny 45 FTDI Hello Step Response Code
- Attiny 44 FTDI Hello Step Response Code
--> in use on the Puppet Master board
Puppet Master Board - First Attempt
Learning From My Mistakes
There was a issue with this initial board and I could not get
the modified code for the attiny 44 to work with the Hello.Step.45.py
code. I attempted triage, but the board remained flaky. There seemed to
be an issue between the ground pin on the chip and power.
In addition, I realized that I routed the Tx to Rx wrong on the FTDI header pins. I was using jumpers to work around it initially, but after the version 1 board's persistent flakiness, I decided to make a version 2 board.
I am not including the schematics & Eagle files for version 1 because of these issues.
The version 2 board (see below) with the correctly mapped FTDI header and some slightly shifted components to allow for more space between some problematic components. This board version has been flashed with the attiny 44 code and works with Hello.Step.45.py code.
Download the Eagle files:
PuppetMaster v.2.0 Board
PuppetMaster v.2.0 Schematic
I ran out of time in the lab and did not get a chance to mill out the board. Instead, I scored the permiter with a utility knife and broke off the excess edge. Then I used a rasp to file the edges down.
I wanted to try to make a resistive sensor that would read finger beds. I tried a few designs, but each failed miserably. I did get some readings of of flexing a copper sheet - but after getting Neil's guidance on the subject - I decided to use a capacitive sensor instead. (See Board 2 update below)
The following pictures in this section are of board v.2 not working very well with the addition of a glove to attach the board and sensors to.
Board 2 UPDATE
Changed to capacitive sensor. Used copper on thumb and finger
tip and grounded the thumb (in accordance with Neil's comments during
the project triage.
The change in output is now in a more approprate range - see
Added a ground pin for the thumb to the board.
Download the Eagle files:
I had hoped to mill Puppet Master board v.3 and connect it to an infrared transmitter and receiver so I could sent a control signal to something. Unfortunately I am having alot of trouble getting the Modela to mill boards without chewing them to pieces.
Infrared Boards - Receiver / Transmitter
I began with David Mellis' infrared networking boards and modified them to work with a attiny84 (we are all out of the 45 and 44's in the lab and all the suppliers were backordered).
I did get one version of a set of infrared boards milled and stuffed (after many hours of fighting with the modela). I was able to flash program the transmitter board, but receiver board had some problems and the usbtiny couldn't connect to the microcontroller.
Shawn Wallace looked it over and noticed that Mellis had used a jumper wire to connect a trace that was left out of the schematic. I was putting the jumper in - when the headers came off the board.
Here are the pictures I took before the header came off (receiver on the left - transmitter on the right):
Although I did not get as far as I intended with the current Puppet Master prototype, future versions will likely include other features and multiple technologies capable of controlling multiple device types - making it a true universal remote. The board and power supply will be worn around the wrist like a watch or bracelet. Making weight and a small form factor a major design consideration for future versions of this project)
In future versions - the user of PuppetMaster will be able to input information via gestures and remote control objects by using sensors. One way to do this is to use sensors embedded in the fingertips, most likely using 6-axis sensors (3 axis gyro, 3 axis accelerometer) to the device. Another way is a Kinnect or SixthSense style camera setup.
I intend to study signed languages (such as ASL or American Sign Language) and incorporate communication styles from previous signed languges into machine control gestural language. See the Puppet Master Gestural Language Proposal page.