Project Overview
This project required the integration of at least one continuous input sensor with an interactive game as well as requiring the gamer to "tag" in using an NFC card reader. Our team, Ross Yeager and Ryan Rho, used 1 accelerometer, 1 sport glove, 2 Arduinos, and the Adafruit NFC card breakout board to implement the old arcade game, Breakout, as well as a plotting utility to graph the accelerometer data and visualize gesture recognitions. For our continuous sensing, the accelerometer was used (it is embedded into a glove worn by the player):
-If the player rolls his hand to the left or right, it corresponds to continuous movement left or right in the game for as long as it remains rolled. -A quick upward flick in the Z-axis direction corresponds to a discrete "power" in the game (extending the bar at bottom momentarily).
Image: Device
Images: How to Use the Controller
Utility Tool
Utility Visualizer (X,Y,Z Axes)
The utility was created to visualize the accelerometer data and to show when the selected gestures occurred. The first three plots are the X,Y,and Z axes respectively. The utility was done using the Processing IDE, and the gesture recognition was based on a threshold state machine. Thresholds were determined experimentally, and future improvements of the utility would include user ability to control/adjust thresholds. The right left FSM consisted of the following states:
-NOLR: This state no left or right motion occurs
-LEFT: To get in this state, the threshold must have been crossed downwards and it must remain below the threshold to stay in this state. This state continuously outputs a valid "left" signal and this is visualized in the utility.
-RIGHT: To get in this state, the threshold must have been crossed upwards and it must remain above the threshold to stay in this state. This state continuously outputs a valid "right" signal and this is visualized in the utility.
-LLEFT: Leaves the "left" state. Has an upward crossing of the threshold. Indicates end of left hand rotation gesture.
-LRIGHT: Leaves the "right" state. Has an downward crossing of the threshold. Indicates end of left hand rotation gesture.
The flick motion is detected by looking for an upward crossing of the z-axis threshold. Since it is discrete it only occurs once per gesture (as opposed to left right signals which the user can hold).
Finally a summation plot of all of the axes is at the very bottom of the utility.
Gesture recognition was represented visually in the utility by a shade of magenta/purple.
Utility Video
ARDUINO
Arduino 1: Accelerometer
For the accelerometer, the baud rate is set to 9600 and the accelerometer was wired up to the first 3 analog pins. A2D readings are taken ever loop and sent serially to the host computer in the following format:
Utility Visualizer (X,Y,Z Axes)
The utility was created to visualize the accelerometer data and to show when the selected gestures occurred. The first three plots are the X,Y,and Z axes respectively. The utility was done using the Processing IDE, and the gesture recognition was based on a threshold state machine. Thresholds were determined experimentally, and future improvements of the utility would include user ability to control/adjust thresholds. The right left FSM consisted of the following states:
-NOLR: This state no left or right motion occurs
-LEFT: To get in this state, the threshold must have been crossed downwards and it must remain below the threshold to stay in this state. This state continuously outputs a valid "left" signal and this is visualized in the utility.
-RIGHT: To get in this state, the threshold must have been crossed upwards and it must remain above the threshold to stay in this state. This state continuously outputs a valid "right" signal and this is visualized in the utility.
-LLEFT: Leaves the "left" state. Has an upward crossing of the threshold. Indicates end of left hand rotation gesture.
-LRIGHT: Leaves the "right" state. Has an downward crossing of the threshold. Indicates end of left hand rotation gesture.
The flick motion is detected by looking for an upward crossing of the z-axis threshold. Since it is discrete it only occurs once per gesture (as opposed to left right signals which the user can hold).
Finally a summation plot of all of the axes is at the very bottom of the utility.
Gesture recognition was represented visually in the utility by a shade of magenta/purple.
Utility Video
ARDUINO
Arduino 1: Accelerometer
For the accelerometer, the baud rate is set to 9600 and the accelerometer was wired up to the first 3 analog pins. A2D readings are taken ever loop and sent serially to the host computer in the following format:
"###|###|###\n"
where the numbers are the X-axis, second Y-axis, and Z-axis readings.
Arduino 2: NFC ID
For the second Arduino, the NFC ID breakout board is connected to it using the built in SPI input pins. The baud rate is 9600 and the nfc object is initiated. Using the Adafruit library, when a user swipes his ID the tag is identified and the total score and high score are sent serially as a string to the host computer. If the game is over and the host computer wants to send a new high score to write onto the card, the serial interrupt occurs and the Arduino writes the incoming data to the card instead of reading from it.
We used the classic arcade game, Breakout, for our project game. We used Pygame, a game library for Python. As the basis, we modified a sample game source code in Pygame to suit our project needs and to integrate the sensor and RFID applications into the game. The left and right rotation gestures correspond to left and right movement of the bar, and the upward flick extends the bar to cover more ground momentarily. Points are scored from destroying all of the bricks in the game, and the game is over when the player misses the ball too many times.
where the numbers are the X-axis, second Y-axis, and Z-axis readings.
Arduino 2: NFC ID
For the second Arduino, the NFC ID breakout board is connected to it using the built in SPI input pins. The baud rate is 9600 and the nfc object is initiated. Using the Adafruit library, when a user swipes his ID the tag is identified and the total score and high score are sent serially as a string to the host computer. If the game is over and the host computer wants to send a new high score to write onto the card, the serial interrupt occurs and the Arduino writes the incoming data to the card instead of reading from it.
Software
Game: BREAKOUTWe used the classic arcade game, Breakout, for our project game. We used Pygame, a game library for Python. As the basis, we modified a sample game source code in Pygame to suit our project needs and to integrate the sensor and RFID applications into the game. The left and right rotation gestures correspond to left and right movement of the bar, and the upward flick extends the bar to cover more ground momentarily. Points are scored from destroying all of the bricks in the game, and the game is over when the player misses the ball too many times.
No comments:
Post a Comment