Ian (Yuan Gao) and I worked together to produce a sword fighting game with foam swords, digital sound effects, and a scoring mechanism for our PComp midterm. We had recently learned how to read the data from an accelerometer and how to serially communicate between the arduino and our laptops such that programming on the arduino can effect programming in p5.js and vica versa. We thought a sword fighting game would be a good application of these skills.
After consulting with Tom about our project he suggested we start out by visualizing the output of our accelerometers and using any patterns we noticed as a jumping off point for detecting sword motions like swings and hits. After a few attempts I put together a program in p5 that would graph the accelerometer data in a readable way, similar to what Tom had shown us. At the time we were hoping to use bluetooth communication so I set up bluetooth and attached the arduino to a battery as I tested my accelerometer graph.
Video:AccelerometerGraphing
As we studied (mostly played around) with the data output it became clear that acceleration isn’t always what it seems. Conceptually I understand acceleration as the rate of change of velocity. However, in practice I think I was expecting acceleration to act more like velocity or maybe “energy”. In addition, the constant acceleration due to gravity complicates the “movement” view of acceleration further. We looked into how to manage this data input and get it into a more usable state and we realized that in doing so we were entering the world of complicated math and physics lessons. One option in this regard was to calibrate the accelerometer to sense G force accurately. This would have made our measurements more accurate and legible but wouldn’t have necessarily helped us identify swings and hits. The other option we identified, and ultimately used, was to dive deeper into the meaning of acceleration and its derivatives. While we did research we discovered that a lot of people who work with accelerometers also work with the derivatives of the acceleration: Jerk and Jounce. Combining current accelerometer readings with the Jerk and Jounce allows you to determine points of inflection that could indicate the beginning of a swing, the end of a swing, a hit and other events. We ended up using a threshold value for jerk as the primary component of our swing detection, however it became very clear that it is possible to get accurate and distinct readings for a huge variety of motions depending on how much math and pattern recognition you are willing to do with acceleration, jerk and jounce. As a result, we spent a lot of time tweaking and experimenting with different values for each. In the end we included a lot of smoothing algorithms and this kind of threshold test for playing a sound:
function playbackChopSoundEffect(_jerk_vals, _jounce_vals, sample_ind) {
if ((abs(_jerk_vals.x) >= _jerk_vals.max_x * 0.25 ||
abs(_jerk_vals.y) >= _jerk_vals.max_y * 0.25 ||
abs(_jerk_vals.z) >= _jerk_vals.max_z * 0.25) &&
(abs(_jounce_vals.x) >= _jounce_vals.max_x * 0.33 ||
abs(_jounce_vals.y) >= _jounce_vals.max_y * 0.33 ||
abs(_jounce_vals.z) >= _jounce_vals.max_z * 0.33)) {
if (random(1) > 0.5) swoosh_sounds[sample_ind].play();
else swoosh_sounds[sample_ind + 2].play();
}
}
If we had more time for research we may have been able to rely completely on the accelerometer to detect all of the possible sword events however we settled on having the accelerometer control swing sounds and to use home made switches for hit detection (scoring) and to play a sound when two swords hit each other.
Attaching the accelerometer to the foam swords, building the p5.js sketch to detect and respond to the action and wiring this all up was tricky but took much less time than the calibration of the accelerometer. We wrapped each sword in tin foil and built companion targets (also wrapped in tin foil) to form switches when the opponents word makes contact with your sword or your target. The accelerometer was attached to a mini breadboard which we zip tied to each sword. Unfortunately we didn’t have time to make this wireless so each sword had a long trail of wires connecting them to the arduino. The sketch incorporated some fencing images and sounds that we found online (citation coming) as well as the score count for each player.
I was happy with how this project turned out. Some problems were the lack of wireless which would have made the game more fun due to the increased freedom of movement and the relatively fragile nature of the swords and electronics we fabricated. I’m confident that with more time, resources, and guidance we could make this game fully wireless, self contained (wouldn’t require a PC connection for the sounds and scoring), and more durable.
Here’s a video of it in action: