In 2018 I created a project, Lucid Doodle, that explored working with LED strips to create an interactive light display with a groovy aesthetic. Since then, I’ve switched to mostly web development, and this project seems like a great opportunity to return to my fabrication roots and evolve Lucid Doodle with machine learning.
I used Teachable Machine to develop an audio ML model that is trained to recognize a variety of keywords. I took about 20 audio samples for each keyword, which provided enough accuracy to correctly identify the keyword when spoken, most of the time. Here is a list of what the model currently understands:
I use p5.js to extract the ML model results from Teachable Machine and also to provide an interface to see data live, as shown on the canvas above. I forked a p5 script from here: Yining p5 Teachable Machine and used it as a boilerplate to build from. From here, the incoming data is processed and passed to Arduino via serial communication (p5.serialcontrol).