Microbe Orchestra

IxD

Computer Vision

Machine Learning

Discursive Design

What if microbial activity can be mapped into music?

I will be designing a web application that allows people to observe microbial activity with just their smartphone camera and a plastic lens attachment. The libraries I'm currently using are:

  • tone.js: Music and Synth Library
  • darknet: Computer Vision Neural Network (users train app to identify specific microbe species)
  • tracking.js: Motion and Color CV Tracking Library

More documentation coming soon.

Proof of concept (midterm): 
Using openFrameworks to demonstrate functionality.

Recording of micro-organism activity in order to test ofxCv's blob detection and contour-tracking libraries. The goal is to store up to 10 cells (micro-organisms) from the field-of-view into a vector. Then, attach a sound sample to each cell. You can see the basic structure of how cells are tracked and stored on the screenshot at right.

Contour-tracking of micro-organisms from a river-water culture (OFxCV)

Collaborating with RISD Nature Lab for organism observation: Observing baby and adult sea monkeys (brine shrimp)

Working with the RISD Nature Lab, I'm using the micro-organisms of various aquacultures to observe for this project.