As I’ve been preparing for my residency at DPI I’ve been investigating alternative ways to produce sonic drawings. Several weeks ago, I saw (again) demonstration videos from Johnny Chung Lee’s Wiimote Projects and began thinking about whether my rope&pulley system was the ideal input for the audio drawings I’m going to be working on. I envision drawing on a surface with “traditional” drawing tools (graphite sticks, maybe chalk, or markers) and mapping those large drawing gestures in realtime to audio in order to create sonic marks. The infrared LED pens desribed by Lee could be used as a starting point for a sleeve which I could put around my drawing tools. Although this departs from the rugged physicality of the rope&pulley, I think it will make the drawing process more intuitive.
Over the past two weeks, I’ve been experimenting with a Wiimote. While it was fairly straightforward to get the Wiimote connected to my PC and run some of the demo programs, I’m going to be writing my software in Max/MSP/Jitter, so I’ve been trying to figure out how to transform the coordinates I get back from the camera in the Wiimote into coordinates on a rectangular screen. I don’t think I’ll be projecting any visuals from my computer, but I want to resolve the pen position onto a regular rectangle so I can create a mapping to the sound generation component of my software.
Earlier this morning, I looked back at the Wiimote projects page and found that one of his example programs contained source code — and provided I can follow it, I think I can take what I’ve been learning about matrix operations in Jitter to implement it.
- Johnny Chung Lee Wiimote Projects
- Smarter Presentations: Exploiting Homography in Camera-Projector Systems (academic paper)
- Projective Transform (google search)
- Wii IR (search through Cycling ’74 archives for examples of porting Johnny Lee’s Wiimote whiteboard to Max/MSP/Jitter)
- warpPerspective code: example from OpenCV library used in OpenFrameworks. A little difficult to follow, though.
- Keystoning Forum Discussion (cycling74): example in Jitter demonstrating a method for using textured shaders to apply drastic keystoning to movies.
- Johnny Chung Lee’s Wiimote Whiteboard source code: Contains C# calibration and warping code, though no explanation of how it works.