MUSIC VISUALIZATION IN iTunes – PART II
This second instalment about music visualization using iTunes displays the same information – the frequency spectrum of a music being played, but in a different format. In Part I, the frequencies obtained from the FFT analysis were plotted for both the left and right channels separately in the form of bar graphs. As the frequencies changed, the bar graph got updated. In this second part, the audible frequencies in the 20Hz to 20kHz range are visualized as a function of time. As time progresses, along the horizontal axis, each column shows the distribution of prevalent frequencies at that time. The colour of each pixel corresponds to the prevalence of that frequency. Cold colours, such as blue, show a lack of that frequency, while hot, as in red, colours denote a high value. These colours, similar to Part I, were obtained using a linear interpolation from blue to green to red within the RGB colour cube.
The first, and looking back, very naive implementation drew the coloured columns as time progressed. When the last column, on the far right side of the window got completed, the process repeated itself again starting from the left, erasing whatever was on the screen. So for every screen’s worth, farther along to the right we were, the more columns had to be drawn. This resulted in a serious and noticeable impact in updating the visualization. A much better alternative was to create a blank NSImage and keep displaying it at every update. As time progressed, this NSImage was updated, so at any given time, the cost of updating/displaying remained the same and the slowdown was resolved. The colour corresponding to the frequency is obtained by averaging the left and right channel’s values for a given time.
Without further delay, let’s see the source code. As before, the drawRect function selects which visualization to display.
// image to hold the spectrum as time progresses
NSImage* spectrumImage;
// current time step to draw
int currentStepToDraw=0;
void DrawVisualSpectrumTime(VisualPluginData *visualPluginData)
{
CGRect drawRect;
CGPointstartPoint;
CGPointbarExtent;
CGPoint drawAreaExtent;
// this shouldn't happen but let's be safe
if (visualPluginData->destView==NULL) {
return;
}
// establish the extend of the drawing area
drawRect=[visualPluginData->destView bounds];
drawAreaExtent.x=drawRect.size.width;
drawAreaExtent.y=drawRect.size.height;
// fill background
[[NSColor blackColor] set];
NSRectFill(drawRect);
// draw gray border
CGRectborderRect;
borderRect.origin.x=5.0;
borderRect.origin.y=5.0;
borderRect.size.width=drawAreaExtent.x-10.0;
borderRect.size.height=drawAreaExtent.y-10.0;
[[NSColor grayColor] setStroke];
DrawRoundedRect(borderRect,5.0,5.0,3.0);
// setup parameters
int numSpectrumChannelsToShow=256;
int barSegmentHeight=5.0;
// lock the focus for the image, so we can update it
[spectrumImage lockFocus];
for (int i=0;i<numSpectrumChannelsToShow;++i) {
startPoint.x=10.0;
startPoint.y=10.0;
barExtent.x=barSegmentHeight;
barExtent.y=(drawAreaExtent.y-startPoint.y*2.0)/
numSpectrumChannelsToShow;
if (barExtent.y<1.0) {
barExtent.y=1.0;
}
startPoint.x=startPoint.x+currentStepToDraw*barSegmentHeight;
startPoint.y+=i*(barExtent.y); CGFloatred; CGFloatgreen; CGFloatblue;
// in this case we take the average of the left and right channel //of course we could take the left or right only
float inputValue=(visualPluginData->renderData.spectrumData[0][i]
+visualPluginData->renderData.spectrumData[1][i])*0.5;
// interpolate the RGB cube from blue to red
GetColorColdHot(inputValue,0.0,255.0,&red,&green,&blue);
[[NSColor colorWithDeviceRed:red green:green blue:blue alpha:1] set];
drawRect=NSMakeRect(startPoint.x,startPoint.y,
barSegmentHeight,barExtent.y);
// draw the column as a rectangle for this time instant
DrawRectFilled(drawRect);
}
// unlock the image after we are done updating it
[spectrumImage unlockFocus];
++currentStepToDraw;
// if we are getting too close to the right boundary,
// reset and start at the left again
if ((startPoint.x+currentStepToDraw*barSegmentHeight)>
(2.0*borderRect.size.width-60.0)) {
currentStepToDraw=0;
}
CGPointstartPointSpectrum;
startPointSpectrum.x=10.0;
startPointSpectrum.y=10.0;
// display the image, which hold the accumulated frequency display for
// all the time steps so far displayable on the screen
[spectrumImage drawAtPoint:startPointSpectrum fromRect:NSZeroRect
operation:NSCompositeSourceOver
fraction:1.0];
}
Other miscellaneous functions utilized in the above function:
// This function was obtained from
// http://paulbourke.net/texture_colour/colourramp/
// there are many many interesting algorithms on those webpages
void GetColorColdHot(CGFloat d,CGFloat dmin,CGFloat dmax, CGFloat *r, CGFloat *g, CGFloat *b)
{
CGFloat ds;
if (d<dmin) {
d=dmin;
}
if (d>dmax) {
d=dmax;
}
ds=dmax-dmin;
if (d<(dmin+0.25*ds)) {
*r=0.0;
*g=4.0*(d-dmin)/ds;
*b=1.0;
} else if (d<(dmin+0.5*ds)) {
*r=0.0;
*g=1.0;
*b=1.0+4.0*(dmin+0.25*ds-d)/ds;
} else if (d<(dmin+0.75*ds)) {
*r=4.0*(d-dmin-0.5*ds)/ds;
*g=1.0;
*b=0.0;
} else {
*g=1.0+4.0*(dmin+0.75*ds-d)/ds;
*g=1.0;
*b=0.0;
}
}
Note that these functions are driven by the following, already implemented function:
-(void)drawRect:(NSRect)dirtyRect
{
if ( _visualPluginData != NULL ) {
DrawVisualSpectrumTime(_visualPluginData);
}
}
The image to draw into is set up in the following function:
OSStatus ActivateVisual( VisualPluginData * visualPluginData, VISUAL_PLATFORM_VIEW destView, OptionBits options )
{
....
CGRect drawRect;
CGPoint drawAreaExtent;
drawRect=[visualPluginData->destView bounds];
drawAreaExtent.x=drawRect.size.width-20.0;
drawAreaExtent.y=drawRect.size.height-20.0;
spectrumImage=[[NSImage alloc] initWithSize:
NSMakeSize(drawAreaExtent.x,drawAreaExtent.y)];
....
return status;
}
Note: Apple, iTunes and other hardware/software is trademark of their respective owners.