{"id":391,"date":"2013-11-24T23:52:25","date_gmt":"2013-11-24T23:52:25","guid":{"rendered":"http:\/\/localhost:8888\/?page_id=391"},"modified":"2022-07-06T18:40:28","modified_gmt":"2022-07-06T18:40:28","slug":"music-visualization-part-ii","status":"publish","type":"page","link":"https:\/\/www.amzsaki.com\/?page_id=391","title":{"rendered":"Music visualization – Part II."},"content":{"rendered":"

MUSIC VISUALIZATION IN\u00a0iTunes\u00a0– PART II<\/strong><\/p>\n

\"iTunesViz02large-filtered\"<\/p>\n

This second instalment about music visualization using iTunes displays the same information – the frequency spectrum of a music being played, but in a different format. In Part I, the frequencies obtained from the FFT analysis were plotted for both the left and right channels separately in the form of bar graphs. As the frequencies changed, the bar graph got updated. In this second part, the audible frequencies in the 20Hz to 20kHz range are visualized as a function of time. As time progresses, along the horizontal axis, each column shows the distribution of prevalent frequencies at that time. The colour of each pixel corresponds to the prevalence of that frequency. Cold colours, such as blue, show a lack of that frequency, while hot, as in red, colours denote a high value. These colours, similar to Part I, were obtained using a linear interpolation\u00a0 from blue to green to red within the RGB colour cube.<\/p>\n

The first, and looking back, very naive implementation drew the coloured columns as time progressed. When the last column, on the far right side of the window got completed, the process repeated itself again starting from the left, erasing whatever was on the screen. So for every screen\u2019s worth, farther along to the right we were, the more columns had to be drawn. This resulted in a serious and noticeable impact in updating the visualization. A much better alternative was to create a blank NSImage and keep displaying it at every update. As time progressed, this NSImage was updated, so at any given time, the cost of updating\/displaying remained the same and the slowdown was resolved. The colour corresponding to the frequency is obtained by averaging the left and right channel\u2019s values for a given time.<\/p>\n

Without further delay, let\u2019s see the source code. As before, the drawRect function selects which visualization to display.<\/p>\n

\/\/ image to hold the spectrum as time progresses\nNSImage* spectrumImage;\n\n\/\/ current time step to draw\nint currentStepToDraw=0;\n\nvoid DrawVisualSpectrumTime(VisualPluginData *visualPluginData)\n{\n\n  CGRect drawRect;\n  CGPointstartPoint;\n  CGPointbarExtent;\n  CGPoint drawAreaExtent;\n\n  \/\/ this shouldn't happen but let's be safe\n  if (visualPluginData->destView==NULL) {\n    return;\n  }\n\n  \/\/ establish the extend of the drawing area\n  drawRect=[visualPluginData->destView bounds];\n  drawAreaExtent.x=drawRect.size.width;\n  drawAreaExtent.y=drawRect.size.height;\n\n  \/\/ fill background\n  [[NSColor blackColor] set];\n  NSRectFill(drawRect);\n\n  \/\/ draw gray border\n  CGRectborderRect;\n  borderRect.origin.x=5.0;\n  borderRect.origin.y=5.0;\n  borderRect.size.width=drawAreaExtent.x-10.0;\n  borderRect.size.height=drawAreaExtent.y-10.0;\n\n  [[NSColor grayColor] setStroke];\n  DrawRoundedRect(borderRect,5.0,5.0,3.0);\n\n  \/\/ setup parameters\n  int numSpectrumChannelsToShow=256;\n  int barSegmentHeight=5.0;\n\n  \/\/ lock the focus for the image, so we can update it\n  [spectrumImage lockFocus];\n\n  for (int i=0;i<numSpectrumChannelsToShow;++i) {\n    startPoint.x=10.0;\n    startPoint.y=10.0;\n\n    barExtent.x=barSegmentHeight;\n    barExtent.y=(drawAreaExtent.y-startPoint.y*2.0)\/\n                                             numSpectrumChannelsToShow;\n\n    if (barExtent.y<1.0) { \n      barExtent.y=1.0; \n    } \n    startPoint.x=startPoint.x+currentStepToDraw*barSegmentHeight; \n    startPoint.y+=i*(barExtent.y); CGFloatred; CGFloatgreen; CGFloatblue;                \n    \/\/ in this case we take the average of the left and right channel                 \/\/of course we could take the left or right only \n    float inputValue=(visualPluginData->renderData.spectrumData[0][i]\n                          +visualPluginData->renderData.spectrumData[1][i])*0.5;\n\n    \/\/ interpolate the RGB cube from blue to red\n    GetColorColdHot(inputValue,0.0,255.0,&red,&green,&blue);\n\n    [[NSColor colorWithDeviceRed:red green:green blue:blue alpha:1] set];\n\n    drawRect=NSMakeRect(startPoint.x,startPoint.y,\n                                            barSegmentHeight,barExtent.y);\n\n    \/\/ draw the column as a rectangle for this time instant\n    DrawRectFilled(drawRect);\n  }\n\n  \/\/ unlock the image after we are done updating it\n  [spectrumImage unlockFocus];\n\n  ++currentStepToDraw;\n\n  \/\/ if we are getting too close to the right boundary, \n  \/\/ reset and start at the left again\n  if ((startPoint.x+currentStepToDraw*barSegmentHeight)>\n         (2.0*borderRect.size.width-60.0)) {\n    currentStepToDraw=0;\n  }\n\n  CGPointstartPointSpectrum;\n\n  startPointSpectrum.x=10.0;\n  startPointSpectrum.y=10.0;\n\n  \/\/ display the image, which hold the accumulated frequency display for\n  \/\/ all the time steps so far displayable on the screen\n  [spectrumImage drawAtPoint:startPointSpectrum fromRect:NSZeroRect\n                     operation:NSCompositeSourceOver\n                     fraction:1.0];\n}<\/span><\/pre>\n
Other miscellaneous functions utilized in the above function:<\/p>\n
\/\/ This function was obtained from\n\/\/ http:\/\/paulbourke.net\/texture_colour\/colourramp\/\n\/\/ there are many many interesting algorithms on those webpages\n\nvoid GetColorColdHot(CGFloat d,CGFloat dmin,CGFloat dmax, CGFloat *r, CGFloat *g, CGFloat *b)\n{\n  CGFloat ds; \n\n  if (d<dmin) { \n    d=dmin; \n  } \n  if (d>dmax) {\n    d=dmax;\n  }\n\n  ds=dmax-dmin;\n\n  if (d<(dmin+0.25*ds)) {\n    *r=0.0;\n    *g=4.0*(d-dmin)\/ds;\n    *b=1.0;\n  } else if (d<(dmin+0.5*ds)) {\n    *r=0.0;\n    *g=1.0;\n    *b=1.0+4.0*(dmin+0.25*ds-d)\/ds;\n  } else if (d<(dmin+0.75*ds)) {\n    *r=4.0*(d-dmin-0.5*ds)\/ds;\n    *g=1.0;\n    *b=0.0;\n  } else {\n    *g=1.0+4.0*(dmin+0.75*ds-d)\/ds;\n    *g=1.0;\n    *b=0.0;\n  }\n}<\/span><\/pre>\n
Note that these functions are driven by the following, already implemented function:<\/p>\n
-(void)drawRect:(NSRect)dirtyRect\n{\n  if ( _visualPluginData != NULL ) {\n    DrawVisualSpectrumTime(_visualPluginData);\n  }\n}<\/span><\/pre>\n
The image to draw into is set up in the following function:<\/p>\n
OSStatus ActivateVisual( VisualPluginData * visualPluginData, VISUAL_PLATFORM_VIEW destView, OptionBits options )\n{\n\n  ....\n\n  CGRect drawRect;\n  CGPoint drawAreaExtent;\n\n  drawRect=[visualPluginData->destView bounds];\n  drawAreaExtent.x=drawRect.size.width-20.0;\n  drawAreaExtent.y=drawRect.size.height-20.0;\n\n  spectrumImage=[[NSImage alloc] initWithSize:\n                                 NSMakeSize(drawAreaExtent.x,drawAreaExtent.y)];\n\n  ....\n\n  return status;\n}<\/span><\/pre>\n
Note: Apple, iTunes and other hardware\/software is trademark of their respective owners.<\/p>\n","protected":false},"excerpt":{"rendered":"
MUSIC VISUALIZATION IN\u00a0iTunes\u00a0– PART II This second instalment about music visualization using iTunes displays the same information – the frequency<\/p>\n
Continue readingMusic visualization – Part II.<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":298,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-391","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/www.amzsaki.com\/index.php?rest_route=\/wp\/v2\/pages\/391","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.amzsaki.com\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.amzsaki.com\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.amzsaki.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.amzsaki.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=391"}],"version-history":[{"count":6,"href":"https:\/\/www.amzsaki.com\/index.php?rest_route=\/wp\/v2\/pages\/391\/revisions"}],"predecessor-version":[{"id":3539,"href":"https:\/\/www.amzsaki.com\/index.php?rest_route=\/wp\/v2\/pages\/391\/revisions\/3539"}],"up":[{"embeddable":true,"href":"https:\/\/www.amzsaki.com\/index.php?rest_route=\/wp\/v2\/pages\/298"}],"wp:attachment":[{"href":"https:\/\/www.amzsaki.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=391"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}