User Tools

Site Tools


gsoc2008_waveform_rewrite

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

gsoc2008_waveform_rewrite [2008/04/23 05:39]
wesley created
gsoc2008_waveform_rewrite [2008/07/01 22:17]
rryan
Line 33: Line 33:
 2008. Checkpoints and measurable progress indicators will be used to 2008. Checkpoints and measurable progress indicators will be used to
 ensure development is on schedule. ensure development is on schedule.
 +
 +
 +==== General Design Overview ====
 +
 +(warning: i'm dumping some free-written text that might not make complete sense. i'll clean it up as I go)
 +
 +  * Need a WaveformViewerFactory that produces waveform viewer widgets. This gets rid of the awkward dance done in mixxx/​mixxxview to create a waveform viewer, but then fallback to WVisualSimple if OpenGL is not available.
 +    * In order to do this, we need a standard interface for talking with a waveform viewer widget.
 +    * With enough abstraction,​ it shouldn'​t be too hard to do an OpenGL approach, a QPainter approach, AND a Simple approach (the current non-opengl path).
 +    * This requires changes to the way that Mixxx talks with the waveform viewer. Currently it is a little messy -- I'll be cleaning that up.
 +    * Accordingly -- need two widget types (QGLWidget and QWidget) Unfortunately to formally standardize the Mixxx interface for the two, they may need to use multiple inheritance to inherit from a WAbstractWaveformViewer-ish type that will codify the existence of '​setup()',​ etc. 
 +
 +
 +===== WaveformViewerFactory =====
 +A factory for producing waveform viewer widgets. During Mixxx initialization,​ all Mixxx will do is ask for a waveform widget given the config object. The factory will worry about whether GL is available, etc. 
 +
 +As it stands, Mixxx calculates whether the waveform preference is set and passes this as a boolean to the MixxxView constructor. Also MixxxView worries about setting up shared context between QGLWidgets, and whether to instantiate a GL waveform viewer or a simple one. The factory will hide all of this, and ultimately make Mixxx.cpp/​MixxxView.cpp cleaner. ​
 +
 +===== AbstractWaveformViewer =====
 +A base waveform viewer type that will setup the standard interface for Mixxx to with a generic waveform viewer. The code currently stores a pointer to the waveform viewer as a QObject and then casts wherever necessary to represent the tyep as a waveform object. ​
 +
 +===== WGLWaveformViewer =====
 +
 +A QGLWidget that renders the 
 +
 +===== WWaveformViewer =====
 +
 +===== AbstractWaveformRenderer =====
 +Base type/​interface for a renderer (like the 3 below) to inherit from.
 +
 +===== GLWaveformRenderer =====
 +Uses an OpenGL context to render the waveform using accelerated graphics hardware. Requires direct rendering in Linux, which is a downside.
 +
 +===== WaveformRenderer =====
 +Uses a regular Widget with QPainter to draw the waveform without hardware acceleration. This could throttle the CPU on some systems, so the user may want to disable it. 
 +
 +===== SimpleRenderer =====
 +For the most barebone of systems -- equivalent to the current '​Simple'​ renderer
 +
 +
 +===== Performance Tradeoffs =====
 +
 +One idea that I tried was keeping an entire song's worth of samples in RAM. This allows for instant-access to the data when a seek occurs, so there'​s no time that the waveform display doesn'​t have data. The current downside is that the general process is to loop over the sample buffer for the width of the screen, and draw a line to represent the sample. ​
 +
 +This has proven to be taxing at 60fps. It's not as fast as it could be. In order to make this faster, pre-rendering the entire song to a pixmap or image canvas, is the next step in time/space tradeoff. I've written this up, and QImage and QPixmap seem to get very buggy around 47457x68 pixels (about 9MB of memory). The QPixmaps generate tons of X11 errors about being out of resources. This seems completely unreasonable as apps like firefox routinely store many hundreds of pixmaps in the X server. Just the same, this looks like an area where we can't afford to be relying on a person'​s X server for good behavior. ​
 +
 +I have an idea that is a hybrid of the previous two ideas. I'm going to jot down a loose spec here. It's basically a pixmap or image that is a ring-buffer. The width is about 3 or 4 times the width of the waveform viewer in pixels, and the same height as it. This will hopefully amortize the cost of drawing per frame in the best case to one bitblt, second best case, two bitblts, third best case, 2 bitblts and a few line drawings, and worst case drawing the '​width'​ of lines, which is the current '​best'​ case for the waveform viewer with the draw-lines approach.
 +
 +SampleRingBuffer
 +  * 3-4 times the width of visual waveform viewer
 +  * keeps track of both a read pointer and write pointer. ​
 +  * the read and write pointer originally start at 0. 
 +  * per frame, no more than '​width'​ lines will be drawn onto the buffer. ​
 +  * EACH FRAME: (assuming forward progression) ​
 +    * 1) if the read pointer is less than the write pointer, it will bitblit a rectangle of width min(WP-RP,​width) into the widget
 +    * 2) if the read pointer is greater than the write pointer, it will bitblit a rectangle of width (bufstart+buflen-RP) into the widget. ​
 +    * 3) If the width written is less than the width of the widget, and the WP < RP, then loop to the beginning of the buffer and draw a rectangle of width min(remaining_pixels,​ (wp-bufstart)) into the widget. ​
 +    * 4) If the width written is less than the width of the widget, and the WP > RP, then draw that take in that many samples from the sample buffer, and write them at the write pointer, and increment the write pointer. ​
 +  ​
 +
 +The end effect here is that, for the first frame, you have to do '​width'​ drawlines, but do it in a pixmap, so you save the data. The next frame, bitblit the portion of the previous frame that was on screen into the screen, and do drawlines for the amount that is not in the buffer. If you make the buffer size 4 * the length of the visual waveform, you will always be doing some fast memcopy'​s of the previous image, and then a couple drawlines, and sometimes not even any drawlines. ​
 +
 +A benefit to having a big buffer is that if the user scratches the waveform back and forth, it will not have to seek backwards and re-draw, it can go up to 4 times the size of the waveform back before it has to draw new data. 
 +
 +When a seek occurs, the buffer is invalidated,​ and on the first frame after the seek, you have to do '​width'​ drawlines, but after that one frame you're only drawing a few lines per frame. ​
 +
 +If the user starts playing the audio in reverse, some nasty logic will have to happen that just flips the direction of the read/write pointers. ​
 +
 +
gsoc2008_waveform_rewrite.txt ยท Last modified: 2008/07/01 22:17 by rryan