Educational Vowel Synth and ControlSurface Widget

Some months ago I had the idea of doing a Vowel Synthesizer program which could help children to understand why the vowel triangle has some sense and it also could be used to train foreign language vowels.



The core of the idea is that axes correspond mostly to the position of the first two formants F1 and F2. So by a simple mapping, given a point on the triangle you could synthesize any vowel even those in the middle. Also by analyzing incoming voice from a microphone, the system could place a point in the triangle that identifies how far one is from the intended vowel.



I did a first prototype which synthesized something far from human but at least you can identify the vowels when you place the proper frequencies. As i had a proof of concept i stopped there.



This easter i improved the prototype by adding a new control widget: The ControlSurface, which control two parameters by moving a single point. It is really more handy than having two sliders and it is perfect for parameter exploration, not just for the Vowel Synthesizer.

It will also force a change on the way the CLAM Prototyper address the binding of controls as it controls two parameters and our current system binds a widget to a single inlet.

Realtime MFCC and LPC analysis

As I said on the previous blog entry, too much things done in CLAM this Easter. We fully dropped Fltk visualization module from the main libraries. Zach Welch (one of the active new commers) ported Voice2Midi to Qt4 and we moved a subset of the Qt3 visualization module to the SMSTools and dropped the rest. (Zach prepared a set of patches to port it to Qt4 but let's wait until the rewrite). All those drops mean that CLAM size has been reduced to a half of its former size: less mainteinance and faster entry for the new developers.

A few examples stopped being functional because they used Fltk Plots. One of them the LPC_example. It compiled by disabling visualization rendered the example useless. I decided to convert it into a Network Editor example.

After some hacking i had this piece of network working.



After some designer and prototyper hacking i got this interface:



Trying to understand the meaning of the LPC I also added a new output the spectral envelope:



So it captures the spectral envelope which is gonna be very useful for the vowel synth.

Then a user asked for realtime MFCC. As we already had the processings but using portless interface, and LPC visualization could be reused for MFCC, i also took this task. The result:



Mixing it all in a single analysis interface



Vowel formants are even more clear in MelSpectrum that in MelCepstrum or the Spectrum it self. Another idea to be used for the Educational Vowel Synth.

Realtime Voice Gender Change

This Easter has been very productive for the CLAM framework. Holidays and a bunch of new developers that have aproximated the team during these weeks have accelerated a lot the development which had already been accelerated during last months. As consequence, a lot of dormant to-do's has been addressed and most of them deserve a blog entry (pixmap widgets, realtime LPC/MFCC, Qt3/Fltk drop, Qt4 ports, new visualization widgets, new control widgets...). Just take a look at the CLAM-devel mailing list archives or the development screenshots and you'll see a im not exagerating. So let's start from the beginning.

One of the nicer things I worked on and which is going to be the star on CLAM demos for sure, is the voice Gender Change effect on the NetworkEditor. Since Xavier implemented the offline version, it has been arround for a while on the SMSTools turning Elvis voice into Elvira. It was also on the NetworkEditor but crashing or not even sounding.

So, I am a demo whore, let's make people smile. Let's make it realtime. Pau already did some work as he ported almost all the SMS transformation. I just had to fix some of the internal processings, the Pitch shift, and the SpectralShapeShift. Most of the bugs were about detecting non-harmonic parts and figure out which is the best strategy in those cases. In some case this was detected but the action relied on input to output copies implicit on the offline version but not on the realtime one.

What i was not able to solve is the CombFilter. It added a really extrident noise to the residual part. So I disabled it and it does not sound as perfet but... tada! i started listening my sister voice from the speakers instead of mine when i talked the mic, which is specially frightening, most of you know why.

That's the look the network had after having the system completed.



And a first attemp Prototyper interface for an stand alone application:



As soon as it is available as nightly snapshots try it. It is worth to listen. My first experiments i used it to learn how to do a mixt choir in Ardour with just my own voice.