[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Sc-devel] Machine Listening plans

Hi all,

I wanted to outline my proposed changes to instantiate in the coming weeks, for comments and approval. These are being considered alongside the writing of my scbook chapter on this topic.

1. Potential removal of PV_HainsworthFoote and PV_JensenAndersen as outdated and not worth supporting (there are much better solutions now).

2. Introduction of OSC messaging from Server of float arrays as a natural way to share data with the lang, circumventing the hazards of buffer polling (ie, you can send data only when an event is detected)

3. Introduction of a series of new UGens with the prefix ML_, which run using the crossplatform FFT UGen or an equivalent* as their initial frontend. A new ML_UGens target would be added to the Xcode plugins project (I'd need help making the scons update).

The following are already written and ready to commit:

ML_AutoTrack //AutoTrack converted to run with FFT frontend
ML_Loudness //instantaneous loudness calculation in sones, using equal loudness contours and simple spectral and temporal masking

A problem with the spectral machine listening work is supporting only one sample rate. I've found a way to support 44100, 48000, 88200 and 96000, which should be enough for some flexible use.

And I'm drafting more, for example ML_Kitch (12TET major/minor key detection, though not good enough to commit at the moment), and could convert an ML_Concat, ML_AnalyzeEvents etc. I'd also be happy to discuss the addition of Dan's onset detection work to core, perhaps to FFT2_UGens?

Thoughts? Only step 2 could break anything (I sent the sketch to sc- devel before), but there is the issue of adding classes and one plugin.

* The FFT UGen may have to be further modified to support zero crossing, and in some cases, multiple FFTs are needed by a single UGen, requiring everything to be done in-UGen. But the same cross- platform base would be followed.