Feature-driven sampling

This tutorial shows how to build k-dimensional trees to efficiently perform feature-based search on buffers. In this case, we use it to find the best buffer match for each pitch and velocity value in a MIDI file. In this tutorial, we do the following:

Slice a trumpet buffer into multiple onset-based segments, and analyze each.
Build a tree based on those segments, using pitch and velocity as search features.
Load a MIDI file as a list of events.
For each event, we search the tree for the best match based on pitch and velocity.
We get the deviation between the matched buffer segment and the target pitch.
We transcribe the buffer match, using the MIDI event's onset and apply the pitch deviation.
Finally, we render the entire sequence and apply post-rendering reverb and gain normalization.

feature-driven_sampling.bell
## load source buffer
$source = importaudio('trumpet.wav');
## perform onset detection analysis on buffer
$source = $source.analyze(onsets());
## get list of transient-based onset times
$markers = $source.getkey('onsets');
## split buffers into different segments, based on markers
$segments = $source.splitbuf(@split $markers @mode 2);
## initialize data (will hold pitch and velocity values for tree)
$data = null;
## iterate through segments and filter out unpitched ones
$segments = for $seg in $segments collect (
    ## analyze current segment for pitch and loudness
    $seg = $seg.analyze(pitchmelodia() larm());
    ## get pitch results
    $pitch = $seg.getkey('pitchmelodia');
    ## only accept segments with valid detected pitch
    if $pitch > 0 then (
        ## scale loudness to (pseudo) velocity range
        $velocity = $seg.getkey('larm') * 127;
        ## append pitch and velocity as feature vector
        $data _= [$pitch $velocity];
        ## return segment to collect
        $seg
    ) 
);
## create a k-dimensional tree on segments' pitch and velocity values.
$dataset = dataset($data);
## create standard scaler based on dataset
$scaler = stdscaler($dataset);
## scale features for better search accuracy
$dataset = transform($scaler, $dataset);
$tree = kdtree($dataset, 3); ## 3-nearest-neighbors search tree
## import MIDI file
$events = importmidi('bach.mid');
## MIDI output speed
$speed = 1.25;
## gain envelope for every buffer match
$gainenv = [0 1 0] [1 0 0.25];
## iterate through first 64 MIDI events
for $event in left($events, 64) do (
    ## get pitch, onset, duration, and velocity info from event
    $pitch = $event.getkey('pitch');
    $onset = $event.getkey('onset');
    $duration = $event.getkey('duration');
    $velocity = $event.getkey('velocity');
    ## create target vector (pitch and velocity).
    $target = $pitch $velocity;
    ## normalize target using same scaler as training data
    $target = transform($scaler, $target);
    ## search tree for nearest neighbors
    $result = predict($tree, $target);
    ## randomly select one from nearest neighbors
    $matchindex = choose(getkey($result, 'neighbors'));
    ## extract segment based on match's index
    $match = $segments:$matchindex;
    ## get cents deviation between target pitch and buffer match
    $detune = pitchdiff(getkey($match, 'pitchmelodia'), $pitch);
    ## adjust duration to compensate for resampling-based detuning
    $duration /= c2r($detune);
    ## cap match duration at adjusted target duration
    $match = mapkey($match, 'duration', $x -^ $duration -> min($x, $duration));
    ## transcribe match using MIDI event's onset and pitch deviation
    $match.transcribe(
        @onset $onset / $speed
        @detune $detune
        @gain $gainenv
        @pan rand() * 0.25 + 0.5 ## randomize panning around center
        @pitchkey 'pitchmelodia' ## use detected pitch in timeline 
    ) 
);
## trigger rendering
render(
    @play 1 @process (
        ## apply reverb and normalization
        freeverb(@roomsize 0.8) normalize() 
    ) 
)

Result​

Result