cpd73 wrote:
> My interest was in trying to use Essentia to create a "Don't Stop The
> Music" mixer. I analysed 1600 tracks, with very varied results - and saw
> no easy way of using the data to say this track is similar to that
> track. Very likely thats down to me having literaly no experience in
> machine learning, etc, so none of these models make sense. I was,
> naively, hoping that I could find a track's essentia data then find
> other tracks whose (e.g.) danceability, agressiveness, etc, was within a
> certain range of the seed track. And I totally failed at that! So, in
> short, I'm giving up on this for now.
First, see "Available data" on the HP for a better understanding about
what "essentia data" actually is. Unless writing a classification system
(e.g. using machine learning), the lowlevel music descriptors are
irrelevant. LMS Essentia only uses the high-level classifier models
(which take the lowlevel music descriptors as an input).
These will compute a probability that some data (in this case a audio
track) belongs to a class (e.g. "sad mood") defined by the model (using
Gaia or Tensorflow for the machine learning part makes no difference in
this regard).
So the approach to measure similarity of tracks by positing that if they
are in the same (model-specific) class in n models, they are similar,
may have some merit.
You can try this with the existing DB by getting the attributes and
values for one track and then selecting all tracks with these attributes
and values. Example:
Code:
--------------------
select id,file_name from essentia_tracks where id in (
select distinct etrackid from essentia_values where (
(attrib="esstf.genre_tzanetakis.vggish-audioset" and value="cla") or
(attrib="esstf.voice_instrumental.vggish-audioset" and value="instrumental") or
(attrib="esstf.mood_aggressive.musicnn-mtt" and value="not_aggressive")) and
probability > 0.8
group by etrackid having count(etrackid)=3);
--------------------
How well this will work will depend on how accurate the models are, how
exhaustively a track is characterized by the combination of all classes,
and on how well the models are a fit for the tracks. For moods etc. the
latter will be much less of an issue then for genres. For instance, none
of the genre models work that well for my collection, most likely
because they are trained on representative music collections and thus
contain classes not applicable to my tracks (i.e. genres that are not in
my collection), and also because they are too coarse-grained for my
taste ("rock" or "pop").
One solution would be define custom models and to train them on one's
own collection. I tried that with Gaia, but it either segfaulted, or
crashed with some really unhelpful python exceptions. At some point
later, I therefore decided to write my own system to replace Gaia, and
to make defining/training custom models simple.
Preliminary results are fairly promising, considering that I haven't
really invested time into tweaking it and that the training data isn't
that great (some inconsistent tagging, only tagged at album and not
track level etc.): 'test results'
(https://www.nexus0.net/pub/sw/misc/lmsai.png)
Now that Essentia has Tensorflow support, I may look into training
'musicnn' (https://github.com/jordipons/musicnn-training), but at first
glance, this looks like it's going to be tedious (ancient Tensorflow
versions etc.) and I'd rather continue with my own solution.
> I realise tensorflow would probably give better results, but the
> non-tensorflow version was slow enough!
Tensorflow with 'GPGPU'
(https://en.wikipedia.org/wiki/General-purpose_computing_on_graphics_processing_units)
may actually be faster (depending on you graphic card's CUDA
performance)
'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...
------------------------------------------------------------------------
Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=108278
_______________________________________________
plugins mailing list
[email protected]
http://lists.slimdevices.com/mailman/listinfo/plugins
