Hi,
I am developing an application which analyses audio data, and I have recently
been looking into Sqlite as a possible file format. The result of an analysis
in my application is a hierarchical data set, where each level in the hierarchy
represents a summary of the level below, taking the max of each pair in the
sub-level, in the following way:
251 214
251 54 201 214
251 91 17 54 31 201
214 66
251 18 5 91 11 17 54 16 9 31 201 148 173 214 43 66
Such a structure essentially represents the same data set at different levels
of resolution ('zoom levels', if you like). My first experiments involved a
btree-like structure (actually something closer to an enfilade* or counted
btree**), where the data stored in each node is simply a summary of its child
nodes. Edits to any node at the leaf level propagate up the tree, whilst large
edits simply entail unlinking pointers to subtrees, thus making edits on any
scale generally log-like in nature. This works fine as an in-memory structure,
but since my data sets might potentially grow fairly large (a few hundred MB at
least) I need a disk-based solution. I naively assumed that I might be able to
utilize Sqlite's btree layer in order to implement this more effectively; this
doesn't seem possible, however, given that the btree layer isn't directly
exposed, and in any case it doesn't map onto the user interface in any way that
seems helpful for this task.
I am aware of some of the ways in which hierarchical or tree-like structures
can be represented in a database (adjacency lists, nested sets, materialized
paths, etc.), but none of these seems to offer a good solution. What I'm
experimenting with at present is the idea of entering each node of the
hierarchy into the database as a blob (of say, 1024 bytes), while maintaining a
separate in-memory tree which then maps on to this flat database of nodes (each
node in the tree maintains a pointer to a node in the database).
I would be very interested in
thoughts/observations
on this problem - or even better a solution!
Many thanks in advance,
Christopher
* http://en.wikipedia.org/wiki/Enfilade_(Xanadu)
** http://www.chiark.greenend.org.uk/~sgtatham/algorithms/cbtree.html
_______________________________________________
sqlite-users mailing list
[email protected]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
