I wonder how different would be the API for RocksDB vs., say, Cassandra which Matt Chapman has recommended (which may have some advantages in terms of allowing more configurable/flexible notions of consistency?)
On Tue, Aug 4, 2020 at 4:44 PM Linas Vepstas <[email protected]> wrote: > > > > On Tue, Aug 4, 2020 at 11:51 AM Ben Goertzel <[email protected]> wrote: >> >> Wow! > > > You're welcome. Querying from the database is now supported. The demo is in > https://github.com/opencog/atomspace-rocks/blob/master/examples/query-storage.scm > > At the moment it works, but I'm rethinking the API. Do check it out. > Feedback, opinions, suggestions, etc. invited. > > --linas > >> >> On Tue, Aug 4, 2020, 8:45 AM Linas Vepstas <[email protected]> wrote: >>> >>> >>> >>> On Thu, Jul 30, 2020 at 11:20 AM Ben Goertzel <[email protected]> wrote: >>>> >>>> >>>> -- send a Pattern Matcher query to BackingStore >>>> -- sent the Atom-chunk resulting from the query to Atomspace >>>> >>> >>> So, >>> >>> Someone needed to prove me wrong, and who better to do that but me. I took >>> the weekend to implement a file-based backing store, using RocksDB (which >>> itself is a variant on LevelDB). It's here: >>> https://github.com/opencog/atomspace-rocks >>> >>> -- It works, all of the old persistent store unit tests pass (there are 8 >>> of them) >>> -- its faster than the SQL by factors of 2x to 5x depending on dataset. >>> With tuning, maybe one could do better. (I have no plans to tune, right now) >>> >>> I'm certain I know of a simple/easy way to "send a Pattern Matcher query to >>> BackingStore and send the Atom-chunk resulting from the query to Atomspace" >>> and will implement this afternoon (famous last words...) BTW, you can >>> *already* do this with the cogserver-based network client (i.e. without >>> sql, just the network only) here: >>> https://github.com/opencog/atomspace-cog/blob/master/examples/remote-query.scm >>> >>> By combining these two backends, I think you can get file-backed storage >>> that is also network-enabled. Or rather, you have two key building blocks >>> for exploring both distributed and also decentralized designs. >>> >>> Some background info, from the README: >>> >>> AtomSpace RocksDB Backend >>> ========================= >>> >>> Save and restore AtomSpace contents to a RocksDB database. The RocksDB >>> database is a single-user, local-host-only file-backed database. That >>> means that only one AtomSpace can connect to it at any given moment. >>> >>> In ASCII-art: >>> >>> ``` >>> +-------------+ >>> | AtomSpace | >>> | | >>> +---- API-----+ >>> | | >>> | RocksDB | >>> | files | >>> +-------------+ >>> ``` >>> RocksDB (see https://rocksdb.org/) is an "embeddable persistent key-value >>> store for fast storage." The goal of layering the AtomSpace on top of it >>> is to provide fast persistent storage for the AtomSpace. There are >>> several advantages to doing this: >>> >>> * RocksDB is file-based, and so it is straight-forward to make backup >>> copies of datasets, as well as to share these copies with others. >>> * RocksDB runs locally, and so the overhead of pushing bytes through >>> the network is eliminated. The remaining inefficiencies/bottlenecks >>> have to do with converting between the AtomSpace's natural in-RAM >>> format, and the position-independent format that all databases need. >>> (Here, we say "position-independent" in that the DB format does not >>> contain any C/C++ pointers; all references are managed with local >>> unique ID's.) >>> * RocksDB is a "real" database, and so enables the storage of datasets >>> that might not otherwise fit into RAM. This back-end does not try >>> to guess what your working set is; it is up to you to load, work with >>> and save those Atoms that are important for you. The [examples](examples) >>> demonstrate exactly how that can be done. >>> >>> This backend, together with the CogServer-based >>> [network AtomSpace](https://github.com/opencog/atomspace-cog) >>> backend provides a building-block out of which more complex >>> distributed and/or decentralized AtomSpaces can be built. >>> >>> Status >>> ------ >>> This is **Version 0.8.0**. All unit tests pass. All known issues >>> have been fixed. This could effectively be version 1.0; waiting on >>> user feedback. >>> >>> -- Linas >>> >>> -- >>> Verbogeny is one of the pleasurettes of a creatific thinkerizer. >>> --Peter da Silva >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "opencog" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/opencog/CAHrUA37Agw0cg5gJX1fDffvSAjcW1kq4LdMOSuyknaEC_41F1g%40mail.gmail.com. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "opencog" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/opencog/CACYTDBcnROxkUgppev8cW2LuAbzqvjWXxrrWZvCgvQv3g9Q3eg%40mail.gmail.com. > > > > -- > Verbogeny is one of the pleasurettes of a creatific thinkerizer. > --Peter da Silva > > -- > You received this message because you are subscribed to the Google Groups > "opencog" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/opencog/CAHrUA36cTFc-S7C%3D0SqQgfAGZ1bpVupihVfOs0g6hpD14UtSxw%40mail.gmail.com. -- Ben Goertzel, PhD http://goertzel.org “The only people for me are the mad ones, the ones who are mad to live, mad to talk, mad to be saved, desirous of everything at the same time, the ones who never yawn or say a commonplace thing, but burn, burn, burn like fabulous yellow roman candles exploding like spiders across the stars.” -- Jack Kerouac -- You received this message because you are subscribed to the Google Groups "opencog" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CACYTDBfmiyKhHusd2ThoD6dAYBDdyL73CB%3DJe6w0-aX7WbX_Uw%40mail.gmail.com.
