On Thu, Jul 30, 2020 at 11:20 AM Ben Goertzel <[email protected]> wrote:
> > -- send a Pattern Matcher query to BackingStore > -- sent the Atom-chunk resulting from the query to Atomspace > > So, Someone needed to prove me wrong, and who better to do that but me. I took the weekend to implement a file-based backing store, using RocksDB (which itself is a variant on LevelDB). It's here: https://github.com/opencog/atomspace-rocks -- It works, all of the old persistent store unit tests pass (there are 8 of them) -- its faster than the SQL by factors of 2x to 5x depending on dataset. With tuning, maybe one could do better. (I have no plans to tune, right now) I'm certain I know of a simple/easy way to "send a Pattern Matcher query to BackingStore and send the Atom-chunk resulting from the query to Atomspace" and will implement this afternoon (famous last words...) BTW, you can *already* do this with the cogserver-based network client (i.e. without sql, just the network only) here: https://github.com/opencog/atomspace-cog/blob/master/examples/remote-query.scm By combining these two backends, I think you can get file-backed storage that is also network-enabled. Or rather, you have two key building blocks for exploring both distributed and also decentralized designs. Some background info, from the README: AtomSpace RocksDB Backend ========================= Save and restore AtomSpace contents to a RocksDB database. The RocksDB database is a single-user, local-host-only file-backed database. That means that only one AtomSpace can connect to it at any given moment. In ASCII-art: ``` +-------------+ | AtomSpace | | | +---- API-----+ | | | RocksDB | | files | +-------------+ ``` RocksDB (see https://rocksdb.org/) is an "embeddable persistent key-value store for fast storage." The goal of layering the AtomSpace on top of it is to provide fast persistent storage for the AtomSpace. There are several advantages to doing this: * RocksDB is file-based, and so it is straight-forward to make backup copies of datasets, as well as to share these copies with others. * RocksDB runs locally, and so the overhead of pushing bytes through the network is eliminated. The remaining inefficiencies/bottlenecks have to do with converting between the AtomSpace's natural in-RAM format, and the position-independent format that all databases need. (Here, we say "position-independent" in that the DB format does not contain any C/C++ pointers; all references are managed with local unique ID's.) * RocksDB is a "real" database, and so enables the storage of datasets that might not otherwise fit into RAM. This back-end does not try to guess what your working set is; it is up to you to load, work with and save those Atoms that are important for you. The [examples](examples) demonstrate exactly how that can be done. This backend, together with the CogServer-based [network AtomSpace](https://github.com/opencog/atomspace-cog) backend provides a building-block out of which more complex distributed and/or decentralized AtomSpaces can be built. Status ------ This is **Version 0.8.0**. All unit tests pass. All known issues have been fixed. This could effectively be version 1.0; waiting on user feedback. -- Linas -- Verbogeny is one of the pleasurettes of a creatific thinkerizer. --Peter da Silva -- You received this message because you are subscribed to the Google Groups "opencog" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CAHrUA37Agw0cg5gJX1fDffvSAjcW1kq4LdMOSuyknaEC_41F1g%40mail.gmail.com.
