On Saturday, 17 April 2021 at 20:10:23 UTC+2 linas wrote:
> Hi Ben, > > Thank you! I am seriously behind on my reading list; I have a pink piece > of paper in front of me, right now, on which it is written "read Ben's > latest stuff"! It's right up there with much more mundane chores. > > Some comments; trying to be short, but failing... > > On Sat, Apr 17, 2021 at 11:23 AM Ben Goertzel <[email protected]> wrote: > >> >> 1) A distributed Atomspace (which will also, layered on SNet platform, >> be the foundation a decentralized Atomspaces) >> > > Shall I repeat again? The atomspace is already distributed, and there are > now several different ways in which this works. Actually, N ways, to be > precise. I'll describe them below. Again, I invite anyone and everyone to > try it, use it, find out if or how it meets or doesn't meet current needs > and expectations. (It's come a long ways since the iCog Labs experiments, > and explicitly includes feedback from those experiments, Xabush and Kasim > in particular; they may not think I am listening to them, but I am, and am > taking their comments seriously.). > > >> 2) An "Atomese 2" language , consisting of a set of Atom types for use >> in the Atomspace metagraph (with the flexibility to specify >> sub-metagraphs practically usable as programs, among other >> capabilities), plus some syntactic sugar for human use and a >> corresponding interpreter >> > > Well, I'm hoping that you don't succumb to an urge to rewrite everything > from scratch. I fear that you are tempted to do this. I'm fairly certain I > won't be able to talk you out of it, if that is what you choose to do. Let > me only caution that the software industry has vast amounts of experience > in making this decision, and in most situations, it turns out to be a dire > mistake, and in most of the rest, it turns out to be incredibly costly, > painful, expensive and many years behind schedule. I mention this because I > know that you don't follow generic software development news, so you may > think that I am exaggerating, or that it doesn't apply to you. You should > talk to some neuro-typical software executives and get a second opinion > before you embark on a rewrite. > > You have seen some of our thinking on Distributed Atomspace, and as I >> recall in our last discussion on this you said you had some critiques >> in mind but didn't offer them in detail. >> > > I started reading the document you sent me, but it was full of so many > misconceptions and confusions, it was hard to know how to even start > critiquing it. It was kind of a case of "throw it out and start all over > again". (Oh, the irony) I was (still am) fearful my advice (and any > document I write) would be ignored, so there's not much incentive to work > hard on that. Life is short. > > Perhaps the biggest critique is that a "uuid" is not a central concept, > and that uuids are only needed on communications channels. They don't have > to be "universally unique", they only need to be unique on that channel. > The other critique is that one should never-ever underestimate the cost of > communications: sending data from here to there is always expensive, and > not just a little, but a lot. > > You might not be aware of the AtomSpace RocksDB StorageNode. It is now 10x > or 12x faster than the Postgres Node. This is not because postgres is > "stupid", but because there is a huge cost associated with any storage > medium that uses the network to communicate. Again: the cost of > communications is huge. (well, OK, the atomspace postgres node is > over-engineered; I think I now understand a slimmer and faster way of doing > things. That's a rainy-day project. It rarely rains, here.) > > ---------------- > So. The distributed atomspace -- things you may not be aware of. There is > now a new Atom type: (StorageNode "some://example.com/url/where?ever") > When you open it, you can send Atoms to that URL, or receive them, or even > run pattern queries on the remote system, and get back only the matching > results. You can have as many of these open as you want, and so can copy > atoms around from one to another. You can close and reopen them at any > time, for any reason. Everybody and their grandmother can now run an > AtomSpace node, and merrily exchange Atoms and TruthValues with one-another. > This sounds more like *remote* AtomSpace, not distributed. This is still important for use cases where there is supposed to be a server handling multiple clients that send / receive knowledge from it. The distributed use case would usually be when you have a compute cluster and you want to distribute data and tasks across its units - so for example you might have 100 machines there with a sharded AtomSpace and want to make a query such that all these shards get analysed and results pooled together in the end (a la MapReduce). I do not know much about distributed computing myself but there would have to be some sort of orchestration process, ideally this would all be integrated with modern tools for cluster management like Kubernetes. The need for such a workflow ultimately depends on how much computational power you need for your experiments / applications - I myself do not even think about running experiments on such a large scale (at least not yet)... > > There are four supported URL's (and they pass all unit tests, over a dozen > unit tests each. The code is not "beta", its "production ready" ). > > cog://example.com/ > cog-simple://example.com > rocksdb:///path/to/file/on/disk > postgres://example.com/dataset-name?user=foo&passwd=bar > > The first two talk to remote atomspaces. The cog-simple variant is a > stunningly simple variant. I suppose it's slower under a heavy load, but > not much, I don't think. (its full-featured; the implementation is very > minimalist.) > > The RocksDB node is in the local filesystem. Rocks is facebook's #1 > database; it's very highly tuned for performance, and many popular > databases, including graph databases, use rocksdb "under the covers" to do > the heavy lifting. Basically, it's fast and feature-rich. > > The postgres node is built on top of the same-old postgres code as > before. The only thing new is that you can now open N of these, if you > wish. You can open all of these at the same time, and even move atoms > around between them. It works. I've done it. (Hmm. Oddly enough, I haven't > written a unit test for that, yet. I suppose I should ...) > > --------------------- > So, for a basic distributed atomspace, each of your friends can run an > AtomSpace on their computers, can trade URL's with one-another, "cog:// > example.com" "cog://foo.com/" and "cog://bar.com" and then happily have > all the atomspaces chat with one-another, shipping atoms around every-which > way, running remote pattern matches, etc. > > I suppose you might say that you want something fancier than that... > sure! You'd have to say what that is. (the aforementioned document did > not describe anything fancier than this.) I've got an inkling of > possibilities, but this email is too long already. > > Till later, take care; hope all is going well, and shout if you need > anything. > > -- Linas > > > > -- > Patrick: Are they laughing at us? > Sponge Bob: No, Patrick, they are laughing next to us. > > > -- You received this message because you are subscribed to the Google Groups "opencog" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/25e25346-fa0b-4bbf-907c-a226e6cc5392n%40googlegroups.com.
