On Wed, Aug 4, 2021 at 7:45 AM Michael Duncan <[email protected]> wrote:
> linus, an outline of the potential new atomspace is here > <https://wiki.opencog.org/w/Hyperon:Atomspace>. > Thank you Micheal! I skimmed the proposals. They seem to make sense. Mostly, they are lacking in the next level of detail. They resemble discussions we've been having, on and off, for the last decade, and a number of ways of implementing this stuff have been debated in the past. Some of the stuff has been prototyped and experimented with. Others have had foundations laid, but no further development done. (You said "potential new atomspace" but this is a bit of an exaggeration. There's nothing in those proposals that the Atomspace doesn't already mostly do, or couldn't easily do, with assorted relatively minor tweaks. So it's not a "new atomspace", it's some tweaks to what is already there.) I'm thinking it might be time for having an RFI/RFC process -- "request for implementation" -- detailed documents describing specific features and functions, together with a status for each - accepted, implemented, final, withdrawn. Kind like internet RFC's (requests for comment) or scheme srfi's, or python PEP's (python enhancement proposals). Each RFC would be issued a number (so we can track its status) Each proposal should address some meta-questions: * What makes the current AtomSpace inadequate for the envisioned applications? * Can it be retrofitted with the needed things? * Can the needed things be designed as modules on top of the AtomSpace, or can they be plugins, or shims? Do they need to be integrated? Do they require deep, difficult changes or are they shallow? [Meta comments: My general stance is that the current AtomSpace was designed to be extremely flexible, and to accommodate these kinds of ideas. I'm mostly imagining that implementing the various described new features/functions on the Hyperon wiki page is probably straight-forward (perhaps I'm wrong.). A lot of foundational groundwork has been laid for this kind of stuff. I'd be surprised if these foundations would not be able to accomodate the desired enhancements.] Until there are users clamoring for these things, nothing really gets done. I don't know if anyone has time to move forward on any of this stuff, or not. So let me shout out into the wilderness: Going through the links: -- The old 2017 "distributed atomspace" documents describe several ideas that have already been implemented and explored, Some are done, others were discarded because they did not work (things like UUID's). So these proposals are bit-rotted, they don't adequately describe the current situation. This is a bad situation, since newcomers can't tell the difference between fresh-new-promising, and old-stale-broken. -- Clicking on the first link: "Episodic Memory Design for Virtual Assistant" -- Dagim Sisay, Eyob Yidaw, Solomon Molla -- I see things like "PostGIS for GIS data" with fairly-standard SQL statements. There is another (different) half-finished proposal for a shim, that would allow any arbitrary structured data to be mapped into the AtomSpace. That is, instead of writing an importer to pull bulk data into the AtomSpace, the shim would declare that (for example) Evaluation Predicate "foo" List Concept "bar" Concept "baz" should be mapped to (for example, if its SQL or PostGIS) "SELECT (mambo, jambo) FROM TABLE lumbago WHERE thing=bob;" and that correspondingly, changes to the AtomSpace, for this one particular type of structure, would be mapped to "UPDATE TABLE lumbago SET mambo=bar, jambo=baz;" I think such shims can be created for any data source or sink, not just SQL. (so for example, a shim for attaching to a wikidata triplestore, or some semantic web thingy, or maybe an octree for 3D data, or a point-cloud for 3D data, or robotic SLAM systems.) The point here is that this allows different kinds of datasources to be accessed within the framework of the AtomSpace (and specifically, the query engine), without paying the (huge) overhead of trying to import everything, all in one go (and then running out of RAM). If anyone is actually interested in this, the idea can be developed further. I think it can be made to work, cleanly and simply, without all that much new code ... but I might be mistaken about the difficulty. Without a proof-of-concept, who knows. -- Clicking on the second link: "Episodic memory for OpenCog" - Alexey Potapov -- The ideas there seem a bit more sophisticated, but, after a quick skim, I don't see anything that breaks the current paradigm. There are already a number of inverted indexes and caches; only one of them is explicitly exposed to the user; the rest work under the covers, automatically. The proposal seems to be asking for more of the same. (I skimmed it; maybe I missed something.) Anyway, not much can be done, until there are users whose needs are so urgent, that they have the time for actual discussions, prototyping, development and testing. Until someone really really needs this stuff, it ends up on the back burner, simmering and stewing. -- Clicking on the third link, "Episodic Storage Design" - Vitaly Bogdanov - I see this line: "Taking all of the above into account, the minimal set of operations to support by episodic storage is following:..." followed by a list. That list is very similar to the API half-way down the page of https://wiki.opencog.org/w/StorageNode -- that API has already been implemented, tested, debugged and available in scheme, python and c++ -- So then ..??? Should I update the Hyperion wiki page to say "this part is done already"? That is, the "Episodic Storage Design" has already been implemented"? This is why a formal RFC/RFI/RFE process seems like a good idea -- so that we can track the status of where it's all at. -- Linas > ivan, re my tickmark comment, what i was trying to get at is a graphical > indicator for the root node so one wouldn't have to try to scoll up a node > and fail to know one was at the top of the tree. having a small window > with a representation of the complete tree structure with the portion in > the main display highlighted would also be helpful for navigation. > > i would find this useful for exploring any tree/dag (directed acyclic > graph) in the atomspace. besides inferences traces this includes > ontologies in a declarative knowledge atomspace and the boolean models > output by the moses <https://github.com/opencog/asmoses> evolutionary > learning system. a dag with up to a couple hundred nodes could be > tractably explored by hand. > > in the mozi <https://github.com/MOZI-AI/annotation-service> codebase > there is an atomese to json parser (via scheme), if you are interested i > can find some json examples of the above for you to experiment with. > > On Thursday, July 29, 2021 at 7:06:20 AM UTC-4 linas wrote: > >> On Fri, Jul 23, 2021 at 7:44 AM Ivan V. <[email protected]> wrote: >> >>> Knowing that new Atomspace versions may be coming, I'd like to leave the >>> support option open. >>> >> >> I do not expect any "new AtomSpace versions". >> >> I believe that the mentioned JSON input would work well enough with the >>> incoming versions. >>> >> >> I understand you were talking abbott JSON in a different context, but >> there has been enough general confusion (with other correspondents) that I >> recently added a "what the AtomSpace is not" section to the main README. >> One of the things it says is "the AtomSpace is not JSON". I think it might >> be educational to review this. >> >> cut-n-pste below. >> >> What it isn't >> >> Newcomers often struggle with the AtomSpace, because they bring >> preconceived notions of what they think it should be, and its not that. So, >> a few things it is not. >> >> - >> >> *Its not JSON.* So JSON is a perfectly good way of representing >> structured data. JSON records data as key:value pairs, arranged >> hierarchically, with braces, or as lists, with square brackets. The >> AtomSpace is similar, except that there are no keys! The AtomSpace still >> organizes data hierarchically, and provides lists, but all entries are >> anonymous, nameless. Why? There are performance (CPU and RAM usage) and >> other design tradeoffs in not using explicit named keys in the data >> structure. You can still have named values; it is just that they are not >> required. There are several different ways of importing JSON data into the >> AtomSpace. If your mental model of "data" is JSON, then you will be >> confused by the AtomSpace. >> - >> >> *It's not SQL. It's also not noSQL*. Databases from 50 years ago >> organized structured data into tables, where the key is the label of >> a column, and different values sit in different rows. This is more >> efficient than JSON, if you have many rows: you don't have to store the >> same key over and over again, for each row. Of course, tabular data is >> impractical if you have zillions of tables, each with only one or two >> rows. >> That's one reason why JSON was invented. The AtomSpace was designed to >> store *unstructured* data. You can still store structured data in it; >> there are several different ways of importing tabular data into the >> AtomSpace. If your mental model of "data" is structured data, then you >> will >> be confused by the AtomSpace. >> - >> >> *It's not a vertex+edge store*. (Almost?) all graph databases >> decompose graphs into lists of vertexes and edges. This is just fine, if >> you don't use complex algorithms. The problem with this storage format is >> locality: graph traversal becomes a game of repeatedly looking up a >> specific vertex and then, a specific edge, each located in a large table >> of >> vertexes and edges. This is non-local; it requires large indexes on those >> tables (requires a lot of RAM), and the lookups are CPU consuming. Graph >> traversal can be a bottleneck. The AtomSpace avoids much of this overhead >> by using (hyper-/meta-)graphs. This enables more effective and simpler >> traversal algorithms, which in turn allows more sophisticated search >> features to be implemented. If your mental model of graph data is lists of >> vertexes and edges, then you will be confused by the AtomSpace. >> >> The actual AtomSpace resembles some aspects of all three, without being >> specifically any of them. It tries to be general: it wants to let you work >> with structured data or with unstructured data or with graphs, or any >> mixture of all three, however you please. It does not force any particular >> style. >> >> You can store data as ontologies, or as lambda expressions, or as >> prolog-like logical statements, or as syntactic (BNF-style) productions or >> as constraints or as RDF/OWL style schemas ... you can mix declarative, >> procedural and functional styles ... we don't care. The AtomSpace is meant >> to allow general knowledge representation, in any format. >> >> That said: it means that the AtomSpace is different and unusual. It might >> be a bit outside of the comfort zone for most programmers. It doesn't have >> API's that are instantly recognizable to users of these other systems. >> There is a challenging learning curve involved, here. We're sorry about >> that: if you have ideas for better API's that would allow the AtomSpace to >> look more conventional, and be less intimidating to most programmers, then >> contact us! >> >>> >>> peace, >>> ivan >>> >>> čet, 22. srp 2021. u 00:09 Michael Duncan <[email protected]> napisao >>> je: >>> >>>> this is very cool, ivan! two suggestions would be to indicate on the >>>> ovals with a tick mark where the off-screen nodes are to guide navigation, >>>> and a permanent slider/[+,-] buttons for global zoom control. (i know >>>> these are obvious) >>>> >>>> On Tuesday, July 13, 2021 at 7:15:09 AM UTC-4 [email protected] wrote: >>>> >>>>> While waiting for inference histories, I'm developing an idea about >>>>> turning the inference visualizer into an interactive AtomSpace debugger: >>>>> http://ocog.atspace.cc/. >>>>> >>>>> pon, 12. srp 2021. u 21:43 Ivan V. <[email protected]> napisao je: >>>>> >>>>>> Great, I'm glad you are interested in an experiment. >>>>>> >>>>>> A reasonable step would be for Nil to send you some real PLN and URE >>>>>>> inference histories and see what your visualizer does with them... >>>>>>> >>>>>> >>>>>> Sure, JSON history files would be perfect (we could make this a >>>>>> standard communication pipe between reasoner and visualizer), but I >>>>>> guess I >>>>>> can somehow manage any existing format you are already used to. You know >>>>>> my >>>>>> mail. >>>>>> >>>>>> Currently, the library is AJAX-ing an XML tree structure and standard >>>>>> HTMLs as input, with possibility of using php or other server side >>>>>> scripting technology to interface the input files you pass over here. >>>>>> Would >>>>>> that be ok? I have to mention, with some additional effort, there could >>>>>> be >>>>>> other input options such as reading and evaluating JSON file wrapped into >>>>>> javascript source code file. I'm telling this because this would exclude >>>>>> the requirement of running a HTTP and php server, but if it is not >>>>>> necessary, I'd like to avoid this step and do the php trick. >>>>>> >>>>>> pon, 12. srp 2021. u 20:30 Ben Goertzel <[email protected]> napisao >>>>>> je: >>>>>> >>>>>>> A reasonable step would be for Nil to send you some real PLN and URE >>>>>>> inference histories and see what your visualizer does with them... >>>>>>> >>>>>>> On Mon, Jul 12, 2021, 10:59 AM Ivan V. <[email protected]> wrote: >>>>>>> >>>>>>>> I made a small infinity test <http://ocog.atspace.cc/infinite/> >>>>>>>> too. Each parent virtually has an infinite number of children. Rolling >>>>>>>> ovals around, zooming ovals in, zooming ovals out, ... Surely it's not >>>>>>>> exactly perfect, but I could live with it. >>>>>>>> >>>>>>>> pon, 12. srp 2021. u 17:48 Linas Vepstas <[email protected]> >>>>>>>> napisao je: >>>>>>>> >>>>>>>>> Hi Ivan, >>>>>>>>> >>>>>>>>> On Mon, Jul 12, 2021 at 6:00 AM Ivan V. <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thank you for asking, and my thoughts are pretty obvious. As I >>>>>>>>>> understand, URE and PLN are all about proofs, so my thoughts may go >>>>>>>>>> in that >>>>>>>>>> direction. Suppose we have a natural deduction proof composition: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> * --- --- --- --- --- --- --- --- --- I >>>>>>>>>> J K L M N P Q R ----------------- >>>>>>>>>> ----------------- ----------------- A B >>>>>>>>>> >>>>>>>>>> C----------------------------------------------------------- >>>>>>>>>> X* >>>>>>>>>> >>>>>>>>>> You can already see the tree-like composition, but as it may span >>>>>>>>>> over a very wide and tall area, it may be required to represent it >>>>>>>>>> within >>>>>>>>>> an on-demand scaling system. This example >>>>>>>>>> <http://ocog.atspace.cc/> roughly shows what I have imagined for >>>>>>>>>> proof representation. In the example you can play with ovals, >>>>>>>>>> dragging them >>>>>>>>>> around and in or out the central area, zooming proof parts of the >>>>>>>>>> current >>>>>>>>>> interest. Notice how it is possible to represent and navigate nearly >>>>>>>>>> infinite length proofs, assuming enough memory space. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Re: navigating trees: if you don't already know this, then I >>>>>>>>> suggest that you really, really should study hyperbolic rotations aka >>>>>>>>> mobius transformations on the poincare disk. They implement your >>>>>>>>> example. >>>>>>>>> I recall seeing a demo of this at SIGGRAPH two or three decades ago. >>>>>>>>> As you >>>>>>>>> pan around on the hyperbolic disk, different parts of the graph get >>>>>>>>> magnified at the center. And, like an MC Escher print, the rest of the >>>>>>>>> graph remains compressed at the edges. >>>>>>>>> >>>>>>>>> For scale-free networks, this doesn't work. And from what I can >>>>>>>>> tell, learning really does result in something close to scale-free >>>>>>>>> networks. What this means in practice is that there's one vertex >>>>>>>>> with a >>>>>>>>> million edges coming off of it. There are two, with half-a-million >>>>>>>>> each. >>>>>>>>> Four, with a quarter-million each, and so on. So almost all vertexes >>>>>>>>> have >>>>>>>>> just a handful of edges connected to them, but as you move around, >>>>>>>>> from >>>>>>>>> vertex to vertex, you bump into these monsters. And you can't really >>>>>>>>> draw >>>>>>>>> them: try drawing a vertex with a thousand edges on your 2Kx2K >>>>>>>>> monitor: >>>>>>>>> most of those edges will be less than one pixel from each-other. >>>>>>>>> It'll be >>>>>>>>> just a big blob. >>>>>>>>> >>>>>>>>> It's important to "eat your own dog-food", as they say, or "smoke >>>>>>>>> your own dope": use your own code to solve actual, real-world >>>>>>>>> problems. >>>>>>>>> This very quickly highlights where all that beautiful theory doesn't >>>>>>>>> quite >>>>>>>>> work out in practice. >>>>>>>>> >>>>>>>>> --linas >>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "opencog" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/opencog/CAHrUA34HBqa02JkW9-EVR5OrpSkOWEMGjZBOCPM2vKKpJR2%2B0A%40mail.gmail.com >>>>>>>>> <https://groups.google.com/d/msgid/opencog/CAHrUA34HBqa02JkW9-EVR5OrpSkOWEMGjZBOCPM2vKKpJR2%2B0A%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "opencog" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/opencog/CAB5%3Dj6UgLR5xMP9WeE%2BWOkqBynGTr%2BNQTwsmUq9JrSuU1Sh1ZA%40mail.gmail.com >>>>>>>> <https://groups.google.com/d/msgid/opencog/CAB5%3Dj6UgLR5xMP9WeE%2BWOkqBynGTr%2BNQTwsmUq9JrSuU1Sh1ZA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "opencog" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/opencog/CACYTDBeOoqMRD20KFDYPGG1YfRqT0qW4wehOGtTKSqTb%3D6L2Jw%40mail.gmail.com >>>>>>> <https://groups.google.com/d/msgid/opencog/CACYTDBeOoqMRD20KFDYPGG1YfRqT0qW4wehOGtTKSqTb%3D6L2Jw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "opencog" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/opencog/88dc6d00-07a3-4421-9cc0-9e5e1a8d0052n%40googlegroups.com >>>> <https://groups.google.com/d/msgid/opencog/88dc6d00-07a3-4421-9cc0-9e5e1a8d0052n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "opencog" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/opencog/CAB5%3Dj6UWpdQOir0eZ2Xj%2B8rYW1s6DtCurK%2BvJ9qa2WoxL61MVg%40mail.gmail.com >>> <https://groups.google.com/d/msgid/opencog/CAB5%3Dj6UWpdQOir0eZ2Xj%2B8rYW1s6DtCurK%2BvJ9qa2WoxL61MVg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> >> >> -- >> Patrick: Are they laughing at us? >> Sponge Bob: No, Patrick, they are laughing next to us. >> >> >> -- > You received this message because you are subscribed to the Google Groups > "opencog" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/opencog/1738876c-09a3-42b7-8884-0b4a8449c250n%40googlegroups.com > <https://groups.google.com/d/msgid/opencog/1738876c-09a3-42b7-8884-0b4a8449c250n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- Patrick: Are they laughing at us? Sponge Bob: No, Patrick, they are laughing next to us. -- You received this message because you are subscribed to the Google Groups "opencog" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CAHrUA34P8Rith_M6ppEeh%2BwFfHyXwEf3Ln%2BZ%3DD0gmPuU9NE3NQ%40mail.gmail.com.
