Hi, Javascript *could* manage millions of data particles, but I suspect they have to be stored as a single low-level ByteArray, not to bloat javascript variables and structured array space.
If you are eager to use C, there are some tools (I think one of them called emscripten) that compile C++ code to Webassembly to be run inside a browser. I've seen screenshots of an old 3D Doom game from 90-es and other graphical C++ DOS apps running happily inside a browser, I think even a DOS emulator that supports graphic modes. The whole thing should achieve only 2x slowdown (as they measured) compared to native running in dos box. The point is - the most of what you can run natively, you can run inside a browser with merely 2x slowdown. The only bottleneck would be an Internet connection bandwidth. Personally, I don't have any experience with emscripten, but I guess it shouldn't be too complicated to get it running. It should be only a matter of recompiling existing C++ code. But to return to javascript - although millions of atoms could be theoretically efficiently stored in a single bytearray, Internet connection bandwidth could be a problem because maybe we are talking about hundreds of megabytes. The logical step would be to prepare a CogServer instance filled with those millions of atoms, keep it always running, and then query only what is of the current interest to forward it to a browser. Anyway, who would browse over millions of atoms all at once? One might only be interested in some subset of it, and if that subset can be measured in thousands of atoms, I think javascript should be able to grock it (I tested CogProtoLab on ten thousand lines from your web site, the thing loaded in about 20 seconds on my celeron, but I think I could make it faster if necessary). Then CogServer could be queried by php to extract only what is of the current interest, then AJAX it to a client browser. The only remaining question would be what do we want to show, and how do we want to show it. Do you have any basic glimpse of a kind of visualization you'd like to have? And what user interactions would pair it to be successful? - ivan - sub, 5. ožu 2022. u 20:36 Linas Vepstas <[email protected]> napisao je: > Hi, > > On Sat, Mar 5, 2022 at 12:00 AM Ivan V. <[email protected]> wrote: > >> Hi Linas, >> >> Linas wrote: >> > And, while I have your attention, something completely off-topic, but: >> in the https://github.com/opencog/learn project I have large datasets >> with interesting info in them. I have plenty of ways of seeing this myself, >> but I can't think of any easy way to expose what I see to other people. One >> of the datasets shows relationships between tens of thousands of words, and >> I'm wondering: how can I open this up to some interactive web query or >> browser or chat, here you might ask "find similar words" or ... I dunno I'm >> not even sure how to explain it. The project is going very well, I'm >> getting good results, but I can't figure out how to make it exciting for >> anyone else but me... >> >> If it is only tens of thousands of nodes, I believe they can be >> successfully loaded into a web browser on a faster computer (in between 30 >> second or so to load, depending on how heavy javascript post-processing >> is). Maybe it would be a good idea to break intro examples to a few >> fast-loading ones just to get visitor attention, and a few heavier ones, >> once that you get the visitor attention. >> > > Unfortunately, no. The data sets mostly all have ten to 20 thousand words > in the English language; but to describe the various relationships between > these words requires millions of atoms. The smallest datasets have maybe > only 1 or 2 million atoms; usually, they have 10 to 20 million. > > >> >> Some obvious technical questions I can think of are: >> >> - Do you have any specific data examples? >> >> > I have many hundreds of datasets... varying content, varying quality, > varying parameters. > >> >> - What data structures are you interested in? >> >> Well, there's a big difference between "what I am interested in" and > "what might be impressive to curious onlookers". What I am interested in > is well-taken care of: I have a huge assortment of scripts and tools to > view and extract what I'm interested in. So that's not the problem. > > The problem is this: I feel like I'm making good progress, getting great > results, but I don't know how to communicate that in a way that anyone else > can see. That includes people like Ben, but also people like yourself: > there is the "usual band of suspects" -- the people reading this email, > maybe a dozen past participants, maybe another dozen onlookers. What can > be demoed that might catch their attention, their imagination? > >> >> - What kind of query language can be used to extract those structures? >> >> I've got a large variety of tools. All of them wrap the query engine in > some way, hiding the low-level details. > > The issue is that "those structures" are themselves made out of thousands > or tens of thousands of atoms at a time, so its like "here's a blob of a > thousand atoms, look at how it relates to that one over there" -- where > each blob represents a single word of the English language. > > A long long time ago, there was an opencog chatbot, and people did enjoy > connecting to it and fooling around with it. I wish I could do that again, > but the current datasets are not capable of supporting chat. At this time, > the most you can ask for is for relationships between various words, for > statistics about them. Well, and phrases, too, but that is a bit rockier, > right now. > >> >> - How close those structures are to s-expressions? >> >> Everything in the atomspace is an s-expression.... but you know this, of > course. But atoms are very low-level. It is the assemblage of atoms, the > network that matters. How they attach to one-another. That's where the > intelligence lies. > > -- Linas > >> >> All well, >> - ivan - >> >> -- >> You received this message because you are subscribed to the Google Groups >> "opencog" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/opencog/CAB5%3Dj6V1wjTSy-n552MzXaBQTGKrcEETTa%2B6EzAx%2BVhymA6JHw%40mail.gmail.com >> <https://groups.google.com/d/msgid/opencog/CAB5%3Dj6V1wjTSy-n552MzXaBQTGKrcEETTa%2B6EzAx%2BVhymA6JHw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > > > -- > Patrick: Are they laughing at us? > Sponge Bob: No, Patrick, they are laughing next to us. > > > -- > You received this message because you are subscribed to the Google Groups > "opencog" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/opencog/CAHrUA36KSBL4jYb%3DPtbvSHfbYD5Gt%2BE%2B7s89JR1Ay6jawth7Zg%40mail.gmail.com > <https://groups.google.com/d/msgid/opencog/CAHrUA36KSBL4jYb%3DPtbvSHfbYD5Gt%2BE%2B7s89JR1Ay6jawth7Zg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "opencog" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CAB5%3Dj6WaJTrYYuMPD0x6cza5D2EmWCAa_xXkHTnJRmHKAKpxJA%40mail.gmail.com.
