John, Somebody (I think it was David Hart) told me there is a shareware distributed web crawler already available, but I don't know the details, such as how good or fast it is.
How fast could P2P communication be done on one PC, on average both sending upstream and receiving downstream from servers with fat pipes? Roughly how many msgs a second for cable connected PC's, say at 128byte and 1024byte, and 10K byte message sizes? Decent guestimates on such numbers would help me think about what sort of interesting distributed NL learning tasks could be done with by AGI-at-Home network. (of course once it showed any promise Google would start doing it a thousand times faster, but at least it would be open source). Ed Porter -----Original Message----- From: John G. Rose [mailto:[EMAIL PROTECTED] Sent: Thursday, November 29, 2007 8:31 PM To: agi@v2.listbox.com Subject: RE: Hacker intelligence level [WAS Re: [agi] Funding AGI research] Ed, That is the http protocol, it is a client server request/response communication. Your browser asked for the contents at http://www.nytimes.com. The NY Times server(s) dumped the response stream data to your external IP address. You probably have a NAT'd cable address and NAT'ted again by your local router (if you have one). This communication is mainly one way - except for your original few bytes of http request. For a full ack-nack real-time dynamically addressed protocol there is more involved but say OpenCog could be setup to act as an http server and you could have a http client (browser or whatever) for simplicity in communications. Http is very firewall friendly since it is universally used on the internet. A distributed web crawler is a stretch though.... the communications is more complicated. John > -----Original Message----- > From: Ed Porter [mailto:[EMAIL PROTECTED] > Sent: Thursday, November 29, 2007 6:13 PM > To: agi@v2.listbox.com > Subject: RE: Hacker intelligence level [WAS Re: [agi] Funding AGI > research] > > John, > > Thank you for the info. > > I just did a rough count of all the "IMG SRC="http://"; in the source of > the > NYTimes home page which down loaded to my cable-modem connected computer > in > about 3 seconds. I counted roughly 50 occurrences of that string. I > assume > there a many other downloaded files such as for layout info. Lets guess > a > total of at least 100 files that have to be requested and downloaded and > displayed. That would be about 33 per second. So what could one do with > a > system that could do on average about 20 accesses a second on a > sustained > rate, if a user was leaving it one at night as part of an OpenCog-at- > Home > project. > > It seems to me that that would be enough for some interesting large > corpus > NL work in conjunction with a distributed web crawler. > > Ed Porter > > > -----Original Message----- > From: John G. Rose [mailto:[EMAIL PROTECTED] > Sent: Thursday, November 29, 2007 7:27 PM > To: agi@v2.listbox.com > Subject: RE: Hacker intelligence level [WAS Re: [agi] Funding AGI > research] > > > From: Ed Porter [mailto:[EMAIL PROTECTED] > > As I have said many before, to have brain-level AGI I believe you need > > within several orders of magnitude the representational, > computational, > > and > > interconnect capability of the human mind. > > > > If you had 1 million PC bots on the web, the representational and > > computational power would be there. But what sort of interconnect > would > > you > > have? What is the average cable box connected computers upload > > bandwidth? > > > > Is it about 1MBit/sec? If so that would be a bandwidth of 1TBit/sec. > > But > > presumably only a small percent of that total 1TBit/sec could be > > effectively > > used, say 100Gbits/sec. That's way below brain level, but it is high > > enough > > to do valuable AGI research. > > > > But would even 10% of this total 1Tbit/sec bandwidth be practically > > available? > > > > How many messages a second can a PC upload a second at say 100K, 10K, > > 1K, > > and 128 bytes each? Does anybody know? > > > I've gone through all this while being in VOIP R&D. MANY different > connections at many different bandwidths, latencies, QOS, it's dirty > across > the board. Communications between different points is very non- > homogenous. > There are "deep" connections and "surface" alluding to deep web and > surface > web though network topology is somewhat independent of permissions. The > physical infrastructure of the internet allows for certain extremely > high > bandwidth, low latency connections where the edge is typically lower > bandwidth, higher latency but it does depend on the hop graph, time of > day, > etc.. > > Messages per sec depends on many factors - network topology starting > from pc > bus, to NIC, to LAN switch and router, to other routers to ISPs, between > ISPs, back in other end, etc.. A cable box usually does anywhere from > 64kbit > to 1.4mbit upload depending on things such as provider, protocol, hop > distance, it totally depends... usually a test is required. > > > > On the net, can one bot directly talk to another bot, or does the > > communication have to go through some sort of server (other than those > > provided gratis on the web, such as DNS servers)? > > > > If two bots send messages to a third bot at the same time, does the > net > > infrastructure hold the second of the conflicting messages until the > > first > > has been received, or what? > > This is called protocol and there are many - see RFCs and ITU for > standards > but better ones are custom made. There are connectionless and connection > oriented protocols, broadcast, multicast, C/S, P2P, etc.. Existing > protocol > standards can be extended, piggybacked or parasited. > > Bots can talk direct or go through a server using or not using DNS. Also > depends on topology - is one point (or both) behind a NAT? > > Message simultaneity handling is dependent on protocol. > > > > To me the big hurdle to achieving the equivalent of SETI-at-home AGI > is > > getting the bandwidth necessary to allow the interactive computing of > > large > > amounts of knowledge. If we could solve that problem, then it should > be > > pretty easy to get some great tests going, such as with something like > > OpenCog. > > Like I was saying before - better to design based on what you have to > work > with than trying to do something like fit the human brain design on the > "unbounded nondeterministic" internet grid. I'm not sure though what the > architecture of OpenCog looks like... > > John > > > > ----- > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; > > ----- > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?&; ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=70610395-775e15
