John, Thanks.
Can P2P transmission match the same roughly 27 1Kmsg/sec rate as the client to server upload you discribed? Ed Porter -----Original Message----- From: John G. Rose [mailto:[EMAIL PROTECTED] Sent: Thursday, November 29, 2007 11:40 PM To: agi@v2.listbox.com Subject: RE: Hacker intelligence level [WAS Re: [agi] Funding AGI research] OK for a guestimate take a half-way decent cable connection say Comcast on a good day with DL of 4mbits max and UL of 256kbits max with an undiscriminated protocol, an unknown TCP based protocol, talking to a fat-pipe, low latency server. Assume say 16 byte message header wrappers for all of your 128, 1024 and 10k byte message sizes. So upload is 256kbits, go ahead and saturate it fully with either of your 128+16bytes, 1024+16bytes, and 10k+16bytes packet streams. Using TCP for reliability and assume some overhead say subtract 10% from the saturated value, retransmits, latency. What are we left with? Assume the PC has 1gigbit NIC so it is usually waiting to squeeze out the 256kbits of cable upload capacity. Oh right this is just upstream, DL is 4mbits cable into PC NIC or 1gigbit (assume 60% saturation) so there is > ample PC NIC BW for this. ... So for 256kbits/sec = 256,000 bits/sec, (256,000 bits/sec) / ((1024 + 16)bytes x 8bits/ (message bytes)) = 30.769 messages / sec. So 30.769 messages/sec - 10% = 27.692 messages /sec. About 27.692 message per sec for the 1024 byte message upload stream. Download = 16x UL = 443.072 messages/sec My calculation look right? Note: some Comcast cable connections allow as much as 1.4mbits upload. UL is always way less than DL (dependant on protocol). Other cable companies are similar depends on the company and geographic region... John > -----Original Message----- > From: Ed Porter [mailto:[EMAIL PROTECTED] > Sent: Thursday, November 29, 2007 6:50 PM > To: agi@v2.listbox.com > Subject: RE: Hacker intelligence level [WAS Re: [agi] Funding AGI > research] > > John, > > Somebody (I think it was David Hart) told me there is a shareware > distributed web crawler already available, but I don't know the details, > such as how good or fast it is. > > How fast could P2P communication be done on one PC, on average both > sending > upstream and receiving downstream from servers with fat pipes? Roughly > how > many msgs a second for cable connected PC's, say at 128byte and > 1024byte, > and 10K byte message sizes? > > Decent guestimates on such numbers would help me think about what sort > of > interesting distributed NL learning tasks could be done with by AGI-at- > Home > network. (of course once it showed any promise Google would start doing > it a > thousand times faster, but at least it would be open source). > > Ed Porter > > > -----Original Message----- > From: John G. Rose [mailto:[EMAIL PROTECTED] > Sent: Thursday, November 29, 2007 8:31 PM > To: agi@v2.listbox.com > Subject: RE: Hacker intelligence level [WAS Re: [agi] Funding AGI > research] > > Ed, > > That is the http protocol, it is a client server request/response > communication. Your browser asked for the contents at > http://www.nytimes.com. The NY Times server(s) dumped the response > stream > data to your external IP address. You probably have a NAT'd cable > address > and NAT'ted again by your local router (if you have one). This > communication > is mainly one way - except for your original few bytes of http request. > For > a full ack-nack real-time dynamically addressed protocol there is more > involved but say OpenCog could be setup to act as an http server and you > could have a http client (browser or whatever) for simplicity in > communications. Http is very firewall friendly since it is universally > used > on the internet. > > A distributed web crawler is a stretch though.... the communications is > more > complicated. > > John > > > -----Original Message----- > > From: Ed Porter [mailto:[EMAIL PROTECTED] > > Sent: Thursday, November 29, 2007 6:13 PM > > To: agi@v2.listbox.com > > Subject: RE: Hacker intelligence level [WAS Re: [agi] Funding AGI > > research] > > > > John, > > > > Thank you for the info. > > > > I just did a rough count of all the "IMG SRC="http://"; in the source > of > > the > > NYTimes home page which down loaded to my cable-modem connected > computer > > in > > about 3 seconds. I counted roughly 50 occurrences of that string. I > > assume > > there a many other downloaded files such as for layout info. Lets > guess > > a > > total of at least 100 files that have to be requested and downloaded > and > > displayed. That would be about 33 per second. So what could one do > with > > a > > system that could do on average about 20 accesses a second on a > > sustained > > rate, if a user was leaving it one at night as part of an OpenCog-at- > > Home > > project. > > > > It seems to me that that would be enough for some interesting large > > corpus > > NL work in conjunction with a distributed web crawler. > > > > Ed Porter > > > > > > -----Original Message----- > > From: John G. Rose [mailto:[EMAIL PROTECTED] > > Sent: Thursday, November 29, 2007 7:27 PM > > To: agi@v2.listbox.com > > Subject: RE: Hacker intelligence level [WAS Re: [agi] Funding AGI > > research] > > > > > From: Ed Porter [mailto:[EMAIL PROTECTED] > > > As I have said many before, to have brain-level AGI I believe you > need > > > within several orders of magnitude the representational, > > computational, > > > and > > > interconnect capability of the human mind. > > > > > > If you had 1 million PC bots on the web, the representational and > > > computational power would be there. But what sort of interconnect > > would > > > you > > > have? What is the average cable box connected computers upload > > > bandwidth? > > > > > > Is it about 1MBit/sec? If so that would be a bandwidth of > 1TBit/sec. > > > But > > > presumably only a small percent of that total 1TBit/sec could be > > > effectively > > > used, say 100Gbits/sec. That's way below brain level, but it is high > > > enough > > > to do valuable AGI research. > > > > > > But would even 10% of this total 1Tbit/sec bandwidth be practically > > > available? > > > > > > How many messages a second can a PC upload a second at say 100K, > 10K, > > > 1K, > > > and 128 bytes each? Does anybody know? > > > > > > I've gone through all this while being in VOIP R&D. MANY different > > connections at many different bandwidths, latencies, QOS, it's dirty > > across > > the board. Communications between different points is very non- > > homogenous. > > There are "deep" connections and "surface" alluding to deep web and > > surface > > web though network topology is somewhat independent of permissions. > The > > physical infrastructure of the internet allows for certain extremely > > high > > bandwidth, low latency connections where the edge is typically lower > > bandwidth, higher latency but it does depend on the hop graph, time of > > day, > > etc.. > > > > Messages per sec depends on many factors - network topology starting > > from pc > > bus, to NIC, to LAN switch and router, to other routers to ISPs, > between > > ISPs, back in other end, etc.. A cable box usually does anywhere from > > 64kbit > > to 1.4mbit upload depending on things such as provider, protocol, hop > > distance, it totally depends... usually a test is required. > > > > > > > On the net, can one bot directly talk to another bot, or does the > > > communication have to go through some sort of server (other than > those > > > provided gratis on the web, such as DNS servers)? > > > > > > If two bots send messages to a third bot at the same time, does the > > net > > > infrastructure hold the second of the conflicting messages until the > > > first > > > has been received, or what? > > > > This is called protocol and there are many - see RFCs and ITU for > > standards > > but better ones are custom made. There are connectionless and > connection > > oriented protocols, broadcast, multicast, C/S, P2P, etc.. Existing > > protocol > > standards can be extended, piggybacked or parasited. > > > > Bots can talk direct or go through a server using or not using DNS. > Also > > depends on topology - is one point (or both) behind a NAT? > > > > Message simultaneity handling is dependent on protocol. > > > > > > > To me the big hurdle to achieving the equivalent of SETI-at-home AGI > > is > > > getting the bandwidth necessary to allow the interactive computing > of > > > large > > > amounts of knowledge. If we could solve that problem, then it should > > be > > > pretty easy to get some great tests going, such as with something > like > > > OpenCog. > > > > Like I was saying before - better to design based on what you have to > > work > > with than trying to do something like fit the human brain design on > the > > "unbounded nondeterministic" internet grid. I'm not sure though what > the > > architecture of OpenCog looks like... > > > > John > > > > > > > > ----- > > This list is sponsored by AGIRI: http://www.agiri.org/email > > To unsubscribe or change your options, please go to: > > http://v2.listbox.com/member/?&; > > > > ----- > > This list is sponsored by AGIRI: http://www.agiri.org/email > > To unsubscribe or change your options, please go to: > > http://v2.listbox.com/member/?&; > > ----- > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; > > ----- > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?&; ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=70890062-7f8e5f
