Re: [Bitcoin-development] A bitcoin UDP P2P protocol extension
On Saturday, March 23, 2013 10:42:26 AM Randy Willis wrote: Introducing super-nodes with thousands of connected peers can greatly help here. UDP is connectionless. I would hope any UDP bitcoin protocol doesn't try to emulate a connection. :/ Luke -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] A bitcoin UDP P2P protocol extension
On Sat, Mar 23, 2013 at 10:52 AM, Luke-Jr l...@dashjr.org wrote: On Saturday, March 23, 2013 10:42:26 AM Randy Willis wrote: Introducing super-nodes with thousands of connected peers can greatly help here. UDP is connectionless. I would hope any UDP bitcoin protocol doesn't try to emulate a connection. :/ It depends on the usage. Simply broadcasting a TX or INV to a remote peer does not require a connection, clearly... but you probably want to signal acceptance of those messages somehow. But other uses, like subscribing to a broadcast, does require some notion of an association. In the rough draft, a parallel TCP connection with version/verack sequence is required, and you may make use of it if a connection is needed. But that is just one approach. A more robust, heavyweight UDP P2P might be a hole-punching TCP alternative. It's up to the community and results of experimentation. Bittorrent has evolved a full transfer protocol over UDP, to get around firewalls and the like. -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Upcoming network event: block v2 lock-in
On Saturday, March 23, 2013 5:28:55 PM Jeff Garzik wrote: On Sat, Mar 23, 2013 at 1:09 PM, Luke-Jr l...@dashjr.org wrote: I don't think anyone is mining using bitcoind 0.7 or later? slush, BTC Guild, ozcoin too I think, several others. Not for producing coinbases (where BIP 34 is implemented). -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Upcoming network event: block v2 lock-in
On Sat, Mar 23, 2013 at 1:43 PM, Luke-Jr l...@dashjr.org wrote: On Saturday, March 23, 2013 5:28:55 PM Jeff Garzik wrote: On Sat, Mar 23, 2013 at 1:09 PM, Luke-Jr l...@dashjr.org wrote: I don't think anyone is mining using bitcoind 0.7 or later? slush, BTC Guild, ozcoin too I think, several others. Not for producing coinbases (where BIP 34 is implemented). Sure, that is largely the pool server layer. But it is misleading to imply that bitcoind is nowhere in the stack. -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Upcoming network event: block v2 lock-in
On Saturday, March 23, 2013 5:47:46 PM Jeff Garzik wrote: On Sat, Mar 23, 2013 at 1:43 PM, Luke-Jr l...@dashjr.org wrote: On Saturday, March 23, 2013 5:28:55 PM Jeff Garzik wrote: On Sat, Mar 23, 2013 at 1:09 PM, Luke-Jr l...@dashjr.org wrote: I don't think anyone is mining using bitcoind 0.7 or later? slush, BTC Guild, ozcoin too I think, several others. Not for producing coinbases (where BIP 34 is implemented). Sure, that is largely the pool server layer. But it is misleading to imply that bitcoind is nowhere in the stack. Context is everything. bitcoind is nowhere in the implementation of the miner end of BIP 34. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Upcoming network event: block v2 lock-in
On Saturday, March 23, 2013 6:21:32 PM Jeff Garzik wrote: On Sat, Mar 23, 2013 at 1:51 PM, Luke-Jr l...@dashjr.org wrote: bitcoind is nowhere in the implementation of the miner end of BIP 34. Again, not strictly true. bitcoind's 'getblocktemplate' RPC call used by some supplies the block version, selects transactions for the miner, and other tasks integral to the implementation of the miner and BIP 34. Transaction selection and everything else bitcoind does, is irrelevant to BIP 34. It is incompatible with getblocktemplate for coinbase-creating software to produce v2 blocks without implementing BIP 34 themselves, even if the upstream GBT server specifies it. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] A bitcoin UDP P2P protocol extension
On 03/23/2013 11:24 AM, Jeff Garzik wrote: On Sat, Mar 23, 2013 at 10:52 AM, Luke-Jr l...@dashjr.org wrote: On Saturday, March 23, 2013 10:42:26 AM Randy Willis wrote: Introducing super-nodes with thousands of connected peers can greatly help here. UDP is connectionless. I would hope any UDP bitcoin protocol doesn't try to emulate a connection. :/ It depends on the usage. Simply broadcasting a TX or INV to a remote peer does not require a connection, clearly... but you probably want to signal acceptance of those messages somehow. But other uses, like subscribing to a broadcast, does require some notion of an association. In the rough draft, a parallel TCP connection with version/verack sequence is required, and you may make use of it if a connection is needed. But that is just one approach. A more robust, heavyweight UDP P2P might be a hole-punching TCP alternative. It's up to the community and results of experimentation. Bittorrent has evolved a full transfer protocol over UDP, to get around firewalls and the like. Bittorrent uses UDP in 2 ways and for different reasons. The tracker protocol is now UDP because large trackers are under such enormous strain from short lived HTTP connections (40Gb/s) that there have been instances of upstream routers becoming overloaded from the storm of SYN, ACK and FIN messages. UDP helps solve that. The inter-peer protocol is now UDP because TCP does not play nice in the context of bufferbloat and Bittorrent needs lots of active TCP connections to work, exacerbating the problem. In this instance Bittorrent uses a full userspace TCP stack which just sends w/ UDP. +1 for experimenting with UDP, we might learn some interesting things. It's worthwhile to actually speed test UDP v. TCP because the time to send an INV on an established TCP connection with Nagle disabled may not be significantly longer than that for sending with UDP. Also +1 for experimentation with sending a small transaction instead of an INV, if INVs are not being grouped because we want the fastest possible network propagation, they are mostly overhead anyway. If b/w is more important than propagation speed then of course TCP/Nagle is the way to go. Thanks, Caleb -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] A bitcoin UDP P2P protocol extension
If you're considering a datagram protocol, you might be interested in some more modern alternatives to UDP: UDT: Breaking the Data Transfer Bottleneck http://udt.sourceforge.net/ Stream Control Transmission Protocol http://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol On Sat, Mar 23, 2013 at 12:17 AM, Jeff Garzik jgar...@exmulti.com wrote: Here is a rough draft implementation of a UDP P2P protocol extension for bitcoin: https://github.com/jgarzik/bitcoin/tree/udp http://yyz.us/bitcoin/udp-v0.patch Protocol specification (such that it is): - UDP, bound to same port as TCP P2P (normally 8333) - Active, simultaneous TCP P2P connection required (useful against DoS and other attacks) - Same message format as TCP P2P, same pchMessageStart conventions, etc. (my CNetMessage pull req would be helpful here) - Multiple P2P messages per UDP packet permitted - Max UDP packet size 100*1024 bytes - Advertises NODE_UDP in nServices - New getudpcook TCP P2P command returns a udpcook message, containing a yummy cookie - UDP P2P commands inv, tx and addr are handled as if received via TCP - UDP P2P command udpsub sets a mask, that subscribes to one or more data broadcasts. - When USM_INV_BCAST mask bit is set, receive inv messages via UDP rather than TCP. Project and design goals (or, why do this?): - It is theorized that UDP may be useful for some messages we broadcast throughout the network - A productive discussion STARTS with code, otherwise discussion continues forever. - Investigate improving inv relay speed - Investigate unconditional tx broadcasting via UDP, as inv alternative, for small tx's. - Investigate improving block relay speed (or perhaps block header relay speed) - Open up new design avenues, for P2P patterns more suited to UDP than TCP in general. -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] A bitcoin UDP P2P protocol extension
My first concern was that I and about everyone else only has TCP/UDP port forwarding, but at least for the first: UDT uses UDP to transfer bulk data with its own reliability control and congestion control mechanisms. Multiple UDT flows can share a single UDP port, thus a firewall can open only one UDP port for all UDT connections. The latter appears not so friendly to NAT. On 3/23/2013 3:30 PM, Mark Friedenbach wrote: If you're considering a datagram protocol, you might be interested in some more modern alternatives to UDP: UDT: Breaking the Data Transfer Bottleneck http://udt.sourceforge.net/ Stream Control Transmission Protocol http://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] A bitcoin UDP P2P protocol extension
On Sat, Mar 23, 2013 at 5:57 PM, Jay F j...@outlook.com wrote: My first concern was that I and about everyone else only has TCP/UDP port forwarding, You tunnel it! http://tools.ietf.org/html/draft-tuexen-tsvwg-sctp-dtls-encaps-00 You could do worse to have a data stream that looks like WEBRTC traffic… In some ways SCTP is a pretty optimal transport for Bitcoin like messaging. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] A bitcoin UDP P2P protocol extension
On Sat, Mar 23, 2013 at 9:22 PM, Gregory Maxwell gmaxw...@gmail.com wrote: In some ways SCTP is a pretty optimal transport for Bitcoin like messaging. Darn near everything can be shoehorned into a message So absolutely agreed... in theory. Been an SCTP fan for years. Firewall practices tend to put a damper on cool new protocols like that, though. -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] A bitcoin UDP P2P protocol extension
Nearly all of these new(er) user-mode transports run over UDP, so you can hole-punch and port forward just the same. Some which don't can nevertheless be tunneled, to the same effect. Ultimately I don't have any skin in this game though. Just trying to save someone from reinventing a perfectly good wheel ;) On Sat, Mar 23, 2013 at 5:57 PM, Jay F j...@outlook.com wrote: My first concern was that I and about everyone else only has TCP/UDP port forwarding, but at least for the first: UDT uses UDP to transfer bulk data with its own reliability control and congestion control mechanisms. Multiple UDT flows can share a single UDP port, thus a firewall can open only one UDP port for all UDT connections. The latter appears not so friendly to NAT. On 3/23/2013 3:30 PM, Mark Friedenbach wrote: If you're considering a datagram protocol, you might be interested in some more modern alternatives to UDP: UDT: Breaking the Data Transfer Bottleneck http://udt.sourceforge.net/ Stream Control Transmission Protocol http://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
