The "best practice" here is to prefer pull over push. The remote daemon will always knbow best r -----Original Message----- From: Tim Caswell <[email protected]> Sender: [email protected] Date: Thu, 23 Feb 2012 09:00:30 To: <[email protected]> Reply-To: [email protected] Subject: Re: [nodejs] Best practice for pushing lots of data over TCP to a distant server
Tcp itself handles a lot of the connection issues, so it's fairly reliable. You can still get disconnects at this level and you'll need to write reconnect logic if you want to be that robust. As far as bandwidth throttling, use Stream.prototype.pipe which implements "backpressure". That is, it stops reading from the source when the destination is full and not writable. When the dest is ready for more data, it resumes the stream. This way you won't end up buffering all the data in memory in the likely case that the source can read data faster than the remote dest can receive it. On Wed, Feb 22, 2012 at 9:30 PM, Jules <[email protected]> wrote: > Daemon A needs to push lots of data to Daemon B, which is located on > another continent. The bandwidth available is usually quite high, but > it's highly variable and might drop out entirely at times. > > Is there an accepted "best strategy" for implementing this in Node? > > Does A just write as fast as it can to the socket and hope for the > best? Should it detect errors and re-connect, resuming from the last > offset that B successfully received? > > Does a socket ever become non-writeable? i.e. Does daemon A need to > catch "socket not writeable" exceptions and throttle itself to suit > the available bandwidth? > > I imagine some articles have been written on this topic, but I haven't > found them yet. > > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
