[Monotone-devel] Re: Automate stdio chunk size

2006-07-10 Thread Bruce Stephens
Thomas Moschny [EMAIL PROTECTED] writes: [...] So are there any objections against setting constants::automate_stdio_size to, say, 1MB? Is there anyone('s application) depending on smaller chunks? Should we increase the automate format version number? How about makeing the chunk size

Re: [Monotone-devel] Re: Automate stdio chunk size

2006-07-10 Thread Thomas Moschny
On Monday 10 July 2006 14:45, Bruce Stephens wrote: How about makeing the chunk size settable using a new command (leaving the default as it is)? Or set the default to 1M (or BUFSIZ, or something), and then clients that would deadlock have a way to set it to something smaller. I'm not sure

Re: [Monotone-devel] Re: Automate stdio chunk size

2006-07-10 Thread Thomas Keller
Thomas Moschny wrote After thinking a while about it, it is no longer clear to me, why there is a need for chunked output *at all* ... The reading side of a pipe can always read the data in arbitrarily (and independently of the sender) sized packets, even when using synchronous I/O, by

Re: [Monotone-devel] Re: Automate stdio chunk size

2006-07-10 Thread Timothy Brownawell
On Mon, 2006-07-10 at 16:09 +0200, Thomas Keller wrote: Thomas Moschny wrote After thinking a while about it, it is no longer clear to me, why there is a need for chunked output *at all* ... The reading side of a pipe can always read the data in arbitrarily (and independently of

Re: [Monotone-devel] Re: Automate stdio chunk size

2006-07-10 Thread Thomas Keller
We can't use an in-stream EOF token, because the stream should be binary-safe. So this means prefixing each data chunk with the size of that chunk. A chunk is output when it reaches the maximum size (because having a known maximum size seems convenient), or when the stream is flushed (my

[Monotone-devel] Re: Automate stdio chunk size

2006-07-10 Thread Bruce Stephens
Thomas Keller [EMAIL PROTECTED] writes: Well, the EOF token wouldn't really have to be '\0', just something a parser could distinguish from the normal output flow. F.e. in emails the header is separated from the body by double newlines \n\n. If basic_io would become standard for all output of

Re: [Monotone-devel] Re: Automate stdio chunk size

2006-07-10 Thread Timothy Brownawell
On Mon, 2006-07-10 at 18:50 +0200, Thomas Keller wrote: We can't use an in-stream EOF token, because the stream should be binary-safe. So this means prefixing each data chunk with the size of that chunk. A chunk is output when it reaches the maximum size (because having a known maximum

Re: [Monotone-devel] Re: Automate stdio chunk size

2006-07-10 Thread Thomas Moschny
On Monday 10 July 2006 17:48 Timothy Brownawell wrote: There are changes to inventory in the works, that would require changing the interface version anyway, perhaps we should increase the chunk size at the same time we land that? Yes. And I think we should change the docs (for the new

Re: [Monotone-devel] Re: Automate stdio chunk size

2006-07-10 Thread Nathaniel Smith
On Mon, Jul 10, 2006 at 08:17:36PM +0200, Thomas Moschny wrote: On Monday 10 July 2006 17:48 Timothy Brownawell wrote: There are changes to inventory in the works, that would require changing the interface version anyway, perhaps we should increase the chunk size at the same time we land

Re: [Monotone-devel] Re: Automate stdio chunk size

2006-07-10 Thread Nuno Lucas
On 7/10/06, Nathaniel Smith [EMAIL PROTECTED] wrote: On Mon, Jul 10, 2006 at 08:17:36PM +0200, Thomas Moschny wrote: The point of having an upper-limit is to put an upper bound on how much memory monotone has to use. 1M seems a bit large for this purpose, and I'm astonished if you actually have