Bill, I think that you should be able to design a reasonable loop with a delay to provide the needed flow control without modifying the underlying #primWrite:from:startingAt:count: semantics. That is exactly what I think you should do in this case. Be pragmatic about it; break your 1MB data into digestible pieces, set the pipe writer to be nonblocking, and loop with a delay until the whole thing gets digested by whatever is reading it on the other end of the pipe.
HTH, Dave On Sun, Aug 29, 2010 at 03:47:18PM -0400, Schwab,Wilhelm K wrote: > Dave, > > Why does > > primWrite: id from: stringOrByteArray startingAt: startIndex count: count > "Write count bytes onto this file from the given string or byte array > starting at the given index. Answer the number of bytes written." > > raise an error? Perhaps it or a related method should answer the number of > bytes written. There should be a compromise between locking up the VM and > having no idea how quickly the other side can accept data. Given a count of > what was written (as suggested in the comment, and perhaps zero bytes if the > other side is struggling), then one could design a reasonable loop with a > delay to provide the needed flow control. Otherwise, it's just guesswork. > > Bill > > > > ________________________________________ > From: [email protected] > [[email protected]] On Behalf Of David T. Lewis > [[email protected]] > Sent: Sunday, August 29, 2010 3:05 PM > To: [email protected] > Subject: Re: [Pharo-project] OSProcess - can it pipe 1 MB? > > It does not belong in OSProcess unless it's something that the underlying > operating system actually supports, which is not the case here. As far > as I know, there is no Unix system call for determining how much data a > pipe is ready to accept. > > You'll need to manage your flow control yourself. You may also need to > call #setNonBlocking on the pipe writer, otherwise you can block on > write and lock up the virtual machine(I think you know that already but > just in case). If the process at the other end of your pipe is keeping > up with the data, it won't be a problem, but if you try to write to a > full pipe without pulling something out of the other end, you will lock > up your VM completely unless the stream has been set to non-blocking > mode. > > Dave > > On Sun, Aug 29, 2010 at 02:28:50PM -0400, Schwab,Wilhelm K wrote: > > Dave, > > > > Sure, but then the command shell must then be doing something that > > OSProcess should do: transfer "large" (1MB isn't much these days) blobs in > > pieces that are manageable. The questions are where does it belong, and > > what if anything is necessary to ensure the pipe is ready for the next > > piece? > > > > One thought would be > > > > AttachableFileStream>>nextPutAll:blob > > | in | > > in := ReadStream on:blob. > > [ in atEnd ] whileFalse:[ > > self nextPutAll:( in next:32000). > > ]. > > > > but it should run into the same problem after a few times through the loop. > > It needs some "flow control" and could easily belong elsewhere in the > > hierarch?? The above assumes the (should be changed) silent truncation of > > #next:. > > > > Bill > > > > > > ________________________________________ > > From: [email protected] > > [[email protected]] On Behalf Of David T. Lewis > > [[email protected]] > > Sent: Sunday, August 29, 2010 12:37 PM > > To: [email protected] > > Subject: Re: [Pharo-project] OSProcess - can it pipe 1 MB? > > > > On Sun, Aug 29, 2010 at 11:36:02AM -0400, Schwab,Wilhelm K wrote: > > > I am trying to pipe roughly 1 MB of data into a program on Linux; I know > > > it can handle the load, because I have tested it with by cat-ing a script > > > and the data. I do not yet promise that my Pharo code is correct; if the > > > program did not respond or returned junk, I would look to myself. But I > > > am getting errors suggesting that AttachableFileStream is failing on > > > writing to the pipe, and it's very suspicious that it simply can't handle > > > the size of the string. Any ideas? > > > > > > Bill > > > > Try googling "maximum size of write to linux pipe". Unix pipes > > have a capacity limit, and the write will fail if you try to write > > too large a chunk all at once. > > > > Dave > > > > > > _______________________________________________ > > Pharo-project mailing list > > [email protected] > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > _______________________________________________ > > Pharo-project mailing list > > [email protected] > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > _______________________________________________ > Pharo-project mailing list > [email protected] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > _______________________________________________ > Pharo-project mailing list > [email protected] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list [email protected] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
