In this particular application, I do not have to worry too much about memory usage. The focus is to read as much data from the file system as possible, in one read operation.
I will try increasing the kPoolSize and recompile Node. I will update this thread with results. Thanks guys. - Gill On Friday, 3 August 2012 02:48:49 UTC-7, Dominic wrote: > > Gill, the kPool is a large buffer that ReadStream cuts buffers out of, > I think, to minimize the allocation of memory... > > I was reading through this the other day, and noticed that the default > buffer size is larger than the default kPoolSize. that seems wrong. > > you'd need to recompile node, of course, but making kPoolSize larger > should work. > > On Fri, Aug 3, 2012 at 8:50 AM, Gill <[email protected]> wrote: > > I think this is related: https://github.com/joyent/node/issues/2098 > > > > - Gill > > > > > > On Thursday, 2 August 2012 23:32:39 UTC-7, Gill wrote: > >> > >> Ben, thanks for the reply. I have a doubt that its just a hint, because > >> how come it is exactly 40960 bytes every time. The underlying > filesystem is > >> a custom coded one, which WILL return the exact number of bytes that > were > >> asked for. Line 38 for /lib/fs.js says: > >> > >> var kPoolSize = 40 * 1024; > >> > >> Do you think changing it to 128 * 1024 will change anything? > >> > >> - Gill > >> > >> On Thursday, 2 August 2012 16:29:43 UTC-7, Ben Noordhuis wrote: > >>> > >>> On Thu, Aug 2, 2012 at 9:21 PM, Gill <[email protected]> wrote: > >>> > I have a code where the NodeJS server reads a file and streams it to > >>> > response, it looks like: > >>> > > >>> > var fStream = fs.createReadStream(filePath, {'bufferSize': 128 * > >>> > 1024}); > >>> > fStream.pipe(response); > >>> > > >>> > The issue is, Node reads the file exactly 40960 bytes a time. > However, > >>> > my > >>> > app would be much more efficient (due to reasons not applicable to > this > >>> > question), if it reads 131072 (128 * 1024) bytes at a time. > >>> > > >>> > Is there a way to force Node to read 128 * 1024 bytes at a time from > >>> > the > >>> > stream? > >>> > > >>> > Thanks in advance! > >>> > >>> No. bufferSize is a hint, not an imperative. It's up to the operating > >>> system to honor it. > > > > -- > > Job Board: http://jobs.nodejs.org/ > > Posting guidelines: > > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > > You received this message because you are subscribed to the Google > > Groups "nodejs" group. > > To post to this group, send email to [email protected] > > To unsubscribe from this group, send email to > > [email protected] > > For more options, visit this group at > > http://groups.google.com/group/nodejs?hl=en?hl=en > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
