How about bundling it in parts ? On Mon, Jul 29, 2013 at 6:58 AM, Angel Java Lopez <[email protected]> wrote: > No experience.... but I would measure the file system time in serving your > application. > > You could store the files in a SSD disk (a "flash" disk, sorry, I'm not a > hardware guy). Comparing times will give you more context about the issue. > > > On Mon, Jul 29, 2013 at 10:48 AM, Matt <[email protected]> wrote: >> >> Did you think about a possibility of 4) redesign your application so it's >> not loading 10k modules? That's an INSANE amount of dependencies. Even >> making them parallel in some way is still going to require thousands of >> syscalls. In general I don't think it's going to be a priority for node to >> optimise for startup time, but I could be wrong - I'm not a core team >> member. >> >> >> On Mon, Jul 29, 2013 at 9:28 AM, Benjamin Pasero >> <[email protected]> wrote: >>> >>> Hi, >>> >>> in my environment it is very important to startup a node.js server >>> instance in a very short time. Now, my server has many dependencies (around >>> 10.000 files in node_modules). My measurements show that the pure fact that >>> node.js resolves all require() calls from the entry point is making the >>> startup very slow. In this case, around 500 files are being loaded through >>> require(). The problem is even worse for me when I host node.js from a UNC >>> share where loading files one by one is a real bottleneck. >>> >>> I would like to ask the community if there is any idea for a solution >>> that would speed up require() in node.js. So far I had two ideas: >>> >>> parallelize fs operations in require() >>> allow to resolve require() calls from a zip file >>> implement true async module loading through requireJS or similar >>> >>> Option 1 is the obvious: Instead of fs.readFileSync each required file >>> one by one, do them all in one batch and collect the results. This should >>> increase the performance by an order of magnitude, especially if the disk is >>> slow (as in my UNC case). The only drawback from this solution I could see >>> is that it would no longer be possible to overwrite the behavior of the >>> require() function from within a dependency because file loading would >>> happen in parallel. But this seems like not a typical use case. >>> >>> Option 2 would allow to specify an archive file to resolve require() >>> against. This seems like actually supporting commonjs packages in zip >>> archive form (http://wiki.commonjs.org/wiki/Packages/1.0). The idea is that >>> my entire node_modules folder could be one zip file that gets loaded into >>> memory on startup (at least the file/folder structure) and all require() >>> calls resolve against the archive. >>> >>> Option 3 is something I was hoping to get already using the requireJS >>> node module. However they make clear that module loading is sync to the >>> execution context. So this is not an option currently, but maybe there is a >>> plugin that does this? >>> >>> Any other ideas are greatly appreciated. Maybe there is even a solution >>> out there I could benefit from ? >>> Thanks, >>> Ben >>> >>> -- >>> -- >>> Job Board: http://jobs.nodejs.org/ >>> Posting guidelines: >>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >>> You received this message because you are subscribed to the Google >>> Groups "nodejs" group. >>> To post to this group, send email to [email protected] >>> To unsubscribe from this group, send email to >>> [email protected] >>> For more options, visit this group at >>> http://groups.google.com/group/nodejs?hl=en?hl=en >>> >>> --- >>> You received this message because you are subscribed to the Google Groups >>> "nodejs" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>> >> >> >> -- >> -- >> Job Board: http://jobs.nodejs.org/ >> Posting guidelines: >> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >> You received this message because you are subscribed to the Google >> Groups "nodejs" group. >> To post to this group, send email to [email protected] >> To unsubscribe from this group, send email to >> [email protected] >> For more options, visit this group at >> http://groups.google.com/group/nodejs?hl=en?hl=en >> >> --- >> You received this message because you are subscribed to the Google Groups >> "nodejs" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> > > > -- > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > >
-- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
