I confirm that the approach works really well. It's very easy to use and it
handles lots of dependencies very efficiently. Also, it works great if you
share files between client and server because the same "require" code works
on both sides.
I never took the time to polish my streamline-require library. It's good
enough for our own usage but it has some weaknesses, like the fact that it
extracts the require dependencies with a regexp (I've hit once a case where
it did work but the regexp used a lot of CPU because of heavy
backtracking). Also, my library does not handle some features of require
(if you give it a directory it handles the index.js default but it does not
go as far as looking up the main from the package.json file -- we never
needed this for client side modules).
So, it is good to see someone package a full-blown implementation. Go ahead
Saleem.
Bruno
On Thursday, April 19, 2012 9:16:33 PM UTC+2, Saleem Abdul Hamid wrote:
>
> Hi mgutz,
>
> Thanks for the feedback.
>
>> I'm not convinced that fetching every dependency as needed is more
>> efficient than loading one or two minified/compressed packages.
>>
> Half of the whole point is actually to not fetch every dependency as
> needed. You want to fetch exactly optimized minified/compressed packages
> (whether it is one or two or ten depends on your project) The other half of
> the point is to not:
>
> Pre-package your modules and let the browser already handle caching.
>
> What will you do if two modules have some shared dependencies and you
> don't know which one will be required first? Load two copies of the same
> code? How will you handle hand-packaging your dependencies into modules in
> an app with hundreds of modules? Why optimize something imperfectly and
> manually when it can be automated?
>
> It seems overly complicated.
>>
> This may be because I'm explaining how things work. For the developer, it
> is extremely simple. Just:
> module1 = require('module1');
> For a dependency that your ux requires to be available immediately,
> require('module2',function(err, module2){})
> for a module that your ux allows to have loading time.
>
> Have you looked at connect-assets, snockets
>>
> Connect-assets uses snockets, so I'll group them together. Besides what I
> mentioned above about shared dependencies and loading the same code twice,
> it doesn't appear that different modules are name-spaced separately, which
> is a shortcoming vs. just using requirejs for example. Also, declaring
> dependencies in comments is not that nice... what happens if you're using
> automatic documentation generation? And you're forcing upon yourself an
> inherent compile step, even when writing in javascript. And there's no way
> to reuse that same module on the server. These are a few shortcomings of
> that approach that are solved by mundlejs.
>
> Have you looked at stitch?
>
> Again, the same problem with shared dependencies, which, in case you
> think is not a serious problem, consider the case of a client-side
> framework, where some base classes might be contained in modules and would
> be in the dependency trees of every module leading to serious code
> duplication and maybe doubling, or more, the size of the code running in
> the browser.
>
> Have you looked at ...?
>
> I did extensively look for another module loader before starting mundlejs,
> because, as I said, I actually was using it for a project. I didn't find
> any. Since I posted here, I've discovered that there are at least two
> pre-existing libraries that approach the problem the way I think it should
> be approached and am trying to do with mundlejs-
> 1) Bruno's streamline library that he posted in this thread
> 2) YUI loader, which someone emailed to me in response to this thread
> The reasons I don't drop mundlejs and use one of these two are more
> taste/domain related than any fundamental difference of approach. Every
> other client-side loader/ pre-packager/bundler I think is pretty much of
> limited use.
>
>
> If you have a simple app with a few js files then mundlejs is not
> *necessary* for you. It's my goal to make it nice enough and have so little
> overhead that you still might use it just for the convenience. But if you
> have a single-page web app with >1MB of javascript, I think you really need
> something like mundlejs and that's actually the kind of project it was
> written to support.
>
>
> On Wednesday, April 18, 2012 9:46:52 AM UTC-7, mgutz wrote:
>>
>> I'm not sure how this the end result is different than the many libraries
>> out there. It seems overly complicated. Have you looked at connect-assets,
>> stitch, snockets ...? Pre-package your modules and let the browser already
>> handle caching. I'm not convinced that fetching every dependency as needed
>> is more efficient than loading one or two minified/compressed packages.
>> I've tried many AMD loaders and in the end our projects are simpler without
>> them. Stitch was trivial to patch to support multiple packages something
>> like `require(package/some/path)`
>>
>>
>> On Saturday, March 24, 2012 5:04:52 PM UTC-7, Saleem Abdul Hamid wrote:
>>>
>>> tl;dr - Client-side require with a server-side component that caches
>>> dependencies, bundles them, and caches the bundles. Need feedback on
>>> the concept, syntax. Need suggestions/contributions on implementation.
>>> Although, this works for me, it is almost just a proof-of-concept,
>>> needs work.
>>>
>>>
>>> As part of a project I'm working on, I spent a few hours writing a
>>> little client-side module loader with a server-side component enabling
>>> what I think is a pretty neat meaning to CommonJS module syntax. This
>>> morning I pulled it out of the rest of my project and attempted to
>>> package it in a useful way for others to use.
>>>
>>> The basic idea is this- in your client-side code, you can use require
>>> in either a "synchronous" or asynchronous fashion-
>>> module1 = require('some/path.js');
>>> require('some/other/path.js', function(err,result){module2 =
>>> result;});
>>>
>>> An asynchronous require makes a call to the server component to get
>>> the file in question, but before returning the file, the server parses
>>> it, finds all the synchronous require calls, loads those files as well
>>> and returning the whole thing as a package. That way, when the
>>> original file that was asynchronously loaded is executed and comes to
>>> one of those synchronous require calls, that file is already there,
>>> and the require is actually synchronous.
>>>
>>> At this point, maybe this screencast demo will help to clarify how it
>>> works: http://screencast.com/t/nOU53BRYUAX
>>>
>>> Put another way:
>>> If I async require fileA, and fileA has synchronous dependencies on
>>> fileB, and fileC, and an asynchronous dependency on fileD, the server-
>>> side component will return (in a single "bundle") and keep in memory
>>> fileA, fileB, and fileC, not fileD, and it will execute fileA.
>>> The client-side also separates fetching the files and eval'ing them
>>> (the method of getting files is xhr+eval). So, let's say fileA has
>>> require('fileB'); that executes when the file is parsed and executed
>>> on the client, but require('fileC') is inside a function somewhere.
>>> Then fileA will first be eval'ed, then fileB when it comes across
>>> that, and the text of fileC will just be in memory, not eval'ed until
>>> that function is called or some other require to it is called by any
>>> other part of the program.
>>>
>>> Another example-
>>> fileA has dependencies fileB, fileC, fileD, fileE, fileF
>>> fileG has dependencies fileC, fileE, fileH
>>>
>>> When I call require('fileA', function(err,result){return 'yay';});,
>>> the module loader will load fileA, fileB, fileC, fileD, fileE, and
>>> fileF all in a single bundle.
>>> If I, after that, call require('fileG', function(err,result){return
>>> 'yay';});, the module loader will only load fileG and fileH!
>>>
>>> Hopefully, that's clear....
>>>
>>> The advantages-
>>> Being aware of the difference in synchronous and asynchronous require
>>> in your client-side code make it extremely natural to break all your
>>> client-side code into small reusable chunks- there is no penalty and
>>> you don't have to "optimize" later by deciding what to package
>>> together and what to package separately.
>>> Handling dependencies becomes nothing. You don't have to think about
>>> it.
>>> The server can have a "deployment" mode, where it caches what the
>>> dependencies of a file are and doesn't ever need to parse that file
>>> again.
>>> In "deployment" mode, the server can also cache bundles of multiple
>>> files that are requested together, so when another client requests
>>> that same bundle, it is already in memory.
>>>
>>> To sum up:
>>> xhr+eval-when-necessary client-side module loader
>>> both synchronous-ish and asynchronous require in your client side-code
>>> --the synchronous require is actually a command to the server-side
>>> component to bundle
>>> server-side component
>>> --parses for dependencies and bundles them together
>>> --can cache dependency parsing results and whole bundles
>>>
>>>
>>> So- thoughts? Is this a horrible idea? Are there some gotchas that I'm
>>> missing?
>>>
>>> Specific advice needed-
>>> • How to package this in a way that it can be easily used in other
>>> projects? How can I make it integrate seamlessly with existing servers
>>> and make it compatible with different transport mechanisms?
>>> • How to handle path resolution?
>>> • Suggestions for licensing?
>>> • Suggestions for a name- (Mundlejs is a portmanteau of Module and
>>> Bundle- didn't really think long about it)
>>>
>>> Things that need to be (properly)implemented:
>>> • server-side "parsing" is just a brittle regexp right now:
>>> (line.match /require\('(.*)'\)/)
>>> • neither type of server-side caching is implemented (pretty easy to
>>> do)
>>> • uniquely identify clients and keep the server away of what modules
>>> they already have, so we can just send the diff of cached modules-
>>> currently, I'm sending the entire list of already cached modules with
>>> every xhr call, so the server doesn't load a dependency twice.
>>> • proper compatibility with module specifications (i.e. CommonJS)-
>>> right now, it's just require and module.exports
>>>
>>>
>>> Code is available here: https://github.com/meelash/Mundlejs
>>> To test it:
>>> from Mundlejs/tests/, run
>>> node server.js
>>> visit http://127.0.0.1:1337/ and open your browser console.
>>
>>
>> On Saturday, March 24, 2012 5:04:52 PM UTC-7, Saleem Abdul Hamid wrote:
>>>
>>> tl;dr - Client-side require with a server-side component that caches
>>> dependencies, bundles them, and caches the bundles. Need feedback on
>>> the concept, syntax. Need suggestions/contributions on implementation.
>>> Although, this works for me, it is almost just a proof-of-concept,
>>> needs work.
>>>
>>>
>>> As part of a project I'm working on, I spent a few hours writing a
>>> little client-side module loader with a server-side component enabling
>>> what I think is a pretty neat meaning to CommonJS module syntax. This
>>> morning I pulled it out of the rest of my project and attempted to
>>> package it in a useful way for others to use.
>>>
>>> The basic idea is this- in your client-side code, you can use require
>>> in either a "synchronous" or asynchronous fashion-
>>> module1 = require('some/path.js');
>>> require('some/other/path.js', function(err,result){module2 =
>>> result;});
>>>
>>> An asynchronous require makes a call to the server component to get
>>> the file in question, but before returning the file, the server parses
>>> it, finds all the synchronous require calls, loads those files as well
>>> and returning the whole thing as a package. That way, when the
>>> original file that was asynchronously loaded is executed and comes to
>>> one of those synchronous require calls, that file is already there,
>>> and the require is actually synchronous.
>>>
>>> At this point, maybe this screencast demo will help to clarify how it
>>> works: http://screencast.com/t/nOU53BRYUAX
>>>
>>> Put another way:
>>> If I async require fileA, and fileA has synchronous dependencies on
>>> fileB, and fileC, and an asynchronous dependency on fileD, the server-
>>> side component will return (in a single "bundle") and keep in memory
>>> fileA, fileB, and fileC, not fileD, and it will execute fileA.
>>> The client-side also separates fetching the files and eval'ing them
>>> (the method of getting files is xhr+eval). So, let's say fileA has
>>> require('fileB'); that executes when the file is parsed and executed
>>> on the client, but require('fileC') is inside a function somewhere.
>>> Then fileA will first be eval'ed, then fileB when it comes across
>>> that, and the text of fileC will just be in memory, not eval'ed until
>>> that function is called or some other require to it is called by any
>>> other part of the program.
>>>
>>> Another example-
>>> fileA has dependencies fileB, fileC, fileD, fileE, fileF
>>> fileG has dependencies fileC, fileE, fileH
>>>
>>> When I call require('fileA', function(err,result){return 'yay';});,
>>> the module loader will load fileA, fileB, fileC, fileD, fileE, and
>>> fileF all in a single bundle.
>>> If I, after that, call require('fileG', function(err,result){return
>>> 'yay';});, the module loader will only load fileG and fileH!
>>>
>>> Hopefully, that's clear....
>>>
>>> The advantages-
>>> Being aware of the difference in synchronous and asynchronous require
>>> in your client-side code make it extremely natural to break all your
>>> client-side code into small reusable chunks- there is no penalty and
>>> you don't have to "optimize" later by deciding what to package
>>> together and what to package separately.
>>> Handling dependencies becomes nothing. You don't have to think about
>>> it.
>>> The server can have a "deployment" mode, where it caches what the
>>> dependencies of a file are and doesn't ever need to parse that file
>>> again.
>>> In "deployment" mode, the server can also cache bundles of multiple
>>> files that are requested together, so when another client requests
>>> that same bundle, it is already in memory.
>>>
>>> To sum up:
>>> xhr+eval-when-necessary client-side module loader
>>> both synchronous-ish and asynchronous require in your client side-code
>>> --the synchronous require is actually a command to the server-side
>>> component to bundle
>>> server-side component
>>> --parses for dependencies and bundles them together
>>> --can cache dependency parsing results and whole bundles
>>>
>>>
>>> So- thoughts? Is this a horrible idea? Are there some gotchas that I'm
>>> missing?
>>>
>>> Specific advice needed-
>>> • How to package this in a way that it can be easily used in other
>>> projects? How can I make it integrate seamlessly with existing servers
>>> and make it compatible with different transport mechanisms?
>>> • How to handle path resolution?
>>> • Suggestions for licensing?
>>> • Suggestions for a name- (Mundlejs is a portmanteau of Module and
>>> Bundle- didn't really think long about it)
>>>
>>> Things that need to be (properly)implemented:
>>> • server-side "parsing" is just a brittle regexp right now:
>>> (line.match /require\('(.*)'\)/)
>>> • neither type of server-side caching is implemented (pretty easy to
>>> do)
>>> • uniquely identify clients and keep the server away of what modules
>>> they already have, so we can just send the diff of cached modules-
>>> currently, I'm sending the entire list of already cached modules with
>>> every xhr call, so the server doesn't load a dependency twice.
>>> • proper compatibility with module specifications (i.e. CommonJS)-
>>> right now, it's just require and module.exports
>>>
>>>
>>> Code is available here: https://github.com/meelash/Mundlejs
>>> To test it:
>>> from Mundlejs/tests/, run
>>> node server.js
>>> visit http://127.0.0.1:1337/ and open your browser console.
>>
>>
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
