Re: [Wikitech-l] MW support for Composer equivalent for JavaScript packages

2015-11-06 Thread Daniel Friesen 
On 2015-11-05 1:30 PM, C. Scott Ananian wrote:
> Two other interesting pieces:
>
> 1. http://requirejs.org/ is still the goal-standard for async browser-type
> loading, AFAIK, and there are good packages (`npm install requirejs`) that
> allow interoperability with the "npm style".
requirejs is still built for the same single application type as the
other non-async loaders. You may not be able to get even requirejs to
work with MediaWiki's needed pattern of different packages required by
different extensions all integrated together without the use of node on
the server that needs to combine them together.

> 2. The recently-completed ES6 spec contains a module format.  You can
> already use it via compatibility thunks from many existing module systems.
> It may well see increased use, especially on the web as browsers implement
> the spec (which is happening quickly).  There is resistance in the node
> community to adopting ES6 modules, but it may be that we are at an
> inflection point and ES6 will eventually win out.
ES6 modules have a different pattern for how exports are treated,
especially in regards to the 'default' export.
To handle this case babel inserts the following when you are transpiling
ES6/ES2015 module syntax:
Object.defineProperty(exports, '__esModule', {
  value: true
});

Unless you explicitly enable loose mode any library you develop as an es
module and anything that uses it will instantly break in browsers like IE8.

Use of ES6 import/export is gaining more and more adoption. But this
might be one reason some people are holding back on it.

es modules also have different behavior in regards to cycles that I know
esperanto handles but I'm not sure about the other transpilers.

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]

>   --scott
>
> On Thu, Nov 5, 2015 at 3:24 PM, Daniel Friesen 
> wrote:
>
>> As someone who now works in js/node primarily I might be able to add a
>> little outside insight.
>>
>> IMHO bower was probably killed far before npm@3 came out.
>>
>> - npm has always been more advanced
>> - bower's main isn't as reliable as it is in npm. Even if modules aren't
>> forgetting to set it as much as they used to I still see hard to deal
>> with things like modules that include .sass sources in it.
>> - bower's distribution model is actually very problematic for upstream
>> developers. bower clones git repositories directly expecting to find
>> built/compiled js and has no local build hooks. Few major js libraries
>> work in classic js/css sources anymore, instead having some required
>> build step. Bower makes a complete mess of this. Requiring the build
>> result to be committed with each change, an automated build bot, a
>> second git repo for build results only, or the upstream dev to just not
>> care about bower.
>> - Thanks to the rise of browserify and the like more and more
>> client-only libraries have moved to npm despite people traditionally
>> thinking of it as being for nodejs modules. Most client only libraries
>> now exist in npm. And if you wave the name browserify around you can get
>> just about any js library to start distributing to npm.
>>   - This has actually gone so far that once when I added a contribution
>> to a library I noticed that they were actually forgetting to keep their
>> bower.json up to date.
>>
>> npm@3 is also probably not as important as it's being made out here.
>>
>> npm@3 still doesn't guarantee a tree will be 100% flat. Most of npm@3's
>> changes fix small quirks in front-end usage and stability issues with npm.
>>
>> The 'major' change of 'flatness' is really that when installing `a` that
>> depends on `b`, `b` will be hoisted up to the same level as `a`, if
>> possible, instead of always being installed under `a`. npm@2 did have
>> some annoying quirks during development/updating that could leave a
>> duplicate module until you recreated your node_modules tree. And there
>> was the side case of installing two modules that both depended on
>> something you did not depend on. But that could be dealt with by either
>> adding it as a dep yourself or running dedupe.
>>
>> 
>>
>> The bad news is that while more and more libraries are moving to npm the
>> underlying reason for many being on npm is for the many users using
>> tools like browserify and webpack. So the assumption of many libraries
>> is that when you use npm for client-side libraries is you are using it
>> in a node-style/CommonJS way where require is available other npm
>> dependencies can be used through it and you're not including things in
>> the traditional way of libraries being run one after another on the page
>> in the global scope like in ResourceLoader.
>>
>> It's probably going to get harder and harder to reconcile deps shared
>> between extensions and/or use library code itself without having node
>> installed on servers. For that matter half the advancements in the css
>> space probably won't be making

Re: [Wikitech-l] MW support for Composer equivalent for JavaScript packages

2015-11-06 Thread Bryan Davis 
(such top posting! much discussion!)

It would be great to see some of the salient points of this discussion
captured on the current phab ticket [0] on the topic.

[0]: https://phabricator.wikimedia.org/T107561

Bryan

On Fri, Nov 6, 2015 at 1:03 AM, Daniel Friesen
 wrote:
> On 2015-11-05 1:30 PM, C. Scott Ananian wrote:
>> Two other interesting pieces:
>>
>> 1. http://requirejs.org/ is still the goal-standard for async browser-type
>> loading, AFAIK, and there are good packages (`npm install requirejs`) that
>> allow interoperability with the "npm style".
> requirejs is still built for the same single application type as the
> other non-async loaders. You may not be able to get even requirejs to
> work with MediaWiki's needed pattern of different packages required by
> different extensions all integrated together without the use of node on
> the server that needs to combine them together.
>
>> 2. The recently-completed ES6 spec contains a module format.  You can
>> already use it via compatibility thunks from many existing module systems.
>> It may well see increased use, especially on the web as browsers implement
>> the spec (which is happening quickly).  There is resistance in the node
>> community to adopting ES6 modules, but it may be that we are at an
>> inflection point and ES6 will eventually win out.
> ES6 modules have a different pattern for how exports are treated,
> especially in regards to the 'default' export.
> To handle this case babel inserts the following when you are transpiling
> ES6/ES2015 module syntax:
> Object.defineProperty(exports, '__esModule', {
>   value: true
> });
>
> Unless you explicitly enable loose mode any library you develop as an es
> module and anything that uses it will instantly break in browsers like IE8.
>
> Use of ES6 import/export is gaining more and more adoption. But this
> might be one reason some people are holding back on it.
>
> es modules also have different behavior in regards to cycles that I know
> esperanto handles but I'm not sure about the other transpilers.
>
> ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]
>
>>   --scott
>>
>> On Thu, Nov 5, 2015 at 3:24 PM, Daniel Friesen 
>> wrote:
>>
>>> As someone who now works in js/node primarily I might be able to add a
>>> little outside insight.
>>>
>>> IMHO bower was probably killed far before npm@3 came out.
>>>
>>> - npm has always been more advanced
>>> - bower's main isn't as reliable as it is in npm. Even if modules aren't
>>> forgetting to set it as much as they used to I still see hard to deal
>>> with things like modules that include .sass sources in it.
>>> - bower's distribution model is actually very problematic for upstream
>>> developers. bower clones git repositories directly expecting to find
>>> built/compiled js and has no local build hooks. Few major js libraries
>>> work in classic js/css sources anymore, instead having some required
>>> build step. Bower makes a complete mess of this. Requiring the build
>>> result to be committed with each change, an automated build bot, a
>>> second git repo for build results only, or the upstream dev to just not
>>> care about bower.
>>> - Thanks to the rise of browserify and the like more and more
>>> client-only libraries have moved to npm despite people traditionally
>>> thinking of it as being for nodejs modules. Most client only libraries
>>> now exist in npm. And if you wave the name browserify around you can get
>>> just about any js library to start distributing to npm.
>>>   - This has actually gone so far that once when I added a contribution
>>> to a library I noticed that they were actually forgetting to keep their
>>> bower.json up to date.
>>>
>>> npm@3 is also probably not as important as it's being made out here.
>>>
>>> npm@3 still doesn't guarantee a tree will be 100% flat. Most of npm@3's
>>> changes fix small quirks in front-end usage and stability issues with npm.
>>>
>>> The 'major' change of 'flatness' is really that when installing `a` that
>>> depends on `b`, `b` will be hoisted up to the same level as `a`, if
>>> possible, instead of always being installed under `a`. npm@2 did have
>>> some annoying quirks during development/updating that could leave a
>>> duplicate module until you recreated your node_modules tree. And there
>>> was the side case of installing two modules that both depended on
>>> something you did not depend on. But that could be dealt with by either
>>> adding it as a dep yourself or running dedupe.
>>>
>>> 
>>>
>>> The bad news is that while more and more libraries are moving to npm the
>>> underlying reason for many being on npm is for the many users using
>>> tools like browserify and webpack. So the assumption of many libraries
>>> is that when you use npm for client-side libraries is you are using it
>>> in a node-style/CommonJS way where require is available other npm
>>> dependencies can be used through it and

Re: [Wikitech-l] MW support for Composer equivalent for JavaScript packages

2015-11-05 Thread Trevor Parscal 
The flat approach to NPM is a game changer for us, and a Bower killer.

Timo? Had a lot of insight at the time, I'd like to see him be invoked in
this decision. Any thoughts Timo?

- Trevor

On Thursday, November 5, 2015, Jon Robson  wrote:

> It's been a year now, over 3-6 months. Volker can this be given a
> priority in the next frontend standards meeting. I think the lack of
> any standard is extremely damaging to the project.
>
> On Wed, Jul 22, 2015 at 4:21 PM, Bryan Davis  > wrote:
> > On Wed, Jul 22, 2015 at 12:24 PM, Daniel Werner
> > > wrote:
> >> Hey,
> >>
> >> I just wanted to check in about the status of enabling JavaScript
> package
> >> management usage in MediaWiki. I am basically talking about an
> equivalent
> >> for JS to what we have with Composer for PHP.
> >>
> >> Real-world example:
> >>   The "data-values/value-view" package[0] is defining
> >> "jquery.event.special.eachchange.js":
> >> ValueView/lib/jquery.event/jquery.event.special.eachchange.js
> >>
> >>   Now, recently I needed the same functionality in one of my
> extensions, so
> >> I just copied it over. [1]
> >>
> >> I know that this is the worst way one could do this, but as far as I can
> >> see we don't have that much of a choice right now. Here are the
> alternative
> >> options I can see:
> >>
> >> Moving "jquery.event.special.eachchange.js" out of the
> >> "data-values/value-view" package into its own "WMDE/jquery-eachchange"
> >> package...
> >>
> >> 1. ... and using it in my extension via composer.
> >>   + pro: two or more extensions or other packages requiring this package
> >> will still result in having only one MW-wide installation..
> >>   - con: requires MW specific code which is actually not related to the
> >> MW-independent package to feed the resource loader.
> >>   - con: using Composer to manage pure JavaScript packages! Uuuh, ugly!
> >>
> >> 2. ... and having a build step in other packages using the package,
> pulling
> >> the "WMDE/jquery-eachchange" somewhere into the file structure of the
> >> packages/extensions using it.
> >>   + pro: don't need to abuse composer, we can use "npm", "Bower" or any
> >> other arbitrary JS package manager here.
> >>   - con: got to tell resource loader somehow... (didn't think so much
> about
> >> that yet)
> >>   - con: if more than one extensions or other packages require this
> package
> >> we still end up with the same code twice or more often in one MW
> >> installation.
> >>
> >> 3. Combining 1 and 2: Start with 2, using a JS package manager. Then
> going
> >> to 1, creating a composer package and pulling the
> "WMDE/jquery-eachchange"
> >> package in via some build script.
> >>   + pro: The two pros from 1 + 2
> >>   + pro: ^^
> >>   - con: still got to tell resource loader somehow...
> >>   - con: Overhead; We now create two packages where the Composer one is
> >> just a bridge to the MW-world, still polluting packagist.org. Still
> kind of
> >> ugly and more effort for publishing a package and therefore potentially
> >> scaring programmers away from doing so since they've got better things
> to
> >> do than doing work that could be automated.
> >>
> >> I have not seen Approach 2 and 3 yet. Though I could imagine that the
> >> VisualEditor team has used something like that.
> >> Approach 1 is the way the "data-values/value-view" package itself is
> being
> >> handled. And that package should actually be a MW independent pure JS
> >> package but right now it contains MW specific code and uses composer for
> >> distribution!
> >> There is still another option but that had to be properly implemented:
> >>
> >> 4. Choose one native JS package manager for now and go with it (add
> support
> >> for others later perhaps). Integrate it properly with MW (resource
> loader
> >> to begin with), document how to use it and finally distribute JS code
> >> coming from the MW world but useful for other projects in a way where it
> >> can actually be used in a non-MW context.
> >>
> >> This has already been bugging me when working on Wikidata. Now I'd like
> to
> >> reuse some of the code I have written there without spending hours and
> >> hours with option 3 because there should be support for option 4 rather
> >> sooner or later.
> >> So I am wondering; Does anyone have any thoughts, any alternatives
> perhaps
> >> or is there any roadmap on anything like the option 4 that I have shown?
> >>
> >> Cheers,
> >> Daniel
> >>
> >> [0]: https://packagist.org/packages/data-values/value-view
> >> [1]:
> >>
> https://github.com/DanweDE/mediawiki-ext-UserBitcoinAddresses/blob/master/resources/vendor/jquery.event.special.eachchange.js
> >
> > Option 4 was discussed last October as part of the Librarization
> > project [0]. At the time the front end standards group wasn't ready to
> > pick a winner in the javascript packaging landscape. They did want to
> > revisit that in 3-6

Re: [Wikitech-l] MW support for Composer equivalent for JavaScript packages

2015-11-05 Thread Jon Robson 
It's been a year now, over 3-6 months. Volker can this be given a
priority in the next frontend standards meeting. I think the lack of
any standard is extremely damaging to the project.

On Wed, Jul 22, 2015 at 4:21 PM, Bryan Davis  wrote:
> On Wed, Jul 22, 2015 at 12:24 PM, Daniel Werner
>  wrote:
>> Hey,
>>
>> I just wanted to check in about the status of enabling JavaScript package
>> management usage in MediaWiki. I am basically talking about an equivalent
>> for JS to what we have with Composer for PHP.
>>
>> Real-world example:
>>   The "data-values/value-view" package[0] is defining
>> "jquery.event.special.eachchange.js":
>> ValueView/lib/jquery.event/jquery.event.special.eachchange.js
>>
>>   Now, recently I needed the same functionality in one of my extensions, so
>> I just copied it over. [1]
>>
>> I know that this is the worst way one could do this, but as far as I can
>> see we don't have that much of a choice right now. Here are the alternative
>> options I can see:
>>
>> Moving "jquery.event.special.eachchange.js" out of the
>> "data-values/value-view" package into its own "WMDE/jquery-eachchange"
>> package...
>>
>> 1. ... and using it in my extension via composer.
>>   + pro: two or more extensions or other packages requiring this package
>> will still result in having only one MW-wide installation..
>>   - con: requires MW specific code which is actually not related to the
>> MW-independent package to feed the resource loader.
>>   - con: using Composer to manage pure JavaScript packages! Uuuh, ugly!
>>
>> 2. ... and having a build step in other packages using the package, pulling
>> the "WMDE/jquery-eachchange" somewhere into the file structure of the
>> packages/extensions using it.
>>   + pro: don't need to abuse composer, we can use "npm", "Bower" or any
>> other arbitrary JS package manager here.
>>   - con: got to tell resource loader somehow... (didn't think so much about
>> that yet)
>>   - con: if more than one extensions or other packages require this package
>> we still end up with the same code twice or more often in one MW
>> installation.
>>
>> 3. Combining 1 and 2: Start with 2, using a JS package manager. Then going
>> to 1, creating a composer package and pulling the "WMDE/jquery-eachchange"
>> package in via some build script.
>>   + pro: The two pros from 1 + 2
>>   + pro: ^^
>>   - con: still got to tell resource loader somehow...
>>   - con: Overhead; We now create two packages where the Composer one is
>> just a bridge to the MW-world, still polluting packagist.org. Still kind of
>> ugly and more effort for publishing a package and therefore potentially
>> scaring programmers away from doing so since they've got better things to
>> do than doing work that could be automated.
>>
>> I have not seen Approach 2 and 3 yet. Though I could imagine that the
>> VisualEditor team has used something like that.
>> Approach 1 is the way the "data-values/value-view" package itself is being
>> handled. And that package should actually be a MW independent pure JS
>> package but right now it contains MW specific code and uses composer for
>> distribution!
>> There is still another option but that had to be properly implemented:
>>
>> 4. Choose one native JS package manager for now and go with it (add support
>> for others later perhaps). Integrate it properly with MW (resource loader
>> to begin with), document how to use it and finally distribute JS code
>> coming from the MW world but useful for other projects in a way where it
>> can actually be used in a non-MW context.
>>
>> This has already been bugging me when working on Wikidata. Now I'd like to
>> reuse some of the code I have written there without spending hours and
>> hours with option 3 because there should be support for option 4 rather
>> sooner or later.
>> So I am wondering; Does anyone have any thoughts, any alternatives perhaps
>> or is there any roadmap on anything like the option 4 that I have shown?
>>
>> Cheers,
>> Daniel
>>
>> [0]: https://packagist.org/packages/data-values/value-view
>> [1]:
>> https://github.com/DanweDE/mediawiki-ext-UserBitcoinAddresses/blob/master/resources/vendor/jquery.event.special.eachchange.js
>
> Option 4 was discussed last October as part of the Librarization
> project [0]. At the time the front end standards group wasn't ready to
> pick a winner in the javascript packaging landscape. They did want to
> revisit that in 3-6 months however so maybe it is time to find someone
> to look into the various options and their pros and cons again?
>
> [0]: https://phabricator.wikimedia.org/T807
> --
> Bryan Davis  Wikimedia Foundation
> [[m:User:BDavis_(WMF)]]  Sr Software EngineerBoise, ID USA
> irc: bd808v:415.839.6885 x6855
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
>

Re: [Wikitech-l] MW support for Composer equivalent for JavaScript packages

2015-11-05 Thread Daniel Friesen 
As someone who now works in js/node primarily I might be able to add a
little outside insight.

IMHO bower was probably killed far before npm@3 came out.

- npm has always been more advanced
- bower's main isn't as reliable as it is in npm. Even if modules aren't
forgetting to set it as much as they used to I still see hard to deal
with things like modules that include .sass sources in it.
- bower's distribution model is actually very problematic for upstream
developers. bower clones git repositories directly expecting to find
built/compiled js and has no local build hooks. Few major js libraries
work in classic js/css sources anymore, instead having some required
build step. Bower makes a complete mess of this. Requiring the build
result to be committed with each change, an automated build bot, a
second git repo for build results only, or the upstream dev to just not
care about bower.
- Thanks to the rise of browserify and the like more and more
client-only libraries have moved to npm despite people traditionally
thinking of it as being for nodejs modules. Most client only libraries
now exist in npm. And if you wave the name browserify around you can get
just about any js library to start distributing to npm.
  - This has actually gone so far that once when I added a contribution
to a library I noticed that they were actually forgetting to keep their
bower.json up to date.

npm@3 is also probably not as important as it's being made out here.

npm@3 still doesn't guarantee a tree will be 100% flat. Most of npm@3's
changes fix small quirks in front-end usage and stability issues with npm.

The 'major' change of 'flatness' is really that when installing `a` that
depends on `b`, `b` will be hoisted up to the same level as `a`, if
possible, instead of always being installed under `a`. npm@2 did have
some annoying quirks during development/updating that could leave a
duplicate module until you recreated your node_modules tree. And there
was the side case of installing two modules that both depended on
something you did not depend on. But that could be dealt with by either
adding it as a dep yourself or running dedupe.



The bad news is that while more and more libraries are moving to npm the
underlying reason for many being on npm is for the many users using
tools like browserify and webpack. So the assumption of many libraries
is that when you use npm for client-side libraries is you are using it
in a node-style/CommonJS way where require is available other npm
dependencies can be used through it and you're not including things in
the traditional way of libraries being run one after another on the page
in the global scope like in ResourceLoader.

It's probably going to get harder and harder to reconcile deps shared
between extensions and/or use library code itself without having node
installed on servers. For that matter half the advancements in the css
space probably won't be making their way to php either.

Though I do have some hints to things that might open up some
possibilities.

Firstly browserify can build standalone versions of libraries. It's
possible you could make tweaks to that pattern and have a development
build step that would convert npm packages into modules built to run in
a new type of ResourceLoader that would probably be a hybrid of the
current RL and npm. However you would need to take care to exclude
things or else you'll end up with a script that will duplicate the
source of a bunch of other modules. You also have to be wary of handling
the cases where npm intentionally duplicates a library.

This actually may be even more difficult than it sounds due to npm
packages that require modules inside a package rather than just the main
module. You might need to go even deeper into browserify, substituting
one of the last steps so it outputs a different format with objects that
have metadata on the individual modules intended for use in a special
loader instead of as a plain script tag.

Current RL on its own just won't work for this. Globals don't work well
in the module pattern and so the ability to get a result from require()
and allow npm modules to require() others are necessary. Circular
requires also need consideration.

Whether you want a RL api that has a bridge to the npm ecosystem or want
to go all-in to the modern js landscape and build a more
CommonJS/node/npm-like RL api that loads things according to MW's use
cases will be the important bike-shedding.

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]

On 2015-11-05 8:32 AM, Trevor Parscal wrote:
> The flat approach to NPM is a game changer for us, and a Bower killer.
>
> Timo? Had a lot of insight at the time, I'd like to see him be invoked in
> this decision. Any thoughts Timo?
>
> - Trevor
>
> On Thursday, November 5, 2015, Jon Robson  wrote:
>
>> It's been a year now, over 3-6 months. Volker can this be given a
>> priority in the next frontend standards meeting. I think the

Re: [Wikitech-l] MW support for Composer equivalent for JavaScript packages

2015-11-05 Thread C. Scott Ananian 
Note that as part of https://phabricator.wikimedia.org/T114457 I'm
experimenting with distributing mediawiki-core itself as an npm-installable
package.  It ought to be possible to package up extensions, including
Parsoid and VE, using npm as well.
 --scott

On Thu, Nov 5, 2015 at 11:32 AM, Trevor Parscal 
wrote:

> The flat approach to NPM is a game changer for us, and a Bower killer.
>
> Timo? Had a lot of insight at the time, I'd like to see him be invoked in
> this decision. Any thoughts Timo?
>
> - Trevor
>
> On Thursday, November 5, 2015, Jon Robson  wrote:
>
> > It's been a year now, over 3-6 months. Volker can this be given a
> > priority in the next frontend standards meeting. I think the lack of
> > any standard is extremely damaging to the project.
> >
> > On Wed, Jul 22, 2015 at 4:21 PM, Bryan Davis  > > wrote:
> > > On Wed, Jul 22, 2015 at 12:24 PM, Daniel Werner
> > > > wrote:
> > >> Hey,
> > >>
> > >> I just wanted to check in about the status of enabling JavaScript
> > package
> > >> management usage in MediaWiki. I am basically talking about an
> > equivalent
> > >> for JS to what we have with Composer for PHP.
> > >>
> > >> Real-world example:
> > >>   The "data-values/value-view" package[0] is defining
> > >> "jquery.event.special.eachchange.js":
> > >> ValueView/lib/jquery.event/jquery.event.special.eachchange.js
> > >>
> > >>   Now, recently I needed the same functionality in one of my
> > extensions, so
> > >> I just copied it over. [1]
> > >>
> > >> I know that this is the worst way one could do this, but as far as I
> can
> > >> see we don't have that much of a choice right now. Here are the
> > alternative
> > >> options I can see:
> > >>
> > >> Moving "jquery.event.special.eachchange.js" out of the
> > >> "data-values/value-view" package into its own "WMDE/jquery-eachchange"
> > >> package...
> > >>
> > >> 1. ... and using it in my extension via composer.
> > >>   + pro: two or more extensions or other packages requiring this
> package
> > >> will still result in having only one MW-wide installation..
> > >>   - con: requires MW specific code which is actually not related to
> the
> > >> MW-independent package to feed the resource loader.
> > >>   - con: using Composer to manage pure JavaScript packages! Uuuh,
> ugly!
> > >>
> > >> 2. ... and having a build step in other packages using the package,
> > pulling
> > >> the "WMDE/jquery-eachchange" somewhere into the file structure of the
> > >> packages/extensions using it.
> > >>   + pro: don't need to abuse composer, we can use "npm", "Bower" or
> any
> > >> other arbitrary JS package manager here.
> > >>   - con: got to tell resource loader somehow... (didn't think so much
> > about
> > >> that yet)
> > >>   - con: if more than one extensions or other packages require this
> > package
> > >> we still end up with the same code twice or more often in one MW
> > >> installation.
> > >>
> > >> 3. Combining 1 and 2: Start with 2, using a JS package manager. Then
> > going
> > >> to 1, creating a composer package and pulling the
> > "WMDE/jquery-eachchange"
> > >> package in via some build script.
> > >>   + pro: The two pros from 1 + 2
> > >>   + pro: ^^
> > >>   - con: still got to tell resource loader somehow...
> > >>   - con: Overhead; We now create two packages where the Composer one
> is
> > >> just a bridge to the MW-world, still polluting packagist.org. Still
> > kind of
> > >> ugly and more effort for publishing a package and therefore
> potentially
> > >> scaring programmers away from doing so since they've got better things
> > to
> > >> do than doing work that could be automated.
> > >>
> > >> I have not seen Approach 2 and 3 yet. Though I could imagine that the
> > >> VisualEditor team has used something like that.
> > >> Approach 1 is the way the "data-values/value-view" package itself is
> > being
> > >> handled. And that package should actually be a MW independent pure JS
> > >> package but right now it contains MW specific code and uses composer
> for
> > >> distribution!
> > >> There is still another option but that had to be properly implemented:
> > >>
> > >> 4. Choose one native JS package manager for now and go with it (add
> > support
> > >> for others later perhaps). Integrate it properly with MW (resource
> > loader
> > >> to begin with), document how to use it and finally distribute JS code
> > >> coming from the MW world but useful for other projects in a way where
> it
> > >> can actually be used in a non-MW context.
> > >>
> > >> This has already been bugging me when working on Wikidata. Now I'd
> like
> > to
> > >> reuse some of the code I have written there without spending hours and
> > >> hours with option 3 because there should be support for option 4
> rather
> > >> sooner or later.
> > >> So I am wondering; Does anyone have any thoughts, any alternatives
> > perhaps
> >

Re: [Wikitech-l] MW support for Composer equivalent for JavaScript packages

2015-11-05 Thread C. Scott Ananian 
Two other interesting pieces:

1. http://requirejs.org/ is still the goal-standard for async browser-type
loading, AFAIK, and there are good packages (`npm install requirejs`) that
allow interoperability with the "npm style".

2. The recently-completed ES6 spec contains a module format.  You can
already use it via compatibility thunks from many existing module systems.
It may well see increased use, especially on the web as browsers implement
the spec (which is happening quickly).  There is resistance in the node
community to adopting ES6 modules, but it may be that we are at an
inflection point and ES6 will eventually win out.
  --scott

On Thu, Nov 5, 2015 at 3:24 PM, Daniel Friesen 
wrote:

> As someone who now works in js/node primarily I might be able to add a
> little outside insight.
>
> IMHO bower was probably killed far before npm@3 came out.
>
> - npm has always been more advanced
> - bower's main isn't as reliable as it is in npm. Even if modules aren't
> forgetting to set it as much as they used to I still see hard to deal
> with things like modules that include .sass sources in it.
> - bower's distribution model is actually very problematic for upstream
> developers. bower clones git repositories directly expecting to find
> built/compiled js and has no local build hooks. Few major js libraries
> work in classic js/css sources anymore, instead having some required
> build step. Bower makes a complete mess of this. Requiring the build
> result to be committed with each change, an automated build bot, a
> second git repo for build results only, or the upstream dev to just not
> care about bower.
> - Thanks to the rise of browserify and the like more and more
> client-only libraries have moved to npm despite people traditionally
> thinking of it as being for nodejs modules. Most client only libraries
> now exist in npm. And if you wave the name browserify around you can get
> just about any js library to start distributing to npm.
>   - This has actually gone so far that once when I added a contribution
> to a library I noticed that they were actually forgetting to keep their
> bower.json up to date.
>
> npm@3 is also probably not as important as it's being made out here.
>
> npm@3 still doesn't guarantee a tree will be 100% flat. Most of npm@3's
> changes fix small quirks in front-end usage and stability issues with npm.
>
> The 'major' change of 'flatness' is really that when installing `a` that
> depends on `b`, `b` will be hoisted up to the same level as `a`, if
> possible, instead of always being installed under `a`. npm@2 did have
> some annoying quirks during development/updating that could leave a
> duplicate module until you recreated your node_modules tree. And there
> was the side case of installing two modules that both depended on
> something you did not depend on. But that could be dealt with by either
> adding it as a dep yourself or running dedupe.
>
> 
>
> The bad news is that while more and more libraries are moving to npm the
> underlying reason for many being on npm is for the many users using
> tools like browserify and webpack. So the assumption of many libraries
> is that when you use npm for client-side libraries is you are using it
> in a node-style/CommonJS way where require is available other npm
> dependencies can be used through it and you're not including things in
> the traditional way of libraries being run one after another on the page
> in the global scope like in ResourceLoader.
>
> It's probably going to get harder and harder to reconcile deps shared
> between extensions and/or use library code itself without having node
> installed on servers. For that matter half the advancements in the css
> space probably won't be making their way to php either.
>
> Though I do have some hints to things that might open up some
> possibilities.
>
> Firstly browserify can build standalone versions of libraries. It's
> possible you could make tweaks to that pattern and have a development
> build step that would convert npm packages into modules built to run in
> a new type of ResourceLoader that would probably be a hybrid of the
> current RL and npm. However you would need to take care to exclude
> things or else you'll end up with a script that will duplicate the
> source of a bunch of other modules. You also have to be wary of handling
> the cases where npm intentionally duplicates a library.
>
> This actually may be even more difficult than it sounds due to npm
> packages that require modules inside a package rather than just the main
> module. You might need to go even deeper into browserify, substituting
> one of the last steps so it outputs a different format with objects that
> have metadata on the individual modules intended for use in a special
> loader instead of as a plain script tag.
>
> Current RL on its own just won't work for this. Globals don't work well
> in the module pattern and so the ability to get a result from require()
>

Re: [Wikitech-l] MW support for Composer equivalent for JavaScript packages

2015-08-04 Thread Joaquin Oltra Hernandez 
We should choose something, any of them is better than nothing, and
manually copying files.

My vote goes to npm, now that it tries (in v3) its best to flatten the
node_modules folder. See the changelog (
https://github.com/npm/npm/releases/tag/v3.0.0)

*Flat, flat, flat!*



Your dependencies will now be installed *maximally flat*. Insofar as is
 possible, all of your dependencies, and their dependencies, and THEIR
 dependencies will be installed in your project's node_modules folder with
 no nesting. You'll only see modules nested underneath one another when two
 (or more) modules have conflicting dependencies.


And it is the biggest JS package manager.

On Thu, Jul 23, 2015 at 1:21 AM, Bryan Davis bd...@wikimedia.org wrote:

 On Wed, Jul 22, 2015 at 12:24 PM, Daniel Werner
 daniel.a.r.wer...@gmail.com wrote:
  Hey,
 
  I just wanted to check in about the status of enabling JavaScript package
  management usage in MediaWiki. I am basically talking about an equivalent
  for JS to what we have with Composer for PHP.
 
  Real-world example:
The data-values/value-view package[0] is defining
  jquery.event.special.eachchange.js:
  ValueView/lib/jquery.event/jquery.event.special.eachchange.js
 
Now, recently I needed the same functionality in one of my extensions,
 so
  I just copied it over. [1]
 
  I know that this is the worst way one could do this, but as far as I can
  see we don't have that much of a choice right now. Here are the
 alternative
  options I can see:
 
  Moving jquery.event.special.eachchange.js out of the
  data-values/value-view package into its own WMDE/jquery-eachchange
  package...
 
  1. ... and using it in my extension via composer.
+ pro: two or more extensions or other packages requiring this package
  will still result in having only one MW-wide installation..
- con: requires MW specific code which is actually not related to the
  MW-independent package to feed the resource loader.
- con: using Composer to manage pure JavaScript packages! Uuuh, ugly!
 
  2. ... and having a build step in other packages using the package,
 pulling
  the WMDE/jquery-eachchange somewhere into the file structure of the
  packages/extensions using it.
+ pro: don't need to abuse composer, we can use npm, Bower or any
  other arbitrary JS package manager here.
- con: got to tell resource loader somehow... (didn't think so much
 about
  that yet)
- con: if more than one extensions or other packages require this
 package
  we still end up with the same code twice or more often in one MW
  installation.
 
  3. Combining 1 and 2: Start with 2, using a JS package manager. Then
 going
  to 1, creating a composer package and pulling the
 WMDE/jquery-eachchange
  package in via some build script.
+ pro: The two pros from 1 + 2
+ pro: ^^
- con: still got to tell resource loader somehow...
- con: Overhead; We now create two packages where the Composer one is
  just a bridge to the MW-world, still polluting packagist.org. Still
 kind of
  ugly and more effort for publishing a package and therefore potentially
  scaring programmers away from doing so since they've got better things to
  do than doing work that could be automated.
 
  I have not seen Approach 2 and 3 yet. Though I could imagine that the
  VisualEditor team has used something like that.
  Approach 1 is the way the data-values/value-view package itself is
 being
  handled. And that package should actually be a MW independent pure JS
  package but right now it contains MW specific code and uses composer for
  distribution!
  There is still another option but that had to be properly implemented:
 
  4. Choose one native JS package manager for now and go with it (add
 support
  for others later perhaps). Integrate it properly with MW (resource loader
  to begin with), document how to use it and finally distribute JS code
  coming from the MW world but useful for other projects in a way where it
  can actually be used in a non-MW context.
 
  This has already been bugging me when working on Wikidata. Now I'd like
 to
  reuse some of the code I have written there without spending hours and
  hours with option 3 because there should be support for option 4 rather
  sooner or later.
  So I am wondering; Does anyone have any thoughts, any alternatives
 perhaps
  or is there any roadmap on anything like the option 4 that I have shown?
 
  Cheers,
  Daniel
 
  [0]: https://packagist.org/packages/data-values/value-view
  [1]:
 
 https://github.com/DanweDE/mediawiki-ext-UserBitcoinAddresses/blob/master/resources/vendor/jquery.event.special.eachchange.js

 Option 4 was discussed last October as part of the Librarization
 project [0]. At the time the front end standards group wasn't ready to
 pick a winner in the javascript packaging landscape. They did want to
 revisit that in 3-6 months however so maybe it is time to find someone
 to look into the various options and their pros and cons again?

 [0]:

[Wikitech-l] MW support for Composer equivalent for JavaScript packages

2015-07-22 Thread Daniel Werner 
Hey,

I just wanted to check in about the status of enabling JavaScript package
management usage in MediaWiki. I am basically talking about an equivalent
for JS to what we have with Composer for PHP.

Real-world example:
  The data-values/value-view package[0] is defining
jquery.event.special.eachchange.js:
ValueView/lib/jquery.event/jquery.event.special.eachchange.js

  Now, recently I needed the same functionality in one of my extensions, so
I just copied it over. [1]

I know that this is the worst way one could do this, but as far as I can
see we don't have that much of a choice right now. Here are the alternative
options I can see:

Moving jquery.event.special.eachchange.js out of the
data-values/value-view package into its own WMDE/jquery-eachchange
package...

1. ... and using it in my extension via composer.
  + pro: two or more extensions or other packages requiring this package
will still result in having only one MW-wide installation..
  - con: requires MW specific code which is actually not related to the
MW-independent package to feed the resource loader.
  - con: using Composer to manage pure JavaScript packages! Uuuh, ugly!

2. ... and having a build step in other packages using the package, pulling
the WMDE/jquery-eachchange somewhere into the file structure of the
packages/extensions using it.
  + pro: don't need to abuse composer, we can use npm, Bower or any
other arbitrary JS package manager here.
  - con: got to tell resource loader somehow... (didn't think so much about
that yet)
  - con: if more than one extensions or other packages require this package
we still end up with the same code twice or more often in one MW
installation.

3. Combining 1 and 2: Start with 2, using a JS package manager. Then going
to 1, creating a composer package and pulling the WMDE/jquery-eachchange
package in via some build script.
  + pro: The two pros from 1 + 2
  + pro: ^^
  - con: still got to tell resource loader somehow...
  - con: Overhead; We now create two packages where the Composer one is
just a bridge to the MW-world, still polluting packagist.org. Still kind of
ugly and more effort for publishing a package and therefore potentially
scaring programmers away from doing so since they've got better things to
do than doing work that could be automated.

I have not seen Approach 2 and 3 yet. Though I could imagine that the
VisualEditor team has used something like that.
Approach 1 is the way the data-values/value-view package itself is being
handled. And that package should actually be a MW independent pure JS
package but right now it contains MW specific code and uses composer for
distribution!
There is still another option but that had to be properly implemented:

4. Choose one native JS package manager for now and go with it (add support
for others later perhaps). Integrate it properly with MW (resource loader
to begin with), document how to use it and finally distribute JS code
coming from the MW world but useful for other projects in a way where it
can actually be used in a non-MW context.

This has already been bugging me when working on Wikidata. Now I'd like to
reuse some of the code I have written there without spending hours and
hours with option 3 because there should be support for option 4 rather
sooner or later.
So I am wondering; Does anyone have any thoughts, any alternatives perhaps
or is there any roadmap on anything like the option 4 that I have shown?

Cheers,
Daniel

[0]: https://packagist.org/packages/data-values/value-view
[1]:
https://github.com/DanweDE/mediawiki-ext-UserBitcoinAddresses/blob/master/resources/vendor/jquery.event.special.eachchange.js
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] MW support for Composer equivalent for JavaScript packages

2015-07-22 Thread Bryan Davis 
On Wed, Jul 22, 2015 at 12:24 PM, Daniel Werner
daniel.a.r.wer...@gmail.com wrote:
 Hey,

 I just wanted to check in about the status of enabling JavaScript package
 management usage in MediaWiki. I am basically talking about an equivalent
 for JS to what we have with Composer for PHP.

 Real-world example:
   The data-values/value-view package[0] is defining
 jquery.event.special.eachchange.js:
 ValueView/lib/jquery.event/jquery.event.special.eachchange.js

   Now, recently I needed the same functionality in one of my extensions, so
 I just copied it over. [1]

 I know that this is the worst way one could do this, but as far as I can
 see we don't have that much of a choice right now. Here are the alternative
 options I can see:

 Moving jquery.event.special.eachchange.js out of the
 data-values/value-view package into its own WMDE/jquery-eachchange
 package...

 1. ... and using it in my extension via composer.
   + pro: two or more extensions or other packages requiring this package
 will still result in having only one MW-wide installation..
   - con: requires MW specific code which is actually not related to the
 MW-independent package to feed the resource loader.
   - con: using Composer to manage pure JavaScript packages! Uuuh, ugly!

 2. ... and having a build step in other packages using the package, pulling
 the WMDE/jquery-eachchange somewhere into the file structure of the
 packages/extensions using it.
   + pro: don't need to abuse composer, we can use npm, Bower or any
 other arbitrary JS package manager here.
   - con: got to tell resource loader somehow... (didn't think so much about
 that yet)
   - con: if more than one extensions or other packages require this package
 we still end up with the same code twice or more often in one MW
 installation.

 3. Combining 1 and 2: Start with 2, using a JS package manager. Then going
 to 1, creating a composer package and pulling the WMDE/jquery-eachchange
 package in via some build script.
   + pro: The two pros from 1 + 2
   + pro: ^^
   - con: still got to tell resource loader somehow...
   - con: Overhead; We now create two packages where the Composer one is
 just a bridge to the MW-world, still polluting packagist.org. Still kind of
 ugly and more effort for publishing a package and therefore potentially
 scaring programmers away from doing so since they've got better things to
 do than doing work that could be automated.

 I have not seen Approach 2 and 3 yet. Though I could imagine that the
 VisualEditor team has used something like that.
 Approach 1 is the way the data-values/value-view package itself is being
 handled. And that package should actually be a MW independent pure JS
 package but right now it contains MW specific code and uses composer for
 distribution!
 There is still another option but that had to be properly implemented:

 4. Choose one native JS package manager for now and go with it (add support
 for others later perhaps). Integrate it properly with MW (resource loader
 to begin with), document how to use it and finally distribute JS code
 coming from the MW world but useful for other projects in a way where it
 can actually be used in a non-MW context.

 This has already been bugging me when working on Wikidata. Now I'd like to
 reuse some of the code I have written there without spending hours and
 hours with option 3 because there should be support for option 4 rather
 sooner or later.
 So I am wondering; Does anyone have any thoughts, any alternatives perhaps
 or is there any roadmap on anything like the option 4 that I have shown?

 Cheers,
 Daniel

 [0]: https://packagist.org/packages/data-values/value-view
 [1]:
 https://github.com/DanweDE/mediawiki-ext-UserBitcoinAddresses/blob/master/resources/vendor/jquery.event.special.eachchange.js

Option 4 was discussed last October as part of the Librarization
project [0]. At the time the front end standards group wasn't ready to
pick a winner in the javascript packaging landscape. They did want to
revisit that in 3-6 months however so maybe it is time to find someone
to look into the various options and their pros and cons again?

[0]: https://phabricator.wikimedia.org/T807
-- 
Bryan Davis  Wikimedia Foundationbd...@wikimedia.org
[[m:User:BDavis_(WMF)]]  Sr Software EngineerBoise, ID USA
irc: bd808v:415.839.6885 x6855

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l