Re: [gentoo-user] npm: ERR! cb() never called!
On 18/09/2015 00:48, Michael Orlitzky wrote: > On 09/17/2015 05:13 PM, Alan McKinnon wrote: >> >> Slightly OT, but the general idea of package management isn't hard. >> >> Put the stuff you should have in a list, then compare what you should >> have to what you do have. Go get and install what you don't have, then >> make a record that you did it. >> >> Everything needed to get these basics right has been known for 40 years >> or more - fellows like Wirth and Dijkstra figured it all out way back when. >> >> Sure, there's always modern stumbling blocks (like why we have subslots) >> but that's extra to the essential basics. >> >> So why oh why do the latest generation of wunderkinds (not) always get >> it so completely WRONG? 3 runs to fetch all the deps? I suppose wget and >> curl don't actually do what I think they do then >> > > Heavy on bad words and light on solutions, but it made me feel better: > > http://michael.orlitzky.com/articles/motherfuckers_need_package_management.php I read it, and now I also feel much better :-) -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] npm: ERR! cb() never called!
On 18/09/2015 05:34, Alec Ten Harmsel wrote: >>> This sucks, but it seems to be the way a lot of web stuff is deployed >>> > > these days. > All these dynamic languages suffer from the fun problem that developers > that don't write enough tests have essentially no guarantee that their > code actually parses. I can't count how many times I've run Ruby, > Python, or Bash scripts only to have 'variable not found' or type > errors; all of the things that compilers do really well[1]. I well remember flameeyes' bitching about packaging ruby gems. One problem kept coming up over and over: tests would fail, usually because the dev used some code buried deep in his own workstation, and that HAD NEVER BEEN RELEASED. Braindeaddeaddead -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] npm: ERR! cb() never called!
On Fri, 18 Sep 2015 08:57:55 +0200, Alan McKinnon wrote: > > Heavy on bad words and light on solutions, but it made me feel better: > > > > http://michael.orlitzky.com/articles/motherfuckers_need_package_management.php > > > > I read it, and now I also feel much better :-) +1 It's a shame that those it is aimed at will either never read it or read it and think it doesn't apply to them :( -- Neil Bothwick Oxymoron: Reagan memoirs. pgpgpuQHTEUJH.pgp Description: OpenPGP digital signature
[gentoo-user] npm: ERR! cb() never called!
Anyone here familiar with driving nodejs and npm?
I'm trying to write an ebuild for a musicbrainz mirror server and "npm
install" keeps erroring out with one of two errors:
1. The install does finish but npm doesn't get around to exiting,
verified by stopping the emerge, running npm install manually and seeing
that it does nothing. When it stalls strace shows the last call was
(poll,), which implies a race condition.
2. More and more often now I get the dreaded "npm: ERR! cb() never
called!" error message which Google and stackoverflow say has been an
ongoing issue for 3 years now. If I keep retrying it eventually
succeeds, implying a race condition of some sort.
I do this in src_prepare(): [1]
src_prepare() {
npm install npm || die
npm install || die
}
following the lead of the ony other ebuild in the tree that does npm
install - bokeh-0.7.1
The package.json file from musicbrainz.com is this, it looks OK to me:
{
"name": "musicbrainz-server",
"version": "0.0.0",
"description": "package.json for keeping track of musicbrainz-server
nodejs dependencies",
"repository": {
"type": "git",
"url": "https://github.com/metabrainz/musicbrainz-server.git;
},
"license": "GPLv2+",
"dependencies": {
"aclass": "0.5.1",
"babel": "^5.8.21",
"babel-core": "^5.8.22",
"babelify": "^6.1.3",
"balanced-match": "0.2.0",
"cookie": "^0.1.2",
"envify": "^3.4.0",
"filesize": "2.0.4",
"gulp": "^3.8.7",
"gulp-less": "^3.0.0",
"gulp-rev": "^2.0.1",
"gulp-streamify": "^0.0.5",
"immutable": "3.6.4",
"jed": "1.1.0",
"jquery": "1.11.2",
"jquery.browser": "gabceb/jquery-browser-plugin#e4a01fd",
"knockout": "mwiencek/knockout#a8f12df",
"knockout-arraytransforms": "^2.0.0",
"less-plugin-clean-css": "^1.4.0",
"leven": "^1.0.2",
"lodash": "^3.9.3",
"parse-stack": "0.1.3",
"po2json": "^0.3.2",
"q": "^1.1.1",
"rcss": "0.1.4",
"react": "0.13.1",
"shelljs": "^0.3.0",
"tablesorter": "Mottie/tablesorter#430f8c5",
"through2": "^0.6.1",
"uglifyify": "^3.0.1",
"vinyl": "^0.4.6",
"vinyl-source-stream": "^1.0.0",
"yarb": "^0.4.4"
},
"devDependencies": {
"eslint": "^0.24.0",
"eslint-plugin-react": "^2.6.4",
"gulp-watch": "^4.2.1",
"tape": "^4.0.0"
},
"private": true
}
--
Alan McKinnon
alan.mckin...@gmail.com
Re: [gentoo-user] npm: ERR! cb() never called!
On 09/17/2015 10:03 AM, Alan McKinnon wrote:
> Anyone here familiar with driving nodejs and npm?
>
> I'm trying to write an ebuild for a musicbrainz mirror server and "npm
> install" keeps erroring out with one of two errors:
>
> 1. The install does finish but npm doesn't get around to exiting,
> verified by stopping the emerge, running npm install manually and seeing
> that it does nothing. When it stalls strace shows the last call was
> (poll,), which implies a race condition.
>
> 2. More and more often now I get the dreaded "npm: ERR! cb() never
> called!" error message which Google and stackoverflow say has been an
> ongoing issue for 3 years now. If I keep retrying it eventually
> succeeds, implying a race condition of some sort.
>
I went through this wonderful experience a few weeks ago. You're not
allowed to access the network in src_prepare, so that might be
contributing to your weirdness. I came up with two options:
1) Run `npm install` on your dev machine, and then package up the result
as a tarball. Generate the manifest from the tarball, and then in your
src_install, just copy stuff over.
src_install(){
local npm_module_dir="/usr/$(get_libdir)/node/${PN}"
insinto "${npm_module_dir}"
doins -r whatever
...
}
This is the lazy way, but avoids you having to package 1,000 other
things all written by people who just "learned to code" by googling HOW
DO I HTML5.
2) The right way to do it is to use an eclass and install all of the
dependencies using separate packages. As you can imagine, this is a
nightmare if you have more than a few dependencies (looks like you do).
I started an eclass for npm packages. I left the overlay here:
https://github.com/orlitzky/npm
but no one else seemed interested in having it in-tree, and the whole
ecosystem is kind of scary to me anyway.
So, for the large package I need, I'm doing it the lazy way: npm install
on my machine, and make an ebuild for the resulting huge tarball.
Re: [gentoo-user] npm: ERR! cb() never called!
On 17/09/2015 20:50, Michael Orlitzky wrote:
> On 09/17/2015 10:03 AM, Alan McKinnon wrote:
>> Anyone here familiar with driving nodejs and npm?
>>
>> I'm trying to write an ebuild for a musicbrainz mirror server and "npm
>> install" keeps erroring out with one of two errors:
>>
>> 1. The install does finish but npm doesn't get around to exiting,
>> verified by stopping the emerge, running npm install manually and seeing
>> that it does nothing. When it stalls strace shows the last call was
>> (poll,), which implies a race condition.
>>
>> 2. More and more often now I get the dreaded "npm: ERR! cb() never
>> called!" error message which Google and stackoverflow say has been an
>> ongoing issue for 3 years now. If I keep retrying it eventually
>> succeeds, implying a race condition of some sort.
>>
>
Hi Michael,
I reached pretty much the same conclusions as you. And yes, it is
src_prepare() not allowing network access that caused most of the
weirdness. I must have had the packages cached causing them to install
correctly the first time. I've since cleaned the npm cache and of course
now they fail.
After I posted my mail I went through the npm bugs at bgo and the
unstated message from the gentoo devs is loud and clear - "you are not
putting that shit in the tree"
So I took the easiest possible way out: deleted src_prepare() and issued
an elog to "cd $inst_dir && npm install, and do not use -g"
> I went through this wonderful experience a few weeks ago. You're not
> allowed to access the network in src_prepare, so that might be
> contributing to your weirdness. I came up with two options:
>
> 1) Run `npm install` on your dev machine, and then package up the result
> as a tarball. Generate the manifest from the tarball, and then in your
> src_install, just copy stuff over.
>
> src_install(){
> local npm_module_dir="/usr/$(get_libdir)/node/${PN}"
> insinto "${npm_module_dir}"
> doins -r whatever
> ...
> }
>
>
> This is the lazy way, but avoids you having to package 1,000 other
> things all written by people who just "learned to code" by googling HOW
> DO I HTML5.
Agreed. The quality of node software is atrocious, and the package
manager is even worse. read "npm faq", it is most enlightening and gives
insight into people's heads
>
> 2) The right way to do it is to use an eclass and install all of the
> dependencies using separate packages. As you can imagine, this is a
> nightmare if you have more than a few dependencies (looks like you do).
Errr, no :-)
g-cpan is bad enough and those ebuilds are mostly template-able. At
least CPAN modules mostly respond correctly to make && make install.
That node stuff doesn't look like it will ever package sanely.
>
> I started an eclass for npm packages. I left the overlay here:
>
> https://github.com/orlitzky/npm
>
> but no one else seemed interested in having it in-tree, and the whole
> ecosystem is kind of scary to me anyway.
Indeed.
>
> So, for the large package I need, I'm doing it the lazy way: npm install
> on my machine, and make an ebuild for the resulting huge tarball.
Thanks for the feedback
--
Alan McKinnon
alan.mckin...@gmail.com
Re: [gentoo-user] npm: ERR! cb() never called!
On Thu, Sep 17, 2015 at 09:24:39PM +0200, Alan McKinnon wrote: > On 17/09/2015 20:50, Michael Orlitzky wrote: > > On 09/17/2015 10:03 AM, Alan McKinnon wrote: > >> Anyone here familiar with driving nodejs and npm? > >> > >> I'm trying to write an ebuild for a musicbrainz mirror server and "npm > >> install" keeps erroring out with one of two errors: Haha, npm. First time I ever ran npm, it required 3 runs before it actually managed to fetch all the dependencies. Hopefully it's better now. > > > > 2) The right way to do it is to use an eclass and install all of the > > dependencies using separate packages. As you can imagine, this is a > > nightmare if you have more than a few dependencies (looks like you do). > > Errr, no :-) > > g-cpan is bad enough and those ebuilds are mostly template-able. At > least CPAN modules mostly respond correctly to make && make install. > That node stuff doesn't look like it will ever package sanely. Unfortunately, the right way with nodejs/ruby web stuff is to use the tooling specific to the language. If this[1] is what you're trying to deploy, I feel sorry. If I was serious about deploying this, I would: 1. Fork the repo and add a remote on my own server 2. Add your custom configuration 3. Write a small shell script that 1. Runs `git pull` from your own infrastructure 2. Installs perl/node deps locally 3. Runs the gulp build 4. Runs plackup 4. Add an init script that runs that start script This sucks, but it seems to be the way a lot of web stuff is deployed these days. Hope this helps, Alec [1] https://github.com/metabrainz/musicbrainz-server
Re: [gentoo-user] npm: ERR! cb() never called!
On 17/09/2015 22:53, Alec Ten Harmsel wrote: > On Thu, Sep 17, 2015 at 09:24:39PM +0200, Alan McKinnon wrote: >> On 17/09/2015 20:50, Michael Orlitzky wrote: >>> On 09/17/2015 10:03 AM, Alan McKinnon wrote: Anyone here familiar with driving nodejs and npm? I'm trying to write an ebuild for a musicbrainz mirror server and "npm install" keeps erroring out with one of two errors: > > Haha, npm. First time I ever ran npm, it required 3 runs before it > actually managed to fetch all the dependencies. Hopefully it's better > now. Slightly OT, but the general idea of package management isn't hard. Put the stuff you should have in a list, then compare what you should have to what you do have. Go get and install what you don't have, then make a record that you did it. Everything needed to get these basics right has been known for 40 years or more - fellows like Wirth and Dijkstra figured it all out way back when. Sure, there's always modern stumbling blocks (like why we have subslots) but that's extra to the essential basics. So why oh why do the latest generation of wunderkinds (not) always get it so completely WRONG? 3 runs to fetch all the deps? I suppose wget and curl don't actually do what I think they do then > >>> >>> 2) The right way to do it is to use an eclass and install all of the >>> dependencies using separate packages. As you can imagine, this is a >>> nightmare if you have more than a few dependencies (looks like you do). >> >> Errr, no :-) >> >> g-cpan is bad enough and those ebuilds are mostly template-able. At >> least CPAN modules mostly respond correctly to make && make install. >> That node stuff doesn't look like it will ever package sanely. > > Unfortunately, the right way with nodejs/ruby web stuff is to use the > tooling specific to the language. If this[1] is what you're trying to > deploy, I feel sorry. Yes, that's the one > > If I was serious about deploying this, I would: > > 1. Fork the repo and add a remote on my own server > 2. Add your custom configuration > 3. Write a small shell script that > 1. Runs `git pull` from your own infrastructure > 2. Installs perl/node deps locally > 3. Runs the gulp build > 4. Runs plackup > 4. Add an init script that runs that start script I followed that mostly except for forking the repo and writing a small shell script - I much prefer proper ebuilds to hacky scripts. Even though I'm a Linux sysadmin I hate ad-hoc shell scripts with a passion > > This sucks, but it seems to be the way a lot of web stuff is deployed > these days. > > Hope this helps, > > Alec > > [1] https://github.com/metabrainz/musicbrainz-server > -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] npm: ERR! cb() never called!
On 09/17/2015 05:13 PM, Alan McKinnon wrote: > > Slightly OT, but the general idea of package management isn't hard. > > Put the stuff you should have in a list, then compare what you should > have to what you do have. Go get and install what you don't have, then > make a record that you did it. > > Everything needed to get these basics right has been known for 40 years > or more - fellows like Wirth and Dijkstra figured it all out way back when. > > Sure, there's always modern stumbling blocks (like why we have subslots) > but that's extra to the essential basics. > > So why oh why do the latest generation of wunderkinds (not) always get > it so completely WRONG? 3 runs to fetch all the deps? I suppose wget and > curl don't actually do what I think they do then > Heavy on bad words and light on solutions, but it made me feel better: http://michael.orlitzky.com/articles/motherfuckers_need_package_management.php
Re: [gentoo-user] npm: ERR! cb() never called!
On Thu, Sep 17, 2015 at 11:13:10PM +0200, Alan McKinnon wrote:
> On 17/09/2015 22:53, Alec Ten Harmsel wrote:
> > Unfortunately, the right way with nodejs/ruby web stuff is to use the
> > tooling specific to the language. If this[1] is what you're trying to
> > deploy, I feel sorry.
>
> Yes, that's the one
Yay, perl, perl6, and JS. All at the same time.
> >
> > If I was serious about deploying this, I would:
> >
> > 1. Fork the repo and add a remote on my own server
> > 2. Add your custom configuration
> > 3. Write a small shell script that
> > 1. Runs `git pull` from your own infrastructure
> > 2. Installs perl/node deps locally
> > 3. Runs the gulp build
> > 4. Runs plackup
> > 4. Add an init script that runs that start script
>
> I followed that mostly except for forking the repo and writing a small
> shell script - I much prefer proper ebuilds to hacky scripts. Even
> though I'm a Linux sysadmin I hate ad-hoc shell scripts with a passion
I am the same way; I do not like shell scripts. Anything longer than 15
lines or so is written in Ruby, other than my firewall setup script.
I'm not proud of myself for suggesting what I did; just trying to
minimize your pain. Another option could be something along the lines
of:
1. ebuild installs files to `/var/musicbrainz` or wherever
2. Add a small shell script to `/var/musicbrainz` or wherever that:
1. Installs perl/node deps locally
2. Runs the gulp build
3. Runs plackup
3. Add an init script that runs that start script
I guess, since you're not developing musicbrainz thingy, that the git
repo was a bit of overkill.
> > This sucks, but it seems to be the way a lot of web stuff is deployed
> > these days.
All these dynamic languages suffer from the fun problem that developers
that don't write enough tests have essentially no guarantee that their
code actually parses. I can't count how many times I've run Ruby,
Python, or Bash scripts only to have 'variable not found' or type
errors; all of the things that compilers do really well[1].
One of the things that results from this (IMO) is that they bundle
deps/enforce strict versions on stuff because they have to just to run
stuff. For example, a while ago www-apps/jekyll was broken because a gem
it depended on happened to be a newer version and changed the API enough
that jekyll broke. To be fair to the Gentoo developers, jekyll is
~amd64, so I didn't really care.
Alec
[1] For example, below are two *valid* files; one Python, one Ruby. Both
would obviously fail to compile if it was transposed to C/C++/Java, for
good reason.
#!/usr/bin/env ruby
if false
puts hey
else
puts 'hey'
end
#!/usr/bin/env python
if False:
print(hey)
else:
print('hey')
