[
https://issues.apache.org/jira/browse/HADOOP-16018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743220#comment-16743220
]
Steve Loughran commented on HADOOP-16018:
-----------------------------------------
I just did a local build with the "-Pyarn-ui" and it took, and just ran the npm
-install commands locally so it's not that the bower and ember-cli packages
don't exist. At the same time,
{code}
npm http GET https://registry.npmjs.org/babel/plugin-transform-modules-amd
...
npm http 405 https://registry.npmjs.org/babel/plugin-transform-modules-amd
npm http GET
https://registry.npmjs.org/watch-detector/-/watch-detector-0.1.0.tgz
npm ERR! TypeError: Cannot read property 'latest' of undefined
npm ERR! at next (/usr/share/npm/lib/cache.js:687:35)
npm ERR! at /usr/share/npm/lib/cache.js:675:5
npm ERR! at saved
(/usr/share/npm/node_modules/npm-registry-client/lib/get.js:142:7)
npm ERR! at /usr/lib/nodejs/graceful-fs/polyfills.js:133:7
npm ERR! at Object.oncomplete (fs.js:107:15)
npm ERR! If you need help, you may report this log at:
npm ERR! <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR! <[email protected]>
npm ERR! System Linux 4.4.0-138-generic
npm ERR! command "/usr/bin/nodejs" "/usr/bin/npm" "install" "-g" "ember-cli"
npm ERR! cwd /root
npm ERR! node -v v0.10.25
npm ERR! npm -v 1.3.10
npm ERR! type non_object_property_load
{code}
Trying a HEAD on the URL (or a GET) fails with method not allowed
{code}
> curl -I https://registry.npmjs.org/babel/plugin-transform-modules-amd
HTTP/2 405
..
{code}
I'm at my limit of understanding of "npm", whose purpose is to show that every
tool invents a packaging system and while some are good (tex, python), some
aren't.
Any suggestions by ~[~aw]?
> DistCp won't reassemble chunks when blocks per chunk > 0
> --------------------------------------------------------
>
> Key: HADOOP-16018
> URL: https://issues.apache.org/jira/browse/HADOOP-16018
> Project: Hadoop Common
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 3.2.0, 2.9.2
> Reporter: Kai Xie
> Assignee: Kai Xie
> Priority: Major
> Fix For: 3.0.4, 3.2.1, 3.1.3
>
> Attachments: HADOOP-16018-002.patch, HADOOP-16018-branch-2-002.patch,
> HADOOP-16018-branch-2-002.patch, HADOOP-16018-branch-2-003.patch,
> HADOOP-16018-branch-2-004.patch, HADOOP-16018-branch-2-005.patch,
> HADOOP-16018.01.patch
>
>
> I was investigating why hadoop-distcp-2.9.2 won't reassemble chunks of the
> same file when blocks per chunk has been set > 0.
> In the CopyCommitter::commitJob, this logic can prevent chunks from
> reassembling if blocks per chunk is equal to 0:
> {code:java}
> if (blocksPerChunk > 0) {
> concatFileChunks(conf);
> }
> {code}
> Then in CopyCommitter's ctor, blocksPerChunk is initialised from the config:
> {code:java}
> blocksPerChunk = context.getConfiguration().getInt(
> DistCpOptionSwitch.BLOCKS_PER_CHUNK.getConfigLabel(), 0);
> {code}
>
> But here the config key DistCpOptionSwitch.BLOCKS_PER_CHUNK.getConfigLabel()
> will always returns empty string because it is constructed without config
> label:
> {code:java}
> BLOCKS_PER_CHUNK("",
> new Option("blocksperchunk", true, "If set to a positive value, files"
> + "with more blocks than this value will be split into chunks of "
> + "<blocksperchunk> blocks to be transferred in parallel, and "
> + "reassembled on the destination. By default, <blocksperchunk> is "
> + "0 and the files will be transmitted in their entirety without "
> + "splitting. This switch is only applicable when the source file "
> + "system implements getBlockLocations method and the target file "
> + "system implements concat method"))
> {code}
> As a result it will fall back to the default value 0 for blocksPerChunk, and
> prevent the chunks from reassembling.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
