[jira] [Commented] (HADOOP-12928) Update netty to 3.10.5.Final to sync with zookeper
[
https://issues.apache.org/jira/browse/HADOOP-12928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383426#comment-15383426
]
Hadoop QA commented on HADOOP-12928:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
27s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
6s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 27s{color} | {color:orange} root: The patch generated 2 new + 64 unchanged -
1 fixed = 66 total (was 65) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
13s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 30s{color}
| {color:red} hadoop-mapreduce-client-shuffle in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.mapred.TestShuffleHandler |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12818688/HDFS-12928.00.patch |
| JIRA Issue | HADOOP-12928 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux f6a3eaf35a5c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64
[jira] [Commented] (HADOOP-13184) Add "Apache" to Hadoop project logo
[ https://issues.apache.org/jira/browse/HADOOP-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383424#comment-15383424 ] Chris Douglas commented on HADOOP-13184: bq. When the vote ends? I'd hoped to avoid a vote on JIRA, but here we are. I count 9 votes for #1, 7 votes for #4, 2 for #5. [~kspk], could you create the final version of #1? [~curcuru], just to be clear, should the logo have a TM or an (R)? > Add "Apache" to Hadoop project logo > --- > > Key: HADOOP-13184 > URL: https://issues.apache.org/jira/browse/HADOOP-13184 > Project: Hadoop Common > Issue Type: Task >Reporter: Chris Douglas >Assignee: Abhishek > > Many ASF projects include "Apache" in their logo. We should add it to Hadoop. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12912) Add LOG.isDebugEnabled() guard in Progress.set method
[
https://issues.apache.org/jira/browse/HADOOP-12912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383385#comment-15383385
]
Gera Shegalov commented on HADOOP-12912:
Oops I take the above back looking at how isDebugEnabled is implemented
{code}
/**
* Check whether the Log4j Logger used is enabled for DEBUG
priority.
*/
public boolean isDebugEnabled() {
return getLogger().isDebugEnabled();
}
{code}
sorry for the noise!
> Add LOG.isDebugEnabled() guard in Progress.set method
> -
>
> Key: HADOOP-12912
> URL: https://issues.apache.org/jira/browse/HADOOP-12912
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Tsuyoshi Ozawa
>Assignee: Tsuyoshi Ozawa
> Attachments: HADOOP-12912.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11540) Raw Reed-Solomon coder using Intel ISA-L library
[ https://issues.apache.org/jira/browse/HADOOP-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383384#comment-15383384 ] Kai Zheng commented on HADOOP-11540: Hi [~andrew.wang], I noticed you had cut the branch. Would you help check if the latest update has addressed Aron's comments or not? Will you include this in the first release? Thanks for your help. > Raw Reed-Solomon coder using Intel ISA-L library > > > Key: HADOOP-11540 > URL: https://issues.apache.org/jira/browse/HADOOP-11540 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: HDFS-7285 >Reporter: Zhe Zhang >Assignee: Kai Zheng > Labels: hdfs-ec-3.0-must-do > Attachments: HADOOP-11540-initial.patch, HADOOP-11540-v1.patch, > HADOOP-11540-v10.patch, HADOOP-11540-v11.patch, HADOOP-11540-v12.patch, > HADOOP-11540-v2.patch, HADOOP-11540-v4.patch, HADOOP-11540-v5.patch, > HADOOP-11540-v6.patch, HADOOP-11540-v7.patch, HADOOP-11540-v8.patch, > HADOOP-11540-v9.patch, HADOOP-11540-with-11996-codes.patch, Native Erasure > Coder Performance - Intel ISAL-v1.pdf > > > This is to provide RS codec implementation using Intel ISA-L library for > encoding and decoding. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13383) Update release notes for 3.0.0-alpha1
[ https://issues.apache.org/jira/browse/HADOOP-13383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383382#comment-15383382 ] Sangjin Lee commented on HADOOP-13383: -- Thanks Andrew! > Update release notes for 3.0.0-alpha1 > - > > Key: HADOOP-13383 > URL: https://issues.apache.org/jira/browse/HADOOP-13383 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Blocker > Attachments: HADOOP-13383.001.patch, HADOOP-13383.002.patch > > > Per the release instructions (https://wiki.apache.org/hadoop/HowToRelease), > we need to update hadoop-project/src/site/markdown/index.md.vm to reflect the > right versions, new features and big improvements. > I can put together some notes for HADOOP and HDFS, depending on others for > YARN and MR. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12912) Add LOG.isDebugEnabled() guard in Progress.set method
[ https://issues.apache.org/jira/browse/HADOOP-12912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383375#comment-15383375 ] Gera Shegalov commented on HADOOP-12912: isDebugEnabled is worth doing for "best practices" reason imo. I just noticed it in a thread dump. It's much more than a branching instruction. there a bunch of invokevirtual in commons and log4j logging and invokeinterface call behind it. > Add LOG.isDebugEnabled() guard in Progress.set method > - > > Key: HADOOP-12912 > URL: https://issues.apache.org/jira/browse/HADOOP-12912 > Project: Hadoop Common > Issue Type: Bug >Reporter: Tsuyoshi Ozawa >Assignee: Tsuyoshi Ozawa > Attachments: HADOOP-12912.001.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13202) Avoid possible overflow in org.apache.hadoop.util.bloom.BloomFilter#getNBytes
[ https://issues.apache.org/jira/browse/HADOOP-13202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383368#comment-15383368 ] Kai Sasaki commented on HADOOP-13202: - [~ajisakaa] Thank you so much! > Avoid possible overflow in org.apache.hadoop.util.bloom.BloomFilter#getNBytes > - > > Key: HADOOP-13202 > URL: https://issues.apache.org/jira/browse/HADOOP-13202 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 2.7.2 >Reporter: zhengbing li >Assignee: Kai Sasaki > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha2 > > Attachments: HADOOP-13202.01.patch, HADOOP-13202.02.patch, > HADOOP-13202.03.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > Current implementation: > return (vectorSize + 7) / 8; > when vectorSize is 2147483647（the max value of Int), error > :"java.lang.NegativeArraySizeException" will report > the implementation might be changed > return (int)(((long)vectorSize + 7) / 8); -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13301) Millisecond timestamp for FsShell console log and MapReduce jobsummary log
[
https://issues.apache.org/jira/browse/HADOOP-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383344#comment-15383344
]
Hudson commented on HADOOP-13301:
-
SUCCESS: Integrated in Hadoop-trunk-Commit #10116 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/10116/])
HADOOP-13301. Millisecond timestamp for FsShell console log and (aajisaka: rev
92fe2db540a7c563789442355d3bf6aa169a4cce)
* hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
> Millisecond timestamp for FsShell console log and MapReduce jobsummary log
> --
>
> Key: HADOOP-13301
> URL: https://issues.apache.org/jira/browse/HADOOP-13301
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13301.001.patch
>
>
> The log message timestamp on FsShell console show only seconds.
> {noformat}
> $ export HADOOP_ROOT_LOGGER=TRACE,console
> $ hdfs dfs -rm -skipTrash /tmp/2G*
> 16/06/20 16:00:03 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
> Would like to see milliseconds for quick performance turning.
> {noformat}
> 2016-06-20 16:01:42,588 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13240) TestAclCommands.testSetfaclValidations fail
[
https://issues.apache.org/jira/browse/HADOOP-13240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383285#comment-15383285
]
Chris Nauroth commented on HADOOP-13240:
[~jzhuge], thank you for finding a repro and coding a patch. This looks good
to me overall.
{code}
if (aclEntries.size() == 0) {
{code}
A minor nit-pick: please switch to {{aclEntries.isEmpty()}} for increased
readability.
Did you also run {{TestAclCLI}} in the hadoop-hdfs module with this patch
applied? Pre-commit will see this patch as a change in hadoop-common only, so
it will only run the tests in that module.
> TestAclCommands.testSetfaclValidations fail
> ---
>
> Key: HADOOP-13240
> URL: https://issues.apache.org/jira/browse/HADOOP-13240
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.4.1, 2.7.1
> Environment: hadoop 2.4.1,as6.5
>Reporter: linbao111
>Assignee: John Zhuge
>Priority: Minor
> Attachments: HADOOP-13240.001.patch, HADOOP-13240.002.patch
>
>
> mvn test -Djava.net.preferIPv4Stack=true -Dlog4j.rootLogger=DEBUG,console
> -Dtest=TestAclCommands#testSetfaclValidations failed with following message:
> ---
> Test set: org.apache.hadoop.fs.shell.TestAclCommands
> ---
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.599 sec <<<
> FAILURE! - in org.apache.hadoop.fs.shell.TestAclCommands
> testSetfaclValidations(org.apache.hadoop.fs.shell.TestAclCommands) Time
> elapsed: 0.534 sec <<< FAILURE!
> java.lang.AssertionError: setfacl should fail ACL spec missing
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertFalse(Assert.java:68)
> at
> org.apache.hadoop.fs.shell.TestAclCommands.testSetfaclValidations(TestAclCommands.java:81)
> i notice from
> HADOOP-10277,hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/AclEntry.java
> code changed
> should
> hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.javabe
> changed to:
> diff --git
> a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
>
> b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> index b14cd37..463bfcd
> ---
> a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> +++
> b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> @@ -80,7 +80,7 @@ public void testSetfaclValidations() throws Exception {
> "/path" }));
> assertFalse("setfacl should fail ACL spec missing",
> 0 == runCommand(new String[] { "-setfacl", "-m",
> -"", "/path" }));
> +":", "/path" }));
>}
>
>@Test
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13385) Upgrade Avro to 1.8.x
[ https://issues.apache.org/jira/browse/HADOOP-13385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben McCann resolved HADOOP-13385. - Resolution: Duplicate > Upgrade Avro to 1.8.x > - > > Key: HADOOP-13385 > URL: https://issues.apache.org/jira/browse/HADOOP-13385 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ben McCann > > Avro 1.8.x makes generated classes serializable which makes them much easier > to use with Spark. It would be great to upgrade Avro to 1.8.x -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13386) Upgrade Avro to 1.8.x
Ben McCann created HADOOP-13386: --- Summary: Upgrade Avro to 1.8.x Key: HADOOP-13386 URL: https://issues.apache.org/jira/browse/HADOOP-13386 Project: Hadoop Common Issue Type: Sub-task Reporter: Ben McCann Avro 1.8.x makes generated classes serializable which makes them much easier to use with Spark. It would be great to upgrade Avro to 1.8.x -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13385) Upgrade Avro to 1.8.x
Ben McCann created HADOOP-13385: --- Summary: Upgrade Avro to 1.8.x Key: HADOOP-13385 URL: https://issues.apache.org/jira/browse/HADOOP-13385 Project: Hadoop Common Issue Type: Improvement Reporter: Ben McCann Avro 1.8.x makes generated classes serializable which makes them much easier to use with Spark. It would be great to upgrade Avro to 1.8.x -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13301) Millisecond timestamp for FsShell console log and MapReduce jobsummary log
[
https://issues.apache.org/jira/browse/HADOOP-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383258#comment-15383258
]
John Zhuge commented on HADOOP-13301:
-
Thanks [~ajisakaa].
> Millisecond timestamp for FsShell console log and MapReduce jobsummary log
> --
>
> Key: HADOOP-13301
> URL: https://issues.apache.org/jira/browse/HADOOP-13301
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13301.001.patch
>
>
> The log message timestamp on FsShell console show only seconds.
> {noformat}
> $ export HADOOP_ROOT_LOGGER=TRACE,console
> $ hdfs dfs -rm -skipTrash /tmp/2G*
> 16/06/20 16:00:03 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
> Would like to see milliseconds for quick performance turning.
> {noformat}
> 2016-06-20 16:01:42,588 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12527) Upgrade Avro dependency to 1.7.7
[ https://issues.apache.org/jira/browse/HADOOP-12527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383245#comment-15383245 ] Ben McCann commented on HADOOP-12527: - Note that there are no incompatible changes introduced in Avro 1.7.5-1.7.7. See https://github.com/apache/avro/blob/master/CHANGES.txt > Upgrade Avro dependency to 1.7.7 > > > Key: HADOOP-12527 > URL: https://issues.apache.org/jira/browse/HADOOP-12527 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 2.7.1 >Reporter: Jonathan Kelly > > Hadoop has depended upon Avro 1.7.4 for a couple of years now (see > HADOOP-9672), but Apache Spark depends upon what is currently the latest > version of Avro (1.7.7). > This can cause issues if Spark is configured to include the full Hadoop > classpath, as the classpath would then contain both Avro 1.7.4 and 1.7.7, > with the 1.7.4 classes possibly winning depending on ordering. Here is an > example of this issue: > http://stackoverflow.com/questions/33159254/avro-error-on-aws-emr/33403111#33403111 > Would it be possible to upgrade Hadoop's Avro dependency to 1.7.7 now? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12928) Update netty to 3.10.5.Final to sync with zookeper
[ https://issues.apache.org/jira/browse/HADOOP-12928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HADOOP-12928: --- Status: Patch Available (was: Open) > Update netty to 3.10.5.Final to sync with zookeper > -- > > Key: HADOOP-12928 > URL: https://issues.apache.org/jira/browse/HADOOP-12928 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 2.7.2 >Reporter: Hendy Irawan >Assignee: Lei (Eddy) Xu > Attachments: HDFS-12928.00.patch > > > Update netty to 3.7.1.Final because hadoop-client 2.7.2 depends on zookeeper > 3.4.6 which depends on netty 3.7.x. Related to HADOOP-12927 > Pull request: https://github.com/apache/hadoop/pull/85 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12928) Update netty to 3.10.5.Final to sync with zookeper
[
https://issues.apache.org/jira/browse/HADOOP-12928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei (Eddy) Xu updated HADOOP-12928:
---
Attachment: HDFS-12928.00.patch
Hi, [~ceefour] Thanks for reporting this.
Zookeeper has upgraded netty to {{3.10.5 final}}. So I updated the patch to
sync with newest version.
This version also fixes the security vulnerabilities:
[CVE-2014-3488|https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-3488]
> Update netty to 3.10.5.Final to sync with zookeper
> --
>
> Key: HADOOP-12928
> URL: https://issues.apache.org/jira/browse/HADOOP-12928
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 2.7.2
>Reporter: Hendy Irawan
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-12928.00.patch
>
>
> Update netty to 3.7.1.Final because hadoop-client 2.7.2 depends on zookeeper
> 3.4.6 which depends on netty 3.7.x. Related to HADOOP-12927
> Pull request: https://github.com/apache/hadoop/pull/85
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13383) Update release notes for 3.0.0-alpha1
[ https://issues.apache.org/jira/browse/HADOOP-13383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-13383: - Attachment: HADOOP-13383.002.patch Thanks to multiple people for updating the release notes for their features. I've created a new rev based on this input, which I hope is comprehensive. Added a little more intro text and changed the formatting too. Tested by "mvn site:site -Dmaven.javadoc.skip" and viewing the HTML output. > Update release notes for 3.0.0-alpha1 > - > > Key: HADOOP-13383 > URL: https://issues.apache.org/jira/browse/HADOOP-13383 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Blocker > Attachments: HADOOP-13383.001.patch, HADOOP-13383.002.patch > > > Per the release instructions (https://wiki.apache.org/hadoop/HowToRelease), > we need to update hadoop-project/src/site/markdown/index.md.vm to reflect the > right versions, new features and big improvements. > I can put together some notes for HADOOP and HDFS, depending on others for > YARN and MR. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13301) Millisecond timestamp for FsShell console log and MapReduce jobsummary log
[
https://issues.apache.org/jira/browse/HADOOP-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-13301:
---
Resolution: Fixed
Fix Version/s: 3.0.0-alpha2
Status: Resolved (was: Patch Available)
Committed this to trunk. Thanks [~jzhuge] for the contribution and thanks
[~eddyxu] for the review.
> Millisecond timestamp for FsShell console log and MapReduce jobsummary log
> --
>
> Key: HADOOP-13301
> URL: https://issues.apache.org/jira/browse/HADOOP-13301
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13301.001.patch
>
>
> The log message timestamp on FsShell console show only seconds.
> {noformat}
> $ export HADOOP_ROOT_LOGGER=TRACE,console
> $ hdfs dfs -rm -skipTrash /tmp/2G*
> 16/06/20 16:00:03 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
> Would like to see milliseconds for quick performance turning.
> {noformat}
> 2016-06-20 16:01:42,588 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12928) Update netty to 3.10.5.Final to sync with zookeper
[ https://issues.apache.org/jira/browse/HADOOP-12928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HADOOP-12928: --- Summary: Update netty to 3.10.5.Final to sync with zookeper (was: Update netty to 3.7.1.Final to sync with zookeper) > Update netty to 3.10.5.Final to sync with zookeper > -- > > Key: HADOOP-12928 > URL: https://issues.apache.org/jira/browse/HADOOP-12928 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 2.7.2 >Reporter: Hendy Irawan >Assignee: Lei (Eddy) Xu > > Update netty to 3.7.1.Final because hadoop-client 2.7.2 depends on zookeeper > 3.4.6 which depends on netty 3.7.x. Related to HADOOP-12927 > Pull request: https://github.com/apache/hadoop/pull/85 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13301) Millisecond timestamp for FsShell console log
[
https://issues.apache.org/jira/browse/HADOOP-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-13301:
---
Priority: Minor (was: Trivial)
> Millisecond timestamp for FsShell console log
> -
>
> Key: HADOOP-13301
> URL: https://issues.apache.org/jira/browse/HADOOP-13301
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Minor
> Attachments: HADOOP-13301.001.patch
>
>
> The log message timestamp on FsShell console show only seconds.
> {noformat}
> $ export HADOOP_ROOT_LOGGER=TRACE,console
> $ hdfs dfs -rm -skipTrash /tmp/2G*
> 16/06/20 16:00:03 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
> Would like to see milliseconds for quick performance turning.
> {noformat}
> 2016-06-20 16:01:42,588 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13301) Millisecond timestamp for FsShell console log and MapReduce jobsummary log
[
https://issues.apache.org/jira/browse/HADOOP-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-13301:
---
Summary: Millisecond timestamp for FsShell console log and MapReduce
jobsummary log (was: Millisecond timestamp for FsShell console log)
> Millisecond timestamp for FsShell console log and MapReduce jobsummary log
> --
>
> Key: HADOOP-13301
> URL: https://issues.apache.org/jira/browse/HADOOP-13301
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Minor
> Attachments: HADOOP-13301.001.patch
>
>
> The log message timestamp on FsShell console show only seconds.
> {noformat}
> $ export HADOOP_ROOT_LOGGER=TRACE,console
> $ hdfs dfs -rm -skipTrash /tmp/2G*
> 16/06/20 16:00:03 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
> Would like to see milliseconds for quick performance turning.
> {noformat}
> 2016-06-20 16:01:42,588 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13301) Millisecond timestamp for FsShell console log
[
https://issues.apache.org/jira/browse/HADOOP-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-13301:
---
Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change)
Release Note: The time format of console logger and MapReduce job summary
logger is ISO8601 by default to print milliseconds.
> Millisecond timestamp for FsShell console log
> -
>
> Key: HADOOP-13301
> URL: https://issues.apache.org/jira/browse/HADOOP-13301
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Trivial
> Attachments: HADOOP-13301.001.patch
>
>
> The log message timestamp on FsShell console show only seconds.
> {noformat}
> $ export HADOOP_ROOT_LOGGER=TRACE,console
> $ hdfs dfs -rm -skipTrash /tmp/2G*
> 16/06/20 16:00:03 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
> Would like to see milliseconds for quick performance turning.
> {noformat}
> 2016-06-20 16:01:42,588 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12527) Upgrade Avro dependency to 1.7.7
[ https://issues.apache.org/jira/browse/HADOOP-12527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383211#comment-15383211 ] Ben McCann commented on HADOOP-12527: - Hive is already using 1.7.7. See here: https://github.com/apache/hive/blob/master/pom.xml#L112 And it looks like HBase is using 1.7.6: https://github.com/apache/hbase/blob/master/hbase-spark/pom.xml#L45 > Upgrade Avro dependency to 1.7.7 > > > Key: HADOOP-12527 > URL: https://issues.apache.org/jira/browse/HADOOP-12527 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 2.7.1 >Reporter: Jonathan Kelly > > Hadoop has depended upon Avro 1.7.4 for a couple of years now (see > HADOOP-9672), but Apache Spark depends upon what is currently the latest > version of Avro (1.7.7). > This can cause issues if Spark is configured to include the full Hadoop > classpath, as the classpath would then contain both Avro 1.7.4 and 1.7.7, > with the 1.7.4 classes possibly winning depending on ordering. Here is an > example of this issue: > http://stackoverflow.com/questions/33159254/avro-error-on-aws-emr/33403111#33403111 > Would it be possible to upgrade Hadoop's Avro dependency to 1.7.7 now? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13301) Millisecond timestamp for FsShell console log
[
https://issues.apache.org/jira/browse/HADOOP-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383205#comment-15383205
]
Akira Ajisaka commented on HADOOP-13301:
+1, thanks [~jzhuge].
> Millisecond timestamp for FsShell console log
> -
>
> Key: HADOOP-13301
> URL: https://issues.apache.org/jira/browse/HADOOP-13301
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Trivial
> Attachments: HADOOP-13301.001.patch
>
>
> The log message timestamp on FsShell console show only seconds.
> {noformat}
> $ export HADOOP_ROOT_LOGGER=TRACE,console
> $ hdfs dfs -rm -skipTrash /tmp/2G*
> 16/06/20 16:00:03 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
> Would like to see milliseconds for quick performance turning.
> {noformat}
> 2016-06-20 16:01:42,588 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13301) Millisecond timestamp for FsShell console log
[
https://issues.apache.org/jira/browse/HADOOP-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-13301:
---
Hadoop Flags: Incompatible change
Marking this as incompatible change.
> Millisecond timestamp for FsShell console log
> -
>
> Key: HADOOP-13301
> URL: https://issues.apache.org/jira/browse/HADOOP-13301
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Trivial
> Attachments: HADOOP-13301.001.patch
>
>
> The log message timestamp on FsShell console show only seconds.
> {noformat}
> $ export HADOOP_ROOT_LOGGER=TRACE,console
> $ hdfs dfs -rm -skipTrash /tmp/2G*
> 16/06/20 16:00:03 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
> Would like to see milliseconds for quick performance turning.
> {noformat}
> 2016-06-20 16:01:42,588 DEBUG util.Shell: setsid exited with exit code 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12527) Upgrade Avro dependency to 1.7.7
[ https://issues.apache.org/jira/browse/HADOOP-12527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383199#comment-15383199 ] ASF GitHub Bot commented on HADOOP-12527: - Github user benmccann commented on the issue: https://github.com/apache/hadoop/pull/39 No, I don't think it's necessary. It just seems like it'd be good to do at some point > Upgrade Avro dependency to 1.7.7 > > > Key: HADOOP-12527 > URL: https://issues.apache.org/jira/browse/HADOOP-12527 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 2.7.1 >Reporter: Jonathan Kelly > > Hadoop has depended upon Avro 1.7.4 for a couple of years now (see > HADOOP-9672), but Apache Spark depends upon what is currently the latest > version of Avro (1.7.7). > This can cause issues if Spark is configured to include the full Hadoop > classpath, as the classpath would then contain both Avro 1.7.4 and 1.7.7, > with the 1.7.4 classes possibly winning depending on ordering. Here is an > example of this issue: > http://stackoverflow.com/questions/33159254/avro-error-on-aws-emr/33403111#33403111 > Would it be possible to upgrade Hadoop's Avro dependency to 1.7.7 now? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13354) Update WASB driver to use the latest version (4.2.0) of SDK for Microsoft Azure Storage Clients
[ https://issues.apache.org/jira/browse/HADOOP-13354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383163#comment-15383163 ] Dushyanth commented on HADOOP-13354: +1 to HADOOP-13354.004.patch > Update WASB driver to use the latest version (4.2.0) of SDK for Microsoft > Azure Storage Clients > --- > > Key: HADOOP-13354 > URL: https://issues.apache.org/jira/browse/HADOOP-13354 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/azure >Reporter: Sivaguru Sankaridurg > Attachments: HADOOP-13354.001.patch, HADOOP-13354.002.patch, > HADOOP-13354.003.patch, HADOOP-13354.004.patch, Test-Results-With-4.2.0-fixes > > > Update WASB driver to use the latest version (4.2.0) of SDK for Microsoft > Azure Storage Clients. > We are currently using version 2.2.0 of the SDK. > Version 4.2.0 brings some breaking changes. > Need to fix code to resolve all these breaking changes and certify that > everything works properly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-12928) Update netty to 3.7.1.Final to sync with zookeper
[ https://issues.apache.org/jira/browse/HADOOP-12928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu reassigned HADOOP-12928: -- Assignee: Lei (Eddy) Xu > Update netty to 3.7.1.Final to sync with zookeper > - > > Key: HADOOP-12928 > URL: https://issues.apache.org/jira/browse/HADOOP-12928 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 2.7.2 >Reporter: Hendy Irawan >Assignee: Lei (Eddy) Xu > > Update netty to 3.7.1.Final because hadoop-client 2.7.2 depends on zookeeper > 3.4.6 which depends on netty 3.7.x. Related to HADOOP-12927 > Pull request: https://github.com/apache/hadoop/pull/85 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13384) Add a job-conf.xml as a default resource file to load in configuration utils
[ https://issues.apache.org/jira/browse/HADOOP-13384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francois Visconte updated HADOOP-13384: --- Priority: Minor (was: Major) > Add a job-conf.xml as a default resource file to load in configuration utils > - > > Key: HADOOP-13384 > URL: https://issues.apache.org/jira/browse/HADOOP-13384 > Project: Hadoop Common > Issue Type: Improvement > Components: conf >Reporter: Francois Visconte >Priority: Minor > > We would like to make it possible for jobs to include their configuration as > a default resource loaded as a default in ConfigUtil (when available) > That would allow a job to include a job-conf.xml resource file allowing to > configure a job without having to set configuration file from inside the code > (making it a bit easier to track for configuration changes). > FYI We are also using this to inject useful configuration parameters (code > version, main class, parameters, launch time) in order to get these insights > in job.xml files in order to process these data later on (with rumen for > example). > Here is the proposed patch on our fork that I would like to propose > upstream:https://github.com/criteo-forks/hadoop-common/pull/2/commits/9bb3f7ad95989bcd9a41be4fcc0633c2c0b3 > Thanks, -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13382) remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects
[ https://issues.apache.org/jira/browse/HADOOP-13382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HADOOP-13382: --- Hadoop Flags: Reviewed Hello [~mattf]. +1 for the patch. However, I'm not currently set up to run the hadoop-openstack integration tests to validate. I really need to do that one of these days. In the meantime, [~ste...@apache.org], are you interested in doing another review? > remove unneeded commons-httpclient dependencies from POM files in Hadoop and > sub-projects > - > > Key: HADOOP-13382 > URL: https://issues.apache.org/jira/browse/HADOOP-13382 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 2.8.0 >Reporter: Matt Foley >Assignee: Matt Foley > Attachments: HADOOP-13382-branch-2.000.patch, > HADOOP-13382-branch-2.8.000.patch, HADOOP-13382.000.patch > > > In branch-2.8 and later, the patches for various child and related bugs > listed in HADOOP-10105, most recently including HADOOP-11613, HADOOP-12710, > HADOOP-12711, HADOOP-12552, and HDFS-10623, eliminate all use of > "commons-httpclient" from Hadoop and its sub-projects (except for > hadoop-tools/hadoop-openstack; see HADOOP-11614). > However, after incorporating these patches, "commons-httpclient" is still > listed as a dependency in these POM files: > * hadoop-project/pom.xml > * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/pom.xml > We wish to remove these, but since commons-httpclient is still used in many > files in hadoop-tools/hadoop-openstack, we'll need to _add_ the dependency to > * hadoop-tools/hadoop-openstack/pom.xml > (We'll add a note to HADOOP-11614 to undo this when commons-httpclient is > removed from hadoop-openstack.) > In 2.8, this was mostly done by HADOOP-12552, but the version info formerly > inherited from hadoop-project/pom.xml also needs to be added, so that is in > the branch-2.8 version of the patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13181) WASB append support: getPos incorrect
[
https://issues.apache.org/jira/browse/HADOOP-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383079#comment-15383079
]
Dushyanth commented on HADOOP-13181:
+1 to change.
> WASB append support: getPos incorrect
> -
>
> Key: HADOOP-13181
> URL: https://issues.apache.org/jira/browse/HADOOP-13181
> Project: Hadoop Common
> Issue Type: Bug
> Components: azure
>Affects Versions: 2.8.0
>Reporter: Bogdan Raducanu
> Attachments: HADOOP-13181.001.patch, append.java
>
>
> See attached code.
> Cause:
> In NativeAzureFileSystem.java: the append method returns
> {code}
> new FSDataOutputStream(appendStream, statistics)
> {code}
> Instead, it should probably return
> {code}
> new FSDataOutputStream(appendStream, statistics, meta.getLength())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12991) Conflicting default ports in DelegateToFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-12991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-12991:
---
Hadoop Flags: Reviewed
Description:
HADOOP-12304 introduced logic to ensure that the {{DelegateToFileSystem}}
constructor sets the default port to -1:
{code:title=DelegateToFileSystem.java}
protected DelegateToFileSystem(URI theUri, FileSystem theFsImpl,
Configuration conf, String supportedScheme, boolean authorityRequired)
throws IOException, URISyntaxException {
super(theUri, supportedScheme, authorityRequired,
getDefaultPortIfDefined(theFsImpl));
fsImpl = theFsImpl;
fsImpl.initialize(theUri, conf);
fsImpl.statistics = getStatistics();
}
private static int getDefaultPortIfDefined(FileSystem theFsImpl) {
int defaultPort = theFsImpl.getDefaultPort();
return defaultPort != 0 ? defaultPort : -1;
}
{code}
However, {{DelegateToFileSystem#getUriDefaultPort}} returns 0:
{code:title=DelegateToFileSystem.java}
public int getUriDefaultPort() {
return 0;
}
{code}
This breaks {{AbstractFileSystem#checkPath}}:
{code:title=AbstractFileSystem.java}
int thisPort = this.getUri().getPort(); // If using DelegateToFileSystem,
this is -1
int thatPort = uri.getPort(); // This is -1 by default in java.net.URI
if (thatPort == -1) {
thatPort = this.getUriDefaultPort(); // Sets thatPort to 0
}
if (thisPort != thatPort) {
throw new InvalidPathException("Wrong FS: " + path + ", expected: "
+ this.getUri());
}
{code}
Which breaks any subclasses of {{DelegateToFileSystem}} that don't specify a
port.
was:
HADOOP-12304 introduced logic to ensure that the {{DelegateToFileSystem}}
constructor sets the default port to -1:
{code:title=DelegateToFileSystem.java}
protected DelegateToFileSystem(URI theUri, FileSystem theFsImpl,
Configuration conf, String supportedScheme, boolean authorityRequired)
throws IOException, URISyntaxException {
super(theUri, supportedScheme, authorityRequired,
getDefaultPortIfDefined(theFsImpl));
fsImpl = theFsImpl;
fsImpl.initialize(theUri, conf);
fsImpl.statistics = getStatistics();
}
private static int getDefaultPortIfDefined(FileSystem theFsImpl) {
int defaultPort = theFsImpl.getDefaultPort();
return defaultPort != 0 ? defaultPort : -1;
}
{code}
However, {{DelegateToFileSystem#getUriDefaultPort}} returns 0:
{code:title=DelegateToFileSystem.java}
public int getUriDefaultPort() {
return 0;
}
{code}
This breaks {{AbstractFileSystem#checkPath}}:
{code:title=AbstractFileSystem.java}
int thisPort = this.getUri().getPort(); // If using DelegateToFileSystem,
this is -1
int thatPort = uri.getPort(); // This is -1 by default in java.net.URI
if (thatPort == -1) {
thatPort = this.getUriDefaultPort(); // Sets thatPort to 0
}
if (thisPort != thatPort) {
throw new InvalidPathException("Wrong FS: " + path + ", expected: "
+ this.getUri());
}
{code}
Which breaks any subclasses of {{DelegateToFileSystem}} that don't specify a
port (S3n, Wasb(s)).
Updating the description to remove the example, because s3n is not a subclass
of DelegateToFileSystem and wasb has a default port.
> Conflicting default ports in DelegateToFileSystem
> -
>
> Key: HADOOP-12991
> URL: https://issues.apache.org/jira/browse/HADOOP-12991
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 2.7.2
>Reporter: Kevin Hogeland
>Assignee: Kai Sasaki
> Attachments: HADOOP-12991.01.patch, HADOOP-12991.02.patch,
> HADOOP-12991.03.patch
>
>
> HADOOP-12304 introduced logic to ensure that the {{DelegateToFileSystem}}
> constructor sets the default port to -1:
> {code:title=DelegateToFileSystem.java}
> protected DelegateToFileSystem(URI theUri, FileSystem theFsImpl,
> Configuration conf, String supportedScheme, boolean authorityRequired)
> throws IOException, URISyntaxException {
> super(theUri, supportedScheme, authorityRequired,
> getDefaultPortIfDefined(theFsImpl));
> fsImpl = theFsImpl;
> fsImpl.initialize(theUri, conf);
> fsImpl.statistics = getStatistics();
> }
> private static int getDefaultPortIfDefined(FileSystem theFsImpl) {
> int defaultPort = theFsImpl.getDefaultPort();
> return defaultPort != 0 ? defaultPort : -1;
> }
> {code}
> However, {{DelegateToFileSystem#getUriDefaultPort}} returns 0:
> {code:title=DelegateToFileSystem.java}
> public int getUriDefaultPort() {
> return 0;
> }
> {code}
> This breaks {{AbstractFileSystem#checkPath}}:
> {code:title=AbstractFileSystem.java}
> int thisPort = this.getUri().getPort(); // If using DelegateToFileSystem,
> this is -1
> int
[jira] [Commented] (HADOOP-12991) Conflicting default ports in DelegateToFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-12991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383058#comment-15383058
]
Akira Ajisaka commented on HADOOP-12991:
+1, I'll commit this tomorrow if there are no objections.
cc: [~cnauroth]
> Conflicting default ports in DelegateToFileSystem
> -
>
> Key: HADOOP-12991
> URL: https://issues.apache.org/jira/browse/HADOOP-12991
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 2.7.2
>Reporter: Kevin Hogeland
>Assignee: Kai Sasaki
> Attachments: HADOOP-12991.01.patch, HADOOP-12991.02.patch,
> HADOOP-12991.03.patch
>
>
> HADOOP-12304 introduced logic to ensure that the {{DelegateToFileSystem}}
> constructor sets the default port to -1:
> {code:title=DelegateToFileSystem.java}
> protected DelegateToFileSystem(URI theUri, FileSystem theFsImpl,
> Configuration conf, String supportedScheme, boolean authorityRequired)
> throws IOException, URISyntaxException {
> super(theUri, supportedScheme, authorityRequired,
> getDefaultPortIfDefined(theFsImpl));
> fsImpl = theFsImpl;
> fsImpl.initialize(theUri, conf);
> fsImpl.statistics = getStatistics();
> }
> private static int getDefaultPortIfDefined(FileSystem theFsImpl) {
> int defaultPort = theFsImpl.getDefaultPort();
> return defaultPort != 0 ? defaultPort : -1;
> }
> {code}
> However, {{DelegateToFileSystem#getUriDefaultPort}} returns 0:
> {code:title=DelegateToFileSystem.java}
> public int getUriDefaultPort() {
> return 0;
> }
> {code}
> This breaks {{AbstractFileSystem#checkPath}}:
> {code:title=AbstractFileSystem.java}
> int thisPort = this.getUri().getPort(); // If using DelegateToFileSystem,
> this is -1
> int thatPort = uri.getPort(); // This is -1 by default in java.net.URI
> if (thatPort == -1) {
> thatPort = this.getUriDefaultPort(); // Sets thatPort to 0
> }
> if (thisPort != thatPort) {
> throw new InvalidPathException("Wrong FS: " + path + ", expected: "
> + this.getUri());
> }
> {code}
> Which breaks any subclasses of {{DelegateToFileSystem}} that don't specify a
> port (S3n, Wasb(s)).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13123) Permit the default hadoop delegation token file format to be configurable
[ https://issues.apache.org/jira/browse/HADOOP-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382982#comment-15382982 ] Ravi Prakash commented on HADOOP-13123: --- For completeness of this discussion, the patch for HADOOP-12563 was never backported into branch-2 . It exists in trunk only so 3.x releases will have it. I don't know if we really should make the token file format configurable, but that's a discussion at a tangent... I'll save it for later. > Permit the default hadoop delegation token file format to be configurable > - > > Key: HADOOP-13123 > URL: https://issues.apache.org/jira/browse/HADOOP-13123 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Matthew Paduano >Assignee: Matthew Paduano > Attachments: HADOOP-13123.01.patch > > > If one environment updates to using the new dtutil code and accompanying > Credentials code, there is a backward compatibility issue with the default > file format being JAVA. Older clients need to be updated to ask for a file > in the legacy format (FORMAT_JAVA). > As an aid to users in this trap, we can add a configuration property to set > the default file format. When set to FORMAT_JAVA, the new server code will > serve up legacy files without being asked. The default value for this > property will remain FORMAT_PB. But affected users can add this config > option to the services using the newer code. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13202) Avoid possible overflow in org.apache.hadoop.util.bloom.BloomFilter#getNBytes
[ https://issues.apache.org/jira/browse/HADOOP-13202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382918#comment-15382918 ] Hudson commented on HADOOP-13202: - SUCCESS: Integrated in Hadoop-trunk-Commit #10115 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10115/]) HADOOP-13202. Avoid possible overflow in (aajisaka: rev c2bcffb34ebe4399b523a44f69d23761648bd0c4) * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/bloom/TestBloomFilters.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/bloom/BloomFilter.java > Avoid possible overflow in org.apache.hadoop.util.bloom.BloomFilter#getNBytes > - > > Key: HADOOP-13202 > URL: https://issues.apache.org/jira/browse/HADOOP-13202 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 2.7.2 >Reporter: zhengbing li >Assignee: Kai Sasaki > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha2 > > Attachments: HADOOP-13202.01.patch, HADOOP-13202.02.patch, > HADOOP-13202.03.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > Current implementation: > return (vectorSize + 7) / 8; > when vectorSize is 2147483647（the max value of Int), error > :"java.lang.NegativeArraySizeException" will report > the implementation might be changed > return (int)(((long)vectorSize + 7) / 8); -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13202) Avoid possible overflow in org.apache.hadoop.util.bloom.BloomFilter#getNBytes
[ https://issues.apache.org/jira/browse/HADOOP-13202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-13202: --- Resolution: Fixed Fix Version/s: 3.0.0-alpha2 2.7.4 2.8.0 Status: Resolved (was: Patch Available) Committed this to trunk, branch-2, branch-2.8, and branch-2.7. Thanks [~lewuathe] for the contribution and thanks [~bing] for the report. > Avoid possible overflow in org.apache.hadoop.util.bloom.BloomFilter#getNBytes > - > > Key: HADOOP-13202 > URL: https://issues.apache.org/jira/browse/HADOOP-13202 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 2.7.2 >Reporter: zhengbing li >Assignee: Kai Sasaki > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha2 > > Attachments: HADOOP-13202.01.patch, HADOOP-13202.02.patch, > HADOOP-13202.03.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > Current implementation: > return (vectorSize + 7) / 8; > when vectorSize is 2147483647（the max value of Int), error > :"java.lang.NegativeArraySizeException" will report > the implementation might be changed > return (int)(((long)vectorSize + 7) / 8); -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13202) Avoid possible overflow in org.apache.hadoop.util.bloom.BloomFilter#getNBytes
[ https://issues.apache.org/jira/browse/HADOOP-13202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-13202: --- Hadoop Flags: Reviewed Component/s: util Summary: Avoid possible overflow in org.apache.hadoop.util.bloom.BloomFilter#getNBytes (was: Implementation of getNBytes in org.apache.hadoop.util.bloom.BloomFilter might be changed) > Avoid possible overflow in org.apache.hadoop.util.bloom.BloomFilter#getNBytes > - > > Key: HADOOP-13202 > URL: https://issues.apache.org/jira/browse/HADOOP-13202 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 2.7.2 >Reporter: zhengbing li >Assignee: Kai Sasaki > Attachments: HADOOP-13202.01.patch, HADOOP-13202.02.patch, > HADOOP-13202.03.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > Current implementation: > return (vectorSize + 7) / 8; > when vectorSize is 2147483647（the max value of Int), error > :"java.lang.NegativeArraySizeException" will report > the implementation might be changed > return (int)(((long)vectorSize + 7) / 8); -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13383) Update release notes for 3.0.0-alpha1
[ https://issues.apache.org/jira/browse/HADOOP-13383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382801#comment-15382801 ] Akira Ajisaka commented on HADOOP-13383: Thanks Andrew. I'm thinking it's better to add them to the release notes. > Update release notes for 3.0.0-alpha1 > - > > Key: HADOOP-13383 > URL: https://issues.apache.org/jira/browse/HADOOP-13383 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Blocker > Attachments: HADOOP-13383.001.patch > > > Per the release instructions (https://wiki.apache.org/hadoop/HowToRelease), > we need to update hadoop-project/src/site/markdown/index.md.vm to reflect the > right versions, new features and big improvements. > I can put together some notes for HADOOP and HDFS, depending on others for > YARN and MR. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13383) Update release notes for 3.0.0-alpha1
[ https://issues.apache.org/jira/browse/HADOOP-13383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382794#comment-15382794 ] Andrew Wang commented on HADOOP-13383: -- Thanks Akira, all good ideas, and a good reminder to do some related release notes cleanup. I also see some stuff like HADOOP-10950 and MAPREDUCE-5785 related to heap management, do we want to call those out? > Update release notes for 3.0.0-alpha1 > - > > Key: HADOOP-13383 > URL: https://issues.apache.org/jira/browse/HADOOP-13383 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Blocker > Attachments: HADOOP-13383.001.patch > > > Per the release instructions (https://wiki.apache.org/hadoop/HowToRelease), > we need to update hadoop-project/src/site/markdown/index.md.vm to reflect the > right versions, new features and big improvements. > I can put together some notes for HADOOP and HDFS, depending on others for > YARN and MR. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-11858) [JDK8] Set minimum version of Hadoop 3 to JDK 8
[ https://issues.apache.org/jira/browse/HADOOP-11858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-11858: - Hadoop Flags: Incompatible change > [JDK8] Set minimum version of Hadoop 3 to JDK 8 > --- > > Key: HADOOP-11858 > URL: https://issues.apache.org/jira/browse/HADOOP-11858 > Project: Hadoop Common > Issue Type: New Feature > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Blocker > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-11858.001.patch, HADOOP-11858.002.patch, > HADOOP-11858.003.patch, commit.ogv > > > Set minimum version of trunk to JDK 8 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-11858) [JDK8] Set minimum version of Hadoop 3 to JDK 8
[ https://issues.apache.org/jira/browse/HADOOP-11858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-11858: - Release Note: The minimum required JDK version for Hadoop has been increased from JDK7 to JDK8. I moved this to a new issue so it's called out in the release notes / changes as a new feature, hope that's cool with everyone. > [JDK8] Set minimum version of Hadoop 3 to JDK 8 > --- > > Key: HADOOP-11858 > URL: https://issues.apache.org/jira/browse/HADOOP-11858 > Project: Hadoop Common > Issue Type: New Feature > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Blocker > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-11858.001.patch, HADOOP-11858.002.patch, > HADOOP-11858.003.patch, commit.ogv > > > Set minimum version of trunk to JDK 8 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-11858) [JDK8] Set minimum version of Hadoop 3 to JDK 8
[ https://issues.apache.org/jira/browse/HADOOP-11858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-11858: - Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-11090) > [JDK8] Set minimum version of Hadoop 3 to JDK 8 > --- > > Key: HADOOP-11858 > URL: https://issues.apache.org/jira/browse/HADOOP-11858 > Project: Hadoop Common > Issue Type: New Feature > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Blocker > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-11858.001.patch, HADOOP-11858.002.patch, > HADOOP-11858.003.patch, commit.ogv > > > Set minimum version of trunk to JDK 8 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13192) org.apache.hadoop.util.LineReader cannot handle multibyte delimiters correctly
[
https://issues.apache.org/jira/browse/HADOOP-13192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-13192:
---
Attachment: HADOOP-13192.final.patch
Attaching a patch for the commit. (Generated by {{git diff
d0162f2040a4d98fbac51527dfc9447888cb32ab
fc6b50cc574e144fd314dea6c11987c6a384bfa6}})
Thanks [~haitao-tony]!
> org.apache.hadoop.util.LineReader cannot handle multibyte delimiters correctly
> --
>
> Key: HADOOP-13192
> URL: https://issues.apache.org/jira/browse/HADOOP-13192
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.6.2
>Reporter: binde
>Assignee: binde
>Priority: Critical
> Fix For: 2.7.3
>
> Attachments:
> 0001-HADOOP-13192-org.apache.hadoop.util.LineReader-match.patch,
> 0002-fix-bug-hadoop-1392-add-test-case-for-LineReader.patch,
> HADOOP-13192.final.patch
>
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> org.apache.hadoop.util.LineReader.readCustomLine() has a bug,
> when line is bccc, recordDelimiter is aaab, the result should be a,ccc,
> show the code on line 310:
> for (; bufferPosn < bufferLength; ++bufferPosn) {
> if (buffer[bufferPosn] == recordDelimiterBytes[delPosn]) {
> delPosn++;
> if (delPosn >= recordDelimiterBytes.length) {
> bufferPosn++;
> break;
> }
> } else if (delPosn != 0) {
> bufferPosn--;
> delPosn = 0;
> }
> }
> shoud be :
> for (; bufferPosn < bufferLength; ++bufferPosn) {
> if (buffer[bufferPosn] == recordDelimiterBytes[delPosn]) {
> delPosn++;
> if (delPosn >= recordDelimiterBytes.length) {
> bufferPosn++;
> break;
> }
> } else if (delPosn != 0) {
> // - change here - start
> bufferPosn -= delPosn;
> // - change here - end
>
> delPosn = 0;
> }
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13383) Update release notes for 3.0.0-alpha1
[ https://issues.apache.org/jira/browse/HADOOP-13383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382742#comment-15382742 ] Akira Ajisaka commented on HADOOP-13383: It would be better to add ATSv2 and HDFS Disk Balancer. cc: [~arpitagarwal], [~anu] In addition, we need to mention that JDK7 support was dropped. > Update release notes for 3.0.0-alpha1 > - > > Key: HADOOP-13383 > URL: https://issues.apache.org/jira/browse/HADOOP-13383 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Blocker > Attachments: HADOOP-13383.001.patch > > > Per the release instructions (https://wiki.apache.org/hadoop/HowToRelease), > we need to update hadoop-project/src/site/markdown/index.md.vm to reflect the > right versions, new features and big improvements. > I can put together some notes for HADOOP and HDFS, depending on others for > YARN and MR. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13202) Implementation of getNBytes in org.apache.hadoop.util.bloom.BloomFilter might be changed
[ https://issues.apache.org/jira/browse/HADOOP-13202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382715#comment-15382715 ] Akira Ajisaka commented on HADOOP-13202: +1, thanks [~lewuathe]. > Implementation of getNBytes in org.apache.hadoop.util.bloom.BloomFilter might > be changed > > > Key: HADOOP-13202 > URL: https://issues.apache.org/jira/browse/HADOOP-13202 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.7.2 >Reporter: zhengbing li >Assignee: Kai Sasaki > Attachments: HADOOP-13202.01.patch, HADOOP-13202.02.patch, > HADOOP-13202.03.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > Current implementation: > return (vectorSize + 7) / 8; > when vectorSize is 2147483647（the max value of Int), error > :"java.lang.NegativeArraySizeException" will report > the implementation might be changed > return (int)(((long)vectorSize + 7) / 8); -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13073) RawLocalFileSystem does not react on changing umask
[ https://issues.apache.org/jira/browse/HADOOP-13073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382445#comment-15382445 ] Andras Bokor commented on HADOOP-13073: --- Thanks [~ste...@apache.org]! > RawLocalFileSystem does not react on changing umask > --- > > Key: HADOOP-13073 > URL: https://issues.apache.org/jira/browse/HADOOP-13073 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Reporter: Andras Bokor >Assignee: Andras Bokor > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13073.01.patch, HADOOP-13073.02.patch, > HADOOP-13073.03.patch, HADOOP-13073.04.patch > > > FileSystemContractBaseTest#testMkdirsWithUmask is changing umask under the > filesystem. RawLocalFileSystem reads the config on startup so it will not > react if we change the umask. > It blocks [HADOOP-7363|https://issues.apache.org/jira/browse/HADOOP-7363] > since testMkdirsWithUmask test will never work with RawLocalFileSystem. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13384) Add a job-conf.xml as a default resource file to load in configuration utils
Francois Visconte created HADOOP-13384: -- Summary: Add a job-conf.xml as a default resource file to load in configuration utils Key: HADOOP-13384 URL: https://issues.apache.org/jira/browse/HADOOP-13384 Project: Hadoop Common Issue Type: Improvement Components: conf Reporter: Francois Visconte We would like to make it possible for jobs to include their configuration as a default resource loaded as a default in ConfigUtil (when available) That would allow a job to include a job-conf.xml resource file allowing to configure a job without having to set configuration file from inside the code (making it a bit easier to track for configuration changes). FYI We are also using this to inject useful configuration parameters (code version, main class, parameters, launch time) in order to get these insights in job.xml files in order to process these data later on (with rumen for example). Here is the proposed patch on our fork that I would like to propose upstream:https://github.com/criteo-forks/hadoop-common/pull/2/commits/9bb3f7ad95989bcd9a41be4fcc0633c2c0b3 Thanks, -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13073) RawLocalFileSystem does not react on changing umask
[ https://issues.apache.org/jira/browse/HADOOP-13073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382325#comment-15382325 ] Hudson commented on HADOOP-13073: - SUCCESS: Integrated in Hadoop-trunk-Commit #10114 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10114/]) HADOOP-13073 RawLocalFileSystem does not react on changing umask. (stevel: rev 76be82bc0419affbe0103bc4f45c90926f08d0cc) * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystemPermission.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java > RawLocalFileSystem does not react on changing umask > --- > > Key: HADOOP-13073 > URL: https://issues.apache.org/jira/browse/HADOOP-13073 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Reporter: Andras Bokor >Assignee: Andras Bokor > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13073.01.patch, HADOOP-13073.02.patch, > HADOOP-13073.03.patch, HADOOP-13073.04.patch > > > FileSystemContractBaseTest#testMkdirsWithUmask is changing umask under the > filesystem. RawLocalFileSystem reads the config on startup so it will not > react if we change the umask. > It blocks [HADOOP-7363|https://issues.apache.org/jira/browse/HADOOP-7363] > since testMkdirsWithUmask test will never work with RawLocalFileSystem. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13073) RawLocalFileSystem does not react on changing umask
[
https://issues.apache.org/jira/browse/HADOOP-13073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-13073:
Resolution: Fixed
Fix Version/s: 3.0.0-alpha2
Status: Resolved (was: Patch Available)
+1, applied to trunk.
All those log ("not on windows") and return could really be replaced in JUnit 4
with assume() tests, but that's something for the future
> RawLocalFileSystem does not react on changing umask
> ---
>
> Key: HADOOP-13073
> URL: https://issues.apache.org/jira/browse/HADOOP-13073
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13073.01.patch, HADOOP-13073.02.patch,
> HADOOP-13073.03.patch, HADOOP-13073.04.patch
>
>
> FileSystemContractBaseTest#testMkdirsWithUmask is changing umask under the
> filesystem. RawLocalFileSystem reads the config on startup so it will not
> react if we change the umask.
> It blocks [HADOOP-7363|https://issues.apache.org/jira/browse/HADOOP-7363]
> since testMkdirsWithUmask test will never work with RawLocalFileSystem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work started] (HADOOP-13371) S3A globber to use bulk listObject call over recursive directory scan
[
https://issues.apache.org/jira/browse/HADOOP-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HADOOP-13371 started by Steve Loughran.
---
> S3A globber to use bulk listObject call over recursive directory scan
> -
>
> Key: HADOOP-13371
> URL: https://issues.apache.org/jira/browse/HADOOP-13371
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs, fs/s3
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>
> HADOOP-13208 produces O(1) listing of directory trees in
> {{FileSystem.listStatus}} calls, but doesn't do anything for
> {{FileSystem.globStatus()}}, which uses a completely different codepath, one
> which does a selective recursive scan by pattern matching as it goes down,
> filtering out those patterns which don't match. Cost is
> O(matching-directories) + cost of examining the files.
> It should be possible to do the glob status listing in S3A not through the
> filtered treewalk, but through a list + filter operation. This would be an
> O(files) lookup *before any filtering took place*.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-13371) S3A globber to use bulk listObject call over recursive directory scan
[
https://issues.apache.org/jira/browse/HADOOP-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran reassigned HADOOP-13371:
---
Assignee: Steve Loughran
> S3A globber to use bulk listObject call over recursive directory scan
> -
>
> Key: HADOOP-13371
> URL: https://issues.apache.org/jira/browse/HADOOP-13371
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs, fs/s3
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>
> HADOOP-13208 produces O(1) listing of directory trees in
> {{FileSystem.listStatus}} calls, but doesn't do anything for
> {{FileSystem.globStatus()}}, which uses a completely different codepath, one
> which does a selective recursive scan by pattern matching as it goes down,
> filtering out those patterns which don't match. Cost is
> O(matching-directories) + cost of examining the files.
> It should be possible to do the glob status listing in S3A not through the
> filtered treewalk, but through a list + filter operation. This would be an
> O(files) lookup *before any filtering took place*.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12991) Conflicting default ports in DelegateToFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-12991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382228#comment-15382228
]
Hadoop QA commented on HADOOP-12991:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
1s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12818535/HADOOP-12991.03.patch
|
| JIRA Issue | HADOOP-12991 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 789bb6308dfc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 5b4a708 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10021/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10021/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Conflicting default ports in DelegateToFileSystem
> -
>
> Key: HADOOP-12991
> URL: https://issues.apache.org/jira/browse/HADOOP-12991
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 2.7.2
>Reporter: Kevin Hogeland
>Assignee: Kai Sasaki
> Attachments: HADOOP-12991.01.patch, HADOOP-12991.02.patch,
> HADOOP-12991.03.patch
>
>
> HADOOP-12304 introduced logic to ensure that the {{DelegateToFileSystem}}
> constructor sets the default
[jira] [Updated] (HADOOP-12991) Conflicting default ports in DelegateToFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-12991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kai Sasaki updated HADOOP-12991:
Attachment: HADOOP-12991.03.patch
> Conflicting default ports in DelegateToFileSystem
> -
>
> Key: HADOOP-12991
> URL: https://issues.apache.org/jira/browse/HADOOP-12991
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 2.7.2
>Reporter: Kevin Hogeland
>Assignee: Kai Sasaki
> Attachments: HADOOP-12991.01.patch, HADOOP-12991.02.patch,
> HADOOP-12991.03.patch
>
>
> HADOOP-12304 introduced logic to ensure that the {{DelegateToFileSystem}}
> constructor sets the default port to -1:
> {code:title=DelegateToFileSystem.java}
> protected DelegateToFileSystem(URI theUri, FileSystem theFsImpl,
> Configuration conf, String supportedScheme, boolean authorityRequired)
> throws IOException, URISyntaxException {
> super(theUri, supportedScheme, authorityRequired,
> getDefaultPortIfDefined(theFsImpl));
> fsImpl = theFsImpl;
> fsImpl.initialize(theUri, conf);
> fsImpl.statistics = getStatistics();
> }
> private static int getDefaultPortIfDefined(FileSystem theFsImpl) {
> int defaultPort = theFsImpl.getDefaultPort();
> return defaultPort != 0 ? defaultPort : -1;
> }
> {code}
> However, {{DelegateToFileSystem#getUriDefaultPort}} returns 0:
> {code:title=DelegateToFileSystem.java}
> public int getUriDefaultPort() {
> return 0;
> }
> {code}
> This breaks {{AbstractFileSystem#checkPath}}:
> {code:title=AbstractFileSystem.java}
> int thisPort = this.getUri().getPort(); // If using DelegateToFileSystem,
> this is -1
> int thatPort = uri.getPort(); // This is -1 by default in java.net.URI
> if (thatPort == -1) {
> thatPort = this.getUriDefaultPort(); // Sets thatPort to 0
> }
> if (thisPort != thatPort) {
> throw new InvalidPathException("Wrong FS: " + path + ", expected: "
> + this.getUri());
> }
> {code}
> Which breaks any subclasses of {{DelegateToFileSystem}} that don't specify a
> port (S3n, Wasb(s)).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13202) Implementation of getNBytes in org.apache.hadoop.util.bloom.BloomFilter might be changed
[ https://issues.apache.org/jira/browse/HADOOP-13202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382153#comment-15382153 ] Kai Sasaki commented on HADOOP-13202: - [~ajisakaa] Thanks for reviewing! I updated and it worked as expected, since test failure does not look like relate to this patch. > Implementation of getNBytes in org.apache.hadoop.util.bloom.BloomFilter might > be changed > > > Key: HADOOP-13202 > URL: https://issues.apache.org/jira/browse/HADOOP-13202 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.7.2 >Reporter: zhengbing li >Assignee: Kai Sasaki > Attachments: HADOOP-13202.01.patch, HADOOP-13202.02.patch, > HADOOP-13202.03.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > Current implementation: > return (vectorSize + 7) / 8; > when vectorSize is 2147483647（the max value of Int), error > :"java.lang.NegativeArraySizeException" will report > the implementation might be changed > return (int)(((long)vectorSize + 7) / 8); -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13073) RawLocalFileSystem does not react on changing umask
[ https://issues.apache.org/jira/browse/HADOOP-13073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382106#comment-15382106 ] Steve Loughran commented on HADOOP-13073: - don't worry about whitespace; that we just fix on the git apply > RawLocalFileSystem does not react on changing umask > --- > > Key: HADOOP-13073 > URL: https://issues.apache.org/jira/browse/HADOOP-13073 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Reporter: Andras Bokor >Assignee: Andras Bokor > Attachments: HADOOP-13073.01.patch, HADOOP-13073.02.patch, > HADOOP-13073.03.patch, HADOOP-13073.04.patch > > > FileSystemContractBaseTest#testMkdirsWithUmask is changing umask under the > filesystem. RawLocalFileSystem reads the config on startup so it will not > react if we change the umask. > It blocks [HADOOP-7363|https://issues.apache.org/jira/browse/HADOOP-7363] > since testMkdirsWithUmask test will never work with RawLocalFileSystem. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13073) RawLocalFileSystem does not react on changing umask
[
https://issues.apache.org/jira/browse/HADOOP-13073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382015#comment-15382015
]
Hadoop QA commented on HADOOP-13073:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} hadoop-common-project/hadoop-common: The patch
generated 0 new + 31 unchanged - 1 fixed = 31 total (was 32) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
27s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12818511/HADOOP-13073.04.patch
|
| JIRA Issue | HADOOP-13073 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 7ecb203f8b37 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 5b4a708 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10020/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10020/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> RawLocalFileSystem does not react on changing umask
> ---
>
> Key: HADOOP-13073
> URL: https://issues.apache.org/jira/browse/HADOOP-13073
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Attachments: HADOOP-13073.01.patch, HADOOP-13073.02.patch,
> HADOOP-13073.03.patch, HADOOP-13073.04.patch
>
>
>
[jira] [Updated] (HADOOP-9171) Release resources of unpoolable Decompressors
[ https://issues.apache.org/jira/browse/HADOOP-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HADOOP-9171: -- Resolution: Won't Fix Status: Resolved (was: Patch Available) > Release resources of unpoolable Decompressors > - > > Key: HADOOP-9171 > URL: https://issues.apache.org/jira/browse/HADOOP-9171 > Project: Hadoop Common > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Labels: BB2015-05-TBR > Attachments: HDFS-4345.txt > > > Found this when looking into HBASE-7435. > When a Decompressor is returned to the pool in CodecPool.java, we should > probably call end() on it to release its resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13073) RawLocalFileSystem does not react on changing umask
[ https://issues.apache.org/jira/browse/HADOOP-13073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Bokor updated HADOOP-13073: -- Attachment: HADOOP-13073.04.patch 04 to fix whitespace issues. > RawLocalFileSystem does not react on changing umask > --- > > Key: HADOOP-13073 > URL: https://issues.apache.org/jira/browse/HADOOP-13073 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Reporter: Andras Bokor >Assignee: Andras Bokor > Attachments: HADOOP-13073.01.patch, HADOOP-13073.02.patch, > HADOOP-13073.03.patch, HADOOP-13073.04.patch > > > FileSystemContractBaseTest#testMkdirsWithUmask is changing umask under the > filesystem. RawLocalFileSystem reads the config on startup so it will not > react if we change the umask. > It blocks [HADOOP-7363|https://issues.apache.org/jira/browse/HADOOP-7363] > since testMkdirsWithUmask test will never work with RawLocalFileSystem. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
