[jira] [Commented] (HADOOP-12693) Many misusages of assertEquals(expected, actual)
[
https://issues.apache.org/jira/browse/HADOOP-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905348#comment-16905348
]
Daniel Templeton commented on HADOOP-12693:
---
Sounds like we're good for MR and YARN, but what about HADOOP and HDFS?
> Many misusages of assertEquals(expected, actual)
>
>
> Key: HADOOP-12693
> URL: https://issues.apache.org/jira/browse/HADOOP-12693
> Project: Hadoop Common
> Issue Type: Test
>Reporter: Akihiro Suda
>Priority: Trivial
> Attachments: just-rough-approx.txt
>
>
> The first arg of {{org.JUnit.Assert.assertEquals()}} should be an
> {{expected}} value, and the second one should be an {{actual}} value.
> {code}
> void assertEquals(T expected, T actual);
> {code}
> http://junit.org/apidocs/org/junit/Assert.html#assertEquals(java.lang.Object,
> java.lang.Object)
> However, there are so many violations in Hadoop, which can make a misleading
> message like this:
> {code}
> AssertionError: expected: but was:
> {code}.
> Please refer to {{just-rough-approx.txt}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16336) finish variable is unused in ZStandardCompressor
[
https://issues.apache.org/jira/browse/HADOOP-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899019#comment-16899019
]
Daniel Templeton commented on HADOOP-16336:
---
Doh! Didn't think to check for use in the JNI code. My fail. Thanks for the
revert, [~jojochuang].
> finish variable is unused in ZStandardCompressor
>
>
> Key: HADOOP-16336
> URL: https://issues.apache.org/jira/browse/HADOOP-16336
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.3.0
>Reporter: Daniel Templeton
>Priority: Trivial
> Labels: newbie
> Fix For: 3.3.0
>
>
> The boolean {{finish}} variable is unused and can be removed:
> {code:java}
> private boolean finish, finished;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16276) Fix jsvc startup command in hadoop-functions.sh due to jsvc >= 1.0.11 changed default current working directory
[ https://issues.apache.org/jira/browse/HADOOP-16276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16853191#comment-16853191 ] Daniel Templeton commented on HADOOP-16276: --- I actually like the auto-detect with an optional override. We try to do the right thing but let the user have ultimate control. The override is important so that management software can control where the keytabs live. I think the auto-detect is important to maintain backwards compatibility. If someone not using a vendor's tool deploys 3.3 without the auto-detect and without making the right changes, their deploy will break. That said, I think the auto-detect is unnecessarily brittle. Why are we looking for the n-3rd line instead of grepping for "cwd"? > Fix jsvc startup command in hadoop-functions.sh due to jsvc >= 1.0.11 changed > default current working directory > --- > > Key: HADOOP-16276 > URL: https://issues.apache.org/jira/browse/HADOOP-16276 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 3.2.0, 3.1.2 >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Attachments: HADOOP-16276.001.patch, HADOOP-16276.002.patch, > HADOOP-16276.003.patch > > > In CDH6, when we bump jsvc from 1.0.10 to 1.1.0 we hit > *KerberosAuthException: failure to login / LoginException: Unable to obtain > password from user* due to DAEMON-264 and our > *dfs.nfs.keytab.file* config uses a relative path. I will probably file > another jira to issue a warning like *hdfs.keytab not found* before > KerberosAuthException in this case. > The solution is to add *-cwd $(pwd)* in function hadoop_start_secure_daemon > in hadoop-functions.sh but I will have to consider the compatibility with > older jsvc versions <= 1.0.10. Will post the patch after I tested it. > Thanks [~tlipcon] for finding the root cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16336) finish variable is unused in ZStandardCompressor
[
https://issues.apache.org/jira/browse/HADOOP-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated HADOOP-16336:
--
Summary: finish variable is unused in ZStandardCompressor (was: finish
valiable is unused in ZStandardCompressor)
> finish variable is unused in ZStandardCompressor
>
>
> Key: HADOOP-16336
> URL: https://issues.apache.org/jira/browse/HADOOP-16336
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.3.0
>Reporter: Daniel Templeton
>Priority: Trivial
> Labels: newbie
>
> The boolean {{finish}} variable is unused and can be removed:
> {code:java}
> private boolean finish, finished;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16336) finish valiable is unused in ZStandardCompressor
Daniel Templeton created HADOOP-16336:
-
Summary: finish valiable is unused in ZStandardCompressor
Key: HADOOP-16336
URL: https://issues.apache.org/jira/browse/HADOOP-16336
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 3.3.0
Reporter: Daniel Templeton
The boolean {{finish}} variable is unused and can be removed:
{code:java}
private boolean finish, finished;
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15151) MapFile.fix creates a wrong index file in case of block-compressed data file.
[
https://issues.apache.org/jira/browse/HADOOP-15151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819555#comment-16819555
]
Daniel Templeton commented on HADOOP-15151:
---
[~zvenczel], nope. The commit is there, but it's missing the JIRA ID in the
summary:
{noformat}
commit 56872cff92f543bf77206a1324968559dceb7bc2
Author: Chris Douglas
Date: Fri Jan 26 09:06:48 2018 -0800
MapFile.fix creates a wrong index file in case of block-compressed data
file. Contributed by Grigori Rybkine
{noformat}
> MapFile.fix creates a wrong index file in case of block-compressed data file.
> -
>
> Key: HADOOP-15151
> URL: https://issues.apache.org/jira/browse/HADOOP-15151
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Reporter: Grigori Rybkine
>Assignee: Grigori Rybkine
>Priority: Major
> Labels: patch
> Fix For: 2.9.1
>
> Attachments: HADOOP-15151.001.patch, HADOOP-15151.002.patch,
> HADOOP-15151.003.patch, HADOOP-15151.004.patch, HADOOP-15151.004.patch,
> HADOOP-15151.005.patch
>
>
> Index file created with MapFile.fix for an ordered block-compressed data file
> does not allow to find values for keys existing in the data file via the
> MapFile.get method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16156) [Clean-up] Remove NULL check before instanceof and fix checkstyle in InnerNodeImpl
[
https://issues.apache.org/jira/browse/HADOOP-16156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803472#comment-16803472
]
Daniel Templeton commented on HADOOP-16156:
---
[~daryn], that's true. I think the intent here is to clean up latent
checkstyle issues that popped up while trying to remove the extraneous null
check. Point taken, though.
[~shwetayakkali], what I had in mind with my last point above was something
like:
{code}if ((childnode == null) || (path.length == 1)) {
return childnode;
} else if (childnode instanceof InnerNode) {
return ((InnerNode)childnode).getLoc(path[1]);
} else {
return null;
}{code}
> [Clean-up] Remove NULL check before instanceof and fix checkstyle in
> InnerNodeImpl
> --
>
> Key: HADOOP-16156
> URL: https://issues.apache.org/jira/browse/HADOOP-16156
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Shweta
>Assignee: Shweta
>Priority: Minor
> Attachments: HADOOP-16156.001.patch, HADOOP-16156.002.patch,
> HADOOP-16156.003.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16156) [Clean-up] Remove NULL check before instanceof and fix checkstyle in InnerNodeImpl
[
https://issues.apache.org/jira/browse/HADOOP-16156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16798121#comment-16798121
]
Daniel Templeton commented on HADOOP-16156:
---
After a careful review, I have a couple more comments:
# Looks like this change was unnecessary:{code} return new
InnerNodeImpl(parentName,
getPath(this), this, this.getLevel()+1);{code} That line was not over
the 80 char limit. Now, if I were in a mood to reformat code, I'd add a space
on either side of that plus, which would put it over the limit. You can go
either way, but the change in the current patch is superfluous.
# If you're going to reformat this line:{code}if (loc == null ||
loc.length() == 0) return this;{code} then you may as well put some parens
around the clauses in the conditional.
# If you're going to reformat these lines:{code}if (childnode == null)
return null; // non-existing node
if (path.length == 1) return childnode;{code} you should probably combine
them into a single conditional. If {{childnode == null}}, then {{return
childnode}} is the same as {{return null}}. If you do, be sure to add a
comment to explain it.
> [Clean-up] Remove NULL check before instanceof and fix checkstyle in
> InnerNodeImpl
> --
>
> Key: HADOOP-16156
> URL: https://issues.apache.org/jira/browse/HADOOP-16156
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Shweta
>Assignee: Shweta
>Priority: Minor
> Attachments: HADOOP-16156.001.patch, HADOOP-16156.002.patch,
> HADOOP-16156.003.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16156) [Clean-up] Remove NULL check before instanceof and fix checkstyle in InnerNodeImpl
[ https://issues.apache.org/jira/browse/HADOOP-16156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797092#comment-16797092 ] Daniel Templeton commented on HADOOP-16156: --- The latest patch LGTM. I'll do a more careful review and commit it when I get a chance. > [Clean-up] Remove NULL check before instanceof and fix checkstyle in > InnerNodeImpl > -- > > Key: HADOOP-16156 > URL: https://issues.apache.org/jira/browse/HADOOP-16156 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Shweta >Assignee: Shweta >Priority: Minor > Attachments: HADOOP-16156.001.patch, HADOOP-16156.002.patch, > HADOOP-16156.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16156) [Clean-up] Remove NULL check before instanceof and fix checkstyle in InnerNodeImpl
[
https://issues.apache.org/jira/browse/HADOOP-16156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795081#comment-16795081
]
Daniel Templeton commented on HADOOP-16156:
---
Thanks for the patch, [~shwetayakkali]. A few comments:
# You deleted this line:{code} int numOfLeaves =
child.getNumOfLeaves();{code} I don't think that's correct. I assume you were
trying to resolve a complaint about the local variable hiding a class variable.
That's not how to fix it. It's also out of scope for this patch, so I'd leave
it alone.
# You need a space before the brace here:{code}if (index !=-1) {{code}
# You need a newline before the brace here:{code}return children; }{code}
> [Clean-up] Remove NULL check before instanceof and fix checkstyle in
> InnerNodeImpl
> --
>
> Key: HADOOP-16156
> URL: https://issues.apache.org/jira/browse/HADOOP-16156
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Shweta
>Assignee: Shweta
>Priority: Minor
> Attachments: HADOOP-16156.001.patch, HADOOP-16156.002.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795083#comment-16795083
]
Daniel Templeton commented on HADOOP-16167:
---
Not exactly identical. There was a 1-character error in 003. The
{{${ucommand}}} should have been {{${command}}}. Thanks in advance for the
commit.
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch, HADOOP-16167.002.patch,
> HADOOP-16167.003.patch, HADOOP-16167.004.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793853#comment-16793853
]
Daniel Templeton edited comment on HADOOP-16167 at 3/18/19 2:55 PM:
There was an error in the 003 patch. Fixed in 004.
was (Author: templedf):
There was an error in the 002 patch. Fixed in 004.
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch, HADOOP-16167.002.patch,
> HADOOP-16167.003.patch, HADOOP-16167.004.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793853#comment-16793853
]
Daniel Templeton commented on HADOOP-16167:
---
There was an error in the 002 patch. Fixed in 004.
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch, HADOOP-16167.002.patch,
> HADOOP-16167.003.patch, HADOOP-16167.004.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated HADOOP-16167:
--
Attachment: HADOOP-16167.004.patch
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch, HADOOP-16167.002.patch,
> HADOOP-16167.003.patch, HADOOP-16167.004.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated HADOOP-16167:
--
Attachment: HADOOP-16167.003.patch
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch, HADOOP-16167.002.patch,
> HADOOP-16167.003.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated HADOOP-16167:
--
Attachment: HADOOP-16167.002.patch
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch, HADOOP-16167.002.patch,
> HADOOP-16167.003.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated HADOOP-16167:
--
Attachment: (was: HADOOP-16167.002.patch)
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch, HADOOP-16167.003.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791136#comment-16791136
]
Daniel Templeton commented on HADOOP-16167:
---
I just added two patches. 003 is faithful to [~aw]'s intent. 002 uses the
simpler syntax, but is more of a restructure. Any preferences?
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch, HADOOP-16167.002.patch,
> HADOOP-16167.003.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated HADOOP-16167:
--
Attachment: HADOOP-16167.002.patch
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch, HADOOP-16167.002.patch,
> HADOOP-16167.003.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791125#comment-16791125
]
Daniel Templeton commented on HADOOP-16167:
---
[~aw]'s approach is cleaner. I'm a Perl coder, not a bash coder. :) I'll
rework the patch to cover all the failure scenarios with the simpler syntax.
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13656) fs -expunge to take a filesystem
[
https://issues.apache.org/jira/browse/HADOOP-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788794#comment-16788794
]
Daniel Templeton commented on HADOOP-13656:
---
I would also add that the test isn't actually testing much. It's a smoke test
only.
> fs -expunge to take a filesystem
>
>
> Key: HADOOP-13656
> URL: https://issues.apache.org/jira/browse/HADOOP-13656
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs
>Affects Versions: 2.7.3
>Reporter: Steve Loughran
>Assignee: Shweta
>Priority: Minor
> Attachments: HADOOP-13656.001.patch
>
>
> you can't pass in a filesystem or object store to {{fs -expunge}; you have to
> change the default fs
> {code}
> hadoop fs -expunge -D fs.defaultFS=s3a://bucket/
> {code}
> If the command took an optional filesystem argument, it'd be better at
> cleaning up object stores. Given that even deleted object store data runs up
> bills, this could be appreciated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16157) [Clean-up] Remove NULL check before instanceof in AzureNativeFileSystemStore
[ https://issues.apache.org/jira/browse/HADOOP-16157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-16157: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) Thanks for the patch, [~shwetayakkali]. Committed to trunk. > [Clean-up] Remove NULL check before instanceof in AzureNativeFileSystemStore > > > Key: HADOOP-16157 > URL: https://issues.apache.org/jira/browse/HADOOP-16157 > Project: Hadoop Common > Issue Type: Improvement > Components: tools >Reporter: Shweta >Assignee: Shweta >Priority: Minor > Fix For: 3.3.0 > > Attachments: HADOOP-16157.001.patch > > > Null check before instanceof is present on L:1765 and L:2641. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786346#comment-16786346
]
Daniel Templeton commented on HADOOP-16167:
---
Good point, [~eyang]. I think [~aw] was aiming at having consistency in the
scripts for how substitution is done, but in the specific cases where we're
having issues, that's a super easy fix. [~aw], any comments?
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16157) [Clean-up] Remove NULL check before instanceof in AzureNativeFileSystemStore
[ https://issues.apache.org/jira/browse/HADOOP-16157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785117#comment-16785117 ] Daniel Templeton commented on HADOOP-16157: --- LGTM +1 I'll commit when I get a chance. > [Clean-up] Remove NULL check before instanceof in AzureNativeFileSystemStore > > > Key: HADOOP-16157 > URL: https://issues.apache.org/jira/browse/HADOOP-16157 > Project: Hadoop Common > Issue Type: Improvement > Components: tools >Reporter: Shweta >Assignee: Shweta >Priority: Minor > Attachments: HADOOP-16157.001.patch > > > Null check before instanceof is present on L:1765 and L:2641. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated HADOOP-16167:
--
Status: Patch Available (was: Open)
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784982#comment-16784982
]
Daniel Templeton commented on HADOOP-16167:
---
I just posted a quick "bandaid" patch based on [~aw]'s feedback. Anyone want
to review?
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated HADOOP-16167:
--
Attachment: HADOOP-16167.001.patch
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton reassigned HADOOP-16167:
-
Assignee: Daniel Templeton
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.2.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Major
> Attachments: HADOOP-16167.001.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16167) "hadoop CLASSFILE" prints error messages on Ubuntu 18
Daniel Templeton created HADOOP-16167:
-
Summary: "hadoop CLASSFILE" prints error messages on Ubuntu 18
Key: HADOOP-16167
URL: https://issues.apache.org/jira/browse/HADOOP-16167
Project: Hadoop Common
Issue Type: Improvement
Components: scripts
Affects Versions: 3.2.0
Reporter: Daniel Templeton
{noformat}
# hadoop org.apache.hadoop.conf.Configuration
/usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
/usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
/usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
{noformat}
The issue is a regression in bash 4.4. See
[here|http://savannah.gnu.org/support/?109649]. The extraneous output can
break scripts that read the command output.
According to [~aw]:
{quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
hdfs, etc) just needs some special handling when a custom method is being
called. For example, there’s no point in checking to see if it should run with
privileges, so just skip over that. Probably a few other places too.
Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15914) hadoop jar command has no help argument
[
https://issues.apache.org/jira/browse/HADOOP-15914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772422#comment-16772422
]
Daniel Templeton commented on HADOOP-15914:
---
LGTM. [~aw], any concerns? Otherwise I'll +1 and commit.
> hadoop jar command has no help argument
> ---
>
> Key: HADOOP-15914
> URL: https://issues.apache.org/jira/browse/HADOOP-15914
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: Adam Antal
>Assignee: Adam Antal
>Priority: Major
> Attachments: HADOOP-15914.000.patch
>
>
> {{hadoop jar --help}} and {{hadoop jar help}} commands show outputs like this:
> {noformat}
> WARNING: Use "yarn jar" to launch YARN applications.
> JAR does not exist or is not a normal file: /root/--help
> {noformat}
> Only if called with no arguments: {{hadoop jar}} we get the usage text, but
> even in that case we get:
> {noformat}
> WARNING: Use "yarn jar" to launch YARN applications.
> RunJar jarFile [mainClass] args...
> {noformat}
> Where RunJar is wrapped by the hadoop script (so it should not be displayed).
> {{hadoop --help}} displays the following:
> {noformat}
> jar run a jar file. NOTE: please use "yarn jar" to launch YARN
> applications, not this command.
> {noformat}
> which is fine, but {{CommandsManual.md}} tells a bit more information about
> the usage of this command:
> {noformat}
> Usage: hadoop jar [mainClass] args...
> {noformat}
> My suggestion is to add a {{--help}} option to the {{hadoop jar}} command
> that would display this message.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11890) Uber-JIRA: Hadoop should support IPv6
[ https://issues.apache.org/jira/browse/HADOOP-11890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754486#comment-16754486 ] Daniel Templeton commented on HADOOP-11890: --- I'm just wondering the same myself. [~ste...@apache.org], [~shv], [~nkedel], anyone know where this stands? > Uber-JIRA: Hadoop should support IPv6 > - > > Key: HADOOP-11890 > URL: https://issues.apache.org/jira/browse/HADOOP-11890 > Project: Hadoop Common > Issue Type: Improvement > Components: net >Reporter: Nate Edel >Assignee: Nate Edel >Priority: Major > Labels: ipv6 > > Hadoop currently treats IPv6 as unsupported. Track related smaller issues to > support IPv6. > (Current case here is mainly HBase on HDFS, so any suggestions about other > test cases/workload are really appreciated.) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15997) KMS client uses wrong UGI after HADOOP-14445
[
https://issues.apache.org/jira/browse/HADOOP-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721382#comment-16721382
]
Daniel Templeton commented on HADOOP-15997:
---
The change looks good to me. Two minor quibbles. First, I always appreciate
messages in assert statements so that I don't have to read code to know how the
test failed. Second, we seem to have lost some context in the debug logs with
the new patch. There's still enough debug logging to show that something is
happening, but it's no longer clear why.
I would love to get a nod from [~daryn] or [~xiaochen] before we move forward.
> KMS client uses wrong UGI after HADOOP-14445
>
>
> Key: HADOOP-15997
> URL: https://issues.apache.org/jira/browse/HADOOP-15997
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.2.0, 3.0.4, 3.1.2
> Environment: Hadoop 3.0.x (CDH6.x), Kerberized, HDFS at-rest
> encryption, multiple KMS
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HADOOP-15997.001.patch
>
>
> After HADOOP-14445, KMS client always authenticates itself using the
> credentials from login user, rather than current user.
> {noformat}
> 2018-12-07 15:58:30,663 DEBUG [main]
> org.apache.hadoop.crypto.key.kms.KMSClientProvider: Using loginUser when
> Kerberos is enabled but the actual user does not have either KMS Delegation
> Token or Kerberos Credentials
> {noformat}
> The log message {{"Using loginUser when Kerberos is enabled but the actual
> user does not have either KMS Delegation Token or Kerberos Credentials"}} is
> printed because {{KMSClientProvider#containsKmsDt()}} is null when it
> definitely has the kms delegation token.
> In fact, {{KMSClientProvider#containsKmsDt()}} should select delegation token
> using {{clientTokenProvider.selectDelegationToken(creds)}} rather than
> checking if its dtService is in the user credentials.
> This is done correctly in {{KMSClientProvider#createAuthenticatedURL}} though.
> We found this bug when it broke Cloudera's Backup and Disaster Recovery tool.
>
> [~daryn] [~xiaochen] mind taking a look? HADOOP-14445 is a huge patch but it
> is almost perfect except for this bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15997) KMS client uses wrong UGI after HADOOP-14445
[
https://issues.apache.org/jira/browse/HADOOP-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated HADOOP-15997:
--
Status: Patch Available (was: Open)
> KMS client uses wrong UGI after HADOOP-14445
>
>
> Key: HADOOP-15997
> URL: https://issues.apache.org/jira/browse/HADOOP-15997
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.2.0, 3.0.4, 3.1.2
> Environment: Hadoop 3.0.x (CDH6.x), Kerberized, HDFS at-rest
> encryption, multiple KMS
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HADOOP-15997.001.patch
>
>
> After HADOOP-14445, KMS client always authenticates itself using the
> credentials from login user, rather than current user.
> {noformat}
> 2018-12-07 15:58:30,663 DEBUG [main]
> org.apache.hadoop.crypto.key.kms.KMSClientProvider: Using loginUser when
> Kerberos is enabled but the actual user does not have either KMS Delegation
> Token or Kerberos Credentials
> {noformat}
> The log message {{"Using loginUser when Kerberos is enabled but the actual
> user does not have either KMS Delegation Token or Kerberos Credentials"}} is
> printed because {{KMSClientProvider#containsKmsDt()}} is null when it
> definitely has the kms delegation token.
> In fact, {{KMSClientProvider#containsKmsDt()}} should select delegation token
> using {{clientTokenProvider.selectDelegationToken(creds)}} rather than
> checking if its dtService is in the user credentials.
> This is done correctly in {{KMSClientProvider#createAuthenticatedURL}} though.
> We found this bug when it broke Cloudera's Backup and Disaster Recovery tool.
>
> [~daryn] [~xiaochen] mind taking a look? HADOOP-14445 is a huge patch but it
> is almost perfect except for this bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15823) ABFS: Stop requiring client ID and tenant ID for MSI
[ https://issues.apache.org/jira/browse/HADOOP-15823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-15823: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) Thank you for the patch, [~DanielZhou] and for the reviews, [~mackrorysd], [~shwetayakkali], and [~tmarquardt]! Committed to trunk. > ABFS: Stop requiring client ID and tenant ID for MSI > > > Key: HADOOP-15823 > URL: https://issues.apache.org/jira/browse/HADOOP-15823 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.2.0 >Reporter: Sean Mackrory >Assignee: Da Zhou >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-15823-001.patch, HADOOP-15823-002.patch > > > ABFS requires the user to configure the tenant ID and client ID. From my > understanding of MSI, that shouldn't be necessary and is an added requirement > compared to MSI in ADLS. Can that be dropped? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15823) ABFS: Stop requiring client ID and tenant ID for MSI
[ https://issues.apache.org/jira/browse/HADOOP-15823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662857#comment-16662857 ] Daniel Templeton commented on HADOOP-15823: --- Thanks for testing the patch, [~shwetayakkali]. The patch looks good to me as well. +1 I'll go ahead and commit shortly. > ABFS: Stop requiring client ID and tenant ID for MSI > > > Key: HADOOP-15823 > URL: https://issues.apache.org/jira/browse/HADOOP-15823 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.2.0 >Reporter: Sean Mackrory >Assignee: Da Zhou >Priority: Major > Attachments: HADOOP-15823-001.patch, HADOOP-15823-002.patch > > > ABFS requires the user to configure the tenant ID and client ID. From my > understanding of MSI, that shouldn't be necessary and is an added requirement > compared to MSI in ADLS. Can that be dropped? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15706) Typo in compatibility doc: SHOUD -> SHOULD
[
https://issues.apache.org/jira/browse/HADOOP-15706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated HADOOP-15706:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.1.2
3.0.4
3.2.0
Status: Resolved (was: Patch Available)
Thanks for the patch! Committed to trunk, branch-3.1, and branch-3.0.
> Typo in compatibility doc: SHOUD -> SHOULD
> --
>
> Key: HADOOP-15706
> URL: https://issues.apache.org/jira/browse/HADOOP-15706
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Daniel Templeton
>Assignee: Laszlo Kollar
>Priority: Trivial
> Fix For: 3.2.0, 3.0.4, 3.1.2
>
> Attachments: HADOOP-15706.001.patch
>
>
> {quote}% grep SHOUD
> ./hadoop-common-project/hadoop-common/src/site/markdown/Compatibility.md
> Apache Hadoop revisions SHOUD retain binary compatability such that
> end-user{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15706) Typo in compatibility doc: SHOUD -> SHOULD
[
https://issues.apache.org/jira/browse/HADOOP-15706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597615#comment-16597615
]
Daniel Templeton commented on HADOOP-15706:
---
+1, thanks for the patch.
> Typo in compatibility doc: SHOUD -> SHOULD
> --
>
> Key: HADOOP-15706
> URL: https://issues.apache.org/jira/browse/HADOOP-15706
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Daniel Templeton
>Assignee: Laszlo Kollar
>Priority: Trivial
> Attachments: HADOOP-15706.001.patch
>
>
> {quote}% grep SHOUD
> ./hadoop-common-project/hadoop-common/src/site/markdown/Compatibility.md
> Apache Hadoop revisions SHOUD retain binary compatability such that
> end-user{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-15706) Typo in compatibility doc: SHOUD -> SHOULD
[
https://issues.apache.org/jira/browse/HADOOP-15706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton reassigned HADOOP-15706:
-
Assignee: Laszlo Kollar (was: Daniel Templeton)
> Typo in compatibility doc: SHOUD -> SHOULD
> --
>
> Key: HADOOP-15706
> URL: https://issues.apache.org/jira/browse/HADOOP-15706
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Daniel Templeton
>Assignee: Laszlo Kollar
>Priority: Trivial
>
> {quote}% grep SHOUD
> ./hadoop-common-project/hadoop-common/src/site/markdown/Compatibility.md
> Apache Hadoop revisions SHOUD retain binary compatability such that
> end-user{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15705) Typo in the definition of "stable" in the interface classification
[ https://issues.apache.org/jira/browse/HADOOP-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596856#comment-16596856 ] Daniel Templeton commented on HADOOP-15705: --- Filed HADOOP-15706 for the other typo you found. > Typo in the definition of "stable" in the interface classification > -- > > Key: HADOOP-15705 > URL: https://issues.apache.org/jira/browse/HADOOP-15705 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Minor > Fix For: 3.2.0, 3.0.4, 3.1.2 > > Attachments: HADOOP-15705.001.patch > > > "Compatible changes allowed: maintenance (x.Y.0)" > should be > "Compatible changes allowed: maintenance (x.y.Z)" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15706) Typo in compatibility doc: SHOUD -> SHOULD
Daniel Templeton created HADOOP-15706:
-
Summary: Typo in compatibility doc: SHOUD -> SHOULD
Key: HADOOP-15706
URL: https://issues.apache.org/jira/browse/HADOOP-15706
Project: Hadoop Common
Issue Type: Bug
Reporter: Daniel Templeton
Assignee: Daniel Templeton
{quote}% grep SHOUD
./hadoop-common-project/hadoop-common/src/site/markdown/Compatibility.md
Apache Hadoop revisions SHOUD retain binary compatability such that
end-user{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15705) Typo in the definition of "stable" in the interface classification
[ https://issues.apache.org/jira/browse/HADOOP-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-15705: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.2 3.0.4 3.2.0 Status: Resolved (was: Patch Available) Thanks for the review, [~jojochuang]. Committed to trunk, branch-3.1, and branch-3.0. > Typo in the definition of "stable" in the interface classification > -- > > Key: HADOOP-15705 > URL: https://issues.apache.org/jira/browse/HADOOP-15705 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Minor > Fix For: 3.2.0, 3.0.4, 3.1.2 > > Attachments: HADOOP-15705.001.patch > > > "Compatible changes allowed: maintenance (x.Y.0)" > should be > "Compatible changes allowed: maintenance (x.y.Z)" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15705) Typo in the definition of "stable" in the interface classification
[ https://issues.apache.org/jira/browse/HADOOP-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-15705: -- Status: Patch Available (was: Open) > Typo in the definition of "stable" in the interface classification > -- > > Key: HADOOP-15705 > URL: https://issues.apache.org/jira/browse/HADOOP-15705 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Minor > Attachments: HADOOP-15705.001.patch > > > "Compatible changes allowed: maintenance (x.Y.0)" > should be > "Compatible changes allowed: maintenance (x.y.Z)" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15705) Typo in the definition of "stable" in the interface classification
[ https://issues.apache.org/jira/browse/HADOOP-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-15705: -- Attachment: HADOOP-15705.001.patch > Typo in the definition of "stable" in the interface classification > -- > > Key: HADOOP-15705 > URL: https://issues.apache.org/jira/browse/HADOOP-15705 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Minor > Attachments: HADOOP-15705.001.patch > > > "Compatible changes allowed: maintenance (x.Y.0)" > should be > "Compatible changes allowed: maintenance (x.y.Z)" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15705) Typo in the definition of "stable" in the interface classification
Daniel Templeton created HADOOP-15705: - Summary: Typo in the definition of "stable" in the interface classification Key: HADOOP-15705 URL: https://issues.apache.org/jira/browse/HADOOP-15705 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.0.0 Reporter: Daniel Templeton Assignee: Daniel Templeton "Compatible changes allowed: maintenance (x.Y.0)" should be "Compatible changes allowed: maintenance (x.y.Z)" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15698) KMS startup logs don't show
[ https://issues.apache.org/jira/browse/HADOOP-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595662#comment-16595662 ] Daniel Templeton commented on HADOOP-15698: --- +1 from me as well. We don't generally worry about testing logging, because it's seldom straight-forward. > KMS startup logs don't show > --- > > Key: HADOOP-15698 > URL: https://issues.apache.org/jira/browse/HADOOP-15698 > Project: Hadoop Common > Issue Type: Bug > Components: kms >Affects Versions: 3.1.0 >Reporter: Kitti Nanasi >Assignee: Kitti Nanasi >Priority: Major > Attachments: HADOOP-15698.001.patch > > > During KMs startup, log4j logs don't show up resulting in important logs > getting omitted. This happens because log4 initialisation only happens in > KMSWebApp#contextInitialized and logs written before that don't show up. > For example the following log never shows up: > [https://github.com/apache/hadoop/blob/a55d6bba71c81c1c4e9d8cd11f55c78f10a548b0/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/ZKSignerSecretProvider.java#L197-L199] > Another example is that the KMS startup message never shows up in the kms > logs. > Note that this works in the unit tests, because MiniKMS sets the log4j system > property. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15307) Improve NFS error handling: Unsupported verifier flavorAUTH_SYS
[
https://issues.apache.org/jira/browse/HADOOP-15307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506202#comment-16506202
]
Daniel Templeton commented on HADOOP-15307:
---
Thanks, [~gabor.bota]. Two more tiny issues, and I think we're good. First,
there should be a space before the paren in the _if_ statements. I know there
isn't one on the existing _if_ statements, but that's also wrong. You can
either just fix yours or fix them for the whole _if-else_. Second, in the
exception message, please add a colon after "flavor".
> Improve NFS error handling: Unsupported verifier flavorAUTH_SYS
> ---
>
> Key: HADOOP-15307
> URL: https://issues.apache.org/jira/browse/HADOOP-15307
> Project: Hadoop Common
> Issue Type: Bug
> Components: nfs
> Environment: CentOS 7.4, CDH5.13.1, Kerberized Hadoop cluster
>Reporter: Wei-Chiu Chuang
>Assignee: Gabor Bota
>Priority: Major
> Attachments: HADOOP-15307.001.patch, HADOOP-15307.002.patch,
> HADOOP-15307.003.patch, HADOOP-15307.004.patch
>
>
> When NFS gateway starts and if the portmapper request is denied by rpcbind
> for any reason (in our case, /etc/hosts.allow did not have the localhost),
> NFS gateway fails with the following obscure exception:
> {noformat}
> 2018-03-05 12:49:31,976 INFO org.apache.hadoop.oncrpc.SimpleUdpServer:
> Started listening to UDP requests at port 4242 for Rpc program: mountd at
> localhost:4242 with workerCount 1
> 2018-03-05 12:49:31,988 INFO org.apache.hadoop.oncrpc.SimpleTcpServer:
> Started listening to TCP requests at port 4242 for Rpc program: mountd at
> localhost:4242 with workerCount 1
> 2018-03-05 12:49:31,993 TRACE org.apache.hadoop.oncrpc.RpcCall:
> Xid:692394656, messageType:RPC_CALL, rpcVersion:2, program:10, version:2,
> procedure:1, credential:(AuthFlavor:AUTH_NONE),
> verifier:(AuthFlavor:AUTH_NONE)
> 2018-03-05 12:49:31,998 FATAL org.apache.hadoop.mount.MountdBase: Failed to
> start the server. Cause:
> java.lang.UnsupportedOperationException: Unsupported verifier flavorAUTH_SYS
> at
> org.apache.hadoop.oncrpc.security.Verifier.readFlavorAndVerifier(Verifier.java:45)
> at org.apache.hadoop.oncrpc.RpcDeniedReply.read(RpcDeniedReply.java:50)
> at org.apache.hadoop.oncrpc.RpcReply.read(RpcReply.java:67)
> at org.apache.hadoop.oncrpc.SimpleUdpClient.run(SimpleUdpClient.java:71)
> at org.apache.hadoop.oncrpc.RpcProgram.register(RpcProgram.java:130)
> at org.apache.hadoop.oncrpc.RpcProgram.register(RpcProgram.java:101)
> at org.apache.hadoop.mount.MountdBase.start(MountdBase.java:83)
> at org.apache.hadoop.hdfs.nfs.nfs3.Nfs3.startServiceInternal(Nfs3.java:56)
> at org.apache.hadoop.hdfs.nfs.nfs3.Nfs3.startService(Nfs3.java:69)
> at
> org.apache.hadoop.hdfs.nfs.nfs3.PrivilegedNfsGatewayStarter.start(PrivilegedNfsGatewayStarter.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243)
> 2018-03-05 12:49:32,007 INFO org.apache.hadoop.util.ExitUtil: Exiting with
> status 1{noformat}
> Reading the code comment for class Verifier, I think this bug existed since
> its inception
> {code:java}
> /**
> * Base class for verifier. Currently our authentication only supports 3 types
> * of auth flavors: {@link RpcAuthInfo.AuthFlavor#AUTH_NONE}, {@link
> RpcAuthInfo.AuthFlavor#AUTH_SYS},
> * and {@link RpcAuthInfo.AuthFlavor#RPCSEC_GSS}. Thus for verifier we only
> need to handle
> * AUTH_NONE and RPCSEC_GSS
> */
> public abstract class Verifier extends RpcAuthInfo {{code}
> The verifier should also handle AUTH_SYS too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15307) Improve NFS error handling: Unsupported verifier flavorAUTH_SYS
[
https://issues.apache.org/jira/browse/HADOOP-15307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503511#comment-16503511
]
Daniel Templeton commented on HADOOP-15307:
---
I can live with the separate _else if_, but you should add a comment that
explains why you're creating an {{VerifierNone}} for {{AUTH_SYS}}. Looking at
it casually, it looks like a mistake.
> Improve NFS error handling: Unsupported verifier flavorAUTH_SYS
> ---
>
> Key: HADOOP-15307
> URL: https://issues.apache.org/jira/browse/HADOOP-15307
> Project: Hadoop Common
> Issue Type: Bug
> Components: nfs
> Environment: CentOS 7.4, CDH5.13.1, Kerberized Hadoop cluster
>Reporter: Wei-Chiu Chuang
>Assignee: Gabor Bota
>Priority: Major
> Attachments: HADOOP-15307.001.patch, HADOOP-15307.002.patch,
> HADOOP-15307.003.patch
>
>
> When NFS gateway starts and if the portmapper request is denied by rpcbind
> for any reason (in our case, /etc/hosts.allow did not have the localhost),
> NFS gateway fails with the following obscure exception:
> {noformat}
> 2018-03-05 12:49:31,976 INFO org.apache.hadoop.oncrpc.SimpleUdpServer:
> Started listening to UDP requests at port 4242 for Rpc program: mountd at
> localhost:4242 with workerCount 1
> 2018-03-05 12:49:31,988 INFO org.apache.hadoop.oncrpc.SimpleTcpServer:
> Started listening to TCP requests at port 4242 for Rpc program: mountd at
> localhost:4242 with workerCount 1
> 2018-03-05 12:49:31,993 TRACE org.apache.hadoop.oncrpc.RpcCall:
> Xid:692394656, messageType:RPC_CALL, rpcVersion:2, program:10, version:2,
> procedure:1, credential:(AuthFlavor:AUTH_NONE),
> verifier:(AuthFlavor:AUTH_NONE)
> 2018-03-05 12:49:31,998 FATAL org.apache.hadoop.mount.MountdBase: Failed to
> start the server. Cause:
> java.lang.UnsupportedOperationException: Unsupported verifier flavorAUTH_SYS
> at
> org.apache.hadoop.oncrpc.security.Verifier.readFlavorAndVerifier(Verifier.java:45)
> at org.apache.hadoop.oncrpc.RpcDeniedReply.read(RpcDeniedReply.java:50)
> at org.apache.hadoop.oncrpc.RpcReply.read(RpcReply.java:67)
> at org.apache.hadoop.oncrpc.SimpleUdpClient.run(SimpleUdpClient.java:71)
> at org.apache.hadoop.oncrpc.RpcProgram.register(RpcProgram.java:130)
> at org.apache.hadoop.oncrpc.RpcProgram.register(RpcProgram.java:101)
> at org.apache.hadoop.mount.MountdBase.start(MountdBase.java:83)
> at org.apache.hadoop.hdfs.nfs.nfs3.Nfs3.startServiceInternal(Nfs3.java:56)
> at org.apache.hadoop.hdfs.nfs.nfs3.Nfs3.startService(Nfs3.java:69)
> at
> org.apache.hadoop.hdfs.nfs.nfs3.PrivilegedNfsGatewayStarter.start(PrivilegedNfsGatewayStarter.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243)
> 2018-03-05 12:49:32,007 INFO org.apache.hadoop.util.ExitUtil: Exiting with
> status 1{noformat}
> Reading the code comment for class Verifier, I think this bug existed since
> its inception
> {code:java}
> /**
> * Base class for verifier. Currently our authentication only supports 3 types
> * of auth flavors: {@link RpcAuthInfo.AuthFlavor#AUTH_NONE}, {@link
> RpcAuthInfo.AuthFlavor#AUTH_SYS},
> * and {@link RpcAuthInfo.AuthFlavor#RPCSEC_GSS}. Thus for verifier we only
> need to handle
> * AUTH_NONE and RPCSEC_GSS
> */
> public abstract class Verifier extends RpcAuthInfo {{code}
> The verifier should also handle AUTH_SYS too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15299) Bump Hadoop's Jackson 2 dependency 2.9.x
[ https://issues.apache.org/jira/browse/HADOOP-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412302#comment-16412302 ] Daniel Templeton commented on HADOOP-15299: --- The change looks safe enough, as long as you've tested it thoroughly, which it sounds like you have. Given that it closes out some CVEs, I'd say we should get it in. +1 > Bump Hadoop's Jackson 2 dependency 2.9.x > > > Key: HADOOP-15299 > URL: https://issues.apache.org/jira/browse/HADOOP-15299 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.1.0, 3.2.0 >Reporter: Sean Mackrory >Assignee: Sean Mackrory >Priority: Major > Attachments: HADOOP-15299.001.patch > > > There are a few new CVEs open against Jackson 2.7.x. It doesn't (necessarily) > mean Hadoop is vulnerable to the attack - I don't know that it is, but fixes > were released for Jackson 2.8.x and 2.9.x but not 2.7.x (which we're on). We > shouldn't be on an unmaintained line, regardless. HBase is already on 2.9.x, > we have a shaded client now, the API changes are relatively minor and so far > in my testing I haven't seen any problems. I think many of our usual reasons > to hesitate upgrading this dependency don't apply. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14875) Create end user documentation from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367944#comment-16367944 ] Daniel Templeton commented on HADOOP-14875: --- I attached a patch to address your first three concerns. The fourth one is correct as is. > Create end user documentation from the compatibility guidelines > --- > > Key: HADOOP-14875 > URL: https://issues.apache.org/jira/browse/HADOOP-14875 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: AdminCompatibilityGuide.pdf, Compatibility.htm, > HADOOP-14875.001.patch, HADOOP-14875.002.patch, HADOOP-14875.003.patch, > HADOOP-14875.004.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14875) Create end user documentation from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14875: -- Attachment: HADOOP-14875.004.patch > Create end user documentation from the compatibility guidelines > --- > > Key: HADOOP-14875 > URL: https://issues.apache.org/jira/browse/HADOOP-14875 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: AdminCompatibilityGuide.pdf, Compatibility.htm, > HADOOP-14875.001.patch, HADOOP-14875.002.patch, HADOOP-14875.003.patch, > HADOOP-14875.004.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-15160) Confusing text in http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html
[ https://issues.apache.org/jira/browse/HADOOP-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton reassigned HADOOP-15160: - Assignee: Daniel Templeton > Confusing text in > http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html > > > Key: HADOOP-15160 > URL: https://issues.apache.org/jira/browse/HADOOP-15160 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Jim Showalter >Assignee: Daniel Templeton >Priority: Minor > > The text in wire formats, policy, is confusing. > First, there are two subsections with the same heading: > The following changes to a .proto file SHALL be considered incompatible: > The following changes to a .proto file SHALL be considered incompatible: > Second, one of the items listed under the first of those two headings seems > like it is a compatible change, not an incompatible change: > Delete an optional field as long as the optional field has reasonable > defaults to allow deletions -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15160) Confusing text in http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html
[ https://issues.apache.org/jira/browse/HADOOP-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318994#comment-16318994 ] Daniel Templeton commented on HADOOP-15160: --- Thanks for the heads up, [~steve_l]. I agree that it sounds like that line ended up under the wrong heading. The duplicate headings also give mo pause. I'll have to dig in to see where exactly that went south, but I'm on vacation until next Tuesday. I'll take a look then. Feel free to assign it to me. > Confusing text in > http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html > > > Key: HADOOP-15160 > URL: https://issues.apache.org/jira/browse/HADOOP-15160 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Jim Showalter >Priority: Minor > > The text in wire formats, policy, is confusing. > First, there are two subsections with the same heading: > The following changes to a .proto file SHALL be considered incompatible: > The following changes to a .proto file SHALL be considered incompatible: > Second, one of the items listed under the first of those two headings seems > like it is a compatible change, not an incompatible change: > Delete an optional field as long as the optional field has reasonable > defaults to allow deletions -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13493) Compatibility Docs should clarify the policy for what takes precedence when a conflict is found
[
https://issues.apache.org/jira/browse/HADOOP-13493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273002#comment-16273002
]
Daniel Templeton commented on HADOOP-13493:
---
I also pulled it back into branch-3.0.0.
> Compatibility Docs should clarify the policy for what takes precedence when a
> conflict is found
> ---
>
> Key: HADOOP-13493
> URL: https://issues.apache.org/jira/browse/HADOOP-13493
> Project: Hadoop Common
> Issue Type: Task
> Components: documentation
>Affects Versions: 2.7.2
>Reporter: Robert Kanter
>Assignee: Daniel Templeton
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HADOOP-13493.001.patch, HADOOP-13493.002.patch
>
>
> The Compatibility Docs
> (https://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/Compatibility.html#Java_API)
> list the policies for Private, Public, not annotated, etc Classes and
> members, but it doesn't say what happens when there's a conflict. We should
> try obviously try to avoid this situation, but it would be good to explicitly
> state what takes precedence.
> As an example, until YARN-3225 made it consistent, {{RefreshNodesRequest}}
> looked like this:
> {code:java}
> @Private
> @Stable
> public abstract class RefreshNodesRequest {
> @Public
> @Stable
> public static RefreshNodesRequest newInstance() {
> RefreshNodesRequest request =
> Records.newRecord(RefreshNodesRequest.class);
> return request;
> }
> }
> {code}
> Note that the class is marked {{\@Private}}, but the method is marked
> {{\@Public}}.
> In this example, I'd say that the class level should have priority.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14875) Create end user documentation from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266286#comment-16266286 ] Daniel Templeton commented on HADOOP-14875: --- Any other comments, [~anu], [~rkanter], [~steve_l], [~leftnoteasy], or [~sunilg]? > Create end user documentation from the compatibility guidelines > --- > > Key: HADOOP-14875 > URL: https://issues.apache.org/jira/browse/HADOOP-14875 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: AdminCompatibilityGuide.pdf, Compatibility.htm, > HADOOP-14875.001.patch, HADOOP-14875.002.patch, HADOOP-14875.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14876: -- Fix Version/s: (was: 3.0.1) 3.0.0 > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Fix For: 3.0.0, 3.1.0 > > Attachments: Compatibility.pdf, DownstreamDev.pdf, > HADOOP-14876.001.patch, HADOOP-14876.002.patch, HADOOP-14876.003.patch, > HADOOP-14876.004.patch, HADOOP-14876.005.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13493) Compatibility Docs should clarify the policy for what takes precedence when a conflict is found
[
https://issues.apache.org/jira/browse/HADOOP-13493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated HADOOP-13493:
--
Attachment: HADOOP-13493.002.patch
You're right--that patch was useless. :) Try this.
> Compatibility Docs should clarify the policy for what takes precedence when a
> conflict is found
> ---
>
> Key: HADOOP-13493
> URL: https://issues.apache.org/jira/browse/HADOOP-13493
> Project: Hadoop Common
> Issue Type: Task
> Components: documentation
>Affects Versions: 2.7.2
>Reporter: Robert Kanter
>Assignee: Daniel Templeton
>Priority: Critical
> Attachments: HADOOP-13493.001.patch, HADOOP-13493.002.patch
>
>
> The Compatibility Docs
> (https://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/Compatibility.html#Java_API)
> list the policies for Private, Public, not annotated, etc Classes and
> members, but it doesn't say what happens when there's a conflict. We should
> try obviously try to avoid this situation, but it would be good to explicitly
> state what takes precedence.
> As an example, until YARN-3225 made it consistent, {{RefreshNodesRequest}}
> looked like this:
> {code:java}
> @Private
> @Stable
> public abstract class RefreshNodesRequest {
> @Public
> @Stable
> public static RefreshNodesRequest newInstance() {
> RefreshNodesRequest request =
> Records.newRecord(RefreshNodesRequest.class);
> return request;
> }
> }
> {code}
> Note that the class is marked {{\@Private}}, but the method is marked
> {{\@Public}}.
> In this example, I'd say that the class level should have priority.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13493) Compatibility Docs should clarify the policy for what takes precedence when a conflict is found
[
https://issues.apache.org/jira/browse/HADOOP-13493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated HADOOP-13493:
--
Target Version/s: 3.0.0, 3.1.0 (was: 3.1.0)
> Compatibility Docs should clarify the policy for what takes precedence when a
> conflict is found
> ---
>
> Key: HADOOP-13493
> URL: https://issues.apache.org/jira/browse/HADOOP-13493
> Project: Hadoop Common
> Issue Type: Task
> Components: documentation
>Affects Versions: 2.7.2
>Reporter: Robert Kanter
>Assignee: Daniel Templeton
>Priority: Critical
> Attachments: HADOOP-13493.001.patch
>
>
> The Compatibility Docs
> (https://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/Compatibility.html#Java_API)
> list the policies for Private, Public, not annotated, etc Classes and
> members, but it doesn't say what happens when there's a conflict. We should
> try obviously try to avoid this situation, but it would be good to explicitly
> state what takes precedence.
> As an example, until YARN-3225 made it consistent, {{RefreshNodesRequest}}
> looked like this:
> {code:java}
> @Private
> @Stable
> public abstract class RefreshNodesRequest {
> @Public
> @Stable
> public static RefreshNodesRequest newInstance() {
> RefreshNodesRequest request =
> Records.newRecord(RefreshNodesRequest.class);
> return request;
> }
> }
> {code}
> Note that the class is marked {{\@Private}}, but the method is marked
> {{\@Public}}.
> In this example, I'd say that the class level should have priority.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16263036#comment-16263036 ] Daniel Templeton commented on HADOOP-14876: --- [~andrew.wang], can we pull this in for the respin of 3.0.0 RC? > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Fix For: 3.1.0, 3.0.1 > > Attachments: Compatibility.pdf, DownstreamDev.pdf, > HADOOP-14876.001.patch, HADOOP-14876.002.patch, HADOOP-14876.003.patch, > HADOOP-14876.004.patch, HADOOP-14876.005.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14875) Create end user documentation from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14875: -- Attachment: AdminCompatibilityGuide.pdf Updated PDF > Create end user documentation from the compatibility guidelines > --- > > Key: HADOOP-14875 > URL: https://issues.apache.org/jira/browse/HADOOP-14875 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: AdminCompatibilityGuide.pdf, Compatibility.htm, > HADOOP-14875.001.patch, HADOOP-14875.002.patch, HADOOP-14875.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14875) Create end user documentation from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14875: -- Attachment: (was: AdminCompatibilityGuide.htm) > Create end user documentation from the compatibility guidelines > --- > > Key: HADOOP-14875 > URL: https://issues.apache.org/jira/browse/HADOOP-14875 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: AdminCompatibilityGuide.pdf, Compatibility.htm, > HADOOP-14875.001.patch, HADOOP-14875.002.patch, HADOOP-14875.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14875) Create end user documentation from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14875: -- Attachment: HADOOP-14875.003.patch Updated patch to address comments. > Create end user documentation from the compatibility guidelines > --- > > Key: HADOOP-14875 > URL: https://issues.apache.org/jira/browse/HADOOP-14875 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: AdminCompatibilityGuide.htm, Compatibility.htm, > HADOOP-14875.001.patch, HADOOP-14875.002.patch, HADOOP-14875.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14876: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.1 3.1.0 Status: Resolved (was: Patch Available) Thanks for the reviews, @steve_l, [~anu], and [~rkanter]. Committed to trunk and branch-3.0. 3.0 is already rolling out, so this will have to go into 3.0.1. We therefore still have time to amend things if anyone has any late-breaking changes they'd like to see. (Sorry, [~steve_l], I just realized I only let this one sit overnight before the push. When I pushed it I thought it was 24 hours.) > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Fix For: 3.1.0, 3.0.1 > > Attachments: Compatibility.pdf, DownstreamDev.pdf, > HADOOP-14876.001.patch, HADOOP-14876.002.patch, HADOOP-14876.003.patch, > HADOOP-14876.004.patch, HADOOP-14876.005.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252164#comment-16252164 ] Daniel Templeton commented on HADOOP-14876: --- [~anu], [~steve_l], [~rkanter], any other comments? > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: Compatibility.pdf, DownstreamDev.pdf, > HADOOP-14876.001.patch, HADOOP-14876.002.patch, HADOOP-14876.003.patch, > HADOOP-14876.004.patch, HADOOP-14876.005.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14876: -- Attachment: (was: Compatibility.pdf) > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: Compatibility.pdf, DownstreamDev.pdf, > HADOOP-14876.001.patch, HADOOP-14876.002.patch, HADOOP-14876.003.patch, > HADOOP-14876.004.patch, HADOOP-14876.005.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14876: -- Attachment: Compatibility.pdf DownstreamDev.pdf Updated the PDFs as well. > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: Compatibility.pdf, DownstreamDev.pdf, > HADOOP-14876.001.patch, HADOOP-14876.002.patch, HADOOP-14876.003.patch, > HADOOP-14876.004.patch, HADOOP-14876.005.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14876: -- Attachment: (was: DownstreamDev.pdf) > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: Compatibility.pdf, DownstreamDev.pdf, > HADOOP-14876.001.patch, HADOOP-14876.002.patch, HADOOP-14876.003.patch, > HADOOP-14876.004.patch, HADOOP-14876.005.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14876: -- Attachment: HADOOP-14876.005.patch Thanks for the thoughtful review, [~anu]. bq. Use cases Matrix OK bq. It would be good to define which kind of releases are possible Fair point bq. Semantic compatibility Good point. I don't think this goes in the API section or the config section. I've added a new section to address functional compatibility. bq. may I suggest that we add the word "current" Done bq. "that are meaningful to Hadoop" – This is a very loose definition The intent is indeed that no env vars that are interpreted by Hadoop will change in their interpretation between minor releases. I'll see if I can come up with a clearer way to say that. bq. As a non-native English language speaker, I wonder if this statement is ambiguous I don't think it is. It's phrased that way because the compat spec is intended as a spec. "held stable" is definitely easier to read, but it's not specific enough. Looks like I left native dependencies out of the dev guide. I've added it in now. bq. I don't think that we can treat S3 or Kerberos as internal protocols. bq. Did you mean to write, default service ports instead of fixed? Good catch. bq. New transport mechanisms MUST only be introduced with minor or major version changes. I'm OK with removing this constraint. I no longer remember why it's there. :) bq. Should we just remove the automation phrase? I think the problem is less the automation phrase and more the policy below it. I agree that restricting log output changes to minor releases is too draconian. We'll end up excluding patches from being backported for dubious reasons. I've relaxed the policy to Unstable. bq. If we have an upgrade path, I submit that this should be possible. I updated the data node section to talk about upgrade paths instead. bq. Are we sure that 3.0 release is entirely complaint to this spec? I wouldn't be surprised if it weren't. The slaves.txt file isn't explicitly covered by this spec. The CLI policy is only concerned with command output. I think slaves.txt should fall under the configuration policy, though it's not currently covered there. Given that there are several non-XML config files now, it seems that's an oversight I should correct. When you're asking if the change is compatible, do you mean the renaming of the file? bq. Do you have a case where this has happened? If not, we should allow this change. "user-accessible" is an extensive term. The policy is below in the policy section. There we specifically enumerate the things that can't change in a maintenance release and declare everything else free to change. I also added a section at the top to better explain the structure of the document. bq. We should have a full list of supported version documented somewhere. I agree. There is no such document that I know of. > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: Compatibility.pdf, DownstreamDev.pdf, > HADOOP-14876.001.patch, HADOOP-14876.002.patch, HADOOP-14876.003.patch, > HADOOP-14876.004.patch, HADOOP-14876.005.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14875) Create end user documentation from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14875: -- Status: Patch Available (was: Open) > Create end user documentation from the compatibility guidelines > --- > > Key: HADOOP-14875 > URL: https://issues.apache.org/jira/browse/HADOOP-14875 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: AdminCompatibilityGuide.htm, Compatibility.htm, > HADOOP-14875.001.patch, HADOOP-14875.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14875) Create end user documentation from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14875: -- Attachment: AdminCompatibilityGuide.htm Compatibility.htm HADOOP-14875.002.patch Corrected client JARs list and added PDFs of the resulting docs. > Create end user documentation from the compatibility guidelines > --- > > Key: HADOOP-14875 > URL: https://issues.apache.org/jira/browse/HADOOP-14875 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: AdminCompatibilityGuide.htm, Compatibility.htm, > HADOOP-14875.001.patch, HADOOP-14875.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14876: -- Attachment: HADOOP-14876.004.patch Corrected client library list. > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: Compatibility.pdf, DownstreamDev.pdf, > HADOOP-14876.001.patch, HADOOP-14876.002.patch, HADOOP-14876.003.patch, > HADOOP-14876.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14875) Create end user documentation from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14875: -- Attachment: HADOOP-14875.001.patch First pass. It borrows heavily from HADOOP-14876 in some parts. > Create end user documentation from the compatibility guidelines > --- > > Key: HADOOP-14875 > URL: https://issues.apache.org/jira/browse/HADOOP-14875 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: HADOOP-14875.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14876: -- Attachment: HADOOP-14876.003.patch This should resolve all your comments, [~rkanter]. I also corrected all the other spelling mistakes you missed. :) > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: Compatibility.pdf, DownstreamDev.pdf, > HADOOP-14876.001.patch, HADOOP-14876.002.patch, HADOOP-14876.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14875) Create end user documentation from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16219206#comment-16219206 ] Daniel Templeton commented on HADOOP-14875: --- I just need time to get it finished. Coming soon. > Create end user documentation from the compatibility guidelines > --- > > Key: HADOOP-14875 > URL: https://issues.apache.org/jira/browse/HADOOP-14875 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14876: -- Attachment: Compatibility.pdf DownstreamDev.pdf Here are the new docs post v3 of this patch. > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: Compatibility.pdf, DownstreamDev.pdf, > HADOOP-14876.001.patch, HADOOP-14876.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216333#comment-16216333 ] Daniel Templeton edited comment on HADOOP-14876 at 10/24/17 5:22 AM: - Attached are the new docs post v2 of this patch. was (Author: templedf): Here are the new docs post v3 of this patch. > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: Compatibility.pdf, DownstreamDev.pdf, > HADOOP-14876.001.patch, HADOOP-14876.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14876: -- Attachment: HADOOP-14876.002.patch [~steve_l], I addressed what I could. There are still a few open questions in my previous comment. I looked for the non-normative (file system) definition bit and couldn't find anything. I went through the 2.7 doc and all of the JIRA comments on the relevant JIRAs. Maybe I'm just blind? > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: HADOOP-14876.001.patch, HADOOP-14876.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14968) YarnClientImpl.getRootQueueInfos() should not do a recursive call to rmClient.getQueueInfo()
Daniel Templeton created HADOOP-14968:
-
Summary: YarnClientImpl.getRootQueueInfos() should not do a
recursive call to rmClient.getQueueInfo()
Key: HADOOP-14968
URL: https://issues.apache.org/jira/browse/HADOOP-14968
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0-beta1
Reporter: Daniel Templeton
Priority: Minor
{code}
QueueInfo rootQueue =
rmClient.getQueueInfo(getQueueInfoRequest(ROOT, false, true, true))
.getQueueInfo();
getChildQueues(rootQueue, queues, false);
{code}
The final parameter to {{getQueueInfoRequest()}} should match the final
parameter to {{getChildQueues()}}. They should both be false in this case.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208606#comment-16208606 ] Daniel Templeton commented on HADOOP-14876: --- Thanks, [~steve_l]. bq. add: sometimes things are marked as private... Excellent point. bq. We are not in a position to tell people not to use @Private, not given we consider doing so essential even for basic example yarn apps. We should file JIRAs to either fixed dist shell or make the private things public. bq. What does it mean if something is tagged as LimitedPrivate... I agree that Limited-Private is an issue. The only way that I can see it making sense is if it's held to only components of Hadoop itself. Otherwise, as you say, it's Public. (Of course, depending on how one looks at it, keeping it within Hadoop isn't any different from Private.) For extra-Hadoop Limited-Private annotations, I'd file JIRAs to make those interfaces Public. bq. What does it mean if a release removes/changes something you depended on... Sure. This falls in line with your first point. I'll add a section on interacting with the Hadoop community. bq. L113. yeah right. It's usually the first port of call, & if you think otherwise, you're not writing enough downstream code. I know it is, but that's broken. Reading the code creates unnecessary dependencies on implementation. I'm doing something wrong if I don't discourage people from reading our code. It's then on us as a community to provide reasonable alternatives. bq. the original Compatibility.md calls out that some bits of the system have non-normative specifications... I'll take a look and see what I may have trimmed out. bq. maybe mention StreamCapabilities.hasCapability ... OK bq. L160 ... Good catch. bq. L189 ... Uh... Setup logging to go to a file instead of the console? bq. L208 ... Good point. bq. L229 ... I'll update it. bq. 298 ... OK bq. No statement on supported operating systems... I left that out because I didn't think it would affect downstream developers much. If you disagree, I'll add it. bq. Concurrency ... What do you mean with concurrency? As a downstream developer experiences Hadoop's concurrency, I would assume it would just show up as performance/latency. > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: HADOOP-14876.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-10584) ActiveStandbyElector goes down if ZK quorum become unavailable
[ https://issues.apache.org/jira/browse/HADOOP-10584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-10584: -- Priority: Major (was: Critical) Release Note: Since this was mostly resolved in YARN-5677, I'm lowering the priority. > ActiveStandbyElector goes down if ZK quorum become unavailable > -- > > Key: HADOOP-10584 > URL: https://issues.apache.org/jira/browse/HADOOP-10584 > Project: Hadoop Common > Issue Type: Bug > Components: ha >Affects Versions: 2.4.0 >Reporter: Karthik Kambatla >Assignee: Daniel Templeton > Attachments: HADOOP-10584.prelim.patch, hadoop-10584-prelim.patch, > rm.log > > > ActiveStandbyElector retries operations for a few times. If the ZK quorum > itself is down, it goes down and the daemons will have to be brought up > again. > Instead, it should log the fact that it is unable to talk to ZK, call > becomeStandby on its client, and continue to attempt connecting to ZK. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204683#comment-16204683 ] Daniel Templeton commented on HADOOP-14876: --- Oh, forgot to mention that in creating the developer doc I found some areas in the compat spec that are inconsistent and/or underspecified. They're also corrected in this patch. I've also started on HADOOP-14875, and the same is true there as well. > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: HADOOP-14876.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204683#comment-16204683 ] Daniel Templeton edited comment on HADOOP-14876 at 10/14/17 3:41 PM: - Oh, forgot to mention that in creating the developer doc I found some areas in the compat spec that are inconsistent and/or underspecified. They're also corrected in this patch. I've also started on HADOOP-14875, and the same is true there. was (Author: templedf): Oh, forgot to mention that in creating the developer doc I found some areas in the compat spec that are inconsistent and/or underspecified. They're also corrected in this patch. I've also started on HADOOP-14875, and the same is true there as well. > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: HADOOP-14876.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14876: -- Status: Patch Available (was: Open) > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: HADOOP-14876.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14876) Create downstream developer docs from the compatibility guidelines
[ https://issues.apache.org/jira/browse/HADOOP-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14876: -- Attachment: HADOOP-14876.001.patch Here's a first pass at a doc. It's still a little rough, but even so I think this is a big step forward for our downstream consumers. CC [~rkanter], [~steve_l], [~kasha], [~jlowe], [~andrew.wang] > Create downstream developer docs from the compatibility guidelines > -- > > Key: HADOOP-14876 > URL: https://issues.apache.org/jira/browse/HADOOP-14876 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-beta1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > Attachments: HADOOP-14876.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13493) Compatibility Docs should clarify the policy for what takes precedence when a conflict is found
[
https://issues.apache.org/jira/browse/HADOOP-13493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated HADOOP-13493:
--
Attachment: HADOOP-13493.001.patch
See if this is enough.
> Compatibility Docs should clarify the policy for what takes precedence when a
> conflict is found
> ---
>
> Key: HADOOP-13493
> URL: https://issues.apache.org/jira/browse/HADOOP-13493
> Project: Hadoop Common
> Issue Type: Task
> Components: documentation
>Affects Versions: 2.7.2
>Reporter: Robert Kanter
>Assignee: Daniel Templeton
>Priority: Critical
> Attachments: HADOOP-13493.001.patch
>
>
> The Compatibility Docs
> (https://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/Compatibility.html#Java_API)
> list the policies for Private, Public, not annotated, etc Classes and
> members, but it doesn't say what happens when there's a conflict. We should
> try obviously try to avoid this situation, but it would be good to explicitly
> state what takes precedence.
> As an example, until YARN-3225 made it consistent, {{RefreshNodesRequest}}
> looked like this:
> {code:java}
> @Private
> @Stable
> public abstract class RefreshNodesRequest {
> @Public
> @Stable
> public static RefreshNodesRequest newInstance() {
> RefreshNodesRequest request =
> Records.newRecord(RefreshNodesRequest.class);
> return request;
> }
> }
> {code}
> Note that the class is marked {{\@Private}}, but the method is marked
> {{\@Public}}.
> In this example, I'd say that the class level should have priority.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13493) Compatibility Docs should clarify the policy for what takes precedence when a conflict is found
[
https://issues.apache.org/jira/browse/HADOOP-13493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated HADOOP-13493:
--
Status: Patch Available (was: Open)
> Compatibility Docs should clarify the policy for what takes precedence when a
> conflict is found
> ---
>
> Key: HADOOP-13493
> URL: https://issues.apache.org/jira/browse/HADOOP-13493
> Project: Hadoop Common
> Issue Type: Task
> Components: documentation
>Affects Versions: 2.7.2
>Reporter: Robert Kanter
>Assignee: Daniel Templeton
>Priority: Critical
> Attachments: HADOOP-13493.001.patch
>
>
> The Compatibility Docs
> (https://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/Compatibility.html#Java_API)
> list the policies for Private, Public, not annotated, etc Classes and
> members, but it doesn't say what happens when there's a conflict. We should
> try obviously try to avoid this situation, but it would be good to explicitly
> state what takes precedence.
> As an example, until YARN-3225 made it consistent, {{RefreshNodesRequest}}
> looked like this:
> {code:java}
> @Private
> @Stable
> public abstract class RefreshNodesRequest {
> @Public
> @Stable
> public static RefreshNodesRequest newInstance() {
> RefreshNodesRequest request =
> Records.newRecord(RefreshNodesRequest.class);
> return request;
> }
> }
> {code}
> Note that the class is marked {{\@Private}}, but the method is marked
> {{\@Public}}.
> In this example, I'd say that the class level should have priority.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14938) Configuration.updatingResource map should be initialized lazily
[
https://issues.apache.org/jira/browse/HADOOP-14938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201444#comment-16201444
]
Daniel Templeton commented on HADOOP-14938:
---
>From what I've read, you're almost correct on DCL. The one difference is that
>the JVM may do operator reordering, which can cause the value to be returned
>before the cache flushing is complete (or something like that) unless the
>value is volatile.
LGTM, but someone else should take a closer look. I keep doing these reviews
while on the run.
> Configuration.updatingResource map should be initialized lazily
> ---
>
> Key: HADOOP-14938
> URL: https://issues.apache.org/jira/browse/HADOOP-14938
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Misha Dmitriev
>Assignee: Misha Dmitriev
> Attachments: HADOOP-14938.01.patch, HADOOP-14938.02.patch
>
>
> Using jxray (www.jxray.com), I've analyzed a heap dump of YARN RM running in
> a big cluster. The tool uncovered several inefficiencies in the RM memory. It
> turns out that one of the biggest sources of memory waste, responsible for
> almost 1/4 of used memory, is empty ConcurrentHashMap instances in
> org.apache.hadoop.conf.Configuration.updatingResource:
> {code}
> 905,551K (24.0%): java.util.concurrent.ConcurrentHashMap: 22118 / 100% of
> empty 905,551K (24.0%)
> ↖org.apache.hadoop.conf.Configuration.updatingResource
> ↖{j.u.WeakHashMap}.keys
> ↖Java Static org.apache.hadoop.conf.Configuration.REGISTRY
> {code}
> That is, there are 22118 empty ConcurrentHashMaps here, and they collectively
> waste ~905MB of memory. This is caused by eager initialization of these maps.
> To address this problem, we should initialize them lazily.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14938) Configuration.updatingResource map should be initialized lazily
[
https://issues.apache.org/jira/browse/HADOOP-14938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199630#comment-16199630
]
Daniel Templeton commented on HADOOP-14938:
---
A couple of comments:
# It would be nice to have some braces around the inner _if_ in
{{putIntoUpdatingResource()}}
# For the DCL to work in {{putIntoUpdatingResource()}}, {{updatingResource}}
has to be _volatile_.
> Configuration.updatingResource map should be initialized lazily
> ---
>
> Key: HADOOP-14938
> URL: https://issues.apache.org/jira/browse/HADOOP-14938
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Misha Dmitriev
>Assignee: Misha Dmitriev
> Attachments: HADOOP-14938.01.patch
>
>
> Using jxray (www.jxray.com), I've analyzed a heap dump of YARN RM running in
> a big cluster. The tool uncovered several inefficiencies in the RM memory. It
> turns out that one of the biggest sources of memory waste, responsible for
> almost 1/4 of used memory, is empty ConcurrentHashMap instances in
> org.apache.hadoop.conf.Configuration.updatingResource:
> {code}
> 905,551K (24.0%): java.util.concurrent.ConcurrentHashMap: 22118 / 100% of
> empty 905,551K (24.0%)
> ↖org.apache.hadoop.conf.Configuration.updatingResource
> ↖{j.u.WeakHashMap}.keys
> ↖Java Static org.apache.hadoop.conf.Configuration.REGISTRY
> {code}
> That is, there are 22118 empty ConcurrentHashMaps here, and they collectively
> waste ~905MB of memory. This is caused by eager initialization of these maps.
> To address this problem, we should initialize them lazily.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14459) SerializationFactory shouldn't throw a NullPointerException if the serializations list is not defined
[ https://issues.apache.org/jira/browse/HADOOP-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16191250#comment-16191250 ] Daniel Templeton commented on HADOOP-14459: --- There is a test-patch profile for Maven, but it's a pain to use. It's usually easier to just post the patch and let Jenkins do the work. > SerializationFactory shouldn't throw a NullPointerException if the > serializations list is not defined > - > > Key: HADOOP-14459 > URL: https://issues.apache.org/jira/browse/HADOOP-14459 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0-alpha4 >Reporter: Nandor Kollar >Assignee: Nandor Kollar >Priority: Minor > Fix For: 3.0.0, 3.1.0 > > Attachments: HADOOP-14459_2.patch, HADOOP-14459_3.patch, > HADOOP-14459_4.patch, HADOOP-14459_5.patch, HADOOP-14459_6.patch, > HADOOP-14459.patch > > > The SerializationFactory throws an NPE if > CommonConfigurationKeys.IO_SERIALIZATIONS_KEY is not defined in the config. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14459) SerializationFactory shouldn't throw a NullPointerException if the serializations list is not defined
[ https://issues.apache.org/jira/browse/HADOOP-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14459: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.0 3.0.0 Status: Resolved (was: Patch Available) Thanks for the patches, [~nkollar]. Committed to trunk and branch-3.0. > SerializationFactory shouldn't throw a NullPointerException if the > serializations list is not defined > - > > Key: HADOOP-14459 > URL: https://issues.apache.org/jira/browse/HADOOP-14459 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0-alpha4 >Reporter: Nandor Kollar >Assignee: Nandor Kollar >Priority: Minor > Fix For: 3.0.0, 3.1.0 > > Attachments: HADOOP-14459_2.patch, HADOOP-14459_3.patch, > HADOOP-14459_4.patch, HADOOP-14459_5.patch, HADOOP-14459_6.patch, > HADOOP-14459.patch > > > The SerializationFactory throws an NPE if > CommonConfigurationKeys.IO_SERIALIZATIONS_KEY is not defined in the config. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14459) SerializationFactory shouldn't throw a NullPointerException if the serializations list is not defined
[ https://issues.apache.org/jira/browse/HADOOP-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14459: -- Affects Version/s: 3.0.0-alpha4 > SerializationFactory shouldn't throw a NullPointerException if the > serializations list is not defined > - > > Key: HADOOP-14459 > URL: https://issues.apache.org/jira/browse/HADOOP-14459 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0-alpha4 >Reporter: Nandor Kollar >Assignee: Nandor Kollar >Priority: Minor > Attachments: HADOOP-14459_2.patch, HADOOP-14459_3.patch, > HADOOP-14459_4.patch, HADOOP-14459_5.patch, HADOOP-14459_6.patch, > HADOOP-14459.patch > > > The SerializationFactory throws an NPE if > CommonConfigurationKeys.IO_SERIALIZATIONS_KEY is not defined in the config. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14459) SerializationFactory shouldn't throw a NullPointerException if the serializations list is not defined
[ https://issues.apache.org/jira/browse/HADOOP-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189876#comment-16189876 ] Daniel Templeton commented on HADOOP-14459: --- Yes, there is. It runs when you press the Submit Patch button. My bad. I should have noticed that there wasn't a Jenkins report. I pressed the button for you. Expect a report in a few hours. > SerializationFactory shouldn't throw a NullPointerException if the > serializations list is not defined > - > > Key: HADOOP-14459 > URL: https://issues.apache.org/jira/browse/HADOOP-14459 > Project: Hadoop Common > Issue Type: Bug >Reporter: Nandor Kollar >Assignee: Nandor Kollar >Priority: Minor > Attachments: HADOOP-14459_2.patch, HADOOP-14459_3.patch, > HADOOP-14459_4.patch, HADOOP-14459_5.patch, HADOOP-14459_6.patch, > HADOOP-14459.patch > > > The SerializationFactory throws an NPE if > CommonConfigurationKeys.IO_SERIALIZATIONS_KEY is not defined in the config. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14459) SerializationFactory shouldn't throw a NullPointerException if the serializations list is not defined
[ https://issues.apache.org/jira/browse/HADOOP-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14459: -- Status: Patch Available (was: Open) > SerializationFactory shouldn't throw a NullPointerException if the > serializations list is not defined > - > > Key: HADOOP-14459 > URL: https://issues.apache.org/jira/browse/HADOOP-14459 > Project: Hadoop Common > Issue Type: Bug >Reporter: Nandor Kollar >Assignee: Nandor Kollar >Priority: Minor > Attachments: HADOOP-14459_2.patch, HADOOP-14459_3.patch, > HADOOP-14459_4.patch, HADOOP-14459_5.patch, HADOOP-14459_6.patch, > HADOOP-14459.patch > > > The SerializationFactory throws an NPE if > CommonConfigurationKeys.IO_SERIALIZATIONS_KEY is not defined in the config. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14459) SerializationFactory shouldn't throw a NullPointerException if the serializations list is not defined
[
https://issues.apache.org/jira/browse/HADOOP-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189843#comment-16189843
]
Daniel Templeton commented on HADOOP-14459:
---
Oops, wait. In my last pass before committing, I noticed that there's now an
extra space after the colon in the for loop. Since I'm gonna make you post a
new patch for that, you should also change {{X}} to {code}{@code
X}{code} in the javadocs.
> SerializationFactory shouldn't throw a NullPointerException if the
> serializations list is not defined
> -
>
> Key: HADOOP-14459
> URL: https://issues.apache.org/jira/browse/HADOOP-14459
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Nandor Kollar
>Assignee: Nandor Kollar
>Priority: Minor
> Attachments: HADOOP-14459_2.patch, HADOOP-14459_3.patch,
> HADOOP-14459_4.patch, HADOOP-14459_5.patch, HADOOP-14459.patch
>
>
> The SerializationFactory throws an NPE if
> CommonConfigurationKeys.IO_SERIALIZATIONS_KEY is not defined in the config.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14459) SerializationFactory shouldn't throw a NullPointerException if the serializations list is not defined
[ https://issues.apache.org/jira/browse/HADOOP-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189723#comment-16189723 ] Daniel Templeton commented on HADOOP-14459: --- LGTM +1 > SerializationFactory shouldn't throw a NullPointerException if the > serializations list is not defined > - > > Key: HADOOP-14459 > URL: https://issues.apache.org/jira/browse/HADOOP-14459 > Project: Hadoop Common > Issue Type: Bug >Reporter: Nandor Kollar >Assignee: Nandor Kollar >Priority: Minor > Attachments: HADOOP-14459_2.patch, HADOOP-14459_3.patch, > HADOOP-14459_4.patch, HADOOP-14459_5.patch, HADOOP-14459.patch > > > The SerializationFactory throws an NPE if > CommonConfigurationKeys.IO_SERIALIZATIONS_KEY is not defined in the config. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14897) Loosen compatibility guidelines for native dependencies
[ https://issues.apache.org/jira/browse/HADOOP-14897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14897: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.0 3.0.0-beta1 Status: Resolved (was: Patch Available) Thanks to [~chris.douglas] for filing the issue and the review. Thanks also to [~miklos.szeg...@cloudera.com] and [~aw] for reviews. Committed to trunk and branch-3.0. > Loosen compatibility guidelines for native dependencies > --- > > Key: HADOOP-14897 > URL: https://issues.apache.org/jira/browse/HADOOP-14897 > Project: Hadoop Common > Issue Type: Bug > Components: documentation, native >Reporter: Chris Douglas >Assignee: Daniel Templeton >Priority: Blocker > Fix For: 3.0.0-beta1, 3.1.0 > > Attachments: HADOOP-14897.001.patch, HADOOP-14897.002.patch > > > Within a major version, the compatibility guidelines forbid raising the > minimum required version of any native dependency or tool required to build > native components. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14897) Loosen compatibility guidelines for native dependencies
[ https://issues.apache.org/jira/browse/HADOOP-14897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177620#comment-16177620 ] Daniel Templeton commented on HADOOP-14897: --- Anyone else? CC [~steve_l], [~aw]. If not, I'll commit tonight (CET) or tomorrow. > Loosen compatibility guidelines for native dependencies > --- > > Key: HADOOP-14897 > URL: https://issues.apache.org/jira/browse/HADOOP-14897 > Project: Hadoop Common > Issue Type: Bug > Components: documentation, native >Reporter: Chris Douglas >Assignee: Daniel Templeton >Priority: Blocker > Attachments: HADOOP-14897.001.patch, HADOOP-14897.002.patch > > > Within a major version, the compatibility guidelines forbid raising the > minimum required version of any native dependency or tool required to build > native components. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14897) Loosen compatibility guidelines for native dependencies
[ https://issues.apache.org/jira/browse/HADOOP-14897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14897: -- Attachment: HADOOP-14897.002.patch Fixed the link. [~miklos.szeg...@cloudera.com], any comments? > Loosen compatibility guidelines for native dependencies > --- > > Key: HADOOP-14897 > URL: https://issues.apache.org/jira/browse/HADOOP-14897 > Project: Hadoop Common > Issue Type: Bug > Components: documentation, native >Reporter: Chris Douglas >Assignee: Daniel Templeton >Priority: Blocker > Attachments: HADOOP-14897.001.patch, HADOOP-14897.002.patch > > > Within a major version, the compatibility guidelines forbid raising the > minimum required version of any native dependency or tool required to build > native components. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14897) Loosen compatibility guidelines for native dependencies
[ https://issues.apache.org/jira/browse/HADOOP-14897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14897: -- Status: Patch Available (was: Open) > Loosen compatibility guidelines for native dependencies > --- > > Key: HADOOP-14897 > URL: https://issues.apache.org/jira/browse/HADOOP-14897 > Project: Hadoop Common > Issue Type: Bug > Components: documentation, native >Reporter: Chris Douglas >Assignee: Daniel Templeton >Priority: Blocker > Attachments: HADOOP-14897.001.patch, HADOOP-14897.002.patch > > > Within a major version, the compatibility guidelines forbid raising the > minimum required version of any native dependency or tool required to build > native components. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13714) Tighten up our compatibility guidelines for Hadoop 3
[ https://issues.apache.org/jira/browse/HADOOP-13714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-13714: -- Fix Version/s: 3.1.0 > Tighten up our compatibility guidelines for Hadoop 3 > > > Key: HADOOP-13714 > URL: https://issues.apache.org/jira/browse/HADOOP-13714 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.3 >Reporter: Karthik Kambatla >Assignee: Daniel Templeton >Priority: Blocker > Fix For: 3.0.0-beta1, 3.1.0 > > Attachments: Compatibility.pdf, HADOOP-13714.001.patch, > HADOOP-13714.002.patch, HADOOP-13714.003.patch, HADOOP-13714.004.patch, > HADOOP-13714.005.patch, HADOOP-13714.006.patch, HADOOP-13714.007.patch, > HADOOP-13714.008.patch, HADOOP-13714.WIP-001.patch, > InterfaceClassification.pdf > > > Our current compatibility guidelines are incomplete and loose. For many > categories, we do not have a policy. It would be nice to actually define > those policies so our users know what to expect and the developers know what > releases to target their changes. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14897) Loosen compatibility guidelines for native dependencies
[ https://issues.apache.org/jira/browse/HADOOP-14897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-14897: -- Attachment: HADOOP-14897.001.patch > Loosen compatibility guidelines for native dependencies > --- > > Key: HADOOP-14897 > URL: https://issues.apache.org/jira/browse/HADOOP-14897 > Project: Hadoop Common > Issue Type: Bug > Components: documentation, native >Reporter: Chris Douglas >Assignee: Daniel Templeton >Priority: Blocker > Attachments: HADOOP-14897.001.patch > > > Within a major version, the compatibility guidelines forbid raising the > minimum required version of any native dependency or tool required to build > native components. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
