[jira] [Updated] (HDFS-15111) stopStandbyServices() should log which service state it is transitioning from.
[
https://issues.apache.org/jira/browse/HDFS-15111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xieming Li updated HDFS-15111:
--
Description:
Trying to transition Observer to Standby state. {{stopStandbyServices()}} logs
that it is stopping/starting Standby services.
# {{startStandbyServices()}} should log which state it is transitioning TO.
# {{stopStandbyServices()}} should log which state it is transitioning FROM.
was:
Trying to transition Observer to Standby state. Both {{stopStandbyServices()}}
and {{startStandbyServices()}} log that they are stopping/starting Standby
services.
# {{startStandbyServices()}} should log which state it is transitioning TO.
# {{stopStandbyServices()}} should log which state it is transitioning FROM.
> stopStandbyServices() should log which service state it is transitioning from.
> --
>
> Key: HDFS-15111
> URL: https://issues.apache.org/jira/browse/HDFS-15111
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, logging
>Affects Versions: 2.10.0
>Reporter: Konstantin Shvachko
>Assignee: Xieming Li
>Priority: Major
> Labels: newbie++
> Attachments: HDFS-15111.001.patch, HDFS-15111.002.patch
>
>
> Trying to transition Observer to Standby state. {{stopStandbyServices()}}
> logs that it is stopping/starting Standby services.
> # {{startStandbyServices()}} should log which state it is transitioning TO.
> # {{stopStandbyServices()}} should log which state it is transitioning FROM.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15111) stopStandbyServices() should log which service state it is transitioning from.
[
https://issues.apache.org/jira/browse/HDFS-15111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xieming Li updated HDFS-15111:
--
Summary: stopStandbyServices() should log which service state it is
transitioning from. (was: start / stopStandbyServices() should log which
service it is transitioning to/from.)
> stopStandbyServices() should log which service state it is transitioning from.
> --
>
> Key: HDFS-15111
> URL: https://issues.apache.org/jira/browse/HDFS-15111
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, logging
>Affects Versions: 2.10.0
>Reporter: Konstantin Shvachko
>Assignee: Xieming Li
>Priority: Major
> Labels: newbie++
> Attachments: HDFS-15111.001.patch, HDFS-15111.002.patch
>
>
> Trying to transition Observer to Standby state. Both
> {{stopStandbyServices()}} and {{startStandbyServices()}} log that they are
> stopping/starting Standby services.
> # {{startStandbyServices()}} should log which state it is transitioning TO.
> # {{stopStandbyServices()}} should log which state it is transitioning FROM.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-15152) Create light weight INode to store in INodeMap
hemanthboyina created HDFS-15152: Summary: Create light weight INode to store in INodeMap Key: HDFS-15152 URL: https://issues.apache.org/jira/browse/HDFS-15152 Project: Hadoop HDFS Issue Type: Sub-task Reporter: hemanthboyina Assignee: hemanthboyina create a light weight INode to store in INodeMap -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15111) start / stopStandbyServices() should log which service it is transitioning to/from.
[
https://issues.apache.org/jira/browse/HDFS-15111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029569#comment-17029569
]
Ayush Saxena commented on HDFS-15111:
-
I think the logging should be either standby or observer only, There are other
states like initializing or Stopping, in that case IMO we should log standby
only, It won't make sense stopping services started for Initializing state. So
we should ensure log only standby or observer
> start / stopStandbyServices() should log which service it is transitioning
> to/from.
> ---
>
> Key: HDFS-15111
> URL: https://issues.apache.org/jira/browse/HDFS-15111
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, logging
>Affects Versions: 2.10.0
>Reporter: Konstantin Shvachko
>Assignee: Xieming Li
>Priority: Major
> Labels: newbie++
> Attachments: HDFS-15111.001.patch, HDFS-15111.002.patch
>
>
> Trying to transition Observer to Standby state. Both
> {{stopStandbyServices()}} and {{startStandbyServices()}} log that they are
> stopping/starting Standby services.
> # {{startStandbyServices()}} should log which state it is transitioning TO.
> # {{stopStandbyServices()}} should log which state it is transitioning FROM.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15111) start / stopStandbyServices() should log which service it is transitioning to/from.
[
https://issues.apache.org/jira/browse/HDFS-15111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029482#comment-17029482
]
Konstantin Shvachko commented on HDFS-15111:
Thanks for the patch [~risyomei].
* It is better to use placeholders in LOG statements, like
{code:java}
LOG.info("Stopping services started for {} state", curState);{code}
* For {{curState}} you can make it of type {{HAServiceState}}, and use
{{FSNamesystem.getState()}}, which already takes care of the context null value.
> start / stopStandbyServices() should log which service it is transitioning
> to/from.
> ---
>
> Key: HDFS-15111
> URL: https://issues.apache.org/jira/browse/HDFS-15111
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, logging
>Affects Versions: 2.10.0
>Reporter: Konstantin Shvachko
>Assignee: Xieming Li
>Priority: Major
> Labels: newbie++
> Attachments: HDFS-15111.001.patch, HDFS-15111.002.patch
>
>
> Trying to transition Observer to Standby state. Both
> {{stopStandbyServices()}} and {{startStandbyServices()}} log that they are
> stopping/starting Standby services.
> # {{startStandbyServices()}} should log which state it is transitioning TO.
> # {{stopStandbyServices()}} should log which state it is transitioning FROM.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15148) dfs.namenode.send.qop.enabled should not apply to primary NN port
[
https://issues.apache.org/jira/browse/HDFS-15148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029436#comment-17029436
]
Konstantin Shvachko commented on HDFS-15148:
+1 on v04 patch.
I don't see a jira targeting fixing
{{TestDelegationTokensWithHA.testObserverReadProxyProviderWithDT}} on trunk
There is one HDFS-14893 for branch-2. Worth checking if we got the same problem
on trunk. Otherwise please create a new one for the test.
> dfs.namenode.send.qop.enabled should not apply to primary NN port
> -
>
> Key: HDFS-15148
> URL: https://issues.apache.org/jira/browse/HDFS-15148
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.10.1, 3.3.1
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-15148.001.patch, HDFS-15148.002.patch,
> HDFS-15148.003.patch, HDFS-15148.004.patch
>
>
> In HDFS-13617, NameNode can be configured to wrap its established QOP into
> block access token as an encrypted message. Later on DataNode will use this
> message to create SASL connection. But this new behavior should only apply to
> new auxiliary NameNode ports, not the primary port (the one configured in
> fs.defaultFS), as it may cause conflicting behavior with existing other SASL
> related configuration (e.g. dfs.data.transfer.protection). Since this
> configure is introduced for to auxiliary ports only, we should restrict this
> new behavior to not apply to primary port.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14743) Enhance INodeAttributeProvider/ AccessControlEnforcer Interface in HDFS to support Authorization of mkdir, rm, rmdir, copy, move etc...
[ https://issues.apache.org/jira/browse/HDFS-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029332#comment-17029332 ] Arpit Agarwal commented on HDFS-14743: -- One potential downside of thread-locals is if we forget to save a new operation, then stale state can be passed to the authorizer plugin. This is impossible with parameter passing. > Enhance INodeAttributeProvider/ AccessControlEnforcer Interface in HDFS to > support Authorization of mkdir, rm, rmdir, copy, move etc... > --- > > Key: HDFS-14743 > URL: https://issues.apache.org/jira/browse/HDFS-14743 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Affects Versions: 3.1.0 >Reporter: Ramesh Mani >Assignee: Wei-Chiu Chuang >Priority: Critical > Attachments: HDFS-14743 Enhance INodeAttributeProvider_ > AccessControlEnforcer Interface.pdf > > > Enhance INodeAttributeProvider / AccessControlEnforcer Interface in HDFS to > support Authorization of mkdir, rm, rmdir, copy, move etc..., this should > help the implementors of the interface like Apache Ranger's HDFS > Authorization plugin to authorize and audit those command sets. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14743) Enhance INodeAttributeProvider/ AccessControlEnforcer Interface in HDFS to support Authorization of mkdir, rm, rmdir, copy, move etc...
[ https://issues.apache.org/jira/browse/HDFS-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029329#comment-17029329 ] Arpit Agarwal commented on HDFS-14743: -- Thanks for sharing the PoC [~weichiu] . I understand why you chose thread-locals to store the operation info, it avoids significant refactoring of the code. I don't have a strong opinion on which is better, thread-locals vs passing parameters all the way through. [~xyao] [~aengineer] do you have an opinion? > Enhance INodeAttributeProvider/ AccessControlEnforcer Interface in HDFS to > support Authorization of mkdir, rm, rmdir, copy, move etc... > --- > > Key: HDFS-14743 > URL: https://issues.apache.org/jira/browse/HDFS-14743 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Affects Versions: 3.1.0 >Reporter: Ramesh Mani >Assignee: Wei-Chiu Chuang >Priority: Critical > Attachments: HDFS-14743 Enhance INodeAttributeProvider_ > AccessControlEnforcer Interface.pdf > > > Enhance INodeAttributeProvider / AccessControlEnforcer Interface in HDFS to > support Authorization of mkdir, rm, rmdir, copy, move etc..., this should > help the implementors of the interface like Apache Ranger's HDFS > Authorization plugin to authorize and audit those command sets. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work started] (HDFS-14743) Enhance INodeAttributeProvider/ AccessControlEnforcer Interface in HDFS to support Authorization of mkdir, rm, rmdir, copy, move etc...
[ https://issues.apache.org/jira/browse/HDFS-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-14743 started by Wei-Chiu Chuang. -- > Enhance INodeAttributeProvider/ AccessControlEnforcer Interface in HDFS to > support Authorization of mkdir, rm, rmdir, copy, move etc... > --- > > Key: HDFS-14743 > URL: https://issues.apache.org/jira/browse/HDFS-14743 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Affects Versions: 3.1.0 >Reporter: Ramesh Mani >Assignee: Wei-Chiu Chuang >Priority: Critical > Attachments: HDFS-14743 Enhance INodeAttributeProvider_ > AccessControlEnforcer Interface.pdf > > > Enhance INodeAttributeProvider / AccessControlEnforcer Interface in HDFS to > support Authorization of mkdir, rm, rmdir, copy, move etc..., this should > help the implementors of the interface like Apache Ranger's HDFS > Authorization plugin to authorize and audit those command sets. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9786) HttpFS doesn't write the proxyuser information in logfile
[
https://issues.apache.org/jira/browse/HDFS-9786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029270#comment-17029270
]
Ahmed Hussein commented on HDFS-9786:
-
[~ayushtkn] and [~weichiu], the description of the jira is a little bit
confusing. Do you know what is "HttpFS GW"? and what is the wrong values being
reported here?
As far as I can see The values in the MDC do not get picked by the logger. On
the other hand, FSNamesystem construct the audit message manually
"{{FSNamesystem.logAuditEvent()}}".
> HttpFS doesn't write the proxyuser information in logfile
> -
>
> Key: HDFS-9786
> URL: https://issues.apache.org/jira/browse/HDFS-9786
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Heesoo Kim
>Assignee: Ahmed Hussein
>Priority: Major
>
> According to the httpfs-log4j.properties, the log pattern indicates that
> {code}
> log4j.appender.httpfsaudit.layout.ConversionPattern=%d{ISO8601} %5p
> [%X{hostname}][%X{user}:%X{doAs}] %X{op} %m%n
> {code}
> However, the httpfsaudit doesn't write right information for user and
> proxyuser information. It is better to write ugi on audit log in HttpFS GW.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15148) dfs.namenode.send.qop.enabled should not apply to primary NN port
[
https://issues.apache.org/jira/browse/HDFS-15148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029185#comment-17029185
]
Chen Liang commented on HDFS-15148:
---
{{testObserverReadProxyProviderWithDT}} fail is unrelated and fails even
without this patch. We should look into fixing this test fail, but that should
be in another jira. [~shv] mind taking a look at v004 patch?
> dfs.namenode.send.qop.enabled should not apply to primary NN port
> -
>
> Key: HDFS-15148
> URL: https://issues.apache.org/jira/browse/HDFS-15148
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.10.1, 3.3.1
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-15148.001.patch, HDFS-15148.002.patch,
> HDFS-15148.003.patch, HDFS-15148.004.patch
>
>
> In HDFS-13617, NameNode can be configured to wrap its established QOP into
> block access token as an encrypted message. Later on DataNode will use this
> message to create SASL connection. But this new behavior should only apply to
> new auxiliary NameNode ports, not the primary port (the one configured in
> fs.defaultFS), as it may cause conflicting behavior with existing other SASL
> related configuration (e.g. dfs.data.transfer.protection). Since this
> configure is introduced for to auxiliary ports only, we should restrict this
> new behavior to not apply to primary port.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15147) LazyPersistTestCase wait logic is error pruned
[
https://issues.apache.org/jira/browse/HDFS-15147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029155#comment-17029155
]
Íñigo Goiri commented on HDFS-15147:
I was also doing the same comparison.
I was focusing on the main offenders (>1 minute).
What I saw is that the timing was pretty random; most cases were slower than 10
minutes but there were faster ones too.
Anyway, I think it is reasonable to move this forward.
> LazyPersistTestCase wait logic is error pruned
> --
>
> Key: HDFS-15147
> URL: https://issues.apache.org/jira/browse/HDFS-15147
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Minor
> Attachments: HDFS-15147-branch-2.10.001.patch, HDFS-15147.001.patch,
> HDFS-15147.002.patch, HDFS-15147.003.patch
>
>
> {{LazyPersistTestCase}} has some issues hat lead to inconsistent result of
> the test cases:
> * the wait periods to change of status is too long. It reaches 10 secs in
> some cases.
> * triggerBlockReport() only triggers FBR of DN with index 0. This is counter
> intuitive because the JUnit tests restart the DN assuming that the restarted
> DN will send a FBR. However, this never happens because the DN will get a new
> index post restart.
> {code:java}
> protected final void triggerBlockReport()
> throws IOException, InterruptedException {
> // Trigger block report to NN
> DataNodeTestUtils.triggerBlockReport(cluster.getDataNodes().get(0));
> Thread.sleep(10 * 1000);
> }
> {code}
> [~inigoiri] suggested that we propagate the findings and fixes from
> HDFS-13179 and HDFS-15144 into {{LazyPersistTestCase.java}}. This will
> eventually reduce the runtime and make the test cases more stable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15147) LazyPersistTestCase wait logic is error pruned
[
https://issues.apache.org/jira/browse/HDFS-15147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029072#comment-17029072
]
Ahmed Hussein commented on HDFS-15147:
--
[~elgoiri], Thanks for the review.
I have a qq, I compared the runtime of each individual JUnit to evaluate the
effect of the code changes. How do you evaluate the execution time? I see that
the total 9 minutes and 16 seconds on the link. However, when I compare that to
the history of the [PreCommit-HDFS-Build
list|https://builds.apache.org/job/PreCommit-HDFS-Build/28732/testReport/org.apache.hadoop.hdfs.server.datanode.fsdataset.impl/history/],
the set of test cases look different in each entry. I wonder if this is the
only indication to compare runtimes?
> LazyPersistTestCase wait logic is error pruned
> --
>
> Key: HDFS-15147
> URL: https://issues.apache.org/jira/browse/HDFS-15147
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Minor
> Attachments: HDFS-15147-branch-2.10.001.patch, HDFS-15147.001.patch,
> HDFS-15147.002.patch, HDFS-15147.003.patch
>
>
> {{LazyPersistTestCase}} has some issues hat lead to inconsistent result of
> the test cases:
> * the wait periods to change of status is too long. It reaches 10 secs in
> some cases.
> * triggerBlockReport() only triggers FBR of DN with index 0. This is counter
> intuitive because the JUnit tests restart the DN assuming that the restarted
> DN will send a FBR. However, this never happens because the DN will get a new
> index post restart.
> {code:java}
> protected final void triggerBlockReport()
> throws IOException, InterruptedException {
> // Trigger block report to NN
> DataNodeTestUtils.triggerBlockReport(cluster.getDataNodes().get(0));
> Thread.sleep(10 * 1000);
> }
> {code}
> [~inigoiri] suggested that we propagate the findings and fixes from
> HDFS-13179 and HDFS-15144 into {{LazyPersistTestCase.java}}. This will
> eventually reduce the runtime and make the test cases more stable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14651) DeadNodeDetector checks dead node periodically
[ https://issues.apache.org/jira/browse/HDFS-14651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029061#comment-17029061 ] Ahmed Hussein commented on HDFS-14651: -- Thanks [~leosun08], Feel free to take over HDFS-15149. > DeadNodeDetector checks dead node periodically > -- > > Key: HDFS-14651 > URL: https://issues.apache.org/jira/browse/HDFS-14651 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Lisheng Sun >Assignee: Lisheng Sun >Priority: Major > Fix For: 3.3.0 > > Attachments: HDFS-14651.001.patch, HDFS-14651.002.patch, > HDFS-14651.003.patch, HDFS-14651.004.patch, HDFS-14651.005.patch, > HDFS-14651.006.patch, HDFS-14651.007.patch, HDFS-14651.008.patch > > > DeadNodeDetector checks dead node periodically. > DeadNodeDetector periodically detect the Node in DeadNodeDetector#deadnode, > If the access is successful, the Node will be moved from > DeadNodeDetector#deadnode. Continuous detection of the dead node is > necessary. The DataNode need rejoin the cluster due to a service > restart/machine repair. The DataNode may be permanently excluded if there is > no added probe mechanism. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15111) start / stopStandbyServices() should log which service it is transitioning to/from.
[
https://issues.apache.org/jira/browse/HDFS-15111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028929#comment-17028929
]
Hadoop QA commented on HDFS-15111:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
46s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 32s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 39s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}106m 23s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}168m 52s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport |
| | hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage |
| | hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | HDFS-15111 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12992478/HDFS-15111.002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 8fdc3254da1e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 1e3a0b0 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_232 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/28735/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/28735/testReport/ |
| Max. process+thread
[jira] [Commented] (HDFS-15111) start / stopStandbyServices() should log which service it is transitioning to/from.
[
https://issues.apache.org/jira/browse/HDFS-15111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028904#comment-17028904
]
Ayush Saxena commented on HDFS-15111:
-
Thanx [~risyomei] for the update.
Seems we are changing only for {{stopStandbyServices}} only, start already logs
the state. So we can update the description accordingly. For the log message
use sl4j semantics.
> start / stopStandbyServices() should log which service it is transitioning
> to/from.
> ---
>
> Key: HDFS-15111
> URL: https://issues.apache.org/jira/browse/HDFS-15111
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, logging
>Affects Versions: 2.10.0
>Reporter: Konstantin Shvachko
>Assignee: Xieming Li
>Priority: Major
> Labels: newbie++
> Attachments: HDFS-15111.001.patch, HDFS-15111.002.patch
>
>
> Trying to transition Observer to Standby state. Both
> {{stopStandbyServices()}} and {{startStandbyServices()}} log that they are
> stopping/starting Standby services.
> # {{startStandbyServices()}} should log which state it is transitioning TO.
> # {{stopStandbyServices()}} should log which state it is transitioning FROM.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15111) start / stopStandbyServices() should log which service it is transitioning to/from.
[
https://issues.apache.org/jira/browse/HDFS-15111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xieming Li updated HDFS-15111:
--
Status: Open (was: Patch Available)
Sorry, I overlooked the state when NN is starting up.
Re-uploaded an another patch.
> start / stopStandbyServices() should log which service it is transitioning
> to/from.
> ---
>
> Key: HDFS-15111
> URL: https://issues.apache.org/jira/browse/HDFS-15111
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, logging
>Affects Versions: 2.10.0
>Reporter: Konstantin Shvachko
>Assignee: Xieming Li
>Priority: Major
> Labels: newbie++
> Attachments: HDFS-15111.001.patch, HDFS-15111.002.patch
>
>
> Trying to transition Observer to Standby state. Both
> {{stopStandbyServices()}} and {{startStandbyServices()}} log that they are
> stopping/starting Standby services.
> # {{startStandbyServices()}} should log which state it is transitioning TO.
> # {{stopStandbyServices()}} should log which state it is transitioning FROM.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15111) start / stopStandbyServices() should log which service it is transitioning to/from.
[
https://issues.apache.org/jira/browse/HDFS-15111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xieming Li updated HDFS-15111:
--
Attachment: HDFS-15111.002.patch
Status: Patch Available (was: Open)
> start / stopStandbyServices() should log which service it is transitioning
> to/from.
> ---
>
> Key: HDFS-15111
> URL: https://issues.apache.org/jira/browse/HDFS-15111
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, logging
>Affects Versions: 2.10.0
>Reporter: Konstantin Shvachko
>Assignee: Xieming Li
>Priority: Major
> Labels: newbie++
> Attachments: HDFS-15111.001.patch, HDFS-15111.002.patch
>
>
> Trying to transition Observer to Standby state. Both
> {{stopStandbyServices()}} and {{startStandbyServices()}} log that they are
> stopping/starting Standby services.
> # {{startStandbyServices()}} should log which state it is transitioning TO.
> # {{stopStandbyServices()}} should log which state it is transitioning FROM.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15111) start / stopStandbyServices() should log which service it is transitioning to/from.
[
https://issues.apache.org/jira/browse/HDFS-15111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028824#comment-17028824
]
Ayush Saxena commented on HDFS-15111:
-
Thanx [~risyomei] for the patch. Seems the test failures are related. The state
seems to be null
> start / stopStandbyServices() should log which service it is transitioning
> to/from.
> ---
>
> Key: HDFS-15111
> URL: https://issues.apache.org/jira/browse/HDFS-15111
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, logging
>Affects Versions: 2.10.0
>Reporter: Konstantin Shvachko
>Assignee: Xieming Li
>Priority: Major
> Labels: newbie++
> Attachments: HDFS-15111.001.patch
>
>
> Trying to transition Observer to Standby state. Both
> {{stopStandbyServices()}} and {{startStandbyServices()}} log that they are
> stopping/starting Standby services.
> # {{startStandbyServices()}} should log which state it is transitioning TO.
> # {{stopStandbyServices()}} should log which state it is transitioning FROM.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15136) In secure mode when Cookies are not set in request header leads to exception flood in DEBUG log
[ https://issues.apache.org/jira/browse/HDFS-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028802#comment-17028802 ] Ayush Saxena commented on HDFS-15136: - Thanx [~prasad-acit] for the report and fix. Instead of entire trace, you can just change to have just the message, that would suppress the logging. The changes in the UT seems unrelated, doesn't verify changes done here, We can probably remove them. For changing the log statement use the sl4j semantics > In secure mode when Cookies are not set in request header leads to exception > flood in DEBUG log > --- > > Key: HDFS-15136 > URL: https://issues.apache.org/jira/browse/HDFS-15136 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Renukaprasad C >Assignee: Renukaprasad C >Priority: Major > Attachments: HDFS-15136.0001.patch, HDFS-15136.0002.patch > > > In debug mode below exception gets logged when Cookie is not set in the > request header. This exception stack gets repeated and and has no meaning > here. > Instead, log the error in debug mode and continue without throw/catch/log of > exception. > 2020-01-20 18:25:57,792 DEBUG > org.apache.hadoop.security.UserGroupInformation: PrivilegedAction > as:test/t...@hadoop.com (auth:KERBEROS) > from:org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:518) > 2020-01-20 18:25:57,792 DEBUG > org.apache.hadoop.hdfs.web.URLConnectionFactory: open AuthenticatedURL > connection > https://IP:PORT/getJournal?jid=hacluster=295=-64%3A39449123%3A1579244618105%3Amyhacluster=true > 2020-01-20 18:25:57,803 DEBUG > org.apache.hadoop.security.authentication.client.KerberosAuthenticator: JDK > performed authentication on our behalf. > 2020-01-20 18:25:57,803 DEBUG > org.apache.hadoop.security.authentication.client.AuthenticatedURL: Cannot > parse cookie header: > java.lang.IllegalArgumentException: Empty cookie header string > at java.net.HttpCookie.parseInternal(HttpCookie.java:826) > at java.net.HttpCookie.parse(HttpCookie.java:202) > at java.net.HttpCookie.parse(HttpCookie.java:178) > at > org.apache.hadoop.security.authentication.client.AuthenticatedURL$AuthCookieHandler.put(AuthenticatedURL.java:99) > at > org.apache.hadoop.security.authentication.client.AuthenticatedURL.extractToken(AuthenticatedURL.java:390) > at > org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:197) > at > org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:348) > at > org.apache.hadoop.hdfs.web.URLConnectionFactory.openConnection(URLConnectionFactory.java:186) > at > org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:470) > at > org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:464) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at > org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:518) > at > org.apache.hadoop.security.SecurityUtil.doAsCurrentUser(SecurityUtil.java:512) > at > org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog.getInputStream(EditLogFileInputStream.java:463) > at > org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.init(EditLogFileInputStream.java:157) > at > org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOpImpl(EditLogFileInputStream.java:208) > at > org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOp(EditLogFileInputStream.java:266) > at > org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) > at > org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151) > at > org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:198) > at > org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) > at > org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151) > at > org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:198) > at > org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) > at >
