[jira] [Updated] (YARN-10884) EntityGroupFSTimelineStore fails to parse log files which has empty owner
[
https://issues.apache.org/jira/browse/YARN-10884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-10884:
-
Labels: (was: pull-request-available)
> EntityGroupFSTimelineStore fails to parse log files which has empty owner
> -
>
> Key: YARN-10884
> URL: https://issues.apache.org/jira/browse/YARN-10884
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: timelineserver
>Affects Versions: 3.3.1
>Reporter: Prabhu Joseph
>Assignee: SwathiChandrashekar
>Priority: Major
> Fix For: 3.3.1
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Due to [HADOOP-17848|https://issues.apache.org/jira/browse/HADOOP-17848] -
> Wasb FileSystem sets owner as empty during append operation.
> ATS1.5 fails to read such files with below error
> {code:java}
> java.lang.IllegalArgumentException: Null user
> at
> org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1271)
> at
> org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1258)
> at
> org.apache.hadoop.yarn.server.timeline.LogInfo.parsePath(LogInfo.java:141)
> at
> org.apache.hadoop.yarn.server.timeline.LogInfo.parseForStore(LogInfo.java:114)
> at
> org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:701)
> at
> org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:675)
> at
> org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$ActiveLogParser.run(EntityGroupFSTimelineStore.java:888)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
> It gets ownership of the file to check ACL. In case of disabled ACL check,
> this is not required. Will suggest to add anonymous user in case of empty
> user.
> {code}
> if (owner.isEmpty()) {
> user = "anonymous";
> } else {
> user = owner;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10884) EntityGroupFSTimelineStore fails to parse log files which has empty owner
[
https://issues.apache.org/jira/browse/YARN-10884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-10884:
-
Description:
Due to [HADOOP-17848|https://issues.apache.org/jira/browse/HADOOP-17848] - Wasb
FileSystem sets owner as empty during append operation.
ATS1.5 fails to read such files with below error
{code:java}
java.lang.IllegalArgumentException: Null user
at
org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1271)
at
org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1258)
at
org.apache.hadoop.yarn.server.timeline.LogInfo.parsePath(LogInfo.java:141)
at
org.apache.hadoop.yarn.server.timeline.LogInfo.parseForStore(LogInfo.java:114)
at
org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:701)
at
org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:675)
at
org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$ActiveLogParser.run(EntityGroupFSTimelineStore.java:888)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){code}
It gets ownership of the file to check ACL. In case of disabled ACL check, this
is not required. Will suggest to add anonymous user in case of empty user.
{code}
if (owner.isEmpty()) {
user = "anonymous";
} else {
user = owner;
}
{code}
was:
Due to [HADOOP-17848|https://issues.apache.org/jira/browse/HADOOP-17848] Hadoop
NativeAzureFileSystem append removes ownership set on the file - ASF JIRA
(apache.org)] - Wasb FileSystem sets owner as empty during append operation.
ATS1.5 fails to read such files with below error
{code:java}
java.lang.IllegalArgumentException: Null user
at
org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1271)
at
org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1258)
at
org.apache.hadoop.yarn.server.timeline.LogInfo.parsePath(LogInfo.java:141)
at
org.apache.hadoop.yarn.server.timeline.LogInfo.parseForStore(LogInfo.java:114)
at
org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:701)
at
org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:675)
at
org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$ActiveLogParser.run(EntityGroupFSTimelineStore.java:888)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){code}
It gets ownership of the file to check ACL. In case of disabled ACL check, this
is not required. Will suggest to add anonymous user in case of empty user.
{code}
if (owner.isEmpty()) {
user = "anonymous";
} else {
user = owner;
}
{code}
> EntityGroupFSTimelineStore fails to parse log files which has empty owner
> -
>
> Key: YARN-10884
> URL: https://issues.apache.org/jira/browse/YARN-10884
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: timelineserver
>Affects Versions: 3.3.1
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> Due to [HADOOP-17848|https://issues.apache.org/jira/browse/HADOOP-17848] -
> Wasb FileSystem sets owner as empty during append operation.
> ATS1.5 fails to read such files with below error
> {code:java}
>
[jira] [Updated] (YARN-10884) EntityGroupFSTimelineStore fails to parse log files which has empty owner
[
https://issues.apache.org/jira/browse/YARN-10884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-10884:
-
Description:
Due to [HADOOP-17848|https://issues.apache.org/jira/browse/HADOOP-17848] Hadoop
NativeAzureFileSystem append removes ownership set on the file - ASF JIRA
(apache.org)] - Wasb FileSystem sets owner as empty during append operation.
ATS1.5 fails to read such files with below error
{code:java}
java.lang.IllegalArgumentException: Null user
at
org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1271)
at
org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1258)
at
org.apache.hadoop.yarn.server.timeline.LogInfo.parsePath(LogInfo.java:141)
at
org.apache.hadoop.yarn.server.timeline.LogInfo.parseForStore(LogInfo.java:114)
at
org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:701)
at
org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:675)
at
org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$ActiveLogParser.run(EntityGroupFSTimelineStore.java:888)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){code}
It gets ownership of the file to check ACL. In case of disabled ACL check, this
is not required. Will suggest to add anonymous user in case of empty user.
{code}
if (owner.isEmpty()) {
user = "anonymous";
} else {
user = owner;
}
{code}
was:
Due to [HADOOP-17848|[HADOOP-17848] Hadoop NativeAzureFileSystem append removes
ownership set on the file - ASF JIRA (apache.org)] - Wasb FileSystem sets owner
as empty during append operation.
ATS1.5 fails to read such files with below error
{code:java}
java.lang.IllegalArgumentException: Null user
at
org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1271)
at
org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1258)
at
org.apache.hadoop.yarn.server.timeline.LogInfo.parsePath(LogInfo.java:141)
at
org.apache.hadoop.yarn.server.timeline.LogInfo.parseForStore(LogInfo.java:114)
at
org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:701)
at
org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:675)
at
org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$ActiveLogParser.run(EntityGroupFSTimelineStore.java:888)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){code}
It gets ownership of the file to check ACL. In case of disabled ACL check, this
is not required. Will suggest to add anonymous user in case of empty user.
{code}
if (owner.isEmpty()) {
user = "anonymous";
} else {
user = owner;
}
{code}
> EntityGroupFSTimelineStore fails to parse log files which has empty owner
> -
>
> Key: YARN-10884
> URL: https://issues.apache.org/jira/browse/YARN-10884
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: timelineserver
>Affects Versions: 3.3.1
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> Due to [HADOOP-17848|https://issues.apache.org/jira/browse/HADOOP-17848]
> Hadoop NativeAzureFileSystem append removes ownership set on the file - ASF
> JIRA (apache.org)] -
