[jira] [Updated] (YARN-9875) FSSchedulerConfigurationStore fails to update with hdfs path
[
https://issues.apache.org/jira/browse/YARN-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-9875:
Fix Version/s: 3.2.3
Backported to branch-3.2.
> FSSchedulerConfigurationStore fails to update with hdfs path
>
>
> Key: YARN-9875
> URL: https://issues.apache.org/jira/browse/YARN-9875
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.0, 3.2.3
>
> Attachments: YARN-9875-001.patch, YARN-9875-002.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> FSSchedulerConfigurationStore fails to update with hdfs path -
> "java.io.IOException: Filesystem closed"
> *RM Logs:*
> {code}
> 2019-10-06 16:50:40,829 INFO
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore:
> write temp capacity configuration fail,
> schedulerConfigFile=hdfs:/tmp/yarn/system/capacity-scheduler.xml.1570380640828.tmp
> java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:475)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1232)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1214)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1196)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1134)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:530)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:527)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:541)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:468)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1136)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1116)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1005)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:993)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.writeTmpConfig(FSSchedulerConfigurationStore.java:251)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.logMutation(FSSchedulerConfigurationStore.java:130)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.MutableCSConfigurationProvider.logAndApplyMutation(MutableCSConfigurationProvider.java:153)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2597)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2587)
> {code}
> *Repro:*
> {code:java}
> yarn-site.xml:
>
> yarn.scheduler.configuration.fs.path
> hdfs:///tmp/yarn/system
>
>
> yarn.scheduler.configuration.store.class
> fs
>
> [yarn@yarndocker-1 yarn]$ cat /tmp/abc.xml
>
>
> root.default
>
>
> priority
> 10
>
>
>
>
> [yarn@yarndocker-1 yarn]$ curl -v -X PUT -d @/tmp/abc.xml -H "Content-type:
> application/xml"
> 'http://yarndocker-1:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
> Filesystem closed
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9875) FSSchedulerConfigurationStore fails to update with hdfs path
[
https://issues.apache.org/jira/browse/YARN-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated YARN-9875:
-
Labels: pull-request-available (was: )
> FSSchedulerConfigurationStore fails to update with hdfs path
>
>
> Key: YARN-9875
> URL: https://issues.apache.org/jira/browse/YARN-9875
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.0
>
> Attachments: YARN-9875-001.patch, YARN-9875-002.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> FSSchedulerConfigurationStore fails to update with hdfs path -
> "java.io.IOException: Filesystem closed"
> *RM Logs:*
> {code}
> 2019-10-06 16:50:40,829 INFO
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore:
> write temp capacity configuration fail,
> schedulerConfigFile=hdfs:/tmp/yarn/system/capacity-scheduler.xml.1570380640828.tmp
> java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:475)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1232)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1214)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1196)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1134)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:530)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:527)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:541)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:468)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1136)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1116)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1005)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:993)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.writeTmpConfig(FSSchedulerConfigurationStore.java:251)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.logMutation(FSSchedulerConfigurationStore.java:130)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.MutableCSConfigurationProvider.logAndApplyMutation(MutableCSConfigurationProvider.java:153)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2597)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2587)
> {code}
> *Repro:*
> {code:java}
> yarn-site.xml:
>
> yarn.scheduler.configuration.fs.path
> hdfs:///tmp/yarn/system
>
>
> yarn.scheduler.configuration.store.class
> fs
>
> [yarn@yarndocker-1 yarn]$ cat /tmp/abc.xml
>
>
> root.default
>
>
> priority
> 10
>
>
>
>
> [yarn@yarndocker-1 yarn]$ curl -v -X PUT -d @/tmp/abc.xml -H "Content-type:
> application/xml"
> 'http://yarndocker-1:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
> Filesystem closed
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9875) FSSchedulerConfigurationStore fails to update with hdfs path
[
https://issues.apache.org/jira/browse/YARN-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9875:
Description:
FSSchedulerConfigurationStore fails to update with hdfs path -
"java.io.IOException: Filesystem closed"
*RM Logs:*
{code}
2019-10-06 16:50:40,829 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore:
write temp capacity configuration fail,
schedulerConfigFile=hdfs:/tmp/yarn/system/capacity-scheduler.xml.1570380640828.tmp
java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:475)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1232)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1214)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1196)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1134)
at
org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:530)
at
org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:527)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:541)
at
org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:468)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1136)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1116)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1005)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:993)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.writeTmpConfig(FSSchedulerConfigurationStore.java:251)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.logMutation(FSSchedulerConfigurationStore.java:130)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.MutableCSConfigurationProvider.logAndApplyMutation(MutableCSConfigurationProvider.java:153)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2597)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2587)
{code}
*Repro:*
{code:java}
yarn-site.xml:
yarn.scheduler.configuration.fs.path
hdfs:///tmp/yarn/system
yarn.scheduler.configuration.store.class
fs
[yarn@yarndocker-1 yarn]$ cat /tmp/abc.xml
root.default
priority
10
[yarn@yarndocker-1 yarn]$ curl -v -X PUT -d @/tmp/abc.xml -H "Content-type:
application/xml"
'http://yarndocker-1:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
Filesystem closed
{code}
was:
FSSchedulerConfigurationStore fails to update with hdfs path -
"java.io.IOException: Filesystem closed"
*RM Logs:*
{code}
2019-10-06 16:50:40,829 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore:
write temp capacity configuration fail,
schedulerConfigFile=hdfs:/tmp/yarn/system/capacity-scheduler.xml.1570380640828.tmp
java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:475)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1232)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1214)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1196)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1134)
at
org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:530)
at
org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:527)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:541)
at
org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:468)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1136)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1116)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1005)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:993)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.writeTmpConfig(FSSchedulerConfigurationStore.java:251)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.logMutation(FSSchedulerConfigurationStore.java:130)
at
[jira] [Updated] (YARN-9875) FSSchedulerConfigurationStore fails to update with hdfs path
[
https://issues.apache.org/jira/browse/YARN-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9875:
Description:
FSSchedulerConfigurationStore fails to update with hdfs path -
"java.io.IOException: Filesystem closed"
*RM Logs:*
{code}
2019-10-06 16:50:40,829 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore:
write temp capacity configuration fail,
schedulerConfigFile=hdfs:/tmp/yarn/system/capacity-scheduler.xml.1570380640828.tmp
java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:475)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1232)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1214)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1196)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1134)
at
org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:530)
at
org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:527)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:541)
at
org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:468)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1136)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1116)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1005)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:993)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.writeTmpConfig(FSSchedulerConfigurationStore.java:251)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.logMutation(FSSchedulerConfigurationStore.java:130)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.MutableCSConfigurationProvider.logAndApplyMutation(MutableCSConfigurationProvider.java:153)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2597)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2587)
{code}
*Repro:*
{code:java}
yarn-site.xml:
yarn.scheduler.configuration.fs.path
hdfs:///tmp/yarn/system
yarn.scheduler.configuration.store.class/name>
fs
[yarn@yarndocker-1 yarn]$ cat /tmp/abc.xml
root.default
priority
10
[yarn@yarndocker-1 yarn]$ curl -v -X PUT -d @/tmp/abc.xml -H "Content-type:
application/xml"
'http://yarndocker-1:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
Filesystem closed
{code}
was:
FSSchedulerConfigurationStore fails to update with hdfs path.
*Repro:*
{code:java}
yarn-site.xml:
yarn.scheduler.configuration.fs.path
hdfs:///tmp/yarn/system
yarn.scheduler.configuration.store.class/name>
fs
[yarn@yarndocker-1 yarn]$ cat /tmp/abc.xml
root.default
priority
10
[yarn@yarndocker-1 yarn]$ curl -v -X PUT -d @/tmp/abc.xml -H "Content-type:
application/xml"
'http://yarndocker-1:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
Filesystem closed
RM Logs:
2019-10-06 16:50:40,829 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore:
write temp capacity configuration fail,
schedulerConfigFile=hdfs:/tmp/yarn/system/capacity-scheduler.xml.1570380640828.tmp
java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:475)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1232)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1214)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1196)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1134)
at
org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:530)
at
org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:527)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:541)
at
org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:468)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1136)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1116)
at
[jira] [Updated] (YARN-9875) FSSchedulerConfigurationStore fails to update with hdfs path
[
https://issues.apache.org/jira/browse/YARN-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9875:
Attachment: YARN-9875-002.patch
> FSSchedulerConfigurationStore fails to update with hdfs path
>
>
> Key: YARN-9875
> URL: https://issues.apache.org/jira/browse/YARN-9875
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-9875-001.patch, YARN-9875-002.patch
>
>
> FSSchedulerConfigurationStore fails to update with hdfs path.
> *Repro:*
> {code:java}
> yarn-site.xml:
>
> yarn.scheduler.configuration.fs.path
> hdfs:///tmp/yarn/system
>
>
> yarn.scheduler.configuration.store.class/name>
> fs
>
> [yarn@yarndocker-1 yarn]$ cat /tmp/abc.xml
>
>
> root.default
>
>
> priority
> 10
>
>
>
>
> [yarn@yarndocker-1 yarn]$ curl -v -X PUT -d @/tmp/abc.xml -H "Content-type:
> application/xml"
> 'http://yarndocker-1:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
> Filesystem closed
> RM Logs:
> 2019-10-06 16:50:40,829 INFO
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore:
> write temp capacity configuration fail,
> schedulerConfigFile=hdfs:/tmp/yarn/system/capacity-scheduler.xml.1570380640828.tmp
> java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:475)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1232)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1214)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1196)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1134)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:530)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:527)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:541)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:468)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1136)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1116)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1005)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:993)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.writeTmpConfig(FSSchedulerConfigurationStore.java:251)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.logMutation(FSSchedulerConfigurationStore.java:130)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.MutableCSConfigurationProvider.logAndApplyMutation(MutableCSConfigurationProvider.java:153)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2597)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2587)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9875) FSSchedulerConfigurationStore fails to update with hdfs path
[
https://issues.apache.org/jira/browse/YARN-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9875:
Attachment: YARN-9875-001.patch
> FSSchedulerConfigurationStore fails to update with hdfs path
>
>
> Key: YARN-9875
> URL: https://issues.apache.org/jira/browse/YARN-9875
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-9875-001.patch
>
>
> FSSchedulerConfigurationStore fails to update with hdfs path.
> *Repro:*
> {code:java}
> yarn-site.xml:
>
> yarn.scheduler.configuration.fs.path
> hdfs:///tmp/yarn/system
>
>
> yarn.scheduler.configuration.store.class/name>
> fs
>
> [yarn@yarndocker-1 yarn]$ cat /tmp/abc.xml
>
>
> root.default
>
>
> priority
> 10
>
>
>
>
> [yarn@yarndocker-1 yarn]$ curl -v -X PUT -d @/tmp/abc.xml -H "Content-type:
> application/xml"
> 'http://yarndocker-1:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
> Filesystem closed
> RM Logs:
> 2019-10-06 16:50:40,829 INFO
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore:
> write temp capacity configuration fail,
> schedulerConfigFile=hdfs:/tmp/yarn/system/capacity-scheduler.xml.1570380640828.tmp
> java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:475)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1232)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1214)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1196)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1134)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:530)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:527)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:541)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:468)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1136)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1116)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1005)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:993)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.writeTmpConfig(FSSchedulerConfigurationStore.java:251)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.logMutation(FSSchedulerConfigurationStore.java:130)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.MutableCSConfigurationProvider.logAndApplyMutation(MutableCSConfigurationProvider.java:153)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2597)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2587)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9875) FSSchedulerConfigurationStore fails to update with hdfs path
[
https://issues.apache.org/jira/browse/YARN-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9875:
Affects Version/s: 3.3.0
> FSSchedulerConfigurationStore fails to update with hdfs path
>
>
> Key: YARN-9875
> URL: https://issues.apache.org/jira/browse/YARN-9875
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> FSSchedulerConfigurationStore fails to update with hdfs path.
> *Repro:*
> {code:java}
> yarn-site.xml:
>
> yarn.scheduler.configuration.fs.path
> hdfs:///tmp/yarn/system
>
>
> yarn.scheduler.configuration.store.class/name>
> fs
>
> [yarn@yarndocker-1 yarn]$ cat /tmp/abc.xml
>
>
> root.default
>
>
> priority
> 10
>
>
>
>
> [yarn@yarndocker-1 yarn]$ curl -v -X PUT -d @/tmp/abc.xml -H "Content-type:
> application/xml"
> 'http://yarndocker-1:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
> Filesystem closed
> RM Logs:
> 2019-10-06 16:50:40,829 INFO
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore:
> write temp capacity configuration fail,
> schedulerConfigFile=hdfs:/tmp/yarn/system/capacity-scheduler.xml.1570380640828.tmp
> java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:475)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1232)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1214)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1196)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1134)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:530)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:527)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:541)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:468)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1136)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1116)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1005)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:993)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.writeTmpConfig(FSSchedulerConfigurationStore.java:251)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.logMutation(FSSchedulerConfigurationStore.java:130)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.MutableCSConfigurationProvider.logAndApplyMutation(MutableCSConfigurationProvider.java:153)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2597)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2587)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9875) FSSchedulerConfigurationStore fails to update with hdfs path
[
https://issues.apache.org/jira/browse/YARN-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9875:
Component/s: capacityscheduler
> FSSchedulerConfigurationStore fails to update with hdfs path
>
>
> Key: YARN-9875
> URL: https://issues.apache.org/jira/browse/YARN-9875
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> FSSchedulerConfigurationStore fails to update with hdfs path.
> *Repro:*
> {code:java}
> yarn-site.xml:
>
> yarn.scheduler.configuration.fs.path
> hdfs:///tmp/yarn/system
>
>
> yarn.scheduler.configuration.store.class/name>
> fs
>
> [yarn@yarndocker-1 yarn]$ cat /tmp/abc.xml
>
>
> root.default
>
>
> priority
> 10
>
>
>
>
> [yarn@yarndocker-1 yarn]$ curl -v -X PUT -d @/tmp/abc.xml -H "Content-type:
> application/xml"
> 'http://yarndocker-1:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
> Filesystem closed
> RM Logs:
> 2019-10-06 16:50:40,829 INFO
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore:
> write temp capacity configuration fail,
> schedulerConfigFile=hdfs:/tmp/yarn/system/capacity-scheduler.xml.1570380640828.tmp
> java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:475)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1232)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1214)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1196)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1134)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:530)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:527)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:541)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:468)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1136)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1116)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1005)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:993)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.writeTmpConfig(FSSchedulerConfigurationStore.java:251)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.logMutation(FSSchedulerConfigurationStore.java:130)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.MutableCSConfigurationProvider.logAndApplyMutation(MutableCSConfigurationProvider.java:153)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2597)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2587)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
