[jira] [Commented] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[
https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124163#comment-16124163
]
Ajay Yadav commented on HDFS-12221:
---
Hi [~eddyxu], without that we get tag in first line along with xml
declaration tag.
{code:xml}
{code}
> Replace xcerces in XmlEditsVisitor
> ---
>
> Key: HDFS-12221
> URL: https://issues.apache.org/jira/browse/HDFS-12221
> Project: Hadoop HDFS
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha4
>Reporter: Lei (Eddy) Xu
>Assignee: Ajay Yadav
> Attachments: editsStored, fsimage_hdfs-12221.xml,
> HDFS-12221.01.patch, HDFS-12221.02.patch, HDFS-12221.03.patch,
> HDFS-12221.04.patch, HDFS-12221.05.patch
>
>
> XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR
> shading easier (HADOOP-14672)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-12293) DataNode should log file name on disk error
[
https://issues.apache.org/jira/browse/HDFS-12293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ajay Yadav reassigned HDFS-12293:
-
Assignee: Ajay Yadav
> DataNode should log file name on disk error
> ---
>
> Key: HDFS-12293
> URL: https://issues.apache.org/jira/browse/HDFS-12293
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Ajay Yadav
> Labels: newbie
>
> Found the following error message in precommit build
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/488/testReport/junit/org.apache.hadoop.hdfs.server.datanode/TestDataNodeVolumeFailureReporting/testSuccessiveVolumeFailures/
> {noformat}
> 2017-08-10 09:36:53,619 [DataXceiver for client
> DFSClient_NONMAPREDUCE_670847838_18 at /127.0.0.1:55851 [Receiving block
> BP-219227751-172.17.0.2-1502357801473:blk_1073741829_1005]] WARN
> datanode.DataNode (BlockReceiver.java:(287)) - IOException in
> BlockReceiver constructor. Cause is
> java.io.IOException: Not a directory
> at java.io.UnixFileSystem.createFileExclusively(Native Method)
> at java.io.File.createNewFile(File.java:1012)
> at
> org.apache.hadoop.hdfs.server.datanode.FileIoProvider.createFile(FileIoProvider.java:302)
> at
> org.apache.hadoop.hdfs.server.datanode.DatanodeUtil.createFileWithExistsCheck(DatanodeUtil.java:69)
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.createRbwFile(BlockPoolSlice.java:306)
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.createRbwFile(FsVolumeImpl.java:933)
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.createRbw(FsVolumeImpl.java:1202)
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.createRbw(FsDatasetImpl.java:1356)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.(BlockReceiver.java:215)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.getBlockReceiver(DataXceiver.java:1291)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:758)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:173)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:107)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290)
> {noformat}
> It is not known what file was being created.
> What's interesting is that {{DatanodeUtil#createFileWithExistsCheck}} does
> carry file name in the exception message, but the exception handler at
> {{DataTransfer#run()}} and {{BlockReceiver#BlockReceiver}} ignores it:
> {code:title=BlockReceiver#BlockReceiver}
> // check if there is a disk error
> IOException cause = DatanodeUtil.getCauseIfDiskError(ioe);
> DataNode.LOG.warn("IOException in BlockReceiver constructor"
> + (cause == null ? "" : ". Cause is "), cause);
> if (cause != null) {
> ioe = cause;
> // Volume error check moved to FileIoProvider
> }
> {code}
> The logger should print the file name in addition to the cause.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12281) Ozone: Ozone-default.xml has 3 properties that do not match the default Config value
[ https://issues.apache.org/jira/browse/HDFS-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12281: -- Description: # XML Property: ozone.handler.type Value - local ## Config Name: OZONE_HANDLER_TYPE_DEFAULT - distributed # XML Property: ozone.scm.handler.count.key - 20 ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT - 10 # XML Property: ozone.metastore.impl- RocksDB ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT- OZONE_METADATA_STORE_IMPL_ROCKSDB was: # XML Property: ozone.handler.type Value - local ## Config Name: OZONE_HANDLER_TYPE_DEFAULT - distributed # XML Property: ozone.scm.handler.count.key - 20 ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT - 10 # XML Property: ozone.metastore.impl- RocksDB ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT- LevelDB > Ozone: Ozone-default.xml has 3 properties that do not match the default > Config value > > > Key: HDFS-12281 > URL: https://issues.apache.org/jira/browse/HDFS-12281 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Fix For: HDFS-7240 > > Attachments: HDFS-12281-HDFS-7240.01.patch, > HDFS-12281-HDFS-7240.02.patch > > > # XML Property: ozone.handler.type Value - local > ## Config Name: OZONE_HANDLER_TYPE_DEFAULT - distributed > # XML Property: ozone.scm.handler.count.key - 20 > ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT - 10 > # XML Property: ozone.metastore.impl- RocksDB > ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT- > OZONE_METADATA_STORE_IMPL_ROCKSDB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12281) Ozone: Ozone-default.xml has 3 properties that do not match the default Config value
[ https://issues.apache.org/jira/browse/HDFS-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122149#comment-16122149 ] Ajay Yadav commented on HDFS-12281: --- Hi [~cheersyang], updated the patch with RocksDB as default implementation for metastore. > Ozone: Ozone-default.xml has 3 properties that do not match the default > Config value > > > Key: HDFS-12281 > URL: https://issues.apache.org/jira/browse/HDFS-12281 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Fix For: HDFS-7240 > > Attachments: HDFS-12281-HDFS-7240.01.patch, > HDFS-12281-HDFS-7240.02.patch > > > # XML Property: ozone.handler.type Value - local > ## Config Name: OZONE_HANDLER_TYPE_DEFAULT - distributed > # XML Property: ozone.scm.handler.count.key - 20 > ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT - 10 > # XML Property: ozone.metastore.impl- RocksDB > ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT- LevelDB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12281) Ozone: Ozone-default.xml has 3 properties that do not match the default Config value
[ https://issues.apache.org/jira/browse/HDFS-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12281: -- Attachment: HDFS-12281-HDFS-7240.02.patch > Ozone: Ozone-default.xml has 3 properties that do not match the default > Config value > > > Key: HDFS-12281 > URL: https://issues.apache.org/jira/browse/HDFS-12281 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Fix For: HDFS-7240 > > Attachments: HDFS-12281-HDFS-7240.01.patch, > HDFS-12281-HDFS-7240.02.patch > > > # XML Property: ozone.handler.type Value - local > ## Config Name: OZONE_HANDLER_TYPE_DEFAULT - distributed > # XML Property: ozone.scm.handler.count.key - 20 > ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT - 10 > # XML Property: ozone.metastore.impl- RocksDB > ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT- LevelDB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12281) Ozone: Ozone-default.xml has 3 properties that do not match the default Config value
[ https://issues.apache.org/jira/browse/HDFS-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120650#comment-16120650 ] Ajay Yadav commented on HDFS-12281: --- [~anu],[~xyao] plz review the patch. > Ozone: Ozone-default.xml has 3 properties that do not match the default > Config value > > > Key: HDFS-12281 > URL: https://issues.apache.org/jira/browse/HDFS-12281 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Fix For: HDFS-7240 > > Attachments: HDFS-12281-HDFS-7240.01.patch > > > # XML Property: ozone.handler.type Value - local > ## Config Name: OZONE_HANDLER_TYPE_DEFAULT - distributed > # XML Property: ozone.scm.handler.count.key - 20 > ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT - 10 > # XML Property: ozone.metastore.impl- RocksDB > ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT- LevelDB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12281) Ozone: Ozone-default.xml has 3 properties that do not match the default Config value
[ https://issues.apache.org/jira/browse/HDFS-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12281: -- Status: Patch Available (was: Open) > Ozone: Ozone-default.xml has 3 properties that do not match the default > Config value > > > Key: HDFS-12281 > URL: https://issues.apache.org/jira/browse/HDFS-12281 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Fix For: HDFS-7240 > > Attachments: HDFS-12281-HDFS-7240.01.patch > > > # XML Property: ozone.handler.type Value - local > ## Config Name: OZONE_HANDLER_TYPE_DEFAULT - distributed > # XML Property: ozone.scm.handler.count.key - 20 > ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT - 10 > # XML Property: ozone.metastore.impl- RocksDB > ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT- LevelDB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12281) Ozone: Ozone-default.xml has 3 properties that do not match the default Config value
[ https://issues.apache.org/jira/browse/HDFS-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12281: -- Description: #XML Property: ozone.handler.type Value - local ## Config Name: OZONE_HANDLER_TYPE_DEFAULT - distributed # XML Property: ozone.scm.handler.count.key - 20 ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT - 10 # XML Property: ozone.metastore.impl- RocksDB ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT- LevelDB was: # XML Property: ozone.handler.type Value:local ## Config Name: OZONE_HANDLER_TYPE_DEFAULT: distributed # XML Property: ozone.scm.handler.count.key : 20 ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT: 10 # XML Property: ozone.metastore.impl: RocksDB ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT: LevelDB > Ozone: Ozone-default.xml has 3 properties that do not match the default > Config value > > > Key: HDFS-12281 > URL: https://issues.apache.org/jira/browse/HDFS-12281 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Fix For: HDFS-7240 > > > #XML Property: ozone.handler.type Value - local > ## Config Name: OZONE_HANDLER_TYPE_DEFAULT - distributed > # XML Property: ozone.scm.handler.count.key - 20 > ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT - 10 > # XML Property: ozone.metastore.impl- RocksDB > ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT- LevelDB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12281) Ozone: Ozone-default.xml has 3 properties that do not match the default Config value
[ https://issues.apache.org/jira/browse/HDFS-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12281: -- Attachment: HDFS-12281-HDFS-7240.01.patch > Ozone: Ozone-default.xml has 3 properties that do not match the default > Config value > > > Key: HDFS-12281 > URL: https://issues.apache.org/jira/browse/HDFS-12281 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Fix For: HDFS-7240 > > Attachments: HDFS-12281-HDFS-7240.01.patch > > > # XML Property: ozone.handler.type Value - local > ## Config Name: OZONE_HANDLER_TYPE_DEFAULT - distributed > # XML Property: ozone.scm.handler.count.key - 20 > ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT - 10 > # XML Property: ozone.metastore.impl- RocksDB > ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT- LevelDB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12281) Ozone: Ozone-default.xml has 3 properties that do not match the default Config value
[ https://issues.apache.org/jira/browse/HDFS-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12281: -- Description: # XML Property: ozone.handler.type Value - local ## Config Name: OZONE_HANDLER_TYPE_DEFAULT - distributed # XML Property: ozone.scm.handler.count.key - 20 ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT - 10 # XML Property: ozone.metastore.impl- RocksDB ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT- LevelDB was: #XML Property: ozone.handler.type Value - local ## Config Name: OZONE_HANDLER_TYPE_DEFAULT - distributed # XML Property: ozone.scm.handler.count.key - 20 ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT - 10 # XML Property: ozone.metastore.impl- RocksDB ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT- LevelDB > Ozone: Ozone-default.xml has 3 properties that do not match the default > Config value > > > Key: HDFS-12281 > URL: https://issues.apache.org/jira/browse/HDFS-12281 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Fix For: HDFS-7240 > > > # XML Property: ozone.handler.type Value - local > ## Config Name: OZONE_HANDLER_TYPE_DEFAULT - distributed > # XML Property: ozone.scm.handler.count.key - 20 > ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT - 10 > # XML Property: ozone.metastore.impl- RocksDB > ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT- LevelDB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12281) Ozone: Ozone-default.xml has 3 properties that do not match the default Config value
[ https://issues.apache.org/jira/browse/HDFS-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12281: -- Description: # XML Property: ozone.handler.type Value:local ## Config Name: OZONE_HANDLER_TYPE_DEFAULT: distributed # XML Property: ozone.scm.handler.count.key : 20 ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT: 10 # XML Property: ozone.metastore.impl: RocksDB ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT: LevelDB was: # XML Property: ozone.handler.type Value:local #* Config Name: OZONE_HANDLER_TYPE_DEFAULT: distributed # XML Property: ozone.scm.handler.count.key : 20 #* Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT: 10 # XML Property: ozone.metastore.impl: RocksDB #* Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT: LevelDB > Ozone: Ozone-default.xml has 3 properties that do not match the default > Config value > > > Key: HDFS-12281 > URL: https://issues.apache.org/jira/browse/HDFS-12281 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Fix For: HDFS-7240 > > > # XML Property: ozone.handler.type Value:local > ## Config Name: OZONE_HANDLER_TYPE_DEFAULT: distributed > # XML Property: ozone.scm.handler.count.key : 20 > ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT: 10 > # XML Property: ozone.metastore.impl: RocksDB > ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT: LevelDB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12281) Ozone: Ozone-default.xml has 3 properties that do not match the default Config value
[ https://issues.apache.org/jira/browse/HDFS-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12281: -- Description: # XML Property: ozone.handler.type Value:local #* Config Name: OZONE_HANDLER_TYPE_DEFAULT: distributed # XML Property: ozone.scm.handler.count.key : 20 #* Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT: 10 # XML Property: ozone.metastore.impl: RocksDB #* Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT: LevelDB was: # XML Property: ozone.handler.type Value:local ## Config Name: OZONE_HANDLER_TYPE_DEFAULT ## Config Value: distributed # XML Property: ozone.scm.handler.count.key : 20 ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT: 10 # XML Property: ozone.metastore.impl: RocksDB ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT: LevelDB > Ozone: Ozone-default.xml has 3 properties that do not match the default > Config value > > > Key: HDFS-12281 > URL: https://issues.apache.org/jira/browse/HDFS-12281 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Fix For: HDFS-7240 > > > # XML Property: ozone.handler.type Value:local > #* Config Name: OZONE_HANDLER_TYPE_DEFAULT: distributed > # XML Property: ozone.scm.handler.count.key : 20 > #* Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT: 10 > # XML Property: ozone.metastore.impl: RocksDB > #* Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT: LevelDB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12281) Ozone: Ozone-default.xml has 3 properties that do not match the default Config value
[ https://issues.apache.org/jira/browse/HDFS-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12281: -- Description: # XML Property: ozone.handler.type Value:local ## Config Name: OZONE_HANDLER_TYPE_DEFAULT ## Config Value: distributed # XML Property: ozone.scm.handler.count.key : 20 ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT: 10 # XML Property: ozone.metastore.impl: RocksDB ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT: LevelDB > Ozone: Ozone-default.xml has 3 properties that do not match the default > Config value > > > Key: HDFS-12281 > URL: https://issues.apache.org/jira/browse/HDFS-12281 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Fix For: HDFS-7240 > > > # XML Property: ozone.handler.type Value:local > ## Config Name: OZONE_HANDLER_TYPE_DEFAULT > ## Config Value: distributed > # XML Property: ozone.scm.handler.count.key : 20 > ## Config Name: OZONE_SCM_HANDLER_COUNT_DEFAULT: 10 > # XML Property: ozone.metastore.impl: RocksDB > ## Config Name: OZONE_METADATA_STORE_IMPL_DEFAULT: LevelDB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12254) Upgrade JUnit from 4 to 5 in hadoop-hdfs
[ https://issues.apache.org/jira/browse/HDFS-12254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120316#comment-16120316 ] Ajay Yadav commented on HDFS-12254: --- hi [~ajisakaa] , As i understand , we can do it in two ways. # 1. Update the junit dependency in hadoop-main to junit5 with junit-jupiter-engine. which will require changes in most of the test cases to move them to new junit5 api and platform. (More riskier) # 2. Update the junit dependency in hadoop-main to junit5 while maintaining backward compatibility to test cases built in junit4 using (junit-vintage-engine). As a next step we can create new test cases using junit 5 api and move old test cases to junit5 in steps. This will be incremental change with less risk to breaking old test cases. Any ideas, suggestions on this? > Upgrade JUnit from 4 to 5 in hadoop-hdfs > > > Key: HDFS-12254 > URL: https://issues.apache.org/jira/browse/HDFS-12254 > Project: Hadoop HDFS > Issue Type: Test > Components: test >Reporter: Akira Ajisaka >Assignee: Ajay Yadav > > Feel free to create sub-tasks for each module. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120268#comment-16120268 ] Ajay Yadav commented on HDFS-12221: --- [~andrew.wang], [~eddyxu] Added updated binary file to resolve the HDFS test which is failing. (i.e TestOfflineEditsViewer#testStored). Please test the patch with updated binary file "editsStored" > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: editsStored, fsimage_hdfs-12221.xml, > HDFS-12221.01.patch, HDFS-12221.02.patch, HDFS-12221.03.patch, > HDFS-12221.04.patch, HDFS-12221.05.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Attachment: editsStored HDFS-12221.05.patch > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: editsStored, fsimage_hdfs-12221.xml, > HDFS-12221.01.patch, HDFS-12221.02.patch, HDFS-12221.03.patch, > HDFS-12221.04.patch, HDFS-12221.05.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Attachment: (was: HDFS-12221.05.patch) > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch, HDFS-12221.03.patch, HDFS-12221.04.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Attachment: HDFS-12221.05.patch > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch, HDFS-12221.03.patch, HDFS-12221.04.patch, > HDFS-12221.05.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Attachment: (was: HDFS-12221.05.patch) > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch, HDFS-12221.03.patch, HDFS-12221.04.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Attachment: HDFS-12221.05.patch > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch, HDFS-12221.03.patch, HDFS-12221.04.patch, > HDFS-12221.05.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119338#comment-16119338 ] Ajay Yadav edited comment on HDFS-12221 at 8/9/17 3:03 AM: --- i checked the findbugs, yarn and mapreduce errors. They seems unrelated to patch. was (Author: ajayydv): i checked the findbugs, hadoop-hdfs,yarn, mapreduce errors. They seems unrelated to patch. > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch, HDFS-12221.03.patch, HDFS-12221.04.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119338#comment-16119338 ] Ajay Yadav commented on HDFS-12221: --- i checked the findbugs, hadoop-hdfs,yarn, mapreduce errors. They seems unrelated to patch. > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch, HDFS-12221.03.patch, HDFS-12221.04.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Comment: was deleted (was: checked the test failures for hdfs,yarn and hdfs. They are not related to patch.) > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch, HDFS-12221.03.patch, HDFS-12221.04.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119269#comment-16119269 ] Ajay Yadav commented on HDFS-12221: --- checked the test failures for hdfs,yarn and hdfs. They are not related to patch. > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch, HDFS-12221.03.patch, HDFS-12221.04.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12281) Ozone: Ozone-default.xml has 3 properties that do not match the default Config value
[ https://issues.apache.org/jira/browse/HDFS-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12281: -- Tags: Ozone > Ozone: Ozone-default.xml has 3 properties that do not match the default > Config value > > > Key: HDFS-12281 > URL: https://issues.apache.org/jira/browse/HDFS-12281 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Fix For: HDFS-7240 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12281) Ozone-default.xml has 3 properties that do not match the default Config value
Ajay Yadav created HDFS-12281: - Summary: Ozone-default.xml has 3 properties that do not match the default Config value Key: HDFS-12281 URL: https://issues.apache.org/jira/browse/HDFS-12281 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: HDFS-7240 Reporter: Ajay Yadav Fix For: HDFS-7240 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-12281) Ozone-default.xml has 3 properties that do not match the default Config value
[ https://issues.apache.org/jira/browse/HDFS-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav reassigned HDFS-12281: - Assignee: Ajay Yadav > Ozone-default.xml has 3 properties that do not match the default Config value > - > > Key: HDFS-12281 > URL: https://issues.apache.org/jira/browse/HDFS-12281 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: HDFS-7240 >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Fix For: HDFS-7240 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Attachment: HDFS-12221.04.patch > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch, HDFS-12221.03.patch, HDFS-12221.04.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12238) Ozone: Add valid trace ID check in sendCommandAsync
[
https://issues.apache.org/jira/browse/HDFS-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ajay Yadav updated HDFS-12238:
--
Status: Patch Available (was: In Progress)
> Ozone: Add valid trace ID check in sendCommandAsync
> ---
>
> Key: HDFS-12238
> URL: https://issues.apache.org/jira/browse/HDFS-12238
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Ajay Yadav
> Labels: newbie
> Attachments: HDFS-12238-HDFS-7240.01.patch
>
>
> In the function {{XceiverClientHandler#sendCommandAsync}} we should add a
> check
> {code}
> if(StringUtils.isEmpty(request.getTraceID())) {
> throw new IllegalArgumentException("Invalid trace ID");
> }
> {code}
> To ensure that ozone clients always send a valid trace ID. However, when you
> do that a set of current tests that do add a valid trace ID will fail. So we
> need to fix these tests too.
> {code}
> TestContainerMetrics.testContainerMetrics
> TestOzoneContainer.testBothGetandPutSmallFile
> TestOzoneContainer.testCloseContainer
> TestOzoneContainer.testOzoneContainerViaDataNode
> TestOzoneContainer.testXcieverClientAsync
> TestOzoneContainer.testCreateOzoneContainer
> TestOzoneContainer.testDeleteContainer
> TestContainerServer.testClientServer
> TestContainerServer.testClientServerWithContainerDispatcher
> TestKeys.testPutAndGetKeyWithDnRestart
> {code}
> This is based on a comment from [~vagarychen] in HDFS-11580.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12238) Ozone: Add valid trace ID check in sendCommandAsync
[
https://issues.apache.org/jira/browse/HDFS-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118624#comment-16118624
]
Ajay Yadav commented on HDFS-12238:
---
[~anu] Plz review the patch.
> Ozone: Add valid trace ID check in sendCommandAsync
> ---
>
> Key: HDFS-12238
> URL: https://issues.apache.org/jira/browse/HDFS-12238
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Ajay Yadav
> Labels: newbie
> Attachments: HDFS-12238-HDFS-7240.01.patch
>
>
> In the function {{XceiverClientHandler#sendCommandAsync}} we should add a
> check
> {code}
> if(StringUtils.isEmpty(request.getTraceID())) {
> throw new IllegalArgumentException("Invalid trace ID");
> }
> {code}
> To ensure that ozone clients always send a valid trace ID. However, when you
> do that a set of current tests that do add a valid trace ID will fail. So we
> need to fix these tests too.
> {code}
> TestContainerMetrics.testContainerMetrics
> TestOzoneContainer.testBothGetandPutSmallFile
> TestOzoneContainer.testCloseContainer
> TestOzoneContainer.testOzoneContainerViaDataNode
> TestOzoneContainer.testXcieverClientAsync
> TestOzoneContainer.testCreateOzoneContainer
> TestOzoneContainer.testDeleteContainer
> TestContainerServer.testClientServer
> TestContainerServer.testClientServerWithContainerDispatcher
> TestKeys.testPutAndGetKeyWithDnRestart
> {code}
> This is based on a comment from [~vagarychen] in HDFS-11580.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work started] (HDFS-12238) Ozone: Add valid trace ID check in sendCommandAsync
[
https://issues.apache.org/jira/browse/HDFS-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HDFS-12238 started by Ajay Yadav.
-
> Ozone: Add valid trace ID check in sendCommandAsync
> ---
>
> Key: HDFS-12238
> URL: https://issues.apache.org/jira/browse/HDFS-12238
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Ajay Yadav
> Labels: newbie
> Attachments: HDFS-12238-HDFS-7240.01.patch
>
>
> In the function {{XceiverClientHandler#sendCommandAsync}} we should add a
> check
> {code}
> if(StringUtils.isEmpty(request.getTraceID())) {
> throw new IllegalArgumentException("Invalid trace ID");
> }
> {code}
> To ensure that ozone clients always send a valid trace ID. However, when you
> do that a set of current tests that do add a valid trace ID will fail. So we
> need to fix these tests too.
> {code}
> TestContainerMetrics.testContainerMetrics
> TestOzoneContainer.testBothGetandPutSmallFile
> TestOzoneContainer.testCloseContainer
> TestOzoneContainer.testOzoneContainerViaDataNode
> TestOzoneContainer.testXcieverClientAsync
> TestOzoneContainer.testCreateOzoneContainer
> TestOzoneContainer.testDeleteContainer
> TestContainerServer.testClientServer
> TestContainerServer.testClientServerWithContainerDispatcher
> TestKeys.testPutAndGetKeyWithDnRestart
> {code}
> This is based on a comment from [~vagarychen] in HDFS-11580.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12238) Ozone: Add valid trace ID check in sendCommandAsync
[
https://issues.apache.org/jira/browse/HDFS-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ajay Yadav updated HDFS-12238:
--
Attachment: HDFS-12238-HDFS-7240.01.patch
> Ozone: Add valid trace ID check in sendCommandAsync
> ---
>
> Key: HDFS-12238
> URL: https://issues.apache.org/jira/browse/HDFS-12238
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Ajay Yadav
> Labels: newbie
> Attachments: HDFS-12238-HDFS-7240.01.patch
>
>
> In the function {{XceiverClientHandler#sendCommandAsync}} we should add a
> check
> {code}
> if(StringUtils.isEmpty(request.getTraceID())) {
> throw new IllegalArgumentException("Invalid trace ID");
> }
> {code}
> To ensure that ozone clients always send a valid trace ID. However, when you
> do that a set of current tests that do add a valid trace ID will fail. So we
> need to fix these tests too.
> {code}
> TestContainerMetrics.testContainerMetrics
> TestOzoneContainer.testBothGetandPutSmallFile
> TestOzoneContainer.testCloseContainer
> TestOzoneContainer.testOzoneContainerViaDataNode
> TestOzoneContainer.testXcieverClientAsync
> TestOzoneContainer.testCreateOzoneContainer
> TestOzoneContainer.testDeleteContainer
> TestContainerServer.testClientServer
> TestContainerServer.testClientServerWithContainerDispatcher
> TestKeys.testPutAndGetKeyWithDnRestart
> {code}
> This is based on a comment from [~vagarychen] in HDFS-11580.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Status: Patch Available (was: Open) > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch, HDFS-12221.03.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Status: Open (was: Patch Available) > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch, HDFS-12221.03.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Status: Patch Available (was: Open) > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch, HDFS-12221.03.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Status: Open (was: Patch Available) > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch, HDFS-12221.03.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[
https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117438#comment-16117438
]
Ajay Yadav commented on HDFS-12221:
---
[~fight4gold] Uploaded the patch again after fixing the conflicts. "
${runningWithNative}" is not there in
new patch. Please review.
> Replace xcerces in XmlEditsVisitor
> ---
>
> Key: HDFS-12221
> URL: https://issues.apache.org/jira/browse/HDFS-12221
> Project: Hadoop HDFS
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha4
>Reporter: Lei (Eddy) Xu
>Assignee: Ajay Yadav
> Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch,
> HDFS-12221.02.patch, HDFS-12221.03.patch
>
>
> XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR
> shading easier (HADOOP-14672)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Attachment: HDFS-12221.03.patch > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch, HDFS-12221.03.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12264) DataNode uses a deprecated method IoUtils#cleanup.
[ https://issues.apache.org/jira/browse/HDFS-12264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117258#comment-16117258 ] Ajay Yadav commented on HDFS-12264: --- [~aagarwa],[~anu] Could you please review patch for HDFS-12264. > DataNode uses a deprecated method IoUtils#cleanup. > -- > > Key: HDFS-12264 > URL: https://issues.apache.org/jira/browse/HDFS-12264 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Attachments: HDFS-12264.01.patch > > > DataNode uses a deprecated method IoUtils#cleanup. It can be replaced with > IoUtils#cleanupWithLogger. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12264) DataNode uses a deprecated method IoUtils#cleanup.
[ https://issues.apache.org/jira/browse/HDFS-12264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12264: -- Attachment: HDFS-12264.01.patch > DataNode uses a deprecated method IoUtils#cleanup. > -- > > Key: HDFS-12264 > URL: https://issues.apache.org/jira/browse/HDFS-12264 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Attachments: HDFS-12264.01.patch > > > DataNode uses a deprecated method IoUtils#cleanup. It can be replaced with > IoUtils#cleanupWithLogger. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12264) DataNode uses a deprecated method IoUtils#cleanup.
[ https://issues.apache.org/jira/browse/HDFS-12264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12264: -- Status: Patch Available (was: In Progress) > DataNode uses a deprecated method IoUtils#cleanup. > -- > > Key: HDFS-12264 > URL: https://issues.apache.org/jira/browse/HDFS-12264 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Attachments: HDFS-12264.01.patch > > > DataNode uses a deprecated method IoUtils#cleanup. It can be replaced with > IoUtils#cleanupWithLogger. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work started] (HDFS-12264) DataNode uses a deprecated method IoUtils#cleanup.
[ https://issues.apache.org/jira/browse/HDFS-12264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-12264 started by Ajay Yadav. - > DataNode uses a deprecated method IoUtils#cleanup. > -- > > Key: HDFS-12264 > URL: https://issues.apache.org/jira/browse/HDFS-12264 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Yadav >Assignee: Ajay Yadav > Attachments: HDFS-12264.01.patch > > > DataNode uses a deprecated method IoUtils#cleanup. It can be replaced with > IoUtils#cleanupWithLogger. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-12238) Ozone: Add valid trace ID check in sendCommandAsync
[
https://issues.apache.org/jira/browse/HDFS-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ajay Yadav reassigned HDFS-12238:
-
Assignee: Ajay Yadav
> Ozone: Add valid trace ID check in sendCommandAsync
> ---
>
> Key: HDFS-12238
> URL: https://issues.apache.org/jira/browse/HDFS-12238
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Ajay Yadav
> Labels: newbie
>
> In the function {{XceiverClientHandler#sendCommandAsync}} we should add a
> check
> {code}
> if(StringUtils.isEmpty(request.getTraceID())) {
> throw new IllegalArgumentException("Invalid trace ID");
> }
> {code}
> To ensure that ozone clients always send a valid trace ID. However, when you
> do that a set of current tests that do add a valid trace ID will fail. So we
> need to fix these tests too.
> {code}
> TestContainerMetrics.testContainerMetrics
> TestOzoneContainer.testBothGetandPutSmallFile
> TestOzoneContainer.testCloseContainer
> TestOzoneContainer.testOzoneContainerViaDataNode
> TestOzoneContainer.testXcieverClientAsync
> TestOzoneContainer.testCreateOzoneContainer
> TestOzoneContainer.testDeleteContainer
> TestContainerServer.testClientServer
> TestContainerServer.testClientServerWithContainerDispatcher
> TestKeys.testPutAndGetKeyWithDnRestart
> {code}
> This is based on a comment from [~vagarychen] in HDFS-11580.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12262) Ozone: KSM: Reduce default handler thread count from 200
[ https://issues.apache.org/jira/browse/HDFS-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115188#comment-16115188 ] Ajay Yadav commented on HDFS-12262: --- Thanks [~xyao], changed name as per documentation. > Ozone: KSM: Reduce default handler thread count from 200 > > > Key: HDFS-12262 > URL: https://issues.apache.org/jira/browse/HDFS-12262 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Ajay Yadav >Priority: Minor > Fix For: HDFS-7240 > > Attachments: HDFS-12262-HDFS-7240.01.patch > > > KSMConfigKeys#OZONE_KSM_HANDLER_COUNT_DEFAULT is currently 200. It should be > a much smaller value like (20) by default and customized according to the > size of cluster such as the 20Log(N) where N is the size of the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12262) Ozone: KSM: Reduce default handler thread count from 200
[ https://issues.apache.org/jira/browse/HDFS-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12262: -- Attachment: HDFS-12262-HDFS-7240.01.patch > Ozone: KSM: Reduce default handler thread count from 200 > > > Key: HDFS-12262 > URL: https://issues.apache.org/jira/browse/HDFS-12262 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Ajay Yadav >Priority: Minor > Fix For: HDFS-7240 > > Attachments: HDFS-12262-HDFS-7240.01.patch > > > KSMConfigKeys#OZONE_KSM_HANDLER_COUNT_DEFAULT is currently 200. It should be > a much smaller value like (20) by default and customized according to the > size of cluster such as the 20Log(N) where N is the size of the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12262) Ozone: KSM: Reduce default handler thread count from 200
[ https://issues.apache.org/jira/browse/HDFS-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12262: -- Attachment: (was: HDFS-12262.01.patch) > Ozone: KSM: Reduce default handler thread count from 200 > > > Key: HDFS-12262 > URL: https://issues.apache.org/jira/browse/HDFS-12262 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Ajay Yadav >Priority: Minor > Fix For: HDFS-7240 > > > KSMConfigKeys#OZONE_KSM_HANDLER_COUNT_DEFAULT is currently 200. It should be > a much smaller value like (20) by default and customized according to the > size of cluster such as the 20Log(N) where N is the size of the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12262) Ozone: KSM: Reduce default handler thread count from 200
[ https://issues.apache.org/jira/browse/HDFS-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12262: -- Fix Version/s: HDFS-7240 Status: Patch Available (was: Open) > Ozone: KSM: Reduce default handler thread count from 200 > > > Key: HDFS-12262 > URL: https://issues.apache.org/jira/browse/HDFS-12262 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Ajay Yadav >Priority: Minor > Fix For: HDFS-7240 > > Attachments: HDFS-12262.01.patch > > > KSMConfigKeys#OZONE_KSM_HANDLER_COUNT_DEFAULT is currently 200. It should be > a much smaller value like (20) by default and customized according to the > size of cluster such as the 20Log(N) where N is the size of the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115183#comment-16115183 ] Ajay Yadav commented on HDFS-12221: --- [~andrew.wang], [~eddyxu] Updated the patch with changes in pom file as well. Plz review. > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Attachment: HDFS-12221.02.patch > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Release Note: New patch with changes in maven dependency. (removed apache xcerces as dependency) Status: Patch Available (was: Open) > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch, > HDFS-12221.02.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Status: Open (was: Patch Available) > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12262) Ozone: KSM: Reduce default handler thread count from 200
[ https://issues.apache.org/jira/browse/HDFS-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12262: -- Status: Open (was: Patch Available) > Ozone: KSM: Reduce default handler thread count from 200 > > > Key: HDFS-12262 > URL: https://issues.apache.org/jira/browse/HDFS-12262 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Ajay Yadav >Priority: Minor > Attachments: HDFS-12262.01.patch > > > KSMConfigKeys#OZONE_KSM_HANDLER_COUNT_DEFAULT is currently 200. It should be > a much smaller value like (20) by default and customized according to the > size of cluster such as the 20Log(N) where N is the size of the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12162) Update listStatus document to describe the behavior when the argument is a file
[ https://issues.apache.org/jira/browse/HDFS-12162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12162: -- Status: Patch Available (was: In Progress) > Update listStatus document to describe the behavior when the argument is a > file > --- > > Key: HDFS-12162 > URL: https://issues.apache.org/jira/browse/HDFS-12162 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs, httpfs >Reporter: Yongjun Zhang >Assignee: Ajay Yadav > Attachments: HDFS-12162.01.patch, Screen Shot 2017-08-03 at 11.01.46 > AM.png, Screen Shot 2017-08-03 at 11.02.19 AM.png > > > The listStatus method can take in either directory path or file path as > input, however, currently both the javadoc and external document describe it > as only taking directory as input. This jira is to update the document about > the behavior when the argument is a file path. > Thanks [~xiaochen] for the review and discussion in HDFS-12139, creating this > jira is the result of our discussion there. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12162) Update listStatus document to describe the behavior when the argument is a file
[ https://issues.apache.org/jira/browse/HDFS-12162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12162: -- Status: In Progress (was: Patch Available) > Update listStatus document to describe the behavior when the argument is a > file > --- > > Key: HDFS-12162 > URL: https://issues.apache.org/jira/browse/HDFS-12162 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs, httpfs >Reporter: Yongjun Zhang >Assignee: Ajay Yadav > Attachments: HDFS-12162.01.patch, Screen Shot 2017-08-03 at 11.01.46 > AM.png, Screen Shot 2017-08-03 at 11.02.19 AM.png > > > The listStatus method can take in either directory path or file path as > input, however, currently both the javadoc and external document describe it > as only taking directory as input. This jira is to update the document about > the behavior when the argument is a file path. > Thanks [~xiaochen] for the review and discussion in HDFS-12139, creating this > jira is the result of our discussion there. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12264) DataNode uses a deprecated method IoUtils#cleanup.
Ajay Yadav created HDFS-12264: - Summary: DataNode uses a deprecated method IoUtils#cleanup. Key: HDFS-12264 URL: https://issues.apache.org/jira/browse/HDFS-12264 Project: Hadoop HDFS Issue Type: Improvement Reporter: Ajay Yadav Assignee: Ajay Yadav DataNode uses a deprecated method IoUtils#cleanup. It can be replaced with IoUtils#cleanupWithLogger. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12262) Ozone: KSM: Reduce default handler thread count from 200
[ https://issues.apache.org/jira/browse/HDFS-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12262: -- Attachment: HDFS-12262.01.patch > Ozone: KSM: Reduce default handler thread count from 200 > > > Key: HDFS-12262 > URL: https://issues.apache.org/jira/browse/HDFS-12262 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Ajay Yadav >Priority: Minor > Attachments: HDFS-12262.01.patch > > > KSMConfigKeys#OZONE_KSM_HANDLER_COUNT_DEFAULT is currently 200. It should be > a much smaller value like (20) by default and customized according to the > size of cluster such as the 20Log(N) where N is the size of the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12262) Ozone: KSM: Reduce default handler thread count from 200
[ https://issues.apache.org/jira/browse/HDFS-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12262: -- Status: Patch Available (was: Open) Hi [~xyao], Attaching the patch, please review. > Ozone: KSM: Reduce default handler thread count from 200 > > > Key: HDFS-12262 > URL: https://issues.apache.org/jira/browse/HDFS-12262 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Ajay Yadav >Priority: Minor > > KSMConfigKeys#OZONE_KSM_HANDLER_COUNT_DEFAULT is currently 200. It should be > a much smaller value like (20) by default and customized according to the > size of cluster such as the 20Log(N) where N is the size of the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Status: Patch Available (was: Open) > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Status: Open (was: Patch Available) > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Attachment: (was: edits_hdfs-12221.patch) > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: fsimage_hdfs-12221.xml, HDFS-12221.01.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12189) TestPread#testPreadFailureWithChangedBlockLocations fails intermittently
[
https://issues.apache.org/jira/browse/HDFS-12189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113703#comment-16113703
]
Ajay Yadav commented on HDFS-12189:
---
Hi [~brahmareddy] , i tested it in a loop (20 times). Didn't failed. Do you
have more information about failure of this case?
> TestPread#testPreadFailureWithChangedBlockLocations fails intermittently
>
>
> Key: HDFS-12189
> URL: https://issues.apache.org/jira/browse/HDFS-12189
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Reporter: Brahma Reddy Battula
>
> {noformat}
> java.lang.ArrayIndexOutOfBoundsException: 1
> at
> org.apache.hadoop.hdfs.TestPread.doPreadTestWithChangedLocations(TestPread.java:656)
> at
> org.apache.hadoop.hdfs.TestPread.testPreadFailureWithChangedBlockLocations(TestPread.java:566)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113581#comment-16113581 ] Ajay Yadav commented on HDFS-12221: --- [~eddyxu],[~andrew.wang] If we agree on removing xerces dependency then i can update the patch with changes in pom files. > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: edits_hdfs-12221.patch, fsimage_hdfs-12221.xml, > HDFS-12221.01.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113348#comment-16113348 ] Ajay Yadav edited comment on HDFS-12221 at 8/3/17 7:34 PM: --- [~andrew.wang] I checked for references within hdfs and other projects but can't find one. I agree, we shall remove it as dependency if its not used anywhere else. was (Author: ajayydv): [~andrew.wang] I checked for references within hdfs and other projects but can't find one. I agree, we should remove at as dependency if its not used anywhere else. > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: edits_hdfs-12221.patch, fsimage_hdfs-12221.xml, > HDFS-12221.01.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113348#comment-16113348 ] Ajay Yadav commented on HDFS-12221: --- [~andrew.wang] I checked for references within hdfs and other projects but can't find one. I agree, we should remove at as dependency if its not used anywhere else. > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: edits_hdfs-12221.patch, fsimage_hdfs-12221.xml, > HDFS-12221.01.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12162) Update listStatus document to describe the behavior when the argument is a file
[ https://issues.apache.org/jira/browse/HDFS-12162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12162: -- Attachment: HDFS-12162.01.patch Screen Shot 2017-08-03 at 11.02.19 AM.png Screen Shot 2017-08-03 at 11.01.46 AM.png > Update listStatus document to describe the behavior when the argument is a > file > --- > > Key: HDFS-12162 > URL: https://issues.apache.org/jira/browse/HDFS-12162 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs, httpfs >Reporter: Yongjun Zhang >Assignee: Ajay Yadav > Attachments: HDFS-12162.01.patch, Screen Shot 2017-08-03 at 11.01.46 > AM.png, Screen Shot 2017-08-03 at 11.02.19 AM.png > > > The listStatus method can take in either directory path or file path as > input, however, currently both the javadoc and external document describe it > as only taking directory as input. This jira is to update the document about > the behavior when the argument is a file path. > Thanks [~xiaochen] for the review and discussion in HDFS-12139, creating this > jira is the result of our discussion there. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12162) Update listStatus document to describe the behavior when the argument is a file
[ https://issues.apache.org/jira/browse/HDFS-12162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12162: -- Status: Patch Available (was: Open) [~yzhangal] Attaching the patch for change in WebHDFS.md (added an entry for listing file) and javadoc change in FSOperations. Please review it. > Update listStatus document to describe the behavior when the argument is a > file > --- > > Key: HDFS-12162 > URL: https://issues.apache.org/jira/browse/HDFS-12162 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs, httpfs >Reporter: Yongjun Zhang >Assignee: Ajay Yadav > > The listStatus method can take in either directory path or file path as > input, however, currently both the javadoc and external document describe it > as only taking directory as input. This jira is to update the document about > the behavior when the argument is a file path. > Thanks [~xiaochen] for the review and discussion in HDFS-12139, creating this > jira is the result of our discussion there. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Attachment: edits_hdfs-12221.patch fsimage_hdfs-12221.xml HDFS-12221.01.patch > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: edits_hdfs-12221.patch, fsimage_hdfs-12221.xml, > HDFS-12221.01.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HDFS-12221: -- Status: Patch Available (was: Open) > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > Attachments: edits_hdfs-12221.patch, fsimage_hdfs-12221.xml, > HDFS-12221.01.patch > > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113177#comment-16113177 ] Ajay Yadav commented on HDFS-12221: --- [~eddyxu] Thanks for confirming. Attaching the patch and output of fsimage and edit logs after the changes. Please review. > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-12254) Upgrade JUnit from 4 to 5 in hadoop-hdfs
[ https://issues.apache.org/jira/browse/HDFS-12254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav reassigned HDFS-12254: - Assignee: Ajay Yadav > Upgrade JUnit from 4 to 5 in hadoop-hdfs > > > Key: HDFS-12254 > URL: https://issues.apache.org/jira/browse/HDFS-12254 > Project: Hadoop HDFS > Issue Type: Test > Components: test >Reporter: Akira Ajisaka >Assignee: Ajay Yadav > > Feel free to create sub-tasks for each module. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109549#comment-16109549 ] Ajay Yadav commented on HDFS-12221: --- [~eddyxu] I am planning to work on this. Could you plz elaborate on what new XML capability you are referring to. Do we want to just replace OutputFormat and XMLSerializer from xcerces with jdk inbuilt classes? > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-12221) Replace xcerces in XmlEditsVisitor
[ https://issues.apache.org/jira/browse/HDFS-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav reassigned HDFS-12221: - Assignee: Ajay Yadav > Replace xcerces in XmlEditsVisitor > --- > > Key: HDFS-12221 > URL: https://issues.apache.org/jira/browse/HDFS-12221 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0-alpha4 >Reporter: Lei (Eddy) Xu >Assignee: Ajay Yadav > > XmlEditsVisitor should use new XML capability in the newer JDK, to make JAR > shading easier (HADOOP-14672) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12173) MiniDFSCluster cannot reliably use NameNode#stop
[
https://issues.apache.org/jira/browse/HDFS-12173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108346#comment-16108346
]
Ajay Yadav edited comment on HDFS-12173 at 8/1/17 4:19 PM:
---
[~daryn], Could you please share more information on this. Since Namenode has
only one stop method does it make sense to update Stop method itself to call
HAState#setState(context, stopState) where stopState obj is set to STOPPING
state?
was (Author: ajayydv):
[~daryn], Could you please share more information on this. Since Namenode has
only one stop method does it make sense to update Stop method itself to call
HAState#setState(context, HAServiceState.STOPPING)?
> MiniDFSCluster cannot reliably use NameNode#stop
>
>
> Key: HDFS-12173
> URL: https://issues.apache.org/jira/browse/HDFS-12173
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.7.0
>Reporter: Daryn Sharp
>
> Sporadic test failures occur because {{NameNode#stop}} used by the mini
> cluster does not properly manage the HA context's state. It directly calls
> {{HAState#exitState(context)}} instead of
> {{HAState#setState(context,state)}}. The latter will properly lock the
> namesystem and update the ha state while locked, while the former does not.
> The result is that while the cluster is stopping, the lock is released and
> any queued rpc calls think the NN is still active and are processed while the
> NN is in an unstable half-stopped state.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12173) MiniDFSCluster cannot reliably use NameNode#stop
[
https://issues.apache.org/jira/browse/HDFS-12173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108346#comment-16108346
]
Ajay Yadav commented on HDFS-12173:
---
[~daryn], Could you please share more information on this. Since Namenode has
only one stop method does it make sense to update Stop method itself to call
HAState#setState(context, HAServiceState.STOPPING)?
> MiniDFSCluster cannot reliably use NameNode#stop
>
>
> Key: HDFS-12173
> URL: https://issues.apache.org/jira/browse/HDFS-12173
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.7.0
>Reporter: Daryn Sharp
>
> Sporadic test failures occur because {{NameNode#stop}} used by the mini
> cluster does not properly manage the HA context's state. It directly calls
> {{HAState#exitState(context)}} instead of
> {{HAState#setState(context,state)}}. The latter will properly lock the
> namesystem and update the ha state while locked, while the former does not.
> The result is that while the cluster is stopping, the lock is released and
> any queued rpc calls think the NN is still active and are processed while the
> NN is in an unstable half-stopped state.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12162) Update listStatus document to describe the behavior when the argument is a file
[ https://issues.apache.org/jira/browse/HDFS-12162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108081#comment-16108081 ] Ajay Yadav commented on HDFS-12162: --- Hi [~yzhangal], Are you referring to webhdfs documentation? Do you mean we should update webhdfs documentation to include an example for file path. Thanks!! > Update listStatus document to describe the behavior when the argument is a > file > --- > > Key: HDFS-12162 > URL: https://issues.apache.org/jira/browse/HDFS-12162 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs, httpfs >Reporter: Yongjun Zhang >Assignee: Ajay Yadav > > The listStatus method can take in either directory path or file path as > input, however, currently both the javadoc and external document describe it > as only taking directory as input. This jira is to update the document about > the behavior when the argument is a file path. > Thanks [~xiaochen] for the review and discussion in HDFS-12139, creating this > jira is the result of our discussion there. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-12162) Update listStatus document to describe the behavior when the argument is a file
[ https://issues.apache.org/jira/browse/HDFS-12162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav reassigned HDFS-12162: - Assignee: Ajay Yadav > Update listStatus document to describe the behavior when the argument is a > file > --- > > Key: HDFS-12162 > URL: https://issues.apache.org/jira/browse/HDFS-12162 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs, httpfs >Reporter: Yongjun Zhang >Assignee: Ajay Yadav > > The listStatus method can take in either directory path or file path as > input, however, currently both the javadoc and external document describe it > as only taking directory as input. This jira is to update the document about > the behavior when the argument is a file path. > Thanks [~xiaochen] for the review and discussion in HDFS-12139, creating this > jira is the result of our discussion there. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12222) Add EC information to BlockLocation
[
https://issues.apache.org/jira/browse/HDFS-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107835#comment-16107835
]
Ajay Yadav commented on HDFS-1:
---
Hi [~andrew.wang], For usages like FIF BlockLocation does it make more sense to
expose parity blocks and data blocks via different functions in
LocatedFileStatus? BlockLocation seems to be lower abstraction for
FileInputFormat.
{code} if (file instanceof LocatedFileStatus) {
blkLocations = ((LocatedFileStatus) file).getDataBlockLocations();
} else {
blkLocations = fs.getFileDataBlockLocations(file, 0, length);
}
{code}
> Add EC information to BlockLocation
> ---
>
> Key: HDFS-1
> URL: https://issues.apache.org/jira/browse/HDFS-1
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha1
>Reporter: Andrew Wang
> Labels: hdfs-ec-3.0-nice-to-have
>
> HDFS applications query block location information to compute splits. One
> example of this is FileInputFormat:
> https://github.com/apache/hadoop/blob/d4015f8628dd973c7433639451a9acc3e741d2a2/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java#L346
> You see bits of code like this that calculate offsets as follows:
> {noformat}
> long bytesInThisBlock = blkLocations[startIndex].getOffset() +
> blkLocations[startIndex].getLength() - offset;
> {noformat}
> EC confuses this since the block locations include parity block locations as
> well, which are not part of the logical file length. This messes up the
> offset calculation and thus topology/caching information too.
> Applications can figure out what's a parity block by reading the EC policy
> and then parsing the schema, but it'd be a lot better if we exposed this more
> generically in BlockLocation instead.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11458) testRateLimitingDuringDataNodeStartup UT fails with Timed out
[
https://issues.apache.org/jira/browse/HDFS-11458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107637#comment-16107637
]
Ajay Yadav commented on HDFS-11458:
---
Hi [~yeshavora] I can't reproduce this issue in OSx. If it is still failing
could you please provide more details to reproduce this issue.
> testRateLimitingDuringDataNodeStartup UT fails with Timed out
> -
>
> Key: HDFS-11458
> URL: https://issues.apache.org/jira/browse/HDFS-11458
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Yesha Vora
> Attachments: testRateLimitingDuringDataNodeStartup.log
>
>
> testRateLimitingDuringDataNodeStartup Test is failing as below.
> {code}
> Error Message
> Timed out waiting for condition. Thread diagnostics:
> Timestamp: 2017-02-20 09:17:15,298
> "IPC Server handler 8 on 40022" daemon prio=5 tid=909 timed_waiting
> java.lang.Thread.State: TIMED_WAITING
> at sun.misc.Unsafe.park(Native Method)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at
> org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:218)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2317)
> "IPC Server listener on 46888" daemon prio=5 tid=1204 runnable
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
> at org.apache.hadoop.ipc.Server$Listener.run(Server.java:906)
> "AsyncAppender-Dispatcher-Thread-32" daemon prio=5 tid=48 in Object.wait()
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:503)
> at
> org.apache.log4j.AsyncAppender$Dispatcher.run(AsyncAppender.java:548)
> at java.lang.Thread.run(Thread.java:722)
> "java.util.concurrent.ThreadPoolExecutor$Worker@1d1e71f9[State = -1, empty
> queue]" daemon prio=5 tid=1250 timed_waiting{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11454) TestUpdatePipelineWithSnapshots#testUpdatePipelineAfterDelete fails with bind exception
[
https://issues.apache.org/jira/browse/HDFS-11454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107624#comment-16107624
]
Ajay Yadav commented on HDFS-11454:
---
Hi [~brahmareddy], I can't reproduce this issue in OSx. If it is still failing
could you please provide more details to reproduce this issue.
> TestUpdatePipelineWithSnapshots#testUpdatePipelineAfterDelete fails with bind
> exception
> ---
>
> Key: HDFS-11454
> URL: https://issues.apache.org/jira/browse/HDFS-11454
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Reporter: Brahma Reddy Battula
>
> {noformat}
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:433)
> at sun.nio.ch.Net.bind(Net.java:425)
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at
> org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:317)
> at
> org.apache.hadoop.http.HttpServer2.bindListener(HttpServer2.java:1100)
> at
> org.apache.hadoop.http.HttpServer2.bindForSinglePort(HttpServer2.java:1131)
> at
> org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:1193)
> at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:1049)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:169)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:885)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:721)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:947)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:926)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1635)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:2080)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:2054)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots.testUpdatePipelineAfterDelete(TestUpdatePipelineWithSnapshots.java:100)
> {noformat}
> *reference*
> https://builds.apache.org/job/PreCommit-HDFS-Build/18434/testReport/
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
