Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

+1 on RC1 - Verified signatures - Built from source (RHEL7) - Ran single node cluster and tested basic HDFS commands Thanks for putting this together Jonathan! On Mon, Oct 28, 2019 at 7:54 PM zhankun tang wrote: > Thanks for the efforts! Jonathan! > > +1 (non-binding) on RC1. > - Set up a

Re: [VOTE] Moving Submarine to a separate Apache project proposal

My late +1. Me and my colleagues have been involved in Submarine development (in collaboration with our TonY project). I think Submarine will be valuable as a new Apache project. On Fri, Sep 6, 2019 at 10:57 PM Dinesh Chitlangia wrote: > +1 > > -Dinesh > > > > > On Fri, Sep 6, 2019 at 11:23 PM

Re: [VOTE] Merge YARN-8200 to branch-2 and branch-3.0

+1 (binding) As Jonathan said this feature in branch-2 has been running stably for over a year. Therefore I’m supportive to the merge On Mon, Aug 26, 2019 at 2:00 PM Jim Brennan wrote: > +1 (non-binding). > I have built branch-2 with the latest YARN-8200 patch > (YARN-8200-branch-2.003.patch).

Re: [VOTE] Moving branch-2 precommit/nightly test builds to java 8

/ > https://builds.apache.org/view/H-L/view/Hadoop/job/PreCommit-YARN-Build/ > > https://builds.apache.org/view/H-L/view/Hadoop/job/PreCommit-MAPREDUCE-Build/ > > [3] > > https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/ > > Jonathan

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

gt; Wangda Tan > > [1] > > https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E > > > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Re: [DISCUSS] Making submarine to different release model like Ozone

+1 on the proposal and looking forward to the progress of the project! On Thu, Jan 31, 2019 at 10:51 PM Weiwei Yang wrote: > Thanks for proposing this Wangda, my +1 as well. > It is amazing to see the progress made in Submarine last year, the > community grows fast and quiet collaborative. I

Re: [VOTE - 2] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

run for 7 days until Wed Dec 21. > > Thanks, > --Konstantin > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Re: [VOTE] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

s testing progresses, we continue fixing non-critical bugs like > HDFS-14116. > > I attached a unified patch to the umbrella jira for the review and Jenkins > build. > Please vote on this thread. The vote will run for 7 days until Wed Dec 12. > > Thanks, > --Konstantin > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Re: [VOTE] Release Apache Hadoop 2.7.6 (RC0)

ays > > ending 04/16/2018. > > > > My up to date public key is available from: > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > Thanks, > > --Konstantin > > > --------- > To

[jira] [Created] (HDFS-12888) NameNode web UI shows stale config values after cli refresh

Zhe Zhang created HDFS-12888: Summary: NameNode web UI shows stale config values after cli refresh Key: HDFS-12888 URL: https://issues.apache.org/jira/browse/HDFS-12888 Project: Hadoop HDFS

[jira] [Reopened] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-12502: -- > nntop should support a category based on FilesInGetListing

Re: [VOTE] Merge feature branch YARN-5734 (API based scheduler configuration) to trunk, branch-3.0, branch-2

entation are supported. > > There were 15 subtasks completed for this feature. > > Huge thanks to everyone who helped with reviews, commits, guidance, and > technical discussion/design, including Carlo Curino, Xuan Gong, Subru > Krishnan, Min Shen, Konstantin Shvachko, Carl Steinbach, Wan

[jira] [Created] (HDFS-12502) nntop should support category based on FilesInGetListingOps

Zhe Zhang created HDFS-12502: Summary: nntop should support category based on FilesInGetListingOps Key: HDFS-12502 URL: https://issues.apache.org/jira/browse/HDFS-12502 Project: Hadoop HDFS

[jira] [Created] (HDFS-12379) NameNode getListing should use FileStatus instead of HdfsFileStatus

Zhe Zhang created HDFS-12379: Summary: NameNode getListing should use FileStatus instead of HdfsFileStatus Key: HDFS-12379 URL: https://issues.apache.org/jira/browse/HDFS-12379 Project: Hadoop HDFS

[jira] [Created] (HDFS-12345) Scale testing HDFS NameNode

Zhe Zhang created HDFS-12345: Summary: Scale testing HDFS NameNode Key: HDFS-12345 URL: https://issues.apache.org/jira/browse/HDFS-12345 Project: Hadoop HDFS Issue Type: Bug Reporter

[jira] [Created] (HDFS-12284) Router support for Kerberos and delegation tokens

Zhe Zhang created HDFS-12284: Summary: Router support for Kerberos and delegation tokens Key: HDFS-12284 URL: https://issues.apache.org/jira/browse/HDFS-12284 Project: Hadoop HDFS Issue Type

Re: [RESULT] [VOTE] Release Apache Hadoop 2.7.4 (RC0)

s. > Thank you everybody for contributing to the release, testing it, and > voting. > > Binding +1s (7) > Zhe Zhang > Jason Lowe > Eric Payne > Sunil G > Akira Ajisaka > Chris Douglas > Konstantin Shvachko > > Non-binding +1s (13) > John Zhuge > Surendra Lilh

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

wrote: > >>> > >>> > > >>> > > On Jul 31, 2017, at 11:20 AM, Konstantin Shvachko < > >>> shv.had...@gmail.com> > >>> > wrote: > >>> > > > >>> > > https://wiki.apache.org/hadoop/HowToReleasePreDSBCR > >>> > > >>> > FYI: > >>> > > >>> > If you are using ASF Jenkins to create an ASF release > >>> > artifact, it's pretty much an automatic vote failure as any such > >>> release is > >>> > in violation of ASF policy. > >>> > > >>> > > >>> > >> > >> > > - > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

to make sure people don't > use > > > >>> Jenkins option for building. > > > >>> > > > >>> A side note. This particular build is broken anyways, so no worries > > > there. > > > >>> I think though it would be useful to have it working for testing > and > > > as a > > > >>> packaging standard. > > > >>> > > > >>> Thanks, > > > >>> --Konstantin > > > >>> > > > >>> On Mon, Jul 31, 2017 at 11:40 AM, Allen Wittenauer < > > > >>> a...@effectivemachines.com > > > >>> > wrote: > > > >>> > > > >>> > > > > >>> > > On Jul 31, 2017, at 11:20 AM, Konstantin Shvachko < > > > >>> shv.had...@gmail.com> > > > >>> > wrote: > > > >>> > > > > > >>> > > https://wiki.apache.org/hadoop/HowToReleasePreDSBCR > > > >>> > > > > >>> > FYI: > > > >>> > > > > >>> > If you are using ASF Jenkins to create an ASF > > release > > > >>> > artifact, it's pretty much an automatic vote failure as any such > > > >>> release is > > > >>> > in violation of ASF policy. > > > >>> > > > > >>> > > > > >>> > > > >> > > > >> > > > > > > > > > -- > > *Zhou, Ye **周晔* > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Re: About 2.7.4 Release

67 <https://issues.apache.org/jira/browse/YARN-4367>. SLS > YARN-4612 <https://issues.apache.org/jira/browse/YARN-4612>. SLS > > - Node labeling > MAPREDUCE-6304 <https://issues.apache.org/jira/browse/MAPREDUCE-6304> > YARN-2943 <https://issues.apache.org/jira/browse/YARN-2

[jira] [Created] (HDFS-11743) Revert HDFS-7933 from branch-2.7 (fsck reporting decommissioning replicas)

Zhe Zhang created HDFS-11743: Summary: Revert HDFS-7933 from branch-2.7 (fsck reporting decommissioning replicas) Key: HDFS-11743 URL: https://issues.apache.org/jira/browse/HDFS-11743 Project: Hadoop

[jira] [Created] (HDFS-11732) Backport HDFS-8498 to branch-2.7: Blocks can be committed with wrong size

Zhe Zhang created HDFS-11732: Summary: Backport HDFS-8498 to branch-2.7: Blocks can be committed with wrong size Key: HDFS-11732 URL: https://issues.apache.org/jira/browse/HDFS-11732 Project: Hadoop HDFS

[jira] [Reopened] (HDFS-9005) Provide configuration support for upgrade domain

[ https://issues.apache.org/jira/browse/HDFS-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-9005: - > Provide configuration support for upgrade dom

[jira] [Reopened] (HDFS-8873) Allow the directoryScanner to be rate-limited

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-8873: - > Allow the directoryScanner to be rate-limi

[jira] [Created] (HDFS-11709) StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully

Zhe Zhang created HDFS-11709: Summary: StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully Key: HDFS-11709 URL: https://issues.apache.org/jira/browse/HDFS-11709 Project: Hadoop

[jira] [Reopened] (HDFS-8872) Reporting of missing blocks is different in fsck and namenode ui/metasave

[ https://issues.apache.org/jira/browse/HDFS-8872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-8872: - > Reporting of missing blocks is different in fsck and namenode ui/metas

[jira] [Resolved] (HDFS-8872) Reporting of missing blocks is different in fsck and namenode ui/metasave

[ https://issues.apache.org/jira/browse/HDFS-8872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8872. - Resolution: Duplicate > Reporting of missing blocks is different in fsck and namenode ui/metas

[jira] [Resolved] (HDFS-8015) Erasure Coding: local and remote block writer for coding work in DataNode

[ https://issues.apache.org/jira/browse/HDFS-8015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8015. - Resolution: Duplicate Resolving this issue since HDFS-9719 already introduced {{StripedBlockWriter

[jira] [Resolved] (HDFS-8014) Erasure Coding: local and remote block reader for coding work in DataNode

[ https://issues.apache.org/jira/browse/HDFS-8014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8014. - Resolution: Duplicate Resolving the issue as HDFS-9719 already creates {{StripedBlockReader}} > Eras

[jira] [Created] (HDFS-11345) Document the configuration key for FSNamesystem lock fairness

Zhe Zhang created HDFS-11345: Summary: Document the configuration key for FSNamesystem lock fairness Key: HDFS-11345 URL: https://issues.apache.org/jira/browse/HDFS-11345 Project: Hadoop HDFS

[jira] [Created] (HDFS-11157) Enhance documentation around cpLock

Zhe Zhang created HDFS-11157: Summary: Enhance documentation around cpLock Key: HDFS-11157 URL: https://issues.apache.org/jira/browse/HDFS-11157 Project: Hadoop HDFS Issue Type: Improvement

Jenkins patch-compile fails on trunk

https://builds.apache.org/job/PreCommit-HDFS-Build/17517/artifact/patchprocess/patch-compile-root.txt Looks related to yarn-ui? The HDFS-10872 patch doesn't touch YARN, but the patch-compile failed at YARN. Thanks, Zhe

Re: Multiple node Hadoop cluster on Docker for test and debugging

uster/blob/master/README.md> > > I would be glad if you use this tool for developing and debugging and make > our development more efficient. > Please give me any feedbacks to me. Thanks you! > > > Kai Sasaki > mail: lewua...@me.com <mailto:lewua...@me.com> > github: https://github.com/Lewuathe <https://github.com/Lewuathe> > > > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Re: Updated 2.8.0-SNAPSHOT artifact

> https://repository.apache.org/content/repositories/snapshots/org/ap > >>>> ach > >>>> e/hadoop/hadoop-yarn-api/ > >>>> > >>>> Are there any plans to start producing updated SNAPSHOT artifacts > >>>> for current hadoop d

[jira] [Reopened] (HDFS-7964) Add support for async edit logging

[ https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-7964: - I think this'd be a very useful addition to branch-2.8 and branch-2.7. Especially since the patch has been

[jira] [Resolved] (HDFS-1499) mv the namenode NameSpace and BlocksMap to hbase to save the namenode memory

[ https://issues.apache.org/jira/browse/HDFS-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-1499. - Resolution: Duplicate Resolving the old JIRA since many similar JIRAs have been raised, including HDFS

[jira] [Created] (HDFS-11051) Test Balancer behavior when some block moves are slow

Zhe Zhang created HDFS-11051: Summary: Test Balancer behavior when some block moves are slow Key: HDFS-11051 URL: https://issues.apache.org/jira/browse/HDFS-11051 Project: Hadoop HDFS Issue Type

Re: Listing large directories via WebHDFS

Andrew did some pagination features for webhdfs/httpfs via > https://issues.apache.org/jira/browse/HDFS-10784 and > https://issues.apache.org/jira/browse/HDFS-10823, to provide better > control. > > Best, > > -Xiao > > On Wed, Oct 19, 2016 at 2:08 PM, Zhe Zha

Listing large directories via WebHDFS

Hi, The regular HDFS client (DistributedFileSystem) throttles the workload of listing large directories by dividing the work into batches, something like below: {code} // fetch the first batch of entries in the directory DirectoryListing thisListing = dfs.listPaths( src,

[jira] [Created] (HDFS-10977) Balancer should query NameNode with a timeout

Zhe Zhang created HDFS-10977: Summary: Balancer should query NameNode with a timeout Key: HDFS-10977 URL: https://issues.apache.org/jira/browse/HDFS-10977 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-10967) Add configuration for BlockPlacementPolicy to deprioritize near-full DataNodes

Zhe Zhang created HDFS-10967: Summary: Add configuration for BlockPlacementPolicy to deprioritize near-full DataNodes Key: HDFS-10967 URL: https://issues.apache.org/jira/browse/HDFS-10967 Project: Hadoop

[jira] [Created] (HDFS-10966) Enhance Dispatcher logic on deciding when to give up a source DataNode

Zhe Zhang created HDFS-10966: Summary: Enhance Dispatcher logic on deciding when to give up a source DataNode Key: HDFS-10966 URL: https://issues.apache.org/jira/browse/HDFS-10966 Project: Hadoop HDFS

[jira] [Reopened] (HDFS-10745) Directly resolve paths into INodesInPath

[ https://issues.apache.org/jira/browse/HDFS-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-10745: -- Sorry to reopen the JIRA, testing branch-2.7 patch. > Directly resolve paths into INodesInP

[jira] [Resolved] (HDFS-10859) TestBalancer#testUnknownDatanodeSimple and testBalancerWithKeytabs are flaky in branch-2.7

[ https://issues.apache.org/jira/browse/HDFS-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-10859. -- Resolution: Duplicate Thanks for the pointer Xiao. HDFS-10716 indeed solves the problem

[jira] [Reopened] (HDFS-10744) Internally optimize path component resolution

[ https://issues.apache.org/jira/browse/HDFS-10744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-10744: -- Sorry for re-opening this. Triggering Jenkins for branch-2.7 patch. > Internally optimize path compon

[jira] [Reopened] (HDFS-10673) Optimize FSPermissionChecker's internal path usage

[ https://issues.apache.org/jira/browse/HDFS-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-10673: -- Sorry to reopen the JIRA. I want to test the branch-2.7 patch on Jenkins. > Optimize FSPermissionChecke

[jira] [Created] (HDFS-10859) TestBalancer#testUnknownDatanodeSimple and testBalancerWithKeytabs fail in branch-2.7

Zhe Zhang created HDFS-10859: Summary: TestBalancer#testUnknownDatanodeSimple and testBalancerWithKeytabs fail in branch-2.7 Key: HDFS-10859 URL: https://issues.apache.org/jira/browse/HDFS-10859 Project

[jira] [Reopened] (HDFS-8818) Allow Balancer to run faster

[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-8818: - Sorry to reopen this one. I think this is a valid improvement for branch-2.7 and I'm trying to backport

[jira] [Created] (HDFS-10854) Remove createStripedFile and addBlockToFile by creating real EC files

Zhe Zhang created HDFS-10854: Summary: Remove createStripedFile and addBlockToFile by creating real EC files Key: HDFS-10854 URL: https://issues.apache.org/jira/browse/HDFS-10854 Project: Hadoop HDFS

[jira] [Reopened] (HDFS-10662) Optimize UTF8 string/byte conversions

[ https://issues.apache.org/jira/browse/HDFS-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-10662: -- Sorry to reopen the JIRA. I'm backporting to branch-2.7 and it was quite messy. [~daryn] [~kihwal] I'd

[jira] [Reopened] (HDFS-10655) Fix path related byte array conversion bugs

[ https://issues.apache.org/jira/browse/HDFS-10655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-10655: -- Sorry to reopen it, I want to backport it to branch-2.7 and have a full Jenkins run before doing

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

+1 (non-binding) Did the following on 7 RHEL 6.6 servers - Downloaded and built from source - Downloaded and verified checksum of the binary tar.gz file - Setup a cluster with 1 NN and 6 DNs - Tried regular HDFS commands - Tried EC commands (listPolicies, getPolicy, setPolicy), they work fine -

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

Thanks Andrew for the great work! It's really exciting to finally see a Hadoop 3 RC. I noticed CHANGES and RELEASENOTES markdown files which were not in previous RCs like 2.7.3. What are good tools to verify them? I tried reading them on IntelliJ but format looks odd. I'm still testing the RC: -

[jira] [Created] (HDFS-10809) getNumEncryptionZones causes NPE in branch-2.7

Zhe Zhang created HDFS-10809: Summary: getNumEncryptionZones causes NPE in branch-2.7 Key: HDFS-10809 URL: https://issues.apache.org/jira/browse/HDFS-10809 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-10798) Make the threshold of reporting FSNamesystem lock contention configurable

Zhe Zhang created HDFS-10798: Summary: Make the threshold of reporting FSNamesystem lock contention configurable Key: HDFS-10798 URL: https://issues.apache.org/jira/browse/HDFS-10798 Project: Hadoop HDFS

[jira] [Reopened] (HDFS-7933) fsck should also report decommissioning replicas.

[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-7933: - > fsck should also report decommissioning repli

Re: NFS Gateway Performance

t; NFS Server , all data is going through Single Machine So this causing > performance issue .. I tried with small data set in case of huge data set > how it will perform ?? > > > Direct HDFS put gives better performance .. Any Suggestion ? > > > Regards, > Senthil

Re: HDFS Balancer - Balancing Effectively ?

Thanks Nicholas for the pointers; the first 2 links are not working. On Thu, Aug 11, 2016 at 10:09 AM Tsz Wo Sze wrote: > Hi Senthi > The Balancer performance was improved dramatically recently [1]. I am not > sure if you know about the new conf and parameters; see

[jira] [Reopened] (HDFS-9804) Allow long-running Balancer to login with keytab

[ https://issues.apache.org/jira/browse/HDFS-9804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-9804: - > Allow long-running Balancer to login with key

Re: HDFS Balancer - Recommended Threshold Value

hold ?? Is there any way to pass param > only to move blocks from Over Utilized ( 98-100%) to under utilized ? > > > Pls advise! > > > > > Regards, > Senthil > -- Zhe Zhang Apache Hadoop Commiter http://zhe-thoughts.github.io/about/ | @oldcap

[jira] [Created] (HDFS-10680) TestUTF8 fails in branch-2.6

Zhe Zhang created HDFS-10680: Summary: TestUTF8 fails in branch-2.6 Key: HDFS-10680 URL: https://issues.apache.org/jira/browse/HDFS-10680 Project: Hadoop HDFS Issue Type: Bug

[jira] [Resolved] (HDFS-10653) Optimize conversion from path string to components

[ https://issues.apache.org/jira/browse/HDFS-10653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-10653. -- Resolution: Fixed Local test-patch shows 2 failures but those tests fail even without the patch. I'll

[jira] [Reopened] (HDFS-10653) Optimize conversion from path string to components

[ https://issues.apache.org/jira/browse/HDFS-10653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-10653: -- Sorry, have to reopen the issue otherwise Jenkins won't run. > Optimize conversion from path str

[jira] [Reopened] (HDFS-10534) NameNode WebUI should display DataNode usage rate with a certain percentile

[ https://issues.apache.org/jira/browse/HDFS-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-10534: -- > NameNode WebUI should display DataNode usage rate with a certain percent

[jira] [Created] (HDFS-10534) NameNode WebUI should display DataNode usage rate with a certain percentile

Zhe Zhang created HDFS-10534: Summary: NameNode WebUI should display DataNode usage rate with a certain percentile Key: HDFS-10534 URL: https://issues.apache.org/jira/browse/HDFS-10534 Project: Hadoop

Re: [DISCUSS] Increased use of feature branches

Thanks for the notes Andrew, Junping, Karthik. Here are some of my understandings: - Trunk is the "latest and greatest" of Hadoop. If a user starts using Hadoop today, without legacy workloads, trunk is what he/she should use. - Therefore, each commit to trunk should be transactional -- atomic,

[jira] [Created] (HDFS-10458) getFileEncryptionInfo should return quickly for non-encrypted cluster

Zhe Zhang created HDFS-10458: Summary: getFileEncryptionInfo should return quickly for non-encrypted cluster Key: HDFS-10458 URL: https://issues.apache.org/jira/browse/HDFS-10458 Project: Hadoop HDFS

Re: Rise of the Dragon

Seems like someone tried to "clone" all erasure coding JIRAs. On Wed, Mar 16, 2016 at 12:30 PM Colin P. McCabe wrote: > Who is creating all these dragon JIRAs? > > Colin >

[jira] [Created] (HDFS-9844) Correct path creation in getTrashRoot to handle root dir

Zhe Zhang created HDFS-9844: --- Summary: Correct path creation in getTrashRoot to handle root dir Key: HDFS-9844 URL: https://issues.apache.org/jira/browse/HDFS-9844 Project: Hadoop HDFS Issue Type

Re: Looking to a Hadoop 3 release

Thanks Andrew for driving the effort! +1 (non-binding) on starting the 3.0 release process now with 3.0 as an alpha. I wanted to echo Andrew's point that backporting EC to branch-2 is a lot of work. Considering that no concrete backporting plan has been proposed, it seems quite uncertain whether

[jira] [Created] (HDFS-9799) Reimplement getCurrentTrashDir to remove incompatibility

Zhe Zhang created HDFS-9799: --- Summary: Reimplement getCurrentTrashDir to remove incompatibility Key: HDFS-9799 URL: https://issues.apache.org/jira/browse/HDFS-9799 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-9785) Remove unused TrashPolicy#getInstance code

Zhe Zhang created HDFS-9785: --- Summary: Remove unused TrashPolicy#getInstance code Key: HDFS-9785 URL: https://issues.apache.org/jira/browse/HDFS-9785 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Resolved] (HDFS-9172) Erasure Coding: Move DFSStripedIO stream related classes to hadoop-hdfs-client

[ https://issues.apache.org/jira/browse/HDFS-9172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-9172. - Resolution: Invalid > Erasure Coding: Move DFSStripedIO stream related classes to hadoop-hdfs-cli

[jira] [Created] (HDFS-9698) Long running Balancer should renew TGT

Zhe Zhang created HDFS-9698: --- Summary: Long running Balancer should renew TGT Key: HDFS-9698 URL: https://issues.apache.org/jira/browse/HDFS-9698 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-9688) Test the effect of nested encryption zones in HDFS downgrade

Zhe Zhang created HDFS-9688: --- Summary: Test the effect of nested encryption zones in HDFS downgrade Key: HDFS-9688 URL: https://issues.apache.org/jira/browse/HDFS-9688 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-9644) Update encryption documentation to reflect nested EZs

Zhe Zhang created HDFS-9644: --- Summary: Update encryption documentation to reflect nested EZs Key: HDFS-9644 URL: https://issues.apache.org/jira/browse/HDFS-9644 Project: Hadoop HDFS Issue Type

Re: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

as Elliot stated, this will delay the upgrade to 2.9 for some customers. On Tue, Dec 8, 2015 at 2:54 PM, Zhe Zhang <zhezh...@cloudera.com> wrote: > Hi Vinod, > > Thanks for the update. After seeing the comment from Elliot I checked with > Ming Ma and Daryn Sharp offline, to get f

[jira] [Created] (HDFS-9576) HTrace: collect path/offset/length information on read and write operations

Zhe Zhang created HDFS-9576: --- Summary: HTrace: collect path/offset/length information on read and write operations Key: HDFS-9576 URL: https://issues.apache.org/jira/browse/HDFS-9576 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-9578) Incorrect default value (typo) in hdfs-default.xml

[ https://issues.apache.org/jira/browse/HDFS-9578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-9578. - Resolution: Duplicate Thanks for reporting this [~tianyin]. It has been fixed in trunk already

[jira] [Resolved] (HDFS-7691) Handle hflush and hsync in the best optimal way possible during online Erasure encoding

[ https://issues.apache.org/jira/browse/HDFS-7691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7691. - Resolution: Duplicate Thanks Vinay for confirming. Let's track the hflush related efforts under HDFS-7661

[jira] [Resolved] (HDFS-9529) Extend Erasure Code to support POWER Chip acceleration

[ https://issues.apache.org/jira/browse/HDFS-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-9529. - Resolution: Duplicate Thanks Qijun for confirming this. Resolving this one as dup. > Extend Erasure C

Re: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

having some questions about the stability of EC code. Maybe we should give Ming/Daryn some more time to reply to this thread? I think we should also address Elliot's comments above regarding major vs. minor releases, how that impacts the 2.9 upgrade timing etc. Thanks, --- Zhe Zhang On Tue, Dec 8

[jira] [Resolved] (HDFS-9496) Erasure coding: an erasure codec throughput benchmark tool

[ https://issues.apache.org/jira/browse/HDFS-9496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-9496. - Resolution: Duplicate Thanks Rui for clarifying this. > Erasure coding: an erasure codec through

[jira] [Created] (HDFS-9403) Erasure coding: some EC tests are missing timeout

Zhe Zhang created HDFS-9403: --- Summary: Erasure coding: some EC tests are missing timeout Key: HDFS-9403 URL: https://issues.apache.org/jira/browse/HDFS-9403 Project: Hadoop HDFS Issue Type: Sub

Update on potential Gerrit integration

Hi, Per our earlier discussion on the "Github integration for Hadoop" thread, I surveyed HBase, Flume, and HTrace about their interest on Gerrit. The feedbacks have been positive: HBase:

[jira] [Created] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

Zhe Zhang created HDFS-9405: --- Summary: When starting a file, NameNode should generate EDEK in a separate thread Key: HDFS-9405 URL: https://issues.apache.org/jira/browse/HDFS-9405 Project: Hadoop HDFS

[jira] [Created] (HDFS-9386) Erasure coding: updateBlockForPipeline sometimes returns non-striped block for striped file

Zhe Zhang created HDFS-9386: --- Summary: Erasure coding: updateBlockForPipeline sometimes returns non-striped block for striped file Key: HDFS-9386 URL: https://issues.apache.org/jira/browse/HDFS-9386

Re: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

To add to the above discussion on umbrella JIRA: the COMMON side changes of EC have been tracked under HADOOP-11264 (before merge) and HADOOP-11842 (after merge). --- Zhe Zhang On Tue, Nov 3, 2015 at 4:40 PM, Vinod Vavilapalli <vino...@hortonworks.com> wrote: > That makes sense. &

Re: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

Thanks Vinod for the proposal and Andrew/Jing for the comments. 2.9 sounds a good plan. As Andrew pointed out, 2.8 is already quite big. And even when disabled, EC logic has been baked in NN pretty deeply. Do we have a tentative date or estimate for 2.9? --- Zhe Zhang On Mon, Nov 2, 2015 at 1

[jira] [Created] (HDFS-9344) QUEUE_WITH_CORRUPT_BLOCKS is no longer needed

Zhe Zhang created HDFS-9344: --- Summary: QUEUE_WITH_CORRUPT_BLOCKS is no longer needed Key: HDFS-9344 URL: https://issues.apache.org/jira/browse/HDFS-9344 Project: Hadoop HDFS Issue Type: Bug

[jira] [Resolved] (HDFS-9344) QUEUE_WITH_CORRUPT_BLOCKS is no longer needed

[ https://issues.apache.org/jira/browse/HDFS-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-9344. - Resolution: Invalid Thanks Xiao! Good analysis! Closing this as invalid. > QUEUE_WITH_CORRUPT_BLO

[jira] [Created] (HDFS-9329) TestBootstrapStandby#testRateThrottling is flaky because fsimage size is smaller than IO buffer size

Zhe Zhang created HDFS-9329: --- Summary: TestBootstrapStandby#testRateThrottling is flaky because fsimage size is smaller than IO buffer size Key: HDFS-9329 URL: https://issues.apache.org/jira/browse/HDFS-9329

[jira] [Created] (HDFS-9280) Document NFS gateway export point parameter

Zhe Zhang created HDFS-9280: --- Summary: Document NFS gateway export point parameter Key: HDFS-9280 URL: https://issues.apache.org/jira/browse/HDFS-9280 Project: Hadoop HDFS Issue Type: Improvement

Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk

Thanks Vinay for capturing the issue and Uma for offering the help. --- Zhe Zhang On Mon, Oct 5, 2015 at 12:19 PM, Gangumalla, Uma <uma.ganguma...@intel.com> wrote: > Vinay, > > > I would merge them as part of HDFS-9182. > > Thanks, > Uma > > > &g

[jira] [Resolved] (HDFS-7285) Erasure Coding Support inside HDFS

[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7285. - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 I just did a final {{git merge

Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk

Thanks everyone who has participated in this discussion. With 7 +1's (5 binding and 2 non-binding), and no -1, this vote has passed. I will do a final 'git merge' with trunk and work with Andrew to merge the branch to trunk. I'll update on this thread when the merge is done. --- Zhe Zhang

[jira] [Created] (HDFS-9119) Discrepancy between edit log tailing interval and RPC timeout for transitionToActive

Zhe Zhang created HDFS-9119: --- Summary: Discrepancy between edit log tailing interval and RPC timeout for transitionToActive Key: HDFS-9119 URL: https://issues.apache.org/jira/browse/HDFS-9119 Project

[jira] [Created] (HDFS-9097) Erasure coding: update EC command "-s" flag to "-p" when specifying policy

Zhe Zhang created HDFS-9097: --- Summary: Erasure coding: update EC command "-s" flag to "-p" when specifying policy Key: HDFS-9097 URL: https://issues.apache.org/jira/browse/HDFS-9097

[jira] [Created] (HDFS-9098) Erasure coding: emulate race conditions among striped streamers in write pipeline

Zhe Zhang created HDFS-9098: --- Summary: Erasure coding: emulate race conditions among striped streamers in write pipeline Key: HDFS-9098 URL: https://issues.apache.org/jira/browse/HDFS-9098 Project: Hadoop

[jira] [Resolved] (HDFS-8373) Ec files can't be deleted into Trash because of that Trash isn't EC zone.

[ https://issues.apache.org/jira/browse/HDFS-8373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8373. - Resolution: Not A Problem With HDFS-8833 we should be able to delete EC files into Trash. > Ec fi

  1   2   3   >