[jira] [Resolved] (HDFS-17181) WebHDFS not considering whether a DN is good when called from outside the cluster

[ https://issues.apache.org/jira/browse/HDFS-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-17181. Fix Version/s: 3.5.0 Resolution: Fixed > WebHDFS not considering whether a

[jira] [Resolved] (HDFS-17024) Potential data race introduced by HDFS-15865

[ https://issues.apache.org/jira/browse/HDFS-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-17024. Fix Version/s: 3.4.0 Resolution: Fixed > Potential data race introduced by H

Join us at the Storage User Group Meetup!

Hi Please join us at the Storage Meetup at Cloudera's office next Wednesday. https://www.meetup.com/futureofdata-sanfrancisco/events/295917033/ We have HDFS developers from Uber join us to talk about optimizing HDFS for high density disks, and developers from Cloudera to talk about Apache Ozone

HADOOP-18207 hadoop-logging module about to land

Hi, I am preparing to resolve HADOOP-18207 ( https://github.com/apache/hadoop/pull/5717). This change affects all modules. With this change, it will eliminate almost all the direct log4j usage. As always, landing such a big piece is tricky. I

Re: [DISCUSS][HDFS] Add rust binding for libhdfs

Inline On Sat, Jul 15, 2023 at 5:04 AM Ayush Saxena wrote: > Forwarding from dev@hadoop to relevant ML > > Original mail: > https://lists.apache.org/thread/r5rcmc7lwwvkysj0320myxltsyokp9kq > > -Ayush > > On 2023/07/15 09:18:42 Xuanwo wrote: > > Hello, everyone. > > > > I'm the maintainer of

[jira] [Created] (HDFS-17080) Fix ec connection leak (GitHub PR#5807)

Wei-Chiu Chuang created HDFS-17080: -- Summary: Fix ec connection leak (GitHub PR#5807) Key: HDFS-17080 URL: https://issues.apache.org/jira/browse/HDFS-17080 Project: Hadoop HDFS Issue Type

[ANNOUNCE] Apache Hadoop 3.3.6 release

On behalf of the Apache Hadoop Project Management Committee, I am pleased to announce the release of Apache Hadoop 3.3.6. It contains 117 bug fixes, improvements and enhancements since 3.3.5. Users of Apache Hadoop 3.3.5 and earlier should upgrade to this release.

Re: [VOTE] Release Apache Hadoop 3.3.6 RC1

> -Ayush > > > [1] https://github.com/ayushtkn/hadoop-ozone/actions/runs/5282707769 > [2] https://github.com/apache/tez/pull/285#issuecomment-1590962978 > > On Sat, 24 Jun 2023 at 09:43, Nilotpal Nandi > wrote: > >> +1 (Non-binding). >> Thanks a lot Wei-Chiu

Re: [VOTE] Release Apache Hadoop 3.3.6 RC1

hed a pseudo distributed cluster using released binary packages, > > done some HDFS dir/file basic opeations. > > + Run grep, pi and wordcount MR tasks on the pseudo cluster. > > > > Bests, > > Sammi Chen > > > > 发件人: Wei-C

Clean up old Hadoop release tarballs

https://dist.apache.org/repos/dist/release/hadoop/common/ has Hadoop release tarballs 3.3.1 ~ 3.3.5. I plan to remove the tarballs from 3.3.1 to 3.3.4 and leave only 3.3.5 (and the upcoming 3.3.6). Shout out if you have something depending on the old release tarballs (you shouldn't) Other release

DockerHub admin for Apache Hadoop

Ayush and I have acquired the DockerHub admin privilege for the Hadoop project in order to facilitate the release of Hadoop 3.3.6. Apache Infra allows only two seats per project. So if you need something, let Ayush and I know and we will make it happen for you. If you are a Docker guru and can't

Re: [VOTE] Release Apache Hadoop 3.3.6 RC1

built from the source tarball on Rocky Linux 8 (x86_64) and OpenJDK > 8 > >> > with native profile enabled. > >> >+ launched pseudo distributed cluster including kms and httpfs with > >> > Kerberos and SSL enabled. > >> >+ created encryption

Re: [VOTE] Release Apache Hadoop 3.3.6 RC1

Bumping this thread to the top. If you are verifying the release, please vote on this thread. RC0 and RC1 are exactly the same. The only material difference is the Changelog. Thanks!! On Sun, Jun 18, 2023 at 5:52 PM Wei-Chiu Chuang wrote: > I am inviting anyone to try and vote on this rele

[VOTE] Release Apache Hadoop 3.3.6 RC1

I am inviting anyone to try and vote on this release candidate. Note: This is exactly the same as RC0, except the CHANGELOG. The RC is available at: https://home.apache.org/~weichiu/hadoop-3.3.6-RC1-amd64/ (for amd64) https://home.apache.org/~weichiu/hadoop-3.3.6-RC1-arm64/ (for arm64) Git tag:

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

ill be prepared, > right? > If true, I would like to try and vote on the next RC. > Just notice that some JIRAs are not included and need to revert some PRs to > pass HBase verification which are mentioned above. > > Best Regards, > - He Xiaoqiao > > > On Fri, Jun

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

ey for gs://fake-in-memory-test-b ucket/contract-test/testOpenFileApplyAsyncRead "fs.option.openfile.length" On Wed, Jun 14, 2023 at 5:01 PM Wei-Chiu Chuang wrote: > The hbase-filesystem tests passed after reverting HADOOP-18596 > <https://issues.apache.org/jira/b

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

It's branching off branch-3.3 On Thu, Jun 15, 2023 at 3:18 AM Steve Loughran wrote: > Which branch is -3.3.6 off? 3.3.5 or 3.3? > > I'm travelling for the next few days and unlikely to be able to test this; > will do my best > > On Wed, 14 Jun 2023 at 07:27, Wei-Chiu Chuang

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

not the end of the world. I think we can address it by adding an incompatible change flag and a release note. On Wed, Jun 14, 2023 at 3:55 PM Wei-Chiu Chuang wrote: > Cross referenced git history and jira. Changelog needs some update > > Not in the release > >1. HDFS-16858 <https:/

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

59->AbstractContractDistCpTest.distCpUpdateDeepDirectoryStructure:334->AbstractContractDistCpTest.assertCounterInRange:294->Assert.assertTrue:41->Assert.fail:88 Files Copied value 2 above maximum 1 [INFO] [ERROR] Tests run: 240, Failures: 3, Errors: 0, Skipped: 58 Ozone test in progress. Wil

[VOTE] Release Apache Hadoop 3.3.6 RC0

I am inviting anyone to try and vote on this release candidate. Note: This is built off branch-3.3.6 plus PR#5741 (aws sdk update) and PR#5740 (LICENSE file update) The RC is available at: https://home.apache.org/~weichiu/hadoop-3.3.6-RC0-amd64/ (for amd64)

Re: [DISCUSS] Hadoop 3.3.6 release planning

setSafeMode() is exposed through SafeMode interface. On Thu, Jun 8, 2023 at 11:11 AM Wei-Chiu Chuang wrote: > Thanks for comments > > Looking at jiras fixed in 3.3.6 and 3.3.9 (my bad, forgot that most > commits landing in branch-3.3 was 3.3.9), most are okay. We have about 119 >

Fwd: [jira] [Created] (HADOOP-18768) Integrating Apache Hadoop into OSS-Fuzz

Are there Hadoop committers who would like to help triage bug reports from OSS-Fuzz? -- Forwarded message - From: Henry Lin (Jira) Date: Mon, Jun 12, 2023 at 9:10 AM Subject: [jira] [Created] (HADOOP-18768) Integrating Apache Hadoop into OSS-Fuzz To: Henry Lin created

Re: [DISCUSS] Mysql-connector-java is GPL licensed; Hadoop can't use it.

; > .. Owen > > > On Fri, Jun 9, 2023 at 1:58 AM Wei-Chiu Chuang > > wrote: > > > Hi community, > > > > While preparing for 3.3.6 RC, I realized the mysql-connector-java > > dependency added by HADOOP-18535 > > <https://issues.apache.org/jira/b

[DISCUSS] Mysql-connector-java is GPL licensed; Hadoop can't use it.

Hi community, While preparing for 3.3.6 RC, I realized the mysql-connector-java dependency added by HADOOP-18535 is GPL licensed. Source: https://github.com/mysql/mysql-connector-j/blob/release/8.0/LICENSE See legal discussion at Apache

Re: [DISCUSS] Hadoop 3.3.6 release planning

gt; > Cherry-pick YARN-11482 to branch-3.3 and mark 3.3.6 as the fixed > version. > > > > > > > > so far only 8 jiras were resolved in the branch-3.3 line. > > > > > > > > > > > > If we should consider both 3.3.6 and 3.3.9 (which is from > r

[jira] [Created] (HDFS-17024) Potential data race introduced by HDFS-15865

Wei-Chiu Chuang created HDFS-17024: -- Summary: Potential data race introduced by HDFS-15865 Key: HDFS-17024 URL: https://issues.apache.org/jira/browse/HDFS-17024 Project: Hadoop HDFS Issue

Re: Call for Presentations, Community Over Code 2023

There's also a call for presentation for Community over Code Asia 2023 https://www.bagevent.com/event/cocasia-2023-EN Happening Aug 18-20. CfP due by 6/6 On Tue, May 9, 2023 at 8:39 PM Ayush Saxena wrote: > Forwarding from dev@hadoop to the dev ML which we use. > > The actual mail lies here:

[DISCUSS] Hadoop 3.3.6 release planning

Hi community, I'd like to kick off the discussion around Hadoop 3.3.6 release plan. I'm being selfish but my intent for 3.3.6 is to have the new APIs in HADOOP-18671 added so we can have HBase to adopt this new API. Other than that, perhaps

Re: [DISCUSS] hadoop branch-3.3+ going to java11 only

My random thoughts. Probably bad takes: There are projects experimenting with JDK17 now. JDK11 active support will end in 6 months. If it's already hard to migrate from JDK8 why not retarget JDK17. On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena wrote: > I know Jersey upgrade as a blocker. Some

Re: [DISCUSS] Move HDFS specific APIs to FileSystem abstration

I think moving up interfaces to FileSystem or some abstract FileSystem class has a few benefits: 1. Application can potentially be made FS-agnostic, with hasPathCapabilities() check. At least, make the code to compile. 2. We will be able to add a contract test to ensure behavior is expected. The

Re: [DISCUSS] Move HDFS specific APIs to FileSystem abstration

of us who can build the > entire stack can do it, but that gradually adds more divergence from the > open source libraries, reduces the test coverage overall and only increases > maintenance costs over time. > > steve > > On Thu, 16 Mar 2023 at 20:56, Wei-Chiu Chuang wrote:

[DISCUSS] Move HDFS specific APIs to FileSystem abstration

Hi, Stephen and I are working on a project to make HBase to run on Ozone. HBase, born out of the Hadoop project, depends on a number of HDFS specific APIs, including recoverLease() and isInSafeMode(). The HBase community [1] strongly voiced that they don't want the project to have direct

[jira] [Resolved] (HDFS-16947) RBF NamenodeHeartbeatService to report error for not being able to register namenode in state store

[ https://issues.apache.org/jira/browse/HDFS-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16947. Resolution: Fixed > RBF NamenodeHeartbeatService to report error for not being a

Re: Request to create an ASF Jira account

I've submitted a request to create the account. You should receive an email shortly. On Tue, Feb 14, 2023 at 1:40 PM ravindra dingankar < ravindradingan...@gmail.com> wrote: > Hi, > > I am part of LinkedIn's HDFS team, and would like to start contributing to > HDFS and be part of the mailing

[jira] [Resolved] (HDFS-16873) FileStatus compareTo does not specify ordering

[ https://issues.apache.org/jira/browse/HDFS-16873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16873. Fix Version/s: 3.4.0 Resolution: Fixed > FileStatus compareTo does not spec

[jira] [Resolved] (HDFS-16871) DiskBalancer process may throws IllegalArgumentException when the target DataNode has capital letter in hostname

[ https://issues.apache.org/jira/browse/HDFS-16871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16871. Fix Version/s: 3.4.0 Resolution: Fixed > DiskBalancer process may thr

[jira] [Resolved] (HDFS-16854) TestDFSIO to support non-default file system

[ https://issues.apache.org/jira/browse/HDFS-16854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16854. Resolution: Duplicate > TestDFSIO to support non-default file sys

[jira] [Resolved] (HDFS-16839) It should consider EC reconstruction work when we determine if a node is busy

[ https://issues.apache.org/jira/browse/HDFS-16839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16839. Resolution: Fixed > It should consider EC reconstruction work when we determine if a n

[jira] [Created] (HDFS-16854) TestDFSIO to support non-default file system

Wei-Chiu Chuang created HDFS-16854: -- Summary: TestDFSIO to support non-default file system Key: HDFS-16854 URL: https://issues.apache.org/jira/browse/HDFS-16854 Project: Hadoop HDFS Issue

Re: Code coverage report on github PRs

I believe most of them can be added by us using GitHub Workflow. There's a marketplace for these tools and most of them are free for open source projects. On Wed, Nov 23, 2022 at 11:43 AM Ayush Saxena wrote: > A simple Infra ticket I suppose should get it done for us, eg. >

[jira] [Resolved] (HDFS-9536) OOM errors during parallel upgrade to Block-ID based layout

[ https://issues.apache.org/jira/browse/HDFS-9536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-9536. --- Resolution: Duplicate I believe this is no longer an issue after HDFS-15937 and HDFS-15610

[DISCUSS] Supporting partial file rewrite/compose

There were a number of discussions that happened during ApacheCon. In the spirit of the Apache Way, I am taking the conversation online, sharing with the larger community and also capturing requirements. Credits to Owen who started this discussion. There are a number of scenarios where users want

[DISCUSS] Hadoop 3.3.5 release planning

Bumping this up. Adding the [DISCUSS] text to make this message stand out of your inbox. I certainly missed this message and only realized 3.3.5 has more than just security updates. What was the issue with the ARM64 build? I was able to publish ARM64 build for 3.3.1 release without problems.

Re: Hadoop BoF at ApacheCon NA'22

t; On Sat, Sep 24, 2022 at 8:52 PM 俊平堵 wrote: > > > > > Yes. It should be a short talk and I can only present remotely for this > > > time. That would be great if you help to coordinate. :) > > > > > > Thanks, > > > > > > Junping &g

Re: Hadoop BoF at ApacheCon NA'22

/1_ha1BFeEyIkAJtl5tJ8Z4Vf5Md_0rN23RCrXkyNCY1c/edit?usp=sharing Please add more details here. On Thu, Sep 22, 2022 at 10:27 PM 俊平堵 wrote: > Thanks Wei-Chiu. I am happy to share the status of Hadoop Meetups in China > (2019-2022) if that is a suitable topic. :) > > Thanks, > > Junping > >

Hadoop BoF at ApacheCon NA'22

We've not had a physical event for a long long time and we're way overdue for one. I'm excited to announce we've reserved a room at the upcoming ApacheCon for Birds-of-Feather on October 4th from 17:50-18:30 CDT in Rhythms I. I was also told that participants can stay after that until the hotel

[jira] [Resolved] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.

[ https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-4043. --- Resolution: Fixed > Namenode Kerberos Login does not use proper hostname for host qualif

[jira] [Created] (HDFS-16730) Update the doc that append to EC files is supported

Wei-Chiu Chuang created HDFS-16730: -- Summary: Update the doc that append to EC files is supported Key: HDFS-16730 URL: https://issues.apache.org/jira/browse/HDFS-16730 Project: Hadoop HDFS

[jira] [Created] (HDFS-16727) Consider reading chunk files using MappedByteBuffer

Wei-Chiu Chuang created HDFS-16727: -- Summary: Consider reading chunk files using MappedByteBuffer Key: HDFS-16727 URL: https://issues.apache.org/jira/browse/HDFS-16727 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-16619) Fix HttpHeaders.Values And HttpHeaders.Names Deprecated Import.

[ https://issues.apache.org/jira/browse/HDFS-16619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16619. Resolution: Fixed > Fix HttpHeaders.Values And HttpHeaders.Names Deprecated Imp

[jira] [Resolved] (HDFS-16595) Slow peer metrics - add median, mad and upper latency limits

[ https://issues.apache.org/jira/browse/HDFS-16595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16595. Fix Version/s: 3.4.0 Resolution: Fixed > Slow peer metrics - add median,

[jira] [Resolved] (HDFS-16583) DatanodeAdminDefaultMonitor can get stuck in an infinite loop

[ https://issues.apache.org/jira/browse/HDFS-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16583. Resolution: Fixed > DatanodeAdminDefaultMonitor can get stuck in an infinite l

[jira] [Resolved] (HDFS-16603) Improve DatanodeHttpServer With Netty recommended method

[ https://issues.apache.org/jira/browse/HDFS-16603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16603. Resolution: Fixed > Improve DatanodeHttpServer With Netty recommended met

[jira] [Resolved] (HDFS-16585) Add @VisibleForTesting in Dispatcher.java after HDFS-16268

[ https://issues.apache.org/jira/browse/HDFS-16585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16585. Fix Version/s: 3.4.0 3.2.4 3.3.4 Resolution

[jira] [Resolved] (HDFS-16583) DatanodeAdminDefaultMonitor can get stuck in an infinite loop

[ https://issues.apache.org/jira/browse/HDFS-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16583. Resolution: Fixed > DatanodeAdminDefaultMonitor can get stuck in an infinite l

[jira] [Created] (HDFS-16585) Add @VisibleForTesting in Dispatcher.java after HDFS-16268

Wei-Chiu Chuang created HDFS-16585: -- Summary: Add @VisibleForTesting in Dispatcher.java after HDFS-16268 Key: HDFS-16585 URL: https://issues.apache.org/jira/browse/HDFS-16585 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-16520) Improve EC pread: avoid potential reading whole block

[ https://issues.apache.org/jira/browse/HDFS-16520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16520. Resolution: Fixed Merged the PR and cherrypicked into branch-3.3. Thanks! > Improve

Re: [DISCUSS] Enabling all platform builds in CI for all Hadoop PRs

Running builds for all platforms for each and every PR seems too excessive. How about doing all platform builds in the nightly jobs? On Fri, May 6, 2022 at 8:02 AM Steve Loughran wrote: > I'm not enthusiastic here as it not only makes the builds slower, it > reduces the #of builds we can

[jira] [Resolved] (HDFS-16521) DFS API to retrieve slow datanodes

[ https://issues.apache.org/jira/browse/HDFS-16521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16521. Resolution: Fixed > DFS API to retrieve slow datano

Re: [DISCUSS] Hadoop on Windows

Great! Sorry I missed the earlier discussion thread. Is there a target version for this support? I assume the milestone is still in a dev branch? On Thu, Apr 28, 2022 at 8:26 AM Gautham Banasandra wrote: > Hi Hadoop devs, > > I would like to announce that we recently reached a new milestone -

[jira] [Resolved] (HDFS-16551) Backport HADOOP-17588 to 3.3 and other active old branches.

[ https://issues.apache.org/jira/browse/HDFS-16551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16551. Fix Version/s: 2.10.2 3.2.4 Resolution: Fixed Done. Thanks

[jira] [Resolved] (HDFS-16389) Improve NNThroughputBenchmark test mkdirs

[ https://issues.apache.org/jira/browse/HDFS-16389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16389. Fix Version/s: 3.4.0 Resolution: Fixed > Improve NNThroughputBenchmark test mkd

[jira] [Resolved] (HDFS-16535) SlotReleaser should reuse the domain socket based on socket paths

[ https://issues.apache.org/jira/browse/HDFS-16535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16535. Fix Version/s: 3.3.3 3.4.0 Resolution: Fixed Merged. Thanks

Re: Institutions running kerberized Hadoop clusters

Last time I checked (~2 years ago), there were thousands of Kerberized clusters among Cloudera customers. The largest ones had a few thousand nodes. What are you looking for? On Wed, Apr 13, 2022 at 7:22 AM Santosh Marella wrote: > Hey folks, > > Just curious if we have a list of

Re: [VOTE] Release Apache Hadoop 3.2.3 - RC0

aarch64 support is only introduced in/after 3.3.0 On Thu, Mar 17, 2022 at 2:27 PM Emil Ejbyfeldt wrote: > Hi, > > > There is no aarch64 artifact in the release candidate. Is this something > that is intended? > > Best, > Emil Ejbyfeldt > > On 14/03/2022 08:14, Masatake Iwasaki wrote: > > Hi

[jira] [Resolved] (HDFS-16502) Reconfigure Block Invalidate limit

[ https://issues.apache.org/jira/browse/HDFS-16502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16502. Fix Version/s: 3.4.0 3.3.3 Resolution: Fixed > Reconfigure Bl

[ANNOUNCE] New Hadoop PMC Sun Chao

On behalf of the Apache Hadoop PMC, I am pleased to announce that Sun Chao(sunchao) has accepted the PMC's invitation to become a PMC on the project. We appreciate all of Sun's generous contributions thus far and look forward to his continued involvement. Congratulations and welcome, Sun!

Re: [ANNOUNCE] Apache Hadoop 3.3.2 release

> > Many thanks to Viraj Jasani, Michael Stack, Masatake Iwasaki, Xiaoqiao He, > Mukund Madhav Thakur, Wei-Chiu Chuang, Steve Loughran, Akira Ajisaka and > other folks who helped for this release process. > > Best Regards, > Chao >

[jira] [Resolved] (HDFS-16422) Fix thread safety of EC decoding during concurrent preads

[ https://issues.apache.org/jira/browse/HDFS-16422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16422. Fix Version/s: 3.4.0 3.2.3 3.3.3 Resolution

[jira] [Resolved] (HDFS-16437) ReverseXML processor doesn't accept XML files without the SnapshotDiffSection.

[ https://issues.apache.org/jira/browse/HDFS-16437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16437. Fix Version/s: 3.4.0 Resolution: Fixed Merged. Thanks [~it_singer

[jira] [Resolved] (HDFS-16423) balancer should not get blocks on stale storages

[ https://issues.apache.org/jira/browse/HDFS-16423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16423. Fix Version/s: 3.3.3 Resolution: Fixed > balancer should not get blocks on st

[jira] [Resolved] (HDFS-16403) Improve FUSE IO performance by supporting FUSE parameter max_background

[ https://issues.apache.org/jira/browse/HDFS-16403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16403. Resolution: Fixed Thank you [~cndaimin] for the great work and the excellent performance

[jira] [Reopened] (HDFS-16423) balancer should not get blocks on stale storages

[ https://issues.apache.org/jira/browse/HDFS-16423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reopened HDFS-16423: Reopen to backport this to lower branches. > balancer should not get blocks on stale stora

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC2

I'll find time to check out the RC bits. I just feel bad that the tarball is now more than 600MB in size. On Fri, Jan 21, 2022 at 2:23 AM Steve Loughran wrote: > *+1 binding.* > > reviewed binaries, source, artifacts in the staging maven repository in > downstream builds. all good. > > *## test

Re: [DISCUSS] Migrate hadoop from log4j1 to log4j2

+1 I think it makes sense to use reload4j in maint releases. I have a draft PR doing this (https://github.com/apache/hadoop/pull/3906) log4j2 in Hadoop 3.4.0 makes sense to me. There could be incompatibilities introduced by log4j2, but I feel we should at least make it 3.4.0 a "preview" release,

Re: Hadoop-3.2.3 Release Update

Is this still making progress? On Tue, Oct 5, 2021 at 8:45 PM Brahma Reddy Battula wrote: > Hi Akira, > > Thanks for your email!! > > I am evaluating the CVE’s which needs to go for this release.. > > Will update soon!! > > > On Tue, 5 Oct 2021 at 1:46 PM, Akira Ajisaka wrote: > > > Hi Brahma,

Re: Next Mandarin Hadoop Online Meetup Jan 6th.

/JaNm70lZQGCZdlFzh9ZbsfrR7MJ7Nazb2g6NCtYPqsRLWtyEhLfgwXOppzMR3csp.HqRJNGXUGSaPu1qw Access Passcode: 4g1ZF&%f On Mon, Jan 3, 2022 at 5:39 PM Wei-Chiu Chuang wrote: > Hello community, > > This week we'll going to have Tao Li (tomscut) speaking about the > experience of operating HDFS at BIGO. See you on Thursday! > > 题目：《HDFS在BIG

Re: Next Mandarin Hadoop Online Meetup Jan 6th.

Just a gentle reminder this is happening now. On Mon, Jan 3, 2022 at 5:39 PM Wei-Chiu Chuang wrote: > Hello community, > > This week we'll going to have Tao Li (tomscut) speaking about the > experience of operating HDFS at BIGO. See you on Thursday! > > 题目：《HDFS在BIGO的实践》 >

Next Mandarin Hadoop Online Meetup Jan 6th.

Hello community, This week we'll going to have Tao Li (tomscut) speaking about the experience of operating HDFS at BIGO. See you on Thursday! 题目：《HDFS在BIGO的实践》

[jira] [Resolved] (HDFS-16317) Backport HDFS-14729 for branch-3.2

[ https://issues.apache.org/jira/browse/HDFS-16317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16317. Fix Version/s: 3.2.3 Resolution: Fixed Merged the commit into branch-3.2 and branch

Apache Hadoop and CVE-2021-44228 Log4JShell vulnerability

Hi, Given the widespread attention to the recent log4j vulnerability (CVE-2021-44228), I'd like to share an update from the Hadoop developer community regarding the incident. As you probably know, Apache Hadoop depends on the log4j library to keep log files. The highlighted vulnerability

[jira] [Reopened] (HDFS-16384) Upgrade Netty to 4.1.72.Final

[ https://issues.apache.org/jira/browse/HDFS-16384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reopened HDFS-16384: > Upgrade Netty to 4.1.72.Final > - > >

Trunk broken by HDFS-16384

My bad. There was a transitive dependency problem in the PR causing trunk to fail the build. The commit has since been reverted. Sorry for the inconvenience.

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC0

Thanks a lot for pushing it forward! A few things I noticed that we should incorporate: 1. the overview page of the doc is for the Hadoop 3.0 release. It would be best to base the doc on top of Hadoop 3.3.0 overview page. (it's a miss on my part... The overview page of 3.3.1 wasn't updated) 2.

Re: Next Hadoop community online meetup (Mandarin)

in English so the wider audience can read) Please find today's recording here: https://cloudera.zoom.us/rec/share/MOwRDPP50eL7gRfNazfjqAXdl_7HwAaPH460sONmmlx_tLQrftVjjjCBkVvE-9sf.GRb9Kp_16xOqgzNu Passcode: L^n30&%d On Thu, Dec 9, 2021 at 2:10 PM Wei-Chiu Chuang wrote: > We are having a

Re: Next Hadoop community online meetup (Mandarin)

We are having a little technical difficulties. For those who are late, we are using this link instead: https://cloudera.zoom.us/j/93901387554 On Thu, Dec 9, 2021 at 12:55 PM Takanobu Asanuma wrote: > Thanks for the information! > > 2021年12月9日(木) 13:52 Wei-Chiu Chuang : > >>

Re: Next Hadoop community online meetup (Mandarin)

Hello, this talk will start in about an hour. (I'm sorry forgot to add this is 2PM Beijing time) On Thu, Dec 9, 2021 at 11:00 AM Takanobu Asanuma wrote: > What is the timezone? > > 2021年11月29日(月) 14:50 Wei-Chiu Chuang : > >> Hello HDFS devs >> >> After a l

Next Hadoop community online meetup (Mandarin)

Hello HDFS devs After a long hiatus, I'm happy to share that we'll have Yiyang from Shopee who is going to talk about their experience of using HDFS at Shopee. This is a Mandarin talk. 题目：《HDFS在Shopee的演进》

[jira] [Resolved] (HDFS-16337) Show start time of Datanode on Web

[ https://issues.apache.org/jira/browse/HDFS-16337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16337. Fix Version/s: 3.4.0 Resolution: Fixed > Show start time of Datanode on

[jira] [Resolved] (HDFS-16241) Standby close reconstruction thread

[ https://issues.apache.org/jira/browse/HDFS-16241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16241. Fix Version/s: 3.3.2 3.2.3 3.4.0 Resolution

ApacheCon@Home Big Data tracks recordings!

For those who missed the live Apache@Home Big Data tracks, the video recordings are being uploaded to the official ASF channel! Big Data: https://www.youtube.com/playlist?list=PLU2OcwpQkYCzXcumE9UxNirLF1IYLmARj Big Data Ozone:

Re: Hadoop-3.2.3 Release Update

Hi to raise the awareness, it looks like reverting the FoldedTreeSet HDFS-13671 breaks TestBlockManager in branch-3.2. Branch-3.3 is good. tracking jira: HDFS-16258 On Tue, Oct 5, 2021 at 8:45

[jira] [Created] (HDFS-16258) HDFS-13671 breaks TestBlockManager in branch-3.2

Wei-Chiu Chuang created HDFS-16258: -- Summary: HDFS-13671 breaks TestBlockManager in branch-3.2 Key: HDFS-16258 URL: https://issues.apache.org/jira/browse/HDFS-16258 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-16238) Improve comments related to EncryptionZoneManager

[ https://issues.apache.org/jira/browse/HDFS-16238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16238. Fix Version/s: 3.4.0 Resolution: Fixed Thanks [~vjasani] [~hexiaoqiao

[jira] [Resolved] (HDFS-16232) Fix java doc for BlockReaderRemote#newBlockReader

[ https://issues.apache.org/jira/browse/HDFS-16232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16232. Fix Version/s: 3.4.0 Resolution: Fixed > Fix java doc for BlockReaderRem

[jira] [Created] (HDFS-16233) Do not use exception handler to implement copy-on-write for EnumCounters

Wei-Chiu Chuang created HDFS-16233: -- Summary: Do not use exception handler to implement copy-on-write for EnumCounters Key: HDFS-16233 URL: https://issues.apache.org/jira/browse/HDFS-16233 Project

[jira] [Resolved] (HDFS-16192) ViewDistributedFileSystem#rename wrongly using src in the place of dst.

[ https://issues.apache.org/jira/browse/HDFS-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16192. Fix Version/s: 3.3.2 3.4.0 Resolution: Fixed Thanks

[jira] [Resolved] (HDFS-16173) Improve CopyCommands#Put#executor queue configurability

[ https://issues.apache.org/jira/browse/HDFS-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16173. Resolution: Fixed > Improve CopyCommands#Put#executor queue configurabil

[jira] [Resolved] (HDFS-16175) Improve the configurable value of Server #PURGE_INTERVAL_NANOS

[ https://issues.apache.org/jira/browse/HDFS-16175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16175. Fix Version/s: 3.2.4 3.3.2 3.4.0 Resolution

[jira] [Resolved] (HDFS-16180) FsVolumeImpl.nextBlock should consider that the block meta file has been deleted.

[ https://issues.apache.org/jira/browse/HDFS-16180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16180. Fix Version/s: 3.4.0 Resolution: Fixed > FsVolumeImpl.nextBlock should consi

  1   2   3   4   5   6   7   8   >