[PR] Request to PR review

Hi, HDFS-DEVs, Anyone have time to help review the following pullrequest? Thanks. HDFS-12749 DN may not send block report to NN after NN restart HDFS-13473 DataNode update BlockKeys using mode

Issue about NameNode Fatal

Hi Devs, Anyone meet NameNode fatal since editlog not sync in time then lead to process unexpectedly exit. I have met several times and try to dig the root cause but no result. This JIRA(HDFS-10943 ) is trace this issue. welcome any suggestions

[VOTE]: Support for RBF data locality Solution

Hi forks, The current implementation of RBF is not sensitive about data locality, since NameNode could not get real client hostname by invoke Server#getRemoteAddress when RPC request forward by Router to NameNode. Therefore, it will lead to several challenges, for instance, - a. Client could

Re: [VOTE]: Support for RBF data locality Solution

ach A. > > On Thu, 11 Apr 2019, 12:23 pm Akira Ajisaka, wrote: > >> The Approach A looks good to me. >> >> Thanks, >> Akira >> >> On Thu, Apr 11, 2019 at 2:30 PM Xiaoqiao He wrote: >> > >> > Hi forks, >> > >> > The current i

Re: [DISCUSS] Merge HDFS-13891(RBF) to trunk

Thanks Brahma for starting the thread. +1 for merging. He Xiaoqiao On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota wrote: > Thanks Brahma initiating this. > +1(non-binding) for merge. > > @Uber we have almost all changes specially rbf security in production for a > while now without issues. >

Re: [VOTE] Merge HDFS-13891(RBF) to trunk

gt; > https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E > > > On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He wrote: > > > Thanks Brahma for starting the thread. > > +1 for merging. > > >

Re: HDFS Performance Improvement JIRAs: reviewers wanted

Thanks Wei-Chiu for your great work. All JIRAs listed is very valuable and I would like to try my best to participate to review and give some feedback. Another side, I think there are also some helpful JIRAs but not digged up. Does the spreadsheet support to insert more candidate JIRAs about

Re: Aug Hadoop Community Meetup in China

Thanks Junping for organizing the event. Looking forward to the coming meetup. BTW, It is hoped to keep running regular Hadoop Contributors Meetups in China. On Fri, Jul 19, 2019 at 12:35 AM Wanqiang Ji wrote: > Thanks for organize this event! > I always looking forward to meet folks in the

Re: [DISCUSS] Prefer Github PR Integration over patch in JIRA

Thanks Wei-Chiu for starting this discussion. +1 (non-binding) for turning code review to GitHub PR and keep other comments/discussions on JIRA. 1. In my experience, JIRA is better at tracking and recording information. 2. It is confused that no guide (`How to contribute`) about submit PR to JIRA

Re: [ANNOUNCE] New Apache Hadoop Committer - Stephen O'Donnell

Congratulations Stephen! - Hexiaoqiao On Thu, Mar 5, 2020 at 11:08 AM Masatake Iwasaki < iwasak...@oss.nttdata.co.jp> wrote: > Congratulations! > > Masatake Iwasaki > > > On 2020/03/04 5:11, Wei-Chiu Chuang wrote: > > In bcc: general@ > > > > It's my pleasure to announce that Stephen O'Donnell

Re: [Hadoop-3.3 Release update]- branch-3.3 has created

Hi Brahma, Thanks for your great works here. About HDFS-12862 , I try to raise another JIRA (HDFS-15272) to track backport branch-3.1, close this one to decouple release progress. Thanks, Hexiaoqiao On Sun, Apr 12, 2020 at 2:49 AM Brahma Reddy

Re: Client Caller Context Through Router(RBF)

Hi Ayush, Thank you for summary and re-launching this discussion. Both of JIRA's links are very helpful for pushing it forward. I would like to underline that this is a serious issue and very hope that we could get the solution here. As far as know, there are many users who deploy RBF meet this

Re: TestBalancerWithNodeGroup is failing due to HDFS-13183

Thanks Wei-Chiu for the information, Just update the JIRA ID: HDFS-13183. I am working on HDFS-13183 now, it is expected to be finished today. And welcome anymore comments and suggestions if interested Balancer improvement. Thanks Wei-Chiu. On Thu, May 21, 2020 at 2:47 AM Wei-Chiu Chuang

Re: [DISCUSS] Hadoop 3.2.2 release

iao, > > Thanks for volunteering. > I could help to guide you through the process. > > Thanks, > Sunil > > On Thu, Sep 3, 2020 at 9:09 AM Xiaoqiao He wrote: > >> Thanks Wei-Chiu for initiating this. >> >> It's been nearly a year since Hadoop-3.2.1 released(201

Re: [VOTE] End of Life Hadoop 2.9

+1 On Tue, Sep 1, 2020 at 10:14 AM Ayush Saxena wrote: > +1 > > -Ayush > > > On 01-Sep-2020, at 7:23 AM, Akira Ajisaka wrote: > > > > +1 > > > > -Akira > > > >> On Tue, Sep 1, 2020 at 4:09 AM Wei-Chiu Chuang > wrote: > >> > >> Dear fellow Hadoop developers, > >> > >> Given the overwhelming

Re: This Week's Hadoop Storage Online Meetup (APAC Mandarin)

Thanks Wei-Chiu for your detailed summary. The slides is available now, please reference to https://drive.google.com/file/d/1eaE8vSEhIli0H3j2eDiPJNYuKAC0MFgu. Any questions or concerns are welcome. Thanks. Regards He Xiaoqiao

Re: [DISCUSS] Hadoop 3.2.2 release

Thanks Wei-Chiu for initiating this. It's been nearly a year since Hadoop-3.2.1 released(2019 Sep 22). We have several important fixes landed in branch-3.2 (around 37 blockers/critical issues)[1]. There are also 3 blockers/critical pending issues and 3 other pending issues on 3.2.2[2]. I am

Re: Hadoop 3.2.2 Release Code Freeze Plan

; Glad to see this is moving along. I noticed your dashboard has private > filters and therefore the results are not visible publicly. > > On Tue, Sep 29, 2020 at 10:02 PM Xiaoqiao He > wrote: > > > Hi All, > > > > Plan to code frozen for Hadoop-3.2.2 relea

Re: Hadoop 3.2.2 Release Code Freeze Plan

Xiaoqiao He wrote: > Thanks to Wei-Chiu for your reminder, the dashboard and filters have been > updated. It is visible to the public now. > > Thanks, Regards. > > [1] > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12335948 > > > On Fri, Oct 2, 2

Re: [E] Re: Hadoop 3.2.2 Release Code Freeze Plan

, Oct 16, 2020 at 6:43 AM Sunil Govindan wrote: >> >>> Looks good to me. >>> >>> Thanks Xiaoquio. >>> >>> + Sunil >>> >>> On Fri, Oct 16, 2020 at 12:36 PM Xiaoqiao He >>> wrote: >>> >>> > Thanks S

Re: [E] Re: Hadoop 3.2.2 Release Code Freeze Plan

it in 3.2.2. It's a blocking issue though, so I think it'd be > unfortunate if we missed this chance to include it in a 3.2.x maintenance > release. (Same for YARN-8992, which is a related JIRA.) > > Best, > Steve > > On Sat, Oct 17, 2020 at 12:25 PM Xiaoqiao He > wrote: &

Re: Hadoop 3.2.2 Release Code Freeze Plan

/branch-3.2 On Fri, Oct 16, 2020 at 12:03 PM Sunil Govindan wrote: > Thank you > > Did we also cut the branch already ? > > Sunil > > On Fri, 16 Oct 2020 at 9:14 AM, Xiaoqiao He wrote: > >> Hi All, >> >> All issues targeted to 3.2.2 have been resolved from no

Mandarin Hadoop Storage Online Meetup this week (APAC Mandarin)

Hi All, We are happy to have Jinglun(李经伦) leading discussion on HDFS Federation Balancer Tools implementation at this week's Hadoop storage online meetup (APAC Mandarin). Time/Date: 10/14 10PM (US West Coast PST) 10/15 1PM (Beijing, China CST) Join Zoom Meeting

Jenkins does not work for long time for HDFS sub-project

Hi all, It seems that Jenkins does not work for over 20 days for the HDFS sub-project. And there is no entrance to start testing a patch manually[1]. Not sure If I missed something. Anyone who know if there are any issues? Thanks. Best Regards, He Xiaoqiao [1]

Re: Jenkins does not work for long time for HDFS sub-project

of weeks > back. > > This is the new link : > https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/ > > Give a check, if there is any problem there as well. > > -Ayush > > > On Fri, 21 Aug 2020 at 12:36, Xiaoqiao He wrote: > >> Hi all, >> >> It seems t

Re: [DISCUSS] fate of branch-2.9

+1 for putting 2.9 release lines to EOL. Thanks, Hexiaoqiao On Thu, Aug 27, 2020 at 3:14 PM Mingliang Liu wrote: > +1 for putting 2.9 lines to EOL. > > Let's focus on 2.10 releases for Hadoop 2. Also is there any plan for > 2.10.1? It has been 11 months since 2.10 first release. > > Thanks, >

Hadoop 3.2.2 Release Code Freeze Plan

Hi All, Plan to code frozen for Hadoop-3.2.2 release at 2020/10/15. From now on, most of the issues have been resolved. And there are two JIRA still open which target to 3.2.2 traced by [1]. * https://issues.apache.org/jira/browse/YARN-10244 * https://issues.apache.org/jira/browse/HADOOP-17287

Re: [ANNOUNCE] Lisheng Sun is a new Apache Hadoop Committer

Congrats! Best Regards, He Xiaoqiao On Thu, Sep 24, 2020 at 9:08 AM Guanghao Zhang wrote: > Congratulations, Lisheng! >

Re: [ANNOUNCE] Hui Fei is a new Apache Hadoop Committer

Congrats! Best Regards, He Xiaoqiao On Thu, Sep 24, 2020 at 10:03 AM Sammi Chen wrote: > Congratulations to Hui ! > > On Thu, Sep 24, 2020 at 2:07 AM Wei-Chiu Chuang > wrote: > > > I am pleased to announce that Hui Fei has accepted the invitation to > become > > a Hadoop committer. > > > > He

Re: Skipping this week’s APAC Hadoop storage online meetup

Thanks Wei-Chiu for your reminder. Considering there is about 10 days vacation next month. We will plan to host the next APAC Hadoop storage online meetup at 2020/10/15 13:00. If any changes will inform in time. Thanks - He Xiaoqiao On Thu, Sep 24, 2020 at 1:10 AM Wei-Chiu Chuang wrote: > The

Re: [VOTE] Moving Ozone to a separated Apache project

+1 - He Xiaoqiao On Sat, Sep 26, 2020 at 2:20 AM Weiwei Yang wrote: > +1 > > Weiwei > > On Friday, September 25, 2020, Dinesh Chitlangia > wrote: > > > +1 > > > > Thanks, > > Dinesh > > > > On Fri, Sep 25, 2020 at 2:00 AM Elek, Marton wrote: > > > > > Hi all, > > > > > > Thank you for all

Re: [VOTE] Release Apache Hadoop 3.3.0 - RC0

Thanks Brahma Reddy Battula for your great work here. Stephen fixed lease leak in namenode, and it is ready now: https://issues.apache.org/jira/browse/HDFS-14498. I think this affects 3.3.0-RC0. Would you check this? Sorry for reporting it so late. On Mon, Jul 13, 2020 at 2:53 PM Takanobu Asanuma

Re: [VOTE] Release Apache Hadoop 3.3.0 - RC0

t; Thanks for bringing this to my attention. > > It's too rare to occur this scenario and a couple of releases went after > HDFS-4882(broken jira). > > So,You think it's fine to include in the next release? Or if you still > insist we can go for another RC... > > > On Mon,

Re: [ANNOUNCE] New Apache Hadoop Committer - He Xiaoqiao

Thanks all! So glad to join the Hadoop committer family. Will continue to make contributions to the projects. Best Regards, He Xiaoqiao On Fri, Jun 12, 2020 at 3:04 PM Wangda Tan wrote: > Congrats! > > Best, > Wangda Tan > > On Thu, Jun 11, 2020 at 5:55 PM Tao Yang > wrote: > > >

Re: [ANNOUNCE] New Apache Hadoop PMC member : Ayush Saxena

Congratulations Ayush! Regards, He Xiaoqiao On Thu, Jul 23, 2020 at 9:18 AM Sheng Liu wrote: > Congrats, and thanks for your help. > > Sree Vaddi 于2020年7月23日周四 上午3:59写道： > > > Congratulations, Ayush.Keep up the good work. > > > > > > Thank you./Sree > > > > > > > > On Wednesday, July 22,

Mandarin Hadoop Storage Online Meetup next week (APAC Mandarin)

Hi All, We are happy to invite Feilong He from Intel leading discussion on HDFS Provided Storage Read/Write Mount Support On-the fly at next week's Hadoop storage online meetup (APAC Mandarin). Time/Date: 2021/01/20 10PM (US West Coast PST) 2021/01/21 1PM (Beijing, China CST) Join Zoom

[VOTE] Release Apache Hadoop 3.2.2 - RC3

Hi folks, The release candidate (RC3) for Hadoop-3.2.2 is available now. There are 22 commits[1] differences between RC3 and RC2[2]. The RC3 is available at: http://people.apache.org/~hexiaoqiao/hadoop-3.2.2-RC3 The RC3 tag in github is here:

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC3

0 at 9:09 AM Chao Sun wrote: > > > > > > Thanks Xiaoqiao. I just committed the fix to branch-3.2 (and will port > it > > > to other branches as well later). Could you cherry-pick it to > branch-3.2.2 > > > and start another RC? really appre

[VOTE] Release Apache Hadoop 3.2.2 - RC4

Hi folks, The release candidate (RC4) for Hadoop-3.2.2 is available now. There are 10 commits[1] differences between RC4 and RC3[2]. The RC4 is available at: http://people.apache.org/~hexiaoqiao/hadoop-3.2.2-RC4 The RC4 tag in github is here:

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC3

e notice. > > Best, > Chao > > On Mon, Nov 30, 2020 at 7:37 AM Xiaoqiao He wrote: > >> Hi folks, >> >> The release candidate (RC3) for Hadoop-3.2.2 is available now. >> There are 22 commits[1] differences between RC3 and RC2[2]. >> >> The RC3 i

Re: Regarding HDFS-15567. HDFS should expose msync() API to allow downstream applications call it explicitly

Thanks Steve very much for your discussion here. Leave some comments inline. Will focus on this thread to wait for the final conclusion to decide if we should prepare another release candidate of 3.2.2. Thanks Steve and Chao again for your warm discussions. On Sat, Dec 12, 2020 at 7:18 PM Steve

Re: [E] Re: [VOTE] Release Apache Hadoop 3.2.2 - RC4

tps-3A__issues.apache.org_jira_browse_YARN-2D10450=DwIBaQ=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw=nSlLXPsCxZGl0VV03dBWreCNrSH0SsNAZzmjRWO-2Zg=QEHMGtEbBz5Gn7mW4UsGlc-wNZ8ugZwiFQBy2pTx-Fw= >> > > >. >> > > > In the trunk, I

Re: [E] Re: [VOTE] Release Apache Hadoop 3.2.2 - RC4

> I feel that we can call a new RC as YARN-10540 is backported already. > > Thanks > Sunil > > On Wed, Dec 23, 2020 at 3:54 PM Xiaoqiao He wrote: > > > Thanks Jim, Eric and Sunil, > > > > I tried to backport YARN-10540 to branch-3.2.2 at local and check yarn >

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC4

browse/HADOOP-15930>. >> >> Thanks >> Sunil >> >> On Sat, Dec 19, 2020 at 10:16 AM Sunil Govindan >> wrote: >> >>> Hi Xiaoqiao and Wei-chiu >>> >>> I am a bit confused after seeing both *.sha512 and *.md5 files in the >

Mandarin Hadoop Storage Online Meetup this week (APAC Mandarin)

Hi All, We are happy to invite Huang Haibin(黄海滨) leading discussion on HDFS Slow Node Improvement Practice at Xiaomi at next week's Hadoop storage online meetup (APAC Mandarin). Time/Date: 12/23 10PM (US West Coast PST) 12/24 1PM (Beijing, China CST) Join Zoom Meeting

Re: Regarding HDFS-15567. HDFS should expose msync() API to allow downstream applications call it explicitly

. And if we can get what the API does >> > defined in a way which works, it shouldn't need changing. >> > >> > (which reminds me, I do need to check that RC out, don't I?) >> > >> > On Sun, 13 Dec 2020 at 09:00, Xiaoqiao He >> wrote: >>

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC2

nd the webUI etc. Everything >>> looks fine. The only strange thing, was on the Namenode webUI under the >>> overview section - the "unknown" is a little strange, but I am not sure >>> what is usually in there: >>> >>> Version: 3.2.2, rUnknown &

[VOTE] Release Apache Hadoop 3.2.2 - RC2

Hi folks, The release candidate (RC2) for Hadoop-3.2.2 is available now. There are two commits[1] differences between RC2 and RC1[2](Thanks Akira Ajisaka for the report.): * revert HADOOP-17306. * include HDFS-15643. The RC2 is available at: http://people.apache.org/~hexiaoqiao/hadoop-3.2.2-RC2

Mandarin Hadoop Storage Online Meetup this week (APAC Mandarin)

Hi All, We are happy to invite Hu Haiyang(胡海洋) leading discussion on HDFS Erasure Coding Practice at DiDi at this week's Hadoop storage online meetup (APAC Mandarin). Time/Date: 11/11 10PM (US West Coast PST) 11/12 1PM (Beijing, China CST) Join Zoom Meeting

[VOTE] Release Apache Hadoop 3.2.2 - RC1

Hi folks, I have put together a release candidate (RC1) for Hadoop-3.2.2. It contains *473[1]* fixed jira issues since 3.2.1 release which also include many features and improvements(Please reference the full set of release notes). The RC is available at:

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC1

saw that the revert of HADOOP-17306 is now pushed to > the 3.2.2 branch as well. > > Let's do one more RC. Thanks @Xiaoqiao He for > helping. > > Thanks > Sunil > > On Fri, Nov 6, 2020 at 6:58 AM Akira Ajisaka wrote: > >> -1 >> >> - YARN resource loc

Re: Regarding HDFS-15567. HDFS should expose msync() API to allow downstream applications call it explicitly

Hi Steve and Chao, Konstatin has updated HDFS-15751 and attached patch for documentation msync API, Would you mind to TAL. I want to involve this update to 3.2.2 release, but for now it seems there are not enough committers to review. Please check when you have bandwidth. Thanks all. - He

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC5

Hi folks, The vote[1] passed with 4 binging +1's and 4 non-binding +1's.[2] Thanks all for pushing this. I'm going to publish the release. binding +1's: - Akira Ajisaka - Wei-Chiu Chuang - Masatake Iwasaki - Sunil Govindan non-binding +1's: - Xiaoqiao He - Hui Fei - Lisheng Sun - Chao Sun

[ANNOUNCE] Apache Hadoop 3.2.2 release

Hi All, It gives me great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop 3.2.2. Apache Hadoop 3.2.2 includes 516 fixes since the previous Hadoop 3.2.1 release. * For major changes included in Hadoop 3.2.2, please refer to Hadoop 3.2.2 main page [1]. *

Re: Regarding HDFS-15567. HDFS should expose msync() API to allow downstream applications call it explicitly

Hi Konstantin and Steve, HDFS-15751 has committed to related branches including 3.2.2. I will prepare 3.2.2-RC shortly. Please let me know if any other issues for doc or deployment do you meet. Thanks. - He Xiaoqiao On Wed, Dec 30, 2020 at 6:40 PM Xiaoqiao He wrote: > Hi Steve and C

[VOTE] Release Apache Hadoop 3.2.2 - RC5

Hi folks, The release candidate (RC5) for Hadoop-3.2.2 is available now. There are 4 commits[1] differences between RC5 and RC4[2]. The RC5 is available at: http://people.apache.org/~hexiaoqiao/hadoop-3.2.2-RC5 The RC5 tag in github is here:

Re: [E] Re: [VOTE] Release Apache Hadoop 3.2.2 - RC4

l, - He Xiaoqiao [1] https://lists.apache.org/thread.html/r6c2817770e99362da79ebfaa0b6a774c5fca0f31e0dcc24df398eda0%40%3Chdfs-dev.hadoop.apache.org%3E On Thu, Dec 24, 2020 at 11:30 AM Xiaoqiao He wrote: > Thanks Sunil, it makes sense to me. Will backport and prepare another RC > shortly. > > - He Xiaoqiao > &

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

+0. - Built successfully from tag release-3.3.1-RC3 on Ubuntu 20.04 - Verified signature and checksum - Deployed pseudo-distributed cluster with 3 nodes - Ran basic HDFS shell commands and sample MR Jobs, it worked well. - Browsed NN/DN/RM/NM UI. *YARN UI2 DOES NOT work from my side.* NOTE:

Re: [DISCUSS] Change project style guidelines to allow line length 100

+1 for <= 100 chars long per line length. On Thu, May 20, 2021 at 10:28 AM Masatake Iwasaki < iwasak...@oss.nttdata.co.jp> wrote: > I'm +1 too. > I feel 80 characters limit tends to degrade readability by introducing > useless line breaks. > > > >

Re: [VOTE] hadoop-thirdparty 1.1.0-RC0

+1 - Verified checksums and signatures. - Checked CHANGELOG / RELEASE NOTES. - Built from source with jdk1.8.0_202. - Built Hadoop trunk with hadoop-thirdparty 1.1.0 and checked with pseudo distribution hadoop cluster deployment. Thanks, - He Xiaoqiao On Fri, May 14, 2021 at 6:16 PM Viraj

Re: [DISCUSS] which release lines should we still consider actively maintained?

Thanks Sean and Akira for discussion here. > As you commented, the confluence wiki pages for Hadoop releases were out of date. Updated [1]. Just update Roadmap[1] for 3.2.2 and updated/added issues collections for 3.2.3[2] which will be useful if you are interested in becoming a release manager

Re: [VOTE] Release Apache Hadoop Thirdparty 1.1.1 RC0

+1 - Verified checksums and signatures. - Checked CHANGELOG / RELEASE NOTES. - Built from source with jdk1.8.0_202. - Built Hadoop trunk with hadoop-thirdparty 1.1.1 and checked with pseudo distribution hadoop cluster deployment. Thanks Wei-Chiu for your work on this. I am confused about the

Re: [VOTE] Release Apache Hadoop Thirdparty 1.1.1 RC0

p in > the thirdparty 1 1 1. > > On Fri, May 28, 2021 at 11:23 AM Xiaoqiao He > wrote: > >> +1 >> >> - Verified checksums and signatures. >> - Checked CHANGELOG / RELEASE NOTES. >> - Built from source with jdk1.8.0_202. >> - Built Hadoop trunk with

Re: [VOTE] Hadoop 3.1.x EOL

+1 On Thu, Jun 3, 2021 at 4:18 PM Bharat Viswanadham wrote: > +1 > > Thanks, > Bharat > > > On Thu, Jun 3, 2021 at 1:00 PM Viraj Jasani wrote: > > > +1 (non-binding) > > > > On Thu, 3 Jun 2021 at 12:21 PM, Wei-Chiu Chuang > > wrote: > > > > > +1 > > > > > > On Thu, Jun 3, 2021 at 2:14 PM

Re: [DISCUSS] Looking for help fix flaky tests

Thanks Wei-Chiu raise this discussion here. I would like to be involved to fix them. > For example, Dynamometer test are failing because it fetches from a mirror that no longer exists. For Dynamometer, I am not sure why not fetch resources from apache directly, any load consideration? or we

Re: [DISCUSS] Hadoop 3.3.2 release?

+1, Thanks Chao for bringing this up. - He Xiaoqiao On Thu, Sep 9, 2021 at 8:32 AM tom lee wrote: > +1 (non-binding) > > Ayush Saxena 于2021年9月9日周四 上午1:12写道： > > > +1, > > Thanx Chao for volunteering!!! > > > > > On 07-Sep-2021, at 10:36 PM, Chao Sun wrote: > > > > > > Hi all, > > > > > > It

Re: Hadoop-3.2.3 Release Update

>>> In these patches, how about backporting the following issues to > >> >>> branch-3.2 and branch-3.2.3? > >> >>> - HADOOP-14922 > >> >>> - HADOOP-15939 > >> >>> - HADOOP-17569 > >> >>> > >> &

Re: Hadoop-3.2.3 Release Update

. Best Regards, - He Xiaoqiao On Tue, Sep 14, 2021 at 3:39 PM Xiaoqiao He wrote: > Hi Brahma, HDFS-15160 has checked in branch-3.2 & branch-3.2.3. FYI. > > On Tue, Sep 14, 2021 at 3:52 AM Brahma Reddy Battula > wrote: > >> Hi All, >> >> Waiting for the fol

Re: Hadoop-3.2.3 Release Update

Thanks Brahma for initiating this and making hadoop-3.2.3 release happen. I would like to validate the HBase project (both the latest release and trunk branch). Chao Sun will validate the Spark Project (Got in touch with Chao already). once RC is out. Thanks and Regards, - He Xiaoqiao On Tue,

Re: Hadoop-3.2.3 Release Update

fine. FYI. Regards, - He Xiaoqiao On Thu, Sep 16, 2021 at 10:52 PM Brahma Reddy Battula wrote: > Please go ahead. Let me know any help required on review. > > On Tue, Sep 14, 2021 at 6:57 PM Xiaoqiao He wrote: > >> Hi Brahma, >> >> I plan to involve HDFS-14997 an

Re: [DISCUSS] Hadoop 3.2.3 release

Thanks Brahma for volunteering and driving this release plan. I just created a dashboard for 3.2.3 release[1]. I would like to support for this release line if need. (cc Brahma) Thanks. Regards, - He Xiaoqiao [1] https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336167 On

Re: [DISCUSS] Hadoop 3.2.3 release

cc @dev mail-list. On Wed, Jul 28, 2021 at 5:11 PM Xiaoqiao He wrote: > Hi Brahma, > > I just created version 3.2.4, and changed all unresolved issues (target > version/s: 3.2.3) to 3.2.4 after checking both of them are not blocker > issues. Dashboard[1] is clean now. >

Re: [ANNOUNCE] Apache Hadoop 3.3.2 release

release, and everyone in > the > > Apache Hadoop community! This release is a direct result of your great > > contributions. > > > > Many thanks to everyone who helped in this release process! > > > > Many thanks to Viraj Jasani, Michael Stack, Masatake Iwasaki, Xiaoqiao > He, > > Mukund Madhav Thakur, Wei-Chiu Chuang, Steve Loughran, Akira Ajisaka and > > other folks who helped for this release process. > > > > Best Regards, > > Chao > > >

Re: [ANNOUNCE] Apache Hadoop 3.3.2 release

t and 3.3.2 showed up in the list under the > > "Documentation" tab. > > > > In https://hadoop.apache.org, under the "Documentation" tab, the > > "Latest" and "Stable" are already updated to 3.3.2, but somehow the > > announcement is not

Re: Hadoop-3.2.3 Release Update

just for reload4j migration if we > need. > https://issues.apache.org/jira/browse/HADOOP-18088 > > Thanks, > Masatake Iwasaki > > On 2022/03/10 12:23, Xiaoqiao He wrote: > > Thanks Masatake for reaching out. > > > > Not sure if Brahma has bandwidth to pu

Re: Hadoop-3.2.3 Release Update

t; > >>> I am evaluating the CVE’s which needs to go for this release.. > >>> > >>> Will update soon!! > >>> > >>> > >>> On Tue, 5 Oct 2021 at 1:46 PM, Akira Ajisaka > >> wrote: > >>> > >>>> Hi Brahm

Re: [ANNOUNCE] New Hadoop PMC Sun Chao

Congratulations, Sun Chao! Best Regards, - He Xiaoqiao On Wed, Mar 9, 2022 at 1:06 PM tom lee wrote: > Congratulations, Sun! > > Wei-Chiu Chuang 于2022年3月9日周三 11:47写道： > > > On behalf of the Apache Hadoop PMC, I am pleased to announce that Sun > > Chao(sunchao) has accepted the PMC's

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC5

+1(binding) * Built on CentOS 8 and OpenJDK 8 by `mvn clean package -DskipTests -Pnative -Pdist -Dtar` from source. * Verified signature and checksums. * Setup pseudo cluster with HDFS and YARN. * Run simple FsShell - mkdir/put/get/mv/rm. * Submit example mr applications and check the result - Pi

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC4

Thanks Chao for your great work and thanks Viraj's careful checks here. After checking to build from source and setting up a pseudo cluster based on the RC4, it seems that it is almost ready for release-3.3.2 beside update changelog, right? Please feel free to share information if you meet any

Regarding HDFS-15382.Split FsDatasetImpl from blockpool lock to blockpool volume lock

Hi guys, I always hear that DataNode throughput could not improve recently because of the global coarse-grain read-write lock on DataNod especially for federation deployment. Actually about more than one years ago, we split it into a fine-grain read-write lock for BlockPool/Volume grain in my

Re: [VOTE] Release Apache Hadoop 3.2.3 - RC1

+1(binding) * Verified signature and checksum of the source tarball. * Built the source code on Ubuntu and OpenJDK 11 by `mvn clean package -DskipTests -Pnative -Pdist -Dtar`. * Setup pseudo cluster with HDFS and YARN. * Run simple FsShell - mkdir/put/get/mv/rm and check the result. * Run example

Re: MODERATE for hdfs-iss...@hadoop.apache.org

Hi Sergey Onuchin, Sorry to hear that. But we could not give some suggestions based on the only information you mentioned. If any more on-site information may be better to trace, such as depoy architecture, NameNode log and jstack etc. Based on my practice, I did not receive some cases which

Re: Review request for https://issues.apache.org/jira/browse/HDFS-15413

Hi Bryan, Thanks for reaching out and sorry for the late response for this JIRA issue. I tried to contact one HDFS EC expert to get involved. Hope we can push this forward. Thanks again. Best Regards, - He Xiaoqiao On Fri, Nov 3, 2023 at 12:49 AM Bryan Beaudreault wrote: > Hello devs, > >

Re: 关于hadoop-3.3.1版本使用libhdfs3.so 访问hdfs联邦模式router节点rpc端口有20分钟延时问题

Add hdfs-dev@h.a.o and user@h.a.o On Thu, Oct 26, 2023 at 7:07 PM 王继泽 wrote: > 最近在使用hadoop的过程中，发现了一个情况。 > 当我使用c > api向hdfs联邦模式router节点rpc端口发送请求时，比如说写文件，客户端发送完成请求后，hadoop端需要20分钟延时文件才有字节大小，延时期间不能对文件进行操作。 > > 客户端这边运行结束之后，hadoop端日志大致过程： > 1.namenode接收到客户端的请求，FSEditLog打印日志。 >

[ANNOUNCE] New Hadoop PMC Member - Shilun Fan

On behalf of the Apache Hadoop PMC, I am pleased to announce that Shilun Fan(slfan1989) has accepted the PMC's invitation to become PMC member on the project. We appreciate all of Shilun's generous contributions thus far and look forward to his continued involvement. Congratulations and welcome,

[ANNOUNCE] New Hadoop Committer - Shuyan Zhang

I am pleased to announce that Shuyan Zhang has been elected as a committer on the Apache Hadoop project. We appreciate all of Shuyan's work, and look forward to her/his continued contributions. Congratulations and Welcome, Shuyan! Best Regards, - He Xiaoqiao (On behalf of the Apache Hadoop PMC)

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

+1(binding) * Verified signature and checksum of the source tarball. * Built the source code on Ubuntu and OpenJDK 11 by `mvn clean package -DskipTests -Pnative -Pdist -Dtar`. * Setup pseudo cluster with HDFS and YARN. * Run simple FsShell - mkdir/put/get/mv/rm and check the result. * Run example

Re: [DISCUSS] Hadoop 2.10.2 release

Thanks Masatake and Akira for bringing this release up. +1 for 2.10.2 release. Sorry for the late response. Regards, - He Xiaoqiao On Mon, Mar 14, 2022 at 1:17 PM Akira Ajisaka wrote: > > Thank you Masatake. > > +1 to release 2.10.2. > > On Fri, Mar 4, 2022 at 6:52 PM Masatake Iwasaki >

Hadoop community online meetup (Mandarin)

Hi Hadoop folks, We am happy to inform you that we will have Dongfa Huang from ByteDance who is going to talk about their experience of extending NameNode of HDFS at ByteDance. This will be a Mandarin talk. Time/Date: 2022/06/22 10PM (US West Coast PST) 2022/06/23 1PM (Beijing, China CST) Join

Re: [VOTE] Release Apache Hadoop 3.3.5 (RC3)

+1 * Verified signature and checksum of the source tarball. * Built the source code on Ubuntu and OpenJDK 11 by `mvn clean package -DskipTests -Pnative -Pdist -Dtar`. * Setup pseudo cluster with HDFS and YARN. * Run simple FsShell - mkdir/put/get/mv/rm (include EC) and check the result. * Run

Re: [VOTE] Release Apache Hadoop 3.3.6 RC1

Hi, I met a fatal error when building from source on local Mac OSX. It could reproduce stably. Not sure if it is related to my local environment. Try to dig it, but not any conclusion right now. Will feedback once find reasons. Appendix system environment, some more stack information refer to

Re: [VOTE] Release Apache Hadoop 3.3.6 RC1

Addendum: A. Build the release from sources using: `mvn clean install package -DskipTests=true -Dmaven.javadoc.skip=true` B. It works well when using the same command to build from the source branch trunk. On Wed, Jun 21, 2023 at 8:44 PM Xiaoqiao He wrote: > Hi, > > I met a fatal e

Re: [VOTE] Release Apache Hadoop 3.3.6 RC1

+1(binding) * Verified signature and checksum of all source tarballs. * Built source code on Ubuntu and OpenJDK 11 by `mvn clean package -DskipTests -Pnative -Pdist -Dtar`. * Setup pseudo cluster with HDFS and YARN. * Run simple FsShell - mkdir/put/get/mv/rm and check the result. * Run example mr

Re: Fix for SEGV during java app shutfdown (HDFS-13585)

Thanks Ed for your feedback. I think it is better to comment on JIRA HDFS-13585 directly. And feel free to submit PR there if interested and there will be some reviewers coming soon. Please let me know if you will have any questions. Thanks again. Best Regards, - He Xiaoqiao On Sun, Jul 2, 2023

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

Thanks Wei-Chiu for driving this release. The next RC will be prepared, right? If true, I would like to try and vote on the next RC. Just notice that some JIRAs are not included and need to revert some PRs to pass HBase verification which are mentioned above. Best Regards, - He Xiaoqiao On Fri,

Re: [DISCUSS] Hadoop 3.3.6 release planning

Thanks Wei-Chiu for driving this release. Cherry-pick YARN-11482 to branch-3.3 and mark 3.3.6 as the fixed version. so far only 8 jiras were resolved in the branch-3.3 line. If we should consider both 3.3.6 and 3.3.9 (which is from release-3.3.5 discuss)[1] for this release line? I try to query

[INFO] Hadoop-3.4 Release Update

Hi All, Branch trunk has been set to 3.5.0-SNAPSHOT and branch-3.4 is created[1]. Please set the proper fix version while committing jira, 3.5.0 should be priority. If there is any blocker/critical please also backport to branch-3.4 or sync to me or RM Shilun Fan. Shilun will start 3.4.0 RC1

Re: [VOTE] Release Apache Hadoop Thirdparty 1.2.0 (RC1)

at 10:29 PM slfan1989 wrote: > Hi folks, > > Xiaoqiao He and I have put together a release candidate (RC1) for Hadoop > Thirdparty 1.2.0. > > The RC is available at: > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-thirdparty-1.2.0-RC1 > > The RC tag is > ht

Re: [VOTE] Hadoop 3.2.x EOL

t; On Wed, Dec 6, 2023 at 8:39 AM Akira Ajisaka wrote: >> > >> > +1 >> > >> > >> > >> > On Wed, Dec 6, 2023 at 1:10 PM Xiaoqiao He wrote: >> > >> > > Dear Hadoop devs, >> > > >> > > Given the feedb

Re: [DISCUSS][HDFS] Add rust binding for libhdfs

Thanks Xuanwo for your work. I believe it is valuable to enlarge hadoop ecosystem. I am also concerned that it will involve more hard work to release and version match, especially for one who is not familiar with C or Rust. Moreover, I am not aware the difference between `accept hdfs-sys as part

  1   2   3   >