HADOOP-18207 hadoop-logging module about to land

Hi, I am preparing to resolve HADOOP-18207 ( https://github.com/apache/hadoop/pull/5717). This change affects all modules. With this change, it will eliminate almost all the direct log4j usage. As always, landing such a big piece is tricky. I

[DISCUSS] YARN code maintenance

Hi, First of all, I am not familiar with YARN code so I'm not really in the position to make such a claim. But while releasing Hadoop 3.3.6, I found that a number of YARN modules are seldom updated nor maintained. 1. There are hundreds of npm javascript module vulnerability alerts in GitHub

[ANNOUNCE] Apache Hadoop 3.3.6 release

On behalf of the Apache Hadoop Project Management Committee, I am pleased to announce the release of Apache Hadoop 3.3.6. It contains 117 bug fixes, improvements and enhancements since 3.3.5. Users of Apache Hadoop 3.3.5 and earlier should upgrade to this release.

Re: [VOTE] Release Apache Hadoop 3.3.6 RC1

> -Ayush > > > [1] https://github.com/ayushtkn/hadoop-ozone/actions/runs/5282707769 > [2] https://github.com/apache/tez/pull/285#issuecomment-1590962978 > > On Sat, 24 Jun 2023 at 09:43, Nilotpal Nandi > wrote: > >> +1 (Non-binding). >> Thanks a lot Wei-Chiu

Re: [VOTE] Release Apache Hadoop 3.3.6 RC1

hed a pseudo distributed cluster using released binary packages, > > done some HDFS dir/file basic opeations. > > + Run grep, pi and wordcount MR tasks on the pseudo cluster. > > > > Bests, > > Sammi Chen > > > > 发件人: Wei-C

Clean up old Hadoop release tarballs

https://dist.apache.org/repos/dist/release/hadoop/common/ has Hadoop release tarballs 3.3.1 ~ 3.3.5. I plan to remove the tarballs from 3.3.1 to 3.3.4 and leave only 3.3.5 (and the upcoming 3.3.6). Shout out if you have something depending on the old release tarballs (you shouldn't) Other release

DockerHub admin for Apache Hadoop

Ayush and I have acquired the DockerHub admin privilege for the Hadoop project in order to facilitate the release of Hadoop 3.3.6. Apache Infra allows only two seats per project. So if you need something, let Ayush and I know and we will make it happen for you. If you are a Docker guru and can't

Re: [VOTE] Release Apache Hadoop 3.3.6 RC1

built from the source tarball on Rocky Linux 8 (x86_64) and OpenJDK > 8 > >> > with native profile enabled. > >> >+ launched pseudo distributed cluster including kms and httpfs with > >> > Kerberos and SSL enabled. > >> >+ created encryption

Re: [VOTE] Release Apache Hadoop 3.3.6 RC1

Bumping this thread to the top. If you are verifying the release, please vote on this thread. RC0 and RC1 are exactly the same. The only material difference is the Changelog. Thanks!! On Sun, Jun 18, 2023 at 5:52 PM Wei-Chiu Chuang wrote: > I am inviting anyone to try and vote on this rele

[VOTE] Release Apache Hadoop 3.3.6 RC1

I am inviting anyone to try and vote on this release candidate. Note: This is exactly the same as RC0, except the CHANGELOG. The RC is available at: https://home.apache.org/~weichiu/hadoop-3.3.6-RC1-amd64/ (for amd64) https://home.apache.org/~weichiu/hadoop-3.3.6-RC1-arm64/ (for arm64) Git tag:

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

ill be prepared, > right? > If true, I would like to try and vote on the next RC. > Just notice that some JIRAs are not included and need to revert some PRs to > pass HBase verification which are mentioned above. > > Best Regards, > - He Xiaoqiao > > > On Fri, Jun

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

ey for gs://fake-in-memory-test-b ucket/contract-test/testOpenFileApplyAsyncRead "fs.option.openfile.length" On Wed, Jun 14, 2023 at 5:01 PM Wei-Chiu Chuang wrote: > The hbase-filesystem tests passed after reverting HADOOP-18596 > <https://issues.apache.org/jira/b

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

It's branching off branch-3.3 On Thu, Jun 15, 2023 at 3:18 AM Steve Loughran wrote: > Which branch is -3.3.6 off? 3.3.5 or 3.3? > > I'm travelling for the next few days and unlikely to be able to test this; > will do my best > > On Wed, 14 Jun 2023 at 07:27, Wei-Chiu Chuang

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

not the end of the world. I think we can address it by adding an incompatible change flag and a release note. On Wed, Jun 14, 2023 at 3:55 PM Wei-Chiu Chuang wrote: > Cross referenced git history and jira. Changelog needs some update > > Not in the release > >1. HDFS-16858 <https:/

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

59->AbstractContractDistCpTest.distCpUpdateDeepDirectoryStructure:334->AbstractContractDistCpTest.assertCounterInRange:294->Assert.assertTrue:41->Assert.fail:88 Files Copied value 2 above maximum 1 [INFO] [ERROR] Tests run: 240, Failures: 3, Errors: 0, Skipped: 58 Ozone test in progress. Wil

[VOTE] Release Apache Hadoop 3.3.6 RC0

I am inviting anyone to try and vote on this release candidate. Note: This is built off branch-3.3.6 plus PR#5741 (aws sdk update) and PR#5740 (LICENSE file update) The RC is available at: https://home.apache.org/~weichiu/hadoop-3.3.6-RC0-amd64/ (for amd64)

Re: [DISCUSS] Hadoop 3.3.6 release planning

setSafeMode() is exposed through SafeMode interface. On Thu, Jun 8, 2023 at 11:11 AM Wei-Chiu Chuang wrote: > Thanks for comments > > Looking at jiras fixed in 3.3.6 and 3.3.9 (my bad, forgot that most > commits landing in branch-3.3 was 3.3.9), most are okay. We have about 119 >

Fwd: [jira] [Created] (HADOOP-18768) Integrating Apache Hadoop into OSS-Fuzz

Are there Hadoop committers who would like to help triage bug reports from OSS-Fuzz? -- Forwarded message - From: Henry Lin (Jira) Date: Mon, Jun 12, 2023 at 9:10 AM Subject: [jira] [Created] (HADOOP-18768) Integrating Apache Hadoop into OSS-Fuzz To: Henry Lin created

Re: [DISCUSS] Hadoop 3.3.6 release planning

gt; > Cherry-pick YARN-11482 to branch-3.3 and mark 3.3.6 as the fixed > version. > > > > > > > > so far only 8 jiras were resolved in the branch-3.3 line. > > > > > > > > > > > > If we should consider both 3.3.6 and 3.3.9 (which is from > r

Re: Call for Presentations, Community Over Code 2023

There's also a call for presentation for Community over Code Asia 2023 https://www.bagevent.com/event/cocasia-2023-EN Happening Aug 18-20. CfP due by 6/6 On Tue, May 9, 2023 at 8:39 PM Ayush Saxena wrote: > Forwarding from dev@hadoop to the dev ML which we use. > > The actual mail lies here:

[DISCUSS] Hadoop 3.3.6 release planning

Hi community, I'd like to kick off the discussion around Hadoop 3.3.6 release plan. I'm being selfish but my intent for 3.3.6 is to have the new APIs in HADOOP-18671 added so we can have HBase to adopt this new API. Other than that, perhaps

Re: [DISCUSS] hadoop branch-3.3+ going to java11 only

My random thoughts. Probably bad takes: There are projects experimenting with JDK17 now. JDK11 active support will end in 6 months. If it's already hard to migrate from JDK8 why not retarget JDK17. On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena wrote: > I know Jersey upgrade as a blocker. Some

Re: MapReduce Terasort job is slow on Java 11

For the JDK11 case, does everyone on the cluster run on JDK11? or is it the MR job that is on JDK11? We have users running JDK in production. The NN GC performance was the only thing we were aware of. In the past we noticed because JDK11 uses G1GC by default, large NameNode performance was worse

[DISCUSS] Hadoop 3.3.5 release planning

Bumping this up. Adding the [DISCUSS] text to make this message stand out of your inbox. I certainly missed this message and only realized 3.3.5 has more than just security updates. What was the issue with the ARM64 build? I was able to publish ARM64 build for 3.3.1 release without problems.

Re: Hadoop BoF at ApacheCon NA'22

t; On Sat, Sep 24, 2022 at 8:52 PM 俊平堵 wrote: > > > > > Yes. It should be a short talk and I can only present remotely for this > > > time. That would be great if you help to coordinate. :) > > > > > > Thanks, > > > > > > Junping &g

Re: Hadoop BoF at ApacheCon NA'22

/1_ha1BFeEyIkAJtl5tJ8Z4Vf5Md_0rN23RCrXkyNCY1c/edit?usp=sharing Please add more details here. On Thu, Sep 22, 2022 at 10:27 PM 俊平堵 wrote: > Thanks Wei-Chiu. I am happy to share the status of Hadoop Meetups in China > (2019-2022) if that is a suitable topic. :) > > Thanks, > > Junping > >

Hadoop BoF at ApacheCon NA'22

We've not had a physical event for a long long time and we're way overdue for one. I'm excited to announce we've reserved a room at the upcoming ApacheCon for Birds-of-Feather on October 4th from 17:50-18:30 CDT in Rhythms I. I was also told that participants can stay after that until the hotel

Re: [DISCUSS] Enabling all platform builds in CI for all Hadoop PRs

Running builds for all platforms for each and every PR seems too excessive. How about doing all platform builds in the nightly jobs? On Fri, May 6, 2022 at 8:02 AM Steve Loughran wrote: > I'm not enthusiastic here as it not only makes the builds slower, it > reduces the #of builds we can

Re: [DISCUSS] Hadoop on Windows

Great! Sorry I missed the earlier discussion thread. Is there a target version for this support? I assume the milestone is still in a dev branch? On Thu, Apr 28, 2022 at 8:26 AM Gautham Banasandra wrote: > Hi Hadoop devs, > > I would like to announce that we recently reached a new milestone -

Re: Institutions running kerberized Hadoop clusters

Last time I checked (~2 years ago), there were thousands of Kerberized clusters among Cloudera customers. The largest ones had a few thousand nodes. What are you looking for? On Wed, Apr 13, 2022 at 7:22 AM Santosh Marella wrote: > Hey folks, > > Just curious if we have a list of

Re: [VOTE] Release Apache Hadoop 3.2.3 - RC0

aarch64 support is only introduced in/after 3.3.0 On Thu, Mar 17, 2022 at 2:27 PM Emil Ejbyfeldt wrote: > Hi, > > > There is no aarch64 artifact in the release candidate. Is this something > that is intended? > > Best, > Emil Ejbyfeldt > > On 14/03/2022 08:14, Masatake Iwasaki wrote: > > Hi

[ANNOUNCE] New Hadoop PMC Sun Chao

On behalf of the Apache Hadoop PMC, I am pleased to announce that Sun Chao(sunchao) has accepted the PMC's invitation to become a PMC on the project. We appreciate all of Sun's generous contributions thus far and look forward to his continued involvement. Congratulations and welcome, Sun!

Re: [ANNOUNCE] Apache Hadoop 3.3.2 release

> > Many thanks to Viraj Jasani, Michael Stack, Masatake Iwasaki, Xiaoqiao He, > Mukund Madhav Thakur, Wei-Chiu Chuang, Steve Loughran, Akira Ajisaka and > other folks who helped for this release process. > > Best Regards, > Chao >

Looking for help updating npm packages

Hello, Hadoop YARN has extensive use of npm packages. Many of the packages are outdated and even are associated with known CVEs. In many cases, updating the packages is as trivial as clicking on the "Create Dependabot security update" button in the "Dependabot alerts" tab (

[jira] [Created] (YARN-11068) Exclude transitive log4j2 dependency coming from solr-core 8

Wei-Chiu Chuang created YARN-11068: -- Summary: Exclude transitive log4j2 dependency coming from solr-core 8 Key: YARN-11068 URL: https://issues.apache.org/jira/browse/YARN-11068 Project: Hadoop YARN

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC2

I'll find time to check out the RC bits. I just feel bad that the tarball is now more than 600MB in size. On Fri, Jan 21, 2022 at 2:23 AM Steve Loughran wrote: > *+1 binding.* > > reviewed binaries, source, artifacts in the staging maven repository in > downstream builds. all good. > > *## test

Re: [DISCUSS] Migrate hadoop from log4j1 to log4j2

+1 I think it makes sense to use reload4j in maint releases. I have a draft PR doing this (https://github.com/apache/hadoop/pull/3906) log4j2 in Hadoop 3.4.0 makes sense to me. There could be incompatibilities introduced by log4j2, but I feel we should at least make it 3.4.0 a "preview" release,

Re: Hadoop-3.2.3 Release Update

Is this still making progress? On Tue, Oct 5, 2021 at 8:45 PM Brahma Reddy Battula wrote: > Hi Akira, > > Thanks for your email!! > > I am evaluating the CVE’s which needs to go for this release.. > > Will update soon!! > > > On Tue, 5 Oct 2021 at 1:46 PM, Akira Ajisaka wrote: > > > Hi Brahma,

[jira] [Resolved] (YARN-11047) ResourceManager and NodeManager unable to connect to Hbase when ATSv2 is enabled

[ https://issues.apache.org/jira/browse/YARN-11047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved YARN-11047. Resolution: Fixed > ResourceManager and NodeManager unable to connect to Hbase when AT

Apache Hadoop and CVE-2021-44228 Log4JShell vulnerability

Hi, Given the widespread attention to the recent log4j vulnerability (CVE-2021-44228), I'd like to share an update from the Hadoop developer community regarding the incident. As you probably know, Apache Hadoop depends on the log4j library to keep log files. The highlighted vulnerability

Trunk broken by HDFS-16384

My bad. There was a transitive dependency problem in the PR causing trunk to fail the build. The commit has since been reverted. Sorry for the inconvenience.

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC0

Thanks a lot for pushing it forward! A few things I noticed that we should incorporate: 1. the overview page of the doc is for the Hadoop 3.0 release. It would be best to base the doc on top of Hadoop 3.3.0 overview page. (it's a miss on my part... The overview page of 3.3.1 wasn't updated) 2.

ApacheCon@Home Big Data tracks recordings!

For those who missed the live Apache@Home Big Data tracks, the video recordings are being uploaded to the official ASF channel! Big Data: https://www.youtube.com/playlist?list=PLU2OcwpQkYCzXcumE9UxNirLF1IYLmARj Big Data Ozone:

Re: Hadoop-3.2.3 Release Update

Hi to raise the awareness, it looks like reverting the FoldedTreeSet HDFS-13671 breaks TestBlockManager in branch-3.2. Branch-3.3 is good. tracking jira: HDFS-16258 On Tue, Oct 5, 2021 at 8:45

Re: [DISCUSS] Tips for improving productivity, workflow in the Hadoop project?

ves > > costs to configure and debug the precommit jobs. > > https://issues.apache.org/jira/browse/HADOOP-17798 > > 2. Improve the pull request template for the contributors > > https://issues.apache.org/jira/browse/HADOOP-17799 > > > > Regards, > > Akira >

Re: [DISCUSS] Tips for improving productivity, workflow in the Hadoop project?

One more idea that I forgot to mention: We can have the Dependabot to automatically create a pull request whenever a vulnerability is discovered. On Wed, Jul 14, 2021 at 8:42 AM Wei-Chiu Chuang wrote: > So this is just up for discussion and welcome more brainstorming. I'm not > saying I

Re: [DISCUSS] Tips for improving productivity, workflow in the Hadoop project?

ebhooks? > - Just for discussion: is Integrating SonarQube worth the effort of > supporting the GitHub webhooks. SonarQube >is a double edge weapon and it needs dedicated man hours to weed out > the way through the reports. > > On Mon, Jul 12, 2021 at 11:18 PM Wei-Chiu Chuang &g

Re: [DISCUSS] Tips for improving productivity, workflow in the Hadoop project?

Here's another one: We can update the JIRA workflow and add more state. For example, Cassandra has "Review in Progress" and "Need Reviewer" states. INFRA-22049 <https://issues.apache.org/jira/browse/INFRA-22049> On Tue, Jul 13, 2021 at 11:34 AM Wei-Chiu Chuang wr

[DISCUSS] Tips for improving productivity, workflow in the Hadoop project?

I work on multiple projects and learned a bunch from those projects.There are nice add-ons that help with productivity. There are things we can do to help us manage the project better. 1. Add new issue types. We can add "Epic" jira type to organize a set of related jiras. This could be easier to

[ANNOUNCE] Apache Hadoop 3.3.1 release

olks who continued helps for this release process. Best Regards, Wei-Chiu Chuang

Re: [VOTE] Hadoop 3.1.x EOL

Dropped 3.1.4 from website. Removed from https://dist.apache.org/repos/dist/release/hadoop/common/ On Thu, Jun 10, 2021 at 3:42 PM Akira Ajisaka wrote: > This vote has passed with 18 binding +1. I'll update the JIRA and the wiki. > > Thanks all for your participation. > > On Tue, Jun 8, 2021 at

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

> > > +1, >> > > Built from Source. >> > > Successful Native Build on Ubuntu 20.04 >> > > Verified Checksums >> > > Ran basic hdfs shell commands. >> > > Ran simple MR jobs. >> > > Browsed NN,DN,RM and NM UI. >&g

[jira] [Resolved] (YARN-10809) testWithHbaseConfAtHdfsFileSystem consistently failing

[ https://issues.apache.org/jira/browse/YARN-10809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved YARN-10809. Fix Version/s: 3.3.2 3.4.0 Resolution: Fixed

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

ht. >> Created https://issues.apache.org/jira/browse/HADOOP-17746. >> >> This is *not a blocker* >> >> I can prepare a fix and we can have it in so that if any other changes >> come >> in the page will look OK. >> >> >> >> >

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

e release is a > big release, but I get that it’s the current state of the project. > > > On Jun 3, 2021, at 11:30 AM, Wei-Chiu Chuang wrote: > > > > Hello, > > do we want to extend the release vote? I understand a big release like > this > > takes time to

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

is faster by 20% over Hadoop 3.2 library. Looks like we may need some more time to test. How about extending it by a week? On Tue, Jun 1, 2021 at 6:29 PM Wei-Chiu Chuang wrote: > Hi community, > > This is the release candidate RC3 of Apache Hadoop 3.3.1 line. All blocker > issu

Re: [VOTE] Hadoop 3.1.x EOL

+1 On Thu, Jun 3, 2021 at 2:14 PM Akira Ajisaka wrote: > Dear Hadoop developers, > > Given the feedback from the discussion thread [1], I'd like to start > an official vote > thread for the community to vote and start the 3.1 EOL process. > > What this entails: > > (1) an official announcement

[VOTE] Release Apache Hadoop 3.3.1 RC3

Hi community, This is the release candidate RC3 of Apache Hadoop 3.3.1 line. All blocker issues have been resolved [1] again. There are 2 additional issues resolved for RC3: * Revert "MAPREDUCE-7303. Fix TestJobResourceUploader failures after HADOOP-16878 * Revert "HADOOP-16878. FileUtil.copy()

Re: [VOTE] Release Apache Hadoop Thirdparty 1.1.1 RC0

iled Hadoop trunk and branch-3.3 with Hadoop third-party 1.1.1. >> >> -Akira >> >> On Wed, May 26, 2021 at 5:29 PM Wei-Chiu Chuang >> wrote: >> > >> > Hi folks, >> > >> > I have put together a release candidate (RC0) for Hadoo

Re: [VOTE] Release Apache Hadoop Thirdparty 1.1.1 RC0

documents >> - Compiled Hadoop trunk and branch-3.3 with Hadoop third-party 1.1.1. >> >> -Akira >> >> On Wed, May 26, 2021 at 5:29 PM Wei-Chiu Chuang >> wrote: >> > >> > Hi folks, >> > >> > I have put together a release candidate

Re: [VOTE] Release Apache Hadoop 3.3.1 - RC1

irdparty 1.1.1 RC0 and cast a vote. Thanks! On Wed, May 26, 2021 at 9:39 AM Wei-Chiu Chuang wrote: > > Found a few issues building downstreams > > Ozone: HADOOP-17730 <https://issues.apache.org/jira/browse/HADOOP-17730> -- > we may have to make an hadoop-thirdpar

[VOTE] Release Apache Hadoop Thirdparty 1.1.1 RC0

Hi folks, I have put together a release candidate (RC0) for Hadoop Thirdparty 1.1.1 which will be consumed by Hadoop 3.3.1 RC2. The RC is available at: https://people.apache.org/~weichiu/hadoop-thirdparty-1.1.1-RC0/ The RC tag in svn is

Re: [VOTE] Release Apache Hadoop 3.3.1 - RC1

nError: expected null, but was:<1000> at org.apache.hadoop.hbase.TestHBaseConfiguration.testDeprecatedConfigurations(TestHBaseConfiguration.java:150) On Mon, May 24, 2021 at 10:36 PM Wei-Chiu Chuang wrote: > Hi community, > > This is the release candidate RC1 of Apache Had

[VOTE] Release Apache Hadoop 3.3.1 - RC1

Hi community, This is the release candidate RC1 of Apache Hadoop 3.3.1 line. All blocker issues have been resolved. It contains 697 fixed jira issues [2] since 3.3.0 which include a lot of features and improvements(read the full set of release notes). Below feature additions are the highlights

Re: [DISCUSS] which release lines should we still consider actively maintained?

Sean, For reasons I don't understand, I never received emails from your new address in the mailing list. Only Akira's response. I was just able to start a thread like this. I am +1 to EOL 3.1.5. Reason? Spark is already on Hadoop 3.2. Hive and Tez are actively working to support Hadoop 3.3.

Publishing Apache Hadoop 3.3.1-RC0 Preview bits

Hi community, This is the preview bits of the first release candidate of Apache Hadoop 3.3.1 line. It takes tremendous amount of effort to produce a Hadoop RC and we still have a number of unresolved issues [1]. Meanwhile, I'd like to publish preview bits so you guys can check out earlier than

Hadoop Branching: branch-3.3 is now 3.3.2-SNAPSHOT

Hi I created a branch-3.3.1. The branch-3.3. now tracks 3.3.2-SNAPSHOT. Please use Fix version 3.3.2 when cherrypicking your commits to branch-3.3. Regards, Wei-Chiu

Missing options and profiles in the create-release script

Hello, I'm trying to understand Hadoop's create-release script. Looks like it's missing a number of options and profiles that the regular precommit script does: https://github.com/apache/hadoop/blob/trunk/dev-support/bin/create-release#L133

Re: [DISCUSS] Hadoop 3.3.1 release

ys can quickly check out. The first RC will probably not have the aarch64 binary. I don't have much experience with that. Cheers, Wei-Chiu On Thu, May 13, 2021 at 9:04 AM Wei-Chiu Chuang wrote: > Hello it's me again. > > While working on the hadoop-thirdparty release, I would like to

Re: [VOTE] hadoop-thirdparty 1.1.0-RC0

-DskipTests > > - mvn clean install -DskipTests -Psrc > > * Built Hadoop trunk againt hadoop-thirdparty 1.1.0: ok > > > > > > On Thu, May 13, 2021 at 5:25 PM Wei-Chiu Chuang > > wrote: > > > > > Hello my fellow Hadoop developers, > > > > &g

[jira] [Created] (YARN-10778) Bump handlebars from 3.0.7 to 4.7.7 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp

Wei-Chiu Chuang created YARN-10778: -- Summary: Bump handlebars from 3.0.7 to 4.7.7 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp Key: YARN-10778 URL: https://issues.apache.org/jira/browse

[jira] [Resolved] (YARN-10777) Bump node-sass from 4.13.0 to 4.14.1 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp

[ https://issues.apache.org/jira/browse/YARN-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved YARN-10777. Fix Version/s: 3.4.0 Resolution: Fixed > Bump node-sass from 4.13.0 to 4.1

[jira] [Created] (YARN-10777) Bump node-sass from 4.13.0 to 4.14.1 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp

Wei-Chiu Chuang created YARN-10777: -- Summary: Bump node-sass from 4.13.0 to 4.14.1 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp Key: YARN-10777 URL: https://issues.apache.org/jira/browse

[jira] [Created] (YARN-10772) Stable API GetApplicationsRequest#newInstance compatibility broken by YARN-8363

Wei-Chiu Chuang created YARN-10772: -- Summary: Stable API GetApplicationsRequest#newInstance compatibility broken by YARN-8363 Key: YARN-10772 URL: https://issues.apache.org/jira/browse/YARN-10772

Updated Hadoop fix versions

Hi, I found hundreds of discrepancies between Apache JIRA and git history, and updated the jira fix versions according to git history. Some of the noticeable updates: (1) HADOOP-16730 ABFS: Support for Shared Access Signatures (SAS) --> originally a new feature in 3.3.1, is actually in 3.3.0.

[VOTE] hadoop-thirdparty 1.1.0-RC0

Hello my fellow Hadoop developers, I am putting together the first release candidate (RC0) for Hadoop-thirdparty 1.1.0. This is going to be consumed by the upcoming Hadoop 3.3.1 release. The RC is available at: https://people.apache.org/~weichiu/hadoop-thirdparty-1.1.0-RC0/ The RC tag in github

[jira] [Resolved] (YARN-10731) Please upgrade jquery to 3.6.0

[ https://issues.apache.org/jira/browse/YARN-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved YARN-10731. Resolution: Duplicate > Please upgrade jquery to 3.

Re: [DISCUSS] Hadoop 3.3.1 release

the remaining jiras and push out those that don't have much progress today. Thanks all! Weichiu On Tue, Apr 20, 2021 at 12:57 PM Wei-Chiu Chuang wrote: > Billie and Viraj, > > As of today we are at 657 resolved jiras in 3.3.1 > https://issues.apache.org/jira/issues/?filter=-1=project%20in%

Re: [DISCUSS] hadoop-thirdparty 1.1.0 release

with this change so please help review. Thanks! On Tue, Apr 27, 2021 at 2:59 PM Wei-Chiu Chuang wrote: > I'll start preparing the release vote for third-party 1.1.0. Thanks all > for help! > > On Mon, Apr 26, 2021 at 7:20 PM Wei-Chiu Chuang > wrote: > >> Thanks. I created

[DISCUSS] Lots of test failures on JDK11

I see lots of tests that fail consistently on JDK11. https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/160/testReport/ A number of them are YARN tests. I am preparing the 3.3.1 release and in the current state I am not confident in the quality of this release. Please help out!

Re: [DISCUSS] hadoop-thirdparty 1.1.0 release

I'll start preparing the release vote for third-party 1.1.0. Thanks all for help! On Mon, Apr 26, 2021 at 7:20 PM Wei-Chiu Chuang wrote: > Thanks. I created a Jenkins job to upload the SNAPSHOT to Apache nexus > repository. > > https://ci-hadoop.apache.org/view/Hadoop/job/Hadoo

[jira] [Created] (YARN-10752) Shaded guava not found when compiling with profile hbase2.0

Wei-Chiu Chuang created YARN-10752: -- Summary: Shaded guava not found when compiling with profile hbase2.0 Key: YARN-10752 URL: https://issues.apache.org/jira/browse/YARN-10752 Project: Hadoop YARN

Re: [DISCUSS] hadoop-thirdparty 1.1.0 release

Ayush Saxena wrote: > Yep, you have to do it manually > > -Ayush > > On 26-Apr-2021, at 3:23 PM, Wei-Chiu Chuang wrote: > >  > Does anyone know how we publish hadoop-thirdparty SNAPSHOT artifacts? > > The main Hadoop arifacts are published by this job > https:/

Re: [DISCUSS] hadoop-thirdparty 1.1.0 release

apshot, the hadoop code would pick > that up, and watch out everything is safe and clean before release. Unless > you have a better way to verify or already verified!!! > > -Ayush > > > On 23-Apr-2021, at 3:16 PM, Wei-Chiu Chuang > wrote: > > > > Another suggestion: look

Sorry for breaking the build

If you are seeing precommit failures, like this: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project hadoop-annotations: Compilation failure [ERROR] javac: invalid flag: -Xmaxwarns= [ERROR] Usage: javac [ERROR] use -help for

Re: [DISCUSS] hadoop-thirdparty 1.1.0 release

. The version used is too old. On Fri, Apr 23, 2021 at 10:55 AM Wei-Chiu Chuang wrote: > Hi community, > > In preparation of the Hadoop 3.3.1 release, I am starting a thread to > discuss its prerequisite: the release of hadoop-thirdparty 1.1.0. > > My plan: > update guava

[DISCUSS] hadoop-thirdparty 1.1.0 release

Hi community, In preparation of the Hadoop 3.3.1 release, I am starting a thread to discuss its prerequisite: the release of hadoop-thirdparty 1.1.0. My plan: update guava to 30.1.1 (latest). I have the PR ready to merge. Do we want to update protobuf and jaeger? Anything else? I suppose we

Re: [DISCUSS] Hadoop 3.3.1 release

the jira with the label 'release-blocker'. On Tue, Apr 20, 2021 at 2:44 AM Billie Rinaldi wrote: > I was thinking of backporting HADOOP-16948 for 3.3.1. > > Billie > > On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang > wrote: > > > Hello, reviving this thread. > > >

Re: [DISCUSS] Hadoop 3.3.1 release

; >> > > > > > >> >> I will come up with a plan.. Please let me know if anybody > has > > > some > > > > > >> >> features/improvements/bugs that need to be included. > > > > > >> >> >

[jira] [Resolved] (YARN-10730) Please upgrade log4j to log4j2

[ https://issues.apache.org/jira/browse/YARN-10730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved YARN-10730. Fix Version/s: (was: 3.4.0) (was: 3.3.0) Resolution

Re: [DISCUSS] Hadoop 3.3.1 release

gt; >> > Sunil > > >> >> > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack wrote: > > >> >> > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran > > >> >> > > >>

[DISCUSS] Hadoop 3.3.1 release

Hi all, Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated more than 400 changes in the branch-3.3. A number of downstreamers are eagerly waiting for 3.3.1 which addresses the guava version conflict issue.

[jira] [Resolved] (YARN-10575) Hadoop

[ https://issues.apache.org/jira/browse/YARN-10575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved YARN-10575. Resolution: Invalid > Hadoop > -- > > Key

[jira] [Resolved] (YARN-10541) capture the performance metrics of ZKRMStateStore

[ https://issues.apache.org/jira/browse/YARN-10541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved YARN-10541. Resolution: Fixed > capture the performance metrics of ZKRMStateSt

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC5

+1 (binding) On Mon, Jan 4, 2021 at 11:11 PM Akira Ajisaka wrote: > +1 (binding) > > - Verified the checksums and signatures > - Verified the source is the same as the RC5 tag > - Built from source with CentOS 7.9 and Java 1.8.0_275 > - Setup pseudo cluster and ran some FsShell commands > >

Re: [E] Re: [VOTE] Release Apache Hadoop 3.2.2 - RC4

We should target 3.4.0, 3.3.1 3.2.3 (or 3.2.2) I updated and set 3.3.0 released in YARN. Time to clean up some of the old releases. I'll do that. On Mon, Dec 21, 2020 at 4:04 PM Eric Badger wrote: > I've committed https://issues.apache.org/jira/browse/YARN-10540. Xiaoqiao, > feel free to

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC4

Forgot to mention I am +1 (binding) too based on the fact there are no blocker issues and RC3 and RC4 diff. On Mon, Dec 14, 2020 at 1:42 PM Wei-Chiu Chuang wrote: > Diff between RC3 and RC4: > > MAPREDUCE-7284. TestCombineFileInputFormat#testMissingBlocks fails (#2136) > HDFS-134

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC4

Diff between RC3 and RC4: MAPREDUCE-7284. TestCombineFileInputFormat#testMissingBlocks fails (#2136) HDFS-13404. Addendum: RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fail. Contributed by Takanobu Asanuma. HADOOP-16080. hadoop-aws does not work with hadoop-client-api

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC2

I suggest we wait for HDFS-15680 (If I understand it correctly, it's a blocker) I wish to update jetty too in this release, but it's turning out to be more involved than i had anticipated. On Fri, Nov 13, 2020 at 3:22 PM Chao Sun wrote: > +1

[jira] [Resolved] (YARN-10480) replace href tags with ng-href

[ https://issues.apache.org/jira/browse/YARN-10480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved YARN-10480. Fix Version/s: 3.3.1 3.4.0 Resolution: Fixed > replace href t

  1   2   3   >