Re: [DISCUSS] Migrate hadoop from log4j1 to log4j2

Hi Duo, Thank you for starting this discussion. Log4j1.2 bridge seems like a practical short-term solution. However the bridge will silently affect applications that add appenders or filters. NameNode audit logger and metrics come to mind. There may be others. Thanks, Arpit > On Jan 20,

Re: [DISCUSS] which release lines should we still consider actively maintained?

+1 to EOL 3.1.x at least. > On May 23, 2021, at 9:51 PM, Wei-Chiu Chuang > wrote: > > Sean, > > For reasons I don't understand, I never received emails from your new > address in the mailing list. Only Akira's response. > > I was just able to start a thread like this. > > I am +1 to EOL

Re: [VOTE] Apache Hadoop Ozone 0.5.0-beta RC2

+1 binding. - Verified hashes and signatures - Built from source - Deployed to 5 node cluster - Tried ozone shell and filesystem operations - Ran freon stress test for a while with write validation I couldn’t find the RC2 tag in the gitbox repo, although

Re: [DISCUSS] Hadoop 3.3.0 Release include ARM binary

ote: > > Sure, we can't make mandatory while voting and we can upload to downloads > once release vote is passed. > > On Tue, 17 Mar 2020 at 11:24 PM, Arpit Agarwal > wrote: > >>> Sorry,didn't get you...do you mean, once release voting is >>> processed and upload

Re: [DISCUSS] Hadoop 3.3.0 Release include ARM binary

> On Tue, Mar 17, 2020 at 10:59 PM Arpit Agarwal > wrote: > >> Can ARM binaries be provided after the fact? We cannot increase the RM’s >> burden by asking them to generate an extra set of binaries. >> >> >>> On Mar 17, 2020, at 10:23 AM, Brahma Reddy

Re: [DISCUSS] Hadoop 3.3.0 Release include ARM binary

Can ARM binaries be provided after the fact? We cannot increase the RM’s burden by asking them to generate an extra set of binaries. > On Mar 17, 2020, at 10:23 AM, Brahma Reddy Battula wrote: > > + Dev mailing list. > > -- Forwarded message - > From: Brahma Reddy Battula >

Re: [VOTE] Apache Hadoop Ozone 0.5.0-beta RC1

mpiled sources >> * Ran ozone smoke test against both binary and locally compiled versions >> >> Thanks Dinesh for RC1. >> >> -Attila >> >> On Sun, Mar 8, 2020 at 2:34 AM Arpit Agarwal >> wrote: >>> >>> +1 (binding) >>>

Re: [VOTE] Apache Hadoop Ozone 0.5.0-beta RC1

+1 (binding) Verified mds, sha512 Verified signatures Built from source Deployed to 3 node cluster Tried a few ozone shell and filesystem commands Ran freon load generator Thanks Dinesh for putting the RC1 together. > On Mar 6, 2020, at 4:46 PM, Dinesh Chitlangia wrote: > > Hi Folks, > > We

Re: [VOTE] Apache Hadoop Ozone 0.5.0-beta RC0

Hi Dinesh, Thanks for spinning up this RC! Looks like we still had ~15 issues that were tagged as Blockers for 0.5.0 in jira. I’ve moved out most of them, however the remaining 4 look like must fixes.

Re: [DISCUSS] Feature branch for HDFS-14978 In-place Erasure Coding Conversion

+1 > On Jan 23, 2020, at 2:51 PM, Jitendra Pandey wrote: > > +1 for the feature branch. > > On Thu, Jan 23, 2020 at 1:34 PM Wei-Chiu Chuang > wrote: > Hi we are working on a feature to improve Erasure Coding, and I would like > to seek your opinion on creating a feature branch for it.

Re: [VOTE] create ozone-dev and ozone-issues mailing lists

Thanks for kicking this off Marton. Submitted INFRA requests to create the following. The lists should be live soon. - ozone-dev@h.a.o - ozone-issues@h.a.o - ozone-commits@h.a.o > On Oct 31, 2019, at 3:32 AM, Elek, Marton wrote: > > > Thanks for all the votes and

Re: [VOTE] create ozone-dev and ozone-issues mailing lists

+1 > On Oct 27, 2019, at 1:25 AM, Elek, Marton wrote: > > > As discussed earlier in the thread of "Hadoop-Ozone repository mailing list > configurations" [1] I suggested to solve the current misconfiguration problem > with creating separated mailing lists (dev/issues) for Hadoop Ozone. > >

Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

+1 > On Sep 17, 2019, at 2:49 AM, Elek, Marton wrote: > > > > TLDR; I propose to move Ozone related code out from Hadoop trunk and store it > in a separated *Hadoop* git repository apache/hadoop-ozone.git > > > > > When Ozone was adopted as a new Hadoop subproject it was proposed[1] to

Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo

+1 > On Aug 23, 2019, at 7:05 PM, Wangda Tan wrote: > > Hi devs, > > This is a voting thread to move Submarine source code, documentation from > Hadoop repo to a separate Apache Git repo. Which is based on discussions of >

Re: VOTE: Hadoop Ozone 0.4.0-alpha RC2

Thanks for building this RC Ajay. +1 binding. - verified signatures and checksums - built from source - deployed to 3 node cluster, tried out basic operations - ran smoke tests - ran unit tests - LICENSE/NOTICE files look ok There is an extra file in the source root named JenkinsFile. > On

Re: VOTE: Hadoop Ozone 0.4.0-alpha RC1

Thanks Ajay for putting together this RC. Unfortunately HDDS-1425 looks like a blocker. We should make the docker experience smooth for anyone trying out 0.4.0. I’ve just committed Marton’s patch for HDDS-1425 this morning. Let’s roll a new

Re: [DISCUSS] Docker build process

Hi Eric, > Dockerfile is most likely to change to apply the security fix. I am not sure this is always. Marton’s point about revising docker images independent of Hadoop versions is valid. > When maven release is automated through Jenkins, this is a breeze > of clicking a button. Jenkins

Re: [VOTE] Release Apache Hadoop 3.1.2 - RC1

+1 binding for updated source package. - Rechecked signatures and checksums - Source matches release git tag - Built from source > On Feb 5, 2019, at 10:50 AM, Sunil G wrote: > > Thanks Billie for pointing out. > I have updated source by removing patchprocess and extra line create >

Re: [VOTE] Release Apache Hadoop 3.1.2 - RC1

+1 (binding) - Verified signatures - Verified checksums - Built from source - Verified Maven artifacts on staging repo - Deployed 3 node cluster - Tried out HDFS commands, MapReduce jobs. Confirmed the issues Billie pointed out. Not sure if you need to spin up a new RC or you can update the

Re: proposed new repository for hadoop/ozone docker images (+update on docker works)

I’ve requested a new repo hadoop-docker-ozone.git in gitbox. > On Jan 22, 2019, at 4:59 AM, Elek, Marton wrote: > > > > TLDR; > > I proposed to create a separated git repository for ozone docker images > in HDDS-851 (hadoop-docker-ozone.git) > > If there is no objections in the next 3 days

Re: [VOTE] Release Apache Hadoop Ozone 0.3.0-alpha (RC1)

+1 binding. - Verified signatures - Verified checksums - Checked LICENSE/NOTICE files - Built from source - Deployed to three node cluster and ran smoke tests. Thanks Marton for putting up the RC. On 2018/11/14, 9:14 AM, "Elek, Marton" wrote: Hi all, I've created the

Re: [VOTE] Release Apache Hadoop Ozone 0.2.1-alpha (RC0)

Thanks for putting together this release Marton. +1 (binding) - Verified signatures and checksums - LICENSE and NOTICE files exist in source and binary tarballs - Built from source code - Deployed to three node cluster - Tried out Ozone shell commands via 'ozone sh' - Tried out

Re: HADOOP-14163 proposal for new hadoop.apache.org

Requested a new git repo for the site: https://gitbox.apache.org/repos/asf/hadoop-site.git On 9/4/18, 1:33 PM, "Mingliang Liu" wrote: It might be late but I'm +1 on the new site and transition proposal. Thanks Marton. On Fri, Aug 31, 2018 at 1:07 AM Elek, Marton wrote:

Re: HADOOP-14163 proposal for new hadoop.apache.org

+1 Thanks for initiating this Marton. On 8/31/18, 1:07 AM, "Elek, Marton" wrote: Bumping this thread at last time. I have the following proposal: 1. I will request a new git repository hadoop-site.git and import the new site to there (which has exactly the same

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

Things are good. > > I can push this now which will restore trunk to its original. > I can do this if there are no objection. > > - Sunil > > On Fri, Jul 6, 2018 at 11:10 AM Arpit Agarwal > wrote: > > afaict YARN-8435 is s

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

s original. I can do this if there are no objection. - Sunil On Fri, Jul 6, 2018 at 11:10 AM Arpit Agarwal wrote: > afaict YARN-8435 is still in trunk. YARN-7556 and YARN-7451 are not. > > > From: Giovanni Matteo Fumarola > Date: F

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

afaict YARN-8435 is still in trunk. YARN-7556 and YARN-7451 are not. From: Giovanni Matteo Fumarola Date: Friday, July 6, 2018 at 10:59 AM To: Vinod Kumar Vavilapalli Cc: Anu Engineer , Arpit Agarwal , "su...@apache.org" , "yarn-dev@hadoop.apache.org" , "h

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

-1 for the force push. Nothing is broken in trunk. The history looks ugly for two commits and we can live with it. The revert restored the branch to Giovanni's intent. i.e. only YARN-8435 is applied. Verified there is no delta between hashes 0d9804d and 39ad989 (HEAD). 39ad989 2018-07-05

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

com> Date: Monday, April 2, 2018 at 9:25 PM To: Arpit Agarwal <aagar...@hortonworks.com> Cc: Gera Shegalov <ger...@gmail.com>, Sunil G <sun...@apache.org>, "yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>, Hdfs-dev <hdfs-...@hadoop.apache.org>

Re: [VOTE] Release Apache Hadoop 3.0.1 (RC1)

Hi Lei, It looks like the release artefacts have dummy shaded jars. E.g. Repository Path: /org/apache/hadoop/hadoop-client-runtime/3.0.1/hadoop-client-runtime-3.0.1.jar Uploaded by: lei Size: 44.47 KB Uploaded Date:Fri Mar 16 2018 15:50:42 GMT-0700 (PDT) Last Modified:

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

Thanks for putting together this RC, Wangda. The guidance from Apache is to omit MD5s, specifically: > SHOULD NOT supply a MD5 checksum file (because MD5 is too broken). https://www.apache.org/dev/release-distribution#sigs-and-sums On Apr 2, 2018, at 7:03 AM, Wangda Tan

Re: About reset branch-3.1 to trunk before release.

Thanks Wangda. On 3/19/18, 11:38 AM, "Wangda Tan" wrote: Done JIRA fix version update: Moved all JIRAs with fixVersion = 3.2.0 to 3.1.0 except following few fixes (which committed after 49c747ab187d0650143205ba57ca19607ec4c6bd) YARN-8002.

Re: Apache Hadoop 3.0.1 Release plan

Hi Aaron/Lei, Do you plan to roll an RC with an uncommitted fix? That isn't the right approach. This issue has good visibility and enough discussion. If there is a binding veto in effect then the change must be abandoned. Else you should be able to proceed with committing. However, 3.0.0 must

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

:36 AM To: Arpit Agarwal <aagar...@hortonworks.com> Cc: general <gene...@hadoop.apache.org>, "common-...@hadoop.apache.org" <common-...@hadoop.apache.org>, "yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>, "mapreduce-...@hadoo

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

of voting? Best, Andrew On Fri, Nov 17, 2017 at 1:27 PM, Arpit Agarwal <aagar...@hortonworks.com> wrote: > Hi Andrew, > > Thank you for your hard work in getting us to this step. This is our first > major GA release in many years.

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Hi Andrew, Thank you for your hard work in getting us to this step. This is our first major GA release in many years. I feel a 5-day vote window ending over the weekend before thanksgiving may not provide sufficient time to evaluate this RC especially for downstream components. Would you

Re: [VOTE] Release cadence and EOL

The ASF release policy says releases may not be vetoed [1] so the EOL policy sounds unenforceable. Not sure a release cadence is enforceable either since Release Managers are volunteers. 1. https://www.apache.org/dev/release.html#approving-a-release On 1/18/17, 7:06 PM, "Junping Du"

Re: [VOTE] Release Apache Hadoop 2.7.2 RC2

+1 (binding) - verified signatures - Installed pseudo-cluster from binary distribution and ran example MapReduce jobs - Built from sources and reran tests on pseudo-cluster On 1/14/16, 8:57 PM, "Vinod Kumar Vavilapalli" wrote: >Hi all, > >I've created an updated release

Re: [VOTE] Release Apache Hadoop 2.7.2 RC1

Thanks for putting together this release Vinod. +1 (binding) - Verified signatures - Started pseudo-cluster with binary distribution, verified git commit ID - Built sources, deployed pseudo-cluster and ran example map reduce jobs, DistributedShell, HDFS commands. PS: Extra file

Re: [VOTE] Release Apache Hadoop 2.6.3 RC0

+1 (binding) - Verified signatures for source and binary distributions - Built jars from source with java 1.7.0_79 - Deployed single-node pseudo-cluster - Ran example map reduce jobs - Ran hdfs admin commands, verified NN web UI shows expected usages On 12/11/15, 4:16 PM, "Junping Du"

Re: [VOTE] Release Apache Hadoop 2.7.1 RC0

Vinod, thanks for putting together this release. +1 (binding) - Verified signatures - Installed binary release on Centos 6 pseudo cluster * Copied files in and out of HDFS using the shell * Mounted HDFS via NFS and copied a 10GB file in and out over NFS * Ran example MapReduce

Re: Planning Hadoop 2.6.1 release

HDFS candidates for back-porting to Hadoop 2.6.1. The first two were requested in [1]. HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream should be non static HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization HDFS-7009. Active NN and

Re: [VOTE] Release Apache Hadoop 2.7.0 RC0

+1 for calling 2.7.0 an alpha. There are a couple of more issues related to incorrect handling of timestamps. 1. HDFS-8163 - Using monotonicNow for block report scheduling causes test failures on recently restarted systems 2. HDFS-8179 - DFSClient#getServerDefaults returns null within 1 hour of

Re: A 2.7.1 release to follow up 2.7.0

+1 for 2.7.1 and +1 for promoting it to 'stable', assuming it includes no new features or gratuitous improvements. Arpit On 4/9/15, 11:48 AM, Vinod Kumar Vavilapalli vino...@apache.org wrote: Hi all, I feel like we haven't done a great job of maintaining the previous 2.x releases. Seeing

Re: Erratic Jenkins behavior

Sorry I don¹t have much insight into this either. Our Jenkins setup is a bit of a black box to me. Too few of us have access to the build hosts and it makes debugging difficult. Perhaps we should grant access to all committers. On 2/9/15, 11:29 AM, Colin P. McCabe cmcc...@apache.org wrote: I'm

Re: Updates on migration to git

I cloned the new repo, built trunk and branch-2, verified all the branches are present. Also checked a few branches and the recent commit history matches our existing repo. Everything looks good so far. On Tue, Aug 26, 2014 at 1:19 PM, Karthik Kambatla ka...@cloudera.com wrote: The git

Re: [DISCUSS] Switch to log4j 2

The block state change logs are indeed too noisy at INFO and I've not found them useful when troubleshooting. Just filed HDFS-6860 to fix that. This is orthogal to SLF4J migration however moving to SLF4J would help ease the transition to Log4j 2. Thanks for the pointer to HDFS-5421 Aaron,

[DISCUSS] Switch to log4j 2

I don't recall whether this was discussed before. I often find our INFO logging to be too sparse for useful diagnosis. A high performance logging framework will encourage us to log more. Specifically, Asynchronous Loggers look interesting.

Re: [VOTE] Migration from subversion to git for version control

+1 binding. On Tue, Aug 12, 2014 at 6:08 AM, Thomas Graves tgra...@yahoo-inc.com.invalid wrote: +1 (binding). Tom On Friday, August 8, 2014 9:57 PM, Karthik Kambatla ka...@cloudera.com wrote: I have put together this proposal based on recent discussion on this topic. Please vote

[jira] [Created] (YARN-2384) Document YARN multihoming settings

Arpit Agarwal created YARN-2384: --- Summary: Document YARN multihoming settings Key: YARN-2384 URL: https://issues.apache.org/jira/browse/YARN-2384 Project: Hadoop YARN Issue Type: Bug

Re: [DISCUSS] Migrate from svn to git for source control?

+1 to voting on specific workflow(s). On Tue, Aug 5, 2014 at 5:49 PM, Karthik Kambatla ka...@cloudera.com wrote: If we are to start a vote thread, will people prefer a vote thread that includes potential workflows as well? On Tue, Aug 5, 2014 at 5:40 PM, Karthik Kambatla ka...@cloudera.com

Re: Hadoop Code of Incompatible Changes

I cleared out the wiki page and left a forwarding link to the site docs. From a quick scan all the content is included in the site docs. On Tue, Jul 29, 2014 at 2:14 PM, Sandy Ryza sandy.r...@cloudera.com wrote: Eli pointed out to me that this is the up-to-date compatibility guide:

[jira] [Created] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

Arpit Agarwal created YARN-1994: --- Summary: Expose YARN/MR endpoints on multiple interfaces Key: YARN-1994 URL: https://issues.apache.org/jira/browse/YARN-1994 Project: Hadoop YARN Issue Type

Re: [Vote] Merge branch-trunk-win to trunk

, Ramya Sunil, Bikas Saha, Kanna Karanam, John Gordon, Brandon Li, Chris Nauroth, David Lao, Sumadhur Reddy Bolli, Arpit Agarwal, Ahmed El Baz, Mike Liddell, Jing Zhao, Thejas Nair, Steve Maine, Ganeshan Iyer, Raja Aluri, Giridharan Kesavan, Ramya Bharathi Nimmagadda, Daryn Sharp, Arun Murthy, Tsz-Wo