Re: [VOTE] Release hadoop-2.0.3-alpha

2013-02-13 Thread Konstantin Shvachko
+1 Downloaded src and binaries. Built from sources on CentOS 6.3 Ran dfsio, slive, examples, terasort. Tried gridmix, but it failed with some NPEs. Built HBase 0.94 with Hadoop 2.0.3 Ran HBase shell commands. Recompiled and ran different loads of YCSB. Checked documentation and the release notes.

Re: [Vote] Merge branch-trunk-win to trunk

2013-03-01 Thread Konstantin Shvachko
-1 We should have a CI infrastructure in place before we can commit to supporting Windows platform. Eric is right Win/Cygwin was supported since day one. I had a Windows box under my desk running nightly builds back in 2006-07. People were irritated but I was filing windows bugs until 0.22 release

Re: [Vote] Merge branch-trunk-win to trunk

2013-03-01 Thread Konstantin Shvachko
sity of CI and related infrastructure to > support the platform well. Suresh outlined the support to effect this > here: http://s.apache.org/s1 > > Is the commitment to establish this infrastructure after the merge > sufficient? -C > > On Fri, Mar 1, 2013 at 12:18 PM, Konstantin Sh

Re: [Vote] Merge branch-trunk-win to trunk

2013-03-01 Thread Konstantin Shvachko
sh it to Apache Jenkins. Who is the volunteer for this work, please speak up when it can be done. Thanks, --Konstantin On Fri, Mar 1, 2013 at 6:03 PM, sanjay Radia wrote: > > On Mar 1, 2013, at 1:57 PM, Konstantin Shvachko wrote: > >> Commitment is a good thing. >> I think t

Re: [Vote] Merge branch-trunk-win to trunk

2013-03-02 Thread Konstantin Shvachko
ows machine as the nightly build. Such build will let people test their patches for Windows on Jenkins if they don't posses a license for the right version of Windows. I hope this will not turn into extraordinary or impractical effort. Thanks, --Konst > Thanks, > --Matt > > >

Re: [Vote] Merge branch-trunk-win to trunk

2013-03-03 Thread Konstantin Shvachko
> triggered by Jira "Patch Available" state) satisfy your request for > functionality #1 and #2? Yes or no, please. > > Thanks, > --Matt > > > On Sat, Mar 2, 2013 at 7:32 PM, Konstantin Shvachko > wrote: >> >> Hi Matt, >> >> On Sat, Mar

Re: [Vote] Merge branch-trunk-win to trunk

2013-03-04 Thread Konstantin Shvachko
ng them to Jiras. Thanks, --Konstantin > In agile terms, you are the Owner of these requirements. Please give me > owner feedback as to whether my proposed work sounds like it will satisfy > the requirements. > > Thank you, > --Matt > > > On Sun, Mar 3, 2013 at 12:16 PM

Re: [Vote] Merge branch-trunk-win to trunk

2013-03-04 Thread Konstantin Shvachko
an input >> parameter, which would let people run the build on their patches >> chosen from local machine rather than attaching them to Jiras. >> >> Thanks, >> --Konstantin >> >> > In agile terms, you are the Owner of these requirements. Please give me >>

Re: [Vote] Merge branch-trunk-win to trunk

2013-03-25 Thread Konstantin Shvachko
che.org/jira/browse/HADOOP-9359. Giri Kesavan has >> agreed >> > to help with the parts that require Jenkins admin access. >> > >> > Thanks, >> > --Matt >> > >> > >> > >> > On Mon, Mar 4, 2013 at 5:00 PM, Konstantin Shvachko <

Re: Heads up - 2.0.5-beta

2013-04-26 Thread Konstantin Shvachko
Arun, Could you please define the release plan and put it into vote. In accordance with the ByLaws. After this discussion of course. http://hadoop.apache.org/bylaws.html Release Plan Defines the timetable and actions for a release. The plan also nominates a Release Manager. Lazy majority of activ

Re: Heads up - 2.0.5-beta

2013-04-26 Thread Konstantin Shvachko
Arun, Suresh, Very exciting to hear about this final push to stable Hadoop 2. But I have a problem. Either with the plan or with the version number. I'll be arguing for the number change below rather than the plan. 1. Based on features listed by Suresh it looks that you plan a heavy feature-full

Re: Heads up - 2.0.5-beta

2013-04-30 Thread Konstantin Shvachko
x27;s integration testing proved to be very productive. Thanks, --Konstantin On Fri, Apr 26, 2013 at 6:06 PM, Arun C Murthy wrote: > Konstantin, > > On Apr 26, 2013, at 4:34 PM, Konstantin Shvachko wrote: > > > Do you think we can call the version you proposed to release >

Re: Heads up - 2.0.5-beta

2013-05-01 Thread Konstantin Shvachko
If there are no objections, I'll start a vote on this proposal now. Thanks, --Konstantin On Tue, Apr 30, 2013 at 4:28 PM, Konstantin Shvachko wrote: > Hi Arun, > > I am agnostic about version numbers too, as long as the count goes up. > The discussion you are referring to is

Re: Heads up - 2.0.5-beta

2013-05-01 Thread Konstantin Shvachko
On Wed, May 1, 2013 at 1:15 PM, Arun C Murthy wrote: > > On Apr 30, 2013, at 4:28 PM, Konstantin Shvachko wrote: > > > If the next release has to be 2.0.5 I would like to make an alternative > > proposal, which would include > > - stabilization of current 2.0.4 >

Re: Heads up - 2.0.5-beta

2013-05-02 Thread Konstantin Shvachko
tantin Shvachko wrote: > > > On Wed, May 1, 2013 at 1:15 PM, Arun C Murthy > wrote: > >> > >> On Apr 30, 2013, at 4:28 PM, Konstantin Shvachko wrote: > >> > >>> If the next release has to be 2.0.5 I would like to make an alternative > >>>

Re: Heads up - 2.0.5-beta

2013-05-02 Thread Konstantin Shvachko
On Thu, May 2, 2013 at 12:07 AM, Chris Douglas wrote: > Can anyone remember why we vote on release plans? -C To vote on features to include in the release. Thanks, --Konstantin

Re: Heads up - 2.0.5-beta

2013-05-03 Thread Konstantin Shvachko
Hi Arun and Suresh, I am glad my choice of words attracted your attention. I consider this important for the project otherwise I wouldn't waste everybody's time. You tend reacting on a latest message taken out of context, which does not reveal full picture. I'll try here to summarize my proposal a

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Konstantin Shvachko
Arun, I am glad I at least convinced you to finally announce your release plan and put it into vote. Even though it is to overrule the vote that just completed, which you were against and lost, well - Twice. I am glad you removed the NFS feature from the list proposed earlier. I think this vote

Re: [VOTE] - Release 2.0.5-beta

2013-05-17 Thread Konstantin Shvachko
ojects may have had such experience. A clarification on categorizing this action and on voting practices from ASF may help. Thanks, --Konstantin On Wed, May 15, 2013 at 3:36 PM, Konstantin Shvachko wrote: > Arun, > > I am glad I at least convinced you to finally announce your releas

Re: [VOTE] - Release 2.0.5-beta

2013-05-21 Thread Konstantin Shvachko
-1 for the record. This is a great plan for 2.1, which I would gladly support, but not for 2.0.5. I do not see how the previous vote could have been confusing, as it contained a direct quotation of the relative clause of Bylaws. Arun, the format of this vote remains confusing. What is the action

Re: [VOTE] - Release 2.0.5-beta

2013-05-21 Thread Konstantin Shvachko
releases of Hadoop. This is routine development, we are > failing at it, but we will recover by eliminating this pointless > ritual and getting back to producing software. -C > > On Fri, May 17, 2013 at 1:10 PM, Konstantin Shvachko > wrote: > > BCC: general@ > > > >

Re: [PROPOSAL] change in bylaws to remove Release Plan vote

2013-05-22 Thread Konstantin Shvachko
Couldn't reply yesterday. I will try to argue this is a useful action and that keeping it in Bylaws does not change regular release process. - Bylaws do not require to vote on every release plan. If nobody complains then it is a routine process of building a RC and voting on it. - It is useful to

Re: [DISCUSS] Ensuring Consistent Behavior for Alternative Hadoop FileSystems + Workshop

2013-05-24 Thread Konstantin Shvachko
Makes sense, Steve. There are a couple of guys here at WANdisco who will be interested in joining. Thanks, --Konstantin On Fri, May 24, 2013 at 10:15 AM, Milind Bhandarkar < [email protected]> wrote: > Thanks for the initiative, Steve. > > A few folks from Pivotal and our partners would

Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

2013-05-30 Thread Konstantin Shvachko
+1 I verified checksums, the signature, built sources on CentOS, ran tests and a few hadoop commands. Thanks, --Konst On Fri, May 24, 2013 at 8:48 PM, Konstantin Boudnik wrote: > All, > > I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha that I > would > like to release. > > Th

Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

2013-05-30 Thread Konstantin Shvachko
> Why not call this 2.0.5-alpha? Technically, current branch-2 uses 2.0.5-SNAPSHOT and produces maven artifacts with that version. So having another version with the same numbers will be confusing. Therefore 4-level numbers. I thought I mentioned it to you before. Thanks, --Konst On Thu, May 30

Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

2013-05-30 Thread Konstantin Shvachko
Sounds like a plan. Thanks, --Konst On Thu, May 30, 2013 at 8:41 PM, Alejandro Abdelnur wrote: > Konstantin, Cos, > > As we change from 2.0.4.1 to 2.0.5 you'll need to do the following > housekeeping as you work the new RC. > > * rename the svn branch > * update the versions in the POMs > * upd

Re: [VOTE] Release Apache Hadoop 2.0.5-alpha (rc1)

2013-06-02 Thread Konstantin Shvachko
+1 Did basic verification and testing. Thanks, --Konst On Fri, May 31, 2013 at 9:27 PM, Konstantin Boudnik wrote: > All, > > I have created a release candidate (rc1) for hadoop-2.0.5-alpha that I > would > like to release. > > This is a stabilization release that includes fixed for a couple a

Re: [VOTE] Release Apache Hadoop 2.0.5-alpha (rc2)

2013-06-03 Thread Konstantin Shvachko
+1 Thanks, --Konst On Mon, Jun 3, 2013 at 12:51 PM, Konstantin Boudnik wrote: > I have rolled out release candidate (rc2) for hadoop-2.0.5-alpha. > > The difference between rc1 and rc2 is the "optimistic release date" is set > for > 06/06/2013 in the CHANGES.txt files. > > The binary artifact

Re: [VOTE] Release Apache Hadoop 0.23.8

2013-06-03 Thread Konstantin Shvachko
+1 I did basic verification and testing of the rc. Thanks, --Konst On Tue, May 28, 2013 at 9:00 AM, Thomas Graves wrote: > > I've created a release candidate (RC0) for hadoop-0.23.8 that I would like > to release. > > This release is a sustaining release with several important bug fixes in > it

Re: [VOTE] Push back code freeze for 0.21

2009-07-24 Thread Konstantin Shvachko
I would like to clarify the append plans. Right now according to our planning schedule we need about 8 weeks to complete the implementation of the design. This will include all the functionality and unit testing according to the test plan. September 4 deadline gives us about 6 weeks to commit the

Re: Changes to HDFS configuration keys for 21 release

2009-09-28 Thread Konstantin Shvachko
+1. Good plan. --Konstantin Jitendra Nath Pandey wrote: Hi, The changes to configuration keys in HDFS for 21 release are scheduled to be checked in along with the append changes. This change was delayed because configuration changes are too many and merging append code on top of them woul

Re: HDFS-758 in Hadoop-21 , Updates to Namenode health page

2009-11-25 Thread Konstantin Shvachko
+1 I am in favor of committing this to 0.21 because imo it is not a new HDFS feature but rather an improvement of web UI. Allen Wittenauer wrote: > Then you'll have no issues patching other things in 0.21 that are actual > bug fixes that also meet this criteria, right? Or does this only appl

Re: Namespace partitioning using Locality Sensitive Hashing

2010-03-01 Thread Konstantin Shvachko
Hi Ketan, AFAIU, hashing is used to map files and directories into different name-nodes. Suppose you use a simple hash function on a file path h(path), and that files with the same hash value (or within a hash range) are mapped to the same name-node. Then files with the same parent will be rando

Re: Namespace partitioning using Locality Sensitive Hashing

2010-03-01 Thread Konstantin Shvachko
confusing to me) Can you please provide insight into this? Thanks, Ketan On Mon, Mar 1, 2010 at 3:26 PM, Konstantin Shvachko wrote: Hi Ketan, AFAIU, hashing is used to map files and directories into different name-nodes. Suppose you use a simple hash function on a file path h(path), and that

Re: [DISCUSSION] Release process

2010-03-31 Thread Konstantin Shvachko
HDFS 0.20 does not have a reliable append. Also it is (was last time I looked) incompatible with the 0.21 append HDFS-256. That wouldn't be a problem if that was the only incompatibility. But it's not. If 1.0 is re-labeled or re-branched from 0.20 we will have to many incompatibilities going int

Re: [DISCUSSION] Release process

2010-03-31 Thread Konstantin Shvachko
On 3/31/2010 2:19 PM, Doug Cutting wrote: > Konstantin Shvachko wrote: >> I would like to propose a straightforward release of 0.21 from current >> 0.21 branch. > > That could be done too. Would you like to volunteer to drive a release from > the current 0.21 branch

Re: [DISCUSSION]: Integrating SureLogic into Hadoop

2010-05-14 Thread Konstantin Shvachko
This is good news! I found SureLogic stack useful for finding bugs. It was especially helpful in detecting synchronization issues. Good that the licensing issues are cleared out. Thanks, Cos. --Konstantin On 5/14/2010 12:59 PM, Konstantin Boudnik wrote: Here's an update from SureLogic on the li

Re: Multi-Master Hadoop Configuration

2010-10-28 Thread Konstantin Shvachko
Yes. Only one master, called name-node, is managing HDFS, and only one master, called job tracker, is managing MapReduce cluster. You can also read some online documentation, may be even publications. Thanks, --Konstantin On Wed, Oct 27, 2010 at 3:28 PM, Wang, Chengwei wrote: > Thanks for pointi

Hadoop-common-trunk-Commit is failing since 01/19/2011

2011-01-28 Thread Konstantin Shvachko
I see Hadoop-common-trunk-Commit is failing and not sending any emails. It times out on native compilation and aborts. Therefore changes are not integrated, and now it lead to hdfs and mapreduce both not compiling. Can somebody please take a look at this. The last few lines of the build are below.

Re: Hadoop-common-trunk-Commit is failing since 01/19/2011

2011-01-31 Thread Konstantin Shvachko
be restored promptly. Thanks, --Konstantin On Fri, Jan 28, 2011 at 5:53 PM, Konstantin Shvachko wrote: > I see Hadoop-common-trunk-Commit is failing and not sending any emails. > It times out on native compilation and aborts. > Therefore changes are not integrated, and now it lead to

Re: Hadoop-common-trunk-Commit is failing since 01/19/2011

2011-01-31 Thread Konstantin Shvachko
rom HADOOP-6904 is MAPREDUCE-2290, > > which was fixed. Trees from trunk are compiling against each other > > for me (eg each installed to a local maven repo), perhaps the upstream > > maven repo hasn't been updated with the latest bits yet. > > > > Thanks, &

VOTE: Committing HADOOP-6949 to 0.22 branch

2011-03-28 Thread Konstantin Shvachko
HADOOP-6949 introduced a very important optimization to the RPC layer. Based on the benchmarks presented in HDFS-1583 this provides an order of magnitude improvement of current RPC implementation. RPC is a common component of Hadoop projects. Many of them should benefit from this change. But since

Re: VOTE: Committing HADOOP-6949 to 0.22 branch

2011-03-29 Thread Konstantin Shvachko
y. It changes the RPC wire > >>> format, but we already require that clients and servers run identical > >>> builds. No application that ran with a prior version of Hadoop would > be > >>> broken by this change when it upgrades to this version of Hadoop.

Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-03 Thread Konstantin Shvachko
I think its a good idea to release hadoop-0.20.203. It moves Apache Hadoop a step forward. Looks like the technical difficulties are resolved now with latest Arun's commits. Being a superset of hadoop-0.20.2 it can be considered based on one of the official Apache releases. I don't think there was

Re: [VOTE] Release candidate 0.20.203.0-rc0

2011-05-07 Thread Konstantin Shvachko
s the tests. Thanks, --Konstantin On Tue, May 3, 2011 at 1:39 AM, Konstantin Shvachko wrote: > I think its a good idea to release hadoop-0.20.203. It moves Apache Hadoop > a step forward. > > Looks like the technical difficulties are resolved now with latest Arun's > commi

Hadoop and HBase to common Avro base

2011-09-16 Thread Konstantin Shvachko
Guys, Joep is trying to converge Hadoop and HBase under the common Avro version. Seems like a reasonable direction. Could the specialists please take a look. https://issues.apache.org/jira/browse/HADOOP-7646 Thanks, --Konstantin

[ANNOUNCEMENT] Apache Hadoop 0.22.0 release

2011-12-12 Thread Konstantin Shvachko
On December 10, 2011 Hadoop PMC voted to release Hadoop 0.22.0 See http://s.apache.org/COC The release have been brewing for one year. It incorporates over 700 jiras fixed since the release of 0.21.0. It is good to have it finalized. Please note that 0.22.0 release does not support security. For

Re: [ANNOUNCEMENT] Apache Hadoop 0.22.0 release

2011-12-13 Thread Konstantin Shvachko
, Dec 12, 2011 at 3:33 PM, Konstantin Shvachko > wrote: > >> On December 10, 2011 Hadoop PMC voted to release Hadoop 0.22.0 >> See http://s.apache.org/COC >> >> The release have been brewing for one year. >> It incorporates over 700 jiras fixed since the release of 0.21.

Re: [ANNOUNCEMENT] Apache Hadoop 0.22.0 release

2011-12-14 Thread Konstantin Shvachko
but does this become 2.0 then? > > Tom > > > On 12/12/11 2:33 PM, "Konstantin Shvachko" wrote: > >> On December 10, 2011 Hadoop PMC voted to release Hadoop 0.22.0 >> See http://s.apache.org/COC >> >> The release have been brewing for one year. >>

Re: [VOTE] Release Apache Hadoop 2.7.2 RC0

2015-12-15 Thread Konstantin Shvachko
Sorry for bringing this up late. I think we should pick up HDFS-9516 for this release. Rather critical bug fix, but up to you, Vinod. Thanks, --Konst On Wed, Nov 11, 2015 at 8:31 PM, Vinod Kumar Vavilapalli wrote: > Hi all, > > > I've created a release candidate RC0 for Apache Hadoop 2.7.2. > >

Re: [Important] Lots of 2.8.0 commits are in branch-2 only

2015-12-17 Thread Konstantin Shvachko
Sounds like branch-2.8 was cut off prematurely. What is the point of forking off if the release is not imminent. We don't want this thing branching like a banyan again, with each commit going into 5 branches. I think it would be easier to retire branch-2.8 for now, and reset it to branch-2.9 when

Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845

2013-08-12 Thread Konstantin Shvachko
Ok. After installing protobuf 2.5.0 I can compile trunk. But now I cannot compile Hadoop-2 branches. None of them. So if I switch between branches I need to reinstall protobuf? Is there a consensus about going towards protobuf 2.5.0 upgrade in ALL versions? I did not get definite impression there

Re: [VOTE] Release Apache Hadoop 2.0.6-alpha

2013-08-13 Thread Konstantin Shvachko
+1 Verified checksums, signatures. Checked release notes. Built the sources and ran tests. Started a small cluster. Tried hadoop commands, ran a few jobs. Thanks, --Konst On Sat, Aug 10, 2013 at 5:46 PM, Konstantin Boudnik wrote: > All, > > I have created a release candidate (rc0) for hadoop-

Re: [VOTE] Release Apache Hadoop 2.0.6-alpha (RC1)

2013-08-20 Thread Konstantin Shvachko
+1 Did the same as with rc0. Works for me. Thanks, --Konst On Thu, Aug 15, 2013 at 10:29 PM, Konstantin Boudnik wrote: > All, > > I have created a release candidate (rc1) for hadoop-2.0.6-alpha that I > would > like to release. > > This is a stabilization release that includes fixed for a cou

Re: Re-swizzle 2.3

2014-02-10 Thread Konstantin Shvachko
Sorry for the last minute request. Can we add HDFS-4858 to the release, please? It solves pretty important bug related to failover. I can commit momentarily if there are no objections. Thanks, --Konstantin On Mon, Feb 10, 2014 at 4:50 PM, Aaron T. Myers wrote: > Just committed a fix for HDFS-5

Re: HADOOP JIRA near full of contributors, error message not helpful

2014-02-14 Thread Konstantin Shvachko
Had the same problem recently. BTW it's the same for common and HDFS. I ended up adding contributors via jira CLI. Worked well. I guess it's the web UI limitation. Thought this may be useful for others. Thanks, --Konst On Fri, Feb 14, 2014 at 11:30 AM, Jakob Homan wrote: > FYI, ran into troubl

Introducing ConsensusNode and a Coordination Engine

2014-05-29 Thread Konstantin Shvachko
Hello hadoop developers, I just opened two jiras proposing to introduce ConsensusNode into HDFS and a Coordination Engine into Hadoop Common. The latter should benefit HDFS and HBase as well as potentially other projects. See HDFS-6469 and HADOOP-10641 for details. The effort is based on the syst

Re: [VOTE] Change by-laws on release votes: 5 days instead of 7

2014-06-29 Thread Konstantin Shvachko
+1 Thanks, --Konstantin On Tue, Jun 24, 2014 at 1:53 AM, Arun C Murthy wrote: > Folks, > > As discussed, I'd like to call a vote on changing our by-laws to change > release votes from 7 days to 5. > > I've attached the change to by-laws I'm proposing. > > Please vote, the vote will the usua

Re: Meetup invitation: Consensus based replication in Hadoop

2014-07-11 Thread Konstantin Shvachko
would the week of July 7th looks for yall? > > > Tuesday or Thursday look pretty good on our end. > > > > > > Please chime in on your preference either here or reach of directly to > me. > > > Once I have a few RSVPs I will setup an event on Eventbrite or similar. > >

Re: Meetup invitation: Consensus based replication in Hadoop

2014-07-11 Thread Konstantin Shvachko
Ok, or Cos. On Fri, Jul 11, 2014 at 4:41 PM, Konstantin Shvachko wrote: > Few people asked about pick up from Bart. > We can organize pick up from either West Dublin/Pleasanton Station or > Walnut Creek Station. > Whichever gets more requests until Monday 07/14. > Please ping

Re: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

2018-02-22 Thread Konstantin Shvachko
Hi Sanjay, With respect to Ozone my two main concerns were: 1. Wether Ozone can help scaling out the namespace service in handling higher RPC workloads. I think we came to common conclusion that using Ozone as a block management layer is a reasonable path to scaling HDFS. The discussions are in-pr

[DISCUSS] Apache Hadoop 2.7.6 Release Plan

2018-02-22 Thread Konstantin Shvachko
Planning to do a maintenance 2.7.6 release with close to 40 changes since the last release. If there are no objections I'll start preparations. Please let me know if there are blockers: https://issues.apache.org/jira/browse/HDFS-12641?filter=12343253 Thanks, --Konstantin

Re: [DISCUSS] 2.9+ stabilization branch

2018-02-27 Thread Konstantin Shvachko
Thanks Subru for initiating the thread about GPU support. I think the path of taking 2.9 as a base for 2.10 and adding new resource types into it is quite reasonable. That way we can combine stabilization effort on 2.9 with GPUs. Arun, upgrading Java is probably a separate topic. We should discuss

Re: [VOTE] Merging branch HDFS-7240 to trunk

2018-03-18 Thread Konstantin Shvachko
The proposal to add it as a subproject of Hadoop makes sense to me. Thank you Owen. I am glad to have a path for scaling HDFS further, especially as it enters areas like IoT and self-driving cars, where storage requirements are huge. I am not very fond of the name HDSL, though. "Storage Layer" sou

Re: [DISCUSS] Apache Hadoop 2.7.6 Release Plan

2018-04-06 Thread Konstantin Shvachko
Just heads up, working on the release candidate now. It's been a while, I know. But we had some blockers. Thanks, --Konstantin On Thu, Feb 22, 2018 at 5:59 PM, Konstantin Shvachko wrote: > Planning to do a maintenance 2.7.6 release with close to 40 changes since > the last release.

[VOTE] Release Apache Hadoop 2.7.6 (RC0)

2018-04-09 Thread Konstantin Shvachko
Hi everybody, This is the next dot release of Apache Hadoop 2.7 line. The previous one 2.7.5 was released on December 14, 2017. Release 2.7.6 includes critical bug fixes and optimizations. See more details in Release Note: http://home.apache.org/~shv/hadoop-2.7.6-RC0/releasenotes.html The RC0 is

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-10 Thread Konstantin Shvachko
A note to release managers. As discussed in https://issues.apache.org/jira/browse/HADOOP-15205 We are producing release artifacts without sources jars. See e.g. https://repository.apache.org/content/repositories/releases/org/apache/hadoop/hadoop-common/3.1.0/ I believe this has something to do with

Re: [VOTE] Release Apache Hadoop 3.0.2 (RC0)

2018-04-10 Thread Konstantin Shvachko
A note to release managers. As discussed in https://issues.apache.org/jira/browse/HADOOP-15205 We are producing release artifacts without sources jars. See e.g. https://repository.apache.org/content/repositories/releases/org/apache/hadoop/hadoop-common/3.0.1/ Based on what is staged on Nexus for 3.

Re: [VOTE] Release Apache Hadoop 3.0.2 (RC0)

2018-04-13 Thread Konstantin Shvachko
Hi Lei, Did you have any luck with deploy? Could you please post your findings on https://issues.apache.org/jira/browse/HADOOP-15205 Thanks, --Konstantin On Tue, Apr 10, 2018 at 12:10 PM, Lei Xu wrote: > Ajay, thanks for spotting this. > > I am working on fix the deploy. > > On Tue, Apr 10, 20

Re: [VOTE] Release Apache Hadoop 2.7.6 (RC0)

2018-04-17 Thread Konstantin Shvachko
My formal +1 for 2.7.6 RC0 Thanks, --Konstantin On Mon, Apr 9, 2018 at 4:14 PM, Konstantin Shvachko wrote: > Hi everybody, > > This is the next dot release of Apache Hadoop 2.7 line. The previous one > 2.7.5 was released on December 14, 2017. > Release 2.7.6 includes critica

[RESULT] [VOTE] Release Apache Hadoop 2.7.6 (RC0)

2018-04-17 Thread Konstantin Shvachko
Hi everybody, With 4 binding and 4 non-binding +1s and no -1s the vote for Apache Release 2.7.6 passes. Thank you everybody for contributing to the release, testing, and voting. Binding +1s Zhe Zhang Brahma Reddy Battula Jason Lowe Konstantin Shvachko Non-binding +1s Chen Liang Erik Krogen

Re: Hadoop 3.2 Release Plan proposal

2018-10-24 Thread Konstantin Shvachko
I've tried to attract attention to an incompatibility issue through the jira, but it didn't work. So pitching in in this thread. https://issues.apache.org/jira/browse/HDFS-12026 It introduced binary incompatibility, which will prevent people from upgrading from 3.1 to 3.2. I think it can get messy

Re: Hadoop 3.2 Release Plan proposal

2018-10-25 Thread Konstantin Shvachko
ntin for pointing out. > As 3.2 is pretty much on RC level, its better we try to find a good > solution to this issue. > > I ll follow up on this in the jira. > > - Sunil > > On Thu, Oct 25, 2018 at 11:35 AM Konstantin Shvachko > wrote: > >> I've tried to attract

Re: [DISCUSS] Hadoop RPC encryption performance improvements

2018-11-01 Thread Konstantin Shvachko
Hi Wei-Chiu, Thanks for starting the thread and summarizing the problem. Sorry for slow response. We've been looking at the encrypted performance as well and are interested in this effort. We ran some benchmarks locally. Our benchmarks also showed substantial penalty for turning on wire encryption

[VOTE] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

2018-12-05 Thread Konstantin Shvachko
Hi Hadoop developers, I would like to propose to merge to trunk the feature branch HDFS-12943 for Consistent Reads from Standby Node. The feature is intended to scale read RPC workloads. On large clusters reads comprise 95% of all RPCs to the NameNode. We should be able to accommodate higher overa

Re: [VOTE] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

2018-12-06 Thread Konstantin Shvachko
nd why > #2 is not needed for the feature to complete? > 2. Need to fix automatic failover with ZKFC. Currently it does not doesn't > know about ObserverNodes trying to convert them to SBNs. > > Thanks. > --Yongjun > > > On Wed, Dec 5, 2018 at 5:27 PM Konstantin Shvachk

Re: [VOTE] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

2018-12-07 Thread Konstantin Shvachko
te? >> 2. Need to fix automatic failover with ZKFC. Currently it does not doesn't >> know about ObserverNodes trying to convert them to SBNs. >> >> Thanks. >> --Yongjun >> >> >> On Wed, Dec 5, 2018 at 5:27 PM Konstantin Shvachko >> wrot

[Result] [VOTE] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

2018-12-13 Thread Konstantin Shvachko
er of this community. Please check your availability for followup discussions if you choose to get involved with important decisions. On Fri, Dec 7, 2018 at 4:10 PM Konstantin Shvachko wrote: > Hi Daryn, > > Wanted to backup Chen's earlier response to your concerns about rotating

Re: [VOTE] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

2018-12-13 Thread Konstantin Shvachko
luster with real workload. > > Comments? > > --Yongjun > > On Fri, Dec 7, 2018 at 4:10 PM Konstantin Shvachko > wrote: > >> Hi Daryn, >> >> Wanted to backup Chen's earlier response to your concerns about rotating >> calls in the call queue. >&g

[VOTE - 2] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

2018-12-14 Thread Konstantin Shvachko
Hi Hadoop developers, I would like to propose to merge to trunk the feature branch HDFS-12943 for Consistent Reads from Standby Node. The feature is intended to scale read RPC workloads. On large clusters reads comprise 95% of all RPCs to the NameNode. We should be able to accommodate higher overa

[Result] [VOTE - 2] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

2018-12-21 Thread Konstantin Shvachko
Obviously +1 from me. With four binding +1s, two non-binding +1s, and no -1s this vote passes. Thank you folks for working on the feature and for voting. Will do the merge in bit. Thanks, --Konst On Fri, Dec 14, 2018 at 6:16 PM Konstantin Shvachko wrote: > Hi Hadoop developers, > >

Re: [Result] [VOTE - 2] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

2018-12-24 Thread Konstantin Shvachko
e.. Great work. > > -Original Message- > From: Konstantin Shvachko [mailto:[email protected]] > Sent: Saturday, December 22, 2018 6:48 AM > To: Hadoop Common ; hdfs-dev < > [email protected]> > Cc: [email protected]; [email protected]

Re: [DISCUSS] Moving branch-2 to java 8

2019-02-01 Thread Konstantin Shvachko
Just to make sure we are on the same page, as the subject of this thread is too generic and confusing. *The proposal is to move branch-2 Jenkins builds such as precommit to run tests on openJDK-8.* We do not want to break Java 7 source compatibility. The sources and releases will still depend on Ja

Re: [VOTE] Moving branch-2 precommit/nightly test builds to java 8

2019-02-05 Thread Konstantin Shvachko
+1 Makes sense to me. Thanks, --Konst On Mon, Feb 4, 2019 at 6:14 PM Jonathan Hung wrote: > Hello, > > Starting a vote based on the discuss thread [1] for moving branch-2 > precommit/nightly test builds to openjdk8. After this change, the test > phase for precommit builds [2] and branch-2 night

Re: HDFS Scalability Limit?

2019-06-18 Thread Konstantin Shvachko
Hi Wei-Chiu, We run similar Hadoop installations as Kihwal describes. Thanks for sharing Kihwal. Our clusters are also not Federated. So far the growth rate is the same as we reported in our talk last year (slide #5) 2x per year: https://www.slideshare.net/Hadoop_Summit/scaling-hadoop-at-linkedin-

Re: [DISCUSS] Release 3.0.4 or branch-3.0 EOL

2019-08-15 Thread Konstantin Shvachko
Hey Wei-Chiu, Thanks for sharing your plans on Hadoop 3. Good to know. 1. So I understand people don't care about Hadoop 3.0 any more. I see we don't even list this line on the Releases page: https://hadoop.apache.org/releases.html I think it is fair to EOL 3.0 and stop making changes to it. 2.

Re: Make EOL branches to public?

2019-08-25 Thread Konstantin Shvachko
I would also suggest making an explicit commit to the branch stating it is EOL. Thanks, --Konstantin On Tue, Aug 20, 2019 at 7:59 PM Wangda Tan wrote: > Thank you all for suggestions. Let me send a vote email to mark 2.6, 2.7, > 3.0 EOL. > > - Wangda > > On Wed, Aug 21, 2019 at 9:34 AM Akira Aj

Re: [VOTE] Mark 2.6, 2.7, 3.0 release lines EOL

2019-08-27 Thread Konstantin Shvachko
+1 for the proposal. I thought we already EOL-ed 2.6 though some time ago. Thanks, --Konstantin On Tue, Aug 20, 2019 at 8:03 PM Wangda Tan wrote: > Hi all, > > This is a vote thread to mark any versions smaller than 2.7 (inclusive), > and 3.0 EOL. This is based on discussions of [1] > > This d

Re: [VOTE] Merge YARN-8200 to branch-2 and branch-3.0

2019-08-27 Thread Konstantin Shvachko
+1 for the merge. We probably should not bother with branch-3.0 merge since it's been voted EOL. Thanks, --Konstantin On Thu, Aug 22, 2019 at 4:43 PM Jonathan Hung wrote: > Hi folks, > > As per [1], starting a vote to merge YARN-8200 (and YARN-8200.branch3) > feature branch to branch-2 (and br

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-21 Thread Konstantin Shvachko
+1 on RC0. - Verified signatures - Built from sources - Ran unit tests for new features - Checked artifacts on Nexus, made sure the sources are present. Thanks --Konstantin On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung wrote: > Hi folks, > > This is the first release candidate for the first re

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

2019-10-23 Thread Konstantin Shvachko
+1 on RC1 - Verified signatures - Verified maven artifacts on Nexus for sources - Checked rat reports - Checked documentation - Checked packaging contents - Built from sources on RHEL 7 box - Ran unit tests for new HDFS features with Java 8 Thanks, --Konstantin On Tue, Oct 22, 2019 at 2:55 PM Jo

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-11-18 Thread Konstantin Shvachko
Hi Eric, We had a long discussion on this list regarding making the 2.10 release the last of branch-2 releases. We intended 2.10 as a bridge release between Hadoop 2 and 3. We may have bug-fix releases or 2.10, but 2.11 is not in the picture right now, and many people may object this idea. I unde

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-11-27 Thread Konstantin Shvachko
they should >>> commit >>> their patches. >>> >>> Eric >>> >>> [1] >>> >>> https://hadoop.apache.org/docs/r2.10.0/hadoop-project-dist/hadoop-common/Compatibility.html >>> [2] >>> >>> https://hadoop.apach

Re: [VOTE] Moving Ozone to a separated Apache project

2020-09-29 Thread Konstantin Shvachko
+1 Stay safe, --Konstantin On Thu, Sep 24, 2020 at 10:59 PM Elek, Marton wrote: > Hi all, > > Thank you for all the feedback and requests, > > As we discussed in the previous thread(s) [1], Ozone is proposed to be a > separated Apache Top Level Project (TLP) > > The proposal with all the detail

Re: Regarding HDFS-15567. HDFS should expose msync() API to allow downstream applications call it explicitly

2020-12-13 Thread Konstantin Shvachko
Hi Steve, I am not sure I fully understand what is broken here. It is not an incompatible change, right? Could you please explain what you think the process is. Would be best if you could share a link to a document describing it. I would be glad to follow up with tests and documentation that are n

Re: Regarding HDFS-15567. HDFS should expose msync() API to allow downstream applications call it explicitly

2020-12-15 Thread Konstantin Shvachko
both branch-3.2.2 and branch-3.2.3 > missed. And have updated them manually. Please have a look. Thanks. > HADOOP-15691 > HDFS-15464 > HDFS-15478 > HDFS-15567 > HDFS-15574 > HDFS-15583 > HDFS-15628 > YARN-10430 > > Regards, > - He Xiaoqiao > > On Mon, Dec 1

Re: Regarding HDFS-15567. HDFS should expose msync() API to allow downstream applications call it explicitly

2020-12-18 Thread Konstantin Shvachko
make sure it calls msync() on all mount points that enabled observer reads. --Konst On Tue, Dec 15, 2020 at 11:15 AM Steve Loughran wrote: > > > On Sun, 13 Dec 2020 at 21:08, Konstantin Shvachko > wrote: > >> Hi Steve, >> >> I am not sure I fully understand

Re: Regarding HDFS-15567. HDFS should expose msync() API to allow downstream applications call it explicitly

2020-12-24 Thread Konstantin Shvachko
Hi Steve, I created HDFS-15751 <https://issues.apache.org/jira/browse/HDFS-15751> for documenting msync API. Would appreciate your suggestions. Stay safe, --Konstantin On Mon, Dec 21, 2020 at 5:19 AM Steve Loughran wrote: > > > On Fri, 18 Dec 2020 at 23:29, Konstantin Sh

Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

2016-08-03 Thread Konstantin Shvachko
1. I probably missed something but I didn't get it how "alpha"s made their way into release numbers again. This was discussed on several occasions and I thought the common perception was to use just three level numbers for release versioning and avoid branding them. It is particularly confusing to

Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

2016-08-04 Thread Konstantin Shvachko
On Thu, Aug 4, 2016 at 11:20 AM, Andrew Wang wrote: > Hi Konst, thanks for commenting, > > On Wed, Aug 3, 2016 at 11:29 PM, Konstantin Shvachko > wrote: > >> 1. I probably missed something but I didn't get it how "alpha"s made >> their way into re

  1   2   3   >