Re: Hadoop CI with alternate architectures.

[Konstantin Boudnik]

On Wed, May 18, 2016 at 08:29AM, Allen Wittenauer wrote:
>   That’s really a question for infrastructure-...@apache.org . They
>   manage the ASF build infrastructure which Apache Hadoop and lots of
>   other projects utilize.  (Bigtop uses something custom, which I think
>   is funded by Cloudera.)

Nope, Bigtop CI is not sponsored by Cloudera.

Besides, Bigtop is running itw own CI because we are producing a lot of
binary artifacts and don't won't to run a risk of clogging ASF-wide CI.

Cos
--
  Take care,
Konstantin (Cos) Boudnik


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.7.2 RC0

[Konstantin Boudnik]

Vinod, are you Ok with me putting it into 2.7.2?

Cos

On Mon, Nov 30, 2015 at 01:25PM, Vinod Kumar Vavilapalli wrote:
> I think we should get it in, I am still stuck on a couple of HDFS patches.
> 
> But I was not sure overall if the patch was right. Commented on the JIRA 
> regarding the same.
> 
> Thanks
> +Vinod
> 
> > On Nov 27, 2015, at 1:54 AM, Steve Loughran  wrote:
> > 
> > well, I'm not going to block it...it doesn't add anything more to the pom 
> > dependencies that aren't there right now
> > 
> >> On 26 Nov 2015, at 18:51, Konstantin Boudnik  wrote:
> >> 
> >> On Wed, Nov 25, 2015 at 01:51PM, Steve Loughran wrote:
> >>> 
> >>>> On 25 Nov 2015, at 02:48, Konstantin Boudnik  wrote:
> >>>> 
> >>>> Vinod, hopefully it isn't too late for a quick fix to be included into 
> >>>> 2.7.2
> >>>> (sorry for jumping too late on this train). The JIRA in question is
> >>>> HADOOP-12415 and I have committed it to trunk and branch-2 already.
> >>>> 
> >>>> Please let me know if this is still Ok to add into 2.7.2 as it seems to 
> >>>> be
> >>>> going through a respin. Thanks in advance,
> >>>> 
> >>>> Cos
> >>>> 
> >>> 
> >>> 
> >>> can you hold it off until 2.7.3? 2.7.2 was tested and we shoudn't be 
> >>> making
> >>> more changes now, even if its just to the POMs
> >> 
> >> It's up to you guys, the fix is small but pretty critical IMO. The change 
> >> in
> >> question is literally breaking the build from the get go. In Bigtop we had 
> >> to
> >> patch 2.7.1 so we at least can build it into the stack. 
> >> 
> >> For the users using official release tarball it won't simply work unless 
> >> they
> >> are going to patch it themselves.
> >> 
> >> Cos
> > 
> > 
> 


signature.asc
Description: Digital signature

Re: [DISCUSS] Release numbering for stable 2.8 and beyond

[Konstantin Boudnik]

On Mon, Apr 27, 2015 at 10:45AM, Andrew Wang wrote:
> I think Karthik correctly identified the two reasons we might want to
> denote a release as "unstable":
> 
> a) Compatibility. Whether we have freedom to break compatibility before the
> final release, i.e. what "alpha" typically means.

Breaking compatibility within minor releases is a very novel idea, introduced
to the world by Scala, if I am not mistaken. In general though, it is a pretty
bad practice to have 2.8.0 be incompatible with 2.8.1 - this isn't what
normally is expected from a subminor update.

> b) Quality. As Konst says, a release can be allowed to break compatibility
> ("alpha") but itself still be a high quality release.

I don't think Konst has advocated to make subminors incopatible ;)

As for the original question, I'd really go with a normal versioning and
just documenting any stability concerns.

Cos

> We could try and separate out these two concerns when it comes to
> versioning, but I think in reality users only care about prod vs. non-prod,
> which is why many other projects (Linux, HBase, etc) just have two release
> lines: "stable" (compatible/good-quality) vs. "unstable"
> (unknown-compatible/unknown-quality).
> 
> To this end, I don't mind what we choose, as long as the difference between
> stable and unstable is denoted by the version number. I don't like the idea
> of tagging a release as good after the fact (1). The other projects we've
> used as examples make strong statements about their stable release lines,
> and we should too. Our downstreams and end users will appreciate clarity
> from the version number.
> 
> Best,
> Andrew
> 
> On Sun, Apr 26, 2015 at 9:51 AM, Hitesh Shah  wrote:
> 
> > There are a couple of different approaches we could take.
> >
> > How about publishing/releasing bits such as “2.8.0-RC0”. Downstream
> > projects can depend on these and use them normally similar to the approach
> > that they would have taken with release 2.8.0 or 2.8.0-alpha. After some
> > baking ( or more RC suffixed releases), at some point, a proper 2.8.0
> > release could be made? The RC tag clearly indicates to downstream layers
> > that things will be in flux slightly. Trying to distinguish
> > incompatibilities between 2.8.0 and 2.8.1/2.8.2 ( just because 2.8.0 was
> > tagged alpha in documentation ) are likely to be a nightmare to deal with
> > especially for new features introduced in the 2.8 release ( which might get
> > changed slightly based on usage feedback ).
> >
> > Furthermore, considering the release history of hadoop, the likelihood
> > that 2.9.0 will show up before 2.8.2 seems to be very high.
> >
> > With respect to the proposed choice (1), I thought that HBase applied a
> > different approach. The odd-number releases were always unstable and the
> > even number releases were the stable ones. This “could" make things a bit
> > more clear for downstream users without needing to resort to modified
> > versioned numbers ( with alpha/beta tags ) or requiring users to go look up
> > the website to find out which particular version of 2.8.x was the first
> > stable release.
> >
> > thanks
> > — Hitesh
> >
> >
> >
> >
> >
> > On Apr 22, 2015, at 2:17 PM, Vinod Kumar Vavilapalli <
> > vino...@hortonworks.com> wrote:
> >
> > > Forking the thread.
> > >
> > > In the previous 2.7.1 thread [1], there were enough yays to my proposal
> > to wait for a bug-fix release or two before calling a 2.x release stable.
> > There were some concerns about the naming.
> > >
> > > We have two options, taking 2.8 as an example
> > > (1) Release 2.8.0, call it as an alpha in documentation and release
> > notes, wait for a 2.8.1/2.8.2 reasonably stable enough to be called as the
> > first stable release of 2.8.
> > > (2) Release 2.8.0-alpha, 2.8.0-beta etc before culminating in a 2.8.0
> > stable release.
> > >
> > > (1) is what I preferred first up. This is what HBase used to do, and far
> > beyond, in the linux kernel releases. It helps in scenarios where we are
> > forced to downgrade a release, say due to major issues. We can simply
> > announce it as not stable retroactively, change the pointers on our website
> > and move on.
> > >
> > > Thoughts?
> > >
> > > Thanks,
> > > +Vinod
> > >
> > > [1] http://markmail.org/thread/ogzk4phj6wsdpssu
> > >
> > > On Apr 21, 2015, at 4:59 PM, Vinod Kumar Vavilapalli <
> > vino...@hortonworks.com> wrote:
> > >
> > >>
> > >> Sure, I agree it's better to have clear guidelines and scheme. Let me
> > fork this thread about that.
> > >>
> > >> Re 2.7.0, I just forgot about the naming initially though I was clear
> > in the discussion/voting. I so had to end up calling it alpha outside of
> > the release artifact naming.
> > >>
> > >> Thanks
> > >> +Vinod
> > >>
> > >> On Apr 21, 2015, at 4:26 PM, Andrew Wang 
> > wrote:
> > >>
> > >>> I would also like to support Karthik's proposal on the release thread
> > about
> > >>> version numbering. 2.7.0 being "alpha" is pretty confusing since all
> > of the

Re: [VOTE] Release Apache Hadoop 2.7.2 RC0

[Konstantin Boudnik]

On Wed, Nov 25, 2015 at 01:51PM, Steve Loughran wrote:
> 
> > On 25 Nov 2015, at 02:48, Konstantin Boudnik  wrote:
> > 
> > Vinod, hopefully it isn't too late for a quick fix to be included into 2.7.2
> > (sorry for jumping too late on this train). The JIRA in question is
> > HADOOP-12415 and I have committed it to trunk and branch-2 already.
> > 
> > Please let me know if this is still Ok to add into 2.7.2 as it seems to be
> > going through a respin. Thanks in advance,
> > 
> > Cos
> > 
> 
> 
> can you hold it off until 2.7.3? 2.7.2 was tested and we shoudn't be making
> more changes now, even if its just to the POMs

It's up to you guys, the fix is small but pretty critical IMO. The change in
question is literally breaking the build from the get go. In Bigtop we had to
patch 2.7.1 so we at least can build it into the stack. 

For the users using official release tarball it won't simply work unless they
are going to patch it themselves.

Cos


signature.asc
Description: Digital signature

Re: [VOTE] Release Apache Hadoop 2.7.2 RC0

[Konstantin Boudnik]

Vinod, hopefully it isn't too late for a quick fix to be included into 2.7.2
(sorry for jumping too late on this train). The JIRA in question is
HADOOP-12415 and I have committed it to trunk and branch-2 already.

Please let me know if this is still Ok to add into 2.7.2 as it seems to be
going through a respin. Thanks in advance,

Cos

On Wed, Nov 11, 2015 at 08:31PM, Vinod Kumar Vavilapalli wrote:
> Hi all,
> 
> 
> I've created a release candidate RC0 for Apache Hadoop 2.7.2.
> 
> 
> As discussed before, this is the next maintenance release to follow up
> 2.7.1.
> 
> 
> The RC is available for validation at:
> 
> *http://people.apache.org/~vinodkv/hadoop-2.7.2-RC0/
> 
> *
> 
> 
> The RC tag in git is: release-2.7.2-RC0
> 
> 
> The maven artifacts are available via repository.apache.org at
> 
> *https://repository.apache.org/content/repositories/orgapachehadoop-1023/
> 
> *
> 
> 
> As you may have noted, an unusually long 2.6.3 release caused 2.7.2 to slip
> by quite a bit. This release's related discussion threads are linked below:
> [1] and [2].
> 
> 
> Please try the release and vote; the vote will run for the usual 5 days.
> 
> 
> Thanks,
> 
> Vinod
> 
> 
> [1]: 2.7.2 release plan: http://markmail.org/message/oozq3gvd4nhzsaes
> 
> [2]: Planning Apache Hadoop 2.7.2
> http://markmail.org/message/iktqss2qdeykgpqk


signature.asc
Description: Digital signature

[jira] [Resolved] (HADOOP-7730) Allow TestCLI to be run against a cluster

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-7730.

  Resolution: Won't Fix
Release Note: The resolution has been done in the Bigtop a few years 
ago. Closing this one as of irrelevant to the project.
Target Version/s: 1.3.0, 0.22.1  (was: 0.22.1, 1.3.0)

> Allow TestCLI to be run against a cluster
> -
>
> Key: HADOOP-7730
> URL: https://issues.apache.org/jira/browse/HADOOP-7730
> Project: Hadoop Common
>  Issue Type: Test
>  Components: test
>Affects Versions: 0.20.205.0, 0.22.0
>    Reporter: Konstantin Boudnik
>    Assignee: Konstantin Boudnik
> Attachments: HADOOP-7730.patch, HADOOP-7730.trunk.patch, 
> HADOOP-7730.trunk.patch
>
>
> Use the same CLI test to test cluster bits (see HDFS-1762 for more info)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HADOOP-7481) Wire AOP test in Mavenized Hadoop common

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-7481.

  Resolution: Won't Fix
Release Note: Ain't gonna happen clearly: the FI has been removed from the 
source tree, so there's no point of keeping this ticket open.

> Wire AOP test in Mavenized Hadoop common
> 
>
> Key: HADOOP-7481
> URL: https://issues.apache.org/jira/browse/HADOOP-7481
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Alejandro Abdelnur
>Assignee: Konstantin Boudnik
>
> We ned add a Maven profile that activates the AOP injection and runs the 
> necessary AOPed tests. I believe there should be a Maven plugin for doing 
> that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HADOOP-12415) Hadoop build is broken because of missed compile-time dependency

[Konstantin Boudnik (JIRA)]

Konstantin Boudnik created HADOOP-12415:
---

 Summary: Hadoop build is broken because of missed compile-time 
dependency
 Key: HADOOP-12415
 URL: https://issues.apache.org/jira/browse/HADOOP-12415
 Project: Hadoop Common
  Issue Type: Bug
  Components: nfs
Affects Versions: 2.7.1
Reporter: Konstantin Boudnik


As discovered in BIGTOP-2049 {{hadoop-nfs}} module compilation is broken. Looks 
like that HADOOP-11489 is the root-cause of it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HADOOP-11945) Add Apache Ignite (incubating) to the "Hadoop-related projects"

[Konstantin Boudnik (JIRA)]

Konstantin Boudnik created HADOOP-11945:
---

 Summary: Add Apache Ignite (incubating) to the "Hadoop-related 
projects"
 Key: HADOOP-11945
 URL: https://issues.apache.org/jira/browse/HADOOP-11945
 Project: Hadoop Common
  Issue Type: Improvement
  Components: site
Affects Versions: 2.5.2
Reporter: Konstantin Boudnik
Assignee: Konstantin Boudnik
 Fix For: site


Apache Ignite (incubating) has the implementation of HDFS compatible in-memory 
file system (IGFS) and in-memory MapReduce accelerator. These two components 
allow Ignite users to run OLTP/SQL/computational workloads on top of HDFS 
storage.

Let's add this project to the list of Hadoop related ones.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Dropping support for JDK6 in Apache Hadoop

[Konstantin Boudnik]

While it sounds like a topic for a different discussion (or list@), do you
think it would be too crazy to skip JDK7 completely and just go to JDK8
directly? I think downstream components like HBase, Spark and so on would be a
huge driving factor to make the same change in Hadoop and other parts of the
ecosystem.

Thoughts?
  Cos
   
On Tue, Aug 19, 2014 at 11:23AM, Devaraj Das wrote:
> I think we are good on HBase side, at least on the 1.x+ branches. We
> are dropping support for 1.6 from HBase-1.0 onwards.
> 
> On Tue, Aug 19, 2014 at 10:52 AM, Arun C Murthy  wrote:
> > [Apologies for the wide distribution.]
> >
> > Dear HBase/Hive/Pig/Oozie communities,
> >
> >  We, over at Hadoop are considering dropping support for JDK6 this year.
> >
> >  As you maybe aware we just released hadoop-2.5.0 and are now considering 
> > making the next release i.e. hadoop-2.6.0 the *last* release of Apache 
> > Hadoop which supports JDK6. This means, from hadoop-2.7.0 onwards we will 
> > not support JDK6 anymore and we *may* start relying on JDK7-specific apis.
> >
> >  Now, the above releases a proposal and we do not want to pull the trigger 
> > without talking to projects downstream - hence the request for you feedback.
> >
> >  Please feel free to forward this to other communities you might deem to be 
> > at risk from this too.
> >
> > thanks,
> > Arun
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender immediately
> > and delete it from your system. Thank You.
> 
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.

Re: Why Hadoop-trunk-commit always fails?

[Konstantin Boudnik]

If we use maven jar plugin or maven archivers to create any of these then 
adding 
  false
should solve the issue.

Cos

On Mon, Jul 21, 2014 at 01:55PM, Andrew Wang wrote:
> I dug around a bit with Tucu, and I think it's essentially the dependency
> analyzer screwing up with snapshot artifacts. I found a different error for
> HttpFS that looks similar:
> 
> 
> [WARNING]
> Dependency convergence error for
> org.apache.hadoop:hadoop-hdfs:3.0.0-SNAPSHOT paths to dependency are:
> +-org.apache.hadoop:hadoop-hdfs-httpfs:3.0.0-SNAPSHOT
>   +-org.apache.hadoop:hadoop-hdfs:3.0.0-SNAPSHOT
> and
> +-org.apache.hadoop:hadoop-hdfs-httpfs:3.0.0-SNAPSHOT
>   +-org.apache.hadoop:hadoop-hdfs:3.0.0-20140718.221409-4777
> 
> [WARNING] Rule 0:
> org.apache.maven.plugins.enforcer.DependencyConvergence failed with
> message:
> Failed while enforcing releasability the error(s) are [
> Dependency convergence error for
> org.apache.hadoop:hadoop-hdfs:3.0.0-SNAPSHOT paths to dependency are:
> +-org.apache.hadoop:hadoop-hdfs-httpfs:3.0.0-SNAPSHOT
>   +-org.apache.hadoop:hadoop-hdfs:3.0.0-SNAPSHOT
> and
> +-org.apache.hadoop:hadoop-hdfs-httpfs:3.0.0-SNAPSHOT
>   +-org.apache.hadoop:hadoop-hdfs:3.0.0-20140718.221409-4777
> 
> 
> You can see that it sees 3.0.0-SNAPSHOT being used for one, and
> 3.0.0-20140718.221409-4777 for the other (which causes the error). The same
> thing happened in the stuff Ted posted, but for the KMS. Somehow the local
> maven repo is getting screwed up non-deterministically.
> 
> Tucu recommends we remove this check from the post-commit build, and
> instead make it part of the maven job used to build releases. At release
> time, there shouldn't be any ambiguity about version numbers.
> 
> Any brave volunteers out there? I am not a maven maven, but am happy to
> review pom.xml changes that do this, and I'll make sure the maven job used
> to build releases still does the dep check.
> 
> Best,
> Andrew
> 
> 
> 
> 
> On Thu, Jul 17, 2014 at 9:50 PM, Ted Yu  wrote:
> 
> > Here is the warning from enforcer:
> >
> > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence
> > failed with message:
> > Failed while enforcing releasability the error(s) are [
> > Dependency convergence error for
> > org.apache.hadoop:hadoop-auth:3.0.0-20140718.043141-4847 paths to
> > dependency are:
> > +-org.apache.hadoop:hadoop-kms:3.0.0-SNAPSHOT
> >   +-org.apache.hadoop:hadoop-auth:3.0.0-20140718.043141-4847
> > and
> > +-org.apache.hadoop:hadoop-kms:3.0.0-SNAPSHOT
> >   +-org.apache.hadoop:hadoop-common:3.0.0-20140718.043201-4831
> > +-org.apache.hadoop:hadoop-auth:3.0.0-SNAPSHOT
> > and
> > +-org.apache.hadoop:hadoop-kms:3.0.0-SNAPSHOT
> >   +-org.apache.hadoop:hadoop-common:3.0.0-20140718.043201-4831
> > +-org.apache.hadoop:hadoop-auth:3.0.0-SNAPSHOT
> > ]
> >
> > FYI
> >
> >
> > On Thu, Jul 17, 2014 at 9:38 PM, Vinayakumar B 
> > wrote:
> >
> > > Hi,
> > > Hadoop-trunk-commit build always fails with message similar to below.
> > > Anybody knows about this?
> > >
> > > [ERROR] Failed to execute goal
> > > org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
> > > (depcheck) on project hadoop-yarn-server-tests: Some Enforcer rules
> > > have failed. Look above for specific messages explaining why the rule
> > > failed. -> [Help 1]
> > >
> > >
> > >
> > > Regards,
> > > Vinay
> > >
> >

Phone bridge info [Re: Meetup invitation: Consensus based replication in Hadoop]

[Konstantin Boudnik]

To enable remote people to join the conversation, we'll keep open the
following GotoMeeting session. You can join either from your computer or by
using dial-in number. 

 =
Please join my meeting, Jul 15, 2014 at 12:00 PM PDT.
https://www4.gotomeeting.com/join/357791919

2. Use your microphone and speakers (VoIP) - a headset is recommended. Or,
call in using your telephone.

Dial +1 (805) 309-0014
Access Code: 357-791-919
Audio PIN: Shown after joining the meeting

Meeting ID: 357-791-919
 =

BTW, we have one or two spots left, so if you didn't make your reservation -
now is the time.

See you all tomorrow.
  Cos

On Fri, Jul 11, 2014 at 04:33PM, Konstantin Boudnik wrote:
> One more update: it seema that for ppl in SF, who oftentimes might not even
> have a car, getting to San Ramon can represent a certain difficulty.
> 
> So we'll do a shuttle pickup from West Dublin BART station if there's at least
> a few people who want to use the option.
> 
> Please respond directly to me if you're interested before end of Sunday, 13th.
> Cheers,
>   Cos
> 
> On Tue, Jul 08, 2014 at 12:23PM, Konstantin Boudnik wrote:
> > All,
> > 
> > Re-sending this announcement in case it fell through over the long weekend
> > when people were away. We still have seats left, so register soon.
> > 
> > Regards,
> >   Cos
> > 
> > On Wed, Jul 02, 2014 at 06:37PM, Konstantin Boudnik wrote:
> > > We'd like to invite you to the 
> > > Consensus based replication in Hadoop: A deep dive
> > > event that we are happy to hold in our San Ramon office on July 15th at 
> > > noon.
> > > We'd like to accommodate as many people as possible, but I think are 
> > > physically
> > > limited to 30 (+/- a few), so please RSVP to this Eventbrite invitation:
> > > 
> > > https://www.eventbrite.co.uk/e/consensus-based-replication-in-hadoop-a-deep-dive-tickets-12158236613
> > > 
> > > We'll provide pizza and beverages (feel free to express your special 
> > > dietary
> > > requirements if any).
> > > 
> > > See you soon!
> > > With regards,
> > >   Cos
> > > 
> > > On Wed, Jun 18, 2014 at 08:45PM, Konstantin Boudnik wrote:
> > > > Guys,
> > > > 
> > > > In the last a couple of weeks, we had a very good and productive 
> > > > initial round
> > > > of discussions on the JIRAs. I think it is worthy to keep the momentum 
> > > > going
> > > > and have a more detailed conversation. For that, we'd like to host s 
> > > > Hadoop
> > > > developers meetup to get into the bowls of the consensus-based 
> > > > coordination
> > > > implementation for HDFS. The proposed venue is our office in San Ramon, 
> > > > CA.
> > > > 
> > > > Considering that it is already a mid week and the following one looks 
> > > > short
> > > > because of the holidays, how would the week of July 7th looks for yall?
> > > > Tuesday or Thursday look pretty good on our end.
> > > > 
> > > > Please chime in on your preference either here or reach of directly to 
> > > > me.
> > > > Once I have a few RSVPs I will setup an event on Eventbrite or similar.
> > > > 
> > > > Looking forward to your input. Regards,
> > > >   Cos
> > > > 
> > > > On Thu, May 29, 2014 at 02:09PM, Konstantin Shvachko wrote:
> > > > > Hello hadoop developers,
> > > > > 
> > > > > I just opened two jiras proposing to introduce ConsensusNode into 
> > > > > HDFS and
> > > > > a Coordination Engine into Hadoop Common. The latter should benefit 
> > > > > HDFS
> > > > > and  HBase as well as potentially other projects. See HDFS-6469 and
> > > > > HADOOP-10641 for details.
> > > > > The effort is based on the system we built at Wandisco with my 
> > > > > colleagues,
> > > > > who are glad to contribute it to Apache, as quite a few people in the
> > > > > community expressed interest in this ideas and their potential 
> > > > > applications.
> > > > > 
> > > > > We should probably keep technical discussions in the jiras. Here on 
> > > > > the dev
> > > > > list I wanted to touch-base on any logistic issues / questions.
> > > > > - First of all, any ideas and help are very much welcome.
> > > > > - We would like to set up a meetup to discuss this if people are
> > > > > interested. Hadoop Summit next week may be a potential time-place to 
> > > > > meet.
> > > > > Not sure in what form. If not, we can organize one in our San Ramon 
> > > > > office
> > > > > later on.
> > > > > - The effort may take a few months depending on the contributors 
> > > > > schedules.
> > > > > Would it make sense to open a branch for the ConsensusNode work?
> > > > > - APIs and the implementation of the Coordination Engine should be a 
> > > > > fairly
> > > > > independent, so it may be reasonable to add it directly to Hadoop 
> > > > > Common
> > > > > trunk.
> > > > > 
> > > > > Thanks,
> > > > > --Konstantin

Re: Meetup invitation: Consensus based replication in Hadoop

[Konstantin Boudnik]

One more update: it seema that for ppl in SF, who oftentimes might not even
have a car, getting to San Ramon can represent a certain difficulty.

So we'll do a shuttle pickup from West Dublin BART station if there's at least
a few people who want to use the option.

Please respond directly to me if you're interested before end of Sunday, 13th.
Cheers,
  Cos

On Tue, Jul 08, 2014 at 12:23PM, Konstantin Boudnik wrote:
> All,
> 
> Re-sending this announcement in case it fell through over the long weekend
> when people were away. We still have seats left, so register soon.
> 
> Regards,
>   Cos
> 
> On Wed, Jul 02, 2014 at 06:37PM, Konstantin Boudnik wrote:
> > We'd like to invite you to the 
> > Consensus based replication in Hadoop: A deep dive
> > event that we are happy to hold in our San Ramon office on July 15th at 
> > noon.
> > We'd like to accommodate as many people as possible, but I think are 
> > physically
> > limited to 30 (+/- a few), so please RSVP to this Eventbrite invitation:
> > 
> > https://www.eventbrite.co.uk/e/consensus-based-replication-in-hadoop-a-deep-dive-tickets-12158236613
> > 
> > We'll provide pizza and beverages (feel free to express your special dietary
> > requirements if any).
> > 
> > See you soon!
> > With regards,
> >   Cos
> > 
> > On Wed, Jun 18, 2014 at 08:45PM, Konstantin Boudnik wrote:
> > > Guys,
> > > 
> > > In the last a couple of weeks, we had a very good and productive initial 
> > > round
> > > of discussions on the JIRAs. I think it is worthy to keep the momentum 
> > > going
> > > and have a more detailed conversation. For that, we'd like to host s 
> > > Hadoop
> > > developers meetup to get into the bowls of the consensus-based 
> > > coordination
> > > implementation for HDFS. The proposed venue is our office in San Ramon, 
> > > CA.
> > > 
> > > Considering that it is already a mid week and the following one looks 
> > > short
> > > because of the holidays, how would the week of July 7th looks for yall?
> > > Tuesday or Thursday look pretty good on our end.
> > > 
> > > Please chime in on your preference either here or reach of directly to me.
> > > Once I have a few RSVPs I will setup an event on Eventbrite or similar.
> > > 
> > > Looking forward to your input. Regards,
> > >   Cos
> > > 
> > > On Thu, May 29, 2014 at 02:09PM, Konstantin Shvachko wrote:
> > > > Hello hadoop developers,
> > > > 
> > > > I just opened two jiras proposing to introduce ConsensusNode into HDFS 
> > > > and
> > > > a Coordination Engine into Hadoop Common. The latter should benefit HDFS
> > > > and  HBase as well as potentially other projects. See HDFS-6469 and
> > > > HADOOP-10641 for details.
> > > > The effort is based on the system we built at Wandisco with my 
> > > > colleagues,
> > > > who are glad to contribute it to Apache, as quite a few people in the
> > > > community expressed interest in this ideas and their potential 
> > > > applications.
> > > > 
> > > > We should probably keep technical discussions in the jiras. Here on the 
> > > > dev
> > > > list I wanted to touch-base on any logistic issues / questions.
> > > > - First of all, any ideas and help are very much welcome.
> > > > - We would like to set up a meetup to discuss this if people are
> > > > interested. Hadoop Summit next week may be a potential time-place to 
> > > > meet.
> > > > Not sure in what form. If not, we can organize one in our San Ramon 
> > > > office
> > > > later on.
> > > > - The effort may take a few months depending on the contributors 
> > > > schedules.
> > > > Would it make sense to open a branch for the ConsensusNode work?
> > > > - APIs and the implementation of the Coordination Engine should be a 
> > > > fairly
> > > > independent, so it may be reasonable to add it directly to Hadoop Common
> > > > trunk.
> > > > 
> > > > Thanks,
> > > > --Konstantin


signature.asc
Description: Digital signature

Re: Meetup invitation: Consensus based replication in Hadoop

[Konstantin Boudnik]

All,

Re-sending this announcement in case it fell through over the long weekend
when people were away. We still have seats left, so register soon.

Regards,
  Cos

On Wed, Jul 02, 2014 at 06:37PM, Konstantin Boudnik wrote:
> We'd like to invite you to the 
> Consensus based replication in Hadoop: A deep dive
> event that we are happy to hold in our San Ramon office on July 15th at noon.
> We'd like to accommodate as many people as possible, but I think are 
> physically
> limited to 30 (+/- a few), so please RSVP to this Eventbrite invitation:
> 
> https://www.eventbrite.co.uk/e/consensus-based-replication-in-hadoop-a-deep-dive-tickets-12158236613
> 
> We'll provide pizza and beverages (feel free to express your special dietary
> requirements if any).
> 
> See you soon!
> With regards,
>   Cos
> 
> On Wed, Jun 18, 2014 at 08:45PM, Konstantin Boudnik wrote:
> > Guys,
> > 
> > In the last a couple of weeks, we had a very good and productive initial 
> > round
> > of discussions on the JIRAs. I think it is worthy to keep the momentum going
> > and have a more detailed conversation. For that, we'd like to host s Hadoop
> > developers meetup to get into the bowls of the consensus-based coordination
> > implementation for HDFS. The proposed venue is our office in San Ramon, CA.
> > 
> > Considering that it is already a mid week and the following one looks short
> > because of the holidays, how would the week of July 7th looks for yall?
> > Tuesday or Thursday look pretty good on our end.
> > 
> > Please chime in on your preference either here or reach of directly to me.
> > Once I have a few RSVPs I will setup an event on Eventbrite or similar.
> > 
> > Looking forward to your input. Regards,
> >   Cos
> > 
> > On Thu, May 29, 2014 at 02:09PM, Konstantin Shvachko wrote:
> > > Hello hadoop developers,
> > > 
> > > I just opened two jiras proposing to introduce ConsensusNode into HDFS and
> > > a Coordination Engine into Hadoop Common. The latter should benefit HDFS
> > > and  HBase as well as potentially other projects. See HDFS-6469 and
> > > HADOOP-10641 for details.
> > > The effort is based on the system we built at Wandisco with my colleagues,
> > > who are glad to contribute it to Apache, as quite a few people in the
> > > community expressed interest in this ideas and their potential 
> > > applications.
> > > 
> > > We should probably keep technical discussions in the jiras. Here on the 
> > > dev
> > > list I wanted to touch-base on any logistic issues / questions.
> > > - First of all, any ideas and help are very much welcome.
> > > - We would like to set up a meetup to discuss this if people are
> > > interested. Hadoop Summit next week may be a potential time-place to meet.
> > > Not sure in what form. If not, we can organize one in our San Ramon office
> > > later on.
> > > - The effort may take a few months depending on the contributors 
> > > schedules.
> > > Would it make sense to open a branch for the ConsensusNode work?
> > > - APIs and the implementation of the Coordination Engine should be a 
> > > fairly
> > > independent, so it may be reasonable to add it directly to Hadoop Common
> > > trunk.
> > > 
> > > Thanks,
> > > --Konstantin

Meetup invitation: Consensus based replication in Hadoop

[Konstantin Boudnik]

We'd like to invite you to the 
Consensus based replication in Hadoop: A deep dive
event that we are happy to hold in our San Ramon office on July 15th at noon.
We'd like to accommodate as many people as possible, but I think are physically
limited to 30 (+/- a few), so please RSVP to this Eventbrite invitation:

https://www.eventbrite.co.uk/e/consensus-based-replication-in-hadoop-a-deep-dive-tickets-12158236613

We'll provide pizza and beverages (feel free to express your special dietary
requirements if any).

See you soon!
With regards,
  Cos

On Wed, Jun 18, 2014 at 08:45PM, Konstantin Boudnik wrote:
> Guys,
> 
> In the last a couple of weeks, we had a very good and productive initial round
> of discussions on the JIRAs. I think it is worthy to keep the momentum going
> and have a more detailed conversation. For that, we'd like to host s Hadoop
> developers meetup to get into the bowls of the consensus-based coordination
> implementation for HDFS. The proposed venue is our office in San Ramon, CA.
> 
> Considering that it is already a mid week and the following one looks short
> because of the holidays, how would the week of July 7th looks for yall?
> Tuesday or Thursday look pretty good on our end.
> 
> Please chime in on your preference either here or reach of directly to me.
> Once I have a few RSVPs I will setup an event on Eventbrite or similar.
> 
> Looking forward to your input. Regards,
>   Cos
> 
> On Thu, May 29, 2014 at 02:09PM, Konstantin Shvachko wrote:
> > Hello hadoop developers,
> > 
> > I just opened two jiras proposing to introduce ConsensusNode into HDFS and
> > a Coordination Engine into Hadoop Common. The latter should benefit HDFS
> > and  HBase as well as potentially other projects. See HDFS-6469 and
> > HADOOP-10641 for details.
> > The effort is based on the system we built at Wandisco with my colleagues,
> > who are glad to contribute it to Apache, as quite a few people in the
> > community expressed interest in this ideas and their potential applications.
> > 
> > We should probably keep technical discussions in the jiras. Here on the dev
> > list I wanted to touch-base on any logistic issues / questions.
> > - First of all, any ideas and help are very much welcome.
> > - We would like to set up a meetup to discuss this if people are
> > interested. Hadoop Summit next week may be a potential time-place to meet.
> > Not sure in what form. If not, we can organize one in our San Ramon office
> > later on.
> > - The effort may take a few months depending on the contributors schedules.
> > Would it make sense to open a branch for the ConsensusNode work?
> > - APIs and the implementation of the Coordination Engine should be a fairly
> > independent, so it may be reasonable to add it directly to Hadoop Common
> > trunk.
> > 
> > Thanks,
> > --Konstantin

Re: Introducing ConsensusNode and a Coordination Engine

[Konstantin Boudnik]

They are actually listed in the first email on this thread. Here they are:
HADOOP-10641
HDFS-6469

Cos

On Wed, Jun 18, 2014 at 10:30PM, Sujeet Varakhedi wrote:
> Can you point me to the JIRA's
> 
> Sujeet
> 
> 
> On Wed, Jun 18, 2014 at 8:45 PM, Konstantin Boudnik  wrote:
> 
> > Guys,
> >
> > In the last a couple of weeks, we had a very good and productive initial
> > round
> > of discussions on the JIRAs. I think it is worthy to keep the momentum
> > going
> > and have a more detailed conversation. For that, we'd like to host s Hadoop
> > developers meetup to get into the bowls of the consensus-based coordination
> > implementation for HDFS. The proposed venue is our office in San Ramon, CA.
> >
> > Considering that it is already a mid week and the following one looks short
> > because of the holidays, how would the week of July 7th looks for yall?
> > Tuesday or Thursday look pretty good on our end.
> >
> > Please chime in on your preference either here or reach of directly to me.
> > Once I have a few RSVPs I will setup an event on Eventbrite or similar.
> >
> > Looking forward to your input. Regards,
> >   Cos
> >
> > On Thu, May 29, 2014 at 02:09PM, Konstantin Shvachko wrote:
> > > Hello hadoop developers,
> > >
> > > I just opened two jiras proposing to introduce ConsensusNode into HDFS
> > and
> > > a Coordination Engine into Hadoop Common. The latter should benefit HDFS
> > > and  HBase as well as potentially other projects. See HDFS-6469 and
> > > HADOOP-10641 for details.
> > > The effort is based on the system we built at Wandisco with my
> > colleagues,
> > > who are glad to contribute it to Apache, as quite a few people in the
> > > community expressed interest in this ideas and their potential
> > applications.
> > >
> > > We should probably keep technical discussions in the jiras. Here on the
> > dev
> > > list I wanted to touch-base on any logistic issues / questions.
> > > - First of all, any ideas and help are very much welcome.
> > > - We would like to set up a meetup to discuss this if people are
> > > interested. Hadoop Summit next week may be a potential time-place to
> > meet.
> > > Not sure in what form. If not, we can organize one in our San Ramon
> > office
> > > later on.
> > > - The effort may take a few months depending on the contributors
> > schedules.
> > > Would it make sense to open a branch for the ConsensusNode work?
> > > - APIs and the implementation of the Coordination Engine should be a
> > fairly
> > > independent, so it may be reasonable to add it directly to Hadoop Common
> > > trunk.
> > >
> > > Thanks,
> > > --Konstantin
> >

Re: Introducing ConsensusNode and a Coordination Engine

[Konstantin Boudnik]

Guys,

In the last a couple of weeks, we had a very good and productive initial round
of discussions on the JIRAs. I think it is worthy to keep the momentum going
and have a more detailed conversation. For that, we'd like to host s Hadoop
developers meetup to get into the bowls of the consensus-based coordination
implementation for HDFS. The proposed venue is our office in San Ramon, CA.

Considering that it is already a mid week and the following one looks short
because of the holidays, how would the week of July 7th looks for yall?
Tuesday or Thursday look pretty good on our end.

Please chime in on your preference either here or reach of directly to me.
Once I have a few RSVPs I will setup an event on Eventbrite or similar.

Looking forward to your input. Regards,
  Cos

On Thu, May 29, 2014 at 02:09PM, Konstantin Shvachko wrote:
> Hello hadoop developers,
> 
> I just opened two jiras proposing to introduce ConsensusNode into HDFS and
> a Coordination Engine into Hadoop Common. The latter should benefit HDFS
> and  HBase as well as potentially other projects. See HDFS-6469 and
> HADOOP-10641 for details.
> The effort is based on the system we built at Wandisco with my colleagues,
> who are glad to contribute it to Apache, as quite a few people in the
> community expressed interest in this ideas and their potential applications.
> 
> We should probably keep technical discussions in the jiras. Here on the dev
> list I wanted to touch-base on any logistic issues / questions.
> - First of all, any ideas and help are very much welcome.
> - We would like to set up a meetup to discuss this if people are
> interested. Hadoop Summit next week may be a potential time-place to meet.
> Not sure in what form. If not, we can organize one in our San Ramon office
> later on.
> - The effort may take a few months depending on the contributors schedules.
> Would it make sense to open a branch for the ConsensusNode work?
> - APIs and the implementation of the Coordination Engine should be a fairly
> independent, so it may be reasonable to add it directly to Hadoop Common
> trunk.
> 
> Thanks,
> --Konstantin

Re: Re-swizzle 2.3

[Konstantin Boudnik]

Do you guys think that committing 
https://issues.apache.org/jira/browse/HDFS-4858
to branch-2.3 is still Ok? It is a small change that bring fixes broken
timeout behavior of DN to NN RPC.

We have been testing this fix on top of 2.0.6 for a long time now and it seems
to be a real help.

Appreciate the feedback on 2.3 scope. If it is too late then I will commit it
to trunk, branch-2.4 and branch-2 only.

Regards,
  Cos

On Thu, Feb 06, 2014 at 05:44PM, Alejandro Abdelnur wrote:
> yep, the idea is to pull all of them out from branch2.3. things go back to 
> normal then. 
> 
> thanks
> 
> Alejandro
> (phone typing)
> 
> > On Feb 6, 2014, at 17:39, Zhijie Shen  wrote:
> > 
> > Recently I brought 4 JIRAs to branch-2.3, which are MAPREDUCE-5743, 
> > YARN-1628,
> > YARN-1661 and YARN-1689. Recall that we mark test failure fixes as blockers
> > for pior releases as closing to release, thus I brought to branch-2.3
> > MAPREDUCE-5743
> > and YARN-1628 that are the fixes for the test failure on 2.3.0, but didn't
> > marked them as blockers. Please let me know if I should do that.
> > 
> > YARN-1661 is a fix for exit log of DS AppMaster, otherwise the exit log of
> > it will always be failure, which sounds a critical issue to me. Feel free
> > to pull it out if any objects.
> > 
> > YARN-1689 is brought to branch-2.3 as YARN-1493 is still in this branch. It
> > fixes one bug caused by YARN-1493. Those should be included or excluded
> > together upon the decision.
> > 
> > Thanks,
> > Zhijie
> > 
> > 
> >> On Thu, Feb 6, 2014 at 5:15 PM, Sandy Ryza  wrote:
> >> 
> >> +1 to reverting those JIRAs from branch-2.3.  As YARN-1689 is fixing a
> >> problem caused by YARN-1493 I think we can revert it in branch-2.3 as well.
> >> 
> >> I think we should leave them in branch-2 for now.  We can revert if 2.4 is
> >> imminent and they're holding it up, but hopefully the issues they caused
> >> will be fixed by then.
> >> 
> >> -Sandy
> >> 
> >> 
> >> On Thu, Feb 6, 2014 at 5:07 PM, Alejandro Abdelnur  >>> wrote:
> >> 
> >>> Thanks Robert,
> >>> 
> >>> All,
> >>> 
> >>> So it seems that YARN-1493 and YARN-1490 are introducing serious
> >>> regressions.
> >>> 
> >>> I would propose to revert them and the follow up JIRAs from the 2.3
> >> branch
> >>> and keep working on them on trunk/branch-2 until the are stable (I would
> >>> even prefer reverting them from branch-2 not to block a 2.4 if they are
> >> not
> >>> ready in time).
> >>> 
> >>> As I've mentioned before, the list of JIRAs to revert were:
> >>> 
> >>> YARN-1493
> >>> YARN-1490
> >>> YARN-1166
> >>> YARN-1041
> >>> YARN-1566
> >>> 
> >>> Plus 2 additional JIRAs committed since my email on this issue 2 days
> >> ago:
> >>> 
> >>> *YARN-1661
> >>> *YARN-1689 (not sure if this JIRA is related in functionality to the
> >>> previous ones but it is creating conflicts).
> >>> 
> >>> I think we should hold on continuing work on top of something that is
> >>> broken until the broken stuff is fixed.
> >>> 
> >>> Quoting Arun, "Committers - Henceforth, please use extreme caution while
> >>> committing to branch-2.3. Please commit *only* blockers to 2.3."
> >>> 
> >>> YARN-1661 & YARN-1689 are not blockers.
> >>> 
> >>> Unless there are objections, I'll revert all these JIRAs from branch-2.3
> >>> tomorrow around noon and I'll update fixedVersion in the JIRAs.
> >>> 
> >>> I'm inclined to revert them from branch-2 as well.
> >>> 
> >>> Thoughts?
> >>> 
> >>> Thanks.
> >>> 
> >>> 
> >>> On Thu, Feb 6, 2014 at 3:54 PM, Robert Kanter 
> >>> wrote:
> >>> 
>  I think we should revert YARN-1490 from Hadoop 2.3 branch.  I think it
> >>> was
>  causing some strange behavior in the Oozie unit tests:
>  
>  Basically, we use a single MiniMRCluster and MiniDFSCluster across all
> >>> unit
>  tests in a module.  With YARN-1490 we saw that, regardless of test
> >> order,
>  the last few tests would timeout waiting for an MR job to finish; on
> >>> slower
>  machines, the entire test suite would timeout.  Through some digging, I
>  found that we were getting a ton of "Connection refused" Exceptions on
>  LeaseRenewer talking to the NN and a few on the AM talking to the RM.
>  
>  After a bunch of investigation, I found that the problem went away once
>  YARN-1490 was removed.  Though I couldn't figure out the exact problem.
>  Even though this occurred in unit tests, it does make me concerned
> >> that
> >>> it
>  could indicate some bigger issue in a long-running real cluster (where
>  everything isn't running on the same machine) that we haven't seen yet.
>  
>  
>  
>  On Thu, Feb 6, 2014 at 3:06 PM, Karthik Kambatla 
>  wrote:
>  
> > I have marked MAPREDUCE-5744 a blocker for 2.3. Committing it
> >> shortly.
>  Will
> > pull it out of branch-2.3 if anyone objects.
> > 
> > 
> > On Thu, Feb 6, 2014 at 2:04 PM, Arpit Agarwal <
> >>> aagar...@hortonworks.com
> >> wrote:
> > 

Re: Troubles committing

[Konstantin Boudnik]

Wow, seems that my account was blocked after 18 consequent attempts to login
to my account since Jan 15th. I am pretty sure all this activity hasn't been
caused by any of devices. Hence, if you guys see something like this happening
to you - let INFRA know ASAP.

My issue is addressed now - thanks INFRA guys for quick response!
  Cos


On Mon, Jan 20, 2014 at 08:57PM, Konstantin Boudnik wrote:
> Guys,
> 
> anyone experience any issues with committing to Hadoop SVN today? Looks like
> my password is getting rejected by the SVN server (same password works for
> login into people.apache.org). 
> 
> Want to make before troubling INFRA. Thanks in advance!
> -- 
> Take care,
>   Cos
> 

Troubles committing

[Konstantin Boudnik]

Guys,

anyone experience any issues with committing to Hadoop SVN today? Looks like
my password is getting rejected by the SVN server (same password works for
login into people.apache.org). 

Want to make before troubling INFRA. Thanks in advance!
-- 
Take care,
Cos

[jira] [Reopened] (HADOOP-10110) hadoop-auth has a build break due to missing dependency

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-10110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik reopened HADOOP-10110:
-


Needed to be ported back into 2.0.x line as it's breaking the build there as 
well

> hadoop-auth has a build break due to missing dependency
> ---
>
> Key: HADOOP-10110
> URL: https://issues.apache.org/jira/browse/HADOOP-10110
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Affects Versions: 3.0.0, 2.0.6-alpha
>Reporter: Chuan Liu
>Assignee: Chuan Liu
>Priority: Blocker
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HADOOP-10110.patch
>
>
> We have a build break in hadoop-auth if build with maven cache cleaned. The 
> error looks like the follows. The problem exists on both Windows and Linux. 
> If you have old jetty jars in your maven cache, you won't see the error.
> {noformat}
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 1:29.469s
> [INFO] Finished at: Mon Nov 18 12:30:36 PST 2013
> [INFO] Final Memory: 37M/120M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile 
> (default-testCompile) on project hadoop-auth: Compilation failure: 
> Compilation failure:
> [ERROR] 
> /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[84,13]
>  cannot access org.mortbay.component.AbstractLifeCycle
> [ERROR] class file for org.mortbay.component.AbstractLifeCycle not found
> [ERROR] server = new Server(0);
> [ERROR] 
> /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[94,29]
>  cannot access org.mortbay.component.LifeCycle
> [ERROR] class file for org.mortbay.component.LifeCycle not found
> [ERROR] server.getConnectors()[0].setHost(host);
> [ERROR] 
> /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[96,10]
>  cannot find symbol
> [ERROR] symbol  : method start()
> [ERROR] location: class org.mortbay.jetty.Server
> [ERROR] 
> /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[102,12]
>  cannot find symbol
> [ERROR] symbol  : method stop()
> [ERROR] location: class org.mortbay.jetty.Server
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please 
> read the following articles:
> [ERROR] [Help 1] 
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the 
> command
> [ERROR]   mvn  -rf :hadoop-auth
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Re: Release Apache Hadoop 2.0.6-alpha

[Konstantin Boudnik]

Can anyone with admin rights in YARN JIRA project release 2.0.6-alpha version?

Thanks,
  Cos

On Fri, Aug 23, 2013 at 11:44AM, Konstantin Boudnik wrote:
> All,
> 
> I have pushed the bits of 2.0.6-alpha into the open and released staging bits
> in Nexus. Site is updated with the release news. 
> 
> 2.0.6-alpha is officially live now. Thanks everybody for your help!
>   Cos
> 
> On Thu, Aug 22, 2013 at 11:09PM, Konstantin Boudnik wrote:
> > I am clearly +1 (non-binding) on the release.
> > 
> > With 8 +1s (5 binding), no -1s or 0s the vote passes.
> > 
> > Thanks to all who verified the bits, I'll push them out shortly.
> > 
> > Thanks,
> >   Cos
> > 
> > On Thu, Aug 15, 2013 at 10:29PM, Konstantin Boudnik wrote:
> > > All,
> > > 
> > > I have created a release candidate (rc1) for hadoop-2.0.6-alpha that I 
> > > would
> > > like to release.
> > > 
> > > This is a stabilization release that includes fixed for a couple a of 
> > > issues
> > > as outlined on the security list.
> > > 
> > > The RC is available at: 
> > > http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc1/
> > > The RC tag in svn is here: 
> > > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc1
> > > 
> > > The maven artifacts are available via repository.apache.org.
> > > 
> > > The only difference between rc0 and rc1 is ASL added to releasenotes.html 
> > > and
> > > updated release dates in CHANGES.txt files.
> > > 
> > > Please try the release bits and vote; the vote will run for the usual 7 
> > > days.
> > > 
> > > Thanks for your voting
> > >   Cos
> > > 
> > 
> > 
> 
> 


signature.asc
Description: Digital signature

Release Apache Hadoop 2.0.6-alpha

[Konstantin Boudnik]

All,

I have pushed the bits of 2.0.6-alpha into the open and released staging bits
in Nexus. Site is updated with the release news. 

2.0.6-alpha is officially live now. Thanks everybody for your help!
  Cos

On Thu, Aug 22, 2013 at 11:09PM, Konstantin Boudnik wrote:
> I am clearly +1 (non-binding) on the release.
> 
> With 8 +1s (5 binding), no -1s or 0s the vote passes.
> 
> Thanks to all who verified the bits, I'll push them out shortly.
> 
> Thanks,
>   Cos
> 
> On Thu, Aug 15, 2013 at 10:29PM, Konstantin Boudnik wrote:
> > All,
> > 
> > I have created a release candidate (rc1) for hadoop-2.0.6-alpha that I would
> > like to release.
> > 
> > This is a stabilization release that includes fixed for a couple a of issues
> > as outlined on the security list.
> > 
> > The RC is available at: 
> > http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc1/
> > The RC tag in svn is here: 
> > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc1
> > 
> > The maven artifacts are available via repository.apache.org.
> > 
> > The only difference between rc0 and rc1 is ASL added to releasenotes.html 
> > and
> > updated release dates in CHANGES.txt files.
> > 
> > Please try the release bits and vote; the vote will run for the usual 7 
> > days.
> > 
> > Thanks for your voting
> >   Cos
> > 
> 
> 

Re: [VOTE] Release Apache Hadoop 2.0.6-alpha (RC1)

[Konstantin Boudnik]

I am clearly +1 (non-binding) on the release.

With 8 +1s (5 binding), no -1s or 0s the vote passes.

Thanks to all who verified the bits, I'll push them out shortly.

Thanks,
  Cos

On Thu, Aug 15, 2013 at 10:29PM, Konstantin Boudnik wrote:
> All,
> 
> I have created a release candidate (rc1) for hadoop-2.0.6-alpha that I would
> like to release.
> 
> This is a stabilization release that includes fixed for a couple a of issues
> as outlined on the security list.
> 
> The RC is available at: http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc1/
> The RC tag in svn is here: 
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc1
> 
> The maven artifacts are available via repository.apache.org.
> 
> The only difference between rc0 and rc1 is ASL added to releasenotes.html and
> updated release dates in CHANGES.txt files.
> 
> Please try the release bits and vote; the vote will run for the usual 7 days.
> 
> Thanks for your voting
>   Cos
> 




signature.asc
Description: Digital signature

[VOTE] Release Apache Hadoop 2.0.6-alpha (RC1)

[Konstantin Boudnik]

All,

I have created a release candidate (rc1) for hadoop-2.0.6-alpha that I would
like to release.

This is a stabilization release that includes fixed for a couple a of issues
as outlined on the security list.

The RC is available at: http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc1/
The RC tag in svn is here: 
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc1

The maven artifacts are available via repository.apache.org.

The only difference between rc0 and rc1 is ASL added to releasenotes.html and
updated release dates in CHANGES.txt files.

Please try the release bits and vote; the vote will run for the usual 7 days.

Thanks for your voting
  Cos



signature.asc
Description: Digital signature

Re: [VOTE] Release Apache Hadoop 2.0.6-alpha

[Konstantin Boudnik]

Sure Alejandro, not a big deal - I agree. Uploaded RC1 and restartig the vote
right away. Thanks for catching this!

Cos

On Thu, Aug 15, 2013 at 09:08PM, Alejandro Abdelnur wrote:
> it should be straight forward adding the license headers to the release
> notes. please make sure apache-rat:check passes on the RC before publishing
> it.
> 
> Arun, as you are about to cut the new RC for 2.1.0-beta, can you please
> make sure the license headers are used in the releasenotes HTML files?
> 
> Thx
> 
> 
> On Thu, Aug 15, 2013 at 8:02 PM, Konstantin Boudnik  wrote:
> 
> > Alejandro,
> >
> > looking into the source code: it seems that release notes never had license
> > boilerplate in it, hence 2.0.6-alpha doesn't have as well.
> >
> > I have fixed CHANGES with new optimistic date of the release and upload rc1
> > right now.
> >
> > Please let me know if you feel like we need start doing the license for the
> > releasenotes in this release.
> >
> > Thanks,
> >   Cos
> >
> > On Wed, Aug 14, 2013 at 10:40AM, Alejandro Abdelnur wrote:
> > > OK:
> > > * verified MD5
> > > * verified signature
> > > * expanded source tar and did a build
> > > * configured pseudo cluster and run a couple of example MR jobs
> > > * did a few HTTP calls to HTTFS
> > >
> > > NOT OK:
> > > * CHANGES.txt files have 2.0.6 as UNRELEASED, they should have the date
> > the
> > > RC vote ends
> > > * 'mvn apache-rat:check' fails, releasenotes HTML files don't have
> > license
> > > headers,
> > >
> > > I think we need to address the NO OK points (specially the last one),
> > they
> > > are trivial.
> > >
> > > Thanks.
> > >
> > >
> > >
> > > On Sat, Aug 10, 2013 at 5:46 PM, Konstantin Boudnik 
> > wrote:
> > >
> > > > All,
> > > >
> > > > I have created a release candidate (rc0) for hadoop-2.0.6-alpha that I
> > > > would
> > > > like to release.
> > > >
> > > > This is a stabilization release that includes fixed for a couple a of
> > > > issues
> > > > as outlined on the security list.
> > > >
> > > > The RC is available at:
> > > > http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc0/
> > > > The RC tag in svn is here:
> > > >
> > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc0
> > > >
> > > > The maven artifacts are available via repository.apache.org.
> > > >
> > > > Please try the release bits and vote; the vote will run for the usual 7
> > > > days.
> > > >
> > > > Thanks for your voting
> > > >   Cos
> > > >
> > > >
> > >
> > >
> > > --
> > > Alejandro
> >
> 
> 
> 
> -- 
> Alejandro


signature.asc
Description: Digital signature

Re: [VOTE] Release Apache Hadoop 2.0.6-alpha

[Konstantin Boudnik]

Alejandro,

looking into the source code: it seems that release notes never had license
boilerplate in it, hence 2.0.6-alpha doesn't have as well.

I have fixed CHANGES with new optimistic date of the release and upload rc1
right now.

Please let me know if you feel like we need start doing the license for the
releasenotes in this release.

Thanks,
  Cos

On Wed, Aug 14, 2013 at 10:40AM, Alejandro Abdelnur wrote:
> OK:
> * verified MD5
> * verified signature
> * expanded source tar and did a build
> * configured pseudo cluster and run a couple of example MR jobs
> * did a few HTTP calls to HTTFS
> 
> NOT OK:
> * CHANGES.txt files have 2.0.6 as UNRELEASED, they should have the date the
> RC vote ends
> * 'mvn apache-rat:check' fails, releasenotes HTML files don't have license
> headers,
> 
> I think we need to address the NO OK points (specially the last one), they
> are trivial.
> 
> Thanks.
> 
> 
> 
> On Sat, Aug 10, 2013 at 5:46 PM, Konstantin Boudnik  wrote:
> 
> > All,
> >
> > I have created a release candidate (rc0) for hadoop-2.0.6-alpha that I
> > would
> > like to release.
> >
> > This is a stabilization release that includes fixed for a couple a of
> > issues
> > as outlined on the security list.
> >
> > The RC is available at:
> > http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc0/
> > The RC tag in svn is here:
> > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc0
> >
> > The maven artifacts are available via repository.apache.org.
> >
> > Please try the release bits and vote; the vote will run for the usual 7
> > days.
> >
> > Thanks for your voting
> >   Cos
> >
> >
> 
> 
> -- 
> Alejandro


signature.asc
Description: Digital signature

Re: [VOTE] Release Apache Hadoop 2.0.6-alpha

[Konstantin Boudnik]

Darn CHANGES - apparently they just hate me. I agree that needs to be
addressed. I will spin-up rc1 tonight and restart the vote.

Thanks,
  Cos

On Wed, Aug 14, 2013 at 10:40AM, Alejandro Abdelnur wrote:
> OK:
> * verified MD5
> * verified signature
> * expanded source tar and did a build
> * configured pseudo cluster and run a couple of example MR jobs
> * did a few HTTP calls to HTTFS
> 
> NOT OK:
> * CHANGES.txt files have 2.0.6 as UNRELEASED, they should have the date the
> RC vote ends
> * 'mvn apache-rat:check' fails, releasenotes HTML files don't have license
> headers,
> 
> I think we need to address the NO OK points (specially the last one), they
> are trivial.
> 
> Thanks.
> 
> 
> 
> On Sat, Aug 10, 2013 at 5:46 PM, Konstantin Boudnik  wrote:
> 
> > All,
> >
> > I have created a release candidate (rc0) for hadoop-2.0.6-alpha that I
> > would
> > like to release.
> >
> > This is a stabilization release that includes fixed for a couple a of
> > issues
> > as outlined on the security list.
> >
> > The RC is available at:
> > http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc0/
> > The RC tag in svn is here:
> > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc0
> >
> > The maven artifacts are available via repository.apache.org.
> >
> > Please try the release bits and vote; the vote will run for the usual 7
> > days.
> >
> > Thanks for your voting
> >   Cos
> >
> >
> 
> 
> -- 
> Alejandro

[VOTE] Release Apache Hadoop 2.0.6-alpha

[Konstantin Boudnik]

All,

I have created a release candidate (rc0) for hadoop-2.0.6-alpha that I would
like to release.

This is a stabilization release that includes fixed for a couple a of issues
as outlined on the security list.

The RC is available at: http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc0/
The RC tag in svn is here: 
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc0

The maven artifacts are available via repository.apache.org.

Please try the release bits and vote; the vote will run for the usual 7 days.

Thanks for your voting
  Cos



signature.asc
Description: Digital signature

Re: creating 2.2.0 version in JIRA

[Konstantin Boudnik]

Same here.

On Tue, Jul 02, 2013 at 03:54PM, Alejandro Abdelnur wrote:
> We need clarification on this then.
> 
> I was under the impression that branch-2 would be 2.2.0.
> 
> thx
> 
> On Tue, Jul 2, 2013 at 2:38 PM, Jason Lowe  wrote:
> 
> > I thought Arun intends for 2.2.0 to be created off of branch-2.1.0-beta
> > and not off of branch-2.  As I understand it, only critical blockers will
> > be the delta between 2.1.0-beta and 2.2.0 and items checked into branch-2
> > should be marked as  fixed in 2.3.0.
> >
> > Part of the confusion is that currently branch-2 builds as 2.2.0-SNAPSHOT,
> > but I believe Arun intended it to be 2.3.0-SNAPSHOT.
> >
> > Jason
> >
> >
> > On 06/21/2013 12:05 PM, Alejandro Abdelnur wrote:
> >
> >> Thanks Suresh, didn't know that, will do.
> >>
> >>
> >> On Fri, Jun 21, 2013 at 9:48 AM, Suresh Srinivas  >> >wrote:
> >>
> >>  I have added in to HDFS, HADOOP, MAPREDUCE projects. Can someone add it
> >>> for
> >>> YARN?
> >>>
> >>>
> >>> On Fri, Jun 21, 2013 at 9:35 AM, Alejandro Abdelnur  >>>
>  wrote:
>  When Arun created branch-2.1-beta he stated:
> 
>   The expectation is that 2.2.0 will be limited to content in
> >
>  branch-2.1-beta
> 
> > and we stick to stabilizing it henceforth (I've deliberately not
> >
>  created
> >>>
>  2.2.0
> 
> > fix-version on jira yet).
> >
>  I working/committing some JIRAs that I'm putting in branch-2 (testcases
> 
> >>> and
> >>>
>  improvements) but I don't want to put them in branch-2.1-beta as they
>  are
>  not critical and I don't won't add unnecessary noise to the
> 
> >>> branch-2.1-beta
> >>>
>  release work.
> 
>  Currently branch-2 POMs have a version 2.2.0 and the CHANGES.txt files
>  as
>  well.
> 
>  But because we did not create a JIRA version I cannot close those JIRAs.
> 
>  Can we please create the JIRA versions? later we can rename them.
> 
>  Thx
> 
> 
>  --
>  Alejandro
> 
> 
> >>>
> >>> --
> >>> http://hortonworks.com/**download/ 
> >>>
> >>>
> >>
> >>
> >
> 
> 
> -- 
> Alejandro

[VOTE RESULT] Release Apache Hadoop 2.0.5-alpha (rc2)

[Konstantin Boudnik]

I am clearly +1 (non-binding) on the release.

With 8 +1s (3 binding), no -1s or 0s the vote passes.

Thanks to all who verified the bits, I'll push them out shortly.

Thanks,
  Cos

On Mon, Jun 03, 2013 at 12:51PM, Konstantin Boudnik wrote:
> I have rolled out release candidate (rc2) for hadoop-2.0.5-alpha.
> 
> The difference between rc1 and rc2 is the "optimistic release date" is set for
> 06/06/2013 in the CHANGES.txt files.
> 
> The binary artifact is the same - there's no need to rebuild it. The maven
> artifacts are the same.
> 
> The difference between the two RCs:
> 
> svn diff \
>   
> https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc1/ \
>   https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc2/
> 
> New RC builds are uploaded to the web.
> The RC is available at: http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc2/
> The RC tag in svn is here: 
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc2
> 
> I would like to extend the vote for another three days for it is such a minor
> change that doesn't affect anything but the recorded release date. Please
> cast your vote before 06/06/2013 5pm PDT.
> 
> Thanks for your patience!
>   Cos
> 
> On Fri, May 31, 2013 at 09:27PM, Konstantin Boudnik wrote:
> > All,
> > 
> > I have created a release candidate (rc1) for hadoop-2.0.5-alpha that I would
> > like to release.
> > 
> > This is a stabilization release that includes fixed for a couple a of issues
> > discovered in the testing with BigTop 0.6.0 release candidate.
> > 
> > The RC is available at: 
> > http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
> > The RC tag in svn is here: 
> > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc1
> > 
> > The maven artifacts will be available via repository.apache.org on Sat, June
> > 1st, 2013 at 2 pm PDT as outlined here
> > http://s.apache.org/WKD
> > 
> > Please try the release bits and vote; the vote will run for the 3 days,
> > because this is just a version name change. The bits are identical to the 
> > ones
> > voted on before in 
> > http://s.apache.org/2041move
> > 
> > Thanks for your voting
> >   Cos
> > 

Re: [VOTE] Release Apache Hadoop 2.0.5-alpha (rc2)

[Konstantin Boudnik]

Good point Thomas - I have just fixed the name of the file. Nice catch!

Sigh... these three text files... If anyone feels strongly about having them
in place - I will update the tarballs and recalculate the checksums. Or just
leave it as is, perhaps? So, either way is fine with me.

Thanks,
  Cos

On Tue, Jun 04, 2013 at 08:47PM, Thomas Graves wrote:
> Cos,
> 
> The name of the mds file for the binary tar is missing a "." in your
> apache directory: hadoop-2.0.5-alphatar.gz.mds
> <http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc2/hadoop-2.0.5-alphatar
> .gz.mds>
> 
> Contents of it are fine though so you might just add the ".".
> 
> Also the NOTICE.txt, README.txt, and LICENSE.txt files are missing from
> the top level directory in both the src and binary tar ball.
> 
> Everything else looks good. Verified checksums, signatures, built, ran
> single node cluster.
> 
> Thanks,
> Tom
>  
> 
> 
> On 6/3/13 2:51 PM, "Konstantin Boudnik"  wrote:
> 
> >I have rolled out release candidate (rc2) for hadoop-2.0.5-alpha.
> >
> >The difference between rc1 and rc2 is the "optimistic release date" is
> >set for
> >06/06/2013 in the CHANGES.txt files.
> >
> >The binary artifact is the same - there's no need to rebuild it. The maven
> >artifacts are the same.
> >
> >The difference between the two RCs:
> >
> >svn diff \
> >  
> >https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc
> >1/ \
> >  
> >https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc
> >2/
> >
> >New RC builds are uploaded to the web.
> >The RC is available at:
> >http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc2/
> >The RC tag in svn is here:
> >http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc2
> >
> >I would like to extend the vote for another three days for it is such a
> >minor
> >change that doesn't affect anything but the recorded release date. Please
> >cast your vote before 06/06/2013 5pm PDT.
> >
> >Thanks for your patience!
> >  Cos
> >
> >On Fri, May 31, 2013 at 09:27PM, Konstantin Boudnik wrote:
> >> All,
> >> 
> >> I have created a release candidate (rc1) for hadoop-2.0.5-alpha that I
> >>would
> >> like to release.
> >> 
> >> This is a stabilization release that includes fixed for a couple a of
> >>issues
> >> discovered in the testing with BigTop 0.6.0 release candidate.
> >> 
> >> The RC is available at:
> >>http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
> >> The RC tag in svn is here:
> >>http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc
> >>1
> >> 
> >> The maven artifacts will be available via repository.apache.org on Sat,
> >>June
> >> 1st, 2013 at 2 pm PDT as outlined here
> >> http://s.apache.org/WKD
> >> 
> >> Please try the release bits and vote; the vote will run for the 3 days,
> >> because this is just a version name change. The bits are identical to
> >>the ones
> >> voted on before in
> >> http://s.apache.org/2041move
> >> 
> >> Thanks for your voting
> >>   Cos
> >> 
> 

Re: [VOTE] Release Apache Hadoop 2.0.5-alpha (rc2)

[Konstantin Boudnik]

I have rolled out release candidate (rc2) for hadoop-2.0.5-alpha.

The difference between rc1 and rc2 is the "optimistic release date" is set for
06/06/2013 in the CHANGES.txt files.

The binary artifact is the same - there's no need to rebuild it. The maven
artifacts are the same.

The difference between the two RCs:

svn diff \
  https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc1/ \
  https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc2/

New RC builds are uploaded to the web.
The RC is available at: http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc2/
The RC tag in svn is here: 
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc2

I would like to extend the vote for another three days for it is such a minor
change that doesn't affect anything but the recorded release date. Please
cast your vote before 06/06/2013 5pm PDT.

Thanks for your patience!
  Cos

On Fri, May 31, 2013 at 09:27PM, Konstantin Boudnik wrote:
> All,
> 
> I have created a release candidate (rc1) for hadoop-2.0.5-alpha that I would
> like to release.
> 
> This is a stabilization release that includes fixed for a couple a of issues
> discovered in the testing with BigTop 0.6.0 release candidate.
> 
> The RC is available at: http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
> The RC tag in svn is here: 
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc1
> 
> The maven artifacts will be available via repository.apache.org on Sat, June
> 1st, 2013 at 2 pm PDT as outlined here
> http://s.apache.org/WKD
> 
> Please try the release bits and vote; the vote will run for the 3 days,
> because this is just a version name change. The bits are identical to the ones
> voted on before in 
> http://s.apache.org/2041move
> 
> Thanks for your voting
>   Cos
> 


signature.asc
Description: Digital signature

Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

[Konstantin Boudnik]

Ok, if this the consensus on the issue, then tonight I will cut rc2 with only
CHANGES.txt release dates update. After that I will extend the vote once
again.

Sorry for the hassle to all who did voted before.
  Cos

On Mon, Jun 03, 2013 at 12:18AM, Doug Cutting wrote:
> On Sun, Jun 2, 2013 at 9:01 AM, Alejandro Abdelnur  wrote:
> > [...] an easy way to solve it it would be using as release date the date 
> > the vote ends.
> 
> +1 This is what I've done for release candidate builds.
> 
> Doug

Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

[Konstantin Boudnik]

On Sun, Jun 02, 2013 at 09:17PM, Arun C Murthy wrote:
> I do it in the release branch, so the release source tarball has the date.

I guess 2.0.4-alpha was cut differently for whatever reason, 'cause i see the
UNRELEASED date in the source tarball.

As I said - if anyone feels strongly about it I will alter the CHANGES.txt
files

thanks,
  Cos

> Arun
> 
> On Jun 2, 2013, at 6:41 PM, Konstantin Boudnik wrote:
> 
> > On Sun, Jun 02, 2013 at 03:04PM, Arun C Murthy wrote:
> >> I record the 'release date' as the one on which the build is created. Not
> >> sure what Matt actually does on branch-1. Matt?
> > 
> > Arun, but you do this not in the release source tarball, right? Just in the
> > rest of integration branches, ie branch-2 and trunk. Am I correct?
> > 
> > Thanks,
> >  Cos
> > 
> >> hth,
> >> Arun
> >> 
> >> On Jun 2, 2013, at 12:33 PM, Konstantin Boudnik wrote:
> >> 
> >>> This is my first Hadoop release, so I am all ears ;) I looked at how Arun 
> >>> was
> >>> cutting the previous 2.0.x and followed the example. 
> >>> 
> >>> Thanks for your input and help Alejandro!
> >>> 
> >>> Regards,
> >>> Cos
> >>> 
> >>> On Sun, Jun 02, 2013 at 09:01AM, Alejandro Abdelnur wrote:
> >>>> I know we had this issue before. But and easy way to solve it it would be
> >>>> using as release date the date the vote ends. Anyway, not a big deal if 
> >>>> we
> >>>> are not cutting a new rc for other reason
> >>>> 
> >>>> +1 on rc1
> >>>> 
> >>>> Thx
> >>>> 
> >>>> On Jun 2, 2013, at 12:04 AM, Konstantin Boudnik  wrote:
> >>>> 
> >>>>> Alejandro,
> >>>>> 
> >>>>> I believe this is chicken and egg problem: I can't put release date into
> >>>>> unreleased tarball. Looking into 2.0.4-alpha source tarball I see the 
> >>>>> same
> >>>>> situation:
> >>>>> 
> >>>>> Release 2.0.4-alpha - UNRELEASED
> >>>>> 
> >>>>> But in the trunk and branch-2 the release data is in place. I don't 
> >>>>> think this
> >>>>> is an issue. But I would be happy to fix it if this seems to be a 
> >>>>> problem.
> >>>>> 
> >>>>> Thanks,
> >>>>> Cos
> >>>>> 
> >>>>> On Sat, Jun 01, 2013 at 08:04PM, Alejandro Abdelnur wrote:
> >>>>>> On RC1, verified MD5 & signature, built, configured pseudo cluster, 
> >>>>>> run a
> >>>>>> couple of sample jobs, tested HTTPFS.
> >>>>>> 
> >>>>>> CHANGES.txt files contents are correct now. Still, a minor NIT, they 
> >>>>>> have
> >>>>>> 2.0.5 as UNRELEASED, shouldn't they have a date (I would assume the 
> >>>>>> date
> >>>>>> the vote ends).
> >>>>>> 
> >>>>>> Thanks
> >>>>>> 
> >>>>>> 
> >>>>>> On Fri, May 31, 2013 at 9:39 PM, J. Rottinghuis 
> >>>>>> wrote:
> >>>>>> 
> >>>>>>> Thanks for fixing Cos.
> >>>>>>> http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
> >>>>>>> looks good to me.
> >>>>>>> +1 (non-binding)
> >>>>>>> 
> >>>>>>> Thanks,
> >>>>>>> 
> >>>>>>> Joep
> >>>>>>> 
> >>>>>>> 
> >>>>>>> On Fri, May 31, 2013 at 8:25 PM, Konstantin Boudnik 
> >>>>>>> wrote:
> >>>>>>> 
> >>>>>>>> Ok, WRT HDFS-4646 - it is all legit and the code is in 
> >>>>>>>> branch-2.0.4-alpha
> >>>>>>>> and
> >>>>>>>> later. It has been committed as
> >>>>>>>> r1465124
> >>>>>>>> The reason it isn't normally visible because of the weird commit 
> >>>>>>>> message:
> >>>>>>>> 
> >>>>>>>>  svn merge -c1465121 from trunk
> >>>>>>>> 
> 

Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

[Konstantin Boudnik]

On Sun, Jun 02, 2013 at 03:04PM, Arun C Murthy wrote:
> I record the 'release date' as the one on which the build is created. Not
> sure what Matt actually does on branch-1. Matt?

Arun, but you do this not in the release source tarball, right? Just in the
rest of integration branches, ie branch-2 and trunk. Am I correct?

Thanks,
  Cos

> hth,
> Arun
> 
> On Jun 2, 2013, at 12:33 PM, Konstantin Boudnik wrote:
> 
> > This is my first Hadoop release, so I am all ears ;) I looked at how Arun 
> > was
> > cutting the previous 2.0.x and followed the example. 
> > 
> > Thanks for your input and help Alejandro!
> > 
> > Regards,
> >  Cos
> > 
> > On Sun, Jun 02, 2013 at 09:01AM, Alejandro Abdelnur wrote:
> >> I know we had this issue before. But and easy way to solve it it would be
> >> using as release date the date the vote ends. Anyway, not a big deal if we
> >> are not cutting a new rc for other reason
> >> 
> >> +1 on rc1
> >> 
> >> Thx
> >> 
> >> On Jun 2, 2013, at 12:04 AM, Konstantin Boudnik  wrote:
> >> 
> >>> Alejandro,
> >>> 
> >>> I believe this is chicken and egg problem: I can't put release date into
> >>> unreleased tarball. Looking into 2.0.4-alpha source tarball I see the same
> >>> situation:
> >>> 
> >>> Release 2.0.4-alpha - UNRELEASED
> >>> 
> >>> But in the trunk and branch-2 the release data is in place. I don't think 
> >>> this
> >>> is an issue. But I would be happy to fix it if this seems to be a problem.
> >>> 
> >>> Thanks,
> >>> Cos
> >>> 
> >>> On Sat, Jun 01, 2013 at 08:04PM, Alejandro Abdelnur wrote:
> >>>> On RC1, verified MD5 & signature, built, configured pseudo cluster, run a
> >>>> couple of sample jobs, tested HTTPFS.
> >>>> 
> >>>> CHANGES.txt files contents are correct now. Still, a minor NIT, they have
> >>>> 2.0.5 as UNRELEASED, shouldn't they have a date (I would assume the date
> >>>> the vote ends).
> >>>> 
> >>>> Thanks
> >>>> 
> >>>> 
> >>>> On Fri, May 31, 2013 at 9:39 PM, J. Rottinghuis 
> >>>> wrote:
> >>>> 
> >>>>> Thanks for fixing Cos.
> >>>>> http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
> >>>>> looks good to me.
> >>>>> +1 (non-binding)
> >>>>> 
> >>>>> Thanks,
> >>>>> 
> >>>>> Joep
> >>>>> 
> >>>>> 
> >>>>> On Fri, May 31, 2013 at 8:25 PM, Konstantin Boudnik 
> >>>>> wrote:
> >>>>> 
> >>>>>> Ok, WRT HDFS-4646 - it is all legit and the code is in 
> >>>>>> branch-2.0.4-alpha
> >>>>>> and
> >>>>>> later. It has been committed as
> >>>>>> r1465124
> >>>>>> The reason it isn't normally visible because of the weird commit 
> >>>>>> message:
> >>>>>> 
> >>>>>>   svn merge -c1465121 from trunk
> >>>>>> 
> >>>>>> So, we good. I am done with the CHANGES.txt fixed that you guys have
> >>>>> noted
> >>>>>> earlier and will be re-spinning RC1 in a few.
> >>>>>> 
> >>>>>> Cos
> >>>>>> 
> >>>>>> On Fri, May 31, 2013 at 08:07PM, Konstantin Boudnik wrote:
> >>>>>>> Alejandro,
> >>>>>>> 
> >>>>>>> thanks for looking into this. Indeed - I missed the 2.0.5-alpha 
> >>>>>>> section
> >>>>>> in
> >>>>>>> YARN CHANGES.txt. Added now. As for HDFS-4646: apparently I didn't get
> >>>>>> into
> >>>>>>> to branch-2.0.4-alpha back then, although I distinctively remember
> >>>>> doing
> >>>>>> this.
> >>>>>>> Let me pull it into 2.0.5-alpha and update CHANGES.txt to reflect it.
> >>>>>> Also, I
> >>>>>>> will do JIRA in a moment.
> >>>>>>> 
> >>>>>>> Joep, appreciate the thorough examination. I have fixed the dates for
> >>>>> the
> >>>&

Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

[Konstantin Boudnik]

This is my first Hadoop release, so I am all ears ;) I looked at how Arun was
cutting the previous 2.0.x and followed the example. 

Thanks for your input and help Alejandro!

Regards,
  Cos

On Sun, Jun 02, 2013 at 09:01AM, Alejandro Abdelnur wrote:
> I know we had this issue before. But and easy way to solve it it would be
> using as release date the date the vote ends. Anyway, not a big deal if we
> are not cutting a new rc for other reason
> 
> +1 on rc1
> 
> Thx
> 
> On Jun 2, 2013, at 12:04 AM, Konstantin Boudnik  wrote:
> 
> > Alejandro,
> > 
> > I believe this is chicken and egg problem: I can't put release date into
> > unreleased tarball. Looking into 2.0.4-alpha source tarball I see the same
> > situation:
> > 
> > Release 2.0.4-alpha - UNRELEASED
> > 
> > But in the trunk and branch-2 the release data is in place. I don't think 
> > this
> > is an issue. But I would be happy to fix it if this seems to be a problem.
> > 
> > Thanks,
> > Cos
> > 
> > On Sat, Jun 01, 2013 at 08:04PM, Alejandro Abdelnur wrote:
> >> On RC1, verified MD5 & signature, built, configured pseudo cluster, run a
> >> couple of sample jobs, tested HTTPFS.
> >> 
> >> CHANGES.txt files contents are correct now. Still, a minor NIT, they have
> >> 2.0.5 as UNRELEASED, shouldn't they have a date (I would assume the date
> >> the vote ends).
> >> 
> >> Thanks
> >> 
> >> 
> >> On Fri, May 31, 2013 at 9:39 PM, J. Rottinghuis 
> >> wrote:
> >> 
> >>> Thanks for fixing Cos.
> >>> http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
> >>> looks good to me.
> >>> +1 (non-binding)
> >>> 
> >>> Thanks,
> >>> 
> >>> Joep
> >>> 
> >>> 
> >>> On Fri, May 31, 2013 at 8:25 PM, Konstantin Boudnik 
> >>> wrote:
> >>> 
> >>>> Ok, WRT HDFS-4646 - it is all legit and the code is in branch-2.0.4-alpha
> >>>> and
> >>>> later. It has been committed as
> >>>>  r1465124
> >>>> The reason it isn't normally visible because of the weird commit message:
> >>>> 
> >>>>svn merge -c1465121 from trunk
> >>>> 
> >>>> So, we good. I am done with the CHANGES.txt fixed that you guys have
> >>> noted
> >>>> earlier and will be re-spinning RC1 in a few.
> >>>> 
> >>>> Cos
> >>>> 
> >>>> On Fri, May 31, 2013 at 08:07PM, Konstantin Boudnik wrote:
> >>>>> Alejandro,
> >>>>> 
> >>>>> thanks for looking into this. Indeed - I missed the 2.0.5-alpha section
> >>>> in
> >>>>> YARN CHANGES.txt. Added now. As for HDFS-4646: apparently I didn't get
> >>>> into
> >>>>> to branch-2.0.4-alpha back then, although I distinctively remember
> >>> doing
> >>>> this.
> >>>>> Let me pull it into 2.0.5-alpha and update CHANGES.txt to reflect it.
> >>>> Also, I
> >>>>> will do JIRA in a moment.
> >>>>> 
> >>>>> Joep, appreciate the thorough examination. I have fixed the dates for
> >>> the
> >>>>> releases 2.0.4-alpha. As for the top-level readme file - sorry I wasn't
> >>>> aware
> >>>>> about them. As for the binary: I am pretty sure we are only releasing
> >>>> source
> >>>>> code, but I will put binaries into the rc1 respin.
> >>>>> 
> >>>>> I will respin rc1 shortly. Appreciate the feedback!
> >>>>>  Cos
> >>>>> 
> >>>>> On Fri, May 31, 2013 at 05:27PM, Alejandro Abdelnur wrote:
> >>>>>> Verified MD5 & signature, built, configured pseudo cluster, run a
> >>>> couple of
> >>>>>> sample jobs, tested HTTPFS.
> >>>>>> 
> >>>>>> Still, something seems odd.
> >>>>>> 
> >>>>>> The HDFS CHANGES.txt has the following entry under 2.0.5-alpha:
> >>>>>> 
> >>>>>> HDFS-4646. createNNProxyWithClientProtocol ignores configured
> >>> timeout
> >>>>>> value  (Jagane Sundar via cos)
> >>>>>> 
> >>>>>> but I don't see that in the branch.
> >>>>>> 
> &g

Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

[Konstantin Boudnik]

Alejandro,

I believe this is chicken and egg problem: I can't put release date into
unreleased tarball. Looking into 2.0.4-alpha source tarball I see the same
situation:

Release 2.0.4-alpha - UNRELEASED

But in the trunk and branch-2 the release data is in place. I don't think this
is an issue. But I would be happy to fix it if this seems to be a problem.

Thanks,
Cos

On Sat, Jun 01, 2013 at 08:04PM, Alejandro Abdelnur wrote:
> On RC1, verified MD5 & signature, built, configured pseudo cluster, run a
> couple of sample jobs, tested HTTPFS.
> 
> CHANGES.txt files contents are correct now. Still, a minor NIT, they have
> 2.0.5 as UNRELEASED, shouldn't they have a date (I would assume the date
> the vote ends).
> 
> Thanks
> 
> 
> On Fri, May 31, 2013 at 9:39 PM, J. Rottinghuis wrote:
> 
> > Thanks for fixing Cos.
> > http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
> > looks good to me.
> > +1 (non-binding)
> >
> > Thanks,
> >
> > Joep
> >
> >
> > On Fri, May 31, 2013 at 8:25 PM, Konstantin Boudnik 
> > wrote:
> >
> > > Ok, WRT HDFS-4646 - it is all legit and the code is in branch-2.0.4-alpha
> > > and
> > > later. It has been committed as
> > >   r1465124
> > > The reason it isn't normally visible because of the weird commit message:
> > >
> > > svn merge -c1465121 from trunk
> > >
> > > So, we good. I am done with the CHANGES.txt fixed that you guys have
> > noted
> > > earlier and will be re-spinning RC1 in a few.
> > >
> > > Cos
> > >
> > > On Fri, May 31, 2013 at 08:07PM, Konstantin Boudnik wrote:
> > > > Alejandro,
> > > >
> > > > thanks for looking into this. Indeed - I missed the 2.0.5-alpha section
> > > in
> > > > YARN CHANGES.txt. Added now. As for HDFS-4646: apparently I didn't get
> > > into
> > > > to branch-2.0.4-alpha back then, although I distinctively remember
> > doing
> > > this.
> > > > Let me pull it into 2.0.5-alpha and update CHANGES.txt to reflect it.
> > > Also, I
> > > > will do JIRA in a moment.
> > > >
> > > > Joep, appreciate the thorough examination. I have fixed the dates for
> > the
> > > > releases 2.0.4-alpha. As for the top-level readme file - sorry I wasn't
> > > aware
> > > > about them. As for the binary: I am pretty sure we are only releasing
> > > source
> > > > code, but I will put binaries into the rc1 respin.
> > > >
> > > > I will respin rc1 shortly. Appreciate the feedback!
> > > >   Cos
> > > >
> > > > On Fri, May 31, 2013 at 05:27PM, Alejandro Abdelnur wrote:
> > > > > Verified MD5 & signature, built, configured pseudo cluster, run a
> > > couple of
> > > > > sample jobs, tested HTTPFS.
> > > > >
> > > > > Still, something seems odd.
> > > > >
> > > > > The HDFS CHANGES.txt has the following entry under 2.0.5-alpha:
> > > > >
> > > > >  HDFS-4646. createNNProxyWithClientProtocol ignores configured
> > timeout
> > > > > value  (Jagane Sundar via cos)
> > > > >
> > > > > but I don't see that in the branch.
> > > > >
> > > > > And, the YARN CHANGES.txt does not have the 2.0.5-alpha section (it
> > > should
> > > > > be there empty).
> > > > >
> > > > > Cos, can you please look at these 2 things and explain/fix?
> > > > >
> > > > > Thanks.
> > > > >
> > > > >
> > > > >
> > > > > On Fri, May 31, 2013 at 4:04 PM, Konstantin Boudnik 
> > > wrote:
> > > > >
> > > > > > All,
> > > > > >
> > > > > > I have created a release candidate (rc0) for hadoop-2.0.5-alpha
> > that
> > > I
> > > > > > would
> > > > > > like to release.
> > > > > >
> > > > > > This is a stabilization release that includes fixed for a couple a
> > of
> > > > > > issues
> > > > > > discovered in the testing with BigTop 0.6.0 release candidate.
> > > > > >
> > > > > > The RC is available at:
> > > > > > http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc0/
> > > > > > The RC tag in svn is here:
> > > > > >
> > >
> > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc0
> > > > > >
> > > > > > The maven artifacts will be available via repository.apache.org on
> > > Sat,
> > > > > > June
> > > > > > 1st, 2013 at 2 pm PDT as outlined here
> > > > > > http://s.apache.org/WKD
> > > > > >
> > > > > > Please try the release bits and vote; the vote will run for the 3
> > > days,
> > > > > > because this is just a version name change. The bits are identical
> > > to the
> > > > > > ones
> > > > > > voted on before in
> > > > > > http://s.apache.org/2041move
> > > > > >
> > > > > > Thanks for your voting
> > > > > >   Cos
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Alejandro
> > >
> > >
> > >
> >
> 
> 
> 
> -- 
> Alejandro

Re: Heads up: moving from 2.0.4.1-alpha to 2.0.5-alpha

[Konstantin Boudnik]

I changes are completed, I have updated branch-2 and trunk CHANGES.txt files
as planned. Version of the branch-2 is set to 2.1.0-beta; 2.0.5-alpha release
artifacts are deployed to the staging area.

Thanks for your patience, guys!
  Cos

On Fri, May 31, 2013 at 12:45PM, Konstantin Boudnik wrote:
> Guys,
> 
> I will be performing some changes wrt to moving 2.0.4.1 release candidate to
> 2.0.5 space. As outline below by Alejandro:
> 
> 1. I will create new 2.0.5-alpha branch from the current head of 2.0.4-alpha
> that contains 2.0.4.1 changes
> 2. consequently, set the artifacts version on the new branch to be 2.0.5-alpha
> 3. the CHANGES.txt will be updated accordingly on the new 2.0.5 branch
> 4. At this point I can cut an RC and put it out for re-vote. The staging can
> be done after the next two steps.
> 
> I will be doing all these modifications in the next hour or so.
> 
> Tomorrow at 1 pm PDT I would like to:
> 1. update the version of the artifacts on branch-2 to become 2.1.0-SNAPSHOT
> 2. update the CHANGES.txt in the trunk and branch-2 to reflect new version 
> names
> 3. at this point it should safe to do the staging for 2.0.5-alpha RC
> 
> To avoid any collisions during the last two steps - especially 2. - I would
> ask everyone to hold off the modifications of the CHANGES.txt files on trunk
> and branch-2 between 1 pm and 2 pm PDT.
> 
> Please let me know if you see any flaw above, questions.
>   Cos
> 
> > As we change from 2.0.4.1 to 2.0.5 you'll need to do the following
> > housekeeping as you work the new RC.
> > 
> > * rename the svn branch
> > * update the versions in the POMs
> > * update the CHANGES.txt in trunk, branch-2 and the release branch
> > * change the current 2.0.5 version in JIRA to 2.1.0, create a new 2.0.5
> > version, change the fix version of the 2 JIRAs that make the RC
> 
> > I renamed 2.0.5-beta to 2.1.0-beta and 2.0.4.1-alpha to 2.0.5-alpha versions
> > in jira for HADOOP, HDFS, YARN & MAPREDUCE.
> 
> > Please take care of the rest.
> 
> > Also, in branch-2, the version should be 2.1.0-SNAPSHOT.


signature.asc
Description: Digital signature

Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

[Konstantin Boudnik]

Chris,

with 2.0.5-alpha (rc1) out, would you please take a look at the release bits?
I assume the four-digit numbering scheme issue has been resolved now.

Regards,
  Cos

On Thu, May 30, 2013 at 06:18PM, Chris Douglas wrote:
> On Thu, May 30, 2013 at 5:51 PM, Konstantin Boudnik  wrote:
> > I have no issues of changing the version to 2.0.5-alpha and restarting to 
> > vote
> > for the release content, e.g. 2 bug fixes. Shall I call 3 days re-vote 
> > because
> > of the number change?
> 
> +1 Sounds great.
> 
> > Does the result of bylaw vote nullifies the unfinished vote started by Arun?
> > Sorry, I am dense, apparently.
> 
> Yes, nobody should feel bound by either vote. The bylaw change
> clarifies that release plans are for RMs to solicit feedback and gauge
> PMC support for an artifact, not pre-approvals for doing work.
> 
> > Can we limit the vote thread to the merits of the release then?
> 
> Happily.
> 
> > That sound like adding an insult to injury, if my forth-language skills do 
> > not
> > mislead me.
> 
> They do mislead you, or I've expressed the point imprecisely. We can
> take this offline. -C
> 
> >> >> > On Thu, May 30, 2013 at 01:48PM, Chris Douglas wrote:
> >> >> >> On Thu, May 30, 2013 at 10:57 AM, Arun C Murthy 
> >> >> >>  wrote:
> >> >> >> > Why not include MAPREDUCE-4211 as well rather than create one 
> >> >> >> > release per patch?
> >> >> >>
> >> >> >> From Cos's description, it sounded like these were backports of fixes
> >> >> >> to help Sqoop2 and fix some build issues. If it's not just to fixup
> >> >> >> leftover bugs in 2.0.4 *once* so downstream projects can integrate
> >> >> >> against 2.0.4.1, and this a release series, then I've completely
> >> >> >> misunderstood the purpose.
> >> >> >>
> >> >> >> Cos, are you planning 2.0.4.2?
> >> >> >>
> >> >> >> > Also, this is the first time we are seeing a four-numbered scheme 
> >> >> >> > in Hadoop. Why not call this 2.0.5-alpha?
> >> >> >>
> >> >> >> Good point. Since it contains only backports from branch-2, it would
> >> >> >> make sense for it to be an intermediate release.
> >> >> >>
> >> >> >> I shouldn't have to say this, but I'm changing my vote to -1 while we
> >> >> >> work this out. -C
> >> >> >>
> >> >> >> > On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
> >> >> >> >
> >> >> >> >> All,
> >> >> >> >>
> >> >> >> >> I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha 
> >> >> >> >> that I would
> >> >> >> >> like to release.
> >> >> >> >>
> >> >> >> >> This is a stabilization release that includes fixed for a couple 
> >> >> >> >> a of issues
> >> >> >> >> discovered in the testing with BigTop 0.6.0 release candidate.
> >> >> >> >>
> >> >> >> >> The RC is available at: 
> >> >> >> >> http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
> >> >> >> >> The RC tag in svn is here: 
> >> >> >> >> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
> >> >> >> >>
> >> >> >> >> The maven artifacts are available via repository.apache.org.
> >> >> >> >>
> >> >> >> >> Please try the release bits and vote; the vote will run for the 
> >> >> >> >> usual 7 days.
> >> >> >> >>
> >> >> >> >> Thanks for your voting
> >> >> >> >>  Cos
> >> >> >> >>
> >> >> >> >
> >> >> >> >


signature.asc
Description: Digital signature

[VOTE] Release Apache Hadoop 2.0.5-alpha (rc1)

[Konstantin Boudnik]

All,

I have created a release candidate (rc1) for hadoop-2.0.5-alpha that I would
like to release.

This is a stabilization release that includes fixed for a couple a of issues
discovered in the testing with BigTop 0.6.0 release candidate.

The RC is available at: http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
The RC tag in svn is here: 
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc1

The maven artifacts will be available via repository.apache.org on Sat, June
1st, 2013 at 2 pm PDT as outlined here
http://s.apache.org/WKD

Please try the release bits and vote; the vote will run for the 3 days,
because this is just a version name change. The bits are identical to the ones
voted on before in 
http://s.apache.org/2041move

Thanks for your voting
  Cos

Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

[Konstantin Boudnik]

Ok, WRT HDFS-4646 - it is all legit and the code is in branch-2.0.4-alpha and
later. It has been committed as
  r1465124
The reason it isn't normally visible because of the weird commit message:

svn merge -c1465121 from trunk

So, we good. I am done with the CHANGES.txt fixed that you guys have noted
earlier and will be re-spinning RC1 in a few.

Cos

On Fri, May 31, 2013 at 08:07PM, Konstantin Boudnik wrote:
> Alejandro,
> 
> thanks for looking into this. Indeed - I missed the 2.0.5-alpha section in
> YARN CHANGES.txt. Added now. As for HDFS-4646: apparently I didn't get into
> to branch-2.0.4-alpha back then, although I distinctively remember doing this.
> Let me pull it into 2.0.5-alpha and update CHANGES.txt to reflect it. Also, I
> will do JIRA in a moment.
> 
> Joep, appreciate the thorough examination. I have fixed the dates for the
> releases 2.0.4-alpha. As for the top-level readme file - sorry I wasn't aware
> about them. As for the binary: I am pretty sure we are only releasing source
> code, but I will put binaries into the rc1 respin.
> 
> I will respin rc1 shortly. Appreciate the feedback!
>   Cos
> 
> On Fri, May 31, 2013 at 05:27PM, Alejandro Abdelnur wrote:
> > Verified MD5 & signature, built, configured pseudo cluster, run a couple of
> > sample jobs, tested HTTPFS.
> > 
> > Still, something seems odd.
> > 
> > The HDFS CHANGES.txt has the following entry under 2.0.5-alpha:
> > 
> >  HDFS-4646. createNNProxyWithClientProtocol ignores configured timeout
> > value  (Jagane Sundar via cos)
> > 
> > but I don't see that in the branch.
> > 
> > And, the YARN CHANGES.txt does not have the 2.0.5-alpha section (it should
> > be there empty).
> > 
> > Cos, can you please look at these 2 things and explain/fix?
> > 
> > Thanks.
> > 
> > 
> > 
> > On Fri, May 31, 2013 at 4:04 PM, Konstantin Boudnik  wrote:
> > 
> > > All,
> > >
> > > I have created a release candidate (rc0) for hadoop-2.0.5-alpha that I
> > > would
> > > like to release.
> > >
> > > This is a stabilization release that includes fixed for a couple a of
> > > issues
> > > discovered in the testing with BigTop 0.6.0 release candidate.
> > >
> > > The RC is available at:
> > > http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc0/
> > > The RC tag in svn is here:
> > > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc0
> > >
> > > The maven artifacts will be available via repository.apache.org on Sat,
> > > June
> > > 1st, 2013 at 2 pm PDT as outlined here
> > > http://s.apache.org/WKD
> > >
> > > Please try the release bits and vote; the vote will run for the 3 days,
> > > because this is just a version name change. The bits are identical to the
> > > ones
> > > voted on before in
> > > http://s.apache.org/2041move
> > >
> > > Thanks for your voting
> > >   Cos
> > >
> > >
> > 
> > 
> > -- 
> > Alejandro




signature.asc
Description: Digital signature

Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

[Konstantin Boudnik]

Alejandro,

thanks for looking into this. Indeed - I missed the 2.0.5-alpha section in
YARN CHANGES.txt. Added now. As for HDFS-4646: apparently I didn't get into
to branch-2.0.4-alpha back then, although I distinctively remember doing this.
Let me pull it into 2.0.5-alpha and update CHANGES.txt to reflect it. Also, I
will do JIRA in a moment.

Joep, appreciate the thorough examination. I have fixed the dates for the
releases 2.0.4-alpha. As for the top-level readme file - sorry I wasn't aware
about them. As for the binary: I am pretty sure we are only releasing source
code, but I will put binaries into the rc1 respin.

I will respin rc1 shortly. Appreciate the feedback!
  Cos

On Fri, May 31, 2013 at 05:27PM, Alejandro Abdelnur wrote:
> Verified MD5 & signature, built, configured pseudo cluster, run a couple of
> sample jobs, tested HTTPFS.
> 
> Still, something seems odd.
> 
> The HDFS CHANGES.txt has the following entry under 2.0.5-alpha:
> 
>  HDFS-4646. createNNProxyWithClientProtocol ignores configured timeout
> value  (Jagane Sundar via cos)
> 
> but I don't see that in the branch.
> 
> And, the YARN CHANGES.txt does not have the 2.0.5-alpha section (it should
> be there empty).
> 
> Cos, can you please look at these 2 things and explain/fix?
> 
> Thanks.
> 
> 
> 
> On Fri, May 31, 2013 at 4:04 PM, Konstantin Boudnik  wrote:
> 
> > All,
> >
> > I have created a release candidate (rc0) for hadoop-2.0.5-alpha that I
> > would
> > like to release.
> >
> > This is a stabilization release that includes fixed for a couple a of
> > issues
> > discovered in the testing with BigTop 0.6.0 release candidate.
> >
> > The RC is available at:
> > http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc0/
> > The RC tag in svn is here:
> > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc0
> >
> > The maven artifacts will be available via repository.apache.org on Sat,
> > June
> > 1st, 2013 at 2 pm PDT as outlined here
> > http://s.apache.org/WKD
> >
> > Please try the release bits and vote; the vote will run for the 3 days,
> > because this is just a version name change. The bits are identical to the
> > ones
> > voted on before in
> > http://s.apache.org/2041move
> >
> > Thanks for your voting
> >   Cos
> >
> >
> 
> 
> -- 
> Alejandro


signature.asc
Description: Digital signature

[VOTE] Release Apache Hadoop 2.0.5-alpha

[Konstantin Boudnik]

All,

I have created a release candidate (rc0) for hadoop-2.0.5-alpha that I would
like to release.

This is a stabilization release that includes fixed for a couple a of issues
discovered in the testing with BigTop 0.6.0 release candidate.

The RC is available at: http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc0/
The RC tag in svn is here: 
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc0

The maven artifacts will be available via repository.apache.org on Sat, June
1st, 2013 at 2 pm PDT as outlined here
http://s.apache.org/WKD

Please try the release bits and vote; the vote will run for the 3 days,
because this is just a version name change. The bits are identical to the ones
voted on before in 
http://s.apache.org/2041move

Thanks for your voting
  Cos

Re: Heads up: moving from 2.0.4.1-alpha to 2.0.5-alpha

[Konstantin Boudnik]

Indeed! The second part is happening on Saturday, JUN01 2013 1PM-2PM PST. 

On Fri, May 31, 2013 at 01:01PM, Alejandro Abdelnur wrote:
> Cos, just to be clear, this is happening SAT JUN01 1PM-2PM PST, not now
> (FRI MAY31 1PM PST). Correct?
> 
> Thx
> 
> On Fri, May 31, 2013 at 12:45 PM, Konstantin Boudnik  wrote:
> 
> > Guys,
> >
> > I will be performing some changes wrt to moving 2.0.4.1 release candidate
> > to
> > 2.0.5 space. As outline below by Alejandro:
> >
> > 1. I will create new 2.0.5-alpha branch from the current head of
> > 2.0.4-alpha
> > that contains 2.0.4.1 changes
> > 2. consequently, set the artifacts version on the new branch to be
> > 2.0.5-alpha
> > 3. the CHANGES.txt will be updated accordingly on the new 2.0.5 branch
> > 4. At this point I can cut an RC and put it out for re-vote. The staging
> > can
> > be done after the next two steps.
> >
> > I will be doing all these modifications in the next hour or so.
> >
> > Tomorrow at 1 pm PDT I would like to:
> > 1. update the version of the artifacts on branch-2 to become 2.1.0-SNAPSHOT
> > 2. update the CHANGES.txt in the trunk and branch-2 to reflect new version
> > names
> > 3. at this point it should safe to do the staging for 2.0.5-alpha RC
> >
> > To avoid any collisions during the last two steps - especially 2. - I would
> > ask everyone to hold off the modifications of the CHANGES.txt files on
> > trunk
> > and branch-2 between 1 pm and 2 pm PDT.
> >
> > Please let me know if you see any flaw above, questions.
> >   Cos
> >
> > > As we change from 2.0.4.1 to 2.0.5 you'll need to do the following
> > > housekeeping as you work the new RC.
> > >
> > > * rename the svn branch
> > > * update the versions in the POMs
> > > * update the CHANGES.txt in trunk, branch-2 and the release branch
> > > * change the current 2.0.5 version in JIRA to 2.1.0, create a new 2.0.5
> > > version, change the fix version of the 2 JIRAs that make the RC
> >
> > > I renamed 2.0.5-beta to 2.1.0-beta and 2.0.4.1-alpha to 2.0.5-alpha
> > versions
> > > in jira for HADOOP, HDFS, YARN & MAPREDUCE.
> >
> > > Please take care of the rest.
> >
> > > Also, in branch-2, the version should be 2.1.0-SNAPSHOT.
> >
> 
> 
> 
> -- 
> Alejandro

Heads up: moving from 2.0.4.1-alpha to 2.0.5-alpha

[Konstantin Boudnik]

Guys,

I will be performing some changes wrt to moving 2.0.4.1 release candidate to
2.0.5 space. As outline below by Alejandro:

1. I will create new 2.0.5-alpha branch from the current head of 2.0.4-alpha
that contains 2.0.4.1 changes
2. consequently, set the artifacts version on the new branch to be 2.0.5-alpha
3. the CHANGES.txt will be updated accordingly on the new 2.0.5 branch
4. At this point I can cut an RC and put it out for re-vote. The staging can
be done after the next two steps.

I will be doing all these modifications in the next hour or so.

Tomorrow at 1 pm PDT I would like to:
1. update the version of the artifacts on branch-2 to become 2.1.0-SNAPSHOT
2. update the CHANGES.txt in the trunk and branch-2 to reflect new version names
3. at this point it should safe to do the staging for 2.0.5-alpha RC

To avoid any collisions during the last two steps - especially 2. - I would
ask everyone to hold off the modifications of the CHANGES.txt files on trunk
and branch-2 between 1 pm and 2 pm PDT.

Please let me know if you see any flaw above, questions.
  Cos

> As we change from 2.0.4.1 to 2.0.5 you'll need to do the following
> housekeeping as you work the new RC.
> 
> * rename the svn branch
> * update the versions in the POMs
> * update the CHANGES.txt in trunk, branch-2 and the release branch
> * change the current 2.0.5 version in JIRA to 2.1.0, create a new 2.0.5
> version, change the fix version of the 2 JIRAs that make the RC

> I renamed 2.0.5-beta to 2.1.0-beta and 2.0.4.1-alpha to 2.0.5-alpha versions
> in jira for HADOOP, HDFS, YARN & MAPREDUCE.

> Please take care of the rest.

> Also, in branch-2, the version should be 2.1.0-SNAPSHOT.

Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

[Konstantin Boudnik]

Thanks for the headstart Arun!

I will be sending a separate email to the *-dev@ lists outlining the plan and
the schedule of the changes, so we won't step on each other feet, like Vinod
expressed earlier.

Cos

On Fri, May 31, 2013 at 11:20AM, Arun C Murthy wrote:
> 
> On May 30, 2013, at 8:41 PM, Alejandro Abdelnur wrote:
> 
> > Konstantin, Cos,
> > 
> > As we change from 2.0.4.1 to 2.0.5 you'll need to do the following
> > housekeeping as you work the new RC.
> > 
> > * rename the svn branch
> > * update the versions in the POMs
> > * update the CHANGES.txt in trunk, branch-2 and the release branch
> > * change the current 2.0.5 version in JIRA to 2.1.0, create a new 2.0.5
> > version, change the fix version of the 2 JIRAs that make the RC
> 
> I renamed 2.0.5-beta to 2.1.0-beta and 2.0.4.1-alpha to 2.0.5-alpha versions
> in jira for HADOOP, HDFS, YARN & MAPREDUCE.
> 
> Please take care of the rest.
> 
> Also, in branch-2, the version should be 2.1.0-SNAPSHOT.
> 
> thanks,
> Arun
> 
> > 
> > Thanks.
> > 
> > 
> > On Thu, May 30, 2013 at 6:18 PM, Chris Douglas  wrote:
> > 
> >> On Thu, May 30, 2013 at 5:51 PM, Konstantin Boudnik 
> >> wrote:
> >>> I have no issues of changing the version to 2.0.5-alpha and restarting
> >> to vote
> >>> for the release content, e.g. 2 bug fixes. Shall I call 3 days re-vote
> >> because
> >>> of the number change?
> >> 
> >> +1 Sounds great.
> >> 
> >>> Does the result of bylaw vote nullifies the unfinished vote started by
> >> Arun?
> >>> Sorry, I am dense, apparently.
> >> 
> >> Yes, nobody should feel bound by either vote. The bylaw change
> >> clarifies that release plans are for RMs to solicit feedback and gauge
> >> PMC support for an artifact, not pre-approvals for doing work.
> >> 
> >>> Can we limit the vote thread to the merits of the release then?
> >> 
> >> Happily.
> >> 
> >>> That sound like adding an insult to injury, if my forth-language skills
> >> do not
> >>> mislead me.
> >> 
> >> They do mislead you, or I've expressed the point imprecisely. We can
> >> take this offline. -C
> >> 
> >>>>>>> On Thu, May 30, 2013 at 01:48PM, Chris Douglas wrote:
> >>>>>>>> On Thu, May 30, 2013 at 10:57 AM, Arun C Murthy <
> >> a...@hortonworks.com> wrote:
> >>>>>>>>> Why not include MAPREDUCE-4211 as well rather than create one
> >> release per patch?
> >>>>>>>> 
> >>>>>>>> From Cos's description, it sounded like these were backports of
> >> fixes
> >>>>>>>> to help Sqoop2 and fix some build issues. If it's not just to
> >> fixup
> >>>>>>>> leftover bugs in 2.0.4 *once* so downstream projects can integrate
> >>>>>>>> against 2.0.4.1, and this a release series, then I've completely
> >>>>>>>> misunderstood the purpose.
> >>>>>>>> 
> >>>>>>>> Cos, are you planning 2.0.4.2?
> >>>>>>>> 
> >>>>>>>>> Also, this is the first time we are seeing a four-numbered
> >> scheme in Hadoop. Why not call this 2.0.5-alpha?
> >>>>>>>> 
> >>>>>>>> Good point. Since it contains only backports from branch-2, it
> >> would
> >>>>>>>> make sense for it to be an intermediate release.
> >>>>>>>> 
> >>>>>>>> I shouldn't have to say this, but I'm changing my vote to -1
> >> while we
> >>>>>>>> work this out. -C
> >>>>>>>> 
> >>>>>>>>> On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
> >>>>>>>>> 
> >>>>>>>>>> All,
> >>>>>>>>>> 
> >>>>>>>>>> I have created a release candidate (rc0) for
> >> hadoop-2.0.4.1-alpha that I would
> >>>>>>>>>> like to release.
> >>>>>>>>>> 
> >>>>>>>>>> This is a stabilization release that includes fixed for a
> >> couple a of issues
> >>>>>>>>>> discovered in the testing with BigTop 0.6.0 release candidate.
> >>>>>>>>>> 
> >>>>>>>>>> The RC is available at:
> >> http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
> >>>>>>>>>> The RC tag in svn is here:
> >> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
> >>>>>>>>>> 
> >>>>>>>>>> The maven artifacts are available via repository.apache.org.
> >>>>>>>>>> 
> >>>>>>>>>> Please try the release bits and vote; the vote will run for
> >> the usual 7 days.
> >>>>>>>>>> 
> >>>>>>>>>> Thanks for your voting
> >>>>>>>>>> Cos
> >>>>>>>>>> 
> >>>>>>>>> 
> >>>>>>>>> 
> >> 
> > 
> > 
> > 
> > -- 
> > Alejandro
> 
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
> 
> 

Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

[Konstantin Boudnik]

Thanks Alejandro,

that's what my plan for the morning. Thanks for putting together the
check-list - would be easier for me not to miss anything. I am aiming to have
the bits out by noon or so. Appreciate the help!

Cos

On Thu, May 30, 2013 at 08:41PM, Alejandro Abdelnur wrote:
> Konstantin, Cos,
> 
> As we change from 2.0.4.1 to 2.0.5 you'll need to do the following
> housekeeping as you work the new RC.
> 
> * rename the svn branch
> * update the versions in the POMs
> * update the CHANGES.txt in trunk, branch-2 and the release branch
> * change the current 2.0.5 version in JIRA to 2.1.0, create a new 2.0.5
> version, change the fix version of the 2 JIRAs that make the RC
> 
> Thanks.
> 
> 
> On Thu, May 30, 2013 at 6:18 PM, Chris Douglas  wrote:
> 
> > On Thu, May 30, 2013 at 5:51 PM, Konstantin Boudnik 
> > wrote:
> > > I have no issues of changing the version to 2.0.5-alpha and restarting
> > to vote
> > > for the release content, e.g. 2 bug fixes. Shall I call 3 days re-vote
> > because
> > > of the number change?
> >
> > +1 Sounds great.
> >
> > > Does the result of bylaw vote nullifies the unfinished vote started by
> > Arun?
> > > Sorry, I am dense, apparently.
> >
> > Yes, nobody should feel bound by either vote. The bylaw change
> > clarifies that release plans are for RMs to solicit feedback and gauge
> > PMC support for an artifact, not pre-approvals for doing work.
> >
> > > Can we limit the vote thread to the merits of the release then?
> >
> > Happily.
> >
> > > That sound like adding an insult to injury, if my forth-language skills
> > do not
> > > mislead me.
> >
> > They do mislead you, or I've expressed the point imprecisely. We can
> > take this offline. -C
> >
> > >> >> > On Thu, May 30, 2013 at 01:48PM, Chris Douglas wrote:
> > >> >> >> On Thu, May 30, 2013 at 10:57 AM, Arun C Murthy <
> > a...@hortonworks.com> wrote:
> > >> >> >> > Why not include MAPREDUCE-4211 as well rather than create one
> > release per patch?
> > >> >> >>
> > >> >> >> From Cos's description, it sounded like these were backports of
> > fixes
> > >> >> >> to help Sqoop2 and fix some build issues. If it's not just to
> > fixup
> > >> >> >> leftover bugs in 2.0.4 *once* so downstream projects can integrate
> > >> >> >> against 2.0.4.1, and this a release series, then I've completely
> > >> >> >> misunderstood the purpose.
> > >> >> >>
> > >> >> >> Cos, are you planning 2.0.4.2?
> > >> >> >>
> > >> >> >> > Also, this is the first time we are seeing a four-numbered
> > scheme in Hadoop. Why not call this 2.0.5-alpha?
> > >> >> >>
> > >> >> >> Good point. Since it contains only backports from branch-2, it
> > would
> > >> >> >> make sense for it to be an intermediate release.
> > >> >> >>
> > >> >> >> I shouldn't have to say this, but I'm changing my vote to -1
> > while we
> > >> >> >> work this out. -C
> > >> >> >>
> > >> >> >> > On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
> > >> >> >> >
> > >> >> >> >> All,
> > >> >> >> >>
> > >> >> >> >> I have created a release candidate (rc0) for
> > hadoop-2.0.4.1-alpha that I would
> > >> >> >> >> like to release.
> > >> >> >> >>
> > >> >> >> >> This is a stabilization release that includes fixed for a
> > couple a of issues
> > >> >> >> >> discovered in the testing with BigTop 0.6.0 release candidate.
> > >> >> >> >>
> > >> >> >> >> The RC is available at:
> > http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
> > >> >> >> >> The RC tag in svn is here:
> > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
> > >> >> >> >>
> > >> >> >> >> The maven artifacts are available via repository.apache.org.
> > >> >> >> >>
> > >> >> >> >> Please try the release bits and vote; the vote will run for
> > the usual 7 days.
> > >> >> >> >>
> > >> >> >> >> Thanks for your voting
> > >> >> >> >>  Cos
> > >> >> >> >>
> > >> >> >> >
> > >> >> >> >
> >
> 
> 
> 
> -- 
> Alejandro


signature.asc
Description: Digital signature

Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

[Konstantin Boudnik]

On Thu, May 30, 2013 at 05:30PM, Chris Douglas wrote:
> On Thu, May 30, 2013 at 3:25 PM, Konstantin Boudnik  wrote:
> > There's no plans to release anything else at this point - this is a bug-fix
> > release, as I pointed out on a numerous occasions. There's no new features -
> > just 2 fixes.
> 
> If you're worried about extending the voting by a week, I don't think
> anyone can reasonably object to a truncated schedule if the only
> change is the version number. Downstream fixes discovered in Bigtop
> are a sufficient reason for a patch release and I think we'd all like
> them to become routine. Not just in 2.0.x, but in later release
> branches.

I have no issues of changing the version to 2.0.5-alpha and restarting to vote
for the release content, e.g. 2 bug fixes. Shall I call 3 days re-vote because
of the number change?

> > 2.0.5 matter became and still is too controversial at some point. The vote
> > started by Arun to override the results of the Konstantin's vote never been
> > closed.
> 
> For the nth time, neither release plan vote was binding. We recently
> amended the bylaws to eliminate this confusion. There is no ambiguity
> between votes when neither is in scope.

Does the result of bylaw vote nullifies the unfinished vote started by Arun?
Sorry, I am dense, apparently.

> > The downstream projects are handing in the middle of the air because
> > of that confusion.
> 
> Can we please ground our discussion when discussing compatibility and
> bugs? Breathless alarm is not proportional to the severity, here.

Good point. Can we limit the vote thread to the merits of the release then?

> > Have I missed something or you just called me a cheater and a lair right to 
> > my face?
> 
> I called you neither. The prenominate votes were closed, counted, and
> declared to be binding over objections. I'm annoyed that I have to
> toggle my vote to prevent that tactic, but based on recent experience
> I don't expect you to forgo it. I'd be happy to learn my caution is
> unnecessary. -C

That sound like adding an insult to injury, if my forth-language skills do not
mislead me.

Cos

> >> > On Thu, May 30, 2013 at 01:48PM, Chris Douglas wrote:
> >> >> On Thu, May 30, 2013 at 10:57 AM, Arun C Murthy  
> >> >> wrote:
> >> >> > Why not include MAPREDUCE-4211 as well rather than create one release 
> >> >> > per patch?
> >> >>
> >> >> From Cos's description, it sounded like these were backports of fixes
> >> >> to help Sqoop2 and fix some build issues. If it's not just to fixup
> >> >> leftover bugs in 2.0.4 *once* so downstream projects can integrate
> >> >> against 2.0.4.1, and this a release series, then I've completely
> >> >> misunderstood the purpose.
> >> >>
> >> >> Cos, are you planning 2.0.4.2?
> >> >>
> >> >> > Also, this is the first time we are seeing a four-numbered scheme in 
> >> >> > Hadoop. Why not call this 2.0.5-alpha?
> >> >>
> >> >> Good point. Since it contains only backports from branch-2, it would
> >> >> make sense for it to be an intermediate release.
> >> >>
> >> >> I shouldn't have to say this, but I'm changing my vote to -1 while we
> >> >> work this out. -C
> >> >>
> >> >> > On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
> >> >> >
> >> >> >> All,
> >> >> >>
> >> >> >> I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha 
> >> >> >> that I would
> >> >> >> like to release.
> >> >> >>
> >> >> >> This is a stabilization release that includes fixed for a couple a 
> >> >> >> of issues
> >> >> >> discovered in the testing with BigTop 0.6.0 release candidate.
> >> >> >>
> >> >> >> The RC is available at: 
> >> >> >> http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
> >> >> >> The RC tag in svn is here: 
> >> >> >> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
> >> >> >>
> >> >> >> The maven artifacts are available via repository.apache.org.
> >> >> >>
> >> >> >> Please try the release bits and vote; the vote will run for the 
> >> >> >> usual 7 days.
> >> >> >>
> >> >> >> Thanks for your voting
> >> >> >>  Cos
> >> >> >>
> >> >> >
> >> >> >

Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

[Konstantin Boudnik]

On Thu, May 30, 2013 at 04:08PM, Jean-Daniel Cryans wrote:
> FWIW, not that I have a dog in this fight, but the only release with a
> 4th number (not including .0 like the 0.20.20x releases did) we had
> was:
> 
> http://hadoop.6.n7.nabble.com/VOTE-Release-0-17-2-1-rc-0-td13398.html
> 
> 0.17.2 was missing some native libs so 0.17.2.1 was released to fix
> that critical issue instead of calling it .3

Exactly the point - the _bigfix_ release. Thanks for pointing out the
similarities.

Cos

> 
> J-D
> 
> On Thu, May 30, 2013 at 3:52 PM, Konstantin Boudnik  wrote:
> > On Thu, May 30, 2013 at 03:11PM, Matt Foley wrote:
> >> Hi Cos,
> >> I would also request that you renumber the release candidate to just
> >> three-numbers, hence "2.0.5-alpha".
> >>
> >> Arun, are you willing to start the 2.1.x name-space for your next release,
> >> so that 2.0.x-alpha can become an intermediate stabilization branch as Cos
> >> and Konst want?
> >
> > Let's get the facts straight, Matt, please: this "want" has been expressed 
> > in
> > the official vote here http://s.apache.org/ZMf Apparently, 2.0.5-alpha is
> > blocked now because of the another vote that hasn't been closed yet for
> > whatever reason. In order to unblock a number of releases in downstream
> > component I have moved forward with this release. Do you have any material
> > objections to the release that pursue this goal?
> >
> >> I just think that using four-number schemes was symptomatic of the
> >> near-forking we had back in the 0.20.xxx.y days, and I really don't want to
> >> go back there.  Especially since you could say that "0.20.xxx.y" is just
> >> three significant numbers, the leading zero being inconsequential.
> >
> > I dare to remind that forth part of the version is reserved - not in a
> > parallel universe, of course - for "patch level" aka bug fixes. It hardly 
> > can
> > be taken a sign of 'forking' by any definition.
> >
> > Cos
> >
> >> So, would you please consider using 2.0.5-alpha?
> >>
> >> As for the "2.0.5-SNAPSHOT" in the branch-2 versioning, that's standard
> >> usage.  Whoever makes the 2.0.5 release (or any "next" release) is expected
> >> to update the parent branch's SNAPSHOT default versioning, per
> >> HowToReleasePostMavenization#Branching<https://wiki.apache.org/hadoop/HowToReleasePostMavenization#Branching>,
> >> step 6.
> >>
> >> Thanks,
> >> --Matt
> >>
> >>
> >> On Thu, May 30, 2013 at 11:52 AM, Konstantin Boudnik  
> >> wrote:
> >>
> >> > On Thu, May 30, 2013 at 10:57AM, Arun C Murthy wrote:
> >> > > I see you just re-opened MAPREUDCE-5211.
> >> > >
> >> > > Why not include MAPREDUCE-5211 as well rather than create one release
> >> > per patch?
> >> >
> >> > Arun, it is unclear if MAPREDUCE-5211 has implications in 2.0.4 as per
> >> >
> >> > https://issues.apache.org/jira/browse/MAPREDUCE-5211?focusedCommentId=13670574&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13670574
> >> >
> >> > Hence, there's a good chance that it never will be backported. And I 
> >> > don't
> >> > have any plans to created 'a release per patch'.
> >> >
> >> > > Also, this is the first time we are seeing a four-numbered scheme in
> >> > Hadoop.
> >> > > Why not call this 2.0.5-alpha?
> >> >
> >> > There were precedents in four-numbered schemes before: 0.20.20[3-5].0
> >> > comes to
> >> > mind.
> >> >
> >> > As for 2.0.5-alpha: The release numbering games and votes that had
> >> > happened in
> >> > the last few weeks are very confusing. Some of them never been concluded,
> >> > the
> >> > branches are moved and artifact versions seem to be colliding. 2.0.4.x
> >> > seems
> >> > to work well for the stabilization purposes and it will allow to unblock
> >> > downstream and integration projects quickly.
> >> >
> >> > Cos
> >> >
> >> > > Arun
> >> > >
> >> > > On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
> >> > >
> >> > > > All,
> >> > > >
> >> > > > I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha 
> >> > > > that
> >> > I would
> >> > > > like to release.
> >> > > >
> >> > > > This is a stabilization release that includes fixed for a couple a of
> >> > issues
> >> > > > discovered in the testing with BigTop 0.6.0 release candidate.
> >> > > >
> >> > > > The RC is available at:
> >> > http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
> >> > > > The RC tag in svn is here:
> >> > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
> >> > > >
> >> > > > The maven artifacts are available via repository.apache.org.
> >> > > >
> >> > > > Please try the release bits and vote; the vote will run for the usual
> >> > 7 days.
> >> > > >
> >> > > > Thanks for your voting
> >> > > >  Cos
> >> > > >
> >> > >
> >> > >
> >> >

Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

[Konstantin Boudnik]

On Thu, May 30, 2013 at 03:11PM, Matt Foley wrote:
> Hi Cos,
> I would also request that you renumber the release candidate to just
> three-numbers, hence "2.0.5-alpha".
> 
> Arun, are you willing to start the 2.1.x name-space for your next release,
> so that 2.0.x-alpha can become an intermediate stabilization branch as Cos
> and Konst want?

Let's get the facts straight, Matt, please: this "want" has been expressed in
the official vote here http://s.apache.org/ZMf Apparently, 2.0.5-alpha is
blocked now because of the another vote that hasn't been closed yet for
whatever reason. In order to unblock a number of releases in downstream
component I have moved forward with this release. Do you have any material
objections to the release that pursue this goal?

> I just think that using four-number schemes was symptomatic of the
> near-forking we had back in the 0.20.xxx.y days, and I really don't want to
> go back there.  Especially since you could say that "0.20.xxx.y" is just
> three significant numbers, the leading zero being inconsequential.

I dare to remind that forth part of the version is reserved - not in a
parallel universe, of course - for "patch level" aka bug fixes. It hardly can
be taken a sign of 'forking' by any definition.

Cos

> So, would you please consider using 2.0.5-alpha?
> 
> As for the "2.0.5-SNAPSHOT" in the branch-2 versioning, that's standard
> usage.  Whoever makes the 2.0.5 release (or any "next" release) is expected
> to update the parent branch's SNAPSHOT default versioning, per
> HowToReleasePostMavenization#Branching<https://wiki.apache.org/hadoop/HowToReleasePostMavenization#Branching>,
> step 6.
> 
> Thanks,
> --Matt
> 
> 
> On Thu, May 30, 2013 at 11:52 AM, Konstantin Boudnik  wrote:
> 
> > On Thu, May 30, 2013 at 10:57AM, Arun C Murthy wrote:
> > > I see you just re-opened MAPREUDCE-5211.
> > >
> > > Why not include MAPREDUCE-5211 as well rather than create one release
> > per patch?
> >
> > Arun, it is unclear if MAPREDUCE-5211 has implications in 2.0.4 as per
> >
> > https://issues.apache.org/jira/browse/MAPREDUCE-5211?focusedCommentId=13670574&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13670574
> >
> > Hence, there's a good chance that it never will be backported. And I don't
> > have any plans to created 'a release per patch'.
> >
> > > Also, this is the first time we are seeing a four-numbered scheme in
> > Hadoop.
> > > Why not call this 2.0.5-alpha?
> >
> > There were precedents in four-numbered schemes before: 0.20.20[3-5].0
> > comes to
> > mind.
> >
> > As for 2.0.5-alpha: The release numbering games and votes that had
> > happened in
> > the last few weeks are very confusing. Some of them never been concluded,
> > the
> > branches are moved and artifact versions seem to be colliding. 2.0.4.x
> > seems
> > to work well for the stabilization purposes and it will allow to unblock
> > downstream and integration projects quickly.
> >
> > Cos
> >
> > > Arun
> > >
> > > On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
> > >
> > > > All,
> > > >
> > > > I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha that
> > I would
> > > > like to release.
> > > >
> > > > This is a stabilization release that includes fixed for a couple a of
> > issues
> > > > discovered in the testing with BigTop 0.6.0 release candidate.
> > > >
> > > > The RC is available at:
> > http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
> > > > The RC tag in svn is here:
> > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
> > > >
> > > > The maven artifacts are available via repository.apache.org.
> > > >
> > > > Please try the release bits and vote; the vote will run for the usual
> > 7 days.
> > > >
> > > > Thanks for your voting
> > > >  Cos
> > > >
> > >
> > >
> >

Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

[Konstantin Boudnik]

On Thu, May 30, 2013 at 03:18PM, Chris Douglas wrote:
> On Thu, May 30, 2013 at 2:39 PM, Konstantin Boudnik  wrote:
> > There's no misunderstanding Chris - this release is to unblock downstream.
> >
> > As for your question: I don't have a crystal ball; I wish though. I think 
> > the
> > answer depends on will be there more blocking bugs found in the later 
> > releases
> > of Bigtop or other downstream components.
> > This is bugfix release and, I guess, if there are more bugs found in the
> > future - more releases would have to be cut. Isn't this is why the software 
> > is
> > being released?
> 
> Sure, but they're all backports from the release currently marked for
> 2.0.5. Either (a) these are really blocker bugs and we should roll a
> patch release or (b) some bleeding-edge work needs to work around this
> while branch-2 is released in the next few weeks. If it's not severe
> enough to justify disrupting the versioning of snapshot maven
> artifacts in branch-2, then we're clearly not in case (a).
> 
> I thought this was the result of extensive testing, and 2.0.4.1 was a
> release to enable additional integration before 2.0.5. If we plan to
> roll more releases as a subset of the bug fixes committed to branch-2
> then just call it 2.0.5. Please make sure it- and any future,
> intermediate release- is worth the disruption.

There's no plans to release anything else at this point - this is a bug-fix
release, as I pointed out on a numerous occasions. There's no new features -
just 2 fixes.

2.0.5 matter became and still is too controversial at some point. The vote
started by Arun to override the results of the Konstantin's vote never been
closed. The downstream projects are handing in the middle of the air because
of that confusion. 

> > Now, the -1: I am not clear about the justification. What exactly we expect 
> > to
> > "work out"?
> 
> It's become fashionable to close threads and count votes in the middle
> of the discussion. I changed my vote instead of trusting you. -C

Have I missed something or you just called me a cheater and a lair right to my 
face?

Cos

> > On Thu, May 30, 2013 at 01:48PM, Chris Douglas wrote:
> >> On Thu, May 30, 2013 at 10:57 AM, Arun C Murthy  
> >> wrote:
> >> > Why not include MAPREDUCE-4211 as well rather than create one release 
> >> > per patch?
> >>
> >> From Cos's description, it sounded like these were backports of fixes
> >> to help Sqoop2 and fix some build issues. If it's not just to fixup
> >> leftover bugs in 2.0.4 *once* so downstream projects can integrate
> >> against 2.0.4.1, and this a release series, then I've completely
> >> misunderstood the purpose.
> >>
> >> Cos, are you planning 2.0.4.2?
> >>
> >> > Also, this is the first time we are seeing a four-numbered scheme in 
> >> > Hadoop. Why not call this 2.0.5-alpha?
> >>
> >> Good point. Since it contains only backports from branch-2, it would
> >> make sense for it to be an intermediate release.
> >>
> >> I shouldn't have to say this, but I'm changing my vote to -1 while we
> >> work this out. -C
> >>
> >> > On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
> >> >
> >> >> All,
> >> >>
> >> >> I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha that 
> >> >> I would
> >> >> like to release.
> >> >>
> >> >> This is a stabilization release that includes fixed for a couple a of 
> >> >> issues
> >> >> discovered in the testing with BigTop 0.6.0 release candidate.
> >> >>
> >> >> The RC is available at: 
> >> >> http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
> >> >> The RC tag in svn is here: 
> >> >> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
> >> >>
> >> >> The maven artifacts are available via repository.apache.org.
> >> >>
> >> >> Please try the release bits and vote; the vote will run for the usual 7 
> >> >> days.
> >> >>
> >> >> Thanks for your voting
> >> >>  Cos
> >> >>
> >> >
> >> >

Re: Cannot update "PoweredBy" wiki page

[Konstantin Boudnik]

Someone with proper karma needs to add you into ACL for this page.

On Thu, May 30, 2013 at 11:14PM, Adam Kawa wrote:
> Hi,
> 
> When uploading new content (and information about my company), I got the
> exception
> "Sorry, can not save page because "rubbelloselotto.de" is not allowed in
> this wiki."
> 
> How could I solve it?
> 
> Kind regards,
> Adam

Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

[Konstantin Boudnik]

There's no misunderstanding Chris - this release is to unblock downstream.

As for your question: I don't have a crystal ball; I wish though. I think the
answer depends on will be there more blocking bugs found in the later releases
of Bigtop or other downstream components.
This is bugfix release and, I guess, if there are more bugs found in the
future - more releases would have to be cut. Isn't this is why the software is
being released?

Now, the -1: I am not clear about the justification. What exactly we expect to
"work out"?

Cos

On Thu, May 30, 2013 at 01:48PM, Chris Douglas wrote:
> On Thu, May 30, 2013 at 10:57 AM, Arun C Murthy  wrote:
> > Why not include MAPREDUCE-4211 as well rather than create one release per 
> > patch?
> 
> From Cos's description, it sounded like these were backports of fixes
> to help Sqoop2 and fix some build issues. If it's not just to fixup
> leftover bugs in 2.0.4 *once* so downstream projects can integrate
> against 2.0.4.1, and this a release series, then I've completely
> misunderstood the purpose.
> 
> Cos, are you planning 2.0.4.2?
> 
> > Also, this is the first time we are seeing a four-numbered scheme in 
> > Hadoop. Why not call this 2.0.5-alpha?
> 
> Good point. Since it contains only backports from branch-2, it would
> make sense for it to be an intermediate release.
> 
> I shouldn't have to say this, but I'm changing my vote to -1 while we
> work this out. -C
> 
> > On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
> >
> >> All,
> >>
> >> I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha that I 
> >> would
> >> like to release.
> >>
> >> This is a stabilization release that includes fixed for a couple a of 
> >> issues
> >> discovered in the testing with BigTop 0.6.0 release candidate.
> >>
> >> The RC is available at: 
> >> http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
> >> The RC tag in svn is here: 
> >> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
> >>
> >> The maven artifacts are available via repository.apache.org.
> >>
> >> Please try the release bits and vote; the vote will run for the usual 7 
> >> days.
> >>
> >> Thanks for your voting
> >>  Cos
> >>
> >
> >

Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

[Konstantin Boudnik]

On Thu, May 30, 2013 at 10:57AM, Arun C Murthy wrote:
> I see you just re-opened MAPREUDCE-5211.
> 
> Why not include MAPREDUCE-5211 as well rather than create one release per 
> patch?

Arun, it is unclear if MAPREDUCE-5211 has implications in 2.0.4 as per 
https://issues.apache.org/jira/browse/MAPREDUCE-5211?focusedCommentId=13670574&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13670574

Hence, there's a good chance that it never will be backported. And I don't
have any plans to created 'a release per patch'.

> Also, this is the first time we are seeing a four-numbered scheme in Hadoop.
> Why not call this 2.0.5-alpha?

There were precedents in four-numbered schemes before: 0.20.20[3-5].0 comes to
mind.

As for 2.0.5-alpha: The release numbering games and votes that had happened in
the last few weeks are very confusing. Some of them never been concluded, the
branches are moved and artifact versions seem to be colliding. 2.0.4.x seems
to work well for the stabilization purposes and it will allow to unblock
downstream and integration projects quickly.

Cos

> Arun
> 
> On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
> 
> > All,
> > 
> > I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha that I 
> > would
> > like to release.
> > 
> > This is a stabilization release that includes fixed for a couple a of issues
> > discovered in the testing with BigTop 0.6.0 release candidate.
> > 
> > The RC is available at: 
> > http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
> > The RC tag in svn is here: 
> > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
> > 
> > The maven artifacts are available via repository.apache.org.
> > 
> > Please try the release bits and vote; the vote will run for the usual 7 
> > days.
> > 
> > Thanks for your voting
> >  Cos
> > 
> 
> 

[VOTE] Release Apache Hadoop 2.0.4.1-alpha

[Konstantin Boudnik]

All,

I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha that I would
like to release.

This is a stabilization release that includes fixed for a couple a of issues
discovered in the testing with BigTop 0.6.0 release candidate.

The RC is available at: http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
The RC tag in svn is here: 
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0

The maven artifacts are available via repository.apache.org.

Please try the release bits and vote; the vote will run for the usual 7 days.

Thanks for your voting
  Cos



signature.asc
Description: Digital signature

Re: Bugfix release 2.0.4.1

[Konstantin Boudnik]

Thanks guys! I will start the release vote shortly.

Cos

On Thu, May 23, 2013 at 06:23PM, Anatoli Fomenko wrote:
> From Bigtop perspective, Hadoop══2.0.4.1 is unblocked with regards to a
> Sqoop 2 blocker, and the voting can be started.
> 
> Anatoli
> 
> -- Forwarded message --
> From: Konstantin Boudnik 
> Date: Mon, May 20, 2013 at 9:03 PM
> Subject: Re: Bugfix release 2.0.4.1
> To:═common-dev@hadoop.apache.org
> 
> 
> The build is over, all unit tests have passed.
> Roman, the 2.0.4.1 tarballs are available from
> https://builds.apache.org/job/ Hadoop-branch-2.0.4/47/
> 
> For Bigtop to use as the base.
> ═ Cos
> 
> On Mon, May 20, 2013 at 12:15PM, Konstantin Boudnik wrote:
> > I have ran the full set of unit tests on the patched version and I see 3
> > failures in HDFS HA stress tests, which are clearly not related to fix in
> > question. I will commit it shortly.
> >
> > Cos
> >
> > On Thu, May 16, 2013 at 12:03AM, Roman Shaposhnik wrote:
> > > On Wed, May 15, 2013 at 11:09 PM, Vinod Kumar Vavilapalli
> > >  wrote:
> > > >
> > > > It's a little dirty, but Mark and Jarek (presumably from BigTop) ran a 
> > > > patched version with my change at MAPREDUCE-5240.
> > >
> > > I've just updated the patch to apply cleanly on branch-2.0.4 (had to 
> > > refactor
> > > quite a bit of unit tests). Hopefully Cos can apply it═tomorrow═and we
> > > can start testing Bigtop.
> > >
> > > > Otherwise we may end up creating lots of bug-fix releases.
> > >
> > > Agreed. The point of 2.0.4.1 is to unblock Bigtop 0.6.0 release. We're
> > > trying to come with stable base line for Hadoop 2.0.x where
> > > all the downstream components will be running smoothly.
> > >
> > > At this point a MAPREDUCE-5240 is the only thing holding us
> > > up, but once we start our own round of testing other issues
> > > could pop up. We shall see how it goes.
> > >
> > > Thanks,
> > > Roman.══

Re: Bugfix release 2.0.4.1

[Konstantin Boudnik]

The build is over, all unit tests have passed.
Roman, the 2.0.4.1 tarballs are available from
  https://builds.apache.org/job/Hadoop-branch-2.0.4/47/

For Bigtop to use as the base.
  Cos

On Mon, May 20, 2013 at 12:15PM, Konstantin Boudnik wrote:
> I have ran the full set of unit tests on the patched version and I see 3
> failures in HDFS HA stress tests, which are clearly not related to fix in
> question. I will commit it shortly.
> 
> Cos
> 
> On Thu, May 16, 2013 at 12:03AM, Roman Shaposhnik wrote:
> > On Wed, May 15, 2013 at 11:09 PM, Vinod Kumar Vavilapalli
> >  wrote:
> > >
> > > It's a little dirty, but Mark and Jarek (presumably from BigTop) ran a 
> > > patched version with my change at MAPREDUCE-5240.
> > 
> > I've just updated the patch to apply cleanly on branch-2.0.4 (had to 
> > refactor
> > quite a bit of unit tests). Hopefully Cos can apply it tomorrow and we
> > can start testing Bigtop.
> > 
> > > Otherwise we may end up creating lots of bug-fix releases.
> > 
> > Agreed. The point of 2.0.4.1 is to unblock Bigtop 0.6.0 release. We're
> > trying to come with stable base line for Hadoop 2.0.x where
> > all the downstream components will be running smoothly.
> > 
> > At this point a MAPREDUCE-5240 is the only thing holding us
> > up, but once we start our own round of testing other issues
> > could pop up. We shall see how it goes.
> > 
> > Thanks,
> > Roman.


signature.asc
Description: Digital signature

Re: Bugfix release 2.0.4.1

[Konstantin Boudnik]

I have ran the full set of unit tests on the patched version and I see 3
failures in HDFS HA stress tests, which are clearly not related to fix in
question. I will commit it shortly.

Cos

On Thu, May 16, 2013 at 12:03AM, Roman Shaposhnik wrote:
> On Wed, May 15, 2013 at 11:09 PM, Vinod Kumar Vavilapalli
>  wrote:
> >
> > It's a little dirty, but Mark and Jarek (presumably from BigTop) ran a 
> > patched version with my change at MAPREDUCE-5240.
> 
> I've just updated the patch to apply cleanly on branch-2.0.4 (had to refactor
> quite a bit of unit tests). Hopefully Cos can apply it tomorrow and we
> can start testing Bigtop.
> 
> > Otherwise we may end up creating lots of bug-fix releases.
> 
> Agreed. The point of 2.0.4.1 is to unblock Bigtop 0.6.0 release. We're
> trying to come with stable base line for Hadoop 2.0.x where
> all the downstream components will be running smoothly.
> 
> At this point a MAPREDUCE-5240 is the only thing holding us
> up, but once we start our own round of testing other issues
> could pop up. We shall see how it goes.
> 
> Thanks,
> Roman.

Re: [VOTE] - Release 2.0.5-beta

[Konstantin Boudnik]

On Wed, May 15, 2013 at 10:52PM, Suresh Srinivas wrote:
> > Assuming that you are talking about HDFS features when you say
> 
> > > "features going into a beta on a very short short timetable" and
> > > "laundry list" etc,
> > >
> >
> > No, that would not be a correct assumption.
> >
> > So these features are not something that are impulsively developed
> > > and irresponsibly pushed to a release. They have gone through
> > > considerable testing and have been developed over a long time.
> > >
> >
> > There is no need to reframe my comment in combative terms and read in
> > insults that are not there.
> >
> 
> No insult taken. But I want to make a case that feature are not proposed
> lightly and due diligence both during development and testing are done.
> 
> >
> > As I read Arun's mail the plan is to integrate several feature branches
> > into branch-2. That would of course result in brand new never before tested
> > code. I do not believe that should have the label "alpha". This is just my
> > personal opinion. "Shit happens when commits happen." - You know this as
> > well as I. That does not mean I am here to attack or insult you by pointing
> > that out and suggesting more measured alternatives.
> >
> > There is little to gain in engaging in debate club. If you are not
> > interested in hearing these opinions, that is fine, I have received that
> > message already, nothing further need be said.
> 
> 
> Andy, I value your feedback. I am only trying to allay the concerns by
> sharing my perspective.

What I am seeing times and again in these endless discussion threads is this:

  a) "downstream or bigtop: we are seeing a bunch of integration issues with
every new feature introduced/something even a commit made"
  b) "feature developers: no-no, these features are developed for a long time,
tests are ran, no need to be concerned"

The same pattern is repeated times and again. The only conclusion that I can
make out of it, is that either the meaning of "integration testing" is
different for a) and b) or that a) and b) are using very different validation
mechanisms.

Which one is that? I am puzzled.

Bugs are quite expected - Andrew put it very eloquently, actually. But you
only can deal with them effectively if the flow of changes is controlled, e.g.
via smaller and focused releases. The development process has to be
converging, and not fanning-out. Case in point? Sure. 2.0.3-alpha had to be
followed by 2.0.4-alpha release (officially called bugfix release); it - in
turn - requires 2.0.4.1-alpha to make it suitable for other downstream
components. So, it took 2 releases to simply fix issues caused by a bunch of
bugfixes and no major new features being committed into 2.0.3-alpha. These are
just cold facts - not attacking any ones' ego here.

  Cos



signature.asc
Description: Digital signature

Re: [VOTE] - Release 2.0.5-beta

[Konstantin Boudnik]

Guys,

I guess what you're missing is that Bigtop isn't a testing framework for
Hadoop. It is stack framework that verifies that components are dealing with
each other nicely. Every single stack is different: Bigtop 0.5.0 differs from
0.6.0, and so on. Bigtop - as any other ASF project - has its releases that
might or might not be aligned with particular version of Hadoop. Hence, an
ethalon stack needs to be defined first and foremost.

Before we even start talking about running it nightly (another question is on
what hardware, let's not get there for now) let's understand who will can help
with triage'ing test failures? Downstreams, Hadoop or Bigtop?

Judging by a number of other emails there's a number of people on this list
who care plenty about integration issues. Any volunteers to help with
integration testing in the open?

> Is this a previously solved problem?

Yes. The problem is solved by separating actively developed (aka unstable)
release from more mature and less volatile ones. This is what has been
concluded upon two days ago in this voting thread http://s.apache.org/MnU

Cos

On Wed, May 15, 2013 at 04:54PM, Matt Foley wrote:
> Roman, what is your model for how test results from Bigtop should feed back
> into Hadoop-2 development?
> With the understanding that (a) software does have bugs, and (b) you're not
> going to get an SLA on community-sponsored software,
> what are your ideas for how to close the loop better?
> 
> Would "CI" runs of Bigtop against branch-2 be feasible, as Arun suggests?
> How should we accomodate changes in individual components (Hadoop Core, but
> others as well) that may require changes in one or more other components?
> How does Bigtop keep doing a viable nightly build in that chaotic
> environment?
> Is this a previously solved problem?
> 
> Thanks,
> --Matt
> 
> 
> On Wed, May 15, 2013 at 4:47 PM, Arun C Murthy  wrote:
> 
> >
> > On May 15, 2013, at 3:50 PM, Roman Shaposhnik wrote:
> >
> > > Arun,
> > >
> > > am I reading yours answer to my binary question correctly? It is a 'no'.
> >
> > No.
> >
> > >
> > > My reading of your response is that while you appreciate the feedback
> > > Bigtop is providing you're not of an opinion that investigating the level
> > > of stability of Hadoop wrt. downstream any further than what is currently
> > > happening would be a worthy investment of Hadoop's community
> > > (or your personal for that matter) time?
> >
> > Everyone is welcome to contribute in any and all manner. I can't speak for
> > everyone.
> > It would be useful if Bigtop could run regressions on releases here
> > consistently. We've also talked in the past about running Bigtop on
> > branch-2, nightly. Is that something you could help with? You'd earn my
> > personal gratitude.
> >
> > thanks,
> > Arun
> >
> > >
> > > Thanks,
> > > Roman.
> > >
> > > On Wed, May 15, 2013 at 3:02 PM, Arun C Murthy 
> > wrote:
> > >> Roman,
> > >>
> > >> Furthermore, before we rush into finding flaws and scaring kids at
> > night it would be useful to remember one thing:
> > >> Software has *bugs*. We can't block any release till the entire
> > universe validates it, in fact they won't validate it if we don't release
> > since are at the bottom of the stack.
> > >>
> > >> Any help prior to the release is welcome; I know people who work for
> > the same employer as I do have plans to do further testing after we freeze
> > apis via the beta release(s). I hope and pray others can join this effort -
> > thanks to everyone who already has.
> > >>
> > >> Again, freezing APIs and protocols is the primary aim of 2.0.5-beta.
> > There are no guarantees it's 100% bug-free, we can never make such
> > guarantees anyway.
> > >>
> > >> If, and when, we find bugs with 2.0.5-beta I'm more than happy to
> > quickly turn around and make more releases (2.0.6-beta, 2.0.7-beta).
> > Obviously I'll make a call on which bugs are critical - feedback to help me
> > decide is, as always, welcome.
> > >> I've been clear, many times, that we might need more than one beta
> > release to iron out bugs etc.
> > >>
> > >> None of this should be a surprise - this has happened many, many times
> > in the lifetime of this and other projects. 2.0.3-alpha vis-a-vis
> > 2.0.4-alpha is the most recent example - it won't be the last.
> > >>
> > >> So, I hope, concludes this meme.
> > >>
> > >> thanks,
> > >> Arun
> > >>
> > >> On May 15, 2013, at 2:20 PM, Arun C Murthy wrote:
> > >>
> > >>> Great summary, thanks Vinod.
> > >>>
> > >>> On May 15, 2013, at 2:14 PM, Vinod Kumar Vavilapalli wrote:
> > >>>
> > 
> >  Roman, I keep this same argument again and again. Should've refuted
> > earlier.
> > 
> >  Please list down all the issues that BigTop ran into *because of* new
> > features. You continue to argue that new features are destabilizing 2.0.*,
> > which I don't agree with at all. 2.0.3-alpha was the last time major
> > features got merged in, and we found blockers irrespective of those.
> > 
> > >

Re: [VOTE] - Release 2.0.5-beta

[Konstantin Boudnik]

Indeed. I think the root of the issue is deeper. ASF software practices are
great to deal with isolated, relatively contained projects like httpd,
libreoffice, trac, etc. However, Hadoop based stack - essentially, software
aimed at enterprises with bigger scale operations - is a different animal,
that requires balancing of a huge number of moving parts and an unbroken
flow of feedback up the stream. Anyone who have delivered any enterprise
grade software system knows perfectly well how hard is that.

However, in the environment where a release pushed out in the rush
(essentially causing DOA issues downstream), these are got fixed in consequent
releases.  That ironically is likely to contain some other DOAs because an
integration testing - and I mean real world integration system testing - is
done by this small project, that is treated like a toy for adolescent kids.
And there's no other real integration testing happening OPENLY on the full
stack. Despite numerous claims, that is.

Software comes with bugs - this is a somewhat expected phenomena. However, bug
fixes shouldn't be mixed with new features, increasing entropy in the system.
In other words, the development process should fan-in. A process with multiple
consequent stable releases helps to achieve it; and compatibility issues would
be addressed by working on the next major release.

The model above leaves downstream with a choice of sticking to the 3.x or 
switching
to 4.x and so on. Where's having permanent alpha tag is a convenient way to 
control
software project that effectively became a vendor-controlled effort.

And yes - this leads to fragmentation, makes no mistakes about it. Because no
one can sit on the hands for a year and wait until a usable release with all
great features will come about: lot of organizations just silently forking away
to make their own environment suitable for production or sale; some of them
might sporadically contribute something back. And of course - this is not the
aim of Apache project to produce commercial grade platform.

Cos

On Wed, May 15, 2013 at 02:54PM, Roman Shaposhnik wrote:
> On Wed, May 15, 2013 at 2:14 PM, Vinod Kumar Vavilapalli
>  wrote:
> > Please list down all the issues that BigTop ran into *because of* new 
> > features.
> 
> Whether the bug is *because of* new feature or not is a red herring
> for my argument. Please lets drop this distinction. I never used it.
> 
> > You continue to argue that new features are destabilizing 2.0.*,
> > which I don't agree with at all. 2.0.3-alpha was the last time major
> > features got merged in, and we found blockers irrespective of those.
> 
> This is not my argument at all. I apologize if somehow I failed to
> communicate it, but here's what my argument boils down to:
> given *my* experience with Hadoop 2.0.x series and Bigtop
> release every time I try a different release of Hadoop 2.0.x
> I run into issues that scare me. They scare me because
> they are so basic yet they make component like Sqoop
> and Oozie (and I believe Giraph on one occasion)
> pretty much DOA for YARN-base mapreduce implementation.
> 
> In my mind, what that translates into is the fact that nobody
> did *any* real testing of a particular downstream component
> running on a given Hadoop 2.0.x release. Like I said --
> the issues so far make the components in question DOA.

> Effectively the onion of issues remain unpeeled, so to speak.
> 
> What I'm asking on this thread (and somehow nobody is willing
> to give me a straight answer) is whether the Hadoop community
> is willing to invest in peeling this onion of issues somewhat more
> before declaring Hadoop 2.0.5 a beta release.
> 
> Once again it is a binary question -- please give me an answer
> of yes or no.
> 
> > I am not arguing that new features *may* destabilize the branch, but you've 
> > repeatedly stated this as if that were a fact.
> 
> Your list of issues is pretty complete (give or take a few that I didn't file
> but Cos and others did). And I'd be the first one to agree that
> it is not a large list of issues. What scares me is not its size,
> but the fact how basic they are and how the block the *rest*
> of the testing completely.
> 
> To be extra clear -- what scares me about something like
> MAPREDUCE-5240 is not whether it came as a result of
> a merge or was sitting there since day one. What scares
> me is that we've identified it last week and yet Sqoop 2 is
> DOA in its presense.
> 
> How many more issues like that one (regardless of how
> they originated) are in branch-2? Wouldn't we want to
> know before declaring Hadoop 2.0.5 beta?
> 
> Now, knowing would require work -- that's what my
> argument is all about.
> 
> 
> Thanks,
> Roman.

Bugfix release 2.0.4.1

[Konstantin Boudnik]

I'd like to spin-off a bugfix release for 2.0.4-alpha containing fix for
  MAPREDUCE-5240
Vinod has the fix on branch-2 already, so it seems like a no-brainer backport.

However, the issue is blocking Bigtop 0.6.0 from being released on top of
2.0.4-alpha. I am planning to backport the fix into 2.0.4-alpha branch and run
the tests, etc. Please let me know if there are any other critical fixes that
needs to be released along.

If someone with proper JIRA credentials can open 2.0.4.1-alpha version it
would be greatly appreciated!
  
-- 
Thank you,
  Cos

[jira] [Created] (HADOOP-9528) org.apache.hadoop.ha.TestZKFailoverController.testCedeActive is failing intermittently

[Konstantin Boudnik (JIRA)]

Konstantin Boudnik created HADOOP-9528:
--

 Summary: 
org.apache.hadoop.ha.TestZKFailoverController.testCedeActive is failing 
intermittently 
 Key: HADOOP-9528
 URL: https://issues.apache.org/jira/browse/HADOOP-9528
 Project: Hadoop Common
  Issue Type: Bug
  Components: ha
Affects Versions: 2.0.4-alpha
Reporter: Konstantin Boudnik


Unit tests is failing with a message like 
{{Should take ~3 seconds to rejoin. Only took 2276ms before rejoining.}}

A looking into the test shows the only 2.8 seconds and more is considered to be 
{{about 3 seconds}}. Say 2.7 seconds won't be considered {{about 3 seconds}} 
contrary to elementary school arithmetic.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [DISCUSS] Getting to Hadoop 2.X beta

[Konstantin Boudnik]

To add to it, here's what have been experiences as a disconnect during the
release cycle of 2.0.4-alpha using BigTop 0.6.6 as the integration platform and
management stack of choice:

1. Need for improved feedback from the downstream projects
2. Need for improved feedback from the DevOps

Which boils down to getting sufficient buy-in from downstreams and devops
communities that they are now willing to consider branch-2 as being landing
spot for their Hadoop 2.X needs.

I belive the recent effort around 2.0.4-alpha and BigTop 0.6.0 might be
sufficient to convince both communities to:
 1. migrate their Hadoop 2.X profiles to at least 2.0.4-alpha
 2. start to run integration tests against Hadoop 2.X profile
 3. start to publish Maven artifacts
 4. in general agree to treat Hadoop 2.X issues with at least
the same priority as the issues in default profiles are treated (which
are mostly Hadoop 1.X profiles).

that would be a good start. BigTop can certainly help with a lot of
techicalities and moving parts of the process. That would require tighter
coordination between releases, of course.

The question I don't have an answer for is how to engage DevOps community.

Thoughts?
  Cos

On Wed, Apr 24, 2013 at 03:38PM, Roman Shaposhnik wrote:
> On Wed, Apr 24, 2013 at 9:42 AM, Steve Loughran  
> wrote:
> >> There's quite a few preconditions to be met for a piece of
> >> software to reach beta status. Quite a few of them are now
> >> being discussed in a neighboring thread 'Compatibility in
> >> Apache Hadoop' ( http://s.apache.org/VE1 ) but I'd like to
> >> kick off a broader discussion: what do you think a *complete*
> >> list should look like, before we can honestly signal this
> >> change to the rest of the world.
> >>
> >> It would be very nice if the things that are offered as part
> >> of the criteria are easily quantifiable, but lets brainstorm
> >> first anyway. We can always stack-rank and quantify later.
> >>
> > I need to get YARN-117 in before the service lifecycle is frozen forever
> 
> We certainly need to get a list of blocker JIRAs for what would be
> our first beta release. This actually, raises a practical, question:
> what do we tag those?
> 
> Or to put it another way: can we all agree that 2.0.5 should be our
> first beta release?
> 
> Thanks,
> Roman.


signature.asc
Description: Digital signature

Re: [VOTE] Release Apache Hadoop 2.0.4-alpha

[Konstantin Boudnik]

-0

the release is missing HADOOP-9704 that has critical effect on downstream
projects e.g. build are affected. The issue has been raised for the first time
back in 4/10/13 http://is.gd/OGb3GG and never been even sneezed upon.

Cos

On Sat, Apr 13, 2013 at 03:26AM, Arun C Murthy wrote:
> Folks,
> 
> I've created a release candidate (RC2) for hadoop-2.0.4-alpha that I would 
> like to release.
> 
> The RC is available at: 
> http://people.apache.org/~acmurthy/hadoop-2.0.4-alpha-rc2/
> The RC tag in svn is here: 
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4-alpha-rc2
> 
> The maven artifacts are available via repository.apache.org.
> 
> Please try the release and vote; the vote will run for the usual 7 days.
> 
> thanks,
> Arun
> 
> 
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
> 
> 


signature.asc
Description: Digital signature

[jira] [Reopened] (HADOOP-9407) commons-daemon 1.0.3 dependency has bad group id causing build issues

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-9407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik reopened HADOOP-9407:



I think it needs to be added to branch-2.0.4-alpha as well before the release 
goes out.

> commons-daemon 1.0.3 dependency has bad group id causing build issues
> -
>
> Key: HADOOP-9407
> URL: https://issues.apache.org/jira/browse/HADOOP-9407
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Affects Versions: 2.0.3-alpha
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Fix For: 2.0.5-beta
>
> Attachments: HDFS-4497.patch
>
>
> The commons-daemon dependency of the hadoop-hdfs module has been at version 
> 1.0.3 for a while. However, 1.0.3 has a pretty well-known groupId error in 
> its pom ("org.apache.commons" as opposed to "commons-daemon"). This problem 
> has since been corrected on commons-daemon starting 1.0.4.
> This causes build problems for many who depend on hadoop-hdfs directly and 
> indirectly, however. Maven can skip over this metadata inconsistency. But 
> other less forgiving build systems such as ivy and gradle have much harder 
> time working around this problem. For example, in gradle, pretty much the 
> only obvious way to work around this is to override this dependency version.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HADOOP-9405) TestGridmixSummary#testExecutionSummarizer is broken

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-9405.


Resolution: Fixed

I have committed the patch to 2.0.4-alpha branch

> TestGridmixSummary#testExecutionSummarizer is broken
> 
>
> Key: HADOOP-9405
> URL: https://issues.apache.org/jira/browse/HADOOP-9405
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: test, tools
>Affects Versions: 3.0.0, 2.0.4-alpha
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Minor
> Fix For: 3.0.0, 2.0.5-beta, 2.0.4-alpha
>
> Attachments: hdfs-4599-1.patch
>
>
> HADOOP-9252 changed how human readable numbers are printed, and required 
> updating a number of test cases. This one was missed because the Jenkins 
> precommit job apparently isn't running the tests in hadoop-tools.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Reopened] (HADOOP-9405) TestGridmixSummary#testExecutionSummarizer is broken

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik reopened HADOOP-9405:



Need to backport to branch-2.0.4

> TestGridmixSummary#testExecutionSummarizer is broken
> 
>
> Key: HADOOP-9405
> URL: https://issues.apache.org/jira/browse/HADOOP-9405
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: test, tools
>Affects Versions: 3.0.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Minor
> Fix For: 3.0.0, 2.0.5-beta
>
> Attachments: hdfs-4599-1.patch
>
>
> HADOOP-9252 changed how human readable numbers are printed, and required 
> updating a number of test cases. This one was missed because the Jenkins 
> precommit job apparently isn't running the tests in hadoop-tools.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [Vote] Merge branch-trunk-win to trunk

[Konstantin Boudnik]

Here's a daily build for hadoop-2 branch
  https://builds.apache.org/job/Hadoop-branch2

It just builds the full Hadoop project without running any tests (for now).
Can be easily extended to do test runs/artifact deployment, if needed.

Cos

On Mon, Mar 25, 2013 at 07:14PM, Konstantin Boudnik wrote:
> Yes, you are right of course - the mis-merged commit is the cause. Thanks for
> pointing this out!
> 
> I think it would be beneficial if we had branch-2 on going build in the
> Jenkins. I will go ahead and create one tonight.
> 
> Cos
> 
> On Mon, Mar 25, 2013 at 05:09PM, Suresh Srinivas wrote:
> > Adding other mailing lists I missed earlier.
> > 
> > Cos,
> > 
> > There is progress being made on that ticket. Also it has nothing to do with
> > that.
> > 
> > Please follow the discussion here and why this happened due to an invalid
> > commit that was reverted -
> > https://issues.apache.org/jira/browse/HDFS-4615?focusedCommentId=13612650&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13612650
> > 
> > Regards,
> > Suresh
> > 
> > 
> > On Mon, Mar 25, 2013 at 1:17 PM, Konstantin Boudnik  wrote:
> > 
> > > It doesn't look like any progress has been done on the ticket below in the
> > > last 3 weeks. And now branch-2 can't be compiled because of
> > >
> > >
> > > hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java:[895,15]
> > > WINDOWS is not public in org.apache.hadoop.fs.Path; cannot be accessed 
> > > from
> > > outside package
> > >
> > > That's exactly why I was -1'ing this...
> > >   Cos
> > >
> > > On Mon, Mar 04, 2013 at 05:41PM, Matt Foley wrote:
> > > > Thanks, gentlemen.  I've opened and taken responsibility for
> > > > https://issues.apache.org/jira/browse/HADOOP-9359.  Giri Kesavan has
> > > agreed
> > > > to help with the parts that require Jenkins admin access.
> > > >
> > > > Thanks,
> > > > --Matt
> > > >
> > > >
> > > >
> > > > On Mon, Mar 4, 2013 at 5:00 PM, Konstantin Shvachko <
> > > shv.had...@gmail.com>wrote:
> > > >
> > > > > +1 on the merge.
> > > > >
> > > > > I am glad we agreed.
> > > > > Having Jira to track the CI effort is a good idea.
> > > > >
> > > > > Thanks,
> > > > > --Konstantin
> > > > >
> > > > > On Mon, Mar 4, 2013 at 3:29 PM, Matt Foley 
> > > wrote:
> > > > > > Thanks.  I agree Windows -1's in test-patch should not block 
> > > > > > commits.
> > > > > >
> > > > > > --Matt
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Mon, Mar 4, 2013 at 2:30 PM, Konstantin Shvachko <
> > > > > shv.had...@gmail.com>
> > > > > > wrote:
> > > > > >>
> > > > > >> On Mon, Mar 4, 2013 at 12:22 PM, Matt Foley  > > >
> > > > > >> wrote:
> > > > > >> > Konstantine, you have voted -1, and stated some requirements
> > > before
> > > > > >> > you'll
> > > > > >> > withdraw that -1.  As I plan to do work to fulfill those
> > > > > requirements, I
> > > > > >> > want to make sure that what I'm proposing will, in fact, satisfy
> > > you.
> > > > > >> > That's why I'm asking, if we implement full "test-patch"
> > > integration
> > > > > for
> > > > > >> > Windows, does it seem to you that that would provide adequate
> > > support?
> > > > > >>
> > > > > >> Yes.
> > > > > >>
> > > > > >> > I have learned not to presume that my interpretation is correct.
> > >  My
> > > > > >> > interpretation of item #1 is that test-patch provides pre-commit
> > > > > build,
> > > > > >> > so
> > > > > >> > it would satisfy item #1.  But rather than assuming that I am
> > > > > >> > interpreting
> > > > > >> > it correctly, I simply want your agreement that it would, or if
> > > not,
> > > > > >> > clarification why it won&#

Re: [Vote] Merge branch-trunk-win to trunk

[Konstantin Boudnik]

Yes, you are right of course - the mis-merged commit is the cause. Thanks for
pointing this out!

I think it would be beneficial if we had branch-2 on going build in the
Jenkins. I will go ahead and create one tonight.

Cos

On Mon, Mar 25, 2013 at 05:09PM, Suresh Srinivas wrote:
> Adding other mailing lists I missed earlier.
> 
> Cos,
> 
> There is progress being made on that ticket. Also it has nothing to do with
> that.
> 
> Please follow the discussion here and why this happened due to an invalid
> commit that was reverted -
> https://issues.apache.org/jira/browse/HDFS-4615?focusedCommentId=13612650&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13612650
> 
> Regards,
> Suresh
> 
> 
> On Mon, Mar 25, 2013 at 1:17 PM, Konstantin Boudnik  wrote:
> 
> > It doesn't look like any progress has been done on the ticket below in the
> > last 3 weeks. And now branch-2 can't be compiled because of
> >
> >
> > hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java:[895,15]
> > WINDOWS is not public in org.apache.hadoop.fs.Path; cannot be accessed from
> > outside package
> >
> > That's exactly why I was -1'ing this...
> >   Cos
> >
> > On Mon, Mar 04, 2013 at 05:41PM, Matt Foley wrote:
> > > Thanks, gentlemen.  I've opened and taken responsibility for
> > > https://issues.apache.org/jira/browse/HADOOP-9359.  Giri Kesavan has
> > agreed
> > > to help with the parts that require Jenkins admin access.
> > >
> > > Thanks,
> > > --Matt
> > >
> > >
> > >
> > > On Mon, Mar 4, 2013 at 5:00 PM, Konstantin Shvachko <
> > shv.had...@gmail.com>wrote:
> > >
> > > > +1 on the merge.
> > > >
> > > > I am glad we agreed.
> > > > Having Jira to track the CI effort is a good idea.
> > > >
> > > > Thanks,
> > > > --Konstantin
> > > >
> > > > On Mon, Mar 4, 2013 at 3:29 PM, Matt Foley 
> > wrote:
> > > > > Thanks.  I agree Windows -1's in test-patch should not block commits.
> > > > >
> > > > > --Matt
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Mar 4, 2013 at 2:30 PM, Konstantin Shvachko <
> > > > shv.had...@gmail.com>
> > > > > wrote:
> > > > >>
> > > > >> On Mon, Mar 4, 2013 at 12:22 PM, Matt Foley  > >
> > > > >> wrote:
> > > > >> > Konstantine, you have voted -1, and stated some requirements
> > before
> > > > >> > you'll
> > > > >> > withdraw that -1.  As I plan to do work to fulfill those
> > > > requirements, I
> > > > >> > want to make sure that what I'm proposing will, in fact, satisfy
> > you.
> > > > >> > That's why I'm asking, if we implement full "test-patch"
> > integration
> > > > for
> > > > >> > Windows, does it seem to you that that would provide adequate
> > support?
> > > > >>
> > > > >> Yes.
> > > > >>
> > > > >> > I have learned not to presume that my interpretation is correct.
> >  My
> > > > >> > interpretation of item #1 is that test-patch provides pre-commit
> > > > build,
> > > > >> > so
> > > > >> > it would satisfy item #1.  But rather than assuming that I am
> > > > >> > interpreting
> > > > >> > it correctly, I simply want your agreement that it would, or if
> > not,
> > > > >> > clarification why it won't.
> > > > >>
> > > > >> I agree it will satisfy my item #1.
> > > > >> I did not agree in my previous email, but I changed my mind based on
> > > > >> the latest discussion. I have to explain why now.
> > > > >> I was proposing nightly build because I did not want pre-commit
> > build
> > > > >> for Windows block commits to Linux. But if people are fine just
> > ignoring
> > > > >> -1s for the Windows part of the build it should be good.
> > > > >>
> > > > >> > Regarding item #2, it is also my interpretation that test-patch
> > > > provides
> > > > >> > an
> > > > >> > on-demand (perh

Re: [Vote] Merge branch-trunk-win to trunk

[Konstantin Boudnik]

It doesn't look like any progress has been done on the ticket below in the
last 3 weeks. And now branch-2 can't be compiled because of 

hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java:[895,15]
WINDOWS is not public in org.apache.hadoop.fs.Path; cannot be accessed from
outside package

That's exactly why I was -1'ing this...
  Cos

On Mon, Mar 04, 2013 at 05:41PM, Matt Foley wrote:
> Thanks, gentlemen.  I've opened and taken responsibility for
> https://issues.apache.org/jira/browse/HADOOP-9359.  Giri Kesavan has agreed
> to help with the parts that require Jenkins admin access.
> 
> Thanks,
> --Matt
> 
> 
> 
> On Mon, Mar 4, 2013 at 5:00 PM, Konstantin Shvachko 
> wrote:
> 
> > +1 on the merge.
> >
> > I am glad we agreed.
> > Having Jira to track the CI effort is a good idea.
> >
> > Thanks,
> > --Konstantin
> >
> > On Mon, Mar 4, 2013 at 3:29 PM, Matt Foley  wrote:
> > > Thanks.  I agree Windows -1's in test-patch should not block commits.
> > >
> > > --Matt
> > >
> > >
> > >
> > > On Mon, Mar 4, 2013 at 2:30 PM, Konstantin Shvachko <
> > shv.had...@gmail.com>
> > > wrote:
> > >>
> > >> On Mon, Mar 4, 2013 at 12:22 PM, Matt Foley 
> > >> wrote:
> > >> > Konstantine, you have voted -1, and stated some requirements before
> > >> > you'll
> > >> > withdraw that -1.  As I plan to do work to fulfill those
> > requirements, I
> > >> > want to make sure that what I'm proposing will, in fact, satisfy you.
> > >> > That's why I'm asking, if we implement full "test-patch" integration
> > for
> > >> > Windows, does it seem to you that that would provide adequate support?
> > >>
> > >> Yes.
> > >>
> > >> > I have learned not to presume that my interpretation is correct.  My
> > >> > interpretation of item #1 is that test-patch provides pre-commit
> > build,
> > >> > so
> > >> > it would satisfy item #1.  But rather than assuming that I am
> > >> > interpreting
> > >> > it correctly, I simply want your agreement that it would, or if not,
> > >> > clarification why it won't.
> > >>
> > >> I agree it will satisfy my item #1.
> > >> I did not agree in my previous email, but I changed my mind based on
> > >> the latest discussion. I have to explain why now.
> > >> I was proposing nightly build because I did not want pre-commit build
> > >> for Windows block commits to Linux. But if people are fine just ignoring
> > >> -1s for the Windows part of the build it should be good.
> > >>
> > >> > Regarding item #2, it is also my interpretation that test-patch
> > provides
> > >> > an
> > >> > on-demand (perhaps 20-minutes deferred) Jenkins build and unit test,
> > >> > with
> > >> > logs available to the developer, so it would satisfy item #2.  But
> > >> > rather
> > >> > than assuming that I am interpreting it correctly, I simply want your
> > >> > agreement that it would, or if not, clarification why it won't.
> > >>
> > >> It will satisfy my item #2 in the following way:
> > >> I can duplicate your pre-commit build for Windows and add an input
> > >> parameter, which would let people run the build on their patches
> > >> chosen from local machine rather than attaching them to Jiras.
> > >>
> > >> Thanks,
> > >> --Konstantin
> > >>
> > >> > In agile terms, you are the Owner of these requirements.  Please give
> > me
> > >> > owner feedback as to whether my proposed work sounds like it will
> > >> > satisfy
> > >> > the requirements.
> > >> >
> > >> > Thank you,
> > >> > --Matt
> > >> >
> > >> >
> > >> > On Sun, Mar 3, 2013 at 12:16 PM, Konstantin Shvachko
> > >> > 
> > >> > wrote:
> > >> >>
> > >> >> Didn't I explain in details what I am asking for?
> > >> >>
> > >> >> Thanks,
> > >> >> --Konst
> > >> >>
> > >> >> On Sun, Mar 3, 2013 at 11:08 AM, Matt Foley 
> > >> >> wrote:
> > >> >> > Hi Konstantin,
> > >> >> > I'd like to point out two things:
> > >> >> > First, I already committed in this thread (email of Thu, Feb 28,
> > 2013
> > >> >> > at
> > >> >> > 6:01 PM) to providing CI for Windows builds.  So please stop acting
> > >> >> > like
> > >> >> > I'm
> > >> >> > resisting this idea or something.
> > >> >> > Second, you didn't answer my question, you just kvetched about the
> > >> >> > phrasing.
> > >> >> > So I ask again:
> > >> >> >
> > >> >> > Will providing full "test-patch" integration (pre-commit build and
> > >> >> > unit
> > >> >> > test
> > >> >> > triggered by Jira "Patch Available" state) satisfy your request for
> > >> >> > functionality #1 and #2?  Yes or no, please.
> > >> >> >
> > >> >> > Thanks,
> > >> >> > --Matt
> > >> >> >
> > >> >> >
> > >> >> > On Sat, Mar 2, 2013 at 7:32 PM, Konstantin Shvachko
> > >> >> > 
> > >> >> > wrote:
> > >> >> >>
> > >> >> >> Hi Matt,
> > >> >> >>
> > >> >> >> On Sat, Mar 2, 2013 at 12:32 PM, Matt Foley <
> > mfo...@hortonworks.com>
> > >> >> >> wrote:
> > >> >> >> > Konstantin,
> > >> >> >> > I would like to explore what it would take to remove this
> > >> >> >> > perceived
> > >> >> >> > impediment --
> > >> >> >>
> > >> >> >> Glad

[jira] [Resolved] (HADOOP-9399) protoc maven plugin doesn't work on mvn 3.0.2

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-9399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-9399.


  Resolution: Fixed
Release Note: Committed to 2.0.4-alpha branch

> protoc maven plugin doesn't work on mvn 3.0.2
> -
>
> Key: HADOOP-9399
> URL: https://issues.apache.org/jira/browse/HADOOP-9399
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Affects Versions: 3.0.0
>Reporter: Todd Lipcon
>    Assignee: Konstantin Boudnik
>Priority: Minor
> Fix For: 3.0.0, 2.0.5-beta, 2.0.4-alpha
>
> Attachments: hadoop-9399.txt
>
>
> On my machine with mvn 3.0.2, I get a ClassCastException trying to use the 
> maven protoc plugin. The issue seems to be that mvn 3.0.2 sees the List 
> parameter, and doesn't see the generic type argument, and stuffs Strings 
> inside instead. So, we get ClassCastException trying to use the objects as 
> Files.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HADOOP-9299) kerberos name resolution is kicking in even when kerberos is not configured

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-9299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-9299.


   Resolution: Fixed
Fix Version/s: 2.0.4-alpha

Apparently it has been already merged into branch-2 earlier
r1457770 | daryn | 2013-03-18 07:05:09 -0700 (Mon, 18 Mar 2013) | 2 lines

svn merge -c 1457763 FIXES: HADOOP-9299.  kerberos name resolution is kicking 
in even when kerberos is not configured (daryn)

Thanks Daryn! The ticket hasn't been marked as such.


> kerberos name resolution is kicking in even when kerberos is not configured
> ---
>
> Key: HADOOP-9299
> URL: https://issues.apache.org/jira/browse/HADOOP-9299
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.0.3-alpha
>Reporter: Roman Shaposhnik
>Assignee: Daryn Sharp
>Priority: Blocker
> Fix For: 3.0.0, 2.0.5-beta, 2.0.4-alpha
>
> Attachments: HADOOP-9299-branch2.0.4.patch, HADOOP-9299.patch, 
> HADOOP-9299.patch
>
>
> Here's what I'm observing on a fully distributed cluster deployed via Bigtop 
> from the RC0 2.0.3-alpha tarball:
> {noformat}
> 528077-oozie-tucu-W@mr-node] Error starting action [mr-node]. ErrorType 
> [TRANSIENT], ErrorCode [JA009], Message [JA009: 
> org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: 
> No rules applied to yarn/localhost@LOCALREALM
> at 
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier.(AbstractDelegationTokenIdentifier.java:68)
> at 
> org.apache.hadoop.mapreduce.v2.api.MRDelegationTokenIdentifier.(MRDelegationTokenIdentifier.java:51)
> at 
> org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$HSClientProtocolHandler.getDelegationToken(HistoryClientService.java:336)
> at 
> org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getDelegationToken(MRClientProtocolPBServiceImpl.java:210)
> at 
> org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:240)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:454)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1735)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1731)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1729)
> Caused by: 
> org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: 
> No rules applied to yarn/localhost@LOCALREALM
> at 
> org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:378)
> at 
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier.(AbstractDelegationTokenIdentifier.java:66)
> ... 12 more
> ]
> {noformat}
> This is submitting a mapreduce job via Oozie 3.3.1. The reason I think this 
> is a Hadoop issue rather than the oozie one is because when I hack 
> /etc/krb5.conf to be:
> {noformat}
> [libdefaults]
>ticket_lifetime = 600
>default_realm = LOCALHOST
>default_tkt_enctypes = des3-hmac-sha1 des-cbc-crc
>default_tgs_enctypes = des3-hmac-sha1 des-cbc-crc
> [realms]
>LOCALHOST = {
>kdc = localhost:88
>default_domain = .local
>}
> [domain_realm]
>.local = LOCALHOST
> [logging]
>kdc = FILE:/var/log/krb5kdc.log
>admin_server = FILE:/var/log/kadmin.log
>default = FILE:/var/log/krb5lib.log
> {noformat}
> The issue goes away. 
> Now, once again -- the kerberos auth is NOT configured for Hadoop, hence it 
> should NOT pay attention to /etc/krb5.conf to begin with.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Reopened] (HADOOP-9299) kerberos name resolution is kicking in even when kerberos is not configured

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-9299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik reopened HADOOP-9299:



Backporting to 2.0.4-alpha

> kerberos name resolution is kicking in even when kerberos is not configured
> ---
>
> Key: HADOOP-9299
> URL: https://issues.apache.org/jira/browse/HADOOP-9299
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.0.3-alpha
>Reporter: Roman Shaposhnik
>Assignee: Daryn Sharp
>Priority: Blocker
> Fix For: 3.0.0, 2.0.5-beta
>
> Attachments: HADOOP-9299-branch2.0.4.patch, HADOOP-9299.patch, 
> HADOOP-9299.patch
>
>
> Here's what I'm observing on a fully distributed cluster deployed via Bigtop 
> from the RC0 2.0.3-alpha tarball:
> {noformat}
> 528077-oozie-tucu-W@mr-node] Error starting action [mr-node]. ErrorType 
> [TRANSIENT], ErrorCode [JA009], Message [JA009: 
> org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: 
> No rules applied to yarn/localhost@LOCALREALM
> at 
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier.(AbstractDelegationTokenIdentifier.java:68)
> at 
> org.apache.hadoop.mapreduce.v2.api.MRDelegationTokenIdentifier.(MRDelegationTokenIdentifier.java:51)
> at 
> org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$HSClientProtocolHandler.getDelegationToken(HistoryClientService.java:336)
> at 
> org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getDelegationToken(MRClientProtocolPBServiceImpl.java:210)
> at 
> org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:240)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:454)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1735)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1731)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1729)
> Caused by: 
> org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: 
> No rules applied to yarn/localhost@LOCALREALM
> at 
> org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:378)
> at 
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier.(AbstractDelegationTokenIdentifier.java:66)
> ... 12 more
> ]
> {noformat}
> This is submitting a mapreduce job via Oozie 3.3.1. The reason I think this 
> is a Hadoop issue rather than the oozie one is because when I hack 
> /etc/krb5.conf to be:
> {noformat}
> [libdefaults]
>ticket_lifetime = 600
>default_realm = LOCALHOST
>default_tkt_enctypes = des3-hmac-sha1 des-cbc-crc
>default_tgs_enctypes = des3-hmac-sha1 des-cbc-crc
> [realms]
>LOCALHOST = {
>kdc = localhost:88
>default_domain = .local
>}
> [domain_realm]
>.local = LOCALHOST
> [logging]
>kdc = FILE:/var/log/krb5kdc.log
>admin_server = FILE:/var/log/kadmin.log
>default = FILE:/var/log/krb5lib.log
> {noformat}
> The issue goes away. 
> Now, once again -- the kerberos auth is NOT configured for Hadoop, hence it 
> should NOT pay attention to /etc/krb5.conf to begin with.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [Vote] Merge branch-trunk-win to trunk

[Konstantin Boudnik]

Ok, looks like we are converging on this across a few hundred emails ;)

So, as has been stated elsewhere: test-patch will be improved to fully support
Windows; furthermore -1 from Windows' test-patch won't block Linux commits.
This is ok with me.

Can we have a JIRA ticket for that test-patch work assigned to the real owner,
so it can be tracked? I am +1 in this case.

Cos

On Fri, Mar 01, 2013 at 04:57PM, Chris Douglas wrote:
> On Fri, Mar 1, 2013 at 1:57 PM, Konstantin Shvachko
>  wrote:
> > Commitment is a good thing.
> > I think the two builds that I proposed are a prerequisite for Win support.
> > If we commit windows patch people will start breaking it the next day.
> > Which we wont know without the nightly build and wont be able to fix
> > without the on-demand one.
> 
> As several people have pointed out already, the surface of possible
> conflicts is relatively limited, and- as you did in 2007- the devs on
> Windows will report and fix bugs in that platform as they find them.
> CI is important for detecting and preventing bugs, but this isn't
> software we're launching into orbit.
> 
> > Making two builds is less than 2 days work, imho, given that there is
> > a Windows node available and that mvn targets are in place. Correct me
> > if I missed any complications in the process.
> 
> On Fri, Mar 1, 2013 at 3:47 PM, Konstantin Boudnik  wrote:
> > It seems that with the HW in place, the matter of setting at least nightly
> > build is trivial for anyone with up to date Windows knowledge. I wish I 
> > could
> > help. Going without a validation is a recipe for a disaster IMO.
> 
> Fair enough, though that also implies that the window for regressions
> is small, and it leaves little room to doubt that this will receive
> priority. Until it's merged, spurious notifications that the current
> trunk breaks Windows are an awkward introduction to devs' workflow.
> The order of merge/CI is a choice between mild annoyances, really.
> 
> But it might be moot. Giri: you're doing the work on this. When do you
> think it can be complete? -C

Re: [Vote] Merge branch-trunk-win to trunk

[Konstantin Boudnik]

Suresh, I appreciate all the troubles you're going through wrt syncing up the
huge patch for a long time - I really do.

I am not asking to have full test-patch process in place. But I think it is a
real good idea to have a way to run the full test suite once in a while - or
as Konstantin proposing - for a given patch, to make sure that codebase
doesn't bitrot at the edges.

"Official" support has nothing to do with the issue - you're trying to build a
straw man argument around this. What is relevant, on the other hand, is that
Windows is so different from _any_ Unix or pseudo-Unix flavors, including
Windows with Cygwin - that even multi-platform Java has hard hard time
dealing with it. This is enough, IMO, to warrant a separate checkpoint.

I hope I have explained myself better.
  Cos

On Fri, Mar 01, 2013 at 05:55PM, Suresh Srinivas wrote:
> > It seems that with the HW in place, the matter of setting at least nightly
> > build is trivial for anyone with up to date Windows knowledge. I wish I
> > could
> > help. Going without a validation is a recipe for a disaster IMO.
> >
> > -1 until some reasonable solution is implemented.
> >   Cos
> 
> 
> Cos, I have hard time understanding your veto?
> 
> Here is my rationale for merge:
> Currently with all the cross platform support, the merge patch has +1 from
> Jenkins on Linux. So no regression has been introduced in Hadoop on Linux.
> 
> As regards to Windows support I want to make two points:
> 1. Until Jenkins is setup and we are passing all the tests, I am okay, as
> Konstntin proposed, if we do not officially declare Windows as supported. I
> do not want to tie the patch merge with setting up Windows Jenkins. I have
> been maintaining this branch for a long time and keeping it in sync with
> trunk is non-trivial.
> 2. After Jenkins is setup, there is a concern in the community about -1
> from Windows hindering patch commits. As others have already suggested in
> the thread, I am okay committing new patches even if -1 is posted by
> Jenkins on Windows. The team that worked on Windows will fix the issues. I
> do not see many such issues cropping up.


signature.asc
Description: Digital signature

Re: [Vote] Merge branch-trunk-win to trunk

[Konstantin Boudnik]

It seems that with the HW in place, the matter of setting at least nightly
build is trivial for anyone with up to date Windows knowledge. I wish I could
help. Going without a validation is a recipe for a disaster IMO.

-1 until some reasonable solution is implemented.
  Cos

On Fri, Mar 01, 2013 at 01:57PM, Konstantin Shvachko wrote:
> Commitment is a good thing.
> I think the two builds that I proposed are a prerequisite for Win support.
> If we commit windows patch people will start breaking it the next day.
> Which we wont know without the nightly build and wont be able to fix
> without the on-demand one.
> 
> Making two builds is less than 2 days work, imho, given that there is
> a Windows node available and that mvn targets are in place. Correct me
> if I missed any complications in the process.
> 
> Thanks,
> --Konst
> 
> On Fri, Mar 1, 2013 at 1:28 PM, Chris Douglas  wrote:
> > Konstantin-
> >
> > There's no debate on the necessity of CI and related infrastructure to
> > support the platform well. Suresh outlined the support to effect this
> > here: http://s.apache.org/s1
> >
> > Is the commitment to establish this infrastructure after the merge
> > sufficient? -C
> >
> > On Fri, Mar 1, 2013 at 12:18 PM, Konstantin Shvachko
> >  wrote:
> >> -1
> >> We should have a CI infrastructure in place before we can commit to
> >> supporting Windows platform.
> >>
> >> Eric is right Win/Cygwin was supported since day one.
> >> I had a Windows box under my desk running nightly builds back in 2006-07.
> >> People were irritated but I was filing windows bugs until 0.22 release.
> >> Times changing and I am glad to see wider support for Win platform.
> >>
> >> But in order to make it work you guys need to put the CI process in place
> >>
> >> 1. windows jenkins build: could be nightly or PreCommit.
> >> - Nightly would mean that changes can be committed to trunk based on
> >> linux PreCommit build. And people will file bugs if the change broke
> >> Windows nightly build.
> >> - PreCommit-win build will mean automatic reporting failed tests to
> >> respective jira blocking commits the same way as it is now with linux
> >> PreCommit builds.
> >> We should discuss which way is more efficient for developers.
> >>
> >> 2. On-demand-windows Jenkins build.
> >> I see it as a build to which I can attach my patch and the build will
> >> run my changes on a dedicated windows box.
> >> That way people can test their changes without having personal windows 
> >> nodes.
> >>
> >> I think this is the minimal set of requirement for us to be able to
> >> commit to the new platform.
> >> Right now I see only one windows related build
> >> https://builds.apache.org/view/Hadoop/job/Hadoop-1-win/
> >> Which was failing since Sept 8, 2012 and did not run in the last month.
> >>
> >> Thanks,
> >> --Konst
> >>
> >> On Thu, Feb 28, 2013 at 8:47 PM, Eric Baldeschwieler
> >>  wrote:
> >>> +1 (non-binding)
> >>>
> >>> A few of observations:
> >>>
> >>> - Windows has actually been a supported platform for Hadoop since 0.1 .  
> >>> Doug championed supporting windows then and we've continued to do it with 
> >>> varying vigor over time.  To my knowledge we've never made a decision to 
> >>> drop windows support.  The change here is improving our support and 
> >>> dropping the requirement of cigwin.  We had Nutch windows users on the 
> >>> list in 2006 and we've been supporting windows FS requirements since 
> >>> inception.
> >>>
> >>> - A little pragmatism will go a long way.  As a community we've got to 
> >>> stay committed to keeping hadoop simple (so it does work on many 
> >>> platforms) and extending it to take advantage of key emerging OS/hardware 
> >>> features, such as containers, new FSs, virtualization, flash ...  We 
> >>> should all plan to let new features & optimizations emerge that don't 
> >>> work everywhere, if they are compelling and central to hadoop's mission 
> >>> of being THE best fabric for storing and processing big data.
> >>>
> >>> - A UI project like KDE has to deal with the MANY differences between 
> >>> windows and linux UI APIs.  Hadoop faces no such complex challenge and 
> >>> hence can be maintained from a single codeline IMO.  It is mostly 
> >>> abstracted from the OS APIs via Java and our design choices.  Where it is 
> >>> not we can continue to add plugable abstractions.
> >>>

Re: [Vote] Merge branch-trunk-win to trunk

[Konstantin Boudnik]

On Thu, Feb 28, 2013 at 03:08PM, sanjay Radia wrote:
> +1
> Java has done the bulk of the work in making Hadoop multi-platform.
> Windows specific code is a tiny percentage of the code.
> Jeninks support for windows is going help us keep the platform portable going 
> forward.
> I expect that the vast majority of new commits have  no problems. I propose
> that we start by fixing problems that Jenkins raises but not block new
> commits for too long if the author does not have a windows box or if a
> volunteer does not step up.

Considering a typical set of software most of the people here work with it
would be completely inappropriate to block commits for failing Windows
specific features. After all, Microsoft never did bother to check what
features or compatibilty matters they have broke in Java and elsewhere, so why
should we?

I believe this kind of rules have to be set and discussed before the merge is
done.

Cheers,
  Cos


signature.asc
Description: Digital signature

Re: ANNOUNCEMENT: Project Rhino: Enhanced Data Protection for the Apache Hadoop Ecosystem

[Konstantin Boudnik]

[yanking away most of the cross-posts...]

An interesting cross component project Avik. Any plans to incubate it in Apache?

Cos

On Mon, Feb 25, 2013 at 11:46PM, Dey, Avik wrote:
> Project Rhino
> 
> As the Apache Hadoop ecosystem extends into new markets and sees new use
> cases with security and compliance challenges, the benefits of processing
> sensitive and legally protected data with Hadoop must be coupled with
> protection for private information that limits performance impact. Project
> Rhino is our open source
> effort to enhance the existing data protection capabilities of the Hadoop
> ecosystem to address these challenges, and contribute the code back to
> Apache.
> 
> The core of the Apache Hadoop ecosystem as it is commonly understood is:
> 
> - Core: A set of shared libraries
> - HDFS: The Hadoop filesystem
> - MapReduce: Parallel computation framework
> - ZooKeeper: Configuration management and coordination
> - HBase: Column-oriented database on HDFS
> - Hive: Data warehouse on HDFS with SQL-like access
> - Pig: Higher-level programming language for Hadoop computations
> - Oozie: Orchestration and workflow management
> - Mahout: A library of machine learning and data mining algorithms
> - Flume: Collection and import of log and event data
> - Sqoop: Imports data from relational databases
> 
> These components are all separate projects and therefore cross cutting 
> concerns like authN, authZ, a consistent security policy framework, 
> consistent authorization model and audit coverage are loosely coordinated. 
> Some security features expected by our customers, such as encryption, are 
> simply missing. Our aim is to take a full stack view and work with the 
> individual projects toward consistent concepts and capabilities, filling gaps 
> as we go.
> 
> Our initial goals are:
> 
> 1) Framework support for encryption and key management
> 
> There is currently no framework support for encryption or key management. We 
> will add this support into Hadoop Core and integrate it across the ecosystem.
> 
> 2) A common authorization framework for the Hadoop ecosystem
> 
> Each component currently has its own authorization engine. We will abstract 
> the common functions into a reusable authorization framework with a 
> consistent interface. Where appropriate we will either modify an existing 
> engine to work within this framework, or we will plug in a common default 
> engine. Therefore we also must normalize how security policy is expressed and 
> applied by each component. Core, HDFS, ZooKeeper, and HBase currently support 
> simple access control lists (ACLs) composed of users and groups. We see this 
> as a good starting point. Where necessary we will modify components so they 
> each offer equivalent functionality, and build support into others.
> 
> 3) Token based authentication and single sign on
> 
> Core, HDFS, ZooKeeper, and HBase currently support Kerberos authentication at 
> the RPC layer, via SASL. However this does not provide valuable attributes 
> such as group membership, classification level, organizational identity, or 
> support for user defined attributes. Hadoop components must interrogate 
> external resources for discovering these attributes and at scale this is 
> problematic. There is also no consistent delegation model. HDFS has a simple 
> delegation capability, and only Oozie can take limited advantage of it. We 
> will implement a common token based authentication framework to decouple 
> internal user and service authentication from external mechanisms used to 
> support it (like Kerberos).
> 
> 4) Extend HBase support for ACLs to the cell level
> 
> Currently HBase supports setting access controls at the table or column 
> family level. However, many use cases would benefit from the additional 
> capability to do this on a per cell basis. In fact for many users dealing 
> with sensitive information the ability to do this is crucial.
> 
> 5) Improve audit logging
> 
> Audit messages from various Hadoop components do not use a unified or even 
> consistently formatted format. This makes analysis of logs for verifying 
> compliance or taking corrective action difficult. We will build a common 
> audit logging facility as part of the common authorization framework work. We 
> will also build a set of common audit log processing tools for transforming 
> them to different industry standard formats, for supporting compliance 
> verification, and for triggering responses to policy violations.
> 
> Current JIRAs:
> 
> As part of this ongoing effort we are contributing our work to-date against 
> the JIRAs listed below. As you may appreciate, the goals for Project Rhino 
> covers a number of different Apache projects, the scope of work is 
> significant and likely to only increase as we get additional community input. 
> We also appreciate that there may be others in the Apache community that may 
> be working on some of this 

Re: [VOTE] Release hadoop-2.0.3-alpha

[Konstantin Boudnik]

On Tue, Feb 12, 2013 at 07:44PM, Konstantin Boudnik wrote:
> We've created BigTop stack with 2.0.3 as the base. Ran YCSB, slive, and some
> other loads on up to 20 nodes clusters as a part of the release validation.
> Two issues were noted:
> 
> - due to the known issue with Jetty we are seeing MR jobs hanging here and
>   there

The issue referred here is reported as MAPREDUCE-2980

> - without properly configured capacity-scheduler.xml resource manager throws
>   exception on the startup (BIGTOP-841).
> 
> +1
> 
> Cos
> 
> On Wed, Feb 06, 2013 at 07:59PM, Arun C Murthy wrote:
> > Folks,
> > 
> > I've created a release candidate (rc0) for hadoop-2.0.3-alpha that I would 
> > like to release.
> > 
> > This release contains several major enhancements such as QJM for HDFS HA,
> > multi-resource scheduling for YARN, YARN ResourceManager restart etc.  Also
> > YARN has achieved significant stability at scale (more details from Y! folks
> > here: http://s.apache.org/VYO).
> > 
> > The RC is available at: 
> > http://people.apache.org/~acmurthy/hadoop-2.0.3-alpha-rc0/
> > The RC tag in svn is here: 
> > http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.3-alpha-rc0/
> > 
> > The maven artifacts are available via repository.apache.org.
> > 
> > Please try the release and vote; the vote will run for the usual 7 days.
> > 
> > thanks,
> > Arun
> > 
> > 
> > 
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> > 
> > 




signature.asc
Description: Digital signature

Re: [VOTE] Release hadoop-2.0.3-alpha

[Konstantin Boudnik]

We've created BigTop stack with 2.0.3 as the base. Ran YCSB, slive, and some
other loads on up to 20 nodes clusters as a part of the release validation.
Two issues were noted:

- due to the known issue with Jetty we are seeing MR jobs hanging here and
  there
- without properly configured capacity-scheduler.xml resource manager throws
  exception on the startup (BIGTOP-841).

+1

Cos

On Wed, Feb 06, 2013 at 07:59PM, Arun C Murthy wrote:
> Folks,
> 
> I've created a release candidate (rc0) for hadoop-2.0.3-alpha that I would 
> like to release.
> 
> This release contains several major enhancements such as QJM for HDFS HA,
> multi-resource scheduling for YARN, YARN ResourceManager restart etc.  Also
> YARN has achieved significant stability at scale (more details from Y! folks
> here: http://s.apache.org/VYO).
> 
> The RC is available at: 
> http://people.apache.org/~acmurthy/hadoop-2.0.3-alpha-rc0/
> The RC tag in svn is here: 
> http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.3-alpha-rc0/
> 
> The maven artifacts are available via repository.apache.org.
> 
> Please try the release and vote; the vote will run for the usual 7 days.
> 
> thanks,
> Arun
> 
> 
> 
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
> 
> 


signature.asc
Description: Digital signature

Re: [VOTE] Release hadoop-2.0.3-alpha

[Konstantin Boudnik]

The issue with the configuration is raised (and adressed) in 
  https://issues.apache.org/jira/browse/BIGTOP-841

Cos

On Fri, Feb 08, 2013 at 04:25PM, Aaron T. Myers wrote:
> +1 (binding)
> 
> I downloaded the src tar ball, built it with the native bits enabled,
> started up a little cluster, and ran some sample jobs. Things worked as
> expected. I also verified the signatures on the source artifact.
> 
> I did bump into one little issue, but I don't think it should be considered
> a blocker. When I first tried to start up the RM, it failed to start with
> this error:
> 
> 13/02/08 16:00:31 FATAL resourcemanager.ResourceManager: Error starting
> ResourceManager
> java.lang.IllegalStateException: Queue configuration missing child queue
> names for root
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.parseQueue(CapacityScheduler.java:328)
>  at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initializeQueues(CapacityScheduler.java:255)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:220)
>  at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.init(ResourceManager.java:226)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:710)
> 
> And then this on shutdown:
> 
> 13/02/08 16:00:31 INFO service.CompositeService: Error stopping
> ResourceManager
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.stop(ResourceManager.java:590)
>  at
> org.apache.hadoop.yarn.service.CompositeService$CompositeServiceShutdownHook.run(CompositeService.java:122)
> at
> org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
> 
> Presumably this is because I don't have the CapacityScheduler queues
> configured at all, and the default scheduler is now the CapacityScheduler.
> To work around this for my testing, I switched to the FairScheduler and the
> RM came up just fine.
> 
> 
> --
> Aaron T. Myers
> Software Engineer, Cloudera
> 
> 
> On Wed, Feb 6, 2013 at 7:59 PM, Arun C Murthy  wrote:
> 
> > Folks,
> >
> > I've created a release candidate (rc0) for hadoop-2.0.3-alpha that I would
> > like to release.
> >
> > This release contains several major enhancements such as QJM for HDFS HA,
> > multi-resource scheduling for YARN, YARN ResourceManager restart etc.
> > Also YARN has achieved significant stability at scale (more details from
> > Y! folks here: http://s.apache.org/VYO).
> >
> > The RC is available at:
> > http://people.apache.org/~acmurthy/hadoop-2.0.3-alpha-rc0/
> > The RC tag in svn is here:
> > http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.3-alpha-rc0/
> >
> > The maven artifacts are available via repository.apache.org.
> >
> > Please try the release and vote; the vote will run for the usual 7 days.
> >
> > thanks,
> > Arun
> >
> >
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
> >


signature.asc
Description: Digital signature

[jira] [Created] (HADOOP-9231) Parametrize staging URL for the uniformity of distributionManagement

[Konstantin Boudnik (JIRA)]

Konstantin Boudnik created HADOOP-9231:
--

 Summary: Parametrize staging URL for the uniformity of 
distributionManagement
 Key: HADOOP-9231
 URL: https://issues.apache.org/jira/browse/HADOOP-9231
 Project: Hadoop Common
  Issue Type: Bug
  Components: build
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Konstantin Boudnik
Assignee: Konstantin Boudnik


The build's {{distributionManagement}} section currently uses parametrization 
for the snapshot repository. It is convenient and allows to override the value 
from a developer's custom profile.

The same isn't available for release artifacts to make the parametrization 
symmetric for both types.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Hadoop build slaves software

[Konstantin Boudnik]

Do I hear puppet? :)

Cos

On Fri, Jan 04, 2013 at 11:08AM, Todd Lipcon wrote:
> I agree -- I'd like to see us have a shell script of some sort which,
> given a prefix, downloads and installs the needed toolchain
> dependencies.
> 
> We could then download that script onto the build machines and install
> into something like /opt/hadoop-toolchain/
> AFAIK the only real dependencies we have where the Ubuntu packages are
> too old are protoc and maven, so shouldnt be too tough.
> 
> -Todd
> 
> On Fri, Jan 4, 2013 at 10:59 AM, Rajiv Chittajallu  
> wrote:
> > asf008 has been up for a while. It was probably just added as a slave.
> >
> > All the dependencies should probably be installed in a build_prefix, to
> > avoid conflict to OS specific packages and allows multiple projects to
> > build on the same machines. This is an better alternative to
> > provisioning vms for unique builds.
> >
> > -rajive
> >
> > Giridharan Kesavan wrote on 01/04/13 at 09:31:55 -0800:
> >>   Im on it
> >>
> >>   -Giri
> >>
> >>   On Thu, Jan 3, 2013 at 11:24 PM, Todd Lipcon <[1]t...@cloudera.com> 
> >> wrote:
> >>
> >> Hey folks,
> >>
> >> It looks like hadoop8 has recently come back online as a build slave,
> >> but is failing all the builds because it has an ancient version of
> >> protobuf (2.2.0):
> >> todd@asf008:~$ protoc  --version
> >> libprotoc 2.2.0
> >>
> >> In contrast, other slaves have 2.4.1:
> >> todd@asf001:~$ protoc --version
> >> libprotoc 2.4.1
> >>
> >> asf001 has the newer protoc in /usr/local/bin but asf008 does not.
> >> Does anyone know how software is meant to be deployed on these build
> >> slaves? I'm happy to download and install protobuf 2.4.1 into
> >> /usr/local on asf008 if manual installation is the name of the game,
> >> but it seems like we should be doing something a little more
> >> reproducible than one-off builds by rando developers to manage our
> >> toolchain on the Jenkins slaves.
> >> -Todd
> >> --
> >> Todd Lipcon
> >> Software Engineer, Cloudera
> >>
> >>References
> >>
> >>   Visible links
> >>   1. mailto:t...@cloudera.com
> 
> 
> 
> -- 
> Todd Lipcon
> Software Engineer, Cloudera

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

[Konstantin Boudnik]

On Sat, Dec 01, 2012 at 10:07PM, Eric Yang wrote:
> -1, +1, -1
> 
> Python has fairly inconsistent support across all major OS vendors.  It is
> hard to get it right unless the scripts are all designed to make use of
> Python 2.4.  However, Python 2.4 doesn't have necessary OS features to make
> Python useful in runtime or build environment unless you write a lot of
> custom modules.  Which defeats the purpose to use python as intermediate
> layer to do OS dependent work.  Jruby may be a better choice.

JRuby? Really? Groovy is already there and it is really a Java dialect unlike
JRuby. And yes - it is quite suitable for build things, considering the use of
it in BigTop

Cos

> On Sat, Dec 1, 2012 at 12:28 PM, Joep Rottinghuis 
> wrote:
> 
> > 0, 0, -1 (non-binding)
> >
> > Joep
> >
> > On Nov 24, 2012, at 12:13 PM, Matt Foley  wrote:
> >
> > > For discussion, please see previous thread "[PROPOSAL] introduce Python
> > as
> > > build-time and run-time dependency for Hadoop and throughout Hadoop
> > stack".
> > >
> > > This vote consists of three separate items:
> > >
> > > 1. Contributors shall be allowed to use Python as a platform-independent
> > > scripting language for build-time tasks, and add Python as a build-time
> > > dependency.
> > > Please vote +1, 0, -1.
> > >
> > > 2. Contributors shall be encouraged to use Maven tasks in combination
> > with
> > > either plug-ins or Groovy scripts to do cross-platform build-time tasks,
> > > even under ant in Hadoop-1.
> > > Please vote +1, 0, -1.
> > >
> > > 3. Contributors shall be allowed to use Python as a platform-independent
> > > scripting language for run-time tasks, and add Python as a run-time
> > > dependency.
> > > Please vote +1, 0, -1.
> > >
> > > Note that voting -1 on #1 and +1 on #2 essentially REQUIRES contributors
> > to
> > > use Maven plug-ins or Groovy as the only means of cross-platform
> > build-time
> > > tasks, or to simply continue using platform-dependent scripts as is being
> > > done today.
> > >
> > > Vote closes at 12:30pm PST on Saturday 1 December.
> > > -
> > > Personally, my vote is +1, +1, +1.
> > > I think #2 is preferable to #1, but still has many unknowns in it, and
> > > until those are worked out I don't want to delay moving to cross-platform
> > > scripts for build-time tasks.
> > >
> > > Best regards,
> > > --Matt
> >

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

[Konstantin Boudnik]

On Sat, Dec 01, 2012 at 10:44AM, Steve Loughran wrote:
> On 1 December 2012 01:08, Eli Collins  wrote:
> 
> > -1, 0, -1
> >
> > IIUC the only platform we plan to add support for that we can't easily
> > support today (w/o an emulation layer like cygwin) is Windows, and it
> > seems like making the bash scripts simpler and having parallel bat
> > files is IMO a better approach.
> >
> >
> WinNT Bat/CMD files are the worst possible scripting language invented. At
> the very least, .py should be the language of choice there

Compare to the OS in question - it isn't _that_ bad ;)

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

[Konstantin Boudnik]

-1, +1, -1

Thanks

On Sat, Nov 24, 2012 at 12:13PM, Matt Foley wrote:
> For discussion, please see previous thread "[PROPOSAL] introduce Python as
> build-time and run-time dependency for Hadoop and throughout Hadoop stack".
> 
> This vote consists of three separate items:
> 
> 1. Contributors shall be allowed to use Python as a platform-independent
> scripting language for build-time tasks, and add Python as a build-time
> dependency.
> Please vote +1, 0, -1.
> 
> 2. Contributors shall be encouraged to use Maven tasks in combination with
> either plug-ins or Groovy scripts to do cross-platform build-time tasks,
> even under ant in Hadoop-1.
> Please vote +1, 0, -1.
> 
> 3. Contributors shall be allowed to use Python as a platform-independent
> scripting language for run-time tasks, and add Python as a run-time
> dependency.
> Please vote +1, 0, -1.
> 
> Note that voting -1 on #1 and +1 on #2 essentially REQUIRES contributors to
> use Maven plug-ins or Groovy as the only means of cross-platform build-time
> tasks, or to simply continue using platform-dependent scripts as is being
> done today.
> 
> Vote closes at 12:30pm PST on Saturday 1 December.
> -
> Personally, my vote is +1, +1, +1.
> I think #2 is preferable to #1, but still has many unknowns in it, and
> until those are worked out I don't want to delay moving to cross-platform
> scripts for build-time tasks.
> 
> Best regards,
> --Matt

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

[Konstantin Boudnik]

If we decide to go with Maven then there's no point to complicate the
picture with jython. This time I will keep the offensive about *yton to myself
;)

Cos

On Sat, Nov 24, 2012 at 10:26PM, Radim Kolar wrote:
> we have not discussed advantages of stand alone python vs
> jython-in-maven pom
> 
> http://code.google.com/p/jy-maven-plugin/
> 
> language is about same, and it does not needs to have installed,
> which is advantage on windows.

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

[Konstantin Boudnik]

On Thu, Nov 22, 2012 at 12:58AM, Radim Kolar wrote:
> I know that Python compatibility can be worked around. I used Python
> for few years and wrote about 70k LOC in it until it started to
> irritate me that every new version has incompatibilities such as 2.4
> vs 2.3 vs 2.5 and it makes maintaining and testing way harder then
> it should be. Its not just compatibility with missing library
> functions. sometimes even expression evaluated to different value
> under new version. This was similar to php 4 to php 5 migration.
> Today i have 3 versions of python installed because of software
> requirements.
> 
> For simple scripts it can probably work if you stick to some common subset.
> 
> Scripting via maven plugin has advantage that user do not needs to
> install anything, there is couple of languages available: scala,
> groovy, jelly, jruby. Maybe jython too.

pretty much all of the j* in JSR223 land is abomination of one sort or
another, actually :)

Cos

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

[Konstantin Boudnik]

Ditto...

On Wed, Nov 21, 2012 at 01:14PM, Matt Foley wrote:
> Cos,
> Please see in-line.
> 
> On Wed, Nov 21, 2012 at 12:00 PM, Konstantin Boudnik  wrote:
> 
> > I like Alejandro's idea about Maven for a few of reasons:
> >   - bringing in a scripting environment which is known for its
> >   inter-version idiosyncrasies just because Windows can't handle trivial
> >   shell scripting looks like an overkill to me
> 
> Excuse me?  Can we at least try not to belittle other people's platforms on
> a public Apache forum?  There's nothing trivial about implementing shell on
> Windows, as cygwin regrettably proved.

Belittle? Hardly ;) Because we all know very well why shell is so awkward to
implement on any Windows system.

> >   - relative to above, there's a chance that Python's pre-requisites used
> >   in Hadoop might get into a conflict with some other components in the
> >   stack.  This will be a nightmare for the integrator projects i.e. Bigtop
> 
> Said Bigtop project actually uses python, does it not?

It does, Matt. The main concern I have is at some point Hadoop's Python might
all of a sudden be of a different version than the one in BigTop. And all the
hell will break lose compatibility wise. What would be the solution then?

> >   - Maven is de-facto standard for Java stacks
> >
> 
> Sure -- except for when Ant was the de-facto standard for Java stacks.  And

Arguable. Yet beyond the point.

> let's remember what maven and ant are/were the de-facto standard for:
>  Doing builds.  Not scripting everything that needs scripting.

Arguable as well, due to the very definition of a build system.

> >   - Maven has built-in scripting language (Groovy) if some plugins aren't
> > sufficient for achieving whatever goals
> 
> Are you proposing Groovy as a better scripting language than Python?

I am proposing Groovy is a better language than Python. Because, in part, it
goes far beyond scripting. And doesn't have permanent runtime backward
compatibility issues. What was the last time JDK had backward compatibility
problems?

> > Addressing Matt's later point about non-Mavenized Hadoop-1 line: it uses
> > Maven
> > stuff suchs as deploy/install via custom ant tasks. Same approach would
> > work
> > for saveVersion.sh and others, I am sure.
> 
> Current ant scripts in Hadoop seem to use maven only for artifact
> management via the maven repository.  If I'm missing something, please
> point it out.  The ant build task currently calls out to saveVersion.sh.
> Having it call out to maven, which then calls out to a plug-in and/or a
> Groovy script, doesn't sound like an improvement to me.  And it's a way

At least it it guaranteed to work everywhere. And all we need in this case is
an extra jar file that can be pulled down through the same ivy/maven
dependency mechanism.

In case of Python you'd have to make sure that you're having the right version
of the interpreter and runtime. And you will have to do it manually or have an
extra requirement expressed via a system maintenance DSL.

> different use of maven than currently in the Hadoop-1 line, not a
> continuation of established practice.

The main point of my argument expressed in a lesser than 100 words: adding
Python that is inconsistent across different Linux distros and has a history
of backward incompatibilities (2.6 vs 2.5, 3.0 vs earlier, etc.) doesn't seem
to leverage the benefit of having a somewhat easier build in Windows.

Perhaps, we can do a more format benefit analysis by just comparing the
number of Hadoop installations on MS Win vs. Unix's.

Cos

> > On Wed, Nov 21, 2012 at 11:25AM, Alejandro Abdelnur wrote:
> > > Hey Matt,
> > >
> > > We already require java/mvn/protoc/cmake/forrest (forrest is hopefully on
> > > its way out with the move of docs to APT)
> > >
> > > Why not do a maven-plugin to do that?
> > >
> > > Colin already has something to simplify all the cmake calls from the
> > builds
> > > using a maven-plugin (https://issues.apache.org/jira/browse/HADOOP-8887)
> > >
> > > We could do the same with protoc, thus simplifying the POMs.
> > >
> > > The saveVersion.sh seems like another prime candidate for a maven plugin,
> > > and in this case it would not require external tools.
> > >
> > > Does this make sense?
> > >
> > > Thx
> > >
> > > On Wed, Nov 21, 2012 at 11:15 AM, Matt Foley  wrote:
> > >
> > > > This discussion started in
> > > > HADOOP-8924<https://issues.apache.org/jira/browse/HADOOP-8924>
> > > > , where it was proposed to 

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

[Konstantin Boudnik]

On Wed, Nov 21, 2012 at 09:46PM, Radim Kolar wrote:
> 
> >Why not do a maven-plugin to do that?
> maven plugins are difficult to maintain. its better to use inline
> scripts, with something like this:
> 
> http://docs.codehaus.org/display/GMAVEN/Home;jsessionid=E29093B96230BBB4461F02A1718A6B71

Exactly my point, thank you!

Cos



signature.asc
Description: Digital signature

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

[Konstantin Boudnik]

I like Alejandro's idea about Maven for a few of reasons:
  - bringing in a scripting environment which is known for its inter-version
idiosyncrasies just because Windows can't handle trivial shell scripting
looks like an overkill to me
  - relative to above, there's a chance that Python's pre-requisites used in
Hadoop might get into a conflict with some other components in the stack.
This will be a nightmare for the integrator projects i.e. Bigtop
  - Maven is de-facto standard for Java stacks
  - Maven has built-in scripting language (Groovy) if some plugins aren't
sufficient for achieving whatever goals

Addressing Matt's later point about non-Mavenized Hadoop-1 line: it uses Maven
stuff suchs as deploy/install via custom ant tasks. Same approach would work
for saveVersion.sh and others, I am sure.

Cos

On Wed, Nov 21, 2012 at 11:25AM, Alejandro Abdelnur wrote:
> Hey Matt,
> 
> We already require java/mvn/protoc/cmake/forrest (forrest is hopefully on
> its way out with the move of docs to APT)
> 
> Why not do a maven-plugin to do that?
> 
> Colin already has something to simplify all the cmake calls from the builds
> using a maven-plugin (https://issues.apache.org/jira/browse/HADOOP-8887)
> 
> We could do the same with protoc, thus simplifying the POMs.
> 
> The saveVersion.sh seems like another prime candidate for a maven plugin,
> and in this case it would not require external tools.
> 
> Does this make sense?
> 
> Thx
> 
> On Wed, Nov 21, 2012 at 11:15 AM, Matt Foley  wrote:
> 
> > This discussion started in
> > HADOOP-8924
> > , where it was proposed to replace the build-time utility "saveVersion.sh"
> > with a python script.  This would require Python as a build-time
> > dependency.  Here's the background:
> >
> > Those of us involved in the branch-1-win port of Hadoop to Windows without
> > use of Cygwin, have faced the issue of frequent use of shell scripts
> > throughout the system, both in build time (eg, the utility
> > "saveVersion.sh"),
> > and run time (config files like "hadoop-env.sh" and the start/stop scripts
> > in "bin/*" ).  Similar usages exist throughout the Hadoop stack, in all
> > projects.
> >
> > The vast majority of these shell scripts do not do anything platform
> > specific; they can be expressed in a posix-conforming way.  Therefore, it
> > seems to us that it makes sense to start using a cross-platform scripting
> > language, such as python, in place of shell for these purposes.  For those
> > rare occasions where platform-specific functionality really is needed,
> > python also supports quite a lot of platform-specific functionality on both
> > Linux and Windows; but where that is inadequate, one could still
> > conditionally invoke a platform-specific module written in shell (for
> > Linux/*nix) or powershell or bat (for Windows).
> >
> > The primary motive for moving to a cross-platform scripting language is
> > maintainability.  The alternative would be to maintain two complete suites
> > of scripts, one for Linux and one for Windows (and perhaps others in the
> > future).  We want to avoid the need to update dual modules in two different
> > languages when functionality changes, especially given that many Linux
> > developers are not familiar with powershell or bat, and many Windows
> > developers are not familiar with shell or bash.
> >
> > Regarding the choice of python:
> >
> >- There are already a few instances of python usage in Hadoop, such as
> >the utility (currently broken) "relnotes.py", and massive usage of
> > python
> >in the examples/ and contrib/ directories.
> >- Python is also used in Bigtop build-time.
> >- The Python language is available for free on essentially all
> >platforms, under an Apache-compatible
> > license.
> >
> >- It is supported in Eclipse and similar IDEs.
> >- Most importantly, it is widely accepted as a reasonably good OO
> >scripting language, and it is easily learned by anyone who already knows
> >shell or perl, or other common scripting languages.
> >- On the Tiobe index of programming language
> > popularity<
> > http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html>,
> >which seeks to measure the relative number of software engineers who
> > know
> >and use each language, Python far exceeds Perl and Ruby.  The only more
> >well-known scripting languages are PHP and Visual Basic, neither of
> > which
> >seems a prime candidate for this use.
> >
> > For build-time usage, I think we should immediately approve python as a
> > build-time dependency, and allow people who are motivated to do so, to open
> > jiras for migrating existing build-time shell scripts to python.
> >
> > For run-time, there is likely to be a lot more discussion.  Lots of folks,
> > including me, aren't real happy with use of active scripts for
> > configuration, and various ot

[jira] [Resolved] (HADOOP-8417) HADOOP-6963 didn't update hadoop-core-pom-template.xml

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-8417.


Resolution: Fixed

Committed to branch-1.1. Thank you Zhihong.

> HADOOP-6963 didn't update hadoop-core-pom-template.xml
> --
>
> Key: HADOOP-8417
> URL: https://issues.apache.org/jira/browse/HADOOP-8417
> Project: Hadoop Common
>  Issue Type: Bug
>Affects Versions: 1.0.3
>Reporter: Zhihong Ted Yu
>Assignee: Zhihong Ted Yu
> Fix For: 1.1.0
>
> Attachments: hadoop-8417-v2.txt, hadoop-8417.txt
>
>
> HADOOP-6963 introduced commons-io 2.1 in ivy.xml but forgot to update the 
> hadoop-core-pom-template.xml.
> This has caused map reduce jobs in downstream projects to fail with:
> {code}
> Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.FileUtils
>   at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>   ... 15 more
> {code}
> This caused a regression for 1.0.3 because downstream projects used to not 
> directly depend on commons-io

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [ANNOUNCE] Hadoop branch-1.1 and Release Plan for Hadoop-1.1.0

[Konstantin Boudnik]

Also, I have updated HADOOP-8417 against 1.1.0 and we need to include it.
Otherwise, 1.1 will have the same issues for the downstream projects as 1.0.3
had.

Cos

On Sat, Jul 07, 2012 at 05:52PM, Konstantin Boudnik wrote:
> Matt,
> 
> Thanks for the update.
> 
> HADOOP-8399 would be beneficial for BigTop release and it is marked for 1.1.0
> release. The patch is available for a while now and if someone can review I'd
> go ahead and commit it today. 
> 
> I am working on the content of 0.3.1 BigTop release and will shortly post the
> vote for it. Once Hadoop 1.1 rc is cut we'll start testing it with the rest of
> the stack.
> 
> Cos
> 
> On Fri, Jul 06, 2012 at 02:24PM, Matt Foley wrote:
> > Hi Cos,
> > the query string didn't come thru on the link you sent, but the jira query
> > I use is:
> > project in (HADOOP,HDFS,MAPREDUCE) and (("Target Version/s" = '1.1.0'
> > and (fixVersion != '1.1.0' or fixVersion is EMPTY)) or (fixVersion =
> > '1.1.0' and "Target Version/s" is EMPTY)) and (status != Closed and status
> > != Resolved) ORDER BY KEY
> > 
> > You're correct that there are quite a few, currently 107, open jiras
> > originally targeted for 1.1.0 that do not have committed fixes.  Many of
> > these are just the inherited backlog of previously identified work.  I need
> > to move them to "Target Version/s" = 1.1.1.
> > 
> > Folks have requested that the following currently open jiras be included in
> > 1.1.0:
> > 
> > HADOOP-8417 - HADOOP-6963 didn't update hadoop-core-pom-template.xml
> > HADOOP-8445 - Token should not print the password in toString
> > HDFS-96 - HDFS does not support blocks greater than 2GB
> > HDFS-3596 - Improve FSEditLog pre-allocation in branch-1
> > MAPREDUCE-2903 - Map Tasks graph is throwing XML Parse error when Job is
> > executed with 0 maps
> > MAPREDUCE-2129 - Job may hang if
> > mapreduce.job.committer.setup.cleanup.needed=false and
> > mapreduce.map/reduce.failures.maxpercent>0
> > ---
> > MAPREDUCE-4049 - plugin for generic shuffle service
> > HADOOP-7823 - port HADOOP-4012 to branch-1 (splitting support for bzip2)
> > 
> > The first six are simple patches that I am comfortable including.
> > The last two are complex patches that have not yet been committed.
> > I am planning to defer those two to 1.1.1.
> > 
> > Beyond that, I'm going to cut 1.1.0-rc0 from the current state of
> > branch-1.1.
> > I'm planning to do that this weekend.  This is obviously delayed from the
> > previous plan, for which I apologize.
> > 
> > Comments welcome.
> > --Matt
> > 
> > 
> > On Tue, Jul 3, 2012 at 8:32 PM, Konstantin Boudnik  wrote:
> > 
> > > Hi Matt.
> > >
> > > I am picking up the hat of BigTop's maintainer for Hadoop 1.x line. And I
> > > wanted to sync up with about the Hadoop 1.1 release outlook, progress,
> > > what help
> > > you might need, etc.
> > >
> > > I see a few jiras left open in the release
> > > http://is.gd/OyuaNQ
> > > Is this the correct representation of the current status?
> > > How I can help from BigTop side (I haven't yet finalized the stack's
> > > versions), etc. Looking forward for your input. Thanks.
> > >
> > >   Cos
> > >
> > > On Fri, May 25, 2012 at 02:49PM, Matt Foley wrote:
> > > > Greetings.  With the approval of a public vote on common-dev@, I have
> > > > branched Hadoop branch-1 to create branch-1.1.  From this, I will create
> > > a
> > > > release candidate RC-0 for Hadoop-1.1.0, hopefully to be available
> > > shortly
> > > > after this weekend.
> > > >
> > > > There are over 80 patches in branch-1, over and above the contents of
> > > > hadoop-1.0.3.  So I anticipate that some stabilization will be needed,
> > > > before the RC can be approved as a 1.1.0 release.  Your participation in
> > > > assuring a stable RC is very important.  When it becomes available,
> > > please
> > > > download it and work with it to determine whether it is stable enough to
> > > > release, and report issues found.  My colleagues and I will do likewise,
> > > of
> > > > course, but no one company can adequately exercise a new release with
> > > this
> > > > many new contributions.
> > > >
> > > > There are two outstanding issue that are not yet committed, but I know
> > > the
> > > > contributors hope to see in 1.1.0:
> > > > MAPREDUCE-4049 <https://issues.apache.org/jira/browse/MAPREDUCE-4049
> > > >
> > > > HADOOP-4012 <https://issues.apache.org/jira/browse/HADOOP-4012>
> > > > Assuming there is an RC-1, and that these two patches can be committed
> > > > during stabilization of RC-0, I will plan to incorporate these 
> > > > additional
> > > > items in RC-1.
> > > >
> > > > Best regards,
> > > > --Matt
> > > > Release Manager
> > >




signature.asc
Description: Digital signature

Re: [ANNOUNCE] Hadoop branch-1.1 and Release Plan for Hadoop-1.1.0

[Konstantin Boudnik]

Matt,

Thanks for the update.

HADOOP-8399 would be beneficial for BigTop release and it is marked for 1.1.0
release. The patch is available for a while now and if someone can review I'd
go ahead and commit it today. 

I am working on the content of 0.3.1 BigTop release and will shortly post the
vote for it. Once Hadoop 1.1 rc is cut we'll start testing it with the rest of
the stack.

Cos

On Fri, Jul 06, 2012 at 02:24PM, Matt Foley wrote:
> Hi Cos,
> the query string didn't come thru on the link you sent, but the jira query
> I use is:
> project in (HADOOP,HDFS,MAPREDUCE) and (("Target Version/s" = '1.1.0'
> and (fixVersion != '1.1.0' or fixVersion is EMPTY)) or (fixVersion =
> '1.1.0' and "Target Version/s" is EMPTY)) and (status != Closed and status
> != Resolved) ORDER BY KEY
> 
> You're correct that there are quite a few, currently 107, open jiras
> originally targeted for 1.1.0 that do not have committed fixes.  Many of
> these are just the inherited backlog of previously identified work.  I need
> to move them to "Target Version/s" = 1.1.1.
> 
> Folks have requested that the following currently open jiras be included in
> 1.1.0:
> 
> HADOOP-8417 - HADOOP-6963 didn't update hadoop-core-pom-template.xml
> HADOOP-8445 - Token should not print the password in toString
> HDFS-96 - HDFS does not support blocks greater than 2GB
> HDFS-3596 - Improve FSEditLog pre-allocation in branch-1
> MAPREDUCE-2903 - Map Tasks graph is throwing XML Parse error when Job is
> executed with 0 maps
> MAPREDUCE-2129 - Job may hang if
> mapreduce.job.committer.setup.cleanup.needed=false and
> mapreduce.map/reduce.failures.maxpercent>0
> ---
> MAPREDUCE-4049 - plugin for generic shuffle service
> HADOOP-7823 - port HADOOP-4012 to branch-1 (splitting support for bzip2)
> 
> The first six are simple patches that I am comfortable including.
> The last two are complex patches that have not yet been committed.
> I am planning to defer those two to 1.1.1.
> 
> Beyond that, I'm going to cut 1.1.0-rc0 from the current state of
> branch-1.1.
> I'm planning to do that this weekend.  This is obviously delayed from the
> previous plan, for which I apologize.
> 
> Comments welcome.
> --Matt
> 
> 
> On Tue, Jul 3, 2012 at 8:32 PM, Konstantin Boudnik  wrote:
> 
> > Hi Matt.
> >
> > I am picking up the hat of BigTop's maintainer for Hadoop 1.x line. And I
> > wanted to sync up with about the Hadoop 1.1 release outlook, progress,
> > what help
> > you might need, etc.
> >
> > I see a few jiras left open in the release
> > http://is.gd/OyuaNQ
> > Is this the correct representation of the current status?
> > How I can help from BigTop side (I haven't yet finalized the stack's
> > versions), etc. Looking forward for your input. Thanks.
> >
> >   Cos
> >
> > On Fri, May 25, 2012 at 02:49PM, Matt Foley wrote:
> > > Greetings.  With the approval of a public vote on common-dev@, I have
> > > branched Hadoop branch-1 to create branch-1.1.  From this, I will create
> > a
> > > release candidate RC-0 for Hadoop-1.1.0, hopefully to be available
> > shortly
> > > after this weekend.
> > >
> > > There are over 80 patches in branch-1, over and above the contents of
> > > hadoop-1.0.3.  So I anticipate that some stabilization will be needed,
> > > before the RC can be approved as a 1.1.0 release.  Your participation in
> > > assuring a stable RC is very important.  When it becomes available,
> > please
> > > download it and work with it to determine whether it is stable enough to
> > > release, and report issues found.  My colleagues and I will do likewise,
> > of
> > > course, but no one company can adequately exercise a new release with
> > this
> > > many new contributions.
> > >
> > > There are two outstanding issue that are not yet committed, but I know
> > the
> > > contributors hope to see in 1.1.0:
> > > MAPREDUCE-4049 <https://issues.apache.org/jira/browse/MAPREDUCE-4049
> > >
> > > HADOOP-4012 <https://issues.apache.org/jira/browse/HADOOP-4012>
> > > Assuming there is an RC-1, and that these two patches can be committed
> > > during stabilization of RC-0, I will plan to incorporate these additional
> > > items in RC-1.
> > >
> > > Best regards,
> > > --Matt
> > > Release Manager
> >


signature.asc
Description: Digital signature

Re: [ANNOUNCE] Hadoop branch-1.1 and Release Plan for Hadoop-1.1.0

[Konstantin Boudnik]

Hi Matt.

I am picking up the hat of BigTop's maintainer for Hadoop 1.x line. And I
wanted to sync up with about the Hadoop 1.1 release outlook, progress, what help
you might need, etc.

I see a few jiras left open in the release 
http://is.gd/OyuaNQ
Is this the correct representation of the current status?
How I can help from BigTop side (I haven't yet finalized the stack's
versions), etc. Looking forward for your input. Thanks.

  Cos

On Fri, May 25, 2012 at 02:49PM, Matt Foley wrote:
> Greetings.  With the approval of a public vote on common-dev@, I have
> branched Hadoop branch-1 to create branch-1.1.  From this, I will create a
> release candidate RC-0 for Hadoop-1.1.0, hopefully to be available shortly
> after this weekend.
> 
> There are over 80 patches in branch-1, over and above the contents of
> hadoop-1.0.3.  So I anticipate that some stabilization will be needed,
> before the RC can be approved as a 1.1.0 release.  Your participation in
> assuring a stable RC is very important.  When it becomes available, please
> download it and work with it to determine whether it is stable enough to
> release, and report issues found.  My colleagues and I will do likewise, of
> course, but no one company can adequately exercise a new release with this
> many new contributions.
> 
> There are two outstanding issue that are not yet committed, but I know the
> contributors hope to see in 1.1.0:
> MAPREDUCE-4049 
> HADOOP-4012 
> Assuming there is an RC-1, and that these two patches can be committed
> during stabilization of RC-0, I will plan to incorporate these additional
> items in RC-1.
> 
> Best regards,
> --Matt
> Release Manager


signature.asc
Description: Digital signature

Re: Modification of wiki page "Distributions and Commercial Support"

[Konstantin Boudnik]

Sorry, did what? The page doesn't have any references to the BigTop as of now.

Cos

On Thu, Jun 28, 2012 at 07:11PM, Olivier Lamy wrote:
> Thanks!
> Note: Harsh J  did it and give me karma to modify the page.
> 
> --
> Olivier
> 
> 
> 2012/6/28 Konstantin Boudnik :
> > I will, I have the access.
> >
> > Cos
> >
> > On Thu, Jun 28, 2012 at 09:52AM, Roman Shaposhnik wrote:
> >> On Thu, Jun 28, 2012 at 3:27 AM, Olivier Lamy  wrote:
> >> > Hi,
> >> > What is the process to modify the page
> >> > http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support
> >> > ?
> >>
> >> On a related note, I would like to add Apache Bigtop (incubating)
> >> Hadoop distribution
> >> to that wiki page.
> >>
> >> Can anybody help me with that? The wiki seems to be locked.
> >>
> >> Thanks,
> >> Roman.
> 
> 
> 
> -- 
> Olivier Lamy
> Talend: http://coders.talend.com
> http://twitter.com/olamy | http://linkedin.com/in/olamy

Re: Modification of wiki page "Distributions and Commercial Support"

[Konstantin Boudnik]

I will, I have the access.

Cos

On Thu, Jun 28, 2012 at 09:52AM, Roman Shaposhnik wrote:
> On Thu, Jun 28, 2012 at 3:27 AM, Olivier Lamy  wrote:
> > Hi,
> > What is the process to modify the page
> > http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support
> > ?
> 
> On a related note, I would like to add Apache Bigtop (incubating)
> Hadoop distribution
> to that wiki page.
> 
> Can anybody help me with that? The wiki seems to be locked.
> 
> Thanks,
> Roman.

[jira] [Resolved] (HADOOP-7084) Remove java5 dependencies from site's build

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-7084.


Resolution: Duplicate

This is a dup of Hadoop-7072

> Remove java5 dependencies from site's build
> ---
>
> Key: HADOOP-7084
> URL: https://issues.apache.org/jira/browse/HADOOP-7084
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: build
>    Reporter: Konstantin Boudnik
>    Assignee: Konstantin Boudnik
>
> Java5 dependency needs to be removed from 
> http://svn.apache.org/repos/asf/hadoop/site/build.xml

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira