Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-12-12 Thread Konstantin Boudnik
On Sat, Dec 01, 2012 at 10:44AM, Steve Loughran wrote: On 1 December 2012 01:08, Eli Collins e...@cloudera.com wrote: -1, 0, -1 IIUC the only platform we plan to add support for that we can't easily support today (w/o an emulation layer like cygwin) is Windows, and it seems like

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-12-12 Thread Konstantin Boudnik
On Sat, Dec 01, 2012 at 10:07PM, Eric Yang wrote: -1, +1, -1 Python has fairly inconsistent support across all major OS vendors. It is hard to get it right unless the scripts are all designed to make use of Python 2.4. However, Python 2.4 doesn't have necessary OS features to make Python

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-12-04 Thread Matt Foley
Hi Doug, I didn't read your email until this morning, but I spent time overnight thinking about the Apache Way and reached similar conclusions. While tooling is broader in scope than a single code change, it is a technical choice that we all have to live with. More importantly, Community over

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-12-04 Thread Radim Kolar
result of vote is to close https://issues.apache.org/jira/browse/HADOOP-9073 and write groovy in pom.xml variant (option number 2)?

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-12-04 Thread Alejandro Abdelnur
i've been playing around writing a couple of maven plugins, one to replace saveversion.sh and the other to invoke protoc. they both work in windows standard cmd (no cygwin required). together with hadoop-8887 they would remove most of the scripting done the poms. (they also work in linux and

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-12-04 Thread Matt Foley
There's already a jira: HADOOP-8924https://issues.apache.org/jira/browse/HADOOP-8924 On Tue, Dec 4, 2012 at 1:00 PM, Alejandro Abdelnur t...@cloudera.comwrote: i've been playing around writing a couple of maven plugins, one to replace saveversion.sh and the other to invoke protoc. they both

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-12-03 Thread Tom White
+1, +1, -1 Tom On Sat, Nov 24, 2012 at 8:13 PM, Matt Foley ma...@apache.org wrote: For discussion, please see previous thread [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. This vote consists of three separate items: 1.

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-12-03 Thread Doug Cutting
On Mon, Dec 3, 2012 at 11:21 AM, Matt Foley mfo...@hortonworks.com wrote: It is intended to be a technical discussion, in the sense of the bylaws statement (in section Roles and Responsibilities: Committers), Committers may cast binding votes on any technical discussion regarding any

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-12-03 Thread Doug Cutting
Hadoop's bylaws do draw finer distinctions than the Apache voting guidelines document, but we follow the same general principles that are described there. As I understand it, the rationale for using consensus for code is that everyone needs to agree on everything in the codebase or we've

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-12-01 Thread Steve Loughran
On 1 December 2012 01:08, Eli Collins e...@cloudera.com wrote: -1, 0, -1 IIUC the only platform we plan to add support for that we can't easily support today (w/o an emulation layer like cygwin) is Windows, and it seems like making the bash scripts simpler and having parallel bat files is

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-12-01 Thread Steve Loughran
On 30 November 2012 13:40, Radim Kolar h...@filez.com wrote: inline ant scripts =0. Ant's versioning is stricter; you can pull down the exact Jar versions, and some of us in the Ant team worked very hard to get it going everywhere. You don't gain anything by going to .py there are sh

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-12-01 Thread Joep Rottinghuis
0, 0, -1 (non-binding) Joep On Nov 24, 2012, at 12:13 PM, Matt Foley ma...@apache.org wrote: For discussion, please see previous thread [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. This vote consists of three separate items:

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-30 Thread Steve Loughran
On 30 November 2012 00:29, Radim Kolar h...@filez.com wrote: * What else in the current build, besides saveVersion.sh, you see as candidate to be migrated to Phyton? inline ant scripts =0. Ant's versioning is stricter; you can pull down the exact Jar versions, and some of us in the Ant

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-30 Thread Steve Loughran
On 30 November 2012 12:57, Luke Lu l...@apache.org wrote: I'd like to change my binding vote to -1, -0, -1. Considering the hadoop stack/ecosystem as a whole, I think the best cross platform scripting language to adopt is jruby for following reasons: 1. HBase already adopted jruby for HBase

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-30 Thread Luke Lu
On Fri, Nov 30, 2012 at 5:29 AM, Steve Loughran ste...@hortonworks.comwrote: where do you set your env vars... and what about JAVA_HOME There should be only two env vars (JAVA_HOME and HADOOP_HOME) to deal with in the native scripts (.bat on windows and .sh on unix platforms) to boostrap jruby

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-30 Thread Luke Lu
On Fri, Nov 30, 2012 at 5:29 AM, Steve Loughran ste...@hortonworks.comwrote: Yes, you need python at build time, but that's no worse than having a protoc compiler, gcc and the automake toolchain. The problem is that python is known to have _backward_ compatibility issues on various

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-30 Thread Doug Cutting
-1, +1, -1 Run- build-time scripting should be limited to operations that are impossible in Java. These should not be complex nor should we encourage more complexity in them. A parallel set of simple .bat files for such operations seems preferable to adding a Python dependency. Doug On Sat,

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-30 Thread Radim Kolar
inline ant scripts =0. Ant's versioning is stricter; you can pull down the exact Jar versions, and some of us in the Ant team worked very hard to get it going everywhere. You don't gain anything by going to .py there are sh scripts inside maven ant plugin stuff

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-30 Thread Jitendra Pandey
+1, +1, +1 On Fri, Nov 30, 2012 at 5:40 AM, Radim Kolar h...@filez.com wrote: inline ant scripts =0. Ant's versioning is stricter; you can pull down the exact Jar versions, and some of us in the Ant team worked very hard to get it going everywhere. You don't gain anything by going to

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-30 Thread Raja Aluri
+1, +1, +1 (non binding) It makes it a lot easier to make build tools (that cannot be developed easily using maven) work across non-unix like platforms (especially windows). Raja On Sat, Nov 24, 2012 at 12:13 PM, Matt Foley ma...@apache.org wrote: For discussion, please see previous thread

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-30 Thread Eli Collins
-1, 0, -1 IIUC the only platform we plan to add support for that we can't easily support today (w/o an emulation layer like cygwin) is Windows, and it seems like making the bash scripts simpler and having parallel bat files is IMO a better approach. On Sat, Nov 24, 2012 at 12:13 PM, Matt Foley

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-29 Thread Matt Foley
Hi Alejandro, Please see in-line below. On Mon, Nov 26, 2012 at 1:52 PM, Alejandro Abdelnur t...@cloudera.com wrote: Matt, The scope of this vote seems different from what was discussed in the PROPOSAL thread. In the PROPOSAL thread you indicated this was for Hadoop1 because it is ANT

RE: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-29 Thread Ivan Mitic
+1, +1, +1 (some comments inline) -Original Message- From: mfo...@hortonworks.com [mailto:mfo...@hortonworks.com] On Behalf Of Matt Foley Sent: Saturday, November 24, 2012 12:13 PM To: common-dev@hadoop.apache.org Subject: [VOTE] introduce Python as build-time and run-time dependency for

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-29 Thread Radim Kolar
* What else in the current build, besides saveVersion.sh, you see as candidate to be migrated to Phyton? inline ant scripts

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-29 Thread Alejandro Abdelnur
Matt, Let me repost my previous questions and a few more. I'd appreciate your answers, as it will help me understand the full impact this would have in Hadoop and related projects. * Phyton as runtime requirement. Are you planing to migrate all BASH scripts provided by Hadoop (or dynamically

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-29 Thread Matt Foley
Python as runtime requirement. Are you planing to migrate all BASH scripts provided by Hadoop (or dynamically created -ie launcher scripts) to Python? I don't intend to mandate use of Python. Rather, I want there to be a cross-platform option available. Things that are best done in

RE: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-29 Thread Mahadevan Venkatraman
+1, +1, +1 (non-binding) Supporting Comments: Build-time scripts: Using a platform independent language such as python (or maven in certain cases) will greatly help in reducing build breaks and improve on build script maintainability. Run-time scripts: Most run-time scripts are end-user

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-29 Thread Matt Foley
Hello again. Crossed in the mail. * What kind of tasks you envision Python scripts will enable that are not possible today? The point isn't to open brave new worlds. The point is to avoid the nightmare of having to maintain multiple parallel scripts doing the SAME THING in multiple scripting

RE: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-29 Thread Chuan Liu
+1 +1 +1 Agree with Matt on the code maintainability. I think on one side we have Shell which is a script language and OS dependent, e.g. as in bash vs powershell; on the other side we have Java which is not a script language and OS independent. I would accept any script language that can fix

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-29 Thread Bikas Saha
+1, +1, +1 (non-binding) We have had promising results for 1 and 2 when porting to Windows. 3 would allow us to remove platform dependencies from test code. Agree that there might be some nuanced operations that require OS specific environments but this would lead to keeping them at a minimum.

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-27 Thread Roman Shaposhnik
On Sat, Nov 24, 2012 at 12:13 PM, Matt Foley ma...@apache.org wrote: For discussion, please see previous thread [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. Perhaps I'm missing something, but I can't possibly imagine how a vote on a

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-26 Thread Robert Evans
+1, +1, 0 On 11/24/12 2:13 PM, Matt Foley ma...@apache.org wrote: For discussion, please see previous thread [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. This vote consists of three separate items: 1. Contributors shall be allowed to

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-26 Thread Adam Berry
0, +1, -1 (non-binding) Also, it feels like maybe the discussion should have been kept open a little longer, thanksgiving holidays last week meant that people may have missed it. Cheers, Adam On Nov 26, 2012, at 10:16 AM, Robert Evans wrote: +1, +1, 0 On 11/24/12 2:13 PM, Matt Foley

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-26 Thread Radim Kolar
-1, +1, -1

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-26 Thread Konstantin Boudnik
-1, +1, -1 Thanks On Sat, Nov 24, 2012 at 12:13PM, Matt Foley wrote: For discussion, please see previous thread [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. This vote consists of three separate items: 1. Contributors shall be

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-26 Thread Giridharan Kesavan
+1, +1, +1 -Giri On Sat, Nov 24, 2012 at 12:13 PM, Matt Foley ma...@apache.org wrote: For discussion, please see previous thread [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. This vote consists of three separate items: 1.

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-26 Thread Radim Kolar
In the PROPOSAL thread you indicated this was for Hadoop1 because it is ANT based. And the main reason was to remove saveVersion.sh. Your #3 was not discussed in the proposal, was it? it was part of original proposal but not discussed much because language war was more attractive option. You

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-25 Thread Steve Loughran
On 24 November 2012 20:13, Matt Foley ma...@apache.org wrote: For discussion, please see previous thread [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. This vote consists of three separate items: 1. Contributors shall be allowed to

[VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-24 Thread Matt Foley
For discussion, please see previous thread [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. This vote consists of three separate items: 1. Contributors shall be allowed to use Python as a platform-independent scripting language for