I am good with Scheme 2. We are finding a fair number of issues trying to move from Pig 0.8.1 to 0.9, and I don't think those issues are fixed in 10, either.. not sure that this "stabilization" process has happened yet.
D On Mon, Oct 24, 2011 at 11:59 AM, Daniel Dai <da...@hortonworks.com> wrote: > Yes, we need a versioning scheme. There are two versioning scheme I can > think of: > > Scheme 1: > <major>.<patch> > <major> will be the feature rich release every 3 month > <patch> will be the bug fix release when necessary > > Nov release will be 1.0, Feb release will be 2.0. There will be 1.1, 2.1 > etc > for bug fixes. > > Scheme 2: > <major>.<minor>.<patch> > Most of our 3 month release will be counted as <minor> release unless there > are major user facing/disruptive changes. > > Nov release will be 1.0.0, Feb release will be 1.1.0. There will be 1.0.1, > 1.1.1 etc for bug fixes. > > I personally prefer scheme 2, increasing major version too frequently might > be confusing to users. How's other folks feel? > > Daniel > > > On Sat, Oct 22, 2011 at 2:31 AM, Gianmarco De Francisci Morales < > g...@apache.org> wrote: > > > Hi, > > > > just my 2 cents. > > > > I think the issue here is not 1.0 vs 0.10, but what's the versioning > scheme > > we want to use for Pig. > > Up to now it has been just an increasing number after a '0.' prefix, > > changed > > when the community felt it was time. I think this works well for a small > > project, but it is somewhat fuzzy. > > > > I like the idea of having <major>.<minor>.<patch> versions like many > other > > projects. It's a very clear and almost standard way of versioning a piece > > of > > software. It has clear rules on when to change each of the numbers, and > > lets > > the user get an idea of backward compatibility at a glance. > > > > So, to conclude, I am in favor of going 1.0 (or 1.0.0) as long as we > decide > > a clear versioning policy (whichever it is). > > So that the 1.0 milestone would mark the beginning of our new policy. > > > > Cheers, > > -- > > Gianmarco > > > > > > > > On Fri, Oct 21, 2011 at 23:10, <milind.bhandar...@emc.com> wrote: > > > > > If one were to rewrite input and output formats to use the webhdfs:// > > > APIs, this would not be an issue, right ? > > > > > > - milind > > > > > > > > > On 10/21/11 1:50 PM, "Santhosh Srinivasan" <s...@yahoo-inc.com> wrote: > > > > > > >If I was not clear in my earlier email, I apologize for the lack of > > > >clarity. I am no longer in favour of waiting for Hadoop API stability > > > >across Hadoop versions. It's a pipe dream. > > > > > > > >When we had PigInputFormat and PigOutputFormat, your reasoning would > be > > > >spot on. I am concerned about the following. Our tight integration > with > > > >Hadoop due to the use of Input and Output format might lead to a break > > in > > > >backward compatibility. I am not sure if the comparison with that of > > Java > > > >is valid. Probably a majority of the users don't use JNI. Its very > hard > > > >to use Pig without writing custom load and store functions. The > default > > > >load and store don't suffice for a majority of use cases that I have > > > >observed. > > > > > > > >I am trying to get all factors that might influence this decision. > From > > > >the few emails that have been exchanged since yesterday, we have the > > > >following factors: > > > > > > > >1. Hadoop 0.20.205 (support for Append) > > > >2. Hadoop 0.22 > > > >3. Hadoop 0.23 > > > >4. Maturity of the new parser > > > >5. Stability of the new logical plan > > > >6. Other components in the eco system. > > > > - Avro (1.5.4, 1.4.1, ...) > > > > - Cassandra (1.0.0, 0.8.7, ...) > > > > - Chukwa (0.4.0, 0.3.0, ...) > > > > - Hama (0.3.0, 0.2.0, ...) > > > > - Hbase (0.90.4, 0.90.3, 0.90.2, 0.90.1, ...) > > > > - Hive (Releases - 0.7.1, 0.7.0, 0.6.0, ...) > > > > - Zookeeper (3.3.3, 3.3.2, 3.2.2, 3.1.2, ...) > > > > > > > >Santhosh > > > > > > > > > > > >-----Original Message----- > > > >From: Thejas Nair [mailto:the...@hortonworks.com] > > > >Sent: Friday, October 21, 2011 11:22 AM > > > >To: dev@pig.apache.org > > > >Subject: Re: Next Pig release proposal > > > > > > > > > > > >Santosh, > > > >I thought you meant API stability for hadoop across major versions, > but > > I > > > >guess you are referring to stability within 0.23 versions. But > argument > > > >applies to that as well, if 0.23.1 is not compatible with 0.23.0, we > > need > > > >to call the release for 0.23.1 as 'pig 1.x for 0.23.1 api' . > > > > > > > >We just need to communicate to the users that the > > > >InputFormat/OutputFormat api's (and any anything else we expose from > > > >hadoop) depends on the hadoop version they are using. > > > > > > > >I think it is just like different JNI libraries that you would write > for > > > >different OS. But the java version remains the same across OSs. > > > > > > > >-Thejas > > > > > > > > > > > >On 10/21/11 10:59 AM, Santhosh Srinivasan wrote: > > > >> Thejas, > > > >> > > > >> I guess you did not read my email completely. You are referring to > the > > > >>premise without examining the conclusion. I am repasting my entire > > email > > > >>to avoid confusion (I hate truncated references). If you could > respond > > > >>again, it will bring us onto the same page. > > > >> > > > >> <email> > > > >> > > > >> Ref: http://tinyurl.com/4ng8upa (last discussion on 1.0) > > > >> > > > >> How far have we progressed from our last discussion in March. There > > was > > > >>no consensus on the 1.0 release. Opinions ranged from having more > > > >>releases to bake in the maturity of the new parser and logical plan > > > >>changes to compatibility with Hadoop API (was compared to Social > > > >>Security - a very hot topic these days). > > > >> > > > >> My concerns were around Hadoop API stability. I have heard that the > > > >>APIs will not be stable for at least 1 year. This is taking me away > > from > > > >>the Hadoop API stability factor (They passed healthcare in that > > > >>duration. Really!) Do we want compatibility with 0.23 as a gating > > factor > > > >>- not sure if this is anywhere close to getting done in the near > > future. > > > >>Will we support append (0.20.205)? > > > >> > > > >> Btw, Hbase has been doing 0.90.1, 0.90.2, etc. So we can take a look > > at > > > >>this option too. > > > >> > > > >> Santhosh > > > >> > > > >> > > > >> > > > >> -----Original Message----- > > > >> From: Olga Natkovich [mailto:ol...@yahoo-inc.com] > > > >> Sent: Thursday, October 20, 2011 4:40 PM > > > >> To: dev@pig.apache.org > > > >> Subject: Next Pig release proposal > > > >> > > > >> Hi, > > > >> > > > >> Here is what I propose we do for the next Pig release: > > > >> > > > >> > > > >> (1) Branch early next week - we have major features and many bug > > > >>fixes in and will be fixing remaining bugs on the branch > > > >> > > > >> (2) Publish the release by 11/15 - that will give us a couple of > > > >>weeks to stabilize the branch and get last minute bug fixes in > > > >> > > > >> (3) Make this release a 1.0 release. Reasons to go for 1.0 and > not > > > >>0.10 > > > >> > > > >> a. This release has minimal number of features and was focused > > on > > > >>code stabilization and bug fixes. We believe it will be a stable > > release > > > >> > > > >> <email/> > > > >> > > > >> Thanks, > > > >> Santhosh > > > >> > > > >> -----Original Message----- > > > >> From: Thejas Nair [mailto:the...@hortonworks.com] > > > >> Sent: Friday, October 21, 2011 10:45 AM > > > >> To: dev@pig.apache.org > > > >> Subject: Re: Next Pig release proposal > > > >> > > > >> On 10/20/11 4:58 PM, Santhosh Srinivasan wrote: > > > >>> Ref: http://tinyurl.com/4ng8upa (last discussion on 1.0) > > > >>> > > > >>> How far have we progressed from our last discussion in March. There > > > >>>was no consensus on the 1.0 release. Opinions ranged from having > more > > > >>>releases to bake in the maturity of the new parser and logical plan > > > >>>changes to compatibility with Hadoop API (was compared to Social > > > >>>Security - a very hot topic these days). > > > >>> > > > >>> My concerns were around Hadoop API stability. > > > >> > > > >> Over the next year or so, there are going to be two API versions of > > > >>hadoop to be supported - 0.20.x api's and 0.23 apis, as we will have > > > >>userbase on both. > > > >> > > > >> I think it is just a matter of releasing pig 1.0 for 0.20.x api's > and > > > >>1.0 for 0.23.x api's. We will have to come up with a numbering > scheme > > > >>that reflects 'for hadoop version X' in our pig releases, regardless > of > > > >>it being 0.10 or 1.0. > > > >> > > > >> As there will be support for different api's of hadoop in pig > > releases, > > > >>I don't see a reason why the hadoop api stability should stop pig > from > > > >>going 1.0 . > > > >> > > > >> -Thejas > > > > > > > > > > > > > > > > >
