Yes, we need a versioning scheme. There are two versioning scheme I can think of:
Scheme 1: <major>.<patch> <major> will be the feature rich release every 3 month <patch> will be the bug fix release when necessary Nov release will be 1.0, Feb release will be 2.0. There will be 1.1, 2.1 etc for bug fixes. Scheme 2: <major>.<minor>.<patch> Most of our 3 month release will be counted as <minor> release unless there are major user facing/disruptive changes. Nov release will be 1.0.0, Feb release will be 1.1.0. There will be 1.0.1, 1.1.1 etc for bug fixes. I personally prefer scheme 2, increasing major version too frequently might be confusing to users. How's other folks feel? Daniel On Sat, Oct 22, 2011 at 2:31 AM, Gianmarco De Francisci Morales < g...@apache.org> wrote: > Hi, > > just my 2 cents. > > I think the issue here is not 1.0 vs 0.10, but what's the versioning scheme > we want to use for Pig. > Up to now it has been just an increasing number after a '0.' prefix, > changed > when the community felt it was time. I think this works well for a small > project, but it is somewhat fuzzy. > > I like the idea of having <major>.<minor>.<patch> versions like many other > projects. It's a very clear and almost standard way of versioning a piece > of > software. It has clear rules on when to change each of the numbers, and > lets > the user get an idea of backward compatibility at a glance. > > So, to conclude, I am in favor of going 1.0 (or 1.0.0) as long as we decide > a clear versioning policy (whichever it is). > So that the 1.0 milestone would mark the beginning of our new policy. > > Cheers, > -- > Gianmarco > > > > On Fri, Oct 21, 2011 at 23:10, <milind.bhandar...@emc.com> wrote: > > > If one were to rewrite input and output formats to use the webhdfs:// > > APIs, this would not be an issue, right ? > > > > - milind > > > > > > On 10/21/11 1:50 PM, "Santhosh Srinivasan" <s...@yahoo-inc.com> wrote: > > > > >If I was not clear in my earlier email, I apologize for the lack of > > >clarity. I am no longer in favour of waiting for Hadoop API stability > > >across Hadoop versions. It's a pipe dream. > > > > > >When we had PigInputFormat and PigOutputFormat, your reasoning would be > > >spot on. I am concerned about the following. Our tight integration with > > >Hadoop due to the use of Input and Output format might lead to a break > in > > >backward compatibility. I am not sure if the comparison with that of > Java > > >is valid. Probably a majority of the users don't use JNI. Its very hard > > >to use Pig without writing custom load and store functions. The default > > >load and store don't suffice for a majority of use cases that I have > > >observed. > > > > > >I am trying to get all factors that might influence this decision. From > > >the few emails that have been exchanged since yesterday, we have the > > >following factors: > > > > > >1. Hadoop 0.20.205 (support for Append) > > >2. Hadoop 0.22 > > >3. Hadoop 0.23 > > >4. Maturity of the new parser > > >5. Stability of the new logical plan > > >6. Other components in the eco system. > > > - Avro (1.5.4, 1.4.1, ...) > > > - Cassandra (1.0.0, 0.8.7, ...) > > > - Chukwa (0.4.0, 0.3.0, ...) > > > - Hama (0.3.0, 0.2.0, ...) > > > - Hbase (0.90.4, 0.90.3, 0.90.2, 0.90.1, ...) > > > - Hive (Releases - 0.7.1, 0.7.0, 0.6.0, ...) > > > - Zookeeper (3.3.3, 3.3.2, 3.2.2, 3.1.2, ...) > > > > > >Santhosh > > > > > > > > >-----Original Message----- > > >From: Thejas Nair [mailto:the...@hortonworks.com] > > >Sent: Friday, October 21, 2011 11:22 AM > > >To: dev@pig.apache.org > > >Subject: Re: Next Pig release proposal > > > > > > > > >Santosh, > > >I thought you meant API stability for hadoop across major versions, but > I > > >guess you are referring to stability within 0.23 versions. But argument > > >applies to that as well, if 0.23.1 is not compatible with 0.23.0, we > need > > >to call the release for 0.23.1 as 'pig 1.x for 0.23.1 api' . > > > > > >We just need to communicate to the users that the > > >InputFormat/OutputFormat api's (and any anything else we expose from > > >hadoop) depends on the hadoop version they are using. > > > > > >I think it is just like different JNI libraries that you would write for > > >different OS. But the java version remains the same across OSs. > > > > > >-Thejas > > > > > > > > >On 10/21/11 10:59 AM, Santhosh Srinivasan wrote: > > >> Thejas, > > >> > > >> I guess you did not read my email completely. You are referring to the > > >>premise without examining the conclusion. I am repasting my entire > email > > >>to avoid confusion (I hate truncated references). If you could respond > > >>again, it will bring us onto the same page. > > >> > > >> <email> > > >> > > >> Ref: http://tinyurl.com/4ng8upa (last discussion on 1.0) > > >> > > >> How far have we progressed from our last discussion in March. There > was > > >>no consensus on the 1.0 release. Opinions ranged from having more > > >>releases to bake in the maturity of the new parser and logical plan > > >>changes to compatibility with Hadoop API (was compared to Social > > >>Security - a very hot topic these days). > > >> > > >> My concerns were around Hadoop API stability. I have heard that the > > >>APIs will not be stable for at least 1 year. This is taking me away > from > > >>the Hadoop API stability factor (They passed healthcare in that > > >>duration. Really!) Do we want compatibility with 0.23 as a gating > factor > > >>- not sure if this is anywhere close to getting done in the near > future. > > >>Will we support append (0.20.205)? > > >> > > >> Btw, Hbase has been doing 0.90.1, 0.90.2, etc. So we can take a look > at > > >>this option too. > > >> > > >> Santhosh > > >> > > >> > > >> > > >> -----Original Message----- > > >> From: Olga Natkovich [mailto:ol...@yahoo-inc.com] > > >> Sent: Thursday, October 20, 2011 4:40 PM > > >> To: dev@pig.apache.org > > >> Subject: Next Pig release proposal > > >> > > >> Hi, > > >> > > >> Here is what I propose we do for the next Pig release: > > >> > > >> > > >> (1) Branch early next week - we have major features and many bug > > >>fixes in and will be fixing remaining bugs on the branch > > >> > > >> (2) Publish the release by 11/15 - that will give us a couple of > > >>weeks to stabilize the branch and get last minute bug fixes in > > >> > > >> (3) Make this release a 1.0 release. Reasons to go for 1.0 and not > > >>0.10 > > >> > > >> a. This release has minimal number of features and was focused > on > > >>code stabilization and bug fixes. We believe it will be a stable > release > > >> > > >> <email/> > > >> > > >> Thanks, > > >> Santhosh > > >> > > >> -----Original Message----- > > >> From: Thejas Nair [mailto:the...@hortonworks.com] > > >> Sent: Friday, October 21, 2011 10:45 AM > > >> To: dev@pig.apache.org > > >> Subject: Re: Next Pig release proposal > > >> > > >> On 10/20/11 4:58 PM, Santhosh Srinivasan wrote: > > >>> Ref: http://tinyurl.com/4ng8upa (last discussion on 1.0) > > >>> > > >>> How far have we progressed from our last discussion in March. There > > >>>was no consensus on the 1.0 release. Opinions ranged from having more > > >>>releases to bake in the maturity of the new parser and logical plan > > >>>changes to compatibility with Hadoop API (was compared to Social > > >>>Security - a very hot topic these days). > > >>> > > >>> My concerns were around Hadoop API stability. > > >> > > >> Over the next year or so, there are going to be two API versions of > > >>hadoop to be supported - 0.20.x api's and 0.23 apis, as we will have > > >>userbase on both. > > >> > > >> I think it is just a matter of releasing pig 1.0 for 0.20.x api's and > > >>1.0 for 0.23.x api's. We will have to come up with a numbering scheme > > >>that reflects 'for hadoop version X' in our pig releases, regardless of > > >>it being 0.10 or 1.0. > > >> > > >> As there will be support for different api's of hadoop in pig > releases, > > >>I don't see a reason why the hadoop api stability should stop pig from > > >>going 1.0 . > > >> > > >> -Thejas > > > > > > > > > > >
