Re: Un-deprecate the old MapReduce API?
It sounds like there's no strong objection to un-deprecating the old
API in 0.20 - I'll create a patch for this (see
https://issues.apache.org/jira/browse/MAPREDUCE-1734).
0.21 is less clear cut. However, if the new API were marked as
Evolving, then it's odd, to say the least, if the old API were
deprecated since it would send a confusing message to users. There
seems to be consensus that the new API is Evolving (please comment on
https://issues.apache.org/jira/browse/MAPREDUCE-1623 to discuss
whether all of the new API should be marked Evolving, which the latest
patch does). Indeed, the new API hasn't seen widespread use yet, so it
still seems premature to deprecate the old API in 0.21. I've opened
https://issues.apache.org/jira/browse/MAPREDUCE-1735 where we can
discuss this particular case further.
Cheers,
Tom
On Fri, Apr 23, 2010 at 9:21 AM, Alan Gates ga...@yahoo-inc.com wrote:
I don't have any issue with un-deprecating the old APIs. I agree if changes
are needed it's better to mark the new APIs to reflect it. I just hope
those changes can be kept as backward compatible as possible. In particular
with Job, Pig uses that in some of it's APIs that it has declared stable
(LoadFunc, StoreFunc).
Alan.
On Apr 22, 2010, at 11:30 PM, Arun C Murthy wrote:
Alan,
On Apr 22, 2010, at 12:12 PM, Alan Gates wrote:
Speaking for one power user (Pig) that did move to the new APIs, moving
that interface to evolving is a little unsettling. Is there a feel for how
much the new API is going to change?
The intent isn't to mark the 'new' apis as 'Evolving' to change them
willy-nilly... please don't read it so!
This is just a pragmatic proposal to reflect that the 'old' apis will, for
lack of stabilization of new apis, be supported.
Given that, the new apis could mostly be stable, but for Job and Cluster -
is that reasonable? This will ensure we send the right message all concerned
regarding stability of o.a.h.mapreduce.{Mapper|Reducer|...}. Thoughts?
Arun
Alan.
Re: Un-deprecate the old MapReduce API?
Alan,
On Apr 22, 2010, at 12:12 PM, Alan Gates wrote:
Speaking for one power user (Pig) that did move to the new APIs,
moving that interface to evolving is a little unsettling. Is there
a feel for how much the new API is going to change?
The intent isn't to mark the 'new' apis as 'Evolving' to change them
willy-nilly... please don't read it so!
This is just a pragmatic proposal to reflect that the 'old' apis
will, for lack of stabilization of new apis, be supported.
Given that, the new apis could mostly be stable, but for Job and
Cluster - is that reasonable? This will ensure we send the right
message all concerned regarding stability of o.a.h.mapreduce.{Mapper|
Reducer|...}. Thoughts?
Arun
Alan.
On Apr 21, 2010, at 2:24 PM, Tom White wrote:
The old MapReduce API in org.apache.hadoop.mapred was deprecated in
the 0.20 release series when the new (Context Objects) MapReduce
API
was added in org.apache.hadoop.mapreduce. Unfortunately, the new API
was not complete in 0.20 and most users stayed with the old API. This
has led to the confusing situation where the old API is generally
recommended, even though it is deprecated.
To remedy this situation I suggest that we remove deprecations from
the old API in 0.20 and trunk, and mark the new API as
Evolving (see
MAPREDUCE-1623 for the latter). This would mean a few things:
* The next 0.20 release would have a non-deprecated old API.
* The forthcoming 0.21 release would have a Stable (non-deprecated)
old API, and a Evolving new API.
* For some pre-1.0 release (perhaps 0.22), the old API could be
deprecated again, and the new API marked as Stable.
* In the 1.0 release it would be possible to remove the old API.
Thoughts?
Tom
Re: Un-deprecate the old MapReduce API?
I don't have any issue with un-deprecating the old APIs. I agree if
changes are needed it's better to mark the new APIs to reflect it. I
just hope those changes can be kept as backward compatible as
possible. In particular with Job, Pig uses that in some of it's APIs
that it has declared stable (LoadFunc, StoreFunc).
Alan.
On Apr 22, 2010, at 11:30 PM, Arun C Murthy wrote:
Alan,
On Apr 22, 2010, at 12:12 PM, Alan Gates wrote:
Speaking for one power user (Pig) that did move to the new APIs,
moving that interface to evolving is a little unsettling. Is there
a feel for how much the new API is going to change?
The intent isn't to mark the 'new' apis as 'Evolving' to change them
willy-nilly... please don't read it so!
This is just a pragmatic proposal to reflect that the 'old' apis
will, for lack of stabilization of new apis, be supported.
Given that, the new apis could mostly be stable, but for Job and
Cluster - is that reasonable? This will ensure we send the right
message all concerned regarding stability of o.a.h.mapreduce.{Mapper|
Reducer|...}. Thoughts?
Arun
Alan.
Re: Un-deprecate the old MapReduce API?
+1. Currently there is almost no way to write a moderately complex MR job that doesn't spew deprecation warnings. It serves to endlessly confuse newcomers to Hadoop. On Wed, Apr 21, 2010 at 5:24 PM, Tom White t...@cloudera.com wrote: The old MapReduce API in org.apache.hadoop.mapred was deprecated in the 0.20 release series when the new (Context Objects) MapReduce API was added in org.apache.hadoop.mapreduce. Unfortunately, the new API was not complete in 0.20 and most users stayed with the old API. This has led to the confusing situation where the old API is generally recommended, even though it is deprecated. To remedy this situation I suggest that we remove deprecations from the old API in 0.20 and trunk, and mark the new API as Evolving (see MAPREDUCE-1623 for the latter). This would mean a few things: * The next 0.20 release would have a non-deprecated old API. * The forthcoming 0.21 release would have a Stable (non-deprecated) old API, and a Evolving new API. * For some pre-1.0 release (perhaps 0.22), the old API could be deprecated again, and the new API marked as Stable. * In the 1.0 release it would be possible to remove the old API. Thoughts? Tom -- Eric Sammer phone: +1-917-287-2675 twitter: esammer data: www.cloudera.com
Re: Un-deprecate the old MapReduce API?
On the various pieces, I think: 0.20: -0 for removing the deprecation, +1 for improving the deprecation message with links to the corresponding class. 0.21: new core api should be stable except for Job and Cluster new library code should be evolving -1 for removing the deprecation, we need to 0.22: all of the new api should be stable and the old api deprecated. Currently there is almost no way to write a moderately complex MR job that doesn't spew deprecation warnings. That is false in 0.21. -- Owen
Re: Un-deprecate the old MapReduce API?
+1 On Wed, Apr 21, 2010 at 2:24 PM, Tom White t...@cloudera.com wrote: The old MapReduce API in org.apache.hadoop.mapred was deprecated in the 0.20 release series when the new (Context Objects) MapReduce API was added in org.apache.hadoop.mapreduce. Unfortunately, the new API was not complete in 0.20 and most users stayed with the old API. This has led to the confusing situation where the old API is generally recommended, even though it is deprecated. To remedy this situation I suggest that we remove deprecations from the old API in 0.20 and trunk, and mark the new API as Evolving (see MAPREDUCE-1623 for the latter). This would mean a few things: * The next 0.20 release would have a non-deprecated old API. * The forthcoming 0.21 release would have a Stable (non-deprecated) old API, and a Evolving new API. * For some pre-1.0 release (perhaps 0.22), the old API could be deprecated again, and the new API marked as Stable. * In the 1.0 release it would be possible to remove the old API. Thoughts? Tom
Re: Un-deprecate the old MapReduce API?
Owen O'Malley wrote: 0.21: new core api should be stable except for Job and Cluster new library code should be evolving -1 for removing the deprecation, we need to The Job API is central to the new API, no? Should we encourage applications to move to the new API if it's not entirely stable? Can you provide a more detailed rationale for deprecating the only fully-stable API? Thanks, Doug
Re: Un-deprecate the old MapReduce API?
+1 (mainly to avoid confusing new comers as Eric Sammer correctly indicated). On 4/22/2010 9:15 AM, Doug Cutting wrote: Owen O'Malley wrote: 0.21: new core api should be stable except for Job and Cluster new library code should be evolving -1 for removing the deprecation, we need to The Job API is central to the new API, no? Should we encourage applications to move to the new API if it's not entirely stable? Can you provide a more detailed rationale for deprecating the only fully-stable API? Thanks, Doug
Re: Un-deprecate the old MapReduce API?
+1 for undeprecating. I think it's just being pragmatic. Arun On Apr 21, 2010, at 2:26 PM, Tom White t...@cloudera.com wrote: The old MapReduce API in org.apache.hadoop.mapred was deprecated in the 0.20 release series when the new (Context Objects) MapReduce API was added in org.apache.hadoop.mapreduce. Unfortunately, the new API was not complete in 0.20 and most users stayed with the old API. This has led to the confusing situation where the old API is generally recommended, even though it is deprecated. To remedy this situation I suggest that we remove deprecations from the old API in 0.20 and trunk, and mark the new API as Evolving (see MAPREDUCE-1623 for the latter). This would mean a few things: * The next 0.20 release would have a non-deprecated old API. * The forthcoming 0.21 release would have a Stable (non-deprecated) old API, and a Evolving new API. * For some pre-1.0 release (perhaps 0.22), the old API could be deprecated again, and the new API marked as Stable. * In the 1.0 release it would be possible to remove the old API. Thoughts? Tom
Re: Un-deprecate the old MapReduce API?
Speaking for one power user (Pig) that did move to the new APIs, moving that interface to evolving is a little unsettling. Is there a feel for how much the new API is going to change? Alan. On Apr 21, 2010, at 2:24 PM, Tom White wrote: The old MapReduce API in org.apache.hadoop.mapred was deprecated in the 0.20 release series when the new (Context Objects) MapReduce API was added in org.apache.hadoop.mapreduce. Unfortunately, the new API was not complete in 0.20 and most users stayed with the old API. This has led to the confusing situation where the old API is generally recommended, even though it is deprecated. To remedy this situation I suggest that we remove deprecations from the old API in 0.20 and trunk, and mark the new API as Evolving (see MAPREDUCE-1623 for the latter). This would mean a few things: * The next 0.20 release would have a non-deprecated old API. * The forthcoming 0.21 release would have a Stable (non-deprecated) old API, and a Evolving new API. * For some pre-1.0 release (perhaps 0.22), the old API could be deprecated again, and the new API marked as Stable. * In the 1.0 release it would be possible to remove the old API. Thoughts? Tom
Re: Un-deprecate the old MapReduce API?
+1 for removing deprecation in 0.20. +0 for removing deprecation in 0.21 Thanks Amareshwari On 4/22/10 7:25 PM, Owen O'Malley owen.omal...@gmail.com wrote: On the various pieces, I think: 0.20: -0 for removing the deprecation, +1 for improving the deprecation message with links to the corresponding class. 0.21: new core api should be stable except for Job and Cluster new library code should be evolving -1 for removing the deprecation, we need to 0.22: all of the new api should be stable and the old api deprecated. Currently there is almost no way to write a moderately complex MR job that doesn't spew deprecation warnings. That is false in 0.21. -- Owen
