Re: Un-deprecate the old MapReduce API?
+1. Currently there is almost no way to write a moderately complex MR job that doesn't spew deprecation warnings. It serves to endlessly confuse newcomers to Hadoop. On Wed, Apr 21, 2010 at 5:24 PM, Tom White t...@cloudera.com wrote: The old MapReduce API in org.apache.hadoop.mapred was deprecated in the 0.20 release series when the new (Context Objects) MapReduce API was added in org.apache.hadoop.mapreduce. Unfortunately, the new API was not complete in 0.20 and most users stayed with the old API. This has led to the confusing situation where the old API is generally recommended, even though it is deprecated. To remedy this situation I suggest that we remove deprecations from the old API in 0.20 and trunk, and mark the new API as Evolving (see MAPREDUCE-1623 for the latter). This would mean a few things: * The next 0.20 release would have a non-deprecated old API. * The forthcoming 0.21 release would have a Stable (non-deprecated) old API, and a Evolving new API. * For some pre-1.0 release (perhaps 0.22), the old API could be deprecated again, and the new API marked as Stable. * In the 1.0 release it would be possible to remove the old API. Thoughts? Tom -- Eric Sammer phone: +1-917-287-2675 twitter: esammer data: www.cloudera.com
[jira] Created: (MAPREDUCE-1721) Default output format for RandomTextWriter should be TextOutputFormat
Default output format for RandomTextWriter should be TextOutputFormat - Key: MAPREDUCE-1721 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1721 Project: Hadoop Map/Reduce Issue Type: Bug Components: examples Reporter: Amareshwari Sriramadasu Priority: Minor Fix For: 0.22.0 Default output format for RandomTextWriter should be TextOutputFormat, currently output format is SequenceFileOutputFormat. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (MAPREDUCE-1722) contrib/index - Upgrade to new Hadoop and Lucene API
contrib/index - Upgrade to new Hadoop and Lucene API Key: MAPREDUCE-1722 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1722 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/index Affects Versions: 0.20.2 Reporter: Renaud Delbru Priority: Minor contrib/index is still using the old hadoop API. In addition, lucene 3.x has also a new API. The contrib/index should be updated and based on these new APIs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (MAPREDUCE-1723) Capacity Scheduler should allow configuration of Map Reduce task slots independantly
Capacity Scheduler should allow configuration of Map Reduce task slots independantly --- Key: MAPREDUCE-1723 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1723 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.20.1 Environment: all Reporter: Subramaniam Krishnan Fix For: 0.20.3 The JobTracker UI shows both Failed/Killed Jobs as Failed. The Killed job status has been separated from Failed as part of HADOOP-3924, so the UI needs to be updated to reflect the same. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Un-deprecate the old MapReduce API?
On the various pieces, I think: 0.20: -0 for removing the deprecation, +1 for improving the deprecation message with links to the corresponding class. 0.21: new core api should be stable except for Job and Cluster new library code should be evolving -1 for removing the deprecation, we need to 0.22: all of the new api should be stable and the old api deprecated. Currently there is almost no way to write a moderately complex MR job that doesn't spew deprecation warnings. That is false in 0.21. -- Owen
Re: Un-deprecate the old MapReduce API?
+1 On Wed, Apr 21, 2010 at 2:24 PM, Tom White t...@cloudera.com wrote: The old MapReduce API in org.apache.hadoop.mapred was deprecated in the 0.20 release series when the new (Context Objects) MapReduce API was added in org.apache.hadoop.mapreduce. Unfortunately, the new API was not complete in 0.20 and most users stayed with the old API. This has led to the confusing situation where the old API is generally recommended, even though it is deprecated. To remedy this situation I suggest that we remove deprecations from the old API in 0.20 and trunk, and mark the new API as Evolving (see MAPREDUCE-1623 for the latter). This would mean a few things: * The next 0.20 release would have a non-deprecated old API. * The forthcoming 0.21 release would have a Stable (non-deprecated) old API, and a Evolving new API. * For some pre-1.0 release (perhaps 0.22), the old API could be deprecated again, and the new API marked as Stable. * In the 1.0 release it would be possible to remove the old API. Thoughts? Tom
Re: Un-deprecate the old MapReduce API?
Owen O'Malley wrote: 0.21: new core api should be stable except for Job and Cluster new library code should be evolving -1 for removing the deprecation, we need to The Job API is central to the new API, no? Should we encourage applications to move to the new API if it's not entirely stable? Can you provide a more detailed rationale for deprecating the only fully-stable API? Thanks, Doug
Re: Un-deprecate the old MapReduce API?
+1 (mainly to avoid confusing new comers as Eric Sammer correctly indicated). On 4/22/2010 9:15 AM, Doug Cutting wrote: Owen O'Malley wrote: 0.21: new core api should be stable except for Job and Cluster new library code should be evolving -1 for removing the deprecation, we need to The Job API is central to the new API, no? Should we encourage applications to move to the new API if it's not entirely stable? Can you provide a more detailed rationale for deprecating the only fully-stable API? Thanks, Doug
Re: Un-deprecate the old MapReduce API?
+1 for undeprecating. I think it's just being pragmatic. Arun On Apr 21, 2010, at 2:26 PM, Tom White t...@cloudera.com wrote: The old MapReduce API in org.apache.hadoop.mapred was deprecated in the 0.20 release series when the new (Context Objects) MapReduce API was added in org.apache.hadoop.mapreduce. Unfortunately, the new API was not complete in 0.20 and most users stayed with the old API. This has led to the confusing situation where the old API is generally recommended, even though it is deprecated. To remedy this situation I suggest that we remove deprecations from the old API in 0.20 and trunk, and mark the new API as Evolving (see MAPREDUCE-1623 for the latter). This would mean a few things: * The next 0.20 release would have a non-deprecated old API. * The forthcoming 0.21 release would have a Stable (non-deprecated) old API, and a Evolving new API. * For some pre-1.0 release (perhaps 0.22), the old API could be deprecated again, and the new API marked as Stable. * In the 1.0 release it would be possible to remove the old API. Thoughts? Tom
Re: Un-deprecate the old MapReduce API?
Speaking for one power user (Pig) that did move to the new APIs, moving that interface to evolving is a little unsettling. Is there a feel for how much the new API is going to change? Alan. On Apr 21, 2010, at 2:24 PM, Tom White wrote: The old MapReduce API in org.apache.hadoop.mapred was deprecated in the 0.20 release series when the new (Context Objects) MapReduce API was added in org.apache.hadoop.mapreduce. Unfortunately, the new API was not complete in 0.20 and most users stayed with the old API. This has led to the confusing situation where the old API is generally recommended, even though it is deprecated. To remedy this situation I suggest that we remove deprecations from the old API in 0.20 and trunk, and mark the new API as Evolving (see MAPREDUCE-1623 for the latter). This would mean a few things: * The next 0.20 release would have a non-deprecated old API. * The forthcoming 0.21 release would have a Stable (non-deprecated) old API, and a Evolving new API. * For some pre-1.0 release (perhaps 0.22), the old API could be deprecated again, and the new API marked as Stable. * In the 1.0 release it would be possible to remove the old API. Thoughts? Tom
Re: Un-deprecate the old MapReduce API?
+1 for removing deprecation in 0.20. +0 for removing deprecation in 0.21 Thanks Amareshwari On 4/22/10 7:25 PM, Owen O'Malley owen.omal...@gmail.com wrote: On the various pieces, I think: 0.20: -0 for removing the deprecation, +1 for improving the deprecation message with links to the corresponding class. 0.21: new core api should be stable except for Job and Cluster new library code should be evolving -1 for removing the deprecation, we need to 0.22: all of the new api should be stable and the old api deprecated. Currently there is almost no way to write a moderately complex MR job that doesn't spew deprecation warnings. That is false in 0.21. -- Owen
