Hi Daniel, There are ways to do what you are asking for, but are you sure you are using the right framework for the right problem? Hadoop's premise is that Reducer values are not sorted. If you want all values to be concatenated in a sorted order one way is to read in all values in the reducer (memory limitation) and sort the values and concatenate them. This solution is obviously not scalable and defeats the purpose of Hadoop's parallelism.
The other way is outlined here http://sonerbalkir.blogspot.com/2010/01/simulating-secondary-sort-on-values.html where you put the value as part of the mapper output key and then override the partitioner to make all the key-values with the same prefix (not the entire key) to go to the same reducer. So you are essentially using hadoop to sort the keys and then sending them to the same reducer. -Ayon See My Photos on Flickr Also check out my Blog for answers to commonly asked questions. ________________________________ From: Daniel Yehdego <[email protected]> To: [email protected] Sent: Tuesday, September 20, 2011 8:38 AM Subject: RE: Reducer to concatenate string values Hi Ayon, any idea on my previous question? Regards, Daniel T. Yehdego Computational Science Program University of Texas at El Paso, UTEP [email protected] > From: [email protected] > To: [email protected] > Subject: RE: Reducer to concatenate string values > Date: Tue, 20 Sep 2011 06:06:22 +0000 > > > Hi Ayon, > I am using a C executable as my mapper (streaming), but I am not sure how to > use a reducer that concatenates the values from a mapper in order. > > Regards, > > Daniel T. Yehdego > Computational Science Program > University of Texas at El Paso, UTEP > [email protected] > > > Date: Mon, 19 Sep 2011 22:54:46 -0700 > > From: [email protected] > > Subject: Re: Reducer to concatenate string values > > To: [email protected] > > > > What are you using for your map/reduce? Streaming/Java/Pig/Hive? > > > > -Ayon > > See My Photos on Flickr > > Also check out my Blog for answers to commonly asked questions. > > > > > > > > ________________________________ > > From: Daniel Yehdego <[email protected]> > > To: [email protected] > > Sent: Monday, September 19, 2011 10:43 PM > > Subject: Reducer to concatenate string values > > > > > > Good evening, > > I have a certain value output from a mapper and I want to concatenate the > > string outputs using a Reducer (one reducer).But the order of the > > concatenated string values is not in order. How can I use a reducer that > > receives a value from a mapper output and concatenate the strings in order. > > waiting your response and thanks in advance. > > > > Regards, > > > > Daniel T. Yehdego > > Computational Science Program > > University of Texas at El Paso, UTEP > > [email protected] >
