Hi,
On Thu, Mar 1, 2012 at 00:07, Robert Evans wrote:
> Sorry it has taken me so long to respond. Today has been a very crazy
> day.
>
No worries.
> I am just guessing what your algorithm is for auto-complete.
>
What we have has a lot more features. Yet the basic idea of what we have is
si
Niels,
Sorry it has taken me so long to respond. Today has been a very crazy day.
I am just guessing what your algorithm is for auto-complete. I really don't
know so I will just design a back of the envelope one myself as a starting
point. My guess is that you have a few map/reduce jobs. Th
Robert,
On Tue, Feb 28, 2012 at 23:28, Robert Evans wrote:
> I am not sure I can help with that unless I know better what “a special
> distribution” means.
>
The thing is that this application is a "Auto Complete" feature that has a
key that is "the letters that have been typed so far".
Now fo
Niels,
I am not sure I can help with that unless I know better what "a special
distribution" means. Unless you are doing a massive amount of processing in
your reducer having a partition that is only close to balancing the
distribution is a big win over all of the other options that put the d
Hi Robert,
On Tue, Feb 28, 2012 at 21:41, Robert Evans wrote:
> I would recommend that you do what terrasort does and use a different
> partitioner, to ensure that all keys within a given range will go to a
> single reducer. If your partitioner is set up correctly then all you have
> to do is
I would recommend that you do what terrasort does and use a different
partitioner, to ensure that all keys within a given range will go to a single
reducer. If your partitioner is set up correctly then all you have to do is to
concatenate the files together, if you even need to do that.
Look a
