It is a handy trick, and there's a lot of custom Writables involved in a good MapReduce pipeline, it seems to me. Yes, inside your Writable, use VectorWritable to manage the Vector part.
On Tue, Oct 12, 2010 at 4:23 AM, Lance Norskog <[email protected]> wrote: > I have a M/R project where vectors of two different types are in the > same stream. There . They are not matched 1-to-1, in fact there are > different quantities of each type. The mappers may assign 3 of type A > and 7 of type B to one key. The reducer receives both types and > combines them. > > I'm now using a non-Vector design which adds a "payload" to each > vector, giving the type. This is a really handy trick. In fact I'm > using different payload sets in different stages of a multi-stage M/R > job. > > How would I model this in Mahout? Would I create my own Writeable > tuple with a Vector and a payload object? > > -- > Lance Norskog > [email protected] >
