Great ! I'm writing the same kind of job, very helpfull. Thanks
2013/5/28 Josh Wills <[email protected]> > The Avro-based PTypeFamily has support for reflection-based serialization > of simple types. There's an example in the testAvroReflects() test method > in the page rank integration test: > > > https://github.com/apache/crunch/blob/master/crunch-core/src/it/java/org/apache/crunch/PageRankIT.java > > There are some limits on what you can serialize via Avro reflection (i.e., > you need to keep the fields of the class to primitives and simple > array/collection types), and you need to be sure to include a no-arg > constructor for the class. > > > On Tue, May 28, 2013 at 12:02 PM, Quentin Ambard <[email protected] > > wrote: > >> Hi, >> I'd like to know if there is an example of a paralleleDo that emit a >> string has key, but a more complex object as value, for example the >> following class : >> >> StringAndInteger { >> String myString; >> Integer myInteger; >> } >> >> If I'm not wrong the signature of the function should be something like >> this (I'm reading the values from hbase): >> >> parallelDo(String name, DoFn<Pair<ImmutableBytesWritable, >> Result>, Pair<String, StringAndInteger>> doFn, WritableTableType<String, >> StringAndInteger> type); >> >> I have to say I'm a bit lost with the WritableTypes of the parallelDo >> PTableType, and a nice example would be very welcome ! >> >> Thanks >> >> >> -- >> Quentin Ambard >> > > > > -- > Director of Data Science > Cloudera <http://www.cloudera.com> > Twitter: @josh_wills <http://twitter.com/josh_wills> > -- Quentin Ambard
