Maybe I've not been explicit enough. I've found a solution there and applied it. I just share my code :-)
2016-07-06 13:47 GMT+02:00 Lukasz Cwik <[email protected]>: > To fix your issues with the coder reference, you'll want to apply similar > logic as WithKeys.java > <https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/WithKeys.java#L114> > which > sets the coder on the output PCollection before returning it to the user. > > You'll want to get the output type of the simple function and build a KV > coder from it. The issue is about having a reliable way to get the output > type of the simple function. > > > On Tue, Jul 5, 2016 at 9:41 AM, Pawel Szczur <[email protected]> > wrote: > >> Hi, >> >> Often I encounter a pattern: one gets a big data item, uses some part of >> it as key, another as value. Thus I've created simple helper PTransform: >> https://gist.github.com/orian/265e5494e61de931bbfa885ac60eb539 >> >> This allows me to reuse given function in manu places, sometimes as key >> producer and another time as value producer. >> >> The usage is super simple: >> PCollection<KV<String, String>> out = input.apply( >> MakeKV.of(new SimpleFn(), new Simple2Fn())); >> >> In abstract it takes PCollection<V> and returns PCollection<KV<K,V1>>., >> >> BTW I had problems with: Cannot provide a Coder for type variable T >> because the actual type is unknown due to erasure >> >> I've found: >> 1. >> http://stackoverflow.com/questions/32591914/making-transformations-in-dataflow-generic >> 2. >> http://stackoverflow.com/questions/32591914/making-transformations-in-dataflow-generic >> but neither really made it easy to fix (I've been testing the MakeKV with >> built in Integer type/coder). >> >> I've found a solution in WithKeys.java in SDK. >> >> Hope it helps someone. >> >> Cheers, Pawel >> > >
