Hi again, Sorry for bumping this thread but nobody really came with insight.
Should I be defining my own coders for my objects or is it common practice to use the AvroCoder or maybe some other coder? Best regards, Augusto On 2019/03/21 07:35:07, [email protected] <[email protected]> wrote: > Hi> > > I am trying out Beam to do some data aggregations. Many of the inputs/outputs > of my transforms are complex objects (not super complex, but containing > Maps/Lists/Sets sometimes) so when I was prompted to defined a coder to these > objects I added the annotation @DefaultCoder(AvroCoder.class) and things > worked in my development environment.> > > Now that I am trying to run in on "real" data I notice that after I deployed > it to a spark runner and looking at some thread dumps, many of the threads > were blocked on the following method on the Avro library > (ReflectData.getAccessorsFor). So my question is, did I do the wrong thing by > using the AvroCoder or is there some other coder that easily can solve my > problem?> > > Best regards,> > Augusto> > > > >
