Its not only about testing, I will also need to run things against different datasets. I want to reuse as much of the code as possible to load the same data from a file instead of kafka.
Is there a simple way of loading the data from a File using the same conversion classes that I would use to transfrom them when I read them from kafka or do I have to write a new avro deserializer (InputFormat). On Fri, Feb 12, 2016 at 2:06 AM, Gyula Fóra <gyula.f...@gmail.com> wrote: > Hey, > > A very simple thing you could do is to set up a simple kafka producer in a > java program that will feed the data into a topic. This also has the > additional benefit that you are actually testing against kafka. > > Cheers, > Gyula > > Martin Neumann <mneum...@sics.se> ezt írta (időpont: 2016. febr. 12., P, > 0:20): > >> Hej, >> >> I have a stream program reading data from Kafka where the data is in >> avro. I have my own DeserializationSchema to deal with it. >> >> For testing reasons I want to read a dump from hdfs instead, is there a >> way to use the same DeserializationSchema to read from an avro file stored >> on hdfs? >> >> cheers Martin >> >
