Re: KafkaSimpleJsonExtractor

2018-04-27 Thread Vicky Kak
kaReporterTest. > STARTED > > > > Gradle suite > Gradle test > > org.apache.gobblin.metrics.reporter.KafkaReporterTest. > PASSED > > > > Gradle suite > Gradle test > > org.apache.gobblin.metrics.reporter.KafkaReporterTest. > STARTED > > >

Re: KafkaSimpleJsonExtractor

2018-04-19 Thread Vicky Kak
> Value is null > > > > > > Did you have a chance to research adding annotation, @Alias (“ > KafkaSimpleJsonExtractor”)? > > > > > > *From:* Vicky Kak [mailto:vicky@gmail.com] > *Sent:* Saturday, April 14, 2018 12:19 AM > > *To:* user@

RE: KafkaSimpleJsonExtractor

2018-04-19 Thread Kidwell, Jack
ava'] > Value is null Did you have a chance to research adding annotation, @Alias (“KafkaSimpleJsonExtractor”)? From: Vicky Kak [mailto:vicky@gmail.com] Sent: Saturday, April 14, 2018 12:19 AM To: user@gobblin.incubator.apache.org Subject: Re: KafkaSimpleJsonExtractor Yes yo

Re: KafkaSimpleJsonExtractor

2018-04-13 Thread Vicky Kak
Yes you seem to correct, the code never lies :) Can you try implementing your customised Extractor which will extend KafkaSimpleJsonExtractor and have the Alias annotation too? The other option would be to add the Alias annotation to the KafkaSimpleJsonExtractor, I need to see what implications

RE: KafkaSimpleJsonExtractor

2018-04-13 Thread Kidwell, Jack
: the class must be annotated with @Alias. If class KafkaSimpleJsonExtractor was annotated with an @Alias, I think I could use the UniversalKafaSource along with property, gobblin.source.kafka.extractorType. As you know, KafkaSimpleJsonExtractor. decodeRecord() combines both

Re: KafkaSimpleJsonExtractor

2018-04-12 Thread Vicky Kak
this is done then you can use KafkaSimpleJsonExtractor which was asked at the beginning. So to conclude what I understand is that you don't want the data to be stored in the byte array, you require json (or other format). On Thu, Apr 12, 2018 at 8:09 PM, Kidwell, Jack <jack_kidwel...@comcast.

RE: KafkaSimpleJsonExtractor

2018-04-12 Thread Kidwell, Jack
“new KafkaSimpleExtractor(state)”, and class, KafkaSimpleExtractor, overrides method, decodeRecord, to return just the Kafka message value. return kafkaConsumerRecord.getMessageBytes() Since we want both key and value stored in HDFS, class, KafkaSimpleJsonExtractor, appears to provide

RE: KafkaSimpleJsonExtractor

2018-04-12 Thread Kidwell, Jack
my attention because it extracts both keys and values and puts them in decodedRecord. From: Vicky Kak [mailto:vicky@gmail.com] Sent: Thursday, April 12, 2018 1:17 AM To: user@gobblin.incubator.apache.org Subject: Re: KafkaSimpleJsonExtractor I am not sure what you are asking for. Do you want

KafkaSimpleJsonExtractor

2018-04-11 Thread Kidwell, Jack
Hi, Using gobblin_0.10.0, we want to use module, KafkaSimpleJsonExtractor.java in order to see both kafka record keys and values. How does one configure a job to achieve it?