Re: KafkaSimpleJsonExtractor

2018-04-27 Thread Vicky Kak
kaReporterTest. > STARTED > > > > Gradle suite > Gradle test > > org.apache.gobblin.metrics.reporter.KafkaReporterTest. > PASSED > > > > Gradle suite > Gradle test > > org.apache.gobblin.metrics.reporter.KafkaReporterTest. > STARTED > > >

Re: KafkaSimpleJsonExtractor

2018-04-19 Thread Vicky Kak
> Value is null > > > > > > Did you have a chance to research adding annotation, @Alias (“ > KafkaSimpleJsonExtractor”)? > > > > > > *From:* Vicky Kak [mailto:vicky....@gmail.com] > *Sent:* Saturday, April 14, 2018 12:19 AM > > *To:* user@

RE: KafkaSimpleJsonExtractor

2018-04-19 Thread Kidwell, Jack
ava'] > Value is null Did you have a chance to research adding annotation, @Alias (“KafkaSimpleJsonExtractor”)? From: Vicky Kak [mailto:vicky@gmail.com] Sent: Saturday, April 14, 2018 12:19 AM To: user@gobblin.incubator.apache.org Subject: Re: KafkaSimpleJsonExtractor Yes yo

Re: KafkaSimpleJsonExtractor

2018-04-13 Thread Vicky Kak
); > > > > KafkaRecord record = new KafkaRecord(offset, key, payload); > > > > byte[] decodedRecord = gson.toJson(record).getBytes(CHARSET); > > return decodedRecord; > > } > > > > The architecture diagram leads us to think that the

RE: KafkaSimpleJsonExtractor

2018-04-13 Thread Kidwell, Jack
ByteArrayBasedKafkaRecord. getKeyBytes() and ByteArrayBasedKafkaRecord. getMessageBytes(), these result we seek. Or did we miss something? From: Vicky Kak [mailto:vicky@gmail.com] Sent: Friday, April 13, 2018 12:01 AM To: user@gobblin.incubator.apache.org Subject: Re: KafkaSimpleJsonExtractor Are you not having

Re: KafkaSimpleJsonExtractor

2018-04-12 Thread Vicky Kak
> > We want to extract both key and value from a Kafka message and publish the > combined information to HDFS. Keys contain numeric ids for the strings > contained in values. > > > > Please explain “other subscriber”. Are you proposing a different Kafka > message structure?

RE: KafkaSimpleJsonExtractor

2018-04-12 Thread Kidwell, Jack
eager to learn from you. From: Vicky Kak [mailto:vicky@gmail.com] Sent: Thursday, April 12, 2018 9:45 AM To: user@gobblin.incubator.apache.org Subject: Re: KafkaSimpleJsonExtractor For writing the data to the HDFS you need to take a look at the Writer Implementation, please look for it. T

RE: KafkaSimpleJsonExtractor

2018-04-12 Thread Kidwell, Jack
my attention because it extracts both keys and values and puts them in decodedRecord. From: Vicky Kak [mailto:vicky@gmail.com] Sent: Thursday, April 12, 2018 1:17 AM To: user@gobblin.incubator.apache.org Subject: Re: KafkaSimpleJsonExtractor I am not sure what you are asking for. Do you want