Phil,
What is the "Schema Write Strategy" set to on the AvroRecordSetWriter?
-Bryan
On Tue, Sep 5, 2017 at 10:23 AM, <philippe.gib...@orange.com> wrote:
> Hello Joe
>
> This json content
> {"type":"room","id":"room11","attributes":{"position":"47.100,3.246","surface":223,"norme":"NF"}}
>
> is the input of the PublishKafkaRecord_0_10 processor. This processor have
> among others the following properties set :
>
> Record Reader : JsonTreeReader --> pointing to the ConfluentSchemaRegistry
> Record Writer : AvroRecordSetWriter -> pointing to the
> confluentSchemaRegistry
>
>
> I trace the call to confluent registry and I see the get on the subject (
> Confluent registry side)
> And I also see the binary content on a consumer shell console
> (kafka-console-consumer )
> ---------------
> room
> room1147.100,3.246▒NF
> -------------------------------
>
> Thanks
> Phil
>
> -----Original Message-----
> From: Joe Witt [mailto:joe.w...@gmail.com]
> Sent: mardi 5 septembre 2017 15:26
> To: users@nifi.apache.org
> Subject: Re: Nifi kafka ser/deser problem
>
> Hello
>
> Can you share the details of how you're serializing the data? We'll need to
> understand the configuration of the record writer of the publish kafka
> process to help discuss about reading it back (be that in nifi or otherwise).
>
> Since we support multiple modes of encoding the schema information this is
> the most likely source of difficulty.
>
> Thanks
> Joe
>
> On Tue, Sep 5, 2017 at 8:53 AM, <philippe.gib...@orange.com> wrote:
>> Hello, My SW environment à Nifi 1.4 compiled from sources , on Ubuntu
>> 14.04 and a running Confluent 3.3 Platform (running Confluent
>> Registry and kafkaStreams )
>>
>> I have a process group with a PublishKafkaRecord_0_10 that serialize a
>> record ( with 3 fields)
>>
>> And another process group with a ConsumeKafkaRecord_0_10 that
>> deserialise the record . that works nicely with reader/writer and with the
>> Confluent
>> Registry storing the Avro MetaData.
>>
>> I do not have ser/deser problems because I think Nifi does both jobs
>> ……
>>
>>
>>
>> But I got problems of deserialization if I put some java Kafka client in
>> the game … ( try to consume from Confluent Kafka Streams )
>>
>> I got some deserialization exception :
>>
>> org.apache.kafka.common.errors.SerializationException: Error
>> deserializing Avro message for id -1
>>
>>
>>
>> Any trick to force another serialization ( nifi side) or
>> deserialization ( confluent side ) ?
>>
>>
>>
>> Best
>>
>> Phil
>>
>>
>>
>> ______________________________________________________________________
>> ___________________________________________________
>>
>> Ce message et ses pieces jointes peuvent contenir des informations
>> confidentielles ou privilegiees et ne doivent donc pas etre diffuses,
>> exploites ou copies sans autorisation. Si vous avez recu ce message
>> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi
>> que les pieces jointes. Les messages electroniques etant susceptibles
>> d'alteration, Orange decline toute responsabilite si ce message a ete
>> altere, deforme ou falsifie. Merci.
>>
>> This message and its attachments may contain confidential or
>> privileged information that may be protected by law; they should not
>> be distributed, used or copied without authorisation.
>> If you have received this email in error, please notify the sender and
>> delete this message and its attachments.
>> As emails may be altered, Orange is not liable for messages that have
>> been modified, changed or falsified.
>> Thank you.
>
> _________________________________________________________________________________________________________________________
>
> Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu
> ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration,
> Orange decline toute responsabilite si ce message a ete altere, deforme ou
> falsifie. Merci.
>
> This message and its attachments may contain confidential or privileged
> information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and delete
> this message and its attachments.
> As emails may be altered, Orange is not liable for messages that have been
> modified, changed or falsified.
> Thank you.
>
