subject:"published by PublishKafkaRecord_0

Re: published by PublishKafkaRecord_0_10 doesn't embed schema.

2018-03-26 Thread Joe Witt

There are no community plans to do a 1.4.1 release at this time.


Thanks

On Mon, Mar 26, 2018 at 11:28 AM, Milan Das  wrote:
> Hi Bryan,
> We are using NIFI 1.4.0. Can we backport this fix to NIFI 1.4?
>
> Thanks,
> Milan Das
>
> On 3/26/18, 11:26 AM, "Bryan Bende"  wrote:
>
> Hello,
>
> What version of NiFi are you using?
>
> This should be fixed in 1.5.0:
>
> https://issues.apache.org/jira/browse/NIFI-4639
>
> Thanks,
>
> Bryan
>
>
> On Sun, Mar 25, 2018 at 6:45 PM, Milan Das  wrote:
> > Hello Nifi Users,
> >
> > Apparently, it seems like PublishKafkaRecord_0_10 doesn't embed schema 
> even if it Avro Record writer is configured with “Embed Avro Schema”.
> >
> > I have seen the following post from Bryan Brende.  Wondering if it is a 
> known issue or if I am missing anything here.
> >
> >
> >
> > 
> https://community.hortonworks.com/questions/110652/cant-consume-avro-messages-whcih-are-published-by.html
> >
> >
> >
> >
> >
> > This is how message looks in Kafka-console-consumer, when published 
> using “PublishKafkaRecord_0_11”
> >
> >
> >
> >
> >
> >  $ bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 
> --topic test --from-beginning
> >
> >
> >
> > 
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�zpA
> >
> > 
> �=h*�p��l
> >
> > 
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$+�=�ت��;�.Y7
> >
> > 
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�D�p��"B��
>r0
> >
> > 
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$ekLl�;]�,Y�͙�
> >
> > 
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�z��klŤ�1�'�z�
> >
> > 
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$���ξu��5�V}>�_
> >
> > 
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$��=%��VbK�
> >
> > 
> ��'~���X�controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$��0
> >
> >
> >
> >
> >
> > When I publish the message using my java class output from console 
> consumer prints the avro schema.
> >
> >
> >
> > 
> Objavro.schema�P{"type":"record","name":"ActiveDirectoryRecord","namespace":"com..schema","doc":"for
>  more info, refer to 
> http://docs.splunk.com/Documentation/CIM/4.2.0/User/Resource","fields":[{"name":"action","type":"string","doc":"The
>  action performed on the 
> resource."},{"name":"dest","type":"string","doc":"The target involved in the 
> authentication. May be aliased from more specific fields, such as dest_host, 
> dest_ip, or 
> dest_nt_host."},{"name":"signature_id","type":"int","doc":"Description of the 
> change performed (integer)"},{"name":"time","type":"string","doc":"ISO 8601 
> timestamp of the eventl  >,{"name":"privileges","type":["null",{"type":"array","items":"string"}],"doc":"The
>  list of privileges associated with a Privilege Escalation 
> event","default":null},{"name":"subcode","type":["null","string"],"doc":"The 
> error subcode for 
> auth;��d��g%�z�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00usedid1Pz��
> >
> >
> >
> >
> >
> > Regards,
> >
> > Milan Das
> >
> >
>
>
>

Re: published by PublishKafkaRecord_0_10 doesn't embed schema.

2018-03-26 Thread Bryan Bende

You might be able to get the nifi-kafka-0-10-nar from 1.5.0 and run it in 1.4.0.

On Mon, Mar 26, 2018 at 11:28 AM, Milan Das  wrote:
> Hi Bryan,
> We are using NIFI 1.4.0. Can we backport this fix to NIFI 1.4?
>
> Thanks,
> Milan Das
>
> On 3/26/18, 11:26 AM, "Bryan Bende"  wrote:
>
> Hello,
>
> What version of NiFi are you using?
>
> This should be fixed in 1.5.0:
>
> https://issues.apache.org/jira/browse/NIFI-4639
>
> Thanks,
>
> Bryan
>
>
> On Sun, Mar 25, 2018 at 6:45 PM, Milan Das  wrote:
> > Hello Nifi Users,
> >
> > Apparently, it seems like PublishKafkaRecord_0_10 doesn't embed schema 
> even if it Avro Record writer is configured with “Embed Avro Schema”.
> >
> > I have seen the following post from Bryan Brende.  Wondering if it is a 
> known issue or if I am missing anything here.
> >
> >
> >
> > 
> https://community.hortonworks.com/questions/110652/cant-consume-avro-messages-whcih-are-published-by.html
> >
> >
> >
> >
> >
> > This is how message looks in Kafka-console-consumer, when published 
> using “PublishKafkaRecord_0_11”
> >
> >
> >
> >
> >
> >  $ bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 
> --topic test --from-beginning
> >
> >
> >
> > 
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�zpA
> >
> > 
> �=h*�p��l
> >
> > 
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$+�=�ت��;�.Y7
> >
> > 
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�D�p��"B��
>r0
> >
> > 
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$ekLl�;]�,Y�͙�
> >
> > 
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�z��klŤ�1�'�z�
> >
> > 
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$���ξu��5�V}>�_
> >
> > 
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$��=%��VbK�
> >
> > 
> ��'~���X�controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$��0
> >
> >
> >
> >
> >
> > When I publish the message using my java class output from console 
> consumer prints the avro schema.
> >
> >
> >
> > 
> Objavro.schema�P{"type":"record","name":"ActiveDirectoryRecord","namespace":"com..schema","doc":"for
>  more info, refer to 
> http://docs.splunk.com/Documentation/CIM/4.2.0/User/Resource","fields":[{"name":"action","type":"string","doc":"The
>  action performed on the 
> resource."},{"name":"dest","type":"string","doc":"The target involved in the 
> authentication. May be aliased from more specific fields, such as dest_host, 
> dest_ip, or 
> dest_nt_host."},{"name":"signature_id","type":"int","doc":"Description of the 
> change performed (integer)"},{"name":"time","type":"string","doc":"ISO 8601 
> timestamp of the eventl  >,{"name":"privileges","type":["null",{"type":"array","items":"string"}],"doc":"The
>  list of privileges associated with a Privilege Escalation 
> event","default":null},{"name":"subcode","type":["null","string"],"doc":"The 
> error subcode for 
> auth;��d��g%�z�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00usedid1Pz��
> >
> >
> >
> >
> >
> > Regards,
> >
> > Milan Das
> >
> >
>
>
>

Re: published by PublishKafkaRecord_0_10 doesn't embed schema.

2018-03-26 Thread Milan Das

Hi Bryan,
We are using NIFI 1.4.0. Can we backport this fix to NIFI 1.4?

Thanks,
Milan Das

On 3/26/18, 11:26 AM, "Bryan Bende"  wrote:

Hello,

What version of NiFi are you using?

This should be fixed in 1.5.0:

https://issues.apache.org/jira/browse/NIFI-4639

Thanks,

Bryan


On Sun, Mar 25, 2018 at 6:45 PM, Milan Das  wrote:
> Hello Nifi Users,
>
> Apparently, it seems like PublishKafkaRecord_0_10 doesn't embed schema 
even if it Avro Record writer is configured with “Embed Avro Schema”.
>
> I have seen the following post from Bryan Brende.  Wondering if it is a 
known issue or if I am missing anything here.
>
>
>
> 
https://community.hortonworks.com/questions/110652/cant-consume-avro-messages-whcih-are-published-by.html
>
>
>
>
>
> This is how message looks in Kafka-console-consumer, when published using 
“PublishKafkaRecord_0_11”
>
>
>
>
>
>  $ bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 
--topic test --from-beginning
>
>
>
> 
�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�zpA
>
>   
  �=h*�p��l
>
> 
�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$+�=�ت��;�.Y7
>
> 
�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�D�p��"B��
   r0
>
> 
�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$ekLl�;]�,Y�͙�
>
> 
�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�z��klŤ�1�'�z�
>
> 
�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$���ξu��5�V}>�_
>
> 
�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$��=%��VbK�
>
> 
��'~���X�controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$��0
>
>
>
>
>
> When I publish the message using my java class output from console 
consumer prints the avro schema.
>
>
>
> 
Objavro.schema�P{"type":"record","name":"ActiveDirectoryRecord","namespace":"com..schema","doc":"for
 more info, refer to 
http://docs.splunk.com/Documentation/CIM/4.2.0/User/Resource","fields":[{"name":"action","type":"string","doc":"The
 action performed on the resource."},{"name":"dest","type":"string","doc":"The 
target involved in the authentication. May be aliased from more specific 
fields, such as dest_host, dest_ip, or 
dest_nt_host."},{"name":"signature_id","type":"int","doc":"Description of the 
change performed (integer)"},{"name":"time","type":"string","doc":"ISO 8601 
timestamp of the eventl ,{"name":"privileges","type":["null",{"type":"array","items":"string"}],"doc":"The
 list of privileges associated with a Privilege Escalation 
event","default":null},{"name":"subcode","type":["null","string"],"doc":"The 
error subcode for 
auth;��d��g%�z�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00usedid1Pz��
>
>
>
>
>
> Regards,
>
> Milan Das
>
>

Re: published by PublishKafkaRecord_0_10 doesn't embed schema.

2018-03-26 Thread Mark Payne

Milan,

What version of NiFi are you running? This sounds like NIFI-4639 [1], which was 
resolved in version 1.5.0.

Thanks
-Mark


[1] https://issues.apache.org/jira/browse/NIFI-4639


On Mar 25, 2018, at 6:45 PM, Milan Das 
> wrote:

Hello Nifi Users,

Apparently, it seems like PublishKafkaRecord_0_10 doesn't embed schema even if 
it Avro Record writer is configured with “Embed Avro Schema”.

I have seen the following post from Bryan Brende.  Wondering if it is a known 
issue or if I am missing anything here.



https://community.hortonworks.com/questions/110652/cant-consume-avro-messages-whcih-are-published-by.html





This is how message looks in Kafka-console-consumer, when published using 
“PublishKafkaRecord_0_11”





$ bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test 
--from-beginning



�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�zpA


�=h*�p��l

�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$+�=�ت��;�.Y7

�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�D�p��"B��
   r0

�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$ekLl�;]�,Y�͙�

�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�z��klŤ�1�'�z�

�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$���ξu��5�V}>�_

�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$��=%��VbK�

��'~���X�controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$��0





When I publish the message using my java class output from console consumer 
prints the avro schema.



Objavro.schema�P{"type":"record","name":"ActiveDirectoryRecord","namespace":"com..schema","doc":"for
 more info, refer to 
http://docs.splunk.com/Documentation/CIM/4.2.0/User/Resource","fields":[{"name":"action","type":"string","doc":"The
 action performed on the resource."},{"name":"dest","type":"string","doc":"The 
target involved in the authentication. May be aliased from more specific 
fields, such as dest_host, dest_ip, or 
dest_nt_host."},{"name":"signature_id","type":"int","doc":"Description of the 
change performed (integer)"},{"name":"time","type":"string","doc":"ISO 8601 
timestamp of the eventl ,{"name":"privileges","type":["null",{"type":"array","items":"string"}],"doc":"The
 list of privileges associated with a Privilege Escalation 
event","default":null},{"name":"subcode","type":["null","string"],"doc":"The 
error subcode for 
auth;��d��g%�z�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00usedid1Pz��





Regards,

Milan Das

Re: published by PublishKafkaRecord_0_10 doesn't embed schema.

2018-03-26 Thread Bryan Bende

Hello,

What version of NiFi are you using?

This should be fixed in 1.5.0:

https://issues.apache.org/jira/browse/NIFI-4639

Thanks,

Bryan


On Sun, Mar 25, 2018 at 6:45 PM, Milan Das  wrote:
> Hello Nifi Users,
>
> Apparently, it seems like PublishKafkaRecord_0_10 doesn't embed schema even 
> if it Avro Record writer is configured with “Embed Avro Schema”.
>
> I have seen the following post from Bryan Brende.  Wondering if it is a known 
> issue or if I am missing anything here.
>
>
>
> https://community.hortonworks.com/questions/110652/cant-consume-avro-messages-whcih-are-published-by.html
>
>
>
>
>
> This is how message looks in Kafka-console-consumer, when published using 
> “PublishKafkaRecord_0_11”
>
>
>
>
>
>  $ bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic 
> test --from-beginning
>
>
>
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�zpA
>
>   
>   �=h*�p��l
>
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$+�=�ت��;�.Y7
>
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�D�p��"B��
>r0
>
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$ekLl�;]�,Y�͙�
>
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�z��klŤ�1�'�z�
>
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$���ξu��5�V}>�_
>
> �SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$��=%��VbK�
>
> ��'~���X�controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$��0
>
>
>
>
>
> When I publish the message using my java class output from console consumer 
> prints the avro schema.
>
>
>
> Objavro.schema�P{"type":"record","name":"ActiveDirectoryRecord","namespace":"com..schema","doc":"for
>  more info, refer to 
> http://docs.splunk.com/Documentation/CIM/4.2.0/User/Resource","fields":[{"name":"action","type":"string","doc":"The
>  action performed on the 
> resource."},{"name":"dest","type":"string","doc":"The target involved in the 
> authentication. May be aliased from more specific fields, such as dest_host, 
> dest_ip, or 
> dest_nt_host."},{"name":"signature_id","type":"int","doc":"Description of the 
> change performed (integer)"},{"name":"time","type":"string","doc":"ISO 8601 
> timestamp of the eventl  >,{"name":"privileges","type":["null",{"type":"array","items":"string"}],"doc":"The
>  list of privileges associated with a Privilege Escalation 
> event","default":null},{"name":"subcode","type":["null","string"],"doc":"The 
> error subcode for 
> auth;��d��g%�z�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00usedid1Pz��
>
>
>
>
>
> Regards,
>
> Milan Das
>
>

published by PublishKafkaRecord_0_10 doesn't embed schema.

2018-03-25 Thread Milan Das

Hello Nifi Users,

Apparently, it seems like PublishKafkaRecord_0_10 doesn't embed schema even if 
it Avro Record writer is configured with “Embed Avro Schema”.

I have seen the following post from Bryan Brende.  Wondering if it is a known 
issue or if I am missing anything here.

 

https://community.hortonworks.com/questions/110652/cant-consume-avro-messages-whcih-are-published-by.html

 

 

This is how message looks in Kafka-console-consumer, when published using 
“PublishKafkaRecord_0_11”

 

 

 $ bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test 
--from-beginning

 

�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�zpA

    
�=h*�p��l

�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$+�=�ت��;�.Y7

�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�D�p��"B��
   r0

�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$ekLl�;]�,Y�͙�

�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$�z��klŤ�1�'�z�

�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$���ξu��5�V}>�_

�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$��=%��VbK�

��'~���X�controller1.ad.interset.com�H22018-03-15T09:07:04-04:00CONTROLLER1$��0

 

 

When I publish the message using my java class output from console consumer 
prints the avro schema.

 

Objavro.schema�P{"type":"record","name":"ActiveDirectoryRecord","namespace":"com..schema","doc":"for
 more info, refer to 
http://docs.splunk.com/Documentation/CIM/4.2.0/User/Resource","fields":[{"name":"action","type":"string","doc":"The
 action performed on the resource."},{"name":"dest","type":"string","doc":"The 
target involved in the authentication. May be aliased from more specific 
fields, such as dest_host, dest_ip, or 
dest_nt_host."},{"name":"signature_id","type":"int","doc":"Description of the 
change performed (integer)"},{"name":"time","type":"string","doc":"ISO 8601 
timestamp of the eventl ,{"name":"privileges","type":["null",{"type":"array","items":"string"}],"doc":"The
 list of privileges associated with a Privilege Escalation 
event","default":null},{"name":"subcode","type":["null","string"],"doc":"The 
error subcode for 
auth;��d��g%�z�SUCCESS6controller1.ad.interset.com�H22018-03-15T09:07:04-04:00usedid1Pz��

 

 

Regards,

Milan Das

Re: published by PublishKafkaRecord_0_10 doesn't embed schema.

Re: published by PublishKafkaRecord_0_10 doesn't embed schema.

Re: published by PublishKafkaRecord_0_10 doesn't embed schema.

Re: published by PublishKafkaRecord_0_10 doesn't embed schema.

Re: published by PublishKafkaRecord_0_10 doesn't embed schema.

published by PublishKafkaRecord_0_10 doesn't embed schema.

6 matches

Site Navigation

Mail list logo

Footer information