Re: Ingesting golden gate messages to Hbase using Nifi

2018-11-05 Thread Faisal Durrani
Hi Boris, Thank you for your reply. Let me try explaining my data flow in detail. I am receiving the GG transaction as JSON format through Kafka so I can only use the fields provided by the Kafka handler of GG ( Json plug-gable format). I think you meant RBA value instead of rbc. I don't think

HiveQL processors in Nifi 1.7.0

2018-11-05 Thread jeff whybark
Hello, We recently upgraded from Nifi 1.4 to Nifi 1.7.0 and are suddenly having some issues with both the SelectHiveQL and PutHiveQL processors. Currently we are hosting Nifi 1.7.0 in AWS and are connecting to Hive in AWS EMR (Hive Version 2.1.1). Since the upgrade we’ve noticed that the

Re: Ingesting golden gate messages to Hbase using Nifi

2018-11-05 Thread Boris Tyukin
Hi Faisal, I am not Timothy, but you raise an interesting problem we might face soon as well. I did not expect the situation you described and I thought transaction time would be different. Our intent was to use op_ts to enforce order but another option is to use GG rbc value or oracle rowscn

Re: nifi issue with sftp server

2018-11-05 Thread Alfredo De Luca
Hi Joe.. have you had any thought about this? I don't know what else to do. Cheers On Tue, Oct 30, 2018 at 8:18 PM Alfredo De Luca wrote: > Any thought Joe? thank so much > > On Tue, Oct 30, 2018 at 2:28 PM Alfredo De Luca > wrote: > >> I have attached the nifi properties. Also these are more

Ingesting golden gate messages to Hbase using Nifi

2018-11-05 Thread Faisal Durrani
Hi Timothy , Hope you are doing well. We have been using your data flow( https://community.hortonworks.com/content/kbentry/155527/ingesting-golden-gate-records-from-apache-kafka-an.html# ) with slight modifications to store the data in Hbase. To version the rows we have been using Op_ts of golden