Errors when attempting to use timestamp-millis fields with QueryRecord

I have some input avro with some fields using the timestamp-millis logicalType. I've been attempting to use them with QueryRecord to filter, or otherwise operate on the fields, using timestamp operations and I get errors produced. For instance the following SQL queries: SELECT * FROM FLOWFILE

Re: Different NiFi Node sizes within same cluster

Thanks all for the input. So from what I'm gathering, storage differences of around 5 GB (125 GB vs 130 GB) should not cause any problems/load impacts. Larger storage differences could have load impacts. Differences in CPU and RAM could definitely have load impacts. Luckily my older nodes have the

PutS3Object failing when using non-Latin characters in filename

I kept the default for the object key, which is ${filename} and some of our files have non-Latin characters. The error from AWS is: > The request signature we calculated does not match the signature you provided. Check your key and signing method. (Service: Amazon S3; Status Code: 403; Error

QueryRecord and NULLs

I am struggling for an hour now with a very simple thing. I need to add 3 new fields to a record and set them to NULL but it does not work. I tried null instead - same thing. I checked Calcite docs and I do not see anything special about NULL. And I know you can do it in SQL. This works:

Convert Avro to ORC or JSON processor - retaining the data type

Hi, Nifi version 1.7 We have a dataflow that would get data from Oracle database and load into hive tables. Data flow is something like below: GenerateTableFetch -> ExecuteSQL > AvrotoJson/ORC (we tried both) > PutHDFS > ListHDFS> ReplaceTExt (to build load data query form the file) >

ExecuteSQLRecord and timestamps

Hi guys, we just upgraded to 1.9 and I was excited to start using new ExecuteSQLRecord processor. While I was migrating an older flow, that uses ExecuteSQL processor I've noticed that timestamp/date types are coming as integers not strings like before. Also AVRO schema inferred from a database

Re: PutS3Object failing when using non-Latin characters in filename

The fact that the signatures don’t match may indicate some kind of character normalization or encoding difference with the way AWS handles the input. There is an existing Jira for handling filenames with orthographic marks in FetchFile [1]. [1] https://issues.apache.org/jira/browse/NIFI-6051

Re: Different NiFi Node sizes within same cluster

> The last thing I'm looking to understand is what Byran B brought up, do load > balanced connections take into consideration the load of each node? No, load balanced connection doesn't use load of each node to calculate destination currently. As future improvement ideas. We can implement

Re: Convert Avro to ORC or JSON processor - retaining the data type

Hi Ravi, I looked at following links, Hive does support some logical types like timestamp-millis, but not sure if decimal is supported. https://issues.apache.org/jira/browse/HIVE-8131 https://cwiki.apache.org/confluence/display/Hive/AvroSerDe#AvroSerDe-AvrotoHivetypeconversion If treating the

Re: QueryRecord and NULLs

Using NULLIF can be a workaround. I was able to populate new columns with null. SELECT * ,NULLIF(5, 5) as unit_cerner_alias ,NULLIF(5, 5) as room_cerner_alias ,NULLIF(5, 5) as bed_cerner_alias FROM FLOWFILE On Fri, Mar 8, 2019 at 7:57 AM Boris Tyukin wrote: > > I am struggling for an hour now

Re: Errors when attempting to use timestamp-millis fields with QueryRecord

Hello, I believe this is a known issue. Unfortunately, querying against timestamp column is not supported. https://issues.apache.org/jira/browse/NIFI-5888 I'm working on fixing this at Calcite project, the sql execution engine underneath QueryRecord.