Hello.
I'm using Oracle 11g with nifi 1.6.0.
I'm trying to use QueryDatabaseTable to fetch newly modified/created rows from
a table with a few million records.
The field I use as maximum value column is "sysmodtime", a DATE field. When I
put something like 2018-04-23 00:00:00.0 in "initial.maxvalue.sysmodtime", the
query times out. In a sql command line, it takes several minutes to complete.
The query launched with nifi use as where clause: sysmodtime > timestamp
'2018-04-23 00:00:00.0'.
With to_date, like in "select count(*) from hpsm.probsummarym1 where sysmodtime
> to_date('2016-01-01 00:00:00', 'YYYY-MM-DD HH24;MI:SS')", it takes seconds.
The explanation seems about conversion between DATE type to TIMESTAMP type for
every rows, which obviously takes time.
Is there a workaround? I saw https://issues.apache.org/jira/browse/NIFI-2323
but it does not really helps.
Thanks for any answer/pointer.
Aurélien DEHAY
Big Data Architect
+33 616 815 441
[email protected]
2 rue Hennape - 92735 Nanterre Cedex - France
This electronic transmission (and any attachments thereto) is intended solely
for the use of the addressee(s). It may contain confidential or legally
privileged information. If you are not the intended recipient of this message,
you must delete it immediately and notify the sender. Any unauthorized use or
disclosure of this message is strictly prohibited. Faurecia does not guarantee
the integrity of this transmission and shall therefore never be liable if the
message is altered or falsified nor for any virus, interception or damage to
your system.
