[
https://issues.apache.org/jira/browse/NIFI-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gideon Korir updated NIFI-5888:
-------------------------------
Description:
Add the test below to `org.apache.nifi.processors.standard.TestQueryRecord`
{code:java}
@Test public void testTimestampColumns() throws InitializationException {
final MockRecordParser parser = new MockRecordParser();
parser.addSchemaField("name", RecordFieldType.STRING);
parser.addSchemaField("date_added", RecordFieldType.TIMESTAMP);
parser.addRecord("Tom", Timestamp.valueOf("2018-12-03 09:12:00"));
parser.addRecord("Jerry", Timestamp.valueOf("2018-12-04 10:26:00"));
parser.addRecord("Tom", Timestamp.valueOf("2017-01-03 11:22:00"));
final List<String> colNames = new ArrayList<>();
colNames.add("name");
colNames.add("day_added");
final ResultSetValidatingRecordWriter writer = new
ResultSetValidatingRecordWriter(colNames);
TestRunner runner = getRunner();
runner.addControllerService("parser", parser);
runner.enableControllerService(parser);
runner.addControllerService("writer", writer);
runner.enableControllerService(writer);
runner.setProperty(REL_NAME, "select name, {fn YEAR(date_added)} as day_added
from FLOWFILE"); runner.setProperty(QueryRecord.RECORD_READER_FACTORY,
"parser");
runner.setProperty(QueryRecord.RECORD_WRITER_FACTORY, "writer");
runner.enqueue("");
runner.run();
runner.assertTransferCount(REL_NAME, 1);
}
{code}
*Expected*: Test will pass
*Actual*: fails with ClassCastException with message "java.sql.Timestamp cannot
be cast to java.lang.Long".
//Also used sql: "select name, CAST(date_added, DATE) as day_added from
FLOWFILE" and failed
I (think) I've traced the issue to the NiFi usage of Apache Calcite; according
to Julian Hyde's comment on [this
issue|https://issues.apache.org/jira/browse/CALCITE-1427].
Disclaimer: My knowledge on Calcite is zero to none so I could be completely
wrong.
Currently, QueryRecord uses FlowFileTable which only implements QueryableTable
and TranslateableTable and therefore can not work with java.sql.Timestamp (at
least based on Hyde's comment on the issue). This will mean almost all
RecordReader services (Avro, Csv) would not work with QueryRecord.
was:
Add the test below to `org.apache.nifi.processors.standard.TestQueryRecord`
{code:java}
@Test public void testTimestampColumns() throws InitializationException { final
MockRecordParser parser = new MockRecordParser(); parser.addSchemaField("name",
RecordFieldType.STRING); parser.addSchemaField("date_added",
RecordFieldType.TIMESTAMP); parser.addRecord("Tom",
Timestamp.valueOf("2018-12-03 09:12:00")); parser.addRecord("Jerry",
Timestamp.valueOf("2018-12-04 10:26:00")); parser.addRecord("Tom",
Timestamp.valueOf("2017-01-03 11:22:00")); final List<String> colNames = new
ArrayList<>(); colNames.add("name"); colNames.add("day_added"); final
ResultSetValidatingRecordWriter writer = new
ResultSetValidatingRecordWriter(colNames); TestRunner runner = getRunner();
runner.addControllerService("parser", parser);
runner.enableControllerService(parser); runner.addControllerService("writer",
writer); runner.enableControllerService(writer); runner.setProperty(REL_NAME,
"select name, {fn YEAR(date_added)} as day_added from FLOWFILE");
runner.setProperty(QueryRecord.RECORD_READER_FACTORY, "parser");
runner.setProperty(QueryRecord.RECORD_WRITER_FACTORY, "writer");
runner.enqueue(""); runner.run(); runner.assertTransferCount(REL_NAME, 1);
}
{code}
*Expected*: Test will pass
*Actual*: fails with ClassCastException with message "java.sql.Timestamp cannot
be cast to java.lang.Long".
//Also used sql: "select name, CAST(date_added, DATE) as day_added from
FLOWFILE" and failed
I (think) I've traced the issue to the NiFi usage of Apache Calcite; according
to Julian Hyde's comment on [this
issue|https://issues.apache.org/jira/browse/CALCITE-1427].
Disclaimer: My knowledge on Calcite is zero to none so I could be completely
wrong.
Currently, QueryRecord uses FlowFileTable which only implements QueryableTable
and TranslateableTable and therefore can not work with java.sql.Timestamp (at
least based on Hyde's comment on the issue). This will mean almost all
RecordReader services (Avro, Csv) would not work with QueryRecord.
> QueryRecord processor handling timestamp
> ----------------------------------------
>
> Key: NIFI-5888
> URL: https://issues.apache.org/jira/browse/NIFI-5888
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 1.8.0
> Reporter: Gideon Korir
> Priority: Major
>
> Add the test below to `org.apache.nifi.processors.standard.TestQueryRecord`
>
> {code:java}
> @Test public void testTimestampColumns() throws InitializationException {
> final MockRecordParser parser = new MockRecordParser();
> parser.addSchemaField("name", RecordFieldType.STRING);
> parser.addSchemaField("date_added", RecordFieldType.TIMESTAMP);
> parser.addRecord("Tom", Timestamp.valueOf("2018-12-03 09:12:00"));
> parser.addRecord("Jerry", Timestamp.valueOf("2018-12-04 10:26:00"));
> parser.addRecord("Tom", Timestamp.valueOf("2017-01-03 11:22:00"));
> final List<String> colNames = new ArrayList<>();
> colNames.add("name");
> colNames.add("day_added");
> final ResultSetValidatingRecordWriter writer = new
> ResultSetValidatingRecordWriter(colNames);
> TestRunner runner = getRunner();
> runner.addControllerService("parser", parser);
> runner.enableControllerService(parser);
> runner.addControllerService("writer", writer);
> runner.enableControllerService(writer);
> runner.setProperty(REL_NAME, "select name, {fn YEAR(date_added)} as day_added
> from FLOWFILE"); runner.setProperty(QueryRecord.RECORD_READER_FACTORY,
> "parser");
> runner.setProperty(QueryRecord.RECORD_WRITER_FACTORY, "writer");
> runner.enqueue("");
> runner.run();
> runner.assertTransferCount(REL_NAME, 1);
> }
> {code}
>
> *Expected*: Test will pass
> *Actual*: fails with ClassCastException with message "java.sql.Timestamp
> cannot be cast to java.lang.Long".
> //Also used sql: "select name, CAST(date_added, DATE) as day_added from
> FLOWFILE" and failed
>
> I (think) I've traced the issue to the NiFi usage of Apache Calcite;
> according to Julian Hyde's comment on [this
> issue|https://issues.apache.org/jira/browse/CALCITE-1427].
> Disclaimer: My knowledge on Calcite is zero to none so I could be completely
> wrong.
> Currently, QueryRecord uses FlowFileTable which only implements
> QueryableTable and TranslateableTable and therefore can not work with
> java.sql.Timestamp (at least based on Hyde's comment on the issue). This will
> mean almost all RecordReader services (Avro, Csv) would not work with
> QueryRecord.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
