ASF GitHub Bot commented on NIFI-1706:

Github user ijokarumawak commented on the issue:

    @patricker Thanks for the update, and sorry for my delayed response. I just 
got the chance to review this today.
    This has a conflict with #2504 . I can fix compilation level conflict by 
removing AbstractDatabaseProcessor.onPropertyModified which has been deleted by 
    However, It seems that we need to update other places, too. I encountered 
few unit test failures at 
    [ERROR]   TestGenerateTableFetch.testAddedRows:202 expected:<4> but was:<0>
    [ERROR]   TestGenerateTableFetch.testAddedRowsRightBounded:402 expected:<4> 
but was:<0>
    [ERROR]   TestGenerateTableFetch.testAddedRowsTwoTables:276 expected:<1> 
but was:<0>
    [ERROR]   TestGenerateTableFetch.testAddedRowsWithCustomWhereClause:1112 
expected:<5> but was:<0>
    [ERROR]   TestGenerateTableFetch.testMultipleColumnTypeMissing:1250 
expected:<1> but was:<0>
    I'll continue looking at these test failures. It'd be appreciated if you 
can rebase the PR with the latest master and see how we can fix those, too. 

> Extend QueryDatabaseTable to support arbitrary queries
> ------------------------------------------------------
>                 Key: NIFI-1706
>                 URL: https://issues.apache.org/jira/browse/NIFI-1706
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>    Affects Versions: 1.4.0
>            Reporter: Paul Bormans
>            Assignee: Peter Wicks
>            Priority: Major
>              Labels: features
> The QueryDatabaseTable is able to observe a configured database table for new 
> rows and yield these into the flowfile. The model of an rdbms however is 
> often (if not always) normalized so you would need to join various tables in 
> order to "flatten" the data into useful events for a processing pipeline as 
> can be build with nifi or various tools within the hadoop ecosystem.
> The request is to extend the processor to specify an arbitrary sql query 
> instead of specifying the table name + columns.
> In addition (this may be another issue?) it is desired to limit the number of 
> rows returned per run. Not just because of bandwidth issue's from the nifi 
> pipeline onwards but mainly because huge databases may not be able to return 
> so many records within a reasonable time.

This message was sent by Atlassian JIRA

Reply via email to