ASF GitHub Bot commented on NIFI-1706:

GitHub user ijokarumawak opened a pull request:


    NIFI-1706: Extend QueryDatabaseTable to support arbitrary queries

    Thank you for submitting a contribution to Apache NiFi.
    In order to streamline the review of the contribution we ask you
    to ensure the following steps have been taken:
    ### For all changes:
    - [x] Is there a JIRA ticket associated with this PR? Is it referenced 
         in the commit message?
    - [x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.
    - [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?
    - [ ] Is your initial contribution a single, squashed commit?
    ### For code changes:
    - [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
    - [ ] Have you written or updated unit tests to verify your changes?
    - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
    - [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
    - [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
    - [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?
    ### For documentation related changes:
    - [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?
    ### Note:
    Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ijokarumawak/nifi nifi-1706

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2618
commit 8155c9d1679a664d102b45ba0e87e7f60d33f26f
Author: patricker <patricker@...>
Date:   2017-09-19T05:50:06Z

    NIFI-1706: Extend QueryDatabaseTable to support arbitrary queries
    NIFI-1706 - Only include Maximum Value columns in the type map.
    This closes #2162.
    Signed-off-by: Koji Kawamura <ijokaruma...@apache.org>

commit 7664e6c503bae2968b6d19523ddb33a5be832f5e
Author: Koji Kawamura <ijokarumawak@...>
Date:   2018-04-09T10:48:14Z

    NIFI-1706: Extend QueryDatabaseTable to support arbitrary queries
    - Squashed commits in the previous PR
    - Rebased against the latest master
    - Added stop method to GenerateTableFetch so that it refreshes the
    column type map when it gets restarted
    - Fixed whitespacing around if/for statement
    - Updated expressionLanguageSupported value since it is not auto-merged


> Extend QueryDatabaseTable to support arbitrary queries
> ------------------------------------------------------
>                 Key: NIFI-1706
>                 URL: https://issues.apache.org/jira/browse/NIFI-1706
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>    Affects Versions: 1.4.0
>            Reporter: Paul Bormans
>            Assignee: Peter Wicks
>            Priority: Major
>              Labels: features
> The QueryDatabaseTable is able to observe a configured database table for new 
> rows and yield these into the flowfile. The model of an rdbms however is 
> often (if not always) normalized so you would need to join various tables in 
> order to "flatten" the data into useful events for a processing pipeline as 
> can be build with nifi or various tools within the hadoop ecosystem.
> The request is to extend the processor to specify an arbitrary sql query 
> instead of specifying the table name + columns.
> In addition (this may be another issue?) it is desired to limit the number of 
> rows returned per run. Not just because of bandwidth issue's from the nifi 
> pipeline onwards but mainly because huge databases may not be able to return 
> so many records within a reasonable time.

This message was sent by Atlassian JIRA

Reply via email to