[
https://issues.apache.org/jira/browse/NIFI-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382368#comment-15382368
]
ASF GitHub Bot commented on NIFI-2156:
--------------------------------------
Github user JPercivall commented on the issue:
https://github.com/apache/nifi/pull/642
Alright, after giving it some time and a cup of coffee I realize how off I
was at first, lol. This processor reaches out to the DB asking for the tables.
Then for each table that isn't already stored in state it creates a flowfile.
If the processor is configured to give the count, it needs to send a SQL query
asking for it. If that query fails it will remove the flowfile it created and
continue onto the next table. If successful, the FQN of the table will then be
added to state (after queuing it to transfer).
That realization makes my comment about data loss void (was afraid it would
get stored in state after un-successfully getting the count).
One new comment, would a user want to set an expiration for tables in
state? That way they could get updates on the count of a table every X
seconds/minutes. In it's current form it will get the table once but never
again. You're already storing the timestamp as the value so it should be an
easy addition.
> Add ListDatabaseTables processor
> --------------------------------
>
> Key: NIFI-2156
> URL: https://issues.apache.org/jira/browse/NIFI-2156
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Matt Burgess
> Assignee: Matt Burgess
> Fix For: 1.0.0
>
>
> This processor would use a DatabaseConnectionPool controller service, call
> getTables(), and if the (optional, defaulting-to-false) property "Include Row
> Count" is set, then a "SELECT COUNT(1) from table" would be issued to the
> database. The table catalog, schema, name, type, remarks (and its count if
> specified) will be included as attributes in a zero-content flow file.
> It will also use State Management to only list tables once. If new tables are
> added (and the processor is running), then the new tables' flow files will be
> generated. Changing any property that could affect the list of returned
> tables (such as the DB Connection, catalog, schema pattern, table name
> pattern, or table types) will reset the state and all tables will be fetched
> using the new criteria. The state can also be manually cleared using the
> standard Clear State link on the View State dialog (available on the
> processor's context menu)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
