[
https://issues.apache.org/jira/browse/NIFI-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383281#comment-15383281
]
ASF GitHub Bot commented on NIFI-2156:
--------------------------------------
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/642#discussion_r71249117
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListDatabaseTables.java
---
@@ -211,17 +213,37 @@ public void onTrigger(ProcessContext context,
ProcessSession session) throws Pro
?
context.getProperty(TABLE_TYPES).getValue().split("\\s*,\\s*")
: null;
final boolean includeCount =
context.getProperty(INCLUDE_COUNT).asBoolean();
+ final long refreshInterval =
context.getProperty(REFRESH_INTERVAL).asTimePeriod(TimeUnit.MILLISECONDS);
final StateManager stateManager = context.getStateManager();
final StateMap stateMap;
final Map<String, String> stateMapProperties;
try {
- stateMap = stateManager.getState(Scope.LOCAL);
+ stateMap = stateManager.getState(Scope.CLUSTER);
stateMapProperties = new HashMap<>(stateMap.toMap());
} catch (IOException ioe) {
throw new ProcessException(ioe);
}
+ try {
+ // Refresh state if the interval has elapsed
+ long lastRefreshed = -1;
+ final long currentTime = System.currentTimeMillis();
+ String lastTimestamp =
stateMapProperties.get(this.getIdentifier());
+ if (!StringUtils.isEmpty(lastTimestamp)) {
+ lastRefreshed = Long.parseLong(lastTimestamp);
+ }
+ if (lastRefreshed > 0 && refreshInterval > 0 && currentTime >=
(lastRefreshed + refreshInterval)) {
+ stateManager.clear(Scope.CLUSTER);
+ stateMapProperties.clear();
--- End diff --
Good point, will add.
> Add ListDatabaseTables processor
> --------------------------------
>
> Key: NIFI-2156
> URL: https://issues.apache.org/jira/browse/NIFI-2156
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Matt Burgess
> Assignee: Matt Burgess
> Fix For: 1.0.0
>
>
> This processor would use a DatabaseConnectionPool controller service, call
> getTables(), and if the (optional, defaulting-to-false) property "Include Row
> Count" is set, then a "SELECT COUNT(1) from table" would be issued to the
> database. The table catalog, schema, name, type, remarks (and its count if
> specified) will be included as attributes in a zero-content flow file.
> It will also use State Management to only list tables once. If new tables are
> added (and the processor is running), then the new tables' flow files will be
> generated. Changing any property that could affect the list of returned
> tables (such as the DB Connection, catalog, schema pattern, table name
> pattern, or table types) will reset the state and all tables will be fetched
> using the new criteria. The state can also be manually cleared using the
> standard Clear State link on the View State dialog (available on the
> processor's context menu)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
