[jira] [Commented] (NIFI-11191) Intermittent Failures in HikariCPConnectionPoolTest
[
https://issues.apache.org/jira/browse/NIFI-11191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690109#comment-17690109
]
ASF subversion and git services commented on NIFI-11191:
Commit 1380a3df770bbb323126103f373c20f5b676ea55 in nifi's branch
refs/heads/support/nifi-1.x from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1380a3df77 ]
NIFI-11191 Refactored HikariCPConnectionPoolTest with Mock Driver (#6966)
- Removed Derby dependencies from HikariCP module no longer required for testing
Signed-off-by: Kevin Doran
> Intermittent Failures in HikariCPConnectionPoolTest
> ---
>
> Key: NIFI-11191
> URL: https://issues.apache.org/jira/browse/NIFI-11191
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The {{HikariCPConnectionPoolTest}} fails intermittently due to timeouts when
> attempting to retrieve a connection from the file-backed Derby Data Source:
> {noformat}
> Caused by: java.sql.SQLTimeoutException: Login timeout exceeded.
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
> Source)
> at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
> Source)
> at org.apache.derby.jdbc.InternalDriver.timeLogin(Unknown Source)
> at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
> at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
> at org.apache.derby.jdbc.EmbeddedDriver.connect(Unknown Source)
> at
> com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
> at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
> at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
> at
> com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
> at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
> at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:115)
> at
> com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
> at
> org.apache.nifi.dbcp.HikariCPConnectionPool.getConnection(HikariCPConnectionPool.java:343)
> ... 72 more
> Caused by: ERROR XBDA0: Login timeout exceeded.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown
> Source)
> ... 88 more
> Error: Errors:
> Error:HikariCPConnectionPoolTest.testCreateInsertSelect:172 Process
> Connection retrieval failed
> {noformat}
> The problem appears to be related to file locking and concurrent test
> execution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-11191) Intermittent Failures in HikariCPConnectionPoolTest
[
https://issues.apache.org/jira/browse/NIFI-11191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-11191:
Fix Version/s: 1.21.0
> Intermittent Failures in HikariCPConnectionPoolTest
> ---
>
> Key: NIFI-11191
> URL: https://issues.apache.org/jira/browse/NIFI-11191
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.0.0, 1.21.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The {{HikariCPConnectionPoolTest}} fails intermittently due to timeouts when
> attempting to retrieve a connection from the file-backed Derby Data Source:
> {noformat}
> Caused by: java.sql.SQLTimeoutException: Login timeout exceeded.
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
> Source)
> at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
> Source)
> at org.apache.derby.jdbc.InternalDriver.timeLogin(Unknown Source)
> at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
> at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
> at org.apache.derby.jdbc.EmbeddedDriver.connect(Unknown Source)
> at
> com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
> at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
> at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
> at
> com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
> at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
> at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:115)
> at
> com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
> at
> org.apache.nifi.dbcp.HikariCPConnectionPool.getConnection(HikariCPConnectionPool.java:343)
> ... 72 more
> Caused by: ERROR XBDA0: Login timeout exceeded.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown
> Source)
> ... 88 more
> Error: Errors:
> Error:HikariCPConnectionPoolTest.testCreateInsertSelect:172 Process
> Connection retrieval failed
> {noformat}
> The problem appears to be related to file locking and concurrent test
> execution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-11195) Refactor Identity Mapping to nifi-security-identity
David Handermann created NIFI-11195:
---
Summary: Refactor Identity Mapping to nifi-security-identity
Key: NIFI-11195
URL: https://issues.apache.org/jira/browse/NIFI-11195
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Reporter: David Handermann
Assignee: David Handermann
The {{nifi-security-utils}} module has several classes for translating identity
information based on application properties. This classes are not related to
other cryptographic operations in {{nifi-security-utils}} and should be moved
to separate module to reduce transitive dependencies.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-11193) Improve logging for NiFi Registry GitFlowPersistenceProvider in onConfigured
[ https://issues.apache.org/jira/browse/NIFI-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Doran updated NIFI-11193: --- Fix Version/s: 1.21.0 > Improve logging for NiFi Registry GitFlowPersistenceProvider in onConfigured > > > Key: NIFI-11193 > URL: https://issues.apache.org/jira/browse/NIFI-11193 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry >Affects Versions: 1.20.0 >Reporter: Kevin Doran >Assignee: Kevin Doran >Priority: Minor > Fix For: 2.0.0, 1.21.0 > > Time Spent: 40m > Remaining Estimate: 0h > > This swallowed exception makes it difficult to troubleshoot errors: > https://github.com/apache/nifi/blob/65dbbc9e76d618c538f8fc79e4c0343998ca719d/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/provider/flow/git/GitFlowMetaData.java#L191-L193 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11193) Improve logging for NiFi Registry GitFlowPersistenceProvider in onConfigured
[ https://issues.apache.org/jira/browse/NIFI-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690073#comment-17690073 ] ASF subversion and git services commented on NIFI-11193: Commit 17b4480831a9b0f194f5db74ba72c454242776cb in nifi's branch refs/heads/support/nifi-1.x from Kevin Doran [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=17b4480831 ] NIFI-11193 Improve GitFlowPersisenceProvider logging This closes #6967 Co-authored-by: David Handermann Signed-off-by: David Handermann > Improve logging for NiFi Registry GitFlowPersistenceProvider in onConfigured > > > Key: NIFI-11193 > URL: https://issues.apache.org/jira/browse/NIFI-11193 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry >Affects Versions: 1.20.0 >Reporter: Kevin Doran >Assignee: Kevin Doran >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > > This swallowed exception makes it difficult to troubleshoot errors: > https://github.com/apache/nifi/blob/65dbbc9e76d618c538f8fc79e4c0343998ca719d/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/provider/flow/git/GitFlowMetaData.java#L191-L193 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11193) Improve logging for NiFi Registry GitFlowPersistenceProvider in onConfigured
[ https://issues.apache.org/jira/browse/NIFI-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11193: Fix Version/s: 2.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Improve logging for NiFi Registry GitFlowPersistenceProvider in onConfigured > > > Key: NIFI-11193 > URL: https://issues.apache.org/jira/browse/NIFI-11193 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry >Affects Versions: 1.20.0 >Reporter: Kevin Doran >Assignee: Kevin Doran >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > > This swallowed exception makes it difficult to troubleshoot errors: > https://github.com/apache/nifi/blob/65dbbc9e76d618c538f8fc79e4c0343998ca719d/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/provider/flow/git/GitFlowMetaData.java#L191-L193 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MINIFICPP-2035) Support for nifi flow.json format
[ https://issues.apache.org/jira/browse/MINIFICPP-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690063#comment-17690063 ] Marton Szasz commented on MINIFICPP-2035: - https://github.com/apache/nifi-minifi-cpp/pull/1494 > Support for nifi flow.json format > - > > Key: MINIFICPP-2035 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2035 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Major > Fix For: 0.14.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > The agent should be able to handle a json flow config corresponding to a > serialized VersionedDataflow object. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6967: NIFI-11193 Improve GitFlowPersisenceProvider logging
exceptionfactory closed pull request #6967: NIFI-11193 Improve GitFlowPersisenceProvider logging URL: https://github.com/apache/nifi/pull/6967 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11193) Improve logging for NiFi Registry GitFlowPersistenceProvider in onConfigured
[ https://issues.apache.org/jira/browse/NIFI-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690064#comment-17690064 ] ASF subversion and git services commented on NIFI-11193: Commit e7af4c10dfa47e0f9872cd2534e2d52d963c7cf7 in nifi's branch refs/heads/main from Kevin Doran [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e7af4c10df ] NIFI-11193 Improve GitFlowPersisenceProvider logging This closes #6967 Co-authored-by: David Handermann Signed-off-by: David Handermann > Improve logging for NiFi Registry GitFlowPersistenceProvider in onConfigured > > > Key: NIFI-11193 > URL: https://issues.apache.org/jira/browse/NIFI-11193 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry >Affects Versions: 1.20.0 >Reporter: Kevin Doran >Assignee: Kevin Doran >Priority: Minor > Time Spent: 40m > Remaining Estimate: 0h > > This swallowed exception makes it difficult to troubleshoot errors: > https://github.com/apache/nifi/blob/65dbbc9e76d618c538f8fc79e4c0343998ca719d/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/provider/flow/git/GitFlowMetaData.java#L191-L193 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2035) Support for nifi flow.json format
[ https://issues.apache.org/jira/browse/MINIFICPP-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-2035. - Fix Version/s: 0.14.0 Resolution: Done > Support for nifi flow.json format > - > > Key: MINIFICPP-2035 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2035 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Major > Fix For: 0.14.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > The agent should be able to handle a json flow config corresponding to a > serialized VersionedDataflow object. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2045) Enable synchronous flow file reloading, and orphan content cleanup
[ https://issues.apache.org/jira/browse/MINIFICPP-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-2045. - Fix Version/s: 0.14.0 Resolution: Done > Enable synchronous flow file reloading, and orphan content cleanup > -- > > Key: MINIFICPP-2045 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2045 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Major > Fix For: 0.14.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently, reloading the flow files happens asynchronously which allows > connections to exceed the backpressure threshold. We should also clean up > contents without owner flow files. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2034) Improve CWEL SID lookup caching
[ https://issues.apache.org/jira/browse/MINIFICPP-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-2034. - Resolution: Fixed > Improve CWEL SID lookup caching > --- > > Key: MINIFICPP-2034 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2034 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Ferenc Gerlits >Priority: Major > Fix For: 0.14.0 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > SID lookups can take a long time, especially in larger corporate networks. We > have some caching or the lookup results in MetadataWalker, but it's thrown > away at the end of each batch/trigger. We should move the cache to the > processor. Make sure to cache negative responses as well, i.e. the fact that > an SID doesn't have an associated domain/username. > Optional: configurable cache entry lifetime -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2007) Add compression options for flowfile and content repositories
[ https://issues.apache.org/jira/browse/MINIFICPP-2007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-2007. - Fix Version/s: 0.14.0 Resolution: Done > Add compression options for flowfile and content repositories > - > > Key: MINIFICPP-2007 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2007 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Major > Fix For: 0.14.0 > > Time Spent: 1h > Remaining Estimate: 0h > > RocksDB has compression options with several compression types that can be > used to reduce the size of the database. These options can be used to > decrease the size of our content repository and flowfile repository if > rocksdb database is used to persist data. > These options should be investigated and added to our configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1509: MINIFICPP-2045 - Synchronous flow file restore and orphan content cleanup
szaszm closed pull request #1509: MINIFICPP-2045 - Synchronous flow file restore and orphan content cleanup URL: https://github.com/apache/nifi-minifi-cpp/pull/1509 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1494: MINIFICPP-2035 - NiFi flow json configuration format support
szaszm closed pull request #1494: MINIFICPP-2035 - NiFi flow json configuration format support URL: https://github.com/apache/nifi-minifi-cpp/pull/1494 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1480: MINIFICPP-2007 Add compression options for flowfile and content repo
szaszm closed pull request #1480: MINIFICPP-2007 Add compression options for flowfile and content repo URL: https://github.com/apache/nifi-minifi-cpp/pull/1480 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1502: MINIFICPP-2034 Cache the results of the username lookups in ConsumeWindowsEventLog
szaszm closed pull request #1502: MINIFICPP-2034 Cache the results of the username lookups in ConsumeWindowsEventLog URL: https://github.com/apache/nifi-minifi-cpp/pull/1502 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mcgilman commented on pull request #6968: NIFI-11194 Remove gzip CSS and JS from nifi-web-ui
mcgilman commented on PR #6968: URL: https://github.com/apache/nifi/pull/6968#issuecomment-1433884561 The changes look good. Will review closer and verify... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6968: NIFI-11194 Remove gzip CSS and JS from nifi-web-ui
exceptionfactory commented on PR #6968: URL: https://github.com/apache/nifi/pull/6968#issuecomment-1433877105 The Gzip files provided Jetty with the option to stream files that were already compressed, but this feature is deprecated and disabled by default in Jetty 9.4 as described in [Jetty PR 2385](https://github.com/eclipse/jetty.project/pull/2385). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11191) Intermittent Failures in HikariCPConnectionPoolTest
[
https://issues.apache.org/jira/browse/NIFI-11191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-11191:
Fix Version/s: 2.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Intermittent Failures in HikariCPConnectionPoolTest
> ---
>
> Key: NIFI-11191
> URL: https://issues.apache.org/jira/browse/NIFI-11191
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The {{HikariCPConnectionPoolTest}} fails intermittently due to timeouts when
> attempting to retrieve a connection from the file-backed Derby Data Source:
> {noformat}
> Caused by: java.sql.SQLTimeoutException: Login timeout exceeded.
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
> Source)
> at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
> Source)
> at org.apache.derby.jdbc.InternalDriver.timeLogin(Unknown Source)
> at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
> at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
> at org.apache.derby.jdbc.EmbeddedDriver.connect(Unknown Source)
> at
> com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
> at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
> at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
> at
> com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
> at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
> at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:115)
> at
> com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
> at
> org.apache.nifi.dbcp.HikariCPConnectionPool.getConnection(HikariCPConnectionPool.java:343)
> ... 72 more
> Caused by: ERROR XBDA0: Login timeout exceeded.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown
> Source)
> ... 88 more
> Error: Errors:
> Error:HikariCPConnectionPoolTest.testCreateInsertSelect:172 Process
> Connection retrieval failed
> {noformat}
> The problem appears to be related to file locking and concurrent test
> execution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-11194) Remove unused gzip CSS and JS bundling from nifi-web-ui
[
https://issues.apache.org/jira/browse/NIFI-11194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-11194:
Status: Patch Available (was: Open)
> Remove unused gzip CSS and JS bundling from nifi-web-ui
> ---
>
> Key: NIFI-11194
> URL: https://issues.apache.org/jira/browse/NIFI-11194
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core UI
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{nifi-web-ui}} includes Maven build plugins to prepare CSS and
> JavaScript files for bundling. The current configuration also creates gzip
> versions of each file, which are not used at runtime. This gzip files
> increase the size of the {{nifi-web-ui}} WAR and also take a majority of
> build time relative to other modules. The gzip processing and bundling should
> be removed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #6968: NIFI-11194 Remove gzip CSS and JS from nifi-web-ui
exceptionfactory opened a new pull request, #6968: URL: https://github.com/apache/nifi/pull/6968 # Summary [NIFI-11194](https://issues.apache.org/jira/browse/NIFI-11194) Removes unnecessary gzip processing for CSS and JavaScript files in `nifi-web-ui`, avoiding unnecessary building and packaging steps. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11194) Remove unused gzip CSS and JS bundling from nifi-web-ui
David Handermann created NIFI-11194:
---
Summary: Remove unused gzip CSS and JS bundling from nifi-web-ui
Key: NIFI-11194
URL: https://issues.apache.org/jira/browse/NIFI-11194
Project: Apache NiFi
Issue Type: Improvement
Components: Core UI
Reporter: David Handermann
Assignee: David Handermann
The {{nifi-web-ui}} includes Maven build plugins to prepare CSS and JavaScript
files for bundling. The current configuration also creates gzip versions of
each file, which are not used at runtime. This gzip files increase the size of
the {{nifi-web-ui}} WAR and also take a majority of build time relative to
other modules. The gzip processing and bundling should be removed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] kevdoran merged pull request #6966: NIFI-11191 Refactored HikariCPConnectionPoolTest with Mock Driver
kevdoran merged PR #6966: URL: https://github.com/apache/nifi/pull/6966 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11191) Intermittent Failures in HikariCPConnectionPoolTest
[
https://issues.apache.org/jira/browse/NIFI-11191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690039#comment-17690039
]
ASF subversion and git services commented on NIFI-11191:
Commit 5507bfa4b2e30cc09673d83250e3779dfd0aa109 in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=5507bfa4b2 ]
NIFI-11191 Refactored HikariCPConnectionPoolTest with Mock Driver (#6966)
- Removed Derby dependencies from HikariCP module no longer required for testing
Signed-off-by: Kevin Doran
> Intermittent Failures in HikariCPConnectionPoolTest
> ---
>
> Key: NIFI-11191
> URL: https://issues.apache.org/jira/browse/NIFI-11191
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The {{HikariCPConnectionPoolTest}} fails intermittently due to timeouts when
> attempting to retrieve a connection from the file-backed Derby Data Source:
> {noformat}
> Caused by: java.sql.SQLTimeoutException: Login timeout exceeded.
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
> Source)
> at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
> Source)
> at org.apache.derby.jdbc.InternalDriver.timeLogin(Unknown Source)
> at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
> at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
> at org.apache.derby.jdbc.EmbeddedDriver.connect(Unknown Source)
> at
> com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
> at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
> at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
> at
> com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
> at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
> at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:115)
> at
> com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
> at
> org.apache.nifi.dbcp.HikariCPConnectionPool.getConnection(HikariCPConnectionPool.java:343)
> ... 72 more
> Caused by: ERROR XBDA0: Login timeout exceeded.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown
> Source)
> ... 88 more
> Error: Errors:
> Error:HikariCPConnectionPoolTest.testCreateInsertSelect:172 Process
> Connection retrieval failed
> {noformat}
> The problem appears to be related to file locking and concurrent test
> execution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-11193) Improve logging for NiFi Registry GitFlowPersistenceProvider in onConfigured
[ https://issues.apache.org/jira/browse/NIFI-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Doran updated NIFI-11193: --- Status: Patch Available (was: In Progress) > Improve logging for NiFi Registry GitFlowPersistenceProvider in onConfigured > > > Key: NIFI-11193 > URL: https://issues.apache.org/jira/browse/NIFI-11193 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry >Affects Versions: 1.20.0 >Reporter: Kevin Doran >Assignee: Kevin Doran >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > This swallowed exception makes it difficult to troubleshoot errors: > https://github.com/apache/nifi/blob/65dbbc9e76d618c538f8fc79e4c0343998ca719d/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/provider/flow/git/GitFlowMetaData.java#L191-L193 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] kevdoran commented on pull request #6967: NIFI-11193 Improve GitFlowPersisenceProvider logging
kevdoran commented on PR #6967: URL: https://github.com/apache/nifi/pull/6967#issuecomment-1433771689 Accepted all suggestions, thanks @exceptionfactory -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6967: NIFI-11193 Improve GitFlowPersisenceProvider logging
exceptionfactory commented on code in PR #6967:
URL: https://github.com/apache/nifi/pull/6967#discussion_r1108996488
##
nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/provider/flow/git/GitFlowPersistenceProvider.java:
##
@@ -94,9 +94,12 @@ public void onConfigured(ProviderConfigurationContext
configurationContext) thro
flowStorageDir = new File(flowStorageDirValue);
final boolean localRepoExists =
flowMetaData.localRepoExists(flowStorageDir);
if (remoteRepo != null && !remoteRepo.isEmpty() &&
!localRepoExists){
+logger.info("Validating remote repo '%s'...", remoteRepo);
flowMetaData.remoteRepoExists(remoteRepo);
+logger.info("Cloning remote repo '%s' to '%s'...", remoteRepo,
flowStorageDirValue);
flowMetaData.cloneRepository(flowStorageDir, remoteRepo);
}
+logger.info("Loading remote repo '%s'...", remoteRepo);
Review Comment:
```suggestion
logger.info("Loading remote repository [{}]", remoteRepo);
```
##
nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/provider/flow/git/GitFlowPersistenceProvider.java:
##
@@ -94,9 +94,12 @@ public void onConfigured(ProviderConfigurationContext
configurationContext) thro
flowStorageDir = new File(flowStorageDirValue);
final boolean localRepoExists =
flowMetaData.localRepoExists(flowStorageDir);
if (remoteRepo != null && !remoteRepo.isEmpty() &&
!localRepoExists){
+logger.info("Validating remote repo '%s'...", remoteRepo);
flowMetaData.remoteRepoExists(remoteRepo);
+logger.info("Cloning remote repo '%s' to '%s'...", remoteRepo,
flowStorageDirValue);
Review Comment:
```suggestion
logger.info("Cloning remote repository [{}] to [{}]",
remoteRepo, flowStorageDirValue);
```
##
nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/provider/flow/git/GitFlowPersistenceProvider.java:
##
@@ -94,9 +94,12 @@ public void onConfigured(ProviderConfigurationContext
configurationContext) thro
flowStorageDir = new File(flowStorageDirValue);
final boolean localRepoExists =
flowMetaData.localRepoExists(flowStorageDir);
if (remoteRepo != null && !remoteRepo.isEmpty() &&
!localRepoExists){
+logger.info("Validating remote repo '%s'...", remoteRepo);
Review Comment:
Placeholder values for logging should be `{}` instead of `%s`, also
recommend the wording adjustment:
```suggestion
logger.info("Validating remote repository [{}]", remoteRepo);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] kevdoran opened a new pull request, #6967: NIFI-11193 Improve GitFlowPersisenceProvider logging
kevdoran opened a new pull request, #6967: URL: https://github.com/apache/nifi/pull/6967 # Summary [NIFI-11193](https://issues.apache.org/jira/browse/NIFI-11193) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11193) Improve logging for NiFi Registry GitFlowPersistenceProvider in onConfigured
[ https://issues.apache.org/jira/browse/NIFI-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Doran updated NIFI-11193: --- Summary: Improve logging for NiFi Registry GitFlowPersistenceProvider in onConfigured (was: Improve logging for NiFi Registry FlowGitPersistenceProvider in onConfigured) > Improve logging for NiFi Registry GitFlowPersistenceProvider in onConfigured > > > Key: NIFI-11193 > URL: https://issues.apache.org/jira/browse/NIFI-11193 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry >Affects Versions: 1.20.0 >Reporter: Kevin Doran >Assignee: Kevin Doran >Priority: Minor > > This swallowed exception makes it difficult to troubleshoot errors: > https://github.com/apache/nifi/blob/65dbbc9e76d618c538f8fc79e4c0343998ca719d/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/provider/flow/git/GitFlowMetaData.java#L191-L193 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11193) Improve logging for NiFi Registry FlowGitPersistenceProvider in onConfigured
Kevin Doran created NIFI-11193: -- Summary: Improve logging for NiFi Registry FlowGitPersistenceProvider in onConfigured Key: NIFI-11193 URL: https://issues.apache.org/jira/browse/NIFI-11193 Project: Apache NiFi Issue Type: Improvement Components: NiFi Registry Affects Versions: 1.20.0 Reporter: Kevin Doran Assignee: Kevin Doran This swallowed exception makes it difficult to troubleshoot errors: https://github.com/apache/nifi/blob/65dbbc9e76d618c538f8fc79e4c0343998ca719d/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/provider/flow/git/GitFlowMetaData.java#L191-L193 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11192) If Port moved from parent to child group or vice versa between flow versions, version change can leave nifi in bad state
Mark Payne created NIFI-11192:
-
Summary: If Port moved from parent to child group or vice versa
between flow versions, version change can leave nifi in bad state
Key: NIFI-11192
URL: https://issues.apache.org/jira/browse/NIFI-11192
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Mark Payne
Fix For: 2.0.0, 1.21.0
To reproduce:
Create a Process Group, Parent.
Inside of Parent, create:
* A processor, for example, ReplaceText
* A Process Group, Child.
Inside of Child, create:
* An Input Port, In
* A Processor, for example, UpdateAttribute
* A connection between the two components
Then connect ReplaceText to the Input Port, In.
Save the Parent PG flow as Version 1 of a flow.
Now, create a new Processor, say RouteOnAttribute, within the Parent PG.
Move the destination of the connection from the Input Port to RouteOnAttribute.
Step into the Child PG. Select all components, right-click, and choose "Move to
Parent"
Save Parent PG as Version 2 of the flow.
Now, attempt to Change Version on the Parent Group. Change the version to
Version 1.
The version change will fail with an error: "Failed to perform update flow
request due to 42fb2904-c774-359b-5368-2e48b60ac02d is the destination of
another component" and the logs will have a stack trace:
{code:java}
2023-02-16 15:18:28,830 ERROR [Process Group Update Thread-1]
o.apache.nifi.web.api.FlowUpdateResource Failed to perform update flow request
java.lang.IllegalStateException: 42fb2904-c774-359b-5368-2e48b60ac02d is the
destination of another component
at
org.apache.nifi.controller.AbstractPort.verifyCanDelete(AbstractPort.java:562)
at
org.apache.nifi.controller.AbstractPort.verifyCanDelete(AbstractPort.java:542)
at
org.apache.nifi.groups.StandardProcessGroup.removeInputPort(StandardProcessGroup.java:637)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.removeMissingComponents(StandardVersionedComponentSynchronizer.java:948)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.removeMissingInputPorts(StandardVersionedComponentSynchronizer.java:873)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:410)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:260)
at
org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:556)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:255)
at
org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3972)
at
org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3952)
at
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:435)
at
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
at
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:308)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
at
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at
[jira] [Updated] (NIFI-11191) Intermittent Failures in HikariCPConnectionPoolTest
[
https://issues.apache.org/jira/browse/NIFI-11191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-11191:
Status: Patch Available (was: Open)
> Intermittent Failures in HikariCPConnectionPoolTest
> ---
>
> Key: NIFI-11191
> URL: https://issues.apache.org/jira/browse/NIFI-11191
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{HikariCPConnectionPoolTest}} fails intermittently due to timeouts when
> attempting to retrieve a connection from the file-backed Derby Data Source:
> {noformat}
> Caused by: java.sql.SQLTimeoutException: Login timeout exceeded.
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
> Source)
> at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
> Source)
> at org.apache.derby.jdbc.InternalDriver.timeLogin(Unknown Source)
> at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
> at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
> at org.apache.derby.jdbc.EmbeddedDriver.connect(Unknown Source)
> at
> com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
> at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
> at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
> at
> com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
> at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
> at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:115)
> at
> com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
> at
> org.apache.nifi.dbcp.HikariCPConnectionPool.getConnection(HikariCPConnectionPool.java:343)
> ... 72 more
> Caused by: ERROR XBDA0: Login timeout exceeded.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown
> Source)
> ... 88 more
> Error: Errors:
> Error:HikariCPConnectionPoolTest.testCreateInsertSelect:172 Process
> Connection retrieval failed
> {noformat}
> The problem appears to be related to file locking and concurrent test
> execution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10612) Create new Expression Language method isJson to determine whether an attribute is JSON
[ https://issues.apache.org/jira/browse/NIFI-10612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689938#comment-17689938 ] Daniel Stieglitz commented on NIFI-10612: - [~exceptionfactory] I had a cached version. I now see it. Thanks! > Create new Expression Language method isJson to determine whether an > attribute is JSON > -- > > Key: NIFI-10612 > URL: https://issues.apache.org/jira/browse/NIFI-10612 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Fix For: 1.19.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Create a new Expression Language method isJson to determine whether > attributes are JSON or not. The addition of this method provides a short > circuit of failure before attempting a JsonPath which otherwise would result > in flow files yielding if the attribute is not JSON as detailed in NIFI-10396 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10754) Add Expression Language method to encode a URI
[
https://issues.apache.org/jira/browse/NIFI-10754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689936#comment-17689936
]
Daniel Stieglitz commented on NIFI-10754:
-
[~exceptionfactory] That was it thanks!
> Add Expression Language method to encode a URI
> --
>
> Key: NIFI-10754
> URL: https://issues.apache.org/jira/browse/NIFI-10754
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Major
> Fix For: 1.20.0
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> The Nifi Expression Language urlEncode method replaces white space with a +
> and not %20. That is fine for the query section of a URL but not the path.
> There are many Stackoverflow posts which detail the issue. Here is one,
> [URLEncoder not able to translate space
> character|https://stackoverflow.com/questions/4737841/urlencoder-not-able-to-translate-space-character]
> Our particular scenario is where we build a URL from attributes and then try
> to call the URL with InvokeHTTP
> e.g. of a URL with a space in its path
> {code:java}
> https://somehost/api/v1/somepath
> /actual?att1=something=somethingelse{code}
> from that URL we only encode the part which may have special characters
> {code:java}
> somepath /actual{code}
> urlEncode will convert that to
> {code:java}
> somepath+%2Factual{code}
> The + in the URL path is not the same as a blank space hence the call to
> InvokeHttp fails.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10612) Create new Expression Language method isJson to determine whether an attribute is JSON
[ https://issues.apache.org/jira/browse/NIFI-10612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689922#comment-17689922 ] David Handermann commented on NIFI-10612: - https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#isjson is visible in the current version. > Create new Expression Language method isJson to determine whether an > attribute is JSON > -- > > Key: NIFI-10612 > URL: https://issues.apache.org/jira/browse/NIFI-10612 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Fix For: 1.19.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Create a new Expression Language method isJson to determine whether > attributes are JSON or not. The addition of this method provides a short > circuit of failure before attempting a JsonPath which otherwise would result > in flow files yielding if the attribute is not JSON as detailed in NIFI-10396 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10754) Add Expression Language method to encode a URI
[
https://issues.apache.org/jira/browse/NIFI-10754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689919#comment-17689919
]
David Handermann commented on NIFI-10754:
-
[~dstiegli1] Perhaps you have a cached version?
https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#geturi
> Add Expression Language method to encode a URI
> --
>
> Key: NIFI-10754
> URL: https://issues.apache.org/jira/browse/NIFI-10754
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Major
> Fix For: 1.20.0
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> The Nifi Expression Language urlEncode method replaces white space with a +
> and not %20. That is fine for the query section of a URL but not the path.
> There are many Stackoverflow posts which detail the issue. Here is one,
> [URLEncoder not able to translate space
> character|https://stackoverflow.com/questions/4737841/urlencoder-not-able-to-translate-space-character]
> Our particular scenario is where we build a URL from attributes and then try
> to call the URL with InvokeHTTP
> e.g. of a URL with a space in its path
> {code:java}
> https://somehost/api/v1/somepath
> /actual?att1=something=somethingelse{code}
> from that URL we only encode the part which may have special characters
> {code:java}
> somepath /actual{code}
> urlEncode will convert that to
> {code:java}
> somepath+%2Factual{code}
> The + in the URL path is not the same as a blank space hence the call to
> InvokeHttp fails.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (NIFI-10612) Create new Expression Language method isJson to determine whether an attribute is JSON
[ https://issues.apache.org/jira/browse/NIFI-10612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689912#comment-17689912 ] Daniel Stieglitz edited comment on NIFI-10612 at 2/16/23 6:58 PM: -- [~exceptionfactory] I do not see isJson on NIFI Expression Language [documentation page|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html] was (Author: JIRAUSER294662): [~exceptionfactory] I do not see isJson on on NIFI Expression Language [documentation page|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html] > Create new Expression Language method isJson to determine whether an > attribute is JSON > -- > > Key: NIFI-10612 > URL: https://issues.apache.org/jira/browse/NIFI-10612 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Fix For: 1.19.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Create a new Expression Language method isJson to determine whether > attributes are JSON or not. The addition of this method provides a short > circuit of failure before attempting a JsonPath which otherwise would result > in flow files yielding if the attribute is not JSON as detailed in NIFI-10396 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10612) Create new Expression Language method isJson to determine whether an attribute is JSON
[ https://issues.apache.org/jira/browse/NIFI-10612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689912#comment-17689912 ] Daniel Stieglitz commented on NIFI-10612: - [~exceptionfactory] I do not see isJson on on NIFI Expression Language [documentation page|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html] > Create new Expression Language method isJson to determine whether an > attribute is JSON > -- > > Key: NIFI-10612 > URL: https://issues.apache.org/jira/browse/NIFI-10612 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Fix For: 1.19.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Create a new Expression Language method isJson to determine whether > attributes are JSON or not. The addition of this method provides a short > circuit of failure before attempting a JsonPath which otherwise would result > in flow files yielding if the attribute is not JSON as detailed in NIFI-10396 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10754) Add Expression Language method to encode a URI
[
https://issues.apache.org/jira/browse/NIFI-10754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689910#comment-17689910
]
Daniel Stieglitz commented on NIFI-10754:
-
[~exceptionfactory] I do not see getUri on NIFI Expression Language
[documentation page
|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html]
> Add Expression Language method to encode a URI
> --
>
> Key: NIFI-10754
> URL: https://issues.apache.org/jira/browse/NIFI-10754
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Major
> Fix For: 1.20.0
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> The Nifi Expression Language urlEncode method replaces white space with a +
> and not %20. That is fine for the query section of a URL but not the path.
> There are many Stackoverflow posts which detail the issue. Here is one,
> [URLEncoder not able to translate space
> character|https://stackoverflow.com/questions/4737841/urlencoder-not-able-to-translate-space-character]
> Our particular scenario is where we build a URL from attributes and then try
> to call the URL with InvokeHTTP
> e.g. of a URL with a space in its path
> {code:java}
> https://somehost/api/v1/somepath
> /actual?att1=something=somethingelse{code}
> from that URL we only encode the part which may have special characters
> {code:java}
> somepath /actual{code}
> urlEncode will convert that to
> {code:java}
> somepath+%2Factual{code}
> The + in the URL path is not the same as a blank space hence the call to
> InvokeHttp fails.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #6966: NIFI-11191 Refactored HikariCPConnectionPoolTest with Mock Driver
exceptionfactory opened a new pull request, #6966: URL: https://github.com/apache/nifi/pull/6966 # Summary [NIFI-11191](https://issues.apache.org/jira/browse/NIFI-11191) Refactors the `HikariCPConnectionPoolTest` class using a `MockDriver` class to avoid intermittent failures related to concurrency and file locking when using Apache Derby. The existing test class attempted to work around locking issues using specialized temporary file handling for the Derby database and associated the logs. The `MockDriver` approach avoids these issues and focuses the test on confirming whether the Controller Service returns a Connection. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] tpalfy commented on a diff in pull request #6794: NIFI-10966: Add option to QuerySalesforceObject to run custom query
tpalfy commented on code in PR #6794:
URL: https://github.com/apache/nifi/pull/6794#discussion_r1108665694
##
nifi-nar-bundles/nifi-salesforce-bundle/nifi-salesforce-processors/src/main/java/org/apache/nifi/processors/salesforce/QuerySalesforceObject.java:
##
@@ -224,6 +282,8 @@ protected List
getSupportedPropertyDescriptors() {
return Collections.unmodifiableList(Arrays.asList(
API_URL,
Review Comment:
When switching between PROPERTY_BASED and CUSTOM_QUERY the list of
properties on the processor changes in a confusing manner because dependent and
non-dependent properties are intermingled.
Can we group them a bit better and move the non-dependent properties either
before or after the dependent ones?
See example below.
Also CREATE_ZERO_RECORD_FILES doesn't work with CUSTOM_QUERY.
```java
API_URL,
API_VERSION,
QUERY_TYPE,
CUSTOM_SOQL_QUERY,
SOBJECT_NAME,
FIELD_NAMES,
RECORD_WRITER,
AGE_FIELD,
INITIAL_AGE_FILTER,
AGE_DELAY,
CUSTOM_WHERE_CONDITION,
READ_TIMEOUT,
CREATE_ZERO_RECORD_FILES,
TOKEN_PROVIDER
```
##
nifi-nar-bundles/nifi-salesforce-bundle/nifi-salesforce-processors/src/main/java/org/apache/nifi/processors/salesforce/QuerySalesforceObject.java:
##
@@ -106,13 +111,55 @@
@DefaultSchedule(strategy = SchedulingStrategy.TIMER_DRIVEN, period = "1 min")
public class QuerySalesforceObject extends AbstractProcessor {
+static final AllowableValue PROPERTY_BASED_QUERY = new
AllowableValue("property-based-query", "Property Based Query", "Provide query
by properties.");
+static final AllowableValue CUSTOM_QUERY = new
AllowableValue("custom-query", "Custom Query", "Provide custom SOQL query.");
+
+static final PropertyDescriptor API_URL = new PropertyDescriptor.Builder()
Review Comment:
Why can't we use `API_URL` and `API_VERSION` from
`CommonSalesforceProperties`?
##
nifi-nar-bundles/nifi-salesforce-bundle/nifi-salesforce-processors/src/main/java/org/apache/nifi/processors/salesforce/QuerySalesforceObject.java:
##
@@ -261,6 +321,23 @@ protected Collection
customValidate(ValidationContext validati
@Override
public void onTrigger(final ProcessContext context, final ProcessSession
session) throws ProcessException {
+boolean isCustomQuery =
CUSTOM_QUERY.getValue().equals(context.getProperty(QUERY_TYPE).getValue());
+AtomicReference nextRecordsUrl = new AtomicReference<>();
+AtomicReference totalSize = new AtomicReference<>();
Review Comment:
These `AtomicReference` objects are not used here. I don't think their
creation should be the responsibility of `onTrigger`, rather that of the
downstream methods.
The `nextRecordsUrl` constructor would be explicitly called both in those
methods but I wouldn't consider that a duplication. These being closer to the
`do...while` block they are used in would help understand their purpose much
better.
##
nifi-nar-bundles/nifi-salesforce-bundle/nifi-salesforce-processors/src/main/java/org/apache/nifi/processors/salesforce/QuerySalesforceObject.java:
##
@@ -387,11 +461,63 @@ public void onTrigger(final ProcessContext context, final
ProcessSession session
} while (nextRecordsUrl.get() != null);
}
-private InputStream getResultInputStream(AtomicReference
nextRecordsUrl, String querySObject) {
-if (nextRecordsUrl.get() == null) {
+private void processCustomQuery(ProcessContext context, ProcessSession
session, FlowFile flowFile, AtomicReference nextRecordsUrl,
AtomicReference totalSize) {
+String customQuery =
context.getProperty(CUSTOM_SOQL_QUERY).evaluateAttributeExpressions(flowFile).getValue();
+do {
+FlowFile outgoingFlowFile;
+if (flowFile != null) {
+outgoingFlowFile = session.create(flowFile);
+} else {
+outgoingFlowFile = session.create();
+}
+Map attributes = new HashMap<>();
+try (InputStream response =
getResultInputStream(nextRecordsUrl.get(), customQuery)) {
+outgoingFlowFile = session.write(outgoingFlowFile,
parseHttpResponse(response, nextRecordsUrl, totalSize));
+int recordCount = nextRecordsUrl.get() != null ? 2000 :
Integer.parseInt(totalSize.get()) % 2000;
+attributes.put(CoreAttributes.MIME_TYPE.key(),
"application/json");
+attributes.put(TOTAL_RECORD_COUNT,
String.valueOf(recordCount));
+session.adjustCounter("Salesforce records processed",
recordCount, false);
+session.putAllAttributes(outgoingFlowFile, attributes);
+
[GitHub] [nifi] pvillard31 commented on pull request #6794: NIFI-10966: Add option to QuerySalesforceObject to run custom query
pvillard31 commented on PR #6794: URL: https://github.com/apache/nifi/pull/6794#issuecomment-1433444998 I've been playing with the latest version and this is working as expected. I'm a +1 from a feature usage point of view. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] turcsanyip commented on a diff in pull request #6965: NIFI-11190: Added provenance events to GetHubSpot and GetShopify
turcsanyip commented on code in PR #6965:
URL: https://github.com/apache/nifi/pull/6965#discussion_r1108793306
##
nifi-nar-bundles/nifi-shopify-bundle/nifi-shopify-processors/src/main/java/org/apache/nifi/processors/shopify/GetShopify.java:
##
@@ -327,6 +327,7 @@ public void onTrigger(final ProcessContext context, final
ProcessSession session
if (objectCountHolder.get() > 0) {
flowFile = session.putAttribute(flowFile,
CoreAttributes.MIME_TYPE.key(), "application/json");
session.transfer(flowFile, REL_SUCCESS);
+session.getProvenanceReporter().receive(flowFile,
shopifyRestService.getBaseUrl());
Review Comment:
`baseUrl` only contains the value of the Store Domain property (a hostname
basically). (the variable name could be improved too)
The real base URL should be logged in the provenance event.
The existing `getBaseUri()` method returns a builder containing that base
URL. It could be exposed to the processor class (preferably returning a string
instead of the builder).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-11188) Remove ProcessContext encrypt and decrypt methods
[
https://issues.apache.org/jira/browse/NIFI-11188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-11188:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Remove ProcessContext encrypt and decrypt methods
> -
>
> Key: NIFI-11188
> URL: https://issues.apache.org/jira/browse/NIFI-11188
> Project: Apache NiFi
> Issue Type: Task
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The {{ProcessContext}} interface methods {{encrypt}} and {{decrypt}} were
> deprecated for removal in NiFi 1.20.0. Removing deprecated JMS processors
> from the {{nifi-standard-processors}} module removed references inside the
> project. Removing these interface methods and corresponding implementations
> also reduces the need to pass around references to {{{}PropertyEncryptor{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] asfgit closed pull request #6962: NIFI-11188 Remove ProcessContext encrypt and decrypt methods
asfgit closed pull request #6962: NIFI-11188 Remove ProcessContext encrypt and decrypt methods URL: https://github.com/apache/nifi/pull/6962 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11188) Remove ProcessContext encrypt and decrypt methods
[
https://issues.apache.org/jira/browse/NIFI-11188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689854#comment-17689854
]
ASF subversion and git services commented on NIFI-11188:
Commit 284804ac424557d15ee8cc6ac4c9d5b862e1aafa in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=284804ac42 ]
NIFI-11188 Removed ProcessContext encrypt and decrypt methods
- Removed unnecessary references to PropertyEncryptor from multiple framework
components
Signed-off-by: Pierre Villard
This closes #6962.
> Remove ProcessContext encrypt and decrypt methods
> -
>
> Key: NIFI-11188
> URL: https://issues.apache.org/jira/browse/NIFI-11188
> Project: Apache NiFi
> Issue Type: Task
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{ProcessContext}} interface methods {{encrypt}} and {{decrypt}} were
> deprecated for removal in NiFi 1.20.0. Removing deprecated JMS processors
> from the {{nifi-standard-processors}} module removed references inside the
> project. Removing these interface methods and corresponding implementations
> also reduces the need to pass around references to {{{}PropertyEncryptor{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-11191) Intermittent Failures in HikariCPConnectionPoolTest
David Handermann created NIFI-11191:
---
Summary: Intermittent Failures in HikariCPConnectionPoolTest
Key: NIFI-11191
URL: https://issues.apache.org/jira/browse/NIFI-11191
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Reporter: David Handermann
Assignee: David Handermann
The {{HikariCPConnectionPoolTest}} fails intermittently due to timeouts when
attempting to retrieve a connection from the file-backed Derby Data Source:
{noformat}
Caused by: java.sql.SQLTimeoutException: Login timeout exceeded.
at
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
Source)
at org.apache.derby.jdbc.InternalDriver.timeLogin(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.EmbeddedDriver.connect(Unknown Source)
at
com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
at
com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:115)
at
com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
at
org.apache.nifi.dbcp.HikariCPConnectionPool.getConnection(HikariCPConnectionPool.java:343)
... 72 more
Caused by: ERROR XBDA0: Login timeout exceeded.
at org.apache.derby.iapi.error.StandardException.newException(Unknown
Source)
at
org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown
Source)
... 88 more
Error: Errors:
Error:HikariCPConnectionPoolTest.testCreateInsertSelect:172 Process
Connection retrieval failed
{noformat}
The problem appears to be related to file locking and concurrent test execution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-11190) Add provenance events to GetHubSpot and GetShopify
[ https://issues.apache.org/jira/browse/NIFI-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lehel Boér updated NIFI-11190: -- Status: Patch Available (was: In Progress) > Add provenance events to GetHubSpot and GetShopify > -- > > Key: NIFI-11190 > URL: https://issues.apache.org/jira/browse/NIFI-11190 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Lehel Boér >Assignee: Lehel Boér >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > These processors should send RECEIVE provenance events when a flowfile is > created from external source. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] Lehel44 opened a new pull request, #6965: NIFI-11190: Added provenance events to GetHubSpot and GetShopify
Lehel44 opened a new pull request, #6965: URL: https://github.com/apache/nifi/pull/6965 # Summary [NIFI-11190](https://issues.apache.org/jira/browse/NIFI-11190) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-11190) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1497: MINIFICPP-1869 Metrics-related refactoring
fgerlits commented on code in PR #1497:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1497#discussion_r1107133908
##
libminifi/include/c2/C2MetricsPublisher.h:
##
@@ -24,69 +24,53 @@
#include
#include
#include
-#include
+#include
#include "c2/C2Agent.h"
#include "core/controller/ControllerServiceProvider.h"
#include "properties/Configure.h"
#include "core/logging/Logger.h"
#include "core/state/nodes/MetricsBase.h"
-#include "core/Repository.h"
-#include "core/ContentRepository.h"
#include "core/ProcessGroup.h"
-#include "core/Flow.h"
+#include "core/Core.h"
#include "utils/file/FileSystem.h"
#include "core/state/nodes/ResponseNodeLoader.h"
#include "utils/Id.h"
+#include "core/state/MetricsPublisher.h"
namespace org::apache::nifi::minifi::c2 {
-class C2Client : public core::Flow, public state::response::NodeReporter {
+class C2MetricsPublisher : public state::response::NodeReporter, public
state::MetricsPublisher {
public:
- C2Client(
- std::shared_ptr configuration,
std::shared_ptr provenance_repo,
- std::shared_ptr flow_file_repo,
std::shared_ptr content_repo,
- std::unique_ptr flow_configuration,
std::shared_ptr filesystem,
- std::function request_restart,
- std::shared_ptr logger =
core::logging::LoggerFactory::getLogger());
+ using MetricsPublisher::MetricsPublisher;
+
+ MINIFIAPI static constexpr const char* Description = "Class that provides C2
metrics to the C2Agent";
- void initialize(core::controller::ControllerServiceProvider *controller,
state::Pausable *pause_handler, state::StateMonitor* update_sink);
std::optional
getMetricsNode(const std::string& metrics_class) const override;
std::vector
getHeartbeatNodes(bool include_manifest) const override;
- void stopC2();
- void initializeResponseNodes(core::ProcessGroup* root);
- void clearResponseNodes();
+ /**
+ * Retrieves the agent manifest to be sent as a response to C2 DESCRIBE
manifest
+ * @return the agent manifest response node
+ */
Review Comment:
this comment is the same as in the parent class, so I think it could be
removed
##
libminifi/src/core/state/MetricsPublisherFactory.cpp:
##
@@ -0,0 +1,45 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "core/state/MetricsPublisherFactory.h"
+
+namespace org::apache::nifi::minifi::state {
+
+std::unique_ptr createMetricsPublisher(const std::string&
name, const std::shared_ptr& configuration,
+const std::shared_ptr&
response_node_loader) {
+ auto ptr = core::ClassLoader::getDefaultClassLoader().instantiate(name,
name);
+ if (!ptr) {
+throw std::runtime_error("Configured metrics publisher class \"" + name +
"\" could not be instantiated.");
+ }
+
+ auto metrics_publisher =
utils::dynamic_unique_cast(std::move(ptr));
+ if (!metrics_publisher) {
+throw std::runtime_error("Configured metrics publisher class \"" + name +
"\" could not be instantiated.");
Review Comment:
it would be better to have different exception messages in the two cases, eg:
```suggestion
throw std::runtime_error("Configured metrics publisher class \"" + name
+ "\" is not a MetricsPublisher.");
```
##
libminifi/src/core/state/MetricsPublisherFactory.cpp:
##
@@ -0,0 +1,45 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "core/state/MetricsPublisherFactory.h"
+
+namespace
[jira] [Created] (NIFI-11190) Add provenance events to GetHubSpot and GetShopify
Lehel Boér created NIFI-11190: - Summary: Add provenance events to GetHubSpot and GetShopify Key: NIFI-11190 URL: https://issues.apache.org/jira/browse/NIFI-11190 Project: Apache NiFi Issue Type: Improvement Reporter: Lehel Boér Assignee: Lehel Boér These processors should send RECEIVE provenance events when a flowfile is created from external source. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1490: MINIFICPP-2022 Add valid repository size metrics for all repositories
szaszm commented on code in PR #1490:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1490#discussion_r1108677895
##
libminifi/include/core/Repository.h:
##
@@ -54,28 +54,22 @@ constexpr auto MAX_REPOSITORY_STORAGE_SIZE = 10_MiB;
constexpr auto MAX_REPOSITORY_ENTRY_LIFE_TIME = std::chrono::minutes(10);
constexpr auto REPOSITORY_PURGE_PERIOD = std::chrono::milliseconds(2500);
-class Repository : public core::CoreComponent {
+class Repository : public core::ReportableRepository {
Review Comment:
I would roughly keep the structure, but rename `ReportableRepository` to
`RepositoryMetricsSource`, remove the `CoreComponent` inheritance and the
constructors (e.g. move the inheritance to `Repository`), and practically make
it an interface only, with no data other than the vtable.
After the change, the is-a relationship is sound: `Repository` is a
`RepositoryMetricsSource` (i.e. source of repository metrics).
And `RepositoryMetricsSource` has no requirements other than certain
functions of the interface. Decoupled from component and directory dependencies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1490: MINIFICPP-2022 Add valid repository size metrics for all repositories
szaszm commented on code in PR #1490:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1490#discussion_r1108677895
##
libminifi/include/core/Repository.h:
##
@@ -54,28 +54,22 @@ constexpr auto MAX_REPOSITORY_STORAGE_SIZE = 10_MiB;
constexpr auto MAX_REPOSITORY_ENTRY_LIFE_TIME = std::chrono::minutes(10);
constexpr auto REPOSITORY_PURGE_PERIOD = std::chrono::milliseconds(2500);
-class Repository : public core::CoreComponent {
+class Repository : public core::ReportableRepository {
Review Comment:
I would roughly keep the structure, but rename `ReportableRepository` to
`RepositoryMetricsSource`, remove the `CoreComponent` inheritance and the
constructors (e.g. move the inheritance to `Repository`), and practically make
it an interface only, with no data other than the vtable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-11144) Fix failing tests for ConsumeJMS/PublishJMS
[ https://issues.apache.org/jira/browse/NIFI-11144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689798#comment-17689798 ] ASF subversion and git services commented on NIFI-11144: Commit 0d4f1523fed712623a66dcf52ec42f08a71c7e88 in nifi's branch refs/heads/main from Nandor Soma Abonyi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0d4f1523fe ] NIFI-11144 Fix failing tests for ConsumeJMS/PublishJMS This closes #6930. Signed-off-by: Tamas Palfy > Fix failing tests for ConsumeJMS/PublishJMS > --- > > Key: NIFI-11144 > URL: https://issues.apache.org/jira/browse/NIFI-11144 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Nandor Soma Abonyi >Assignee: Nandor Soma Abonyi >Priority: Major > Labels: JMS > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] tpalfy commented on pull request #6930: NIFI-11144 Fix failing tests for ConsumeJMS/PublishJMS
tpalfy commented on PR #6930: URL: https://github.com/apache/nifi/pull/6930#issuecomment-1433208805 LGTM Thank you for your work @nandorsoma ! Merged into main. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #6930: NIFI-11144 Fix failing tests for ConsumeJMS/PublishJMS
asfgit closed pull request #6930: NIFI-11144 Fix failing tests for ConsumeJMS/PublishJMS URL: https://github.com/apache/nifi/pull/6930 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1503: MINIFICPP-2039 Dust off minificontroller
lordgamez commented on code in PR #1503:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1503#discussion_r1108569955
##
controller/tests/ControllerTests.cpp:
##
@@ -0,0 +1,479 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include "TestBase.h"
+#include "Catch.h"
+#include "io/ClientSocket.h"
+#include "core/Processor.h"
+#include "Controller.h"
+#include "c2/ControllerSocketProtocol.h"
+#include "utils/IntegrationTestUtils.h"
+#include "c2/ControllerSocketMetricsPublisher.h"
+#include "core/controller/ControllerServiceProvider.h"
+#include "controllers/SSLContextService.h"
+
+#include "state/UpdateController.h"
+
+using namespace std::literals::chrono_literals;
+
+namespace org::apache::nifi::minifi::test {
+
+class TestStateController : public minifi::state::StateController {
+ public:
+ TestStateController()
+: is_running(false) {
+ }
+
+ std::string getComponentName() const override {
+return "TestStateController";
+ }
+
+ minifi::utils::Identifier getComponentUUID() const override {
+static auto dummyUUID =
minifi::utils::Identifier::parse("12345678-1234-1234-1234-123456789abc").value();
+return dummyUUID;
+ }
+
+ int16_t start() override {
+is_running = true;
+return 0;
+ }
+
+ int16_t stop() override {
+is_running = false;
+return 0;
+ }
+
+ bool isRunning() const override {
+return is_running;
+ }
+
+ int16_t pause() override {
+return 0;
+ }
+
+ int16_t resume() override {
+return 0;
+ }
+
+ std::atomic is_running;
+};
+
+class TestUpdateSink : public minifi::state::StateMonitor {
+ public:
+ explicit TestUpdateSink(std::shared_ptr controller)
+: is_running(true),
+ clear_calls(0),
+ controller(std::move(controller)),
+ update_calls(0) {
+ }
+
+ void executeOnComponent(const std::string&,
std::function func) override {
+func(*controller);
+ }
+
+ void
executeOnAllComponents(std::function
func) override {
+func(*controller);
+ }
+
+ std::string getComponentName() const override {
+return "TestUpdateSink";
+ }
+
+ minifi::utils::Identifier getComponentUUID() const override {
+static auto dummyUUID =
minifi::utils::Identifier::parse("12345678-1234-1234-1234-123456789abc").value();
+return dummyUUID;
+ }
+
+ int16_t start() override {
+is_running = true;
+return 0;
+ }
+
+ int16_t stop() override {
+is_running = false;
+return 0;
+ }
+
+ bool isRunning() const override {
+return is_running;
+ }
+
+ int16_t pause() override {
+return 0;
+ }
+
+ int16_t resume() override {
+return 0;
+ }
+ std::vector getTraces() override {
+std::vector traces;
+return traces;
+ }
+
+ int16_t drainRepositories() override {
+return 0;
+ }
+
+ std::map>
getDebugInfo() override {
+return {};
+ }
+
+ int16_t clearConnection(const std::string& /*connection*/) override {
+clear_calls++;
+return 0;
+ }
+
+ int16_t applyUpdate(const std::string& /*source*/, const std::string&
/*configuration*/, bool /*persist*/ = false, const std::optional&
/*flow_id*/ = std::nullopt) override {
+update_calls++;
+return 0;
+ }
+
+ int16_t applyUpdate(const std::string& /*source*/, const
std::shared_ptr& /*updateController*/) override {
+return 0;
Review Comment:
I think we should, I suppose this overload was unused in the tests, so the
stub was not implemented for it, updated in
2a82c349ac8f334e2a3f49640f136b26e0b7ce23
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1504: MINIFICPP-1961 Python scripting compatibility with multiple python minor versions
szaszm commented on code in PR #1504:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1504#discussion_r1108552010
##
PROCESSORS.md:
##
@@ -20,10 +20,11 @@
- [DeleteAzureDataLakeStorage](#deleteazuredatalakestorage)
- [DeleteGCSObject](#deletegcsobject)
- [DeleteS3Object](#deletes3object)
+- [ExecuteLuaScript](#executeluascript)
- [ExecuteProcess](#executeprocess)
- [ExecutePythonProcessor](#executepythonprocessor)
+- [ExecutePythonScript](#executepythonscript)
- [ExecuteSQL](#executesql)
-- [ExecuteScript](#executescript)
- [ExtractText](#extracttext)
Review Comment:
We should go back to a single processor in the docs as well.
##
bootstrap.sh:
##
@@ -263,9 +263,8 @@ add_option HTTP_CURL_ENABLED ${TRUE} "DISABLE_CURL"
add_option LIBARCHIVE_ENABLED ${TRUE} "DISABLE_LIBARCHIVE"
add_dependency LIBARCHIVE_ENABLED "libarchive"
-add_option EXECUTE_SCRIPT_ENABLED ${FALSE} "ENABLE_SCRIPTING"
-add_dependency EXECUTE_SCRIPT_ENABLED "python"
-add_dependency EXECUTE_SCRIPT_ENABLED "lua"
+add_option PYTHON_SCRIPTING_ENABLED ${FALSE} "ENABLE_PYTHON_SCRIPTING"
+add_option LUA_SCRIPTING_ENABLED ${FALSE} "ENABLE_LUA_SCRIPTING"
Review Comment:
I imagined the ExecuteScript + script executors architecture with 3
extensions:
- scripting: ExecuteScript without any executors
- python: PythonScriptExecutor and support for python processors
- lua: LuaScriptExecutor and lua processors
How does it work now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-11189) Restarting NiFi after failing to upgrade flow can cause NiFi to fail startup
[
https://issues.apache.org/jira/browse/NIFI-11189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689791#comment-17689791
]
ASF subversion and git services commented on NIFI-11189:
Commit 6f2b9e5f65198b0eb3870d9ae268429803f31a3f in nifi's branch
refs/heads/support/nifi-1.x from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6f2b9e5f65 ]
NIFI-11189: When synchronizing a ProcessGroup to match a VersionedProcessGroup,
do not remove the temporary funnel until the very end. This is important if the
temporary funnel already exists in the flow on startup
Signed-off-by: Bence Simon
This closes #6963
> Restarting NiFi after failing to upgrade flow can cause NiFi to fail startup
>
>
> Key: NIFI-11189
> URL: https://issues.apache.org/jira/browse/NIFI-11189
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 2.0.0, 1.21.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When a user updates a group that is under Version Control from one version to
> another, sometimes the destination of a connection now needs to point to a
> new component. And the current destination gets deleted, while the new
> component hasn't yet been created. To handle this, we create a temporary
> funnel and set the connection’s destination to that funnel.
>
> Once all components are created, we then move the connections to their
> intended destination. Then delete the temporary funnel.
>
> We have an issue, however. If, for some reason, we fail to complete the flow
> upgrade, that funnel may remain. We are not handling this case where this
> temporary funnel already exists in the flow definition very well on restart.
> This can result in an error such as:
> {code:java}
> 2023-02-15 16:40:03,347 WARN [main] org.eclipse.jetty.webapp.WebAppContext
> Failed startup of context
> o.e.j.w.WebAppContext@345af277{nifi-api,/nifi-api,file:///opt/nifi-1.18.0.2.1.5.1001-1/work/jetty/nifi-web-api-1.18.0.2.1.5.1001-1.war/webapp/,UNAVAILABLE}{./work/nar/extensions/nifi-server-nar-1.18.0.2.1.5.1001-1.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.18.0.2.1.5.1001-1.war}
> org.apache.nifi.controller.serialization.FlowSynchronizationException:
> java.lang.IllegalArgumentException: Connection has a destination with
> identifier c594bee4-b49e-34a3-8795-732d890df61f but no component could be
> found in the Process Group with a corresponding identifier
> at
> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:454)
> at
> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.sync(VersionedFlowSynchronizer.java:205)
> at
> org.apache.nifi.controller.serialization.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:42)
> at
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1525)
> at
> org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:104)
> at
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:837)
> at
> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:558)
> at
> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
> at
> org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002)
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:765)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at
>
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1501: MINIFICPP-1887 Add default connection size limits
martinzink commented on code in PR #1501:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1501#discussion_r1108533223
##
libminifi/include/Connection.h:
##
@@ -58,8 +58,8 @@ class Connection : public core::Connectable {
Connection(const Connection ) = delete;
Connection =(const Connection ) = delete;
- static constexpr uint64_t DEFAULT_BACKPRESSURE_SIZE_THRESHOLD = 1;
- static constexpr uint64_t DEFAULT_BACKPRESSURE_DATA_THRESHOLD = 10_MB;
+ static constexpr uint64_t DEFAULT_BACKPRESSURE_THRESHOLD_COUNT = 1;
+ static constexpr uint64_t DEFAULT_BACKPRESSURE_THRESHOLD_DATA_SIZE = 10_MB;
Review Comment:
based on our conversation I've modified it to 2000 / 100MB in
https://github.com/apache/nifi-minifi-cpp/pull/1501/commits/75e3d93ddd845f30c8674ca03b4f62b2bf0faf97
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1480: MINIFICPP-2007 Add compression options for flowfile and content repo
lordgamez commented on code in PR #1480:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1480#discussion_r1108469672
##
conf/minifi.properties:
##
@@ -29,9 +29,11 @@ nifi.provenance.repository.max.storage.time=1 MIN
nifi.provenance.repository.max.storage.size=1 MB
nifi.flowfile.repository.directory.default=${MINIFI_HOME}/flowfile_repository
nifi.flowfile.checkpoint.directory.default=${MINIFI_HOME}/flowfile_checkpoint
+# nifi.flowfile.repository.rocksdb.compression=auto
Review Comment:
I tried switching compression types between restarts and the flow continued
where it left off previously. I could see in the rocksdb log files the used
compression was changed to the new one in the minifi property file so it seems
to me that rocksdb could handle the config change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-11189) Restarting NiFi after failing to upgrade flow can cause NiFi to fail startup
[
https://issues.apache.org/jira/browse/NIFI-11189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689761#comment-17689761
]
ASF subversion and git services commented on NIFI-11189:
Commit fed175d46f794cffae8695df6eb63adab4e8a27a in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=fed175d46f ]
NIFI-11189: When synchronizing a ProcessGroup to match a VersionedProcessGroup,
do not remove the temporary funnel until the very end. This is important if the
temporary funnel already exists in the flow on startup
Signed-off-by: Bence Simon
This closes #6963
> Restarting NiFi after failing to upgrade flow can cause NiFi to fail startup
>
>
> Key: NIFI-11189
> URL: https://issues.apache.org/jira/browse/NIFI-11189
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 2.0.0, 1.21.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When a user updates a group that is under Version Control from one version to
> another, sometimes the destination of a connection now needs to point to a
> new component. And the current destination gets deleted, while the new
> component hasn't yet been created. To handle this, we create a temporary
> funnel and set the connection’s destination to that funnel.
>
> Once all components are created, we then move the connections to their
> intended destination. Then delete the temporary funnel.
>
> We have an issue, however. If, for some reason, we fail to complete the flow
> upgrade, that funnel may remain. We are not handling this case where this
> temporary funnel already exists in the flow definition very well on restart.
> This can result in an error such as:
> {code:java}
> 2023-02-15 16:40:03,347 WARN [main] org.eclipse.jetty.webapp.WebAppContext
> Failed startup of context
> o.e.j.w.WebAppContext@345af277{nifi-api,/nifi-api,file:///opt/nifi-1.18.0.2.1.5.1001-1/work/jetty/nifi-web-api-1.18.0.2.1.5.1001-1.war/webapp/,UNAVAILABLE}{./work/nar/extensions/nifi-server-nar-1.18.0.2.1.5.1001-1.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.18.0.2.1.5.1001-1.war}
> org.apache.nifi.controller.serialization.FlowSynchronizationException:
> java.lang.IllegalArgumentException: Connection has a destination with
> identifier c594bee4-b49e-34a3-8795-732d890df61f but no component could be
> found in the Process Group with a corresponding identifier
> at
> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:454)
> at
> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.sync(VersionedFlowSynchronizer.java:205)
> at
> org.apache.nifi.controller.serialization.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:42)
> at
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1525)
> at
> org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:104)
> at
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:837)
> at
> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:558)
> at
> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
> at
> org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002)
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:765)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at
>
[GitHub] [nifi] asfgit closed pull request #6963: NIFI-11189: When synchronizing a ProcessGroup to match a VersionedPro…
asfgit closed pull request #6963: NIFI-11189: When synchronizing a ProcessGroup to match a VersionedPro… URL: https://github.com/apache/nifi/pull/6963 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] simonbence commented on a diff in pull request #6963: NIFI-11189: When synchronizing a ProcessGroup to match a VersionedPro…
simonbence commented on code in PR #6963:
URL: https://github.com/apache/nifi/pull/6963#discussion_r1108431564
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/flow/synchronization/StandardVersionedComponentSynchronizer.java:
##
@@ -395,76 +395,80 @@ private void synchronize(final ProcessGroup group, final
VersionedProcessGroup p
final Set connectionsWithTempDestination =
updateConnectionDestinations(group, proposed, connectionsByVersionedId);
try {
-final Map funnelsByVersionedId =
componentsById(group, ProcessGroup::getFunnels);
-final Map processorsByVersionedId =
componentsById(group, ProcessGroup::getProcessors);
-final Map inputPortsByVersionedId =
componentsById(group, ProcessGroup::getInputPorts);
-final Map outputPortsByVersionedId =
componentsById(group, ProcessGroup::getOutputPorts);
-final Map labelsByVersionedId =
componentsById(group, ProcessGroup::getLabels, Label::getIdentifier,
Label::getVersionedComponentId);
-final Map rpgsByVersionedId =
componentsById(group, ProcessGroup::getRemoteProcessGroups,
-RemoteProcessGroup::getIdentifier,
RemoteProcessGroup::getVersionedComponentId);
-final Map childGroupsByVersionedId =
componentsById(group, ProcessGroup::getProcessGroups,
ProcessGroup::getIdentifier, ProcessGroup::getVersionedComponentId);
-
-removeMissingProcessors(group, proposed, processorsByVersionedId);
-removeMissingFunnels(group, proposed, funnelsByVersionedId);
-removeMissingInputPorts(group, proposed, inputPortsByVersionedId);
-removeMissingOutputPorts(group, proposed,
outputPortsByVersionedId);
-removeMissingLabels(group, proposed, labelsByVersionedId);
-removeMissingRpg(group, proposed, rpgsByVersionedId);
-removeMissingChildGroups(group, proposed,
childGroupsByVersionedId);
-
-// Synchronize Child Process Groups
-synchronizeChildGroups(group, proposed,
versionedParameterContexts, childGroupsByVersionedId,
parameterProviderReferences, topLevelGroup);
-
-synchronizeFunnels(group, proposed, funnelsByVersionedId);
-synchronizeInputPorts(group, proposed, proposedPortFinalNames,
inputPortsByVersionedId);
-synchronizeOutputPorts(group, proposed, proposedPortFinalNames,
outputPortsByVersionedId);
-synchronizeLabels(group, proposed, labelsByVersionedId);
-synchronizeProcessors(group, proposed, processorsByVersionedId,
topLevelGroup);
-synchronizeRemoteGroups(group, proposed, rpgsByVersionedId);
-} finally {
-// Make sure that we reset the connections
-restoreConnectionDestinations(group, proposed,
connectionsByVersionedId, connectionsWithTempDestination);
-removeTemporaryFunnel(group);
-}
-
-Map newParameters = new HashMap<>();
-if (!proposedParameterContextExistsBeforeSynchronize &&
this.context.getFlowMappingOptions().isMapControllerServiceReferencesToVersionedId())
{
-Map controllerServiceVersionedIdToId =
group.getControllerServices(false)
-.stream()
-.filter(controllerServiceNode ->
controllerServiceNode.getVersionedComponentId().isPresent())
-.collect(Collectors.toMap(
-controllerServiceNode ->
controllerServiceNode.getVersionedComponentId().get(),
-ComponentNode::getIdentifier
-));
-
-ParameterContext parameterContext = group.getParameterContext();
-
-if (parameterContext != null) {
-parameterContext.getParameters().forEach((descriptor,
parameter) -> {
-List
referencedControllerServiceData = parameterContext
-.getParameterReferenceManager()
-.getReferencedControllerServiceData(parameterContext,
descriptor.getName());
-
-if (referencedControllerServiceData.isEmpty()) {
-newParameters.put(descriptor.getName(), parameter);
-} else {
-final Parameter adjustedParameter = new
Parameter(parameter.getDescriptor(),
controllerServiceVersionedIdToId.get(parameter.getValue()));
-newParameters.put(descriptor.getName(),
adjustedParameter);
-}
-});
+try {
+final Map funnelsByVersionedId =
componentsById(group, ProcessGroup::getFunnels);
+final Map processorsByVersionedId =
componentsById(group, ProcessGroup::getProcessors);
+final Map inputPortsByVersionedId =
componentsById(group, ProcessGroup::getInputPorts);
+final Map outputPortsByVersionedId =
componentsById(group,
[GitHub] [nifi] simonbence commented on pull request #6963: NIFI-11189: When synchronizing a ProcessGroup to match a VersionedPro…
simonbence commented on PR #6963: URL: https://github.com/apache/nifi/pull/6963#issuecomment-1433028967 Code changes are looking good and I was able to execute the failure scenario whitout the issue appearing. I reused the test flows from [my previous PR](https://github.com/apache/nifi/pull/6937) and tried to start up NiFi with the "corrupted" flow. It did succeed. I will merge soonish! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1480: MINIFICPP-2007 Add compression options for flowfile and content repo
szaszm commented on code in PR #1480:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1480#discussion_r1108397217
##
conf/minifi.properties:
##
@@ -29,9 +29,11 @@ nifi.provenance.repository.max.storage.time=1 MIN
nifi.provenance.repository.max.storage.size=1 MB
nifi.flowfile.repository.directory.default=${MINIFI_HOME}/flowfile_repository
nifi.flowfile.checkpoint.directory.default=${MINIFI_HOME}/flowfile_checkpoint
+# nifi.flowfile.repository.rocksdb.compression=auto
Review Comment:
Is it possible to change these settings without losing existing repository
data? So basically does it detect existing data in a different format and do
conversion at startup? If not, is this something we could make work, possibly
in a followup?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on pull request #1511: MINIFICPP-1716 Recover core dumps from CI
lordgamez commented on PR #1511: URL: https://github.com/apache/nifi-minifi-cpp/pull/1511#issuecomment-1432842782 One thing to consider is that in case of crashing in macos, ubuntu and ubuntu-clang builds we can pinpoint the line numbers from the crashdump with the RelWithDebInfo build type, but we also use 4 times the cache size of the Release build which will result in more cache misses in the future. Is it worth to have the debuginfo in case of a crash for slower builds in the future? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1504: MINIFICPP-1961 Python scripting compatibility with multiple python minor versions
martinzink commented on code in PR #1504: URL: https://github.com/apache/nifi-minifi-cpp/pull/1504#discussion_r1108201009 ## cmake/MiNiFiOptions.cmake: ## @@ -102,9 +101,8 @@ add_minifi_option(ENABLE_SQL "Enables the SQL Suite of Tools." ON) add_minifi_option(ENABLE_MQTT "Enables the mqtt extension." OFF) add_minifi_option(ENABLE_PCAP "Enables the PCAP extension." OFF) add_minifi_option(ENABLE_LIBRDKAFKA "Enables the librdkafka extension." ON) -add_minifi_option(ENABLE_SCRIPTING "Enables the scripting extensions." OFF) add_minifi_option(ENABLE_LUA_SCRIPTING "Enables lua scripting" OFF) -add_minifi_option(DISABLE_PYTHON_SCRIPTING "Disables python scripting" OFF) +add_minifi_option(ENABLE_PYTHON_SCRIPTING "Enables python scripting" OFF) Review Comment: There was ENABLE_PYTHON which controlled the python bindings in nanofi. We didnt really use that one so I've removed the whole nanofi-python bindings entirely. Prior to this PR, there was ENABLE_SCRIPTING which was OFF by default, and there was two additional flags namely DISABLE_PYTHON_SCRIPTING(OFF), ENABLE_LUA_SCRIPTING(OFF), which only mattered if the ENABLE_SCRIPTING was ON. Instead of this I've added ENABLE_PYTHON_SCRIPTING ~(ENABLE_SCRIPTING && NOT DISABLE_PYTHON_SCRIPTING) and ENABLE_LUA_SCRIPTING ~(ENABLE_SCRIPTING && ENABLE_LUA_SCRIPTING), so long story short by default both lua and python was disabled and they still are disabled by default. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1510: MINIFICPP-2047 added reverseLookup to Expression Language
martinzink commented on code in PR #1510:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1510#discussion_r1108173295
##
EXPRESSIONS.md:
##
@@ -1590,6 +1591,31 @@ to evaluate additional functions against.
${allMatchingAttributes('a.*'):count()} ):gt(3)}` returns true if there are
more than 3 attributes whose names begin with the letter a.
+### reverseLookup
Review Comment:
good point, I've renamed it in
https://github.com/apache/nifi-minifi-cpp/pull/1510/commits/31dacdb37bcf4cdad0f32d6311a047f549920dd4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez opened a new pull request, #1511: MINIFICPP-1716 Recover core dumps from CI
lordgamez opened a new pull request, #1511: URL: https://github.com/apache/nifi-minifi-cpp/pull/1511 Uploads core dumps and binaries when test crashes in CI. Also fixes built type argument passing in docker builds. Unfortunately centos cannot be built with RelWithDebInfo build type in the CI workflow as the build runs out of disk space. https://issues.apache.org/jira/browse/MINIFICPP-1716 - Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
