[jira] [Created] (MINIFICPP-719) Fix bootstrap issues
Mr TheSegfault created MINIFICPP-719: Summary: Fix bootstrap issues Key: MINIFICPP-719 URL: https://issues.apache.org/jira/browse/MINIFICPP-719 Project: NiFi MiNiFi C++ Issue Type: Bug Reporter: Mr TheSegfault Assignee: Mr TheSegfault 1) Allow tests to be disabled. 2) Correctly identify that kafka can or cannot be built on versions of gcc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (NIFI-5956) Allow Disabling Block Cache With HBase Scan Processor
[
https://issues.apache.org/jira/browse/NIFI-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandish Kumar HN reassigned NIFI-5956:
--
Assignee: Sandish Kumar HN
> Allow Disabling Block Cache With HBase Scan Processor
> -
>
> Key: NIFI-5956
> URL: https://issues.apache.org/jira/browse/NIFI-5956
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: BELUGA BEHR
>Assignee: Sandish Kumar HN
>Priority: Major
>
> {quote}
> Scan instances can be set to use the block cache in the RegionServer via the
> setCacheBlocks method. For input Scans to MapReduce jobs, this should be
> false.
> https://hbase.apache.org/book.html#perf.hbase.client.blockcache
> {quote}
> Please add a configurable option to the HBase Scan processor to allow users
> to toggle if the Scan should affect the HBase block cache or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5972) LdapUserGroupProvider logging
[ https://issues.apache.org/jira/browse/NIFI-5972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-5972: -- Description: When users and groups are synced from LDAP, there are four different log warnings when group membership is configured but we encounter users or groups that do not match the configured criteria. The warnings were originally added with the intent to identify issues with the LDAP config, however there are legitimate cases where the scenario is expected. For instance, when a user does not belong to any groups or when the user belongs to a group that is not relevant for NiFi and not identified in the group search. When we encounter these cases the warnings can flood the log files invalid warn messages. (was: When users and groups are synced from LDAP, there are four different log warnings when group membership is configured but we encounter users or groups that do not match the configured criteria. The warnings were originally added with the intent to identify issues with the LDAP config, there are legitimate cases where the case is expected. For instance, when a user does not belong to any groups or when the user belongs to a group that is not relevant for NiFi and not identified in the group search.) > LdapUserGroupProvider logging > - > > Key: NIFI-5972 > URL: https://issues.apache.org/jira/browse/NIFI-5972 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > > When users and groups are synced from LDAP, there are four different log > warnings when group membership is configured but we encounter users or groups > that do not match the configured criteria. The warnings were originally added > with the intent to identify issues with the LDAP config, however there are > legitimate cases where the scenario is expected. For instance, when a user > does not belong to any groups or when the user belongs to a group that is not > relevant for NiFi and not identified in the group search. When we encounter > these cases the warnings can flood the log files invalid warn messages. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5972) LdapUserGroupProvider logging
[ https://issues.apache.org/jira/browse/NIFI-5972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-5972: -- Status: Patch Available (was: Open) > LdapUserGroupProvider logging > - > > Key: NIFI-5972 > URL: https://issues.apache.org/jira/browse/NIFI-5972 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > > When users and groups are synced from LDAP, there are four different log > warnings when group membership is configured but we encounter users or groups > that do not match the configured criteria. The warnings were originally added > with the intent to identify issues with the LDAP config, there are legitimate > cases where the case is expected. For instance, when a user does not belong > to any groups or when the user belongs to a group that is not relevant for > NiFi and not identified in the group search. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5972) LdapUserGroupProvider logging
[ https://issues.apache.org/jira/browse/NIFI-5972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-5972: -- Priority: Trivial (was: Major) > LdapUserGroupProvider logging > - > > Key: NIFI-5972 > URL: https://issues.apache.org/jira/browse/NIFI-5972 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Trivial > > When users and groups are synced from LDAP, there are four different log > warnings when group membership is configured but we encounter users or groups > that do not match the configured criteria. The warnings were originally added > with the intent to identify issues with the LDAP config, there are legitimate > cases where the case is expected. For instance, when a user does not belong > to any groups or when the user belongs to a group that is not relevant for > NiFi and not identified in the group search. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] mcgilman opened a new pull request #3274: NIFI-5972: LdapUserGroupProvider logging
mcgilman opened a new pull request #3274: NIFI-5972: LdapUserGroupProvider logging URL: https://github.com/apache/nifi/pull/3274 NIFI-5972: - Converting some warning log messages to debug as they could possibly be due to a valid scenario like NiFi users belonging to a group that is not relevant to NiFi. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFI-5972) LdapUserGroupProvider logging
Matt Gilman created NIFI-5972: - Summary: LdapUserGroupProvider logging Key: NIFI-5972 URL: https://issues.apache.org/jira/browse/NIFI-5972 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Matt Gilman Assignee: Matt Gilman When users and groups are synced from LDAP, there are four different log warnings when group membership is configured but we encounter users or groups that do not match the configured criteria. The warnings were originally added with the intent to identify issues with the LDAP config, there are legitimate cases where the case is expected. For instance, when a user does not belong to any groups or when the user belongs to a group that is not relevant for NiFi and not identified in the group search. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5971) ExecuteSQL Avro schema: all fields are nullable
Marcio Sugar created NIFI-5971: -- Summary: ExecuteSQL Avro schema: all fields are nullable Key: NIFI-5971 URL: https://issues.apache.org/jira/browse/NIFI-5971 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 1.7.1 Environment: Ubuntu 16.04 Apache NiFi 1.7.1 IBM DB2 for Linux, UNIX and Windows 10.5.0.7, 10.5.0.8 IBM Data Server Driver for JDBC and SQLJ, JDBC 4.0 Driver (db2jcc4.jar) 4.19.26 / v10.5 FP6, 4.19.72 / v10.5 FP9 Reporter: Marcio Sugar JdbcCommon#createSchema creates an Avro schema with nullable types for all fields. It should check with java.sql.ResultSetMetaData#isNullable instead. It's the same issue discussed on dev list [here|https://lists.apache.org/list.html?d...@nifi.apache.org:2018-12] a few months ago. A workaround exists, but it's inconvenient when you have lots of tables to deal with. I like the solution proposed by Matt Burgess, the "Honor Non-Nullable Fields" property. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] thenatog commented on a change in pull request #3273: NIFI-5968 - Added the X-XSS-Protection and Strict-Transport-Security …
thenatog commented on a change in pull request #3273: NIFI-5968 - Added the
X-XSS-Protection and Strict-Transport-Security …
URL: https://github.com/apache/nifi/pull/3273#discussion_r250238788
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/test/java/org/apache/nifi/web/server/JettyServerTest.java
##
@@ -155,29 +155,4 @@ public void
testConfigureSslContextFactoryWithPkcsTrustStore() {
verify(contextFactory).setTrustStoreType(trustStoreType);
verify(contextFactory).setTrustStoreProvider(BouncyCastleProvider.PROVIDER_NAME);
}
-
-@Test
-public void testNoDuplicateXFrameOptions() throws NoSuchFieldException,
IllegalAccessException, ServletException, IOException {
Review comment:
Removed. I think this was simply a bad test and that what it was checking
for is handled by another unit test in HTTPHeaderFiltersTest.
I also think I should probably add some integration tests to check for these
HTTP headers in actual responses from JettyServer? But I didn't know where to
get an example on how we do that. At the least, the tests in
HTTPHeaderFiltersTest checks that each individual filter adds the headers we
expect. It just doesn't check whether the filters are added correctly and used
by JettyServer.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] thenatog commented on a change in pull request #3273: NIFI-5968 - Added the X-XSS-Protection and Strict-Transport-Security …
thenatog commented on a change in pull request #3273: NIFI-5968 - Added the
X-XSS-Protection and Strict-Transport-Security …
URL: https://github.com/apache/nifi/pull/3273#discussion_r250238788
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/test/java/org/apache/nifi/web/server/JettyServerTest.java
##
@@ -155,29 +155,4 @@ public void
testConfigureSslContextFactoryWithPkcsTrustStore() {
verify(contextFactory).setTrustStoreType(trustStoreType);
verify(contextFactory).setTrustStoreProvider(BouncyCastleProvider.PROVIDER_NAME);
}
-
-@Test
-public void testNoDuplicateXFrameOptions() throws NoSuchFieldException,
IllegalAccessException, ServletException, IOException {
Review comment:
Removed. I think this was simply a bad test and that what it was checking
for is handled by another unit test in HTTPHeaderFiltersTest.
I also think I should probably add some integration tests to check for these
HTTP headers in actual responses from JettyServer? But I didn't know where to
get an example on how we do that. At the least, the tests in
HTTPHeaderFiltersTest checks that each individual filter adds the headers we
expect.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] thenatog commented on a change in pull request #3273: NIFI-5968 - Added the X-XSS-Protection and Strict-Transport-Security …
thenatog commented on a change in pull request #3273: NIFI-5968 - Added the
X-XSS-Protection and Strict-Transport-Security …
URL: https://github.com/apache/nifi/pull/3273#discussion_r250238788
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/test/java/org/apache/nifi/web/server/JettyServerTest.java
##
@@ -155,29 +155,4 @@ public void
testConfigureSslContextFactoryWithPkcsTrustStore() {
verify(contextFactory).setTrustStoreType(trustStoreType);
verify(contextFactory).setTrustStoreProvider(BouncyCastleProvider.PROVIDER_NAME);
}
-
-@Test
-public void testNoDuplicateXFrameOptions() throws NoSuchFieldException,
IllegalAccessException, ServletException, IOException {
Review comment:
Removed. I think this was simply a bad test and that what it was checking
for is handled by another unit test in HTTPHeaderFiltersTest.
I also think I should probably add some integration tests to check for these
HTTP headers in actual responses, but I didn't know where to get an example on
how we do that.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] thenatog commented on a change in pull request #3273: NIFI-5968 - Added the X-XSS-Protection and Strict-Transport-Security …
thenatog commented on a change in pull request #3273: NIFI-5968 - Added the
X-XSS-Protection and Strict-Transport-Security …
URL: https://github.com/apache/nifi/pull/3273#discussion_r250236583
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java
##
@@ -602,6 +599,28 @@ private WebAppContext loadWar(final File warFile, final
String contextPath, fina
return webappContext;
}
+private void addHTTPHeaders(WebAppContext webappContext) {
+// Add a filter to set the X-Frame-Options header
+FilterHolder xfoFilter = new FilterHolder(new XFrameOptionsFilter());
+xfoFilter.setName(XFrameOptionsFilter.class.getSimpleName());
+webappContext.addFilter(xfoFilter, "/*",
EnumSet.allOf(DispatcherType.class));
+
+// Add a filter to set the Content Security Policy frame-ancestors
directive
+FilterHolder cspFilter = new FilterHolder(new
ContentSecurityPolicyFilter());
+cspFilter.setName(ContentSecurityPolicyFilter.class.getSimpleName());
+webappContext.addFilter(cspFilter, "/*",
EnumSet.allOf(DispatcherType.class));
+
+// Add a filter to set the HSTS header
Review comment:
I'm not certain of the performance impact. From what I've seen when
debugging Jetty Filters, the call stack is pretty deep. I believe that for each
request, Jetty will iterate through the contexts until it finds a context
match, and then iterate through the attached filters? I'm not certain how
significant the impact of adding a few more filters will be. Do we have a
request response time filter for request/response times or did I imagine that?
We could try using that to check how much longer requests take.
I believe adding X-XSS-Protection will reduce performance when the browser
performs XSS checks?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] thenatog commented on a change in pull request #3273: NIFI-5968 - Added the X-XSS-Protection and Strict-Transport-Security …
thenatog commented on a change in pull request #3273: NIFI-5968 - Added the X-XSS-Protection and Strict-Transport-Security … URL: https://github.com/apache/nifi/pull/3273#discussion_r250225707 ## File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java ## @@ -583,13 +586,7 @@ private WebAppContext loadWar(final File warFile, final String contextPath, fina // configure the max form size (3x the default) webappContext.setMaxFormContentSize(60); -// add a filter to set the X-Frame-Options filter -webappContext.addFilter(new FilterHolder(FRAME_OPTIONS_FILTER), "/*", EnumSet.allOf(DispatcherType.class)); - -// add a filter to set the Content Security Policy frame-ancestors directive -FilterHolder cspFilter = new FilterHolder(new ContentSecurityPolicyFilter()); -cspFilter.setName(ContentSecurityPolicyFilter.class.getSimpleName()); -webappContext.addFilter(cspFilter, "/*", EnumSet.allOf(DispatcherType.class)); +addHTTPHeaders(webappContext); Review comment: Ah yes, I thought about that and then forgot. I'll look more into that. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit closed pull request #475: MINIFICPP-718: Fix issue with docker verify with docker API changes
asfgit closed pull request #475: MINIFICPP-718: Fix issue with docker verify with docker API changes URL: https://github.com/apache/nifi-minifi-cpp/pull/475 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] apiri commented on issue #475: MINIFICPP-718: Fix issue with docker verify with docker API changes
apiri commented on issue #475: MINIFICPP-718: Fix issue with docker verify with docker API changes URL: https://github.com/apache/nifi-minifi-cpp/pull/475#issuecomment-456822386 @phrocker Sorry for the false alarm. It does not seem the script pulls the latest image, so I'm assuming I unfortunately had some dev versions of nifi images that conflicted Doing a scorched earth approach on all those images got everything working swimmingly. Will get this merged. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFI-5970) PutSQL with batch size > 1 + DBCPConnectionPoolLookup results in missing database.name
Bryan Bende created NIFI-5970: - Summary: PutSQL with batch size > 1 + DBCPConnectionPoolLookup results in missing database.name Key: NIFI-5970 URL: https://issues.apache.org/jira/browse/NIFI-5970 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.8.0 Reporter: Bryan Bende When the batch size is greater than 1 we end up passing null for the attributes that get passed in to the DBCPConnectionPoolLookup, this is because the attributes of each flow file could be different: [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/pattern/Put.java#L97] Part of the issue is that at this point in the code we have no idea if we are using a DBCPConnectionPoolLookup that requires the database.name attribute, or just a regular DBCPConnectionPool that doesn't. https://stackoverflow.com/questions/54312773/dbcpconnectionpoollookup-complaining-about-missing-database-name-even-when-it-is -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] phrocker edited a comment on issue #475: MINIFICPP-718: Fix issue with docker verify with docker API changes
phrocker edited a comment on issue #475: MINIFICPP-718: Fix issue with docker verify with docker API changes URL: https://github.com/apache/nifi-minifi-cpp/pull/475#issuecomment-456652571 Thanks. I will do a make clean to ensure I have a clean environment. I imagine I didn't see the issues because I had some stale build artifacts Edit: @apiri I can't replicate those failures. Can you verify that you've completely rebuilt from scratch ( not sure a make clean does it. I removed docker/minificppsource juset in case ) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFI-5969) Many S2S connections in state CLOSE_WAIT
jamescheng created NIFI-5969: Summary: Many S2S connections in state CLOSE_WAIT Key: NIFI-5969 URL: https://issues.apache.org/jira/browse/NIFI-5969 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.8.0 Environment: Linux Reporter: jamescheng Attachments: image-2019-01-23-16-46-33-081.png, image-2019-01-23-17-34-28-533.png Hi, We have a 4-node NiFi cluster. We use RPG to send flowfiles back to cluster. We faced the WEB UI slowness issue, so we found there are many S2S connection in CLOSE_WAIT state. After we restart the NiFi, the S2S connections are keep growing at each node, also, we can see there are many "socket time out exception" in log files. !image-2019-01-23-16-46-33-081.png! !image-2019-01-23-17-34-28-533.png! It may have some issue cause the NiFi does not close s2s connection. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] arpadboda commented on a change in pull request #470: MINIFICPP-706 - RawSiteToSite: remove code duplication
arpadboda commented on a change in pull request #470: MINIFICPP-706 -
RawSiteToSite: remove code duplication
URL: https://github.com/apache/nifi-minifi-cpp/pull/470#discussion_r249948870
##
File path: libminifi/src/sitetosite/RawSocketProtocol.cpp
##
@@ -395,97 +395,37 @@ bool
RawSiteToSiteClient::getPeerList(std::vector ) {
}
}
-int RawSiteToSiteClient::writeRequestType(RequestType type) {
- if (type >= MAX_REQUEST_TYPE)
-return -1;
-
- return peer_->writeUTF(SiteToSiteRequest::RequestTypeStr[type]);
-}
-
-int RawSiteToSiteClient::readRequestType(RequestType ) {
- std::string requestTypeStr;
-
- int ret = peer_->readUTF(requestTypeStr);
-
- if (ret <= 0)
-return ret;
+ int RawSiteToSiteClient::writeRequestType(RequestType type) {
+if (type >= MAX_REQUEST_TYPE)
+ return -1;
- for (int i = NEGOTIATE_FLOWFILE_CODEC; i <= SHUTDOWN; i++) {
-if (SiteToSiteRequest::RequestTypeStr[i] == requestTypeStr) {
- type = (RequestType) i;
- return ret;
-}
+return peer_->writeUTF(SiteToSiteRequest::RequestTypeStr[type]);
}
- return -1;
-}
-
-int RawSiteToSiteClient::readRespond(const std::shared_ptr
, RespondCode , std::string ) {
- uint8_t firstByte;
Review comment:
That's the case.
The implementation here was similar to "ReadResponse", which exists in base
class, where it's required.
The change I made was to call that instead of copy-pasting.
"ReadResponse" implementation however cannot be removed from base as http
client relies on that: it overrides, but calls base implementation in a case.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
