[GitHub] [nifi] feitgraph opened a new pull request #4334: NIFI-6163 Reporting task cannot be set to running when in INVALID state
feitgraph opened a new pull request #4334:
URL: https://github.com/apache/nifi/pull/4334
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
Fixes bug https://jira.apache.org/jira/browse/NIFI-6163
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #808: MINIFICPP-1234 - Enable all archive tests on Windows.
adamdebreceni commented on a change in pull request #808: URL: https://github.com/apache/nifi-minifi-cpp/pull/808#discussion_r439969476 ## File path: libminifi/test/archive-tests/ManipulateArchiveTests.cpp ## @@ -90,9 +89,7 @@ bool run_archive_test(OrderedTestArchive input_archive, OrderedTestArchive outpu plan->runNextProcessor(); // ManipulateArchive plan->runNextProcessor(); // PutFile 2 (manipulated) -std::stringstream ss2; -ss2 << dir2 << "/" << TEST_ARCHIVE_NAME; -std::string output_path = ss2.str(); +std::string output_path =utils::file::FileUtils::concat_path(dir2, TEST_ARCHIVE_NAME); Review comment: done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #808: MINIFICPP-1234 - Enable all archive tests on Windows.
adamdebreceni commented on a change in pull request #808:
URL: https://github.com/apache/nifi-minifi-cpp/pull/808#discussion_r439969614
##
File path: libminifi/include/utils/file/FileManager.h
##
@@ -61,46 +62,25 @@ class FileManager {
}
}
std::string unique_file(const std::string &location, bool keep = false) {
+const std::string& dir = !IsNullOrEmpty(location) ? location :
utils::file::FileUtils::get_temp_directory();
-
-if (!IsNullOrEmpty(location)) {
- std::string file_name = location + FILE_SEPARATOR +
non_repeating_string_generator_.generate();
- while (!verify_not_exist(file_name)) {
-file_name = location + FILE_SEPARATOR +
non_repeating_string_generator_.generate();
- }
- if (!keep)
-unique_files_.push_back(file_name);
- return file_name;
-} else {
- std::string tmpDir = "/tmp";
- #ifdef WIN32
- TCHAR lpTempPathBuffer[MAX_PATH];
- GetTempPath(MAX_PATH, lpTempPathBuffer);
- tmpDir = lpTempPathBuffer;
- #endif
- std::string file_name = tmpDir + FILE_SEPARATOR +
non_repeating_string_generator_.generate();
- while (!verify_not_exist(file_name)) {
-file_name = tmpDir + FILE_SEPARATOR +
non_repeating_string_generator_.generate();
- }
- if (!keep)
-unique_files_.push_back(file_name);
- return file_name;
+std::string file_name = utils::file::FileUtils::concat_path(dir,
non_repeating_string_generator_.generate());
+while (!verify_not_exist(file_name)) {
+ file_name = utils::file::FileUtils::concat_path(dir,
non_repeating_string_generator_.generate());
}
+if (!keep)
+ unique_files_.push_back(file_name);
+return file_name;
}
std::string unique_file(bool keep = false) {
#ifdef BOOST_VERSION
return boost::filesystem::unique_path().native();
#else
- std::string tmpDir = "/tmp";
- #ifdef WIN32
- TCHAR lpTempPathBuffer[MAX_PATH];
- GetTempPath(MAX_PATH, lpTempPathBuffer);
- tmpDir = lpTempPathBuffer;
- #endif
-std::string file_name = tmpDir + FILE_SEPARATOR +
non_repeating_string_generator_.generate();
+ std::string tmpDir = utils::file::FileUtils::get_temp_directory();
+std::string file_name = utils::file::FileUtils::concat_path(tmpDir,
non_repeating_string_generator_.generate());
while (!verify_not_exist(file_name)) {
- file_name = tmpDir + FILE_SEPARATOR +
non_repeating_string_generator_.generate();
+ file_name = utils::file::FileUtils::concat_path(tmpDir,
non_repeating_string_generator_.generate());
}
if (!keep)
unique_files_.push_back(file_name);
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #809: MINIFICPP-1254 - Introduce optional-lite
adamdebreceni commented on a change in pull request #809: URL: https://github.com/apache/nifi-minifi-cpp/pull/809#discussion_r439973992 ## File path: libminifi/src/utils/BackTrace.cpp ## @@ -30,22 +30,24 @@ #include #endif +#include Review comment: +1 for the alias with optional customisation (no pun intended) in our own headers This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #811: MINIFICPP-1257 don't leak kafka messages on failed send to broker
adamdebreceni commented on a change in pull request #811:
URL: https://github.com/apache/nifi-minifi-cpp/pull/811#discussion_r439985423
##
File path: extensions/librdkafka/PublishKafka.h
##
@@ -233,9 +233,13 @@ class PublishKafka : public core::Processor {
const gsl::owner hdrs_copy =
rd_kafka_headers_copy(hdrs.get());
const auto err = rd_kafka_producev(rk_, RD_KAFKA_V_RKT(rkt_),
RD_KAFKA_V_PARTITION(RD_KAFKA_PARTITION_UA),
RD_KAFKA_V_MSGFLAGS(RD_KAFKA_MSG_F_COPY), RD_KAFKA_V_VALUE(buffer.data(),
buflen),
- RD_KAFKA_V_HEADERS(hdrs_copy),
RD_KAFKA_V_KEY(key_.c_str(), key_.size()),
RD_KAFKA_V_OPAQUE(callback_ptr.release()), RD_KAFKA_V_END);
- if (err) {
-// the message only takes ownership of the headers in case of success
+ RD_KAFKA_V_HEADERS(hdrs_copy),
RD_KAFKA_V_KEY(key_.c_str(), key_.size()),
RD_KAFKA_V_OPAQUE(callback_ptr.get()), RD_KAFKA_V_END);
+ if (err == RD_KAFKA_RESP_ERR_NO_ERROR) {
Review comment:
👍
##
File path: extensions/librdkafka/PublishKafka.h
##
@@ -233,9 +233,13 @@ class PublishKafka : public core::Processor {
const gsl::owner hdrs_copy =
rd_kafka_headers_copy(hdrs.get());
const auto err = rd_kafka_producev(rk_, RD_KAFKA_V_RKT(rkt_),
RD_KAFKA_V_PARTITION(RD_KAFKA_PARTITION_UA),
RD_KAFKA_V_MSGFLAGS(RD_KAFKA_MSG_F_COPY), RD_KAFKA_V_VALUE(buffer.data(),
buflen),
- RD_KAFKA_V_HEADERS(hdrs_copy),
RD_KAFKA_V_KEY(key_.c_str(), key_.size()),
RD_KAFKA_V_OPAQUE(callback_ptr.release()), RD_KAFKA_V_END);
- if (err) {
-// the message only takes ownership of the headers in case of success
+ RD_KAFKA_V_HEADERS(hdrs_copy),
RD_KAFKA_V_KEY(key_.c_str(), key_.size()),
RD_KAFKA_V_OPAQUE(callback_ptr.get()), RD_KAFKA_V_END);
+ if (err == RD_KAFKA_RESP_ERR_NO_ERROR) {
+// in case of failure, messageDeliveryCallback is not called and
callback_ptr will delete the callback
+// in case of success, messageDeliveryCallback takes ownership of the
callback, so we no longer need to delete it
Review comment:
I'm not familiar with this processor, I assume messageDeliveryCallback
is async called, could we have a scenario where the messageDeliveryCallback is
not eventually called?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #811: MINIFICPP-1257 don't leak kafka messages on failed send to broker
adamdebreceni commented on a change in pull request #811:
URL: https://github.com/apache/nifi-minifi-cpp/pull/811#discussion_r439988172
##
File path: extensions/librdkafka/PublishKafka.h
##
@@ -233,9 +233,13 @@ class PublishKafka : public core::Processor {
const gsl::owner hdrs_copy =
rd_kafka_headers_copy(hdrs.get());
const auto err = rd_kafka_producev(rk_, RD_KAFKA_V_RKT(rkt_),
RD_KAFKA_V_PARTITION(RD_KAFKA_PARTITION_UA),
RD_KAFKA_V_MSGFLAGS(RD_KAFKA_MSG_F_COPY), RD_KAFKA_V_VALUE(buffer.data(),
buflen),
- RD_KAFKA_V_HEADERS(hdrs_copy),
RD_KAFKA_V_KEY(key_.c_str(), key_.size()),
RD_KAFKA_V_OPAQUE(callback_ptr.release()), RD_KAFKA_V_END);
- if (err) {
-// the message only takes ownership of the headers in case of success
+ RD_KAFKA_V_HEADERS(hdrs_copy),
RD_KAFKA_V_KEY(key_.c_str(), key_.size()),
RD_KAFKA_V_OPAQUE(callback_ptr.get()), RD_KAFKA_V_END);
+ if (err == RD_KAFKA_RESP_ERR_NO_ERROR) {
+// in case of failure, messageDeliveryCallback is not called and
callback_ptr will delete the callback
+// in case of success, messageDeliveryCallback takes ownership of the
callback, so we no longer need to delete it
Review comment:
I'm not familiar with this processor, could we have a scenario where the
messageDeliveryCallback is not eventually called?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #784: MINIFICPP-1206 - Rework and test ExecutePythonProcessor, add in-place script support
adamdebreceni commented on a change in pull request #784:
URL: https://github.com/apache/nifi-minifi-cpp/pull/784#discussion_r440004015
##
File path: libminifi/include/utils/TestUtils.h
##
@@ -0,0 +1,67 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+
+#include "../../test/TestBase.h"
+#include "utils/file/FileUtils.h"
+#include "utils/Environment.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+std::string createTempDir(TestController* testController) {
+ char dirtemplate[] = "/tmp/gt.XX";
+ std::string temp_dir = testController->createTempDirectory(dirtemplate);
+ REQUIRE(!temp_dir.empty());
+ REQUIRE(file::FileUtils::is_directory(temp_dir.c_str()));
+ return temp_dir;
+}
+
+std::string putFileToDir(const std::string& dir_path, const std::string&
file_name, const std::string& content) {
+ std::string file_path(dir_path + file::FileUtils::get_separator() +
file_name);
Review comment:
`file::FileUtils::concat_path` might be used here
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm opened a new pull request #813: MINIFICPP-1252 fix shared library build with gsl
szaszm opened a new pull request #813: URL: https://github.com/apache/nifi-minifi-cpp/pull/813 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Reopened] (MINIFICPP-1252) Introduce gsl-lite
[
https://issues.apache.org/jira/browse/MINIFICPP-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz reopened MINIFICPP-1252:
-
shared library build fails
> Introduce gsl-lite
> --
>
> Key: MINIFICPP-1252
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1252
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Minor
> Fix For: 0.8.0
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Add gsl-lite to minifi c++ so that we can follow the best practices of the
> c++ core guidelines more closely.
> Highlights:
> - {{gsl::owner}}: annotate owner pointers
> - {{gsl::not_null}}: pointer wrapper class with {{!= nullptr}} invariant,
> works on smart pointers
> - {{gsl_Expects}}, {{gsl_Ensures}}: contract checking
> - {{gsl::finally}}: execute callback at the end of the scope
> - {{gsl::narrow}}: checked narrowing conversion, fails on value change
> - {{gsl::narrow_cast}}: searchable annotation for narrowing conversions
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7531) Auto-loading NAR directory does not need RW access
Pierre Villard created NIFI-7531: Summary: Auto-loading NAR directory does not need RW access Key: NIFI-7531 URL: https://issues.apache.org/jira/browse/NIFI-7531 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Pierre Villard Assignee: Pierre Villard If the auto-load NARs directory has not read+write access, NiFi won't start: java.io.IOException: /tmp/nifi-extensions directory does not have read/write privilege at org.apache.nifi.util.FileUtils.ensureDirectoryExistAndCanReadAndWrite(FileUtils.java:51) at org.apache.nifi.nar.NarAutoLoader.start(NarAutoLoader.java:56) at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1051) at org.apache.nifi.NiFi.(NiFi.java:158) at org.apache.nifi.NiFi.(NiFi.java:72) at org.apache.nifi.NiFi.main(NiFi.java:301) Read only access is enough to this directory. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7531) Auto-loading NAR directory does not need RW access
[
https://issues.apache.org/jira/browse/NIFI-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-7531:
-
Description:
If the auto-load NARs directory has not read+write access, NiFi won't start:
{code:java}
java.io.IOException: /tmp/nifi-extensions directory does not have read/write
privilege
at
org.apache.nifi.util.FileUtils.ensureDirectoryExistAndCanReadAndWrite(FileUtils.java:51)
at org.apache.nifi.nar.NarAutoLoader.start(NarAutoLoader.java:56)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1051)
at org.apache.nifi.NiFi.(NiFi.java:158)
at org.apache.nifi.NiFi.(NiFi.java:72)
at org.apache.nifi.NiFi.main(NiFi.java:301){code}
Read only access is enough to this directory.
was:
If the auto-load NARs directory has not read+write access, NiFi won't start:
java.io.IOException: /tmp/nifi-extensions directory does not have read/write
privilege
at
org.apache.nifi.util.FileUtils.ensureDirectoryExistAndCanReadAndWrite(FileUtils.java:51)
at org.apache.nifi.nar.NarAutoLoader.start(NarAutoLoader.java:56)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1051)
at org.apache.nifi.NiFi.(NiFi.java:158)
at org.apache.nifi.NiFi.(NiFi.java:72)
at org.apache.nifi.NiFi.main(NiFi.java:301)
Read only access is enough to this directory.
> Auto-loading NAR directory does not need RW access
> --
>
> Key: NIFI-7531
> URL: https://issues.apache.org/jira/browse/NIFI-7531
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Major
>
> If the auto-load NARs directory has not read+write access, NiFi won't start:
> {code:java}
> java.io.IOException: /tmp/nifi-extensions directory does not have read/write
> privilege
> at
> org.apache.nifi.util.FileUtils.ensureDirectoryExistAndCanReadAndWrite(FileUtils.java:51)
> at org.apache.nifi.nar.NarAutoLoader.start(NarAutoLoader.java:56)
> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1051)
> at org.apache.nifi.NiFi.(NiFi.java:158)
> at org.apache.nifi.NiFi.(NiFi.java:72)
> at org.apache.nifi.NiFi.main(NiFi.java:301){code}
> Read only access is enough to this directory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] pvillard31 opened a new pull request #4335: NIFI-7531 - changed RW to RO access requirement on autoload NARs dire…
pvillard31 opened a new pull request #4335:
URL: https://github.com/apache/nifi/pull/4335
…ctory
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-7531) Auto-loading NAR directory does not need RW access
[
https://issues.apache.org/jira/browse/NIFI-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-7531:
-
Status: Patch Available (was: Open)
> Auto-loading NAR directory does not need RW access
> --
>
> Key: NIFI-7531
> URL: https://issues.apache.org/jira/browse/NIFI-7531
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> If the auto-load NARs directory has not read+write access, NiFi won't start:
> {code:java}
> java.io.IOException: /tmp/nifi-extensions directory does not have read/write
> privilege
> at
> org.apache.nifi.util.FileUtils.ensureDirectoryExistAndCanReadAndWrite(FileUtils.java:51)
> at org.apache.nifi.nar.NarAutoLoader.start(NarAutoLoader.java:56)
> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1051)
> at org.apache.nifi.NiFi.(NiFi.java:158)
> at org.apache.nifi.NiFi.(NiFi.java:72)
> at org.apache.nifi.NiFi.main(NiFi.java:301){code}
> Read only access is enough to this directory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #809: MINIFICPP-1254 - Introduce optional-lite
szaszm commented on a change in pull request #809:
URL: https://github.com/apache/nifi-minifi-cpp/pull/809#discussion_r440079252
##
File path: libminifi/include/utils/OptionalUtils.h
##
@@ -0,0 +1,44 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+#define LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+
+#include
+
+#include
+#include "utils/GeneralUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+nonstd::optional::type> optional_from_nullable(T&&
obj) {
Review comment:
I recommend introducing an alias to optional in our namespace and
referring to that in our code. This would simplify later extension and
migration.
##
File path: libminifi/include/utils/GeneralUtils.h
##
@@ -61,6 +61,17 @@ T exchange(T& obj, U&& new_value) {
template
using void_t = void;
+template
+using remove_reference_t = typename std::remove_reference::type;
+
+template
+using remove_cv_t = typename std::remove_cv::type;
+
+template
+struct remove_cvref {
+typedef remove_cv_t> type;
+};
Review comment:
Please move `remove_cvref` either outside of `#ifdef` checks, or another
one that checks for C++20 and falls back to `std::remove_cvref` if >= C++20.
##
File path: libminifi/include/utils/GeneralUtils.h
##
@@ -61,6 +61,17 @@ T exchange(T& obj, U&& new_value) {
template
using void_t = void;
+template
+using remove_reference_t = typename std::remove_reference::type;
+
+template
+using remove_cv_t = typename std::remove_cv::type;
Review comment:
Please add a fallback to std to the `#else` branch of this `#ifdef`
block.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #809: MINIFICPP-1254 - Introduce optional-lite
szaszm commented on a change in pull request #809:
URL: https://github.com/apache/nifi-minifi-cpp/pull/809#discussion_r440079252
##
File path: libminifi/include/utils/OptionalUtils.h
##
@@ -0,0 +1,44 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+#define LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+
+#include
+
+#include
+#include "utils/GeneralUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+nonstd::optional::type> optional_from_nullable(T&&
obj) {
Review comment:
I recommend introducing an alias to optional in our namespace and
referring to that in our code. This would simplify later extension and
migration.
```
namespace org::apache::nifi::minifi::utils {
using nonstd::optional;
} // namespace org::apache::nifi::minifi::utils
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #809: MINIFICPP-1254 - Introduce optional-lite
szaszm commented on a change in pull request #809: URL: https://github.com/apache/nifi-minifi-cpp/pull/809#discussion_r440090536 ## File path: thirdparty/optional-lite-3.2.0/LICENSE ## @@ -0,0 +1,23 @@ +Boost Software License - Version 1.0 - August 17th, 2003 Review comment: The filename was `LICENSE.txt` in the original. I think we shouldn't rename things without a good reason. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (MINIFICPP-1260) Make void_t in GeneralUtils forward-compabilble with C++17
Marton Szasz created MINIFICPP-1260: --- Summary: Make void_t in GeneralUtils forward-compabilble with C++17 Key: MINIFICPP-1260 URL: https://issues.apache.org/jira/browse/MINIFICPP-1260 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Marton Szasz Assignee: Marton Szasz -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] szaszm opened a new pull request #814: MINIFICPP-1260 make void_t fwd-compatible with C++17
szaszm opened a new pull request #814: URL: https://github.com/apache/nifi-minifi-cpp/pull/814 ... and make `make_unique` follow the standard version more closely Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #814: MINIFICPP-1260 make void_t fwd-compatible with C++17
szaszm commented on a change in pull request #814:
URL: https://github.com/apache/nifi-minifi-cpp/pull/814#discussion_r440096814
##
File path: libminifi/include/utils/GeneralUtils.h
##
@@ -35,7 +35,7 @@ namespace utils {
#if __cplusplus < 201402L
template
std::unique_ptr make_unique(Args&&... args) {
- return std::unique_ptr{ new T{ std::forward(args)... } };
+ return std::unique_ptr{ new T(std::forward(args)...) };
Review comment:
most significant differences between direct-initialization and
direct-list-initialization:
- direct-list-initialization doesn't allow narrowing conversions
- direct-list-initialization prefers the initializer-list constructor, and
only attempts regular ctor calls when the initializer-list one does not match,
as opposed to doing regular overload resolution with the initializer-list
constructor in the overload set.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #791: MINIFICPP-1177 Improvements to the TailFile processor
fgerlits commented on a change in pull request #791:
URL: https://github.com/apache/nifi-minifi-cpp/pull/791#discussion_r440097068
##
File path: extensions/standard-processors/processors/TailFile.cpp
##
@@ -204,52 +418,78 @@ void TailFile::parseStateFileLine(char *buf) {
}
std::string value = equal;
- key = trimRight(key);
- value = trimRight(value);
+ key = utils::StringUtils::trimRight(key);
+ value = utils::StringUtils::trimRight(value);
if (key == "FILENAME") {
std::string fileLocation, fileName;
if (utils::file::PathUtils::getFileNameAndPath(value, fileLocation,
fileName)) {
logger_->log_debug("State migration received path %s, file %s",
fileLocation, fileName);
- tail_states_.insert(std::make_pair(fileName, TailState { fileLocation,
fileName, 0, 0 }));
+ state.emplace(fileName, TailState{fileLocation, fileName});
} else {
- tail_states_.insert(std::make_pair(value, TailState { fileLocation,
value, 0, 0 }));
+ state.emplace(value, TailState{fileLocation, value});
}
}
if (key == "POSITION") {
// for backwards compatibility
-if (tail_states_.size() != 1) {
+if (tail_states_.size() != std::size_t{1}) {
throw minifi::Exception(ExceptionType::PROCESSOR_EXCEPTION,
"Incompatible state file types");
}
const auto position = std::stoull(value);
logger_->log_debug("Received position %d", position);
-tail_states_.begin()->second.currentTailFilePosition_ = position;
+state.begin()->second.position_ = position;
}
if (key.find(CURRENT_STR) == 0) {
const auto file = key.substr(strlen(CURRENT_STR));
std::string fileLocation, fileName;
if (utils::file::PathUtils::getFileNameAndPath(value, fileLocation,
fileName)) {
- tail_states_[file].path_ = fileLocation;
- tail_states_[file].current_file_name_ = fileName;
+ state[file].path_ = fileLocation;
+ state[file].file_name_ = fileName;
} else {
throw minifi::Exception(ExceptionType::PROCESSOR_EXCEPTION, "State file
contains an invalid file name");
}
}
if (key.find(POSITION_STR) == 0) {
const auto file = key.substr(strlen(POSITION_STR));
-tail_states_[file].currentTailFilePosition_ = std::stoull(value);
+state[file].position_ = std::stoull(value);
}
}
+bool TailFile::recoverState(const std::shared_ptr&
context) {
+ std::map new_tail_states;
+ bool state_load_success = getStateFromStateManager(new_tail_states) ||
+getStateFromLegacyStateFile(context,
new_tail_states);
+ if (!state_load_success) {
+return false;
+ }
+ logger_->log_debug("load state succeeded");
-bool TailFile::recoverState(const std::shared_ptr&
context) {
- bool state_load_success = false;
+ if (tail_mode_ == Mode::SINGLE) {
+if (tail_states_.size() == 1) {
+ auto state_it = tail_states_.begin();
+ const auto it = new_tail_states.find(state_it->first);
+ if (it != new_tail_states.end()) {
+state_it->second = it->second;
+ }
+} else {
+ throw minifi::Exception(ExceptionType::PROCESSOR_EXCEPTION, "This should
never happen: "
Review comment:
I don't think it can happen, as a configuration change will restart the
processor. A newly created processor in Single file mode starts with a
one-element state, and we only update that one item here, we don't add to it.
This `throw` is here only to guard against later code changes which change this
behavior.
As to invalidating the state, I'll need to look at how NiFi does that. I
think there is value in keeping the state of existing files after a
configuration change: for example, if the selection is broadened in Multi file
mode (say from `*.log` to `*.*`), we probably don't want to restart the files
we have already tailed from the beginning. But we should do whatever NiFi
does. Can that be a separate pull request?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] feitgraph commented on pull request #4334: NIFI-6163 Reporting task cannot be set to running when in INVALID state
feitgraph commented on pull request #4334: URL: https://github.com/apache/nifi/pull/4334#issuecomment-644060552 Was able to reproduce it: 1. Add "Scripting report task". After adding it - it is not valid. 2. In Postman try to send requests as in https://issues.apache.org/jira/browse/NIFI-6163 . You will receive OK message. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] feitgraph commented on pull request #4334: NIFI-6163 Reporting task cannot be set to running when in INVALID state
feitgraph commented on pull request #4334: URL: https://github.com/apache/nifi/pull/4334#issuecomment-644061029 Regarding failed workflows - it seems errors are not on my part. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #809: MINIFICPP-1254 - Introduce optional-lite
hunyadi-dev commented on a change in pull request #809:
URL: https://github.com/apache/nifi-minifi-cpp/pull/809#discussion_r440102107
##
File path: libminifi/src/utils/BackTrace.cpp
##
@@ -89,19 +90,9 @@ void pull_trace(uint8_t frames_to_skip /* = 1 */) {
}
}
-/* Determine the symbol name */
-std::string demangled_symbol_name;
+std::string symbol_name;
if (dl_info.dli_sname != nullptr) {
- symbol_name = dl_info.dli_sname;
-
- /* Try to demangle the symbol name */
- demangled_symbol_name = demangle_symbol(symbol_name);
- if (!demangled_symbol_name.empty()) {
-symbol_name = demangled_symbol_name.c_str();
- }
-} else {
- /* If we could not determine the symbol name, we will use the filename
instead */
- symbol_name = file_name;
+ symbol_name =
demangle_symbol(dl_info.dli_sname).value_or(std::string(file_name));
}
Review comment:
Updated as requested.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #809: MINIFICPP-1254 - Introduce optional-lite
hunyadi-dev commented on a change in pull request #809:
URL: https://github.com/apache/nifi-minifi-cpp/pull/809#discussion_r440112121
##
File path: libminifi/include/utils/GeneralUtils.h
##
@@ -61,6 +61,17 @@ T exchange(T& obj, U&& new_value) {
template
using void_t = void;
+template
+using remove_reference_t = typename std::remove_reference::type;
+
+template
+using remove_cv_t = typename std::remove_cv::type;
+
+template
+struct remove_cvref {
+typedef remove_cv_t> type;
+};
Review comment:
As requested.
##
File path: libminifi/include/utils/GeneralUtils.h
##
@@ -61,6 +61,17 @@ T exchange(T& obj, U&& new_value) {
template
using void_t = void;
+template
+using remove_reference_t = typename std::remove_reference::type;
+
+template
+using remove_cv_t = typename std::remove_cv::type;
Review comment:
Updated as requested.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-7532) Preserve file properties(timestamp, permissions etc.) after getFile & putFile
Vivek created NIFI-7532: --- Summary: Preserve file properties(timestamp, permissions etc.) after getFile & putFile Key: NIFI-7532 URL: https://issues.apache.org/jira/browse/NIFI-7532 Project: Apache NiFi Issue Type: Wish Components: Configuration Reporter: Vivek -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7533) Urgent requirement for cdc for Postgres
Vivek created NIFI-7533: --- Summary: Urgent requirement for cdc for Postgres Key: NIFI-7533 URL: https://issues.apache.org/jira/browse/NIFI-7533 Project: Apache NiFi Issue Type: Wish Reporter: Vivek Hi Team, I can see the cdc for mysql in Nifi & was successfully able to use it but now I need to use cdc with Postgres. Can someone suggest if this is possible in Nifi. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #791: MINIFICPP-1177 Improvements to the TailFile processor
arpadboda commented on a change in pull request #791:
URL: https://github.com/apache/nifi-minifi-cpp/pull/791#discussion_r440135550
##
File path: extensions/standard-processors/processors/TailFile.cpp
##
@@ -204,52 +418,78 @@ void TailFile::parseStateFileLine(char *buf) {
}
std::string value = equal;
- key = trimRight(key);
- value = trimRight(value);
+ key = utils::StringUtils::trimRight(key);
+ value = utils::StringUtils::trimRight(value);
if (key == "FILENAME") {
std::string fileLocation, fileName;
if (utils::file::PathUtils::getFileNameAndPath(value, fileLocation,
fileName)) {
logger_->log_debug("State migration received path %s, file %s",
fileLocation, fileName);
- tail_states_.insert(std::make_pair(fileName, TailState { fileLocation,
fileName, 0, 0 }));
+ state.emplace(fileName, TailState{fileLocation, fileName});
} else {
- tail_states_.insert(std::make_pair(value, TailState { fileLocation,
value, 0, 0 }));
+ state.emplace(value, TailState{fileLocation, value});
}
}
if (key == "POSITION") {
// for backwards compatibility
-if (tail_states_.size() != 1) {
+if (tail_states_.size() != std::size_t{1}) {
throw minifi::Exception(ExceptionType::PROCESSOR_EXCEPTION,
"Incompatible state file types");
}
const auto position = std::stoull(value);
logger_->log_debug("Received position %d", position);
-tail_states_.begin()->second.currentTailFilePosition_ = position;
+state.begin()->second.position_ = position;
}
if (key.find(CURRENT_STR) == 0) {
const auto file = key.substr(strlen(CURRENT_STR));
std::string fileLocation, fileName;
if (utils::file::PathUtils::getFileNameAndPath(value, fileLocation,
fileName)) {
- tail_states_[file].path_ = fileLocation;
- tail_states_[file].current_file_name_ = fileName;
+ state[file].path_ = fileLocation;
+ state[file].file_name_ = fileName;
} else {
throw minifi::Exception(ExceptionType::PROCESSOR_EXCEPTION, "State file
contains an invalid file name");
}
}
if (key.find(POSITION_STR) == 0) {
const auto file = key.substr(strlen(POSITION_STR));
-tail_states_[file].currentTailFilePosition_ = std::stoull(value);
+state[file].position_ = std::stoull(value);
}
}
+bool TailFile::recoverState(const std::shared_ptr&
context) {
+ std::map new_tail_states;
+ bool state_load_success = getStateFromStateManager(new_tail_states) ||
+getStateFromLegacyStateFile(context,
new_tail_states);
+ if (!state_load_success) {
+return false;
+ }
+ logger_->log_debug("load state succeeded");
-bool TailFile::recoverState(const std::shared_ptr&
context) {
- bool state_load_success = false;
+ if (tail_mode_ == Mode::SINGLE) {
+if (tail_states_.size() == 1) {
+ auto state_it = tail_states_.begin();
+ const auto it = new_tail_states.find(state_it->first);
+ if (it != new_tail_states.end()) {
+state_it->second = it->second;
+ }
+} else {
+ throw minifi::Exception(ExceptionType::PROCESSOR_EXCEPTION, "This should
never happen: "
Review comment:
Yep, it's fine to do as a follow-up.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #809: MINIFICPP-1254 - Introduce optional-lite
hunyadi-dev commented on a change in pull request #809:
URL: https://github.com/apache/nifi-minifi-cpp/pull/809#discussion_r440152423
##
File path: libminifi/include/utils/OptionalUtils.h
##
@@ -0,0 +1,44 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+#define LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+
+#include
+
+#include
+#include "utils/GeneralUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+nonstd::optional::type> optional_from_nullable(T&&
obj) {
Review comment:
Added:
```c++
template
using optional = nonstd::optional;
const nonstd::nullopt_t nullopt = nonstd::nullopt;
```
However, for this to work there is no good way to refer `nonstd::nullopt`
from outside of the helper namespace. Any improvement idea on this?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #809: MINIFICPP-1254 - Introduce optional-lite
adamdebreceni commented on a change in pull request #809:
URL: https://github.com/apache/nifi-minifi-cpp/pull/809#discussion_r440164112
##
File path: libminifi/include/utils/OptionalUtils.h
##
@@ -0,0 +1,44 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+#define LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+
+#include
+
+#include
+#include "utils/GeneralUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+nonstd::optional::type> optional_from_nullable(T&&
obj) {
Review comment:
why not `using nostd::nullopt;` ?
##
File path: libminifi/include/utils/OptionalUtils.h
##
@@ -0,0 +1,44 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+#define LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+
+#include
+
+#include
+#include "utils/GeneralUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+nonstd::optional::type> optional_from_nullable(T&&
obj) {
Review comment:
why not `using nonstd::nullopt;` ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (MINIFICPP-1248) Unit test CWEL using WIN API
[ https://issues.apache.org/jira/browse/MINIFICPP-1248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-1248: - Assignee: Ferenc Gerlits > Unit test CWEL using WIN API > > > Key: MINIFICPP-1248 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1248 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.7.0 >Reporter: Arpad Boda >Assignee: Ferenc Gerlits >Priority: Major > Fix For: 0.8.0 > > > Windows API provides functionality to report events in C(++) code: > https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-reporteventa > Using this we could create unit tests for CWEL. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1229) Clean up CompressContentTests
[ https://issues.apache.org/jira/browse/MINIFICPP-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Debreceni resolved MINIFICPP-1229. --- Resolution: Fixed > Clean up CompressContentTests > - > > Key: MINIFICPP-1229 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1229 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Minor > Time Spent: 5h 40m > Remaining Estimate: 0h > > Most tests in CompressContentTests were wrapped in a try-catch masking all > errors. We should fix this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1230) Enable MergeFileTests on Windows.
[ https://issues.apache.org/jira/browse/MINIFICPP-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Debreceni resolved MINIFICPP-1230. --- Resolution: Fixed > Enable MergeFileTests on Windows. > - > > Key: MINIFICPP-1230 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1230 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Minor > Time Spent: 1h 10m > Remaining Estimate: 0h > > The only archive test that runs on Windows is the CompressContentTests, turn > MergeFileTests on and fix all arising issues. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1225) Fix flaky HTTP tests.
[ https://issues.apache.org/jira/browse/MINIFICPP-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Debreceni resolved MINIFICPP-1225. --- Resolution: Fixed > Fix flaky HTTP tests. > - > > Key: MINIFICPP-1225 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1225 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Major > Time Spent: 2h 50m > Remaining Estimate: 0h > > Some HTTP tests are sporadically failing, including the HTTPSiteToSiteTests, > their behavior should be consistent. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #809: MINIFICPP-1254 - Introduce optional-lite
hunyadi-dev commented on a change in pull request #809:
URL: https://github.com/apache/nifi-minifi-cpp/pull/809#discussion_r440188628
##
File path: libminifi/include/utils/OptionalUtils.h
##
@@ -0,0 +1,44 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+#define LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+
+#include
+
+#include
+#include "utils/GeneralUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+nonstd::optional::type> optional_from_nullable(T&&
obj) {
Review comment:
Because it is a value, not a type. There is the added value declaration,
but it can only be used by either declaring it in the caller scope or by
providing a fully qualified name.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #809: MINIFICPP-1254 - Introduce optional-lite
adamdebreceni commented on a change in pull request #809:
URL: https://github.com/apache/nifi-minifi-cpp/pull/809#discussion_r440192333
##
File path: libminifi/include/utils/OptionalUtils.h
##
@@ -0,0 +1,44 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+#define LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+
+#include
+
+#include
+#include "utils/GeneralUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+nonstd::optional::type> optional_from_nullable(T&&
obj) {
Review comment:
I am not aware of such restrictions on the using-declaration:
https://godbolt.org/z/w7jbKM
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #809: MINIFICPP-1254 - Introduce optional-lite
szaszm commented on a change in pull request #809:
URL: https://github.com/apache/nifi-minifi-cpp/pull/809#discussion_r440193744
##
File path: libminifi/include/utils/OptionalUtils.h
##
@@ -0,0 +1,44 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+#define LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+
+#include
+
+#include
+#include "utils/GeneralUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+nonstd::optional::type> optional_from_nullable(T&&
obj) {
Review comment:
You can use using-declarations on variables.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #809: MINIFICPP-1254 - Introduce optional-lite
szaszm commented on a change in pull request #809:
URL: https://github.com/apache/nifi-minifi-cpp/pull/809#discussion_r440194759
##
File path: libminifi/include/utils/OptionalUtils.h
##
@@ -21,17 +21,20 @@
#include
#include
-#include "utils/GeneralUtils.h"
+#include "utils/gsl.h"
namespace org {
namespace apache {
namespace nifi {
namespace minifi {
namespace utils {
+template
+using optional = nonstd::optional;
+const nonstd::nullopt_t nullopt = nonstd::nullopt;
Review comment:
As pointed out in the other comment, these could become
using-declarations.
```suggestion
using nonstd::optional;
using nonstd::nullopt;
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #809: MINIFICPP-1254 - Introduce optional-lite
hunyadi-dev commented on a change in pull request #809:
URL: https://github.com/apache/nifi-minifi-cpp/pull/809#discussion_r440196041
##
File path: libminifi/include/utils/OptionalUtils.h
##
@@ -0,0 +1,44 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+#define LIBMINIFI_INCLUDE_UTILS_OPTIONALUTILS_H_
+
+#include
+
+#include
+#include "utils/GeneralUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+nonstd::optional::type> optional_from_nullable(T&&
obj) {
Review comment:
Updated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #809: MINIFICPP-1254 - Introduce optional-lite
hunyadi-dev commented on a change in pull request #809:
URL: https://github.com/apache/nifi-minifi-cpp/pull/809#discussion_r440197225
##
File path: libminifi/include/utils/OptionalUtils.h
##
@@ -21,17 +21,20 @@
#include
#include
-#include "utils/GeneralUtils.h"
+#include "utils/gsl.h"
namespace org {
namespace apache {
namespace nifi {
namespace minifi {
namespace utils {
+template
+using optional = nonstd::optional;
+const nonstd::nullopt_t nullopt = nonstd::nullopt;
Review comment:
Already modified.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #811: MINIFICPP-1257 don't leak kafka messages on failed send to broker
arpadboda commented on a change in pull request #811:
URL: https://github.com/apache/nifi-minifi-cpp/pull/811#discussion_r440227046
##
File path: extensions/librdkafka/PublishKafka.h
##
@@ -233,9 +233,13 @@ class PublishKafka : public core::Processor {
const gsl::owner hdrs_copy =
rd_kafka_headers_copy(hdrs.get());
const auto err = rd_kafka_producev(rk_, RD_KAFKA_V_RKT(rkt_),
RD_KAFKA_V_PARTITION(RD_KAFKA_PARTITION_UA),
RD_KAFKA_V_MSGFLAGS(RD_KAFKA_MSG_F_COPY), RD_KAFKA_V_VALUE(buffer.data(),
buflen),
- RD_KAFKA_V_HEADERS(hdrs_copy),
RD_KAFKA_V_KEY(key_.c_str(), key_.size()),
RD_KAFKA_V_OPAQUE(callback_ptr.release()), RD_KAFKA_V_END);
- if (err) {
-// the message only takes ownership of the headers in case of success
+ RD_KAFKA_V_HEADERS(hdrs_copy),
RD_KAFKA_V_KEY(key_.c_str(), key_.size()),
RD_KAFKA_V_OPAQUE(callback_ptr.get()), RD_KAFKA_V_END);
+ if (err == RD_KAFKA_RESP_ERR_NO_ERROR) {
+// in case of failure, messageDeliveryCallback is not called and
callback_ptr will delete the callback
+// in case of success, messageDeliveryCallback takes ownership of the
callback, so we no longer need to delete it
Review comment:
As far as I understand the API documentation, in case this function
returns no error, the callback should be called.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] r65535 opened a new pull request #4336: NIFI-7515 Added 7Zip support to UnpackContent
r65535 opened a new pull request #4336: URL: https://github.com/apache/nifi/pull/4336 Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Added 7ZIP support to UnpackContent. Will raise another PR/jira ticket for 7Zip support in MergeContent. _Enables X functionality; fixes bug NIFI-._ In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically `master`)? - [x] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on JDK 8? - [ ] Have you verified that the full build is successful on JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-7515) .7z decompress functionality
[ https://issues.apache.org/jira/browse/NIFI-7515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] r65535 reassigned NIFI-7515: Assignee: r65535 > .7z decompress functionality > > > Key: NIFI-7515 > URL: https://issues.apache.org/jira/browse/NIFI-7515 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Core UI >Affects Versions: 1.11.0 >Reporter: Mohammed Saifullah Sarfraz Sanaullah >Assignee: r65535 >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Please add .7z support to decompress such files as it will help us to > extract content for downstream use. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #789: MINIFICPP-1203 - Enable header linting in include directories and resolve linter recommendations
arpadboda closed pull request #789: URL: https://github.com/apache/nifi-minifi-cpp/pull/789 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #811: MINIFICPP-1257 don't leak kafka messages on failed send to broker
arpadboda closed pull request #811: URL: https://github.com/apache/nifi-minifi-cpp/pull/811 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #813: MINIFICPP-1252 fix shared library build with gsl
arpadboda closed pull request #813: URL: https://github.com/apache/nifi-minifi-cpp/pull/813 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] thanatas commented on pull request #4065: NIFI-4239 - Adding CaptureChangePostgreSQL processor to capture data changes (INSERT/UPDATE/DELETE) in PostgreSQL tables via Logical Replicati
thanatas commented on pull request #4065: URL: https://github.com/apache/nifi/pull/4065#issuecomment-644202159 This is great feature, is there any build version of it, unfortunately it has not merged yet? Building is not feasible. Thanks.. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-7534) Consume MQTT Processor. Support EL for Topic Filter property
Andriy Binetsky created NIFI-7534: - Summary: Consume MQTT Processor. Support EL for Topic Filter property Key: NIFI-7534 URL: https://issues.apache.org/jira/browse/NIFI-7534 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 1.11.4 Reporter: Andriy Binetsky It would be great to support EL for Topic Filter property of the Consume MQTT Processor. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1252) Introduce gsl-lite
[
https://issues.apache.org/jira/browse/MINIFICPP-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz resolved MINIFICPP-1252.
-
Resolution: Fixed
> Introduce gsl-lite
> --
>
> Key: MINIFICPP-1252
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1252
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Minor
> Fix For: 0.8.0
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Add gsl-lite to minifi c++ so that we can follow the best practices of the
> c++ core guidelines more closely.
> Highlights:
> - {{gsl::owner}}: annotate owner pointers
> - {{gsl::not_null}}: pointer wrapper class with {{!= nullptr}} invariant,
> works on smart pointers
> - {{gsl_Expects}}, {{gsl_Ensures}}: contract checking
> - {{gsl::finally}}: execute callback at the end of the scope
> - {{gsl::narrow}}: checked narrowing conversion, fails on value change
> - {{gsl::narrow_cast}}: searchable annotation for narrowing conversions
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (MINIFICPP-1252) Introduce gsl-lite
[
https://issues.apache.org/jira/browse/MINIFICPP-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz reopened MINIFICPP-1252:
-
> Introduce gsl-lite
> --
>
> Key: MINIFICPP-1252
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1252
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Minor
> Fix For: 0.8.0
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Add gsl-lite to minifi c++ so that we can follow the best practices of the
> c++ core guidelines more closely.
> Highlights:
> - {{gsl::owner}}: annotate owner pointers
> - {{gsl::not_null}}: pointer wrapper class with {{!= nullptr}} invariant,
> works on smart pointers
> - {{gsl_Expects}}, {{gsl_Ensures}}: contract checking
> - {{gsl::finally}}: execute callback at the end of the scope
> - {{gsl::narrow}}: checked narrowing conversion, fails on value change
> - {{gsl::narrow_cast}}: searchable annotation for narrowing conversions
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1252) Introduce gsl-lite
[
https://issues.apache.org/jira/browse/MINIFICPP-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz resolved MINIFICPP-1252.
-
Resolution: Done
> Introduce gsl-lite
> --
>
> Key: MINIFICPP-1252
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1252
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Minor
> Fix For: 0.8.0
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Add gsl-lite to minifi c++ so that we can follow the best practices of the
> c++ core guidelines more closely.
> Highlights:
> - {{gsl::owner}}: annotate owner pointers
> - {{gsl::not_null}}: pointer wrapper class with {{!= nullptr}} invariant,
> works on smart pointers
> - {{gsl_Expects}}, {{gsl_Ensures}}: contract checking
> - {{gsl::finally}}: execute callback at the end of the scope
> - {{gsl::narrow}}: checked narrowing conversion, fails on value change
> - {{gsl::narrow_cast}}: searchable annotation for narrowing conversions
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] fgerlits opened a new pull request #815: MINIFICPP-1153 Make the CTailFile tests thread-safe
fgerlits opened a new pull request #815: URL: https://github.com/apache/nifi-minifi-cpp/pull/815 ### Description of the changes Before each test in CTailFile tests, change the current working directory to a temporary directory not used by any other test. This is a mix of C and C++, but that was already true of these tests. --- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-7535) Adding nar's to extensions directory unreadable by user running the nifi jvm fails to start nifi
Juan C. Sequeiros created NIFI-7535:
---
Summary: Adding nar's to extensions directory unreadable by user
running the nifi jvm fails to start nifi
Key: NIFI-7535
URL: https://issues.apache.org/jira/browse/NIFI-7535
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Environment: NiFi 1.9
Reporter: Juan C. Sequeiros
If nar's are added to the extensions directory while NiFi is not running and
those nar's do not have read permissions by the user set to run nifi jvm, nifi
fails to start up and throws NPE
{code:java}
2020-06-15 13:34:16,988 WARN [main] org.apache.nifi.web.server.JettyServer
Failed to start web server... shutting down. java.lang.NullPointerException:
null at
org.apache.nifi.documentation.DocGenerator.generate(DocGenerator.java:62) at
org.apache.nifi.web.server.JettyServer.start(JettyServer.java:932) at
org.apache.nifi.NiFi.(NiFi.java:158) at
org.apache.nifi.NiFi.(NiFi.java:72) at
org.apache.nifi.NiFi.main(NiFi.java:297) 2020-06-15 13:34:16,988 INFO
[Thread-1] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...
2020-06-15 13:34:16,989 INFO [Thread-1] org.apache.nifi.NiFi Jetty web server
shutdown completed (nicely or otherwise).
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (NIFI-7337) Can't display any provenance data
[ https://issues.apache.org/jira/browse/NIFI-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFI-7337. -- Fix Version/s: 1.12.0 Resolution: Duplicate > Can't display any provenance data > - > > Key: NIFI-7337 > URL: https://issues.apache.org/jira/browse/NIFI-7337 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.9.2, 1.11.3 > Environment: win10,jdk 1.8.0_131 >Reporter: hoar >Priority: Major > Fix For: 1.12.0 > > > After the processors generated some provenance data, stop all processors. > Restart the processors after stopping for a while( > "nifi.provenance.repository.max.storage.time") and the provenance data is no > longer visible from the interface. > some warning in the log file : > o.a.n.p.store.WriteAheadStorePartition Failed to remove Provenance Event > file .\provenance_repository\0.prov; this file should be cleaned up manually -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7536) Timeouts occurring updating Parameter Context
Mark Payne created NIFI-7536:
Summary: Timeouts occurring updating Parameter Context
Key: NIFI-7536
URL: https://issues.apache.org/jira/browse/NIFI-7536
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
I am encountering the following exception when trying to update a Parameter
Context. The Parameter Context is new and has no Parameters. Adding a new
parameter fails when attempting to check if all affected processors are stopped.
{quote}2020-06-08 16:09:35,329 WARN [Replicate Request Thread-25]
o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET
/nifi-api/process-groups/root/processors to due to
java.net.SocketTimeoutException: timeout
2020-06-08 16:09:35,332 WARN [Replicate Request Thread-25]
o.a.n.c.c.h.r.ThreadPoolRequestReplicator
java.net.SocketTimeoutException: timeout
at okio.Okio$4.newTimeoutException(Okio.java:232)
at okio.AsyncTimeout.exit(AsyncTimeout.java:285)
at okio.AsyncTimeout$2.read(AsyncTimeout.java:241)
at okio.RealBufferedSource.indexOf(RealBufferedSource.java:355)
at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:227)
at okhttp3.internal.http1.Http1Codec.readHeaderLine(Http1Codec.java:215)
at okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:189)
at
okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:88)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at
okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at
okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
at okhttp3.RealCall.execute(RealCall.java:77)
at
org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:146)
at
org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:140)
at
org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:647)
at
org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:839)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketException: Socket closed
at java.net.SocketInputStream.read(SocketInputStream.java:203)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at okio.Okio$2.read(Okio.java:140)
at okio.AsyncTimeout$2.read(AsyncTimeout.java:237)
... 28 common frames omitted
2020-06-08 16:09:35,335 ERROR [Parameter Context Update Thread-1]
o.a.n.web.api.ParameterContextResource Failed to update Parameter Context
{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7536) Timeouts occurring updating Parameter Context
[
https://issues.apache.org/jira/browse/NIFI-7536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136088#comment-17136088
]
Mark Payne commented on NIFI-7536:
--
The following appear to be key details when troubleshooting.
* When creating the Response that describes a Processor, NiFi waits up to 1
millisecond for validation on a Processor to complete. However, the act of
waiting up to 1 millisecond can be fairly expensive and could actually result
in several milliseconds. If it takes 10 milliseconds per processor, for 1,000
processors that equates to 10 seconds already. In reality it's probably a
little less than that, but it's a very significant amount of time for a web
request. We can eliminate this all together and just avoid waiting for
validation to complete at all.
* For each processor that we return in the response, we must also include the
Processor's status. Obtaining the status currently is very inefficient because
it captures a snapshot status of the entire flow, and then only uses the status
for that processor. Again, not a big deal for 1 processor but for 1,000
processors (or many thousands) this because extremely expensive.
* When the Cluster Coordinator receives the response that contains this list
of Processors, it must merge together the responses from all nodes. This JSON
that it receives/parses/merges is huge. The code only really needs a couple of
fields, but the entire Processor data model is used because it already existed.
This parsing and merging takes a few seconds on my laptop with a flow that has
a couple thousand processors. We need to introduce a new REST endpoint that
returns only the information we care about.
When you add up all of these different inefficiencies, it's easy to see that
this works well for a small flow but can take a really long time for a large
flow.
> Timeouts occurring updating Parameter Context
> -
>
> Key: NIFI-7536
> URL: https://issues.apache.org/jira/browse/NIFI-7536
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
>
> I am encountering the following exception when trying to update a Parameter
> Context. The Parameter Context is new and has no Parameters. Adding a new
> parameter fails when attempting to check if all affected processors are
> stopped.
>
>
> {quote}2020-06-08 16:09:35,329 WARN [Replicate Request Thread-25]
> o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET
> /nifi-api/process-groups/root/processors to due to
> java.net.SocketTimeoutException: timeout
> 2020-06-08 16:09:35,332 WARN [Replicate Request Thread-25]
> o.a.n.c.c.h.r.ThreadPoolRequestReplicator
> java.net.SocketTimeoutException: timeout
> at okio.Okio$4.newTimeoutException(Okio.java:232)
> at okio.AsyncTimeout.exit(AsyncTimeout.java:285)
> at okio.AsyncTimeout$2.read(AsyncTimeout.java:241)
> at okio.RealBufferedSource.indexOf(RealBufferedSource.java:355)
> at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:227)
> at okhttp3.internal.http1.Http1Codec.readHeaderLine(Http1Codec.java:215)
> at okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:189)
> at
> okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:88)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> at
> okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
> at
> okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
> at
> okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> at
> okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
> at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
> at okhttp3.RealCall.execute(RealCall.java:77)
> at
> org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:146)
> at
> org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replica
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #810: MINIFICPP-1255 - Set absolute timeout and initialize timeouts
arpadboda commented on a change in pull request #810:
URL: https://github.com/apache/nifi-minifi-cpp/pull/810#discussion_r440371729
##
File path: extensions/http-curl/tests/HTTPHandlers.h
##
@@ -489,30 +489,39 @@ class InvokeHTTPResponse501Handler : public
ServerAwareHandler {
class TimeoutingHTTPHandler : public ServerAwareHandler {
public:
- TimeoutingHTTPHandler(std::chrono::milliseconds wait_ms)
- : wait_(wait_ms) {
+ TimeoutingHTTPHandler(std::vector wait_times)
+ : wait_times_(wait_times) {
}
bool handlePost(CivetServer *, struct mg_connection *conn) {
-std::this_thread::sleep_for(wait_);
-mg_printf(conn, "HTTP/1.1 201 OK\r\nContent-Type:
text/plain\r\nContent-Length: 0\r\nConnection: close\r\n\r\n");
+respond(conn);
return true;
}
bool handleGet(CivetServer *, struct mg_connection *conn) {
-std::this_thread::sleep_for(wait_);
-mg_printf(conn, "HTTP/1.1 201 OK\r\nContent-Type:
text/plain\r\nContent-Length: 0\r\nConnection: close\r\n\r\n");
+respond(conn);
return true;
}
bool handleDelete(CivetServer *, struct mg_connection *conn) {
-std::this_thread::sleep_for(wait_);
-mg_printf(conn, "HTTP/1.1 201 OK\r\nContent-Type:
text/plain\r\nContent-Length: 0\r\nConnection: close\r\n\r\n");
+respond(conn);
return true;
}
bool handlePut(CivetServer *, struct mg_connection *conn) {
-std::this_thread::sleep_for(wait_);
-mg_printf(conn, "HTTP/1.1 201 OK\r\nContent-Type:
text/plain\r\nContent-Length: 0\r\nConnection: close\r\n\r\n");
+respond(conn);
return true;
}
-protected:
- std::chrono::milliseconds wait_;
+ private:
+ void respond(struct mg_connection *conn) {
+if (wait_times_.size() > 0 && wait_times_[0].count() > 0) {
+ std::this_thread::sleep_for(wait_times_[0]);
+}
+int chunk_count = std::max(static_cast(wait_times_.size()) - 1, 0);
+mg_printf(conn, "HTTP/1.1 201 OK\r\nContent-Type:
text/plain\r\nContent-Length: %d\r\nConnection: close\r\n\r\n", chunk_count);
+for (int chunkIdx = 0; chunkIdx < chunk_count; ++chunkIdx) {
+ mg_printf(conn, "a");
Review comment:
Tricky :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-7537) LDAP Connection Timeout Issue
Nathan Gough created NIFI-7537:
--
Summary: LDAP Connection Timeout Issue
Key: NIFI-7537
URL: https://issues.apache.org/jira/browse/NIFI-7537
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.12.0
Reporter: Nathan Gough
Fix For: 1.12.0
The TLS refactor caused an issue with LDAP connection configuration,
specifically the 10 secs but
possibly other numerical fields.
Saw the below exception when attempting to use LDAP on master. Believe it is
related to changes from the SSLContext refactor work in -NIFI-7407.-
{code:java}
2020-06-10 17:58:54,263 ERROR [NiFi Web Server-89]
o.a.n.w.a.c.AdministrationExceptionMapper
org.apache.nifi.admin.service.AdministrationException: Unable to validate the
supplied credentials. Please contact the system administrator.. Returning
Internal Server Error response.2020-06-10 17:58:54,263 ERROR [NiFi Web
Server-89] o.a.n.w.a.c.AdministrationExceptionMapper
org.apache.nifi.admin.service.AdministrationException: Unable to validate the
supplied credentials. Please contact the system administrator.. Returning
Internal Server Error
response.org.apache.nifi.admin.service.AdministrationException: Unable to
validate the supplied credentials. Please contact the system administrator. at
org.apache.nifi.web.api.AccessResource.createAccessToken(AccessResource.java:737)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191)
at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at
org.glassfish.jersey.internal.Errors.process(Errors.java:316) at
org.glassfish.jersey.internal.Errors.process(Errors.java:298) at
org.glassfish.jersey.internal.Errors.process(Errors.java:268) at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256)
at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703)
at
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
at
org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1395)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755) at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1617)
at org.apache.nifi.web.filter.RequestLogger.doFilter(RequestLogger.java:66) at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
at
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:208)
at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
at org.apache.nifi.web.filter.TimerFilter.doFilter(TimerFilter.java:51) at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
at
org.apache.nifi.web.filter.ExceptionFilter.doFilter(Exceptio
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #814: MINIFICPP-1260 make void_t fwd-compatible with C++17
arpadboda closed pull request #814: URL: https://github.com/apache/nifi-minifi-cpp/pull/814 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #815: MINIFICPP-1153 Make the CTailFile tests thread-safe
arpadboda closed pull request #815: URL: https://github.com/apache/nifi-minifi-cpp/pull/815 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #810: MINIFICPP-1255 - Set absolute timeout and initialize timeouts
arpadboda closed pull request #810: URL: https://github.com/apache/nifi-minifi-cpp/pull/810 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #772: MINIFICPP-1153 Fix failing CTailFile Tests
arpadboda closed pull request #772: URL: https://github.com/apache/nifi-minifi-cpp/pull/772 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda commented on pull request #772: MINIFICPP-1153 Fix failing CTailFile Tests
arpadboda commented on pull request #772: URL: https://github.com/apache/nifi-minifi-cpp/pull/772#issuecomment-644364584 Closing this as fixed in scope of 46517012d1cc94e36d4368a2ca1d806a7e48ac6e This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 opened a new pull request #4337: NIFI-7536: Fix to improve performance of updating parameters
markap14 opened a new pull request #4337:
URL: https://github.com/apache/nifi/pull/4337
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-7536) Timeouts occurring updating Parameter Context
[
https://issues.apache.org/jira/browse/NIFI-7536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-7536:
-
Fix Version/s: 1.12.0
Status: Patch Available (was: Open)
> Timeouts occurring updating Parameter Context
> -
>
> Key: NIFI-7536
> URL: https://issues.apache.org/jira/browse/NIFI-7536
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.12.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I am encountering the following exception when trying to update a Parameter
> Context. The Parameter Context is new and has no Parameters. Adding a new
> parameter fails when attempting to check if all affected processors are
> stopped.
>
>
> {quote}2020-06-08 16:09:35,329 WARN [Replicate Request Thread-25]
> o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET
> /nifi-api/process-groups/root/processors to due to
> java.net.SocketTimeoutException: timeout
> 2020-06-08 16:09:35,332 WARN [Replicate Request Thread-25]
> o.a.n.c.c.h.r.ThreadPoolRequestReplicator
> java.net.SocketTimeoutException: timeout
> at okio.Okio$4.newTimeoutException(Okio.java:232)
> at okio.AsyncTimeout.exit(AsyncTimeout.java:285)
> at okio.AsyncTimeout$2.read(AsyncTimeout.java:241)
> at okio.RealBufferedSource.indexOf(RealBufferedSource.java:355)
> at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:227)
> at okhttp3.internal.http1.Http1Codec.readHeaderLine(Http1Codec.java:215)
> at okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:189)
> at
> okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:88)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> at
> okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
> at
> okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
> at
> okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> at
> okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
> at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
> at okhttp3.RealCall.execute(RealCall.java:77)
> at
> org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:146)
> at
> org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:140)
> at
> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:647)
> at
> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:839)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.SocketException: Socket closed
> at java.net.SocketInputStream.read(SocketInputStream.java:203)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
> at sun.security.ssl.InputRecord.read(InputRecord.java:503)
> at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
> at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
> at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
> at okio.Okio$2.read(Okio.java:140)
> at okio.AsyncTimeout$2.read(AsyncTimeout.java:237)
> ... 28 common frames omitted
> 2020-06-08 16:09:35,335 ERROR [Parameter Context Update Thread-1]
> o.a.n.web.api.ParameterContextResource Failed to update Parameter Context
> {quote}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7527) AbstractKuduProcessor deadlocks after TGT refresh
[
https://issues.apache.org/jira/browse/NIFI-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136144#comment-17136144
]
ASF subversion and git services commented on NIFI-7527:
---
Commit e02ffdd99fb3e0f561a50903f491b392a1a505cc in nifi's branch
refs/heads/master from Tamas Palfy
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e02ffdd ]
NIFI-7527 AbstractKuduProcessorrefresh TGT deadlock fix: Redesigned locking.
NIFI-7527 Fixed StackOverFlowError due to pacing issue (recursive login before
loggedIn flag is set).
NIFI-7527 Refactor: removed redundant kudu client creation.
This closes #4330.
Signed-off-by: Peter Turcsanyi
> AbstractKuduProcessor deadlocks after TGT refresh
> --
>
> Key: NIFI-7527
> URL: https://issues.apache.org/jira/browse/NIFI-7527
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Tamas Palfy
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The fix for https://issues.apache.org/jira/browse/NIFI-7453 (PutKudu kerberos
> issue after TGT expires) introduced a new bug: after TGT refresh the
> processor ends up in a deadlock.
> The reason is that the onTrigger initiates a read lock:
> {code:java}
> @Override
> public void onTrigger(final ProcessContext context, final ProcessSession
> session) throws ProcessException {
> kuduClientReadLock.lock();
> try {
> onTrigger(context, session, kuduClientR);
> } finally {
> kuduClientReadLock.unlock();
> }
> }
> {code}
> and while the read lock is in effect, later (in the same stack) - if TGT
> refresh occurs - a write lock is attempted:
> {code:java}
> ...
> public synchronized boolean checkTGTAndRelogin() throws
> LoginException {
> boolean didRelogin = super.checkTGTAndRelogin();
> if (didRelogin) {
> createKuduClient(context);
> }
> return didRelogin;
> }
> ...
> protected void createKuduClient(ProcessContext context) {
> kuduClientWriteLock.lock();
> try {
> if (this.kuduClientR.get() != null) {
> try {
> this.kuduClientR.get().close();
> } catch (KuduException e) {
> getLogger().error("Couldn't close Kudu client.");
> }
> }
> if (kerberosUser != null) {
> final KerberosAction kerberosAction = new
> KerberosAction<>(kerberosUser, () -> buildClient(context), getLogger());
> this.kuduClientR.set(kerberosAction.execute());
> } else {
> this.kuduClientR.set(buildClient(context));
> }
> } finally {
> kuduClientWriteLock.unlock();
> }
> }
> {code}
> This attempt at the write lock will get stuck, waiting for the previous read
> lock to get released.
> (Other threads may have acquired the same read lock but they can release it
> eventually - unless they too try to acquire the write lock themselves.)
> For the fix it seemed to be best to re-evalute the locking logic.
> Previously basically the whole onTrigger logic was encapsulated in a read
> lock, including the checking - and recreating as needed - the Kudu client
> (as explained before).
> It's best to just keep the actual privileged action in the read lock so the
> the refreshing of the TGT and re-creation of the Kudu client can safely be
> done in a write lock before that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7527) AbstractKuduProcessor deadlocks after TGT refresh
[
https://issues.apache.org/jira/browse/NIFI-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136145#comment-17136145
]
ASF subversion and git services commented on NIFI-7527:
---
Commit e02ffdd99fb3e0f561a50903f491b392a1a505cc in nifi's branch
refs/heads/master from Tamas Palfy
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e02ffdd ]
NIFI-7527 AbstractKuduProcessorrefresh TGT deadlock fix: Redesigned locking.
NIFI-7527 Fixed StackOverFlowError due to pacing issue (recursive login before
loggedIn flag is set).
NIFI-7527 Refactor: removed redundant kudu client creation.
This closes #4330.
Signed-off-by: Peter Turcsanyi
> AbstractKuduProcessor deadlocks after TGT refresh
> --
>
> Key: NIFI-7527
> URL: https://issues.apache.org/jira/browse/NIFI-7527
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Tamas Palfy
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The fix for https://issues.apache.org/jira/browse/NIFI-7453 (PutKudu kerberos
> issue after TGT expires) introduced a new bug: after TGT refresh the
> processor ends up in a deadlock.
> The reason is that the onTrigger initiates a read lock:
> {code:java}
> @Override
> public void onTrigger(final ProcessContext context, final ProcessSession
> session) throws ProcessException {
> kuduClientReadLock.lock();
> try {
> onTrigger(context, session, kuduClientR);
> } finally {
> kuduClientReadLock.unlock();
> }
> }
> {code}
> and while the read lock is in effect, later (in the same stack) - if TGT
> refresh occurs - a write lock is attempted:
> {code:java}
> ...
> public synchronized boolean checkTGTAndRelogin() throws
> LoginException {
> boolean didRelogin = super.checkTGTAndRelogin();
> if (didRelogin) {
> createKuduClient(context);
> }
> return didRelogin;
> }
> ...
> protected void createKuduClient(ProcessContext context) {
> kuduClientWriteLock.lock();
> try {
> if (this.kuduClientR.get() != null) {
> try {
> this.kuduClientR.get().close();
> } catch (KuduException e) {
> getLogger().error("Couldn't close Kudu client.");
> }
> }
> if (kerberosUser != null) {
> final KerberosAction kerberosAction = new
> KerberosAction<>(kerberosUser, () -> buildClient(context), getLogger());
> this.kuduClientR.set(kerberosAction.execute());
> } else {
> this.kuduClientR.set(buildClient(context));
> }
> } finally {
> kuduClientWriteLock.unlock();
> }
> }
> {code}
> This attempt at the write lock will get stuck, waiting for the previous read
> lock to get released.
> (Other threads may have acquired the same read lock but they can release it
> eventually - unless they too try to acquire the write lock themselves.)
> For the fix it seemed to be best to re-evalute the locking logic.
> Previously basically the whole onTrigger logic was encapsulated in a read
> lock, including the checking - and recreating as needed - the Kudu client
> (as explained before).
> It's best to just keep the actual privileged action in the read lock so the
> the refreshing of the TGT and re-creation of the Kudu client can safely be
> done in a write lock before that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4330: NIFI-7527 AbstractKuduProcessor refresh TGT deadlock fix
asfgit closed pull request #4330: URL: https://github.com/apache/nifi/pull/4330 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7527) AbstractKuduProcessor deadlocks after TGT refresh
[
https://issues.apache.org/jira/browse/NIFI-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136146#comment-17136146
]
ASF subversion and git services commented on NIFI-7527:
---
Commit e02ffdd99fb3e0f561a50903f491b392a1a505cc in nifi's branch
refs/heads/master from Tamas Palfy
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e02ffdd ]
NIFI-7527 AbstractKuduProcessorrefresh TGT deadlock fix: Redesigned locking.
NIFI-7527 Fixed StackOverFlowError due to pacing issue (recursive login before
loggedIn flag is set).
NIFI-7527 Refactor: removed redundant kudu client creation.
This closes #4330.
Signed-off-by: Peter Turcsanyi
> AbstractKuduProcessor deadlocks after TGT refresh
> --
>
> Key: NIFI-7527
> URL: https://issues.apache.org/jira/browse/NIFI-7527
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Tamas Palfy
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The fix for https://issues.apache.org/jira/browse/NIFI-7453 (PutKudu kerberos
> issue after TGT expires) introduced a new bug: after TGT refresh the
> processor ends up in a deadlock.
> The reason is that the onTrigger initiates a read lock:
> {code:java}
> @Override
> public void onTrigger(final ProcessContext context, final ProcessSession
> session) throws ProcessException {
> kuduClientReadLock.lock();
> try {
> onTrigger(context, session, kuduClientR);
> } finally {
> kuduClientReadLock.unlock();
> }
> }
> {code}
> and while the read lock is in effect, later (in the same stack) - if TGT
> refresh occurs - a write lock is attempted:
> {code:java}
> ...
> public synchronized boolean checkTGTAndRelogin() throws
> LoginException {
> boolean didRelogin = super.checkTGTAndRelogin();
> if (didRelogin) {
> createKuduClient(context);
> }
> return didRelogin;
> }
> ...
> protected void createKuduClient(ProcessContext context) {
> kuduClientWriteLock.lock();
> try {
> if (this.kuduClientR.get() != null) {
> try {
> this.kuduClientR.get().close();
> } catch (KuduException e) {
> getLogger().error("Couldn't close Kudu client.");
> }
> }
> if (kerberosUser != null) {
> final KerberosAction kerberosAction = new
> KerberosAction<>(kerberosUser, () -> buildClient(context), getLogger());
> this.kuduClientR.set(kerberosAction.execute());
> } else {
> this.kuduClientR.set(buildClient(context));
> }
> } finally {
> kuduClientWriteLock.unlock();
> }
> }
> {code}
> This attempt at the write lock will get stuck, waiting for the previous read
> lock to get released.
> (Other threads may have acquired the same read lock but they can release it
> eventually - unless they too try to acquire the write lock themselves.)
> For the fix it seemed to be best to re-evalute the locking logic.
> Previously basically the whole onTrigger logic was encapsulated in a read
> lock, including the checking - and recreating as needed - the Kudu client
> (as explained before).
> It's best to just keep the actual privileged action in the read lock so the
> the refreshing of the TGT and re-creation of the Kudu client can safely be
> done in a write lock before that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7527) AbstractKuduProcessor deadlocks after TGT refresh
[
https://issues.apache.org/jira/browse/NIFI-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-7527:
--
Component/s: Extensions
> AbstractKuduProcessor deadlocks after TGT refresh
> --
>
> Key: NIFI-7527
> URL: https://issues.apache.org/jira/browse/NIFI-7527
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Tamas Palfy
>Assignee: Tamas Palfy
>Priority: Major
> Fix For: 1.12.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The fix for https://issues.apache.org/jira/browse/NIFI-7453 (PutKudu kerberos
> issue after TGT expires) introduced a new bug: after TGT refresh the
> processor ends up in a deadlock.
> The reason is that the onTrigger initiates a read lock:
> {code:java}
> @Override
> public void onTrigger(final ProcessContext context, final ProcessSession
> session) throws ProcessException {
> kuduClientReadLock.lock();
> try {
> onTrigger(context, session, kuduClientR);
> } finally {
> kuduClientReadLock.unlock();
> }
> }
> {code}
> and while the read lock is in effect, later (in the same stack) - if TGT
> refresh occurs - a write lock is attempted:
> {code:java}
> ...
> public synchronized boolean checkTGTAndRelogin() throws
> LoginException {
> boolean didRelogin = super.checkTGTAndRelogin();
> if (didRelogin) {
> createKuduClient(context);
> }
> return didRelogin;
> }
> ...
> protected void createKuduClient(ProcessContext context) {
> kuduClientWriteLock.lock();
> try {
> if (this.kuduClientR.get() != null) {
> try {
> this.kuduClientR.get().close();
> } catch (KuduException e) {
> getLogger().error("Couldn't close Kudu client.");
> }
> }
> if (kerberosUser != null) {
> final KerberosAction kerberosAction = new
> KerberosAction<>(kerberosUser, () -> buildClient(context), getLogger());
> this.kuduClientR.set(kerberosAction.execute());
> } else {
> this.kuduClientR.set(buildClient(context));
> }
> } finally {
> kuduClientWriteLock.unlock();
> }
> }
> {code}
> This attempt at the write lock will get stuck, waiting for the previous read
> lock to get released.
> (Other threads may have acquired the same read lock but they can release it
> eventually - unless they too try to acquire the write lock themselves.)
> For the fix it seemed to be best to re-evalute the locking logic.
> Previously basically the whole onTrigger logic was encapsulated in a read
> lock, including the checking - and recreating as needed - the Kudu client
> (as explained before).
> It's best to just keep the actual privileged action in the read lock so the
> the refreshing of the TGT and re-creation of the Kudu client can safely be
> done in a write lock before that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (NIFI-7527) AbstractKuduProcessor deadlocks after TGT refresh
[
https://issues.apache.org/jira/browse/NIFI-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi reassigned NIFI-7527:
-
Assignee: Tamas Palfy
> AbstractKuduProcessor deadlocks after TGT refresh
> --
>
> Key: NIFI-7527
> URL: https://issues.apache.org/jira/browse/NIFI-7527
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Tamas Palfy
>Assignee: Tamas Palfy
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The fix for https://issues.apache.org/jira/browse/NIFI-7453 (PutKudu kerberos
> issue after TGT expires) introduced a new bug: after TGT refresh the
> processor ends up in a deadlock.
> The reason is that the onTrigger initiates a read lock:
> {code:java}
> @Override
> public void onTrigger(final ProcessContext context, final ProcessSession
> session) throws ProcessException {
> kuduClientReadLock.lock();
> try {
> onTrigger(context, session, kuduClientR);
> } finally {
> kuduClientReadLock.unlock();
> }
> }
> {code}
> and while the read lock is in effect, later (in the same stack) - if TGT
> refresh occurs - a write lock is attempted:
> {code:java}
> ...
> public synchronized boolean checkTGTAndRelogin() throws
> LoginException {
> boolean didRelogin = super.checkTGTAndRelogin();
> if (didRelogin) {
> createKuduClient(context);
> }
> return didRelogin;
> }
> ...
> protected void createKuduClient(ProcessContext context) {
> kuduClientWriteLock.lock();
> try {
> if (this.kuduClientR.get() != null) {
> try {
> this.kuduClientR.get().close();
> } catch (KuduException e) {
> getLogger().error("Couldn't close Kudu client.");
> }
> }
> if (kerberosUser != null) {
> final KerberosAction kerberosAction = new
> KerberosAction<>(kerberosUser, () -> buildClient(context), getLogger());
> this.kuduClientR.set(kerberosAction.execute());
> } else {
> this.kuduClientR.set(buildClient(context));
> }
> } finally {
> kuduClientWriteLock.unlock();
> }
> }
> {code}
> This attempt at the write lock will get stuck, waiting for the previous read
> lock to get released.
> (Other threads may have acquired the same read lock but they can release it
> eventually - unless they too try to acquire the write lock themselves.)
> For the fix it seemed to be best to re-evalute the locking logic.
> Previously basically the whole onTrigger logic was encapsulated in a read
> lock, including the checking - and recreating as needed - the Kudu client
> (as explained before).
> It's best to just keep the actual privileged action in the read lock so the
> the refreshing of the TGT and re-creation of the Kudu client can safely be
> done in a write lock before that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (NIFI-7527) AbstractKuduProcessor deadlocks after TGT refresh
[
https://issues.apache.org/jira/browse/NIFI-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi resolved NIFI-7527.
---
Fix Version/s: 1.12.0
Resolution: Fixed
> AbstractKuduProcessor deadlocks after TGT refresh
> --
>
> Key: NIFI-7527
> URL: https://issues.apache.org/jira/browse/NIFI-7527
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Tamas Palfy
>Assignee: Tamas Palfy
>Priority: Major
> Fix For: 1.12.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The fix for https://issues.apache.org/jira/browse/NIFI-7453 (PutKudu kerberos
> issue after TGT expires) introduced a new bug: after TGT refresh the
> processor ends up in a deadlock.
> The reason is that the onTrigger initiates a read lock:
> {code:java}
> @Override
> public void onTrigger(final ProcessContext context, final ProcessSession
> session) throws ProcessException {
> kuduClientReadLock.lock();
> try {
> onTrigger(context, session, kuduClientR);
> } finally {
> kuduClientReadLock.unlock();
> }
> }
> {code}
> and while the read lock is in effect, later (in the same stack) - if TGT
> refresh occurs - a write lock is attempted:
> {code:java}
> ...
> public synchronized boolean checkTGTAndRelogin() throws
> LoginException {
> boolean didRelogin = super.checkTGTAndRelogin();
> if (didRelogin) {
> createKuduClient(context);
> }
> return didRelogin;
> }
> ...
> protected void createKuduClient(ProcessContext context) {
> kuduClientWriteLock.lock();
> try {
> if (this.kuduClientR.get() != null) {
> try {
> this.kuduClientR.get().close();
> } catch (KuduException e) {
> getLogger().error("Couldn't close Kudu client.");
> }
> }
> if (kerberosUser != null) {
> final KerberosAction kerberosAction = new
> KerberosAction<>(kerberosUser, () -> buildClient(context), getLogger());
> this.kuduClientR.set(kerberosAction.execute());
> } else {
> this.kuduClientR.set(buildClient(context));
> }
> } finally {
> kuduClientWriteLock.unlock();
> }
> }
> {code}
> This attempt at the write lock will get stuck, waiting for the previous read
> lock to get released.
> (Other threads may have acquired the same read lock but they can release it
> eventually - unless they too try to acquire the write lock themselves.)
> For the fix it seemed to be best to re-evalute the locking logic.
> Previously basically the whole onTrigger logic was encapsulated in a read
> lock, including the checking - and recreating as needed - the Kudu client
> (as explained before).
> It's best to just keep the actual privileged action in the read lock so the
> the refreshing of the TGT and re-creation of the Kudu client can safely be
> done in a write lock before that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] arpadboda commented on pull request #808: MINIFICPP-1234 - Enable all archive tests on Windows.
arpadboda commented on pull request #808: URL: https://github.com/apache/nifi-minifi-cpp/pull/808#issuecomment-644408446 There are some conflicts, but other than that, doesn't this depend on #797 ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] naru014 commented on pull request #3511: NIFI-6175 Spark Livy - Improving Livy
naru014 commented on pull request #3511: URL: https://github.com/apache/nifi/pull/3511#issuecomment-644564279 > @naru014 Yes, it can. It also includes Batch processing controller service/processor. The one that comes with NiFi currently has severe limitations. I keep forgetting to get this one merged. I just need to update it for the latest version of NiFI. @patricker Thank you. We have got this build and included in the nifi. But when using a livy session controller service per nifi flow, the flows in some cases dont use the session created by the controller for that flow, but any other available session. Is this expected? So currently instead of creating a livy session controller per flow, I have created one livy session controller on the parent nifi-flow and it is inherited by all the child flows. By doing that, the nifi cluster has say min 2 sessions available and is used by any running flow based on its availability. Please let me know if there is a better way to use it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] naru014 edited a comment on pull request #3511: NIFI-6175 Spark Livy - Improving Livy
naru014 edited a comment on pull request #3511: URL: https://github.com/apache/nifi/pull/3511#issuecomment-644564279 > @naru014 Yes, it can. It also includes Batch processing controller service/processor. The one that comes with NiFi currently has severe limitations. I keep forgetting to get this one merged. I just need to update it for the latest version of NiFI. @patricker Thank you. We have got this built and included in the nifi. But when using a livy session controller service per nifi flow, the flows in some cases dont use the session created by the controller for that flow, but any other available session. Is this expected? So currently instead of creating a livy session controller per flow, I have created one livy session controller on the parent nifi-flow and it is inherited by all the child flows. By doing that, the nifi cluster has say min 2 sessions available and is used by any running flow based on its availability. Please let me know if there is a better way to use it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-1255) Always have non-zero default timeout for network operations.
[ https://issues.apache.org/jira/browse/MINIFICPP-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Debreceni resolved MINIFICPP-1255. --- Assignee: Adam Debreceni Resolution: Fixed > Always have non-zero default timeout for network operations. > > > Key: MINIFICPP-1255 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1255 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > Configure the curl requests with a default timeout and also enable an > absolute timeout, besides the progressMeter. -- This message was sent by Atlassian Jira (v8.3.4#803005)
