[GitHub] [nifi] naddym commented on pull request #4588: NIFI-7904: Support for validation query timeout in DBCP,Hive and HBas…
naddym commented on pull request #4588: URL: https://github.com/apache/nifi/pull/4588#issuecomment-719184116 @mattyb149 Any update on this PR please? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
szaszm commented on a change in pull request #932:
URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514640192
##
File path:
libminifi/src/controllers/keyvalue/PersistableKeyValueStoreService.cpp
##
@@ -29,20 +29,31 @@
PersistableKeyValueStoreService::PersistableKeyValueStoreService(const std::stri
PersistableKeyValueStoreService::~PersistableKeyValueStoreService() = default;
-bool PersistableKeyValueStoreService::setImpl(const std::string& key, const
std::string& value) {
- return set(key, value);
+bool PersistableKeyValueStoreService::setImpl(const utils::Identifier& key,
const std::string& value) {
+ return set(key.to_string(), value);
}
-bool PersistableKeyValueStoreService::getImpl(const std::string& key,
std::string& value) {
- return get(key, value);
+bool PersistableKeyValueStoreService::getImpl(const utils::Identifier& key,
std::string& value) {
+ return get(key.to_string(), value);
}
-bool PersistableKeyValueStoreService::getImpl(std::unordered_map& kvs) {
- return get(kvs);
+bool PersistableKeyValueStoreService::getImpl(std::map& kvs) {
+ std::unordered_map states;
+ if (!get(states)) {
+return false;
+ }
+ kvs.clear();
+ for (const auto& state : states) {
+utils::optional optional_uuid =
utils::Identifier::parse(state.first);
+if (optional_uuid) {
+ kvs[optional_uuid.value()] = state.second;
+}
Review comment:
Sounds like a violation of the Liskov substitution principle. Thanks for
the mitigation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
szaszm commented on a change in pull request #932:
URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514638672
##
File path: libminifi/include/core/CoreComponentState.h
##
@@ -48,13 +49,13 @@ class CoreComponentStateManagerProvider {
public:
virtual ~CoreComponentStateManagerProvider() = default;
- virtual std::shared_ptr
getCoreComponentStateManager(const std::string& uuid) = 0;
+ virtual std::shared_ptr
getCoreComponentStateManager(const utils::Identifier& uuid) = 0;
virtual std::shared_ptr
getCoreComponentStateManager(const CoreComponent& component) {
-return getCoreComponentStateManager(component.getUUIDStr());
+return getCoreComponentStateManager(component.getUUID());
}
- virtual std::unordered_map> getAllCoreComponentStates() = 0;
+ virtual std::map> getAllCoreComponentStates() = 0;
Review comment:
marking as resolved again as per
https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514057657
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
szaszm commented on a change in pull request #932:
URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514638319
##
File path: libminifi/src/ThreadedSchedulingAgent.cpp
##
@@ -110,16 +110,15 @@ void
ThreadedSchedulingAgent::schedule(std::shared_ptr processo
thread_pool_.execute(std::move(functor), future);
}
logger_->log_debug("Scheduled thread %d concurrent workers for for process
%s", processor->getMaxConcurrentTasks(), processor->getName());
- processors_running_.insert(processor->getUUIDStr());
- return;
+ processors_running_.insert(processor->getUUID());
}
void ThreadedSchedulingAgent::stop() {
SchedulingAgent::stop();
std::lock_guard lock(mutex_);
- for (const auto& p : processors_running_) {
-logger_->log_error("SchedulingAgent is stopped before processor was
unscheduled: %s", p);
-thread_pool_.stopTasks(p);
+ for (const auto& processor_id : processors_running_) {
+logger_->log_error("SchedulingAgent is stopped before processor was
unscheduled: %s", processor_id.to_string());
+thread_pool_.stopTasks(processor_id.to_string());
Review comment:
@arpadboda could you confirm or clarify, as the author/major refactorer
of this code?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
szaszm commented on a change in pull request #932:
URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514637504
##
File path: libminifi/include/core/logging/Logger.h
##
@@ -59,11 +59,18 @@ inline char const* conditional_conversion(const
utils::SmallString& arr) {
return arr.c_str();
}
-template
+template::value ||
+std::is_enum::value ||
+std::is_pointer::value>::type>
inline T conditional_conversion(T const& t) {
return t;
}
+inline char const* conditional_conversion(const char* str) {
+ return str;
+}
Review comment:
The reason you couldn't reply to my comment about hash is because it was
a reply to this discussion:
https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r513307831
To this thread: acknowledged, I'm fine with either way.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
szaszm commented on a change in pull request #932:
URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514637504
##
File path: libminifi/include/core/logging/Logger.h
##
@@ -59,11 +59,18 @@ inline char const* conditional_conversion(const
utils::SmallString& arr) {
return arr.c_str();
}
-template
+template::value ||
+std::is_enum::value ||
+std::is_pointer::value>::type>
inline T conditional_conversion(T const& t) {
return t;
}
+inline char const* conditional_conversion(const char* str) {
+ return str;
+}
Review comment:
The reason you couldn't reply to my comment about hash is because it was
a reply to this discussion:
https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r513307831 (You
can reply to the original, but not here.)
To this thread: acknowledged, I'm fine with either way.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-7452) Support adls_gen2_directory in Atlas reporting task
[ https://issues.apache.org/jira/browse/NIFI-7452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-7452: -- Status: Patch Available (was: In Progress) > Support adls_gen2_directory in Atlas reporting task > --- > > Key: NIFI-7452 > URL: https://issues.apache.org/jira/browse/NIFI-7452 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] turcsanyip opened a new pull request #4636: NIFI-7452: Support adls_gen2_directory in Atlas reporting task
turcsanyip opened a new pull request #4636:
URL: https://github.com/apache/nifi/pull/4636
Description of PR
https://issues.apache.org/jira/browse/NIFI-7452
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-7964) PutAzureBlobStorage OutOfMemory Exception
[
https://issues.apache.org/jira/browse/NIFI-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223281#comment-17223281
]
Eric Secules commented on NIFI-7964:
[~jfrazee] This is the first time I can think that i have tried uploading files
this large to blob storage using NiFi.
> PutAzureBlobStorage OutOfMemory Exception
> -
>
> Key: NIFI-7964
> URL: https://issues.apache.org/jira/browse/NIFI-7964
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.1
>Reporter: Eric Secules
>Assignee: Joey Frazee
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> As part of my flow I upload files to azure blob storage. They can be several
> hundred MB in size. I putting a 300 MB file into my flow and it choked on the
> PutAzureBlobStorage processor with the following log message.
> {code:java}
> 2020-10-28 19:34:10,717 ERROR [Timer-Driven Process Thread-6]
> o.a.n.p.a.storage.PutAzureBlobStorage
> PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5]
> PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5] failed to
> process session due to java.lang.OutOfMemoryError: Java heap space; Processor
> Administratively Yielded for 1 sec: java.lang.OutOfMemoryError: Java heap
> space
> java.lang.OutOfMemoryError: Java heap space
> 2020-10-28 19:34:10,717 WARN [Timer-Driven Process Thread-6]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding
> PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5] due to uncaught
> Exception: java.lang.OutOfMemoryError: Java heap space
> java.lang.OutOfMemoryError: Java heap space
> {code}
> I did not expect this to happen because I think the PutAzureBlob processor
> should be streaming the flowfile from disk directly to blob. But this
> behaviour suggests to me that it's getting read into memory in its entirety.
> My JVM heap size is set to 512 MB, which shouldn't be a problem if streaming
> was used to upload to blob storage in chunks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] MikeThomsen commented on pull request #4628: NIFI-7958 ConvertAvroToORC - decimal support
MikeThomsen commented on pull request #4628: URL: https://github.com/apache/nifi/pull/4628#issuecomment-719047603 You need to rebase against `main` and force push to your branch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] MikeThomsen opened a new pull request #4635: NIFI-7821 Added Cassandra-based DMC.
MikeThomsen opened a new pull request #4635:
URL: https://github.com/apache/nifi/pull/4635
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-7968) PutHDFS/PutParquet fail to write to Kerberized HDFS with KMS enabled
Bryan Bende created NIFI-7968:
-
Summary: PutHDFS/PutParquet fail to write to Kerberized HDFS with
KMS enabled
Key: NIFI-7968
URL: https://issues.apache.org/jira/browse/NIFI-7968
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.12.1, 1.12.0
Reporter: Bryan Bende
>From apache slack...
{color:#1d1c1d}My PutHDFS and PutParquet processors are configured to use a
KeytabCredentialsService. I've confirmed that that service is configured
correctly. The server also has the latest core-site and hdfs-site XML
configuration files from the HDFS cluster. However, whenever either of those
processors run, we receive the attached error message.{color}
{code:java}
2020-10-13 21:37:33,547 WARN [Timer-Driven Process Thread-100]
o.a.h.c.k.k.LoadBalancingKMSClientProvider KMS provider at [https://:9393/kms/v1/] threw an IOException:java.io.IOException:
org.apache.hadoop.security.authentication.client.AuthenticationException: Error
while authenticating with endpoint: https://:9393/kms/v1/keyversion/keyname/_eek?eek_op=decryptat
org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:525)
at
org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:826)
at
org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:351)
at
org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:347)
at
org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:172)
at
org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.decryptEncryptedKey(LoadBalancingKMSClientProvider.java:347)
at
org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:532)
at
org.apache.hadoop.hdfs.HdfsKMSUtil.decryptEncryptedDataEncryptionKey(HdfsKMSUtil.java:206)
at
org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:966)
at
org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:947)
at
org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:533)
at
org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:527)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:541)
at
org.apache.nifi.processors.hadoop.PutHDFS$1$1.process(PutHDFS.java:337)
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2324)
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2292)
at org.apache.nifi.processors.hadoop.PutHDFS$1.run(PutHDFS.java:320)
at java.security.AccessController.doPrivileged(Native Method)at
javax.security.auth.Subject.doAs(Subject.java:360)at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710)
at
org.apache.nifi.processors.hadoop.PutHDFS.onTrigger(PutHDFS.java:250)at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1174)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)Caused by:
org.apache.hadoop.security.authentication.client.AuthenticationException: Error
while authenticating with endpoint: https://:9393/kms/v1/keyversion/keyname/_eek?eek_op=decryptat
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
[jira] [Commented] (NIFI-7964) PutAzureBlobStorage OutOfMemory Exception
[
https://issues.apache.org/jira/browse/NIFI-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223083#comment-17223083
]
Joey Frazee commented on NIFI-7964:
---
[~esecules] AFAICT the dependency version hasn't changed in a while and my read
of the CloudBlockBlob class is that this isn't a bug per se but confusing
behavior. Did this work for you (i.e., no OOMEs) with earlier releases (I'll go
back and check if I get some time)?
There's an open PR for NIFI-7677 to upgrade from the currently used
azure-storage to azure-storage-blob which is actively maintained. This could be
a reason to get it moving again.
> PutAzureBlobStorage OutOfMemory Exception
> -
>
> Key: NIFI-7964
> URL: https://issues.apache.org/jira/browse/NIFI-7964
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.1
>Reporter: Eric Secules
>Assignee: Joey Frazee
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> As part of my flow I upload files to azure blob storage. They can be several
> hundred MB in size. I putting a 300 MB file into my flow and it choked on the
> PutAzureBlobStorage processor with the following log message.
> {code:java}
> 2020-10-28 19:34:10,717 ERROR [Timer-Driven Process Thread-6]
> o.a.n.p.a.storage.PutAzureBlobStorage
> PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5]
> PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5] failed to
> process session due to java.lang.OutOfMemoryError: Java heap space; Processor
> Administratively Yielded for 1 sec: java.lang.OutOfMemoryError: Java heap
> space
> java.lang.OutOfMemoryError: Java heap space
> 2020-10-28 19:34:10,717 WARN [Timer-Driven Process Thread-6]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding
> PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5] due to uncaught
> Exception: java.lang.OutOfMemoryError: Java heap space
> java.lang.OutOfMemoryError: Java heap space
> {code}
> I did not expect this to happen because I think the PutAzureBlob processor
> should be streaming the flowfile from disk directly to blob. But this
> behaviour suggests to me that it's getting read into memory in its entirety.
> My JVM heap size is set to 512 MB, which shouldn't be a problem if streaming
> was used to upload to blob storage in chunks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] jfrazee commented on a change in pull request #4613: NiFi-7819 - Add Zookeeper client TLS (external zookeeper) for cluster state management
jfrazee commented on a change in pull request #4613:
URL: https://github.com/apache/nifi/pull/4613#discussion_r510327902
##
File path:
nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/NiFiProperties.java
##
@@ -1335,6 +1335,13 @@ public boolean isZooKeeperClientSecure() {
return Boolean.parseBoolean(clientSecure);
}
+public boolean isZooKeeperTlsConfigurationPresent() {
Review comment:
Just to sanity check. This doesn't
include`NiFiProperties.ZOOKEEPER_CLIENT_SECURE` since it's possible to have
cluster ZK security without state management ZK security and vice versa?
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/state/manager/StandardStateManagerProvider.java
##
@@ -76,6 +77,8 @@ private StandardStateManagerProvider(final StateProvider
localStateProvider, fin
this.clusterStateProvider = clusterStateProvider;
}
+public static Map tlsPropertyMappings;
Review comment:
I don't think this gets used anywhere.
```suggestion
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] mtien-apache commented on pull request #4512: NIFI-1121: Support making properties dependent upon one another
mtien-apache commented on pull request #4512: URL: https://github.com/apache/nifi/pull/4512#issuecomment-718869459 I fixed the error and submitted a patch to markap14. I added another check for dependent values. Thank you. @bbende This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-registry] kevdoran commented on pull request #311: NIFIREG-429 Detect if older flyway table is already present in databa…
kevdoran commented on pull request #311: URL: https://github.com/apache/nifi-registry/pull/311#issuecomment-718867936 thanks, @bbende - will review... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Issue Comment Deleted] (NIFI-7903) UpdateAttribute advanced configuration error "Unable to load the rule list and evalaution criteria."
[ https://issues.apache.org/jira/browse/NIFI-7903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristen Guthier updated NIFI-7903: -- Comment: was deleted (was: Missed requirement to build with Java 8) > UpdateAttribute advanced configuration error "Unable to load the rule list > and evalaution criteria." > > > Key: NIFI-7903 > URL: https://issues.apache.org/jira/browse/NIFI-7903 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.13.0 > Environment: Ubuntu 18.04, OpenJDK 11.0.8 >Reporter: Kristen Guthier >Priority: Major > > Opening UpdateAttribute>Configure>Advanced in nifi.1.13.0-SNAPSHOT causes > error "Unable to load the rule list and evalaution criteria." > If that error is ignored and a rule added/named, when a condition expression > is created, a configuration error is reported: >content="text/html;charset=utf-8"/> Error 500 > java.lang.NullPointerException HTTP ERROR 500 > java.lang.NullPointerException > URI:/nifi-update-attribute-ui-1.13.0-SNAPSHOT/api/criteria/rules/conditions > STATUS:500 > MESSAGE:java.lang.NullPointerException > SERVLET:api CAUSED > BY:java.lang.NullPointerException Caused > by:java.lang.NullPointerException at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) > at > org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1395) > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755) at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1617) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:100) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:59) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:100) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:59) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:100) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:59) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:100) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:59) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:100) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:59) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:96) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) > at > org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177) > at > org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347) >
[jira] [Reopened] (NIFI-7903) UpdateAttribute advanced configuration error "Unable to load the rule list and evalaution criteria."
[ https://issues.apache.org/jira/browse/NIFI-7903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristen Guthier reopened NIFI-7903: --- > UpdateAttribute advanced configuration error "Unable to load the rule list > and evalaution criteria." > > > Key: NIFI-7903 > URL: https://issues.apache.org/jira/browse/NIFI-7903 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.13.0 > Environment: Ubuntu 18.04, OpenJDK 11.0.8 >Reporter: Kristen Guthier >Priority: Major > > Opening UpdateAttribute>Configure>Advanced in nifi.1.13.0-SNAPSHOT causes > error "Unable to load the rule list and evalaution criteria." > If that error is ignored and a rule added/named, when a condition expression > is created, a configuration error is reported: >content="text/html;charset=utf-8"/> Error 500 > java.lang.NullPointerException HTTP ERROR 500 > java.lang.NullPointerException > URI:/nifi-update-attribute-ui-1.13.0-SNAPSHOT/api/criteria/rules/conditions > STATUS:500 > MESSAGE:java.lang.NullPointerException > SERVLET:api CAUSED > BY:java.lang.NullPointerException Caused > by:java.lang.NullPointerException at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) > at > org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1395) > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755) at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1617) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:100) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:59) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:100) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:59) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:100) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:59) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:100) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:59) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:100) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:59) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:96) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) > at > org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177) > at > org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347) > at >
[GitHub] [nifi-registry] bbende opened a new pull request #311: NIFIREG-429 Detect if older flyway table is already present in databa…
bbende opened a new pull request #311:
URL: https://github.com/apache/nifi-registry/pull/311
…se, if so then set Flyway config to keep using that
Thank you for submitting a contribution to Apache NiFi Registry.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFIREG-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFIREG-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi-registry` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-registry-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-registry-assembly`?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] r65535 opened a new pull request #4634: NIFI-7967: Added regex support for attribute header selection on HandleHTTPResponse
r65535 opened a new pull request #4634:
URL: https://github.com/apache/nifi/pull/4634
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [x] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFIREG-429) Flyway error when upgrading from older release to 0.8.0
Bryan Bende created NIFIREG-429:
---
Summary: Flyway error when upgrading from older release to 0.8.0
Key: NIFIREG-429
URL: https://issues.apache.org/jira/browse/NIFIREG-429
Project: NiFi Registry
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Bryan Bende
Assignee: Bryan Bende
Fix For: 0.9.0
When taking an H2 database from 0.3.0 and dropping it into 0.8.0 or
0.9.0-SNAPSHOT, it produces the following error:
{code:java}
Caused by: org.flywaydb.core.api.FlywayException: Found non-empty schema(s)
"PUBLIC" without schema history table! Use baseline() or set baselineOnMigrate
to true to initialize the schema history table.
at org.flywaydb.core.Flyway$1.execute(Flyway.java:177)
at org.flywaydb.core.Flyway$1.execute(Flyway.java:149)
at org.flywaydb.core.Flyway.execute(Flyway.java:511)
at org.flywaydb.core.Flyway.migrate(Flyway.java:149)
at
org.apache.nifi.registry.db.CustomFlywayMigrationStrategy.migrate(CustomFlywayMigrationStrategy.java:88)
at
org.springframework.boot.autoconfigure.flyway.FlywayMigrationInitializer.afterPropertiesSet(FlywayMigrationInitializer.java:62)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1855)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1792)
... 98 common frames omitted {code}
Looks like the issue is because Flyway changed their default table name at some
point.
https://github.com/flyway/flyway/issues/1848
The issue does not happen going from 0.7.0 to 0.8.0, or 0.3.0 to 0.7.0, which
is probably because 0.8.0 changed the Flyway version from 5.x to 6.x.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7967) Regex support for response headers on HandleHTTPResponse
r65535 created NIFI-7967: Summary: Regex support for response headers on HandleHTTPResponse Key: NIFI-7967 URL: https://issues.apache.org/jira/browse/NIFI-7967 Project: Apache NiFi Issue Type: Improvement Reporter: r65535 Assignee: r65535 HandleHTTPResponse supports adding attributes as HTTP headers using dynamic properties, but it'd be nice to also support header selection by regex (like PostHTTP). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7966) putkudu 1.12.0 memory leak
[ https://issues.apache.org/jira/browse/NIFI-7966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222965#comment-17222965 ] Pierre Villard commented on NIFI-7966: -- Hi [~suyuji] - Can you please share the configuration of the processor (all tabs)? > putkudu 1.12.0 memory leak > -- > > Key: NIFI-7966 > URL: https://issues.apache.org/jira/browse/NIFI-7966 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.12.0 >Reporter: yj >Priority: Critical > Attachments: histo.txt, image-2020-10-29-15-29-03-910.png, kudu01.png > > > !image-2020-10-29-15-29-03-910.png! > see histo.txt -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] bbende commented on a change in pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
bbende commented on a change in pull request #4629:
URL: https://github.com/apache/nifi/pull/4629#discussion_r514338031
##
File path:
nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/SecurityUtil.java
##
@@ -141,4 +143,37 @@ public static boolean isSecurityEnabled(final
Configuration config) {
Validate.notNull(config);
return
KERBEROS.equalsIgnoreCase(config.get(HADOOP_SECURITY_AUTHENTICATION));
}
+
+public static T
callWithUgi(CallableThrowingException
ugiProvider, CallableThrowingException function) throws
IOException {
Review comment:
I haven't looked too closely, but I was also wondering about the need
for `CallableThrowingException `, isn't it basically providing the same thing
as `PrivilegedExceptionAction` ?
I would think the code inside the client services could just be:
```
getUgi().doAs((PrivilegedExceptionAction)() -> {
...
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] adenes commented on a change in pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
adenes commented on a change in pull request #4629:
URL: https://github.com/apache/nifi/pull/4629#discussion_r514307394
##
File path:
nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/SecurityUtil.java
##
@@ -141,4 +143,37 @@ public static boolean isSecurityEnabled(final
Configuration config) {
Validate.notNull(config);
return
KERBEROS.equalsIgnoreCase(config.get(HADOOP_SECURITY_AUTHENTICATION));
}
+
+public static T
callWithUgi(CallableThrowingException
ugiProvider, CallableThrowingException function) throws
IOException {
Review comment:
wow, the first point from this comment has been fixed even before I
submitted the comment :) Thanks @tpalfy
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] adenes commented on a change in pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
adenes commented on a change in pull request #4629:
URL: https://github.com/apache/nifi/pull/4629#discussion_r514300383
##
File path:
nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/SecurityUtil.java
##
@@ -141,4 +143,37 @@ public static boolean isSecurityEnabled(final
Configuration config) {
Validate.notNull(config);
return
KERBEROS.equalsIgnoreCase(config.get(HADOOP_SECURITY_AUTHENTICATION));
}
+
+public static T
callWithUgi(CallableThrowingException
ugiProvider, CallableThrowingException function) throws
IOException {
Review comment:
In my opinion passing in a `UserGroupProvider` instance instead of the
`ugiProvider` would make no difference from the functionality point of view but
it'd simplify the usage/improve the readability.
As for the second parameter I'm not fully sure why we need the
`CallableThrowingException` class. For me simple `Callable` seems to be ok too,
even though it doesn't specify its exception type, but I don't see the need for
that.
##
File path:
nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/SecurityUtil.java
##
@@ -141,4 +143,37 @@ public static boolean isSecurityEnabled(final
Configuration config) {
Validate.notNull(config);
return
KERBEROS.equalsIgnoreCase(config.get(HADOOP_SECURITY_AUTHENTICATION));
}
+
+public static T
callWithUgi(CallableThrowingException
ugiProvider, CallableThrowingException function) throws
IOException {
+try {
+return ugiProvider.call().doAs((PrivilegedExceptionAction)() ->
function.call());
Review comment:
minor nit: `() -> function.call()` could be replaced with method
reference, i.e. `function::call`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Comment Edited] (NIFI-7957) Nifi Content Repo Viewer not working with OIDC
[ https://issues.apache.org/jira/browse/NIFI-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222893#comment-17222893 ] Bryan Bende edited comment on NIFI-7957 at 10/29/20, 1:09 PM: -- Please don't submit bugs against an area of the code you have patched. We will be wasting our time looking into issues we could never reproduce without your code. was (Author: bende): Please don't submit bugs against an area of the code you have patched. We will be wasting out time looking into issues we could never reproduce without your code. > Nifi Content Repo Viewer not working with OIDC > -- > > Key: NIFI-7957 > URL: https://issues.apache.org/jira/browse/NIFI-7957 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.12.1 >Reporter: Jenil Shah >Priority: Major > Attachments: Screenshot from 2020-10-28 15-11-57.png, Screenshot from > 2020-10-28 15-15-05.png > > > I have enabled OIDC auth in nifi and given proper access permission to user > to view content of flow file.Now I try to see content of flow file from NIFI > UI but it is giving me error. > In ideal flow of nifi,nifi fetch the token using rest api and pass this token > as Authentication header in all subsequent request but when we try to view > content of flowfile it is opening new tab in which all those things are not > happening so Authorization header is not passed in request which is generated > from new tab. This causes auth problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-7957) Nifi Content Repo Viewer not working with OIDC
[ https://issues.apache.org/jira/browse/NIFI-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende resolved NIFI-7957. --- Resolution: Not A Bug > Nifi Content Repo Viewer not working with OIDC > -- > > Key: NIFI-7957 > URL: https://issues.apache.org/jira/browse/NIFI-7957 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.12.1 >Reporter: Jenil Shah >Priority: Major > Attachments: Screenshot from 2020-10-28 15-11-57.png, Screenshot from > 2020-10-28 15-15-05.png > > > I have enabled OIDC auth in nifi and given proper access permission to user > to view content of flow file.Now I try to see content of flow file from NIFI > UI but it is giving me error. > In ideal flow of nifi,nifi fetch the token using rest api and pass this token > as Authentication header in all subsequent request but when we try to view > content of flowfile it is opening new tab in which all those things are not > happening so Authorization header is not passed in request which is generated > from new tab. This causes auth problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7957) Nifi Content Repo Viewer not working with OIDC
[ https://issues.apache.org/jira/browse/NIFI-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222893#comment-17222893 ] Bryan Bende commented on NIFI-7957: --- Please don't submit bugs against an area of the code you have patched. We will be wasting out time looking into issues we could never reproduce without your code. > Nifi Content Repo Viewer not working with OIDC > -- > > Key: NIFI-7957 > URL: https://issues.apache.org/jira/browse/NIFI-7957 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.12.1 >Reporter: Jenil Shah >Priority: Major > Attachments: Screenshot from 2020-10-28 15-11-57.png, Screenshot from > 2020-10-28 15-15-05.png > > > I have enabled OIDC auth in nifi and given proper access permission to user > to view content of flow file.Now I try to see content of flow file from NIFI > UI but it is giving me error. > In ideal flow of nifi,nifi fetch the token using rest api and pass this token > as Authentication header in all subsequent request but when we try to view > content of flowfile it is opening new tab in which all those things are not > happening so Authorization header is not passed in request which is generated > from new tab. This causes auth problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (NIFI-7047) Flow file content is corrupt
[ https://issues.apache.org/jira/browse/NIFI-7047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222847#comment-17222847 ] Shevtsov Ihor edited comment on NIFI-7047 at 10/29/20, 11:40 AM: - Hi! I have the same problem and know how to reproduce it. When you have some queue before MergeContent processor and some of nodes are shutdown, when you come it back to life, you have the phantom or corrupt files in queue. Maybe the problem can occur when enabled LoadBalancer is enabled. was (Author: ingvar89): Hi! I have the same problem and know how to reproduce it. When you have some queue before merge content and some of nodes are shutdown, when you come it back to life, you have the phantom or corrupt files in queue. > Flow file content is corrupt > > > Key: NIFI-7047 > URL: https://issues.apache.org/jira/browse/NIFI-7047 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.9.2 > Environment: Cent OS (Linux) >Reporter: Shashikant Udapi >Priority: Critical > > The content of my flowfile seems corrupted. This happens intermittently and > bunch of flowfile contents from the few of the content repository section > seems corrupted. Looks like the flowfile content is picked from wrong offset. > > We work with json data format in most of the places. And these flowfile > content has broken json content. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7047) Flow file content is corrupt
[ https://issues.apache.org/jira/browse/NIFI-7047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222847#comment-17222847 ] Shevtsov Ihor commented on NIFI-7047: - Hi! I have the same problem and know how to reproduce it. When you have some queue before merge content and some of nodes are shutdown, when you come it back to life, you have the phantom or corrupt files in queue. > Flow file content is corrupt > > > Key: NIFI-7047 > URL: https://issues.apache.org/jira/browse/NIFI-7047 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.9.2 > Environment: Cent OS (Linux) >Reporter: Shashikant Udapi >Priority: Critical > > The content of my flowfile seems corrupted. This happens intermittently and > bunch of flowfile contents from the few of the content repository section > seems corrupted. Looks like the flowfile content is picked from wrong offset. > > We work with json data format in most of the places. And these flowfile > content has broken json content. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
adamdebreceni commented on a change in pull request #932:
URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514186240
##
File path: libminifi/src/provenance/Provenance.cpp
##
@@ -346,25 +352,33 @@ bool ProvenanceEventRecord::DeSerialize(const uint8_t
*buffer, const size_t buff
}
for (uint32_t i = 0; i < number; i++) {
- std::string parentUUID;
- ret = outStream.read(parentUUID);
+ std::string parentUUIDStr;
+ ret = outStream.read(parentUUIDStr);
if (ret <= 0) {
return false;
}
- this->addParentUuid(parentUUID);
+ utils::optional parentUUID =
utils::Identifier::parse(parentUUIDStr);
+ if (!parentUUID) {
+return false;
+ }
+ this->addParentUuid(parentUUID.value());
}
number = 0;
ret = outStream.read(number);
if (ret != 4) {
return false;
}
for (uint32_t i = 0; i < number; i++) {
- std::string childUUID;
- ret = outStream.read(childUUID);
+ std::string childUUIDStr;
+ ret = outStream.read(childUUIDStr);
if (ret <= 0) {
return false;
}
- this->addChildUuid(childUUID);
+ utils::optional childUUID =
utils::Identifier::parse(childUUIDStr);
+ if (!childUUID) {
+return false;
+ }
+ this->addChildUuid(childUUID.value());
Review comment:
added, also added an `OutputStream:read` overload for symmetry
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
adamdebreceni commented on a change in pull request #932: URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514185977 ## File path: nanofi/src/cxx/Plan.cpp ## @@ -304,11 +304,11 @@ std::shared_ptr ExecutionPlan::connectProcessors(std::shared connection->setSource(src_proc); utils::Identifier uuid_copy, uuid_copy_next; - src_proc->getUUID(uuid_copy); + uuid_copy = src_proc->getUUID(); connection->setSourceUUID(uuid_copy); Review comment: done ## File path: libminifi/test/TestBase.cpp ## @@ -104,9 +104,9 @@ std::shared_ptr TestPlan::addProcessor(const std::shared_ptrsetDestination(processor); utils::Identifier uuid_copy, uuid_copy_next; -last->getUUID(uuid_copy); +uuid_copy = last->getUUID(); connection->setSourceUUID(uuid_copy); -processor->getUUID(uuid_copy_next); +uuid_copy_next = processor->getUUID(); connection->setDestinationUUID(uuid_copy_next); Review comment: done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
fgerlits commented on a change in pull request #932: URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514181694 ## File path: libminifi/src/provenance/Provenance.cpp ## @@ -128,7 +128,7 @@ bool ProvenanceEventRecord::Serialize(org::apache::nifi::minifi::io::BufferStrea return false; } - ret = outStream.write(this->flow_uuid_); + ret = outStream.write(this->flow_uuid_.to_string()); Review comment: these `.to_string()`s are no longer needed here, right? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
adamdebreceni commented on a change in pull request #932:
URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514137427
##
File path:
libminifi/src/controllers/keyvalue/PersistableKeyValueStoreService.cpp
##
@@ -29,20 +29,29 @@
PersistableKeyValueStoreService::PersistableKeyValueStoreService(const std::stri
PersistableKeyValueStoreService::~PersistableKeyValueStoreService() = default;
-bool PersistableKeyValueStoreService::setImpl(const std::string& key, const
std::string& value) {
- return set(key, value);
+bool PersistableKeyValueStoreService::setImpl(const utils::Identifier& key,
const std::string& value) {
+ return set(key.to_string(), value);
}
-bool PersistableKeyValueStoreService::getImpl(const std::string& key,
std::string& value) {
- return get(key, value);
+bool PersistableKeyValueStoreService::getImpl(const utils::Identifier& key,
std::string& value) {
+ return get(key.to_string(), value);
}
-bool PersistableKeyValueStoreService::getImpl(std::unordered_map& kvs) {
- return get(kvs);
+bool PersistableKeyValueStoreService::getImpl(std::map& kvs) {
+ std::unordered_map states;
+ if (!get(states)) {
+return false;
+ }
+ kvs.clear();
+ for (const auto& state : states) {
+const auto uuid = utils::Identifier::parse(state.first);
+kvs[uuid.value()] = state.second;
Review comment:
rewrote it, to ignore such cases, but log errors, (should be refactored
to contain an instance of `KeyValueStoreService` instead of extending it)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
adamdebreceni commented on a change in pull request #932:
URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514136812
##
File path: libminifi/src/ThreadedSchedulingAgent.cpp
##
@@ -110,16 +110,15 @@ void
ThreadedSchedulingAgent::schedule(std::shared_ptr processo
thread_pool_.execute(std::move(functor), future);
}
logger_->log_debug("Scheduled thread %d concurrent workers for for process
%s", processor->getMaxConcurrentTasks(), processor->getName());
- processors_running_.insert(processor->getUUIDStr());
- return;
+ processors_running_.insert(processor->getUUID());
}
void ThreadedSchedulingAgent::stop() {
SchedulingAgent::stop();
std::lock_guard lock(mutex_);
for (const auto& p : processors_running_) {
-logger_->log_error("SchedulingAgent is stopped before processor was
unscheduled: %s", p);
-thread_pool_.stopTasks(p);
+logger_->log_error("SchedulingAgent is stopped before processor was
unscheduled: %s", p.to_string());
Review comment:
done
##
File path:
libminifi/src/controllers/keyvalue/PersistableKeyValueStoreService.cpp
##
@@ -29,20 +29,29 @@
PersistableKeyValueStoreService::PersistableKeyValueStoreService(const std::stri
PersistableKeyValueStoreService::~PersistableKeyValueStoreService() = default;
-bool PersistableKeyValueStoreService::setImpl(const std::string& key, const
std::string& value) {
- return set(key, value);
+bool PersistableKeyValueStoreService::setImpl(const utils::Identifier& key,
const std::string& value) {
+ return set(key.to_string(), value);
}
-bool PersistableKeyValueStoreService::getImpl(const std::string& key,
std::string& value) {
- return get(key, value);
+bool PersistableKeyValueStoreService::getImpl(const utils::Identifier& key,
std::string& value) {
+ return get(key.to_string(), value);
}
-bool PersistableKeyValueStoreService::getImpl(std::unordered_map& kvs) {
- return get(kvs);
+bool PersistableKeyValueStoreService::getImpl(std::map& kvs) {
+ std::unordered_map states;
+ if (!get(states)) {
+return false;
+ }
+ kvs.clear();
+ for (const auto& state : states) {
+const auto uuid = utils::Identifier::parse(state.first);
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-7966) putkudu 1.12.0 memory leak
[ https://issues.apache.org/jira/browse/NIFI-7966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yj updated NIFI-7966: - Priority: Critical (was: Major) > putkudu 1.12.0 memory leak > -- > > Key: NIFI-7966 > URL: https://issues.apache.org/jira/browse/NIFI-7966 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.12.0 >Reporter: yj >Priority: Critical > Attachments: histo.txt, image-2020-10-29-15-29-03-910.png, kudu01.png > > > !image-2020-10-29-15-29-03-910.png! > see histo.txt -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
adamdebreceni commented on a change in pull request #932: URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514079865 ## File path: libminifi/include/controllers/keyvalue/AbstractCoreComponentStateManagerProvider.h ## @@ -59,10 +60,10 @@ class AbstractCoreComponentStateManagerProvider : public std::enable_shared_from }; protected: - virtual bool setImpl(const std::string& key, const std::string& value) = 0; - virtual bool getImpl(const std::string& key, std::string& value) = 0; - virtual bool getImpl(std::unordered_map& kvs) = 0; - virtual bool removeImpl(const std::string& key) = 0; + virtual bool setImpl(const utils::Identifier& key, const std::string& value) = 0; + virtual bool getImpl(const utils::Identifier& key, std::string& value) = 0; + virtual bool getImpl(std::map& kvs) = 0; + virtual bool removeImpl(const utils::Identifier& key) = 0; Review comment: renamed the parameter, also added type alias for the `CoreComponentState` to `unordered_map` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
adamdebreceni commented on a change in pull request #932:
URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514069155
##
File path: libminifi/include/core/logging/Logger.h
##
@@ -59,11 +59,18 @@ inline char const* conditional_conversion(const
utils::SmallString& arr) {
return arr.c_str();
}
-template
+template::value ||
+std::is_enum::value ||
+std::is_pointer::value>::type>
inline T conditional_conversion(T const& t) {
return t;
}
+inline char const* conditional_conversion(const char* str) {
+ return str;
+}
Review comment:
for this comment, it is needed so we can allow
[this](https://godbolt.org/z/9drc8f) to compile, and now that I wrote this
down, I realize I could simply remove the `const&` qualifier on the template
```
#include
struct A{
static constexpr const char* STR = "abcd";
};
template::value ||
std::is_enum::value ||
std::is_pointer::value>::type>
inline T conditional_conversion(T const& t) {
return t;
}
// inline char const* conditional_conversion(const char* str) {
// return str;
// }
int main(){
std::cout << conditional_conversion(A::STR) << std::endl;
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
adamdebreceni commented on a change in pull request #932:
URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514063453
##
File path:
libminifi/src/controllers/keyvalue/PersistableKeyValueStoreService.cpp
##
@@ -29,20 +29,31 @@
PersistableKeyValueStoreService::PersistableKeyValueStoreService(const std::stri
PersistableKeyValueStoreService::~PersistableKeyValueStoreService() = default;
-bool PersistableKeyValueStoreService::setImpl(const std::string& key, const
std::string& value) {
- return set(key, value);
+bool PersistableKeyValueStoreService::setImpl(const utils::Identifier& key,
const std::string& value) {
+ return set(key.to_string(), value);
}
-bool PersistableKeyValueStoreService::getImpl(const std::string& key,
std::string& value) {
- return get(key, value);
+bool PersistableKeyValueStoreService::getImpl(const utils::Identifier& key,
std::string& value) {
+ return get(key.to_string(), value);
}
-bool PersistableKeyValueStoreService::getImpl(std::unordered_map& kvs) {
- return get(kvs);
+bool PersistableKeyValueStoreService::getImpl(std::map& kvs) {
+ std::unordered_map states;
+ if (!get(states)) {
+return false;
+ }
+ kvs.clear();
+ for (const auto& state : states) {
+utils::optional optional_uuid =
utils::Identifier::parse(state.first);
+if (optional_uuid) {
+ kvs[optional_uuid.value()] = state.second;
+}
Review comment:
since a `PerisistableKeyValueStoreService (PKVSS)` ISA
`KeyValueStoreService (KVSS)` (publicly inherits) which can store arbitrary
`std::string` keys, it is not impossible to cast a `PKVSS` to a `KVSS`, add
some "invalid" key, and "mess up" the `PKVSS`, I would refactor this to not
inherit from `KVSS` but contain an instance of it, for now I will add a log
message
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
adamdebreceni commented on a change in pull request #932:
URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514060941
##
File path: libminifi/src/ThreadedSchedulingAgent.cpp
##
@@ -110,16 +110,15 @@ void
ThreadedSchedulingAgent::schedule(std::shared_ptr processo
thread_pool_.execute(std::move(functor), future);
}
logger_->log_debug("Scheduled thread %d concurrent workers for for process
%s", processor->getMaxConcurrentTasks(), processor->getName());
- processors_running_.insert(processor->getUUIDStr());
- return;
+ processors_running_.insert(processor->getUUID());
}
void ThreadedSchedulingAgent::stop() {
SchedulingAgent::stop();
std::lock_guard lock(mutex_);
- for (const auto& p : processors_running_) {
-logger_->log_error("SchedulingAgent is stopped before processor was
unscheduled: %s", p);
-thread_pool_.stopTasks(p);
+ for (const auto& processor_id : processors_running_) {
+logger_->log_error("SchedulingAgent is stopped before processor was
unscheduled: %s", processor_id.to_string());
+thread_pool_.stopTasks(processor_id.to_string());
Review comment:
yes that seems to be the case, so processors with greater than one
maxConcurrentTasks share the same `task_status_` flag, this is probably not
intentional
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
adamdebreceni commented on a change in pull request #932:
URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514060941
##
File path: libminifi/src/ThreadedSchedulingAgent.cpp
##
@@ -110,16 +110,15 @@ void
ThreadedSchedulingAgent::schedule(std::shared_ptr processo
thread_pool_.execute(std::move(functor), future);
}
logger_->log_debug("Scheduled thread %d concurrent workers for for process
%s", processor->getMaxConcurrentTasks(), processor->getName());
- processors_running_.insert(processor->getUUIDStr());
- return;
+ processors_running_.insert(processor->getUUID());
}
void ThreadedSchedulingAgent::stop() {
SchedulingAgent::stop();
std::lock_guard lock(mutex_);
- for (const auto& p : processors_running_) {
-logger_->log_error("SchedulingAgent is stopped before processor was
unscheduled: %s", p);
-thread_pool_.stopTasks(p);
+ for (const auto& processor_id : processors_running_) {
+logger_->log_error("SchedulingAgent is stopped before processor was
unscheduled: %s", processor_id.to_string());
+thread_pool_.stopTasks(processor_id.to_string());
Review comment:
yes that seems to be the case, (so processors with greater than one
maxConcurrentTasks share the same `task_status_` flag)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #932: MINIFICPP-1395 - Use Identifier instead of its stringified form wherever possible
adamdebreceni commented on a change in pull request #932:
URL: https://github.com/apache/nifi-minifi-cpp/pull/932#discussion_r514057657
##
File path: libminifi/include/core/logging/Logger.h
##
@@ -59,11 +59,18 @@ inline char const* conditional_conversion(const
utils::SmallString& arr) {
return arr.c_str();
}
-template
+template::value ||
+std::is_enum::value ||
+std::is_pointer::value>::type>
inline T conditional_conversion(T const& t) {
return t;
}
+inline char const* conditional_conversion(const char* str) {
+ return str;
+}
Review comment:
for some reason I cannot reply to your comment about `std::hash`
specialization:
from [google
guidelines](https://google.github.io/styleguide/cppguide.html#std_hash)
> Do not define specializations of std::hash.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-7966) putkudu 1.12.0 memory leak
[ https://issues.apache.org/jira/browse/NIFI-7966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yj updated NIFI-7966: - Description: !image-2020-10-29-15-29-03-910.png! see histo.txt was: !image-2020-10-29-15-29-03-910.png! ssee histo.txt > putkudu 1.12.0 memory leak > -- > > Key: NIFI-7966 > URL: https://issues.apache.org/jira/browse/NIFI-7966 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.12.0 >Reporter: yj >Priority: Major > Attachments: histo.txt, image-2020-10-29-15-29-03-910.png, kudu01.png > > > !image-2020-10-29-15-29-03-910.png! > see histo.txt -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7966) putkudu 1.12.0 memory leak
yj created NIFI-7966: Summary: putkudu 1.12.0 memory leak Key: NIFI-7966 URL: https://issues.apache.org/jira/browse/NIFI-7966 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.12.0 Reporter: yj Attachments: histo.txt, image-2020-10-29-15-29-03-910.png, kudu01.png !image-2020-10-29-15-29-03-910.png! ssee histo.txt -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7949) Add tag HCFS to HDFS Processors
[ https://issues.apache.org/jira/browse/NIFI-7949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated NIFI-7949: - Status: Patch Available (was: Open) > Add tag HCFS to HDFS Processors > --- > > Key: NIFI-7949 > URL: https://issues.apache.org/jira/browse/NIFI-7949 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.12.1 >Reporter: Siyao Meng >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Adding tag "HCFS" to all HDFS Processors: > - DeleteHDFS > - FetchHDFS > - GetHDFS > - GetHDFSFileInfo > - GetHDFSSequenceFile > - ListHDFS > - MoveHDFS > - PutHDFS > --- > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hadoop-nar/1.12.1/org.apache.nifi.processors.hadoop.PutHDFS/index.html > The PutHDFS processor can actually write to any [Hadoop Compatible File > System > (HCFS)|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/filesystem/introduction.html], > where HDFS is merely one of them. > Especially with the emergence of Hadoop Ozone and others, the name of the > processor might be a bit inaccurate and misleading. > What do you think? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] smengcl opened a new pull request #4633: NIFI-7949: Add tag HCFS to HDFS Processors
smengcl opened a new pull request #4633:
URL: https://github.com/apache/nifi/pull/4633
Description of PR
https://issues.apache.org/jira/browse/NIFI-7949
Adding tag "HCFS" to all HDFS Processors:
- DeleteHDFS
- FetchHDFS
- GetHDFS
- GetHDFSFileInfo
- GetHDFSSequenceFile
- ListHDFS
- MoveHDFS
- PutHDFS
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
https://issues.apache.org/jira/browse/NIFI-7949
- [x] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [x] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-7949) Add tag HCFS to HDFS Processors
[ https://issues.apache.org/jira/browse/NIFI-7949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222715#comment-17222715 ] Siyao Meng commented on NIFI-7949: -- Thanks [~joewitt] and [~bbende]. I'm +1 on adding a "HCFS" tag for now. > Add tag HCFS to HDFS Processors > --- > > Key: NIFI-7949 > URL: https://issues.apache.org/jira/browse/NIFI-7949 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.12.1 >Reporter: Siyao Meng >Priority: Major > > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hadoop-nar/1.12.1/org.apache.nifi.processors.hadoop.PutHDFS/index.html > The PutHDFS processor can actually write to any [Hadoop Compatible File > System > (HCFS)|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/filesystem/introduction.html], > where HDFS is merely one of them. > Especially with the emergence of Hadoop Ozone and others, the name of the > processor might be a bit inaccurate and misleading. > What do you think? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7949) Add tag HCFS to HDFS Processors
[ https://issues.apache.org/jira/browse/NIFI-7949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated NIFI-7949: - Description: Adding tag "HCFS" to all HDFS Processors: - DeleteHDFS - FetchHDFS - GetHDFS - GetHDFSFileInfo - GetHDFSSequenceFile - ListHDFS - MoveHDFS - PutHDFS --- https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hadoop-nar/1.12.1/org.apache.nifi.processors.hadoop.PutHDFS/index.html The PutHDFS processor can actually write to any [Hadoop Compatible File System (HCFS)|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/filesystem/introduction.html], where HDFS is merely one of them. Especially with the emergence of Hadoop Ozone and others, the name of the processor might be a bit inaccurate and misleading. What do you think? was: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hadoop-nar/1.12.1/org.apache.nifi.processors.hadoop.PutHDFS/index.html The PutHDFS processor can actually write to any [Hadoop Compatible File System (HCFS)|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/filesystem/introduction.html], where HDFS is merely one of them. Especially with the emergence of Hadoop Ozone and others, the name of the processor might be a bit inaccurate and misleading. What do you think? > Add tag HCFS to HDFS Processors > --- > > Key: NIFI-7949 > URL: https://issues.apache.org/jira/browse/NIFI-7949 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.12.1 >Reporter: Siyao Meng >Priority: Major > > Adding tag "HCFS" to all HDFS Processors: > - DeleteHDFS > - FetchHDFS > - GetHDFS > - GetHDFSFileInfo > - GetHDFSSequenceFile > - ListHDFS > - MoveHDFS > - PutHDFS > --- > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hadoop-nar/1.12.1/org.apache.nifi.processors.hadoop.PutHDFS/index.html > The PutHDFS processor can actually write to any [Hadoop Compatible File > System > (HCFS)|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/filesystem/introduction.html], > where HDFS is merely one of them. > Especially with the emergence of Hadoop Ozone and others, the name of the > processor might be a bit inaccurate and misleading. > What do you think? -- This message was sent by Atlassian Jira (v8.3.4#803005)
