[jira] [Updated] (NIFI-13287) Add note to msal4j dependency in Azure bundle's pom
[
https://issues.apache.org/jira/browse/NIFI-13287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-13287:
---
Status: Patch Available (was: Open)
> Add note to msal4j dependency in Azure bundle's pom
> ---
>
> Key: NIFI-13287
> URL: https://issues.apache.org/jira/browse/NIFI-13287
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{com.azure:azure-identity}} in {{nifi-azure-nar}} has a dependency on
> {{com.microsoft.azure:msal4j}} and requires a given version of it. It could
> bring that version transitively but {{msal4j}} is also used directly by
> {{nifi-azure-graph-authorizer}} and the {{msal4j}} version needs to be
> configured in that pom. Unfortunately, {{azure-sdk-bom}} does not control the
> {{msal4j}} version (I think because it is a {{com.microsoft.azure}} artifact,
> not {{{}com.azure{}}}).
> Configuring the {{azure-identity}} (via the BOM) and {{msal4j}} versions
> separately caused dependency issues multiple times in the past (NIFI-9305,
> NIFI-13181). I will add a "heads-up" note in the pom to keep these versions
> consistent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-13287) Add note to msal4j dependency in Azure bundle's pom
Peter Turcsanyi created NIFI-13287:
--
Summary: Add note to msal4j dependency in Azure bundle's pom
Key: NIFI-13287
URL: https://issues.apache.org/jira/browse/NIFI-13287
Project: Apache NiFi
Issue Type: Improvement
Reporter: Peter Turcsanyi
Assignee: Peter Turcsanyi
{{com.azure:azure-identity}} in {{nifi-azure-nar}} has a dependency on
{{com.microsoft.azure:msal4j}} and requires a given version of it. It could
bring that version transitively but {{msal4j}} is also used directly by
{{nifi-azure-graph-authorizer}} and the {{msal4j}} version needs to be
configured in that pom. Unfortunately,
{{azure-sdk-bom}} does not control the {{msal4j}} version (I think because it
is a {{com.microsoft.azure}} artifact, not {{{}com.azure{}}}).
Configuring the {{azure-identity}} (via the BOM) and {{msal4j}} versions
separately caused dependency issues multiple times in the past (NIFI-9305,
NIFI-13181). I will add a "heads-up" note in the pom to keep these versions
consistent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13287) Add note to msal4j dependency in Azure bundle's pom
[
https://issues.apache.org/jira/browse/NIFI-13287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-13287:
---
Description:
{{com.azure:azure-identity}} in {{nifi-azure-nar}} has a dependency on
{{com.microsoft.azure:msal4j}} and requires a given version of it. It could
bring that version transitively but {{msal4j}} is also used directly by
{{nifi-azure-graph-authorizer}} and the {{msal4j}} version needs to be
configured in that pom. Unfortunately, {{azure-sdk-bom}} does not control the
{{msal4j}} version (I think because it is a {{com.microsoft.azure}} artifact,
not {{{}com.azure{}}}).
Configuring the {{azure-identity}} (via the BOM) and {{msal4j}} versions
separately caused dependency issues multiple times in the past (NIFI-9305,
NIFI-13181). I will add a "heads-up" note in the pom to keep these versions
consistent.
was:
{{com.azure:azure-identity}} in {{nifi-azure-nar}} has a dependency on
{{com.microsoft.azure:msal4j}} and requires a given version of it. It could
bring that version transitively but {{msal4j}} is also used directly by
{{nifi-azure-graph-authorizer}} and the {{msal4j}} version needs to be
configured in that pom. Unfortunately,
{{azure-sdk-bom}} does not control the {{msal4j}} version (I think because it
is a {{com.microsoft.azure}} artifact, not {{{}com.azure{}}}).
Configuring the {{azure-identity}} (via the BOM) and {{msal4j}} versions
separately caused dependency issues multiple times in the past (NIFI-9305,
NIFI-13181). I will add a "heads-up" note in the pom to keep these versions
consistent.
> Add note to msal4j dependency in Azure bundle's pom
> ---
>
> Key: NIFI-13287
> URL: https://issues.apache.org/jira/browse/NIFI-13287
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Minor
>
> {{com.azure:azure-identity}} in {{nifi-azure-nar}} has a dependency on
> {{com.microsoft.azure:msal4j}} and requires a given version of it. It could
> bring that version transitively but {{msal4j}} is also used directly by
> {{nifi-azure-graph-authorizer}} and the {{msal4j}} version needs to be
> configured in that pom. Unfortunately, {{azure-sdk-bom}} does not control the
> {{msal4j}} version (I think because it is a {{com.microsoft.azure}} artifact,
> not {{{}com.azure{}}}).
> Configuring the {{azure-identity}} (via the BOM) and {{msal4j}} versions
> separately caused dependency issues multiple times in the past (NIFI-9305,
> NIFI-13181). I will add a "heads-up" note in the pom to keep these versions
> consistent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-13268) ConsumeGCPPubSub Stopped Working with M3
[
https://issues.apache.org/jira/browse/NIFI-13268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848662#comment-17848662
]
Peter Turcsanyi commented on NIFI-13268:
Those google dependencies come transitively via google-auth-library-oauth2-http
so they are included in the services-api nar implicitly.
Regarding grpc-*:
* grpc-api and grpc-context: google-auth-library-oauth2-http pulls them too
* grpc-core and grpc-util: grpc-api has runtime dependency on these jars
(explained
[here|https://github.com/apache/nifi/pull/8491#issuecomment-1992542785]), it
looks a design issue in the grpc library because the api should not depend on
implementation jars (not even at runtime)
So I added grpc-core and grpc-util in the services-api nar and then also added
grpc-api and grpc-context explicitly in order to keep all grpc-* "together".
> ConsumeGCPPubSub Stopped Working with M3
>
>
> Key: NIFI-13268
> URL: https://issues.apache.org/jira/browse/NIFI-13268
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 2.0.0-M3
>Reporter: Emin
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> After updating from 1.26 to 2.0.0M3, getting the following for
> `ConsumeGCPPubSub`
>
> {{nifi-1 | 2024-05-18 23:00:02,157 INFO [NiFi Web Server-405]
> o.a.n.c.s.StandardProcessScheduler Running once
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,158 INFO [NiFi Web Server-405]
> o.a.n.controller.StandardProcessorNode Starting
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,170 ERROR [Timer-Driven Process Thread-1]
> o.a.n.p.gcp.pubsub.ConsumeGCPubSub
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | at
> java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586) nifi-1 | at
> java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:679)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1240)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
> nifi-1 | at
> java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309) nifi-1 |
> at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
> nifi-1 | at io.grpc.ServiceProviders.loadAll(ServiceProviders.java:67) nifi-1
> | at
> io.grpc.ManagedChannelRegistry.getDefaultRegistry(ManagedChannelRegistry.java:101)
> nifi-1 | at
> io.grpc.ManagedChannelProvider.provider(ManagedChannelProvider.java:43)
> nifi-1 | at
> io.grpc.ManagedChannelBuilder.forAddress(ManagedChannelBuilder.java:44)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:404)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.(ChannelPool.java:107)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.create(ChannelPool.java:85)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:243)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:237)
> nifi-1 | at
> com.google.api.gax.rpc.ClientContext.create(ClientContext.java:230) nifi-1 |
> at
> com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.create(GrpcSubscriberStub.java:287)
> nifi-1 | at
> org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub.getSubscriber(ConsumeGCPubSub.java:286)
> nifi-1 | at
> org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub.onScheduled(ConsumeGCPubSub.java:134)
> nifi-1 | at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> nifi-1 | at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133)
> nifi-1 | at
>
[jira] [Commented] (NIFI-13268) ConsumeGCPPubSub Stopped Working with M3
[
https://issues.apache.org/jira/browse/NIFI-13268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848635#comment-17848635
]
Peter Turcsanyi commented on NIFI-13268:
Yes, I excluded the dependencies where the processor jar is packaged into the
nar. The other option would have been to exclude the dependencies in the
processor's pom but in that case the exclude list needs to be added in all
google dependencies (storage, pubsub, etc.). Also, in this case the excluded
dependencies need to be re-added with provided scope and it would make the
tests run successfully again.
> ConsumeGCPPubSub Stopped Working with M3
>
>
> Key: NIFI-13268
> URL: https://issues.apache.org/jira/browse/NIFI-13268
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 2.0.0-M3
>Reporter: Emin
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> After updating from 1.26 to 2.0.0M3, getting the following for
> `ConsumeGCPPubSub`
>
> {{nifi-1 | 2024-05-18 23:00:02,157 INFO [NiFi Web Server-405]
> o.a.n.c.s.StandardProcessScheduler Running once
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,158 INFO [NiFi Web Server-405]
> o.a.n.controller.StandardProcessorNode Starting
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,170 ERROR [Timer-Driven Process Thread-1]
> o.a.n.p.gcp.pubsub.ConsumeGCPubSub
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | at
> java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586) nifi-1 | at
> java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:679)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1240)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
> nifi-1 | at
> java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309) nifi-1 |
> at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
> nifi-1 | at io.grpc.ServiceProviders.loadAll(ServiceProviders.java:67) nifi-1
> | at
> io.grpc.ManagedChannelRegistry.getDefaultRegistry(ManagedChannelRegistry.java:101)
> nifi-1 | at
> io.grpc.ManagedChannelProvider.provider(ManagedChannelProvider.java:43)
> nifi-1 | at
> io.grpc.ManagedChannelBuilder.forAddress(ManagedChannelBuilder.java:44)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:404)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.(ChannelPool.java:107)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.create(ChannelPool.java:85)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:243)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:237)
> nifi-1 | at
> com.google.api.gax.rpc.ClientContext.create(ClientContext.java:230) nifi-1 |
> at
> com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.create(GrpcSubscriberStub.java:287)
> nifi-1 | at
> org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub.getSubscriber(ConsumeGCPubSub.java:286)
> nifi-1 | at
> org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub.onScheduled(ConsumeGCPubSub.java:134)
> nifi-1 | at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> nifi-1 | at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:78)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:55)
> nifi-1 | at
> org.apache.nifi.controller.StandardProcessorNode.lambda$initiateStart$10(StandardProcessorNode.java:1668)
> nifi-1 | at
[jira] [Updated] (NIFI-13268) ConsumeGCPPubSub Stopped Working with M3
[
https://issues.apache.org/jira/browse/NIFI-13268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-13268:
---
Status: Patch Available (was: In Progress)
> ConsumeGCPPubSub Stopped Working with M3
>
>
> Key: NIFI-13268
> URL: https://issues.apache.org/jira/browse/NIFI-13268
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 2.0.0-M3
>Reporter: Emin
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> After updating from 1.26 to 2.0.0M3, getting the following for
> `ConsumeGCPPubSub`
>
> {{nifi-1 | 2024-05-18 23:00:02,157 INFO [NiFi Web Server-405]
> o.a.n.c.s.StandardProcessScheduler Running once
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,158 INFO [NiFi Web Server-405]
> o.a.n.controller.StandardProcessorNode Starting
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,170 ERROR [Timer-Driven Process Thread-1]
> o.a.n.p.gcp.pubsub.ConsumeGCPubSub
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | at
> java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586) nifi-1 | at
> java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:679)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1240)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
> nifi-1 | at
> java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309) nifi-1 |
> at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
> nifi-1 | at io.grpc.ServiceProviders.loadAll(ServiceProviders.java:67) nifi-1
> | at
> io.grpc.ManagedChannelRegistry.getDefaultRegistry(ManagedChannelRegistry.java:101)
> nifi-1 | at
> io.grpc.ManagedChannelProvider.provider(ManagedChannelProvider.java:43)
> nifi-1 | at
> io.grpc.ManagedChannelBuilder.forAddress(ManagedChannelBuilder.java:44)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:404)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.(ChannelPool.java:107)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.create(ChannelPool.java:85)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:243)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:237)
> nifi-1 | at
> com.google.api.gax.rpc.ClientContext.create(ClientContext.java:230) nifi-1 |
> at
> com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.create(GrpcSubscriberStub.java:287)
> nifi-1 | at
> org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub.getSubscriber(ConsumeGCPubSub.java:286)
> nifi-1 | at
> org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub.onScheduled(ConsumeGCPubSub.java:134)
> nifi-1 | at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> nifi-1 | at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:78)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:55)
> nifi-1 | at
> org.apache.nifi.controller.StandardProcessorNode.lambda$initiateStart$10(StandardProcessorNode.java:1668)
> nifi-1 | at org.apache.nifi.engine.FlowEngine$3.call(FlowEngine.java:123)
> nifi-1 | at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) nifi-1 |
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> nifi-1 | at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> nifi-1 | at
>
[jira] [Commented] (NIFI-13268) ConsumeGCPPubSub Stopped Working with M3
[
https://issues.apache.org/jira/browse/NIFI-13268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848598#comment-17848598
]
Peter Turcsanyi commented on NIFI-13268:
Restored the GRPC dependencies from NIFI-10707.
I did some dependency clean-up in NIFI-10707 in order to eliminate a
ClassNotFoundException warning and also to avoid duplicated jars in
nifi-gcp-services-api-nar and nifi-gcp-nar bundles. The
grpc-[api|context|core|util] jars are required in nifi-gcp-services-api-nar and
as they are present there, I excluded them (and also other duplications) from
nifi-gcp-nar (further details
[here|https://github.com/apache/nifi/pull/8491#issuecomment-1992542785]).
NIFI-12998 removed grpc-* dependencies from the services-api nar's pom but they
are expected by the processors nar at runtime.
The IT tests would not have caught the bug. The issue with the tests (either
using the public cloud service or testcontainers) is that NiFi's TestRunner
runs the processor using a single classloader that can see all the dependencies
including the ones with provided or test scopes and which may be excluded from
the final nar. Generally it cannot surface nar classloading issues. It should
be tested in a NiFi runtime with nar bundles. Not sure, we have tests like
this. The nifi-system-tests module works with a real-like NiFi environment but
it is rather for framework tests, as far as I know.
In any case, using [https://java.testcontainers.org/modules/gcloud/] would be
an improvement and a step forward. I suggest adding it in a separate Jira
because it is not related closely to the current bug.
> ConsumeGCPPubSub Stopped Working with M3
>
>
> Key: NIFI-13268
> URL: https://issues.apache.org/jira/browse/NIFI-13268
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 2.0.0-M3
>Reporter: Emin
>Assignee: Peter Turcsanyi
>Priority: Major
>
> After updating from 1.26 to 2.0.0M3, getting the following for
> `ConsumeGCPPubSub`
>
> {{nifi-1 | 2024-05-18 23:00:02,157 INFO [NiFi Web Server-405]
> o.a.n.c.s.StandardProcessScheduler Running once
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,158 INFO [NiFi Web Server-405]
> o.a.n.controller.StandardProcessorNode Starting
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,170 ERROR [Timer-Driven Process Thread-1]
> o.a.n.p.gcp.pubsub.ConsumeGCPubSub
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | at
> java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586) nifi-1 | at
> java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:679)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1240)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
> nifi-1 | at
> java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309) nifi-1 |
> at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
> nifi-1 | at io.grpc.ServiceProviders.loadAll(ServiceProviders.java:67) nifi-1
> | at
> io.grpc.ManagedChannelRegistry.getDefaultRegistry(ManagedChannelRegistry.java:101)
> nifi-1 | at
> io.grpc.ManagedChannelProvider.provider(ManagedChannelProvider.java:43)
> nifi-1 | at
> io.grpc.ManagedChannelBuilder.forAddress(ManagedChannelBuilder.java:44)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:404)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.(ChannelPool.java:107)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.create(ChannelPool.java:85)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:243)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:237)
> nifi-1 | at
> com.google.api.gax.rpc.ClientContext.create(ClientContext.java:230) nifi-1 |
> at
> com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.create(GrpcSubscriberStub.java:287)
> nifi-1 | at
>
[jira] [Assigned] (NIFI-13268) ConsumeGCPPubSub Stopped Working with M3
[
https://issues.apache.org/jira/browse/NIFI-13268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi reassigned NIFI-13268:
--
Assignee: Peter Turcsanyi
> ConsumeGCPPubSub Stopped Working with M3
>
>
> Key: NIFI-13268
> URL: https://issues.apache.org/jira/browse/NIFI-13268
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Emin
>Assignee: Peter Turcsanyi
>Priority: Major
>
> After updating from 1.26 to 2.0.0M3, getting the following for
> `ConsumeGCPPubSub`
>
> {{nifi-1 | 2024-05-18 23:00:02,157 INFO [NiFi Web Server-405]
> o.a.n.c.s.StandardProcessScheduler Running once
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,158 INFO [NiFi Web Server-405]
> o.a.n.controller.StandardProcessorNode Starting
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,170 ERROR [Timer-Driven Process Thread-1]
> o.a.n.p.gcp.pubsub.ConsumeGCPubSub
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | at
> java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586) nifi-1 | at
> java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:679)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1240)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
> nifi-1 | at
> java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309) nifi-1 |
> at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
> nifi-1 | at io.grpc.ServiceProviders.loadAll(ServiceProviders.java:67) nifi-1
> | at
> io.grpc.ManagedChannelRegistry.getDefaultRegistry(ManagedChannelRegistry.java:101)
> nifi-1 | at
> io.grpc.ManagedChannelProvider.provider(ManagedChannelProvider.java:43)
> nifi-1 | at
> io.grpc.ManagedChannelBuilder.forAddress(ManagedChannelBuilder.java:44)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:404)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.(ChannelPool.java:107)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.create(ChannelPool.java:85)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:243)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:237)
> nifi-1 | at
> com.google.api.gax.rpc.ClientContext.create(ClientContext.java:230) nifi-1 |
> at
> com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.create(GrpcSubscriberStub.java:287)
> nifi-1 | at
> org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub.getSubscriber(ConsumeGCPubSub.java:286)
> nifi-1 | at
> org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub.onScheduled(ConsumeGCPubSub.java:134)
> nifi-1 | at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> nifi-1 | at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:78)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:55)
> nifi-1 | at
> org.apache.nifi.controller.StandardProcessorNode.lambda$initiateStart$10(StandardProcessorNode.java:1668)
> nifi-1 | at org.apache.nifi.engine.FlowEngine$3.call(FlowEngine.java:123)
> nifi-1 | at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) nifi-1 |
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> nifi-1 | at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> nifi-1 | at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> nifi-1 | at
[jira] [Resolved] (NIFI-10964) FetchSmb Processor should have the option to delete file after being fetched
[ https://issues.apache.org/jira/browse/NIFI-10964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi resolved NIFI-10964. Resolution: Duplicate > FetchSmb Processor should have the option to delete file after being fetched > > > Key: NIFI-10964 > URL: https://issues.apache.org/jira/browse/NIFI-10964 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Sam Williams >Priority: Minor > > It would be good to have the option to delete a file once the FetchSmb > processor has collected the file. This could either be a whole new processor, > or just a property on the existing FetSmb processor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13072) MonitorActivity processor generates a false positive on a node restart
[ https://issues.apache.org/jira/browse/NIFI-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-13072: --- Fix Version/s: 2.0.0-M3 1.27.0 Resolution: Fixed Status: Resolved (was: Patch Available) > MonitorActivity processor generates a false positive on a node restart > -- > > Key: NIFI-13072 > URL: https://issues.apache.org/jira/browse/NIFI-13072 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 2.0.0-M1, 1.18.0, 1.19.0, 1.20.0, 1.19.1, 1.21.0, > 1.22.0, 1.23.0, 1.24.0, 1.23.1, 1.23.2, 1.25.0, 2.0.0-M2 >Reporter: Rajmund Takacs >Assignee: Rajmund Takacs >Priority: Major > Fix For: 2.0.0-M3, 1.27.0 > > Attachments: MonitorActivity_Flow_Example_NiFi_1.18_OSS.json > > Time Spent: 1h > Remaining Estimate: 0h > > If the monitoring scope is configured to "cluster", then there are multiple > restart cases when markers are incorrectly generated: > h4. CASE 1 > * The flow is already inactive, and there is no traffic. The processor is > still running and scheduled on all nodes. > * We restart any of the NiFi nodes in the cluster. > * *All nodes, except the restarted one, generates activation marker.* (Reason > for this is, that the common timestamp is updated by the restarted node > during initialization.) > h4. CASE 2: > * The flow is already inactive, and there is no traffic. The processor is > still running and scheduled on all nodes. Reporting is done by all nodes. > * We temporarily stop any of the nodes. > * We start traffic on any other node, so that an activation marker is > generated. > * We start the previously stopped node again. > * *This node does not send activation marker.* (Reason for this is that the > initial state is always active.) > Expected behavior for stop/start cases would be: > * Single node restart alone should not affect the flow activity state. > * If a marker has already been sent before stopping the processor, it should > not be repeated on next start, unless asked for, or there is a change in > activity state. > * If the flow was active during a weekend shutdown, it should not begin with > an inactivity marker right after startup on monday. Although it should send > inactivity marker after the threshold has expired since startup. > * If the flow was inactive during a weekend shutdown, it should send > activation marker after started up again with traffic. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13181) Azure Blob and ADLS processors throw NoSuchMethodError when Service Principal is used
[
https://issues.apache.org/jira/browse/NIFI-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844753#comment-17844753
]
Peter Turcsanyi commented on NIFI-13181:
[~joewitt] No worries. I'm afraid there are no integration tests for Service
Principal authentication because it would require a specific setup on Azure.
> Azure Blob and ADLS processors throw NoSuchMethodError when Service Principal
> is used
> -
>
> Key: NIFI-13181
> URL: https://issues.apache.org/jira/browse/NIFI-13181
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.26.0
>Reporter: Zoltán Kornél Török
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I spent some time testing nifi 1.26 and found this error, when tried to use
> blobstorage related processors, like ListAzureBlobStorage_v12 .
> {code:java}
> 2024-05-08 09:09:06,416 WARN reactor.core.Exceptions: throwIfFatal detected a
> jvm fatal exception, which is thrown and logged below:
> java.lang.NoSuchMethodError:
> com.microsoft.aad.msal4j.ConfidentialClientApplication$Builder.logPii(Z)Lcom/microsoft/aad/msal4j/AbstractApplicationBase$Builder;
> at
> com.azure.identity.implementation.IdentityClientBase.getConfidentialClient(IdentityClientBase.java:233)
> at
> com.azure.identity.implementation.IdentityClient.lambda$getConfidentialClientApplication$4(IdentityClient.java:130)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:45)
> at
> reactor.core.publisher.MonoCacheTime.subscribeOrReturn(MonoCacheTime.java:143)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
> at reactor.core.publisher.MonoUsing.subscribe(MonoUsing.java:102)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at
> reactor.core.publisher.MonoFromFluxOperator.subscribe(MonoFromFluxOperator.java:81)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:263)
> at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:203)
> at
> reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:203)
> at
> reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
> at reactor.core.publisher.Flux.subscribe(Flux.java:8628)
> at reactor.core.publisher.Flux.blockLast(Flux.java:2760)
> at
> com.azure.core.util.paging.ContinuablePagedByIteratorBase.requestPage(ContinuablePagedByIteratorBase.java:102)
> at
> com.azure.core.util.paging.ContinuablePagedByItemIterable$ContinuablePagedByItemIterator.(ContinuablePagedByItemIterable.java:75)
> at
> com.azure.core.util.paging.ContinuablePagedByItemIterable.iterator(ContinuablePagedByItemIterable.java:55)
> at
> com.azure.core.util.paging.ContinuablePagedIterable.iterator(ContinuablePagedIterable.java:141)
> at
> org.apache.nifi.processors.azure.storage.ListAzureBlobStorage_v12.performListing(ListAzureBlobStorage_v12.java:230)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.lambda$listByTrackingEntities$11(AbstractListProcessor.java:1126)
> at
> org.apache.nifi.processor.util.list.ListedEntityTracker.trackEntities(ListedEntityTracker.java:272)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingEntities(AbstractListProcessor.java:1124)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:529)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1361)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:247)
> at
>
[jira] [Commented] (NIFI-13181) Azure Blob and ADLS processors throw NoSuchMethodError when Service Principal is used
[
https://issues.apache.org/jira/browse/NIFI-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844673#comment-17844673
]
Peter Turcsanyi commented on NIFI-13181:
Root case: {{azure-identity}} was updated to 1.12.0 via {{azure-sdk-bom}}
update and it requires {{{}msal4j:1.15.0{}}}. {{msal4j}} is also used in
{{nifi-azure-graph-authorizer}} as a direct dependency. Due to this,
{{azure-identity}} cannot bring the dependency version it would need but we
have to configure it manually. I will add a "heads-up" comment for
{{}} in pom.xml and also specify the version in
{{{}{}}}. Will do it in a separate jira for both 2.x and
1.x. On 2.x, the version mismatch has already been fixed in NIFI-13144.
> Azure Blob and ADLS processors throw NoSuchMethodError when Service Principal
> is used
> -
>
> Key: NIFI-13181
> URL: https://issues.apache.org/jira/browse/NIFI-13181
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.26.0
>Reporter: Zoltán Kornél Török
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I spent some time testing nifi 1.26 and found this error, when tried to use
> blobstorage related processors, like ListAzureBlobStorage_v12 .
> {code:java}
> 2024-05-08 09:09:06,416 WARN reactor.core.Exceptions: throwIfFatal detected a
> jvm fatal exception, which is thrown and logged below:
> java.lang.NoSuchMethodError:
> com.microsoft.aad.msal4j.ConfidentialClientApplication$Builder.logPii(Z)Lcom/microsoft/aad/msal4j/AbstractApplicationBase$Builder;
> at
> com.azure.identity.implementation.IdentityClientBase.getConfidentialClient(IdentityClientBase.java:233)
> at
> com.azure.identity.implementation.IdentityClient.lambda$getConfidentialClientApplication$4(IdentityClient.java:130)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:45)
> at
> reactor.core.publisher.MonoCacheTime.subscribeOrReturn(MonoCacheTime.java:143)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
> at reactor.core.publisher.MonoUsing.subscribe(MonoUsing.java:102)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at
> reactor.core.publisher.MonoFromFluxOperator.subscribe(MonoFromFluxOperator.java:81)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:263)
> at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:203)
> at
> reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:203)
> at
> reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
> at reactor.core.publisher.Flux.subscribe(Flux.java:8628)
> at reactor.core.publisher.Flux.blockLast(Flux.java:2760)
> at
> com.azure.core.util.paging.ContinuablePagedByIteratorBase.requestPage(ContinuablePagedByIteratorBase.java:102)
> at
> com.azure.core.util.paging.ContinuablePagedByItemIterable$ContinuablePagedByItemIterator.(ContinuablePagedByItemIterable.java:75)
> at
> com.azure.core.util.paging.ContinuablePagedByItemIterable.iterator(ContinuablePagedByItemIterable.java:55)
> at
> com.azure.core.util.paging.ContinuablePagedIterable.iterator(ContinuablePagedIterable.java:141)
> at
> org.apache.nifi.processors.azure.storage.ListAzureBlobStorage_v12.performListing(ListAzureBlobStorage_v12.java:230)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.lambda$listByTrackingEntities$11(AbstractListProcessor.java:1126)
> at
> org.apache.nifi.processor.util.list.ListedEntityTracker.trackEntities(ListedEntityTracker.java:272)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingEntities(AbstractListProcessor.java:1124)
> at
>
[jira] [Updated] (NIFI-13181) Azure Blob and ADLS processors throw NoSuchMethodExceptions when Service Principal is used
[
https://issues.apache.org/jira/browse/NIFI-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-13181:
---
Summary: Azure Blob and ADLS processors throw NoSuchMethodExceptions when
Service Principal is used (was: Azure BlobStorage processors throws
NoSuchMethodExceptions)
> Azure Blob and ADLS processors throw NoSuchMethodExceptions when Service
> Principal is used
> --
>
> Key: NIFI-13181
> URL: https://issues.apache.org/jira/browse/NIFI-13181
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.26.0
>Reporter: Zoltán Kornél Török
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I spent some time testing nifi 1.26 and found this error, when tried to use
> blobstorage related processors, like ListAzureBlobStorage_v12 .
> {code:java}
> 2024-05-08 09:09:06,416 WARN reactor.core.Exceptions: throwIfFatal detected a
> jvm fatal exception, which is thrown and logged below:
> java.lang.NoSuchMethodError:
> com.microsoft.aad.msal4j.ConfidentialClientApplication$Builder.logPii(Z)Lcom/microsoft/aad/msal4j/AbstractApplicationBase$Builder;
> at
> com.azure.identity.implementation.IdentityClientBase.getConfidentialClient(IdentityClientBase.java:233)
> at
> com.azure.identity.implementation.IdentityClient.lambda$getConfidentialClientApplication$4(IdentityClient.java:130)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:45)
> at
> reactor.core.publisher.MonoCacheTime.subscribeOrReturn(MonoCacheTime.java:143)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
> at reactor.core.publisher.MonoUsing.subscribe(MonoUsing.java:102)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at
> reactor.core.publisher.MonoFromFluxOperator.subscribe(MonoFromFluxOperator.java:81)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:263)
> at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:203)
> at
> reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:203)
> at
> reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
> at reactor.core.publisher.Flux.subscribe(Flux.java:8628)
> at reactor.core.publisher.Flux.blockLast(Flux.java:2760)
> at
> com.azure.core.util.paging.ContinuablePagedByIteratorBase.requestPage(ContinuablePagedByIteratorBase.java:102)
> at
> com.azure.core.util.paging.ContinuablePagedByItemIterable$ContinuablePagedByItemIterator.(ContinuablePagedByItemIterable.java:75)
> at
> com.azure.core.util.paging.ContinuablePagedByItemIterable.iterator(ContinuablePagedByItemIterable.java:55)
> at
> com.azure.core.util.paging.ContinuablePagedIterable.iterator(ContinuablePagedIterable.java:141)
> at
> org.apache.nifi.processors.azure.storage.ListAzureBlobStorage_v12.performListing(ListAzureBlobStorage_v12.java:230)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.lambda$listByTrackingEntities$11(AbstractListProcessor.java:1126)
> at
> org.apache.nifi.processor.util.list.ListedEntityTracker.trackEntities(ListedEntityTracker.java:272)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingEntities(AbstractListProcessor.java:1124)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:529)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1361)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:247)
> at
>
[jira] [Updated] (NIFI-13181) Azure Blob and ADLS processors throw NoSuchMethodError when Service Principal is used
[
https://issues.apache.org/jira/browse/NIFI-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-13181:
---
Summary: Azure Blob and ADLS processors throw NoSuchMethodError when
Service Principal is used (was: Azure Blob and ADLS processors throw
NoSuchMethodExceptions when Service Principal is used)
> Azure Blob and ADLS processors throw NoSuchMethodError when Service Principal
> is used
> -
>
> Key: NIFI-13181
> URL: https://issues.apache.org/jira/browse/NIFI-13181
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.26.0
>Reporter: Zoltán Kornél Török
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I spent some time testing nifi 1.26 and found this error, when tried to use
> blobstorage related processors, like ListAzureBlobStorage_v12 .
> {code:java}
> 2024-05-08 09:09:06,416 WARN reactor.core.Exceptions: throwIfFatal detected a
> jvm fatal exception, which is thrown and logged below:
> java.lang.NoSuchMethodError:
> com.microsoft.aad.msal4j.ConfidentialClientApplication$Builder.logPii(Z)Lcom/microsoft/aad/msal4j/AbstractApplicationBase$Builder;
> at
> com.azure.identity.implementation.IdentityClientBase.getConfidentialClient(IdentityClientBase.java:233)
> at
> com.azure.identity.implementation.IdentityClient.lambda$getConfidentialClientApplication$4(IdentityClient.java:130)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:45)
> at
> reactor.core.publisher.MonoCacheTime.subscribeOrReturn(MonoCacheTime.java:143)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
> at reactor.core.publisher.MonoUsing.subscribe(MonoUsing.java:102)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at
> reactor.core.publisher.MonoFromFluxOperator.subscribe(MonoFromFluxOperator.java:81)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:263)
> at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:203)
> at
> reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:203)
> at
> reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
> at reactor.core.publisher.Flux.subscribe(Flux.java:8628)
> at reactor.core.publisher.Flux.blockLast(Flux.java:2760)
> at
> com.azure.core.util.paging.ContinuablePagedByIteratorBase.requestPage(ContinuablePagedByIteratorBase.java:102)
> at
> com.azure.core.util.paging.ContinuablePagedByItemIterable$ContinuablePagedByItemIterator.(ContinuablePagedByItemIterable.java:75)
> at
> com.azure.core.util.paging.ContinuablePagedByItemIterable.iterator(ContinuablePagedByItemIterable.java:55)
> at
> com.azure.core.util.paging.ContinuablePagedIterable.iterator(ContinuablePagedIterable.java:141)
> at
> org.apache.nifi.processors.azure.storage.ListAzureBlobStorage_v12.performListing(ListAzureBlobStorage_v12.java:230)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.lambda$listByTrackingEntities$11(AbstractListProcessor.java:1126)
> at
> org.apache.nifi.processor.util.list.ListedEntityTracker.trackEntities(ListedEntityTracker.java:272)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingEntities(AbstractListProcessor.java:1124)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:529)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1361)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:247)
> at
>
[jira] [Updated] (NIFI-13181) Azure BlobStorage processors throws NoSuchMethodExceptions
[
https://issues.apache.org/jira/browse/NIFI-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-13181:
---
Status: Patch Available (was: In Progress)
> Azure BlobStorage processors throws NoSuchMethodExceptions
> --
>
> Key: NIFI-13181
> URL: https://issues.apache.org/jira/browse/NIFI-13181
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.26.0
>Reporter: Zoltán Kornél Török
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I spent some time testing nifi 1.26 and found this error, when tried to use
> blobstorage related processors, like ListAzureBlobStorage_v12 .
> {code:java}
> 2024-05-08 09:09:06,416 WARN reactor.core.Exceptions: throwIfFatal detected a
> jvm fatal exception, which is thrown and logged below:
> java.lang.NoSuchMethodError:
> com.microsoft.aad.msal4j.ConfidentialClientApplication$Builder.logPii(Z)Lcom/microsoft/aad/msal4j/AbstractApplicationBase$Builder;
> at
> com.azure.identity.implementation.IdentityClientBase.getConfidentialClient(IdentityClientBase.java:233)
> at
> com.azure.identity.implementation.IdentityClient.lambda$getConfidentialClientApplication$4(IdentityClient.java:130)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:45)
> at
> reactor.core.publisher.MonoCacheTime.subscribeOrReturn(MonoCacheTime.java:143)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
> at reactor.core.publisher.MonoUsing.subscribe(MonoUsing.java:102)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at
> reactor.core.publisher.MonoFromFluxOperator.subscribe(MonoFromFluxOperator.java:81)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:263)
> at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:203)
> at
> reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:203)
> at
> reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
> at reactor.core.publisher.Flux.subscribe(Flux.java:8628)
> at reactor.core.publisher.Flux.blockLast(Flux.java:2760)
> at
> com.azure.core.util.paging.ContinuablePagedByIteratorBase.requestPage(ContinuablePagedByIteratorBase.java:102)
> at
> com.azure.core.util.paging.ContinuablePagedByItemIterable$ContinuablePagedByItemIterator.(ContinuablePagedByItemIterable.java:75)
> at
> com.azure.core.util.paging.ContinuablePagedByItemIterable.iterator(ContinuablePagedByItemIterable.java:55)
> at
> com.azure.core.util.paging.ContinuablePagedIterable.iterator(ContinuablePagedIterable.java:141)
> at
> org.apache.nifi.processors.azure.storage.ListAzureBlobStorage_v12.performListing(ListAzureBlobStorage_v12.java:230)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.lambda$listByTrackingEntities$11(AbstractListProcessor.java:1126)
> at
> org.apache.nifi.processor.util.list.ListedEntityTracker.trackEntities(ListedEntityTracker.java:272)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingEntities(AbstractListProcessor.java:1124)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:529)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1361)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:247)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
>
[jira] [Assigned] (NIFI-13181) Azure BlobStorage processors throws NoSuchMethodExceptions
[
https://issues.apache.org/jira/browse/NIFI-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi reassigned NIFI-13181:
--
Assignee: Peter Turcsanyi
> Azure BlobStorage processors throws NoSuchMethodExceptions
> --
>
> Key: NIFI-13181
> URL: https://issues.apache.org/jira/browse/NIFI-13181
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.26.0
>Reporter: Zoltán Kornél Török
>Assignee: Peter Turcsanyi
>Priority: Major
>
> I spent some time testing nifi 1.26 and found this error, when tried to use
> blobstorage related processors, like ListAzureBlobStorage_v12 .
> {code:java}
> 2024-05-08 09:09:06,416 WARN reactor.core.Exceptions: throwIfFatal detected a
> jvm fatal exception, which is thrown and logged below:
> java.lang.NoSuchMethodError:
> com.microsoft.aad.msal4j.ConfidentialClientApplication$Builder.logPii(Z)Lcom/microsoft/aad/msal4j/AbstractApplicationBase$Builder;
> at
> com.azure.identity.implementation.IdentityClientBase.getConfidentialClient(IdentityClientBase.java:233)
> at
> com.azure.identity.implementation.IdentityClient.lambda$getConfidentialClientApplication$4(IdentityClient.java:130)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:45)
> at
> reactor.core.publisher.MonoCacheTime.subscribeOrReturn(MonoCacheTime.java:143)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
> at reactor.core.publisher.MonoUsing.subscribe(MonoUsing.java:102)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at
> reactor.core.publisher.MonoFromFluxOperator.subscribe(MonoFromFluxOperator.java:81)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:263)
> at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:203)
> at
> reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:203)
> at
> reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
> at reactor.core.publisher.Flux.subscribe(Flux.java:8628)
> at reactor.core.publisher.Flux.blockLast(Flux.java:2760)
> at
> com.azure.core.util.paging.ContinuablePagedByIteratorBase.requestPage(ContinuablePagedByIteratorBase.java:102)
> at
> com.azure.core.util.paging.ContinuablePagedByItemIterable$ContinuablePagedByItemIterator.(ContinuablePagedByItemIterable.java:75)
> at
> com.azure.core.util.paging.ContinuablePagedByItemIterable.iterator(ContinuablePagedByItemIterable.java:55)
> at
> com.azure.core.util.paging.ContinuablePagedIterable.iterator(ContinuablePagedIterable.java:141)
> at
> org.apache.nifi.processors.azure.storage.ListAzureBlobStorage_v12.performListing(ListAzureBlobStorage_v12.java:230)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.lambda$listByTrackingEntities$11(AbstractListProcessor.java:1126)
> at
> org.apache.nifi.processor.util.list.ListedEntityTracker.trackEntities(ListedEntityTracker.java:272)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingEntities(AbstractListProcessor.java:1124)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:529)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1361)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:247)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at
>
[jira] [Updated] (NIFI-12890) Refactor HadoopDBCPConnectionPool to extend AbstractDBCPConnectionPool
[ https://issues.apache.org/jira/browse/NIFI-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12890: --- Fix Version/s: 2.0.0-M3 1.26.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Refactor HadoopDBCPConnectionPool to extend AbstractDBCPConnectionPool > -- > > Key: NIFI-12890 > URL: https://issues.apache.org/jira/browse/NIFI-12890 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Lehel Boér >Assignee: Lehel Boér >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > The AbstractDBCPConnectionPool was designed to avoid code duplication within > the DBCP services. It encompasses shared variables and is constructed in a > way that it contains numerous methods which can be overridden if we wish to > subclass from it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12231) Add completion strategy to FetchSmb
[ https://issues.apache.org/jira/browse/NIFI-12231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12231: --- Component/s: Extensions (was: Core Framework) > Add completion strategy to FetchSmb > --- > > Key: NIFI-12231 > URL: https://issues.apache.org/jira/browse/NIFI-12231 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.23.2 >Reporter: Jens M Kofoed >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The newly added ListSmb and FetchSmb has been very much appreciated. But > sadly the FetchSmb does not have the same options like other Fetch processors. > FetchFTP and FetchSFTP process has a Completion Strategy where you can choose > to Move or Delete the file after it has been fetched. > It would be very helpfull if FetchSmb gets the same possibilities -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12231) Add completion strategy to FetchSmb
[ https://issues.apache.org/jira/browse/NIFI-12231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12231: --- Status: Patch Available (was: In Progress) > Add completion strategy to FetchSmb > --- > > Key: NIFI-12231 > URL: https://issues.apache.org/jira/browse/NIFI-12231 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.23.2 >Reporter: Jens M Kofoed >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The newly added ListSmb and FetchSmb has been very much appreciated. But > sadly the FetchSmb does not have the same options like other Fetch processors. > FetchFTP and FetchSFTP process has a Completion Strategy where you can choose > to Move or Delete the file after it has been fetched. > It would be very helpfull if FetchSmb gets the same possibilities -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13014) Remove unused Calcite Avatica Versions from Code Coverage Module
[ https://issues.apache.org/jira/browse/NIFI-13014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-13014: --- Fix Version/s: 2.0.0-M3 Resolution: Fixed Status: Resolved (was: Patch Available) > Remove unused Calcite Avatica Versions from Code Coverage Module > > > Key: NIFI-13014 > URL: https://issues.apache.org/jira/browse/NIFI-13014 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Fix For: 2.0.0-M3 > > Time Spent: 20m > Remaining Estimate: 0h > > The nifi-code-coverage module includes several dependency version > declarations to ensure version resolution aligns with runtime versions. > Following the removal of Hive 3 components, the version references for > Calcite Avatica can be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13012) Upgraded Tika to 2.9.2
[ https://issues.apache.org/jira/browse/NIFI-13012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-13012: --- Fix Version/s: 2.0.0-M3 1.26.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgraded Tika to 2.9.2 > -- > > Key: NIFI-13012 > URL: https://issues.apache.org/jira/browse/NIFI-13012 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Apache Tika dependencies should be upgraded to > [2.9.2|https://downloads.apache.org/tika/2.9.2/CHANGES-2.9.2.txt] to > incorporate several transitive version upgrades. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13013) Upgrade ActiveMQ to 6.1.1
[ https://issues.apache.org/jira/browse/NIFI-13013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-13013: --- Fix Version/s: 2.0.0-M3 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade ActiveMQ to 6.1.1 > - > > Key: NIFI-13013 > URL: https://issues.apache.org/jira/browse/NIFI-13013 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Fix For: 2.0.0-M3 > > Time Spent: 20m > Remaining Estimate: 0h > > Apache ActiveMQ dependencies for testing in JMS Processors should be upgraded > to > [6.1.1|https://activemq.apache.org/components/classic/download/classic-06-01-01] > to incorporate several transitive dependency upgrades. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12984) Bump Snowflake Ingest SDK to 2.1.0
[ https://issues.apache.org/jira/browse/NIFI-12984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12984: --- Fix Version/s: 2.0.0-M3 1.26.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Bump Snowflake Ingest SDK to 2.1.0 > -- > > Key: NIFI-12984 > URL: https://issues.apache.org/jira/browse/NIFI-12984 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Labels: backport-needed > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Bump Snowflake Ingest SDK to 2.1.0. We keep the JDBC version as-is given the > release notes. See: > [https://docs.snowflake.com/en/release-notes/clients-drivers/ingest-java-sdk-2024] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-12231) Add completion strategy to FetchSmb
[ https://issues.apache.org/jira/browse/NIFI-12231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi reassigned NIFI-12231: -- Assignee: Peter Turcsanyi > Add completion strategy to FetchSmb > --- > > Key: NIFI-12231 > URL: https://issues.apache.org/jira/browse/NIFI-12231 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.23.2 >Reporter: Jens M Kofoed >Assignee: Peter Turcsanyi >Priority: Major > > The newly added ListSmb and FetchSmb has been very much appreciated. But > sadly the FetchSmb does not have the same options like other Fetch processors. > FetchFTP and FetchSFTP process has a Completion Strategy where you can choose > to Move or Delete the file after it has been fetched. > It would be very helpfull if FetchSmb gets the same possibilities -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12895) GetSmbFile and PutSmbFile has forced 5 sec timeout
[
https://issues.apache.org/jira/browse/NIFI-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830121#comment-17830121
]
Peter Turcsanyi commented on NIFI-12895:
[~andersns] Thanks for your suggestion! Yes, it was that easy to add the
property.
> GetSmbFile and PutSmbFile has forced 5 sec timeout
> --
>
> Key: NIFI-12895
> URL: https://issues.apache.org/jira/browse/NIFI-12895
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.25.0, 2.0.0-M2
>Reporter: Anders
>Assignee: Peter Turcsanyi
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems like timeout was introduced with the SmbjClientProviderService, and
> SmbUtils.java defines the withTimeout setting from the underlying smbj
> library.
> In SmbProperties.java, the default timeout value is set to 5 sec and you can
> configure it in the SmbjClientProviderService.
> However, the older GetSmbFile and PutSmbFile processors uses SmbUtils.java as
> well, but the timeout configuration item is not exposed in their config ui.
> The timeout setting should be added to the UI for GetSmbFile and PutSmbFile.
> I think it's enough to add the following lines to PutSmbFile.java and
> GetSmbFile.java:
> {code}
> ...
> import static org.apache.nifi.smb.common.SmbProperties.TIMEOUT;
> ...
> descriptors.add(TIMEOUT);
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12895) GetSmbFile and PutSmbFile has forced 5 sec timeout
[
https://issues.apache.org/jira/browse/NIFI-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12895:
---
Status: Patch Available (was: Open)
> GetSmbFile and PutSmbFile has forced 5 sec timeout
> --
>
> Key: NIFI-12895
> URL: https://issues.apache.org/jira/browse/NIFI-12895
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 2.0.0-M2, 1.25.0
>Reporter: Anders
>Assignee: Peter Turcsanyi
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems like timeout was introduced with the SmbjClientProviderService, and
> SmbUtils.java defines the withTimeout setting from the underlying smbj
> library.
> In SmbProperties.java, the default timeout value is set to 5 sec and you can
> configure it in the SmbjClientProviderService.
> However, the older GetSmbFile and PutSmbFile processors uses SmbUtils.java as
> well, but the timeout configuration item is not exposed in their config ui.
> The timeout setting should be added to the UI for GetSmbFile and PutSmbFile.
> I think it's enough to add the following lines to PutSmbFile.java and
> GetSmbFile.java:
> {code}
> ...
> import static org.apache.nifi.smb.common.SmbProperties.TIMEOUT;
> ...
> descriptors.add(TIMEOUT);
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12895) GetSmbFile and PutSmbFile has forced 5 sec timeout
[
https://issues.apache.org/jira/browse/NIFI-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12895:
---
Affects Version/s: 2.0.0-M2
> GetSmbFile and PutSmbFile has forced 5 sec timeout
> --
>
> Key: NIFI-12895
> URL: https://issues.apache.org/jira/browse/NIFI-12895
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.25.0, 2.0.0-M2
>Reporter: Anders
>Assignee: Peter Turcsanyi
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems like timeout was introduced with the SmbjClientProviderService, and
> SmbUtils.java defines the withTimeout setting from the underlying smbj
> library.
> In SmbProperties.java, the default timeout value is set to 5 sec and you can
> configure it in the SmbjClientProviderService.
> However, the older GetSmbFile and PutSmbFile processors uses SmbUtils.java as
> well, but the timeout configuration item is not exposed in their config ui.
> The timeout setting should be added to the UI for GetSmbFile and PutSmbFile.
> I think it's enough to add the following lines to PutSmbFile.java and
> GetSmbFile.java:
> {code}
> ...
> import static org.apache.nifi.smb.common.SmbProperties.TIMEOUT;
> ...
> descriptors.add(TIMEOUT);
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12895) GetSmbFile and PutSmbFile has forced 5 sec timeout
[
https://issues.apache.org/jira/browse/NIFI-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12895:
---
Priority: Minor (was: Major)
> GetSmbFile and PutSmbFile has forced 5 sec timeout
> --
>
> Key: NIFI-12895
> URL: https://issues.apache.org/jira/browse/NIFI-12895
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.25.0
>Reporter: Anders
>Assignee: Peter Turcsanyi
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems like timeout was introduced with the SmbjClientProviderService, and
> SmbUtils.java defines the withTimeout setting from the underlying smbj
> library.
> In SmbProperties.java, the default timeout value is set to 5 sec and you can
> configure it in the SmbjClientProviderService.
> However, the older GetSmbFile and PutSmbFile processors uses SmbUtils.java as
> well, but the timeout configuration item is not exposed in their config ui.
> The timeout setting should be added to the UI for GetSmbFile and PutSmbFile.
> I think it's enough to add the following lines to PutSmbFile.java and
> GetSmbFile.java:
> {code}
> ...
> import static org.apache.nifi.smb.common.SmbProperties.TIMEOUT;
> ...
> descriptors.add(TIMEOUT);
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12895) GetSmbFile and PutSmbFile has forced 5 sec timeout
[
https://issues.apache.org/jira/browse/NIFI-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12895:
---
Issue Type: Improvement (was: Bug)
> GetSmbFile and PutSmbFile has forced 5 sec timeout
> --
>
> Key: NIFI-12895
> URL: https://issues.apache.org/jira/browse/NIFI-12895
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.25.0
>Reporter: Anders
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems like timeout was introduced with the SmbjClientProviderService, and
> SmbUtils.java defines the withTimeout setting from the underlying smbj
> library.
> In SmbProperties.java, the default timeout value is set to 5 sec and you can
> configure it in the SmbjClientProviderService.
> However, the older GetSmbFile and PutSmbFile processors uses SmbUtils.java as
> well, but the timeout configuration item is not exposed in their config ui.
> The timeout setting should be added to the UI for GetSmbFile and PutSmbFile.
> I think it's enough to add the following lines to PutSmbFile.java and
> GetSmbFile.java:
> {code}
> ...
> import static org.apache.nifi.smb.common.SmbProperties.TIMEOUT;
> ...
> descriptors.add(TIMEOUT);
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12895) GetSmbFile and PutSmbFile has forced 5 sec timeout
[
https://issues.apache.org/jira/browse/NIFI-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12895:
---
Component/s: Extensions
> GetSmbFile and PutSmbFile has forced 5 sec timeout
> --
>
> Key: NIFI-12895
> URL: https://issues.apache.org/jira/browse/NIFI-12895
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.25.0
>Reporter: Anders
>Assignee: Peter Turcsanyi
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems like timeout was introduced with the SmbjClientProviderService, and
> SmbUtils.java defines the withTimeout setting from the underlying smbj
> library.
> In SmbProperties.java, the default timeout value is set to 5 sec and you can
> configure it in the SmbjClientProviderService.
> However, the older GetSmbFile and PutSmbFile processors uses SmbUtils.java as
> well, but the timeout configuration item is not exposed in their config ui.
> The timeout setting should be added to the UI for GetSmbFile and PutSmbFile.
> I think it's enough to add the following lines to PutSmbFile.java and
> GetSmbFile.java:
> {code}
> ...
> import static org.apache.nifi.smb.common.SmbProperties.TIMEOUT;
> ...
> descriptors.add(TIMEOUT);
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-12895) GetSmbFile and PutSmbFile has forced 5 sec timeout
[
https://issues.apache.org/jira/browse/NIFI-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi reassigned NIFI-12895:
--
Assignee: Peter Turcsanyi
> GetSmbFile and PutSmbFile has forced 5 sec timeout
> --
>
> Key: NIFI-12895
> URL: https://issues.apache.org/jira/browse/NIFI-12895
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.25.0
>Reporter: Anders
>Assignee: Peter Turcsanyi
>Priority: Major
>
> It seems like timeout was introduced with the SmbjClientProviderService, and
> SmbUtils.java defines the withTimeout setting from the underlying smbj
> library.
> In SmbProperties.java, the default timeout value is set to 5 sec and you can
> configure it in the SmbjClientProviderService.
> However, the older GetSmbFile and PutSmbFile processors uses SmbUtils.java as
> well, but the timeout configuration item is not exposed in their config ui.
> The timeout setting should be added to the UI for GetSmbFile and PutSmbFile.
> I think it's enough to add the following lines to PutSmbFile.java and
> GetSmbFile.java:
> {code}
> ...
> import static org.apache.nifi.smb.common.SmbProperties.TIMEOUT;
> ...
> descriptors.add(TIMEOUT);
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12936) ListGCSBucket should reset its tracking state after configuration change
[ https://issues.apache.org/jira/browse/NIFI-12936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12936: --- Status: Patch Available (was: Open) > ListGCSBucket should reset its tracking state after configuration change > > > Key: NIFI-12936 > URL: https://issues.apache.org/jira/browse/NIFI-12936 > Project: Apache NiFi > Issue Type: Bug >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Same as NIFI-12732 but for ListGCSBucket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12936) ListGCSBucket should reset its tracking state after configuration change
Peter Turcsanyi created NIFI-12936: -- Summary: ListGCSBucket should reset its tracking state after configuration change Key: NIFI-12936 URL: https://issues.apache.org/jira/browse/NIFI-12936 Project: Apache NiFi Issue Type: Bug Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi Same as NIFI-12732 but for ListGCSBucket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12933) Rearrange properties on GCP processors
[
https://issues.apache.org/jira/browse/NIFI-12933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12933:
---
Status: Patch Available (was: Open)
> Rearrange properties on GCP processors
> --
>
> Key: NIFI-12933
> URL: https://issues.apache.org/jira/browse/NIFI-12933
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Parent classes of the GCP processors define some common properties and they
> are added to the processors via
> {{{}descriptors.addAll(super.getSupportedPropertyDescriptors()){}}}.
> This practice leads to weird property ordering in most cases and it is not
> worth using it. E.g. "Number of retries" precedes "Bucket" and "Key"
> properties in PutGCSObject.
> Get rid of {{super.getSupportedPropertyDescriptors()}} and reorder the
> properties in a more reasonable way.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12933) Rearrange properties on GCP processors
Peter Turcsanyi created NIFI-12933:
--
Summary: Rearrange properties on GCP processors
Key: NIFI-12933
URL: https://issues.apache.org/jira/browse/NIFI-12933
Project: Apache NiFi
Issue Type: Improvement
Reporter: Peter Turcsanyi
Assignee: Peter Turcsanyi
Parent classes of the GCP processors define some common properties and they are
added to the processors via
{{{}descriptors.addAll(super.getSupportedPropertyDescriptors()){}}}.
This practice leads to weird property ordering in most cases and it is not
worth using it. E.g. "Number of retries" precedes "Bucket" and "Key" properties
in PutGCSObject.
Get rid of {{super.getSupportedPropertyDescriptors()}} and reorder the
properties in a more reasonable way.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12928) Add Simple Write strategy in PutAzureDataLakeStorage
[ https://issues.apache.org/jira/browse/NIFI-12928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12928: --- Status: Patch Available (was: Open) > Add Simple Write strategy in PutAzureDataLakeStorage > > > Key: NIFI-12928 > URL: https://issues.apache.org/jira/browse/NIFI-12928 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > NIFI-8248 introduced temporary location for file uploads in > PutAzureDataLakeStorage. The file is uploaded into a temporary directory > first and then moved to the final location. > The motivation behind this was that the file upload with Azure client API is > a 2-step process: first an empty file is created, then the content is > uploaded. The unfinished empty file would be available for listing / > fetching. Also, if the put processor is in 'replace' mode and the uploading > fails, then old content is lost as the file was already cleared. > However, this behaviour is not desired in all cases and writing the file > directly to its final location would be preferred. E.g. when ACL is used for > permission handling and the temporary folder would need extra administration > (and the files are not listed/read in parallel so there is no issue with the > partially written files). > Add 'Writing Strategy' property with 'Write and Rename' (current behaviour, > default) and 'Simple Write' options. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12928) Add Simple Write strategy in PutAzureDataLakeStorage
Peter Turcsanyi created NIFI-12928: -- Summary: Add Simple Write strategy in PutAzureDataLakeStorage Key: NIFI-12928 URL: https://issues.apache.org/jira/browse/NIFI-12928 Project: Apache NiFi Issue Type: Improvement Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi NIFI-8248 introduced temporary location for file uploads in PutAzureDataLakeStorage. The file is uploaded into a temporary directory first and then moved to the final location. The motivation behind this was that the file upload with Azure client API is a 2-step process: first an empty file is created, then the content is uploaded. The unfinished empty file would be available for listing / fetching. Also, if the put processor is in 'replace' mode and the uploading fails, then old content is lost as the file was already cleared. However, this behaviour is not desired in all cases and writing the file directly to its final location would be preferred. E.g. when ACL is used for permission handling and the temporary folder would need extra administration (and the files are not listed/read in parallel so there is no issue with the partially written files). Add 'Writing Strategy' property with 'Write and Rename' (current behaviour, default) and 'Simple Write' options. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12837) Add DFS setting to smb processors
[
https://issues.apache.org/jira/browse/NIFI-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12837:
---
Affects Version/s: 2.0.0-M2
Status: Patch Available (was: In Progress)
> Add DFS setting to smb processors
> -
>
> Key: NIFI-12837
> URL: https://issues.apache.org/jira/browse/NIFI-12837
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 2.0.0-M2, 1.25.0
>Reporter: Anders
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The hierynomus/smbj library has a setting for enabling DFS which is disabled
> by default:
> https://github.com/hierynomus/smbj/blob/f25d5c5478a5b73e9ba4202dcfb365974e15367e/src/main/java/com/hierynomus/smbj/SmbConfig.java#L106C17-L106C39
> This appears to cause problems in some SMB configurations.
> Patched
> https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> to test in my environment with:
> {code}
> $ git diff
> nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> diff --git
> a/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
>
> b/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> index 0895abfae0..eac765 100644
> ---
> a/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> +++
> b/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> @@ -46,6 +46,8 @@ public final class SmbUtils {
> }
> }
> +configBuilder.withDfsEnabled(true);
> +
> if (context.getProperty(USE_ENCRYPTION).isSet()) {
>
> configBuilder.withEncryptData(context.getProperty(USE_ENCRYPTION).asBoolean());
> }
> {code}
> This appeared to resolve the issue.
> It would be very useful if this setting was available to toggle in the UI for
> all SMB processors.
> Without this setting, I get a *STATUS_PATH_NOT_COVERED* error.
> Somewhat related hierynomus/smbj github issues:
> https://github.com/hierynomus/smbj/issues/152
> https://github.com/hierynomus/smbj/issues/419
> This setting should be made available in the following processors and
> services:
> * GetSmbFile
> * PutSmbFile
> * SmbjClientProviderService
> Edit: It might require some more changes to handle the connections and
> sessions correctly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-12837) Add DFS setting to smb processors
[
https://issues.apache.org/jira/browse/NIFI-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi reassigned NIFI-12837:
--
Assignee: Peter Turcsanyi
> Add DFS setting to smb processors
> -
>
> Key: NIFI-12837
> URL: https://issues.apache.org/jira/browse/NIFI-12837
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.25.0
>Reporter: Anders
>Assignee: Peter Turcsanyi
>Priority: Major
>
> The hierynomus/smbj library has a setting for enabling DFS which is disabled
> by default:
> https://github.com/hierynomus/smbj/blob/f25d5c5478a5b73e9ba4202dcfb365974e15367e/src/main/java/com/hierynomus/smbj/SmbConfig.java#L106C17-L106C39
> This appears to cause problems in some SMB configurations.
> Patched
> https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> to test in my environment with:
> {code}
> $ git diff
> nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> diff --git
> a/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
>
> b/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> index 0895abfae0..eac765 100644
> ---
> a/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> +++
> b/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> @@ -46,6 +46,8 @@ public final class SmbUtils {
> }
> }
> +configBuilder.withDfsEnabled(true);
> +
> if (context.getProperty(USE_ENCRYPTION).isSet()) {
>
> configBuilder.withEncryptData(context.getProperty(USE_ENCRYPTION).asBoolean());
> }
> {code}
> This appeared to resolve the issue.
> It would be very useful if this setting was available to toggle in the UI for
> all SMB processors.
> Without this setting, I get a *STATUS_PATH_NOT_COVERED* error.
> Somewhat related hierynomus/smbj github issues:
> https://github.com/hierynomus/smbj/issues/152
> https://github.com/hierynomus/smbj/issues/419
> This setting should be made available in the following processors and
> services:
> * GetSmbFile
> * PutSmbFile
> * SmbjClientProviderService
> Edit: It might require some more changes to handle the connections and
> sessions correctly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12837) Add DFS setting to smb processors
[
https://issues.apache.org/jira/browse/NIFI-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17826782#comment-17826782
]
Peter Turcsanyi commented on NIFI-12837:
[~andersns] Thanks for filing this feature request!
I did some investigation around DFS support in smbj and how it could be used in
NiFi a few months ago. You are right that turning it on is quite
straightforward. However, I ran into two bugs in the library (there may be
[others|https://github.com/hierynomus/smbj/issues?q=is%3Aissue+is%3Aopen+DFS]
too):
* [https://github.com/hierynomus/smbj/issues/796] - Stale connections in
SMBClient.connectionTable
* [https://github.com/hierynomus/smbj/issues/717] - Null Pointer when opening
DFS link
They should be fixed on the smbj side. Suggestions were provided how to handle
them, but still no answers.
Fortunately, there is a workaround to the first one that can be implemented in
NiFi.
The second one is a bit specific:
- affects the List processor only
- occurs only when the DFS root or a parent directory (not a link) is listed;
when Directory property is set to a link directly, it works
- managed to replicate on Samba DFS share and standalone Windows instances; it
seems to work in Windows AD Domain where the DFS namespace is hosted by Domain
Controller(s)
Having said that, I don't feel smbj's DFS support pretty robust but we can go
ahead and add it in the SMB processors as an "experimental" feature.
> Add DFS setting to smb processors
> -
>
> Key: NIFI-12837
> URL: https://issues.apache.org/jira/browse/NIFI-12837
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.25.0
>Reporter: Anders
>Priority: Major
>
> The hierynomus/smbj library has a setting for enabling DFS which is disabled
> by default:
> https://github.com/hierynomus/smbj/blob/f25d5c5478a5b73e9ba4202dcfb365974e15367e/src/main/java/com/hierynomus/smbj/SmbConfig.java#L106C17-L106C39
> This appears to cause problems in some SMB configurations.
> Patched
> https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> to test in my environment with:
> {code}
> $ git diff
> nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> diff --git
> a/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
>
> b/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> index 0895abfae0..eac765 100644
> ---
> a/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> +++
> b/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java
> @@ -46,6 +46,8 @@ public final class SmbUtils {
> }
> }
> +configBuilder.withDfsEnabled(true);
> +
> if (context.getProperty(USE_ENCRYPTION).isSet()) {
>
> configBuilder.withEncryptData(context.getProperty(USE_ENCRYPTION).asBoolean());
> }
> {code}
> This appeared to resolve the issue.
> It would be very useful if this setting was available to toggle in the UI for
> all SMB processors.
> Without this setting, I get a *STATUS_PATH_NOT_COVERED* error.
> Somewhat related hierynomus/smbj github issues:
> https://github.com/hierynomus/smbj/issues/152
> https://github.com/hierynomus/smbj/issues/419
> This setting should be made available in the following processors and
> services:
> * GetSmbFile
> * PutSmbFile
> * SmbjClientProviderService
> Edit: It might require some more changes to handle the connections and
> sessions correctly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10707) Add Proxy Configuration Service to new PutBigQuery processor
[ https://issues.apache.org/jira/browse/NIFI-10707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-10707: --- Status: Patch Available (was: Open) > Add Proxy Configuration Service to new PutBigQuery processor > > > Key: NIFI-10707 > URL: https://issues.apache.org/jira/browse/NIFI-10707 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Csaba Bejan >Assignee: Peter Turcsanyi >Priority: Major > Labels: BigQuery, GCP > Time Spent: 10m > Remaining Estimate: 0h > > The new PutBigQuery processor targeting the new Write API introduced as part > of this ticket: https://issues.apache.org/jira/browse/NIFI-10403 should be > extended with Proxy Configuration Service capability. As discussed on an > earlier PR Proxy configuration needs to be added on gRPC level as well which > could be a more involved effort: https://github.com/apache/nifi/pull/6580 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-10707) Add Proxy Configuration Service to new PutBigQuery processor
[ https://issues.apache.org/jira/browse/NIFI-10707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi reassigned NIFI-10707: -- Assignee: Peter Turcsanyi > Add Proxy Configuration Service to new PutBigQuery processor > > > Key: NIFI-10707 > URL: https://issues.apache.org/jira/browse/NIFI-10707 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Csaba Bejan >Assignee: Peter Turcsanyi >Priority: Major > Labels: BigQuery, GCP > > The new PutBigQuery processor targeting the new Write API introduced as part > of this ticket: https://issues.apache.org/jira/browse/NIFI-10403 should be > extended with Proxy Configuration Service capability. As discussed on an > earlier PR Proxy configuration needs to be added on gRPC level as well which > could be a more involved effort: https://github.com/apache/nifi/pull/6580 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-10883) SnowflakeComputingConnectionPool fails on Java 21
[
https://issues.apache.org/jira/browse/NIFI-10883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi resolved NIFI-10883.
Resolution: Information Provided
> SnowflakeComputingConnectionPool fails on Java 21
> -
>
> Key: NIFI-10883
> URL: https://issues.apache.org/jira/browse/NIFI-10883
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
>
> Stack trace:
> {code:java}
> 2022-11-28 12:03:05,585 ERROR [Timer-Driven Process Thread-1]
> o.a.n.p.standard.PutDatabaseRecord
> PutDatabaseRecord[id=c8f95698-77b8-379e-1a39-f4f9fa48bd91] Failed to put
> Records to database for
> StandardFlowFileRecord[uuid=41f457f5-3d38-426c-98e3-63ec75b57548,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1669633380959-1, container=default,
> section=1], offset=0,
> length=25],offset=0,name=41f457f5-3d38-426c-98e3-63ec75b57548,size=25].
> Routing to failure.
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> Cannot create PoolableConnectionFactory (isValid() returned false)
> at
> org.apache.nifi.dbcp.AbstractDBCPConnectionPool.getConnection(AbstractDBCPConnectionPool.java:573)
> at
> org.apache.nifi.dbcp.AbstractDBCPConnectionPool.getConnection(AbstractDBCPConnectionPool.java:550)
> at org.apache.nifi.dbcp.DBCPService.getConnection(DBCPService.java:55)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:568)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:105)
> at jdk.proxy12/jdk.proxy12.$Proxy88.getConnection(Unknown Source)
> at
> org.apache.nifi.processors.standard.PutDatabaseRecord.onTrigger(PutDatabaseRecord.java:487)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1356)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:246)
> at
> org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:59)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.base/java.lang.Thread.run(Thread.java:833)
> Caused by: java.sql.SQLException: Cannot create PoolableConnectionFactory
> (isValid() returned false)
> at
> org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:653)
> at
> org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:531)
> at
> org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:731)
> at
> org.apache.nifi.dbcp.AbstractDBCPConnectionPool.getConnection(AbstractDBCPConnectionPool.java:560)
> ... 21 common frames omitted
> Caused by: java.sql.SQLException: isValid() returned false
> at
> org.apache.commons.dbcp2.PoolableConnection.validate(PoolableConnection.java:335)
> at
> org.apache.commons.dbcp2.PoolableConnectionFactory.validateConnection(PoolableConnectionFactory.java:653)
> at
> org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:109)
> at
> org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:649)
> ... 24 common frames omitted
> {code}
> Environment:
> - compiled with 1.8.0_312-b07 (Temurin)
> -- run on 1.8.0_312-b07 (Temurin) => no error
> -- run on Temurin-11.0.17+8 => no error
> -- run on Temurin-17.0.5+8 => failure
> - compiled with Temurin-17.0.5+8
> -- run on Temurin-17.0.5+8 => failure
--
This
[jira] [Commented] (NIFI-10883) SnowflakeComputingConnectionPool fails on Java 21
[
https://issues.apache.org/jira/browse/NIFI-10883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823618#comment-17823618
]
Peter Turcsanyi commented on NIFI-10883:
[SNOW-917458|https://github.com/snowflakedb/snowflake-jdbc/issues/1512] has
been solved in snowflake-jdbc:3.14.4 and NiFi 2.0 will use the fixed version
(NIFI-12497).
If SnowflakeComputingConnectionPool is used with ExecuteSQL[Record] (and also
in other cases when InaccessibleObjectException occurs), then the following
entry needs to be added in bootstrap.conf:
{code:java}
java.arg.add-opens.java.nio=--add-opens=java.base/java.nio=ALL-UNNAMED
{code}
As nifi-snowflake nars are not part of the default distribution bundle of NiFi,
bootstrap.conf does not contain the setting by default. It needs to be added on
demand.
> SnowflakeComputingConnectionPool fails on Java 21
> -
>
> Key: NIFI-10883
> URL: https://issues.apache.org/jira/browse/NIFI-10883
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
>
> Stack trace:
> {code:java}
> 2022-11-28 12:03:05,585 ERROR [Timer-Driven Process Thread-1]
> o.a.n.p.standard.PutDatabaseRecord
> PutDatabaseRecord[id=c8f95698-77b8-379e-1a39-f4f9fa48bd91] Failed to put
> Records to database for
> StandardFlowFileRecord[uuid=41f457f5-3d38-426c-98e3-63ec75b57548,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1669633380959-1, container=default,
> section=1], offset=0,
> length=25],offset=0,name=41f457f5-3d38-426c-98e3-63ec75b57548,size=25].
> Routing to failure.
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> Cannot create PoolableConnectionFactory (isValid() returned false)
> at
> org.apache.nifi.dbcp.AbstractDBCPConnectionPool.getConnection(AbstractDBCPConnectionPool.java:573)
> at
> org.apache.nifi.dbcp.AbstractDBCPConnectionPool.getConnection(AbstractDBCPConnectionPool.java:550)
> at org.apache.nifi.dbcp.DBCPService.getConnection(DBCPService.java:55)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:568)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:105)
> at jdk.proxy12/jdk.proxy12.$Proxy88.getConnection(Unknown Source)
> at
> org.apache.nifi.processors.standard.PutDatabaseRecord.onTrigger(PutDatabaseRecord.java:487)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1356)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:246)
> at
> org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:59)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.base/java.lang.Thread.run(Thread.java:833)
> Caused by: java.sql.SQLException: Cannot create PoolableConnectionFactory
> (isValid() returned false)
> at
> org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:653)
> at
> org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:531)
> at
> org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:731)
> at
> org.apache.nifi.dbcp.AbstractDBCPConnectionPool.getConnection(AbstractDBCPConnectionPool.java:560)
> ... 21 common frames omitted
> Caused by: java.sql.SQLException: isValid() returned false
> at
> org.apache.commons.dbcp2.PoolableConnection.validate(PoolableConnection.java:335)
> at
>
[jira] [Updated] (NIFI-12846) AWS Assume Role Credentials with VPCE Endpoint URL cannot handle the Region
[
https://issues.apache.org/jira/browse/NIFI-12846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12846:
---
Status: Patch Available (was: Open)
> AWS Assume Role Credentials with VPCE Endpoint URL cannot handle the Region
>
>
> Key: NIFI-12846
> URL: https://issues.apache.org/jira/browse/NIFI-12846
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.25.0
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case of custom Endpoint URLs, the AWS client library may be able to parse
> the Region from the URL but can not handle VPCE URLs (e.g.
> [https://vpce-{*}-{*}***-eu-central-1a.sts.eu-central-1.vpce.amazonaws.com).|https://vpce-%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A-%2A%2A%2A%2A%2A%2A%2A%2A-eu-central-1a.sts.eu-central-1.vpce.amazonaws.com)./]
> {code:java}
> 2024-02-27 13:13:04,102 ERROR [Timer-Driven Process Thread-1]
> o.apache.nifi.processors.aws.s3.ListS3
> ListS3[id=d5e08c19-a155-3b34-e9e6-dbd70e048cd1] Failed to list contents of
> bucket
> com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException:
> Credential should be scoped to a valid region. (Service:
> AWSSecurityTokenService; Status Code: 403; Error Code: SignatureDoesNotMatch;
> Request ID: 7820b219-dee5-4989-8d0c-46523
> 1469705; Proxy: null)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
> at
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
> at
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.doInvoke(AWSSecurityTokenServiceClient.java:1731)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1698)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1687)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.executeAssumeRole(AWSSecurityTokenServiceClient.java:532)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.assumeRole(AWSSecurityTokenServiceClient.java:501)
> ... {code}
> Use the explicit Region property (added in NIFI-10791) for VPCE endpoints.
> The issue affects the 1.x line only. In NiFi 2.0, the AWS processors
> (including the credential handling) have been refactored (NIFI-12144).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12846) AWS Assume Role Credentials with VPCE Endpoint URL cannot handle the Region
[
https://issues.apache.org/jira/browse/NIFI-12846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12846:
---
Description:
In case of custom Endpoint URLs, the AWS client library may be able to parse
the Region from the URL but can not handle VPCE URLs (e.g.
[https://vpce-{*}-{*}***-eu-central-1a.sts.eu-central-1.vpce.amazonaws.com).|https://vpce-%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A-%2A%2A%2A%2A%2A%2A%2A%2A-eu-central-1a.sts.eu-central-1.vpce.amazonaws.com)./]
{code:java}
2024-02-27 13:13:04,102 ERROR [Timer-Driven Process Thread-1]
o.apache.nifi.processors.aws.s3.ListS3
ListS3[id=d5e08c19-a155-3b34-e9e6-dbd70e048cd1] Failed to list contents of
bucket
com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException:
Credential should be scoped to a valid region. (Service:
AWSSecurityTokenService; Status Code: 403; Error Code: SignatureDoesNotMatch;
Request ID: 7820b219-dee5-4989-8d0c-46523
1469705; Proxy: null)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
at
com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.doInvoke(AWSSecurityTokenServiceClient.java:1731)
at
com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1698)
at
com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1687)
at
com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.executeAssumeRole(AWSSecurityTokenServiceClient.java:532)
at
com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.assumeRole(AWSSecurityTokenServiceClient.java:501)
... {code}
Use the explicit Region property (added in NIFI-10791) for VPCE endpoints.
The issue affects the 1.x line only. In NiFi 2.0, the AWS processors (including
the credential handling) have been refactored (NIFI-12144).
was:
In case of custom Endpoint URLs, the AWS client library may be able to parse
the Region from the URL but can not handle VPCE URLs (e.g.
[https://vpce-*--eu-central-1a.sts.eu-central-1.vpce.amazonaws.com).|https://vpce-%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A-%2A%2A%2A%2A%2A%2A%2A%2A-eu-central-1a.sts.eu-central-1.vpce.amazonaws.com)./]
{code:java}
2024-02-27 13:13:04,102 ERROR [Timer-Driven Process Thread-1]
o.apache.nifi.processors.aws.s3.ListS3
ListS3[id=d5e08c19-a155-3b34-e9e6-dbd70e048cd1] Failed to list contents of
bucket
com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException:
Credential should be scoped to a valid region. (Service:
AWSSecurityTokenService; Status Code: 403; Error Code: SignatureDoesNotMatch;
Request ID: 7820b219-dee5-4989-8d0c-46523
1469705; Proxy: null)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
at
[jira] [Updated] (NIFI-12846) AWS Assume Role Credentials with VPCE Endpoint URL cannot handle the Region
[
https://issues.apache.org/jira/browse/NIFI-12846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12846:
---
Affects Version/s: 1.25.0
> AWS Assume Role Credentials with VPCE Endpoint URL cannot handle the Region
>
>
> Key: NIFI-12846
> URL: https://issues.apache.org/jira/browse/NIFI-12846
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.25.0
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
>
> In case of custom Endpoint URLs, the AWS client library may be able to parse
> the Region from the URL but can not handle VPCE URLs (e.g.
> [https://vpce-*--eu-central-1a.sts.eu-central-1.vpce.amazonaws.com).|https://vpce-%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A-%2A%2A%2A%2A%2A%2A%2A%2A-eu-central-1a.sts.eu-central-1.vpce.amazonaws.com)./]
> {code:java}
> 2024-02-27 13:13:04,102 ERROR [Timer-Driven Process Thread-1]
> o.apache.nifi.processors.aws.s3.ListS3
> ListS3[id=d5e08c19-a155-3b34-e9e6-dbd70e048cd1] Failed to list contents of
> bucket
> com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException:
> Credential should be scoped to a valid region. (Service:
> AWSSecurityTokenService; Status Code: 403; Error Code: SignatureDoesNotMatch;
> Request ID: 7820b219-dee5-4989-8d0c-46523
> 1469705; Proxy: null)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
> at
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
> at
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.doInvoke(AWSSecurityTokenServiceClient.java:1731)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1698)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1687)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.executeAssumeRole(AWSSecurityTokenServiceClient.java:532)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.assumeRole(AWSSecurityTokenServiceClient.java:501)
> ... {code}
> Use the explicit Region property (added in NIFI-10791) for VPCE endpoints.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-12846) AWS Assume Role Credentials with VPCE Endpoint URL cannot handle the Region
[
https://issues.apache.org/jira/browse/NIFI-12846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi reassigned NIFI-12846:
--
Assignee: Peter Turcsanyi
> AWS Assume Role Credentials with VPCE Endpoint URL cannot handle the Region
>
>
> Key: NIFI-12846
> URL: https://issues.apache.org/jira/browse/NIFI-12846
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
>
> In case of custom Endpoint URLs, the AWS client library may be able to parse
> the Region from the URL but can not handle VPCE URLs (e.g.
> [https://vpce-*--eu-central-1a.sts.eu-central-1.vpce.amazonaws.com).|https://vpce-%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A-%2A%2A%2A%2A%2A%2A%2A%2A-eu-central-1a.sts.eu-central-1.vpce.amazonaws.com)./]
> {code:java}
> 2024-02-27 13:13:04,102 ERROR [Timer-Driven Process Thread-1]
> o.apache.nifi.processors.aws.s3.ListS3
> ListS3[id=d5e08c19-a155-3b34-e9e6-dbd70e048cd1] Failed to list contents of
> bucket
> com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException:
> Credential should be scoped to a valid region. (Service:
> AWSSecurityTokenService; Status Code: 403; Error Code: SignatureDoesNotMatch;
> Request ID: 7820b219-dee5-4989-8d0c-46523
> 1469705; Proxy: null)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
> at
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
> at
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.doInvoke(AWSSecurityTokenServiceClient.java:1731)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1698)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1687)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.executeAssumeRole(AWSSecurityTokenServiceClient.java:532)
> at
> com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.assumeRole(AWSSecurityTokenServiceClient.java:501)
> ... {code}
> Use the explicit Region property (added in NIFI-10791) for VPCE endpoints.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12846) AWS Assume Role Credentials with VPCE Endpoint URL cannot handle the Region
Peter Turcsanyi created NIFI-12846:
--
Summary: AWS Assume Role Credentials with VPCE Endpoint URL cannot
handle the Region
Key: NIFI-12846
URL: https://issues.apache.org/jira/browse/NIFI-12846
Project: Apache NiFi
Issue Type: Bug
Reporter: Peter Turcsanyi
In case of custom Endpoint URLs, the AWS client library may be able to parse
the Region from the URL but can not handle VPCE URLs (e.g.
[https://vpce-*--eu-central-1a.sts.eu-central-1.vpce.amazonaws.com).|https://vpce-%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A%2A-%2A%2A%2A%2A%2A%2A%2A%2A-eu-central-1a.sts.eu-central-1.vpce.amazonaws.com)./]
{code:java}
2024-02-27 13:13:04,102 ERROR [Timer-Driven Process Thread-1]
o.apache.nifi.processors.aws.s3.ListS3
ListS3[id=d5e08c19-a155-3b34-e9e6-dbd70e048cd1] Failed to list contents of
bucket
com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException:
Credential should be scoped to a valid region. (Service:
AWSSecurityTokenService; Status Code: 403; Error Code: SignatureDoesNotMatch;
Request ID: 7820b219-dee5-4989-8d0c-46523
1469705; Proxy: null)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
at
com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.doInvoke(AWSSecurityTokenServiceClient.java:1731)
at
com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1698)
at
com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1687)
at
com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.executeAssumeRole(AWSSecurityTokenServiceClient.java:532)
at
com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.assumeRole(AWSSecurityTokenServiceClient.java:501)
... {code}
Use the explicit Region property (added in NIFI-10791) for VPCE endpoints.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12671) Implement AWS option for FileResourceService
[ https://issues.apache.org/jira/browse/NIFI-12671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12671: --- Fix Version/s: 2.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Implement AWS option for FileResourceService > > > Key: NIFI-12671 > URL: https://issues.apache.org/jira/browse/NIFI-12671 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Balázs Gerner >Assignee: Balázs Gerner >Priority: Major > Fix For: 2.0.0 > > Time Spent: 2h > Remaining Estimate: 0h > > Add S3 based implementation for the FileResourceService interface to be able > to fetch data directly from AWS. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12782) Migrate GCS processors' Proxy properties to ProxyConfigurationService
[ https://issues.apache.org/jira/browse/NIFI-12782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12782: --- Status: Patch Available (was: Open) > Migrate GCS processors' Proxy properties to ProxyConfigurationService > - > > Key: NIFI-12782 > URL: https://issues.apache.org/jira/browse/NIFI-12782 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Get rid of the obsolete processor level Proxy properties in all GCS > processors and add migration code to convert them to > ProxyConfigurationService. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12782) Migrate GCS processors' Proxy properties to ProxyConfigurationService
Peter Turcsanyi created NIFI-12782: -- Summary: Migrate GCS processors' Proxy properties to ProxyConfigurationService Key: NIFI-12782 URL: https://issues.apache.org/jira/browse/NIFI-12782 Project: Apache NiFi Issue Type: Improvement Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi Get rid of the obsolete processor level Proxy properties in all GCS processors and add migration code to convert them to ProxyConfigurationService. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12775) Provenance FileResource should be renamed
[ https://issues.apache.org/jira/browse/NIFI-12775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12775: --- Summary: Provenance FileResource should be renamed (was: Provenance FIleResource shoud be renamed) > Provenance FileResource should be renamed > - > > Key: NIFI-12775 > URL: https://issues.apache.org/jira/browse/NIFI-12775 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Lehel Boér >Assignee: Lehel Boér >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The org.apache.nifi.provenance.FileResource should be renamed > because it clashes with org.apache.nifi.fileresource.service.api.FileResource. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12775) Provenance FIleResource shoud be renamed
[ https://issues.apache.org/jira/browse/NIFI-12775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi resolved NIFI-12775. Fix Version/s: 2.0.0 Resolution: Fixed > Provenance FIleResource shoud be renamed > > > Key: NIFI-12775 > URL: https://issues.apache.org/jira/browse/NIFI-12775 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Lehel Boér >Assignee: Lehel Boér >Priority: Major > Fix For: 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The org.apache.nifi.provenance.FileResource should be renamed > because it clashes with org.apache.nifi.fileresource.service.api.FileResource. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12775) Provenance FIleResource shoud be renamed
[ https://issues.apache.org/jira/browse/NIFI-12775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12775: --- Priority: Minor (was: Major) > Provenance FIleResource shoud be renamed > > > Key: NIFI-12775 > URL: https://issues.apache.org/jira/browse/NIFI-12775 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Lehel Boér >Assignee: Lehel Boér >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The org.apache.nifi.provenance.FileResource should be renamed > because it clashes with org.apache.nifi.fileresource.service.api.FileResource. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12771) Restore Client-side KMS encryption strategy in StandardS3EncryptionService
[ https://issues.apache.org/jira/browse/NIFI-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12771: --- Status: Patch Available (was: Open) > Restore Client-side KMS encryption strategy in StandardS3EncryptionService > -- > > Key: NIFI-12771 > URL: https://issues.apache.org/jira/browse/NIFI-12771 > Project: Apache NiFi > Issue Type: Bug >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > NIFI-12144 partially removed "Client-side KMS" encryption strategy. The > strategy class and some code were deleted but the option is still available > on the UI, just not works. > It seems it happened unintentionally. Bring back the functionality. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12771) Restore Client-side KMS encryption strategy in StandardS3EncryptionService
Peter Turcsanyi created NIFI-12771: -- Summary: Restore Client-side KMS encryption strategy in StandardS3EncryptionService Key: NIFI-12771 URL: https://issues.apache.org/jira/browse/NIFI-12771 Project: Apache NiFi Issue Type: Bug Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi NIFI-12144 partially removed "Client-side KMS" encryption strategy. The strategy class and some code were deleted but the option is still available on the UI, just not works. It seems it happened unintentionally. Bring back the functionality. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12769) Update Copyright for 2024
[ https://issues.apache.org/jira/browse/NIFI-12769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12769: --- Status: Patch Available (was: Open) > Update Copyright for 2024 > - > > Key: NIFI-12769 > URL: https://issues.apache.org/jira/browse/NIFI-12769 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12769) Update Copyright for 2024
Peter Turcsanyi created NIFI-12769: -- Summary: Update Copyright for 2024 Key: NIFI-12769 URL: https://issues.apache.org/jira/browse/NIFI-12769 Project: Apache NiFi Issue Type: Improvement Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12766) Fix Region handling in AssumeRoleCredentialsStrategy in AWSCredentialsProviderControllerService
[
https://issues.apache.org/jira/browse/NIFI-12766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12766:
---
Status: Patch Available (was: Open)
> Fix Region handling in AssumeRoleCredentialsStrategy in
> AWSCredentialsProviderControllerService
> ---
>
> Key: NIFI-12766
> URL: https://issues.apache.org/jira/browse/NIFI-12766
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Either region or custom endpoint configuration (including the region) can be
> set on {{AWSSecurityTokenServiceClientBuilder}}.
>
> {code:java}
> 2024-02-09 17:44:16,084 ERROR [Timer-Driven Process Thread-2]
> o.a.n.c.s.StandardControllerServiceNode
> StandardControllerServiceNode[service=AWSCredentialsProviderService[id=1b0b4a21-e6fa-3e75-88b2-11e7583673a4],
> name=AWSCredentialsProviderControllerService (STS+Endpoint), active=true]
> Failed to invoke @OnEnabled method
> java.lang.IllegalStateException: Only one of Region or EndpointConfiguration
> may be set.
> at
> com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:450)
> at
> com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
> at
> com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
> at
> org.apache.nifi.processors.aws.credentials.provider.factory.strategies.AssumeRoleCredentialsStrategy.getDerivedCredentialsProvider(AssumeRoleCredentialsStrategy.java:190)
> at
> org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.createCredentialsProvider(AWSCredentialsProviderControllerService.java:382)
> at
> org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.onConfigured(AWSCredentialsProviderControllerService.java:371)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12766) Fix Region handling in AssumeRoleCredentialsStrategy in AWSCredentialsProviderControllerService
[
https://issues.apache.org/jira/browse/NIFI-12766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12766:
---
Description:
Either region or custom endpoint configuration (including the region) can be
set on {{AWSSecurityTokenServiceClientBuilder}}.
{code:java}
2024-02-09 17:44:16,084 ERROR [Timer-Driven Process Thread-2]
o.a.n.c.s.StandardControllerServiceNode
StandardControllerServiceNode[service=AWSCredentialsProviderService[id=1b0b4a21-e6fa-3e75-88b2-11e7583673a4],
name=AWSCredentialsProviderControllerService (STS+Endpoint), active=true]
Failed to invoke @OnEnabled method
java.lang.IllegalStateException: Only one of Region or EndpointConfiguration
may be set.
at
com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:450)
at
com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
at
com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
at
org.apache.nifi.processors.aws.credentials.provider.factory.strategies.AssumeRoleCredentialsStrategy.getDerivedCredentialsProvider(AssumeRoleCredentialsStrategy.java:190)
at
org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.createCredentialsProvider(AWSCredentialsProviderControllerService.java:382)
at
org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.onConfigured(AWSCredentialsProviderControllerService.java:371)
{code}
was:
Either region or custom endpoint configuration (including the region) can be
set on
{{{}AWSSecurityTokenServiceClientBuilder{}}}.
{code:java}
2024-02-09 17:44:16,084 ERROR [Timer-Driven Process Thread-2]
o.a.n.c.s.StandardControllerServiceNode
StandardControllerServiceNode[service=AWSCredentialsProviderService[id=1b0b4a21-e6fa-3e75-88b2-11e7583673a4],
name=AWSCredentialsProviderControllerService (STS+Endpoint), active=true]
Failed to invoke @OnEnabled method
java.lang.IllegalStateException: Only one of Region or EndpointConfiguration
may be set.
at
com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:450)
at
com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
at
com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
at
org.apache.nifi.processors.aws.credentials.provider.factory.strategies.AssumeRoleCredentialsStrategy.getDerivedCredentialsProvider(AssumeRoleCredentialsStrategy.java:190)
at
org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.createCredentialsProvider(AWSCredentialsProviderControllerService.java:382)
at
org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.onConfigured(AWSCredentialsProviderControllerService.java:371)
{code}
> Fix Region handling in AssumeRoleCredentialsStrategy in
> AWSCredentialsProviderControllerService
> ---
>
> Key: NIFI-12766
> URL: https://issues.apache.org/jira/browse/NIFI-12766
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
>
> Either region or custom endpoint configuration (including the region) can be
> set on {{AWSSecurityTokenServiceClientBuilder}}.
>
> {code:java}
> 2024-02-09 17:44:16,084 ERROR [Timer-Driven Process Thread-2]
> o.a.n.c.s.StandardControllerServiceNode
> StandardControllerServiceNode[service=AWSCredentialsProviderService[id=1b0b4a21-e6fa-3e75-88b2-11e7583673a4],
> name=AWSCredentialsProviderControllerService (STS+Endpoint), active=true]
> Failed to invoke @OnEnabled method
> java.lang.IllegalStateException: Only one of Region or EndpointConfiguration
> may be set.
> at
> com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:450)
> at
> com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
> at
> com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
> at
> org.apache.nifi.processors.aws.credentials.provider.factory.strategies.AssumeRoleCredentialsStrategy.getDerivedCredentialsProvider(AssumeRoleCredentialsStrategy.java:190)
> at
> org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.createCredentialsProvider(AWSCredentialsProviderControllerService.java:382)
> at
> org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.onConfigured(AWSCredentialsProviderControllerService.java:371)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12766) Fix Region handling in AssumeRoleCredentialsStrategy in AWSCredentialsProviderControllerService
Peter Turcsanyi created NIFI-12766:
--
Summary: Fix Region handling in AssumeRoleCredentialsStrategy in
AWSCredentialsProviderControllerService
Key: NIFI-12766
URL: https://issues.apache.org/jira/browse/NIFI-12766
Project: Apache NiFi
Issue Type: Bug
Reporter: Peter Turcsanyi
Assignee: Peter Turcsanyi
Either region or custom endpoint configuration (including the region) can be
set on
{{{}AWSSecurityTokenServiceClientBuilder{}}}.
{code:java}
2024-02-09 17:44:16,084 ERROR [Timer-Driven Process Thread-2]
o.a.n.c.s.StandardControllerServiceNode
StandardControllerServiceNode[service=AWSCredentialsProviderService[id=1b0b4a21-e6fa-3e75-88b2-11e7583673a4],
name=AWSCredentialsProviderControllerService (STS+Endpoint), active=true]
Failed to invoke @OnEnabled method
java.lang.IllegalStateException: Only one of Region or EndpointConfiguration
may be set.
at
com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:450)
at
com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
at
com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
at
org.apache.nifi.processors.aws.credentials.provider.factory.strategies.AssumeRoleCredentialsStrategy.getDerivedCredentialsProvider(AssumeRoleCredentialsStrategy.java:190)
at
org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.createCredentialsProvider(AWSCredentialsProviderControllerService.java:382)
at
org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService.onConfigured(AWSCredentialsProviderControllerService.java:371)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12736) Fix migration of credential properties in AWS processors
[
https://issues.apache.org/jira/browse/NIFI-12736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12736:
---
Status: Patch Available (was: Open)
> Fix migration of credential properties in AWS processors
>
>
> Key: NIFI-12736
> URL: https://issues.apache.org/jira/browse/NIFI-12736
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0-M2
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In AWS processors the processor level credential properties were removed in
> 2.0 in favour of AWS Credentials Provider Service. Migration code was added
> in NIFI-12220.
> The 1.x processors allowed to configure both the processor level Access Key
> ID + Secret Access Key and AWS Credentials Provider Service properties
> ({{{}customValidate(){}}} did not check it for some reason). In this case the
> controller service took precedence.
> The migration process first checks if Access Key ID + Secret Access Key are
> configured and if they are, it creates a new controller service even if there
> was another one already in use (and the new may have outdated migrated
> credential values as the pre-existing service was used effectively in the
> flow).
> Modify the migration logic to avoid controller service duplication and to use
> the existing service.
> Also modify the default credential to
> [anonymous|https://github.com/apache/nifi/blob/ce56117e43c7cdce37ca76d54550a2b45acb508a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java#L415]
> as it was used if no properties were configured at all.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12736) Fix migration of credential properties in AWS processors
[
https://issues.apache.org/jira/browse/NIFI-12736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12736:
---
Affects Version/s: 2.0.0-M2
> Fix migration of credential properties in AWS processors
>
>
> Key: NIFI-12736
> URL: https://issues.apache.org/jira/browse/NIFI-12736
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0-M2
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Minor
>
> In AWS processors the processor level credential properties were removed in
> 2.0 in favour of AWS Credentials Provider Service. Migration code was added
> in NIFI-12220.
> The 1.x processors allowed to configure both the processor level Access Key
> ID + Secret Access Key and AWS Credentials Provider Service properties
> ({{{}customValidate(){}}} did not check it for some reason). In this case the
> controller service took precedence.
> The migration process first checks if Access Key ID + Secret Access Key are
> configured and if they are, it creates a new controller service even if there
> was another one already in use (and the new may have outdated migrated
> credential values as the pre-existing service was used effectively in the
> flow).
> Modify the migration logic to avoid controller service duplication and to use
> the existing service.
> Also modify the default credential to
> [anonymous|https://github.com/apache/nifi/blob/ce56117e43c7cdce37ca76d54550a2b45acb508a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java#L415]
> as it was used if no properties were configured at all.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12736) Fix migration of credential properties in AWS processors
Peter Turcsanyi created NIFI-12736:
--
Summary: Fix migration of credential properties in AWS processors
Key: NIFI-12736
URL: https://issues.apache.org/jira/browse/NIFI-12736
Project: Apache NiFi
Issue Type: Bug
Reporter: Peter Turcsanyi
Assignee: Peter Turcsanyi
In AWS processors the processor level credential properties were removed in 2.0
in favour of AWS Credentials Provider Service. Migration code was added in
NIFI-12220.
The 1.x processors allowed to configure both the processor level Access Key ID
+ Secret Access Key and AWS Credentials Provider Service properties
({{{}customValidate(){}}} did not check it for some reason). In this case the
controller service took precedence.
The migration process first checks if Access Key ID + Secret Access Key are
configured and if they are, it creates a new controller service even if there
was another one already in use (and the new may have outdated migrated
credential values as the pre-existing service was used effectively in the flow).
Modify the migration logic to avoid controller service duplication and to use
the existing service.
Also modify the default credential to
[anonymous|https://github.com/apache/nifi/blob/ce56117e43c7cdce37ca76d54550a2b45acb508a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java#L415]
as it was used if no properties were configured at all.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12441) Add No Tracking Strategy to ListS3
[ https://issues.apache.org/jira/browse/NIFI-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12441: --- Fix Version/s: 1.26.0 > Add No Tracking Strategy to ListS3 > -- > > Key: NIFI-12441 > URL: https://issues.apache.org/jira/browse/NIFI-12441 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Julien G. >Assignee: Julien G. >Priority: Major > Fix For: 2.0.0-M2, 1.26.0 > > Time Spent: 50m > Remaining Estimate: 0h > > As was done for ListGCS in > [NIFI-11891|https://issues.apache.org/jira/browse/NIFI-11891], I want to make > it possible to have no state in ListS3 so that I can periodically retrieve > all my objects in S3. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12732) ListS3 should reset its tracking state after configuration change
[
https://issues.apache.org/jira/browse/NIFI-12732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12732:
---
Status: Patch Available (was: Open)
> ListS3 should reset its tracking state after configuration change
> -
>
> Key: NIFI-12732
> URL: https://issues.apache.org/jira/browse/NIFI-12732
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{List}} processors can reset their listing tracking state (stored in the
> component state or in a distributed cache depending on the {{{}Listing
> Strategy{}}}) when a relevant property has been modified (e.g. the directory
> to list). The functionality was implemented in {{{}AbstractListProcessor{}}},
> the child processors only need to provide the property list when the reset
> should occur.
> {{ListS3}} is not a child of {{AbstractListProcessor}} and the listing logic
> was implemented separately. The reset function is missing in the processor.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12732) ListS3 should reset its tracking state after configuration change
Peter Turcsanyi created NIFI-12732:
--
Summary: ListS3 should reset its tracking state after
configuration change
Key: NIFI-12732
URL: https://issues.apache.org/jira/browse/NIFI-12732
Project: Apache NiFi
Issue Type: Bug
Reporter: Peter Turcsanyi
Assignee: Peter Turcsanyi
{{List}} processors can reset their listing tracking state (stored in the
component state or in a distributed cache depending on the {{{}Listing
Strategy{}}}) when a relevant property has been modified (e.g. the directory to
list). The functionality was implemented in {{{}AbstractListProcessor{}}}, the
child processors only need to provide the property list when the reset should
occur.
{{ListS3}} is not a child of {{AbstractListProcessor}} and the listing logic
was implemented separately. The reset function is missing in the processor.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-12715) Update Snowflake SDKs
[ https://issues.apache.org/jira/browse/NIFI-12715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi resolved NIFI-12715. Fix Version/s: 2.0.0 1.26.0 Resolution: Fixed > Update Snowflake SDKs > - > > Key: NIFI-12715 > URL: https://issues.apache.org/jira/browse/NIFI-12715 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M2 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Fix For: 2.0.0, 1.26.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Update the following code: > net.snowflake » snowflake-ingest-sdk from 2.0.4 to 2.0.5 > net.snowflake » snowflake-jdbc from 3.14.4 to 3.14.5 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12642) Add local file upload option in PutS3 processor
[ https://issues.apache.org/jira/browse/NIFI-12642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12642: --- Fix Version/s: 2.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add local file upload option in PutS3 processor > --- > > Key: NIFI-12642 > URL: https://issues.apache.org/jira/browse/NIFI-12642 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Balázs Gerner >Assignee: Balázs Gerner >Priority: Major > Fix For: 2.0.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Similarly to PutAzure*Storage processors, inside the *PutS3Object* processor, > add option to upload file from the local filesystem directly to AWS S3 > by-passing the content repo. > The two available file resource transfer sources are: > * FlowFile's Content (default, current behaviour) > * Local File -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12643) Add local file upload option in PutGCS processor
[ https://issues.apache.org/jira/browse/NIFI-12643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12643: --- Fix Version/s: 2.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add local file upload option in PutGCS processor > > > Key: NIFI-12643 > URL: https://issues.apache.org/jira/browse/NIFI-12643 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Balázs Gerner >Assignee: Balázs Gerner >Priority: Major > Fix For: 2.0.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Similarly to PutAzure*Storage processors, inside the *PutGCSObject* > processor, add option to upload file from the local filesystem directly to > GCS storage by-passing the content repo. > The two available file resource transfer sources are: > * FlowFile's Content (default, current behaviour) > * Local File -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-12452) Improve support for Enum & DescribedValue for allowableValues
[
https://issues.apache.org/jira/browse/NIFI-12452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi reassigned NIFI-12452:
--
Assignee: endzeit
> Improve support for Enum & DescribedValue for allowableValues
> -
>
> Key: NIFI-12452
> URL: https://issues.apache.org/jira/browse/NIFI-12452
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: endzeit
>Assignee: endzeit
>Priority: Major
> Fix For: 2.0.0-M2
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> The {{PropertyDescriptor.Builder}} supports providing a {{Class}} that is
> both an {{Enum}} and {{{}DescribedValue{}}}. This improves type safety, by
> avoiding the passing of sheer arbitrary {{String}} values.
> I'd like to propose extensions to both the {{PropertyDescriptor.Builder}}
> class as well as the {{PropertyValue}} interface.
>
> The {{PropertyDescriptor.Builder}} should allow not only a raw {{String}} to
> be provided as {{{}defaultValue(){}}}, but also provide an overload that
> instead accepts an {{{}DescribedValue{}}}.
> {code:java}
> public Builder defaultValue(final DescribedValue value) {code}
> This allows to replace
> {code:java}
> .allowableValues(Foo.class)
> .defaultValue(Foo.BAR.getValue()) {code}
> with
> {code:java}
> .allowableValues(Foo.class)
> .defaultValue(Foo.BAR) {code}
>
> The {{PropertyValue}} should allow to receive the value as one of the Enum
> constants, similar to one of the existing {{as...}} methods.
> {code:java}
> & DescribedValue> E asAllowableValue(Class clazz) {code}
> This way processor implementations rely on type-safe mappings of allowable
> values instead of matching on {{String}} values manually.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-8278) Add Credentials Type property to ADLSCredentialsControllerService
[
https://issues.apache.org/jira/browse/NIFI-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807468#comment-17807468
]
Peter Turcsanyi commented on NIFI-8278:
---
Thanks for the heads-up [~bbende]! Yes, it is due to
PROXY_CONFIGURATION_SERVICE because it is used by both the controller services
and the processors but I forgot about the latter and added dependsOn based on
the usage in the services. 'credentials-type' is a new property in the ADLS
service and it was never present on the processors. Just the
PROXY_CONFIGURATION_SERVICE property definition was shared. Will fix it in new
PR.
> Add Credentials Type property to ADLSCredentialsControllerService
> -
>
> Key: NIFI-8278
> URL: https://issues.apache.org/jira/browse/NIFI-8278
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Fix For: 2.0.0-M2
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{ADLSCredentialsControllerService}} supports different authentication modes:
> Account Key, SAS Token, Managed Identity, Service Principal.
> All these modes have their own properties and the configuration is not really
> straightforward.
> Add new {{Authentication Type}} property with the enumerated values of
> authentication types. Use {{dependsOn()}} for the other properties in order
> to show only the relevant ones. Also support the current logic with default
> type AUTO for backward compatibility.
> Update:
> Add the feature only in NiFi 2.0. In that case, no AUTO option is needed for
> supporting the legacy configuration. The old-style config can be migrated
> using the new property migration feature (NIFI-12139).
> Use {{Credentials Type}} for the property name (common property with the
> existing {{AzureStorageCredentialsControllerService_v12}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-12594) ListS3 minimum object age filter not observed when entity state tracking is used
[
https://issues.apache.org/jira/browse/NIFI-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi resolved NIFI-12594.
Fix Version/s: 1.25.0
2.0.0
Assignee: Peter Kimberley
Resolution: Fixed
> ListS3 minimum object age filter not observed when entity state tracking is
> used
>
>
> Key: NIFI-12594
> URL: https://issues.apache.org/jira/browse/NIFI-12594
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.24.0, 2.0.0
> Environment: Docker, on-prem S3
>Reporter: Peter Kimberley
>Assignee: Peter Kimberley
>Priority: Major
> Fix For: 1.25.0, 2.0.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> When ListS3 is configured to use the {{Tracking Entities}} listing strategy,
> the following is observed:
> # Configure ListS3 with a Minimum Object Age of {{1 hour.}} Ensure processor
> is stopped.
> # Create a new FlowFile with GenerateFlowFile and run once
> # Put the FlowFile to an S3 bucket with PutS3
> # Open ListS3 configuration
> # Click Verify. UI reports: ??Successfully listed contents of bucket name>, finding 0 objects matching the filter.??
> # Run ListS3 once. Flowfile is retrieved, even though the 1 hour interval
> has not yet elapsed.
> The issue is the ListS3 {{Minimum Object Age}} property is not being observed
> when using the Tracking Entities listing strategy. When using {{{}Tracking
> Timestamps{}}}, the processor behaves as expected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12555) UI - Property with .dependsOn() incorrectly being displayed
[ https://issues.apache.org/jira/browse/NIFI-12555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17804690#comment-17804690 ] Peter Turcsanyi commented on NIFI-12555: [~markbean] Thanks for the context and good to hear that changing the order is working for you! > UI - Property with .dependsOn() incorrectly being displayed > --- > > Key: NIFI-12555 > URL: https://issues.apache.org/jira/browse/NIFI-12555 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.24.0 >Reporter: Mark Bean >Priority: Major > Attachments: TestCustomProcessor.java > > > Consider PROPERTY_1 with: > .dependsOn(PROPERTY_2, "foo") > > In the UI, when Property 2 is set to a value of "bar", Property 1 should not > be displayed. However, when the configuration is first opened, Property 1 is > displayed. The UI correctly updates only after re-selecting "bar" for > Property 2 - or even just opening its value and choosing "Cancel". In other > words, after taking any action to change any property, then Property 1 is > correctly removed from the UI. > This appears to be a problem only for the initial display of component > properties, and not when they are refreshed and/or dependency rules are > evaluated. > -It is possible this is related to changes introduced in NIFI-11593. Note > that the above behavior occurs even when PROPERTY_2 has specific allowed > values.- > The same behavior is seen in 1.21.0, and NIFI-11593 has a fix version of > 1.22.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12576) MQTT processors should allow path component in Broker URI
[ https://issues.apache.org/jira/browse/NIFI-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12576: --- Status: Patch Available (was: Open) > MQTT processors should allow path component in Broker URI > - > > Key: NIFI-12576 > URL: https://issues.apache.org/jira/browse/NIFI-12576 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > MQTT processors validate the Broker URI against a couple of rules and do not > allow URIs containing path component at the end. > RabbitMQ exposes its websocket endpoint via ws://myhost:15675/ws by default. > It cannot be configured on the processors now. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12576) MQTT processors should allow path component in Broker URI
Peter Turcsanyi created NIFI-12576: -- Summary: MQTT processors should allow path component in Broker URI Key: NIFI-12576 URL: https://issues.apache.org/jira/browse/NIFI-12576 Project: Apache NiFi Issue Type: Improvement Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi MQTT processors validate the Broker URI against a couple of rules and do not allow URIs containing path component at the end. RabbitMQ exposes its websocket endpoint via ws://myhost:15675/ws by default. It cannot be configured on the processors now. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12555) UI - Property with .dependsOn() incorrectly being displayed
[ https://issues.apache.org/jira/browse/NIFI-12555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803634#comment-17803634 ] Peter Turcsanyi commented on NIFI-12555: Thanks [~markbean] for the example! I have also seen this behaviour but it only happens when the dependent property precedes the "master" on the UI which is quite unusual in my opinion. So I simply fixed it via changing the order of the properties: "master" first, then the dependent one(s). I don't say it could not be improved but I'm not sure it would worth. Do you have a real business use case where the dependent property should go first? > UI - Property with .dependsOn() incorrectly being displayed > --- > > Key: NIFI-12555 > URL: https://issues.apache.org/jira/browse/NIFI-12555 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.24.0 >Reporter: Mark Bean >Priority: Major > Attachments: TestCustomProcessor.java > > > Consider PROPERTY_1 with: > .dependsOn(PROPERTY_2, "foo") > > In the UI, when Property 2 is set to a value of "bar", Property 1 should not > be displayed. However, when the configuration is first opened, Property 1 is > displayed. The UI correctly updates only after re-selecting "bar" for > Property 2 - or even just opening its value and choosing "Cancel". In other > words, after taking any action to change any property, then Property 1 is > correctly removed from the UI. > This appears to be a problem only for the initial display of component > properties, and not when they are refreshed and/or dependency rules are > evaluated. > -It is possible this is related to changes introduced in NIFI-11593. Note > that the above behavior occurs even when PROPERTY_2 has specific allowed > values.- > The same behavior is seen in 1.21.0, and NIFI-11593 has a fix version of > 1.22.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-8278) Add Credentials Type property to ADLSCredentialsControllerService
[
https://issues.apache.org/jira/browse/NIFI-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-8278:
--
Status: Patch Available (was: Open)
> Add Credentials Type property to ADLSCredentialsControllerService
> -
>
> Key: NIFI-8278
> URL: https://issues.apache.org/jira/browse/NIFI-8278
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{ADLSCredentialsControllerService}} supports different authentication modes:
> Account Key, SAS Token, Managed Identity, Service Principal.
> All these modes have their own properties and the configuration is not really
> straightforward.
> Add new {{Authentication Type}} property with the enumerated values of
> authentication types. Use {{dependsOn()}} for the other properties in order
> to show only the relevant ones. Also support the current logic with default
> type AUTO for backward compatibility.
> Update:
> Add the feature only in NiFi 2.0. In that case, no AUTO option is needed for
> supporting the legacy configuration. The old-style config can be migrated
> using the new property migration feature (NIFI-12139).
> Use {{Credentials Type}} for the property name (common property with the
> existing {{AzureStorageCredentialsControllerService_v12}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-8278) Add Credentials Type property to ADLSCredentialsControllerService
[
https://issues.apache.org/jira/browse/NIFI-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-8278:
--
Summary: Add Credentials Type property to ADLSCredentialsControllerService
(was: Add Authentication Type property to ADLSCredentialsControllerService)
> Add Credentials Type property to ADLSCredentialsControllerService
> -
>
> Key: NIFI-8278
> URL: https://issues.apache.org/jira/browse/NIFI-8278
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
>
> {{ADLSCredentialsControllerService}} supports different authentication modes:
> Account Key, SAS Token, Managed Identity, Service Principal.
> All these modes have their own properties and the configuration is not really
> straightforward.
> Add new {{Authentication Type}} property with the enumerated values of
> authentication types. Use {{dependsOn()}} for the other properties in order
> to show only the relevant ones. Also support the current logic with default
> type AUTO for backward compatibility.
> Update:
> Add the feature only in NiFi 2.0. In that case, no AUTO option is needed for
> supporting the legacy configuration. The old-style config can be migrated
> using the new property migration feature (NIFI-12139).
> Use {{Credentials Type}} for the property name (common property with the
> existing {{AzureStorageCredentialsControllerService_v12}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-8278) Add Authentication Type property to ADLSCredentialsControllerService
[
https://issues.apache.org/jira/browse/NIFI-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-8278:
--
Description:
{{ADLSCredentialsControllerService}} supports different authentication modes:
Account Key, SAS Token, Managed Identity, Service Principal.
All these modes have their own properties and the configuration is not really
straightforward.
Add new {{Authentication Type}} property with the enumerated values of
authentication types. Use {{dependsOn()}} for the other properties in order to
show only the relevant ones. Also support the current logic with default type
AUTO for backward compatibility.
Update:
Add the feature only in NiFi 2.0. In that case, no AUTO option is needed for
supporting the legacy configuration. The old-style config can be migrated using
the new property migration feature (NIFI-12139).
Use {{Credentials Type}} for the property name (common property with the
existing {{AzureStorageCredentialsControllerService_v12}}).
was:
{{ADLSCredentialsControllerService}} supports different authentication modes:
Account Key, SAS Token, Managed Identity, Service Principal.
All these modes have their own properties and the configuration is not really
straightforward.
Add new {{Authentication Type}} property with the enumerated values of
authentication types. Use {{dependsOn()}} for the other properties in order to
show only the relevant ones. Also support the current logic with default type
AUTO for backward compatibility.
> Add Authentication Type property to ADLSCredentialsControllerService
>
>
> Key: NIFI-8278
> URL: https://issues.apache.org/jira/browse/NIFI-8278
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
>
> {{ADLSCredentialsControllerService}} supports different authentication modes:
> Account Key, SAS Token, Managed Identity, Service Principal.
> All these modes have their own properties and the configuration is not really
> straightforward.
> Add new {{Authentication Type}} property with the enumerated values of
> authentication types. Use {{dependsOn()}} for the other properties in order
> to show only the relevant ones. Also support the current logic with default
> type AUTO for backward compatibility.
> Update:
> Add the feature only in NiFi 2.0. In that case, no AUTO option is needed for
> supporting the legacy configuration. The old-style config can be migrated
> using the new property migration feature (NIFI-12139).
> Use {{Credentials Type}} for the property name (common property with the
> existing {{AzureStorageCredentialsControllerService_v12}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12527) Remove deprecated Session Cache Size property from JMS processors
[ https://issues.apache.org/jira/browse/NIFI-12527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12527: --- Status: Patch Available (was: Open) > Remove deprecated Session Cache Size property from JMS processors > - > > Key: NIFI-12527 > URL: https://issues.apache.org/jira/browse/NIFI-12527 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > PublishJMS and ConsumeJMS processors' Session Cache Size property is not used > / has no effect. It was marked as deprecated and can be removed in 2.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12527) Remove deprecated Session Cache Size property from JMS processors
Peter Turcsanyi created NIFI-12527: -- Summary: Remove deprecated Session Cache Size property from JMS processors Key: NIFI-12527 URL: https://issues.apache.org/jira/browse/NIFI-12527 Project: Apache NiFi Issue Type: Improvement Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi PublishJMS and ConsumeJMS processors' Session Cache Size property is not used / has no effect. It was marked as deprecated and can be removed in 2.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12525) Add property migration for the removed Consumer Hostname property in ConsumeAzureEventHub
[ https://issues.apache.org/jira/browse/NIFI-12525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12525: --- Status: Patch Available (was: Open) > Add property migration for the removed Consumer Hostname property in > ConsumeAzureEventHub > - > > Key: NIFI-12525 > URL: https://issues.apache.org/jira/browse/NIFI-12525 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12525) Add property migration for the removed Consumer Hostname property in ConsumeAzureEventHub
Peter Turcsanyi created NIFI-12525: -- Summary: Add property migration for the removed Consumer Hostname property in ConsumeAzureEventHub Key: NIFI-12525 URL: https://issues.apache.org/jira/browse/NIFI-12525 Project: Apache NiFi Issue Type: Improvement Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12521) Upgrade ca.uhn.hapi dependencies to 2.5.1
[ https://issues.apache.org/jira/browse/NIFI-12521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12521: --- Description: Upgrade ca.uhn.hapi dependencies to 2.5.1 The new version of the library is Java 11 bytecode so it is applicable to NiFi 2.x only. was:Upgrade ca.uhn.hapi dependencies to 2.5.1 > Upgrade ca.uhn.hapi dependencies to 2.5.1 > - > > Key: NIFI-12521 > URL: https://issues.apache.org/jira/browse/NIFI-12521 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade ca.uhn.hapi dependencies to 2.5.1 > The new version of the library is Java 11 bytecode so it is applicable to > NiFi 2.x only. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12520) ExtractHL7Attributes processor ignores repeatable field values
[ https://issues.apache.org/jira/browse/NIFI-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi resolved NIFI-12520. Fix Version/s: 1.25.0 2.0.0 Resolution: Fixed > ExtractHL7Attributes processor ignores repeatable field values > -- > > Key: NIFI-12520 > URL: https://issues.apache.org/jira/browse/NIFI-12520 > Project: Apache NiFi > Issue Type: Bug >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Fix For: 1.25.0, 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > When there is a repeatable field with multiple values the processor gets only > the first element from the value list and ignores the rest. This is causing > data loss after the attribute extraction. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12515) Remove deprecated Client Auth property from AMQP processors
[ https://issues.apache.org/jira/browse/NIFI-12515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12515: --- Status: Patch Available (was: Open) > Remove deprecated Client Auth property from AMQP processors > --- > > Key: NIFI-12515 > URL: https://issues.apache.org/jira/browse/NIFI-12515 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > PublishAMQP and ConsumeAMQP processors' Client Auth property is not used / > has no effect. It was marked as deprecated and can be removed in 2.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12515) Remove deprecated Client Auth property from AMQP processors
Peter Turcsanyi created NIFI-12515: -- Summary: Remove deprecated Client Auth property from AMQP processors Key: NIFI-12515 URL: https://issues.apache.org/jira/browse/NIFI-12515 Project: Apache NiFi Issue Type: Improvement Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi PublishAMQP and ConsumeAMQP processors' Client Auth property is not used / has no effect. It was marked as deprecated and can be removed in 2.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-6730) Basic QOS in AMQP
[ https://issues.apache.org/jira/browse/NIFI-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi resolved NIFI-6730. --- Fix Version/s: 1.25.0 2.0.0 Resolution: Fixed > Basic QOS in AMQP > - > > Key: NIFI-6730 > URL: https://issues.apache.org/jira/browse/NIFI-6730 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Matthew Formosa >Priority: Minor > Fix For: 1.25.0, 2.0.0 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > The current implementation of AMQP does not support Basic QOS. The purpose of > this task is, therefore, to allow Basic QOS support. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12509) Change default TTL for HazelcastMapCacheClient
[ https://issues.apache.org/jira/browse/NIFI-12509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12509: --- Fix Version/s: 1.25.0 2.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Change default TTL for HazelcastMapCacheClient > -- > > Key: NIFI-12509 > URL: https://issues.apache.org/jira/browse/NIFI-12509 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Simon Bence >Assignee: Simon Bence >Priority: Major > Fix For: 1.25.0, 2.0.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Current default TTL is 0, which in terms of Hazelcast means that the cache > will be ever-growing. Having Max Size Policy on the imap (backs the cache) > can be reached via instance (server side) so it looks to be a valid setup to > user 0 TTL. This makes it unreasonable to not allow the value at this point. > Note: `EmbeddedHazelcastCacheManager` has currently no opportunity to set > such a Max Size Policy. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12497) Upgrade snowflake-jdbc to 3.14.4
[
https://issues.apache.org/jira/browse/NIFI-12497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12497:
---
Status: Patch Available (was: In Progress)
> Upgrade snowflake-jdbc to 3.14.4
>
>
> Key: NIFI-12497
> URL: https://issues.apache.org/jira/browse/NIFI-12497
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{snowflake-jdbc:3.14.4}} is supposed to fix issue with Java 21
> ([SNOW-917458|https://github.com/snowflakedb/snowflake-jdbc/issues/1512],
> NIFI-10883).
> Compatibility with {{snowflake-ingest-sdk}} has to be checked also.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12497) Upgrade snowflake-jdbc to 3.14.4
Peter Turcsanyi created NIFI-12497:
--
Summary: Upgrade snowflake-jdbc to 3.14.4
Key: NIFI-12497
URL: https://issues.apache.org/jira/browse/NIFI-12497
Project: Apache NiFi
Issue Type: Improvement
Reporter: Peter Turcsanyi
Assignee: Peter Turcsanyi
{{snowflake-jdbc:3.14.4}} is supposed to fix issue with Java 21
([SNOW-917458|https://github.com/snowflakedb/snowflake-jdbc/issues/1512],
NIFI-10883).
Compatibility with {{snowflake-ingest-sdk}} has to be checked also.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-11992) Processor and sink service for filing tickets in Zendesk
[ https://issues.apache.org/jira/browse/NIFI-11992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi resolved NIFI-11992. Fix Version/s: 1.25.0 2.0.0 Resolution: Fixed > Processor and sink service for filing tickets in Zendesk > > > Key: NIFI-11992 > URL: https://issues.apache.org/jira/browse/NIFI-11992 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Fix For: 1.25.0, 2.0.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > Create a put processor and a sink service which is capable of creating > tickets in Zendesk. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12412) ConsumeAzureEventHub should support Blob checkpoint store access via proxy
[ https://issues.apache.org/jira/browse/NIFI-12412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12412: --- Status: Patch Available (was: Open) > ConsumeAzureEventHub should support Blob checkpoint store access via proxy > -- > > Key: NIFI-12412 > URL: https://issues.apache.org/jira/browse/NIFI-12412 > Project: Apache NiFi > Issue Type: Bug >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > NIFI-6240 introduced proxy support for AzureEventHub processors and applied > the proxy configuration for the Event Hubs API calls. ConsumeAzureEventHub > also needs access to Blob Storage for storing checkpoints. The proxy > configuration should be used for the Blob API calls too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12412) ConsumeAzureEventHub should support Blob checkpoint store access via proxy
Peter Turcsanyi created NIFI-12412: -- Summary: ConsumeAzureEventHub should support Blob checkpoint store access via proxy Key: NIFI-12412 URL: https://issues.apache.org/jira/browse/NIFI-12412 Project: Apache NiFi Issue Type: Bug Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi NIFI-6240 introduced proxy support for AzureEventHub processors and applied the proxy configuration for the Event Hubs API calls. ConsumeAzureEventHub also needs access to Blob Storage for storing checkpoints. The proxy configuration should be used for the Blob API calls too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12364) Upgrade snowflake-ingest-sdk to 2.0.4 and snowflake-jdbc to 3.14.3
[ https://issues.apache.org/jira/browse/NIFI-12364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12364: --- Summary: Upgrade snowflake-ingest-sdk to 2.0.4 and snowflake-jdbc to 3.14.3 (was: Upgrade snowflake-ingest-sdk to 2.0.4) > Upgrade snowflake-ingest-sdk to 2.0.4 and snowflake-jdbc to 3.14.3 > -- > > Key: NIFI-12364 > URL: https://issues.apache.org/jira/browse/NIFI-12364 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.23.2 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > Upgrade snowflake-ingest-sdk to 2.0.4 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12364) Upgrade snowflake-ingest-sdk to 2.0.4 and snowflake-jdbc to 3.14.3
[ https://issues.apache.org/jira/browse/NIFI-12364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12364: --- Description: Upgrade snowflake-ingest-sdk to 2.0.4 and snowflake-jdbc to 3.14.3 (was: Upgrade snowflake-ingest-sdk to 2.0.4) > Upgrade snowflake-ingest-sdk to 2.0.4 and snowflake-jdbc to 3.14.3 > -- > > Key: NIFI-12364 > URL: https://issues.apache.org/jira/browse/NIFI-12364 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.23.2 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > Upgrade snowflake-ingest-sdk to 2.0.4 and snowflake-jdbc to 3.14.3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12306) ConsumeAzureEventHub should not log partition ownership changes at error level
[
https://issues.apache.org/jira/browse/NIFI-12306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12306:
---
Fix Version/s: 1.latest
2.latest
> ConsumeAzureEventHub should not log partition ownership changes at error level
> --
>
> Key: NIFI-12306
> URL: https://issues.apache.org/jira/browse/NIFI-12306
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Minor
> Fix For: 1.latest, 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Azure Event Hub client instances run a load balancing algorithm which checks
> the distribution of the partitions among the consumer group members and can
> rebalance the partition assignments.
> This algorithm runs independently on each client node and if the node owns
> less partitions than the fair share (typically after start-up), it "steals" a
> partition from another node.
> The other node only recognizes it via the broken connection (2 consumers from
> the same consumer group cannot consume the same partition at the same time)
> so basically it is an error event on that side. It seems it is by design and
> will not change in the Azure client lib.
> However, we can handle this type of "error" differently on the NiFi side and
> do not log it at error level but only info. The current error bulletin is
> quite confusing for the user.
> Error message:
> {code:java}
> 2023-11-01 13:19:46,613 ERROR [partition-pump-1-22]
> o.a.n.p.a.eventhub.ConsumeAzureEventHub
> ConsumeAzureEventHub[id=7d57cb75-018b-1000--42a1ec17] Receive Events
> failed Namespace [turcsanyi-eventhub-ns.servicebus.windows.net] Event Hub
> [turcsanyi-eventhub-2] Consumer Group [$default] Partition [1]
> com.azure.core.amqp.exception.AmqpException: New receiver
> '7661792b-acf1-49b8-9474-2ac06ebc22d9' with higher epoch of '0' is created
> hence current receiver '8bec0a38-2d58-46d1-ac37-4cb80bd7ba29' with epoch '0'
> is getting disconnected. If you are recreating the receiver, make sure a
> higher epoch is used.
> TrackingId:ddb1571668bf0010179a6542425c_G13S3_B13S1,
> SystemTracker:turcsanyi-eventhub-ns:eventhub:turcsanyi-eventhub-2~16383|$default,
> Timestamp:2023-11-01T12:19:46, errorContext[NAMESPACE:
> turcsanyi-eventhub-ns.servicebus.windows.net. ERROR CONTEXT: N/A, PATH:
> turcsanyi-eventhub-2/ConsumerGroups/$default/Partitions/1, REFERENCE_ID:
> 1_b1a39c_1698841180105, LINK_CREDIT: 300]
> at
> com.azure.core.amqp.implementation.ExceptionUtil.toException(ExceptionUtil.java:85)
> at
> com.azure.core.amqp.implementation.handler.LinkHandler.handleRemoteLinkClosed(LinkHandler.java:124)
> ...
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12306) ConsumeAzureEventHub should not log partition ownership changes at error level
[
https://issues.apache.org/jira/browse/NIFI-12306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12306:
---
Status: Patch Available (was: Open)
> ConsumeAzureEventHub should not log partition ownership changes at error level
> --
>
> Key: NIFI-12306
> URL: https://issues.apache.org/jira/browse/NIFI-12306
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Minor
> Fix For: 1.latest, 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Azure Event Hub client instances run a load balancing algorithm which checks
> the distribution of the partitions among the consumer group members and can
> rebalance the partition assignments.
> This algorithm runs independently on each client node and if the node owns
> less partitions than the fair share (typically after start-up), it "steals" a
> partition from another node.
> The other node only recognizes it via the broken connection (2 consumers from
> the same consumer group cannot consume the same partition at the same time)
> so basically it is an error event on that side. It seems it is by design and
> will not change in the Azure client lib.
> However, we can handle this type of "error" differently on the NiFi side and
> do not log it at error level but only info. The current error bulletin is
> quite confusing for the user.
> Error message:
> {code:java}
> 2023-11-01 13:19:46,613 ERROR [partition-pump-1-22]
> o.a.n.p.a.eventhub.ConsumeAzureEventHub
> ConsumeAzureEventHub[id=7d57cb75-018b-1000--42a1ec17] Receive Events
> failed Namespace [turcsanyi-eventhub-ns.servicebus.windows.net] Event Hub
> [turcsanyi-eventhub-2] Consumer Group [$default] Partition [1]
> com.azure.core.amqp.exception.AmqpException: New receiver
> '7661792b-acf1-49b8-9474-2ac06ebc22d9' with higher epoch of '0' is created
> hence current receiver '8bec0a38-2d58-46d1-ac37-4cb80bd7ba29' with epoch '0'
> is getting disconnected. If you are recreating the receiver, make sure a
> higher epoch is used.
> TrackingId:ddb1571668bf0010179a6542425c_G13S3_B13S1,
> SystemTracker:turcsanyi-eventhub-ns:eventhub:turcsanyi-eventhub-2~16383|$default,
> Timestamp:2023-11-01T12:19:46, errorContext[NAMESPACE:
> turcsanyi-eventhub-ns.servicebus.windows.net. ERROR CONTEXT: N/A, PATH:
> turcsanyi-eventhub-2/ConsumerGroups/$default/Partitions/1, REFERENCE_ID:
> 1_b1a39c_1698841180105, LINK_CREDIT: 300]
> at
> com.azure.core.amqp.implementation.ExceptionUtil.toException(ExceptionUtil.java:85)
> at
> com.azure.core.amqp.implementation.handler.LinkHandler.handleRemoteLinkClosed(LinkHandler.java:124)
> ...
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
