[jira] [Commented] (NIFI-7296) BST TimeZone parsing fails, breaking webgui and API
[
https://issues.apache.org/jira/browse/NIFI-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103006#comment-17103006
]
Dmitry Ibragimov commented on NIFI-7296:
I've finally found workaround to add
{code:java}
java.arg.10=-Djava.locale.providers=COMPAT
{code}
string to bootstrap.conf
> BST TimeZone parsing fails, breaking webgui and API
> ---
>
> Key: NIFI-7296
> URL: https://issues.apache.org/jira/browse/NIFI-7296
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Affects Versions: 1.11.3
> Environment: Nifi 1.11.3 running on
> jre-11-openjdk-11.0.4.11-1.el7_7.x86_64 and RHEL 8
> cluster of 6 servers
>Reporter: Michael Percival
>Priority: Blocker
>
> Since clocks have changed in the UK and we have moved to BST, API calls and
> browsing to the web gui fails with a 'An unexpected error has occurred.
> Please check the logs for additional details.' error. reviewing the
> nifi-user.log shows the below when attempting to access the webgui, appears
> the timezone is not being parsed properly by the web server, see below:
> Caused by: java.time.format.DateTimeParseException: Text '12:23:17 BST' could
> not be parsed: null
> at
> java.base/java.time.format.DateTimeFormatter.createError(DateTimeFormatter.java:2017)
> at
> java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1952)
> at java.base/java.time.LocalDateTime.parse(LocalDateTime.java:492)
> at
> org.apache.nifi.web.api.dto.util.TimeAdapter.unmarshal(TimeAdapter.java:55)
> at
> org.apache.nifi.web.api.dto.util.TimeAdapter.unmarshal(TimeAdapter.java:33)
> at
> com.fasterxml.jackson.module.jaxb.AdapterConverter.convert(AdapterConverter.java:35)
> at
> com.fasterxml.jackson.databind.deser.std.StdDelegatingDeserializer.convertValue(StdDelegatingDeserializ$
> at
> com.fasterxml.jackson.databind.deser.std.StdDelegatingDeserializer.deserialize(StdDelegatingDeserialize$
> at
> com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)
> ... 122 common frames omitted
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7296) BST TimeZone parsing fails, breaking webgui and API
[
https://issues.apache.org/jira/browse/NIFI-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102962#comment-17102962
]
Dmitry Ibragimov commented on NIFI-7296:
I faced with the same issue with MSK timezone:
CentOS 7.8.2003, java-11-openjdk-11.0.7.10-4.el7_8.x86_64, NiFi 11.4.1 secured
cluster after login UI shows error:
An unexpected error has occurred. Please check the logs for additional details.
In nifi-user.log:
{code:java}
2020-05-09 00:37:41,805 ERROR [NiFi Web Server-25]
o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
java.lang.RuntimeException: Failed to parse response as entity
of type class org.apache.nifi.web.api.entity.FlowConfigurationEntity. Returning
Internal Server Error response.
java.lang.RuntimeException: Failed to parse response as entity of type class
org.apache.nifi.web.api.entity.FlowConfigurationEntity
at
org.apache.nifi.cluster.coordination.http.replication.okhttp.JacksonResponse.readEntity(JacksonResponse.java:103)
at
org.apache.nifi.cluster.coordination.http.endpoints.AbstractSingleDTOEndpoint.merge(AbstractSingleDTOEndpoint.java:38)
at
org.apache.nifi.cluster.coordination.http.StandardHttpResponseMapper.mapResponses(StandardHttpResponseMapper.java:212)
at
org.apache.nifi.cluster.coordination.http.replication.StandardAsyncClusterResponse.getMergedResponse(StandardAsyncClusterResponse.java:219)
at
org.apache.nifi.cluster.coordination.http.replication.StandardAsyncClusterResponse.awaitMergedResponse(StandardAsyncClusterResponse.java:236)
at
org.apache.nifi.web.api.ApplicationResource.replicateNodeResponse(ApplicationResource.java:1102)
at
org.apache.nifi.web.api.ApplicationResource.replicateNodeResponse(ApplicationResource.java:1074)
at
org.apache.nifi.web.api.ApplicationResource.replicate(ApplicationResource.java:1065)
at
org.apache.nifi.web.api.ApplicationResource.replicate(ApplicationResource.java:1049)
at
org.apache.nifi.web.api.ApplicationResource.replicate(ApplicationResource.java:1024)
at
org.apache.nifi.web.api.FlowResource.getFlowConfig(FlowResource.java:294)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191)
at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104)
at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268)
at org.glassfish.jersey.internal.Errors.process(Errors.java:316)
at org.glassfish.jersey.internal.Errors.process(Errors.java:298)
at org.glassfish.jersey.internal.Errors.process(Errors.java:268)
at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289)
at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256)
at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703)
at
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416)
at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:876)
at
[GitHub] [nifi] markap14 commented on a change in pull request #4232: NIFI-7392: Initial commit for the ValidateJSON processor
markap14 commented on a change in pull request #4232:
URL: https://github.com/apache/nifi/pull/4232#discussion_r422373363
##
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
##
@@ -0,0 +1,215 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+
+import com.fasterxml.jackson.core.JsonParseException;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.networknt.schema.JsonSchema;
+import com.networknt.schema.JsonSchemaFactory;
+import com.networknt.schema.ValidationMessage;
+import com.networknt.schema.SpecVersion.VersionFlag;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"JSON", "schema", "validation"})
+@WritesAttributes({
+@WritesAttribute(attribute = "validatejson.invalid.error", description =
"If the flow file is routed to the invalid relationship "
++ "the attribute will contain the error message resulting from the
validation failure.")
+})
+@CapabilityDescription("Validates the contents of FlowFiles against a
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final String ERROR_ATTRIBUTE_KEY =
"validatejson.invalid.error";
+
+public static final AllowableValue SCHEMA_VERSION_4 = new
AllowableValue("V4");
+public static final AllowableValue SCHEMA_VERSION_6 = new
AllowableValue("V6");
+public static final AllowableValue SCHEMA_VERSION_7 = new
AllowableValue("V7");
+public static final AllowableValue SCHEMA_VERSION_V201909 = new
AllowableValue("V201909");
+
+public static final PropertyDescriptor SCHEMA_VERSION = new
PropertyDescriptor
+.Builder().name("SCHEMA_VERSION")
+.displayName("Schema Version")
+.description("The JSON schema specification")
+.required(true)
+.allowableValues(SCHEMA_VERSION_4, SCHEMA_VERSION_6, SCHEMA_VERSION_7,
SCHEMA_VERSION_V201909)
+.defaultValue(SCHEMA_VERSION_V201909.getValue())
+.build();
+
+public static final PropertyDescriptor SCHEMA_TEXT = new PropertyDescriptor
+.Builder().name("SCHEMA_TEXT")
+.displayName("Schema Text")
+.description("The text of a JSON schema")
+.required(true)
+.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully
[jira] [Created] (NIFI-7435) JoltTransformReccord throws ClassCastException
Pavel created NIFI-7435:
---
Summary: JoltTransformReccord throws ClassCastException
Key: NIFI-7435
URL: https://issues.apache.org/jira/browse/NIFI-7435
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.11.4
Reporter: Pavel
Attachments: Snímek z 2020-05-08 21-33-08.png
I would like to implement a user case as shown in the picture in the attachment:
!Snímek z 2020-05-08 21-33-08.png|width=936,height=181!
If the records are non-empty, everything works.
However, if I first send first empty and then filled in, an exception is
raised. In reverse order it works.
Even manual schema setup for the input JsonTreeReader did not help.
Test use case:
input flowfile:
{code:java}
[{
"CODE_ONE":null,
"CODE_TWO":null
},
{
"CODE_ONE":"a",
"CODE_TWO":null
}]{code}
{{jolt:}}
{code:java}
[
{
"operation": "shift",
"spec": {
"CODE_*": "CODES",
"*": "&"
}
}
]
{code}
log:
JoltTransformRecord[id=f596e13d-0171-1000-7957-5abd51aa5ea0] Failed to write
MapRecord[\{CODES=[Ljava.lang.Object;@6589e3eb}] with schema ["CODES" :
"ARRAY[STRING]"] as a JSON Object due to java.lang.ClassCastException: null
JoltTransformRecord[id=f596e13d-0171-1000-7957-5abd51aa5ea0] Unable to
transform
StandardFlowFileRecord[uuid=db74cbdf-673b-4af4-ae3e-cab5915aa46e,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1588966927548-947, container=default,
section=947], offset=766896,
length=120],offset=0,name=db74cbdf-673b-4af4-ae3e-cab5915aa46e,size=120] due to
java.lang.ClassCastException: null
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] tpalfy commented on a change in pull request #4223: NIFI-7369 Adding big decimal support for record handling in order to avoid missing precision when reading in records
tpalfy commented on a change in pull request #4223:
URL: https://github.com/apache/nifi/pull/4223#discussion_r422237710
##
File path:
nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-avro-record-utils/src/main/java/org/apache/nifi/avro/AvroTypeUtil.java
##
@@ -256,6 +256,12 @@ private static Schema buildAvroSchema(final DataType
dataType, final String fiel
case LONG:
schema = Schema.create(Type.LONG);
break;
+case BIGDECIMAL:
+// One more byte than below to allow the dot in the string
representation
+schema = Schema.createFixed(fieldName + "Type", null,
"org.apache.nifi",39);
Review comment:
Not sure where the concept that this should be a fixed sized type (with
namespace) comes from.
Also where did the 38 maximum come from? I remember seeing such a max in
ORC, but not sure about Avro.
##
File path:
nifi-nar-bundles/nifi-hive-bundle/nifi-hive3-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java
##
@@ -284,6 +290,11 @@ public static TypeInfo getOrcField(DataType dataType,
boolean hiveFieldNames) th
|| RecordFieldType.STRING.equals(fieldType)) {
return getPrimitiveOrcTypeFromPrimitiveFieldType(dataType);
}
+
+if (RecordFieldType.BIGDECIMAL.equals(fieldType)) {
+// 38 is the maximum allowed precision and 19 digit is needed to
represent long values
Review comment:
Not sure I understand these constraints. With this, no bigger than 19
digit numbers (apart from decimal part) can be used.
All-in-all I'm wondering if the `RecordFieldType.BIGDECIMAL` can stay as it
is, without being able to convey precision and scale information.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-6925) JoltTransformRecord, PutESHttpRecord, PutHive3Streaming fail with various RecordReaders
[ https://issues.apache.org/jira/browse/NIFI-6925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel updated NIFI-6925: Attachment: (was: Snímek z 2020-05-08 21-26-27.png) > JoltTransformRecord, PutESHttpRecord, PutHive3Streaming fail with various > RecordReaders > --- > > Key: NIFI-6925 > URL: https://issues.apache.org/jira/browse/NIFI-6925 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.11.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > When using JoltTransformRecord with readers such as AvroReader or CSVReader, > an error occurs when trying to transfer the original input flowfile to the > "original" relationship. The error indicates that it is reading from multiple > callbacks or that the input stream is still open. It happens to work with > JsonTreeReader because the underlying JSON parser closes the stream when no > more records are available. > This is a result of not explicitly closing the input stream before trying to > transfer the input flow file. It was not caught in the unit tests because > MockProcessSession doesn't keep track of the open reader count, so that > should be fixed as well. Fixing MockProcessSession will cause > TestJoltTransformRecord to fail before the fix for this case is applied. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6925) JoltTransformRecord, PutESHttpRecord, PutHive3Streaming fail with various RecordReaders
[ https://issues.apache.org/jira/browse/NIFI-6925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel updated NIFI-6925: Attachment: Snímek z 2020-05-08 21-26-27.png > JoltTransformRecord, PutESHttpRecord, PutHive3Streaming fail with various > RecordReaders > --- > > Key: NIFI-6925 > URL: https://issues.apache.org/jira/browse/NIFI-6925 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.11.0 > > Attachments: Snímek z 2020-05-08 21-26-27.png > > Time Spent: 1h 10m > Remaining Estimate: 0h > > When using JoltTransformRecord with readers such as AvroReader or CSVReader, > an error occurs when trying to transfer the original input flowfile to the > "original" relationship. The error indicates that it is reading from multiple > callbacks or that the input stream is still open. It happens to work with > JsonTreeReader because the underlying JSON parser closes the stream when no > more records are available. > This is a result of not explicitly closing the input stream before trying to > transfer the input flow file. It was not caught in the unit tests because > MockProcessSession doesn't keep track of the open reader count, so that > should be fixed as well. Fixing MockProcessSession will cause > TestJoltTransformRecord to fail before the fix for this case is applied. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] alopresto commented on a change in pull request #4192: NIFI-7321 - Allow NiFi admins to configure whether Jetty will send th…
alopresto commented on a change in pull request #4192:
URL: https://github.com/apache/nifi/pull/4192#discussion_r422300486
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java
##
@@ -908,6 +909,7 @@ private ServerConnector
createUnconfiguredSslServerConnector(Server server, Http
final HttpConfiguration httpsConfiguration = new
HttpConfiguration(httpConfiguration);
httpsConfiguration.setSecureScheme("https");
httpsConfiguration.setSecurePort(port);
+
httpConfiguration.setSendServerVersion(props.shouldSendServerVersion());
Review comment:
Why is this variable `httpConfiguration` vs. `httpsConfiguration`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-registry] alopresto commented on pull request #277: NIFIREG-387 - Upgraded spring boot and some additional depedencies re…
alopresto commented on pull request #277: URL: https://github.com/apache/nifi-registry/pull/277#issuecomment-625950245 @kevdoran did you need another look at this or were updating the `NOTICE` file and going to `2.10.3` the only reservations you had? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFIREG-382) Upgrade minimist version
[ https://issues.apache.org/jira/browse/NIFIREG-382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman resolved NIFIREG-382. - Fix Version/s: 1.0.0 Resolution: Fixed > Upgrade minimist version > > > Key: NIFIREG-382 > URL: https://issues.apache.org/jira/browse/NIFIREG-382 > Project: NiFi Registry > Issue Type: Improvement >Affects Versions: 0.6.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Major > Labels: dependencies > Fix For: 1.0.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Upgrade the version of minimist to 1.2.3 or 0.2.1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-registry] mcgilman commented on pull request #275: NIFIREG-382 Upgraded versions of dependencies.
mcgilman commented on pull request #275: URL: https://github.com/apache/nifi-registry/pull/275#issuecomment-625932231 This has been handled by #278 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-registry] mcgilman commented on pull request #278: [NIFIREG-382] upgrade frontend deps
mcgilman commented on pull request #278: URL: https://github.com/apache/nifi-registry/pull/278#issuecomment-625931752 Thanks for the PR @scottyaslan! Thanks for the review @mtien-apache! This has been merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-registry] mcgilman commented on pull request #278: [NIFIREG-382] upgrade frontend deps
mcgilman commented on pull request #278: URL: https://github.com/apache/nifi-registry/pull/278#issuecomment-625926219 Will review... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7434) Support override endpointSuffix in PutAzureBlobStorage NiFi processor
[ https://issues.apache.org/jira/browse/NIFI-7434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102756#comment-17102756 ] Muazma Zahid commented on NIFI-7434: [~sjyang18] I think its better if we can update the suffixendpoint for all AzureBlobStorage processors together, you can edit the title accordingly. > Support override endpointSuffix in PutAzureBlobStorage NiFi processor > - > > Key: NIFI-7434 > URL: https://issues.apache.org/jira/browse/NIFI-7434 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Seokwon Yang >Assignee: Seokwon Yang >Priority: Major > > In order to support different Azure cloud environments (e.g., GovCloud, Azure > Stack) and testing, the PutAzureBlobStorage processor should support > overriding the endpoint to something other than blob.core.windows.net -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] KuKuDeCheng commented on pull request #4243: NIFI-7410 Fix CLOB would be unreadable code in ExecuteSql or QueryDataBaseTable
KuKuDeCheng commented on pull request #4243: URL: https://github.com/apache/nifi/pull/4243#issuecomment-625922802 > Is there a way to add a unit test for this? Maybe inserting Chinese characters into a Derby table with a CLOB column and reading the results? Thanks for reply. I will add a unit test for this situation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-7423) Upgrade jquery version
[ https://issues.apache.org/jira/browse/NIFI-7423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman resolved NIFI-7423. --- Fix Version/s: 1.12.0 Resolution: Fixed > Upgrade jquery version > -- > > Key: NIFI-7423 > URL: https://issues.apache.org/jira/browse/NIFI-7423 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.11.4 >Reporter: M Tien >Assignee: M Tien >Priority: Major > Labels: dependency > Fix For: 1.12.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Upgrade dependency version of jquery. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mcgilman commented on pull request #4258: NIFI-7423 Upgraded jquery dependency version.
mcgilman commented on pull request #4258: URL: https://github.com/apache/nifi/pull/4258#issuecomment-625922140 Thanks for the PR @mtien-apache! Thanks for the review @andrewmlim! This has been merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7423) Upgrade jquery version
[ https://issues.apache.org/jira/browse/NIFI-7423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102752#comment-17102752 ] ASF subversion and git services commented on NIFI-7423: --- Commit 647a9a60b73794106d70941e1ab62f7f25040350 in nifi's branch refs/heads/master from mtien [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=647a9a6 ] NIFI-7423 Upgraded jquery dependency version. NIFI-7423 Upgraded jquery dependency version to latest 3.5.1. This closes #4258 > Upgrade jquery version > -- > > Key: NIFI-7423 > URL: https://issues.apache.org/jira/browse/NIFI-7423 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.11.4 >Reporter: M Tien >Assignee: M Tien >Priority: Major > Labels: dependency > Time Spent: 1h > Remaining Estimate: 0h > > Upgrade dependency version of jquery. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7423) Upgrade jquery version
[ https://issues.apache.org/jira/browse/NIFI-7423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102751#comment-17102751 ] ASF subversion and git services commented on NIFI-7423: --- Commit 647a9a60b73794106d70941e1ab62f7f25040350 in nifi's branch refs/heads/master from mtien [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=647a9a6 ] NIFI-7423 Upgraded jquery dependency version. NIFI-7423 Upgraded jquery dependency version to latest 3.5.1. This closes #4258 > Upgrade jquery version > -- > > Key: NIFI-7423 > URL: https://issues.apache.org/jira/browse/NIFI-7423 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.11.4 >Reporter: M Tien >Assignee: M Tien >Priority: Major > Labels: dependency > Time Spent: 1h > Remaining Estimate: 0h > > Upgrade dependency version of jquery. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7390) Avro writing fails when narrow data type arrives
[ https://issues.apache.org/jira/browse/NIFI-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-7390: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Avro writing fails when narrow data type arrives > > > Key: NIFI-7390 > URL: https://issues.apache.org/jira/browse/NIFI-7390 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions > Environment: Nifi 1.9 but probably other versions >Reporter: Simon Bence >Assignee: Simon Bence >Priority: Major > Labels: Record > Fix For: 1.12.0 > > Time Spent: 20m > Remaining Estimate: 0h > > We expect a Long during writing the Avro record, but try to write in an > Integer and as there is no inheritance between them, it fails with class cast > exception. > The reason we try to write Integer: the incoming number from JSON is parsed > by Jackson and unmarshalled based on the value, not the expected schema. If > the actual value fits to Integer, it will use that, otherwise it will be a > Long. It has no knowledge of the expected schema. In a later point when we > consider the input valid (we do allow “narrow number types” both in strict > and non strict validation), the Avro specific writer thinks it a Long based > on the validation schema and fails to write it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7390) Avro writing fails when narrow data type arrives
[ https://issues.apache.org/jira/browse/NIFI-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102749#comment-17102749 ] ASF subversion and git services commented on NIFI-7390: --- Commit 66b175f405e727aba650f67872292ff8c8ccb14d in nifi's branch refs/heads/master from Bence Simon [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=66b175f ] NIFI-7390 Covering Avro type conversion in case of map withing Record Signed-off-by: Matthew Burgess This closes #4256 > Avro writing fails when narrow data type arrives > > > Key: NIFI-7390 > URL: https://issues.apache.org/jira/browse/NIFI-7390 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions > Environment: Nifi 1.9 but probably other versions >Reporter: Simon Bence >Assignee: Simon Bence >Priority: Major > Labels: Record > Fix For: 1.12.0 > > Time Spent: 20m > Remaining Estimate: 0h > > We expect a Long during writing the Avro record, but try to write in an > Integer and as there is no inheritance between them, it fails with class cast > exception. > The reason we try to write Integer: the incoming number from JSON is parsed > by Jackson and unmarshalled based on the value, not the expected schema. If > the actual value fits to Integer, it will use that, otherwise it will be a > Long. It has no knowledge of the expected schema. In a later point when we > consider the input valid (we do allow “narrow number types” both in strict > and non strict validation), the Avro specific writer thinks it a Long based > on the validation schema and fails to write it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mattyb149 commented on pull request #4256: NIFI-7390 Covering Avro type conversion in case of map withing Record
mattyb149 commented on pull request #4256: URL: https://github.com/apache/nifi/pull/4256#issuecomment-625920588 +1 LGTM, ran contrib-check and a few tests on a live NiFi (with ValidateRecord and ConvertRecord). Thanks for the fix! Merging to master This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-7434) Support override endpointSuffix in PutAzureBlobStorage NiFi processor
Seokwon Yang created NIFI-7434: -- Summary: Support override endpointSuffix in PutAzureBlobStorage NiFi processor Key: NIFI-7434 URL: https://issues.apache.org/jira/browse/NIFI-7434 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Seokwon Yang In order to support different Azure cloud environments (e.g., GovCloud, Azure Stack) and testing, the PutAzureBlobStorage processor should support overriding the endpoint to something other than blob.core.windows.net -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-7434) Support override endpointSuffix in PutAzureBlobStorage NiFi processor
[ https://issues.apache.org/jira/browse/NIFI-7434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seokwon Yang reassigned NIFI-7434: -- Assignee: Seokwon Yang > Support override endpointSuffix in PutAzureBlobStorage NiFi processor > - > > Key: NIFI-7434 > URL: https://issues.apache.org/jira/browse/NIFI-7434 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Seokwon Yang >Assignee: Seokwon Yang >Priority: Major > > In order to support different Azure cloud environments (e.g., GovCloud, Azure > Stack) and testing, the PutAzureBlobStorage processor should support > overriding the endpoint to something other than blob.core.windows.net -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7433) Update NiFi Toolkit docs to include new commands from 1.11.x
Andrew Grande created NIFI-7433: --- Summary: Update NiFi Toolkit docs to include new commands from 1.11.x Key: NIFI-7433 URL: https://issues.apache.org/jira/browse/NIFI-7433 Project: Apache NiFi Issue Type: Task Reporter: Andrew Grande https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#nifi_CLI Missing parameter context commands, maybe a few more new ones, needs a refresh. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-7312) Search function does not work for variable registry in root process group
[ https://issues.apache.org/jira/browse/NIFI-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Gyori reassigned NIFI-7312: - Assignee: Peter Gyori (was: Simon Bence) > Search function does not work for variable registry in root process group > - > > Key: NIFI-7312 > URL: https://issues.apache.org/jira/browse/NIFI-7312 > Project: Apache NiFi > Issue Type: Bug >Reporter: Peter Turcsanyi >Assignee: Peter Gyori >Priority: Major > > https://github.com/apache/nifi/pull/4123#discussion_r401883313 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] jfrazee commented on a change in pull request #4164: NIFI-5481 - Add new providers of protected sensitive configuration values
jfrazee commented on a change in pull request #4164: URL: https://github.com/apache/nifi/pull/4164#discussion_r422196412 ## File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-properties-loader/src/main/java/org/apache/nifi/properties/sensitive/SensitivePropertyProvider.java ## @@ -14,10 +14,20 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.nifi.properties; +package org.apache.nifi.properties.sensitive; Review comment: I agree that the package layout is clearer (esp. in the context of the other work). And I'm skeptical that this would have been an extension point previously since you'd have to have changed other parts of the codebase, but nonetheless they're public. What are your thoughts on leaving the old stuff in place and deprecating them for a few releases? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #777: MINIFICPP-1216 - Controller Services Integration test is unstable
szaszm commented on a change in pull request #777:
URL: https://github.com/apache/nifi-minifi-cpp/pull/777#discussion_r422195976
##
File path: libminifi/src/FlowController.cpp
##
@@ -248,9 +248,9 @@ int16_t FlowController::stop(bool force, uint64_t
timeToWait) {
this->timer_scheduler_->stop();
this->event_scheduler_->stop();
this->cron_scheduler_->stop();
+thread_pool_.shutdown();
Review comment:
Given the above discussion, the changes look reasonable to me. However,
the discussion revealed that this code has numerous hidden interconnections:
processors may run after initiating stop => both repositories and controller
services need to be stopped after the thread pool, controller services need to
be stopped because of db locking, affected C2 which I don't understand in
detail.
Could you add a code comment mentioning the existence of these to the future
reader/developer? While not all of it is strictly in the scope of this fix, I
think the value/effort ratio would be outstanding here, making it worth doing
IMO.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (MINIFICPP-1218) C2 metrics simplification introduced an undefined and unused member function
Marton Szasz created MINIFICPP-1218:
---
Summary: C2 metrics simplification introduced an undefined and
unused member function
Key: MINIFICPP-1218
URL: https://issues.apache.org/jira/browse/MINIFICPP-1218
Project: Apache NiFi MiNiFi C++
Issue Type: Bug
Reporter: Marton Szasz
Assignee: Marton Szasz
{{FlowController::getAgentInformation()}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7432) Replace method in Redis DMC client service does not set TTL
Bryan Bende created NIFI-7432: - Summary: Replace method in Redis DMC client service does not set TTL Key: NIFI-7432 URL: https://issues.apache.org/jira/browse/NIFI-7432 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.11.4 Reporter: Bryan Bende The Redis DMC client service has a TTL property that was added after the initial implementation. The "replace" method does not set the TTL, but should be. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7243) SFTP Processors - Exhausted available authentication methods
[ https://issues.apache.org/jira/browse/NIFI-7243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102585#comment-17102585 ] Nathan Gough commented on NIFI-7243: Just to confirm, you were unable to use NiFI to list files on test.rebex.net:22 with username=demo and password=password? > SFTP Processors - Exhausted available authentication methods > > > Key: NIFI-7243 > URL: https://issues.apache.org/jira/browse/NIFI-7243 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.10.0, 1.11.0, 1.11.1, 1.11.3 > Environment: PRO >Reporter: Raúl >Assignee: Nathan Gough >Priority: Critical > Labels: SFTP, listsftp > Attachments: Screen Shot 2020-05-07 at 5.28.29 PM.png > > > In version 1.8 this works correctly. In superiors no. > > Any of the processors: ListSFTP, FetchSFTP, GetSFTP, PutSFTP when connecting > to an sftp shows the error: > 2020-03-10 15:12:51,596 ERROR [Timer-Driven Process Thread-3] > o.a.nifi.processors.standard.ListSFTP > ListSFTP[id=af72555e-0170-1000-c80d-7e992c9de544] Failed to perform listing > on remote host due to Exhausted available authentication methods: > net.schmizz.sshj.userauth.UserAuthException: Exhausted available > authentication methods > net.schmizz.sshj.userauth.UserAuthException: Exhausted available > authentication methods > at net.schmizz.sshj.SSHClient.auth(SSHClient.java:230) > > These tests that I commented have been made from nifi, both in windows and > linux environment and fails in both. > 1 - Strict Host Key Checking: false > 2 - Strict Host Key Checking: true > Host Key File: C:\Users\usuario\.ssh\known_hosts --> (ssh-rsa, > ssh-dss and ecdsa-sha2-nistp256) > > From the machines, both in windows and in linux, if an sftp is made from the > command line it connects without problems. But with Nifi he fails to connect. > > Any idea about how can I resolve this error? > > Thank you in advance, > log > - > 2020-03-10 15:12:51,596 ERROR [Timer-Driven Process Thread-3] > o.a.nifi.processors.standard.ListSFTP > ListSFTP[id=af72555e-0170-1000-c80d-7e992c9de544] Failed to perform listing > on remote host due to Exhausted available authentication methods: > net.schmizz.sshj.userauth.UserAuthException: Exhausted available > authentication methods2020-03-10 15:12:51,596 ERROR [Timer-Driven Process > Thread-3] o.a.nifi.processors.standard.ListSFTP > ListSFTP[id=af72555e-0170-1000-c80d-7e992c9de544] Failed to perform listing > on remote host due to Exhausted available authentication methods: > net.schmizz.sshj.userauth.UserAuthException: Exhausted available > authentication methodsnet.schmizz.sshj.userauth.UserAuthException: Exhausted > available authentication methods at > net.schmizz.sshj.SSHClient.auth(SSHClient.java:230) at > org.apache.nifi.processors.standard.util.SFTPTransfer.getSFTPClient(SFTPTransfer.java:606) > at > org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:233) > at > org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:196) > at > org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:106) > at > org.apache.nifi.processors.standard.ListSFTP.performListing(ListSFTP.java:146) > at > org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingTimestamps(AbstractListProcessor.java:472) > at > org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:414) > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176) > at > org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213) > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) > at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748)2020-03-10 15:12:51,601 DEBUG > [Timer-Driven Process Thread-3]
[GitHub] [nifi] ashish0mittal0 commented on pull request #4261: NIFI-7424: PutSQL updated, commit and rollback allowed only if autocommit set false for the connection
ashish0mittal0 commented on pull request #4261: URL: https://github.com/apache/nifi/pull/4261#issuecomment-625789213 Please suggest how may I get the pull request merged into the master, since the failure of the workflow is not related to the code change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (MINIFICPP-1217) RPG should configure http client with reasonable timeouts
Arpad Boda created MINIFICPP-1217: - Summary: RPG should configure http client with reasonable timeouts Key: MINIFICPP-1217 URL: https://issues.apache.org/jira/browse/MINIFICPP-1217 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Affects Versions: 0.7.0 Reporter: Arpad Boda Assignee: Adam Debreceni Fix For: 0.8.0 Currently when transmitting flowfiles via HTTP, only connection timeout is configured. Which means that RPG can hang in case the connection is interrupted. We should use proper timeouts. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] ashish0mittal0 commented on pull request #4261: NIFI-7424: PutSQL updated, commit and rollback allowed only if autocommit set false for the connection
ashish0mittal0 commented on pull request #4261: URL: https://github.com/apache/nifi/pull/4261#issuecomment-625753662 **ci-workflow / Ubuntu - JDK 1.8 FR (pull_request)** Failed, because of the following error ``` [ERROR] COMPILATION ERROR : [ERROR] error: error reading /home/runner/.m2/repository/org/apache/sis/core/sis-utility/1.0/sis-utility-1.0.jar; error in opening zip file [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project nifi-media-processors: Compilation failure [ERROR] error: error reading /home/runner/.m2/repository/org/apache/sis/core/sis-utility/1.0/sis-utility-1.0.jar; error in opening zip file [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :nifi-media-processors ##[error]Process completed with exit code 1. ``` After inspecting **Check Maven Org Cache** the following errors with warning was identified- ``` gzip: stdin: unexpected end of file /bin/tar: Unexpected EOF in archive /bin/tar: Unexpected EOF in archive /bin/tar: Error is not recoverable: exiting now [warning]Tar failed with error: The process '/bin/tar' failed with exit code 2 ``` It seems like an issue with loading repository. And the code changes also doesn't relatewith this issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (MINIFICPP-1177) Rewrite TailFile
[ https://issues.apache.org/jira/browse/MINIFICPP-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-1177: - Assignee: Ferenc Gerlits (was: Alex Marmer) > Rewrite TailFile > > > Key: MINIFICPP-1177 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1177 > Project: Apache NiFi MiNiFi C++ > Issue Type: Task >Reporter: Dániel Bakai >Assignee: Ferenc Gerlits >Priority: Major > Fix For: 0.8.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Our TailFile implementation, especially in the handling of rollover and > multiple file mode is buggy, and has significant, erroneus deviations from > the NiFi implementation. It has been patched time and time again, but it is > still not up to par. Since this is the processor used for one of our primary > business use cases, log parsing, it should be the best implementation we can > achieve. We should review how NiFi works and reimplement this processor based > on that, utilizing our recent processor design best practices. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7431) Kafka consumers don't support zstd compression type 4
Gonzalo Aguilar created NIFI-7431: - Summary: Kafka consumers don't support zstd compression type 4 Key: NIFI-7431 URL: https://issues.apache.org/jira/browse/NIFI-7431 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 1.11.4 Reporter: Gonzalo Aguilar Fix For: 1.12.0 When trying to consume from a topic compressed with zstd it breaks with an exception: Os ha dado esto a alguno? java.lang.IllegalArgumentException: Unknown compression type id: 4at org.apache.kafka.common.record.CompressionType.forId(CompressionType.java:160) at org.apache.kafka.common.record.DefaultRecordBatch.compressionType(DefaultRecordBatch.java:212) at org.apache.kafka.common.record.AbstractRecordBatch.isCompressed(AbstractRecordBatch.java:33) at org.apache.kafka.common.record.DefaultRecordBatch.isCompressed(DefaultRecordBatch.java:96) at org.apache.kafka.common.record.DefaultRecordBatch.streamingIterator(DefaultRecordBatch.java:334) at org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.nextFetchedRecord(Fetcher.java:1144) at org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.fetchRecords(Fetcher.java:1181) at org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.access$1500(Fetcher.java:1035) at org.apache.kafka.clients.consumer.internals.Fetcher.fetchRecords(Fetcher.java:544) at org.apache.kafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:505) at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1230) at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1187) at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1115) at org.apache.nifi.processors.kafka.pubsub.ConsumerLease.poll(ConsumerLease.java:177) at org.apache.nifi.processors.kafka.pubsub.ConsumeKafkaRecord_2_0.onTrigger(ConsumeKafkaRecord_2_0.java:394) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) This si because kafka client libraries are outdated and requires kafka client 2.1.0 or above. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] ashish0mittal0 opened a new pull request #4261: NIFI-7424: PutSQL updated, commit and rollback allowed only if autocommit set false for the connection
ashish0mittal0 opened a new pull request #4261: URL: https://github.com/apache/nifi/pull/4261 Enables PutSQL processor to route flow-files to the relationships when AUTO_COMMIT set TRUE; fixes bug NIFI-7424. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7243) SFTP Processors - Exhausted available authentication methods
[ https://issues.apache.org/jira/browse/NIFI-7243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102333#comment-17102333 ] Raúl commented on NIFI-7243: Hello [~thenatog], as I mentioned any processor fails: ListSFTP, FetchSFTP, GetSFTP, PutSFTP. In version 1.8 this works correctly. In superiors no. If I connect with another client, for example Winscp works correctly. Thanks > SFTP Processors - Exhausted available authentication methods > > > Key: NIFI-7243 > URL: https://issues.apache.org/jira/browse/NIFI-7243 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.10.0, 1.11.0, 1.11.1, 1.11.3 > Environment: PRO >Reporter: Raúl >Assignee: Nathan Gough >Priority: Critical > Labels: SFTP, listsftp > Attachments: Screen Shot 2020-05-07 at 5.28.29 PM.png > > > In version 1.8 this works correctly. In superiors no. > > Any of the processors: ListSFTP, FetchSFTP, GetSFTP, PutSFTP when connecting > to an sftp shows the error: > 2020-03-10 15:12:51,596 ERROR [Timer-Driven Process Thread-3] > o.a.nifi.processors.standard.ListSFTP > ListSFTP[id=af72555e-0170-1000-c80d-7e992c9de544] Failed to perform listing > on remote host due to Exhausted available authentication methods: > net.schmizz.sshj.userauth.UserAuthException: Exhausted available > authentication methods > net.schmizz.sshj.userauth.UserAuthException: Exhausted available > authentication methods > at net.schmizz.sshj.SSHClient.auth(SSHClient.java:230) > > These tests that I commented have been made from nifi, both in windows and > linux environment and fails in both. > 1 - Strict Host Key Checking: false > 2 - Strict Host Key Checking: true > Host Key File: C:\Users\usuario\.ssh\known_hosts --> (ssh-rsa, > ssh-dss and ecdsa-sha2-nistp256) > > From the machines, both in windows and in linux, if an sftp is made from the > command line it connects without problems. But with Nifi he fails to connect. > > Any idea about how can I resolve this error? > > Thank you in advance, > log > - > 2020-03-10 15:12:51,596 ERROR [Timer-Driven Process Thread-3] > o.a.nifi.processors.standard.ListSFTP > ListSFTP[id=af72555e-0170-1000-c80d-7e992c9de544] Failed to perform listing > on remote host due to Exhausted available authentication methods: > net.schmizz.sshj.userauth.UserAuthException: Exhausted available > authentication methods2020-03-10 15:12:51,596 ERROR [Timer-Driven Process > Thread-3] o.a.nifi.processors.standard.ListSFTP > ListSFTP[id=af72555e-0170-1000-c80d-7e992c9de544] Failed to perform listing > on remote host due to Exhausted available authentication methods: > net.schmizz.sshj.userauth.UserAuthException: Exhausted available > authentication methodsnet.schmizz.sshj.userauth.UserAuthException: Exhausted > available authentication methods at > net.schmizz.sshj.SSHClient.auth(SSHClient.java:230) at > org.apache.nifi.processors.standard.util.SFTPTransfer.getSFTPClient(SFTPTransfer.java:606) > at > org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:233) > at > org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:196) > at > org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:106) > at > org.apache.nifi.processors.standard.ListSFTP.performListing(ListSFTP.java:146) > at > org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingTimestamps(AbstractListProcessor.java:472) > at > org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:414) > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176) > at > org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213) > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) > at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at
