[jira] [Commented] (NIFIREG-188) Login by hitting Enter in the password field in web UI
[ https://issues.apache.org/jira/browse/NIFIREG-188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933089#comment-16933089 ] Kotaro Terada commented on NIFIREG-188: --- Hi [~kdoran] , There is a code conflict, I just rebased it onto master and re-pushed to GitHub. Would you please look into it? [https://github.com/apache/nifi-registry/pull/147] > Login by hitting Enter in the password field in web UI > -- > > Key: NIFIREG-188 > URL: https://issues.apache.org/jira/browse/NIFIREG-188 > Project: NiFi Registry > Issue Type: Improvement >Affects Versions: 0.1.0, 0.2.0 >Reporter: Julian Gimbel >Assignee: Kotaro Terada >Priority: Minor > > While logging in to Nifi Registry it should be possible to hit enter in the > password field to login instead of using the mouse or tab on to the login > button. > Unfortunately I am not experienced enough with angular to do it myself. > It is not as simple as just adding a tag around the dialog fields and > buttons. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (NIFI-6686) ResizeImage Processor Does Not Handle Some Exceptions Cleanly
[ https://issues.apache.org/jira/browse/NIFI-6686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Knight updated NIFI-6686: - Comment: was deleted (was: I am working on digging up an image to replicate this.) > ResizeImage Processor Does Not Handle Some Exceptions Cleanly > - > > Key: NIFI-6686 > URL: https://issues.apache.org/jira/browse/NIFI-6686 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.9.2 > Environment: Ubuntu OS, OpenJDK 1.8 >Reporter: Matthew Knight >Priority: Major > Labels: beginner, imageio > Attachments: peppers.jpg > > Original Estimate: 48h > Remaining Estimate: 48h > > The ResizeImage Processor's process method which performs the actual image > handles IOExceptions and performs some validation, but it does not handle all > exception types cleanly. The attached image (peppers.jpg) throws an > IllegalArgumentException. The offending flowfile gets penalized, but > whenever it is reprocessed this error will just happen again because the > image itself is the source of the problem and that has not changed. > Processing time becomes wasted as the same file is processed repeatedly at > every penalty duration with the same result. ImageResize exceptions should > be caught and handled explicitly as REL_FAILURE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6680) Nifi PutKudu processor - Convert record field names to lowercase
[
https://issues.apache.org/jira/browse/NIFI-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933356#comment-16933356
]
Tamas Palfy commented on NIFI-6680:
---
Wondering if it would make sense to actually enhance the
{{org.apache.nifi.serialization.record.RecordSchema}} interface with something
like this:
{code:java}
/**
* @param mutator formatting logic to run on all fields
* @return the names of the fields after applying the mutator to each
*/
List getFieldNames(Function mutator);
{code}
Could be an overkill only for this case but the idea would be that similar
issues may come up in other places. And the concrete issue could vary (e.g. fix
width, prefix/suffix, encoding, to name a few).
> Nifi PutKudu processor - Convert record field names to lowercase
>
>
> Key: NIFI-6680
> URL: https://issues.apache.org/jira/browse/NIFI-6680
> Project: Apache NiFi
> Issue Type: Wish
> Components: Extensions
>Affects Versions: 1.9.2
>Reporter: Kevin J McCarthy
>Priority: Major
> Original Estimate: 1h
> Time Spent: 0.5h
> Remaining Estimate: 0.5h
>
> Kudu supports case sensitive column names while query tools built over Kudu
> (eg. Impala) do not. If a table is created via Impala, which is the most
> common use case, all column names are converted to lowercase. If an incoming
> record in a FlowFile contains uppercase column names, the columns won't match
> the schema of the destination table unless the schema for the record is
> modified to have lowercase field names.
> Rather than converting the schema of the record, I propose field names are
> converted to lowercase before they are put into Kudu.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] bakaid opened a new pull request #649: MINIFICPP-1024 - Fix third party dependencies
bakaid opened a new pull request #649: MINIFICPP-1024 - Fix third party dependencies URL: https://github.com/apache/nifi-minifi-cpp/pull/649 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-6686) ResizeImage Processor Does Not Handle Some Exceptions Cleanly
[ https://issues.apache.org/jira/browse/NIFI-6686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Knight updated NIFI-6686: - Description: The ResizeImage Processor's process method which performs the actual image handles IOExceptions and performs some validation, but it does not handle all exception types cleanly. The attached image (peppers.jpg) throws an IllegalArgumentException. The offending flowfile gets penalized, but whenever it is reprocessed this error will just happen again because the image itself is the source of the problem and that has not changed. Processing time becomes wasted as the same file is processed repeatedly at every penalty duration with the same result. ImageResize exceptions should be caught and handled explicitly as REL_FAILURE. (was: The ResizeImage Processor's process method which performs the actual image resize throws an IOException which is not directly caught. The offending flowfile gets penalized, but whenever it is reprocessed this error will just happen again because the image itself is the source of the problem and that has not changed. Processing time becomes wasted as the same file is processed repeatedly at every penalty duration with the same result. ImageResize IOExceptions should be caught and handled explicitly as REL_FAILURE.) > ResizeImage Processor Does Not Handle Some Exceptions Cleanly > - > > Key: NIFI-6686 > URL: https://issues.apache.org/jira/browse/NIFI-6686 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.9.2 > Environment: Ubuntu OS, OpenJDK 1.8 >Reporter: Matthew Knight >Priority: Major > Labels: beginner, imageio > Attachments: peppers.jpg > > Original Estimate: 48h > Remaining Estimate: 48h > > The ResizeImage Processor's process method which performs the actual image > handles IOExceptions and performs some validation, but it does not handle all > exception types cleanly. The attached image (peppers.jpg) throws an > IllegalArgumentException. The offending flowfile gets penalized, but > whenever it is reprocessed this error will just happen again because the > image itself is the source of the problem and that has not changed. > Processing time becomes wasted as the same file is processed repeatedly at > every penalty duration with the same result. ImageResize exceptions should > be caught and handled explicitly as REL_FAILURE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6312) AMQP processors seem to have thread cleanup issues
[ https://issues.apache.org/jira/browse/NIFI-6312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933358#comment-16933358 ] Batuhan Bayrakçı commented on NIFI-6312: We have encountered with this situation. The problem was about the queue not exists nifi looking for to consume. After queue has been created, connection problem had been resolved. > AMQP processors seem to have thread cleanup issues > -- > > Key: NIFI-6312 > URL: https://issues.apache.org/jira/browse/NIFI-6312 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.9.0 >Reporter: Robert Bruno >Priority: Major > > At a minimum the ConsumeAMQP processor exhibits this behavior but the > PublishAMQP may as well. > If ConsumeAMQP is listening to a working AMQP server and then that server > name is no longer resolvable errors begin to show up in logs saying the > hostname can't be resolve. This is expected. > What isn't expected is if you then turn off the processor or even delete the > processor the error message persists. The only way to resolve this is > restarting the nifi node. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mcgilman commented on issue #3747: NIFI-6558 Add Parameters to User Guide & Admin Guide
mcgilman commented on issue #3747: NIFI-6558 Add Parameters to User Guide & Admin Guide URL: https://github.com/apache/nifi/pull/3747#issuecomment-533138600 Will review... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] asfgit closed pull request #3745: Fix syntax error in schema example
asfgit closed pull request #3745: Fix syntax error in schema example URL: https://github.com/apache/nifi/pull/3745 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] MikeThomsen commented on issue #3745: Fix syntax error in schema example
MikeThomsen commented on issue #3745: Fix syntax error in schema example URL: https://github.com/apache/nifi/pull/3745#issuecomment-533096998 Merged. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-6686) ResizeImage Processor Does Not Handle Some Exceptions Cleanly
[ https://issues.apache.org/jira/browse/NIFI-6686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Knight updated NIFI-6686: - Attachment: peppers.jpg > ResizeImage Processor Does Not Handle Some Exceptions Cleanly > - > > Key: NIFI-6686 > URL: https://issues.apache.org/jira/browse/NIFI-6686 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.9.2 > Environment: Ubuntu OS, OpenJDK 1.8 >Reporter: Matthew Knight >Priority: Major > Labels: beginner, imageio > Attachments: peppers.jpg > > Original Estimate: 48h > Remaining Estimate: 48h > > The ResizeImage Processor's process method which performs the actual image > resize throws an IOException which is not directly caught. The offending > flowfile gets penalized, but whenever it is reprocessed this error will just > happen again because the image itself is the source of the problem and that > has not changed. Processing time becomes wasted as the same file is > processed repeatedly at every penalty duration with the same result. > ImageResize IOExceptions should be caught and handled explicitly as > REL_FAILURE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] bbende commented on issue #3726: NIFI-5816 Switch SFTP processors to use SSHJ
bbende commented on issue #3726: NIFI-5816 Switch SFTP processors to use SSHJ URL: https://github.com/apache/nifi/pull/3726#issuecomment-533123035 I did write a new unit test for SFTPTransfer that tests every method in the class using the SSHTestServer, and I ran that unit test against the old implementation and the new one to prove everything works the same. As far as testing the key size, I didn't do that through a unit test, but from a quick look at the SSHTestServer it seems like it may be possible using `setPublickeyAuthenticator` instead of the password authenticator. Also, thanks for setting up that test server :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6680) Nifi PutKudu processor - Convert record field names to lowercase
[ https://issues.apache.org/jira/browse/NIFI-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933369#comment-16933369 ] Joseph Witt commented on NIFI-6680: --- At least as shown in this example use case the need for mutation of the column names is an issue of a given system we're interfacing with. So to that end I'd lean toward having this mutation only occur at that point. If we were to find this was a common case or something we'd want to do/have understood in the record interfaces themselves then I'd think what you're showing is a good next step. > Nifi PutKudu processor - Convert record field names to lowercase > > > Key: NIFI-6680 > URL: https://issues.apache.org/jira/browse/NIFI-6680 > Project: Apache NiFi > Issue Type: Wish > Components: Extensions >Affects Versions: 1.9.2 >Reporter: Kevin J McCarthy >Priority: Major > Original Estimate: 1h > Time Spent: 0.5h > Remaining Estimate: 0.5h > > Kudu supports case sensitive column names while query tools built over Kudu > (eg. Impala) do not. If a table is created via Impala, which is the most > common use case, all column names are converted to lowercase. If an incoming > record in a FlowFile contains uppercase column names, the columns won't match > the schema of the destination table unless the schema for the record is > modified to have lowercase field names. > Rather than converting the schema of the record, I propose field names are > converted to lowercase before they are put into Kudu. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] bakaid commented on issue #649: MINIFICPP-1024 - Fix third party dependencies
bakaid commented on issue #649: MINIFICPP-1024 - Fix third party dependencies URL: https://github.com/apache/nifi-minifi-cpp/pull/649#issuecomment-533152557 > Looks good. I'll run some tests across platforms with various builds enabling and disabling features before merging. thanks! Thanks, let me know if anything fails. This is a less complete fix than MINIFICPP-1022 will be, but it should make the builds running properly again. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6312) AMQP processors seem to have thread cleanup issues
[ https://issues.apache.org/jira/browse/NIFI-6312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933365#comment-16933365 ] Joseph Witt commented on NIFI-6312: --- In my review of the code for this processor and interaction with a proper AMQP server I believe this processor requires significant rework. It is useable for some cases but it needs to be largely rewritten. > AMQP processors seem to have thread cleanup issues > -- > > Key: NIFI-6312 > URL: https://issues.apache.org/jira/browse/NIFI-6312 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.9.0 >Reporter: Robert Bruno >Priority: Major > > At a minimum the ConsumeAMQP processor exhibits this behavior but the > PublishAMQP may as well. > If ConsumeAMQP is listening to a working AMQP server and then that server > name is no longer resolvable errors begin to show up in logs saying the > hostname can't be resolve. This is expected. > What isn't expected is if you then turn off the processor or even delete the > processor the error message persists. The only way to resolve this is > restarting the nifi node. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6686) ResizeImage Processor Does Not Handle Some Exceptions Cleanly
[ https://issues.apache.org/jira/browse/NIFI-6686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933391#comment-16933391 ] Matthew Knight commented on NIFI-6686: -- I am working on a fix for this for my purposes, opening this issue to possibly be able to commit that fix back. > ResizeImage Processor Does Not Handle Some Exceptions Cleanly > - > > Key: NIFI-6686 > URL: https://issues.apache.org/jira/browse/NIFI-6686 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.9.2 > Environment: Ubuntu OS, OpenJDK 1.8 >Reporter: Matthew Knight >Priority: Major > Labels: beginner, imageio > Attachments: peppers.jpg > > Original Estimate: 48h > Remaining Estimate: 48h > > The ResizeImage Processor's process method which performs the actual image > handles IOExceptions and performs some validation, but it does not handle all > exception types cleanly. The attached image (peppers.jpg) throws an > IllegalArgumentException. The offending flowfile gets penalized, but > whenever it is reprocessed this error will just happen again because the > image itself is the source of the problem and that has not changed. > Processing time becomes wasted as the same file is processed repeatedly at > every penalty duration with the same result. ImageResize exceptions should > be caught and handled explicitly as REL_FAILURE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] tpalfy commented on a change in pull request #3748: NIFI-6680: Nifi PutKudu processor - Convert record field names to low…
tpalfy commented on a change in pull request #3748: NIFI-6680: Nifi PutKudu
processor - Convert record field names to low…
URL: https://github.com/apache/nifi/pull/3748#discussion_r326256769
##
File path:
nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-processors/src/test/java/org/apache/nifi/processors/kudu/TestPutKudu.java
##
@@ -103,6 +103,7 @@ private void setUpTestRunner(TestRunner testRunner) throws
InitializationExcepti
testRunner.setProperty(PutKudu.KUDU_MASTERS, DEFAULT_MASTERS);
testRunner.setProperty(PutKudu.SKIP_HEAD_LINE, SKIP_HEAD_LINE);
testRunner.setProperty(PutKudu.IGNORE_NULL, "true");
+testRunner.setProperty(PutKudu.LOWERCASE_FIELD_NAMES, "true");
Review comment:
Unfortunately the current tests are not capable of verifying the correct
behaviour regarding this change.
Wether these settings are omitted or or changed to `"false"` makes no
difference.
This `setProperty` could be removed (a bit misleading as it doesn't really
do anything in the current state) and a couple targeted tests could be added,
something like these:
```java
@Test
public void testBuildPartialRowWithLowercaseKuduColumn() {
String kuduColumn = "name";
final String recordField = "NAME";
boolean lowercaseFields = true;
testBuildPartialRow(kuduColumn, recordField, lowercaseFields);
}
@Test
public void testBuildPartialRowWithUppercaseKuduColumn() {
String kuduColumn = "NAME";
final String recordField = "NAME";
boolean lowercaseFields = false;
testBuildPartialRow(kuduColumn, recordField, lowercaseFields);
}
private void testBuildPartialRow(String kuduColumn, String recordField,
boolean lowercaseFields) {
// GIVEN
final Schema kuduSchema = new Schema(Arrays.asList(
new ColumnSchema.ColumnSchemaBuilder(kuduColumn,
Type.STRING).nullable(true).build()
));
final RecordSchema schema = new SimpleRecordSchema(Arrays.asList(
new RecordField(recordField,
RecordFieldType.STRING.getDataType())
));
Map values = new HashMap<>() {{
put(recordField, "foo");
}};
PartialRow row = kuduSchema.newPartialRow();
// WHEN
processor.buildPartialRow(
kuduSchema,
row,
new MapRecord(schema, values),
schema.getFieldNames(),
true,
lowercaseFields
);
// THEN
assertEquals("foo", row.getString(kuduColumn));
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #3748: NIFI-6680: Nifi PutKudu processor - Convert record field names to low…
tpalfy commented on a change in pull request #3748: NIFI-6680: Nifi PutKudu
processor - Convert record field names to low…
URL: https://github.com/apache/nifi/pull/3748#discussion_r326260268
##
File path:
nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-processors/src/test/java/org/apache/nifi/processors/kudu/TestPutKudu.java
##
@@ -412,6 +413,7 @@ private void buildPartialRow(Long id, String name, Short
age) {
kuduSchema.newPartialRow(),
new MapRecord(schema, values),
schema.getFieldNames(),
+true,
Review comment:
I think setting `lowercaseFields` to `true` is a bit misleading. At the
moment this only works because
1. All the `fieldNames` in the test are already in lower case.
2. There are no assertions that the `buildPartialRow` actually did set the
values correctly.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (MINIFICPP-1034) MQTT processors doesn't work
Arpad Boda created MINIFICPP-1034: - Summary: MQTT processors doesn't work Key: MINIFICPP-1034 URL: https://issues.apache.org/jira/browse/MINIFICPP-1034 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Affects Versions: 0.6.0 Reporter: Arpad Boda Assignee: Arpad Boda Fix For: 0.7.0 Neither ConsumeMQTT nor PublishMQTT call AbstractMQTT::Initialize, leaving the member called "sslEnabled_" uninitialized. Which means we try to connect securely or plain tcp based on memory garbage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] kjmccarthy commented on a change in pull request #3748: NIFI-6680: Nifi PutKudu processor - Convert record field names to low…
kjmccarthy commented on a change in pull request #3748: NIFI-6680: Nifi PutKudu processor - Convert record field names to low… URL: https://github.com/apache/nifi/pull/3748#discussion_r326264483 ## File path: nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-processors/src/test/java/org/apache/nifi/processors/kudu/TestPutKudu.java ## @@ -103,6 +103,7 @@ private void setUpTestRunner(TestRunner testRunner) throws InitializationExcepti testRunner.setProperty(PutKudu.KUDU_MASTERS, DEFAULT_MASTERS); testRunner.setProperty(PutKudu.SKIP_HEAD_LINE, SKIP_HEAD_LINE); testRunner.setProperty(PutKudu.IGNORE_NULL, "true"); +testRunner.setProperty(PutKudu.LOWERCASE_FIELD_NAMES, "true"); Review comment: Thanks @tpalfy, I will add a test to verify the correct behavior. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6665) NiFi toolkit bind issue with cli.bat file
[ https://issues.apache.org/jira/browse/NIFI-6665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933628#comment-16933628 ] Bryan Bende commented on NIFI-6665: --- You would to build NiFi from the master branch and then get the result toolkit assembly from nifi-toolkit/nifi-toolkit-assembly/target > NiFi toolkit bind issue with cli.bat file > - > > Key: NIFI-6665 > URL: https://issues.apache.org/jira/browse/NIFI-6665 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Affects Versions: 1.8.0, 1.9.2 >Reporter: Sridhar >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > When any cli command executed with cli.bat file ( like list bucket in > registry etc..) Exception comes with like " class not found" > Issue is with line : > *SET JAVA_PARAMS=-cp %LIB_DIR%\* %JAVA_OPTS% > org.apache.nifi.toolkit.cli.CLIMain* > please make change above in cli.bat file as below. > *SET JAVA_PARAMS=-cp %LIB_DIR%\* %JAVA_OPTS% --add-modules=java.xml.bind > org.apache.nifi.toolkit.cli.CLIMain* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (NIFI-6665) NiFi toolkit bind issue with cli.bat file
[ https://issues.apache.org/jira/browse/NIFI-6665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933628#comment-16933628 ] Bryan Bende edited comment on NIFI-6665 at 9/19/19 5:53 PM: You would have to build NiFi from the master branch and then get the result toolkit assembly from nifi-toolkit/nifi-toolkit-assembly/target was (Author: bende): You would to build NiFi from the master branch and then get the result toolkit assembly from nifi-toolkit/nifi-toolkit-assembly/target > NiFi toolkit bind issue with cli.bat file > - > > Key: NIFI-6665 > URL: https://issues.apache.org/jira/browse/NIFI-6665 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Affects Versions: 1.8.0, 1.9.2 >Reporter: Sridhar >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > When any cli command executed with cli.bat file ( like list bucket in > registry etc..) Exception comes with like " class not found" > Issue is with line : > *SET JAVA_PARAMS=-cp %LIB_DIR%\* %JAVA_OPTS% > org.apache.nifi.toolkit.cli.CLIMain* > please make change above in cli.bat file as below. > *SET JAVA_PARAMS=-cp %LIB_DIR%\* %JAVA_OPTS% --add-modules=java.xml.bind > org.apache.nifi.toolkit.cli.CLIMain* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-5816) SFTP cannot connect due to JSch limitations
[ https://issues.apache.org/jira/browse/NIFI-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933469#comment-16933469 ] ASF subversion and git services commented on NIFI-5816: --- Commit ff6a7d95613fc01601d3f60b5fb976c10d89949f in nifi's branch refs/heads/master from Bryan Bende [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ff6a7d9 ] NIFI-5816 This closes #3726. Remove unused SFTP classes that reference Jsch NIFI-5816 Switching SFTP processors from JSCH to SSHJ NIFI-5816 LICENSE/NOTICIE updates to reflect changing from JSch to SSHJ Signed-off-by: Joe Witt > SFTP cannot connect due to JSch limitations > --- > > Key: NIFI-5816 > URL: https://issues.apache.org/jira/browse/NIFI-5816 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.8.0 >Reporter: Laurenceau Julien >Assignee: Bryan Bende >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Hi, > The JSch library used for SFTP does not support HostKeyAlgorithms=ed25519 > whereas it is the current standard. This make SFTP / SSH unusable when > dealing with recent openssh config. > On dbeaver project they switched to sshj. > [https://github.com/dbeaver/dbeaver/issues/2202] > [https://community.hortonworks.com/answers/226377/view.html] > > https://stackoverflow.com/questions/2003419/com-jcraft-jsch-jschexception-unknownhostkey > One more argument against JSch is that it does not support rsa key length > other than default (2048). > ssh-keygen -o -t rsa -b 4096 -f id_rsa -> does not work with nifi > ssh-keygen -t rsa -f id_rsa -> works with nifi > Thanks and regards > JL > PS : sorry but I do not know nifi deep enough to fill all fields. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-5816) SFTP cannot connect due to JSch limitations
[ https://issues.apache.org/jira/browse/NIFI-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933468#comment-16933468 ] ASF subversion and git services commented on NIFI-5816: --- Commit ff6a7d95613fc01601d3f60b5fb976c10d89949f in nifi's branch refs/heads/master from Bryan Bende [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ff6a7d9 ] NIFI-5816 This closes #3726. Remove unused SFTP classes that reference Jsch NIFI-5816 Switching SFTP processors from JSCH to SSHJ NIFI-5816 LICENSE/NOTICIE updates to reflect changing from JSch to SSHJ Signed-off-by: Joe Witt > SFTP cannot connect due to JSch limitations > --- > > Key: NIFI-5816 > URL: https://issues.apache.org/jira/browse/NIFI-5816 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.8.0 >Reporter: Laurenceau Julien >Assignee: Bryan Bende >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Hi, > The JSch library used for SFTP does not support HostKeyAlgorithms=ed25519 > whereas it is the current standard. This make SFTP / SSH unusable when > dealing with recent openssh config. > On dbeaver project they switched to sshj. > [https://github.com/dbeaver/dbeaver/issues/2202] > [https://community.hortonworks.com/answers/226377/view.html] > > https://stackoverflow.com/questions/2003419/com-jcraft-jsch-jschexception-unknownhostkey > One more argument against JSch is that it does not support rsa key length > other than default (2048). > ssh-keygen -o -t rsa -b 4096 -f id_rsa -> does not work with nifi > ssh-keygen -t rsa -f id_rsa -> works with nifi > Thanks and regards > JL > PS : sorry but I do not know nifi deep enough to fill all fields. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-5816) SFTP cannot connect due to JSch limitations
[ https://issues.apache.org/jira/browse/NIFI-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933470#comment-16933470 ] ASF subversion and git services commented on NIFI-5816: --- Commit ff6a7d95613fc01601d3f60b5fb976c10d89949f in nifi's branch refs/heads/master from Bryan Bende [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ff6a7d9 ] NIFI-5816 This closes #3726. Remove unused SFTP classes that reference Jsch NIFI-5816 Switching SFTP processors from JSCH to SSHJ NIFI-5816 LICENSE/NOTICIE updates to reflect changing from JSch to SSHJ Signed-off-by: Joe Witt > SFTP cannot connect due to JSch limitations > --- > > Key: NIFI-5816 > URL: https://issues.apache.org/jira/browse/NIFI-5816 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.8.0 >Reporter: Laurenceau Julien >Assignee: Bryan Bende >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Hi, > The JSch library used for SFTP does not support HostKeyAlgorithms=ed25519 > whereas it is the current standard. This make SFTP / SSH unusable when > dealing with recent openssh config. > On dbeaver project they switched to sshj. > [https://github.com/dbeaver/dbeaver/issues/2202] > [https://community.hortonworks.com/answers/226377/view.html] > > https://stackoverflow.com/questions/2003419/com-jcraft-jsch-jschexception-unknownhostkey > One more argument against JSch is that it does not support rsa key length > other than default (2048). > ssh-keygen -o -t rsa -b 4096 -f id_rsa -> does not work with nifi > ssh-keygen -t rsa -f id_rsa -> works with nifi > Thanks and regards > JL > PS : sorry but I do not know nifi deep enough to fill all fields. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] markap14 commented on issue #3724: NIFI-6640 - UNION/CHOICE types not handled correctly
markap14 commented on issue #3724: NIFI-6640 - UNION/CHOICE types not handled correctly URL: https://github.com/apache/nifi/pull/3724#issuecomment-533172096 Thanks @tpalfy things are looking much better performance-wise. Things look good to me. Will merge. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6665) NiFi toolkit bind issue with cli.bat file
[ https://issues.apache.org/jira/browse/NIFI-6665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933492#comment-16933492 ] Bryan Bende commented on NIFI-6665: --- Also, so far all released versions of NiFi only support Java 8. There are changes in master to support > Java 8, so I would like to see if the new toolkit from master (which will be 1.10.0) works without needing this change. > NiFi toolkit bind issue with cli.bat file > - > > Key: NIFI-6665 > URL: https://issues.apache.org/jira/browse/NIFI-6665 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Affects Versions: 1.8.0, 1.9.2 >Reporter: Sridhar >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > When any cli command executed with cli.bat file ( like list bucket in > registry etc..) Exception comes with like " class not found" > Issue is with line : > *SET JAVA_PARAMS=-cp %LIB_DIR%\* %JAVA_OPTS% > org.apache.nifi.toolkit.cli.CLIMain* > please make change above in cli.bat file as below. > *SET JAVA_PARAMS=-cp %LIB_DIR%\* %JAVA_OPTS% --add-modules=java.xml.bind > org.apache.nifi.toolkit.cli.CLIMain* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (MINIFICPP-1035) MQTT processors doesn't export security-related properties
Arpad Boda created MINIFICPP-1035: - Summary: MQTT processors doesn't export security-related properties Key: MINIFICPP-1035 URL: https://issues.apache.org/jira/browse/MINIFICPP-1035 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Affects Versions: 0.6.0 Reporter: Arpad Boda Assignee: Arpad Boda MQTT processors doesn't export security related properties, which means they can only connect via plain TCP. It's not going to be fixed on scope of MINIFICPP-1034 as the feature seems to be untested. This ticket is a follow up to verify secure communication before enabling related properties. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6665) NiFi toolkit bind issue with cli.bat file
[ https://issues.apache.org/jira/browse/NIFI-6665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933465#comment-16933465 ] Bryan Bende commented on NIFI-6665: --- I'm confused because the concept "modules" only exists in Java 9 and greater, so if you were running with Java 8 then adding the JAXB modules shouldn't be necessary. > NiFi toolkit bind issue with cli.bat file > - > > Key: NIFI-6665 > URL: https://issues.apache.org/jira/browse/NIFI-6665 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Affects Versions: 1.8.0, 1.9.2 >Reporter: Sridhar >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > When any cli command executed with cli.bat file ( like list bucket in > registry etc..) Exception comes with like " class not found" > Issue is with line : > *SET JAVA_PARAMS=-cp %LIB_DIR%\* %JAVA_OPTS% > org.apache.nifi.toolkit.cli.CLIMain* > please make change above in cli.bat file as below. > *SET JAVA_PARAMS=-cp %LIB_DIR%\* %JAVA_OPTS% --add-modules=java.xml.bind > org.apache.nifi.toolkit.cli.CLIMain* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #3724: NIFI-6640 - UNION/CHOICE types not handled correctly
asfgit closed pull request #3724: NIFI-6640 - UNION/CHOICE types not handled correctly URL: https://github.com/apache/nifi/pull/3724 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6665) NiFi toolkit bind issue with cli.bat file
[ https://issues.apache.org/jira/browse/NIFI-6665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933576#comment-16933576 ] Muazma Zahid commented on NIFI-6665: Is there a way to test 1.10.0 with newer Java and test that new toolkit would work? > NiFi toolkit bind issue with cli.bat file > - > > Key: NIFI-6665 > URL: https://issues.apache.org/jira/browse/NIFI-6665 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Affects Versions: 1.8.0, 1.9.2 >Reporter: Sridhar >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > When any cli command executed with cli.bat file ( like list bucket in > registry etc..) Exception comes with like " class not found" > Issue is with line : > *SET JAVA_PARAMS=-cp %LIB_DIR%\* %JAVA_OPTS% > org.apache.nifi.toolkit.cli.CLIMain* > please make change above in cli.bat file as below. > *SET JAVA_PARAMS=-cp %LIB_DIR%\* %JAVA_OPTS% --add-modules=java.xml.bind > org.apache.nifi.toolkit.cli.CLIMain* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6689) nf-parameter-contexts.js?1.10.0-SNAPSHOT:1717 Uncaught TypeError: Cannot read property 'referencingComponents' of undefined
[ https://issues.apache.org/jira/browse/NIFI-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFI-6689: -- Status: Patch Available (was: In Progress) > nf-parameter-contexts.js?1.10.0-SNAPSHOT:1717 Uncaught TypeError: Cannot read > property 'referencingComponents' of undefined > --- > > Key: NIFI-6689 > URL: https://issues.apache.org/jira/browse/NIFI-6689 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Blocker > Fix For: 1.10.0 > > Attachments: Screen Shot 2019-09-19 at 12.11.54 PM.png > > Time Spent: 10m > Remaining Estimate: 0h > > Repro steps: > Create a parameter context. Remove modify permissions for the user on this > parameter context. Configure the PG you are in to use the parameter context > you just created. Add a processor. Configure a processor property to > reference a parameter. Click to go to view that parameter. Close all the > dialogs and open the parameter context shell. Attempt to open a different > parameter context. You will see the error. > > !Screen Shot 2019-09-19 at 12.11.54 PM.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] scottyaslan opened a new pull request #3752: [NIFI-6689] reset private var on dialog close
scottyaslan opened a new pull request #3752: [NIFI-6689] reset private var on
dialog close
URL: https://github.com/apache/nifi/pull/3752
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and
JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #650: MINIFICPP-1034 - MQTT processors doesnt work
arpadboda commented on a change in pull request #650: MINIFICPP-1034 - MQTT
processors doesnt work
URL: https://github.com/apache/nifi-minifi-cpp/pull/650#discussion_r326269891
##
File path: extensions/mqtt/processors/AbstractMQTTProcessor.cpp
##
@@ -46,32 +46,15 @@ core::Property AbstractMQTTProcessor::SecurityCA("Security
CA", "File or directo
core::Property AbstractMQTTProcessor::SecurityCert("Security Cert", "Path to
client's public key (PEM) used for authentication", "");
core::Property AbstractMQTTProcessor::SecurityPrivateKey("Security Private
Key", "Path to client's private key (PEM) used for authentication", "");
core::Property AbstractMQTTProcessor::SecurityPrivateKeyPassWord("Security
Pass Phrase", "Private key passphrase", "");
-core::Relationship AbstractMQTTProcessor::Success("success", "FlowFiles that
are sent successfully to the destination are transferred to this relationship");
-core::Relationship AbstractMQTTProcessor::Failure("failure", "FlowFiles that
failed to send to the destination are transferred to this relationship");
-void AbstractMQTTProcessor::initialize() {
- // Set the supported properties
- std::set properties;
- properties.insert(BrokerURL);
- properties.insert(CleanSession);
- properties.insert(ClientID);
- properties.insert(UserName);
- properties.insert(PassWord);
- properties.insert(KeepLiveInterval);
- properties.insert(ConnectionTimeOut);
- properties.insert(QOS);
- properties.insert(Topic);
- setSupportedProperties(properties);
- // Set the supported relationships
- std::set relationships;
- relationships.insert(Success);
- relationships.insert(Failure);
- setSupportedRelationships(relationships);
- MQTTClient_SSLOptions sslopts_ = MQTTClient_SSLOptions_initializer;
- sslEnabled_ = false;
Review comment:
Too bad it was only initialized here, but none of the baseclasses called
this function.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #650: MINIFICPP-1034 - MQTT processors doesnt work
arpadboda commented on a change in pull request #650: MINIFICPP-1034 - MQTT
processors doesnt work
URL: https://github.com/apache/nifi-minifi-cpp/pull/650#discussion_r326269553
##
File path: extensions/mqtt/processors/AbstractMQTTProcessor.cpp
##
@@ -46,32 +46,15 @@ core::Property AbstractMQTTProcessor::SecurityCA("Security
CA", "File or directo
core::Property AbstractMQTTProcessor::SecurityCert("Security Cert", "Path to
client's public key (PEM) used for authentication", "");
core::Property AbstractMQTTProcessor::SecurityPrivateKey("Security Private
Key", "Path to client's private key (PEM) used for authentication", "");
core::Property AbstractMQTTProcessor::SecurityPrivateKeyPassWord("Security
Pass Phrase", "Private key passphrase", "");
-core::Relationship AbstractMQTTProcessor::Success("success", "FlowFiles that
are sent successfully to the destination are transferred to this relationship");
-core::Relationship AbstractMQTTProcessor::Failure("failure", "FlowFiles that
failed to send to the destination are transferred to this relationship");
-void AbstractMQTTProcessor::initialize() {
- // Set the supported properties
- std::set properties;
- properties.insert(BrokerURL);
- properties.insert(CleanSession);
- properties.insert(ClientID);
- properties.insert(UserName);
- properties.insert(PassWord);
- properties.insert(KeepLiveInterval);
- properties.insert(ConnectionTimeOut);
- properties.insert(QOS);
- properties.insert(Topic);
- setSupportedProperties(properties);
- // Set the supported relationships
- std::set relationships;
- relationships.insert(Success);
- relationships.insert(Failure);
- setSupportedRelationships(relationships);
- MQTTClient_SSLOptions sslopts_ = MQTTClient_SSLOptions_initializer;
Review comment:
This was just hiding instead of initializing the member. So it had no chance
to work, but it didn't pop up given the security properties are not exported,
so it's never referenced.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (NIFI-6640) Schema Inference of UNION/CHOICE types not handled correctly
[
https://issues.apache.org/jira/browse/NIFI-6640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933474#comment-16933474
]
ASF subversion and git services commented on NIFI-6640:
---
Commit 34112519c2dde19d704ef624e62e51b399cf1ce7 in nifi's branch
refs/heads/master from Tamas Palfy
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3411251 ]
NIFI-6640 - UNION/CHOICE types not handled correctly
3 important changes:
1. FieldTypeInference had a bug when dealing with multiple datatypes for
the same field where some (but not all) were in a wider-than-the-other
relationship.
Before: Some datatypes could be lost. String was wider than any other.
After: Consistent behaviour. String is NOT wider than any other.
2. Choosing a datatype for a value from a ChoiceDataType:
Before it chose the first compatible datatype as the basis of conversion.
After change it tries to find the most suitable datatype.
3. Conversion of a value of avro union type:
Before it chose the first compatible datatype as the basis of conversion.
After change it tries to find the most suitable datatype.
Change: In the RecordFieldType enum moved TIMESTAMP ahead of DATE.
This closes #3724.
Signed-off-by: Mark Payne
> Schema Inference of UNION/CHOICE types not handled correctly
>
>
> Key: NIFI-6640
> URL: https://issues.apache.org/jira/browse/NIFI-6640
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Tamas Palfy
>Assignee: Tamas Palfy
>Priority: Major
> Labels: Record, inference, schema
> Attachments: NIFI-6640.template.xml
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> When reading the following CSV:
> {code}
> Id|Value
> 1|3
> 2|3.75
> 3|3.85
> 4|8
> 5|2.0
> 6|4.0
> 7|some_string
> {code}
> And try to channel through a {{ConvertRecord}} processor, the following
> exception is thrown:
> {code}
> 2019-09-06 18:25:48,936 ERROR [Timer-Driven Process Thread-2]
> o.a.n.processors.standard.ConvertRecord
> ConvertRecord[id=07635c71-016d-1000-3847-ff916164b32a] Failed to process
> StandardFlowFileRecord[uuid=4b4ab01a-b349-4f83-9b25-6a58d0b29
> 7c1,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1567786888281-1, container=default,
> section=1], offset=326669,
> length=56],offset=0,name=4b4ab01a-b349-4f83-9b25-6a58d0b297c1,size=56]; will
> route to failure: org.apa
> che.nifi.processor.exception.ProcessException: Could not parse incoming data
> org.apache.nifi.processor.exception.ProcessException: Could not parse
> incoming data
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:170)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2925)
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor.onTrigger(AbstractRecordProcessor.java:122)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:205)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.nifi.serialization.MalformedRecordException: Error
> while getting next record. Root cause:
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [some_string] of type class j
> ava.lang.String for field Value to any of the following available Sub-Types
> for a Choice: [FLOAT, INT]
> at
> org.apache.nifi.csv.CSVRecordReader.nextRecord(CSVRecordReader.java:119)
> at
> org.apache.nifi.serialization.RecordReader.nextRecord(RecordReader.java:50)
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:156)
> ... 13 common frames omitted
> Caused by:
>
[jira] [Updated] (NIFI-5816) SFTP cannot connect due to JSch limitations
[ https://issues.apache.org/jira/browse/NIFI-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-5816: -- Fix Version/s: 1.10.0 > SFTP cannot connect due to JSch limitations > --- > > Key: NIFI-5816 > URL: https://issues.apache.org/jira/browse/NIFI-5816 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.8.0 >Reporter: Laurenceau Julien >Assignee: Bryan Bende >Priority: Minor > Fix For: 1.10.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Hi, > The JSch library used for SFTP does not support HostKeyAlgorithms=ed25519 > whereas it is the current standard. This make SFTP / SSH unusable when > dealing with recent openssh config. > On dbeaver project they switched to sshj. > [https://github.com/dbeaver/dbeaver/issues/2202] > [https://community.hortonworks.com/answers/226377/view.html] > > https://stackoverflow.com/questions/2003419/com-jcraft-jsch-jschexception-unknownhostkey > One more argument against JSch is that it does not support rsa key length > other than default (2048). > ssh-keygen -o -t rsa -b 4096 -f id_rsa -> does not work with nifi > ssh-keygen -t rsa -f id_rsa -> works with nifi > Thanks and regards > JL > PS : sorry but I do not know nifi deep enough to fill all fields. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-5816) SFTP cannot connect due to JSch limitations
[ https://issues.apache.org/jira/browse/NIFI-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-5816: -- Resolution: Fixed Status: Resolved (was: Patch Available) +1 merged to master > SFTP cannot connect due to JSch limitations > --- > > Key: NIFI-5816 > URL: https://issues.apache.org/jira/browse/NIFI-5816 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.8.0 >Reporter: Laurenceau Julien >Assignee: Bryan Bende >Priority: Minor > Fix For: 1.10.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Hi, > The JSch library used for SFTP does not support HostKeyAlgorithms=ed25519 > whereas it is the current standard. This make SFTP / SSH unusable when > dealing with recent openssh config. > On dbeaver project they switched to sshj. > [https://github.com/dbeaver/dbeaver/issues/2202] > [https://community.hortonworks.com/answers/226377/view.html] > > https://stackoverflow.com/questions/2003419/com-jcraft-jsch-jschexception-unknownhostkey > One more argument against JSch is that it does not support rsa key length > other than default (2048). > ssh-keygen -o -t rsa -b 4096 -f id_rsa -> does not work with nifi > ssh-keygen -t rsa -f id_rsa -> works with nifi > Thanks and regards > JL > PS : sorry but I do not know nifi deep enough to fill all fields. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-6640) Schema Inference of UNION/CHOICE types not handled correctly
[
https://issues.apache.org/jira/browse/NIFI-6640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne resolved NIFI-6640.
--
Fix Version/s: 1.10.0
Resolution: Fixed
> Schema Inference of UNION/CHOICE types not handled correctly
>
>
> Key: NIFI-6640
> URL: https://issues.apache.org/jira/browse/NIFI-6640
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Tamas Palfy
>Assignee: Tamas Palfy
>Priority: Major
> Labels: Record, inference, schema
> Fix For: 1.10.0
>
> Attachments: NIFI-6640.template.xml
>
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> When reading the following CSV:
> {code}
> Id|Value
> 1|3
> 2|3.75
> 3|3.85
> 4|8
> 5|2.0
> 6|4.0
> 7|some_string
> {code}
> And try to channel through a {{ConvertRecord}} processor, the following
> exception is thrown:
> {code}
> 2019-09-06 18:25:48,936 ERROR [Timer-Driven Process Thread-2]
> o.a.n.processors.standard.ConvertRecord
> ConvertRecord[id=07635c71-016d-1000-3847-ff916164b32a] Failed to process
> StandardFlowFileRecord[uuid=4b4ab01a-b349-4f83-9b25-6a58d0b29
> 7c1,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1567786888281-1, container=default,
> section=1], offset=326669,
> length=56],offset=0,name=4b4ab01a-b349-4f83-9b25-6a58d0b297c1,size=56]; will
> route to failure: org.apa
> che.nifi.processor.exception.ProcessException: Could not parse incoming data
> org.apache.nifi.processor.exception.ProcessException: Could not parse
> incoming data
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:170)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2925)
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor.onTrigger(AbstractRecordProcessor.java:122)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:205)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.nifi.serialization.MalformedRecordException: Error
> while getting next record. Root cause:
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [some_string] of type class j
> ava.lang.String for field Value to any of the following available Sub-Types
> for a Choice: [FLOAT, INT]
> at
> org.apache.nifi.csv.CSVRecordReader.nextRecord(CSVRecordReader.java:119)
> at
> org.apache.nifi.serialization.RecordReader.nextRecord(RecordReader.java:50)
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:156)
> ... 13 common frames omitted
> Caused by:
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [some_string] of type class java.lang.String for field
> Value to any of the following available Sub-Types for a Choice: [FLOAT, INT
> ]
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:166)
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:116)
> at
> org.apache.nifi.csv.AbstractCSVRecordReader.convert(AbstractCSVRecordReader.java:86)
> at
> org.apache.nifi.csv.CSVRecordReader.nextRecord(CSVRecordReader.java:105)
> ... 15 common frames omitted
> {code}
> The problem is that {{FieldTypeInference}} has both a list of
> {{possibleDataTypes}} and a {{singleDataType}} and as long as an added
> dataType is not in a "wider" relationship with the previous types it is added
> to the {{possibleDataTypes}}. But once a "wider" type is added, it actually
> gets set as the {{singleDataType}} and the {{possibleDataTypes}} remains
> intact.
>
[GitHub] [nifi-minifi-cpp] arpadboda opened a new pull request #650: MINIFICPP-1034 - MQTT processors doesnt work
arpadboda opened a new pull request #650: MINIFICPP-1034 - MQTT processors doesnt work URL: https://github.com/apache/nifi-minifi-cpp/pull/650 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #650: MINIFICPP-1034 - MQTT processors doesnt work
arpadboda commented on a change in pull request #650: MINIFICPP-1034 - MQTT
processors doesnt work
URL: https://github.com/apache/nifi-minifi-cpp/pull/650#discussion_r326273570
##
File path: extensions/mqtt/processors/AbstractMQTTProcessor.cpp
##
@@ -46,32 +46,15 @@ core::Property AbstractMQTTProcessor::SecurityCA("Security
CA", "File or directo
core::Property AbstractMQTTProcessor::SecurityCert("Security Cert", "Path to
client's public key (PEM) used for authentication", "");
core::Property AbstractMQTTProcessor::SecurityPrivateKey("Security Private
Key", "Path to client's private key (PEM) used for authentication", "");
core::Property AbstractMQTTProcessor::SecurityPrivateKeyPassWord("Security
Pass Phrase", "Private key passphrase", "");
Review comment:
These are not exported, but clearly never been tested as well, so created
https://issues.apache.org/jira/browse/MINIFICPP-1035 to address this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (NIFI-6689) nf-parameter-contexts.js?1.10.0-SNAPSHOT:1717 Uncaught TypeError: Cannot read property 'referencingComponents' of undefined
Scott Aslan created NIFI-6689: - Summary: nf-parameter-contexts.js?1.10.0-SNAPSHOT:1717 Uncaught TypeError: Cannot read property 'referencingComponents' of undefined Key: NIFI-6689 URL: https://issues.apache.org/jira/browse/NIFI-6689 Project: Apache NiFi Issue Type: Sub-task Components: Core UI Reporter: Scott Aslan Assignee: Scott Aslan Fix For: 1.10.0 Attachments: Screen Shot 2019-09-19 at 12.11.54 PM.png Repro steps: Create a parameter context. Remove modify permissions for the user on this parameter context. Configure the PG you are in to use the parameter context you just created. Add a processor. Configure a processor property to reference a parameter. Click to go to view that parameter. Close all the dialogs and open the parameter context shell. Attempt to open a different parameter context. You will see the error. !Screen Shot 2019-09-19 at 12.11.54 PM.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-6690) Limit GoTo parameter action to parameters that exist in the active context.
Robert Fellows created NIFI-6690:
Summary: Limit GoTo parameter action to parameters that exist in
the active context.
Key: NIFI-6690
URL: https://issues.apache.org/jira/browse/NIFI-6690
Project: Apache NiFi
Issue Type: Improvement
Components: Core UI
Affects Versions: 1.10.0
Reporter: Robert Fellows
Currently, on the processor controller-service configuration dialogs, if a
value of a property matches the pattern of what a parameter reference looks
like *[1]*, the action to goto that parameter is available. However, if that
parameter does not exist in the parameter context, it can't find it to
highlight it in the table.
The goto action should only be available if the parameter reference matches a
parameter that already exists in the parameter context.
{{*[1]* #\{param name}}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #3726: NIFI-5816 Switch SFTP processors to use SSHJ
asfgit closed pull request #3726: NIFI-5816 Switch SFTP processors to use SSHJ URL: https://github.com/apache/nifi/pull/3726 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-4012) Azure Event Hub UI typos and cleanup
[ https://issues.apache.org/jira/browse/NIFI-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933563#comment-16933563 ] Shayne Burgess commented on NIFI-4012: -- We are starting to use this processor and the finding a lot of simple issues in the descriptions. There was a previous effort to address this but it has not made progress as it is also addressing other things not related to the issues with the text of the processor. I put up a patch that just addresses the descriptions. If you guys would prefer me to integrate this into the other patch instead I can but I think there are valid concerns with that patch and this is an easy fix. > Azure Event Hub UI typos and cleanup > > > Key: NIFI-4012 > URL: https://issues.apache.org/jira/browse/NIFI-4012 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andrew Grande >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-6692) Kudu Processors throw NullPointer without Kerberos
Sam Hjelmfelt created NIFI-6692: --- Summary: Kudu Processors throw NullPointer without Kerberos Key: NIFI-6692 URL: https://issues.apache.org/jira/browse/NIFI-6692 Project: Apache NiFi Issue Type: Bug Reporter: Sam Hjelmfelt A KerberosCredentialService is not required by the AbstractKuduProcessor, but if it is not set, a null pointer exception is thrown. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFIREG-320) Add standard HTTP security headers
Nathan Gough created NIFIREG-320: Summary: Add standard HTTP security headers Key: NIFIREG-320 URL: https://issues.apache.org/jira/browse/NIFIREG-320 Project: NiFi Registry Issue Type: Improvement Affects Versions: 0.5.0 Reporter: Nathan Gough Assignee: Nathan Gough Fix For: 1.0.0 Add HTTP security headers that are commonly used in most brower-based applications. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6682) SplitJson does not work if the input is null
[ https://issues.apache.org/jira/browse/NIFI-6682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933683#comment-16933683 ] Matthew Knight commented on NIFI-6682: -- I'm having trouble reproducing this in unit test form, what specifically is the content of the json files that cause this issue? > SplitJson does not work if the input is null > - > > Key: NIFI-6682 > URL: https://issues.apache.org/jira/browse/NIFI-6682 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.9.2 >Reporter: Frank Chen >Priority: Major > Attachments: image-2019-09-17-18-18-19-860.png, > image-2019-09-17-18-33-33-708.png > > > When the input is null, the SplitJson processor should output to failure > route rather than report error.As a result, the flowfile is not processed in > the queue all the time. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (NIFI-6682) SplitJson does not work if the input is null
[ https://issues.apache.org/jira/browse/NIFI-6682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Knight updated NIFI-6682: - Comment: was deleted (was: I'm having trouble reproducing this in unit test form, what specifically is the content of the json files that cause this issue?) > SplitJson does not work if the input is null > - > > Key: NIFI-6682 > URL: https://issues.apache.org/jira/browse/NIFI-6682 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.9.2 >Reporter: Frank Chen >Priority: Major > Attachments: image-2019-09-17-18-18-19-860.png, > image-2019-09-17-18-33-33-708.png > > > When the input is null, the SplitJson processor should output to failure > route rather than report error.As a result, the flowfile is not processed in > the queue all the time. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] eduardofontes commented on a change in pull request #3746: NIFI-6679 - Patch MoveHDFS
eduardofontes commented on a change in pull request #3746: NIFI-6679 - Patch
MoveHDFS
URL: https://github.com/apache/nifi/pull/3746#discussion_r326355843
##
File path:
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/MoveHDFS.java
##
@@ -357,9 +357,10 @@ public Object run() {
if (destinationExists) {
switch (processorConfig.getConflictResolution()) {
case REPLACE_RESOLUTION:
-if (hdfs.delete(file, false)) {
+// Remove destination file (newFile) to
replace
Review comment:
Ok @tpalfy . Thank you! I've just put your code in MoveHDFS test class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] granthenke commented on issue #3755: NIFI-6692: Fixing NullPointerException when not using Kerberos
granthenke commented on issue #3755: NIFI-6692: Fixing NullPointerException when not using Kerberos URL: https://github.com/apache/nifi/pull/3755#issuecomment-533312015 LGTM. Ideally there would be a test validating this, but I think it's okay to add it later when unifying the Kudu client building logic. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] mcgilman commented on issue #3753: NIFI-6558 Improved import/export versioned flows section and also edi…
mcgilman commented on issue #3753: NIFI-6558 Improved import/export versioned flows section and also edi… URL: https://github.com/apache/nifi/pull/3753#issuecomment-533291645 Will review... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] andrewmlim commented on issue #3747: NIFI-6558 Add Parameters to User Guide & Admin Guide
andrewmlim commented on issue #3747: NIFI-6558 Add Parameters to User Guide & Admin Guide URL: https://github.com/apache/nifi/pull/3747#issuecomment-533291339 Closing this one in favor or https://github.com/apache/nifi/pull/3753. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] andrewmlim closed pull request #3747: NIFI-6558 Add Parameters to User Guide & Admin Guide
andrewmlim closed pull request #3747: NIFI-6558 Add Parameters to User Guide & Admin Guide URL: https://github.com/apache/nifi/pull/3747 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFI-6691) UpdateRecord fails to update Record Schema when adding new "inner" field
Mark Payne created NIFI-6691: Summary: UpdateRecord fails to update Record Schema when adding new "inner" field Key: NIFI-6691 URL: https://issues.apache.org/jira/browse/NIFI-6691 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Mark Payne Assignee: Mark Payne If UpdateRecord is used to add a field to a Record, the output Record should have a schema that consists of the input record's schema modified to account for any newly added fields/types. For example, if the input Record contains 3 fields, "street", "city", and "state", and a new field, "fullAddress" is to be added by UpdateRecord, the output Record should contain all 4 fields. This works already. But if the field to add is not at "root" level (for example, if the root is a Record that has an array of type Address that possesses all of these fields) then the resulting Record has a Schema with an "Address" field that still contains only street, city, and state – not the new "fullAddress" field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] andrewmlim opened a new pull request #3753: NIFI-6558 Improved import/export versioned flows section and also edi…
andrewmlim opened a new pull request #3753: NIFI-6558 Improved import/export versioned flows section and also edi… URL: https://github.com/apache/nifi/pull/3753 …ted managing local changes section This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] markap14 opened a new pull request #3754: NIFI-6691: Fixed bug in MapRecord's algorithm for incorporating Inact…
markap14 opened a new pull request #3754: NIFI-6691: Fixed bug in MapRecord's
algorithm for incorporating Inact…
URL: https://github.com/apache/nifi/pull/3754
…ive Fields, by recursing into any 'child fields' that contain records; also
fixed bug in UpdateRecord that caused it to incorrectly map the first result to
all elements if the key of the property points to an array
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and
JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (NIFI-6691) UpdateRecord fails to update Record Schema when adding new "inner" field
[ https://issues.apache.org/jira/browse/NIFI-6691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-6691: - Fix Version/s: 1.10.0 Status: Patch Available (was: Open) > UpdateRecord fails to update Record Schema when adding new "inner" field > > > Key: NIFI-6691 > URL: https://issues.apache.org/jira/browse/NIFI-6691 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.10.0 > > Time Spent: 10m > Remaining Estimate: 0h > > If UpdateRecord is used to add a field to a Record, the output Record should > have a schema that consists of the input record's schema modified to account > for any newly added fields/types. For example, if the input Record contains 3 > fields, "street", "city", and "state", and a new field, "fullAddress" is to > be added by UpdateRecord, the output Record should contain all 4 fields. > This works already. But if the field to add is not at "root" level (for > example, if the root is a Record that has an array of type Address that > possesses all of these fields) then the resulting Record has a Schema with an > "Address" field that still contains only street, city, and state – not the > new "fullAddress" field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6665) NiFi toolkit bind issue with cli.bat file
[ https://issues.apache.org/jira/browse/NIFI-6665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933666#comment-16933666 ] Muazma Zahid commented on NIFI-6665: Just tested, in the master build this issue is resolved. It does still exist with Java 8. > NiFi toolkit bind issue with cli.bat file > - > > Key: NIFI-6665 > URL: https://issues.apache.org/jira/browse/NIFI-6665 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Affects Versions: 1.8.0, 1.9.2 >Reporter: Sridhar >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > When any cli command executed with cli.bat file ( like list bucket in > registry etc..) Exception comes with like " class not found" > Issue is with line : > *SET JAVA_PARAMS=-cp %LIB_DIR%\* %JAVA_OPTS% > org.apache.nifi.toolkit.cli.CLIMain* > please make change above in cli.bat file as below. > *SET JAVA_PARAMS=-cp %LIB_DIR%\* %JAVA_OPTS% --add-modules=java.xml.bind > org.apache.nifi.toolkit.cli.CLIMain* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] SamHjelmfelt opened a new pull request #3755: NIFI-6692: Fixing NullPointerException when not using Kerberos
SamHjelmfelt opened a new pull request #3755: NIFI-6692: Fixing
NullPointerException when not using Kerberos
URL: https://github.com/apache/nifi/pull/3755
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
A KerberosCredentialService is not required by the AbstractKuduProcessor,
but if it is not set, a null pointer exception is thrown. The change mirrors
the KuduLookupService.
https://issues.apache.org/jira/browse/NIFI-6692
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ X ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ X ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ X ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ X ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ X ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ X ] Have you verified that the full build is successful on both JDK 8
and JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] asfgit closed pull request #3754: NIFI-6691: Fixed bug in MapRecord's algorithm for incorporating Inact…
asfgit closed pull request #3754: NIFI-6691: Fixed bug in MapRecord's algorithm for incorporating Inact… URL: https://github.com/apache/nifi/pull/3754 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-4166) Create toolkit module to generate and build Swagger API library for NiFi
[ https://issues.apache.org/jira/browse/NIFI-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-4166: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Create toolkit module to generate and build Swagger API library for NiFi > > > Key: NIFI-4166 > URL: https://issues.apache.org/jira/browse/NIFI-4166 > Project: Apache NiFi > Issue Type: New Feature > Components: Tools and Build >Reporter: Joe Skora >Assignee: Joe Skora >Priority: Minor > Fix For: 1.10.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Create a new toolkit module to generate the Swagger API library based on the > current REST API annotations in the NiFi source by way of the Swagger Codegen > Maven Plugin. This should make it easier access to access the REST API from > Java code or Groovy scripts. > Swagger Codegen support other languages, so this could be expanded for > additional API client types. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6583) need a way to report nifi metrics to Azure Log Analytics
[ https://issues.apache.org/jira/browse/NIFI-6583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-6583: -- Fix Version/s: (was: 1.10.0) > need a way to report nifi metrics to Azure Log Analytics > > > Key: NIFI-6583 > URL: https://issues.apache.org/jira/browse/NIFI-6583 > Project: Apache NiFi > Issue Type: Improvement > Components: Examples >Affects Versions: 1.9.2 >Reporter: Seokwon Yang >Priority: Minor > Labels: patch > Original Estimate: 168h > Time Spent: 20m > Remaining Estimate: 167h 40m > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6583) need a way to report nifi metrics to Azure Log Analytics
[ https://issues.apache.org/jira/browse/NIFI-6583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933996#comment-16933996 ] Joseph Witt commented on NIFI-6583: --- once there is a PR and review traction associated with this JIRA the fix version can be established > need a way to report nifi metrics to Azure Log Analytics > > > Key: NIFI-6583 > URL: https://issues.apache.org/jira/browse/NIFI-6583 > Project: Apache NiFi > Issue Type: Improvement > Components: Examples >Affects Versions: 1.9.2 >Reporter: Seokwon Yang >Priority: Minor > Labels: patch > Original Estimate: 168h > Time Spent: 20m > Remaining Estimate: 167h 40m > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #3560: NIFI-6369: Updated JSON Readers to convert String values to Date/Time…
asfgit closed pull request #3560: NIFI-6369: Updated JSON Readers to convert String values to Date/Time… URL: https://github.com/apache/nifi/pull/3560 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6604) Reduce the size of the generated assemblies
[ https://issues.apache.org/jira/browse/NIFI-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934011#comment-16934011 ] Joseph Witt commented on NIFI-6604: --- Removed nifi-kafka-0-8-nar nifi-flume-nar nifi-media-nar nifi-druid-controller-service-api-nar nifi-druid-nar nifi-other-graph-services-nar nifi-kite-nar > Reduce the size of the generated assemblies > --- > > Key: NIFI-6604 > URL: https://issues.apache.org/jira/browse/NIFI-6604 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Reporter: Aldrin Piri >Assignee: Aldrin Piri >Priority: Blocker > Labels: migration-guide, release-notes > Fix For: 1.10.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > As per the discussion on > https://lists.apache.org/thread.html/d45f33805dcb8096914a06853bc3b8eca1acf50c2ae7b68e75eabfe8@ > we must reduce the assembly size such that we can publish the convenience > binaries as part of the release. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6604) Reduce the size of the generated assemblies
[ https://issues.apache.org/jira/browse/NIFI-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934010#comment-16934010 ] Joseph Witt commented on NIFI-6604: --- also removed kite-nar from the list as well > Reduce the size of the generated assemblies > --- > > Key: NIFI-6604 > URL: https://issues.apache.org/jira/browse/NIFI-6604 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Reporter: Aldrin Piri >Assignee: Aldrin Piri >Priority: Blocker > Labels: migration-guide, release-notes > Fix For: 1.10.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > As per the discussion on > https://lists.apache.org/thread.html/d45f33805dcb8096914a06853bc3b8eca1acf50c2ae7b68e75eabfe8@ > we must reduce the assembly size such that we can publish the convenience > binaries as part of the release. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6604) Reduce the size of the generated assemblies
[ https://issues.apache.org/jira/browse/NIFI-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934015#comment-16934015 ] ASF subversion and git services commented on NIFI-6604: --- Commit be1acd07c59e256e9633cd5958f0b3356b5873f3 in nifi's branch refs/heads/master from Aldrin Piri [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=be1acd0 ] NIFI-6604: This closes #3682. Removing large NARs from generated assemblies to reduce size. Removed items consist of: * nifi-kafka-0-8-nar * nifi-flume-nar * nifi-media-nar * nifi-druid-nar * nifi-druid-controller-service-api-nar * nifi-other-graph-services-nar * nifi-kite-nar > Reduce the size of the generated assemblies > --- > > Key: NIFI-6604 > URL: https://issues.apache.org/jira/browse/NIFI-6604 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Reporter: Aldrin Piri >Assignee: Aldrin Piri >Priority: Blocker > Labels: migration-guide, release-notes > Fix For: 1.10.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > As per the discussion on > https://lists.apache.org/thread.html/d45f33805dcb8096914a06853bc3b8eca1acf50c2ae7b68e75eabfe8@ > we must reduce the assembly size such that we can publish the convenience > binaries as part of the release. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6604) Reduce the size of the generated assemblies
[ https://issues.apache.org/jira/browse/NIFI-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-6604: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Reduce the size of the generated assemblies > --- > > Key: NIFI-6604 > URL: https://issues.apache.org/jira/browse/NIFI-6604 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Reporter: Aldrin Piri >Assignee: Aldrin Piri >Priority: Blocker > Labels: migration-guide, release-notes > Fix For: 1.10.0 > > Time Spent: 40m > Remaining Estimate: 0h > > As per the discussion on > https://lists.apache.org/thread.html/d45f33805dcb8096914a06853bc3b8eca1acf50c2ae7b68e75eabfe8@ > we must reduce the assembly size such that we can publish the convenience > binaries as part of the release. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6691) UpdateRecord fails to update Record Schema when adding new "inner" field
[ https://issues.apache.org/jira/browse/NIFI-6691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-6691: -- Resolution: Fixed Status: Resolved (was: Patch Available) +1 merged to master > UpdateRecord fails to update Record Schema when adding new "inner" field > > > Key: NIFI-6691 > URL: https://issues.apache.org/jira/browse/NIFI-6691 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.10.0 > > Time Spent: 20m > Remaining Estimate: 0h > > If UpdateRecord is used to add a field to a Record, the output Record should > have a schema that consists of the input record's schema modified to account > for any newly added fields/types. For example, if the input Record contains 3 > fields, "street", "city", and "state", and a new field, "fullAddress" is to > be added by UpdateRecord, the output Record should contain all 4 fields. > This works already. But if the field to add is not at "root" level (for > example, if the root is a Record that has an array of type Address that > possesses all of these fields) then the resulting Record has a Schema with an > "Address" field that still contains only street, city, and state – not the > new "fullAddress" field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6691) UpdateRecord fails to update Record Schema when adding new "inner" field
[ https://issues.apache.org/jira/browse/NIFI-6691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933977#comment-16933977 ] ASF subversion and git services commented on NIFI-6691: --- Commit d0371a5ef1f1abe59b7f1de310f21fb93e8fbacb in nifi's branch refs/heads/master from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d0371a5 ] NIFI-6691: This closes #3754. Fixed bug in MapRecord's algorithm for incorporating Inactive Fields, by recursing into any 'child fields' that contain records; also fixed bug in UpdateRecord that caused it to incorrectly map the first result to all elements if the key of the property points to an array > UpdateRecord fails to update Record Schema when adding new "inner" field > > > Key: NIFI-6691 > URL: https://issues.apache.org/jira/browse/NIFI-6691 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.10.0 > > Time Spent: 10m > Remaining Estimate: 0h > > If UpdateRecord is used to add a field to a Record, the output Record should > have a schema that consists of the input record's schema modified to account > for any newly added fields/types. For example, if the input Record contains 3 > fields, "street", "city", and "state", and a new field, "fullAddress" is to > be added by UpdateRecord, the output Record should contain all 4 fields. > This works already. But if the field to add is not at "root" level (for > example, if the root is a Record that has an array of type Address that > possesses all of these fields) then the resulting Record has a Schema with an > "Address" field that still contains only street, city, and state – not the > new "fullAddress" field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-5973) NiFi: Add 'unix' or 'hadoop' user-group-provider
[ https://issues.apache.org/jira/browse/NIFI-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933993#comment-16933993 ] Joseph Witt commented on NIFI-5973: --- [~alopresto][~tmelhase] i see this is merged but not resolved and there is a comment. can you clarify the status? > NiFi: Add 'unix' or 'hadoop' user-group-provider > > > Key: NIFI-5973 > URL: https://issues.apache.org/jira/browse/NIFI-5973 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Sean Roberts >Assignee: Troy Melhase >Priority: Major > Labels: authentication, authorization, identity > Fix For: 1.10.0 > > Time Spent: 18h 10m > Remaining Estimate: 0h > > NiFi currently supports these group mappings: > - File > - LDAP > Benefits of Unix or Hadoop based group identity: > - File is now allowed any many environments as it's not integrated with the > corporations identity system (typically AD or another LDAP). > - LDAP adds a lot of complexity, overhead, and tons of up-front config per > environment. > - Most services, including those related to Hadoop, are moving to Unix based > group lookups. > How it works and some possible method to implement: > - Linux users/groups come from the "Name Service Switch" (aka "nss"). > - Here are the Linux native commands to query it: > -- show all users: `getent passwd` > -- show specific user: `getent group %s` > -- show all groups: `getent group` > -- show specific group: `getent group %s` > - hadoop-common has it's own libraries for accessing `nss`. Example of Knox's > implementation: > https://github.com/apache/knox/blob/master/gateway-provider-identity-assertion-hadoop-groups/src/main/java/org/apache/knox/gateway/identityasserter/hadoop/groups/filter/HadoopGroupProviderFilter.java > - ranger usersync calls `getent` directly: > https://github.com/apache/ranger/blob/da29d1929a54b2b579a74da32e5ea074d0f8e15d/ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java#L49-L51 > - there are modules in most programming languages to access `nss`. > Advantages of this method: > - ability to get users/groups from multiple LDAP directories. _(very common > requirement)_ > - little to no configuration: There is nothing customer or cluster specific > to configure. (i.e. it "just works" with the default configs). > - ease of change: if all services use this approach, you don't have to update > dozens of services every time LDAP changes. > - a lot less overhead: The OS has the users/groups. All of the services > running the OS simply check locally for users/groups. Spares the LDAP servers > and our machines from doing all the lookups. > - group names are guaranteed to be consistent across services. > - as this is becoming the standard in Knox, it makes KnoxSSO more stable. > - less concern about LDAP credentials all over the place. > - easier to pass security/compliance tests since we are utilising the > customers existing identity infrastructure. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6145) Make groovy File functions available for FlowFile to simplify scripting
[ https://issues.apache.org/jira/browse/NIFI-6145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-6145: -- Fix Version/s: (was: 1.10.0) > Make groovy File functions available for FlowFile to simplify scripting > --- > > Key: NIFI-6145 > URL: https://issues.apache.org/jira/browse/NIFI-6145 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Dmitry Lukyanov >Assignee: Matt Burgess >Priority: Minor > Time Spent: 40m > Remaining Estimate: 0h > > I'd like to add groovy File functions to a FlowFile class insige groovy > script to make work with flowfie streams similar like we work with File in > groovy. > Object withInputStream(Closure closure) > Create a new InputStream for this file and passes it into the closure. > Object withOutputStream(Closure closure) > Creates a new OutputStream for this file and passes it into the closure. > Object withReader(String charset, Closure closure) > Create a new BufferedReader for this file using the specified charset and > then passes it into the closure, ensuring the reader is closed after the > closure returns. > Object withWriter(String charset, Closure closure) > Creates a new BufferedWriter for this file, passes it to the closure, and > ensures the stream is flushed and closed after the closure returns. > etc... > > ready to contribute -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6604) Reduce the size of the generated assemblies
[ https://issues.apache.org/jira/browse/NIFI-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-6604: -- Labels: migration-guide release-notes (was: ) > Reduce the size of the generated assemblies > --- > > Key: NIFI-6604 > URL: https://issues.apache.org/jira/browse/NIFI-6604 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Reporter: Aldrin Piri >Assignee: Aldrin Piri >Priority: Blocker > Labels: migration-guide, release-notes > Fix For: 1.10.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > As per the discussion on > https://lists.apache.org/thread.html/d45f33805dcb8096914a06853bc3b8eca1acf50c2ae7b68e75eabfe8@ > we must reduce the assembly size such that we can publish the convenience > binaries as part of the release. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #3682: NIFI-6604: Removing large NARs from generated assemblies to reduce size.
asfgit closed pull request #3682: NIFI-6604: Removing large NARs from generated assemblies to reduce size. URL: https://github.com/apache/nifi/pull/3682 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6520) Upgrade to latest Apache Tika 1.22
[ https://issues.apache.org/jira/browse/NIFI-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934028#comment-16934028 ] Joseph Witt commented on NIFI-6520: --- update tika version and remove extraneous test assertions > Upgrade to latest Apache Tika 1.22 > -- > > Key: NIFI-6520 > URL: https://issues.apache.org/jira/browse/NIFI-6520 > Project: Apache NiFi > Issue Type: Task >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Major > Fix For: 1.10.0 > > Time Spent: 10m > Remaining Estimate: 0h > > [CVE-2019-10088] OOM from a crafted Zip File in Apache Tika's > RecursiveParserWrapper > [CVE-2019-10093] Denial of Service in Apache Tika's 2003ml and 2006ml Parsers > [CVE-2019-10094] StackOverflow from Crafted Package/Compressed Files in > Apache Tika's RecursiveParserWrapper > Not really a blocker but upgrading is generally extremely easy and we should > knock it out. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] joewitt opened a new pull request #3756: NIFI-6520 update to tika 1.22
joewitt opened a new pull request #3756: NIFI-6520 update to tika 1.22
URL: https://github.com/apache/nifi/pull/3756
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and
JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (NIFI-6686) ResizeImage Processor Does Not Handle Some Exceptions Cleanly
[ https://issues.apache.org/jira/browse/NIFI-6686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Knight updated NIFI-6686: - Summary: ResizeImage Processor Does Not Handle Some Exceptions Cleanly (was: ResizeImage Processor Throws Uncaught IOException) > ResizeImage Processor Does Not Handle Some Exceptions Cleanly > - > > Key: NIFI-6686 > URL: https://issues.apache.org/jira/browse/NIFI-6686 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.9.2 > Environment: Ubuntu OS, OpenJDK 1.8 >Reporter: Matthew Knight >Priority: Major > Labels: beginner, imageio > Original Estimate: 48h > Remaining Estimate: 48h > > The ResizeImage Processor's process method which performs the actual image > resize throws an IOException which is not directly caught. The offending > flowfile gets penalized, but whenever it is reprocessed this error will just > happen again because the image itself is the source of the problem and that > has not changed. Processing time becomes wasted as the same file is > processed repeatedly at every penalty duration with the same result. > ImageResize IOExceptions should be caught and handled explicitly as > REL_FAILURE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-5816) SFTP cannot connect due to JSch limitations
[ https://issues.apache.org/jira/browse/NIFI-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933936#comment-16933936 ] ASF subversion and git services commented on NIFI-5816: --- Commit 13381c225439709bd054f560228ba3158f1c2755 in nifi's branch refs/heads/master from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=13381c2 ] NIFI-5816 converting test to integration test as it is test environment specific > SFTP cannot connect due to JSch limitations > --- > > Key: NIFI-5816 > URL: https://issues.apache.org/jira/browse/NIFI-5816 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.8.0 >Reporter: Laurenceau Julien >Assignee: Bryan Bende >Priority: Minor > Fix For: 1.10.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Hi, > The JSch library used for SFTP does not support HostKeyAlgorithms=ed25519 > whereas it is the current standard. This make SFTP / SSH unusable when > dealing with recent openssh config. > On dbeaver project they switched to sshj. > [https://github.com/dbeaver/dbeaver/issues/2202] > [https://community.hortonworks.com/answers/226377/view.html] > > https://stackoverflow.com/questions/2003419/com-jcraft-jsch-jschexception-unknownhostkey > One more argument against JSch is that it does not support rsa key length > other than default (2048). > ssh-keygen -o -t rsa -b 4096 -f id_rsa -> does not work with nifi > ssh-keygen -t rsa -f id_rsa -> works with nifi > Thanks and regards > JL > PS : sorry but I do not know nifi deep enough to fill all fields. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-5816) SFTP cannot connect due to JSch limitations
[ https://issues.apache.org/jira/browse/NIFI-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt resolved NIFI-5816. --- Resolution: Fixed have pushed a commit which converts the problematic test class to an integration test > SFTP cannot connect due to JSch limitations > --- > > Key: NIFI-5816 > URL: https://issues.apache.org/jira/browse/NIFI-5816 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.8.0 >Reporter: Laurenceau Julien >Assignee: Bryan Bende >Priority: Minor > Fix For: 1.10.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Hi, > The JSch library used for SFTP does not support HostKeyAlgorithms=ed25519 > whereas it is the current standard. This make SFTP / SSH unusable when > dealing with recent openssh config. > On dbeaver project they switched to sshj. > [https://github.com/dbeaver/dbeaver/issues/2202] > [https://community.hortonworks.com/answers/226377/view.html] > > https://stackoverflow.com/questions/2003419/com-jcraft-jsch-jschexception-unknownhostkey > One more argument against JSch is that it does not support rsa key length > other than default (2048). > ssh-keygen -o -t rsa -b 4096 -f id_rsa -> does not work with nifi > ssh-keygen -t rsa -f id_rsa -> works with nifi > Thanks and regards > JL > PS : sorry but I do not know nifi deep enough to fill all fields. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-6693) ControllerSearchServiceTest failing to find parameter/paramter context on Linux machine
Joseph Witt created NIFI-6693: - Summary: ControllerSearchServiceTest failing to find parameter/paramter context on Linux machine Key: NIFI-6693 URL: https://issues.apache.org/jira/browse/NIFI-6693 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.10.0 Environment: Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-03T15:39:06-04:00) Maven home: /development/apache-maven-3.5.0 Java version: 1.8.0_191, vendor: Oracle Corporation Java home: /usr/java/jdk1.8.0_191-amd64/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "5.2.9-200.fc30.x86_64", arch: "amd64", family: "unix" Reporter: Joseph Witt Assignee: Matt Gilman Fix For: 1.10.0 [ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.329 s <<< FAILURE! - in org.apache.nifi.web.controller.ControllerSearchServiceTest [ERROR] testSearchParameterContext(org.apache.nifi.web.controller.ControllerSearchServiceTest) Time elapsed: 0.014 s <<< FAILURE! java.lang.AssertionError at org.apache.nifi.web.controller.ControllerSearchServiceTest.testSearchParameterContext(ControllerSearchServiceTest.java:374) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6693) ControllerSearchServiceTest failing to find parameter/paramter context on Linux machine
[ https://issues.apache.org/jira/browse/NIFI-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933970#comment-16933970 ] Joseph Witt commented on NIFI-6693: --- the test always fails including when only building that single module. the result of the parameter context is zero found. no parameters found either. both are tested by an assertion to have at least a single result for 'foo'. on osx this works. not clear to me how this could be system to system specific. > ControllerSearchServiceTest failing to find parameter/paramter context on > Linux machine > --- > > Key: NIFI-6693 > URL: https://issues.apache.org/jira/browse/NIFI-6693 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.10.0 > Environment: Apache Maven 3.5.0 > (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-03T15:39:06-04:00) > Maven home: /development/apache-maven-3.5.0 > Java version: 1.8.0_191, vendor: Oracle Corporation > Java home: /usr/java/jdk1.8.0_191-amd64/jre > Default locale: en_US, platform encoding: UTF-8 > OS name: "linux", version: "5.2.9-200.fc30.x86_64", arch: "amd64", family: > "unix" >Reporter: Joseph Witt >Assignee: Matt Gilman >Priority: Blocker > Fix For: 1.10.0 > > > [ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.329 > s <<< FAILURE! - in org.apache.nifi.web.controller.ControllerSearchServiceTest > [ERROR] > testSearchParameterContext(org.apache.nifi.web.controller.ControllerSearchServiceTest) > Time elapsed: 0.014 s <<< FAILURE! > java.lang.AssertionError > at > org.apache.nifi.web.controller.ControllerSearchServiceTest.testSearchParameterContext(ControllerSearchServiceTest.java:374) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-6694) Travis build is unstable due to Hive3-exec access issue
Joseph Witt created NIFI-6694: - Summary: Travis build is unstable due to Hive3-exec access issue Key: NIFI-6694 URL: https://issues.apache.org/jira/browse/NIFI-6694 Project: Apache NiFi Issue Type: Bug Environment: Travis-CI Reporter: Joseph Witt Assignee: Jeff Storck Fix For: 1.10.0 If this is a result of caching issues/etc.. should we disable caching? I think this occurred recently and required manual changes to work. [~jtstorck] what options do we have? [ERROR] error: error reading /home/travis/.m2/repository/org/apache/hive/hive-exec/3.1.0/hive-exec-3.1.0.jar; error in opening zip file [INFO] [INFO] --- maven-compiler-plugin:3.8.1:testCompile (groovy-tests) @ nifi-hive3-processors --- [INFO] Changes detected - recompiling the module! [INFO] Using Groovy-Eclipse compiler to compile both Java and Groovy files [INFO] Compiling in a forked process using /home/travis/.m2/repository/org/codehaus/groovy/groovy-eclipse-batch/2.5.4-01/groovy-eclipse-batch-2.5.4-01.jar [INFO] [INFO] --- maven-resources-plugin:3.0.2:testResources (default-testResources) @ nifi-livy-processors --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] [INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ nifi-livy-processors --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 3 source files to /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-spark-bundle/nifi-livy-processors/target/test-classes [INFO] - [ERROR] COMPILATION ERROR : [INFO] - [ERROR] Failure executing groovy-eclipse compiler: -- 1. ERROR in /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive3-processors/src/test/java/org/apache/nifi/util/orc/TestNiFiOrcUtils.java (at line 0) /* ^ The type org.apache.hadoop.hive.ql.metadata.Table cannot be resolved. It is indirectly referenced from required .class files -- 1 problem (1 error) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6694) Travis build is unstable due to Hive3-exec access issue
[ https://issues.apache.org/jira/browse/NIFI-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933974#comment-16933974 ] Joseph Witt commented on NIFI-6694: --- https://travis-ci.org/apache/nifi/jobs/587280103 that is an example of a build that failed for most things due to this hive-exec jar thing. but one of the builds did work... > Travis build is unstable due to Hive3-exec access issue > --- > > Key: NIFI-6694 > URL: https://issues.apache.org/jira/browse/NIFI-6694 > Project: Apache NiFi > Issue Type: Bug > Environment: Travis-CI >Reporter: Joseph Witt >Assignee: Jeff Storck >Priority: Critical > Fix For: 1.10.0 > > > If this is a result of caching issues/etc.. should we disable caching? I > think this occurred recently and required manual changes to work. > [~jtstorck] what options do we have? > [ERROR] error: error reading > /home/travis/.m2/repository/org/apache/hive/hive-exec/3.1.0/hive-exec-3.1.0.jar; > error in opening zip file > [INFO] > [INFO] --- maven-compiler-plugin:3.8.1:testCompile (groovy-tests) @ > nifi-hive3-processors --- > [INFO] Changes detected - recompiling the module! > [INFO] Using Groovy-Eclipse compiler to compile both Java and Groovy files > [INFO] Compiling in a forked process using > /home/travis/.m2/repository/org/codehaus/groovy/groovy-eclipse-batch/2.5.4-01/groovy-eclipse-batch-2.5.4-01.jar > [INFO] > [INFO] --- maven-resources-plugin:3.0.2:testResources (default-testResources) > @ nifi-livy-processors --- > [INFO] Using 'UTF-8' encoding to copy filtered resources. > [INFO] > [INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ > nifi-livy-processors --- > [INFO] Changes detected - recompiling the module! > [INFO] Compiling 3 source files to > /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-spark-bundle/nifi-livy-processors/target/test-classes > [INFO] - > [ERROR] COMPILATION ERROR : > [INFO] - > [ERROR] Failure executing groovy-eclipse compiler: > -- > 1. ERROR in > /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive3-processors/src/test/java/org/apache/nifi/util/orc/TestNiFiOrcUtils.java > (at line 0) > /* > ^ > The type org.apache.hadoop.hive.ql.metadata.Table cannot be resolved. It is > indirectly referenced from required .class files > -- > 1 problem (1 error) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6056) Cassandra services NAR incorrectly bundles nifi-mock jar
[ https://issues.apache.org/jira/browse/NIFI-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-6056: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Cassandra services NAR incorrectly bundles nifi-mock jar > > > Key: NIFI-6056 > URL: https://issues.apache.org/jira/browse/NIFI-6056 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.8.0, 1.9.0 >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Trivial > Fix For: 1.10.0 > > > The cassandra services NAR incorrectly bundles nifi-mock jar which should > only be used for testing: > work/nar//extensions/nifi-cassandra-services-nar-1.9.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-mock-1.9.0.jar -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6313) PutGCSObject performance seems slow
[ https://issues.apache.org/jira/browse/NIFI-6313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-6313: -- Fix Version/s: (was: 1.10.0) > PutGCSObject performance seems slow > --- > > Key: NIFI-6313 > URL: https://issues.apache.org/jira/browse/NIFI-6313 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Affects Versions: 1.9.2 >Reporter: Jasper Knulst >Priority: Major > > The PutGCSObject processor to transfer files to Google Cloud Platform bucket > has bad transfer speeds. > It is impossible to put any hard figures on the throughput as it seems > dependent on: > -Network location of the Nifi node (situated in GC or not) > -Network bandwidth > -Nifi node specs > > After performing benchmarks on multiple Nifi clusters (ranging from test > setups to prod. sites) the throughput can range from 8MB/s to 800MB/s. > Slow really means, slow in comparison to gsutil. If you run gsutil directly > from the Nifi node the throughput speed goes up 5 to 8 times (without > 'parallel_composite_upload') and up to 16 times faster with > 'parallel_composite_upload'. > > The GC Java API on which Nifi's GCS processors are built, does not have the > same optimizations as gsutil and maybe isn't supported/maintained. The > Storage.create method is even deprecated. > Still there must be ways to speed up transfers the GCS by implementing > parallel composite uploads in chuncks and config options on the GCS > processors -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6313) PutGCSObject performance seems slow
[ https://issues.apache.org/jira/browse/NIFI-6313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933999#comment-16933999 ] Joseph Witt commented on NIFI-6313: --- [~markap14] the GCE massive scale testing you did - was it with this processor? [~jasperknulst] this is important to progress but it needs a PR and review traction before we set the fix version. > PutGCSObject performance seems slow > --- > > Key: NIFI-6313 > URL: https://issues.apache.org/jira/browse/NIFI-6313 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Affects Versions: 1.9.2 >Reporter: Jasper Knulst >Priority: Major > Fix For: 1.10.0 > > > The PutGCSObject processor to transfer files to Google Cloud Platform bucket > has bad transfer speeds. > It is impossible to put any hard figures on the throughput as it seems > dependent on: > -Network location of the Nifi node (situated in GC or not) > -Network bandwidth > -Nifi node specs > > After performing benchmarks on multiple Nifi clusters (ranging from test > setups to prod. sites) the throughput can range from 8MB/s to 800MB/s. > Slow really means, slow in comparison to gsutil. If you run gsutil directly > from the Nifi node the throughput speed goes up 5 to 8 times (without > 'parallel_composite_upload') and up to 16 times faster with > 'parallel_composite_upload'. > > The GC Java API on which Nifi's GCS processors are built, does not have the > same optimizations as gsutil and maybe isn't supported/maintained. The > Storage.create method is even deprecated. > Still there must be ways to speed up transfers the GCS by implementing > parallel composite uploads in chuncks and config options on the GCS > processors -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] joewitt commented on issue #3682: NIFI-6604: Removing large NARs from generated assemblies to reduce size.
joewitt commented on issue #3682: NIFI-6604: Removing large NARs from generated assemblies to reduce size. URL: https://github.com/apache/nifi/pull/3682#issuecomment-533388331 @bbende for now i'd say we just drop them. but we could later add profile to include them for convenience sake. or just let folks pull them as needed since were moving to the a la carte model with versioned extensions with the registry anyway This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6369) ValidateRecord routes JSON records to 'invalid' when validating a timestamp, even if format is valid
[
https://issues.apache.org/jira/browse/NIFI-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934006#comment-16934006
]
ASF subversion and git services commented on NIFI-6369:
---
Commit e18d9ce1e8d576a21630dcd67e9260ff5c3f9bf0 in nifi's branch
refs/heads/master from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e18d9ce ]
NIFI-6369: This closes #3560. Updated JSON Readers to convert String values to
Date/Time/Timestamp objects when appropriate according to the schema and the
configured pattern
NIFI-6939: Fix to WriteJsonRecord to deal with date/time/timestamp when no
format is explicitly set
> ValidateRecord routes JSON records to 'invalid' when validating a timestamp,
> even if format is valid
>
>
> Key: NIFI-6369
> URL: https://issues.apache.org/jira/browse/NIFI-6369
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.10.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> If a JSON Reader (either JsonPath or JsonTree reader) is used with
> ValidateRecord and the schema indicates that a field is a timestamp (e.g.,
> {"name": "timestamp", "type": \{"type": "long",
> "logicalType":"timestamp-millis"}}
> ) with the JSON having a value like "2019/06/10 12:00:00Z" and a format of
> "/MM/dd HH:mm:ss'Z'" The processor considers this invalid, even when
> setting "strict type checking" to "false".
> This appears to be due to the way that ValidateRecord calls
> RecordReader.nextReader() - it calls it "reader.nextRecord( /* coerce types
> **/* false, /* drop unknown types **/* false)"
> When this is called, the Record Reader checks the 'coerce types' flag and
> determines that it should not coerce the string "2019/06/10 12:00:00Z" into a
> timestamp. However, for any date/time/timestamp fields, if the value is a
> String and it matches the configured pattern, the Record Readers should go
> ahead and convert the value into the appropriate java.sql.Date,
> java.sql.Time, java.sql.Timestamp object because that's clearly the intent of
> the schema and formatting being applied.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-6369) ValidateRecord routes JSON records to 'invalid' when validating a timestamp, even if format is valid
[
https://issues.apache.org/jira/browse/NIFI-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Witt updated NIFI-6369:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
+1 merged to master
> ValidateRecord routes JSON records to 'invalid' when validating a timestamp,
> even if format is valid
>
>
> Key: NIFI-6369
> URL: https://issues.apache.org/jira/browse/NIFI-6369
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> If a JSON Reader (either JsonPath or JsonTree reader) is used with
> ValidateRecord and the schema indicates that a field is a timestamp (e.g.,
> {"name": "timestamp", "type": \{"type": "long",
> "logicalType":"timestamp-millis"}}
> ) with the JSON having a value like "2019/06/10 12:00:00Z" and a format of
> "/MM/dd HH:mm:ss'Z'" The processor considers this invalid, even when
> setting "strict type checking" to "false".
> This appears to be due to the way that ValidateRecord calls
> RecordReader.nextReader() - it calls it "reader.nextRecord( /* coerce types
> **/* false, /* drop unknown types **/* false)"
> When this is called, the Record Reader checks the 'coerce types' flag and
> determines that it should not coerce the string "2019/06/10 12:00:00Z" into a
> timestamp. However, for any date/time/timestamp fields, if the value is a
> String and it matches the configured pattern, the Record Readers should go
> ahead and convert the value into the appropriate java.sql.Date,
> java.sql.Time, java.sql.Timestamp object because that's clearly the intent of
> the schema and formatting being applied.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] SamHjelmfelt commented on issue #3755: NIFI-6692: Fixing NullPointerException when not using Kerberos
SamHjelmfelt commented on issue #3755: NIFI-6692: Fixing NullPointerException when not using Kerberos URL: https://github.com/apache/nifi/pull/3755#issuecomment-58921 Build failed due to the test: "org.apache.nifi.processors.standard.util.TestSFTPTransferWithSSHTestServer" failing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6675) JsonPath put expression language support
[ https://issues.apache.org/jira/browse/NIFI-6675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933906#comment-16933906 ] ASF subversion and git services commented on NIFI-6675: --- Commit 79e4f360f489764ab95b416658844c18cc7a7998 in nifi's branch refs/heads/master from mans2singh [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=79e4f36 ] NIFI-6675 - This closes #3741. Added support for JsonPath put operation > JsonPath put expression language support > > > Key: NIFI-6675 > URL: https://issues.apache.org/jira/browse/NIFI-6675 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.9.2 > Environment: All >Reporter: Mans Singh >Assignee: Mans Singh >Priority: Minor > Labels: expression-language, jsonpath, put > Fix For: 1.10.0 > > Original Estimate: 12h > Time Spent: 10m > Remaining Estimate: 11h 50m > > Add nifi-expressions support for JsonPath put operations -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-6675) JsonPath put expression language support
[ https://issues.apache.org/jira/browse/NIFI-6675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt resolved NIFI-6675. --- Resolution: Fixed +1 merged to master - thanks > JsonPath put expression language support > > > Key: NIFI-6675 > URL: https://issues.apache.org/jira/browse/NIFI-6675 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.9.2 > Environment: All >Reporter: Mans Singh >Assignee: Mans Singh >Priority: Minor > Labels: expression-language, jsonpath, put > Fix For: 1.10.0 > > Original Estimate: 12h > Time Spent: 20m > Remaining Estimate: 11h 40m > > Add nifi-expressions support for JsonPath put operations -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #3741: NIFI-6675 - Added support for JsonPath put operation
asfgit closed pull request #3741: NIFI-6675 - Added support for JsonPath put operation URL: https://github.com/apache/nifi/pull/3741 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] joewitt commented on issue #3643: NIFI-6542: Upgrade nifi-hadoop-bundles Hadoop version dependency to 3…
joewitt commented on issue #3643: NIFI-6542: Upgrade nifi-hadoop-bundles Hadoop version dependency to 3… URL: https://github.com/apache/nifi/pull/3643#issuecomment-533368284 the yetus one is fine but we need to add the dnsjava entry to the LICENSE file of the nifi-assembly/LICENSE and the nifi-hadoop-*-nar that this gets bundled into. The entry we want to add comes straight from https://github.com/dnsjava/dnsjava/blob/master/LICENSE and you can follow the examples of other licenses. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6542) Upgrade nifi-hadoop-bundles Hadoop version dependency to 3.2.0
[ https://issues.apache.org/jira/browse/NIFI-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933915#comment-16933915 ] Joseph Witt commented on NIFI-6542: --- fix version can be added once the PR is ready to roll. it just needs licensing updates. > Upgrade nifi-hadoop-bundles Hadoop version dependency to 3.2.0 > -- > > Key: NIFI-6542 > URL: https://issues.apache.org/jira/browse/NIFI-6542 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.10.0, 1.9.2 >Reporter: David Mollitor >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > Upgrade to Hadoop libraries to 3.2 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6542) Upgrade nifi-hadoop-bundles Hadoop version dependency to 3.2.0
[ https://issues.apache.org/jira/browse/NIFI-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-6542: -- Fix Version/s: (was: 1.10.0) > Upgrade nifi-hadoop-bundles Hadoop version dependency to 3.2.0 > -- > > Key: NIFI-6542 > URL: https://issues.apache.org/jira/browse/NIFI-6542 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.10.0, 1.9.2 >Reporter: David Mollitor >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > Upgrade to Hadoop libraries to 3.2 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (NIFI-5816) SFTP cannot connect due to JSch limitations
[ https://issues.apache.org/jira/browse/NIFI-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt reopened NIFI-5816: --- reopening to address a range of test failures being observed in Travis and linux. The test with the ssh server appears to be better suited as an Integration test as certain aspects like expected file permission/etc.. may differ from system to system. > SFTP cannot connect due to JSch limitations > --- > > Key: NIFI-5816 > URL: https://issues.apache.org/jira/browse/NIFI-5816 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.8.0 >Reporter: Laurenceau Julien >Assignee: Bryan Bende >Priority: Minor > Fix For: 1.10.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Hi, > The JSch library used for SFTP does not support HostKeyAlgorithms=ed25519 > whereas it is the current standard. This make SFTP / SSH unusable when > dealing with recent openssh config. > On dbeaver project they switched to sshj. > [https://github.com/dbeaver/dbeaver/issues/2202] > [https://community.hortonworks.com/answers/226377/view.html] > > https://stackoverflow.com/questions/2003419/com-jcraft-jsch-jschexception-unknownhostkey > One more argument against JSch is that it does not support rsa key length > other than default (2048). > ssh-keygen -o -t rsa -b 4096 -f id_rsa -> does not work with nifi > ssh-keygen -t rsa -f id_rsa -> works with nifi > Thanks and regards > JL > PS : sorry but I do not know nifi deep enough to fill all fields. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-5816) SFTP cannot connect due to JSch limitations
[ https://issues.apache.org/jira/browse/NIFI-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933932#comment-16933932 ] Joseph Witt commented on NIFI-5816: --- [ERROR] Failures: [ERROR] TestSFTPTransferWithSSHTestServer.testGetListingSimple:154 expected: but was: [INFO] there was a range of other errors seen for that test class on Linux as well. all of it seems related to different behaviors we'd see on different systems. the test class looks valuable but as an integration test. > SFTP cannot connect due to JSch limitations > --- > > Key: NIFI-5816 > URL: https://issues.apache.org/jira/browse/NIFI-5816 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.8.0 >Reporter: Laurenceau Julien >Assignee: Bryan Bende >Priority: Minor > Fix For: 1.10.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Hi, > The JSch library used for SFTP does not support HostKeyAlgorithms=ed25519 > whereas it is the current standard. This make SFTP / SSH unusable when > dealing with recent openssh config. > On dbeaver project they switched to sshj. > [https://github.com/dbeaver/dbeaver/issues/2202] > [https://community.hortonworks.com/answers/226377/view.html] > > https://stackoverflow.com/questions/2003419/com-jcraft-jsch-jschexception-unknownhostkey > One more argument against JSch is that it does not support rsa key length > other than default (2048). > ssh-keygen -o -t rsa -b 4096 -f id_rsa -> does not work with nifi > ssh-keygen -t rsa -f id_rsa -> works with nifi > Thanks and regards > JL > PS : sorry but I do not know nifi deep enough to fill all fields. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
