[jira] [Comment Edited] (NIFI-10143) ListFTP/GetFTP has issues with danish special characters (UTF-8)
[
https://issues.apache.org/jira/browse/NIFI-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557242#comment-17557242
]
Jens M Kofoed edited comment on NIFI-10143 at 6/22/22 5:38 AM:
---
[~pgrey] I have made some more test.
Testing with windows built in command line ftp. After connecting to the ftp
server I see this: "200 OPTS UTF8 command successful - UTF8 encoding now ON"
Testing different scenario with non-english words works fine with danish
charecters. With my currently selected character set, I can't see arabic
characters in the command prompt, but FTP are still able to handle files with
Arabic characters, and upload them just fine.
So it seems like MS FTP are handling utf8 just fine. But I still have issues
with NIFI accessing MS FTP. But it seems like NIFI don't have issue with
FileZilla FTP Server. So where the different is, I don't know.
Testing from my Ubuntu 20.04 with lftp, it also works fine with non-english
letters agains MS FTP.
Unfortunately we are not able to switch to FileZilla FTP Server since it does
not support MS-AD and NTFS rights
was (Author: jmkofoed):
[~pgrey] I have made some more test.
Testing with windows built in command line ftp. After connecting to the ftp
server I see this: "200 OPTS UTF8 command successful - UTF8 encoding now ON"
Testing different scenario with non-english words works fine with danish
charecters. With my currently selected character set, I can't see arabic
characters in the command prompt, but FTP are still able to handle files with
Arabic characters, and upload them just fine.
So it seems like MS FTP are handling utf8 just fine. But I still have issues
with NIFI accessing MS FTP. But it seems like NIFI don't have issue with
FileZilla FTP Server. So where the different is, I don't know.
Unfortunately we are not able to switch to FileZilla FTP Server since it does
not support MS-AD and NTFS rights
> ListFTP/GetFTP has issues with danish special characters (UTF-8)
>
>
> Key: NIFI-10143
> URL: https://issues.apache.org/jira/browse/NIFI-10143
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.16.2
> Environment: NIFI Version 1.16.2, Java 8
>Reporter: Jens M Kofoed
>Assignee: Paul Grey
>Priority: Major
>
> ListFTP and GetFTP has issues listing file/directories including danish
> charaters (æøå).
> The ftp server is MS IIS FTP, and UTF-8 has been set to true.
> with a small test flow, I used a GenerateFlowfile and added 2 Properties.
> filename="test.txt" and path="æøå". The output is going to a PutFTP, with
> Remote path set to ${path}
> The PutFTP does indeed create a subfolder at the ftp server, named æøå. But...
> With a ListFTP or GetFTP process, looking into the same root folder gives an
> error.
> ListFTP[id=e4c43ea1-2d5d-1fea--89c101ce] Unable to get listing from
> /���; skipping: java.io.IOException: Cannot list files for non-existent
> directory /���
>
> Testing the old case from NIFI-8250. I got issues with the arabic charaters
> too.
> PutFTP[id=e4c43ef3-2d5d-1fea--ea9da690] Unable to transfer
> FlowFile[filename=اختبار] to remote host 192.168.1.2 due to IOException
> thrown from PutFTP[id=e4c43ef3-2d5d-1fea--ea9da690]:
> java.io.IOException: Failed to store file .اختبار to /test/اختبار due to: 550
> The filename, directory name, or volume label syntax is incorrect.
> : java.io.IOException: Failed to store file .اختبار to /test/اختبار due to:
> 550 The filename, directory name, or volume label syntax is incorrect.
> ; routing to failure: {}
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (NIFI-10143) ListFTP/GetFTP has issues with danish special characters (UTF-8)
[
https://issues.apache.org/jira/browse/NIFI-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557242#comment-17557242
]
Jens M Kofoed commented on NIFI-10143:
--
[~pgrey] I have made some more test.
Testing with windows built in command line ftp. After connecting to the ftp
server I see this: "200 OPTS UTF8 command successful - UTF8 encoding now ON"
Testing different scenario with non-english words works fine with danish
charecters. With my currently selected character set, I can't see arabic
characters in the command prompt, but FTP are still able to handle files with
Arabic characters, and upload them just fine.
So it seems like MS FTP are handling utf8 just fine. But I still have issues
with NIFI accessing MS FTP. But it seems like NIFI don't have issue with
FileZilla FTP Server. So where the different is, I don't know.
Unfortunately we are not able to switch to FileZilla FTP Server since it does
not support MS-AD and NTFS rights
> ListFTP/GetFTP has issues with danish special characters (UTF-8)
>
>
> Key: NIFI-10143
> URL: https://issues.apache.org/jira/browse/NIFI-10143
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.16.2
> Environment: NIFI Version 1.16.2, Java 8
>Reporter: Jens M Kofoed
>Assignee: Paul Grey
>Priority: Major
>
> ListFTP and GetFTP has issues listing file/directories including danish
> charaters (æøå).
> The ftp server is MS IIS FTP, and UTF-8 has been set to true.
> with a small test flow, I used a GenerateFlowfile and added 2 Properties.
> filename="test.txt" and path="æøå". The output is going to a PutFTP, with
> Remote path set to ${path}
> The PutFTP does indeed create a subfolder at the ftp server, named æøå. But...
> With a ListFTP or GetFTP process, looking into the same root folder gives an
> error.
> ListFTP[id=e4c43ea1-2d5d-1fea--89c101ce] Unable to get listing from
> /���; skipping: java.io.IOException: Cannot list files for non-existent
> directory /���
>
> Testing the old case from NIFI-8250. I got issues with the arabic charaters
> too.
> PutFTP[id=e4c43ef3-2d5d-1fea--ea9da690] Unable to transfer
> FlowFile[filename=اختبار] to remote host 192.168.1.2 due to IOException
> thrown from PutFTP[id=e4c43ef3-2d5d-1fea--ea9da690]:
> java.io.IOException: Failed to store file .اختبار to /test/اختبار due to: 550
> The filename, directory name, or volume label syntax is incorrect.
> : java.io.IOException: Failed to store file .اختبار to /test/اختبار due to:
> 550 The filename, directory name, or volume label syntax is incorrect.
> ; routing to failure: {}
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1354: MINIFICPP-1860 Integrate readability clang-tidy checks in CI
szaszm commented on code in PR #1354:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1354#discussion_r903204296
##
extensions/http-curl/client/HTTPClient.cpp:
##
@@ -437,13 +438,13 @@ void HTTPClient::setFollowRedirects(bool follow) {
}
bool HTTPClient::isValidHttpHeaderField(std::string_view field_name) {
- if (field_name.size() == 0) {
+ if (field_name.empty()) {
return false;
}
// RFC822 3.1.2: The field-name must be composed of printable ASCII
characters
// (i.e., characters that have values between 33. and 126., decimal,
except colon).
- for (auto ch : field_name) {
+ for (auto ch : field_name) { // NOLINT(readability-use-anyofallof)
Review Comment:
```suggestion
return std::all_of(std::begin(field_name), std::end(field_name), [](char
c) { return c >= 33 && c <= 126 && c != ':'; });
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (MINIFICPP-1867) tzdata is missing from docker-minimal
[ https://issues.apache.org/jira/browse/MINIFICPP-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1867: Resolution: Fixed Status: Resolved (was: Patch Available) > tzdata is missing from docker-minimal > - > > Key: MINIFICPP-1867 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1867 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Fix For: 0.13.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > tzdata is required for timezone features (cron scheduler, > expression-language), but it is missing on docker-minimal image (based on > alpine) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1356: MINIFICPP-1867 added tzdata to docker-minimal
szaszm closed pull request #1356: MINIFICPP-1867 added tzdata to docker-minimal URL: https://github.com/apache/nifi-minifi-cpp/pull/1356 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1353: MINIFICPP-1863 fixing duplicate entry in PutGCSObject::properties
szaszm closed pull request #1353: MINIFICPP-1863 fixing duplicate entry in PutGCSObject::properties URL: https://github.com/apache/nifi-minifi-cpp/pull/1353 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9983) Add new attribute to indicate the routed relation in QueryRecord processor
[ https://issues.apache.org/jira/browse/NIFI-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-9983: - Fix Version/s: 1.17.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add new attribute to indicate the routed relation in QueryRecord processor > -- > > Key: NIFI-9983 > URL: https://issues.apache.org/jira/browse/NIFI-9983 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Julien G. >Priority: Major > Fix For: 1.17.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > I think the QueryRecord should provide the same functionality as the > [RouteOnAttribute|https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/RouteOnAttribute.java] > processor. > And one of the features provided by the processor is the writing of the > output relation in an attribute. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (NIFI-9983) Add new attribute to indicate the routed relation in QueryRecord processor
[ https://issues.apache.org/jira/browse/NIFI-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557142#comment-17557142 ] ASF subversion and git services commented on NIFI-9983: --- Commit bc84532a8b27eae2e7f3ab284473c1cbc9035be6 in nifi's branch refs/heads/main from Juldrixx [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=bc84532a8b ] NIFI-9983 Add output relation name attribute in QueryRecord processor (#6011) * NIFI-9983 Add output relation name attribute > Add new attribute to indicate the routed relation in QueryRecord processor > -- > > Key: NIFI-9983 > URL: https://issues.apache.org/jira/browse/NIFI-9983 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Julien G. >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > > I think the QueryRecord should provide the same functionality as the > [RouteOnAttribute|https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/RouteOnAttribute.java] > processor. > And one of the features provided by the processor is the writing of the > output relation in an attribute. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [nifi] markap14 merged pull request #6011: NIFI-9983 Add output relation name attribute in QueryRecord processor
markap14 merged PR #6011: URL: https://github.com/apache/nifi/pull/6011 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on pull request #6011: NIFI-9983 Add output relation name attribute in QueryRecord processor
markap14 commented on PR #6011: URL: https://github.com/apache/nifi/pull/6011#issuecomment-1162404504 Thanks @juldrixx changes look good to me. I'm a +1, will merge to main. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9996) ListFile does not list root directory when Recurse Subdirectories is active and Path Filter is specified
[ https://issues.apache.org/jira/browse/NIFI-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-9996: - Fix Version/s: 1.17.0 Resolution: Fixed Status: Resolved (was: Patch Available) > ListFile does not list root directory when Recurse Subdirectories is active > and Path Filter is specified > > > Key: NIFI-9996 > URL: https://issues.apache.org/jira/browse/NIFI-9996 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.16.1 >Reporter: Andrea Molteni >Assignee: Paul Grey >Priority: Major > Fix For: 1.17.0 > > Attachments: image-2022-05-06-18-35-31-937.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > ListFile does not list root directory content when Recurse Subdirectories is > active and Path Filter is specified. > you need to create a directory structure like the following: > -MYDIR > --AAA > --BBB > --CCC > --LOG > if you want to LIST all the *.TXT files from MYDIR and subdirectories ( > excluding LOG dir) i have to configure the list file processor as following > !image-2022-05-06-18-35-31-937.png! > Once activated, the processor will list file from all subdirs excluding LOG > directory and the root dir content too. > In prevous nifi version ( 1.13.1) the root folder was listed too. > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (NIFI-9996) ListFile does not list root directory when Recurse Subdirectories is active and Path Filter is specified
[ https://issues.apache.org/jira/browse/NIFI-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557136#comment-17557136 ] ASF subversion and git services commented on NIFI-9996: --- Commit f9beb507fdbde55c62170949393aeb542fe23b39 in nifi's branch refs/heads/main from greyp9 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=f9beb507fd ] NIFI-9996 - ListFile does not list root directory when Recurse Subdirectories is active and Path Filter is specified (#6110) > ListFile does not list root directory when Recurse Subdirectories is active > and Path Filter is specified > > > Key: NIFI-9996 > URL: https://issues.apache.org/jira/browse/NIFI-9996 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.16.1 >Reporter: Andrea Molteni >Assignee: Paul Grey >Priority: Major > Fix For: 1.17.0 > > Attachments: image-2022-05-06-18-35-31-937.png > > Time Spent: 40m > Remaining Estimate: 0h > > ListFile does not list root directory content when Recurse Subdirectories is > active and Path Filter is specified. > you need to create a directory structure like the following: > -MYDIR > --AAA > --BBB > --CCC > --LOG > if you want to LIST all the *.TXT files from MYDIR and subdirectories ( > excluding LOG dir) i have to configure the list file processor as following > !image-2022-05-06-18-35-31-937.png! > Once activated, the processor will list file from all subdirs excluding LOG > directory and the root dir content too. > In prevous nifi version ( 1.13.1) the root folder was listed too. > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [nifi] markap14 merged pull request #6110: NIFI-9996 - ListFile does not list root directory when Recurse Subdir…
markap14 merged PR #6110: URL: https://github.com/apache/nifi/pull/6110 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on pull request #6110: NIFI-9996 - ListFile does not list root directory when Recurse Subdir…
markap14 commented on PR #6110: URL: https://github.com/apache/nifi/pull/6110#issuecomment-1162382417 Looks good to me as well. Thanks @greyp9 and @emiliosetiadarma +1 will merge to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-10106) Fix potential NPE in LookupRecord
[ https://issues.apache.org/jira/browse/NIFI-10106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFI-10106. --- Fix Version/s: 1.17.0 Resolution: Fixed > Fix potential NPE in LookupRecord > - > > Key: NIFI-10106 > URL: https://issues.apache.org/jira/browse/NIFI-10106 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > Fix For: 1.17.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > _LookupRecord_ can throw a _NullPointerException_ during > _createLookupCoordinates_ when one of the record path evaluation results in a > FieldValue that has no RecordField. > This can occur for example when a dynamic property is set that evaluates to a > LiteralValuePath, like a _toString_ [RecordPath Domain-Specific Language > (DSL)|https://localhost:18443/nifi-docs/html/record-path-guide.html] call. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (NIFI-10106) Fix potential NPE in LookupRecord
[ https://issues.apache.org/jira/browse/NIFI-10106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557129#comment-17557129 ] ASF subversion and git services commented on NIFI-10106: Commit c08996515bbf76253563e6f050b32dd8549919e7 in nifi's branch refs/heads/main from tpalfy [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c08996515b ] NIFI-10106 Avoid NPE in LookupRecord.createLookupCoordinates (#6111) > Fix potential NPE in LookupRecord > - > > Key: NIFI-10106 > URL: https://issues.apache.org/jira/browse/NIFI-10106 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > _LookupRecord_ can throw a _NullPointerException_ during > _createLookupCoordinates_ when one of the record path evaluation results in a > FieldValue that has no RecordField. > This can occur for example when a dynamic property is set that evaluates to a > LiteralValuePath, like a _toString_ [RecordPath Domain-Specific Language > (DSL)|https://localhost:18443/nifi-docs/html/record-path-guide.html] call. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [nifi] markap14 commented on pull request #6111: NIFI-10106 Avoid NPE in LookupRecord.createLookupCoordinates
markap14 commented on PR #6111: URL: https://github.com/apache/nifi/pull/6111#issuecomment-1162359250 Thanks for fixing @tpalfy +1 merged to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 merged pull request #6111: NIFI-10106 Avoid NPE in LookupRecord.createLookupCoordinates
markap14 merged PR #6111: URL: https://github.com/apache/nifi/pull/6111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10109) Preserve Field Order in JsonTreeReader Map Records
[
https://issues.apache.org/jira/browse/NIFI-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-10109:
--
Fix Version/s: 1.17.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Preserve Field Order in JsonTreeReader Map Records
> --
>
> Key: NIFI-10109
> URL: https://issues.apache.org/jira/browse/NIFI-10109
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Fix For: 1.17.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> The current implementation of the {{JsonTreeReader}} reads JSON Object nodes
> into a {{HashMap}} for representation as a {{{}MapRecord{}}}. Schemas
> generated from JSON Objects retain field order, but the internal {{Map}}
> representation does not preserve the field order due to the non-deterministic
> use of {{{}HashMap{}}}. Changing the internal representation to a
> {{LinkedHashMap}} would preserve field order for methods such as
> {{{}Record.toMap(){}}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (NIFI-10147) Remove Support for Java 8
[ https://issues.apache.org/jira/browse/NIFI-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557127#comment-17557127 ] Pierre Villard commented on NIFI-10147: --- Something I would consider for removal with NiFi 2.0 is the concept of variables. > Remove Support for Java 8 > - > > Key: NIFI-10147 > URL: https://issues.apache.org/jira/browse/NIFI-10147 > Project: Apache NiFi > Issue Type: Epic > Components: Core Framework, MiNiFi, NiFi Registry, NiFi Stateless >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Although multiple vendors have planned extended support for Java 8, a number > of projects are in the process of deprecating and removing support for Java > 8. These projects include the following: > - [Jetty 9.4|https://github.com/eclipse/jetty.project/issues/7958] > - [Spring > 5|https://spring.io/blog/2021/09/02/a-java-17-and-jakarta-ee-9-baseline-for-spring-framework-6] > - [OpenSAML 3|https://shibboleth.atlassian.net/wiki/spaces/OSAML/overview] > Apache NiFi should also remove support for Java 8 in order to maintain active > support for various components. > Removing support for Java 8 requires a new major version release and will > also require several build configuration changes including the following: > - Updating Maven source and compiler settings > - Updating minimum Java build version > - Removing optional Maven build profiles > - Updating GitHub workflows > Initial discussion on a migration strategy included a tentative target of the > end of calendar year 2022. > Implementation should not proceed until after a minor release of Apache NiFi > 1.17.0. > Mailing list discussion: > [https://lists.apache.org/thread/mm1xf3b9nvrcgytb92oy3swvvc45fl34] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (NIFI-10109) Preserve Field Order in JsonTreeReader Map Records
[
https://issues.apache.org/jira/browse/NIFI-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557126#comment-17557126
]
ASF subversion and git services commented on NIFI-10109:
Commit 5febd47c72db79a0dd063b662ed8f23c2d9530be in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=5febd47c72 ]
NIFI-10109 Changed HashMap to LinkedHashMap in JsonTreeRowRecordReader (#6112)
- Added unit test to confirm preservation of field ordering
> Preserve Field Order in JsonTreeReader Map Records
> --
>
> Key: NIFI-10109
> URL: https://issues.apache.org/jira/browse/NIFI-10109
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The current implementation of the {{JsonTreeReader}} reads JSON Object nodes
> into a {{HashMap}} for representation as a {{{}MapRecord{}}}. Schemas
> generated from JSON Objects retain field order, but the internal {{Map}}
> representation does not preserve the field order due to the non-deterministic
> use of {{{}HashMap{}}}. Changing the internal representation to a
> {{LinkedHashMap}} would preserve field order for methods such as
> {{{}Record.toMap(){}}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [nifi] markap14 commented on pull request #6112: NIFI-10109 Preserve Field Order in JsonTreeReader
markap14 commented on PR #6112: URL: https://github.com/apache/nifi/pull/6112#issuecomment-1162356351 Thanks @exceptionfactory LGTM +1 merged to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 merged pull request #6112: NIFI-10109 Preserve Field Order in JsonTreeReader
markap14 merged PR #6112: URL: https://github.com/apache/nifi/pull/6112 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10146) Upgrade Apache Tika to 2.4.1
[
https://issues.apache.org/jira/browse/NIFI-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-10146:
--
Fix Version/s: 1.17.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Upgrade Apache Tika to 2.4.1
>
>
> Key: NIFI-10146
> URL: https://issues.apache.org/jira/browse/NIFI-10146
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Labels: dependency-upgrade
> Fix For: 1.17.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Apache Tika
> [2.4.1|https://dist.apache.org/repos/dist/release/tika/2.4.1/CHANGES-2.4.1.txt]
> includes several improvements and also includes a newer version of
> {{metadata-extractor}}, which resolves CVE-2022-24613 and CVE-2022-24614
> related to processing crafted JPEG files.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (NIFI-10146) Upgrade Apache Tika to 2.4.1
[
https://issues.apache.org/jira/browse/NIFI-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557122#comment-17557122
]
ASF subversion and git services commented on NIFI-10146:
Commit 47121614a205d99f53a619a656456757056fb595 in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=47121614a2 ]
NIFI-10146 Upgraded Apache Tika from 2.4.0 to 2.4.1
- Removed direct dependency on metadata-extractor to inherit newer transitive
version
Signed-off-by: Pierre Villard
This closes #6140.
> Upgrade Apache Tika to 2.4.1
>
>
> Key: NIFI-10146
> URL: https://issues.apache.org/jira/browse/NIFI-10146
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Labels: dependency-upgrade
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Apache Tika
> [2.4.1|https://dist.apache.org/repos/dist/release/tika/2.4.1/CHANGES-2.4.1.txt]
> includes several improvements and also includes a newer version of
> {{metadata-extractor}}, which resolves CVE-2022-24613 and CVE-2022-24614
> related to processing crafted JPEG files.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [nifi] asfgit closed pull request #6140: NIFI-10146 Upgrade Apache Tika from 2.4.0 to 2.4.1
asfgit closed pull request #6140: NIFI-10146 Upgrade Apache Tika from 2.4.0 to 2.4.1 URL: https://github.com/apache/nifi/pull/6140 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10148) When Process Group has FlowFile Concurrency of "Single FlowFile Per Node", can occasionally let two FlowFiles in at the same time.
[
https://issues.apache.org/jira/browse/NIFI-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557120#comment-17557120
]
ASF subversion and git services commented on NIFI-10148:
Commit ac8c1b0326fc83b3e574bf9e7eb8ad1a26f7c918 in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ac8c1b0326 ]
NIFI-10148: Fixed bug in SwappablePriorityQueue, in which the Active Queue Size
could be decremented before the Unacknowledged Queue Size is incremented when
FlowFiles are polled. This can result in the SwappablePrioriotyQueue
incorrectly returning a value of true from the isEmpty() method. Additionally,
updated the NiFiSystemIT so that if waiting for a particular queue size we
periodically log size of all queues, which aids in debugging, and added
necessary methods to FlowClient to make that happen.
This closes #6143
Signed-off-by: David Handermann
> When Process Group has FlowFile Concurrency of "Single FlowFile Per Node",
> can occasionally let two FlowFiles in at the same time.
> --
>
> Key: NIFI-10148
> URL: https://issues.apache.org/jira/browse/NIFI-10148
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Labels: SingleFlowFileConcurrency, batch, concurrency,
> system-tests
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When a Process Group is configured with a FlowFile Concurrency of "Single
> FlowFile Per Node", only a single FlowFile should enter the Process Group at
> a time. No other FlowFile should enter on that node until the PG is empty.
> However, occasionally, a second FlowFile appears to be allowed through. This
> is evidenced by the intermittent failure of the {{BatchFlowBetweenGroupsIT}}
> system test. Recently, the system tests failed with the following stack trace:
> {code:java}
> java.util.concurrent.TimeoutException:
> testSingleConcurrencyAndBatchOutputToBatchInputOutput() timed out after 5
> minutes
> at
> org.junit.jupiter.engine.extension.TimeoutInvocation.createTimeoutException(TimeoutInvocation.java:70)
> at
> org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:59)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
> at
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
> at
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
> at
> org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
> at
>
[jira] [Updated] (NIFI-10148) When Process Group has FlowFile Concurrency of "Single FlowFile Per Node", can occasionally let two FlowFiles in at the same time.
[
https://issues.apache.org/jira/browse/NIFI-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-10148:
Fix Version/s: 1.17.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> When Process Group has FlowFile Concurrency of "Single FlowFile Per Node",
> can occasionally let two FlowFiles in at the same time.
> --
>
> Key: NIFI-10148
> URL: https://issues.apache.org/jira/browse/NIFI-10148
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Labels: SingleFlowFileConcurrency, batch, concurrency,
> system-tests
> Fix For: 1.17.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When a Process Group is configured with a FlowFile Concurrency of "Single
> FlowFile Per Node", only a single FlowFile should enter the Process Group at
> a time. No other FlowFile should enter on that node until the PG is empty.
> However, occasionally, a second FlowFile appears to be allowed through. This
> is evidenced by the intermittent failure of the {{BatchFlowBetweenGroupsIT}}
> system test. Recently, the system tests failed with the following stack trace:
> {code:java}
> java.util.concurrent.TimeoutException:
> testSingleConcurrencyAndBatchOutputToBatchInputOutput() timed out after 5
> minutes
> at
> org.junit.jupiter.engine.extension.TimeoutInvocation.createTimeoutException(TimeoutInvocation.java:70)
> at
> org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:59)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
> at
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
> at
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
> at
> org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
> at
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
> at
> org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
> at
>
[GitHub] [nifi] exceptionfactory closed pull request #6143: NIFI-10148: Fixed bug in SwappablePriorityQueue, in which the Active …
exceptionfactory closed pull request #6143: NIFI-10148: Fixed bug in SwappablePriorityQueue, in which the Active … URL: https://github.com/apache/nifi/pull/6143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10141) JSONRecordSetWriter when unable to write record should convey its schema in log message
[
https://issues.apache.org/jira/browse/NIFI-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-10141:
--
Fix Version/s: 1.17.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> JSONRecordSetWriter when unable to write record should convey its schema in
> log message
> ---
>
> Key: NIFI-10141
> URL: https://issues.apache.org/jira/browse/NIFI-10141
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Matt Burgess
>Priority: Major
> Fix For: 1.17.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> If the JSON Record Set Writer fails to write out a Record, we get a log
> message generated by the following line:
> {code:java}
> logger.error("Failed to write {} with schema {} as a JSON Object due to {}",
> new Object[] {record, record.getSchema(), e.toString(), e}); {code}
> This indicates the Record and its schema, but it can lead to confusion if the
> Record Writer was created with a different schema. We should also include in
> this log message the schema that the Record Writer itself is using.
> Otherwise, we end up with output that looks like this:
> {code:java}
> Failed to write MapRecord[{VER=1, UDT=1654949958385,
> IID=Yzk0ZjUwNDczYjkyM2Q0YjoxNjU0OTQ5OTU4Mzg1,
> USI=00016549499409380af094b69c808340, UID=9807013334,
> DAT=JGABPMJAY\u00091551868547373\u00091, ROT=0, CAT=sms, DVM=Redmi:Redmi Note
> 8 Pro, AID=c94f59473b923d4b, LDT=1654949958385}]
> with schema [\"AID\" : \"STRING\", \"CAT\" : \"STRING\", \"DAT\" :
> \"STRING\", \"DVM\" : \"STRING\", \"IID\" : \"STRING\", \"LDT\" : \"LONG\",
> \"ROT\" : \"INT\", \"UDT\" : \"LONG\", \"UID\" : \"STRING\", \"USI\" :
> \"STRING\", \"VER\" : \"INT\"]
> as a JSON Object due to
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [1654949958385] of type class java.lang.Long to Integer
> for field LDT as it causes an arithmetic overflow (the value is too large,
> e.g.):
> {code}
> Here, it's attempting to convert a LONG object to an integer and failing.
> However, it's very unclear from this log message why it would attempt to
> convert the value {{1654949958385}} to an INT because the schema says it's a
> LONG. Eventually, we discovered that the cause is that the Record Writer was
> created with a schema indicating the field is an INT, not a LONG.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (NIFI-10141) JSONRecordSetWriter when unable to write record should convey its schema in log message
[
https://issues.apache.org/jira/browse/NIFI-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557118#comment-17557118
]
ASF subversion and git services commented on NIFI-10141:
Commit b60b9b8e29be4839b93c4baeacdd63802a693c29 in nifi's branch
refs/heads/main from Matt Burgess
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b60b9b8e29 ]
NIFI-10141: Fix WriteJsonResult logging to show writer's schema (#6142)
* NIFI-10141: Fix WriteJsonResult logging to show writer's schema
> JSONRecordSetWriter when unable to write record should convey its schema in
> log message
> ---
>
> Key: NIFI-10141
> URL: https://issues.apache.org/jira/browse/NIFI-10141
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Matt Burgess
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> If the JSON Record Set Writer fails to write out a Record, we get a log
> message generated by the following line:
> {code:java}
> logger.error("Failed to write {} with schema {} as a JSON Object due to {}",
> new Object[] {record, record.getSchema(), e.toString(), e}); {code}
> This indicates the Record and its schema, but it can lead to confusion if the
> Record Writer was created with a different schema. We should also include in
> this log message the schema that the Record Writer itself is using.
> Otherwise, we end up with output that looks like this:
> {code:java}
> Failed to write MapRecord[{VER=1, UDT=1654949958385,
> IID=Yzk0ZjUwNDczYjkyM2Q0YjoxNjU0OTQ5OTU4Mzg1,
> USI=00016549499409380af094b69c808340, UID=9807013334,
> DAT=JGABPMJAY\u00091551868547373\u00091, ROT=0, CAT=sms, DVM=Redmi:Redmi Note
> 8 Pro, AID=c94f59473b923d4b, LDT=1654949958385}]
> with schema [\"AID\" : \"STRING\", \"CAT\" : \"STRING\", \"DAT\" :
> \"STRING\", \"DVM\" : \"STRING\", \"IID\" : \"STRING\", \"LDT\" : \"LONG\",
> \"ROT\" : \"INT\", \"UDT\" : \"LONG\", \"UID\" : \"STRING\", \"USI\" :
> \"STRING\", \"VER\" : \"INT\"]
> as a JSON Object due to
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [1654949958385] of type class java.lang.Long to Integer
> for field LDT as it causes an arithmetic overflow (the value is too large,
> e.g.):
> {code}
> Here, it's attempting to convert a LONG object to an integer and failing.
> However, it's very unclear from this log message why it would attempt to
> convert the value {{1654949958385}} to an INT because the schema says it's a
> LONG. Eventually, we discovered that the cause is that the Record Writer was
> created with a schema indicating the field is an INT, not a LONG.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [nifi] markap14 commented on pull request #6142: NIFI-10141: Fix WriteJsonResult logging to show writer's schema
markap14 commented on PR #6142: URL: https://github.com/apache/nifi/pull/6142#issuecomment-1162344541 Thanks @mattyb149 +1 merged to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 merged pull request #6142: NIFI-10141: Fix WriteJsonResult logging to show writer's schema
markap14 merged PR #6142: URL: https://github.com/apache/nifi/pull/6142 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a diff in pull request #6131: NIFI-9822 - ConsumeKafkaRecord allows writing out Kafka record key
markap14 commented on code in PR #6131:
URL: https://github.com/apache/nifi/pull/6131#discussion_r902981711
##
nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-6-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumeKafkaRecord_2_6.java:
##
@@ -212,6 +217,31 @@ public class ConsumeKafkaRecord_2_6 extends
AbstractProcessor implements Verifia
.defaultValue("UTF-8")
.required(false)
.build();
+static final PropertyDescriptor CONSUME_STRATEGY = new
PropertyDescriptor.Builder()
+.name("consume-strategy")
+.displayName("Consume Strategy")
Review Comment:
This doesn't really control how we consume data from Kafka. Rather, it
controls how we write out the data. So perhaps "Output Strategy" is more
suitable?
##
nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-6-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/PublishKafkaRecord_2_6.java:
##
@@ -271,7 +273,21 @@ public class PublishKafkaRecord_2_6 extends
AbstractProcessor implements Verifia
.defaultValue("UTF-8")
.required(false)
.build();
-
+static final PropertyDescriptor PUBLISH_STRATEGY = new
PropertyDescriptor.Builder()
Review Comment:
With the addition of this property, I think some of the existing properties
can also be hidden if using the wrapper. Specifically, just like in the
consumer, we should make MESSAGE_KEY_FIELD and ATTRIBUTE_NAME_REGEX dependent
properties because it doesn't make sense to specify those if using the Wrapper.
##
nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-6-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/KafkaProcessorUtils.java:
##
@@ -110,6 +110,12 @@ public final class KafkaProcessorUtils {
"When unable to publish a FlowFile to Kafka, the FlowFile will be
placed back on the top of its queue so that it will be the next FlowFile tried
again. " +
"For dataflows where ordering of FlowFiles is important, this
strategy can be used along with ensuring that the each processor in the
dataflow uses only a single Concurrent Task.");
+static final AllowableValue WRITE_VALUE_ONLY = new
AllowableValue("write-value-only", "Write Value Only", "Write only the Kafka
Record value.");
+static final AllowableValue USE_WRAPPER = new
AllowableValue("use-wrapper", "Use Wrapper", "Write the Kafka Record key,
value, headers, and metadata.");
Review Comment:
Perhaps we should mention in the description explicitly that these fields
will be wrapped in a wrapper element, and to see Processor Usage for more
information.
##
nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-6-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.ConsumeKafkaRecord_2_6/additionalDetails.html:
##
@@ -256,5 +256,69 @@ SASL_SSL
ssl.client.auth property.
+Output Modes
+
+This processor (NiFi 1.17+) offers multiple output strategies
(configured via processor property 'Consume
+Strategy') for converting Kafka records into FlowFiles.
+
+Consume Strategy 'Write Value Only' (the default) emits
flowfile records containing only the Kafka
+record value.
+
+Consume Strategy 'Use Wrapper' (new) emits flowfile
records containing the Kafka record key, value,
+and headers, as well as additional metadata from the Kafka
record.
+
+
+
+
+The record schema that is used when 'Use Wrapper' is active is
as follows:
+
+key: one of RECORD, STRING, or BYTE_ARRAY (based on
processor configuration property 'Key Format')
+
+value: RECORD
+headers: MAP (STRING, STRING)
+metadata: RECORD
+
+topic: STRING
+partition: INTEGER
+offset: LONG
+timestamp: TIMESTAMP
+
+
+
+
+If the Consume Strategy property 'Use Wrapper' is active, an
additional processor configuration property
+('Key Format') is activated. This property is used to
fine-tune the transformation of the incoming Kafka
+record. The possible values for 'Key Format' are 'Byte Array',
'String', or 'Record'.
+
+'Byte Array' supplies the Kafka Record Key bytes unchanged
from the incoming Kafka record.
+'String' converts the Kafka Record Key bytes into a string
using the UTF-8 character encoding.
+(Failure to parse the key bytes as UTF-8 will result in
the record being routed to the
+'parse.failure' relationship.)
+
+'Record' converts the Kafka
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1356: MINIFICPP-1867 added tzdata to docker-minimal
szaszm commented on code in PR #1356:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1356#discussion_r902947218
##
docker/Dockerfile:
##
@@ -175,6 +175,7 @@ RUN addgroup -g ${GID} ${USER} && adduser -u ${UID} -D -G
${USER} -g "" ${USER}
# Copy built minifi distribution from builder
COPY --from=build --chown=${USER}:${USER} ${MINIFI_VERSIONED_HOME}
${MINIFI_HOME}
COPY --from=build --chown=${USER}:${USER}
${MINIFI_BASE_DIR}/docker/conf/minifi-log.properties
${MINIFI_HOME}/conf/minifi-log.properties
+RUN echo "UTC" > /etc/timezone
Review Comment:
fair, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1356: MINIFICPP-1867 added tzdata to docker-minimal
martinzink commented on code in PR #1356:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1356#discussion_r902942020
##
docker/Dockerfile:
##
@@ -175,6 +175,7 @@ RUN addgroup -g ${GID} ${USER} && adduser -u ${UID} -D -G
${USER} -g "" ${USER}
# Copy built minifi distribution from builder
COPY --from=build --chown=${USER}:${USER} ${MINIFI_VERSIONED_HOME}
${MINIFI_HOME}
COPY --from=build --chown=${USER}:${USER}
${MINIFI_BASE_DIR}/docker/conf/minifi-log.properties
${MINIFI_HOME}/conf/minifi-log.properties
+RUN echo "UTC" > /etc/timezone
Review Comment:
good catch, I've changed it to use setup-timezone instead in
https://github.com/apache/nifi-minifi-cpp/pull/1356/commits/85d7603993968ad0e6351edeee063073b840ce55
From what I've gathered its a common problem, and its present on all
containers.
e.g.
```
$ date
Tue 21 Jun 20:28:05 CEST 2022
$ docker run -it ubuntu:22.04 date
Tue Jun 21 18:28:07 UTC 2022
$ docker run -it centos:7 date
Tue Jun 21 18:28:14 UTC 2022
```
If someone wants to change this they can easily build an image from ours.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a diff in pull request #6115: NIFI-10108 Processor scheduling via parameter
markap14 commented on code in PR #6115:
URL: https://github.com/apache/nifi/pull/6115#discussion_r902915510
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java:
##
@@ -1209,33 +1194,112 @@ protected Collection
computeValidationErrors(final ValidationC
}
@Override
-protected List validateConfig() {
+public List validateConfig() {
final List results = new ArrayList<>();
final ParameterContext parameterContext = getParameterContext();
-if (parameterContext == null && !this.parameterReferences.isEmpty()) {
+if (parameterContext == null &&
!this.configurationParameterReferences.isEmpty()) {
results.add(new ValidationResult.Builder()
-.subject(getName())
+.subject(RUN_SCHEDULE)
+.input("Parameter Context")
.valid(false)
.explanation("Processor configuration references one or
more Parameters but no Parameter Context is currently set on the Process
Group.")
.build());
} else {
-for (final ParameterReference paramRef : parameterReferences) {
-if
(!parameterContext.getParameter(paramRef.getParameterName()).isPresent() ) {
+for (final ParameterReference paramRef :
configurationParameterReferences) {
+final Optional parameterRef =
parameterContext.getParameter(paramRef.getParameterName());
+if (!parameterRef.isPresent() ) {
results.add(new ValidationResult.Builder()
-.subject(getName())
+.subject(RUN_SCHEDULE)
+.input(paramRef.getParameterName())
.valid(false)
.explanation("Processor configuration references
Parameter '" + paramRef.getParameterName() +
"' but the currently selected Parameter
Context does not have a Parameter with that name")
.build());
+} else {
+final ParameterDescriptor parameterDescriptor =
parameterRef.get().getDescriptor();
+if (parameterDescriptor.isSensitive()) {
+results.add(new ValidationResult.Builder()
+.subject(RUN_SCHEDULE)
+.input(parameterDescriptor.getName())
+.valid(false)
+.explanation("Processor configuration cannot
reference sensitive parameters")
+.build());
+}
}
}
-}
+final String schedulingPeriod = getSchedulingPeriod();
+final String evaluatedSchedulingPeriod =
evaluateParameter(schedulingPeriod);
+
+if (evaluatedSchedulingPeriod != null) {
Review Comment:
This should never be `null`, should it?
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java:
##
@@ -1969,17 +2033,12 @@ public void setMaxBackoffPeriod(String
maxBackoffPeriod) {
}
@Override
-public String evaluateSchedulingPeriodFromParameter(final String
schedulingPeriod) {
+public String evaluateParameter(final String parameter) {
Review Comment:
I don't think the argument here is really a parameter is it? It's some value
that may or may not contain parameters. Perhaps should call it
`evaluateParameters` and call the argument `value`?
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/controller/AbstractComponentNode.java:
##
@@ -1084,12 +1092,29 @@ public void onParametersModified(final Map updatedParam
}
// If this component is affected by the Parameter change, we need to
re-validate
+componentAffected |=
isConfigurationParameterModified(updatedParameters);
if (componentAffected) {
logger.debug("Configuration of {} changed due to an update to
Parameter Context. Resetting validation state", this);
resetValidationState();
}
}
+protected void increaseReferenceCounts(final String parameterName) {
+parameterReferenceCounts.merge(parameterName, 1, (a, b) -> a == -1 ?
null : a + b);
+}
+
+protected void decreaseReferenceCounts(final String parameterName) {
+parameterReferenceCounts.merge(parameterName, -1, (a, b) -> a == 1 ?
null : a + b);
+}
+
+/**
+ * Verifies whether any referenced configuration parameter has been
updated and its effective value has changed.
+ * If yes, the component needs to be re-validated.
+ *
[GitHub] [nifi] mr1716 commented on pull request #6144: NIFI-10149 Update Apache-JSP To 11.0.9
mr1716 commented on PR #6144: URL: https://github.com/apache/nifi/pull/6144#issuecomment-1162157875 @exceptionfactory Thanks. Much appreciated. That helps. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-10149) Update Apache JSP To 11.0.9
[ https://issues.apache.org/jira/browse/NIFI-10149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R resolved NIFI-10149. --- Resolution: Invalid > Update Apache JSP To 11.0.9 > --- > > Key: NIFI-10149 > URL: https://issues.apache.org/jira/browse/NIFI-10149 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.16.1, 1.16.2, 1.16.3 >Reporter: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Looking at the Maven dependencies, it was seen that Apache JSP 9.4 uses a > vulnerable version of > [jetty-schemas|https://mvnrepository.com/artifact/org.eclipse.jetty.toolchain/jetty-schemas], > which is 5.2. Updating this specific version to 11.0.9 of Apache JSP, will > remove this dependency -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1356: MINIFICPP-1867 added tzdata to docker-minimal
szaszm commented on code in PR #1356:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1356#discussion_r902925494
##
docker/Dockerfile:
##
@@ -175,6 +175,7 @@ RUN addgroup -g ${GID} ${USER} && adduser -u ${UID} -D -G
${USER} -g "" ${USER}
# Copy built minifi distribution from builder
COPY --from=build --chown=${USER}:${USER} ${MINIFI_VERSIONED_HOME}
${MINIFI_HOME}
COPY --from=build --chown=${USER}:${USER}
${MINIFI_BASE_DIR}/docker/conf/minifi-log.properties
${MINIFI_HOME}/conf/minifi-log.properties
+RUN echo "UTC" > /etc/timezone
Review Comment:
I'm also thinking whether we should somehow use the host system's timezone
instead of hardcoding UTC in the docker image, but I don't know how to
accomplish that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-10147) Remove Support for Java 8
[ https://issues.apache.org/jira/browse/NIFI-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557042#comment-17557042 ] David Handermann commented on NIFI-10147: - That's a good question [~kdoran]. Too many changes could become difficult to rebase, however, the initial scope of changes should be minimal. Given that automated builds already include Java 11 and Java 17, initial removal of Java 8 should require minimal changes. Removing optional Maven build profiles could be a bit more involved, and there may be some incremental work that could be accomplished on the current main branch prior to removing Java 8 support. For now, it seems better to wait on creating a separate branch, but open to additional reasons why it might be helpful sooner rather than later. > Remove Support for Java 8 > - > > Key: NIFI-10147 > URL: https://issues.apache.org/jira/browse/NIFI-10147 > Project: Apache NiFi > Issue Type: Epic > Components: Core Framework, MiNiFi, NiFi Registry, NiFi Stateless >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Although multiple vendors have planned extended support for Java 8, a number > of projects are in the process of deprecating and removing support for Java > 8. These projects include the following: > - [Jetty 9.4|https://github.com/eclipse/jetty.project/issues/7958] > - [Spring > 5|https://spring.io/blog/2021/09/02/a-java-17-and-jakarta-ee-9-baseline-for-spring-framework-6] > - [OpenSAML 3|https://shibboleth.atlassian.net/wiki/spaces/OSAML/overview] > Apache NiFi should also remove support for Java 8 in order to maintain active > support for various components. > Removing support for Java 8 requires a new major version release and will > also require several build configuration changes including the following: > - Updating Maven source and compiler settings > - Updating minimum Java build version > - Removing optional Maven build profiles > - Updating GitHub workflows > Initial discussion on a migration strategy included a tentative target of the > end of calendar year 2022. > Implementation should not proceed until after a minor release of Apache NiFi > 1.17.0. > Mailing list discussion: > [https://lists.apache.org/thread/mm1xf3b9nvrcgytb92oy3swvvc45fl34] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1356: MINIFICPP-1867 added tzdata to docker-minimal
szaszm commented on code in PR #1356:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1356#discussion_r902921623
##
docker/Dockerfile:
##
@@ -175,6 +175,7 @@ RUN addgroup -g ${GID} ${USER} && adduser -u ${UID} -D -G
${USER} -g "" ${USER}
# Copy built minifi distribution from builder
COPY --from=build --chown=${USER}:${USER} ${MINIFI_VERSIONED_HOME}
${MINIFI_HOME}
COPY --from=build --chown=${USER}:${USER}
${MINIFI_BASE_DIR}/docker/conf/minifi-log.properties
${MINIFI_HOME}/conf/minifi-log.properties
+RUN echo "UTC" > /etc/timezone
Review Comment:
According to [this](https://wiki.alpinelinux.org/wiki/Setting_the_timezone),
the manual method of setting the timezone is considered obsolete, and they
recommend using
[`setup-timezone`](https://wiki.alpinelinux.org/wiki/Alpine_setup_scripts#setup-timezone)
instead.
I'm assuming writing to `/etc/timezone` only sets the timezone for musl
libc, and linking/copying a compiled timezone info file from
/usr/share/zoneinfo to /etc/localtime only sets the timezone of glibc. Please
use the `setup-timezone` utility unless you think I missed something.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1356: MINIFICPP-1867 added tzdata to docker-minimal
szaszm commented on code in PR #1356:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1356#discussion_r902921623
##
docker/Dockerfile:
##
@@ -175,6 +175,7 @@ RUN addgroup -g ${GID} ${USER} && adduser -u ${UID} -D -G
${USER} -g "" ${USER}
# Copy built minifi distribution from builder
COPY --from=build --chown=${USER}:${USER} ${MINIFI_VERSIONED_HOME}
${MINIFI_HOME}
COPY --from=build --chown=${USER}:${USER}
${MINIFI_BASE_DIR}/docker/conf/minifi-log.properties
${MINIFI_HOME}/conf/minifi-log.properties
+RUN echo "UTC" > /etc/timezone
Review Comment:
According to [this](https://wiki.alpinelinux.org/wiki/Setting_the_timezone),
the manual method of setting the timezone is considered obsolete, and they
recommend using
[`setup-timezone`](https://wiki.alpinelinux.org/wiki/Alpine_setup_scripts#setup-timezone)
instead.
I'm assuming this only sets the timezone for musl libc, while linking a
compiled timezone info file from /usr/share/zoneinfo to /etc/localtime only
sets the timezone of glibc. Please use the `setup-timezone` utility unless you
think I missed something.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-10147) Remove Support for Java 8
[ https://issues.apache.org/jira/browse/NIFI-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557038#comment-17557038 ] Kevin Doran commented on NIFI-10147: Do we want to make a 2.0 development branch in the central code repository? I'm thinking it would be a place that experimental work could begin, to be continually rebased on main, until we are ready to release 2.0, at which point we could push the 2.0 development branch to main. Would this be helpful to start making progress? Would rebases be too complicated to sustain this work? > Remove Support for Java 8 > - > > Key: NIFI-10147 > URL: https://issues.apache.org/jira/browse/NIFI-10147 > Project: Apache NiFi > Issue Type: Epic > Components: Core Framework, MiNiFi, NiFi Registry, NiFi Stateless >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Although multiple vendors have planned extended support for Java 8, a number > of projects are in the process of deprecating and removing support for Java > 8. These projects include the following: > - [Jetty 9.4|https://github.com/eclipse/jetty.project/issues/7958] > - [Spring > 5|https://spring.io/blog/2021/09/02/a-java-17-and-jakarta-ee-9-baseline-for-spring-framework-6] > - [OpenSAML 3|https://shibboleth.atlassian.net/wiki/spaces/OSAML/overview] > Apache NiFi should also remove support for Java 8 in order to maintain active > support for various components. > Removing support for Java 8 requires a new major version release and will > also require several build configuration changes including the following: > - Updating Maven source and compiler settings > - Updating minimum Java build version > - Removing optional Maven build profiles > - Updating GitHub workflows > Initial discussion on a migration strategy included a tentative target of the > end of calendar year 2022. > Implementation should not proceed until after a minor release of Apache NiFi > 1.17.0. > Mailing list discussion: > [https://lists.apache.org/thread/mm1xf3b9nvrcgytb92oy3swvvc45fl34] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (NIFI-10149) Update Apache JSP To 11.0.9
[ https://issues.apache.org/jira/browse/NIFI-10149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557026#comment-17557026 ] David Handermann commented on NIFI-10149: - [~msr1716], as mentioned on the associated pull request, jetty-schemas 5.2 is the latest version available, and it is not vulnerable to any issues associated with Jetty 9.4. In addition, the Apache JSP library is a repackaged library for Jetty and does not have the vulnerabilities associated with Apache Tomcat server, as described in the dependency check suppression configuration: https://github.com/apache/nifi/blob/main/nifi-dependency-check-maven/suppressions.xml#L73 > Update Apache JSP To 11.0.9 > --- > > Key: NIFI-10149 > URL: https://issues.apache.org/jira/browse/NIFI-10149 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.16.1, 1.16.2, 1.16.3 >Reporter: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Looking at the Maven dependencies, it was seen that Apache JSP 9.4 uses a > vulnerable version of > [jetty-schemas|https://mvnrepository.com/artifact/org.eclipse.jetty.toolchain/jetty-schemas], > which is 5.2. Updating this specific version to 11.0.9 of Apache JSP, will > remove this dependency -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [nifi] exceptionfactory closed pull request #6144: NIFI-10149 Update Apache-JSP To 11.0.9
exceptionfactory closed pull request #6144: NIFI-10149 Update Apache-JSP To 11.0.9 URL: https://github.com/apache/nifi/pull/6144 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6144: NIFI-10149 Update Apache-JSP To 11.0.9
exceptionfactory commented on PR #6144: URL: https://github.com/apache/nifi/pull/6144#issuecomment-1162086464 @mr1716 I'm not sure what it reporting `jetty-schemas` 5.2 as a vulnerable dependency, but it is a false positive. The [jetty-schemas 5.2](https://search.maven.org/artifact/org.eclipse.jetty.toolchain/jetty-schemas) dependency is the latest version available, it does not contain code, only XML Schema Definitions. In addition, the changes proposed will not work, because Jetty 11 has many additional changes over 9.4, and the Jetty version is managed through the `jetty.version` property. This is a case where more detailed evaluation is necessary, so closing this pull request. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mr1716 opened a new pull request, #6144: NIFI-10147 Update Apache-JSP To 11.0.9
mr1716 opened a new pull request, #6144: URL: https://github.com/apache/nifi/pull/6144 # Summary [NIFI-10147]([NIFI-10147](https://issues.apache.org/jira/browse/NIFI-10147)) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira]([NIFI-10147](https://issues.apache.org/jira/browse/NIFI-10147)) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [X] JDK 8 - [X] JDK 11 - [X] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10149) Update Apache JSP To 11.0.9
Mike R created NIFI-10149: - Summary: Update Apache JSP To 11.0.9 Key: NIFI-10149 URL: https://issues.apache.org/jira/browse/NIFI-10149 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.16.3, 1.16.2, 1.16.1 Reporter: Mike R Looking at the Maven dependencies, it was seen that Apache JSP 9.4 uses a vulnerable version of [jetty-schemas|https://mvnrepository.com/artifact/org.eclipse.jetty.toolchain/jetty-schemas], which is 5.2. Updating this specific version to 11.0.9 of Apache JSP, will remove this dependency -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [nifi] markap14 commented on a diff in pull request #6117: NIFI-10111: Added option to unpack NARs into an Uber Jar and updated …
markap14 commented on code in PR #6117:
URL: https://github.com/apache/nifi/pull/6117#discussion_r902876581
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/nifi.properties:
##
@@ -39,6 +39,7 @@ nifi.nar.library.directory=${nifi.nar.library.directory}
nifi.nar.library.autoload.directory=${nifi.nar.library.autoload.directory}
nifi.nar.working.directory=${nifi.nar.working.directory}
nifi.documentation.working.directory=${nifi.documentation.working.directory}
+nifi.nar.unpack.uber.jar=${nifi.nar.unpack.uber.jar}
Review Comment:
Not necessarily. With nifi.properties, we tend to offer a lot more
properties because it's easy to configure them and because it's a very mature
"product." With stateless, this will be important for many cases, and it's
going to be hard to expose this in a way that makes sense to most users. I
expect this property will go away from NiFi in the future and we'll always do
this, but wanted to make it configurable just in case there are any problems.
WIth Stateless we take a few more liberties than with NiFi just because it's
newer and faster evolving.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on a diff in pull request #6117: NIFI-10111: Added option to unpack NARs into an Uber Jar and updated …
mattyb149 commented on code in PR #6117:
URL: https://github.com/apache/nifi/pull/6117#discussion_r902862490
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/nifi.properties:
##
@@ -39,6 +39,7 @@ nifi.nar.library.directory=${nifi.nar.library.directory}
nifi.nar.library.autoload.directory=${nifi.nar.library.autoload.directory}
nifi.nar.working.directory=${nifi.nar.working.directory}
nifi.documentation.working.directory=${nifi.documentation.working.directory}
+nifi.nar.unpack.uber.jar=${nifi.nar.unpack.uber.jar}
Review Comment:
Shouldn't this (also) be added to stateless.properties?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (MINIFICPP-1867) tzdata is missing from docker-minimal
[ https://issues.apache.org/jira/browse/MINIFICPP-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-1867: --- Status: Patch Available (was: Open) https://github.com/apache/nifi-minifi-cpp/pull/1356 > tzdata is missing from docker-minimal > - > > Key: MINIFICPP-1867 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1867 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Fix For: 0.13.0 > > Time Spent: 10m > Remaining Estimate: 0h > > tzdata is required for timezone features (cron scheduler, > expression-language), but it is missing on docker-minimal image (based on > alpine) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (NIFI-10148) When Process Group has FlowFile Concurrency of "Single FlowFile Per Node", can occasionally let two FlowFiles in at the same time.
[
https://issues.apache.org/jira/browse/NIFI-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-10148:
--
Status: Patch Available (was: Open)
> When Process Group has FlowFile Concurrency of "Single FlowFile Per Node",
> can occasionally let two FlowFiles in at the same time.
> --
>
> Key: NIFI-10148
> URL: https://issues.apache.org/jira/browse/NIFI-10148
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Labels: SingleFlowFileConcurrency, batch, concurrency,
> system-tests
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When a Process Group is configured with a FlowFile Concurrency of "Single
> FlowFile Per Node", only a single FlowFile should enter the Process Group at
> a time. No other FlowFile should enter on that node until the PG is empty.
> However, occasionally, a second FlowFile appears to be allowed through. This
> is evidenced by the intermittent failure of the {{BatchFlowBetweenGroupsIT}}
> system test. Recently, the system tests failed with the following stack trace:
> {code:java}
> java.util.concurrent.TimeoutException:
> testSingleConcurrencyAndBatchOutputToBatchInputOutput() timed out after 5
> minutes
> at
> org.junit.jupiter.engine.extension.TimeoutInvocation.createTimeoutException(TimeoutInvocation.java:70)
> at
> org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:59)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
> at
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
> at
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
> at
> org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
> at
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
> at
> org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
> at
>
[GitHub] [nifi] markap14 opened a new pull request, #6143: NIFI-10148: Fixed bug in SwappablePriorityQueue, in which the Active …
markap14 opened a new pull request, #6143: URL: https://github.com/apache/nifi/pull/6143 …Queue Size could be decremented before the Unacknowledged Queue Size is incremented when FlowFiles are polled. This can result in the SwappablePrioriotyQueue incorrectly returning a value of true from the isEmpty() method. Additionally, updated the NiFiSystemIT so that if waiting for a particular queue size we periodically log size of all queues, which aids in debugging, and added necessary methods to FlowClient to make that happen. # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10148) When Process Group has FlowFile Concurrency of "Single FlowFile Per Node", can occasionally let two FlowFiles in at the same time.
[
https://issues.apache.org/jira/browse/NIFI-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-10148:
--
Labels: SingleFlowFileConcurrency batch concurrency system-tests (was: )
> When Process Group has FlowFile Concurrency of "Single FlowFile Per Node",
> can occasionally let two FlowFiles in at the same time.
> --
>
> Key: NIFI-10148
> URL: https://issues.apache.org/jira/browse/NIFI-10148
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Labels: SingleFlowFileConcurrency, batch, concurrency,
> system-tests
>
> When a Process Group is configured with a FlowFile Concurrency of "Single
> FlowFile Per Node", only a single FlowFile should enter the Process Group at
> a time. No other FlowFile should enter on that node until the PG is empty.
> However, occasionally, a second FlowFile appears to be allowed through. This
> is evidenced by the intermittent failure of the {{BatchFlowBetweenGroupsIT}}
> system test. Recently, the system tests failed with the following stack trace:
> {code:java}
> java.util.concurrent.TimeoutException:
> testSingleConcurrencyAndBatchOutputToBatchInputOutput() timed out after 5
> minutes
> at
> org.junit.jupiter.engine.extension.TimeoutInvocation.createTimeoutException(TimeoutInvocation.java:70)
> at
> org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:59)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
> at
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
> at
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
> at
> org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
> at
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
> at
> org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
> at
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
> at
>
[jira] [Created] (NIFI-10148) When Process Group has FlowFile Concurrency of "Single FlowFile Per Node", can occasionally let two FlowFiles in at the same time.
Mark Payne created NIFI-10148:
-
Summary: When Process Group has FlowFile Concurrency of "Single
FlowFile Per Node", can occasionally let two FlowFiles in at the same time.
Key: NIFI-10148
URL: https://issues.apache.org/jira/browse/NIFI-10148
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
When a Process Group is configured with a FlowFile Concurrency of "Single
FlowFile Per Node", only a single FlowFile should enter the Process Group at a
time. No other FlowFile should enter on that node until the PG is empty.
However, occasionally, a second FlowFile appears to be allowed through. This is
evidenced by the intermittent failure of the {{BatchFlowBetweenGroupsIT}}
system test. Recently, the system tests failed with the following stack trace:
{code:java}
java.util.concurrent.TimeoutException:
testSingleConcurrencyAndBatchOutputToBatchInputOutput() timed out after 5
minutes
at
org.junit.jupiter.engine.extension.TimeoutInvocation.createTimeoutException(TimeoutInvocation.java:70)
at
org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:59)
at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at
org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at
org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at
org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at
org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at
org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at
[jira] [Resolved] (NIFI-8751) Process Group concurrency configuration does not update when deploying to another environment
[ https://issues.apache.org/jira/browse/NIFI-8751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFI-8751. -- Resolution: Information Provided This appears to be due to running too old a version of NiFi Registry. Closing as Information Provided. If issues occur, we can re-open. > Process Group concurrency configuration does not update when deploying to > another environment > - > > Key: NIFI-8751 > URL: https://issues.apache.org/jira/browse/NIFI-8751 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.12.1 >Reporter: naveen kumar saharan >Priority: Major > > We have PG, where we have used the Process Group FlowFile Concurrency as > single flow file per node, the same e are not able to see when checked into > registry and pulling the PG in another environment. > the setting gets reset to unbounded. > > PG A checked into registry > PG B is inside PG A. > > PG B has flowfile concurrency configuration setting as Single Flowfile per > node. > > Now, when pulling from registry PG B in new environment reset the PG flowfile > configuration > > > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [nifi] markap14 commented on pull request #6142: NIFI-10141: Fix WriteJsonResult logging to show writer's schema
markap14 commented on PR #6142: URL: https://github.com/apache/nifi/pull/6142#issuecomment-1161985810 @mattyb149 I think it's important to include both the record's schema and the writer's schema. This way, if there's a mismatch it becomes clear -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink opened a new pull request, #1356: MINIFICPP-1867 added tzdata to docker-minimal
martinzink opened a new pull request, #1356: URL: https://github.com/apache/nifi-minifi-cpp/pull/1356 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10141) JSONRecordSetWriter when unable to write record should convey its schema in log message
[
https://issues.apache.org/jira/browse/NIFI-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-10141:
Status: Patch Available (was: In Progress)
> JSONRecordSetWriter when unable to write record should convey its schema in
> log message
> ---
>
> Key: NIFI-10141
> URL: https://issues.apache.org/jira/browse/NIFI-10141
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Matt Burgess
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> If the JSON Record Set Writer fails to write out a Record, we get a log
> message generated by the following line:
> {code:java}
> logger.error("Failed to write {} with schema {} as a JSON Object due to {}",
> new Object[] {record, record.getSchema(), e.toString(), e}); {code}
> This indicates the Record and its schema, but it can lead to confusion if the
> Record Writer was created with a different schema. We should also include in
> this log message the schema that the Record Writer itself is using.
> Otherwise, we end up with output that looks like this:
> {code:java}
> Failed to write MapRecord[{VER=1, UDT=1654949958385,
> IID=Yzk0ZjUwNDczYjkyM2Q0YjoxNjU0OTQ5OTU4Mzg1,
> USI=00016549499409380af094b69c808340, UID=9807013334,
> DAT=JGABPMJAY\u00091551868547373\u00091, ROT=0, CAT=sms, DVM=Redmi:Redmi Note
> 8 Pro, AID=c94f59473b923d4b, LDT=1654949958385}]
> with schema [\"AID\" : \"STRING\", \"CAT\" : \"STRING\", \"DAT\" :
> \"STRING\", \"DVM\" : \"STRING\", \"IID\" : \"STRING\", \"LDT\" : \"LONG\",
> \"ROT\" : \"INT\", \"UDT\" : \"LONG\", \"UID\" : \"STRING\", \"USI\" :
> \"STRING\", \"VER\" : \"INT\"]
> as a JSON Object due to
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [1654949958385] of type class java.lang.Long to Integer
> for field LDT as it causes an arithmetic overflow (the value is too large,
> e.g.):
> {code}
> Here, it's attempting to convert a LONG object to an integer and failing.
> However, it's very unclear from this log message why it would attempt to
> convert the value {{1654949958385}} to an INT because the schema says it's a
> LONG. Eventually, we discovered that the cause is that the Record Writer was
> created with a schema indicating the field is an INT, not a LONG.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [nifi] mattyb149 opened a new pull request, #6142: NIFI-10141: Fix WriteJsonResult logging to show writer's schema
mattyb149 opened a new pull request, #6142: URL: https://github.com/apache/nifi/pull/6142 # Summary [NIFI-10141](https://issues.apache.org/jira/browse/NIFI-10141) The current error logging when a field cannot be written out uses the record's (i.e the record reader's) schema, which is not an accurate depiction of why the field cannot be written. Instead the record writer's schema should be used in the logging statement to show the schema used to write the record. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [x] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (MINIFICPP-1867) tzdata is missing from docker-minimal
Martin Zink created MINIFICPP-1867: -- Summary: tzdata is missing from docker-minimal Key: MINIFICPP-1867 URL: https://issues.apache.org/jira/browse/MINIFICPP-1867 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Martin Zink Assignee: Martin Zink Fix For: 0.13.0 tzdata is required for timezone features (cron scheduler, expression-language), but it is missing on docker-minimal image (based on alpine) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (NIFI-10141) JSONRecordSetWriter when unable to write record should convey its schema in log message
[
https://issues.apache.org/jira/browse/NIFI-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess reassigned NIFI-10141:
---
Assignee: Matt Burgess
> JSONRecordSetWriter when unable to write record should convey its schema in
> log message
> ---
>
> Key: NIFI-10141
> URL: https://issues.apache.org/jira/browse/NIFI-10141
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Matt Burgess
>Priority: Major
>
> If the JSON Record Set Writer fails to write out a Record, we get a log
> message generated by the following line:
> {code:java}
> logger.error("Failed to write {} with schema {} as a JSON Object due to {}",
> new Object[] {record, record.getSchema(), e.toString(), e}); {code}
> This indicates the Record and its schema, but it can lead to confusion if the
> Record Writer was created with a different schema. We should also include in
> this log message the schema that the Record Writer itself is using.
> Otherwise, we end up with output that looks like this:
> {code:java}
> Failed to write MapRecord[{VER=1, UDT=1654949958385,
> IID=Yzk0ZjUwNDczYjkyM2Q0YjoxNjU0OTQ5OTU4Mzg1,
> USI=00016549499409380af094b69c808340, UID=9807013334,
> DAT=JGABPMJAY\u00091551868547373\u00091, ROT=0, CAT=sms, DVM=Redmi:Redmi Note
> 8 Pro, AID=c94f59473b923d4b, LDT=1654949958385}]
> with schema [\"AID\" : \"STRING\", \"CAT\" : \"STRING\", \"DAT\" :
> \"STRING\", \"DVM\" : \"STRING\", \"IID\" : \"STRING\", \"LDT\" : \"LONG\",
> \"ROT\" : \"INT\", \"UDT\" : \"LONG\", \"UID\" : \"STRING\", \"USI\" :
> \"STRING\", \"VER\" : \"INT\"]
> as a JSON Object due to
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [1654949958385] of type class java.lang.Long to Integer
> for field LDT as it causes an arithmetic overflow (the value is too large,
> e.g.):
> {code}
> Here, it's attempting to convert a LONG object to an integer and failing.
> However, it's very unclear from this log message why it would attempt to
> convert the value {{1654949958385}} to an INT because the schema says it's a
> LONG. Eventually, we discovered that the cause is that the Record Writer was
> created with a schema indicating the field is an INT, not a LONG.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (NIFI-5005) MergeRecord processor ignoring writer schema for JsonRecordSetWriter output
[ https://issues.apache.org/jira/browse/NIFI-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-5005: --- Affects Version/s: (was: 1.5.0) Status: Patch Available (was: In Progress) > MergeRecord processor ignoring writer schema for JsonRecordSetWriter output > --- > > Key: NIFI-5005 > URL: https://issues.apache.org/jira/browse/NIFI-5005 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Nick Pettyjohn >Assignee: Matt Burgess >Priority: Major > Attachments: MergeContent_JsonOutput.xml, test.csv, test_schema.avsc > > Time Spent: 10m > Remaining Estimate: 0h > > Issue noticed when using a MergeRecord processor with a Record Reader of > CSVReader and a Record Writer of JsonRecordSetWriter. > The CSVReader is configured with a Schema Access Strategy of "Use String > Fields From Header". The JsonRecordSetWriter is given an Avro schema in the > Schema Text property that contains a mix of string, double, and long value > types. > Running sample csv data through the MergeRecord processor produces JSON in > which all values are quoted, despite the Avro schema specifying otherwise. > However, when using the ConvertRecord processor with the same Reader/Writer > config, the output JSON records use the typing given in the avro schema, > keeping long and float values unquoted. > The attached template, with corresponding avro schema and sample input csv > file will demonstrate the issue. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [nifi] mattyb149 opened a new pull request, #6141: NIFI-5005: Fix MergeRecord to honor writer's schema
mattyb149 opened a new pull request, #6141: URL: https://github.com/apache/nifi/pull/6141 # Summary [NIFI-5005](https://issues.apache.org/jira/browse/NIFI-5005) Changes MergeRecord to use the writer's schema instead of the reader's schema in order to determine output fields, names, data types, etc. The writerFactory can still make use of the reader's schema if need be, but whatever the writerFactory produces for a schema is what is used to write out the records. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x] Build completed using `mvn clean install -P contrib-check` - [x] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10140) Versioned component synchronizer not handling deleted parameters
[ https://issues.apache.org/jira/browse/NIFI-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Gresock updated NIFI-10140: --- Fix Version/s: 1.17.0 > Versioned component synchronizer not handling deleted parameters > > > Key: NIFI-10140 > URL: https://issues.apache.org/jira/browse/NIFI-10140 > Project: Apache NiFi > Issue Type: Bug >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Major > Fix For: 1.17.0 > > Time Spent: 40m > Remaining Estimate: 0h > > In the StandardVersionedComponentSynchronizer, there needs to be handling for > deleted parameters similar to what is in VersionedFlowSynchronizer where it > produces a null value for any parameter names that are in the existing > context, but not in the submitted parameters. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (NIFI-10140) Versioned component synchronizer not handling deleted parameters
[ https://issues.apache.org/jira/browse/NIFI-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Gresock updated NIFI-10140: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Versioned component synchronizer not handling deleted parameters > > > Key: NIFI-10140 > URL: https://issues.apache.org/jira/browse/NIFI-10140 > Project: Apache NiFi > Issue Type: Bug >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > In the StandardVersionedComponentSynchronizer, there needs to be handling for > deleted parameters similar to what is in VersionedFlowSynchronizer where it > produces a null value for any parameter names that are in the existing > context, but not in the submitted parameters. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [nifi] asfgit closed pull request #6138: NIFI-10140 Handle deleted parameters in StandardVersionedComponentSyn…
asfgit closed pull request #6138: NIFI-10140 Handle deleted parameters in StandardVersionedComponentSyn… URL: https://github.com/apache/nifi/pull/6138 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10140) Versioned component synchronizer not handling deleted parameters
[ https://issues.apache.org/jira/browse/NIFI-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556946#comment-17556946 ] ASF subversion and git services commented on NIFI-10140: Commit 1de9c727790df6aba5997bdc2f7cb12ac4e0657f in nifi's branch refs/heads/main from Bryan Bende [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1de9c72779 ] NIFI-10140 Handle deleted parameters in StandardVersionedComponentSynchronizer Signed-off-by: Joe Gresock This closes #6138. > Versioned component synchronizer not handling deleted parameters > > > Key: NIFI-10140 > URL: https://issues.apache.org/jira/browse/NIFI-10140 > Project: Apache NiFi > Issue Type: Bug >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > In the StandardVersionedComponentSynchronizer, there needs to be handling for > deleted parameters similar to what is in VersionedFlowSynchronizer where it > produces a null value for any parameter names that are in the existing > context, but not in the submitted parameters. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (NIFI-10145) Upgrade Hadoop to 3.3.3
[ https://issues.apache.org/jira/browse/NIFI-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556930#comment-17556930 ] David Handermann commented on NIFI-10145: - [~msr1716], no SLF4J is the primary logging facade library used across all NiFi projects, and version 1.7.36 is the latest version as of this writing. > Upgrade Hadoop to 3.3.3 > --- > > Key: NIFI-10145 > URL: https://issues.apache.org/jira/browse/NIFI-10145 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Multiple extension components should be upgraded to depend on Hadoop > [3.3.3|https://hadoop.apache.org/docs/r3.3.3/index.html] libraries, which > incorporates a number of updates, such as improved support for Java 11 and > better library handling for Guava and compression codecs. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (NIFI-10145) Upgrade Hadoop to 3.3.3
[ https://issues.apache.org/jira/browse/NIFI-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556926#comment-17556926 ] Mike R commented on NIFI-10145: --- Hey, would this remove all of the uses of slf4j version 1.7.36 in NiFi? > Upgrade Hadoop to 3.3.3 > --- > > Key: NIFI-10145 > URL: https://issues.apache.org/jira/browse/NIFI-10145 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Multiple extension components should be upgraded to depend on Hadoop > [3.3.3|https://hadoop.apache.org/docs/r3.3.3/index.html] libraries, which > incorporates a number of updates, such as improved support for Java 11 and > better library handling for Guava and compression codecs. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (NIFI-10147) Remove Support for Java 8
David Handermann created NIFI-10147: --- Summary: Remove Support for Java 8 Key: NIFI-10147 URL: https://issues.apache.org/jira/browse/NIFI-10147 Project: Apache NiFi Issue Type: Epic Components: Core Framework, MiNiFi, NiFi Registry, NiFi Stateless Reporter: David Handermann Assignee: David Handermann Fix For: 2.0.0 Although multiple vendors have planned extended support for Java 8, a number of projects are in the process of deprecating and removing support for Java 8. These projects include the following: - [Jetty 9.4|https://github.com/eclipse/jetty.project/issues/7958] - [Spring 5|https://spring.io/blog/2021/09/02/a-java-17-and-jakarta-ee-9-baseline-for-spring-framework-6] - [OpenSAML 3|https://shibboleth.atlassian.net/wiki/spaces/OSAML/overview] Apache NiFi should also remove support for Java 8 in order to maintain active support for various components. Removing support for Java 8 requires a new major version release and will also require several build configuration changes including the following: - Updating Maven source and compiler settings - Updating minimum Java build version - Removing optional Maven build profiles - Updating GitHub workflows Initial discussion on a migration strategy included a tentative target of the end of calendar year 2022. Implementation should not proceed until after a minor release of Apache NiFi 1.17.0. Mailing list discussion: [https://lists.apache.org/thread/mm1xf3b9nvrcgytb92oy3swvvc45fl34] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (NIFI-8971) Remove Legacy FlowFile Attributes
[ https://issues.apache.org/jira/browse/NIFI-8971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-8971: --- Summary: Remove Legacy FlowFile Attributes (was: For NiFi 2.0, remove support for legacy attributes) > Remove Legacy FlowFile Attributes > - > > Key: NIFI-8971 > URL: https://issues.apache.org/jira/browse/NIFI-8971 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andrew Atwood >Assignee: Aakash Chauhan >Priority: Minor > Labels: beginner > Fix For: 2.0.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > I believe the following attributes are considered legacy and only maintained > for backwards compatibility. As part of removing technical debt in 2.0 > release, support for them should be removed. > * nf.file.name > * nf.file.path > * content-encoding > * content-type > I found handling for them in MergeContent, UnpackContent, and > ListenHTTPServlet. There is also handling for them in PostHTTP, but that > processor is deprecated and planned to be removed in 2.0 release. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (NIFI-8981) Review TODO Comments and Deprecations
[ https://issues.apache.org/jira/browse/NIFI-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-8981: --- Summary: Review TODO Comments and Deprecations (was: For NiFi 2.0, go through TODOs) > Review TODO Comments and Deprecations > - > > Key: NIFI-8981 > URL: https://issues.apache.org/jira/browse/NIFI-8981 > Project: Apache NiFi > Issue Type: Task >Reporter: Andrew Atwood >Priority: Minor > Fix For: 2.0.0 > > > Once NiFi 2.0 plans are stable, all of the TODOs in the project should be > evaluated (Hopefully assignee's IDE can show all TODOs). > For example, currently Java 11 requirement is being planned for NiFi 2.0 > release, and I find lots of TODO comments saying to remove something when > minimum Java version is 11. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (NIFI-9982) Enrich SplitXML processor
[ https://issues.apache.org/jira/browse/NIFI-9982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-9982: --- Fix Version/s: (was: 2.0.0) > Enrich SplitXML processor > - > > Key: NIFI-9982 > URL: https://issues.apache.org/jira/browse/NIFI-9982 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.16.1 >Reporter: Jasper Knulst >Priority: Major > Labels: XML > > The Nifi SplitXML processor can currently split an XML like: > (starting from this base XML) > > > > .. > .. > > > > in as many child XML's as there are b-elements, like: > > .. (new FF) > > .. > .. > .. (new FF) > > What you want though is extra options to say: > I want child XML's with x-number of b-elements, or even give me x equally > sized outbound flowfiles out of the input XML, maintaining the XML envelope: > > > > > .. ( new FF) > > > > > .. ( new FF) > .. > .. > > > > > The splits can subsequently be load balanced to all Nifi nodes. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (NIFI-10146) Upgrade Apache Tika to 2.4.1
[
https://issues.apache.org/jira/browse/NIFI-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-10146:
Status: Patch Available (was: Open)
> Upgrade Apache Tika to 2.4.1
>
>
> Key: NIFI-10146
> URL: https://issues.apache.org/jira/browse/NIFI-10146
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Labels: dependency-upgrade
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Apache Tika
> [2.4.1|https://dist.apache.org/repos/dist/release/tika/2.4.1/CHANGES-2.4.1.txt]
> includes several improvements and also includes a newer version of
> {{metadata-extractor}}, which resolves CVE-2022-24613 and CVE-2022-24614
> related to processing crafted JPEG files.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [nifi] exceptionfactory opened a new pull request, #6140: NIFI-10146 Upgrade Apache Tika from 2.4.0 to 2.4.1
exceptionfactory opened a new pull request, #6140: URL: https://github.com/apache/nifi/pull/6140 # Summary [NIFI-10146](https://issues.apache.org/jira/browse/NIFI-10146) Upgrades Apache Tika dependencies from 2.4.0 to 2.4.1 and removes the direct dependency on `metadata-extractor` to inherit a newer transitive version, which resolves CVE-2022-24613 and CVE-2022-24613 related to processing crafted JPEG files. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10146) Upgrade Apache Tika to 2.4.1
David Handermann created NIFI-10146:
---
Summary: Upgrade Apache Tika to 2.4.1
Key: NIFI-10146
URL: https://issues.apache.org/jira/browse/NIFI-10146
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework, Extensions
Reporter: David Handermann
Assignee: David Handermann
Apache Tika
[2.4.1|https://dist.apache.org/repos/dist/release/tika/2.4.1/CHANGES-2.4.1.txt]
includes several improvements and also includes a newer version of
{{metadata-extractor}}, which resolves CVE-2022-24613 and CVE-2022-24614
related to processing crafted JPEG files.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (MINIFICPP-1866) Secure connection for ListenSyslog, ListenTCP
Martin Zink created MINIFICPP-1866: -- Summary: Secure connection for ListenSyslog, ListenTCP Key: MINIFICPP-1866 URL: https://issues.apache.org/jira/browse/MINIFICPP-1866 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Martin Zink Fix For: 0.13.0 It would be a great if ListenSyslog and the upcoming ListenTCP could make use of the SSLContextService and even force (optionally) client certificates aswell. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (MINIFICPP-1843) Implement PutElasticsearchJson and ElasticSearchClientServiceImpl
[ https://issues.apache.org/jira/browse/MINIFICPP-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-1843: --- Fix Version/s: 0.13.0 > Implement PutElasticsearchJson and ElasticSearchClientServiceImpl > - > > Key: MINIFICPP-1843 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1843 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Marton Szasz >Assignee: Martin Zink >Priority: Major > Fix For: 0.13.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > It would be useful to use the elastic REST API to implement the functionality > provided by PutElasticsearchJson, in MiNiFi C++. It's already possible to use > MiNiFi C++ with Elasticsearch, but it requires digging deep in the elastic > docs, and manually using the REST API with InvokeHTTP. This would be a simple > HTTP client processor that makes it easier for the users to use it. > It can go either in the http-curl extension or its own separate one, since it > doesn't require a new dependency, only libcurl and libressl. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (MINIFICPP-1863) PutGCSObject has NumberOfRetries twice in properties()
[ https://issues.apache.org/jira/browse/MINIFICPP-1863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-1863: --- Status: Patch Available (was: Open) https://github.com/apache/nifi-minifi-cpp/pull/1353 > PutGCSObject has NumberOfRetries twice in properties() > -- > > Key: MINIFICPP-1863 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1863 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > MINIFICPP-1803 duplicated NumberOfRetries in properties() which causes > problems in C2 describe operation. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (MINIFICPP-1863) PutGCSObject has NumberOfRetries twice in properties()
[ https://issues.apache.org/jira/browse/MINIFICPP-1863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-1863: --- Fix Version/s: 0.13.0 > PutGCSObject has NumberOfRetries twice in properties() > -- > > Key: MINIFICPP-1863 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1863 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Fix For: 0.13.0 > > Time Spent: 20m > Remaining Estimate: 0h > > MINIFICPP-1803 duplicated NumberOfRetries in properties() which causes > problems in C2 describe operation. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (MINIFICPP-1700) Change leftover integers to std::chrono from MINIFICPP-1688
[ https://issues.apache.org/jira/browse/MINIFICPP-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-1700: --- Status: Patch Available (was: Open) [https://github.com/apache/nifi-minifi-cpp/pull/1352] > Change leftover integers to std::chrono from MINIFICPP-1688 > --- > > Key: MINIFICPP-1700 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1700 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > > MINIFICPP-1688 left out some integer->std::chrono refactors. > e.g. > [extensions/sftp/processors/ListSFTP.cpp|https://github.com/apache/nifi-minifi-cpp/pull/1225/files/f487526c2ac5b8037a369b53b23e42c7a5c9cb89#diff-2a26794f434d15c4c7911b91d7bf66a6e8069d49856da2b30c556439adb1b67c] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (MINIFICPP-1700) Change leftover integers to std::chrono from MINIFICPP-1688
[ https://issues.apache.org/jira/browse/MINIFICPP-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-1700: --- Fix Version/s: 0.13.0 > Change leftover integers to std::chrono from MINIFICPP-1688 > --- > > Key: MINIFICPP-1700 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1700 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > Fix For: 0.13.0 > > > MINIFICPP-1688 left out some integer->std::chrono refactors. > e.g. > [extensions/sftp/processors/ListSFTP.cpp|https://github.com/apache/nifi-minifi-cpp/pull/1225/files/f487526c2ac5b8037a369b53b23e42c7a5c9cb89#diff-2a26794f434d15c4c7911b91d7bf66a6e8069d49856da2b30c556439adb1b67c] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (MINIFICPP-1842) getTimeStr should use std::chrono
[ https://issues.apache.org/jira/browse/MINIFICPP-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-1842: --- Fix Version/s: 0.13.0 > getTimeStr should use std::chrono > - > > Key: MINIFICPP-1842 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1842 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > Fix For: 0.13.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (NIFI-10143) ListFTP/GetFTP has issues with danish special characters (UTF-8)
[
https://issues.apache.org/jira/browse/NIFI-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Grey reassigned NIFI-10143:
Assignee: Paul Grey
> ListFTP/GetFTP has issues with danish special characters (UTF-8)
>
>
> Key: NIFI-10143
> URL: https://issues.apache.org/jira/browse/NIFI-10143
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.16.2
> Environment: NIFI Version 1.16.2, Java 8
>Reporter: Jens M Kofoed
>Assignee: Paul Grey
>Priority: Major
>
> ListFTP and GetFTP has issues listing file/directories including danish
> charaters (æøå).
> The ftp server is MS IIS FTP, and UTF-8 has been set to true.
> with a small test flow, I used a GenerateFlowfile and added 2 Properties.
> filename="test.txt" and path="æøå". The output is going to a PutFTP, with
> Remote path set to ${path}
> The PutFTP does indeed create a subfolder at the ftp server, named æøå. But...
> With a ListFTP or GetFTP process, looking into the same root folder gives an
> error.
> ListFTP[id=e4c43ea1-2d5d-1fea--89c101ce] Unable to get listing from
> /���; skipping: java.io.IOException: Cannot list files for non-existent
> directory /���
>
> Testing the old case from NIFI-8250. I got issues with the arabic charaters
> too.
> PutFTP[id=e4c43ef3-2d5d-1fea--ea9da690] Unable to transfer
> FlowFile[filename=اختبار] to remote host 192.168.1.2 due to IOException
> thrown from PutFTP[id=e4c43ef3-2d5d-1fea--ea9da690]:
> java.io.IOException: Failed to store file .اختبار to /test/اختبار due to: 550
> The filename, directory name, or volume label syntax is incorrect.
> : java.io.IOException: Failed to store file .اختبار to /test/اختبار due to:
> 550 The filename, directory name, or volume label syntax is incorrect.
> ; routing to failure: {}
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (NIFI-10145) Upgrade Hadoop to 3.3.3
[ https://issues.apache.org/jira/browse/NIFI-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10145: Status: Patch Available (was: Open) > Upgrade Hadoop to 3.3.3 > --- > > Key: NIFI-10145 > URL: https://issues.apache.org/jira/browse/NIFI-10145 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Multiple extension components should be upgraded to depend on Hadoop > [3.3.3|https://hadoop.apache.org/docs/r3.3.3/index.html] libraries, which > incorporates a number of updates, such as improved support for Java 11 and > better library handling for Guava and compression codecs. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [nifi] exceptionfactory opened a new pull request, #6139: NIFI-10145 Upgrade Hadoop from 3.3.2 to 3.3.3
exceptionfactory opened a new pull request, #6139: URL: https://github.com/apache/nifi/pull/6139 # Summary [NIFI-10145](https://issues.apache.org/jira/browse/NIFI-10145) Upgrades shared Hadoop library versions from 3.3.2 to [3.3.3](https://hadoop.apache.org/docs/r3.3.3/index.html). Hadoop 3.3.3 replaced `slf4j-log4j12` with `slf4j-reload4j`, requiring a new exclusion to avoid dependencies on Reload4j. Additional changes include updating the following bundles to use the same shared Hadoop version: - nifi-hive3-processors from version 3.1.1 - nifi-accumulo-bundle from version 3.1.1 - nifi-atlas-bundle from version 3.3.2 - nifi-spark-receiver from version 3.3.2 # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10145) Upgrade Hadoop to 3.3.3
David Handermann created NIFI-10145: --- Summary: Upgrade Hadoop to 3.3.3 Key: NIFI-10145 URL: https://issues.apache.org/jira/browse/NIFI-10145 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: David Handermann Assignee: David Handermann Multiple extension components should be upgraded to depend on Hadoop [3.3.3|https://hadoop.apache.org/docs/r3.3.3/index.html] libraries, which incorporates a number of updates, such as improved support for Java 11 and better library handling for Guava and compression codecs. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Comment Edited] (NIFI-7900) Add AWS session token to AWSCredentialsProvider
[ https://issues.apache.org/jira/browse/NIFI-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556874#comment-17556874 ] iain smith edited comment on NIFI-7900 at 6/21/22 12:39 PM: [~john.wise] - Assuming your use case above is not running on an EC2 instance, could you also use default credentials, by setting 'Use Default Credentials' to 'true' then configuring your periodic STS-endpoint credentials retrieving flow to write the access key id, secret access key and session token into the AWS default credentials file $HOME/.aws/credentials (in the format shown below)? [default] aws_access_key_id = aws_secret_access_key = aws_session_token = X or set them as environment variables with the correct names maybe? That should also be picked up as long as 'Use Default Credentials' is 'true' was (Author: JIRAUSER291337): [~john.wise] - Could this be used in your use case above, by setting 'Use Default Credentials' to 'true' then configuring your periodic STS-endpoint credentials retrieving flow to write the access key id, secret access key and session token into the AWS default credentials file $HOME/.aws/credentials (in the format shown below)? [default] aws_access_key_id = aws_secret_access_key = aws_session_token = X or set them as environment variables with the correct names maybe? That should also be picked up as long as 'Use Default Credentials' is 'true' > Add AWS session token to AWSCredentialsProvider > --- > > Key: NIFI-7900 > URL: https://issues.apache.org/jira/browse/NIFI-7900 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.9.2, 1.12.1 >Reporter: Jody >Assignee: Peter Turcsanyi >Priority: Major > > As a NiFi user, I want to use AWS processors, e.g. PutS3Object processor, > with temporary credentials to allow connecting to secure AWS environments > that make use of the AWS Security Token Service. > > The NiFi AWSCredentialsProviderControllerService is giving an option to add > the required fields for using temporary credentials. While access key id and > secret access key properties can be configured, the property "session token" > is not available. The session token property must be provided when temporary > credentials are used. If the session token is not presented, an error will be > thrown: "The AWS Access Key Id you provided does not exist in our records. > (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId" -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (NIFI-10143) ListFTP/GetFTP has issues with danish special characters (UTF-8)
[
https://issues.apache.org/jira/browse/NIFI-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556877#comment-17556877
]
David Handermann commented on NIFI-10143:
-
Thanks for reporting this issue [~jmkofoed].
The changes for NIFI-6266 should have maintained support for UTF-8 handling in
the FTPClient through the UTF-8 Encoding property, but the changes moved
property configuration to the StandardFTPClientProvider class:
[https://github.com/apache/nifi/commit/95abc35463d43ea9ee62dc041f85dca38579fec2#diff-815ac6b9740e1a8327479cb5784d20016372250d543c7c5204a31feff659e53fR149]
As you noted, however, it sounds like the new implementation did not maintain
the expected behavior.
> ListFTP/GetFTP has issues with danish special characters (UTF-8)
>
>
> Key: NIFI-10143
> URL: https://issues.apache.org/jira/browse/NIFI-10143
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.16.2
> Environment: NIFI Version 1.16.2, Java 8
>Reporter: Jens M Kofoed
>Priority: Major
>
> ListFTP and GetFTP has issues listing file/directories including danish
> charaters (æøå).
> The ftp server is MS IIS FTP, and UTF-8 has been set to true.
> with a small test flow, I used a GenerateFlowfile and added 2 Properties.
> filename="test.txt" and path="æøå". The output is going to a PutFTP, with
> Remote path set to ${path}
> The PutFTP does indeed create a subfolder at the ftp server, named æøå. But...
> With a ListFTP or GetFTP process, looking into the same root folder gives an
> error.
> ListFTP[id=e4c43ea1-2d5d-1fea--89c101ce] Unable to get listing from
> /���; skipping: java.io.IOException: Cannot list files for non-existent
> directory /���
>
> Testing the old case from NIFI-8250. I got issues with the arabic charaters
> too.
> PutFTP[id=e4c43ef3-2d5d-1fea--ea9da690] Unable to transfer
> FlowFile[filename=اختبار] to remote host 192.168.1.2 due to IOException
> thrown from PutFTP[id=e4c43ef3-2d5d-1fea--ea9da690]:
> java.io.IOException: Failed to store file .اختبار to /test/اختبار due to: 550
> The filename, directory name, or volume label syntax is incorrect.
> : java.io.IOException: Failed to store file .اختبار to /test/اختبار due to:
> 550 The filename, directory name, or volume label syntax is incorrect.
> ; routing to failure: {}
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (NIFI-9908) MiNiFi C2 test coverage
[ https://issues.apache.org/jira/browse/NIFI-9908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Csaba Bejan updated NIFI-9908: -- Summary: MiNiFi C2 test coverage (was: MiNiFi integration with c2-client-service + tests) > MiNiFi C2 test coverage > --- > > Key: NIFI-9908 > URL: https://issues.apache.org/jira/browse/NIFI-9908 > Project: Apache NiFi > Issue Type: Sub-task > Components: C2, MiNiFi >Reporter: Csaba Bejan >Priority: Major > > Add test coverage for the newly introduced C2 functionality and look for > opportunities to refactor. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (NIFI-7900) Add AWS session token to AWSCredentialsProvider
[ https://issues.apache.org/jira/browse/NIFI-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556874#comment-17556874 ] iain smith commented on NIFI-7900: -- [~john.wise] - Could this be used in your use case above, by setting 'Use Default Credentials' to 'true' then configuring your periodic STS-endpoint credentials retrieving flow to write the access key id, secret access key and session token into the AWS default credentials file $HOME/.aws/credentials (in the format shown below)? [default] aws_access_key_id = aws_secret_access_key = aws_session_token = X or set them as environment variables with the correct names maybe? That should also be picked up as long as 'Use Default Credentials' is 'true' > Add AWS session token to AWSCredentialsProvider > --- > > Key: NIFI-7900 > URL: https://issues.apache.org/jira/browse/NIFI-7900 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.9.2, 1.12.1 >Reporter: Jody >Assignee: Peter Turcsanyi >Priority: Major > > As a NiFi user, I want to use AWS processors, e.g. PutS3Object processor, > with temporary credentials to allow connecting to secure AWS environments > that make use of the AWS Security Token Service. > > The NiFi AWSCredentialsProviderControllerService is giving an option to add > the required fields for using temporary credentials. While access key id and > secret access key properties can be configured, the property "session token" > is not available. The session token property must be provided when temporary > credentials are used. If the session token is not presented, an error will be > thrown: "The AWS Access Key Id you provided does not exist in our records. > (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId" -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (NIFI-9908) MiNiFi integration with c2-client-service + tests
[ https://issues.apache.org/jira/browse/NIFI-9908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Csaba Bejan updated NIFI-9908: -- Description: Add test coverage for the newly introduced C2 functionality and look for opportunities to refactor. (was: Validate that MiNiFi implementation works well with the basic c2-client-service. Document the setup steps someone needs to take to start a purely apache setup. In addition look into opportunities to have some kind of integration test with minifi working together with the s2-client-service. Look into integration with github actions as that is tied to Apache NiFi.) > MiNiFi integration with c2-client-service + tests > - > > Key: NIFI-9908 > URL: https://issues.apache.org/jira/browse/NIFI-9908 > Project: Apache NiFi > Issue Type: Sub-task > Components: C2, MiNiFi >Reporter: Csaba Bejan >Priority: Major > > Add test coverage for the newly introduced C2 functionality and look for > opportunities to refactor. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [nifi] timeabarna commented on pull request #6115: NIFI-10108 Processor scheduling via parameter
timeabarna commented on PR #6115: URL: https://github.com/apache/nifi/pull/6115#issuecomment-1161681954 Hello @markap14, Thanks for your review. I've modified the PR based on your recommendation. Can you please take another look? Thanks in advance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (NIFI-7900) Add AWS session token to AWSCredentialsProvider
[ https://issues.apache.org/jira/browse/NIFI-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556793#comment-17556793 ] iain smith edited comment on NIFI-7900 at 6/21/22 12:24 PM: update - the above use case (accessing an EC2 instance's temporary AWS credentials granted by its associated instance profile) is already supported, as per the documentation at [https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.15.3/org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService/index.html] - so there is no need to try to retrieve and manually use the credentials from the EC2 metadata. All that is required is to set the AWSCredentialsProviderControllerService 'Use Default Credentials' to 'true', and ensure that no other conflicting properties are set (eg. 'Credentials File'). was (Author: JIRAUSER291337): I've just noticed on [https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.15.3/org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService/index.html] "Uses default credentials without configuration. Default credentials support EC2 instance profile/role, default user profile, environment variables, etc" Is using the temporary credentials from an EC2 instance profile/role already supported? (going to test this) > Add AWS session token to AWSCredentialsProvider > --- > > Key: NIFI-7900 > URL: https://issues.apache.org/jira/browse/NIFI-7900 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.9.2, 1.12.1 >Reporter: Jody >Assignee: Peter Turcsanyi >Priority: Major > > As a NiFi user, I want to use AWS processors, e.g. PutS3Object processor, > with temporary credentials to allow connecting to secure AWS environments > that make use of the AWS Security Token Service. > > The NiFi AWSCredentialsProviderControllerService is giving an option to add > the required fields for using temporary credentials. While access key id and > secret access key properties can be configured, the property "session token" > is not available. The session token property must be provided when temporary > credentials are used. If the session token is not presented, an error will be > thrown: "The AWS Access Key Id you provided does not exist in our records. > (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId" -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] (NIFI-7900) Add AWS session token to AWSCredentialsProvider
[ https://issues.apache.org/jira/browse/NIFI-7900 ] iain smith deleted comment on NIFI-7900: -- was (Author: JIRAUSER291337): This really should be supported for Nifi running on EC2 instances... we should not have to create an IAM user with static API keys for Nifi to use, when the EC2 already provides local API credentials which are more secure (ie. regularly auto-rotated, and not likely to set off irritating security non-compliance alerts!) > Add AWS session token to AWSCredentialsProvider > --- > > Key: NIFI-7900 > URL: https://issues.apache.org/jira/browse/NIFI-7900 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.9.2, 1.12.1 >Reporter: Jody >Assignee: Peter Turcsanyi >Priority: Major > > As a NiFi user, I want to use AWS processors, e.g. PutS3Object processor, > with temporary credentials to allow connecting to secure AWS environments > that make use of the AWS Security Token Service. > > The NiFi AWSCredentialsProviderControllerService is giving an option to add > the required fields for using temporary credentials. While access key id and > secret access key properties can be configured, the property "session token" > is not available. The session token property must be provided when temporary > credentials are used. If the session token is not presented, an error will be > thrown: "The AWS Access Key Id you provided does not exist in our records. > (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId" -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (NIFI-7712) Remove cache key from cache server
[ https://issues.apache.org/jira/browse/NIFI-7712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556849#comment-17556849 ] oskarryn commented on NIFI-7712: I concur, that's a duplicate of NIFI-4173. Very sad that this feature is still unavailable after 5 years. > Remove cache key from cache server > -- > > Key: NIFI-7712 > URL: https://issues.apache.org/jira/browse/NIFI-7712 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Chandraprakash kabra >Priority: Major > > Hi Team, > I am using distributedmap cache and my requirement is I just would like to > remove cache key from cache server after fetching it from > fetchdistributedmapcache. > So is there any inbuild processor and functionality available or I have to > write script for that. I have seem some of the issue but they are suggesting > script but I am in favour of inbuild processor if we can just pass the key > and remove it. > Every 24 hour new cache entry would come and old is also there so I would > like to remove old. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (MINIFICPP-1864) Splunk docker tests fail in CI
[ https://issues.apache.org/jira/browse/MINIFICPP-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi resolved MINIFICPP-1864. -- Fix Version/s: 0.13.0 Resolution: Fixed > Splunk docker tests fail in CI > -- > > Key: MINIFICPP-1864 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1864 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Minor > Fix For: 0.13.0 > > Attachments: splunk_failure.log > > Time Spent: 20m > Remaining Estimate: 0h > > See logs attached for more info -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [nifi-minifi-cpp] martinzink closed pull request #1355: MINIFICPP-1864 Fix Splunk tests with newer container image
martinzink closed pull request #1355: MINIFICPP-1864 Fix Splunk tests with newer container image URL: https://github.com/apache/nifi-minifi-cpp/pull/1355 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
