[jira] [Created] (NIFI-7689) QID 11827 Security Headers
Jon McAlexander created NIFI-7689: - Summary: QID 11827 Security Headers Key: NIFI-7689 URL: https://issues.apache.org/jira/browse/NIFI-7689 Project: Apache NiFi Issue Type: Bug Components: Core Framework Reporter: Jon McAlexander Looking for appropriate document to configure the HTTP Security Headers to satisfy a Qualsys ID 11827 Finding on Apache Nifi. Please advise. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7525) Create a nullValue() Subjectless Functions and support null type on Expression Language
[ https://issues.apache.org/jira/browse/NIFI-7525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabrizio Spataro updated NIFI-7525: --- Summary: Create a nullValue() Subjectless Functions and support null type on Expression Language (was: Create a null() Subjectless Functions and support null type on Expression Language) > Create a nullValue() Subjectless Functions and support null type on > Expression Language > --- > > Key: NIFI-7525 > URL: https://issues.apache.org/jira/browse/NIFI-7525 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Fabrizio Spataro >Priority: Minor > > Reading (and using) nifi expression language, could be useful have a function > to set a "null value" easily! > Today, to resolve this, we can apply some tricks, for example: use an > UpdateRecord processor with an unknown RecordPath value or convert record in > text (json) and apply ReplaceText processor using a regex + *null* value. > > This issue will propose you a new stateless function, called *nullValue()* > to retrieve a real "null value" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-7688) NiFi Does Not Run on Windows 10 Pro
[ https://issues.apache.org/jira/browse/NIFI-7688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto resolved NIFI-7688. - Fix Version/s: 1.12.0 Resolution: Duplicate > NiFi Does Not Run on Windows 10 Pro > --- > > Key: NIFI-7688 > URL: https://issues.apache.org/jira/browse/NIFI-7688 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.11.4 > Environment: WIndows 10 Pro >Reporter: Tim Wollin >Priority: Major > Fix For: 1.12.0 > > > The check on line 1040 of RunNiFi.java is always false on Win10 Pro. On > Win10 Pro OpenJDK 11 outputs: > d:\Development\nifi\nifi-1.11.4\bin>java --version > openjdk 11 2018-09-25 > OpenJDK Runtime Environment 18.9 (build 11+28) > OpenJDK 64-Bit Server VM 18.9 (build 11+28, mixed mode) > System.getProperty("java.version") is always set to "11". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7688) NiFi Does Not Run on Windows 10 Pro
[ https://issues.apache.org/jira/browse/NIFI-7688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167514#comment-17167514 ] Joey Frazee commented on NIFI-7688: --- The pattern for all releases since 9 has been for the first GA release to use a bare number (e.g., 11 vs. 11.0 or 11.0.0) so it's probably not specific to Windows. > NiFi Does Not Run on Windows 10 Pro > --- > > Key: NIFI-7688 > URL: https://issues.apache.org/jira/browse/NIFI-7688 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.11.4 > Environment: WIndows 10 Pro >Reporter: Tim Wollin >Priority: Major > > The check on line 1040 of RunNiFi.java is always false on Win10 Pro. On > Win10 Pro OpenJDK 11 outputs: > d:\Development\nifi\nifi-1.11.4\bin>java --version > openjdk 11 2018-09-25 > OpenJDK Runtime Environment 18.9 (build 11+28) > OpenJDK 64-Bit Server VM 18.9 (build 11+28, mixed mode) > System.getProperty("java.version") is always set to "11". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7688) NiFi Does Not Run on Windows 10 Pro
Tim Wollin created NIFI-7688: Summary: NiFi Does Not Run on Windows 10 Pro Key: NIFI-7688 URL: https://issues.apache.org/jira/browse/NIFI-7688 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.11.4 Environment: WIndows 10 Pro Reporter: Tim Wollin The check on line 1040 of RunNiFi.java is always false on Win10 Pro. On Win10 Pro OpenJDK 11 outputs: d:\Development\nifi\nifi-1.11.4\bin>java --version openjdk 11 2018-09-25 OpenJDK Runtime Environment 18.9 (build 11+28) OpenJDK 64-Bit Server VM 18.9 (build 11+28, mixed mode) System.getProperty("java.version") is always set to "11". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7687) NiFi fails to start when running on first release of JDK major version
Andy LoPresto created NIFI-7687: --- Summary: NiFi fails to start when running on first release of JDK major version Key: NIFI-7687 URL: https://issues.apache.org/jira/browse/NIFI-7687 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.11.4 Reporter: Andy LoPresto Assignee: Andy LoPresto Fix For: 1.12.0 When the JDK reports the version without any minor component (e.g. as {{11}}), the application fails to start. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6976) Native integrations of Apache Nifi with Delta Lake
[ https://issues.apache.org/jira/browse/NIFI-6976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto updated NIFI-6976: Affects Version/s: (was: 1.12.0) > Native integrations of Apache Nifi with Delta Lake > -- > > Key: NIFI-6976 > URL: https://issues.apache.org/jira/browse/NIFI-6976 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Martin >Priority: Major > Labels: ACID, delta, integration, spark, storage-layer > > "Delta Lake is an open-source storage layer that brings ACID transactions to > Apache Spar and big data workloads" ([https://delta.io/]) > > NiFi already offers many features that make it unique. A Delta integration > would complement this scope in a sensible way. > > Source: [https://github.com/delta-io/delta] > ## Update ## > Here is a demo how this looks at Streamsets. Having the same feature set on > Nifi would we awesome. > Video: [https://youtu.be/VLd_qOrKrTI] > > Here you can find all available integrations right now. > [https://docs.delta.io/0.5.0/integrations.html] > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7640) HandleHttpRequest - define temporary files location
[ https://issues.apache.org/jira/browse/NIFI-7640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167503#comment-17167503 ] Andy LoPresto commented on NIFI-7640: - PR 4414 was linked to this Jira but only added documentation for the {{HandleHttpRequest}} rather than implementing this property. Is there a separate PR for that? > HandleHttpRequest - define temporary files location > --- > > Key: NIFI-7640 > URL: https://issues.apache.org/jira/browse/NIFI-7640 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Tamás Bunth >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > A property should be added to the processor allowing users to define where > temporary files are located. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7640) HandleHttpRequest - define temporary files location
[ https://issues.apache.org/jira/browse/NIFI-7640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167502#comment-17167502 ] ASF subversion and git services commented on NIFI-7640: --- Commit 455f48fce4fbbae19df27a7702d49f1912f8753a in nifi's branch refs/heads/main from Tamás Bunth [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=455f48f ] NIFI-7640 Add documentation: temporary directory (#4414) NiFi uses the Java IO temporary directory for storing HTTP multipart files when using HandleHttpRequest processor. The directory can be overwritten with Java command line parameter. > HandleHttpRequest - define temporary files location > --- > > Key: NIFI-7640 > URL: https://issues.apache.org/jira/browse/NIFI-7640 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Tamás Bunth >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > A property should be added to the processor allowing users to define where > temporary files are located. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFIREG-292) Add database implementations of UserGroupProvider and AccessPolicyProvider
[ https://issues.apache.org/jira/browse/NIFIREG-292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167468#comment-17167468 ] Kevin Doran commented on NIFIREG-292: - [~ChrisEng] Ideally, perhaps, however, the different clustering model used by NiFi, where changes to the state are replicated across all nodes, mean that the existing File-based providers are sufficient in NiFi for HA, whereas the NiFi Registry clustering approach relies on an external, central store for all state (ie, the database) > Add database implementations of UserGroupProvider and AccessPolicyProvider > -- > > Key: NIFIREG-292 > URL: https://issues.apache.org/jira/browse/NIFIREG-292 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Major > Time Spent: 3h > Remaining Estimate: 0h > > We should offer database backed implementations of UserGroupProvider and > AccessPolicyProvider as an alternative to the file-based impls. We have LDAP > and Ranger alternatives, but for people not using those, the DB impls would > be a good way to get the data off the local filesystem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] sylmorin-gbif closed pull request #4433: NIFI-528 add support to specify timeout in ExecuteProcess
sylmorin-gbif closed pull request #4433: URL: https://github.com/apache/nifi/pull/4433 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] sylmorin-gbif opened a new pull request #4440: NIFI-528 add support to specify timeout in ExecuteProcess
sylmorin-gbif opened a new pull request #4440: URL: https://github.com/apache/nifi/pull/4440 From the unfinished PR #1755 by @medjessy Fixing the indentation issues as requested in PR comments This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #853: MINIFICPP-952 - Use GitHub Actions
fgerlits commented on a change in pull request #853: URL: https://github.com/apache/nifi-minifi-cpp/pull/853#discussion_r462286271 ## File path: .github/workflows/ci.yml ## @@ -0,0 +1,329 @@ +name: "MiNiFi-CPP CI" +on: [push, pull_request, workflow_dispatch] +jobs: + macos_xcode_10_3: +name: "macos-xcode10.3" +runs-on: macos-10.15 +timeout-minutes: 60 +steps: + - id: checkout +uses: actions/checkout@v2 + - id: cache +uses: actions/cache@v2 +with: + path: | +~/Library/Caches/Homebrew + key: xcode10.3-ccache-${{github.ref}}-${{github.sha}} + restore-keys: | +xcode10.3-ccache-${{github.ref}}- +xcode10.3-ccache-refs/heads/main- + - id: install_dependencies +run: brew install ccache ossp-uuid boost flex openssl python lua libpcap xz libssh2 + - id: setup_env +run: | + echo -e "127.0.0.1\t$HOSTNAME" | sudo tee -a /etc/hosts > /dev/null + echo "::set-env name=CMAKE_BUILD_OPTIONS::-DENABLE_PCAP=TRUE -DENABLE_LUA_SCRIPTING=1" + sudo xcode-select -switch /Applications/Xcode_10.3.app + echo "::set-env name=PATH::`brew --prefix`/opt/ccache/libexec:$PATH" + - id: build +run: ./bootstrap.sh -e -t && cd build && cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_RULE_MESSAGES=OFF -DSTRICT_GSL_CHECKS=ON .. && cmake --build . --parallel 4 && make test ARGS="--timeout 300 -j4 --output-on-failure" && make linter + macos_xcode_11_2_1: +name: "macos-xcode11.2.1" +runs-on: macos-10.15 +timeout-minutes: 60 +steps: + - id: checkout +uses: actions/checkout@v2 + - id: cache +uses: actions/cache@v2 +with: + path: | +~/Library/Caches/Homebrew + key: xcode11.2.1-ccache-${{github.ref}}-${{github.sha}} + restore-keys: | +xcode11.2.1-ccache-${{github.ref}}- +xcode11.2.1-ccache-refs/heads/main- + - id: install_dependencies +run: brew install ccache ossp-uuid boost flex openssl python lua libpcap xz libssh2 + - id: setup_env +run: | + echo -e "127.0.0.1\t$HOSTNAME" | sudo tee -a /etc/hosts > /dev/null + echo "::set-env name=CMAKE_BUILD_OPTIONS::-DENABLE_PCAP=TRUE -DENABLE_LUA_SCRIPTING=1" + sudo xcode-select -switch /Applications/Xcode_11.2.1.app + echo "::set-env name=PATH::`brew --prefix`/opt/ccache/libexec:$PATH" + - id: build +run: ./bootstrap.sh -e -t && cd build && cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_RULE_MESSAGES=OFF -DSTRICT_GSL_CHECKS=ON .. && cmake --build . --parallel 4 && make test ARGS="--timeout 300 -j4 --output-on-failure" && make linter + windows: +name: "windows" +runs-on: windows-2016 +timeout-minutes: 90 +steps: + - id: checkout +uses: actions/checkout@v2 + - id: build +run: | + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\Tools\;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\;%path% + PATH=C:\Windows\Microsoft.NET\Framework\v4.0.30319;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin;%path% + PATH=C:\Program Files (x86)\Windows Kits\10\bin\x86;%path% + PATH=C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x86;%path% + PATH=C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.6.1 Tools\;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Performance Tools;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\bin\Roslyn;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\VC\VCPackages;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\HostX86\x86;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;%path% + PATH=C:\Windows\System32\;%path% + win_build_vs.bat build /CI +shell: cmd + ubuntu_16_04: +name: "ubuntu-16.04" +runs-on: ubuntu-16.04 +timeout-minutes: 60 +steps: + - id: checkout +uses: actions/checkout@v2 + - id: cache +uses: actions/cache@v2 +with: +
[GitHub] [nifi] tirkarthi commented on pull request #4437: NIFI-7682 Check for request object to be null.
tirkarthi commented on pull request #4437: URL: https://github.com/apache/nifi/pull/4437#issuecomment-665776665 Thanks @pvillard31 . Just curious if there are test cases for the web API where this can be tested? I tam new to Java. I tried using okhttpclient to test by passing empty payload but couldn't setup the nifi server inside tests. I tried using `testPut` method but it doesn't accept null to be passed for the endpoint. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev opened a new pull request #855: MINIFICPP-1288 - Remove FlowController::reload() as it is both flawed and unused
hunyadi-dev opened a new pull request #855: URL: https://github.com/apache/nifi-minifi-cpp/pull/855 Move FlowController::reload() where it belongs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-7686) Support multiple key columns in database lookup services
Ferdinand created NIFI-7686: --- Summary: Support multiple key columns in database lookup services Key: NIFI-7686 URL: https://issues.apache.org/jira/browse/NIFI-7686 Project: Apache NiFi Issue Type: Wish Components: Core Framework Reporter: Ferdinand {{SimpleDatabaseLookupService }}and{{ DatabaseRecordLookupService}} require to configure a _"Lookup Key Column"_. However, sometimes the database schema might require you to lookup a record or attribute identified by a composite primary key, i.e. using multiple keys. Wouldn't it be convenient to be able to specify any amount of lookup key columns and {{AND}}-concatenate the respective values in the [query's {{WHERE}}-clause?|https://github.com/apache/nifi/blob/716ba992f5b4641c20465638d62d1e3ffe91d118/nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services/src/main/java/org/apache/nifi/lookup/db/SimpleDatabaseLookupService.java#L139]?] Instead of a single string, {{LOOKUP_KEY_COLUMN}} (and the lookups processors' respective fields) may accept a comma-separated list of column names / identifiers. What are your thoughts on this? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7686) Support multiple key columns in database lookup services
[ https://issues.apache.org/jira/browse/NIFI-7686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferdinand updated NIFI-7686: Description: {{SimpleDatabaseLookupService and DatabaseRecordLookupService}} require to configure a _"Lookup Key Column"_. However, sometimes the database schema might require you to lookup a record or attribute identified by a composite primary key, i.e. using multiple keys. Wouldn't it be convenient to be able to specify any amount of lookup key columns and {{AND}}-concatenate the respective values in the [query's {{WHERE}}-clause?|https://github.com/apache/nifi/blob/716ba992f5b4641c20465638d62d1e3ffe91d118/nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services/src/main/java/org/apache/nifi/lookup/db/SimpleDatabaseLookupService.java#L139]? Instead of a single string, {{LOOKUP_KEY_COLUMN}} (and the lookups processors' respective fields) may accept a comma-separated list of column names / identifiers. What are your thoughts on this? was: {{SimpleDatabaseLookupService }}and{{ DatabaseRecordLookupService}} require to configure a _"Lookup Key Column"_. However, sometimes the database schema might require you to lookup a record or attribute identified by a composite primary key, i.e. using multiple keys. Wouldn't it be convenient to be able to specify any amount of lookup key columns and {{AND}}-concatenate the respective values in the [query's {{WHERE}}-clause?|https://github.com/apache/nifi/blob/716ba992f5b4641c20465638d62d1e3ffe91d118/nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services/src/main/java/org/apache/nifi/lookup/db/SimpleDatabaseLookupService.java#L139]?] Instead of a single string, {{LOOKUP_KEY_COLUMN}} (and the lookups processors' respective fields) may accept a comma-separated list of column names / identifiers. What are your thoughts on this? > Support multiple key columns in database lookup services > > > Key: NIFI-7686 > URL: https://issues.apache.org/jira/browse/NIFI-7686 > Project: Apache NiFi > Issue Type: Wish > Components: Core Framework >Reporter: Ferdinand >Priority: Minor > > {{SimpleDatabaseLookupService and DatabaseRecordLookupService}} require to > configure a _"Lookup Key Column"_. However, sometimes the database schema > might require you to lookup a record or attribute identified by a composite > primary key, i.e. using multiple keys. Wouldn't it be convenient to be able > to specify any amount of lookup key columns and {{AND}}-concatenate the > respective values in the [query's > {{WHERE}}-clause?|https://github.com/apache/nifi/blob/716ba992f5b4641c20465638d62d1e3ffe91d118/nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services/src/main/java/org/apache/nifi/lookup/db/SimpleDatabaseLookupService.java#L139]? > Instead of a single string, {{LOOKUP_KEY_COLUMN}} (and the lookups > processors' respective fields) may accept a comma-separated list of column > names / identifiers. > What are your thoughts on this? > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-6332) PutS3 Processor adding Cache-Control property
[ https://issues.apache.org/jira/browse/NIFI-6332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Nguyen reassigned NIFI-6332: - Assignee: Kent Nguyen > PutS3 Processor adding Cache-Control property > -- > > Key: NIFI-6332 > URL: https://issues.apache.org/jira/browse/NIFI-6332 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.8.0 >Reporter: Dirk Arends >Assignee: Kent Nguyen >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > The PutS3 Processor does not allow you to set the Cache-Control header. > My use case is to be able to control the Cache-Control header on objects > intended to be served via S3 website hosting. After looking at other issues I > found NIFI-2810 which references [Github PR > 1034|https://github.com/apache/nifi/pull/1034] which added the property > Content-Type to the PutS3Object. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-7664) PutS3 Processor adding Content-Disposition property
[ https://issues.apache.org/jira/browse/NIFI-7664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Nguyen reassigned NIFI-7664: - Assignee: Kent Nguyen > PutS3 Processor adding Content-Disposition property > --- > > Key: NIFI-7664 > URL: https://issues.apache.org/jira/browse/NIFI-7664 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Dirk Arends >Assignee: Kent Nguyen >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > The PutS3 Processor does not allow you to set the Content-Disposition header. > My use case is to be able to control the Content-Disposition header on > objects intended to be served via S3 website hosting. > This is related to NIFI-6332 and both changes can be use NIFI-2810 as an > example of how to implement the changes as NIFI-2810 adds the property > Content-Type to the PutS3Object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-7685) Add UTF8 support for FetchFTP
[ https://issues.apache.org/jira/browse/NIFI-7685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denes Arvay reassigned NIFI-7685: - Assignee: Denes Arvay > Add UTF8 support for FetchFTP > - > > Key: NIFI-7685 > URL: https://issues.apache.org/jira/browse/NIFI-7685 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Luca Giovannini >Assignee: Denes Arvay >Priority: Major > > When FetchFTP tries to fetch a file with name containing non-ASCII characters > (e.g. “Xml_APE001_unit*à*immobiliareguarda.xml”), the process succeeds > but the corresponding flowfile is empty. > > The issue is the same as the one in [1] but it just wasn't fixed for > FetchFTP. > [1] > [https://issues.apache.org/jira/browse/NIFI-4137|https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FNIFI-4137=02%7C01%7CLuca.Giovannini%40dedagroup.it%7C35da53a2e1db4aadb69a08d82800df73%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637303331502181485=ETqdW2puaeZKwDl1cr4raz8oll29qGFSN6fqVKO9%2FI4%3D=0] > > Thank you! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mattyb149 closed pull request #4439: NIFI-7683: Fixed bug that resulted in a byte[] being allocated and he…
mattyb149 closed pull request #4439: URL: https://github.com/apache/nifi/pull/4439 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on pull request #844: WIP: Use agent_version.cpp
adamdebreceni commented on pull request #844: URL: https://github.com/apache/nifi-minifi-cpp/pull/844#issuecomment-664949365 closing in favor of #853 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 opened a new pull request #4439: NIFI-7683: Fixed bug that resulted in a byte[] being allocated and he…
markap14 opened a new pull request #4439: URL: https://github.com/apache/nifi/pull/4439 …ld onto by a member variable that was unprotected. This caused the byte array to be modified by two different threads concurrently, which can potentially cause corruption of FlowFile's data Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Description of PR _Enables X functionality; fixes bug NIFI-._ In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically `main`)? - [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on JDK 8? - [ ] Have you verified that the full build is successful on JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni opened a new pull request #853: MINIFICPP-952 - Use GitHub Actions
adamdebreceni opened a new pull request #853: URL: https://github.com/apache/nifi-minifi-cpp/pull/853 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] tirkarthi opened a new pull request #4437: NIFI-7682 Check for request object to be null.
tirkarthi opened a new pull request #4437: URL: https://github.com/apache/nifi/pull/4437 Description of PR Check for request object to be null before accessing process group ID to fix NullPointerException. Fixes NIFI-7682 ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically `main`)? - [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on JDK 8? - [ ] Have you verified that the full build is successful on JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4435: NIFI-7669 Changed custom PBE AEAD algorithm to derive key once rather…
pvillard31 commented on pull request #4435: URL: https://github.com/apache/nifi/pull/4435#issuecomment-664856323 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-7685) Add UTF8 support for FetchFTP
Luca Giovannini created NIFI-7685: - Summary: Add UTF8 support for FetchFTP Key: NIFI-7685 URL: https://issues.apache.org/jira/browse/NIFI-7685 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Luca Giovannini When FetchFTP tries to fetch a file with name containing non-ASCII characters (e.g. “Xml_APE001_unit*à*immobiliareguarda.xml”), the process succeeds but the corresponding flowfile is empty. The issue is the same as the one in [1] but it just wasn't fixed for FetchFTP. [1] [https://issues.apache.org/jira/browse/NIFI-4137|https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FNIFI-4137=02%7C01%7CLuca.Giovannini%40dedagroup.it%7C35da53a2e1db4aadb69a08d82800df73%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637303331502181485=ETqdW2puaeZKwDl1cr4raz8oll29qGFSN6fqVKO9%2FI4%3D=0] Thank you! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] fgerlits opened a new pull request #854: MINIFICPP-1310 Fix the CMakeSettings.json file used by Visual Studio
fgerlits opened a new pull request #854: URL: https://github.com/apache/nifi-minifi-cpp/pull/854 * Change build type to 32-bit RelWithDebInfo * Copy all the variable settings from win_build_vs.bat * Set up the search path for mc.exe correctly (mixing individual items and lists doesn't work) * Update Windows.md --- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #853: MINIFICPP-952 - Use GitHub Actions
hunyadi-dev commented on a change in pull request #853: URL: https://github.com/apache/nifi-minifi-cpp/pull/853#discussion_r461472418 ## File path: .github/workflows/ci.yml ## @@ -0,0 +1,331 @@ +name: "MiNiFi-CPP CI" +on: [push, pull_request, workflow_dispatch] +jobs: + macos_xcode_10_3: +name: "macos-xcode10.3" +runs-on: macos-10.15 +timeout-minutes: 60 +steps: + - id: checkout +uses: actions/checkout@v2 + - id: cache +uses: actions/cache@v2 +with: + path: | +~/.ccache +~/Library/Caches/Homebrew + key: xcode10.3-ccache-${{github.ref}}-${{github.sha}} + restore-keys: | +xcode10.3-ccache-${{github.ref}}- +xcode10.3-ccache-refs/heads/main- + - id: install_dependencies +run: brew install ccache ossp-uuid boost flex openssl doxygen graphviz python lua libusb libpng libpcap xz libssh2 + - id: setup_env +run: | + echo -e "127.0.0.1\t$HOSTNAME" | sudo tee -a /etc/hosts > /dev/null + echo "::set-env name=CMAKE_BUILD_OPTIONS::-DENABLE_PCAP=TRUE -DENABLE_LUA_SCRIPTING=1" + sudo xcode-select -switch /Applications/Xcode_10.3.app + echo "::set-env name=PATH::`brew --prefix`/opt/ccache/libexec:$PATH" + - id: build +run: ./bootstrap.sh -e -t && cd build && cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_RULE_MESSAGES=OFF -DSTRICT_GSL_CHECKS=ON .. && cmake --build . --parallel 4 && sudo make test ARGS="--timeout 300 -j4 --output-on-failure" && make linter Review comment: Why do we need to run `./bootsrap.sh` if we definethe cmake flags manually anyway? Also why do we `sudo` on `make test`? ## File path: .github/workflows/ci.yml ## @@ -0,0 +1,331 @@ +name: "MiNiFi-CPP CI" +on: [push, pull_request, workflow_dispatch] +jobs: + macos_xcode_10_3: +name: "macos-xcode10.3" +runs-on: macos-10.15 +timeout-minutes: 60 +steps: + - id: checkout +uses: actions/checkout@v2 + - id: cache +uses: actions/cache@v2 +with: + path: | +~/.ccache +~/Library/Caches/Homebrew + key: xcode10.3-ccache-${{github.ref}}-${{github.sha}} + restore-keys: | +xcode10.3-ccache-${{github.ref}}- +xcode10.3-ccache-refs/heads/main- + - id: install_dependencies +run: brew install ccache ossp-uuid boost flex openssl doxygen graphviz python lua libusb libpng libpcap xz libssh2 + - id: setup_env +run: | + echo -e "127.0.0.1\t$HOSTNAME" | sudo tee -a /etc/hosts > /dev/null + echo "::set-env name=CMAKE_BUILD_OPTIONS::-DENABLE_PCAP=TRUE -DENABLE_LUA_SCRIPTING=1" + sudo xcode-select -switch /Applications/Xcode_10.3.app + echo "::set-env name=PATH::`brew --prefix`/opt/ccache/libexec:$PATH" + - id: build +run: ./bootstrap.sh -e -t && cd build && cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_RULE_MESSAGES=OFF -DSTRICT_GSL_CHECKS=ON .. && cmake --build . --parallel 4 && sudo make test ARGS="--timeout 300 -j4 --output-on-failure" && make linter + macos_xcode_11_2_1: +name: "macos-xcode11.2.1" +runs-on: macos-10.15 +timeout-minutes: 60 +steps: + - id: checkout +uses: actions/checkout@v2 + - id: cache +uses: actions/cache@v2 +with: + path: | +~/.ccache +~/Library/Caches/Homebrew + key: xcode11.2.1-ccache-${{github.ref}}-${{github.sha}} + restore-keys: | +xcode11.2.1-ccache-${{github.ref}}- +xcode11.2.1-ccache-refs/heads/main- + - id: install_dependencies +run: brew install ccache ossp-uuid boost flex openssl doxygen graphviz python lua libusb libpng libpcap xz libssh2 + - id: setup_env +run: | + echo -e "127.0.0.1\t$HOSTNAME" | sudo tee -a /etc/hosts > /dev/null + echo "::set-env name=CMAKE_BUILD_OPTIONS::-DENABLE_PCAP=TRUE -DENABLE_LUA_SCRIPTING=1" + sudo xcode-select -switch /Applications/Xcode_11.2.1.app + echo "::set-env name=PATH::`brew --prefix`/opt/ccache/libexec:$PATH" + - id: build +run: ./bootstrap.sh -e -t && cd build && cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_RULE_MESSAGES=OFF -DSTRICT_GSL_CHECKS=ON .. && cmake --build . --parallel 4 && sudo make test ARGS="--timeout 300 -j4 --output-on-failure" && make linter + windows: +name: "windows" +runs-on: windows-2016 +timeout-minutes: 90 Review comment: Why is the timeout longer for windows? ## File path: .github/workflows/ci.yml ## @@ -0,0 +1,331 @@ +name: "MiNiFi-CPP CI" +on: [push, pull_request, workflow_dispatch] +jobs: + macos_xcode_10_3: +name: "macos-xcode10.3" +runs-on: macos-10.15 +timeout-minutes: 60 +steps: + - id:
[jira] [Commented] (NIFI-7669) Add flow protection key caching mechanism for derived keys
[ https://issues.apache.org/jira/browse/NIFI-7669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166993#comment-17166993 ] ASF subversion and git services commented on NIFI-7669: --- Commit 716ba992f5b4641c20465638d62d1e3ffe91d118 in nifi's branch refs/heads/main from Andy LoPresto [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=716ba99 ] NIFI-7669 Changed custom PBE AEAD algorithm to derive key once rather than on every encrypt/decrypt operation, leading to substantial performance gains. Updated documentation. Added unit tests. NIFI-7669 Moved time-based encryption tests to integration tests to avoid running during CI builds. NIFI-7669 Fixed failing test due to nifi.properties initialization. Signed-off-by: Pierre Villard This closes #4435. > Add flow protection key caching mechanism for derived keys > -- > > Key: NIFI-7669 > URL: https://issues.apache.org/jira/browse/NIFI-7669 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Core Framework >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Major > Labels: caching, encryption, kdf, performance, security > Time Spent: 10m > Remaining Estimate: 0h > > The specific algorithm introduced in NIFI-7638 introduces a ~1 sec delay in > every encryption operation (which occurs during every flow synchronization > and serialization to disk) due to the Argon2 KDF process. This is an > acceptable tradeoff for security-conscious users at this time, but can be > improved through a key caching mechanism in memory. Deriving the key once at > application startup and using it directly will remove this delay, and the key > cannot change without an application restart. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7669) Add flow protection key caching mechanism for derived keys
[ https://issues.apache.org/jira/browse/NIFI-7669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166992#comment-17166992 ] ASF subversion and git services commented on NIFI-7669: --- Commit 716ba992f5b4641c20465638d62d1e3ffe91d118 in nifi's branch refs/heads/main from Andy LoPresto [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=716ba99 ] NIFI-7669 Changed custom PBE AEAD algorithm to derive key once rather than on every encrypt/decrypt operation, leading to substantial performance gains. Updated documentation. Added unit tests. NIFI-7669 Moved time-based encryption tests to integration tests to avoid running during CI builds. NIFI-7669 Fixed failing test due to nifi.properties initialization. Signed-off-by: Pierre Villard This closes #4435. > Add flow protection key caching mechanism for derived keys > -- > > Key: NIFI-7669 > URL: https://issues.apache.org/jira/browse/NIFI-7669 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Core Framework >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Major > Labels: caching, encryption, kdf, performance, security > Time Spent: 10m > Remaining Estimate: 0h > > The specific algorithm introduced in NIFI-7638 introduces a ~1 sec delay in > every encryption operation (which occurs during every flow synchronization > and serialization to disk) due to the Argon2 KDF process. This is an > acceptable tradeoff for security-conscious users at this time, but can be > improved through a key caching mechanism in memory. Deriving the key once at > application startup and using it directly will remove this delay, and the key > cannot change without an application restart. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on pull request #829: WIP: MINIFICPP-1279 - FlowFile shouldn't inherit from Connectable
hunyadi-dev commented on pull request #829: URL: https://github.com/apache/nifi-minifi-cpp/pull/829#issuecomment-665079380 Minor, but this PR is only named WIP, but is not a draft. Also targets `master`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4434: NIFI-7679 Changed flow fingerprint masking logic to derive a static k…
pvillard31 commented on pull request #4434: URL: https://github.com/apache/nifi/pull/4434#issuecomment-664621928 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4438: NIFI-7340: Adding ListAzureDataLakeStorage
pvillard31 commented on pull request #4438: URL: https://github.com/apache/nifi/pull/4438#issuecomment-665109263 Code change LGTM, left a very minor comment, but I won't have the chance to give it a try before a while. @MuazmaZ - I know you looked at previous ADLS related work, are you able to have a look at it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] jfrazee commented on a change in pull request #4438: NIFI-7340: Adding ListAzureDataLakeStorage
jfrazee commented on a change in pull request #4438: URL: https://github.com/apache/nifi/pull/4438#discussion_r461795133 ## File path: nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/ListAzureDataLakeStorage.java ## @@ -0,0 +1,243 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.azure.storage; + +import com.azure.storage.file.datalake.DataLakeFileSystemClient; +import com.azure.storage.file.datalake.DataLakeServiceClient; +import com.azure.storage.file.datalake.models.ListPathsOptions; +import org.apache.commons.lang3.RegExUtils; +import org.apache.commons.lang3.StringUtils; +import org.apache.commons.lang3.exception.ExceptionUtils; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.InputRequirement.Requirement; +import org.apache.nifi.annotation.behavior.PrimaryNodeOnly; +import org.apache.nifi.annotation.behavior.Stateful; +import org.apache.nifi.annotation.behavior.TriggerSerially; +import org.apache.nifi.annotation.behavior.WritesAttribute; +import org.apache.nifi.annotation.behavior.WritesAttributes; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.SeeAlso; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnStopped; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.components.state.Scope; +import org.apache.nifi.context.PropertyContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.processor.util.list.AbstractListProcessor; +import org.apache.nifi.processors.azure.storage.utils.ADLSFileInfo; +import org.apache.nifi.serialization.record.RecordSchema; + +import java.io.IOException; +import java.util.Arrays; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.regex.Pattern; +import java.util.stream.Collectors; + +import static org.apache.nifi.processor.util.list.ListedEntityTracker.INITIAL_LISTING_TARGET; +import static org.apache.nifi.processor.util.list.ListedEntityTracker.TRACKING_STATE_CACHE; +import static org.apache.nifi.processor.util.list.ListedEntityTracker.TRACKING_TIME_WINDOW; +import static org.apache.nifi.processors.azure.AbstractAzureDataLakeStorageProcessor.ADLS_CREDENTIALS_SERVICE; +import static org.apache.nifi.processors.azure.AbstractAzureDataLakeStorageProcessor.DIRECTORY; +import static org.apache.nifi.processors.azure.AbstractAzureDataLakeStorageProcessor.FILESYSTEM; +import static org.apache.nifi.processors.azure.AbstractAzureDataLakeStorageProcessor.getStorageClient; + +@PrimaryNodeOnly +@TriggerSerially +@Tags({"azure", "microsoft", "cloud", "storage", "adlsgen2", "datalake"}) +@SeeAlso({PutAzureDataLakeStorage.class, DeleteAzureDataLakeStorage.class, FetchAzureDataLakeStorage.class}) +@CapabilityDescription("Lists directory in an Azure Data Lake Storage Gen 2 filesystem") +@WritesAttributes({ +@WritesAttribute(attribute = "azure.filesystem", description = "The name of the Azure File System"), +@WritesAttribute(attribute = "azure.filePath", description = "The full path of the Azure File"), +@WritesAttribute(attribute = "azure.directory", description = "The name of the Azure Directory"), +@WritesAttribute(attribute = "azure.filename", description = "The name of the Azure File"), +@WritesAttribute(attribute = "azure.length", description = "The length of the Azure File"), +@WritesAttribute(attribute = "azure.lastModified", description = "The last modification time of the Azure File"), +@WritesAttribute(attribute = "azure.etag", description = "The ETag of the Azure File") +}) +@InputRequirement(Requirement.INPUT_FORBIDDEN)
[GitHub] [nifi] alopresto opened a new pull request #4434: NIFI-7679 Changed flow fingerprint masking logic to derive a static k…
alopresto opened a new pull request #4434: URL: https://github.com/apache/nifi/pull/4434 …ey once (slowly) from nifi.sensitive.props.key and use a (fast) HMAC/SHA-256 operation during fingerprinting to resolve unacceptable delays. Added unit tests. Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Description of PR _Enables X functionality; fixes bug NIFI-._ In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically `main`)? - [x] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [x] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on JDK 8? - [x] Have you verified that the full build is successful on JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on a change in pull request #4438: NIFI-7340: Adding ListAzureDataLakeStorage
pvillard31 commented on a change in pull request #4438: URL: https://github.com/apache/nifi/pull/4438#discussion_r461579592 ## File path: nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/ListAzureDataLakeStorage.java ## @@ -0,0 +1,243 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.azure.storage; + +import com.azure.storage.file.datalake.DataLakeFileSystemClient; +import com.azure.storage.file.datalake.DataLakeServiceClient; +import com.azure.storage.file.datalake.models.ListPathsOptions; +import org.apache.commons.lang3.RegExUtils; +import org.apache.commons.lang3.StringUtils; +import org.apache.commons.lang3.exception.ExceptionUtils; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.InputRequirement.Requirement; +import org.apache.nifi.annotation.behavior.PrimaryNodeOnly; +import org.apache.nifi.annotation.behavior.Stateful; +import org.apache.nifi.annotation.behavior.TriggerSerially; +import org.apache.nifi.annotation.behavior.WritesAttribute; +import org.apache.nifi.annotation.behavior.WritesAttributes; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.SeeAlso; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnStopped; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.components.state.Scope; +import org.apache.nifi.context.PropertyContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.processor.util.list.AbstractListProcessor; +import org.apache.nifi.processors.azure.storage.utils.ADLSFileInfo; +import org.apache.nifi.serialization.record.RecordSchema; + +import java.io.IOException; +import java.util.Arrays; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.regex.Pattern; +import java.util.stream.Collectors; + +import static org.apache.nifi.processor.util.list.ListedEntityTracker.INITIAL_LISTING_TARGET; +import static org.apache.nifi.processor.util.list.ListedEntityTracker.TRACKING_STATE_CACHE; +import static org.apache.nifi.processor.util.list.ListedEntityTracker.TRACKING_TIME_WINDOW; +import static org.apache.nifi.processors.azure.AbstractAzureDataLakeStorageProcessor.ADLS_CREDENTIALS_SERVICE; +import static org.apache.nifi.processors.azure.AbstractAzureDataLakeStorageProcessor.DIRECTORY; +import static org.apache.nifi.processors.azure.AbstractAzureDataLakeStorageProcessor.FILESYSTEM; +import static org.apache.nifi.processors.azure.AbstractAzureDataLakeStorageProcessor.getStorageClient; + +@PrimaryNodeOnly +@TriggerSerially +@Tags({"azure", "microsoft", "cloud", "storage", "adlsgen2", "datalake"}) +@SeeAlso({PutAzureDataLakeStorage.class, DeleteAzureDataLakeStorage.class, FetchAzureDataLakeStorage.class}) +@CapabilityDescription("Lists directory in an Azure Data Lake Storage Gen 2 filesystem") +@WritesAttributes({ +@WritesAttribute(attribute = "azure.filesystem", description = "The name of the Azure File System"), +@WritesAttribute(attribute = "azure.filePath", description = "The full path of the Azure File"), +@WritesAttribute(attribute = "azure.directory", description = "The name of the Azure Directory"), +@WritesAttribute(attribute = "azure.filename", description = "The name of the Azure File"), +@WritesAttribute(attribute = "azure.length", description = "The length of the Azure File"), +@WritesAttribute(attribute = "azure.lastModified", description = "The last modification time of the Azure File"), +@WritesAttribute(attribute = "azure.etag", description = "The ETag of the Azure File") +}) +@InputRequirement(Requirement.INPUT_FORBIDDEN)
[GitHub] [nifi-minifi-cpp] adamdebreceni closed pull request #844: WIP: Use agent_version.cpp
adamdebreceni closed pull request #844: URL: https://github.com/apache/nifi-minifi-cpp/pull/844 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #4437: NIFI-7682 Check for request object to be null.
asfgit closed pull request #4437: URL: https://github.com/apache/nifi/pull/4437 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on a change in pull request #4436: NIFI-7680 Added convenience methods for creating XML DocumentBuilder …
mattyb149 commented on a change in pull request #4436: URL: https://github.com/apache/nifi/pull/4436#discussion_r461745355 ## File path: nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/NotificationServiceManager.java ## @@ -27,7 +44,9 @@ import org.apache.nifi.components.PropertyValue; import org.apache.nifi.components.ValidationContext; import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.parameter.ParameterLookup; import org.apache.nifi.registry.VariableRegistry; +import org.apache.nifi.security.xml.XmlUtils; Review comment: Now that nifi-bootstrap depends on nifi-security-utils, we'll have to make sure it's in the `lib/bootstrap/` directory in the assembly: nifi-assembly/src/main/assembly/common.xml line 31 insert: `nifi-security-utils` ## File path: nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/NotificationServiceManager.java ## @@ -27,7 +44,9 @@ import org.apache.nifi.components.PropertyValue; import org.apache.nifi.components.ValidationContext; import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.parameter.ParameterLookup; import org.apache.nifi.registry.VariableRegistry; +import org.apache.nifi.security.xml.XmlUtils; Review comment: Looks like there's a better solution by just making it available via nifi-framework-authorization, will try with the latest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #4434: NIFI-7679 Changed flow fingerprint masking logic to derive a static k…
asfgit closed pull request #4434: URL: https://github.com/apache/nifi/pull/4434 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] alopresto opened a new pull request #4436: NIFI-7680 Added convenience methods for creating XML DocumentBuilder …
alopresto opened a new pull request #4436: URL: https://github.com/apache/nifi/pull/4436 …instances. Added unit tests. Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Description of PR _Makes creating `DocumentBuilder` instances easier._ In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically `main`)? - [x] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [x] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on JDK 8? - [x] Have you verified that the full build is successful on JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] alopresto opened a new pull request #4435: NIFI-7679 Changed custom PBE AEAD algorithm to derive key once rather…
alopresto opened a new pull request #4435: URL: https://github.com/apache/nifi/pull/4435 … than on every encrypt/decrypt operation, leading to substantial performance gains. Updated documentation. Added unit tests. Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Description of PR _Improves performance of custom encryption algorithms by deriving key once during initialization rather than on-demand during frequent encryption/decryption operations._ In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically `main`)? - [x] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [x] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on JDK 8? - [ ] Have you verified that the full build is successful on JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] turcsanyip commented on a change in pull request #4438: NIFI-7340: Adding ListAzureDataLakeStorage
turcsanyip commented on a change in pull request #4438: URL: https://github.com/apache/nifi/pull/4438#discussion_r461881207 ## File path: nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/ListAzureDataLakeStorage.java ## @@ -0,0 +1,243 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.azure.storage; + +import com.azure.storage.file.datalake.DataLakeFileSystemClient; +import com.azure.storage.file.datalake.DataLakeServiceClient; +import com.azure.storage.file.datalake.models.ListPathsOptions; +import org.apache.commons.lang3.RegExUtils; +import org.apache.commons.lang3.StringUtils; +import org.apache.commons.lang3.exception.ExceptionUtils; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.InputRequirement.Requirement; +import org.apache.nifi.annotation.behavior.PrimaryNodeOnly; +import org.apache.nifi.annotation.behavior.Stateful; +import org.apache.nifi.annotation.behavior.TriggerSerially; +import org.apache.nifi.annotation.behavior.WritesAttribute; +import org.apache.nifi.annotation.behavior.WritesAttributes; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.SeeAlso; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnStopped; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.components.state.Scope; +import org.apache.nifi.context.PropertyContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.processor.util.list.AbstractListProcessor; +import org.apache.nifi.processors.azure.storage.utils.ADLSFileInfo; +import org.apache.nifi.serialization.record.RecordSchema; + +import java.io.IOException; +import java.util.Arrays; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.regex.Pattern; +import java.util.stream.Collectors; + +import static org.apache.nifi.processor.util.list.ListedEntityTracker.INITIAL_LISTING_TARGET; +import static org.apache.nifi.processor.util.list.ListedEntityTracker.TRACKING_STATE_CACHE; +import static org.apache.nifi.processor.util.list.ListedEntityTracker.TRACKING_TIME_WINDOW; +import static org.apache.nifi.processors.azure.AbstractAzureDataLakeStorageProcessor.ADLS_CREDENTIALS_SERVICE; +import static org.apache.nifi.processors.azure.AbstractAzureDataLakeStorageProcessor.DIRECTORY; +import static org.apache.nifi.processors.azure.AbstractAzureDataLakeStorageProcessor.FILESYSTEM; +import static org.apache.nifi.processors.azure.AbstractAzureDataLakeStorageProcessor.getStorageClient; + +@PrimaryNodeOnly +@TriggerSerially +@Tags({"azure", "microsoft", "cloud", "storage", "adlsgen2", "datalake"}) +@SeeAlso({PutAzureDataLakeStorage.class, DeleteAzureDataLakeStorage.class, FetchAzureDataLakeStorage.class}) +@CapabilityDescription("Lists directory in an Azure Data Lake Storage Gen 2 filesystem") +@WritesAttributes({ +@WritesAttribute(attribute = "azure.filesystem", description = "The name of the Azure File System"), +@WritesAttribute(attribute = "azure.filePath", description = "The full path of the Azure File"), +@WritesAttribute(attribute = "azure.directory", description = "The name of the Azure Directory"), +@WritesAttribute(attribute = "azure.filename", description = "The name of the Azure File"), +@WritesAttribute(attribute = "azure.length", description = "The length of the Azure File"), +@WritesAttribute(attribute = "azure.lastModified", description = "The last modification time of the Azure File"), +@WritesAttribute(attribute = "azure.etag", description = "The ETag of the Azure File") +}) +@InputRequirement(Requirement.INPUT_FORBIDDEN)
[GitHub] [nifi] mattyb149 closed pull request #4436: NIFI-7680 Added convenience methods for creating XML DocumentBuilder …
mattyb149 closed pull request #4436: URL: https://github.com/apache/nifi/pull/4436 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4437: NIFI-7682 Check for request object to be null.
pvillard31 commented on pull request #4437: URL: https://github.com/apache/nifi/pull/4437#issuecomment-665228007 Merged to main, thanks @tirkarthi This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #4436: NIFI-7680 Added convenience methods for creating XML DocumentBuilder …
mattyb149 commented on pull request #4436: URL: https://github.com/apache/nifi/pull/4436#issuecomment-665317911 +1 LGTM, ran a few different scenarios with components that do XML processing. Thanks for the improvement! Merging to master This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda commented on pull request #829: WIP: MINIFICPP-1279 - FlowFile shouldn't inherit from Connectable
arpadboda commented on pull request #829: URL: https://github.com/apache/nifi-minifi-cpp/pull/829#issuecomment-665116201 > Minor, but this PR is only named WIP, but is not a draft. Also targets `master`. Which makes me happy as it surely cannot be merged. :) It is just a POC, will be closed soon, should be properly done in #839 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] energy2522 commented on pull request #4065: NIFI-4239 - Adding CaptureChangePostgreSQL processor to capture data changes (INSERT/UPDATE/DELETE) in PostgreSQL tables via Logical Replica
energy2522 commented on pull request #4065: URL: https://github.com/apache/nifi/pull/4065#issuecomment-664983810 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #4439: NIFI-7683: Fixed bug that resulted in a byte[] being allocated and he…
mattyb149 commented on pull request #4439: URL: https://github.com/apache/nifi/pull/4439#issuecomment-665198735 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] turcsanyip opened a new pull request #4438: NIFI-7340: Adding ListAzureDataLakeStorage
turcsanyip opened a new pull request #4438: URL: https://github.com/apache/nifi/pull/4438 Also added validator for Directory Name property in AbstractAzureDataLakeStorageProcessor https://issues.apache.org/jira/browse/NIFI-7340 Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Description of PR _Enables X functionality; fixes bug NIFI-._ In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically `main`)? - [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on JDK 8? - [ ] Have you verified that the full build is successful on JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] MuazmaZ edited a comment on pull request #4438: NIFI-7340: Adding ListAzureDataLakeStorage
MuazmaZ edited a comment on pull request #4438: URL: https://github.com/apache/nifi/pull/4438#issuecomment-665449257 @turcsanyip @pvillard31 I have not been able to build this PR getting this ERROR: [ERROR] Failed to execute goal pl.project13.maven:git-commit-id-plugin:4.0.0:revision (default) on project nifi-api: .git directory is not found! Please specify a valid [dotGitDirectory] in your pom.xml For Azure bundle I am getting: ..\processors\src\main\java\org\apache\nifi\processors\azure\storage\utils\BlobInfo.java:[121,4] error: method does not override or implement a method from a supertype This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] MuazmaZ commented on pull request #4438: NIFI-7340: Adding ListAzureDataLakeStorage
MuazmaZ commented on pull request #4438: URL: https://github.com/apache/nifi/pull/4438#issuecomment-665186559 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7669) Add flow protection key caching mechanism for derived keys
[ https://issues.apache.org/jira/browse/NIFI-7669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7669: - Fix Version/s: 1.12.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add flow protection key caching mechanism for derived keys > -- > > Key: NIFI-7669 > URL: https://issues.apache.org/jira/browse/NIFI-7669 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Core Framework >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Major > Labels: caching, encryption, kdf, performance, security > Fix For: 1.12.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The specific algorithm introduced in NIFI-7638 introduces a ~1 sec delay in > every encryption operation (which occurs during every flow synchronization > and serialization to disk) due to the Argon2 KDF process. This is an > acceptable tradeoff for security-conscious users at this time, but can be > improved through a key caching mechanism in memory. Deriving the key once at > application startup and using it directly will remove this delay, and the key > cannot change without an application restart. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] joewitt commented on pull request #4435: NIFI-7669 Changed custom PBE AEAD algorithm to derive key once rather…
joewitt commented on pull request #4435: URL: https://github.com/apache/nifi/pull/4435#issuecomment-665127728 @alopresto maybe just convert them to integration tests which we dont run in CI/CD but folks do tend to run for validating RCs? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] alopresto commented on pull request #4435: NIFI-7669 Changed custom PBE AEAD algorithm to derive key once rather…
alopresto commented on pull request #4435: URL: https://github.com/apache/nifi/pull/4435#issuecomment-665125294 It seems the CI/CD machines are overloaded and/or lower performance than many of the developer environments. I'll ignore those tests with a comment that they can be enabled locally to prove the separation of the key derivation from the encrypt/decrypt operation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7669) Add flow protection key caching mechanism for derived keys
[ https://issues.apache.org/jira/browse/NIFI-7669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166991#comment-17166991 ] ASF subversion and git services commented on NIFI-7669: --- Commit 716ba992f5b4641c20465638d62d1e3ffe91d118 in nifi's branch refs/heads/main from Andy LoPresto [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=716ba99 ] NIFI-7669 Changed custom PBE AEAD algorithm to derive key once rather than on every encrypt/decrypt operation, leading to substantial performance gains. Updated documentation. Added unit tests. NIFI-7669 Moved time-based encryption tests to integration tests to avoid running during CI builds. NIFI-7669 Fixed failing test due to nifi.properties initialization. Signed-off-by: Pierre Villard This closes #4435. > Add flow protection key caching mechanism for derived keys > -- > > Key: NIFI-7669 > URL: https://issues.apache.org/jira/browse/NIFI-7669 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Core Framework >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Major > Labels: caching, encryption, kdf, performance, security > Time Spent: 10m > Remaining Estimate: 0h > > The specific algorithm introduced in NIFI-7638 introduces a ~1 sec delay in > every encryption operation (which occurs during every flow synchronization > and serialization to disk) due to the Argon2 KDF process. This is an > acceptable tradeoff for security-conscious users at this time, but can be > improved through a key caching mechanism in memory. Deriving the key once at > application startup and using it directly will remove this delay, and the key > cannot change without an application restart. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #853: MINIFICPP-952 - Use GitHub Actions
adamdebreceni commented on a change in pull request #853: URL: https://github.com/apache/nifi-minifi-cpp/pull/853#discussion_r461486303 ## File path: .github/workflows/ci.yml ## @@ -0,0 +1,331 @@ +name: "MiNiFi-CPP CI" +on: [push, pull_request, workflow_dispatch] +jobs: + macos_xcode_10_3: +name: "macos-xcode10.3" +runs-on: macos-10.15 +timeout-minutes: 60 +steps: + - id: checkout +uses: actions/checkout@v2 + - id: cache +uses: actions/cache@v2 +with: + path: | +~/.ccache +~/Library/Caches/Homebrew + key: xcode10.3-ccache-${{github.ref}}-${{github.sha}} + restore-keys: | +xcode10.3-ccache-${{github.ref}}- +xcode10.3-ccache-refs/heads/main- + - id: install_dependencies +run: brew install ccache ossp-uuid boost flex openssl doxygen graphviz python lua libusb libpng libpcap xz libssh2 + - id: setup_env +run: | + echo -e "127.0.0.1\t$HOSTNAME" | sudo tee -a /etc/hosts > /dev/null + echo "::set-env name=CMAKE_BUILD_OPTIONS::-DENABLE_PCAP=TRUE -DENABLE_LUA_SCRIPTING=1" + sudo xcode-select -switch /Applications/Xcode_10.3.app + echo "::set-env name=PATH::`brew --prefix`/opt/ccache/libexec:$PATH" + - id: build +run: ./bootstrap.sh -e -t && cd build && cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_RULE_MESSAGES=OFF -DSTRICT_GSL_CHECKS=ON .. && cmake --build . --parallel 4 && sudo make test ARGS="--timeout 300 -j4 --output-on-failure" && make linter + macos_xcode_11_2_1: +name: "macos-xcode11.2.1" +runs-on: macos-10.15 +timeout-minutes: 60 +steps: + - id: checkout +uses: actions/checkout@v2 + - id: cache +uses: actions/cache@v2 +with: + path: | +~/.ccache +~/Library/Caches/Homebrew + key: xcode11.2.1-ccache-${{github.ref}}-${{github.sha}} + restore-keys: | +xcode11.2.1-ccache-${{github.ref}}- +xcode11.2.1-ccache-refs/heads/main- + - id: install_dependencies +run: brew install ccache ossp-uuid boost flex openssl doxygen graphviz python lua libusb libpng libpcap xz libssh2 + - id: setup_env +run: | + echo -e "127.0.0.1\t$HOSTNAME" | sudo tee -a /etc/hosts > /dev/null + echo "::set-env name=CMAKE_BUILD_OPTIONS::-DENABLE_PCAP=TRUE -DENABLE_LUA_SCRIPTING=1" + sudo xcode-select -switch /Applications/Xcode_11.2.1.app + echo "::set-env name=PATH::`brew --prefix`/opt/ccache/libexec:$PATH" + - id: build +run: ./bootstrap.sh -e -t && cd build && cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_RULE_MESSAGES=OFF -DSTRICT_GSL_CHECKS=ON .. && cmake --build . --parallel 4 && sudo make test ARGS="--timeout 300 -j4 --output-on-failure" && make linter + windows: +name: "windows" +runs-on: windows-2016 +timeout-minutes: 90 Review comment: because it takes longer to build on windows on the github provided agents ## File path: .github/workflows/ci.yml ## @@ -0,0 +1,331 @@ +name: "MiNiFi-CPP CI" +on: [push, pull_request, workflow_dispatch] +jobs: + macos_xcode_10_3: +name: "macos-xcode10.3" +runs-on: macos-10.15 +timeout-minutes: 60 +steps: + - id: checkout +uses: actions/checkout@v2 + - id: cache +uses: actions/cache@v2 +with: + path: | +~/.ccache +~/Library/Caches/Homebrew + key: xcode10.3-ccache-${{github.ref}}-${{github.sha}} + restore-keys: | +xcode10.3-ccache-${{github.ref}}- +xcode10.3-ccache-refs/heads/main- + - id: install_dependencies +run: brew install ccache ossp-uuid boost flex openssl doxygen graphviz python lua libusb libpng libpcap xz libssh2 + - id: setup_env +run: | + echo -e "127.0.0.1\t$HOSTNAME" | sudo tee -a /etc/hosts > /dev/null + echo "::set-env name=CMAKE_BUILD_OPTIONS::-DENABLE_PCAP=TRUE -DENABLE_LUA_SCRIPTING=1" + sudo xcode-select -switch /Applications/Xcode_10.3.app + echo "::set-env name=PATH::`brew --prefix`/opt/ccache/libexec:$PATH" + - id: build +run: ./bootstrap.sh -e -t && cd build && cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_RULE_MESSAGES=OFF -DSTRICT_GSL_CHECKS=ON .. && cmake --build . --parallel 4 && sudo make test ARGS="--timeout 300 -j4 --output-on-failure" && make linter Review comment: `./bootstrap.sh` does more than just define cmake flags, will remove `sudo` and we'll see if it breaks anything ## File path: extensions/standard-processors/tests/unit/TailFileTests.cpp ## @@ -1521,8 +1521,8 @@ TEST_CASE("TailFile interprets the lookup frequency property correctly", "[multi
[GitHub] [nifi] turcsanyip commented on pull request #4438: NIFI-7340: Adding ListAzureDataLakeStorage
turcsanyip commented on pull request #4438: URL: https://github.com/apache/nifi/pull/4438#issuecomment-665472441 @MuazmaZ Please try `mvn clean install -DskipTests ` in the project root. The BlobInfo error is due to changes in the parent class (`ListableEntity` in `nifi-processor-utils` module). I have no idea about the `git-commit-id-plugin` error. It is a recent change too and a full build might help too. Please first check if `.git` directory exists in your project root (it should if not deleted manually). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #852: MINIFICPP-1307 - Add attribute-based merge test
arpadboda closed pull request #852: URL: https://github.com/apache/nifi-minifi-cpp/pull/852 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on pull request #852: MINIFICPP-1307 - Add attribute-based merge test
adamdebreceni commented on pull request #852: URL: https://github.com/apache/nifi-minifi-cpp/pull/852#issuecomment-664812916 the shortcomings of this test and other MergeFileTests and CompressContentTests will be addressed in [this](https://issues.apache.org/jira/browse/MINIFICPP-1308) ticket This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #853: MINIFICPP-952 - Use GitHub Actions
fgerlits commented on a change in pull request #853: URL: https://github.com/apache/nifi-minifi-cpp/pull/853#discussion_r461633515 ## File path: .github/workflows/ci.yml ## @@ -0,0 +1,329 @@ +name: "MiNiFi-CPP CI" +on: [push, pull_request, workflow_dispatch] +jobs: + macos_xcode_10_3: +name: "macos-xcode10.3" +runs-on: macos-10.15 +timeout-minutes: 60 +steps: + - id: checkout +uses: actions/checkout@v2 + - id: cache +uses: actions/cache@v2 +with: + path: | +~/Library/Caches/Homebrew + key: xcode10.3-ccache-${{github.ref}}-${{github.sha}} + restore-keys: | +xcode10.3-ccache-${{github.ref}}- +xcode10.3-ccache-refs/heads/main- + - id: install_dependencies +run: brew install ccache ossp-uuid boost flex openssl python lua libpcap xz libssh2 + - id: setup_env +run: | + echo -e "127.0.0.1\t$HOSTNAME" | sudo tee -a /etc/hosts > /dev/null + echo "::set-env name=CMAKE_BUILD_OPTIONS::-DENABLE_PCAP=TRUE -DENABLE_LUA_SCRIPTING=1" + sudo xcode-select -switch /Applications/Xcode_10.3.app + echo "::set-env name=PATH::`brew --prefix`/opt/ccache/libexec:$PATH" + - id: build +run: ./bootstrap.sh -e -t && cd build && cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_RULE_MESSAGES=OFF -DSTRICT_GSL_CHECKS=ON .. && cmake --build . --parallel 4 && make test ARGS="--timeout 300 -j4 --output-on-failure" && make linter + macos_xcode_11_2_1: +name: "macos-xcode11.2.1" +runs-on: macos-10.15 +timeout-minutes: 60 +steps: + - id: checkout +uses: actions/checkout@v2 + - id: cache +uses: actions/cache@v2 +with: + path: | +~/Library/Caches/Homebrew + key: xcode11.2.1-ccache-${{github.ref}}-${{github.sha}} + restore-keys: | +xcode11.2.1-ccache-${{github.ref}}- +xcode11.2.1-ccache-refs/heads/main- + - id: install_dependencies +run: brew install ccache ossp-uuid boost flex openssl python lua libpcap xz libssh2 + - id: setup_env +run: | + echo -e "127.0.0.1\t$HOSTNAME" | sudo tee -a /etc/hosts > /dev/null + echo "::set-env name=CMAKE_BUILD_OPTIONS::-DENABLE_PCAP=TRUE -DENABLE_LUA_SCRIPTING=1" + sudo xcode-select -switch /Applications/Xcode_11.2.1.app + echo "::set-env name=PATH::`brew --prefix`/opt/ccache/libexec:$PATH" + - id: build +run: ./bootstrap.sh -e -t && cd build && cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_RULE_MESSAGES=OFF -DSTRICT_GSL_CHECKS=ON .. && cmake --build . --parallel 4 && make test ARGS="--timeout 300 -j4 --output-on-failure" && make linter + windows: +name: "windows" +runs-on: windows-2016 +timeout-minutes: 90 +steps: + - id: checkout +uses: actions/checkout@v2 + - id: build +run: | + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\Tools\;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\;%path% + PATH=C:\Windows\Microsoft.NET\Framework\v4.0.30319;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin;%path% + PATH=C:\Program Files (x86)\Windows Kits\10\bin\x86;%path% + PATH=C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x86;%path% + PATH=C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.6.1 Tools\;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Performance Tools;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\bin\Roslyn;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\VC\VCPackages;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\HostX86\x86;%path% + PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;%path% + PATH=C:\Windows\System32\;%path% + win_build_vs.bat build /CI +shell: cmd + ubuntu_16_04: +name: "ubuntu-16.04" +runs-on: ubuntu-16.04 +timeout-minutes: 60 +steps: + - id: checkout +uses: actions/checkout@v2 + - id: cache +uses: actions/cache@v2 +with: +