[jira] [Updated] (NIFI-12296) Write the data to Apache Doris in a StreamLoad fashion
[ https://issues.apache.org/jira/browse/NIFI-12296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiyang updated NIFI-12296: -- Description: A processor was added to write data to Apache Doris as a StreamLoad. Unlike the others, StreamLoad is a high performance, low latency data import mode, and supports transactions (controlled by labels). This processor solves several problems: 1. The CaptureChangeMySQL processor can be used directly to read the data out, and the data can be written to Doris after simple processing 2. Support large Batch writes (increase Batch Size and Buffer Size) 3. Support dynamic delete operation (if parse to op operation DELETE case) 4. Support the operation of multi-database and multi-table Deficiencies: 1. Do not implement all the test cases (section have been implemented, but CI | CD after the merger will have conflict, there is no solution, need advice or Suggestions) 2. The data reading operation needs to be reviewed or provided by the management (not sure whether it is reasonable) 3. As for the suggestion of using HttpClient part, I may not know much about WebClientServiceProvider and I need some time to understand this implementation was: A processor was added to write data to Apache Doris as a StreamLoad. Unlike the others, StreamLoad is a high performance, low latency data import mode, and supports transactions (controlled by labels). This processor solves several problems: 1. The CaptureChangeMySQL processor can be used directly to read the data out, and the data can be written to Doris after simple processing 2. Support large Batch writes (increase Batch Size and Buffer Size) 3. Support dynamic delete operation (if parse to op operation DELETE case) 4. Support the operation of multi-database and multi-table Deficiencies: 1. Do not implement all the test cases (section have been implemented, but CI | CD after the merger will have conflict, there is no solution, need advice or Suggestions) 2. The data reading operation needs to be reviewed or provided by the management (not sure whether it is reasonable) > Write the data to Apache Doris in a StreamLoad fashion > -- > > Key: NIFI-12296 > URL: https://issues.apache.org/jira/browse/NIFI-12296 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: xiyang >Priority: Minor > Time Spent: 40m > Remaining Estimate: 0h > > A processor was added to write data to Apache Doris as a StreamLoad. Unlike > the others, StreamLoad is a high performance, low latency data import mode, > and supports transactions (controlled by labels). This processor solves > several problems: > 1. The CaptureChangeMySQL processor can be used directly to read the data > out, and the data can be written to Doris after simple processing > 2. Support large Batch writes (increase Batch Size and Buffer Size) > 3. Support dynamic delete operation (if parse to op operation DELETE case) > 4. Support the operation of multi-database and multi-table > Deficiencies: > 1. Do not implement all the test cases (section have been implemented, but CI > | CD after the merger will have conflict, there is no solution, need advice > or Suggestions) > 2. The data reading operation needs to be reviewed or provided by the > management (not sure whether it is reasonable) > 3. As for the suggestion of using HttpClient part, I may not know much about > WebClientServiceProvider and I need some time to understand this > implementation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12296) Write the data to Apache Doris in a StreamLoad fashion
[ https://issues.apache.org/jira/browse/NIFI-12296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiyang updated NIFI-12296: -- Description: A processor was added to write data to Apache Doris as a StreamLoad. Unlike the others, StreamLoad is a high performance, low latency data import mode, and supports transactions (controlled by labels). This processor solves several problems: 1. The CaptureChangeMySQL processor can be used directly to read the data out, and the data can be written to Doris after simple processing 2. Support large Batch writes (increase Batch Size and Buffer Size) 3. Support dynamic delete operation (if parse to op operation DELETE case) 4. Support the operation of multi-database and multi-table Deficiencies: 1. Do not implement all the test cases (section have been implemented, but CI | CD after the merger will have conflict, there is no solution, need advice or Suggestions) 2. The data reading operation needs to be reviewed or provided by the management (not sure whether it is reasonable) was: https://github.com/apache/nifi/pull/7959 As shown in the link, the basic functionality is implemented and some decorative functionality is missing, but I'm not sure if my code is spec compliant or if something is missing. What should I do next to make this work.Looking forward to your reply. best regards > Write the data to Apache Doris in a StreamLoad fashion > -- > > Key: NIFI-12296 > URL: https://issues.apache.org/jira/browse/NIFI-12296 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: xiyang >Priority: Minor > Time Spent: 40m > Remaining Estimate: 0h > > A processor was added to write data to Apache Doris as a StreamLoad. Unlike > the others, StreamLoad is a high performance, low latency data import mode, > and supports transactions (controlled by labels). This processor solves > several problems: > 1. The CaptureChangeMySQL processor can be used directly to read the data > out, and the data can be written to Doris after simple processing > 2. Support large Batch writes (increase Batch Size and Buffer Size) > 3. Support dynamic delete operation (if parse to op operation DELETE case) > 4. Support the operation of multi-database and multi-table > Deficiencies: > 1. Do not implement all the test cases (section have been implemented, but CI > | CD after the merger will have conflict, there is no solution, need advice > or Suggestions) > 2. The data reading operation needs to be reviewed or provided by the > management (not sure whether it is reasonable) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12296 Write the data to Apache Doris in a StreamLoad fashion [nifi]
wanghangyu817 opened a new pull request, #8003: URL: https://github.com/apache/nifi/pull/8003 # Summary [NIFI-12296](https://issues.apache.org/jira/browse/NIFI-12296) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-12296` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-12296` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12339 Fix Property Decryption for Migrated Components [nifi]
exceptionfactory opened a new pull request, #8002: URL: https://github.com/apache/nifi/pull/8002 # Summary [NIFI-12339](https://issues.apache.org/jira/browse/NIFI-12339) Resolves recursive sensitive property encryption issues on the main branch, introduced with changes for handling migration of properties in [NIFI-12301](https://issues.apache.org/jira/browse/NIFI-12301). The problem was the result of passing encrypted property values to migration methods, which subsequently resulted in encrypting a value that was already encrypted. The resolution decrypts original property values prior to registering created extensions so that subsequent property migration methods have access to the decrypted property values. The resolution can be verified using the `InvokeHTTP` Processor and creating a sensitive dynamic property named `Authorization`. After creating the property with an arbitrary value, restarting NiFi maintains the same length for the encrypted string in `flow.json.gz`, as opposed to growing the encrypted string length after each restart operation. New test methods for the Versioned Component Synchronizer also verify that migration methods for Processors receive decrypted values. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-12292) PutElasticsearchHttp exception with JSON strings over 20 MB
[
https://issues.apache.org/jira/browse/NIFI-12292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gregory M. Foreman resolved NIFI-12292.
---
Resolution: Won't Fix
> PutElasticsearchHttp exception with JSON strings over 20 MB
> ---
>
> Key: NIFI-12292
> URL: https://issues.apache.org/jira/browse/NIFI-12292
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.23.2
>Reporter: Gregory M. Foreman
>Priority: Major
>
> PutElasticsearchHttp throws an exception when reading JSON documents that
> contain string fields over 20 MB:
> {code:java}
> PutElasticsearchHttp[id=] Flow file content is not valid JSON, penalizing
> and transferring to failure. {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12292) PutElasticsearchHttp exception with JSON strings over 20 MB
[
https://issues.apache.org/jira/browse/NIFI-12292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784614#comment-17784614
]
Gregory M. Foreman commented on NIFI-12292:
---
Thanks for clarifying. I will close this ticket. Opened NIFI-12343.
> PutElasticsearchHttp exception with JSON strings over 20 MB
> ---
>
> Key: NIFI-12292
> URL: https://issues.apache.org/jira/browse/NIFI-12292
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.23.2
>Reporter: Gregory M. Foreman
>Priority: Major
>
> PutElasticsearchHttp throws an exception when reading JSON documents that
> contain string fields over 20 MB:
> {code:java}
> PutElasticsearchHttp[id=] Flow file content is not valid JSON, penalizing
> and transferring to failure. {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12343) PutElasticsearchJson exception with JSON strings over 20 MB
Gregory M. Foreman created NIFI-12343:
-
Summary: PutElasticsearchJson exception with JSON strings over 20
MB
Key: NIFI-12343
URL: https://issues.apache.org/jira/browse/NIFI-12343
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.23.2
Reporter: Gregory M. Foreman
PutElasticsearchJson throws an exception when reading JSON documents that
contain string fields over 20 MB:
{code:java}
PutElasticsearchJson[id=] No FlowFiles successfully parsed for sending to
Elasticsearch
PutElasticsearchJson[id=] Could not read FlowFile content valid JSON.:
com.fasterxml.jackson.core.exc.StreamConstraintsException: String length
(20050553) exceeds the maximum length (2000)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12339) Sensitive Dynamic Properties not properly decrypted, resulting in wrong property value and ever-growing flow.json.gz
[
https://issues.apache.org/jira/browse/NIFI-12339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-12339:
Fix Version/s: (was: 1.24.0)
> Sensitive Dynamic Properties not properly decrypted, resulting in wrong
> property value and ever-growing flow.json.gz
>
>
> Key: NIFI-12339
> URL: https://issues.apache.org/jira/browse/NIFI-12339
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: David Handermann
>Priority: Blocker
> Fix For: 2.0.0
>
>
> To replication, create an InvokeHTTP Processor. Add a Sensitive Dynamic
> Property named "Authorization" with a value of "Bearer
> fsi8y3ofysp9f8ncp9nupnu8p3s9nu3s9" (it's ok that the value is nonsense).
> Apply the changes.
> Check the flow.json.gz:
> {code:java}
> cat conf/flow.json.gz | gunzip - | jq | grep Authorization{code}
> Restart NiFi.
> The value is no longer correct. And if you run the {{cat}} command above,
> you'll see the value has doubled in length. After restarting several times we
> can see this:
> {code:java}
> nifi-2.0.0-SNAPSHOT $ cat conf/flow.json.gz | gunzip - | jq | grep
> Authorization
> "Authorization":
> "enc{f1f9ba180c6468ff8ce393955034e69383739de54b44ef42b1bf2050c2639e83815d940b8a0cf9f5bc65bdf36f7df59bff9d7e69fa02f0ccc25c8b381684550c8fc6b6a8c570998064ef730f05b0dc}",
> -- restart --nifi-2.0.0-SNAPSHOT $ cat conf/flow.json.gz | gunzip - | jq |
> grep Authorization
> "Authorization":
> "enc{e4455b884d07a7156397d2f60ce3a2f44be909084403f5a84af205bae2af6dbfa2adf47a33d6663799ab523915e9323064554030236b928d5b1684b0a9d635b6589d878b731c35ae1560fbef5627a433b23fb331657e66af355ac356a1c9cd1435c0836a4ecb872966c2852aa3b13e179da1a0f7898c64173b27363458c01dbf7c8595a5dfe9ab798834568c9e0a52fefaf03f6f9d1bdf6ad230fea7cf1e8663a78a6b964d945c729d9ae678e2eaba8910d02373cd9acd08e7a047e0c676ee8a13e9c0}",
> -- restart --nifi-2.0.0-SNAPSHOT $ cat conf/flow.json.gz | gunzip - | jq |
> grep Authorization
> "Authorization":
> "enc{1aeb6970c1ff7f10b88f5b94a2c0cfa70c179638eb976ff7580f5b2546a64b4d96ae834afff9d01cae79c98b9ca4d73af604eab5e95013047e79c152d3e90b3c556e054f9478713eb156da41477d59668902c606f3f300e9804b8a504712822b5f072a5a596c2ba1706520f0163ce8bf0a51dbaf84ee9359c60e55df029dec700725ff1ac599774d4271d5c390ad49d4b350d21bee9f2c235a81f5356d85279db7b4e335bc11fc0d6bf1045a6d2610ff61d8b9da931fc026d356a3d9a9b738312d283c01740757a286e5eb9ad675daa14a391d3df694eaeeb6c66085976a88c86a08052b3eb046e622e5346205bc1e38bfe4aed2ff130595688e4b72d217f29a5c24a28bc06c7bb55e4fd2d25fea15ce523e92b8d721e9a9c0d08ab6d1634cb027658c868feacd89462796b604db7dc55cc2bba7c650f77148bad4ec7328ae8dbeed743420b5b640061f36ed8c8c1db200bbe6a241d6eb370cb024a5881fc734d722e2f1091f1ffa178ad841a4859c9dc734b66a628fbfeb8c3f0a1e5d02e28ce3e2c04737ab5b92d032fafe21ebe5abd542731228b394356bb5b547c68517f972864351022d2ef1118426}",
> -- restart --
> nifi-2.0.0-SNAPSHOT $ cat conf/flow.json.gz | gunzip - | jq | grep
> Authorization
> "Authorization":
>
[jira] [Commented] (NIFI-12342) Remove extraneous 'relativePath' entry in pom.xml for Apache parent
[ https://issues.apache.org/jira/browse/NIFI-12342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784572#comment-17784572 ] Joe Witt commented on NIFI-12342: - This explains what that means: https://robintegg.com/2019/01/20/why-does-spring-initializr-set-the-parent-pom-relativepath-to-empty.html As does this: https://www.baeldung.com/maven-relativepath All happened in this commit during initial nifi release: https://github.com/apache/nifi/commit/a2f63c7c886495ccee52a448e7a9625ad65ad1f9#diff-d9daf395e85220dd6e54928b5d2cec7c7a2dbcd00c231b4128b92e57b52a314a Seems like it is fine to be there. Mostly just an optimization to skip checking locally first. But should not break anything having it though seeing some build issues for this in another env. > Remove extraneous 'relativePath' entry in pom.xml for Apache parent > --- > > Key: NIFI-12342 > URL: https://issues.apache.org/jira/browse/NIFI-12342 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.latest > > > This relativePath entry dates back to 2015. Dont recall why we needed it but > it is problematic now. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12342) Remove extraneous 'relativePath' entry in pom.xml for Apache parent
[ https://issues.apache.org/jira/browse/NIFI-12342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-12342: Fix Version/s: 2.latest (was: 2.0.0) > Remove extraneous 'relativePath' entry in pom.xml for Apache parent > --- > > Key: NIFI-12342 > URL: https://issues.apache.org/jira/browse/NIFI-12342 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.latest > > > This relativePath entry dates back to 2015. Dont recall why we needed it but > it is problematic now. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12342) Remove extraneous 'relativePath' entry in pom.xml for Apache parent
Joe Witt created NIFI-12342: --- Summary: Remove extraneous 'relativePath' entry in pom.xml for Apache parent Key: NIFI-12342 URL: https://issues.apache.org/jira/browse/NIFI-12342 Project: Apache NiFi Issue Type: Task Reporter: Joe Witt Assignee: Joe Witt Fix For: 2.0.0 This relativePath entry dates back to 2015. Dont recall why we needed it but it is problematic now. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12292) PutElasticsearchHttp exception with JSON strings over 20 MB
[
https://issues.apache.org/jira/browse/NIFI-12292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784568#comment-17784568
]
Bryan Bende commented on NIFI-12292:
I don't think PutElasticSearchHttp was addressed since the Jira is still open,
but also the ES HTTP processors are marked as Deprecated on the 1.x line, and
most active development is happening for 2.0.0 on the main line, and on the
main line they have been removed. So most likely no one invested the time since
the processors are going away.
> PutElasticsearchHttp exception with JSON strings over 20 MB
> ---
>
> Key: NIFI-12292
> URL: https://issues.apache.org/jira/browse/NIFI-12292
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.23.2
>Reporter: Gregory M. Foreman
>Priority: Major
>
> PutElasticsearchHttp throws an exception when reading JSON documents that
> contain string fields over 20 MB:
> {code:java}
> PutElasticsearchHttp[id=] Flow file content is not valid JSON, penalizing
> and transferring to failure. {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12292) PutElasticsearchHttp exception with JSON strings over 20 MB
[
https://issues.apache.org/jira/browse/NIFI-12292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784561#comment-17784561
]
Gregory M. Foreman commented on NIFI-12292:
---
[~bbende] do you happen to know if PutElasticsearchHttp (this one) was
addressed?
> PutElasticsearchHttp exception with JSON strings over 20 MB
> ---
>
> Key: NIFI-12292
> URL: https://issues.apache.org/jira/browse/NIFI-12292
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.23.2
>Reporter: Gregory M. Foreman
>Priority: Major
>
> PutElasticsearchHttp throws an exception when reading JSON documents that
> contain string fields over 20 MB:
> {code:java}
> PutElasticsearchHttp[id=] Flow file content is not valid JSON, penalizing
> and transferring to failure. {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12212: Upgrading DynamoDB processors to AWS SDK 2.x [nifi]
emiliosetiadarma commented on PR #7911: URL: https://github.com/apache/nifi/pull/7911#issuecomment-1804253684 I would think so, I don't have write access to this repository so I can't merge it, but it looks good to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12212: Upgrading DynamoDB processors to AWS SDK 2.x [nifi]
gresockj commented on PR #7911: URL: https://github.com/apache/nifi/pull/7911#issuecomment-1804020812 > Changes look good to me and I was able to spin up a simple flow to get, put (both using and not using the records) and delete stuff from DynamoDB. Ok, great! Are we good to merge? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12292) PutElasticsearchHttp exception with JSON strings over 20 MB
[
https://issues.apache.org/jira/browse/NIFI-12292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784505#comment-17784505
]
Bryan Bende commented on NIFI-12292:
This one is specific to the ObjectMapper used to unmarshall the flow.json, so
it does not related to any processors.
I think we already addressed the processor issues that you ran into though..
https://issues.apache.org/jira/browse/NIFI-12127
https://issues.apache.org/jira/browse/NIFI-12117
There might a couple of others that I don't have the links to.
> PutElasticsearchHttp exception with JSON strings over 20 MB
> ---
>
> Key: NIFI-12292
> URL: https://issues.apache.org/jira/browse/NIFI-12292
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.23.2
>Reporter: Gregory M. Foreman
>Priority: Major
>
> PutElasticsearchHttp throws an exception when reading JSON documents that
> contain string fields over 20 MB:
> {code:java}
> PutElasticsearchHttp[id=] Flow file content is not valid JSON, penalizing
> and transferring to failure. {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12333 MiNiFi property override through env and system variables [nifi]
ferencerdei commented on PR #7997: URL: https://github.com/apache/nifi/pull/7997#issuecomment-1804007639 Thanks for the comments, I've applied them. Also needed to rename / create a new minifi-commons-utils module, hope it doesn't make the change too noisy. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12333 MiNiFi property override through env and system variables [nifi]
ferencerdei commented on code in PR #7997:
URL: https://github.com/apache/nifi/pull/7997#discussion_r1388140508
##
minifi/minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-runtime/src/main/java/org/apache/nifi/minifi/util/MiNiFiProperties.java:
##
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.util;
+
+import static
org.apache.nifi.minifi.commons.status.util.PropertyUtil.resolvePropertyValue;
+
+import java.io.BufferedInputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.InputStream;
+import java.util.Optional;
+import java.util.Properties;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+import org.apache.nifi.util.NiFiProperties;
+
+/**
+ * Extends NiFi properties functionality with System and Environment property
override possibility. The property resolution also works with
+ * dots and hyphens that are not supported in some shells.
+ */
+public class MiNiFiProperties extends NiFiProperties {
Review Comment:
renamed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12333 MiNiFi property override through env and system variables [nifi]
ferencerdei commented on code in PR #7997:
URL: https://github.com/apache/nifi/pull/7997#discussion_r1388139084
##
minifi/minifi-commons/minifi-utils/src/main/java/org/apache/nifi/minifi/commons/status/util/PropertyUtil.java:
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.commons.status.util;
+
+import java.util.Map;
+import java.util.Optional;
+import java.util.stream.Stream;
+
+public class PropertyUtil {
+private static final String DOT = ".";
+private static final String UNDERSCORE = "_";
+private static final String HYPHEN = "-";
+
+private PropertyUtil() {
+
+}
+
+public static String resolvePropertyValue(String name, Map
properties) {
+return Optional.of(name)
+.map(n -> getPropertyValue(n, properties))
+.or(() -> Optional.of(name.toUpperCase())
+.filter(uppercasedName -> !name.equals(uppercasedName))
+.map(n -> getPropertyValue(n, properties)))
+.orElse(null);
+}
+
+private static String getPropertyValue(String name, Map properties) {
+return Stream.concat(
+Stream.of(name).filter(properties::containsKey),
+Stream.of(name.replace(DOT, UNDERSCORE), name.replace(HYPHEN,
UNDERSCORE), name.replace(DOT, UNDERSCORE).replace(HYPHEN, UNDERSCORE))
+.distinct()
+.filter(candidate -> !name.equals(candidate) &&
properties.containsKey(candidate)))
Review Comment:
refactored as suggested
##
minifi/minifi-commons/minifi-utils/src/main/java/org/apache/nifi/minifi/commons/status/util/PropertyUtil.java:
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.commons.status.util;
+
+import java.util.Map;
+import java.util.Optional;
+import java.util.stream.Stream;
+
+public class PropertyUtil {
+private static final String DOT = ".";
+private static final String UNDERSCORE = "_";
+private static final String HYPHEN = "-";
+
+private PropertyUtil() {
+
+}
+
+public static String resolvePropertyValue(String name, Map
properties) {
+return Optional.of(name)
+.map(n -> getPropertyValue(n, properties))
+.or(() -> Optional.of(name.toUpperCase())
+.filter(uppercasedName -> !name.equals(uppercasedName))
+.map(n -> getPropertyValue(n, properties)))
+.orElse(null);
+}
+
+private static String getPropertyValue(String name, Map properties) {
+return Stream.concat(
+Stream.of(name).filter(properties::containsKey),
+Stream.of(name.replace(DOT, UNDERSCORE), name.replace(HYPHEN,
UNDERSCORE), name.replace(DOT, UNDERSCORE).replace(HYPHEN, UNDERSCORE))
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12333 MiNiFi property override through env and system variables [nifi]
ferencerdei commented on code in PR #7997:
URL: https://github.com/apache/nifi/pull/7997#discussion_r1388138427
##
minifi/minifi-commons/minifi-utils/src/main/java/org/apache/nifi/minifi/commons/status/util/PropertyUtil.java:
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.commons.status.util;
+
+import java.util.Map;
+import java.util.Optional;
+import java.util.stream.Stream;
+
+public class PropertyUtil {
+private static final String DOT = ".";
+private static final String UNDERSCORE = "_";
+private static final String HYPHEN = "-";
+
+private PropertyUtil() {
+
+}
+
+public static String resolvePropertyValue(String name, Map
properties) {
+return Optional.of(name)
+.map(n -> getPropertyValue(n, properties))
+.or(() -> Optional.of(name.toUpperCase())
Review Comment:
discussed it in a separate channel. As the utility approach does not allow
this easily we leave it as it is.
##
minifi/minifi-commons/minifi-utils/src/main/java/org/apache/nifi/minifi/commons/status/util/PropertyUtil.java:
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.commons.status.util;
+
+import java.util.Map;
+import java.util.Optional;
+import java.util.stream.Stream;
+
+public class PropertyUtil {
+private static final String DOT = ".";
+private static final String UNDERSCORE = "_";
+private static final String HYPHEN = "-";
+
+private PropertyUtil() {
+
+}
+
+public static String resolvePropertyValue(String name, Map
properties) {
+return Optional.of(name)
+.map(n -> getPropertyValue(n, properties))
+.or(() -> Optional.of(name.toUpperCase())
+.filter(uppercasedName -> !name.equals(uppercasedName))
+.map(n -> getPropertyValue(n, properties)))
+.orElse(null);
+}
+
+private static String getPropertyValue(String name, Map properties) {
+return Stream.concat(
+Stream.of(name).filter(properties::containsKey),
+Stream.of(name.replace(DOT, UNDERSCORE), name.replace(HYPHEN,
UNDERSCORE), name.replace(DOT, UNDERSCORE).replace(HYPHEN, UNDERSCORE))
+.distinct()
+.filter(candidate -> !name.equals(candidate) &&
properties.containsKey(candidate)))
+.findFirst()
+.map(properties::get)
+.map(String::valueOf)
+.orElse(null);
Review Comment:
done
##
minifi/minifi-commons/minifi-utils/src/main/java/org/apache/nifi/minifi/commons/status/util/PropertyUtil.java:
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the
Re: [PR] NIFI-12333 MiNiFi property override through env and system variables [nifi]
ferencerdei commented on code in PR #7997:
URL: https://github.com/apache/nifi/pull/7997#discussion_r1388137175
##
minifi/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/service/BootstrapProperties.java:
##
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.bootstrap.service;
+
+import static
org.apache.nifi.minifi.commons.status.util.PropertyUtil.resolvePropertyValue;
+
+import java.util.Optional;
+import java.util.Properties;
+
+/**
+ * Extends Properties functionality with System and Environment property
override possibility. The property resolution also works with
+ * dots and hyphens that are not supported in some shells.
+ */
+public class BootstrapProperties extends Properties {
+
+public BootstrapProperties() {
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12333 MiNiFi property override through env and system variables [nifi]
ferencerdei commented on code in PR #7997:
URL: https://github.com/apache/nifi/pull/7997#discussion_r1388136691
##
minifi/minifi-commons/minifi-utils/src/test/java/org/apache/nifi/minifi/commons/status/util/PropertyUtilTest.java:
##
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.commons.status.util;
+
+import static
org.apache.nifi.minifi.commons.status.util.PropertyUtil.resolvePropertyValue;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNull;
+
+import java.util.Map;
+import org.junit.jupiter.api.Test;
+
+class PropertyUtilTest {
+
+private static final String PROPERTY_KEY_WITH_DOTS = "property.foo.bar";
+private static final String PROPERTY_KEY_WITH_HYPHENS = "property-foo-bar";
+private static final String PROPERTY_KEY_WITH_DOTS_AND_HYPHENS =
"property.foo-bar";
+private static final String PROPERTY_KEY_WITH_UNDERSCORE =
"property_foo_bar";
+private static final String VALUE = "value";
+
+@Test
+void
testResolveParameterValueReturnsNullWhenPropertiesDoesNotContainTheKey() {
+assertNull(resolvePropertyValue(PROPERTY_KEY_WITH_DOTS, Map.of()));
+}
+
+@Test
Review Comment:
Done
##
minifi/minifi-commons/minifi-utils/src/main/java/org/apache/nifi/minifi/commons/status/util/PropertyUtil.java:
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.commons.status.util;
+
+import java.util.Map;
+import java.util.Optional;
+import java.util.stream.Stream;
+
+public class PropertyUtil {
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-12292) PutElasticsearchHttp exception with JSON strings over 20 MB
[
https://issues.apache.org/jira/browse/NIFI-12292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784500#comment-17784500
]
Gregory M. Foreman commented on NIFI-12292:
---
[~bbende] I reviewed:
[NIFI-12323 Configure StreamReadConstraints on the ObjectMapper used b… by
bbende · Pull Request #7985 · apache/nifi ·
GitHub|https://github.com/apache/nifi/pull/7985/files#diff-aa64084868d7f234108c98fa298f7e7980400dc02c4ebe07166c0a7005e5f386R157]
I like the solution. Would support it being applied to the processors impacted
by the Jackson library change.
> PutElasticsearchHttp exception with JSON strings over 20 MB
> ---
>
> Key: NIFI-12292
> URL: https://issues.apache.org/jira/browse/NIFI-12292
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.23.2
>Reporter: Gregory M. Foreman
>Priority: Major
>
> PutElasticsearchHttp throws an exception when reading JSON documents that
> contain string fields over 20 MB:
> {code:java}
> PutElasticsearchHttp[id=] Flow file content is not valid JSON, penalizing
> and transferring to failure. {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-12341) Typo in Expression Language Guide
[
https://issues.apache.org/jira/browse/NIFI-12341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne resolved NIFI-12341.
---
Resolution: Fixed
> Typo in Expression Language Guide
> -
>
> Key: NIFI-12341
> URL: https://issues.apache.org/jira/browse/NIFI-12341
> Project: Apache NiFi
> Issue Type: Bug
> Components: Documentation Website
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Trivial
> Fix For: 2.0.0
>
>
> From Slack:
> {quote}Hello, quick question in the expression language documentation
> ([https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#math])
> there is this example
> {{${literal(64):toDouble():math("cbrt"):toNumber():math("max", 5)}}} but the
> {{toDouble}} doesn't exist in NiFi, shouldn't it be {{toDecimal}} instead? Or
> I'm missing something?{quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12341) Typo in Expression Language Guide
[
https://issues.apache.org/jira/browse/NIFI-12341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784496#comment-17784496
]
ASF subversion and git services commented on NIFI-12341:
Commit 3b6c482db3f53a7d9090ff2e4375e4a8de0643f1 in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3b6c482db3 ]
NIFI-12341: Fixed typo in expression language guide
> Typo in Expression Language Guide
> -
>
> Key: NIFI-12341
> URL: https://issues.apache.org/jira/browse/NIFI-12341
> Project: Apache NiFi
> Issue Type: Bug
> Components: Documentation Website
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Trivial
> Fix For: 2.0.0
>
>
> From Slack:
> {quote}Hello, quick question in the expression language documentation
> ([https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#math])
> there is this example
> {{${literal(64):toDouble():math("cbrt"):toNumber():math("max", 5)}}} but the
> {{toDouble}} doesn't exist in NiFi, shouldn't it be {{toDecimal}} instead? Or
> I'm missing something?{quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12341) Typo in Expression Language Guide
Mark Payne created NIFI-12341:
-
Summary: Typo in Expression Language Guide
Key: NIFI-12341
URL: https://issues.apache.org/jira/browse/NIFI-12341
Project: Apache NiFi
Issue Type: Bug
Components: Documentation Website
Reporter: Mark Payne
Assignee: Mark Payne
Fix For: 2.0.0
>From Slack:
{quote}Hello, quick question in the expression language documentation
([https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#math])
there is this example
{{${literal(64):toDouble():math("cbrt"):toNumber():math("max", 5)}}} but the
{{toDouble}} doesn't exist in NiFi, shouldn't it be {{toDecimal}} instead? Or
I'm missing something?{quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] NIFI-12340: Only call PythonBridge.onProcessorRemoved for python base… [nifi]
markap14 opened a new pull request, #8001: URL: https://github.com/apache/nifi/pull/8001 …d processors, not all processors. Do so in a background virtual thread. Also specified explicit versions for openai dependencies to ensure correctness, since the newly released 1.x is not yet compatible with the latest langchain # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12340) Only call PythonBridge.onProcessorRemoved for Python Processors and do in background
Mark Payne created NIFI-12340: - Summary: Only call PythonBridge.onProcessorRemoved for Python Processors and do in background Key: NIFI-12340 URL: https://issues.apache.org/jira/browse/NIFI-12340 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 2.0.0 Reporter: Mark Payne Assignee: Mark Payne Fix For: 2.0.0 Every time that a Processor is removed, we are calling PythonBridge.onProcessorRemoved(). But we only need to do this for Python based processors. Additionally, we should do this in a background (virtual) thread because it needs to communicate over sockets to the Python process and there's no need to wait on that to complete in order to Reload a processor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-12339) Sensitive Dynamic Properties not properly decrypted, resulting in wrong property value and ever-growing flow.json.gz
[
https://issues.apache.org/jira/browse/NIFI-12339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann reassigned NIFI-12339:
---
Assignee: David Handermann
> Sensitive Dynamic Properties not properly decrypted, resulting in wrong
> property value and ever-growing flow.json.gz
>
>
> Key: NIFI-12339
> URL: https://issues.apache.org/jira/browse/NIFI-12339
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: David Handermann
>Priority: Blocker
> Fix For: 2.0.0, 1.24.0
>
>
> To replication, create an InvokeHTTP Processor. Add a Sensitive Dynamic
> Property named "Authorization" with a value of "Bearer
> fsi8y3ofysp9f8ncp9nupnu8p3s9nu3s9" (it's ok that the value is nonsense).
> Apply the changes.
> Check the flow.json.gz:
> {code:java}
> cat conf/flow.json.gz | gunzip - | jq | grep Authorization{code}
> Restart NiFi.
> The value is no longer correct. And if you run the {{cat}} command above,
> you'll see the value has doubled in length. After restarting several times we
> can see this:
> {code:java}
> nifi-2.0.0-SNAPSHOT $ cat conf/flow.json.gz | gunzip - | jq | grep
> Authorization
> "Authorization":
> "enc{f1f9ba180c6468ff8ce393955034e69383739de54b44ef42b1bf2050c2639e83815d940b8a0cf9f5bc65bdf36f7df59bff9d7e69fa02f0ccc25c8b381684550c8fc6b6a8c570998064ef730f05b0dc}",
> -- restart --nifi-2.0.0-SNAPSHOT $ cat conf/flow.json.gz | gunzip - | jq |
> grep Authorization
> "Authorization":
> "enc{e4455b884d07a7156397d2f60ce3a2f44be909084403f5a84af205bae2af6dbfa2adf47a33d6663799ab523915e9323064554030236b928d5b1684b0a9d635b6589d878b731c35ae1560fbef5627a433b23fb331657e66af355ac356a1c9cd1435c0836a4ecb872966c2852aa3b13e179da1a0f7898c64173b27363458c01dbf7c8595a5dfe9ab798834568c9e0a52fefaf03f6f9d1bdf6ad230fea7cf1e8663a78a6b964d945c729d9ae678e2eaba8910d02373cd9acd08e7a047e0c676ee8a13e9c0}",
> -- restart --nifi-2.0.0-SNAPSHOT $ cat conf/flow.json.gz | gunzip - | jq |
> grep Authorization
> "Authorization":
> "enc{1aeb6970c1ff7f10b88f5b94a2c0cfa70c179638eb976ff7580f5b2546a64b4d96ae834afff9d01cae79c98b9ca4d73af604eab5e95013047e79c152d3e90b3c556e054f9478713eb156da41477d59668902c606f3f300e9804b8a504712822b5f072a5a596c2ba1706520f0163ce8bf0a51dbaf84ee9359c60e55df029dec700725ff1ac599774d4271d5c390ad49d4b350d21bee9f2c235a81f5356d85279db7b4e335bc11fc0d6bf1045a6d2610ff61d8b9da931fc026d356a3d9a9b738312d283c01740757a286e5eb9ad675daa14a391d3df694eaeeb6c66085976a88c86a08052b3eb046e622e5346205bc1e38bfe4aed2ff130595688e4b72d217f29a5c24a28bc06c7bb55e4fd2d25fea15ce523e92b8d721e9a9c0d08ab6d1634cb027658c868feacd89462796b604db7dc55cc2bba7c650f77148bad4ec7328ae8dbeed743420b5b640061f36ed8c8c1db200bbe6a241d6eb370cb024a5881fc734d722e2f1091f1ffa178ad841a4859c9dc734b66a628fbfeb8c3f0a1e5d02e28ce3e2c04737ab5b92d032fafe21ebe5abd542731228b394356bb5b547c68517f972864351022d2ef1118426}",
> -- restart --
> nifi-2.0.0-SNAPSHOT $ cat conf/flow.json.gz | gunzip - | jq | grep
> Authorization
> "Authorization":
>
[jira] [Created] (NIFI-12339) Sensitive Dynamic Properties not properly decrypted, resulting in wrong property value and ever-growing flow.json.gz
Mark Payne created NIFI-12339:
-
Summary: Sensitive Dynamic Properties not properly decrypted,
resulting in wrong property value and ever-growing flow.json.gz
Key: NIFI-12339
URL: https://issues.apache.org/jira/browse/NIFI-12339
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Mark Payne
Fix For: 2.0.0, 1.24.0
To replication, create an InvokeHTTP Processor. Add a Sensitive Dynamic
Property named "Authorization" with a value of "Bearer
fsi8y3ofysp9f8ncp9nupnu8p3s9nu3s9" (it's ok that the value is nonsense). Apply
the changes.
Check the flow.json.gz:
{code:java}
cat conf/flow.json.gz | gunzip - | jq | grep Authorization{code}
Restart NiFi.
The value is no longer correct. And if you run the {{cat}} command above,
you'll see the value has doubled in length. After restarting several times we
can see this:
{code:java}
nifi-2.0.0-SNAPSHOT $ cat conf/flow.json.gz | gunzip - | jq | grep Authorization
"Authorization":
"enc{f1f9ba180c6468ff8ce393955034e69383739de54b44ef42b1bf2050c2639e83815d940b8a0cf9f5bc65bdf36f7df59bff9d7e69fa02f0ccc25c8b381684550c8fc6b6a8c570998064ef730f05b0dc}",
-- restart --nifi-2.0.0-SNAPSHOT $ cat conf/flow.json.gz | gunzip - | jq |
grep Authorization
"Authorization":
"enc{e4455b884d07a7156397d2f60ce3a2f44be909084403f5a84af205bae2af6dbfa2adf47a33d6663799ab523915e9323064554030236b928d5b1684b0a9d635b6589d878b731c35ae1560fbef5627a433b23fb331657e66af355ac356a1c9cd1435c0836a4ecb872966c2852aa3b13e179da1a0f7898c64173b27363458c01dbf7c8595a5dfe9ab798834568c9e0a52fefaf03f6f9d1bdf6ad230fea7cf1e8663a78a6b964d945c729d9ae678e2eaba8910d02373cd9acd08e7a047e0c676ee8a13e9c0}",
-- restart --nifi-2.0.0-SNAPSHOT $ cat conf/flow.json.gz | gunzip - | jq |
grep Authorization
"Authorization":
"enc{1aeb6970c1ff7f10b88f5b94a2c0cfa70c179638eb976ff7580f5b2546a64b4d96ae834afff9d01cae79c98b9ca4d73af604eab5e95013047e79c152d3e90b3c556e054f9478713eb156da41477d59668902c606f3f300e9804b8a504712822b5f072a5a596c2ba1706520f0163ce8bf0a51dbaf84ee9359c60e55df029dec700725ff1ac599774d4271d5c390ad49d4b350d21bee9f2c235a81f5356d85279db7b4e335bc11fc0d6bf1045a6d2610ff61d8b9da931fc026d356a3d9a9b738312d283c01740757a286e5eb9ad675daa14a391d3df694eaeeb6c66085976a88c86a08052b3eb046e622e5346205bc1e38bfe4aed2ff130595688e4b72d217f29a5c24a28bc06c7bb55e4fd2d25fea15ce523e92b8d721e9a9c0d08ab6d1634cb027658c868feacd89462796b604db7dc55cc2bba7c650f77148bad4ec7328ae8dbeed743420b5b640061f36ed8c8c1db200bbe6a241d6eb370cb024a5881fc734d722e2f1091f1ffa178ad841a4859c9dc734b66a628fbfeb8c3f0a1e5d02e28ce3e2c04737ab5b92d032fafe21ebe5abd542731228b394356bb5b547c68517f972864351022d2ef1118426}",
-- restart --
nifi-2.0.0-SNAPSHOT $ cat conf/flow.json.gz | gunzip - | jq | grep Authorization
"Authorization":
"enc{7446029842eae314646fb90250ed4d9e2425bb81694a91408623e2a49f987df74bd8e08e1996b955f70bcc1a961a7356025ee6459bde7d7cec2235e66ff88cb2addb5365c4d46a3df0684d392cd170e2a1105ab6f7e05d03b90f8e6b50fedd904f1043ff13b49817f70b316c89978ca4d4c3a137627728ebd576aa72250d2716915a1ed1aa587ae6dde8a2e560482c3bd33c527904e9a06b92e52e55349bf6907834fc14cc0cbf3c3002def88e1e8b7f05f1302ceece2659fa4f9b4dfdbd7f3baf140a7be105671a17a51e5c15c09a2c233f7255d78ca7761402fbf0053f0e0f108231c1cef427315d5d2fd60f5de78d44c8e6ee159d2fe702a946ab3cf404f96146920e95aedd186a9db85b216eb582d56d899fe7a91df801e95759c05580c263a4870ebe75304fed8484388a7b1706dd6ece7404aae044df57508bb837ac36c9a4118d0efd5b0661ea765485af26f11bf39cc0ee3450dc6533c1adc2205321035107895bc4408183e14b167b2a0a4e3cdf95d1a8742325067c059c96ca21a2ca14a36212f2dd47eef394a7f93db050a14bc561638aa667e01ac1a801de3f9e51ef157eb886a2e99a02834f618dafd56f72a469d4f51a472d0aff632b697696fb50ee47cdc7bb4b0c0dbc4135ca58e8562f3308c0d45b8c983c38967f8cb45b7f82e01d5ce291ca7001150a2e630939db902135cf6249ebfa3319b72b46fcbed82cf30dd3a6a2318a3f4c1cb5f1b90c821b4df1caf5777bc85c6e637dcc9b397aca2f29cc2f6fc6ca227a2a7692a66c88f3e719e0cfe05f6aaef79a99e1336f90c8886c9c30917577a320818f6bd0f790f0f145181bf665232224fbb6094efa3b1782869c714babdc196f2cf253f3f1b112d59f9306c71bd909d7fc3a895739f1731c2831f3bb65d1847583a99fada5f77c00bd433776a6c3edbf837059be26ad5105f175b0e81ea58fc6092ff0593c882c713e6b2e6464cb063848866a594c1e992d5fdf121e4a26e844512d21012bf4fe3dba1cfd3cdd9200ec034301643f79a1eec65b71f4c17d3cccf63c779736fb741dfc61a661c46b63e2bb74209164c6f7b060e311a3c954198b32d19fd7751485b840931dfb913411b4c3b6b3b05e8c886af1518cc41ef31f0f3af7c8e6ef0adcb1dd769657a51c5c52dd495b9f1b3d2f86bb70d86dc94f7bb764eb65fda0c28f0bf7f0f990a1f8b382b37f976184ccbc315bf25ed2eedf5cf75b6d9f07a12d3c5ca04a4c549692ef20}",{code}
Eventually this caused OOME on my NiFi instance. When checking the
{{flow.json.gz}} the file was 70 MB! After removing that property value, the
size shrank to 29 KB.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11671) JoinEnrichment SQL strategy doesn't allow attributes in join statement
[
https://issues.apache.org/jira/browse/NIFI-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784417#comment-17784417
]
Philipp Korniets commented on NIFI-11671:
-
[~markap14] [~jgresock] - this is still a problem - can you please take a look?
> JoinEnrichment SQL strategy doesn't allow attributes in join statement
> --
>
> Key: NIFI-11671
> URL: https://issues.apache.org/jira/browse/NIFI-11671
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.18.0, 1.20.0, 1.23.0
>Reporter: Philipp Korniets
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> We use ForkEnrichement - JoinEnrichment pattern and want to include filtering
> in join SQL. Filter value is coming from FlowFile attribute
> {code:sql}
> ${test} = 'NewValue'
> SELECT original.*, enrichment.*,'${test}'
> FROM original
> LEFT OUTER JOIN enrichment
> ON original.Underlying = enrichment.Underlying
> WHERE enrichment.MyField = '${test}'
> {code}
> However this doesnt work because JoinEnrichment doesnt use
> evaluateAttributeExpressions
> Additionally in version 1.18,1.23 - doesnt allow whole query to be passed as
> attribute.
> !screenshot-1.png|width=692,height=431!
>
> {code:java}
> 2023-06-28 11:07:16,611 ERROR [Timer-Driven Process Thread-7]
> o.a.n.processors.standard.JoinEnrichment
> JoinEnrichment[id=dbe156ac-0187-1000-4477-0183899e0432] Failed to join
> 'original' FlowFile
> StandardFlowFileRecord[uuid=2ab9f6ad-73a5-4763-b25e-fd26c44835e1,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1687948831976-629,
> container=default, section=629], offset=8334082,
> length=600557],offset=0,name=lmr_SY08C41-1_S_514682_20230627.csv,size=600557]
> and 'enrichment' FlowFile
> StandardFlowFileRecord[uuid=e4bb7769-fdce-4dfe-af18-443676103035,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1687949723375-631,
> container=default, section=631], offset=5362822,
> length=1999502],offset=0,name=lmr_SY08C41-1_S_514682_20230627.csv,size=1999502];
> routing to failure
> java.sql.SQLException: Error while preparing statement [${instrumentJoinSQL}]
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:224)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement(CalciteConnectionImpl.java:203)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement(CalciteConnectionImpl.java:99)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareStatement(AvaticaConnection.java:178)
> at
> org.apache.nifi.processors.standard.enrichment.SqlJoinCache.createCalciteParameters(SqlJoinCache.java:91)
> at
> org.apache.nifi.processors.standard.enrichment.SqlJoinCache.getCalciteParameters(SqlJoinCache.java:65)
> at
> org.apache.nifi.processors.standard.enrichment.SqlJoinStrategy.join(SqlJoinStrategy.java:49)
> at
> org.apache.nifi.processors.standard.JoinEnrichment.processBin(JoinEnrichment.java:387)
> at
> org.apache.nifi.processor.util.bin.BinFiles.processBins(BinFiles.java:233)
> at
> org.apache.nifi.processors.standard.JoinEnrichment.processBins(JoinEnrichment.java:503)
> at
> org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:193)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1354)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:246)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: java.lang.RuntimeException: parse failed: Encountered "$" at line
> 1, column 1.
> Was expecting one of:
> "ABS" ...
> {code}
> As I understand issue is in following line of code
>
[jira] [Resolved] (NIFI-12335) MiNiFi startup failure
[
https://issues.apache.org/jira/browse/NIFI-12335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Erdei resolved NIFI-12335.
-
Resolution: Fixed
> MiNiFi startup failure
> --
>
> Key: NIFI-12335
> URL: https://issues.apache.org/jira/browse/NIFI-12335
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Ferenc Erdei
>Assignee: Ferenc Erdei
>Priority: Major
> Labels: minifi-java
> Fix For: 2.0.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If the current flow is empty and I don't set nifi.minifi.security properties,
> then I see the following error in the minifi-app.log and the MiNiFi can not
> start:
> {code:java}
> java.lang.Exception: Unable to load flow due to:
> org.apache.nifi.controller.serialization.FlowSynchronizationException:
> java.lang.NullPointerException: Cannot invoke "java.util.List.iterator()"
> because the return value of
> "org.apache.nifi.controller.flow.VersionedDataflow.getReportingTasks()" is
> null{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12333 MiNiFi property override through env and system variables [nifi]
ferencerdei commented on code in PR #7997:
URL: https://github.com/apache/nifi/pull/7997#discussion_r1387676651
##
minifi/minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-runtime/src/main/java/org/apache/nifi/minifi/util/MiNiFiProperties.java:
##
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.util;
+
+import static
org.apache.nifi.minifi.commons.status.util.PropertyUtil.resolvePropertyValue;
+
+import java.io.BufferedInputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.InputStream;
+import java.util.Optional;
+import java.util.Properties;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+import org.apache.nifi.util.NiFiProperties;
+
+/**
+ * Extends NiFi properties functionality with System and Environment property
override possibility. The property resolution also works with
+ * dots and hyphens that are not supported in some shells.
+ */
+public class MiNiFiProperties extends NiFiProperties {
+
+private final Properties properties = new Properties();
Review Comment:
It must extend the NiFiProperties because this is used inside the NiFi core,
so we use polymorphism there.
I went with the same approach we had in
NifiProperties.createBasicNiFiProperties. I agree it's not ideal, we'd need to
switch to org.apache.nifi.properties.NiFiPropertiesLoader (or an equivalent
MiNiFi implementation). I'm planning to do this in a separate story where I add
property encryption functionality to the bootstrap / minifi properties files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12333 MiNiFi property override through env and system variables [nifi]
bejancsaba commented on code in PR #7997:
URL: https://github.com/apache/nifi/pull/7997#discussion_r1387659456
##
minifi/minifi-commons/minifi-utils/src/main/java/org/apache/nifi/minifi/commons/status/util/PropertyUtil.java:
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.commons.status.util;
+
+import java.util.Map;
+import java.util.Optional;
+import java.util.stream.Stream;
+
+public class PropertyUtil {
+private static final String DOT = ".";
+private static final String UNDERSCORE = "_";
+private static final String HYPHEN = "-";
+
+private PropertyUtil() {
+
+}
+
+public static String resolvePropertyValue(String name, Map
properties) {
+return Optional.of(name)
+.map(n -> getPropertyValue(n, properties))
+.or(() -> Optional.of(name.toUpperCase())
Review Comment:
There is a lot happening here, wouldn't it be cleaner to do uppercase for
both name and the properties keys? I know it would mean extra iteration over
properties but would make the code cleaner in my opinion. What do you think?
##
minifi/minifi-commons/minifi-utils/src/main/java/org/apache/nifi/minifi/commons/status/util/PropertyUtil.java:
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.commons.status.util;
+
+import java.util.Map;
+import java.util.Optional;
+import java.util.stream.Stream;
+
+public class PropertyUtil {
+private static final String DOT = ".";
+private static final String UNDERSCORE = "_";
+private static final String HYPHEN = "-";
+
+private PropertyUtil() {
+
+}
+
+public static String resolvePropertyValue(String name, Map
properties) {
+return Optional.of(name)
+.map(n -> getPropertyValue(n, properties))
+.or(() -> Optional.of(name.toUpperCase())
+.filter(uppercasedName -> !name.equals(uppercasedName))
+.map(n -> getPropertyValue(n, properties)))
+.orElse(null);
+}
+
+private static String getPropertyValue(String name, Map properties) {
+return Stream.concat(
+Stream.of(name).filter(properties::containsKey),
+Stream.of(name.replace(DOT, UNDERSCORE), name.replace(HYPHEN,
UNDERSCORE), name.replace(DOT, UNDERSCORE).replace(HYPHEN, UNDERSCORE))
+.distinct()
+.filter(candidate -> !name.equals(candidate) &&
properties.containsKey(candidate)))
+.findFirst()
+.map(properties::get)
+.map(String::valueOf)
+.orElse(null);
Review Comment:
You could keep it as optional here as well and flatmap on the caller side as
needed.
##
minifi/minifi-commons/minifi-utils/src/main/java/org/apache/nifi/minifi/commons/status/util/PropertyUtil.java:
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by
