[GitHub] nifi pull request #2476: Add feature convertJsonToUpsertSQL

[dogHere]

GitHub user dogHere opened a pull request:

https://github.com/apache/nifi/pull/2476

Add feature convertJsonToUpsertSQL

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dogHere/nifi master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2476.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2476


commit 6f073f4a0d8ed6118d51dec24f0abdb0098b333a
Author: doghere 
Date:   2018-02-12T17:39:27Z

add convertJsonToUpsertSQL

commit c6971f6b6a573b4800b69797354efee22568e93c
Author: doghere 
Date:   2018-02-12T17:43:20Z

Merge branch 'master' of github.com:doghere/nifi

commit 0a1935431c6b218293fb89364c4035672739475b
Author: doghere 
Date:   2018-02-13T05:22:24Z

upsert if target is empty just insert

commit 27c3a5dd7f5d9cceae97064bc02ece85540e5e40
Author: doghere 
Date:   2018-02-16T05:24:02Z

fix putsql conflits

commit a1b836300475765f2285fec78c184959e252744d
Author: doghere 
Date:   2018-02-16T05:25:29Z

fix conflits on convertJSON and JdbcCommon

commit a4f1cb4268fffe8e24a869f3f210401ebd79b0a8
Author: doghere 
Date:   2018-02-16T05:25:38Z

fix conflits on convertJSON and JdbcCommon

commit 32137fe740ac34f2723fd5eb01fc99cec0f35ae1
Author: doghere 
Date:   2018-02-16T05:50:25Z

rebuild convertJson




---

[jira] [Updated] (NIFI-4872) NIFI component high resource usage annotation

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-4872:
--
Status: Patch Available  (was: In Progress)

> NIFI component high resource usage annotation
> -
>
> Key: NIFI-4872
> URL: https://issues.apache.org/jira/browse/NIFI-4872
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework, Core UI
>Affects Versions: 1.5.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Critical
>
> NiFi Processors currently have no means to relay whether or not they have may 
> be resource intensive or not. The idea here would be to introduce an 
> Annotation that can be added to Processors that indicate they may cause high 
> memory, disk, CPU, or network usage. For instance, any Processor that reads 
> the FlowFile contents into memory (like many XML Processors for instance) may 
> cause high memory usage. What ultimately determines if there is high 
> memory/disk/cpu/network usage will depend on the FlowFiles being processed. 
> With many of these components in the dataflow, it increases the risk of 
> OutOfMemoryErrors and performance degradation.
> The annotation should support one value from a fixed list of: CPU, Disk, 
> Memory, Network.  It should also allow the developer to provide a custom 
> description of the scenario that the component would fall under the high 
> usage category.  The annotation should be able to be specified multiple 
> times, for as many resources as it has the potential to be high usage.
> By marking components with this new Annotation, we can update the generated 
> Processor documentation to include this fact.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4882) CSVRecordReader should utilize specified date/time/timestamp format at its convertSimpleIfPossible method

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366542#comment-16366542
 ] 

ASF GitHub Bot commented on NIFI-4882:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2473#discussion_r168657570
  
--- Diff: 
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/AbstractCSVRecordReader.java
 ---
@@ -0,0 +1,110 @@
+package org.apache.nifi.csv;
+
+
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.record.DataType;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.serialization.record.util.DataTypeUtils;
+import org.apache.nifi.serialization.record.RecordFieldType;
+import java.text.DateFormat;
+import java.util.function.Supplier;
+
+abstract public class AbstractCSVRecordReader implements RecordReader {
+
+protected final ComponentLog logger;
+protected final boolean hasHeader;
+protected final boolean ignoreHeader;
+
+protected final Supplier LAZY_DATE_FORMAT;
+protected final Supplier LAZY_TIME_FORMAT;
+protected final Supplier LAZY_TIMESTAMP_FORMAT;
+
+protected final String dateFormat;
+protected final String timeFormat;
+protected final String timestampFormat;
+
+protected final RecordSchema schema;
+
+AbstractCSVRecordReader(final ComponentLog logger, final RecordSchema 
schema, final boolean hasHeader, final boolean ignoreHeader,
+final String dateFormat, final String 
timeFormat, final String timestampFormat) {
+this.logger = logger;
+this.schema = schema;
+this.hasHeader = hasHeader;
+this.ignoreHeader = ignoreHeader;
+this.dateFormat = (dateFormat == null) ? 
RecordFieldType.DATE.getDefaultFormat() : dateFormat;
+this.timeFormat = (timeFormat == null) ? 
RecordFieldType.TIME.getDefaultFormat(): timeFormat;
+this.timestampFormat = (timestampFormat == null) ? 
RecordFieldType.TIMESTAMP.getDefaultFormat(): timestampFormat;
+
+final DateFormat df = (dateFormat == null) ? null : 
DataTypeUtils.getDateFormat(dateFormat);
+final DateFormat tf = (timeFormat == null) ? null : 
DataTypeUtils.getDateFormat(timeFormat);
+final DateFormat tsf = (timestampFormat == null) ? null : 
DataTypeUtils.getDateFormat(timestampFormat);
--- End diff --

I believe these three can omit null check using ternary operator since 
String formats variables will not be null.


> CSVRecordReader should utilize specified date/time/timestamp format at its 
> convertSimpleIfPossible method
> -
>
> Key: NIFI-4882
> URL: https://issues.apache.org/jira/browse/NIFI-4882
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Koji Kawamura
>Assignee: Derek Straka
>Priority: Major
>
> CSVRecordReader.convertSimpleIfPossible method is used by ValidateRecord. The 
> method does not coerce values to the target schema field type if the raw 
> string representation in the input CSV file is not compatible.
> The type compatibility check is implemented as follows. But it does not use 
> user specified date/time/timestamp format:
> {code}
> // This will return 'false' for input '01/01/1900' when user 
> specified custom format 'MM/dd/'
> if (DataTypeUtils.isCompatibleDataType(trimmed, dataType)) {
> // The LAZY_DATE_FORMAT should be used to check 
> compatibility, too.
> return DataTypeUtils.convertType(trimmed, dataType, 
> LAZY_DATE_FORMAT, LAZY_TIME_FORMAT, LAZY_TIMESTAMP_FORMAT, fieldName);
> } else {
> return value;
> }
> {code}
> If input date strings have different format than the default format 
> '-MM-dd', then ValidateRecord processor can not validate input records.
> JacksonCSVRecordReader has the identical methods with CSVRecordReader. Those 
> classes should have an abstract class.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4882) CSVRecordReader should utilize specified date/time/timestamp format at its convertSimpleIfPossible method

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366541#comment-16366541
 ] 

ASF GitHub Bot commented on NIFI-4882:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2473#discussion_r168663334
  
--- Diff: 
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/AbstractCSVRecordReader.java
 ---
@@ -0,0 +1,110 @@
+package org.apache.nifi.csv;
--- End diff --

Please add License header comment.


> CSVRecordReader should utilize specified date/time/timestamp format at its 
> convertSimpleIfPossible method
> -
>
> Key: NIFI-4882
> URL: https://issues.apache.org/jira/browse/NIFI-4882
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Koji Kawamura
>Assignee: Derek Straka
>Priority: Major
>
> CSVRecordReader.convertSimpleIfPossible method is used by ValidateRecord. The 
> method does not coerce values to the target schema field type if the raw 
> string representation in the input CSV file is not compatible.
> The type compatibility check is implemented as follows. But it does not use 
> user specified date/time/timestamp format:
> {code}
> // This will return 'false' for input '01/01/1900' when user 
> specified custom format 'MM/dd/'
> if (DataTypeUtils.isCompatibleDataType(trimmed, dataType)) {
> // The LAZY_DATE_FORMAT should be used to check 
> compatibility, too.
> return DataTypeUtils.convertType(trimmed, dataType, 
> LAZY_DATE_FORMAT, LAZY_TIME_FORMAT, LAZY_TIMESTAMP_FORMAT, fieldName);
> } else {
> return value;
> }
> {code}
> If input date strings have different format than the default format 
> '-MM-dd', then ValidateRecord processor can not validate input records.
> JacksonCSVRecordReader has the identical methods with CSVRecordReader. Those 
> classes should have an abstract class.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2473: NIFI-4882: Resolve issue with parsing custom date, ...

[ijokarumawak]

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2473#discussion_r168663334
  
--- Diff: 
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/AbstractCSVRecordReader.java
 ---
@@ -0,0 +1,110 @@
+package org.apache.nifi.csv;
--- End diff --

Please add License header comment.


---

[GitHub] nifi pull request #2473: NIFI-4882: Resolve issue with parsing custom date, ...

[ijokarumawak]

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2473#discussion_r168657570
  
--- Diff: 
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/AbstractCSVRecordReader.java
 ---
@@ -0,0 +1,110 @@
+package org.apache.nifi.csv;
+
+
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.record.DataType;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.serialization.record.util.DataTypeUtils;
+import org.apache.nifi.serialization.record.RecordFieldType;
+import java.text.DateFormat;
+import java.util.function.Supplier;
+
+abstract public class AbstractCSVRecordReader implements RecordReader {
+
+protected final ComponentLog logger;
+protected final boolean hasHeader;
+protected final boolean ignoreHeader;
+
+protected final Supplier LAZY_DATE_FORMAT;
+protected final Supplier LAZY_TIME_FORMAT;
+protected final Supplier LAZY_TIMESTAMP_FORMAT;
+
+protected final String dateFormat;
+protected final String timeFormat;
+protected final String timestampFormat;
+
+protected final RecordSchema schema;
+
+AbstractCSVRecordReader(final ComponentLog logger, final RecordSchema 
schema, final boolean hasHeader, final boolean ignoreHeader,
+final String dateFormat, final String 
timeFormat, final String timestampFormat) {
+this.logger = logger;
+this.schema = schema;
+this.hasHeader = hasHeader;
+this.ignoreHeader = ignoreHeader;
+this.dateFormat = (dateFormat == null) ? 
RecordFieldType.DATE.getDefaultFormat() : dateFormat;
+this.timeFormat = (timeFormat == null) ? 
RecordFieldType.TIME.getDefaultFormat(): timeFormat;
+this.timestampFormat = (timestampFormat == null) ? 
RecordFieldType.TIMESTAMP.getDefaultFormat(): timestampFormat;
+
+final DateFormat df = (dateFormat == null) ? null : 
DataTypeUtils.getDateFormat(dateFormat);
+final DateFormat tf = (timeFormat == null) ? null : 
DataTypeUtils.getDateFormat(timeFormat);
+final DateFormat tsf = (timestampFormat == null) ? null : 
DataTypeUtils.getDateFormat(timestampFormat);
--- End diff --

I believe these three can omit null check using ternary operator since 
String formats variables will not be null.


---

[jira] [Updated] (NIFI-4882) CSVRecordReader should utilize specified date/time/timestamp format at its convertSimpleIfPossible method

[Koji Kawamura (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Koji Kawamura updated NIFI-4882:

Status: Patch Available  (was: Open)

[~derekstraka] I've added 'contributor' role to your account for NiFi JIRA, and 
assigned you to this.

> CSVRecordReader should utilize specified date/time/timestamp format at its 
> convertSimpleIfPossible method
> -
>
> Key: NIFI-4882
> URL: https://issues.apache.org/jira/browse/NIFI-4882
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Koji Kawamura
>Assignee: Derek Straka
>Priority: Major
>
> CSVRecordReader.convertSimpleIfPossible method is used by ValidateRecord. The 
> method does not coerce values to the target schema field type if the raw 
> string representation in the input CSV file is not compatible.
> The type compatibility check is implemented as follows. But it does not use 
> user specified date/time/timestamp format:
> {code}
> // This will return 'false' for input '01/01/1900' when user 
> specified custom format 'MM/dd/'
> if (DataTypeUtils.isCompatibleDataType(trimmed, dataType)) {
> // The LAZY_DATE_FORMAT should be used to check 
> compatibility, too.
> return DataTypeUtils.convertType(trimmed, dataType, 
> LAZY_DATE_FORMAT, LAZY_TIME_FORMAT, LAZY_TIMESTAMP_FORMAT, fieldName);
> } else {
> return value;
> }
> {code}
> If input date strings have different format than the default format 
> '-MM-dd', then ValidateRecord processor can not validate input records.
> JacksonCSVRecordReader has the identical methods with CSVRecordReader. Those 
> classes should have an abstract class.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (NIFI-4882) CSVRecordReader should utilize specified date/time/timestamp format at its convertSimpleIfPossible method

[Koji Kawamura (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Koji Kawamura reassigned NIFI-4882:
---

Assignee: Derek Straka

> CSVRecordReader should utilize specified date/time/timestamp format at its 
> convertSimpleIfPossible method
> -
>
> Key: NIFI-4882
> URL: https://issues.apache.org/jira/browse/NIFI-4882
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Koji Kawamura
>Assignee: Derek Straka
>Priority: Major
>
> CSVRecordReader.convertSimpleIfPossible method is used by ValidateRecord. The 
> method does not coerce values to the target schema field type if the raw 
> string representation in the input CSV file is not compatible.
> The type compatibility check is implemented as follows. But it does not use 
> user specified date/time/timestamp format:
> {code}
> // This will return 'false' for input '01/01/1900' when user 
> specified custom format 'MM/dd/'
> if (DataTypeUtils.isCompatibleDataType(trimmed, dataType)) {
> // The LAZY_DATE_FORMAT should be used to check 
> compatibility, too.
> return DataTypeUtils.convertType(trimmed, dataType, 
> LAZY_DATE_FORMAT, LAZY_TIME_FORMAT, LAZY_TIMESTAMP_FORMAT, fieldName);
> } else {
> return value;
> }
> {code}
> If input date strings have different format than the default format 
> '-MM-dd', then ValidateRecord processor can not validate input records.
> JacksonCSVRecordReader has the identical methods with CSVRecordReader. Those 
> classes should have an abstract class.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4246) OAuth 2 Authorization support - Client Credentials Grant

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366529#comment-16366529
 ] 

ASF GitHub Bot commented on NIFI-4246:
--

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2085#discussion_r168661430
  
--- Diff: 
nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/AbstractOAuthControllerService.java
 ---
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.oauth;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.Map;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.reporting.InitializationException;
+
+
+public abstract class AbstractOAuthControllerService
+extends AbstractControllerService implements OAuth2ClientService {
+
+protected String accessToken = null;
+protected String refreshToken = null;
+protected String tokenType = null;
+protected long expiresIn = -1;
+protected long expiresTime = -1;
+protected long lastResponseTimestamp = -1;
+protected Map extraHeaders = new HashMap();
+protected String authUrl = null;
+protected long expireTimeSafetyNetSeconds = -1;
+protected String accessTokenRespName = null;
+protected String expireTimeRespName = null;
+protected String expireInRespName = null;
+protected String tokenTypeRespName = null;
+protected String scopeRespName = null;
+
+public static final PropertyDescriptor AUTH_SERVER_URL = new 
PropertyDescriptor
+.Builder().name("OAuth2 Authorization Server URL")
+.displayName("OAuth2 Authorization Server")
+.description("OAuth2 Authorization Server that grants access 
to the protected resources on the behalf of the resource owner.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor 
RESPONSE_ACCESS_TOKEN_FIELD_NAME = new PropertyDescriptor
+.Builder().name("JSON response 'access_token' name")
+.displayName("JSON response 'access_token' name")
+.description("Name of the field in the JSON response that 
contains the access token. IETF OAuth2 spec default is 'access_token' if your 
API provider's" +
+" response field is different this is where you can 
change that.")
+.defaultValue("access_token")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor RESPONSE_EXPIRE_TIME_FIELD_NAME 
= new PropertyDescriptor
+.Builder().name("JSON response 'expire_time' name")
+.displayName("JSON response 'expire_time' name")
+.description("Name of the field in the JSON response that 
contains the expire time. IETF OAuth2 spec default is 'expire_time' if your API 
provider's" +
+" response field is different this is where you can 
change that.")
+.defaultValue("expire_time")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor RESPONSE_EXPIRE_IN_FIELD_NAME = 
new PropertyDescriptor
+.Builder().name("JSON response 'expire_in' name")
+.displayName("JSON response 'expire_in' name")
+.description("Name of the field in the JSON response that 
contains the expire in. IETF OAuth2 spec default is 'expire_in' 

[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2085#discussion_r168661430
  
--- Diff: 
nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/AbstractOAuthControllerService.java
 ---
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.oauth;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.Map;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.reporting.InitializationException;
+
+
+public abstract class AbstractOAuthControllerService
+extends AbstractControllerService implements OAuth2ClientService {
+
+protected String accessToken = null;
+protected String refreshToken = null;
+protected String tokenType = null;
+protected long expiresIn = -1;
+protected long expiresTime = -1;
+protected long lastResponseTimestamp = -1;
+protected Map extraHeaders = new HashMap();
+protected String authUrl = null;
+protected long expireTimeSafetyNetSeconds = -1;
+protected String accessTokenRespName = null;
+protected String expireTimeRespName = null;
+protected String expireInRespName = null;
+protected String tokenTypeRespName = null;
+protected String scopeRespName = null;
+
+public static final PropertyDescriptor AUTH_SERVER_URL = new 
PropertyDescriptor
+.Builder().name("OAuth2 Authorization Server URL")
+.displayName("OAuth2 Authorization Server")
+.description("OAuth2 Authorization Server that grants access 
to the protected resources on the behalf of the resource owner.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor 
RESPONSE_ACCESS_TOKEN_FIELD_NAME = new PropertyDescriptor
+.Builder().name("JSON response 'access_token' name")
+.displayName("JSON response 'access_token' name")
+.description("Name of the field in the JSON response that 
contains the access token. IETF OAuth2 spec default is 'access_token' if your 
API provider's" +
+" response field is different this is where you can 
change that.")
+.defaultValue("access_token")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor RESPONSE_EXPIRE_TIME_FIELD_NAME 
= new PropertyDescriptor
+.Builder().name("JSON response 'expire_time' name")
+.displayName("JSON response 'expire_time' name")
+.description("Name of the field in the JSON response that 
contains the expire time. IETF OAuth2 spec default is 'expire_time' if your API 
provider's" +
+" response field is different this is where you can 
change that.")
+.defaultValue("expire_time")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor RESPONSE_EXPIRE_IN_FIELD_NAME = 
new PropertyDescriptor
+.Builder().name("JSON response 'expire_in' name")
+.displayName("JSON response 'expire_in' name")
+.description("Name of the field in the JSON response that 
contains the expire in. IETF OAuth2 spec default is 'expire_in' if your API 
provider's" +
+" response field is different this is where you can 
change that.")
+.defaultValue("expire_in")
+.required(true)
+

[jira] [Commented] (NIFI-4872) NIFI component high resource usage annotation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366275#comment-16366275
 ] 

ASF GitHub Bot commented on NIFI-4872:
--

GitHub user jtstorck opened a pull request:

https://github.com/apache/nifi/pull/2475

NIFI-4872 Added annotation for specifying scenarios in which components can 
cause high usage of system resources.

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jtstorck/nifi NIFI-4872

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2475.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2475


commit 4906f2d1545f94c1ec264fc0a65615412b81cbe9
Author: Jeff Storck 
Date:   2018-02-15T18:12:49Z

NIFI-4872 Added annotation for specifying scenarios in which components can 
cause high usage of system resources.

commit ec85dadc21c9081297d6dcb1ae0424c33ed6f42b
Author: Jeff Storck 
Date:   2018-02-15T20:03:39Z

NIFI-4872 Initial set of components marked with the 
HighResourceUsageScenario annotation.




> NIFI component high resource usage annotation
> -
>
> Key: NIFI-4872
> URL: https://issues.apache.org/jira/browse/NIFI-4872
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework, Core UI
>Affects Versions: 1.5.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Critical
>
> NiFi Processors currently have no means to relay whether or not they have may 
> be resource intensive or not. The idea here would be to introduce an 
> Annotation that can be added to Processors that indicate they may cause high 
> memory, disk, CPU, or network usage. For instance, any Processor that reads 
> the FlowFile contents into memory (like many XML Processors for instance) may 
> cause high memory usage. What ultimately determines if there is high 
> memory/disk/cpu/network usage will depend on the FlowFiles being processed. 
> With many of these components in the dataflow, it increases the risk of 
> OutOfMemoryErrors and performance degradation.
> The annotation should support one value from a fixed list of: CPU, Disk, 
> Memory, Network.  It should also allow the developer to provide a custom 
> description of the scenario that the component would fall under the high 
> usage category.  The annotation should be able to be specified multiple 
> times, for as many resources as it has the potential to be high usage.
> By marking components with this new Annotation, we can update the generated 
> Processor documentation to include this fact.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2475: NIFI-4872 Added annotation for specifying scenarios...

[jtstorck]

GitHub user jtstorck opened a pull request:

https://github.com/apache/nifi/pull/2475

NIFI-4872 Added annotation for specifying scenarios in which components can 
cause high usage of system resources.

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jtstorck/nifi NIFI-4872

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2475.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2475


commit 4906f2d1545f94c1ec264fc0a65615412b81cbe9
Author: Jeff Storck 
Date:   2018-02-15T18:12:49Z

NIFI-4872 Added annotation for specifying scenarios in which components can 
cause high usage of system resources.

commit ec85dadc21c9081297d6dcb1ae0424c33ed6f42b
Author: Jeff Storck 
Date:   2018-02-15T20:03:39Z

NIFI-4872 Initial set of components marked with the 
HighResourceUsageScenario annotation.




---

[jira] [Commented] (NIFI-4880) Add the ability to utilize aliases in Avro to Avro record conversion

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366190#comment-16366190
 ] 

ASF GitHub Bot commented on NIFI-4880:
--

GitHub user derekstraka opened a pull request:

https://github.com/apache/nifi/pull/2474

NIFI-4880: Add the ability to map record based on the aliases

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [X] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [X] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [X] Is your initial contribution a single, squashed commit?

### For code changes:
- [X] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/derekstraka/nifi NIFI-4880

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2474.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2474


commit 8fa6cfe5729fab36b11fbac9c6ea5f9dd975da0f
Author: Derek Straka 
Date:   2018-02-15T19:56:40Z

NIFI-4880: Add the ability to map record based on the aliases

Signed-off-by: Derek Straka 




> Add the ability to utilize aliases in Avro to Avro record conversion
> 
>
> Key: NIFI-4880
> URL: https://issues.apache.org/jira/browse/NIFI-4880
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.5.0
>Reporter: Derek Straka
>Priority: Minor
>
> Currently, the Avro to Avro conversions will ignore fields that are not 
> mapped verbatim.  In avro schemas, it is possible for fields to be aliased to 
> one another.  It would be useful if a 1:1 mapping was not available, the 
> aliases for the field were searched to locate a prospective value and then 
> add the default.
>  
> The functionality can be accomplished by adding some logic to 
> AvroTypeUtil::createAvroRecord.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2474: NIFI-4880: Add the ability to map record based on t...

[derekstraka]

GitHub user derekstraka opened a pull request:

https://github.com/apache/nifi/pull/2474

NIFI-4880: Add the ability to map record based on the aliases

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [X] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [X] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [X] Is your initial contribution a single, squashed commit?

### For code changes:
- [X] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/derekstraka/nifi NIFI-4880

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2474.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2474


commit 8fa6cfe5729fab36b11fbac9c6ea5f9dd975da0f
Author: Derek Straka 
Date:   2018-02-15T19:56:40Z

NIFI-4880: Add the ability to map record based on the aliases

Signed-off-by: Derek Straka 




---

[GitHub] nifi-minifi-cpp pull request #264: Minificpp 400

[apiri]

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/264#discussion_r168574559
  
--- Diff: debian.sh ---
@@ -0,0 +1,90 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#!/bin/bash
+verify_enable(){
+  feature="$1"
+  feature_status=${!1}
+  if [ "$feature" = "USB_ENABLED" ]; then
+echo "false"
+  else
+echo "true"
+  fi
+}
+add_os_flags() {
+  CMAKE_BUILD_COMMAND="${CMAKE_BUILD_COMMAND} -DFAIL_ON_WARNINGS= "
+}
+bootstrap_cmake(){
+  sudo apt-get -y install cmake
+}
+build_deps(){
+  ## need to account for debian
+  sudo apt-get install libssl1.0-dev > /dev/null
+  RETVAL=$?
+  if ["$RETVAL" -ne "0"]; then  
+ sudo apt-get install libssl-dev > /dev/null
+  fi
+  COMMAND="sudo apt-get -y install cmake gcc g++ zlib1g-dev uuid uuid-dev"
+  export DEBIAN_FRONTEND=noninteractive
+  INSTALLED=()
+  sudo apt-get -y update
+  for option in "${OPTIONS[@]}" ; do
+option_value="${!option}"
+if [ "$option_value" = "${TRUE}" ]; then
+  # option is enabled
+  FOUND_VALUE=""
+  for cmake_opt in "${DEPENDENCIES[@]}" ; do
+KEY=${cmake_opt%%:*}
+VALUE=${cmake_opt#*:}
+if [ "$KEY" = "$option" ]; then
+  FOUND_VALUE="$VALUE"
+  if [ "$FOUND_VALUE" = "libcurl" ]; then
--- End diff --

we also need to grab bz2-devel.  without that I was getting a segfault on 
CompressContent.  installing that and rebuilding and everything looked good


---

[jira] [Commented] (NIFI-2630) Allow PublishJMS processor to create TextMessages

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366102#comment-16366102
 ] 

ASF GitHub Bot commented on NIFI-2630:
--

Github user mosermw commented on the issue:

https://github.com/apache/nifi/pull/2458
  
Added a third commit to improve documentation.


> Allow PublishJMS processor to create TextMessages
> -
>
> Key: NIFI-2630
> URL: https://issues.apache.org/jira/browse/NIFI-2630
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 0.6.0
>Reporter: James Anderson
>Assignee: Michael Moser
>Priority: Minor
>  Labels: patch
> Attachments: 
> 0001-NIFI-2630-Allow-PublishJMS-processor-to-create-TextM.patch
>
>
> Create a new configuration option for PublishJMS that allows the processor to 
> be configured to emit instances of TextMessages as well as BytesMessage.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2458: NIFI-2630 Allow PublishJMS to send TextMessages

[mosermw]

Github user mosermw commented on the issue:

https://github.com/apache/nifi/pull/2458
  
Added a third commit to improve documentation.


---

[jira] [Commented] (NIFI-4882) CSVRecordReader should utilize specified date/time/timestamp format at its convertSimpleIfPossible method

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366077#comment-16366077
 ] 

ASF GitHub Bot commented on NIFI-4882:
--

Github user derekstraka commented on the issue:

https://github.com/apache/nifi/pull/2473
  
/cc @ijokarumawak


> CSVRecordReader should utilize specified date/time/timestamp format at its 
> convertSimpleIfPossible method
> -
>
> Key: NIFI-4882
> URL: https://issues.apache.org/jira/browse/NIFI-4882
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Koji Kawamura
>Priority: Major
>
> CSVRecordReader.convertSimpleIfPossible method is used by ValidateRecord. The 
> method does not coerce values to the target schema field type if the raw 
> string representation in the input CSV file is not compatible.
> The type compatibility check is implemented as follows. But it does not use 
> user specified date/time/timestamp format:
> {code}
> // This will return 'false' for input '01/01/1900' when user 
> specified custom format 'MM/dd/'
> if (DataTypeUtils.isCompatibleDataType(trimmed, dataType)) {
> // The LAZY_DATE_FORMAT should be used to check 
> compatibility, too.
> return DataTypeUtils.convertType(trimmed, dataType, 
> LAZY_DATE_FORMAT, LAZY_TIME_FORMAT, LAZY_TIMESTAMP_FORMAT, fieldName);
> } else {
> return value;
> }
> {code}
> If input date strings have different format than the default format 
> '-MM-dd', then ValidateRecord processor can not validate input records.
> JacksonCSVRecordReader has the identical methods with CSVRecordReader. Those 
> classes should have an abstract class.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2473: NIFI-4882: Resolve issue with parsing custom date, time, a...

[derekstraka]

Github user derekstraka commented on the issue:

https://github.com/apache/nifi/pull/2473
  
/cc @ijokarumawak


---

[jira] [Commented] (NIFI-4864) Additional Resources property pointing at a directory won't find new JARs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366073#comment-16366073
 ] 

ASF GitHub Bot commented on NIFI-4864:
--

Github user bbende commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2470#discussion_r168568930
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java
 ---
@@ -3447,11 +3448,32 @@ public void startReportingTask(final 
ReportingTaskNode reportingTaskNode) {
 if (isTerminated()) {
 throw new IllegalStateException("Cannot start reporting task " 
+ reportingTaskNode.getIdentifier() + " because the controller is terminated");
 }
-
+validateAdditionalResourcesFingerprint(reportingTaskNode);
 reportingTaskNode.verifyCanStart();
 processScheduler.schedule(reportingTaskNode);
 }
 
+private void validateAdditionalResourcesFingerprint(ReportingTaskNode 
reportingTaskNode){
--- End diff --

It seems like this method is very similar to the code needed in the 
controller service provider and the standard process group... since all three 
node types extend from AbstractConfiguredComponent, would it be possible to 
create a single method there like `void reloadIfNecessary()` (or some better 
name) ? 

The logic for calculating the fingerprint, comparing to the existing one, 
and reloading if different should all be able to implemented generically for 
any AbstractConfiguredComponent since the existing fingerprint is already 
stored there.


> Additional Resources property pointing at a directory won't find new JARs
> -
>
> Key: NIFI-4864
> URL: https://issues.apache.org/jira/browse/NIFI-4864
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0
>Reporter: Bryan Bende
>Assignee: zenfenan
>Priority: Minor
>
> If you have a processor/Controller Service/Reporting Task that has a property 
> with dynamicallyModifiesClasspath(true) and you set the value to a directory, 
> the resources in that directory will only be calculated when that property 
> changes. This means if you added JARs to the directory later, and stopped and 
> started your processor, those new JARs still won't be available. You would 
> have to change the property to a new directory, or back and forth to some 
> other directory, to force a recalculation.
> The setProperties method in AbstractConfiguredComponent is where it looks at 
> incoming property changes and determines if any were for classpath related 
> properties and then calls reload accordingly.
> We would need to consider the case where setProperties is never even being 
> called, someone just stops and starts the processor and would want to pick up 
> any new JARs added.
> A possible solution might be to computer some kind of hash/fingerprint of the 
> URLs each time reload is called, and then when starting the processor we 
> could recompute the fingerprint and compare it to the previous one. If they 
> are different then we call reload before starting the component.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2470: NIFI-4864 Fixing additional resources property poin...

[bbende]

Github user bbende commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2470#discussion_r168568930
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java
 ---
@@ -3447,11 +3448,32 @@ public void startReportingTask(final 
ReportingTaskNode reportingTaskNode) {
 if (isTerminated()) {
 throw new IllegalStateException("Cannot start reporting task " 
+ reportingTaskNode.getIdentifier() + " because the controller is terminated");
 }
-
+validateAdditionalResourcesFingerprint(reportingTaskNode);
 reportingTaskNode.verifyCanStart();
 processScheduler.schedule(reportingTaskNode);
 }
 
+private void validateAdditionalResourcesFingerprint(ReportingTaskNode 
reportingTaskNode){
--- End diff --

It seems like this method is very similar to the code needed in the 
controller service provider and the standard process group... since all three 
node types extend from AbstractConfiguredComponent, would it be possible to 
create a single method there like `void reloadIfNecessary()` (or some better 
name) ? 

The logic for calculating the fingerprint, comparing to the existing one, 
and reloading if different should all be able to implemented generically for 
any AbstractConfiguredComponent since the existing fingerprint is already 
stored there.


---

[jira] [Commented] (NIFI-4864) Additional Resources property pointing at a directory won't find new JARs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366050#comment-16366050
 ] 

ASF GitHub Bot commented on NIFI-4864:
--

Github user derekstraka commented on the issue:

https://github.com/apache/nifi/pull/2470
  
/cc ijokarumawak 


> Additional Resources property pointing at a directory won't find new JARs
> -
>
> Key: NIFI-4864
> URL: https://issues.apache.org/jira/browse/NIFI-4864
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0
>Reporter: Bryan Bende
>Assignee: zenfenan
>Priority: Minor
>
> If you have a processor/Controller Service/Reporting Task that has a property 
> with dynamicallyModifiesClasspath(true) and you set the value to a directory, 
> the resources in that directory will only be calculated when that property 
> changes. This means if you added JARs to the directory later, and stopped and 
> started your processor, those new JARs still won't be available. You would 
> have to change the property to a new directory, or back and forth to some 
> other directory, to force a recalculation.
> The setProperties method in AbstractConfiguredComponent is where it looks at 
> incoming property changes and determines if any were for classpath related 
> properties and then calls reload accordingly.
> We would need to consider the case where setProperties is never even being 
> called, someone just stops and starts the processor and would want to pick up 
> any new JARs added.
> A possible solution might be to computer some kind of hash/fingerprint of the 
> URLs each time reload is called, and then when starting the processor we 
> could recompute the fingerprint and compare it to the previous one. If they 
> are different then we call reload before starting the component.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2470: NIFI-4864 Fixing additional resources property pointing to...

[derekstraka]

Github user derekstraka commented on the issue:

https://github.com/apache/nifi/pull/2470
  
/cc ijokarumawak 


---

[jira] [Commented] (NIFI-4882) CSVRecordReader should utilize specified date/time/timestamp format at its convertSimpleIfPossible method

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366047#comment-16366047
 ] 

ASF GitHub Bot commented on NIFI-4882:
--

GitHub user derekstraka opened a pull request:

https://github.com/apache/nifi/pull/2473

NIFI-4882: Resolve issue with parsing custom date, time, and timestamp 
formats in CSV files

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [X] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [X] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [X] Is your initial contribution a single, squashed commit?

### For code changes:
- [X] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [X] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/derekstraka/nifi NIFI-4882

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2473.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2473


commit aa5ee01944eca25b2850d7a00fed8beea7d306c5
Author: Derek Straka 
Date:   2018-02-15T18:05:19Z

NIFI-4882: Resolve issue with parsing custom date, time, and timestamp 
formats in CSV files

Refactor the two existing CSV classes to inherit from an abstract base 
class that parses the data
Add unit tests for calls to nextRecord that do not coerce types

Signed-off-by: Derek Straka 




> CSVRecordReader should utilize specified date/time/timestamp format at its 
> convertSimpleIfPossible method
> -
>
> Key: NIFI-4882
> URL: https://issues.apache.org/jira/browse/NIFI-4882
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Koji Kawamura
>Priority: Major
>
> CSVRecordReader.convertSimpleIfPossible method is used by ValidateRecord. The 
> method does not coerce values to the target schema field type if the raw 
> string representation in the input CSV file is not compatible.
> The type compatibility check is implemented as follows. But it does not use 
> user specified date/time/timestamp format:
> {code}
> // This will return 'false' for input '01/01/1900' when user 
> specified custom format 'MM/dd/'
> if (DataTypeUtils.isCompatibleDataType(trimmed, dataType)) {
> // The LAZY_DATE_FORMAT should be used to check 
> compatibility, too.
> return DataTypeUtils.convertType(trimmed, dataType, 
> LAZY_DATE_FORMAT, LAZY_TIME_FORMAT, LAZY_TIMESTAMP_FORMAT, fieldName);
> } else {
> return value;
> }
> {code}
> If input date strings have different format than the default format 
> '-MM-dd', then ValidateRecord processor can not validate input records.
> JacksonCSVRecordReader has the identical methods with CSVRecordReader. Those 
> classes should have an abstract class.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2473: NIFI-4882: Resolve issue with parsing custom date, ...

[derekstraka]

GitHub user derekstraka opened a pull request:

https://github.com/apache/nifi/pull/2473

NIFI-4882: Resolve issue with parsing custom date, time, and timestamp 
formats in CSV files

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [X] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [X] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [X] Is your initial contribution a single, squashed commit?

### For code changes:
- [X] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [X] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/derekstraka/nifi NIFI-4882

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2473.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2473


commit aa5ee01944eca25b2850d7a00fed8beea7d306c5
Author: Derek Straka 
Date:   2018-02-15T18:05:19Z

NIFI-4882: Resolve issue with parsing custom date, time, and timestamp 
formats in CSV files

Refactor the two existing CSV classes to inherit from an abstract base 
class that parses the data
Add unit tests for calls to nextRecord that do not coerce types

Signed-off-by: Derek Straka 




---

[jira] [Commented] (NIFI-3502) Upgrade D3 to the latest 4.x version

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366043#comment-16366043
 ] 

ASF GitHub Bot commented on NIFI-3502:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/2471
  
Just pushed a new commit that addresses @andrewmlim findings.


> Upgrade D3 to the latest 4.x version
> 
>
> Key: NIFI-3502
> URL: https://issues.apache.org/jira/browse/NIFI-3502
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Scott Aslan
>Assignee: Matt Gilman
>Priority: Major
>
> The NIFI canvas web application is using version 3.x of the D3 library which 
> is a major version behind and is due to be upgraded. This will be a bit of an 
> effort as the API's have changed in 4.x.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2471: NIFI-3502: Upgrading D3 version

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/2471
  
Just pushed a new commit that addresses @andrewmlim findings.


---

[jira] [Commented] (NIFI-4884) Failed to import flow from registry when processor has CRON schedule

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366004#comment-16366004
 ] 

ASF GitHub Bot commented on NIFI-4884:
--

GitHub user bbende opened a pull request:

https://github.com/apache/nifi/pull/2472

NIFI-4884 Fixing ordering during import from registry so that we set …

…schedule strategy on a processor before setting scheduling duration

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bbende/nifi NIFI-4884

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2472.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2472


commit 15ac547d6b7d1d2541456b0c380b38b731f29c4b
Author: Bryan Bende 
Date:   2018-02-15T17:42:47Z

NIFI-4884 Fixing ordering during import from registry so that we set 
schedule strategy on a processor before setting scheduling duration




> Failed to import flow from registry when processor has CRON schedule
> 
>
> Key: NIFI-4884
> URL: https://issues.apache.org/jira/browse/NIFI-4884
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.5.0
>Reporter: Bryan Bende
>Assignee: Bryan Bende
>Priority: Major
>
> Steps to reproduce:
>  * Make a new process group
>  * Go into new PG
>  * Create a GenerateFlowFile processor
>  * Set scheduling strategy to CRON
>  * Start Version Control
>  * Create a new PG by importing V1 that was saved in previous step
> This will produce a modal pop up saying that the cron expression is not a 
> valid scheduling strategy and will result in an empty PG.
> Tested on a large PG with 50+ processors where only one had CRON, and still 
> results in empty PG with nothing imported.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2472: NIFI-4884 Fixing ordering during import from regist...

[bbende]

GitHub user bbende opened a pull request:

https://github.com/apache/nifi/pull/2472

NIFI-4884 Fixing ordering during import from registry so that we set …

…schedule strategy on a processor before setting scheduling duration

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bbende/nifi NIFI-4884

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2472.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2472


commit 15ac547d6b7d1d2541456b0c380b38b731f29c4b
Author: Bryan Bende 
Date:   2018-02-15T17:42:47Z

NIFI-4884 Fixing ordering during import from registry so that we set 
schedule strategy on a processor before setting scheduling duration




---

[jira] [Commented] (NIFI-4884) Failed to import flow from registry when processor has CRON schedule

[Bryan Bende (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365990#comment-16365990
 ] 

Bryan Bende commented on NIFI-4884:
---

I believe the issue is that when we apply values from the versioned processor 
to the actual processor, we are setting the schedule duration before the 
schedule strategy.

Testing a change and will submit a PR if it works.

> Failed to import flow from registry when processor has CRON schedule
> 
>
> Key: NIFI-4884
> URL: https://issues.apache.org/jira/browse/NIFI-4884
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.5.0
>Reporter: Bryan Bende
>Assignee: Bryan Bende
>Priority: Major
>
> Steps to reproduce:
>  * Make a new process group
>  * Go into new PG
>  * Create a GenerateFlowFile processor
>  * Set scheduling strategy to CRON
>  * Start Version Control
>  * Create a new PG by importing V1 that was saved in previous step
> This will produce a modal pop up saying that the cron expression is not a 
> valid scheduling strategy and will result in an empty PG.
> Tested on a large PG with 50+ processors where only one had CRON, and still 
> results in empty PG with nothing imported.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (NIFI-4884) Failed to import flow from registry when processor has CRON schedule

[Bryan Bende (JIRA)]

Bryan Bende created NIFI-4884:
-

 Summary: Failed to import flow from registry when processor has 
CRON schedule
 Key: NIFI-4884
 URL: https://issues.apache.org/jira/browse/NIFI-4884
 Project: Apache NiFi
  Issue Type: Bug
Affects Versions: 1.5.0
Reporter: Bryan Bende
Assignee: Bryan Bende


Steps to reproduce:
 * Make a new process group
 * Go into new PG
 * Create a GenerateFlowFile processor
 * Set scheduling strategy to CRON
 * Start Version Control
 * Create a new PG by importing V1 that was saved in previous step

This will produce a modal pop up saying that the cron expression is not a valid 
scheduling strategy and will result in an empty PG.

Tested on a large PG with 50+ processors where only one had CRON, and still 
results in empty PG with nothing imported.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (NIFI-4857) Record components do not support String <-> byte[] conversions

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman reassigned NIFI-4857:
-

Assignee: Matt Burgess

> Record components do not support String <-> byte[] conversions
> --
>
> Key: NIFI-4857
> URL: https://issues.apache.org/jira/browse/NIFI-4857
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
>
> When trying to perform a conversion of a field between a String and a byte 
> array, various errors are reporting (depending on where the conversion is 
> taking place). Here are some examples:
> 1) CSVReader, if a column with String values is specified in the schema as 
> "bytes"
> 2) ConvertRecord, if an input field is of type String and the output field is 
> of type "bytes"
> 3) ConvertRecord, if an input field is of type "bytes" and the output field 
> is of type "String"
> Many/most/all of the record components use utility methods to convert values, 
> I believe these methods need to be updated to support conversion between 
> String and byte[] values.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (NIFIREG-145) nifi-registry.sh is not strictly sh compatible

[Kevin Doran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFIREG-145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Doran resolved NIFIREG-145.
-
   Resolution: Fixed
Fix Version/s: 0.2.0

> nifi-registry.sh is not strictly sh compatible
> --
>
> Key: NIFIREG-145
> URL: https://issues.apache.org/jira/browse/NIFIREG-145
> Project: NiFi Registry
>  Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: Kevin Doran
>Priority: Minor
> Fix For: 0.2.0
>
>
> {{nifi-registry.sh}} uses some the {{bash}}-specific commands/functions that 
> are not part of {{sh}}.
> For example, [the use of {{source}} on line 
> 88|https://github.com/apache/nifi-registry/blob/rel/nifi-registry-0.1.0/nifi-registry-resources/src/main/resources/bin/nifi-registry.sh#L88],
>  which [does not exist in 
> {{sh}}|https://stackoverflow.com/questions/13702425/source-command-not-found-in-sh-shell/13702462].
> On platforms where the system shell ({{sh}}) symlinks to something other than 
> {{bash}} (apparently it points to {{dash}} on some Debian/Ubuntu 
> distributions), this script can fail with the error message:
> {noformat}
> sh: source: not found
> {noformat}
> We should change the script to be strictly {{sh}}-compatible for portability 
> or change the specified shell on the shebang line to {{#!/bin/bash}}
> 
> As a workaround for those encountering this issue in nifi-registry-0.1.0:
>  * change line 1 of nifi-registry.sh to {{#!/bin/bash}}; or
>  * change line 88 of nifi-registry.sh to use {{.}} instead of {{source}}; or
>  * leave nifi-registry.sh unmodified, but execute it using bash, ie:
> {noformat}
> bash -e /path/to/nifi-registry/bin/nifi-registry.sh [run|start|stop|...]
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-registry pull request #102: "source" doesn't exist in regular /bin/sh

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-registry/pull/102


---

[jira] [Updated] (NIFIREG-145) nifi-registry.sh is not strictly sh compatible

[Kevin Doran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFIREG-145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Doran updated NIFIREG-145:

Description: 
{{nifi-registry.sh}} uses some the {{bash}}-specific commands/functions that 
are not part of {{sh}}.

For example, [the use of {{source}} on line 
88|https://github.com/apache/nifi-registry/blob/rel/nifi-registry-0.1.0/nifi-registry-resources/src/main/resources/bin/nifi-registry.sh#L88],
 which [does not exist in 
{{sh}}|https://stackoverflow.com/questions/13702425/source-command-not-found-in-sh-shell/13702462].

On platforms where the system shell ({{sh}}) symlinks to something other than 
{{bash}} (apparently it points to {{dash}} on some Debian/Ubuntu 
distributions), this script can fail with the error message:
{noformat}
sh: source: not found
{noformat}
We should change the script to be strictly {{sh}}-compatible for portability or 
change the specified shell on the shebang line to {{#!/bin/bash}}



As a workaround for those encountering this issue in nifi-registry-0.1.0:
 * change line 1 of nifi-registry.sh to {{#!/bin/bash}}; or
 * change line 88 of nifi-registry.sh to use {{.}} instead of {{source}}; or
 * leave nifi-registry.sh unmodified, but execute it using bash, ie:

{noformat}
bash -e /path/to/nifi-registry/bin/nifi-registry.sh [run|start|stop|...]
{noformat}


  was:
{{nifi-registry.sh}} uses some the {{bash}}-specific commands/functions that 
are not part of {{sh}}.

For example, [the use of {{source}} on line 
88|https://github.com/apache/nifi-registry/blob/rel/nifi-registry-0.1.0/nifi-registry-resources/src/main/resources/bin/nifi-registry.sh#L88],
 which [does not exist in 
{{sh}}|https://stackoverflow.com/questions/13702425/source-command-not-found-in-sh-shell/13702462].

On platforms where the system shell ({{sh}}) symlinks to something other than 
{{bash}} (apparently it points to {{dash}} on some Debian/Ubuntu 
distributions), this script can fail with the error message: 

{noformat}
sh: source: not found
{noformat}

We should change the script to be strictly {{sh}}-compatible for portability or 
change the specified shell on the shebang line to {{#!/bin/bash}}


> nifi-registry.sh is not strictly sh compatible
> --
>
> Key: NIFIREG-145
> URL: https://issues.apache.org/jira/browse/NIFIREG-145
> Project: NiFi Registry
>  Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: Kevin Doran
>Priority: Minor
>
> {{nifi-registry.sh}} uses some the {{bash}}-specific commands/functions that 
> are not part of {{sh}}.
> For example, [the use of {{source}} on line 
> 88|https://github.com/apache/nifi-registry/blob/rel/nifi-registry-0.1.0/nifi-registry-resources/src/main/resources/bin/nifi-registry.sh#L88],
>  which [does not exist in 
> {{sh}}|https://stackoverflow.com/questions/13702425/source-command-not-found-in-sh-shell/13702462].
> On platforms where the system shell ({{sh}}) symlinks to something other than 
> {{bash}} (apparently it points to {{dash}} on some Debian/Ubuntu 
> distributions), this script can fail with the error message:
> {noformat}
> sh: source: not found
> {noformat}
> We should change the script to be strictly {{sh}}-compatible for portability 
> or change the specified shell on the shebang line to {{#!/bin/bash}}
> 
> As a workaround for those encountering this issue in nifi-registry-0.1.0:
>  * change line 1 of nifi-registry.sh to {{#!/bin/bash}}; or
>  * change line 88 of nifi-registry.sh to use {{.}} instead of {{source}}; or
>  * leave nifi-registry.sh unmodified, but execute it using bash, ie:
> {noformat}
> bash -e /path/to/nifi-registry/bin/nifi-registry.sh [run|start|stop|...]
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFIREG-145) nifi-registry.sh is not strictly sh compatible

[Kevin Doran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFIREG-145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Doran updated NIFIREG-145:

Affects Version/s: 0.1.0

> nifi-registry.sh is not strictly sh compatible
> --
>
> Key: NIFIREG-145
> URL: https://issues.apache.org/jira/browse/NIFIREG-145
> Project: NiFi Registry
>  Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: Kevin Doran
>Priority: Minor
>
> {{nifi-registry.sh}} uses some the {{bash}}-specific commands/functions that 
> are not part of {{sh}}.
> For example, [the use of {{source}} on line 
> 88|https://github.com/apache/nifi-registry/blob/rel/nifi-registry-0.1.0/nifi-registry-resources/src/main/resources/bin/nifi-registry.sh#L88],
>  which [does not exist in 
> {{sh}}|https://stackoverflow.com/questions/13702425/source-command-not-found-in-sh-shell/13702462].
> On platforms where the system shell ({{sh}}) symlinks to something other than 
> {{bash}} (apparently it points to {{dash}} on some Debian/Ubuntu 
> distributions), this script can fail with the error message: 
> {noformat}
> sh: source: not found
> {noformat}
> We should change the script to be strictly {{sh}}-compatible for portability 
> or change the specified shell on the shebang line to {{#!/bin/bash}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-registry issue #102: "source" doesn't exist in regular /bin/sh

[NKame]

Github user NKame commented on the issue:

https://github.com/apache/nifi-registry/pull/102
  
I've tested the version with /bin/sh + . (instead of source): working on my 
setup, better than having a dependency towards bash.


---

[jira] [Created] (NIFIREG-145) nifi-registry.sh is not strictly sh compatible

[Kevin Doran (JIRA)]

Kevin Doran created NIFIREG-145:
---

 Summary: nifi-registry.sh is not strictly sh compatible
 Key: NIFIREG-145
 URL: https://issues.apache.org/jira/browse/NIFIREG-145
 Project: NiFi Registry
  Issue Type: Bug
Reporter: Kevin Doran


{{nifi-registry.sh}} uses some the {{bash}}-specific commands/functions that 
are not part of {{sh}}.

For example, [the use of {{source}} on line 
88|https://github.com/apache/nifi-registry/blob/rel/nifi-registry-0.1.0/nifi-registry-resources/src/main/resources/bin/nifi-registry.sh#L88],
 which [does not exist in 
{{sh}}|https://stackoverflow.com/questions/13702425/source-command-not-found-in-sh-shell/13702462].

On platforms where the system shell ({{sh}}) symlinks to something other than 
{{bash}} (apparently it points to {{dash}} on some Debian/Ubuntu 
distributions), this script can fail with the error message: 

{noformat}
sh: source: not found
{noformat}

We should change the script to be strictly {{sh}}-compatible for portability or 
change the specified shell on the shebang line to {{#!/bin/bash}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (MINIFICPP-405) RPG Binding to Local Network Interface

[bqiu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

bqiu updated MINIFICPP-405:
---
Description: 
Remote Process Group for site2site bind to a local interface name

http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#configure-site-to-site-client-nifi-instance.
 YAML config via 
https://github.com/apache/nifi-minifi/blob/master/minifi-docs/src/main/markdown/System_Admin_Guide.md#remote-process-groups-1

  was:
Remote Process Group for site2site bind to a local interface name

 


> RPG Binding to Local Network Interface
> --
>
> Key: MINIFICPP-405
> URL: https://issues.apache.org/jira/browse/MINIFICPP-405
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 1.0.0
>Reporter: bqiu
>Assignee: bqiu
>Priority: Minor
> Fix For: 1.0.0
>
>
> Remote Process Group for site2site bind to a local interface name
> http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#configure-site-to-site-client-nifi-instance.
>  YAML config via 
> https://github.com/apache/nifi-minifi/blob/master/minifi-docs/src/main/markdown/System_Admin_Guide.md#remote-process-groups-1



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (MINIFICPP-405) RPG Binding to Local Network Interface

[bqiu (JIRA)]

bqiu created MINIFICPP-405:
--

 Summary: RPG Binding to Local Network Interface
 Key: MINIFICPP-405
 URL: https://issues.apache.org/jira/browse/MINIFICPP-405
 Project: NiFi MiNiFi C++
  Issue Type: Improvement
Affects Versions: 1.0.0
Reporter: bqiu
Assignee: bqiu
 Fix For: 1.0.0


Remote Process Group for site2site bind to a local interface name

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (MINIFICPP-404) HTTP Proxy Support for HTTP Site to Site

[bqiu (JIRA)]

bqiu created MINIFICPP-404:
--

 Summary: HTTP Proxy Support for HTTP Site to Site
 Key: MINIFICPP-404
 URL: https://issues.apache.org/jira/browse/MINIFICPP-404
 Project: NiFi MiNiFi C++
  Issue Type: Improvement
Affects Versions: 1.0.0
Reporter: bqiu
Assignee: bqiu
 Fix For: 1.0.0


HTTP Proxy Support for HTTP Site to Site

http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#configure-site-to-site-client-nifi-instance.
 support for this in YAML config via 
https://github.com/apache/nifi-minifi/blob/master/minifi-docs/src/main/markdown/System_Admin_Guide.md#remote-process-groups-1.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (MINIFICPP-403) Enable tagging of flowfiles with flow metadata information in C++

[bqiu (JIRA)]

bqiu created MINIFICPP-403:
--

 Summary: Enable tagging of flowfiles with flow metadata 
information in C++
 Key: MINIFICPP-403
 URL: https://issues.apache.org/jira/browse/MINIFICPP-403
 Project: NiFi MiNiFi C++
  Issue Type: Improvement
Affects Versions: 1.0.0
Reporter: bqiu
Assignee: bqiu
 Fix For: 1.0.0


Provide framework level support to tag flowfiles with metadata about the flow 
that created them.

Design proposal

Right now, MiNiFi support core attributes like 

// FlowFile Attribute
enum FlowAttribute {
 // The flowfile's path indicates the relative directory to which a FlowFile 
belongs and does not contain the filename
 PATH = 0,
 // The flowfile's absolute path indicates the absolute directory to which a 
FlowFile belongs and does not contain the filename
 ABSOLUTE_PATH,
 // The filename of the FlowFile. The filename should not contain any directory 
structure.
 FILENAME,
 // A unique UUID assigned to this FlowFile.
 UUID,
 // A numeric value indicating the FlowFile priority
 priority,
 // The MIME Type of this FlowFile
 MIME_TYPE,
 // Specifies the reason that a FlowFile is being discarded
 DISCARD_REASON,
 // Indicates an identifier other than the FlowFile's UUID that is known to 
refer to this FlowFile.
 ALTERNATE_IDENTIFIER,
 MAX_FLOW_ATTRIBUTES
};

So one approach is in the flow YAML file, specific the list of core flow 
attributes along with the processors that inject/import/create the flow files.
When flow was created/imported/injected by this processor, we can apply these 
core attributes to the new flow.
Also user can define their own core attributes template and EL for populate 
value for these core attributes, for example protocol, TTL, record route 
(expected route), key, version, etc.

In current implementation, FILENAME, PATH and UUID are required attributes when 
flow was created, others are optional

// Populate the default attributes

addKeyedAttribute(FILENAME,
std::to_string(getTimeNano()));

addKeyedAttribute(PATH, DEFAULT_FLOWFILE_PATH);

addKeyedAttribute(UUID,
getUUIDStr())

So if user specify the optional meta flow info section for the processor with 
the key/value pairs as above, MiNiFI will add these key attributes to the flow 
when flow was created by this processor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-registry issue #102: "source" doesn't exist in regular /bin/sh

[kevdoran]

Github user kevdoran commented on the issue:

https://github.com/apache/nifi-registry/pull/102
  
Hi @NKame, thanks for the contribution. It seems the issue is the use of 
the [`source` function on line 
88](https://github.com/apache/nifi-registry/blob/rel/nifi-registry-0.1.0/nifi-registry-resources/src/main/resources/bin/nifi-registry.sh#L88),
 which is a `bash` builtin.

While changing the shebang line to specify bash would fix this issue, I 
think we could also modify the script to remove any bash builtins and maintain 
sh/posix compatibility.

Could you try replacing line 88 in the script with this and see if that 
fixes the issue on your platform:

. /etc/os-release


---

[jira] [Commented] (NIFI-4882) CSVRecordReader should utilize specified date/time/timestamp format at its convertSimpleIfPossible method

[Derek Straka (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365791#comment-16365791
 ] 

Derek Straka commented on NIFI-4882:


I'll make a patchset for this with the proposed refactor.

> CSVRecordReader should utilize specified date/time/timestamp format at its 
> convertSimpleIfPossible method
> -
>
> Key: NIFI-4882
> URL: https://issues.apache.org/jira/browse/NIFI-4882
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Koji Kawamura
>Priority: Major
>
> CSVRecordReader.convertSimpleIfPossible method is used by ValidateRecord. The 
> method does not coerce values to the target schema field type if the raw 
> string representation in the input CSV file is not compatible.
> The type compatibility check is implemented as follows. But it does not use 
> user specified date/time/timestamp format:
> {code}
> // This will return 'false' for input '01/01/1900' when user 
> specified custom format 'MM/dd/'
> if (DataTypeUtils.isCompatibleDataType(trimmed, dataType)) {
> // The LAZY_DATE_FORMAT should be used to check 
> compatibility, too.
> return DataTypeUtils.convertType(trimmed, dataType, 
> LAZY_DATE_FORMAT, LAZY_TIME_FORMAT, LAZY_TIMESTAMP_FORMAT, fieldName);
> } else {
> return value;
> }
> {code}
> If input date strings have different format than the default format 
> '-MM-dd', then ValidateRecord processor can not validate input records.
> JacksonCSVRecordReader has the identical methods with CSVRecordReader. Those 
> classes should have an abstract class.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-registry issue #102: "source" doesn't exist in regular /bin/sh

[NKame]

Github user NKame commented on the issue:

https://github.com/apache/nifi-registry/pull/102
  
Thought it was not the real problem on my setup, but it was: reopening.


---

[GitHub] nifi-registry pull request #102: "source" doesn't exist in regular /bin/sh

[NKame]

GitHub user NKame reopened a pull request:

https://github.com/apache/nifi-registry/pull/102

"source" doesn't exist in regular /bin/sh



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/NKame/nifi-registry patch-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-registry/pull/102.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #102


commit fcbbeb971e9b1dd89479644f737ba0cceb69efea
Author: NKame 
Date:   2018-02-15T15:31:30Z

"source" doesn't exist in regular /bin/sh




---

[GitHub] nifi-registry pull request #102: "source" doesn't exist in regular /bin/sh

[NKame]

Github user NKame closed the pull request at:

https://github.com/apache/nifi-registry/pull/102


---

[GitHub] nifi-registry pull request #102: "source" doesn't exist in regular /bin/sh

[NKame]

GitHub user NKame opened a pull request:

https://github.com/apache/nifi-registry/pull/102

"source" doesn't exist in regular /bin/sh



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/NKame/nifi-registry patch-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-registry/pull/102.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #102


commit fcbbeb971e9b1dd89479644f737ba0cceb69efea
Author: NKame 
Date:   2018-02-15T15:31:30Z

"source" doesn't exist in regular /bin/sh




---

[GitHub] nifi-minifi-cpp issue #264: Minificpp 400

[apiri]

Github user apiri commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/264
  
reviewing


---

[jira] [Updated] (MINIFICPP-382) Add SUSE support to bootstrap process.

[Aldrin Piri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aldrin Piri updated MINIFICPP-382:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Add SUSE support to bootstrap process. 
> ---
>
> Key: MINIFICPP-382
> URL: https://issues.apache.org/jira/browse/MINIFICPP-382
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: marco polo
>Assignee: marco polo
>Priority: Major
> Fix For: 0.5.0
>
>
> Add support to bootstrap process. 
>  
> Currently have tested on OpenSUSE and SLES12. 
>  
> SLES12/OpenSUSE – build and tested, verifying SiteToSite
> SLES11 – TBD



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-382) Add SUSE support to bootstrap process.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365703#comment-16365703
 ] 

ASF GitHub Bot commented on MINIFICPP-382:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/260


> Add SUSE support to bootstrap process. 
> ---
>
> Key: MINIFICPP-382
> URL: https://issues.apache.org/jira/browse/MINIFICPP-382
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: marco polo
>Assignee: marco polo
>Priority: Major
> Fix For: 0.5.0
>
>
> Add support to bootstrap process. 
>  
> Currently have tested on OpenSUSE and SLES12. 
>  
> SLES12/OpenSUSE – build and tested, verifying SiteToSite
> SLES11 – TBD



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp pull request #260: MINIFICPP-382: Implement SUSE release sup...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/260


---

[jira] [Commented] (NIFI-4859) Swagger Spec VersionControlInformationDTO missing SYNC_FAILURE state

[Kevin Doran (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365623#comment-16365623
 ] 

Kevin Doran commented on NIFI-4859:
---

Good to know. Thanks [~chaffelson]

> Swagger Spec VersionControlInformationDTO missing SYNC_FAILURE state
> 
>
> Key: NIFI-4859
> URL: https://issues.apache.org/jira/browse/NIFI-4859
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Flow Versioning
>Affects Versions: 1.5.0
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Minor
> Fix For: 1.6.0
>
>
> It is possible to get a Versioned Process Group into a SYNC_FAILURE state, 
> but this is not an allowable state in the code generated from the swagger.json
> This prevents versioned objects from being manipulated via the API in some 
> use cases.
> {noformat}
> @state.setter
> def state(self, state):
> """
> Sets the state of this VersionControlInformationDTO.
> The current state of the Process Group, as it relates to the Versioned Flow
> :param state: The state of this VersionControlInformationDTO.
> :type: str
> """
> allowed_values = ["LOCALLY_MODIFIED_DESCENDANT", "LOCALLY_MODIFIED", "STALE", 
> "LOCALLY_MODIFIED_AND_STALE", "UP_TO_DATE"]
> if state not in allowed_values:
> raise ValueError(
> "Invalid value for `state` ({0}), must be one of {1}"
> > .format(state, allowed_values)
> )
> E ValueError: Invalid value for `state` (SYNC_FAILURE), must be one of 
> ['LOCALLY_MODIFIED_DESCENDANT', 'LOCALLY_MODIFIED', 'STALE', 
> 'LOCALLY_MODIFIED_AND_STALE', 'UP_TO_DATE']
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-382) Add SUSE support to bootstrap process.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365601#comment-16365601
 ] 

ASF GitHub Bot commented on MINIFICPP-382:
--

Github user apiri commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/260
  
@phrocker great, thanks will get this merged. sorry didn't notice the 
update.


> Add SUSE support to bootstrap process. 
> ---
>
> Key: MINIFICPP-382
> URL: https://issues.apache.org/jira/browse/MINIFICPP-382
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: marco polo
>Assignee: marco polo
>Priority: Major
> Fix For: 0.5.0
>
>
> Add support to bootstrap process. 
>  
> Currently have tested on OpenSUSE and SLES12. 
>  
> SLES12/OpenSUSE – build and tested, verifying SiteToSite
> SLES11 – TBD



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp issue #260: MINIFICPP-382: Implement SUSE release support fo...

[apiri]

Github user apiri commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/260
  
@phrocker great, thanks will get this merged. sorry didn't notice the 
update.


---

[jira] [Comment Edited] (NIFI-4859) Swagger Spec VersionControlInformationDTO missing SYNC_FAILURE state

[Daniel Chaffelson (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365571#comment-16365571
 ] 

Daniel Chaffelson edited comment on NIFI-4859 at 2/15/18 2:16 PM:
--

This also applies to ProcessGroupStatusSnapshotDTO, specifically the 
VersionedFlowState

And the ProcessGroupEntity, and the FlowBreadCrumbEntity


was (Author: chaffelson):
This also applies to ProcessGroupStatusSnapshotDTO, specifically the 
VersionedFlowState

> Swagger Spec VersionControlInformationDTO missing SYNC_FAILURE state
> 
>
> Key: NIFI-4859
> URL: https://issues.apache.org/jira/browse/NIFI-4859
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Flow Versioning
>Affects Versions: 1.5.0
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Minor
> Fix For: 1.6.0
>
>
> It is possible to get a Versioned Process Group into a SYNC_FAILURE state, 
> but this is not an allowable state in the code generated from the swagger.json
> This prevents versioned objects from being manipulated via the API in some 
> use cases.
> {noformat}
> @state.setter
> def state(self, state):
> """
> Sets the state of this VersionControlInformationDTO.
> The current state of the Process Group, as it relates to the Versioned Flow
> :param state: The state of this VersionControlInformationDTO.
> :type: str
> """
> allowed_values = ["LOCALLY_MODIFIED_DESCENDANT", "LOCALLY_MODIFIED", "STALE", 
> "LOCALLY_MODIFIED_AND_STALE", "UP_TO_DATE"]
> if state not in allowed_values:
> raise ValueError(
> "Invalid value for `state` ({0}), must be one of {1}"
> > .format(state, allowed_values)
> )
> E ValueError: Invalid value for `state` (SYNC_FAILURE), must be one of 
> ['LOCALLY_MODIFIED_DESCENDANT', 'LOCALLY_MODIFIED', 'STALE', 
> 'LOCALLY_MODIFIED_AND_STALE', 'UP_TO_DATE']
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4859) Swagger Spec VersionControlInformationDTO missing SYNC_FAILURE state

[Daniel Chaffelson (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365571#comment-16365571
 ] 

Daniel Chaffelson commented on NIFI-4859:
-

This also applies to ProcessGroupStatusSnapshotDTO, specifically the 
VersionedFlowState

> Swagger Spec VersionControlInformationDTO missing SYNC_FAILURE state
> 
>
> Key: NIFI-4859
> URL: https://issues.apache.org/jira/browse/NIFI-4859
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Flow Versioning
>Affects Versions: 1.5.0
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Minor
> Fix For: 1.6.0
>
>
> It is possible to get a Versioned Process Group into a SYNC_FAILURE state, 
> but this is not an allowable state in the code generated from the swagger.json
> This prevents versioned objects from being manipulated via the API in some 
> use cases.
> {noformat}
> @state.setter
> def state(self, state):
> """
> Sets the state of this VersionControlInformationDTO.
> The current state of the Process Group, as it relates to the Versioned Flow
> :param state: The state of this VersionControlInformationDTO.
> :type: str
> """
> allowed_values = ["LOCALLY_MODIFIED_DESCENDANT", "LOCALLY_MODIFIED", "STALE", 
> "LOCALLY_MODIFIED_AND_STALE", "UP_TO_DATE"]
> if state not in allowed_values:
> raise ValueError(
> "Invalid value for `state` ({0}), must be one of {1}"
> > .format(state, allowed_values)
> )
> E ValueError: Invalid value for `state` (SYNC_FAILURE), must be one of 
> ['LOCALLY_MODIFIED_DESCENDANT', 'LOCALLY_MODIFIED', 'STALE', 
> 'LOCALLY_MODIFIED_AND_STALE', 'UP_TO_DATE']
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp pull request #264: Minificpp 400

[phrocker]

GitHub user phrocker opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/264

Minificpp 400

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [ ] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file?
- [ ] If applicable, have you updated the NOTICE file?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-400

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/264.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #264


commit 6640b27a52d0174b27007cac5e7d53e13df50caf
Author: Marc Parisi 
Date:   2018-02-14T19:47:29Z

MINIFICPP-400: Add Debian specific limitations on the SSL version, update 
the docs, and resolve Debian test failure

commit 93154323627655518db3b0ff757f07cf2cc160bc
Author: Marc Parisi 
Date:   2018-02-15T13:29:54Z

MINIFICPP-31: Update broken test as a result of failed merge.

Original committer introduced a fix for this :

https://raw.githubusercontent.com/achristianson/nifi-minifi-cpp/efd701594f43eecc2e2e5d058bc60337a35b7b55/libminifi/test/unit/UpdateAttributeTests.cpp

but I must have missed this when merging so I am correcting the error.




---

[jira] [Comment Edited] (NIFI-4197) *GCP processors do not expose proxy settings

[Julian Gimbel (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365504#comment-16365504
 ] 

Julian Gimbel edited comment on NIFI-4197 at 2/15/18 1:14 PM:
--

Two additional attributes proxy_user and proxy_password for authenticated proxy 
usage would be awesome:

 
{code:java}
final String authUser = "user";
final String authPassword = "password";
Authenticator.setDefault(
   new Authenticator() {
  @Override
  public PasswordAuthentication getPasswordAuthentication() {
 return new PasswordAuthentication(
authUser, authPassword.toCharArray()
 );
  }
   }
);

{code}
 

 


was (Author: jugi):
Two additional attributes proxy_user and proxy_password for authenticated proxy 
usage would be awesome:

final String authUser = "user";
final String authPassword = "password"; 
Authenticator.setDefault( new Authenticator() { 
   @Override public PasswordAuthentication getPasswordAuthentication() {
        return new PasswordAuthentication( authUser, 
                      authPassword.toCharArray()); } } ); 

> *GCP processors do not expose proxy settings
> 
>
> Key: NIFI-4197
> URL: https://issues.apache.org/jira/browse/NIFI-4197
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Andre F de Miranda
>Assignee: Andre F de Miranda
>Priority: Major
> Fix For: 1.5.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4197) *GCP processors do not expose proxy settings

[Julian Gimbel (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365504#comment-16365504
 ] 

Julian Gimbel commented on NIFI-4197:
-

Two additional attributes proxy_user and proxy_password for authenticated proxy 
usage would be awesome:

final String authUser = "user";
final String authPassword = "password"; 
Authenticator.setDefault( new Authenticator() { 
   @Override public PasswordAuthentication getPasswordAuthentication() {
        return new PasswordAuthentication( authUser, 
                      authPassword.toCharArray()); } } ); 

> *GCP processors do not expose proxy settings
> 
>
> Key: NIFI-4197
> URL: https://issues.apache.org/jira/browse/NIFI-4197
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Andre F de Miranda
>Assignee: Andre F de Miranda
>Priority: Major
> Fix For: 1.5.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-382) Add SUSE support to bootstrap process.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365482#comment-16365482
 ] 

ASF GitHub Bot commented on MINIFICPP-382:
--

Github user phrocker commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/260
  
@apiri I made an update to the readme last night. 


https://github.com/apache/nifi-minifi-cpp/pull/260/commits/7a3e81066c007053adc77a315f490e44671cbd34


> Add SUSE support to bootstrap process. 
> ---
>
> Key: MINIFICPP-382
> URL: https://issues.apache.org/jira/browse/MINIFICPP-382
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: marco polo
>Assignee: marco polo
>Priority: Major
> Fix For: 0.5.0
>
>
> Add support to bootstrap process. 
>  
> Currently have tested on OpenSUSE and SLES12. 
>  
> SLES12/OpenSUSE – build and tested, verifying SiteToSite
> SLES11 – TBD



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp issue #260: MINIFICPP-382: Implement SUSE release support fo...

[phrocker]

Github user phrocker commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/260
  
@apiri I made an update to the readme last night. 


https://github.com/apache/nifi-minifi-cpp/pull/260/commits/7a3e81066c007053adc77a315f490e44671cbd34


---

[jira] [Created] (NIFI-4883) VaridateRecord processor should be able to use different schema for valid and invalid records

[Koji Kawamura (JIRA)]

Koji Kawamura created NIFI-4883:
---

 Summary: VaridateRecord processor should be able to use different 
schema for valid and invalid records
 Key: NIFI-4883
 URL: https://issues.apache.org/jira/browse/NIFI-4883
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Reporter: Koji Kawamura


ValidateRecord uses the same record schema for both valid and invalid outgoing 
FlowFiles. When the processor writes invalid records using the schema which is 
incompatible for the invalid records will not be able to write such records and 
throws Exception like this:

{code}
2018-02-15 10:15:09,311 ERROR [Timer-Driven Process Thread-2] 
o.a.n.processors.standard.ValidateRecord ValidateRecord[id
=2f5c7438-b406-3811-82f4-4d8122cc87eb] 
ValidateRecord[id=2f5c7438-b406-3811-82f4-4d8122cc87eb] failed to process 
session
 due to java.lang.NumberFormatException: For input string: "01/01/1900": {}
java.lang.NumberFormatException: For input string: "01/01/1900"
at 
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:589)
at java.lang.Long.parseLong(Long.java:631)
at 
org.apache.nifi.serialization.record.util.DataTypeUtils.toLong(DataTypeUtils.java:822)
at 
org.apache.nifi.avro.AvroTypeUtil.convertToAvroObject(AvroTypeUtil.java:494)
at 
org.apache.nifi.avro.AvroTypeUtil.createAvroRecord(AvroTypeUtil.java:456)
at 
org.apache.nifi.avro.WriteAvroResultWithSchema.writeRecord(WriteAvroResultWithSchema.java:60)
at 
org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59)
at 
org.apache.nifi.processors.standard.ValidateRecord.onTrigger(ValidateRecord.java:328)
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at 
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
{code}

ValidateRecord processor should support different schema for invalid FlowFiles. 
It could be all String type auto generated schema.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (NIFI-4882) CSVRecordReader should utilize specified date/time/timestamp format at its convertSimpleIfPossible method

[Koji Kawamura (JIRA)]

Koji Kawamura created NIFI-4882:
---

 Summary: CSVRecordReader should utilize specified 
date/time/timestamp format at its convertSimpleIfPossible method
 Key: NIFI-4882
 URL: https://issues.apache.org/jira/browse/NIFI-4882
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Reporter: Koji Kawamura


CSVRecordReader.convertSimpleIfPossible method is used by ValidateRecord. The 
method does not coerce values to the target schema field type if the raw string 
representation in the input CSV file is not compatible.

The type compatibility check is implemented as follows. But it does not use 
user specified date/time/timestamp format:

{code}
// This will return 'false' for input '01/01/1900' when user 
specified custom format 'MM/dd/'
if (DataTypeUtils.isCompatibleDataType(trimmed, dataType)) {
// The LAZY_DATE_FORMAT should be used to check 
compatibility, too.
return DataTypeUtils.convertType(trimmed, dataType, 
LAZY_DATE_FORMAT, LAZY_TIME_FORMAT, LAZY_TIMESTAMP_FORMAT, fieldName);
} else {
return value;
}
{code}

If input date strings have different format than the default format 
'-MM-dd', then ValidateRecord processor can not validate input records.

JacksonCSVRecordReader has the identical methods with CSVRecordReader. Those 
classes should have an abstract class.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4367) InvokedScriptedProcessor does not support scripted processor that extends AbstractProcessor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365241#comment-16365241
 ] 

ASF GitHub Bot commented on NIFI-4367:
--

Github user mgaido91 commented on the issue:

https://github.com/apache/nifi/pull/2201
  
kindly ping @frett27 


> InvokedScriptedProcessor does not support scripted processor that extends 
> AbstractProcessor
> ---
>
> Key: NIFI-4367
> URL: https://issues.apache.org/jira/browse/NIFI-4367
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Patrice Freydiere
>Priority: Major
>  Labels: InvokeScriptedProcessor, validation
>
> InvokeScriptedProcessor pass his ValidationContext to the inner script, 
> validate call
> InvokeScriptedProcessor Line 465 :final 
> Collection instanceResults = instance.validate(context);
>  
> The problem is that the invokedscript pass the ScriptFile PropertyDescriptor 
> that is validated, if the script derived from the 
> AbstractConfigurableComponent, (because the AbstractConfigurableComponent 
> validate all the context properties).
> The context should be refined to remove the InvokeScriptedProcessor 
> Properties.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2201: NIFI-4367 Fix on processor for permit deriving script clas...

[mgaido91]

Github user mgaido91 commented on the issue:

https://github.com/apache/nifi/pull/2201
  
kindly ping @frett27 


---