[GitHub] nifi pull request #2875: NIFI-5351 Support for Decimals in FetchParquet
Github user joetrite closed the pull request at: https://github.com/apache/nifi/pull/2875 ---
[GitHub] nifi pull request #2875: NIFI-5351 Support for Decimals in FetchParquet
GitHub user joetrite opened a pull request: https://github.com/apache/nifi/pull/2875 NIFI-5351 Support for Decimals in FetchParquet Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/joetrite/nifi nifi-5351 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2875.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2875 commit ca1d7de926b0ee9445de667277b203e56cece817 Author: Joe Trite Date: 2018-07-10T23:53:09Z NIFI-5351 Support for Decimals in FetchParquet ---
[GitHub] nifi pull request #2711: NIFI-1705 - Adding AttributesToCSV processor
Github user joetrite commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2711#discussion_r193808025
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AttributesToCSV.java
---
@@ -0,0 +1,335 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.text.StringEscapeUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.HashSet;
+import java.util.List;
+import java.util.LinkedHashMap;
+import java.util.LinkedHashSet;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+import java.util.Collections;
+import java.util.Arrays;
+import java.util.ArrayList;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"csv", "attributes", "flowfile"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Generates a CSV representation of the input
FlowFile Attributes. The resulting CSV " +
+"can be written to either a newly generated attribute named
'CSVAttributes' or written to the FlowFile as content. " +
+"If the attribute value contains a comma, newline or double quote,
then the attribute value will be " +
+"escaped with double quotes. Any double quote characters in the
attribute value are escaped with " +
+"another double quote.")
+@WritesAttributes({
+@WritesAttribute(attribute = "CSVSchema", description = "CSV
representation of the Schema"),
+@WritesAttribute(attribute = "CSVData", description = "CSV
representation of Attributes")
+})
+
+public class AttributesToCSV extends AbstractProcessor {
+private static final String DATA_ATTRIBUTE_NAME = "CSVData";
+private static final String SCHEMA_ATTRIBUTE_NAME = "CSVSchema";
+private static final String OUTPUT_SEPARATOR = ",";
+private static final String OUTPUT_MIME_TYPE = "text/csv";
+private static final String SPLIT_REGEX = OUTPUT_SEPARATOR +
"(?=(?:[^\"]*\"[^\"]*\")*[^\"]*$)";
+
+static final AllowableValue OUTPUT_OVERWRITE_CONTENT = new
AllowableValue("flowfile-content", "flowfile-content", "The resulting CSV
string will be placed into the content of the flowfile." +
+"Existing flowfile context will be overwritten. 'CSVData' will
not be written to at all (neither null nor
[GitHub] nifi issue #2711: NIFI-1705 - Adding AttributesToCSV processor
Github user joetrite commented on the issue: https://github.com/apache/nifi/pull/2711 @bdesert good catch, we should add the option to push the header to attribute or content based on the selection. The header should also be csv format so that when the header and data are in the output file they will have the same schema. ---
[GitHub] nifi pull request #2711: NIFI-1705 - Adding AttributesToCSV processor
Github user joetrite commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2711#discussion_r190733381
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AttributesToCSV.java
---
@@ -0,0 +1,272 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.lang3.StringEscapeUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Collections;
+import java.util.stream.Collectors;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"csv", "attributes", "flowfile"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Generates a CSV representation of the input
FlowFile Attributes. The resulting CSV " +
+"can be written to either a newly generated attribute named
'CSVAttributes' or written to the FlowFile as content. " +
+"If the attribute value contains a comma, newline or double quote,
then the attribute value will be " +
+"escaped with double quotes. Any double quote characters in the
attribute value are escaped with " +
+"another double quote. If the attribute value does not contain a
comma, newline or double quote, then the " +
+"attribute value is returned unchanged.")
+@WritesAttribute(attribute = "CSVAttributes", description = "CSV
representation of Attributes")
+public class AttributesToCSV extends AbstractProcessor {
+
+private static final String OUTPUT_NEW_ATTRIBUTE =
"flowfile-attribute";
+private static final String OUTPUT_OVERWRITE_CONTENT =
"flowfile-content";
+private static final String OUTPUT_ATTRIBUTE_NAME = "CSVAttributes";
+private static final String OUTPUT_SEPARATOR = ",";
+private static final String OUTPUT_MIME_TYPE = "text/csv";
+
+
+public static final PropertyDescriptor ATTRIBUTES_LIST = new
PropertyDescriptor.Builder()
+.name("attribute-list")
+.displayName("Attribute List")
+.description("Comma separated list of attributes to be
included in the resulting CSV. If this value " +
+"is left empty then all existing Attributes will be
included. This list of attributes is " +
+"case sensitive and does not support attribute names
that contain commas. If an attribute
[GitHub] nifi pull request #2711: NIFI-1705 - Adding AttributesToCSV processor
Github user joetrite commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2711#discussion_r190733355
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AttributesToCSV.java
---
@@ -0,0 +1,272 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.lang3.StringEscapeUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Collections;
+import java.util.stream.Collectors;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"csv", "attributes", "flowfile"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Generates a CSV representation of the input
FlowFile Attributes. The resulting CSV " +
+"can be written to either a newly generated attribute named
'CSVAttributes' or written to the FlowFile as content. " +
+"If the attribute value contains a comma, newline or double quote,
then the attribute value will be " +
+"escaped with double quotes. Any double quote characters in the
attribute value are escaped with " +
+"another double quote. If the attribute value does not contain a
comma, newline or double quote, then the " +
+"attribute value is returned unchanged.")
+@WritesAttribute(attribute = "CSVAttributes", description = "CSV
representation of Attributes")
+public class AttributesToCSV extends AbstractProcessor {
+
+private static final String OUTPUT_NEW_ATTRIBUTE =
"flowfile-attribute";
+private static final String OUTPUT_OVERWRITE_CONTENT =
"flowfile-content";
+private static final String OUTPUT_ATTRIBUTE_NAME = "CSVAttributes";
+private static final String OUTPUT_SEPARATOR = ",";
+private static final String OUTPUT_MIME_TYPE = "text/csv";
+
+
+public static final PropertyDescriptor ATTRIBUTES_LIST = new
PropertyDescriptor.Builder()
+.name("attribute-list")
+.displayName("Attribute List")
+.description("Comma separated list of attributes to be
included in the resulting CSV. If this value " +
+"is left empty then all existing Attributes will be
included. This list of attributes is " +
+"case sensitive and does not support attribute names
that contain commas. If an attribute
[GitHub] nifi issue #2711: NIFI-1705 - Adding AttributesToCSV processor
Github user joetrite commented on the issue: https://github.com/apache/nifi/pull/2711 @bdesert it looks like the regex support was added to AttributesToJSON after copied the code. I'll add regex support to this processor and check if anything else has changed. ---
[GitHub] nifi pull request #2711: NIFI-1705 - Adding AttributesToCSV processor
Github user joetrite commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2711#discussion_r190288780
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AttributesToCSV.java
---
@@ -0,0 +1,272 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.lang3.StringEscapeUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Collections;
+import java.util.stream.Collectors;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"csv", "attributes", "flowfile"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Generates a CSV representation of the input
FlowFile Attributes. The resulting CSV " +
+"can be written to either a newly generated attribute named
'CSVAttributes' or written to the FlowFile as content. " +
+"If the attribute value contains a comma, newline or double quote,
then the attribute value will be " +
+"escaped with double quotes. Any double quote characters in the
attribute value are escaped with " +
+"another double quote. If the attribute value does not contain a
comma, newline or double quote, then the " +
+"attribute value is returned unchanged.")
+@WritesAttribute(attribute = "CSVAttributes", description = "CSV
representation of Attributes")
+public class AttributesToCSV extends AbstractProcessor {
+
+private static final String OUTPUT_NEW_ATTRIBUTE =
"flowfile-attribute";
+private static final String OUTPUT_OVERWRITE_CONTENT =
"flowfile-content";
+private static final String OUTPUT_ATTRIBUTE_NAME = "CSVAttributes";
+private static final String OUTPUT_SEPARATOR = ",";
+private static final String OUTPUT_MIME_TYPE = "text/csv";
+
+
+public static final PropertyDescriptor ATTRIBUTES_LIST = new
PropertyDescriptor.Builder()
+.name("attribute-list")
+.displayName("Attribute List")
+.description("Comma separated list of attributes to be
included in the resulting CSV. If this value " +
+"is left empty then all existing Attributes will be
included. This list of attributes is " +
+"case sensitive and does not support attribute names
that contain commas. If an attribute specified in the list is not found it will
be emitted " +
--- End diff --
will do, thanks for the feedback.
---
[GitHub] nifi pull request #2711: NIFI-1705 - Adding AttributesToCSV processor
Github user joetrite commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2711#discussion_r190286828
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AttributesToCSV.java
---
@@ -0,0 +1,272 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.lang3.StringEscapeUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Collections;
+import java.util.stream.Collectors;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"csv", "attributes", "flowfile"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Generates a CSV representation of the input
FlowFile Attributes. The resulting CSV " +
+"can be written to either a newly generated attribute named
'CSVAttributes' or written to the FlowFile as content. " +
+"If the attribute value contains a comma, newline or double quote,
then the attribute value will be " +
+"escaped with double quotes. Any double quote characters in the
attribute value are escaped with " +
+"another double quote. If the attribute value does not contain a
comma, newline or double quote, then the " +
+"attribute value is returned unchanged.")
+@WritesAttribute(attribute = "CSVAttributes", description = "CSV
representation of Attributes")
+public class AttributesToCSV extends AbstractProcessor {
+
+private static final String OUTPUT_NEW_ATTRIBUTE =
"flowfile-attribute";
+private static final String OUTPUT_OVERWRITE_CONTENT =
"flowfile-content";
+private static final String OUTPUT_ATTRIBUTE_NAME = "CSVAttributes";
+private static final String OUTPUT_SEPARATOR = ",";
+private static final String OUTPUT_MIME_TYPE = "text/csv";
+
+
+public static final PropertyDescriptor ATTRIBUTES_LIST = new
PropertyDescriptor.Builder()
+.name("attribute-list")
+.displayName("Attribute List")
+.description("Comma separated list of attributes to be
included in the resulting CSV. If this value " +
+"is left empty then all existing Attributes will be
included. This list of attributes is " +
+"case sensitive and does not support attribute names
that contain commas. If an attribute
[GitHub] nifi issue #1589: NIFI-1705 Adding AttributesToCSV processor
Github user joetrite commented on the issue: https://github.com/apache/nifi/pull/1589 i drove this one off a cliff :). New pull request raised. ---
[GitHub] nifi pull request #2711: NIFI-1705 - Adding AttributesToCSV processor
GitHub user joetrite opened a pull request: https://github.com/apache/nifi/pull/2711 NIFI-1705 - Adding AttributesToCSV processor New pull request replacing https://github.com/apache/nifi/pull/1589 You can merge this pull request into a Git repository by running: $ git pull https://github.com/joetrite/nifi nifi-1705 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2711.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2711 commit 6ca49f3c0cddcede2028aeecd41f69d21e8f1c15 Author: Joe Trite <joetrite@...> Date: 2018-05-16T18:32:05Z Squashed commit of the following: commit 9c31e45d3f94bf1e73c87379f315f6559e6a23f4 Author: Joe Trite <joetr...@gmail.com> Date: Mon Mar 13 07:55:19 2017 -0400 NIFI-1705 Adding AttributesToCSV processor commit 5e9afa9ccf4276c7a2318b761ded77fd514a60d9 Merge: 3177eb185 74cbfc4b6 Author: Joe Trite <joetr...@gmail.com> Date: Sat Mar 4 08:12:39 2017 -0500 Merge remote-tracking branch 'origin/master' # Conflicts: # nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ScanAttribute.java commit 3177eb1852dbc7e70c250c53d038a5bcc5cfa3c5 Author: Joe Trite <joetr...@gmail.com> Date: Sat Mar 4 08:09:06 2017 -0500 NIFI-3497 Converted properties to use displayName, changed validator on demarcator property and created a method for duplicate code. commit 74cbfc4b69ffde1ddb4442b91e160b66ea3c8b6b Merge: a974f78e0 f8cad0f8c Author: Joe Trite <joetr...@gmail.com> Date: Sat Mar 4 07:47:46 2017 -0500 Merge branch 'master' into master commit a974f78e033885455cadd2cbffd7e387cbabc4d7 Author: Joe Trite <joetr...@gmail.com> Date: Sat Mar 4 07:43:02 2017 -0500 NIFI-3497 Converted properties to use displayName, changed validator on demarcator property and created a method for duplicate code. commit 1bfaef8e87eaa6af84b1a80cb680e0a15eef2f8d Merge: 65ed46de9 89ec68d14 Author: Joe Trite <joetr...@gmail.com> Date: Fri Mar 3 08:01:59 2017 -0500 Merge branch 'master' of https://github.com/joetrite/nifi commit 65ed46de9a00518cfe06ecd69bd4f3bbd8d3e662 Author: Joe Trite <joetr...@gmail.com> Date: Fri Feb 24 18:09:36 2017 -0500 NIFI-3497 - fixing Pcontrib issues commit c5d52cf6f0f16496d9ef83411770409c0b2f88d4 Author: Joe Trite <joetr...@gmail.com> Date: Thu Feb 23 10:19:01 2017 -0500 NIFI-3497 - excluding test files Adding new test data files to exclude list. commit b1959341138d3b5004979544fcea86ba36fe6ebb Author: Joe Trite <joetr...@gmail.com> Date: Wed Feb 22 16:48:10 2017 -0500 NIFI-3497 - New dictionary files for test Adding new dictionary files to support metadata dictionary option. commit e296268f39bf213a9e8eaa8298b26556c6efe278 Author: Joe Trite <joetr...@gmail.com> Date: Wed Feb 22 16:46:13 2017 -0500 NIFI-3497 test cases for metadata updates Adding test cases to support metadata option update. commit de7e348e62c0f7c5fbd334106878ca6ac46935af Author: Joe Trite <joetr...@gmail.com> Date: Wed Feb 22 16:36:08 2017 -0500 NIFI-3497 - Added metadata option Added optional to post additional metadata as new attributed if a match is found in the dictionary. commit 89ec68d14bb34cbe65ff9a4d50ff5321fd4ec0ef Author: Joe Trite <joetr...@gmail.com> Date: Fri Feb 24 18:09:36 2017 -0500 NIFI-3497 - fixing Pcontrib issues commit d71426037b142da8ca04dae38952c164d1614806 Author: Joe Trite <joetr...@gmail.com> Date: Thu Feb 23 10:19:01 2017 -0500 NIFI-3497 - excluding test files Adding new test data files to exclude list. commit a7a7b6ace80380416c342809ce95a4f4087bb066 Author: Joe Trite <joetr...@gmail.com> Date: Wed Feb 22 16:48:10 2017 -0500 NIFI-3497 - New dictionary files for test Adding new dictionary files to support metadata dictionary option. commit 8eb54a50193897cf564eb7d222aae35481168af4 Author: Joe Trite <joetr...@gmail.com> Date: Wed Feb 22 16:46:13 2017 -0500 NIFI-3497 test cases for metadata updates Adding test cases to support metadata option update. commit f52e1f2a064b31f87d4165af6075716fa7d55046 Author: Joe Trite <joetr...@gmail.com> Date: Wed Feb 22 16:36:08 2017 -0500 NIFI-3497 - Added metadata option Added optional to post additional metadata as new attributed if a match is found in the dictionary. ---
[GitHub] nifi pull request #1589: NIFI-1705 Adding AttributesToCSV processor
Github user joetrite closed the pull request at: https://github.com/apache/nifi/pull/1589 ---
[GitHub] nifi issue #1589: NIFI-1705 Adding AttributesToCSV processor
Github user joetrite commented on the issue: https://github.com/apache/nifi/pull/1589 @mattyb149 thanks for reviewing, let me have a think on these things. i like the idea of allowing the attribute list to be dynamic. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1589: NIFI-1705 Adding AttributesToCSV processor
GitHub user joetrite opened a pull request: https://github.com/apache/nifi/pull/1589 NIFI-1705 Adding AttributesToCSV processor NIFI-1705 Adding AttributesToCSV processor You can merge this pull request into a Git repository by running: $ git pull https://github.com/joetrite/nifi NIFI-1705 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1589.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1589 commit 9c31e45d3f94bf1e73c87379f315f6559e6a23f4 Author: Joe Trite <joetr...@gmail.com> Date: 2017-03-13T11:55:19Z NIFI-1705 Adding AttributesToCSV processor --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1564: NIFI-3497 ScanAttribute Tagging
Github user joetrite commented on the issue: https://github.com/apache/nifi/pull/1564 @joewitt fixed routing, simplified code + thread safety issue and spelling of tomato :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1564: NIFI-3497 ScanAttribute Tagging
Github user joetrite commented on the issue: https://github.com/apache/nifi/pull/1564 @joewitt i'm made most of these changes, still testings things but one thing i want to note/raise (maybe by design?) in the original code is that if the ff does not contain any attributes that match the attribute pattern regex, then the ff is automatically going to the matched relationship. I would think that if I specify an attribute pattern and none of the attributes match the pattern then the ff would go to unmatched. I'm not planning to change this functionality as it would def break existing flows, just want to note it in case it is a bug/not working as expected. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1564: NIFI-3497 ScanAttribute Tagging
Github user joetrite commented on the issue: https://github.com/apache/nifi/pull/1564 @joewitt thx for the feedback. will work on it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1564: NIFI-3497 ScanAttribute Tagging
Github user joetrite commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1564#discussion_r10431
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ScanAttribute.java
---
@@ -60,13 +64,21 @@
@Tags({"scan", "attributes", "search", "lookup"})
@CapabilityDescription("Scans the specified attributes of FlowFiles,
checking to see if any of their values are "
+ "present within the specified dictionary of terms")
+@WritesAttributes({
+@WritesAttribute(attribute = "dictionary.hit.{n}.attribute",
description = "The attribute name that had a value hit on the dictionary
file."),
--- End diff --
sounds good, changing...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1536: NIFI-3497 ScanAttribute Tagging
Github user joetrite closed the pull request at: https://github.com/apache/nifi/pull/1536 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1536: NIFI-3497 ScanAttribute Tagging
Github user joetrite commented on the issue: https://github.com/apache/nifi/pull/1536 created new pull request #1564 with single commit. closing this. thanks @joewitt @alopresto for all the help and advise. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1564: NIFI-3497 ScanAttribute Tagging
GitHub user joetrite opened a pull request: https://github.com/apache/nifi/pull/1564 NIFI-3497 ScanAttribute Tagging NIFI-3497 Adding tagging option to ScanAttribute You can merge this pull request into a Git repository by running: $ git pull https://github.com/joetrite/nifi NIFI-3497-ScanAttribute-Tagging Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1564.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1564 commit c4b231e13399879fb6de1f3766f291adbc61c0de Author: Joe Trite <joetr...@gmail.com> Date: 2017-03-04T20:17:21Z NIFI-3497 ScanAttribute Tagging --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1536: NIFI-3497 ScanAttribute Tagging
Github user joetrite commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1536#discussion_r104274990
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ScanAttribute.java
---
@@ -97,13 +107,23 @@
.addValidator(StandardValidators.createRegexValidator(0, 1,
false))
.defaultValue(null)
.build();
+public static final PropertyDescriptor
DICTIONARY_ENTRY_METADATA_DEMARCATOR = new PropertyDescriptor.Builder()
+.name("Dictionary Entry Metadata Demarcator")
--- End diff --
ok, cool. i copied and pasted the existing properties which have not
implemented displayName yet. Do you want me to change it for the new
properties that I added and leave the existing properties alone?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1543: NIFI-1750 - updated dict file desc
GitHub user joetrite opened a pull request: https://github.com/apache/nifi/pull/1543 NIFI-1750 - updated dict file desc Clarifying the load/reload strategy of the dictionary file. You can merge this pull request into a Git repository by running: $ git pull https://github.com/joetrite/nifi patch-3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1543.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1543 commit 4c7227c2af7ebdbdac277f595e25f2267627005f Author: Joe Trite <joetr...@gmail.com> Date: 2017-02-24T21:18:54Z NIFI-1750 - updated dict file desc Clarifying the load/reload strategy of the dictionary file. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1536: NIFI-3497 ScanAttribute Tagging
GitHub user joetrite opened a pull request: https://github.com/apache/nifi/pull/1536 NIFI-3497 ScanAttribute Tagging Adding tagging to ScanAttribute. You can merge this pull request into a Git repository by running: $ git pull https://github.com/joetrite/nifi master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1536.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1536 commit f52e1f2a064b31f87d4165af6075716fa7d55046 Author: Joe Trite <joetr...@gmail.com> Date: 2017-02-22T21:36:08Z NIFI-3497 - Added metadata option Added optional to post additional metadata as new attributed if a match is found in the dictionary. commit 8eb54a50193897cf564eb7d222aae35481168af4 Author: Joe Trite <joetr...@gmail.com> Date: 2017-02-22T21:46:13Z NIFI-3497 test cases for metadata updates Adding test cases to support metadata option update. commit a7a7b6ace80380416c342809ce95a4f4087bb066 Author: Joe Trite <joetr...@gmail.com> Date: 2017-02-22T21:48:10Z NIFI-3497 - New dictionary files for test Adding new dictionary files to support metadata dictionary option. commit d71426037b142da8ca04dae38952c164d1614806 Author: Joe Trite <joetr...@gmail.com> Date: 2017-02-23T15:19:01Z NIFI-3497 - excluding test files Adding new test data files to exclude list. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
