[jira] [Created] (NIFI-4909) ExtractHL7Attributes isn't correctly extracting repeating DG1 segments

[Joey Frazee (JIRA)]

Joey Frazee created NIFI-4909:
-

 Summary: ExtractHL7Attributes isn't correctly extracting repeating 
DG1 segments
 Key: NIFI-4909
 URL: https://issues.apache.org/jira/browse/NIFI-4909
 Project: Apache NiFi
  Issue Type: Bug
Affects Versions: 1.5.0, 1.4.0, 1.3.0
Reporter: Joey Frazee
Assignee: Joey Frazee


In HL7 documents with multiple DG1 segments, only the last DG1 segment is 
extracted (sometimes several times though), and the attribute names are 
incorrectly indexed.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4827) Make GetMongo able to use flowfiles for queries

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376231#comment-16376231
 ] 

ASF GitHub Bot commented on NIFI-4827:
--

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2443
  
Two of the 3 builds succeeded, and there are now five tests that cover the 
"read query" behavior so I think the tires have been pretty well kicked on this 
one.


> Make GetMongo able to use flowfiles for queries
> ---
>
> Key: NIFI-4827
> URL: https://issues.apache.org/jira/browse/NIFI-4827
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>Priority: Minor
>
> GetMongo should be able to retrieve a valid query from the flowfile content 
> or allow the incoming flowfile to provide attributes to power EL statements 
> in the Query configuration field. Allowing the body to be used would allow 
> GetMongo to be used in a much more generic way.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2443: NIFI-4827 Added support for reading queries from the flowf...

[MikeThomsen]

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2443
  
Two of the 3 builds succeeded, and there are now five tests that cover the 
"read query" behavior so I think the tires have been pretty well kicked on this 
one.


---

[jira] [Commented] (NIFI-4876) Add Minimum Age Filter to ListS3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376224#comment-16376224
 ] 

ASF GitHub Bot commented on NIFI-4876:
--

GitHub user jvwing opened a pull request:

https://github.com/apache/nifi/pull/2491

NIFI-4876 Adding Min Object Age to ListS3

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [X] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [X] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [X] Is your initial contribution a single, squashed commit?

### For code changes:
- [X] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [X] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [X] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jvwing/nifi NIFI-4876-lists3-minage-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2491.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2491


commit b0fb4c6e541fc2ce8b6f6babf743c75d4f259100
Author: James Wing 
Date:   2018-02-25T18:41:33Z

NIFI-4876 Adding Min Object Age to ListS3




> Add Minimum Age Filter to ListS3
> 
>
> Key: NIFI-4876
> URL: https://issues.apache.org/jira/browse/NIFI-4876
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.5.0
>Reporter: James Wing
>Assignee: James Wing
>Priority: Minor
>
> ListS3 can experience difficulty reading the latest objects in a rapidly 
> changing S3 bucket due to the eventually consistent nature of S3.  Much of 
> this difficulty might be avoided by ignoring objects until a minimum age, 
> even 30 seconds or 1 minute.  I propose to add a Minimum Object Age feature 
> to ListS3, similar to the Minimum File Age in GetFile.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2491: NIFI-4876 Adding Min Object Age to ListS3

[jvwing]

GitHub user jvwing opened a pull request:

https://github.com/apache/nifi/pull/2491

NIFI-4876 Adding Min Object Age to ListS3

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [X] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [X] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [X] Is your initial contribution a single, squashed commit?

### For code changes:
- [X] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [X] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [X] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jvwing/nifi NIFI-4876-lists3-minage-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2491.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2491


commit b0fb4c6e541fc2ce8b6f6babf743c75d4f259100
Author: James Wing 
Date:   2018-02-25T18:41:33Z

NIFI-4876 Adding Min Object Age to ListS3




---

[jira] [Commented] (NIFI-4903) Database Fetch processors cannot handle timestamp fields with Oracle 12+

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376158#comment-16376158
 ] 

ASF GitHub Bot commented on NIFI-4903:
--

Github user zenfenan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2488#discussion_r170459537
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AbstractDatabaseFetchProcessor.java
 ---
@@ -446,7 +446,7 @@ protected static String getLiteralByType(int type, 
String value, String database
 case TIME:
 return "'" + value + "'";
 case TIMESTAMP:
-if ("Oracle".equals(databaseType)) {
+if (!StringUtils.isEmpty(databaseType) && 
databaseType.contains("Oracle")) {
--- End diff --

How about using `equalsIgnoreCase` in checking if databaseType contains 
oracle? It's trivial but a safety check, IMHO, in case some one changes the 
`getName` in `DatabaseAdapter` implementations.


> Database Fetch processors cannot handle timestamp fields with Oracle 12+
> 
>
> Key: NIFI-4903
> URL: https://issues.apache.org/jira/browse/NIFI-4903
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
> Fix For: 1.6.0
>
>
> When using GenerateTableFetch or QueryDatabaseTable with an Oracle 12+ 
> adapter, if the maximum value column is set to a column of type timestamp, 
> then upon the second fetch the following error occurs:
> java.sql.SQLDataException: ORA-01843: not a valid month
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2488: NIFI-4903: Fixed timestamp bug with fetch processor...

[zenfenan]

Github user zenfenan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2488#discussion_r170459537
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AbstractDatabaseFetchProcessor.java
 ---
@@ -446,7 +446,7 @@ protected static String getLiteralByType(int type, 
String value, String database
 case TIME:
 return "'" + value + "'";
 case TIMESTAMP:
-if ("Oracle".equals(databaseType)) {
+if (!StringUtils.isEmpty(databaseType) && 
databaseType.contains("Oracle")) {
--- End diff --

How about using `equalsIgnoreCase` in checking if databaseType contains 
oracle? It's trivial but a safety check, IMHO, in case some one changes the 
`getName` in `DatabaseAdapter` implementations.


---

[jira] [Commented] (NIFI-4906) Add GetHdfsFileInfo Processor

[Sivaprasanna Sethuraman (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376153#comment-16376153
 ] 

Sivaprasanna Sethuraman commented on NIFI-4906:
---

Is ListHDFS not good enough for this? It covers almost all of the required info 
that you have mentioned.

> Add GetHdfsFileInfo Processor
> -
>
> Key: NIFI-4906
> URL: https://issues.apache.org/jira/browse/NIFI-4906
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Ed Berezitsky
>Assignee: Ed Berezitsky
>Priority: Major
>
> Add *GetHdfsFileInfo* Processor to be able to get stats from a file system.
> This processor should support recursive scan, getting information of 
> directories and files.
> _File-level info required_: name, path, length, modified timestamp, last 
> access timestamp, owner, group, permissions.
> _Directory-level info required_: name, path, sum of lengths of files under a 
> dir, count of files under a dir, modified timestamp, last access timestamp, 
> owner, group, permissions.
>  
> The result returned:
>  * in single flow file (in content - a json line per file/dir info);
>  * flow file per each file/dir info (in content as json obj or in set of 
> attributes by the choice).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4503) Connection to support load-balancing strategies

[Phillip Tensa (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376082#comment-16376082
 ] 

Phillip Tensa commented on NIFI-4503:
-

I voted although I think the more appropriate solution would be to just allow 
to use rpg with non-root-leveled input ports. It would allow the same thing 
except it won't allow to do it dynamically unless it would be possible to 
mention the current process group id with expression language.

> Connection to support load-balancing strategies
> ---
>
> Key: NIFI-4503
> URL: https://issues.apache.org/jira/browse/NIFI-4503
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Haimo Liu
>Priority: Major
>
> As an operator, I want to be able to create new list/fetch flows encapsulated 
> within a single process group, that automatically distribute the fetch 
> operations across the nodes in my NiFi cluster, so that I can manage each 
> flow independently from one another and without the need for orchestration 
> with remote process groups.
> It would be great to add the ability for on any given connection have a user 
> be able to select ‘auto balance across cluster’ and it will automatically 
> take care of distributing the objects across the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (NIFI-4901) Json to Avro using Record framework does not support union types with boolean

[Gardella Juan Pablo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gardella Juan Pablo resolved NIFI-4901.
---
Resolution: Invalid

[~markap14] you are right. My fault, I tried to use the valid JSON for 
[JsonDecoder|https://avro.apache.org/docs/current/api/java/org/apache/avro/io/JsonDecoder.html]
 class as the input, and actually this is not required. Thanks for take some 
time to answer the ticket.

> Json to Avro using Record framework does not support union types with boolean
> -
>
> Key: NIFI-4901
> URL: https://issues.apache.org/jira/browse/NIFI-4901
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0
> Environment: ALL
>Reporter: Gardella Juan Pablo
>Priority: Major
> Attachments: optiona-boolean.zip
>
>
> Given the following valid Avro Schema:
> {code}
> {
>"type":"record",
>"name":"foo",
>"fields":[
>   {
>  "name":"isSwap",
>  "type":[
> "boolean",
> "null"
>  ]
>   } 
>]
> }
> {code}
> And the following JSON:
> {code}
> {
>   "isSwap": {
> "boolean": true
>   }
> }
> {code}
> When it is trying to be converted to Avro using ConvertRecord fails with:
> {{org.apache.nifi.serialization.MalformedRecordException: Successfully parsed 
> a JSON object from input but failed to convert into a Record object with the 
> given schema}}
> Attached a repository to reproduce the issue and also included the fix:
> * Run {{mvn clean test}} to reproduce the issue.
> * Run {{mvn clean test -Ppatch}} to test the fix. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4827) Make GetMongo able to use flowfiles for queries

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376076#comment-16376076
 ] 

ASF GitHub Bot commented on NIFI-4827:
--

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2443
  
@mattyb149 I broke the query location unit test down into three distinct 
cases:

* Verify that an empty query w/ no flowfile defaults to timer-driven 
behavior with `{}` as the query.
* Verify that a query param will work when set.
* Verify that it can read from the body w/ EL support.


> Make GetMongo able to use flowfiles for queries
> ---
>
> Key: NIFI-4827
> URL: https://issues.apache.org/jira/browse/NIFI-4827
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>Priority: Minor
>
> GetMongo should be able to retrieve a valid query from the flowfile content 
> or allow the incoming flowfile to provide attributes to power EL statements 
> in the Query configuration field. Allowing the body to be used would allow 
> GetMongo to be used in a much more generic way.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2443: NIFI-4827 Added support for reading queries from the flowf...

[MikeThomsen]

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2443
  
@mattyb149 I broke the query location unit test down into three distinct 
cases:

* Verify that an empty query w/ no flowfile defaults to timer-driven 
behavior with `{}` as the query.
* Verify that a query param will work when set.
* Verify that it can read from the body w/ EL support.


---

[jira] [Assigned] (NIFI-4704) get a view of the process as a list of directories

[Yuri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuri reassigned NIFI-4704:
--

Assignee: Yuri

> get a view of the process as a list of directories
> --
>
> Key: NIFI-4704
> URL: https://issues.apache.org/jira/browse/NIFI-4704
> Project: Apache NiFi
>  Issue Type: Wish
>  Components: Core UI
>Reporter: Alexandre Côté
>Assignee: Yuri
>Priority: Minor
>
> Hello,
> It's very long to move around all our componend of Nifi (nvagiation in the 
> group).
> It could be very usefull to get something like a directory view as we can get 
> one in eclipse for exemple.
> We can imagine in the next step to have some information about the process in 
> that view (state, error, number of flow, ...), and start and stop the process 
> directly in the view for exemple.
> Thank's,
> Alex



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)