[jira] [Created] (NIFI-2857) Document Site 2 Site protocol

[Andre (JIRA)]

Andre created NIFI-2857:
---

 Summary: Document Site 2 Site protocol
 Key: NIFI-2857
 URL: https://issues.apache.org/jira/browse/NIFI-2857
 Project: Apache NiFi
  Issue Type: Bug
Reporter: Andre


It would be great if we could document the Site 2 Site protocol, in special, 
the ability to publish content to RPGs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2849) UI - Policy Management Enhancements

[Andy LoPresto (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15544090#comment-15544090
 ] 

Andy LoPresto commented on NIFI-2849:
-

I would even say for item 3 we default to copying the inherited policy rules. 
If a user invokes that option, they probably expect to make an individual 
change, rather than having to rebuild the existing policy from scratch. 
Deleting the unwanted authorizations listed is easier than remembering and 
rebuilding the previous authorizations. 

> UI - Policy Management Enhancements
> ---
>
> Key: NIFI-2849
> URL: https://issues.apache.org/jira/browse/NIFI-2849
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> 1 - Show component name whenever possible (specifically in the message 
> regarding the inherited policy).
> 2 - Do not allow editing of inherited policies. Require explicit navigation 
> to the component in question. Support linking to the component in the 
> inherited policy message to make this easier.
> 3 - When overriding a policy, allow the user to optionally copy the inherited 
> policy rules.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2852) Expression Language should allow base64 encode and decode

[Joseph Niemiec (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Niemiec updated NIFI-2852:
-
Attachment: 0002-NIFI-2852-base64-expression-langauge-functions.patch

Doc tweaks

> Expression Language should allow base64 encode and decode
> -
>
> Key: NIFI-2852
> URL: https://issues.apache.org/jira/browse/NIFI-2852
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 
> 0001-NIFI-2852-base64-expression-language-functions.patch, 
> 0002-NIFI-2852-base64-expression-langauge-functions.patch
>
>
> currently there is no way to deal with passing base64 data around from 
> attributes. This makes interacting with some REST based systems very hard as 
> we cannot encode dynamic attribute driven payloads. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2852) Expression Language should allow base64 encode and decode

[Joseph Niemiec (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Niemiec updated NIFI-2852:
-
Status: Patch Available  (was: In Progress)

> Expression Language should allow base64 encode and decode
> -
>
> Key: NIFI-2852
> URL: https://issues.apache.org/jira/browse/NIFI-2852
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 
> 0001-NIFI-2852-base64-expression-language-functions.patch, 
> 0002-NIFI-2852-base64-expression-langauge-functions.patch
>
>
> currently there is no way to deal with passing base64 data around from 
> attributes. This makes interacting with some REST based systems very hard as 
> we cannot encode dynamic attribute driven payloads. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2852) Expression Language should allow base64 encode and decode

[Joseph Niemiec (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Niemiec updated NIFI-2852:
-
Status: In Progress  (was: Patch Available)

> Expression Language should allow base64 encode and decode
> -
>
> Key: NIFI-2852
> URL: https://issues.apache.org/jira/browse/NIFI-2852
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 0001-NIFI-2852-base64-expression-language-functions.patch
>
>
> currently there is no way to deal with passing base64 data around from 
> attributes. This makes interacting with some REST based systems very hard as 
> we cannot encode dynamic attribute driven payloads. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2852) Expression Language should allow base64 encode and decode

[Joseph Niemiec (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Niemiec updated NIFI-2852:
-
Status: Patch Available  (was: In Progress)

tested with myself and another user. Main issue was related to redeploying the 
framework nar file after adding your expressions to get them to deploy. 

> Expression Language should allow base64 encode and decode
> -
>
> Key: NIFI-2852
> URL: https://issues.apache.org/jira/browse/NIFI-2852
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 0001-NIFI-2852-base64-expression-language-functions.patch
>
>
> currently there is no way to deal with passing base64 data around from 
> attributes. This makes interacting with some REST based systems very hard as 
> we cannot encode dynamic attribute driven payloads. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1922) Add support for WASB / Azure Blob Storage with HDFS processors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543844#comment-15543844
 ] 

ASF GitHub Bot commented on NIFI-1922:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/626
  
@atilcock not sure why GitHub doesn't show but this PR drifted from the 
master in the sense we no longer specify the hadoop version within the child 
pom.xml

Also, while I am not an expert, I am a bit worried about including Hadoop 
Azure dependency as default. 

I would imagine the dependency simply adds the wasb and wasbs file system 
schemas, but was wondering if tried compiling using the CDH and MapR profiles 
to see if this causes any issues?

Cheers


> Add support for WASB / Azure Blob Storage with HDFS processors
> --
>
> Key: NIFI-1922
> URL: https://issues.apache.org/jira/browse/NIFI-1922
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Joseph Witt
>
> https://hadoop.apache.org/docs/stable2/hadoop-azure/index.html
> Received two mailing list questions on this in the past couple weeks.
> We should look at what things we'd need to do in the processors to make this 
> possible.  Adding the dependency is one, anything else?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #626: NIFI-1922 added support for HDInsight wasb file storage to ...

[trixpan]

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/626
  
@atilcock not sure why GitHub doesn't show but this PR drifted from the 
master in the sense we no longer specify the hadoop version within the child 
pom.xml

Also, while I am not an expert, I am a bit worried about including Hadoop 
Azure dependency as default. 

I would imagine the dependency simply adds the wasb and wasbs file system 
schemas, but was wondering if tried compiling using the CDH and MapR profiles 
to see if this causes any issues?

Cheers


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-1840) Add compression type property in Kite processors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543814#comment-15543814
 ] 

ASF GitHub Bot commented on NIFI-1840:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/409
  
LGTM :+1:


> Add compression type property in Kite processors
> 
>
> Key: NIFI-1840
> URL: https://issues.apache.org/jira/browse/NIFI-1840
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 0.6.1
>Reporter: Pierre Villard
>
> Following
> https://community.hortonworks.com/questions/31017/unable-to-turn-off-snappy-compression-in-nifi-putk.html
> By default Kite uses Snappy compression. It would be interesting to have the 
> ability to change this compression.
> http://kitesdk.org/docs/1.0.0/API-Overview.html
> http://kitesdk.org/docs/1.0.0/apidocs/org/kitesdk/data/DatasetDescriptor.Builder.html#compressionType(java.lang.String)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #409: NIFI-1840 Added compression type property in Kite processor...

[trixpan]

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/409
  
LGTM :+1:


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-1582) New processor to update attributes with state

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543808#comment-15543808
 ] 

ASF GitHub Bot commented on NIFI-1582:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/319
  
@JPercivall welcome back!

Could you rebase this PR?


> New processor to update attributes with state
> -
>
> Key: NIFI-1582
> URL: https://issues.apache.org/jira/browse/NIFI-1582
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Joseph Percivall
>Assignee: Joseph Percivall
>
> This idea was sparked by a thread on the user list and should allow basic 
> data science:
> I expect that in the future I’ll need something a little more sophisticated 
> but for now my problem is very simple:
> I want to be able to trigger an alert (only once) when an attribute in an 
> incoming stream, for instance, goes over a predefined threshold. The 
> Processor should then trigger (only once again) another trigger when the 
> signal goes back to normal (below threshold). Basically a RouteByAttribute 
> but with memory.
> Thanks 
> Claudio
> 
> Hello Claudio,
> Your use-case actually could leverage a couple of recently added features to 
> create a really cool open-source processor. The two key features that were 
> added are State Management and the ability to reference processor specific 
> variables in expression language. You can take a look at RouteText to see 
> both in action. 
> By utilizing both you can create a processor that is configured with multiple 
> Expression language expressions. There would be dynamic properties which 
> would accept expression language and then store the evaluated value via state 
> management. Then there would be a routing property (that supports expression 
> language) that could simply add an attribute to the flowfile with the 
> evaluated value which would allow it to be used by flowing processors for 
> routing.
> This would allow you to do your use-case where you store the value for the 
> incoming stream and route differently once you go over a threshold. It could 
> even allow more complex use-cases. One instance, I believe, would be possible 
> is to have a running average and standard deviation and route data to 
> different locations based on it's standard deviation.
> You can think of this like an UpdateAttribute with the ability to store and 
> calculate variables using expression language.
> Joe



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (NIFI-2855) NiFi Site-To-Site with port forwarding

[Koji Kawamura (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Koji Kawamura reassigned NIFI-2855:
---

Assignee: Koji Kawamura

> NiFi Site-To-Site with port forwarding
> --
>
> Key: NIFI-2855
> URL: https://issues.apache.org/jira/browse/NIFI-2855
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Bryan Rosander
>Assignee: Koji Kawamura
>
> It would be useful to be able to use port forwarding with NiFi Site-To-Site.  
> This would allow NiFi to appear externally to be listening on a privileged 
> port without having been granted elevated permissions.
> For example, an administrator could configure iptables to forward traffic 
> from port 443 to port 9443.  Then users could use NiFi at port 443.  This 
> provides more flexibility as far as firewall configuration is concerned.
> The above scenario causes problems with Site-To-Site though because in a 
> clustered scenario, the nodes will still advertise themselves with port 9443. 
>  This would prevent a Site-To-Site client from being able to talk to them 
> from outside the firewall.
> We need a way (probably a nifi property) to tell NiFi to listen on one port 
> (9443) and advertise another (443) for Site-To-Site purposes to enable this 
> usecase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #319: NIFI-1582 added state to UpdateAttribute as well as updated...

[trixpan]

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/319
  
@JPercivall welcome back!

Could you rebase this PR?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543807#comment-15543807
 ] 

ASF GitHub Bot commented on NIFI-1613:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/293
  
@ToivoAdams is this ready for review?


> ConvertJSONToSQL Drops Type Information
> ---
>
> Key: NIFI-1613
> URL: https://issues.apache.org/jira/browse/NIFI-1613
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 0.4.1, 0.5.1
> Environment: Ubuntu 14.04 LTS
>Reporter: Aaron Stephens
>Assignee: Toivo Adams
>  Labels: ConvertJSONToSQL, Phoenix, SQL
>
> It appears that the ConvertJSONToSQL processor is turning Boolean (and 
> possibly Integer and Float) values into Strings.  This is okay for some 
> drivers (like PostgreSQL) which can coerce a String back into a Boolean, but 
> it causes issues for others (specifically Phoenix in my case).
> {noformat}
> org.apache.phoenix.schema.ConstraintViolationException: 
> org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type 
> mismatch. VARCHAR cannot be coerced to BOOLEAN
> at 
> org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282)
>  ~[na:na]
> at 
> org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na]
> at 
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442)
>  ~[na:na]
> at 
> org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166)
>  ~[na:na]
> at 
> org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166)
>  ~[na:na]
> at 
> org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) 
> ~[na:na]
> at 
> org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) 
> ~[na:na]
> at 
> org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na]
> at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>  ~[nifi-api-0.4.1.jar:0.4.1]
> at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146)
>  ~[nifi-framework-core-0.4.1.jar:0.4.1]
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
>  [nifi-framework-core-0.4.1.jar:0.4.1]
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
>  [nifi-framework-core-0.4.1.jar:0.4.1]
> at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119)
>  [nifi-framework-core-0.4.1.jar:0.4.1]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_79]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
> [na:1.7.0_79]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>  [na:1.7.0_79]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  [na:1.7.0_79]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_79]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_79]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 
> (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN
> at 
> org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71)
>  ~[na:na]
> at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
>  ~[na:na]
> ... 20 common frames omitted
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #512: NIFI-401

[trixpan]

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/512
  
@beugley would you mind rebasing this commit?

Cheers


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-401) New Scheduling strategy (On primary node - CRON )

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543799#comment-15543799
 ] 

ASF GitHub Bot commented on NIFI-401:
-

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/512
  
@beugley would you mind rebasing this commit?

Cheers


> New Scheduling strategy (On primary node - CRON )
> -
>
> Key: NIFI-401
> URL: https://issues.apache.org/jira/browse/NIFI-401
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Matthew Clarke
>Priority: Minor
> Attachments: initial prototype .png
>
>
> Currently the only scheduling strategy supported when a processor is set to 
> use "On primary Node" is Timer Driven.   There should be a second option to 
> allow cron driven On primary Node scheduling strategy.  This would allow 
> users to more control over when a given primary node only processor runs. 
> This would prevent these processors from running when configuration changes 
> or instance restarts occur.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2852) Expression Language should allow base64 encode and decode

[Joseph Niemiec (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Niemiec updated NIFI-2852:
-
Status: Open  (was: Patch Available)

> Expression Language should allow base64 encode and decode
> -
>
> Key: NIFI-2852
> URL: https://issues.apache.org/jira/browse/NIFI-2852
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 0001-NIFI-2852-base64-expression-language-functions.patch
>
>
> currently there is no way to deal with passing base64 data around from 
> attributes. This makes interacting with some REST based systems very hard as 
> we cannot encode dynamic attribute driven payloads. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2852) Expression Language should allow base64 encode and decode

[Joseph Niemiec (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Niemiec updated NIFI-2852:
-
Fix Version/s: 1.1.0
   Status: Patch Available  (was: In Progress)

Attached Patch. 

> Expression Language should allow base64 encode and decode
> -
>
> Key: NIFI-2852
> URL: https://issues.apache.org/jira/browse/NIFI-2852
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 0001-NIFI-2852-base64-expression-language-functions.patch
>
>
> currently there is no way to deal with passing base64 data around from 
> attributes. This makes interacting with some REST based systems very hard as 
> we cannot encode dynamic attribute driven payloads. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2852) Expression Language should allow base64 encode and decode

[Joseph Niemiec (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Niemiec updated NIFI-2852:
-
Attachment: 0001-NIFI-2852-base64-expression-language-functions.patch

> Expression Language should allow base64 encode and decode
> -
>
> Key: NIFI-2852
> URL: https://issues.apache.org/jira/browse/NIFI-2852
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
> Attachments: 0001-NIFI-2852-base64-expression-language-functions.patch
>
>
> currently there is no way to deal with passing base64 data around from 
> attributes. This makes interacting with some REST based systems very hard as 
> we cannot encode dynamic attribute driven payloads. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2852) Expression Language should allow base64 encode and decode

[Joseph Niemiec (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Niemiec updated NIFI-2852:
-
Attachment: .0001-NIFI-2852-base64-expression-language-functions.patch.swp

> Expression Language should allow base64 encode and decode
> -
>
> Key: NIFI-2852
> URL: https://issues.apache.org/jira/browse/NIFI-2852
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
> Attachments: 0001-NIFI-2852-base64-expression-language-functions.patch
>
>
> currently there is no way to deal with passing base64 data around from 
> attributes. This makes interacting with some REST based systems very hard as 
> we cannot encode dynamic attribute driven payloads. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2852) Expression Language should allow base64 encode and decode

[Joseph Niemiec (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Niemiec updated NIFI-2852:
-
Attachment: (was: 
.0001-NIFI-2852-base64-expression-language-functions.patch.swp)

> Expression Language should allow base64 encode and decode
> -
>
> Key: NIFI-2852
> URL: https://issues.apache.org/jira/browse/NIFI-2852
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
> Attachments: 0001-NIFI-2852-base64-expression-language-functions.patch
>
>
> currently there is no way to deal with passing base64 data around from 
> attributes. This makes interacting with some REST based systems very hard as 
> we cannot encode dynamic attribute driven payloads. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #721: Nifi 2398 - Apache Ignite Get Processor

[pvillard31]

Github user pvillard31 commented on the issue:

https://github.com/apache/nifi/pull/721
  
I have played with both Put and Get processors and it seems to work fine. 
However, when stopping/starting processors to change properties (to test 
batching and performances), I got in a situation with this exception:


2016-10-03 23:32:42,633 WARN [Timer-Driven Process Thread-4] 
o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding 
PutIgniteCache[id=4d80623c-0157-1000--179dcc8f] due to uncaught 
Exception: java.lang.IllegalStateException: Data streamer has been closed.
2016-10-03 23:32:42,634 WARN [Timer-Driven Process Thread-4] 
o.a.n.c.t.ContinuallyRunProcessorTask
java.lang.IllegalStateException: Data streamer has been closed.
at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.enterBusy(DataStreamerImpl.java:355)
 ~[na:na]
at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:489)
 ~[na:na]
at 
org.apache.nifi.processors.ignite.cache.PutIgniteCache.onTrigger(PutIgniteCache.java:274)
 ~[na:na]
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
 ~[nifi-api-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1060)
 ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
 [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
 [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:123)
 [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_77]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
[na:1.8.0_77]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
 [na:1.8.0_77]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
 [na:1.8.0_77]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[na:1.8.0_77]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_77]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]


It was not possible to get the flow back to OK unless restarting NiFi. I 
believe something is not closed correctly.

To reproduce, take this template, run all processors, everything should be 
OK, stop GetIgnite processor and restart it, do the same with PutIgnite. You 
should get the error.
Here is the template I use: 
https://gist.github.com/pvillard31/8a0c1d0344bf5bdfdf4cd460e936905c


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #721: Nifi 2398 - Apache Ignite Get Processor

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/721#discussion_r81645352
  
--- Diff: 
nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/src/main/java/org/apache/nifi/processors/ignite/AbstractIgniteProcessor.java
 ---
@@ -92,6 +94,14 @@ public void initializeIgnite(ProcessContext context) {
 return;
 }
 
+List grids = Ignition.allGrids();
+
+if ( grids.size() == 1 ) {
+getLogger().warn("Ignite grid already avaialble");
--- End diff --

I think this should not be a warning because if using Put and Get 
processors on the canvas we will get a warning bulletin although there is no 
issue. Maybe log as info and inform the user we use a shared grid? (also take 
care of the typo).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2849) UI - Policy Management Enhancements

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543237#comment-15543237
 ] 

ASF GitHub Bot commented on NIFI-2849:
--

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/1090

UI - Policy Management Improvements

NIFI-2849:
- Showing process group name when possible.
- Providing a link to jump to the process group defined in the effective 
policy.
- Preventing editing an inherited policy.
- When overriding a policy, allowing the user to indicate if the policy 
should be empty or should copy the user/groups of the inherited policy.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-2849

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1090.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1090


commit 23955f44ddff1b6369f2e71c68f9619682da0642
Author: Matt Gilman 
Date:   2016-10-03T19:50:32Z

NIFI-2849:
- Showing process group name when possible.
- Providing a link to jump to the process group defined in the effective 
policy.
- Preventing editing an inherited policy.
- When overriding a policy, allowing the user to indicate if the policy 
should be empty or should copy the user/groups of the inherited policy.




> UI - Policy Management Enhancements
> ---
>
> Key: NIFI-2849
> URL: https://issues.apache.org/jira/browse/NIFI-2849
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> 1 - Show component name whenever possible (specifically in the message 
> regarding the inherited policy).
> 2 - Do not allow editing of inherited policies. Require explicit navigation 
> to the component in question. Support linking to the component in the 
> inherited policy message to make this easier.
> 3 - When overriding a policy, allow the user to optionally copy the inherited 
> policy rules.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2849) UI - Policy Management Enhancements

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-2849:
--
Status: Patch Available  (was: In Progress)

> UI - Policy Management Enhancements
> ---
>
> Key: NIFI-2849
> URL: https://issues.apache.org/jira/browse/NIFI-2849
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> 1 - Show component name whenever possible (specifically in the message 
> regarding the inherited policy).
> 2 - Do not allow editing of inherited policies. Require explicit navigation 
> to the component in question. Support linking to the component in the 
> inherited policy message to make this easier.
> 3 - When overriding a policy, allow the user to optionally copy the inherited 
> policy rules.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1090: UI - Policy Management Improvements

[mcgilman]

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/1090

UI - Policy Management Improvements

NIFI-2849:
- Showing process group name when possible.
- Providing a link to jump to the process group defined in the effective 
policy.
- Preventing editing an inherited policy.
- When overriding a policy, allowing the user to indicate if the policy 
should be empty or should copy the user/groups of the inherited policy.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-2849

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1090.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1090


commit 23955f44ddff1b6369f2e71c68f9619682da0642
Author: Matt Gilman 
Date:   2016-10-03T19:50:32Z

NIFI-2849:
- Showing process group name when possible.
- Providing a link to jump to the process group defined in the effective 
policy.
- Preventing editing an inherited policy.
- When overriding a policy, allowing the user to indicate if the policy 
should be empty or should copy the user/groups of the inherited policy.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (NIFI-2856) Queue Listing - Cluster node id is missing

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-2856:
-

 Summary: Queue Listing - Cluster node id is missing
 Key: NIFI-2856
 URL: https://issues.apache.org/jira/browse/NIFI-2856
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework, Core UI
Reporter: Matt Gilman
Assignee: Matt Gilman
 Fix For: 1.1.0


The cluster node id appears to be missing when performing a queue listing in 
clustered mode. As a result, subsequent attempts to directly access that 
flowfile fail with a message indicating that

{noformat}The id of the node in the cluster is required{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2838) In the Advanced UI, if the Rule name is very long, the "x" to delete the rule becomes unavailable

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan updated NIFI-2838:
--
Status: Patch Available  (was: In Progress)

https://github.com/apache/nifi/pull/1089

> In the Advanced UI, if the Rule name is very long, the "x" to delete the rule 
> becomes unavailable
> -
>
> Key: NIFI-2838
> URL: https://issues.apache.org/jira/browse/NIFI-2838
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Andrew Lim
>Assignee: Scott Aslan
>Priority: Minor
>  Labels: UI
> Attachments: NIFI-2838_noDeleteIcon.png, NIFI-2838_twoXicons.png
>
>
> As shown in the attached screenshot, when the rule name is very long it 
> extends the whole width of the bar, so the "x" to delete the rule is no 
> longer selectable.
> Note: There is also a scroll bar that appears in this part of the UI, which I 
> thought would possibly reveal the "x", but the scroll bar doesn't appear to 
> add any value.  The width of the rule name bar doesn't extend the width of 
> the space.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2809) Add processors for Google Cloud Storage Fetch/Put/Delete

[Frank Maritato (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Frank Maritato updated NIFI-2809:
-
Attachment: gcp_bundle.patch

Updated patch:
* includes only nifi-gcp-bundle files
* update to nifi-assembly/NOTICE file

> Add processors for Google Cloud Storage Fetch/Put/Delete
> 
>
> Key: NIFI-2809
> URL: https://issues.apache.org/jira/browse/NIFI-2809
> Project: Apache NiFi
>  Issue Type: New Feature
>Affects Versions: 1.1.0
>Reporter: Frank Maritato
> Attachments: gcp_bundle.patch, gcs_bundle.patch
>
>
> Creating a ticket to add nifi processors for Google Cloud Storage similar to 
> how the AWS S3 processors are done:
> * FetchGCSObject
> * PutGCSObject
> * DeleteGCSObject
> I have the code mostly written and will attach the patch file when complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2603) Bringing Some UI Color Back

[Pierre Villard (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543150#comment-15543150
 ] 

Pierre Villard commented on NIFI-2603:
--

[~rmoran] +1

> Bringing Some UI Color Back
> ---
>
> Key: NIFI-2603
> URL: https://issues.apache.org/jira/browse/NIFI-2603
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Peter Wicks
>Priority: Minor
> Attachments: status-icon-color.png
>
>
> In the new 1.0 UI all of the colors associated with status (except the orange 
> triangle) are gone; replaced with a dark gray color.
> I propose bringing back color.  The screenshots are in the format of before 
> on the top and after on the bottom, except were labeled in the picture itself:
>  - Top Status Menu: https://goo.gl/photos/se1JnvhRwU7N4Fap7
>  - Process Group: https://goo.gl/photos/dqjG4KvC6xqxQfgT7
>  - Processes (Running/Stopped/Invalid): 
> https://goo.gl/photos/dSS8vgE2RkrXtc77A
>  - Operate Play/Stop buttons (only on mouse hover): 
> https://goo.gl/photos/Am5SUEEn7G9RjmMe6
>  - Processor/Processor Group Context Menu: 
> https://goo.gl/photos/Jq3qFg4ezaN91qms5
> This is not a "100% done, I've covered everything" before/after list.  I know 
> I need to do the NiFi summary page also at the minimum.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2603) Bringing Some UI Color Back

[Rob Moran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rob Moran updated NIFI-2603:

Attachment: status-icon-color.png

> Bringing Some UI Color Back
> ---
>
> Key: NIFI-2603
> URL: https://issues.apache.org/jira/browse/NIFI-2603
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Peter Wicks
>Priority: Minor
> Attachments: status-icon-color.png
>
>
> In the new 1.0 UI all of the colors associated with status (except the orange 
> triangle) are gone; replaced with a dark gray color.
> I propose bringing back color.  The screenshots are in the format of before 
> on the top and after on the bottom, except were labeled in the picture itself:
>  - Top Status Menu: https://goo.gl/photos/se1JnvhRwU7N4Fap7
>  - Process Group: https://goo.gl/photos/dqjG4KvC6xqxQfgT7
>  - Processes (Running/Stopped/Invalid): 
> https://goo.gl/photos/dSS8vgE2RkrXtc77A
>  - Operate Play/Stop buttons (only on mouse hover): 
> https://goo.gl/photos/Am5SUEEn7G9RjmMe6
>  - Processor/Processor Group Context Menu: 
> https://goo.gl/photos/Jq3qFg4ezaN91qms5
> This is not a "100% done, I've covered everything" before/after list.  I know 
> I need to do the NiFi summary page also at the minimum.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2603) Bringing Some UI Color Back

[Rob Moran (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543067#comment-15543067
 ] 

Rob Moran commented on NIFI-2603:
-

Hi all – I wanted to recommend icon colors for 'running, stopped, invalid, and 
disabled' statuses seen in the flow status bar. These same icons are seen on 
the faces of components so the idea is to use the same colors there as well. I 
believe this is all aligns with past discussion. We won't change colors on 
actions in palettes or context menus. This is purely for status-related icons 
to aid awareness for when things change.

I'm also recommending a unique style for all the icons in the status bar when 
values are 0. The thinking here is a change will be more obvious when going 
from 0 to 1 or more (or the opposite) and helps quickly distinguish differences 
while performing a visual scan across the bar.

See the attached mockup – *status-icon-color.png* – showing the 4 colored 
icons, ones with a 0 value, and examples of the colored icons on the face of a 
processor.

If there is some consensus on the proposed design I'll post color codes and 
other style-related details.

> Bringing Some UI Color Back
> ---
>
> Key: NIFI-2603
> URL: https://issues.apache.org/jira/browse/NIFI-2603
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Peter Wicks
>Priority: Minor
>
> In the new 1.0 UI all of the colors associated with status (except the orange 
> triangle) are gone; replaced with a dark gray color.
> I propose bringing back color.  The screenshots are in the format of before 
> on the top and after on the bottom, except were labeled in the picture itself:
>  - Top Status Menu: https://goo.gl/photos/se1JnvhRwU7N4Fap7
>  - Process Group: https://goo.gl/photos/dqjG4KvC6xqxQfgT7
>  - Processes (Running/Stopped/Invalid): 
> https://goo.gl/photos/dSS8vgE2RkrXtc77A
>  - Operate Play/Stop buttons (only on mouse hover): 
> https://goo.gl/photos/Am5SUEEn7G9RjmMe6
>  - Processor/Processor Group Context Menu: 
> https://goo.gl/photos/Jq3qFg4ezaN91qms5
> This is not a "100% done, I've covered everything" before/after list.  I know 
> I need to do the NiFi summary page also at the minimum.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2825) Remote Process Group should use num of queued flow files in remote nodes to distribute load

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende updated NIFI-2825:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Remote Process Group should use num of queued flow files in remote nodes to 
> distribute load
> ---
>
> Key: NIFI-2825
> URL: https://issues.apache.org/jira/browse/NIFI-2825
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
> Environment: Clustered
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>  Labels: Cluster
> Fix For: 1.1.0
>
>
> A RPG (Site-to-Site client) should take into account of the number of queued 
> flow files in each remote cluster nodes, to calculate the load distribution 
> balance among target nodes. So that it can send more data to nodes those have 
> less queued files, or receive more data from nodes those have more queued 
> files.
> However, after the clustering mechanism has changed since 1.0, this load 
> distribution logic is not handled appropriately. Site-to-Site remote 
> component always returns flow file count as 0. So RPG distribute load evenly, 
> even if a remote cluster has unbalanced load among nodes.
> We need to fix the remote node API to expose queued flow file count correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2825) Remote Process Group should use num of queued flow files in remote nodes to distribute load

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542985#comment-15542985
 ] 

ASF subversion and git services commented on NIFI-2825:
---

Commit 17a36c6fd525dba837e13a71a52ddfd224782fae in nifi's branch 
refs/heads/master from [~ijokarumawak]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=17a36c6 ]

NIFI-2825: Fix S2S getPeers flow file count

- Added ClusterWorkload message to retrieve workload information from a
  cluster coordinator
- Use cluster workload to return queued flow file count to site-to-site
  client so that it can calculate distribution of data transfer

This closes #1084.

Signed-off-by: Bryan Bende 


> Remote Process Group should use num of queued flow files in remote nodes to 
> distribute load
> ---
>
> Key: NIFI-2825
> URL: https://issues.apache.org/jira/browse/NIFI-2825
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
> Environment: Clustered
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>  Labels: Cluster
> Fix For: 1.1.0
>
>
> A RPG (Site-to-Site client) should take into account of the number of queued 
> flow files in each remote cluster nodes, to calculate the load distribution 
> balance among target nodes. So that it can send more data to nodes those have 
> less queued files, or receive more data from nodes those have more queued 
> files.
> However, after the clustering mechanism has changed since 1.0, this load 
> distribution logic is not handled appropriately. Site-to-Site remote 
> component always returns flow file count as 0. So RPG distribute load evenly, 
> even if a remote cluster has unbalanced load among nodes.
> We need to fix the remote node API to expose queued flow file count correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2825) Remote Process Group should use num of queued flow files in remote nodes to distribute load

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542987#comment-15542987
 ] 

ASF GitHub Bot commented on NIFI-2825:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1084


> Remote Process Group should use num of queued flow files in remote nodes to 
> distribute load
> ---
>
> Key: NIFI-2825
> URL: https://issues.apache.org/jira/browse/NIFI-2825
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
> Environment: Clustered
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>  Labels: Cluster
> Fix For: 1.1.0
>
>
> A RPG (Site-to-Site client) should take into account of the number of queued 
> flow files in each remote cluster nodes, to calculate the load distribution 
> balance among target nodes. So that it can send more data to nodes those have 
> less queued files, or receive more data from nodes those have more queued 
> files.
> However, after the clustering mechanism has changed since 1.0, this load 
> distribution logic is not handled appropriately. Site-to-Site remote 
> component always returns flow file count as 0. So RPG distribute load evenly, 
> even if a remote cluster has unbalanced load among nodes.
> We need to fix the remote node API to expose queued flow file count correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1084: NIFI-2825: Fix S2S getPeers flow file count

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1084


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2838) In the Advanced UI, if the Rule name is very long, the "x" to delete the rule becomes unavailable

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542875#comment-15542875
 ] 

ASF GitHub Bot commented on NIFI-2838:
--

GitHub user scottyaslan opened a pull request:

https://github.com/apache/nifi/pull/1089

[NIFI-2838] update width of rule name



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/scottyaslan/nifi NIFI-2838

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1089.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1089






> In the Advanced UI, if the Rule name is very long, the "x" to delete the rule 
> becomes unavailable
> -
>
> Key: NIFI-2838
> URL: https://issues.apache.org/jira/browse/NIFI-2838
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Andrew Lim
>Assignee: Scott Aslan
>Priority: Minor
>  Labels: UI
> Attachments: NIFI-2838_noDeleteIcon.png, NIFI-2838_twoXicons.png
>
>
> As shown in the attached screenshot, when the rule name is very long it 
> extends the whole width of the bar, so the "x" to delete the rule is no 
> longer selectable.
> Note: There is also a scroll bar that appears in this part of the UI, which I 
> thought would possibly reveal the "x", but the scroll bar doesn't appear to 
> add any value.  The width of the rule name bar doesn't extend the width of 
> the space.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1089: [NIFI-2838] update width of rule name

[scottyaslan]

GitHub user scottyaslan opened a pull request:

https://github.com/apache/nifi/pull/1089

[NIFI-2838] update width of rule name



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/scottyaslan/nifi NIFI-2838

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1089.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1089






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (NIFI-2854) Allow Provenance Repository to roll back to a previous implementation

[Mark Payne (JIRA)]

Mark Payne created NIFI-2854:


 Summary: Allow Provenance Repository to roll back to a previous 
implementation
 Key: NIFI-2854
 URL: https://issues.apache.org/jira/browse/NIFI-2854
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
 Fix For: 1.1.0


Currently, if we start up a new version of NiFi and the format of the Data 
Provenance data has changed, we are not able to roll back to a previous version 
of NiFi. If we do, NiFi will fail to read the Provenance Data and not start up. 
We should instead provide the ability to write data to the repository in such a 
way that old versions of the repository will still be able to read the data, so 
that we can roll back



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2853) Improve ListHDFS state tracking

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende updated NIFI-2853:
--
Description: 
Currently ListHDFS tracks two properties in state management, 
"listing.timestamp" and "emitted.timestamp". In the 1.0.0 release, the 
directory property now supports expression language which means the directory 
being listed could dynamically change on any execution of the processor. 

The processor should be changed to store state specific to the directory that 
was listed, for example "listing.timestamp.dir1" and "emitted.timestamp.dir1".

This would also help in a clustered scenario... currently ListHDFS has to be 
run on primary node only, otherwise each node will be overwriting each others 
state and producing unexpected results. With the above improvement, if the 
directory evaluated to a unique path for each node, it would store the state of 
each of those path separately.

  was:
Currently ListHDFS tracks two properties in state management, 
"listing.timestamp" and "emitted.timestamp". In the 1.0.0 release, the 
directory property now supports expression language which means the directory 
being listed could dynamically change on any execution of the processor. The 
processor should store state specific to the directory that was listed, for 
example "listing.timestamp.dir1" and "emitted.timestamp.dir1".

This would also help in a clustered scenario... currently ListHDFS has to be 
run on primary node only, otherwise each node will be overwriting each others 
state and producing unexpected results. With the above improvement, if the 
directory evaluated to a unique path for each node, it would store the state of 
each of those path separately.


> Improve ListHDFS state tracking
> ---
>
> Key: NIFI-2853
> URL: https://issues.apache.org/jira/browse/NIFI-2853
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.0.0
>Reporter: Bryan Bende
>Priority: Minor
>
> Currently ListHDFS tracks two properties in state management, 
> "listing.timestamp" and "emitted.timestamp". In the 1.0.0 release, the 
> directory property now supports expression language which means the directory 
> being listed could dynamically change on any execution of the processor. 
> The processor should be changed to store state specific to the directory that 
> was listed, for example "listing.timestamp.dir1" and "emitted.timestamp.dir1".
> This would also help in a clustered scenario... currently ListHDFS has to be 
> run on primary node only, otherwise each node will be overwriting each others 
> state and producing unexpected results. With the above improvement, if the 
> directory evaluated to a unique path for each node, it would store the state 
> of each of those path separately.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-2853) Improve ListHDFS state tracking

[Bryan Bende (JIRA)]

Bryan Bende created NIFI-2853:
-

 Summary: Improve ListHDFS state tracking
 Key: NIFI-2853
 URL: https://issues.apache.org/jira/browse/NIFI-2853
 Project: Apache NiFi
  Issue Type: Improvement
Affects Versions: 1.0.0
Reporter: Bryan Bende
Priority: Minor


Currently ListHDFS tracks two properties in state management, 
"listing.timestamp" and "emitted.timestamp". In the 1.0.0 release, the 
directory property now supports expression language which means the directory 
being listed could dynamically change on any execution of the processors. The 
processor should store state specific to the directory that was listed, for 
example "listing.timestamp.dir1" and "emitted.timestamp.dir1".

This would also help in a clustered scenario... currently ListHDFS has to be 
run on primary node only, otherwise each node will be overwriting each others 
state and producing unexpected results. With the above improvement, if the 
directory evaluated to a unique path for each node, it would store the state of 
each of those path separately.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2853) Improve ListHDFS state tracking

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende updated NIFI-2853:
--
Description: 
Currently ListHDFS tracks two properties in state management, 
"listing.timestamp" and "emitted.timestamp". In the 1.0.0 release, the 
directory property now supports expression language which means the directory 
being listed could dynamically change on any execution of the processor. The 
processor should store state specific to the directory that was listed, for 
example "listing.timestamp.dir1" and "emitted.timestamp.dir1".

This would also help in a clustered scenario... currently ListHDFS has to be 
run on primary node only, otherwise each node will be overwriting each others 
state and producing unexpected results. With the above improvement, if the 
directory evaluated to a unique path for each node, it would store the state of 
each of those path separately.

  was:
Currently ListHDFS tracks two properties in state management, 
"listing.timestamp" and "emitted.timestamp". In the 1.0.0 release, the 
directory property now supports expression language which means the directory 
being listed could dynamically change on any execution of the processors. The 
processor should store state specific to the directory that was listed, for 
example "listing.timestamp.dir1" and "emitted.timestamp.dir1".

This would also help in a clustered scenario... currently ListHDFS has to be 
run on primary node only, otherwise each node will be overwriting each others 
state and producing unexpected results. With the above improvement, if the 
directory evaluated to a unique path for each node, it would store the state of 
each of those path separately.


> Improve ListHDFS state tracking
> ---
>
> Key: NIFI-2853
> URL: https://issues.apache.org/jira/browse/NIFI-2853
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.0.0
>Reporter: Bryan Bende
>Priority: Minor
>
> Currently ListHDFS tracks two properties in state management, 
> "listing.timestamp" and "emitted.timestamp". In the 1.0.0 release, the 
> directory property now supports expression language which means the directory 
> being listed could dynamically change on any execution of the processor. The 
> processor should store state specific to the directory that was listed, for 
> example "listing.timestamp.dir1" and "emitted.timestamp.dir1".
> This would also help in a clustered scenario... currently ListHDFS has to be 
> run on primary node only, otherwise each node will be overwriting each others 
> state and producing unexpected results. With the above improvement, if the 
> directory evaluated to a unique path for each node, it would store the state 
> of each of those path separately.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1088: NIFI-2841 Refactoring logic in SplitAvro RecordSpli...

[bbende]

GitHub user bbende opened a pull request:

https://github.com/apache/nifi/pull/1088

NIFI-2841 Refactoring logic in SplitAvro RecordSplitter to avoid maki…

…ng two calls in a row to reader.hasNext()

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bbende/nifi NIFI-2841

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1088.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1088


commit b46e700d17240e7fe1f589ef7ca9b41e1a86a2bb
Author: Bryan Bende 
Date:   2016-09-29T17:49:27Z

NIFI-2841 Refactoring logic in SplitAvro RecordSplitter to avoid making two 
calls in a row to reader.hasNext()




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2841) SplitAvro Processor is Broken

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542599#comment-15542599
 ] 

ASF GitHub Bot commented on NIFI-2841:
--

GitHub user bbende opened a pull request:

https://github.com/apache/nifi/pull/1088

NIFI-2841 Refactoring logic in SplitAvro RecordSplitter to avoid maki…

…ng two calls in a row to reader.hasNext()

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bbende/nifi NIFI-2841

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1088.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1088


commit b46e700d17240e7fe1f589ef7ca9b41e1a86a2bb
Author: Bryan Bende 
Date:   2016-09-29T17:49:27Z

NIFI-2841 Refactoring logic in SplitAvro RecordSplitter to avoid making two 
calls in a row to reader.hasNext()




> SplitAvro Processor is Broken
> -
>
> Key: NIFI-2841
> URL: https://issues.apache.org/jira/browse/NIFI-2841
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: David Hicks
>Assignee: Bryan Bende
>Priority: Critical
> Fix For: 1.1.0
>
> Attachments: NIFI-2841.patch
>
>
> This is largely the fault of the Avro DataFileStream reader, but it's making 
> the processor unusable.  The problem appears to occur when you make the 
> following series of calls (which happens because of the splitSize comparison):
> reader.next() -> returns last element
> reader.hasNext() -> returns false
> reader.hasNext() -> returns true
> reader.next() -> EOFException
> org.apache.nifi.processor.exception.ProcessException: IOException thrown from 
> SplitAvro[id=22e03ca4-0151-4474-92fc-040e1fe12ab9]: java.io.EOFException
>   at 
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2013)
>  ~[na:na]
>   at 
> org.apache.nifi.processors.avro.SplitAvro$RecordSplitter$1.process(SplitAvro.java:250)
>  ~[nifi-avro-processors-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
>  ~[na:na]
>   at 
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1822)
>  ~[na:na]
>   at 
> org.apache.nifi.processors.avro.SplitAvro$RecordSplitter.split(SplitAvro.java:236)
>  ~[nifi-avro-processors-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.processors.avro.SplitAvro.onTrigger(SplitAvro.java:202) 
> ~[nifi-avro-processors-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>  [nifi-api-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_101]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_101]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_101]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_101]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_101]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_101]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.io.EOFException: null
>   at 
> org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473) 
> ~[avro-1.7.7.jar:1.7.7]
>   at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128) 
> ~[avro-1.7.7.jar:1.7.7]
>   at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:259) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:363)
>  ~[avro-1.7.7.jar:1.7.7]
>   at 
> 

[jira] [Commented] (NIFI-2841) SplitAvro Processor is Broken

[Bryan Bende (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542589#comment-15542589
 ] 

Bryan Bende commented on NIFI-2841:
---

Thanks for verifying the fix. I'll mark this as patch available then and see if 
anyone wants to review/commit it.

> SplitAvro Processor is Broken
> -
>
> Key: NIFI-2841
> URL: https://issues.apache.org/jira/browse/NIFI-2841
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: David Hicks
>Priority: Critical
> Fix For: 1.1.0
>
> Attachments: NIFI-2841.patch
>
>
> This is largely the fault of the Avro DataFileStream reader, but it's making 
> the processor unusable.  The problem appears to occur when you make the 
> following series of calls (which happens because of the splitSize comparison):
> reader.next() -> returns last element
> reader.hasNext() -> returns false
> reader.hasNext() -> returns true
> reader.next() -> EOFException
> org.apache.nifi.processor.exception.ProcessException: IOException thrown from 
> SplitAvro[id=22e03ca4-0151-4474-92fc-040e1fe12ab9]: java.io.EOFException
>   at 
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2013)
>  ~[na:na]
>   at 
> org.apache.nifi.processors.avro.SplitAvro$RecordSplitter$1.process(SplitAvro.java:250)
>  ~[nifi-avro-processors-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
>  ~[na:na]
>   at 
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1822)
>  ~[na:na]
>   at 
> org.apache.nifi.processors.avro.SplitAvro$RecordSplitter.split(SplitAvro.java:236)
>  ~[nifi-avro-processors-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.processors.avro.SplitAvro.onTrigger(SplitAvro.java:202) 
> ~[nifi-avro-processors-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>  [nifi-api-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_101]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_101]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_101]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_101]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_101]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_101]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.io.EOFException: null
>   at 
> org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473) 
> ~[avro-1.7.7.jar:1.7.7]
>   at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128) 
> ~[avro-1.7.7.jar:1.7.7]
>   at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:259) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:363)
>  ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:355)
>  ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:157) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193)
>  ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183)
>  ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142) 
> ~[avro-1.7.7.jar:1.7.7]
>   at org.apache.avro.file.DataFileStream.next(DataFileStream.java:233) 
> 

[jira] [Updated] (NIFI-2841) SplitAvro Processor is Broken

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende updated NIFI-2841:
--
Fix Version/s: 1.1.0

> SplitAvro Processor is Broken
> -
>
> Key: NIFI-2841
> URL: https://issues.apache.org/jira/browse/NIFI-2841
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: David Hicks
>Priority: Critical
> Fix For: 1.1.0
>
> Attachments: NIFI-2841.patch
>
>
> This is largely the fault of the Avro DataFileStream reader, but it's making 
> the processor unusable.  The problem appears to occur when you make the 
> following series of calls (which happens because of the splitSize comparison):
> reader.next() -> returns last element
> reader.hasNext() -> returns false
> reader.hasNext() -> returns true
> reader.next() -> EOFException
> org.apache.nifi.processor.exception.ProcessException: IOException thrown from 
> SplitAvro[id=22e03ca4-0151-4474-92fc-040e1fe12ab9]: java.io.EOFException
>   at 
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2013)
>  ~[na:na]
>   at 
> org.apache.nifi.processors.avro.SplitAvro$RecordSplitter$1.process(SplitAvro.java:250)
>  ~[nifi-avro-processors-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
>  ~[na:na]
>   at 
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1822)
>  ~[na:na]
>   at 
> org.apache.nifi.processors.avro.SplitAvro$RecordSplitter.split(SplitAvro.java:236)
>  ~[nifi-avro-processors-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.processors.avro.SplitAvro.onTrigger(SplitAvro.java:202) 
> ~[nifi-avro-processors-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>  [nifi-api-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_101]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_101]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_101]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_101]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_101]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_101]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.io.EOFException: null
>   at 
> org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473) 
> ~[avro-1.7.7.jar:1.7.7]
>   at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128) 
> ~[avro-1.7.7.jar:1.7.7]
>   at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:259) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:363)
>  ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:355)
>  ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:157) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193)
>  ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183)
>  ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142) 
> ~[avro-1.7.7.jar:1.7.7]
>   at org.apache.avro.file.DataFileStream.next(DataFileStream.java:233) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.nifi.processors.avro.SplitAvro$RecordSplitter$1$1.process(SplitAvro.java:259)
>  

[jira] [Updated] (NIFI-2826) GetEventHubProcessor Support Enqueue Time

[Joseph Percivall (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Percivall updated NIFI-2826:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Resolved with this commit: 
https://github.com/apache/nifi/commit/34e5a5321ab5e5d02e22932f115be982b3b5b304

> GetEventHubProcessor Support Enqueue Time
> -
>
> Key: NIFI-2826
> URL: https://issues.apache.org/jira/browse/NIFI-2826
> Project: Apache NiFi
>  Issue Type: Improvement
> Environment: centos6, jdk1.8, nifi 1.0.0
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 
> 0001-NIFI-2826-Adding-enqueue-time-to-GetAzureEventHub-pr.patch, 
> 0002-NIFI-2826-converted-from-epoch-to-iso8061-enqueue-in.patch, 
> 0003-NIFI-2826-Adding-enqueue-time-to-GetAzureEventHub-pr.patch, patch.1
>
>
> The current GetEventHubProcessor should support allowing an enqueue time like 
> eventhubs to read messages from the past. This would also support reading 
> messages missed during downtime. Today the behaviour is for the processor to 
> start at the enqueue of when it was started by the user missing any past 
> messages. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2826) GetEventHubProcessor Support Enqueue Time

[Joseph Percivall (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542565#comment-15542565
 ] 

Joseph Percivall commented on NIFI-2826:


Hey [~josephxsxn] you had a couple check style issues (revealed by running "mvn 
clean install -Pcontrib-check") but I fixed them and amended your commit.

+1

Visually verified code and did a contrib check build (passed after my fix was 
added). In a standealone instance, tested using Azure Event hub and IoT hub. 
Properly saw data come in as it was produced and also saw historical data come 
in when the enqueue time was set. Thanks for the contribution Joe! I look 
forward to reviewing more of your commits in the future. I will merge it in.

> GetEventHubProcessor Support Enqueue Time
> -
>
> Key: NIFI-2826
> URL: https://issues.apache.org/jira/browse/NIFI-2826
> Project: Apache NiFi
>  Issue Type: Improvement
> Environment: centos6, jdk1.8, nifi 1.0.0
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 
> 0001-NIFI-2826-Adding-enqueue-time-to-GetAzureEventHub-pr.patch, 
> 0002-NIFI-2826-converted-from-epoch-to-iso8061-enqueue-in.patch, 
> 0003-NIFI-2826-Adding-enqueue-time-to-GetAzureEventHub-pr.patch, patch.1
>
>
> The current GetEventHubProcessor should support allowing an enqueue time like 
> eventhubs to read messages from the past. This would also support reading 
> messages missed during downtime. Today the behaviour is for the processor to 
> start at the enqueue of when it was started by the user missing any past 
> messages. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2826) GetEventHubProcessor Support Enqueue Time

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542568#comment-15542568
 ] 

ASF subversion and git services commented on NIFI-2826:
---

Commit 34e5a5321ab5e5d02e22932f115be982b3b5b304 in nifi's branch 
refs/heads/master from [~josephxsxn]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=34e5a53 ]

NIFI-2826 Adding enqueue time to GetAzureEventHub processor

Signed-off-by: jpercivall 


> GetEventHubProcessor Support Enqueue Time
> -
>
> Key: NIFI-2826
> URL: https://issues.apache.org/jira/browse/NIFI-2826
> Project: Apache NiFi
>  Issue Type: Improvement
> Environment: centos6, jdk1.8, nifi 1.0.0
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 
> 0001-NIFI-2826-Adding-enqueue-time-to-GetAzureEventHub-pr.patch, 
> 0002-NIFI-2826-converted-from-epoch-to-iso8061-enqueue-in.patch, 
> 0003-NIFI-2826-Adding-enqueue-time-to-GetAzureEventHub-pr.patch, patch.1
>
>
> The current GetEventHubProcessor should support allowing an enqueue time like 
> eventhubs to read messages from the past. This would also support reading 
> messages missed during downtime. Today the behaviour is for the processor to 
> start at the enqueue of when it was started by the user missing any past 
> messages. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1083


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542570#comment-15542570
 ] 

ASF GitHub Bot commented on NIFI-2709:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1083


> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft 
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML 
> messages with no RFC2822 headers. I have also tried setting Exchange to force 
> emails to be text only, sadly only the body with some Content-Type: fields 
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used 
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in 
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format

[Andre (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andre updated NIFI-2709:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft 
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML 
> messages with no RFC2822 headers. I have also tried setting Exchange to force 
> emails to be text only, sadly only the body with some Content-Type: fields 
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used 
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in 
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to genera...

[trixpan]

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1083
  
@olegz thanks.

I wasn't sure if the comment was referring to the Java 8 or the whole 
thing. Thanks for clarifying and thank you for reviewing.

Cheers


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542544#comment-15542544
 ] 

ASF GitHub Bot commented on NIFI-2709:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1083
  
@olegz thanks.

I wasn't sure if the comment was referring to the Java 8 or the whole 
thing. Thanks for clarifying and thank you for reviewing.

Cheers


> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft 
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML 
> messages with no RFC2822 headers. I have also tried setting Exchange to force 
> emails to be text only, sadly only the body with some Content-Type: fields 
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used 
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in 
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542536#comment-15542536
 ] 

ASF GitHub Bot commented on NIFI-2709:
--

Github user olegz commented on the issue:

https://github.com/apache/nifi/pull/1083
  
@trixpan I thought I replied (3 hrs ago). Anyway it may not have been 
clear. I am ok, all is good.


> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft 
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML 
> messages with no RFC2822 headers. I have also tried setting Exchange to force 
> emails to be text only, sadly only the body with some Content-Type: fields 
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used 
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in 
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to genera...

[olegz]

Github user olegz commented on the issue:

https://github.com/apache/nifi/pull/1083
  
@trixpan I thought I replied (3 hrs ago). Anyway it may not have been 
clear. I am ok, all is good.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to genera...

[trixpan]

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1083
  
@olegz any feedback or :+1: ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542533#comment-15542533
 ] 

ASF GitHub Bot commented on NIFI-2709:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1083
  
@olegz any feedback or :+1: ?


> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft 
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML 
> messages with no RFC2822 headers. I have also tried setting Exchange to force 
> emails to be text only, sadly only the body with some Content-Type: fields 
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used 
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in 
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (NIFI-1794) For rules that have very long names, the name extends beyond the size of the delete confirmation window

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan reassigned NIFI-1794:
-

Assignee: Scott Aslan

> For rules that have very long names, the name extends beyond the size of the 
> delete confirmation window
> ---
>
> Key: NIFI-1794
> URL: https://issues.apache.org/jira/browse/NIFI-1794
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 0.6.0
>Reporter: Andrew Lim
>Assignee: Scott Aslan
>Priority: Minor
> Attachments: NIFI-1794_deleteLongRule.png
>
>
> 1.  Create a rule with a very long name.
> 2.  Select "x" next to the rule to delete it.
> 3.  In the Delete Confirmation pop-up window, the name of the rule extends 
> beyond the window (and likely beyond the rules window itself).
> Screenshot attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2841) SplitAvro Processor is Broken

[David Hicks (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542468#comment-15542468
 ] 

David Hicks commented on NIFI-2841:
---

Good and bad:
1) The bad is that I can't figure out how to get a file that isn't sensitive 
that this fails on.
2) The good is that I confirmed your patch works.  Thanks for the fast 
turnaround on that.


> SplitAvro Processor is Broken
> -
>
> Key: NIFI-2841
> URL: https://issues.apache.org/jira/browse/NIFI-2841
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: David Hicks
>Priority: Critical
> Attachments: NIFI-2841.patch
>
>
> This is largely the fault of the Avro DataFileStream reader, but it's making 
> the processor unusable.  The problem appears to occur when you make the 
> following series of calls (which happens because of the splitSize comparison):
> reader.next() -> returns last element
> reader.hasNext() -> returns false
> reader.hasNext() -> returns true
> reader.next() -> EOFException
> org.apache.nifi.processor.exception.ProcessException: IOException thrown from 
> SplitAvro[id=22e03ca4-0151-4474-92fc-040e1fe12ab9]: java.io.EOFException
>   at 
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2013)
>  ~[na:na]
>   at 
> org.apache.nifi.processors.avro.SplitAvro$RecordSplitter$1.process(SplitAvro.java:250)
>  ~[nifi-avro-processors-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
>  ~[na:na]
>   at 
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1822)
>  ~[na:na]
>   at 
> org.apache.nifi.processors.avro.SplitAvro$RecordSplitter.split(SplitAvro.java:236)
>  ~[nifi-avro-processors-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.processors.avro.SplitAvro.onTrigger(SplitAvro.java:202) 
> ~[nifi-avro-processors-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>  [nifi-api-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
>  [nifi-framework-core-0.7.0.jar:0.7.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_101]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_101]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_101]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_101]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_101]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_101]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.io.EOFException: null
>   at 
> org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473) 
> ~[avro-1.7.7.jar:1.7.7]
>   at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128) 
> ~[avro-1.7.7.jar:1.7.7]
>   at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:259) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:363)
>  ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:355)
>  ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:157) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193)
>  ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183)
>  ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142) 
> ~[avro-1.7.7.jar:1.7.7]
>   at 

[jira] [Assigned] (NIFI-2838) In the Advanced UI, if the Rule name is very long, the "x" to delete the rule becomes unavailable

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan reassigned NIFI-2838:
-

Assignee: Scott Aslan

> In the Advanced UI, if the Rule name is very long, the "x" to delete the rule 
> becomes unavailable
> -
>
> Key: NIFI-2838
> URL: https://issues.apache.org/jira/browse/NIFI-2838
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Andrew Lim
>Assignee: Scott Aslan
>Priority: Minor
>  Labels: UI
> Attachments: NIFI-2838_noDeleteIcon.png, NIFI-2838_twoXicons.png
>
>
> As shown in the attached screenshot, when the rule name is very long it 
> extends the whole width of the bar, so the "x" to delete the rule is no 
> longer selectable.
> Note: There is also a scroll bar that appears in this part of the UI, which I 
> thought would possibly reveal the "x", but the scroll bar doesn't appear to 
> add any value.  The width of the rule name bar doesn't extend the width of 
> the space.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2843) UI - Process Group Context Menu

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542411#comment-15542411
 ] 

ASF GitHub Bot commented on NIFI-2843:
--

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/1087

Not showing View Configuration context menu option for Process Groups

NIFI-2843:
- Removing the View Configuration menu item from the context menu on 
Process Groups.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-2843

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1087.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1087


commit 136da61a82d71bb57d3c10351009644042e03d5e
Author: Matt Gilman 
Date:   2016-09-30T14:34:43Z

NIFI-2843:
- Removing the View Configuration menu item from the context menu on 
Process Groups.




> UI - Process Group Context Menu
> ---
>
> Key: NIFI-2843
> URL: https://issues.apache.org/jira/browse/NIFI-2843
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.1.0
>
>
> The context menu for the Process Group lists both Configure and View 
> Configuration menu items. Investigate possibly removing the view 
> configuration dialog as the Configure dialog supports read-only mode when the 
> user does not have permissions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2843) UI - Process Group Context Menu

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-2843:
--
Fix Version/s: 1.1.0
   Status: Patch Available  (was: In Progress)

> UI - Process Group Context Menu
> ---
>
> Key: NIFI-2843
> URL: https://issues.apache.org/jira/browse/NIFI-2843
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.1.0
>
>
> The context menu for the Process Group lists both Configure and View 
> Configuration menu items. Investigate possibly removing the view 
> configuration dialog as the Configure dialog supports read-only mode when the 
> user does not have permissions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1087: Not showing View Configuration context menu option ...

[mcgilman]

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/1087

Not showing View Configuration context menu option for Process Groups

NIFI-2843:
- Removing the View Configuration menu item from the context menu on 
Process Groups.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-2843

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1087.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1087


commit 136da61a82d71bb57d3c10351009644042e03d5e
Author: Matt Gilman 
Date:   2016-09-30T14:34:43Z

NIFI-2843:
- Removing the View Configuration menu item from the context menu on 
Process Groups.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1086: Hiding the template upload dialog after form submis...

[mcgilman]

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/1086

Hiding the template upload dialog after form submission

NIFI-2837:
- Immediately hiding the template dialog after clicking the Upload button 
to prevent an accidental re-submission.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-2837

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1086.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1086


commit 19c27784ae51d889e132958d2fbe9af3af716116
Author: Matt Gilman 
Date:   2016-09-29T18:57:07Z

NIFI-2837:
- Immediately hiding the template dialog after clicking the Upload button 
to prevent an accidental re-submission.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-2837) UI - Template upload

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-2837:
--
Status: Patch Available  (was: In Progress)

> UI - Template upload
> 
>
> Key: NIFI-2837
> URL: https://issues.apache.org/jira/browse/NIFI-2837
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> When uploading a template, the form submit button should be removed once 
> clicked to prevent accidental re-submission.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (NIFI-2835) GetAzureEventHub processor should leverage partition offset to better handle restarts

[Joseph Niemiec (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Niemiec reassigned NIFI-2835:


Assignee: Joseph Niemiec

> GetAzureEventHub processor should leverage partition offset to better handle 
> restarts
> -
>
> Key: NIFI-2835
> URL: https://issues.apache.org/jira/browse/NIFI-2835
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Percivall
>Assignee: Joseph Niemiec
>
> The GetAzureEventHub processor utilizes the Azure client that consists of 
> receivers for each partition. The processor stores them in a map[1] that gets 
> cleared every time the processor is stopped[2]. These receivers have 
> partition offsets which keep track of which message it's currently on and 
> which it should receive next. So currently, when the processor is 
> stopped/restarted, any tracking of which message is next to be received is 
> lost.
> If instead of clearing the map each time, we hold onto the receivers, or kept 
> track of the partitionId/Offsets when stopping, (barring any relevant 
> configuration changes) the processor would restart exactly where it left off 
> with no loss of data.
> This would work very well with NIFI-2826.
> [1]https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/eventhub/GetAzureEventHub.java#L122
> [2] 
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/eventhub/GetAzureEventHub.java#L229



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format

[Andre (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andre updated NIFI-2709:

Fix Version/s: 1.1.0

> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft 
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML 
> messages with no RFC2822 headers. I have also tried setting Exchange to force 
> emails to be text only, sadly only the body with some Content-Type: fields 
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used 
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in 
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542224#comment-15542224
 ] 

ASF GitHub Bot commented on NIFI-2709:
--

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81533566
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
 ---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage, 
ProcessContext context, ProcessSessi
 long start = System.nanoTime();
 FlowFile flowFile = processSession.create();
 
-flowFile = processSession.append(flowFile, new 
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException 
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --

actually my bad, i've mistaken it with JMS. Indeed they were introduced for 
1.0 only, so all is good.


> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft 
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML 
> messages with no RFC2822 headers. I have also tried setting Exchange to force 
> emails to be text only, sadly only the body with some Content-Type: fields 
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used 
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in 
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to...

[olegz]

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81533566
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
 ---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage, 
ProcessContext context, ProcessSessi
 long start = System.nanoTime();
 FlowFile flowFile = processSession.create();
 
-flowFile = processSession.append(flowFile, new 
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException 
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --

actually my bad, i've mistaken it with JMS. Indeed they were introduced for 
1.0 only, so all is good.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to...

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81533345
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
 ---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage, 
ProcessContext context, ProcessSessi
 long start = System.nanoTime();
 FlowFile flowFile = processSession.create();
 
-flowFile = processSession.append(flowFile, new 
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException 
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --

@olegz happy to follow you lead but are you sure it isn't 1.0 only? 

If I recall correctly the email processors were being ironed out few days 
prior to 1.0 getting released.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to...

[olegz]

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81532172
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
 ---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage, 
ProcessContext context, ProcessSessi
 long start = System.nanoTime();
 FlowFile flowFile = processSession.create();
 
-flowFile = processSession.append(flowFile, new 
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException 
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --

The email support was introduced prior to 1.0, so if this patch only meant 
for 1.0, then it's fine, but at the moment the JIRA is not versioned and I 
assume the fix should be applied on both branches 0.* and master


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542193#comment-15542193
 ] 

ASF GitHub Bot commented on NIFI-2709:
--

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81532172
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
 ---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage, 
ProcessContext context, ProcessSessi
 long start = System.nanoTime();
 FlowFile flowFile = processSession.create();
 
-flowFile = processSession.append(flowFile, new 
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException 
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --

The email support was introduced prior to 1.0, so if this patch only meant 
for 1.0, then it's fine, but at the moment the JIRA is not versioned and I 
assume the fix should be applied on both branches 0.* and master


> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft 
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML 
> messages with no RFC2822 headers. I have also tried setting Exchange to force 
> emails to be text only, sadly only the body with some Content-Type: fields 
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used 
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in 
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542182#comment-15542182
 ] 

ASF GitHub Bot commented on NIFI-2709:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81531370
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
 ---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage, 
ProcessContext context, ProcessSessi
 long start = System.nanoTime();
 FlowFile flowFile = processSession.create();
 
-flowFile = processSession.append(flowFile, new 
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException 
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --

having said that, if necessary, happy to try to avoid using the lambda


> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft 
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML 
> messages with no RFC2822 headers. I have also tried setting Exchange to force 
> emails to be text only, sadly only the body with some Content-Type: fields 
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used 
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in 
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to...

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81531370
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
 ---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage, 
ProcessContext context, ProcessSessi
 long start = System.nanoTime();
 FlowFile flowFile = processSession.create();
 
-flowFile = processSession.append(flowFile, new 
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException 
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --

having said that, if necessary, happy to try to avoid using the lambda


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542180#comment-15542180
 ] 

ASF GitHub Bot commented on NIFI-2709:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81531262
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestConsumeEmail.java
 ---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.email;
+
+import com.icegreen.greenmail.user.GreenMailUser;
+import com.icegreen.greenmail.util.GreenMail;
+import com.icegreen.greenmail.util.ServerSetupTest;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.Assert;
+import org.springframework.integration.mail.AbstractMailReceiver;
+
+import javax.mail.Message;
+import javax.mail.MessagingException;
+import javax.mail.Session;
+import javax.mail.internet.InternetAddress;
+import javax.mail.internet.MimeMessage;
+import java.lang.reflect.Field;
+import java.util.List;
+import java.util.Properties;
+
+import static org.junit.Assert.assertEquals;
+
+
+public class TestConsumeEmail {
--- End diff --

The initial commit introduced a temporary test unit based on GreenMail, to 
avoid overlap, I gave it a different name. After user confirmed patch worked, I 
migrated some of the test you had created into the new jUnit.

This allowed me to run debug the two test units while I was developing the 
final iteration of the test unit.

I guess Test*.java was used was this naming convention used in other 
processors of the package, happy to rename back to the original if necessary.


> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft 
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML 
> messages with no RFC2822 headers. I have also tried setting Exchange to force 
> emails to be text only, sadly only the body with some Content-Type: fields 
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used 
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in 
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to...

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81531262
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestConsumeEmail.java
 ---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.email;
+
+import com.icegreen.greenmail.user.GreenMailUser;
+import com.icegreen.greenmail.util.GreenMail;
+import com.icegreen.greenmail.util.ServerSetupTest;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.Assert;
+import org.springframework.integration.mail.AbstractMailReceiver;
+
+import javax.mail.Message;
+import javax.mail.MessagingException;
+import javax.mail.Session;
+import javax.mail.internet.InternetAddress;
+import javax.mail.internet.MimeMessage;
+import java.lang.reflect.Field;
+import java.util.List;
+import java.util.Properties;
+
+import static org.junit.Assert.assertEquals;
+
+
+public class TestConsumeEmail {
--- End diff --

The initial commit introduced a temporary test unit based on GreenMail, to 
avoid overlap, I gave it a different name. After user confirmed patch worked, I 
migrated some of the test you had created into the new jUnit.

This allowed me to run debug the two test units while I was developing the 
final iteration of the test unit.

I guess Test*.java was used was this naming convention used in other 
processors of the package, happy to rename back to the original if necessary.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542171#comment-15542171
 ] 

ASF GitHub Bot commented on NIFI-2709:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81530739
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
 ---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage, 
ProcessContext context, ProcessSessi
 long start = System.nanoTime();
 FlowFile flowFile = processSession.create();
 
-flowFile = processSession.append(flowFile, new 
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException 
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --

NiFi 1.0 requires Java 8 or above?


> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft 
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML 
> messages with no RFC2822 headers. I have also tried setting Exchange to force 
> emails to be text only, sadly only the body with some Content-Type: fields 
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used 
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in 
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to...

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81530739
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
 ---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage, 
ProcessContext context, ProcessSessi
 long start = System.nanoTime();
 FlowFile flowFile = processSession.create();
 
-flowFile = processSession.append(flowFile, new 
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException 
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --

NiFi 1.0 requires Java 8 or above?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to...

[olegz]

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81530413
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestConsumeEmail.java
 ---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.email;
+
+import com.icegreen.greenmail.user.GreenMailUser;
+import com.icegreen.greenmail.util.GreenMail;
+import com.icegreen.greenmail.util.ServerSetupTest;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.Assert;
+import org.springframework.integration.mail.AbstractMailReceiver;
+
+import javax.mail.Message;
+import javax.mail.MessagingException;
+import javax.mail.Session;
+import javax.mail.internet.InternetAddress;
+import javax.mail.internet.MimeMessage;
+import java.lang.reflect.Field;
+import java.util.List;
+import java.util.Properties;
+
+import static org.junit.Assert.assertEquals;
+
+
+public class TestConsumeEmail {
--- End diff --

Why was the name of the class changed?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to...

[olegz]

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81530213
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
 ---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage, 
ProcessContext context, ProcessSessi
 long start = System.nanoTime();
 FlowFile flowFile = processSession.create();
 
-flowFile = processSession.append(flowFile, new 
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException 
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --

This makes it incompatible with Java 7


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542168#comment-15542168
 ] 

ASF GitHub Bot commented on NIFI-2709:
--

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81530213
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
 ---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage, 
ProcessContext context, ProcessSessi
 long start = System.nanoTime();
 FlowFile flowFile = processSession.create();
 
-flowFile = processSession.append(flowFile, new 
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException 
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --

This makes it incompatible with Java 7


> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft 
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML 
> messages with no RFC2822 headers. I have also tried setting Exchange to force 
> emails to be text only, sadly only the body with some Content-Type: fields 
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used 
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in 
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542167#comment-15542167
 ] 

ASF GitHub Bot commented on NIFI-2709:
--

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1083#discussion_r81530413
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestConsumeEmail.java
 ---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.email;
+
+import com.icegreen.greenmail.user.GreenMailUser;
+import com.icegreen.greenmail.util.GreenMail;
+import com.icegreen.greenmail.util.ServerSetupTest;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.Assert;
+import org.springframework.integration.mail.AbstractMailReceiver;
+
+import javax.mail.Message;
+import javax.mail.MessagingException;
+import javax.mail.Session;
+import javax.mail.internet.InternetAddress;
+import javax.mail.internet.MimeMessage;
+import java.lang.reflect.Field;
+import java.util.List;
+import java.util.Properties;
+
+import static org.junit.Assert.assertEquals;
+
+
+public class TestConsumeEmail {
--- End diff --

Why was the name of the class changed?


> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft 
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML 
> messages with no RFC2822 headers. I have also tried setting Exchange to force 
> emails to be text only, sadly only the body with some Content-Type: fields 
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used 
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in 
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2848) Queues aren't fairly drained when leading to a single component

[Joseph Gresock (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542104#comment-15542104
 ] 

Joseph Gresock commented on NIFI-2848:
--

As indicated in NIFI-2751, this only applies when connections go to a processor 
that uses processSession.get(int) or processSession.get(FlowFileFilter).  I 
tried it with RouteOnAttribute (which uses processSession.get()), and it does 
round robin correctly.

> Queues aren't fairly drained when leading to a single component
> ---
>
> Key: NIFI-2848
> URL: https://issues.apache.org/jira/browse/NIFI-2848
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Joseph Gresock
> Attachments: Backpressure_prioritization_test.xml, dominate_1.png, 
> dominate_2.png, queue_drain.png
>
>
> Consider the scenario where multiple queues lead to a single component and 
> all of them are full due to back pressure.  With the attached template, it is 
> easily observable that once a single queue starts to drain due to relieved 
> back pressure, it will continue to drain as long as it has incoming flow 
> files.  This means that if there's a constant flow of incoming flow files to 
> this queue, the other queues will never be drained (at least, that's my 
> theory based on several hours of observation).
> To reproduce this: 
> # Load the template into NiFi 1.0.0
> # Play all three GenerateFlowFile processors, but not the UpdateAttribute 
> processor (this simulates backpressure).  Wait until each queue has 1,000 
> flow files (max backpressure)
> # Stop the GenerateFlowFile processors, and play the UpdateAttribute 
> processor (this relieves the backpressure)
> # Observe which queue has started to drain, and start its GenerateFlowFile 
> processor
> # Observe that the other two queues remain full indefinitely, while the 
> draining queue continues to replenish and be drained indefinitely



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (NIFI-2728) Travis CI experiences GC overhead limit exceeded

[Andre (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andre resolved NIFI-2728.
-
Resolution: Fixed
  Assignee: Andre

> Travis CI experiences GC overhead limit exceeded 
> -
>
> Key: NIFI-2728
> URL: https://issues.apache.org/jira/browse/NIFI-2728
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Andre
>Assignee: Andre
>
> travis-ci is currently broken and frequently reporting 
> {{GC overhead limit exceeded}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2448) Hive Processors depend on too recent a Hive version

[Tomasz Domanski (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15541902#comment-15541902
 ] 

Tomasz Domanski commented on NIFI-2448:
---

Why is this ticket marked as superceded by  NIFI-1868 Add support for Hive 
Streaming?

Hive Streaming is one thing, and if you would like to use this processor than 
it is great.
However if you would like to use processors like:

 * SelectHiveQL
 * PutHiveQL
 
They are using Hive Database Connection Pooling Service and this is not working.

So I think this ticket is not resolved by Hive Streaming.
I have now version 1.0.0 installed and I'm not able to connect to hive instance 
on fresh Cloudera Instance (which has Hive 1.1).

> Hive Processors depend on too recent a Hive version
> ---
>
> Key: NIFI-2448
> URL: https://issues.apache.org/jira/browse/NIFI-2448
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Simon Elliston Ball
>Priority: Critical
>
> The new Hive bundle depends on version 2.0.0 of Hive. This means that it can 
> only connect to very recent Hive distributions. 
> Sadly very few people in the field have upgraded their Hive to the latest and 
> greatest, and as per https://issues.apache.org/jira/browse/HIVE-6050 the 
> issue of backward compatibility in the client is still not resolved.
> We should look at lowering the dependency version to allow connections with 
> older Hive distros.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)