[jira] [Commented] (NIFI-4184) I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the PutHDFS Processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095927#comment-16095927
 ] 

ASF GitHub Bot commented on NIFI-4184:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2007


>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the 
> PutHDFS Processor
> --
>
> Key: NIFI-4184
> URL: https://issues.apache.org/jira/browse/NIFI-4184
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: dav
>
>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in order to 
> achieve it i put expressionLanguageSupported(true) on the PropertyDescriptor 
> of REMOTE_GROUP and REMOTE_OWNER



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4184) I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the PutHDFS Processor

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095925#comment-16095925
 ] 

ASF subversion and git services commented on NIFI-4184:
---

Commit d334532b169e53f18765eb20c93ebcee1ac7ade0 in nifi's branch 
refs/heads/master from Davide
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d334532 ]

NIFI-4184: Enabled EL on PutHDFS Remote Group and Owner

I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER, in order to 
achieve it i put expressionLanguageSupported(true) on the PropertyDescriptor of 
REMOTE_GROUP and REMOTE_OWNER

This closes #2007.

Signed-off-by: Davide 
Signed-off-by: Koji Kawamura 


>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the 
> PutHDFS Processor
> --
>
> Key: NIFI-4184
> URL: https://issues.apache.org/jira/browse/NIFI-4184
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: dav
>
>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in order to 
> achieve it i put expressionLanguageSupported(true) on the PropertyDescriptor 
> of REMOTE_GROUP and REMOTE_OWNER



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4184) I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the PutHDFS Processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095921#comment-16095921
 ] 

ASF GitHub Bot commented on NIFI-4184:
--

Github user ijokarumawak commented on the issue:

https://github.com/apache/nifi/pull/2007
  
AppVeyor build has been failing. There are many unit tests those are not 
working well on Windows. We've been trying to improve it but not completed.


>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the 
> PutHDFS Processor
> --
>
> Key: NIFI-4184
> URL: https://issues.apache.org/jira/browse/NIFI-4184
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: dav
>
>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in order to 
> achieve it i put expressionLanguageSupported(true) on the PropertyDescriptor 
> of REMOTE_GROUP and REMOTE_OWNER



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4184) I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the PutHDFS Processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095918#comment-16095918
 ] 

ASF GitHub Bot commented on NIFI-4184:
--

Github user panelladavide commented on the issue:

https://github.com/apache/nifi/pull/2007
  
Thanks for your help and for your patience! Why AppVeyor build failed? When 
Nifi release a new release?


>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the 
> PutHDFS Processor
> --
>
> Key: NIFI-4184
> URL: https://issues.apache.org/jira/browse/NIFI-4184
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: dav
>
>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in order to 
> achieve it i put expressionLanguageSupported(true) on the PropertyDescriptor 
> of REMOTE_GROUP and REMOTE_OWNER



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4184) I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the PutHDFS Processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095910#comment-16095910
 ] 

ASF GitHub Bot commented on NIFI-4184:
--

Github user ijokarumawak commented on the issue:

https://github.com/apache/nifi/pull/2007
  
@panelladavide LGTM, +1. I'm going to merge. Thanks for your contribution!


>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the 
> PutHDFS Processor
> --
>
> Key: NIFI-4184
> URL: https://issues.apache.org/jira/browse/NIFI-4184
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: dav
>
>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in order to 
> achieve it i put expressionLanguageSupported(true) on the PropertyDescriptor 
> of REMOTE_GROUP and REMOTE_OWNER



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4184) I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the PutHDFS Processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095843#comment-16095843
 ] 

ASF GitHub Bot commented on NIFI-4184:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2007#discussion_r128689940
  
--- Diff: 
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java
 ---
@@ -386,11 +388,14 @@ public void process(InputStream in) throws 
IOException {
 });
 }
 
-protected void changeOwner(final ProcessContext context, final 
FileSystem hdfs, final Path name) {
+protected void changeOwner(final ProcessContext context, final 
FileSystem hdfs, final Path name, final FlowFile flowFile) {
 try {
 // Change owner and group of file if configured to do so
-String owner = context.getProperty(REMOTE_OWNER).getValue();
-String group = context.getProperty(REMOTE_GROUP).getValue();
+//String owner = context.getProperty(REMOTE_OWNER).getValue();
+//String group = context.getProperty(REMOTE_GROUP).getValue();
+String owner = 
context.getProperty(REMOTE_OWNER).evaluateAttributeExpressions(flowFile).getValue();
+String group = 
context.getProperty(REMOTE_GROUP).evaluateAttributeExpressions(flowFile).getValue();
+
 if (owner != null || group != null) {
--- End diff --

When I tested with an expression that returns no value, e.g. specify an 
attribute name which does not exist, then the group string became an Empty 
string. And the file was updated with an empty group. To avoid that, Could you 
add following code before this if statement? to make empty string into null:

```
owner = owner == null || owner.isEmpty() ? null : owner;
group = group == null || group.isEmpty() ? null : group;
```


![image](https://user-images.githubusercontent.com/1107620/28451433-3b1718a2-6e28-11e7-9473-282bbfcdd881.png)



>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the 
> PutHDFS Processor
> --
>
> Key: NIFI-4184
> URL: https://issues.apache.org/jira/browse/NIFI-4184
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: dav
>
>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in order to 
> achieve it i put expressionLanguageSupported(true) on the PropertyDescriptor 
> of REMOTE_GROUP and REMOTE_OWNER



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4184) I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the PutHDFS Processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095845#comment-16095845
 ] 

ASF GitHub Bot commented on NIFI-4184:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2007#discussion_r128684447
  
--- Diff: 
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java
 ---
@@ -386,11 +388,14 @@ public void process(InputStream in) throws 
IOException {
 });
 }
 
-protected void changeOwner(final ProcessContext context, final 
FileSystem hdfs, final Path name) {
+protected void changeOwner(final ProcessContext context, final 
FileSystem hdfs, final Path name, final FlowFile flowFile) {
 try {
 // Change owner and group of file if configured to do so
-String owner = context.getProperty(REMOTE_OWNER).getValue();
-String group = context.getProperty(REMOTE_GROUP).getValue();
+//String owner = context.getProperty(REMOTE_OWNER).getValue();
+//String group = context.getProperty(REMOTE_GROUP).getValue();
--- End diff --

Please removed these old lines. We don't have to keep old code unless if 
there's a strong needs.


>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the 
> PutHDFS Processor
> --
>
> Key: NIFI-4184
> URL: https://issues.apache.org/jira/browse/NIFI-4184
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: dav
>
>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in order to 
> achieve it i put expressionLanguageSupported(true) on the PropertyDescriptor 
> of REMOTE_GROUP and REMOTE_OWNER



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4184) I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the PutHDFS Processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095844#comment-16095844
 ] 

ASF GitHub Bot commented on NIFI-4184:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2007#discussion_r128684359
  
--- Diff: 
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java
 ---
@@ -352,7 +354,7 @@ public void process(InputStream in) throws IOException {
 + " to its final filename");
 }
 
-changeOwner(context, hdfs, copyFile);
+changeOwner(context, hdfs, copyFile,flowFile);
--- End diff --

I just added a whitespace between arguments.


>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the 
> PutHDFS Processor
> --
>
> Key: NIFI-4184
> URL: https://issues.apache.org/jira/browse/NIFI-4184
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: dav
>
>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in order to 
> achieve it i put expressionLanguageSupported(true) on the PropertyDescriptor 
> of REMOTE_GROUP and REMOTE_OWNER



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4184) I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the PutHDFS Processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094518#comment-16094518
 ] 

ASF GitHub Bot commented on NIFI-4184:
--

Github user panelladavide commented on the issue:

https://github.com/apache/nifi/pull/2007
  
Thank you so much!


>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the 
> PutHDFS Processor
> --
>
> Key: NIFI-4184
> URL: https://issues.apache.org/jira/browse/NIFI-4184
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: dav
>
>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in order to 
> achieve it i put expressionLanguageSupported(true) on the PropertyDescriptor 
> of REMOTE_GROUP and REMOTE_OWNER



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4184) I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the PutHDFS Processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094103#comment-16094103
 ] 

ASF GitHub Bot commented on NIFI-4184:
--

Github user ijokarumawak commented on the issue:

https://github.com/apache/nifi/pull/2007
  
@panelladavide I don't know any reason for not supporting EL with 
REMOTE_OWNER/GROUP. I think applying EL at incoming FlowFile level is safe, and 
wouldn't break anything because it just sets owner for a specific path.

It would be problematic if it changes FileSystem or HdfsResources instance 
state those are reused from every onTrigger call. But again, setting owner 
should just affect the specified path, so it should be fine.

I guess it's just written that way at the initial contribution and haven't 
been requested until now. 


>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the 
> PutHDFS Processor
> --
>
> Key: NIFI-4184
> URL: https://issues.apache.org/jira/browse/NIFI-4184
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: dav
>
>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in order to 
> achieve it i put expressionLanguageSupported(true) on the PropertyDescriptor 
> of REMOTE_GROUP and REMOTE_OWNER



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4184) I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the PutHDFS Processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091373#comment-16091373
 ] 

ASF GitHub Bot commented on NIFI-4184:
--

Github user panelladavide commented on the issue:

https://github.com/apache/nifi/pull/2007
  
@ijokarumawak Thank you so much! 
As soon as I have time to code I will update my branch. 

I have a question. Why did not you enable the LE in PutHDFS REMOTE_OWNER 
and GROUP_OWNER? 
Is there a specific reason? 


>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the 
> PutHDFS Processor
> --
>
> Key: NIFI-4184
> URL: https://issues.apache.org/jira/browse/NIFI-4184
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: dav
>
>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in order to 
> achieve it i put expressionLanguageSupported(true) on the PropertyDescriptor 
> of REMOTE_GROUP and REMOTE_OWNER



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4184) I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the PutHDFS Processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090987#comment-16090987
 ] 

ASF GitHub Bot commented on NIFI-4184:
--

Github user ijokarumawak commented on the issue:

https://github.com/apache/nifi/pull/2007
  
Hi @panelladavide thanks for your contribution!

In order to enable Expression Language, setting expressionLanguageSupported 
to true is not enough. You also need to evaluate the configured EL. Plus, if 
you need to support EL to use FlowFile attribute, you need to pass a FlowFile 
(the incoming FlowFile in most cases) when EL is evaluated.

Specifically, you need to modify changeOwner method:
- Add EL evaluation like:
 ```
 String owner = 
context.getProperty(REMOTE_OWNER).evaluateAttributeExpressions(flowFile).getValue();
 ```
- Add FlowFile argument, so that it can be passed to 
`evaluateAttributeExpressions(flowFile)`


https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java#L389


>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in the 
> PutHDFS Processor
> --
>
> Key: NIFI-4184
> URL: https://issues.apache.org/jira/browse/NIFI-4184
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: dav
>
>  I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER in order to 
> achieve it i put expressionLanguageSupported(true) on the PropertyDescriptor 
> of REMOTE_GROUP and REMOTE_OWNER



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)