[
https://issues.apache.org/jira/browse/NIFI-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896390#comment-15896390
]
ASF GitHub Bot commented on NIFI-3204:
--------------------------------------
Github user joewitt commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1561#discussion_r104317162
--- Diff:
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/DeleteHDFS.java
---
@@ -141,20 +141,33 @@ public void onTrigger(ProcessContext context,
ProcessSession session) throws Pro
}
Map<String, String> attributes =
Maps.newHashMapWithExpectedSize(2);
+ boolean foundMissingFile = false;
for (Path path : pathList) {
attributes.put("filename", path.getName());
attributes.put("path", path.getParent().toString());
if (fileSystem.exists(path)) {
fileSystem.delete(path,
context.getProperty(RECURSIVE).asBoolean());
+
if (!context.hasIncomingConnection()) {
flowFile = session.create();
+
session.transfer(session.putAllAttributes(flowFile, attributes), REL_SUCCESS);
}
- session.transfer(session.putAllAttributes(flowFile,
attributes), REL_SUCCESS);
+
} else {
getLogger().warn("File (" + path + ") does not exist");
+
if (!context.hasIncomingConnection()) {
flowFile = session.create();
+
session.transfer(session.putAllAttributes(flowFile, attributes), REL_FAILURE);
}
+
+ }
+ }
+ if (context.hasIncomingConnection()) {
+ // TODO we only put the last path deleted, change the
semantic of the processor?
--- End diff --
We should remove this 'TODO' and instead document the behavior you're
describing on the capability decription/and/or the WritesAttribute section. If
it is important for a user to see a listing of deleted items they should not
use the wildcard here but rather use ListHDFS to create an explicit set of
files and this processor should be honoring precisely the items specified.
> delete hdfs processor throws an error stating transfer relationship not
> specified even when all relationships are present
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-3204
> URL: https://issues.apache.org/jira/browse/NIFI-3204
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.0.0, 1.1.0
> Reporter: Arpit Gupta
>
> Following flow was setup
> get file -> extract text -> delete hdfs
> A bunch of files were written each having one line which was the path to
> delete. Some of these path's were files, some were directories and some were
> patterns. Extract text would extract the line and assign to an attribute
> which delete hdfs would use to populate the path to delete.
> However the processor would run into an error when ever it tried to process
> the path which was a pattern matching multiple paths.
> {code}
> 2016-12-14 11:32:43,335 ERROR [Timer-Driven Process Thread-7]
> o.a.nifi.processors.hadoop.DeleteHDFS
> DeleteHDFS[id=fed0acf6-0158-1000-b7ab-8cc724e4142d]
> DeleteHDFS[id=fed0acf6-0158-1000-b7ab-8cc724e4142d] failed to process session
> due to org.apache.nifi.processor.exception
> .FlowFileHandlingException:
> StandardFlowFileRecord[uuid=af8be94a-e527-4203-bb87-a0115f84e582,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1481656798897-1, container=default,
> section=1], offset=6518, length=75],offset=0,name=noyg3p7km8.txt,size=75] tr
> ansfer relationship not specified:
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=af8be94a-e527-4203-bb87-a0115f84e582,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1481656798897-1, container=default,
> sectio
> n=1], offset=6518, length=75],offset=0,name=noyg3p7km8.txt,size=75] transfer
> relationship not specified
> 2016-12-14 11:32:43,335 ERROR [Timer-Driven Process Thread-7]
> o.a.nifi.processors.hadoop.DeleteHDFS
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=af8be94a-e527-4203-bb87-a0115f84e582,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1481656798897-1, container=default,
> section=1], offset=6518, length=75],offse
> t=0,name=noyg3p7km8.txt,size=75] transfer relationship not specified
> at
> org.apache.nifi.controller.repository.StandardProcessSession.checkpoint(StandardProcessSession.java:234)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:304)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:28)
> ~[nifi-api-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
> [nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> [nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> [nifi-framework-core-1.1.0.jar:1.1.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_92]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_92]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_92]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_92]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_92]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_92]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_92]
> 2016-12-14 11:32:43,335 WARN [Timer-Driven Process Thread-7]
> o.a.nifi.processors.hadoop.DeleteHDFS
> DeleteHDFS[id=fed0acf6-0158-1000-b7ab-8cc724e4142d] Processor
> Administratively Yielded for 1 sec due to processing failure
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
