[
https://issues.apache.org/jira/browse/NIFI-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15291190#comment-15291190
]
Bryan Bende commented on NIFI-1902:
-----------------------------------
It seems like there could be two possible solutions, or maybe both... One would
be to not force the ".sf" suffix to be added to filename, and if someone wants
that suffix then they can set the filename using UpdateAttribute. The other
option would be to not use filename as the key... we could have another
property like "Key Attribute" and the value would be the name of the attribute
to use as the key. This way you can still set filename to end in ".sf" and the
key can be something else.
Seems like maybe both of these are a good idea.
> CreateHadoopSequenceFile forces .sf suffix into filename and key
> ----------------------------------------------------------------
>
> Key: NIFI-1902
> URL: https://issues.apache.org/jira/browse/NIFI-1902
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 0.6.1
> Reporter: Bryan Bende
> Priority: Minor
>
> CreateHadoopSequenceFile always adds .sf to the end of the filename:
> <https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/CreateHadoopSequenceFile.java>https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/CreateHadoopSequenceFile.java
> 155 final String fileName =
> flowFile.getAttribute(CoreAttributes.FILENAME.key()) + ".sf";
> 156 flowFile = session.putAttribute(flowFile,
> CoreAttributes.FILENAME.key(), fileName);
> 157 try {
> 158 flowFile = sequenceFileWriter.writeSequenceFile(flowFile,
> session, getConfiguration(), compressionType);
> 159 session.transfer(flowFile, RELATIONSHIP_SUCCESS);
> 160 getLogger().info("Transferred flowfile {} to {}", new
> Object[]{flowFile, RELATIONSHIP_SUCCESS});
> 161 } catch (ProcessException e) {
> 162 getLogger().error("Failed to create Sequence File.
> Transferring {} to 'failure'", new Object[]{flowFile}, e);
> 163 session.transfer(flowFile, RELATIONSHIP_FAILURE);
> 164 }
> Then uses the filename as the key when writing the sequence file:
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/SequenceFileWriterImpl.java
> 117 String key = flowFile.getAttribute(CoreAttributes.FILENAME.key());
> 118 writer.append(new Text(key), inStreamWritable
> There should be a way to control the key so that it doesn't have to have the
> .sf suffix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
