Alessandro D'Armiento created NIFI-6464:
-------------------------------------------
Summary: ListHDFS should support fragment attributes with
strategies
Key: NIFI-6464
URL: https://issues.apache.org/jira/browse/NIFI-6464
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Affects Versions: 1.9.2
Reporter: Alessandro D'Armiento
h2. Current Situation
ListHDFS doesn't support Fragmentation attributes
h2. Improvement Proposal
* Since the processor works on a 1:N semantic (1 input trigger flowfile, N
output flowfiles) it would be nice to support fragmentation attributes (for
example for subsequent merge operations)
** It would be also useful to support different fragmentation strategies, in
order to support multiple user cases. For example, it should be possible to
select:
*** A "one for all" fragmentation strategy which will create a single
fragmentation group. Therefore, all files will have the same
fragment.identifier, the same fragment.count, equal to the total number N of
listed files, and fragment.index ∈ [0, N).
*** A "per subdir" fragmentation strategy which will create different
fragmentation groups, one for each scanned subdirectory of the given path.
Therefore, for each subfolder, flowfiles will have a specific
fragment.identifier, fragment.count will be, for each flowfile, equal to the
number Ni of files in the i-th directory, and fragment.index ∈ [0, Ni).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
