[
https://issues.apache.org/jira/browse/NIFI-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15080634#comment-15080634
]
ASF GitHub Bot commented on NIFI-1118:
--------------------------------------
Github user joewitt commented on a diff in the pull request:
https://github.com/apache/nifi/pull/135#discussion_r48701229
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/SplitText.java
---
@@ -82,10 +84,16 @@
public static final PropertyDescriptor LINE_SPLIT_COUNT = new
PropertyDescriptor.Builder()
.name("Line Split Count")
- .description("The number of lines that will be added to each
split file")
+ .description("The number of lines that will be added to each
split file, excluding header lines")
.required(true)
.addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR)
.build();
+ public static final PropertyDescriptor FRAGMENT_MAX_SIZE = new
PropertyDescriptor.Builder()
+ .name("Maximum Fragment Size")
+ .description("The maximum size of each split file, including
header lines")
--- End diff --
Seems like there could be some interesting edge conditions. For instance
what if a given line is by itself larger than the max? Or, does the max get
exceeded by the size of at most one line then split or is the max exceeded then
a pushback mechanism goes to the previous line and does the split and then
continues? I might be overthinking this but in any case would be good to let
the user know how these edge cases are handled.
> Enable SplitText processor to limit line length and filter header lines
> -----------------------------------------------------------------------
>
> Key: NIFI-1118
> URL: https://issues.apache.org/jira/browse/NIFI-1118
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Mark Bean
> Assignee: Joe Skora
> Fix For: 0.5.0
>
>
> Include the following functionality to the SplitText processor:
> 1) Maximum size limit of the split file(s)
> A new split file will be created if the next line to be added to the current
> split file exceeds a user-defined maximum file size
> 2) Header line marker
> User-defined character(s) can be used to identify the header line(s) of the
> data file rather than a predetermined number of lines
> These changes are additions, not a replacement of any property or behavior.
> In the case of header line marker, the existing property "Header Line Count"
> must be zero for the new property and behavior to be used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
