Hisoka-X commented on code in PR #9608:
URL: https://github.com/apache/seatunnel/pull/9608#discussion_r2242682429
##########
seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/reader/TextReadStrategy.java:
##########
@@ -65,6 +66,106 @@ public class TextReadStrategy extends AbstractReadStrategy {
private int[] indexes;
private String encoding = FileBaseSourceOptions.ENCODING.defaultValue();
+ /** Custom stream divider for splitting text streams by specified
delimiters */
+ public static class StreamLineSplitter {
+ private final String delimiter;
Review Comment:
useless?
##########
seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/reader/TextReadStrategy.java:
##########
@@ -65,6 +66,106 @@ public class TextReadStrategy extends AbstractReadStrategy {
private int[] indexes;
private String encoding = FileBaseSourceOptions.ENCODING.defaultValue();
+ /** Custom stream divider for splitting text streams by specified
delimiters */
+ public static class StreamLineSplitter {
+ private final String delimiter;
+ private final char[] delimiterChars;
+ private final StringBuilder lineBuffer;
+ private int delimiterIndex;
+ private int skipCount;
+ private final long skipHeaderNumber;
+ private final LineProcessor lineProcessor;
+ private final boolean useReadLine;
+
+ public StreamLineSplitter(
+ String delimiter, long skipHeaderNumber, LineProcessor
lineProcessor) {
+ this.delimiter = delimiter;
+ this.delimiterChars = delimiter.toCharArray();
+ this.lineBuffer = new StringBuilder();
+ this.delimiterIndex = 0;
+ this.skipCount = 0;
+ this.skipHeaderNumber = skipHeaderNumber;
+ this.lineProcessor = lineProcessor;
+
+ this.useReadLine = isDefaultLineDelimiter(delimiter);
+ }
+
+ private boolean isDefaultLineDelimiter(String delimiter) {
+ return "\n".equals(delimiter) || "\r\n".equals(delimiter);
Review Comment:
<img width="863" height="411" alt="image"
src="https://github.com/user-attachments/assets/732a5609-a65c-4dea-a1cc-c6af044858f8";
/>
`\r` should be consider too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
