[
https://issues.apache.org/jira/browse/FLINK-14569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964064#comment-16964064
]
kk commented on FLINK-14569:
----------------------------
thank you very much, use the api
"env.readCsvFile(path).fieldDelimiter(fieldDeli).lineDelimiter(lineDeli).types(...)"
or use the "new
CsvAppendTableSourceFactory().createTableSource(false,targetTableDescriptor.toProperties())",
then print the dataset ,get then data alwasys lose 1
record(total:18records,Parallelism:24),but use the api
"env.readTextFile(path)", can get the right result!
> use flink to deal with batch task,when parallelism granter than 1, always 1
> or 2 records loss
> ---------------------------------------------------------------------------------------------
>
> Key: FLINK-14569
> URL: https://issues.apache.org/jira/browse/FLINK-14569
> Project: Flink
> Issue Type: Bug
> Components: API / DataSet
> Affects Versions: 1.7.2
> Environment: hadoop:2.8.5
> flink: 1.7.1
> node machines: 3
> deploy: yarn
> Reporter: kk
> Priority: Critical
>
> when flink read from hdfs file and set the parallelism>1, occasionally， 1 or
> 2 records lose, however the hdfs file bigger or smaller.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
