damccorm opened a new issue, #21620: URL: https://github.com/apache/beam/issues/21620
When a snowflake string column contains new-line, the snowflakeIO fails to read it as a record. Instead two invalid records gets created. I see the issue due to the usage of TextIO; as the exported extract from the snowflake contains the column data enclosed with quoted string, making it a valid CSV. Possibly be solved if FileIO(with custom splitable dofn to handle big file) is used instead of TextIO. Imported from Jira [BEAM-14526](https://issues.apache.org/jira/browse/BEAM-14526). Original Jira may contain additional context. Reported by: Varun Rauthan. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
