Adamyuanyuan commented on issue #10204: URL: https://github.com/apache/seatunnel/issues/10204#issuecomment-3667948001
I think we can consider implementing lightweight data validation on the Seatunnel side, allowing users to configure it optionally. For example, we can add an optional validation module after Source, Transform, and Sink. This module can provide capabilities similar to count(*). However, this can be dangerous in some scenarios. For instance, performing a count(*) on a large Hive table might crash the entire cluster, so this feature should be optional. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
