kanterov commented on pull request #12919: URL: https://github.com/apache/beam/pull/12919#issuecomment-699943118
@Vasu7052 Moving table schema logic into workers will turn a class of deployment errors into the runtime errors, that isn't always desired. From reading the JIRA ticket I can understand your motivation. As a middle-ground, probably you can add `tableSchema` as a nullable property to `ClickHouseIO.Write` that is going to be populated in the `ClickHouseIO.Write.Builder` unless specified explicitly? This way we can specify `TableSchema` in the pipeline graph when fetching from ClickHouse isn't possible. This also enables the future check if input PCollection schema is compatible with table schema, that isn't implemented yet, but possible with the current implementation, and not going to be possible if table schema is unknown during deployment. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
