rahul8383 commented on pull request #11581: URL: https://github.com/apache/beam/pull/11581#issuecomment-622308296
I agree that this is not a scalable solution. Providing a Calcite RelDataType Mapping for every Logical Type defined(which is the solution presented in this PR) by every IO is not scalable. Another approach to solving the problem is: Provide Calcite RelDataType Mapping depending on the Base Type defined in the logical type. But, as CHAR and VARCHAR logical types have Base Type as STRING, we have to choose a default Calcite RelDataType Mapping. The same thing applies to BINARY and VARBINARY logical types. We also have to choose a default RelDataType Mapping if the Base Type is DATETIME. By using this approach, I think we might be missing some built-in Aggregation functions provided by Calcite for specific types. I was also thinking if we can use the IDENTIFIER of the logical type to determine the corresponding Calcite RelDataType. But, as the IDENTIFIER type is String and not an enum, it cannot be used. For example, all the logical types defined by JdbcIO use java.sql.JDBCType name as the IDENTIFIER. Please correct me if my understanding is incorrect. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
