liyafan82 opened a new pull request #8597: URL: https://github.com/apache/arrow/pull/8597
According to the current implementation of JDBC adapter, the conversion between SQL types and Arrow types is hard-coded. This will cause some problems in practice: 1. The appropriate conversion may vary for different databases. For example, for SQL Server, type real corresponds to 4 byte floating point values (https://docs.microsoft.com/en-us/sql/t-sql/data-types/float-and-real-transact-sql?view=sql-server-ver15), whereas for SQLite, real corresponds to 8 byte floating point values (https://www.sqlitetutorial.net/sqlite-data-types/). If the maping is not right, some extra conversion would happen, which can impact performance severely. 2. Our current implementation determines the type conversion solely based on the type ID. However, the appropriate conversion may also depend some other information, like precision and scale. For example, for FLOAT( n ), it should correspond to 4 byte floating point values, if n <= 24, otherwise, it should correspond to 8 byte floating point values. To address the problems, we should allow users to customize the conversion between SQL and Arrow types. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
