yashmayya opened a new issue, #13615: URL: https://github.com/apache/pinot/issues/13615
- https://github.com/apache/pinot/pull/11518 updated the [TypeFactory](https://github.com/apache/pinot/blob/3db93ccc210991ad91b6a5306b3b610d8f5e0507/pinot-query-planner/src/main/java/org/apache/pinot/query/type/TypeFactory.java#L43) (that is used to convert between Pinot's data types to Calcite SQL's types) to map Pinot's `FLOAT` data type to Calcite SQL's `DOUBLE` type. - This is incorrect since Pinot's `FLOAT` data type uses Java's `float` primitive type internally which is a 32 bit floating point type and should map to Calcite's `REAL` type instead (see https://calcite.apache.org/docs/reference.html#data-types). - This leads to precision issues where a simple query like `SELECT aFloatColumn FROM table` returns different values in the v1 and v2 query engines (can be verified via any of the quickstarts). The value returned in the multi-stage query engine is incorrect and won't match the value that was originally ingested. - The reason the original change was introduced was for queries like `SELECT COUNT(*) FROM table WHERE aFloatColumn = 0.05`. However, such queries should be written like `SELECT COUNT(*) FROM table WHERE aFloatColumn = CAST(0.05 AS FLOAT)` instead of incorrectly mapping Pinot's `FLOAT` data type. This is consistent with other standard databases like Postgres and MySQL. - In a query like `SELECT COUNT(*) FROM table WHERE aFloatColumn = 0.05`, the literal `0.05` is inferred as a `DECIMAL` type with fixed precision and scale. During query compilation, Calcite tries to unify the operand types in an expression using a principle of finding a consistent type across the operands [here](https://github.com/apache/calcite/blob/1e424a0d4eeec69e4da0a504cf080cf7a847704d/core/src/main/java/org/apache/calcite/sql2rel/StandardConvertletTable.java#L1187-L1197). For a `DECIMAL` / `REAL` combination, this comes out to be `DOUBLE` [here](https://github.com/apache/calcite/blob/1e424a0d4eeec69e4da0a504cf080cf7a847704d/core/src/main/java/org/apache/calcite/sql2rel/StandardConvertletTable.java#L1187-L1197) and thus there is an additional cast to `DOUBLE` added to the `FLOAT` column in the query plan. This can be avoided by explicitly casting the literal to a `FLOAT` type instead. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
