asolimando commented on PR #4050: URL: https://github.com/apache/hive/pull/4050#issuecomment-1426092206
> > @VenuReddy2103 thanks for your PR, I have a couple of questions: did you try with other backend DBs as well? Did you encounter a similar error? I am curious because this part of the code does not seem to be DB specific, so I'd rather confirm quickly if that's an issue only with MSSQL or if it's more general, and that we don't regress on other databases by using `setBytes()` in place of `setObject()`. > > Have you tried the same code when histogram is actually computed? Did it fail? I am asking because it might be an artifact of the uninitialized `histogram` field. > > Since the `bitvector` field is totally analogous (same data type, `byte array`, and written with `setObject()`), apart from the fact that it is always computed, I am wondering if that's not the real difference, as it's successfully written to MSSQL with the `.setObject()` call. > > Can you update here with the actual return value of `mPartitionColumnStatistics.getHistogram()` and of `mPartitionColumnStatistics.getBitVector()` at the time of the call by putting a breakpoint in the `insertIntoPartColStatTable` function? That would help shed some light on what's actually going on there. > > Thanks! > > @asolimando Thanks for reviewing PR. Please find reply for queries. > > 1. I've tried with Mysql, Postgresql, Oracle and Derby as well. This issue is not observed with them. It is observed only with MSSql. When histogram statistics is disabled, `histogram` field in `MPartitionColumnStatistics` is initialized default with empty byte array. And `mPartitionColumnStatistics.getHistogram()` returns null if `histogram.length` is 0. With `preparedStatement.setObject(18, mPartitionColumnStatistics.getHistogram())` invocation, since the object is null, `SQLServerPreparedStatement.setObject()` inferred the jdbc type as `JDBCType.CHAR`. Hence this issue. Whereas with `preparedStatement.setBytes()`, though null is passed, it don't try to infer type again. Have verified this change with MSSql, MySql, PostgreSql, Oracle and Derby. > Have found a similar issue fix here - https://github.com/trinodb/trino/pull/4846/files#:~:text=statement.setBytes(index%2C%20null)%3B > 2. When the histogram is computed, it didn't fail because value is not null and driver could infer the type correctly. > 3. Yes this issue is not seen with `bitvector` as it is always computed. > 4. Debugger screenshots for `MPartitionColumnStatistics.getBitVector()` and `MPartitionColumnStatistics.getHistogram()` > > <img alt="image" width="974" src="https://user-images.githubusercontent.com/35334869/218059642-40495bcf-a547-48cb-bc5d-0a21697e3c07.png";> > > <img alt="image" width="800" src="https://user-images.githubusercontent.com/35334869/218060043-960bcad7-0578-4b9b-8183-068f9672eaf7.png";> @VenuReddy2103, thanks a lot for checking, it all makes sense to me. One last thing, could you try with the other metastore backend databases if we are still OK with the current fix, both in case of empty (null) and non-empty histogram array? If you can confirm that, then we are good to go. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
![https://user-images.githubusercontent.com/35334869/218059642-40495bcf-a547-48cb-bc5d-0a21697e3c07.png"](https://user-images.githubusercontent.com/35334869/218059642-40495bcf-a547-48cb-bc5d-0a21697e3c07.png"){kind=link}
![https://user-images.githubusercontent.com/35334869/218060043-960bcad7-0578-4b9b-8183-068f9672eaf7.png"](https://user-images.githubusercontent.com/35334869/218060043-960bcad7-0578-4b9b-8183-068f9672eaf7.png"){kind=link}