andygrove opened a new issue, #2946:
URL: https://github.com/apache/datafusion-comet/issues/2946
### Describe the bug
It is getting frustrating trying to get PRs to pass all tests in CI. I see
the hive-1 tests failing even after re-running multiple times.
There are no failure. It looks like a hang or timeout. The last entries in
the log look like this:
```
[info] HivePartitionFilteringSuite(2.0):
20:48:31.545 WARN org.apache.hadoop.hive.metastore.ObjectStore: Version
information not found in metastore. hive.metastore.schema.verification is not
enabled so recording the schema version 2.0.0
20:48:32.044 WARN org.apache.hadoop.hive.ql.session.SessionState:
METASTORE_FILTER_HOOK will be ignored, since
hive.security.authorization.manager is set to instance of HiveAuthorizerFactory.
04:51:45.212 WARN org.apache.spark.sql.hive.client.Shim_v2_0: Caught Hive
MetaException attempting to get partition metadata by filter from Hive. Falling
back to fetching all partition metadata, which will degrade performance.
Modifying your Hive metastore configuration to set
hive.metastore.try.direct.sql to true (if it is not true already) may resolve
this problem. Or you can enable
spark.sql.hive.metastorePartitionPruningFastFallback to alleviate performance
downgrade. Otherwise, to avoid degraded performance you can set
spark.sql.hive.metastorePartitionPruningFall...
MetaException(message:Filtering is supported only on partition keys of type
string)
at
org.apache.hadoop.hive.metastore.parser.ExpressionTree$FilterBuilder.setError(ExpressionTree.java:184)
at
org.apache.hadoop.hive.metastore.parser.ExpressionTree$LeafNode.getJdoFilterPushdownParam(ExpressionTree.java:439)
at
org.apache.hadoop.hive.metastore.parser.ExpressionTree$LeafNode.generateJDOFilterOverPartitions(ExpressionTree.java:356)
at
org.apache.hadoop.hive.metastore.parser.ExpressionTree$LeafNode.generateJDOFilter(ExpressionTree.java:278)
at
org.apache.hadoop.hive.metastore.parser.ExpressionTree.generateJDOFilterFragment(ExpressionTree.java:583)
at
org.apache.hadoop.hive.metastore.ObjectStore.makeQueryFilterString(ObjectStore.java:2704)
at
org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsViaOrmFilter(ObjectStore.java:2312)
[info] - 2.0: getPartitionsByFilter returns all partitions when
spark.sql.hive.metastorePartitionPruningFallbackOnException=true (5 seconds,
426 milliseconds)
[info] - 2.0: getPartitionsByFilter should fail when
spark.sql.hive.metastorePartitionPruningFallbackOnException=false (69
milliseconds)
[info] - 2.0: getPartitionsByFilter: ds<=>20170101 (1 second, 466
milliseconds)
...
05:12:55.055 WARN org.apache.spark.sql.hive.client.Shim_v3_0: Caught Hive
MetaException attempting to get partition metadata by filter from Hive. Falling
back to fetching all partition metadata, which will degrade performance.
Modifying your Hive metastore configuration to set
hive.metastore.try.direct.sql to true (if it is not true already) may resolve
this problem. Or you can enable
spark.sql.hive.metastorePartitionPruningFastFallback to alleviate performance
downgrade. Otherwise, to avoid degraded performance you can set
spark.sql.hive.metastorePartitionPruningFall...
MetaException(message:Filtering is supported only on partition keys of type
string)
at
org.apache.hadoop.hive.metastore.parser.ExpressionTree$FilterBuilder.setError(ExpressionTree.java:184)
at
org.apache.hadoop.hive.metastore.parser.ExpressionTree$LeafNode.getJdoFilterPushdownParam(ExpressionTree.java:437)
at
org.apache.hadoop.hive.metastore.parser.ExpressionTree$LeafNode.generateJDOFilterOverPartitions(ExpressionTree.java:355)
at
org.apache.hadoop.hive.metastore.parser.ExpressionTree$LeafNode.generateJDOFilter(ExpressionTree.java:277)
at
org.apache.hadoop.hive.metastore.parser.ExpressionTree.generateJDOFilterFragment(ExpressionTree.java:581)
at
org.apache.hadoop.hive.metastore.ObjectStore.makeQueryFilterString(ObjectStore.java:3841)
at
org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsViaOrmFilter(ObjectStore.java:3287)
...
### Steps to reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
