sunchao commented on pull request #33382:
URL: https://github.com/apache/spark/pull/33382#issuecomment-881620207
> If possible, I'd like to make this as a disabled configuration instead of
enforcing it. For normal users which has successful queries don't need to
bother with this. And, for the failed queries, can we give a directional guide
at the error message to enable this? Is it difficult?
Hmm you mean introduce another configuration in Spark, and only if that
turned on, we'd fallback to `getAllPartitionsMethod` when
`hive.metastore.try.direct.sql` is true?
```scala
val tryDirectSql =
hive.getMSC.getConfigValue(tryDirectSqlConfVar.varname,
tryDirectSqlConfVar.defaultBoolVal.toString).toBoolean
try {
getPartitionsByFilterMethod.invoke(hive, table, filter)
.asInstanceOf[JArrayList[Partition]]
} catch {
case ex: InvocationTargetException if
ex.getCause.isInstanceOf[MetaException] &&
!tryDirectSql && !enableFallback =>
getAllPartitionsMethod.invoke(hive,
table).asInstanceOf[JSet[Partition]]
case ex: InvocationTargetException if
ex.getCause.isInstanceOf[MetaException] &&
tryDirectSql || enableFallback =>
throw QueryExecutionErrors.getPartitionMetadataByFilterError(ex)
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
