Yihong He created SPARK-51899:
---------------------------------
Summary: Implement an exception allowlist for
spark.catalog.listTables()
Key: SPARK-51899
URL: https://issues.apache.org/jira/browse/SPARK-51899
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: 4.1.0
Reporter: Yihong He
Swallowing all non-fatal errors and unconditionally returning partial results
can lead to inconsistent outcomes, especially in cases involving transient
errors (e.g., a table may return full results in {{{}listTables(){}}}, but then
return partial results in a subsequent call due to a transient error).
Additionally, the {{tableType}} field in the {{Table}} class is not nullable.
While it is technically possible to set {{tableType}} to {{{}null{}}}, doing so
violates the expectations of the {{Table}} class. This may surprise people.
To address these issues, we can implement an exception allowlist based on SQL
states for non-transient errors and avoid returning results for those tables,
rather than returning partial results when exceptions occur.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
