Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/22009#discussion_r209022127
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/SessionConfigSupport.java
---
@@ -27,10 +27,10 @@
@InterfaceStability.Evolving
public interface SessionConfigSupport extends DataSourceV2 {
--- End diff --
@cloud-fan, in the `TableCatalog` addition, #21306, I added this to
configure catalogs. That way, a catalog is how you add configuration for all
tables. Anonymous tables from the `ReadSupportProvider` classes don't really
need this. I think we should remove the session config support in favor of
using the catalog initialization config.
I also add a CaseInsensitiveStringMap for passing config that we should
move to instead of DataSourceOptions. There are two main differences:
1. DataSourceOptions has methods to retrieve paths, table name, etc. that
aren't needed when we use catalogs. Named tables and path-based tables should
use catalogs and not pass these through options. Anonymous tables don't need
standard ways to pass this information.
2. It is more clear what the class does: it provides a string to string
mapping with case insensitive keys. It isn't as obvious what DataSourceOptions
does.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
