JNSimba opened a new pull request, #383: URL: https://github.com/apache/doris-spark-connector/pull/383
# Proposed changes Issue Number: None ## Problem Summary Support reading explicitly qualified `catalog.database.table` tables through the existing REST schema and Arrow Flight SQL read paths. Preserve the catalog when discovering schema and constructing Spark tables, and reuse schema and projection handling for internal and external tables. External reads use one tablet-free Spark partition; two-part table reads retain tablet partitioning. Reject Thrift fallback for three-part reads, preserve user-supplied identifier quoting, and advertise three-part tables as read-only. Add shared MySQL JDBC Catalog fixtures with independent DataFrame, Spark SQL, RDD, and empty-table integration tests, plus focused unit tests and usage documentation. No new Maven dependencies. ## Validation - 98 unit tests passed in the common and Spark 3 shared modules on JDK 8, with zero failures or errors. - IT source compilation passed for Spark 2.4 / Scala 2.11 / JDK 8, Spark 3.3 / Scala 2.12 / JDK 17, and Spark 4.1 / Scala 2.13 / JDK 17. - Live validation on an existing Doris cluster with MySQL 8.0.46 and Spark 3.3 passed: default Arrow mode and port discovery, DataFrame/SQL/RDD reads, one partition, projection/filter/limit, NULLs, empty tables, quoted identifiers, and Thrift rejection. Used a connector JAR built from this branch. - The new Docker ITCase has not been executed locally. It is selected by the existing CI `*ITCase` runs for Spark 2.4, 3.1–3.5, and 4.1. ## Checklist(Required) 1. Does it affect the original behavior: Yes; enables three-part reads and rejects unsupported three-part writes. Existing two-part read modes remain supported. 2. Has unit tests been added: Yes. 3. Has document been added or modified: Yes. 4. Does it need to update dependencies: No. 5. Are there any changes that cannot be rolled back: No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
