hellozepp opened a new pull request, #335: URL: https://github.com/apache/doris-spark-connector/pull/335
What problem does this PR solve? In the Spark 3 connector (spark-doris-connector-spark-3-base), the filter configuration specified by DorisOptions.DORIS_FILTER_QUERY was not being properly applied when no filters were passed from the Spark SQL pushdown. This caused the connector to ignore user-defined filter queries in configuration. What is changed and how it works? Added explicit check for DorisOptions.DORIS_FILTER_QUERY configuration when no filters are provided from Spark SQL Enhanced logging to provide better visibility into filter application process Maintains backward compatibility with existing filter handling logic Before: Filter queries from configuration were ignored when Spark SQL didn't provide filters After: Filter queries from configuration are properly applied as fallback -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
