Kontinuation opened a new pull request, #1662: URL: https://github.com/apache/sedona/pull/1662
## Did you read the Contributor Guide? - Yes, I have read the [Contributor Rules](https://sedona.apache.org/latest-snapshot/community/rule/) and [Contributor Development Guide](https://sedona.apache.org/latest-snapshot/community/develop/) ## Is this PR related to a JIRA ticket? - Yes, the URL of the associated JIRA ticket is https://issues.apache.org/jira/browse/SEDONA-670. The PR name follows the format `[SEDONA-XXX] my subject`. ## What changes were proposed in this PR? This PR works around an internal method incompatibility between open-source Apache Spark and DBR. The readFile method defined by open-source Apache Spark is: ```scala def readFile( conf: Configuration, file: PartitionedFile, parser: JacksonParser, schema: StructType): Iterator[InternalRow] ``` While this function on DBR takes an extra `Option[_]` parameter: ```scala def readFile( conf: Configuration, file: PartitionedFile, parser: JacksonParser, schema: StructType, badRecordsWriter: Option[BadRecordsWriter]): Iterator[InternalRow] ``` We workaround this problem by detecting the number of parameters of the `readFile` function using reflection, and pass the appropriate parameters to them. ## How was this patch tested? Passing existing tests and manually tested on DBR 15.4 LTS. ## Did this PR include necessary documentation updates? - No, this PR does not affect any public API so no need to change the documentation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@sedona.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
