wywen opened a new pull request, #1944: URL: https://github.com/apache/fury/pull/1944
<!-- **Thanks for contributing to Fury.** **If this is your first time opening a PR on fury, you can refer to [CONTRIBUTING.md](https://github.com/apache/fury/blob/main/CONTRIBUTING.md).** Contribution Checklist - The **Apache Fury (incubating)** community has restrictions on the naming of pr titles. You can also find instructions in [CONTRIBUTING.md](https://github.com/apache/fury/blob/main/CONTRIBUTING.md). - Fury has a strong focus on performance. If the PR you submit will have an impact on performance, please benchmark it first and provide the benchmark result here. --> ## What does this PR do? This PR removes the java specific `ExtField` class from the schema and moves the extData mechanism to the internal UnsafeTrait class. This is necessary because `ExtField` is only created internally from `inferSchema` method used by java, and we would potentially import or derive schemas from other sources (e.g XLANG, handwritten schema, arrow-native source) -- those schemas will be incompatible when we attempt to retrieve the cached schema. Removing schema-caching will fix the issue, but create allocations, so after some discussion, we decided to move the mechanism to the internal UnsafeTrait class. This implementation makes changes internal API: - Derived classes from `UnsafeTrait` need to initialize the `extData` cache and define the number of extData slots needed. - The internal `getStruct` method needs to define which slot we use to retrieve `extData`. Other: - pom.xml for fury-format will automatically run tests with appropriate --add-opens flag for arrow ## Related issues <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> ## Does this PR introduce any user-facing change? N/A - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
