nielifeng commented on PR #11503: URL: https://github.com/apache/seatunnel/pull/11503#issuecomment-5041892846
Addressed the Lance blocker in head 7cf4e7aff. Root cause: SeaTunnelRowType.indexOf(fieldName) throws when the persisted Lance dataset contains a field that is absent from the restored checkpoint schema, so the existing fieldIndex >= 0 guard could never reconcile the layouts. The converter now: - projects rows by dataset field name using the non-throwing index lookup - preserves the persisted dataset schema - explicitly materializes missing checkpoint fields as null in the Arrow batch - closes the VectorSchemaRoot if conversion fails Added FragmentConverterTest covering an existing non-nullable c_bytes dataset field missing from the restored runtime row. The c_string value is preserved, c_bytes is present and null, and vector value counts are correct. Validation: - FragmentConverterTest: 1/1 passed - Spotless passed - connector-lance skip-tests verify passed The full LanceSinkTest still requires the Linux Lance JNI library; this macOS x86_64 environment does not ship nativelib/darwin-x86-64/liblance_jni.dylib. Linux CI will run the exact native prepareCommit regression. Please re-review after that lane completes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
