19855134604 opened a new pull request, #38743: URL: https://github.com/apache/spark/pull/38743
### What changes were proposed in this pull request? This PR use profile named `-Puser-defined-protoc` to support that users can build and test `protobuf` module by specifying custom `protoc` executables. ### Why are the changes needed? As described in [SPARK-41215](https://issues.apache.org/jira/browse/SPARK-41215), the latest versions of `protoc` have the minimum version requirements for basic libraries such as `glibc` and `glibcxx`. Because of that it is not possible to test-compile the `protobuf` module out of the box on CentOS 6 or CentOS 7. Instead the following error messages is shown: ``` [ERROR] /home/disk1/spark-ut/spark/connector/protobuf/src/test/resources/protobuf/timestamp.proto [0:0]: /tmp/protoc6599263403262688374.exe: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /tmp/protoc6599263403262688374.exe) [ERROR] /home/disk1/spark-ut/spark/connector/protobuf/src/test/resources/protobuf/timestamp.proto [0:0]: /tmp/protoc6599263403262688374.exe: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.18' not found (required by /tmp/protoc6599263403262688374.exe) [ERROR] /home/disk1/spark-ut/spark/connector/protobuf/src/test/resources/protobuf/timestamp.proto [0:0]: /tmp/protoc6599263403262688374.exe: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.14' not found (required by /tmp/protoc6599263403262688374.exe) [ERROR] /home/disk1/spark-ut/spark/connector/protobuf/src/test/resources/protobuf/timestamp.proto [0:0]: /tmp/protoc6599263403262688374.exe: /usr/lib64/libstdc++.so.6: version `CXXABI_1.3.5' not found (required by /tmp/protoc6599263403262688374.exe) ``` ### Does this PR introduce _any_ user-facing change? No, the way to using official pre-release `protoc` binary files is activated by default. ### How was this patch tested? - Pass GitHub Actions - Manual test on CentOS6u3 and CentOS7u4 ```bash export PROTOBUF_PROTOC_EXEC_PATH=/path-to-protoc-exe ./build/mvn clean install -pl connector/protobuf -Puser-defined-protoc -am -DskipTests ./build/mvn clean test -pl connector/protobuf -Puser-defined-protoc ``` and ```bash export PROTOBUF_PROTOC_EXEC_PATH=/path-to-protoc-exe ./build/sbt clean "protobuf/compile" -Puser-defined-protoc ./build/sbt "protobuf/test" -Puser-defined-protoc ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
