dpengpeng opened a new issue, #1499:
URL: https://github.com/apache/datafusion-comet/issues/1499
### What is the problem the feature request solves?
I compiled the source code of Comet release 0.6.0 on the Ubuntu (18.04.4
LTS) operating system, obtaining a `comet-spark-spark3.4_2.12-0.6.0.jar`. I
then placed this JAR file in the Spark (version: 3.4.1) environment deployed on
a CentOS 7 machine, specifically in the `$SPARK_HOME/jars` directory. I used
the spark-sql local mode to run the program. I had previously created a data
table and inserted data into it. When I enabled Comet and executed the SQL
statement "select * from test;", it failed with the following error:
```
2025-03-11 17:00:37,191 WARN NativeBase: Failed to load comet library
java.lang.UnsatisfiedLinkError: /tmp/libcomet-1273561377384406186.so:
/lib64/libm.so.6: version `GLIBC_2.27' not found (required by
/tmp/libcomet-1273561377384406186.so)
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
~[?:1.8.0_302]
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1934)
~[?:1.8.0_302]
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1817)
~[?:1.8.0_302]
at java.lang.Runtime.load0(Runtime.java:810) ~[?:1.8.0_302]
at java.lang.System.load(System.java:1088) ~[?:1.8.0_302]
```
Upon analysis, I found that the glibc version on the CentOS 7 system is
2.17, while the glibc version used during the compilation of Comet on the
Ubuntu system is 2.27. Therefore, does this indicate that the operating system
used for compiling Comet should be consistent with the one used for running it?
However, I noticed that the Comet official website now only specifies
supported systems as Linux and Apple OSX, without explicitly stating specific
operating systems as before. This suggests that it may not be dependent on the
specific operating system but rather on the CPU architecture. If this were the
case, it would contradict the issue I am currently encountering.
Additionally, I observed another issue: the size of the
`comet-spark-spark3.4_2.12-0.6.0.jar` I compiled is approximately 28 MB,
whereas the published JAR is about 82 MB. Why is there such a difference?
### Describe the potential solution
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
