jiayuasu commented on pull request #494: URL: https://github.com/apache/incubator-sedona/pull/494#issuecomment-741675706
@Imbruced I have successfully made Sedona run on Spark 2.4.7 + Python 3.7. In fact, I am glad that the test failed before. There was a bug in the root pom.xml (sedona-parent). It packaged a wrong jackson into the compiled Sedona jar. It was introduced by PR https://github.com/apache/incubator-sedona/pull/471 This bug will sometimes cause the Scala / Java / Python Sedona fail in the Spark cluster mode. Once I removed this dependency, all test passed. Now as you can see in the GitHub CI test result, 6 checks have passed. The only thing left is the test on Spark 3.0.1 + Python 3.9. Based on my initial test https://github.com/apache/incubator-sedona/runs/1521112458 , the error is `OSError: Could not find library geos_c or load any of its variants ['libgeos_c.so.1', 'libgeos_c.so']` It looks like some of the Sedona Python packages need to be updated. If you think Spark 3.0.1 + Python 3.9 is something easy to fix, please let me know the solution. If you think this will take some time, I will directly merge this PR and leave Python 3.9 support for the future work. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
