[
https://issues.apache.org/jira/browse/SEDONA-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17714951#comment-17714951
]
Adam Binford commented on SEDONA-276:
-------------------------------------
[~umartin] Oh that's not how I took it! Yeah that's pretty much what I'm saying
I was trying to go for, Sedona should only interact with slf4j, and we should
use config files to control the verbosity of tests. It looks like Spark didn't
update to log4j2 until 3.3, so if you try to use Spark's implementation of
log4j for tests you need both log4j1 config files and log4j2 config files to
control the verbosity of all tested versions. I think we're mostly saying the
same thing hah.
I think the only issue with logging as far as 3.4 is that they upgraded to
slf4j 2, which has a new log4j-slf4j2-impl artifact that was causing weird
issues with logj4-slf4j-impl or something like that. Until we're only
supporting 3.3+, it's probably easier just to exclude log4j-slf4j2-impl from
Spark so everything is still using log4j2 at least under the hood for testing?
Once we're only at 3.3+ we could drop specifying the slf4j implementation in
Spark tests.
> Add support for Spark 3.4
> -------------------------
>
> Key: SEDONA-276
> URL: https://issues.apache.org/jira/browse/SEDONA-276
> Project: Apache Sedona
> Issue Type: Improvement
> Reporter: Martin Andersson
> Priority: Major
>
> Spark 3.4.0 was announced on April 14.
> [https://lists.apache.org/thread/o1o9n1t036rmfp45q1jrjs0zzzx0j156]
> Findings from a quick test build using -Dspark.version=3.4.0:
> # Incompatible logging api versions causes test failures.
> # GeoParquet and Geotiff datasources fails to compile due to api changes in
> Spark.
> Setting log4j and slf4j to the same versions as Spark 3.4.0 uses solves 1. I
> don't know how that will affect other Spark versions.
> To solve 2 we might need to use reflection to handle the api changes in Spark.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
