https://github.com/apache/spark/blob/master/sql/connect/client/jvm/src/test/resources/StubClassDummyUdf.scala (commit 408c3df) is the source code for the following jars:
- ./connector/connect/server/src/test/resources/udf_noA.jar - ./connector/connect/client/jvm/src/test/resources/udf2.13.jar - ./connector/connect/client/jvm/src/test/resources/udf2.12.jar https://github.com/apache/spark/blob/master/sql/hive/src/test/resources/regression-test-SPARK-8489/Main.scala and https://github.com/apache/spark/blob/master/sql/hive/src/test/resources/regression-test-SPARK-8489/MyCoolClass.scala are the source files for the following jars: - ./sql/hive/src/test/resources/regression-test-SPARK-8489/test-2.12.jar - ./sql/hive/src/test/resources/regression-test-SPARK-8489/test-2.13.jar - ./sql/hive/src/test/resources/regression-test-SPARK-8489/test-2.11.jar ./sql/core/src/test/resources/SPARK-33084.jar was added in https://github.com/apache/spark/commit/771c538620e66be2d0fb0e383e4aa37b4d29f7eb. The commit contains the source code. I will get the source code for the following jars checked in: - ./core/src/test/resources/TestHelloV3_2.12.jar - ./core/src/test/resources/TestHelloV2_2.12.jar - ./core/src/test/resources/TestHelloV2_2.13.jar - ./core/src/test/resources/TestHelloV3_2.13.jar - ./connector/connect/common/src/test/resources/artifact-tests/junitLargeJar.jar - ./connector/connect/common/src/test/resources/artifact-tests/smallJar.jar - ./connector/connect/client/jvm/src/test/resources/TestHelloV2_2.12.jar - ./connector/connect/client/jvm/src/test/resources/TestHelloV2_2.13.jar - ./data/artifact-tests/junitLargeJar.jar - ./data/artifact-tests/smallJar.jar On Tue, Feb 25, 2025 at 9:22 PM Rozov, Vlad <vro...@amazon.com.invalid> wrote: > Right, the issue does not seem to be new for 3.5 and it is not new for > 3.5.5. Here is the list of all jars I found in the source release: > > ./core/src/test/resources/TestHelloV3_2.12.jar > ./core/src/test/resources/TestHelloV2_2.12.jar > ./core/src/test/resources/TestHelloV2_2.13.jar > ./core/src/test/resources/TestHelloV3_2.13.jar > ./core/src/test/resources/TestUDTF.jar > ./connector/connect/server/src/test/resources/udf_noA.jar > > ./connector/connect/common/src/test/resources/artifact-tests/junitLargeJar.jar > ./connector/connect/common/src/test/resources/artifact-tests/smallJar.jar > ./connector/connect/client/jvm/src/test/resources/TestHelloV2_2.12.jar > ./connector/connect/client/jvm/src/test/resources/TestHelloV2_2.13.jar > ./connector/connect/client/jvm/src/test/resources/udf2.13.jar > ./connector/connect/client/jvm/src/test/resources/udf2.12.jar > ./data/artifact-tests/junitLargeJar.jar > ./data/artifact-tests/smallJar.jar > ./sql/core/src/test/resources/SPARK-33084.jar > ./sql/hive/src/test/noclasspath/hive-test-udfs.jar > ./sql/hive/src/test/resources/regression-test-SPARK-8489/test-2.12.jar > ./sql/hive/src/test/resources/regression-test-SPARK-8489/test-2.13.jar > ./sql/hive/src/test/resources/regression-test-SPARK-8489/test-2.11.jar > ./sql/hive/src/test/resources/SPARK-21101-1.0.jar > ./sql/hive/src/test/resources/data/files/TestSerDe.jar > ./sql/hive/src/test/resources/TestUDTF.jar > ./sql/hive-thriftserver/src/test/resources/TestUDTF.jar > > Thank you, > > Vlad > > > On Feb 25, 2025, at 3:03 PM, Dongjoon Hyun <dongj...@apache.org> wrote: > > > > Just for the record, `TestUDTF.jar` test resource seems to be one of the > oldest examples which Vlad pointed. > > > > > https://github.com/apache/spark/blob/v2.0.0/core/src/test/resources/TestUDTF.jar > > (Added 9 years ago) > > > > $ tar tvf spark-2.0.0.tgz | grep TestUDTF.jar > > -rw-r--r-- 0 jenkins jenkins 1328 Jul 19 2016 > spark-2.0.0/sql/hive/src/test/resources/TestUDTF.jar > > -rw-r--r-- 0 jenkins jenkins 1328 Jul 19 2016 > spark-2.0.0/sql/hive-thriftserver/src/test/resources/TestUDTF.jar > > -rw-r--r-- 0 jenkins jenkins 1328 Jul 19 2016 > spark-2.0.0/core/src/test/resources/TestUDTF.jar > > > > Dongjoon. > > > > > > On 2025/02/25 22:47:07 Dongjoon Hyun wrote: > >> Thank you for reply, Vlad. > >> > >> Do you mean SPARK-44246 violated ASF policy since Apache Spark 3.5.0 by > injecting a unlicensed test jar, `TestHelloV2.jar`? > >> > >> https://issues.apache.org/jira/browse/SPARK-44246 > >> https://github.com/apache/spark/pull/41789 > >> > >> And, it was spread to `TestHelloV3_2.13.jar` via SPARK-44297 in the > same way? > >> > >> https://issues.apache.org/jira/browse/SPARK-44297 > >> https://github.com/apache/spark/pull/41852 > >> > >> > >> Dongjoon. > >> > >> On 2025/02/25 19:53:45 "Rozov, Vlad" wrote: > >>> I am not sure if this was already discussed and noted, so want to > confirm with PMC members: > >>> > >>> I see several (test) JAR files included into the ASF source release > that do not have LICENSE in the MANIFEST or META-INF and do not have the > source code. For example core/src/test/resources/TestHelloV3_2.13.jar. > AFAIK it may be against ASF source release policy to include such jars into > the source release (see "Does the policy apply to binary/object files, such > as executables or JAR files?" in > https://www.apache.org/legal/src-headers.html) > >>> > >>> Thank you, > >>> > >>> Vlad > >>> > >>> On Feb 23, 2025, at 4:37 PM, Dongjoon Hyun <dongjoon.h...@gmail.com> > wrote: > >>> > >>> Please vote on releasing the following candidate as Apache Spark > version 3.5.5. > >>> > >>> The vote is open until February 27th 1AM (PST) and passes if a > majority +1 PMC votes are cast, with a minimum of 3 +1 votes. > >>> > >>> [ ] +1 Release this package as Apache Spark 3.5.5 > >>> [ ] -1 Do not release this package because ... > >>> > >>> To learn more about Apache Spark, please see https://spark.apache.org/ > >>> > >>> The tag to be voted on is v3.5.5-rc1 (commit > 7c29c664cdc9321205a98a14858aaf8daaa19db2) > >>> https://github.com/apache/spark/tree/v3.5.5-rc1 > >>> > >>> The release files, including signatures, digests, etc. can be found at: > >>> https://dist.apache.org/repos/dist/dev/spark/v3.5.5-rc1-bin/ > >>> > >>> Signatures used for Spark RCs can be found in this file: > >>> https://dist.apache.org/repos/dist/dev/spark/KEYS > >>> > >>> The staging repository for this release can be found at: > >>> > https://repository.apache.org/content/repositories/orgapachespark-1476/ > >>> > >>> The documentation corresponding to this release can be found at: > >>> https://dist.apache.org/repos/dist/dev/spark/v3.5.5-rc1-docs/ > >>> > >>> The list of bug fixes going into 3.5.5 can be found at the following > URL: > >>> https://issues.apache.org/jira/projects/SPARK/versions/12355518 > >>> > >>> This release is using the release script of the tag v3.5.5-rc1. > >>> > >>> FAQ > >>> > >>> ========================= > >>> How can I help test this release? > >>> ========================= > >>> > >>> If you are a Spark user, you can help us test this release by taking > >>> an existing Spark workload and running on this release candidate, then > >>> reporting any regressions. > >>> > >>> If you're working in PySpark you can set up a virtual env and install > >>> the current RC and see if anything important breaks, in the Java/Scala > >>> you can add the staging repository to your projects resolvers and test > >>> with the RC (make sure to clean up the artifact cache before/after so > >>> you don't end up building with a out of date RC going forward). > >>> > >>> =========================================== > >>> What should happen to JIRA tickets still targeting 3.5.5? > >>> =========================================== > >>> > >>> The current list of open tickets targeted at 3.5.5 can be found at: > >>> https://issues.apache.org/jira/projects/SPARK and search for "Target > Version/s" = 3.5.5 > >>> > >>> Committers should look at those and triage. Extremely important bug > >>> fixes, documentation, and API tweaks that impact compatibility should > >>> be worked on immediately. Everything else please retarget to an > >>> appropriate release. > >>> > >>> ================== > >>> But my bug isn't fixed? > >>> ================== > >>> > >>> In order to make timely releases, we will typically not hold the > >>> release unless the bug in question is a regression from the previous > >>> release. That being said, if there is something which is a regression > >>> that has not been correctly targeted please ping me or a committer to > >>> help target the issue. > >>> > >>> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >> > >> > > > > --------------------------------------------------------------------- > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > > > >
