Attila Doroszlai created HDDS-17848:
---------------------------------------

             Summary: STS acceptance test needs Iceberg jars at runtime
                 Key: HDDS-17848
                 URL: https://issues.apache.org/jira/browse/HDDS-17848
             Project: Apache Ozone
          Issue Type: Bug
          Components: CI
            Reporter: Attila Doroszlai
            Assignee: Attila Doroszlai


{code:title=https://github.com/adoroszlai/ozone/actions/runs/44148030238/job/134290218488}
Running Spark SQL against Polaris with STS vended credentials...
:: loading settings :: url = 
jar:file:/opt/spark/jars/ivy-2.5.3.jar!/org/apache/ivy/core/settings/ivysettings.xml
Ivy Default Cache set to: /root/.ivy2/cache
The jars for the packages stored in: /root/.ivy2/jars
org.apache.iceberg#iceberg-spark-runtime-3.5_2.12 added as a dependency
org.apache.iceberg#iceberg-aws-bundle added as a dependency
:: resolving dependencies :: 
org.apache.spark#spark-submit-parent-b7569e83-38d5-4548-a99f-65bc398897da;1.0
        confs: [default]
:: resolution report :: resolve 866ms :: artifacts dl 0ms
        :: modules in use:
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   2   |   0   |   0   |   0   ||   0   |   0   |
        ---------------------------------------------------------------------

:: problems summary ::
:::: WARNINGS
                module not found: 
org.apache.iceberg#iceberg-spark-runtime-3.5_2.12;1.10.1

        ==== local-m2-cache: tried

          
file:/root/.m2/repository/org/apache/iceberg/iceberg-spark-runtime-3.5_2.12/1.10.1/iceberg-spark-runtime-3.5_2.12-1.10.1.pom

          -- artifact 
org.apache.iceberg#iceberg-spark-runtime-3.5_2.12;1.10.1!iceberg-spark-runtime-3.5_2.12.jar:

          
file:/root/.m2/repository/org/apache/iceberg/iceberg-spark-runtime-3.5_2.12/1.10.1/iceberg-spark-runtime-3.5_2.12-1.10.1.jar

        ==== local-ivy-cache: tried

          
/root/.ivy2/local/org.apache.iceberg/iceberg-spark-runtime-3.5_2.12/1.10.1/ivys/ivy.xml

          -- artifact 
org.apache.iceberg#iceberg-spark-runtime-3.5_2.12;1.10.1!iceberg-spark-runtime-3.5_2.12.jar:

          
/root/.ivy2/local/org.apache.iceberg/iceberg-spark-runtime-3.5_2.12/1.10.1/jars/iceberg-spark-runtime-3.5_2.12.jar

        ==== central: tried

          
https://repo1.maven.org/maven2/org/apache/iceberg/iceberg-spark-runtime-3.5_2.12/1.10.1/iceberg-spark-runtime-3.5_2.12-1.10.1.pom

          -- artifact 
org.apache.iceberg#iceberg-spark-runtime-3.5_2.12;1.10.1!iceberg-spark-runtime-3.5_2.12.jar:

          
https://repo1.maven.org/maven2/org/apache/iceberg/iceberg-spark-runtime-3.5_2.12/1.10.1/iceberg-spark-runtime-3.5_2.12-1.10.1.jar

        ==== spark-packages: tried

          
https://repos.spark-packages.org/org/apache/iceberg/iceberg-spark-runtime-3.5_2.12/1.10.1/iceberg-spark-runtime-3.5_2.12-1.10.1.pom

          -- artifact 
org.apache.iceberg#iceberg-spark-runtime-3.5_2.12;1.10.1!iceberg-spark-runtime-3.5_2.12.jar:

          
https://repos.spark-packages.org/org/apache/iceberg/iceberg-spark-runtime-3.5_2.12/1.10.1/iceberg-spark-runtime-3.5_2.12-1.10.1.jar

                module not found: org.apache.iceberg#iceberg-aws-bundle;1.10.1

        ==== local-m2-cache: tried

          
file:/root/.m2/repository/org/apache/iceberg/iceberg-aws-bundle/1.10.1/iceberg-aws-bundle-1.10.1.pom

          -- artifact 
org.apache.iceberg#iceberg-aws-bundle;1.10.1!iceberg-aws-bundle.jar:

          
file:/root/.m2/repository/org/apache/iceberg/iceberg-aws-bundle/1.10.1/iceberg-aws-bundle-1.10.1.jar

        ==== local-ivy-cache: tried

          
/root/.ivy2/local/org.apache.iceberg/iceberg-aws-bundle/1.10.1/ivys/ivy.xml

          -- artifact 
org.apache.iceberg#iceberg-aws-bundle;1.10.1!iceberg-aws-bundle.jar:

          
/root/.ivy2/local/org.apache.iceberg/iceberg-aws-bundle/1.10.1/jars/iceberg-aws-bundle.jar

        ==== central: tried

          
https://repo1.maven.org/maven2/org/apache/iceberg/iceberg-aws-bundle/1.10.1/iceberg-aws-bundle-1.10.1.pom

          -- artifact 
org.apache.iceberg#iceberg-aws-bundle;1.10.1!iceberg-aws-bundle.jar:

          
https://repo1.maven.org/maven2/org/apache/iceberg/iceberg-aws-bundle/1.10.1/iceberg-aws-bundle-1.10.1.jar

        ==== spark-packages: tried

          
https://repos.spark-packages.org/org/apache/iceberg/iceberg-aws-bundle/1.10.1/iceberg-aws-bundle-1.10.1.pom

          -- artifact 
org.apache.iceberg#iceberg-aws-bundle;1.10.1!iceberg-aws-bundle.jar:

          
https://repos.spark-packages.org/org/apache/iceberg/iceberg-aws-bundle/1.10.1/iceberg-aws-bundle-1.10.1.jar

                ::::::::::::::::::::::::::::::::::::::::::::::

                ::          UNRESOLVED DEPENDENCIES         ::

                ::::::::::::::::::::::::::::::::::::::::::::::

                :: org.apache.iceberg#iceberg-spark-runtime-3.5_2.12;1.10.1: 
not found

                :: org.apache.iceberg#iceberg-aws-bundle;1.10.1: not found

                ::::::::::::::::::::::::::::::::::::::::::::::



:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
Exception in thread "main" java.lang.RuntimeException: [unresolved dependency: 
org.apache.iceberg#iceberg-spark-runtime-3.5_2.12;1.10.1: not found, unresolved 
dependency: org.apache.iceberg#iceberg-aws-bundle;1.10.1: not found]
        at 
org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1613)
        at 
org.apache.spark.util.DependencyUtils$.resolveMavenDependencies(DependencyUtils.scala:185)
        at 
org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:339)
        at 
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:969)
        at 
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:199)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:222)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91)
        at 
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1125)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1134)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
ERROR: spark-sql exited with status 1
ERROR: Test execution of ozonesecure-ha/test-sts.sh is FAILED!!!!
{code}



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to