sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492456768
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,50 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: java11-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ java11-maven-
Review comment:
Ah, I don't intend to reuse it. I just forgot to change the name.
I've renamed it.
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g
-Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver
\
+ -Dguava.version=25.0-jre -Djava.version=11
-Dtest.default.exclude.tags= -Dtest=none \
Review comment:
Yes. I know the official version of Guava is `14.0.1`. Actually, I
noticed the current ChromeDriver can't use `guava 14.0.1` so I intended to use
`25.0` only for `ChromeUISeleniumSuite` and `ChromeUIHistoryServerSuite`.
Do you have any better idea rather than use `25.0`?
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g
-Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver
\
+ -Dguava.version=25.0-jre -Djava.version=11
-Dtest.default.exclude.tags= -Dtest=none \
Review comment:
Yes. I know the official version of Guava is `14.0.1`. Actually, I
noticed the current ChromeDriver and RemoteWebDriver can't use `guava 14.0.1`
so I intended to use `25.0` only for `ChromeUISeleniumSuite` and
`ChromeUIHistoryServerSuite`.
Do you have any better idea rather than use `25.0`?
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g
-Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver
\
+ -Dguava.version=25.0-jre -Djava.version=11
-Dtest.default.exclude.tags= -Dtest=none \
Review comment:
The first time those tests are added, they works with `guava 14.0.1`.
I believe #28585 (SPARK-31765) requires upgraded RemoteWebDriver and it
requires `25.0-jre`.
That PR is for a small security issue.
> Do we have a JIRA issue for that test suite failure?
Yes. But we need to modify the title.
https://issues.apache.org/jira/browse/SPARK-31996
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g
-Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver
\
+ -Dguava.version=25.0-jre -Djava.version=11
-Dtest.default.exclude.tags= -Dtest=none \
Review comment:
The first time those tests are added, they works with `guava 14.0.1`.
I believe #28585 (SPARK-31765) requires upgraded RemoteWebDriver and it
requires `25.0-jre`.
That PR is for a small security issue.
> Do we have a JIRA issue for that test suite failure?
Yes. But we might to need to modify the title.
https://issues.apache.org/jira/browse/SPARK-31996
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g
-Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver
\
+ -Dguava.version=25.0-jre -Djava.version=11
-Dtest.default.exclude.tags= -Dtest=none \
Review comment:
The first time those tests are added, they works with `guava 14.0.1`.
I believe #28585 (SPARK-31765) requires upgraded RemoteWebDriver and it
requires `25.0-jre`.
That PR is for a small security issue.
> Do we have a JIRA issue for that test suite failure?
Yes. But we might need to modify the title.
https://issues.apache.org/jira/browse/SPARK-31996
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g
-Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver
\
+ -Dguava.version=25.0-jre -Djava.version=11
-Dtest.default.exclude.tags= -Dtest=none \
Review comment:
> BTW, there are many request for Guava update.
Yeah, I know. So, should we revert that PR? It's a security fix but for the
test code, or waiting for Guava updated?
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
Yes. I tried to do so first but I noticed those tests need the newer
Guava so I made a separate job...
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` but as I
expected, tests failed due to the Guava version compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite ***
ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at
org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at
org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at
org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at
org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at
org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at
org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at
org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at
org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at
org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at
org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running
org.apache.spark.deploy.history.ChromeUIHistoryServerSuite:
java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at
org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`.
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` on my
repository but as I expected, tests failed due to the Guava version
compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite ***
ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at
org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at
org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at
org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at
org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at
org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at
org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at
org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at
org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at
org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at
org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running
org.apache.spark.deploy.history.ChromeUIHistoryServerSuite:
java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at
org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`.
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` on my
repository but as I expected, tests failed due to the Guava version
compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite ***
ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at
org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at
org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at
org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at
org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at
org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at
org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at
org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at
org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at
org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at
org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running
org.apache.spark.deploy.history.ChromeUIHistoryServerSuite:
java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at
org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`.
Or, revert that change as I mentioned
[here](https://github.com/apache/spark/pull/29827#discussion_r492484703).
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` on my
repository but as I expected, tests failed due to the Guava version
compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite ***
ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at
org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at
org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at
org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at
org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at
org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at
org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at
org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at
org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at
org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at
org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running
org.apache.spark.deploy.history.ChromeUIHistoryServerSuite:
java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at
org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`.
Or, revert SPARK-31765 as I mentioned
[here](https://github.com/apache/spark/pull/29827#discussion_r492484703).
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` and
running GitHub Actions jobs on my repository but as I expected, tests failed
due to the Guava version compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite ***
ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at
org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at
org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at
org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at
org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at
org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at
org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at
org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at
org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at
org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at
org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running
org.apache.spark.deploy.history.ChromeUIHistoryServerSuite:
java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at
org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`.
Or, revert SPARK-31765 as I mentioned
[here](https://github.com/apache/spark/pull/29827#discussion_r492484703).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
