sarutak commented on a change in pull request #29958:
URL: https://github.com/apache/spark/pull/29958#discussion_r502226000
##########
File path:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
##########
@@ -91,7 +91,7 @@ class KinesisBackedBlockRDD[T: ClassTag](
require(_blockIds.length == arrayOfseqNumberRanges.length,
"Number of blockIds is not equal to the number of sequence number ranges")
- override def isValid(): Boolean = true
+ override def isValid: Boolean = true
Review comment:
Yes. empty parameter list causes compilation error if we build for Scala
2.13 with SBT.
This is what I'd like to detect by CI.
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -294,28 +294,28 @@ jobs:
./build/mvn $MAVEN_CLI_OPTS -DskipTests -Pyarn -Pmesos -Pkubernetes
-Phive -Phive-thriftserver -Phadoop-cloud -Djava.version=11 install
rm -rf ~/.m2/repository/org/apache/spark
- scala-213:
- name: Scala 2.13 build
+ scala-213-sbt:
+ name: Scala 2.13 build with SBT
runs-on: ubuntu-latest
steps:
- name: Checkout Spark repository
uses: actions/checkout@v2
- - name: Cache Maven local repository
+ - name: Cache Ivy local repository
uses: actions/cache@v2
with:
- path: ~/.m2/repository
- key: scala-213-maven-${{ hashFiles('**/pom.xml') }}
+ path: ~/.ivy2/cache
+ key: scala-213-ivy-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }}
restore-keys: |
- scala-213-maven-
+ scala-213-ivy-
- name: Install Java 11
uses: actions/setup-java@v1
with:
java-version: 11
- - name: Build with Maven
+ - name: Build with SBT
run: |
- export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g
-Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
- export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.ivy2/local
mkdir -p ~/.m2
Review comment:
`.m2` is required because `publishLocal` installs artifacts into `.m2`.
But, as I mentioned above, I think we can only do `compile` and
`test:compile` so I'll remove this part.
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -294,28 +294,28 @@ jobs:
./build/mvn $MAVEN_CLI_OPTS -DskipTests -Pyarn -Pmesos -Pkubernetes
-Phive -Phive-thriftserver -Phadoop-cloud -Djava.version=11 install
rm -rf ~/.m2/repository/org/apache/spark
- scala-213:
- name: Scala 2.13 build
+ scala-213-sbt:
+ name: Scala 2.13 build with SBT
runs-on: ubuntu-latest
steps:
- name: Checkout Spark repository
uses: actions/checkout@v2
- - name: Cache Maven local repository
+ - name: Cache Ivy local repository
uses: actions/cache@v2
with:
- path: ~/.m2/repository
- key: scala-213-maven-${{ hashFiles('**/pom.xml') }}
+ path: ~/.ivy2/cache
+ key: scala-213-ivy-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }}
restore-keys: |
- scala-213-maven-
+ scala-213-ivy-
- name: Install Java 11
uses: actions/setup-java@v1
with:
java-version: 11
- - name: Build with Maven
+ - name: Build with SBT
run: |
- export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g
-Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
- export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.ivy2/local
mkdir -p ~/.m2
Review comment:
The existing build job for Scala 2.13 with Maven does `install` so the
new job does `publshLocal`.
`.m2` is required because `publishLocal` installs artifacts into `.m2`.
But I think we can only do `compile` and `test:compile` so I'll remove this
part.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
