yihua commented on code in PR #11644:
URL: https://github.com/apache/hudi/pull/11644#discussion_r1681670425
##########
azure-pipelines-20230430.yml:
##########
@@ -61,7 +61,7 @@ parameters:
default:
- 'hudi-spark-datasource'
- 'hudi-spark-datasource/hudi-spark'
- - 'hudi-spark-datasource/hudi-spark3.2.x'
+ - 'hudi-spark-datasource/hudi-spark3.5.x'
Review Comment:
Keep Azure CI the same as there are test failures that need to be addressed
in #11493.
##########
.github/workflows/bot.yml:
##########
@@ -670,46 +620,46 @@ jobs:
HUDI_VERSION=$(mvn help:evaluate -Dexpression=project.version -q
-DforceStdout)
./packaging/bundle-validation/ci_run.sh hudi_docker_java17
$HUDI_VERSION openjdk17
- integration-tests:
- runs-on: ubuntu-latest
- strategy:
- matrix:
- include:
- - sparkProfile: 'spark2.4'
- sparkArchive: 'spark-2.4.4/spark-2.4.4-bin-hadoop2.7.tgz'
- steps:
- - uses: actions/checkout@v3
- - name: Set up JDK 8
- uses: actions/setup-java@v3
- with:
- java-version: '8'
- distribution: 'temurin'
- architecture: x64
- cache: maven
- - name: Build Project
- env:
- SPARK_PROFILE: ${{ matrix.sparkProfile }}
- SCALA_PROFILE: '-Dscala-2.11 -Dscala.binary.version=2.11'
- run:
- mvn clean install -T 2 $SCALA_PROFILE -D"$SPARK_PROFILE"
-Pintegration-tests -DskipTests=true $MVN_ARGS
- - name: 'UT integ-test'
- env:
- SPARK_PROFILE: ${{ matrix.sparkProfile }}
- SCALA_PROFILE: '-Dscala-2.11 -Dscala.binary.version=2.11'
- run:
- mvn test $SCALA_PROFILE -D"$SPARK_PROFILE" -Pintegration-tests
-DskipUTs=false -DskipITs=true -pl hudi-integ-test $MVN_ARGS
- - name: 'IT'
- env:
- SPARK_PROFILE: ${{ matrix.sparkProfile }}
- SPARK_ARCHIVE: ${{ matrix.sparkArchive }}
- SCALA_PROFILE: '-Dscala-2.11 -Dscala.binary.version=2.11'
- run: |
- echo "Downloading $SPARK_ARCHIVE"
- curl https://archive.apache.org/dist/spark/$SPARK_ARCHIVE
--create-dirs -o $GITHUB_WORKSPACE/$SPARK_ARCHIVE
- tar -xvf $GITHUB_WORKSPACE/$SPARK_ARCHIVE -C $GITHUB_WORKSPACE/
- mkdir /tmp/spark-events/
- SPARK_ARCHIVE_BASENAME=$(basename $SPARK_ARCHIVE)
- export SPARK_HOME=$GITHUB_WORKSPACE/${SPARK_ARCHIVE_BASENAME%.*}
- rm -f $GITHUB_WORKSPACE/$SPARK_ARCHIVE
- docker system prune --all --force
- mvn verify $SCALA_PROFILE -D"$SPARK_PROFILE" -Pintegration-tests -pl
!hudi-flink-datasource/hudi-flink $MVN_ARGS
+# integration-tests:
Review Comment:
Revert the changes now as you're addressing it in a separate PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
