Hi, For me also the test runs but looking at the error, it looks like no work or deduping is done, which is strange. Here is the error ->
[*ERROR*] *Tests **run: 1*, *Failures: 1*, Errors: 0, Skipped: 0, Time elapsed: 8.425 s* <<< FAILURE!* - in org.apache.hudi.cli.integ. *ITTestRepairsCommand* [*ERROR*] org.apache.hudi.cli.integ.ITTestRepairsCommand.testDeduplicateWithReal Time elapsed: 7.588 s <<< FAILURE! org.opentest4j.AssertionFailedError: expected: <200> but was: <210> at org.apache.hudi.cli.integ.ITTestRepairsCommand.testDeduplicateWithReal(ITTestRepairsCommand.java:254) Initially also 210 records are present, so effectively the test runs but without doing anything. There is no other error apart from the above one. I feel integration tests for hudi-cli should also be running in docker like other integration tests rather than running on local spark installation. That would help ensure such issues do not come up in future. Thoughts? On Sun, May 17, 2020 at 7:59 PM hddong <[email protected]> wrote: > Hi Pratyaksh, > > Dose it throws the same Exception? And can you check if sparkLauncher > throws the same Exception. Most time, ITTest failed due to some config of > local spark. > I had got this Exception before, but it run successfully after `mvn clean > package ...`. > > Regards > hddong > > Pratyaksh Sharma <[email protected]> 于2020年5月17日周日 下午8:42写道： > > > Hi hddong, > > > > Strange but nothing seems to work for me. I tried doing mvn clean and > then > > run travis tests. Also I tried running the command `mvn clean package > > -DskipTests -DskipITs -Pspark-shade-unbundle-avro` first and then run the > > test using `mvn -Dtest=ITTestRepairsCommand#testDeduplicateWithReal > > -DfailIfNoTests=false test`. But both of them did not work. I have spark > > installation and I am setting the SPARK_HOME to > > /usr/local/Cellar/apache-spark/2.4.5. > > > > On Sun, May 17, 2020 at 9:00 AM hddong <[email protected]> wrote: > > > > > Hi Pratyaksh, > > > > > > run_travis_tests,sh not run `mvn clean`, You can try to run `mvn > > > clean` manually > > > before integration test. > > > > > > BTW, if you use IDEA, you can do > > > `mvn clean package -DskipTests -DskipITs -Pspark-shade-unbundle-avro` > > > first, > > > then just run integration test in IDEA like unit test does. > > > > > > But, there are something to notice here: you need a runnable spark and > > > SPARK_HOME should in env. > > > > > > Regards > > > hddong > > > > > >
