> On March 23, 2018, 4:33 p.m., Szabolcs Vasas wrote: > > Hi Anna, > > > > Awesome, thanks for the quick update. > > > > I have tried running the third party tests today, it was successful, > > however there is a small problem with the cleanup. > > If I run ./gradlew test first and then the third party tests then > > org.apache.sqoop.manager.sqlserver.SQLServerHiveImportTest.testHiveImportAsParquetWhenTableExistsWithIncompatibleSchema > > fails. This test case succeeds if I run the third party tests only after a > > ./gradlew clean. I think the problem is that SQLServerHiveImportTest > > extends HiveImportTest which does not do a proper Hive metastore database > > cleanup, so a table which has to be created by the test already exists. > > Anyway, changing the Hive metastore DB to an in-memory DB solves the > > problem so I think can incorporate it into your patch > > (https://github.com/szvasas/sqoop/blob/gradletest2/testdata/hcatalog/conf/hive-site.xml#L37). > > I am not sure why but I did not experience this issue when running the > > tests with ant. > > > > An interesting thing I have noticed is that when I try to execute one test > > case Gradle seems to start up lots of JVMs even after the test has > > successfully finished. > > > > ./gradlew test --tests org.apache.sqoop.hive.TestHiveImport > > :test > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; > > support was removed in 8.0 > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option > > MaxPermSize=512m; support was removed in 8.0 > > objc[10180]: Class JavaLaunchHelper is implemented in both > > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/bin/java > > and > > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/lib/libinstrument.dylib. > > One of the two will be used. Which one is undefined. > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; > > support was removed in 8.0 > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option > > MaxPermSize=512m; support was removed in 8.0 > > objc[10183]: Class JavaLaunchHelper is implemented in both > > ... > > ... > > ... > > ... > > > > org.apache.sqoop.hive.TestHiveImport > testDecimalMapColumnHive STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testDecimalMapColumnHive PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelims STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelims PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testGenerateOnly STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testGenerateOnly PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testHiveExitFails STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testHiveExitFails PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testDate STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testDate PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelimsReplacement > > STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelimsReplacement > > PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testCustomDelimiters STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testCustomDelimiters PASSED > > > > org.apache.sqoop.hive.TestHiveImport > > > testHiveDropAndReplaceOptionValidation STARTED > > > > org.apache.sqoop.hive.TestHiveImport > > > testHiveDropAndReplaceOptionValidation PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testCreateOverwriteHiveImport STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testCreateOverwriteHiveImport PASSED > > > > org.apache.sqoop.hive.TestHiveImport > > > testHiveImportAsParquetWhenTableExistsWithIncompatibleSchema STARTED > > > > org.apache.sqoop.hive.TestHiveImport > > > testHiveImportAsParquetWhenTableExistsWithIncompatibleSchema PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testCreateHiveImportAsParquet STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testCreateHiveImportAsParquet PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testCreateOnlyHiveImport STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testCreateOnlyHiveImport PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testAppendHiveImportAsParquet STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testAppendHiveImportAsParquet PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testNormalHiveImport STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testNormalHiveImport PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testNormalHiveImportAsParquet STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testNormalHiveImportAsParquet PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testImportWithBadPartitionKey STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testImportWithBadPartitionKey PASSED > > > > org.apache.sqoop.hive.TestHiveImport > > > testCreateOverwriteHiveImportAsParquet STARTED > > > > org.apache.sqoop.hive.TestHiveImport > > > testCreateOverwriteHiveImportAsParquet PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testImportHiveWithPartitions STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testImportHiveWithPartitions PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testNumeric STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testNumeric PASSED > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; > > support was removed in 8.0 > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option > > MaxPermSize=512m; support was removed in 8.0 > > objc[11118]: Class JavaLaunchHelper is implemented in both > > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/bin/java > > and > > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/lib/libinstrument.dylib. > > One of the two will be used. Which one is undefined. > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; > > support was removed in 8.0 > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option > > MaxPermSize=512m; support was removed in 8.0 > > objc[11119]: Class JavaLaunchHelper is implemented in both > > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/bin/java > > and > > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/lib/libinstrument.dylib. > > One of the two will be used. Which one is undefined. > > ... > > ... > > > > BUILD SUCCESSFUL > > > > Total time: 2 mins 54.654 secs > > > > > > I am not sure what happens here, my guess is that a new JVM is started for > > every test case even if they are not executed. Have you experienced this > > problem too? > > Anna Szonyi wrote: > I've had similar issues running with the --tests option, I usually use > -Dtest.single, unfortunately if you have a buildSrc folder in your project > that seems to mess everything up for test.single, so I can rename that > directory to solve this for now. Oddly enough it doesn't affect > thirdParty.single. > Maybe in the long run we can also try to make test runs independent, so > we don't have to use forkevery 1...
Hey Szabi, Please try with test.single with the new patch - you shouldn't face the same issues! Thanks, Anna - Anna ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66067/#review199896 ----------------------------------------------------------- On April 13, 2018, 9:06 a.m., Anna Szonyi wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/66067/ > ----------------------------------------------------------- > > (Updated April 13, 2018, 9:06 a.m.) > > > Review request for Sqoop. > > > Bugs: Sqoop-3052 > https://issues.apache.org/jira/browse/Sqoop-3052 > > > Repository: sqoop-trunk > > > Description > ------- > > SQOOP-3052: Introduce gradle based build for Sqoop to make it more developer > friendly / open > > > Diffs > ----- > > .gitignore 68cbe28731e613607c208824443d1edf256d9c8a > COMPILING.txt 3b82250488256871352056e9061ad08fabbd7fc5 > build.gradle PRE-CREATION > config/checkstyle/checkstyle-java-header.txt PRE-CREATION > config/checkstyle/checkstyle-noframes.xsl PRE-CREATION > config/checkstyle/checkstyle.xml PRE-CREATION > gradle.properties PRE-CREATION > gradle/customUnixStartScript.txt PRE-CREATION > gradle/customWindowsStartScript.txt PRE-CREATION > gradle/sqoop-package.gradle PRE-CREATION > gradle/sqoop-version-gen.gradle PRE-CREATION > gradle/wrapper/gradle-wrapper.jar PRE-CREATION > gradle/wrapper/gradle-wrapper.properties PRE-CREATION > gradlew PRE-CREATION > gradlew.bat PRE-CREATION > settings.gradle PRE-CREATION > src/scripts/rat-violations.sh 1cfbc1502b24dd1b8b7e7ce21f0b5d1880c06556 > > > Diff: https://reviews.apache.org/r/66067/diff/7/ > > > Testing > ------- > > ran all new tasks, except for internal maven publishing > > Notes: > - To try it out you can call ./gradlew tasks --all to see all the tasks and > compare them to current tasks/artifacts. > - Replaced cobertura with jacoco, as it's easier/cleaner to configure, easier > to combine all test results into a single report. > - Generated pom.xml now has correct dependencies/versions > - Script generation is currently hardcoded and not based on sqoop help, as > previously - though added the possiblity of hooking it in later > > > Thanks, > > Anna Szonyi > >
