> On March 23, 2018, 4:33 p.m., Szabolcs Vasas wrote:
> > Hi Anna,
> > 
> > Awesome, thanks for the quick update.
> > 
> > I have tried running the third party tests today, it was successful, 
> > however there is a small problem with the cleanup.
> > If I run ./gradlew test first and then the third party tests then 
> > org.apache.sqoop.manager.sqlserver.SQLServerHiveImportTest.testHiveImportAsParquetWhenTableExistsWithIncompatibleSchema
> >  fails. This test case succeeds if I run the third party tests only after a 
> > ./gradlew clean. I think the problem is that SQLServerHiveImportTest 
> > extends HiveImportTest which does not do a proper Hive metastore database 
> > cleanup, so a table which has to be created by the test already exists.
> > Anyway, changing the Hive metastore DB to an in-memory DB solves the 
> > problem so I think can incorporate it into your patch 
> > (https://github.com/szvasas/sqoop/blob/gradletest2/testdata/hcatalog/conf/hive-site.xml#L37).
> >  I am not sure why but I did not experience this issue when running the 
> > tests with ant.
> > 
> > An interesting thing I have noticed is that when I try to execute one test 
> > case Gradle seems to start up lots of JVMs even after the test has 
> > successfully finished.
> > 
> > ./gradlew test --tests org.apache.sqoop.hive.TestHiveImport
> > :test
> > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; 
> > support was removed in 8.0
> > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option 
> > MaxPermSize=512m; support was removed in 8.0
> > objc[10180]: Class JavaLaunchHelper is implemented in both 
> > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/bin/java 
> > and 
> > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/lib/libinstrument.dylib.
> >  One of the two will be used. Which one is undefined.
> > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; 
> > support was removed in 8.0
> > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option 
> > MaxPermSize=512m; support was removed in 8.0
> > objc[10183]: Class JavaLaunchHelper is implemented in both 
> > ...
> > ...
> > ...
> > ...
> > 
> > org.apache.sqoop.hive.TestHiveImport > testDecimalMapColumnHive STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testDecimalMapColumnHive PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelims STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelims PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testGenerateOnly STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testGenerateOnly PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testHiveExitFails STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testHiveExitFails PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testDate STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testDate PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelimsReplacement 
> > STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelimsReplacement 
> > PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testCustomDelimiters STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testCustomDelimiters PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > 
> > testHiveDropAndReplaceOptionValidation STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > 
> > testHiveDropAndReplaceOptionValidation PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testCreateOverwriteHiveImport STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testCreateOverwriteHiveImport PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > 
> > testHiveImportAsParquetWhenTableExistsWithIncompatibleSchema STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > 
> > testHiveImportAsParquetWhenTableExistsWithIncompatibleSchema PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testCreateHiveImportAsParquet STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testCreateHiveImportAsParquet PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testCreateOnlyHiveImport STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testCreateOnlyHiveImport PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testAppendHiveImportAsParquet STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testAppendHiveImportAsParquet PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testNormalHiveImport STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testNormalHiveImport PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testNormalHiveImportAsParquet STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testNormalHiveImportAsParquet PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testImportWithBadPartitionKey STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testImportWithBadPartitionKey PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > 
> > testCreateOverwriteHiveImportAsParquet STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > 
> > testCreateOverwriteHiveImportAsParquet PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testImportHiveWithPartitions STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testImportHiveWithPartitions PASSED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testNumeric STARTED
> > 
> > org.apache.sqoop.hive.TestHiveImport > testNumeric PASSED
> > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; 
> > support was removed in 8.0
> > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option 
> > MaxPermSize=512m; support was removed in 8.0
> > objc[11118]: Class JavaLaunchHelper is implemented in both 
> > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/bin/java 
> > and 
> > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/lib/libinstrument.dylib.
> >  One of the two will be used. Which one is undefined.
> > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; 
> > support was removed in 8.0
> > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option 
> > MaxPermSize=512m; support was removed in 8.0
> > objc[11119]: Class JavaLaunchHelper is implemented in both 
> > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/bin/java 
> > and 
> > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/lib/libinstrument.dylib.
> >  One of the two will be used. Which one is undefined.
> > ...
> > ...
> > 
> > BUILD SUCCESSFUL
> > 
> > Total time: 2 mins 54.654 secs
> > 
> > 
> > I am not sure what happens here, my guess is that a new JVM is started for 
> > every test case even if they are not executed. Have you experienced this 
> > problem too?
> 
> Anna Szonyi wrote:
>     I've had similar issues running with the --tests option, I usually use 
> -Dtest.single, unfortunately if you have a buildSrc folder in your project 
> that seems to mess everything up for test.single, so I can rename that 
> directory to solve this for now. Oddly enough it doesn't affect 
> thirdParty.single.
>     Maybe in the long run we can also try to make test runs independent, so 
> we don't have to use forkevery 1...

Hey Szabi,

Please try with test.single with the new patch - you shouldn't face the same 
issues!

Thanks,
Anna


- Anna


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66067/#review199896
-----------------------------------------------------------


On April 13, 2018, 9:06 a.m., Anna Szonyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66067/
> -----------------------------------------------------------
> 
> (Updated April 13, 2018, 9:06 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: Sqoop-3052
>     https://issues.apache.org/jira/browse/Sqoop-3052
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> SQOOP-3052: Introduce gradle based build for Sqoop to make it more developer 
> friendly / open
> 
> 
> Diffs
> -----
> 
>   .gitignore 68cbe28731e613607c208824443d1edf256d9c8a 
>   COMPILING.txt 3b82250488256871352056e9061ad08fabbd7fc5 
>   build.gradle PRE-CREATION 
>   config/checkstyle/checkstyle-java-header.txt PRE-CREATION 
>   config/checkstyle/checkstyle-noframes.xsl PRE-CREATION 
>   config/checkstyle/checkstyle.xml PRE-CREATION 
>   gradle.properties PRE-CREATION 
>   gradle/customUnixStartScript.txt PRE-CREATION 
>   gradle/customWindowsStartScript.txt PRE-CREATION 
>   gradle/sqoop-package.gradle PRE-CREATION 
>   gradle/sqoop-version-gen.gradle PRE-CREATION 
>   gradle/wrapper/gradle-wrapper.jar PRE-CREATION 
>   gradle/wrapper/gradle-wrapper.properties PRE-CREATION 
>   gradlew PRE-CREATION 
>   gradlew.bat PRE-CREATION 
>   settings.gradle PRE-CREATION 
>   src/scripts/rat-violations.sh 1cfbc1502b24dd1b8b7e7ce21f0b5d1880c06556 
> 
> 
> Diff: https://reviews.apache.org/r/66067/diff/7/
> 
> 
> Testing
> -------
> 
> ran all new tasks, except for internal maven publishing
> 
> Notes:
> - To try it out you can call ./gradlew tasks --all to see all the tasks and 
> compare them to current tasks/artifacts.
> - Replaced cobertura with jacoco, as it's easier/cleaner to configure, easier 
> to combine all test results into a single report.
> - Generated pom.xml now has correct dependencies/versions
> - Script generation is currently hardcoded and not based on sqoop help, as 
> previously - though added the possiblity of hooking it in later
> 
> 
> Thanks,
> 
> Anna Szonyi
> 
>

Reply via email to