[jira] [Commented] (SQOOP-3404) Categorize all tests in the project

2018-11-19 Thread Hudson (JIRA) 


[ 
https://issues.apache.org/jira/browse/SQOOP-3404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691553#comment-16691553
 ] 

Hudson commented on SQOOP-3404:
---

FAILURE: Integrated in Jenkins build Sqoop-hadoop200 #1228 (See 
[https://builds.apache.org/job/Sqoop-hadoop200/1228/])
SQOOP-3404: Categorize all tests in the project (fero: 
[https://git-wip-us.apache.org/repos/asf?p=sqoop.git=commit=bb9c2dd85b0cac84503e69906a05c76d6a0413e1])
* (edit) 
src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java
* (edit) 
src/test/org/apache/sqoop/tool/TestS3IncrementalImportOptionValidations.java
* (edit) src/test/org/apache/sqoop/s3/TestS3ImportWithHadoopCredProvider.java
* (edit) src/test/org/apache/sqoop/s3/TestS3TextImport.java
* (edit) 
src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeDatasetBinaryRecord.java
* (edit) src/test/org/apache/sqoop/s3/TestS3AvroImport.java
* (edit) src/test/org/apache/sqoop/s3/TestS3IncrementalAppendParquetImport.java
* (edit) src/test/org/apache/sqoop/manager/TestMainframeManager.java
* (edit) 
src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java
* (edit) src/test/org/apache/sqoop/s3/TestS3ExternalHiveTableImport.java
* (edit) 
src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeFTPFileGdgEntryParser.java
* (edit) src/test/org/apache/sqoop/s3/TestS3IncrementalAppendTextImport.java
* (edit) src/test/org/apache/sqoop/s3/TestS3IncrementalMergeParquetImport.java
* (edit) 
src/test/org/apache/sqoop/s3/TestS3IncrementalAppendSequenceFileImport.java
* (edit) src/test/org/apache/sqoop/s3/TestS3ParquetImport.java
* (edit) 
src/test/org/apache/sqoop/manager/oracle/TestOraOopDBInputSplitGetDebugDetails.java
* (edit) src/test/org/apache/sqoop/s3/TestS3IncrementalAppendAvroImport.java
* (add) src/test/org/apache/sqoop/testcategories/thirdpartytest/S3Test.java
* (edit) src/test/org/apache/sqoop/importjob/SplitByImportTest.java
* (edit) src/test/org/apache/sqoop/s3/TestS3IncrementalMergeTextImport.java
* (edit) src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java
* (edit) COMPILING.txt
* (edit) build.gradle


> Categorize all tests in the project
> ---
>
> Key: SQOOP-3404
> URL: https://issues.apache.org/jira/browse/SQOOP-3404
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: SQOOP-3404.patch
>
>
> SQOOP-3104 has introduced test categories but while it was under review many 
> other patches with new test cases were committed.
> The task is to make sure that all of the new tests are properly categorized 
> and the test tasks are cleaned up.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (SQOOP-3230) Add a build step which starts RDBMS docker containers before the third party tests are executed

2018-11-19 Thread Szabolcs Vasas (JIRA) 


 [ 
https://issues.apache.org/jira/browse/SQOOP-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Szabolcs Vasas resolved SQOOP-3230.
---
Resolution: Won't Do

Resolving this ticked with 'Won't do' as it seems we will choose a different 
path.

The containers will be started up by a script which will be invoked by the CI 
system.

> Add a build step which starts RDBMS docker containers before the third party 
> tests are executed
> ---
>
> Key: SQOOP-3230
> URL: https://issues.apache.org/jira/browse/SQOOP-3230
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Szabolcs Vasas
>Priority: Major
>
> The task is to enable automatic third party test execution using our build 
> tool.
> Ant should start up all the necessary containers before the tests are 
> executed and shut them down after they are completed.
> The solution should address (at least) the below problems:
> * Find a way to start docker containers from ant (probably a Java library)
> * Random port support: we should make the solution more robust by not relying 
> on the default ports exposed by the containers since they may be occupied on 
> the host running the test cases.
> * Health check logic should be implemented: the framework should make sure 
> that the containers are properly started before it executes the test cases.
> * The solution should not be too tied to Ant since Gradle build will be 
> introduced soon



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3404) Categorize all tests in the project

2018-11-19 Thread ASF subversion and git services (JIRA) 


[ 
https://issues.apache.org/jira/browse/SQOOP-3404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691478#comment-16691478
 ] 

ASF subversion and git services commented on SQOOP-3404:


Commit bb9c2dd85b0cac84503e69906a05c76d6a0413e1 in sqoop's branch 
refs/heads/trunk from [~fero]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=bb9c2dd ]

SQOOP-3404: Categorize all tests in the project

(Szabolcs Vasas via Fero Szabo)


> Categorize all tests in the project
> ---
>
> Key: SQOOP-3404
> URL: https://issues.apache.org/jira/browse/SQOOP-3404
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: SQOOP-3404.patch
>
>
> SQOOP-3104 has introduced test categories but while it was under review many 
> other patches with new test cases were committed.
> The task is to make sure that all of the new tests are properly categorized 
> and the test tasks are cleaned up.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69346: Categorize all tests in the project

2018-11-19 Thread Szabolcs Vasas 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69346/
---

(Updated Nov. 19, 2018, 9:10 a.m.)


Review request for Sqoop.


Bugs: SQOOP-3404
https://issues.apache.org/jira/browse/SQOOP-3404


Repository: sqoop-trunk


Description
---

- All tests are categorized now
- Introduced S3Test category as a subcategory of ThirdPartyTest
- Reorganized test tasks: we have unitTest, integrationTest, kerberizedTest, 
test, s3Test, allTest tasks now
- jacocoTestReport task is fixed to contain the coverage information of the 
kerberizedTest task too. This is needed because the kerberizedTest needs the 
forkEvery parameter to be set to 1 and because of that it has to be a separate 
task which generates separate coverage information too. However it is 
automatically triggered after the test task so the invocation is more 
convenient for the tester.


Diffs (updated)
-

  COMPILING.txt 835ba33b1e89158bed0e05698b188ab3323eb881 
  build.gradle cb9eeca74bbf278c3e5fd15de608d8c37c917ddb 
  src/test/org/apache/sqoop/importjob/SplitByImportTest.java 
c6fe4f2e8a80c96ad667f4fe4a26510af96562dc 
  src/test/org/apache/sqoop/manager/TestMainframeManager.java 
c84f05f660c396a06a5031e00abdae77ffbcf2aa 
  
src/test/org/apache/sqoop/manager/oracle/TestOraOopDBInputSplitGetDebugDetails.java
 6f33ad3b650436b7f268b4ef5bfd451bd5e6958e 
  
src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java
 5e558717c0d43301ecbf81a37d5ee3fd35756d65 
  
src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java
 1a6943786834d27f27523f484d76cf678f18cf48 
  
src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeDatasetBinaryRecord.java
 b4cba28c3611400b5c4227a5166b6c91e9152dc4 
  
src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeFTPFileGdgEntryParser.java
 521a04266e8806321fe7aa6a89c064f369174523 
  src/test/org/apache/sqoop/s3/TestS3AvroImport.java 
7f5f5d62c5cab10f932aa22c3a713b13fefc2b58 
  src/test/org/apache/sqoop/s3/TestS3ExternalHiveTableImport.java 
0c3161e5a783446e35f4754124f86715d103ec0b 
  src/test/org/apache/sqoop/s3/TestS3ImportWithHadoopCredProvider.java 
3a0d6365dc20f8eef5bdd67a4a2dc9c68ff74d7f 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendAvroImport.java 
5faf59ea80c48fe025294cabd100e7d176032138 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendParquetImport.java 
a4f986423ea299716a29f9d02f7c8453a7f2ba02 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendSequenceFileImport.java 
d271588c5af060bbc3d301a845f45c46d0f6a2ba 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendTextImport.java 
52d89c775b5f1219471df44d222fd92a59ed408c 
  src/test/org/apache/sqoop/s3/TestS3IncrementalMergeParquetImport.java 
39238c5fab56b54a85dde5aed0d4bb2c77382fa6 
  src/test/org/apache/sqoop/s3/TestS3IncrementalMergeTextImport.java 
597e3def2cc33adebeeb3bc1ee35ad8a7f4b990d 
  src/test/org/apache/sqoop/s3/TestS3ParquetImport.java 
c9785d816d4a7a5870d74c51a9faa229f6d3818e 
  src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java 
bba8b74ebe639df26e977abf377f4904144dcfaa 
  src/test/org/apache/sqoop/s3/TestS3TextImport.java 
114f97cbb8857a7633cae5d030769ac4a90e36aa 
  src/test/org/apache/sqoop/testcategories/thirdpartytest/S3Test.java 
PRE-CREATION 
  src/test/org/apache/sqoop/tool/TestS3IncrementalImportOptionValidations.java 
7745f1b07e6d6c457b0164deeace12587ec058d0 


Diff: https://reviews.apache.org/r/69346/diff/4/

Changes: https://reviews.apache.org/r/69346/diff/3-4/


Testing
---

./gradlew unitTest
./gradlew integrationTest
./gradlew kerberizedTest
./gradlew ... s3Test
./gradlew test
./gradlew ... thirdPartyTest
./gradlew allTest


Thanks,

Szabolcs Vasas

Re: Review Request 69346: Categorize all tests in the project

2018-11-19 Thread Szabolcs Vasas 


> On Nov. 16, 2018, 9:43 p.m., Fero Szabo wrote:
> > COMPILING.txt
> > Line 465 (original), 633 (patched)
> > 
> >
> > I believe you require a fully qualified classname here, right? A 
> > concrete example here might come in handy. What do you think?

You don't have to specify a full class name here, and you can use wildcards 
too. I have updated the example and removed -Dtest.single because it is 
deprecated now.


- Szabolcs


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69346/#review210576
---


On Nov. 19, 2018, 9:10 a.m., Szabolcs Vasas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69346/
> ---
> 
> (Updated Nov. 19, 2018, 9:10 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-3404
> https://issues.apache.org/jira/browse/SQOOP-3404
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> ---
> 
> - All tests are categorized now
> - Introduced S3Test category as a subcategory of ThirdPartyTest
> - Reorganized test tasks: we have unitTest, integrationTest, kerberizedTest, 
> test, s3Test, allTest tasks now
> - jacocoTestReport task is fixed to contain the coverage information of the 
> kerberizedTest task too. This is needed because the kerberizedTest needs the 
> forkEvery parameter to be set to 1 and because of that it has to be a 
> separate task which generates separate coverage information too. However it 
> is automatically triggered after the test task so the invocation is more 
> convenient for the tester.
> 
> 
> Diffs
> -
> 
>   COMPILING.txt 835ba33b1e89158bed0e05698b188ab3323eb881 
>   build.gradle cb9eeca74bbf278c3e5fd15de608d8c37c917ddb 
>   src/test/org/apache/sqoop/importjob/SplitByImportTest.java 
> c6fe4f2e8a80c96ad667f4fe4a26510af96562dc 
>   src/test/org/apache/sqoop/manager/TestMainframeManager.java 
> c84f05f660c396a06a5031e00abdae77ffbcf2aa 
>   
> src/test/org/apache/sqoop/manager/oracle/TestOraOopDBInputSplitGetDebugDetails.java
>  6f33ad3b650436b7f268b4ef5bfd451bd5e6958e 
>   
> src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java
>  5e558717c0d43301ecbf81a37d5ee3fd35756d65 
>   
> src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java
>  1a6943786834d27f27523f484d76cf678f18cf48 
>   
> src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeDatasetBinaryRecord.java
>  b4cba28c3611400b5c4227a5166b6c91e9152dc4 
>   
> src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeFTPFileGdgEntryParser.java
>  521a04266e8806321fe7aa6a89c064f369174523 
>   src/test/org/apache/sqoop/s3/TestS3AvroImport.java 
> 7f5f5d62c5cab10f932aa22c3a713b13fefc2b58 
>   src/test/org/apache/sqoop/s3/TestS3ExternalHiveTableImport.java 
> 0c3161e5a783446e35f4754124f86715d103ec0b 
>   src/test/org/apache/sqoop/s3/TestS3ImportWithHadoopCredProvider.java 
> 3a0d6365dc20f8eef5bdd67a4a2dc9c68ff74d7f 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendAvroImport.java 
> 5faf59ea80c48fe025294cabd100e7d176032138 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendParquetImport.java 
> a4f986423ea299716a29f9d02f7c8453a7f2ba02 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendSequenceFileImport.java 
> d271588c5af060bbc3d301a845f45c46d0f6a2ba 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendTextImport.java 
> 52d89c775b5f1219471df44d222fd92a59ed408c 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalMergeParquetImport.java 
> 39238c5fab56b54a85dde5aed0d4bb2c77382fa6 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalMergeTextImport.java 
> 597e3def2cc33adebeeb3bc1ee35ad8a7f4b990d 
>   src/test/org/apache/sqoop/s3/TestS3ParquetImport.java 
> c9785d816d4a7a5870d74c51a9faa229f6d3818e 
>   src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java 
> bba8b74ebe639df26e977abf377f4904144dcfaa 
>   src/test/org/apache/sqoop/s3/TestS3TextImport.java 
> 114f97cbb8857a7633cae5d030769ac4a90e36aa 
>   src/test/org/apache/sqoop/testcategories/thirdpartytest/S3Test.java 
> PRE-CREATION 
>   
> src/test/org/apache/sqoop/tool/TestS3IncrementalImportOptionValidations.java 
> 7745f1b07e6d6c457b0164deeace12587ec058d0 
> 
> 
> Diff: https://reviews.apache.org/r/69346/diff/4/
> 
> 
> Testing
> ---
> 
> ./gradlew unitTest
> ./gradlew integrationTest
> ./gradlew kerberizedTest
> ./gradlew ... s3Test
> ./gradlew test
> ./gradlew ... thirdPartyTest
> ./gradlew allTest
> 
> 
> Thanks,
> 
> Szabolcs Vasas
> 
>