> On Sept. 11, 2014, 8:25 a.m., Jarek Cecho wrote: > > Pretty much all integration tests are failing for me on missing input files > > (details below). We have currently majority of the tests disabled, enabled > > but failing integration test seems better to me as long as all unit tests > > are passing - and that is the case here, so I'll go ahead and commit this > > patch. > > > > > > > > testBasic(org.apache.sqoop.integration.connector.jdbc.generic.FromRDBMSToHDFSTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.FromRDBMSToHDFSTest/testBasic > > does not exist. > > > > testColumns(org.apache.sqoop.integration.connector.jdbc.generic.FromRDBMSToHDFSTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.FromRDBMSToHDFSTest/testColumns > > does not exist. > > > > testSplitter[id-13-3](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[id-13-3] > > does not exist. > > > > testSplitter[id-13-5](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[id-13-5] > > does not exist. > > > > testSplitter[id-13-10](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[id-13-10] > > does not exist. > > > > testSplitter[id-13-13](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[id-13-13] > > does not exist. > > > > testSplitter[code_name-13-3](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[code_name-13-3] > > does not exist. > > > > testSplitter[code_name-13-5](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[code_name-13-5] > > does not exist. > > > > testSplitter[code_name-13-10](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[code_name-13-10] > > does not exist. > > > > testSplitter[code_name-13-13](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[code_name-13-13] > > does not exist. > > > > testSplitter[version-13-3](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[version-13-3] > > does not exist. > > > > testSplitter[version-13-5](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[version-13-5] > > does not exist. > > > > testSplitter[version-13-10](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[version-13-10] > > does not exist. > > > > testSplitter[version-13-13](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[version-13-13] > > does not exist. > > > > testSplitter[release_date-13-3](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[release_date-13-3] > > does not exist. > > > > testSplitter[release_date-13-5](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[release_date-13-5] > > does not exist. > > > > testSplitter[release_date-13-10](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[release_date-13-10] > > does not exist. > > > > testSplitter[release_date-13-13](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[release_date-13-13] > > does not exist. > > > > testSplitter[lts-2-3](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[lts-2-3] > > does not exist. > > > > testSplitter[lts-2-5](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[lts-2-5] > > does not exist. > > > > testSplitter[lts-2-10](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[lts-2-10] > > does not exist. > > > > testSplitter[lts-2-13](org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest): > > File > > /mapreduce-job-io/org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest/testSplitter[lts-2-13] > > does not exist.
I was able to get this working on my machine actually. The problem I ran into was that a LocalFileSystem was being used and the test cases were trying to create the directory /mapreduce-job-io. > On Sept. 11, 2014, 8:25 a.m., Jarek Cecho wrote: > > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/TableStagedRDBMSTest.java, > > line 33 > > <https://reviews.apache.org/r/25491/diff/4/?file=684803#file684803line33> > > > > I'm not particulary found of this dual testing where we are using HDFS > > connector to verify JDBC connector. > > > > I think that we should create a special test "FROM" connector that we > > can configure as a generator and then "TO" connector that will serve as > > some sort of simple storage that we can query. > > > > This way we can easily test all connectors with the same testing > > infrastructure and we won't be depending that connector A does it's job in > > order to test connector B. > > > > I don't think that it's a show stopper for this JIRA though, so let's > > cover it by separate one. Makes sense. - Abraham ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25491/#review53022 ----------------------------------------------------------- On Sept. 10, 2014, 9:47 p.m., Abraham Elmahrek wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/25491/ > ----------------------------------------------------------- > > (Updated Sept. 10, 2014, 9:47 p.m.) > > > Review request for Sqoop. > > > Bugs: SQOOP-1506 > https://issues.apache.org/jira/browse/SQOOP-1506 > > > Repository: sqoop-SQOOP-1367 > > > Description > ------- > > commit 1d865164dca2dd7f359dff837013acbc3dfe9ace > Author: Abraham Elmahrek <[email protected]> > Date: Mon Sep 8 22:33:42 2014 -0700 > > SQOOP-1506: Sqoop2: From/To: Re-enable all tests > > Left out a couple in TestJdbcRepository. > Left out proper testing in TableStagedRDBMSTest. > Depends on SQOOP-1488. > Removed MapreduceExecutionEngineTest since it seems > to exist in HdfsConnectors. > > :100644 100644 b5e7e61... 731f6cc... M > client/src/test/java/org/apache/sqoop/client/TestSqoopClient.java > :100644 100644 a50c815... c0d678a... M > common/src/main/java/org/apache/sqoop/model/MValidatedElement.java > :000000 100644 0000000... 9d11a2e... A > common/src/main/java/org/apache/sqoop/validation/validators/NullOrContains.java > :000000 100644 0000000... 3c64346... A > common/src/test/java/org/apache/sqoop/validation/validators/TestNullOrContains.java > :100644 100644 a25f344... db79acc... M > connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcPartitioner.java > :100644 100644 ac72546... 72902a2... M > connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/configuration/FromTableForm.java > :100644 100644 4831cf8... eb6fcf1... M > connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestToInitializer.java > :100644 100644 5a924f9... 61676fe... M > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java > :100644 000000 d4aaa0a... 0000000... D > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/StorageType.java > :100644 100644 79cf1f1... 644bbc2... M > connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java > :100644 100644 830606a... 897d3c7... M > core/src/main/java/org/apache/sqoop/framework/configuration/ConnectionConfiguration.java > :100644 100644 90395ac... 69c1b56... M > core/src/test/java/org/apache/sqoop/framework/TestFrameworkValidator.java > :100644 100644 f19e01c... 50daa62... M > core/src/test/java/org/apache/sqoop/repository/TestJdbcRepository.java > :100644 100644 1d60ba3... 3065680... M > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java > :100644 000000 f70e9bd... 0000000... D > execution/mapreduce/src/test/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngineTest.java > :100644 100644 fae9fe8... 9d9b508... M pom.xml > :100644 100644 e2f1675... 9ecc9da... M > test/src/main/java/org/apache/sqoop/test/minicluster/TomcatSqoopMiniCluster.java > :100644 100644 af0f299... 8a699f5... M > test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java > :100644 100644 59c5f15... ec650e1... M > test/src/main/java/org/apache/sqoop/test/utils/HdfsUtils.java > :000000 100644 0000000... c01aa80... A > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java > :000000 100644 0000000... f976f29... A > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java > :000000 100644 0000000... fac7e8b... A > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/PartitionerTest.java > :100644 000000 39b48d8... 0000000... D > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/TableExportTest.java > :100644 000000 9171b8e... 0000000... D > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/TableImportTest.java > :000000 100644 0000000... cb782c7... A > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/TableStagedRDBMSTest.java > :100644 000000 cb028bb... 0000000... D > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/exports/TableStagedExportTest.java > :100644 000000 a0a4022... 0000000... D > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/imports/PartitionerTest.java > :100644 100644 1a7a3a8... 3c01cb0... M > test/src/test/java/org/apache/sqoop/integration/server/SubmissionWithDisabledModelObjectsTest.java > :100644 100644 5ebe95f... cea24b9... M > test/src/test/java/org/apache/sqoop/integration/server/VersionTest.java > > > Diffs > ----- > > client/src/test/java/org/apache/sqoop/client/TestSqoopClient.java b5e7e61 > common/src/main/java/org/apache/sqoop/model/MValidatedElement.java a50c815 > > common/src/main/java/org/apache/sqoop/validation/validators/NullOrContains.java > PRE-CREATION > > common/src/test/java/org/apache/sqoop/validation/validators/TestNullOrContains.java > PRE-CREATION > > connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/configuration/FromTableForm.java > ac72546 > > connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestToInitializer.java > 4831cf8 > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java > 5a924f9 > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/StorageType.java > d4aaa0a > > connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java > 6ed4087 > > connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsBase.java > 0cc2b8b > > core/src/main/java/org/apache/sqoop/framework/configuration/ConnectionConfiguration.java > 830606a > core/src/test/java/org/apache/sqoop/framework/TestFrameworkValidator.java > 90395ac > core/src/test/java/org/apache/sqoop/repository/TestJdbcRepository.java > f19e01c > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java > 1d60ba3 > > execution/mapreduce/src/test/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngineTest.java > f70e9bd > pom.xml fae9fe8 > > test/src/main/java/org/apache/sqoop/test/minicluster/TomcatSqoopMiniCluster.java > e2f1675 > test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java > af0f299 > test/src/main/java/org/apache/sqoop/test/utils/HdfsUtils.java 59c5f15 > > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java > PRE-CREATION > > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java > PRE-CREATION > > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/PartitionerTest.java > PRE-CREATION > > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/TableExportTest.java > 39b48d8 > > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/TableImportTest.java > 9171b8e > > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/TableStagedRDBMSTest.java > PRE-CREATION > > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/exports/TableStagedExportTest.java > cb028bb > > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/imports/PartitionerTest.java > a0a4022 > > test/src/test/java/org/apache/sqoop/integration/server/SubmissionWithDisabledModelObjectsTest.java > 1a7a3a8 > test/src/test/java/org/apache/sqoop/integration/server/VersionTest.java > 5ebe95f > > Diff: https://reviews.apache.org/r/25491/diff/ > > > Testing > ------- > > mvn integration-test passes. > > > Thanks, > > Abraham Elmahrek > >
