[jira] [Comment Edited] (SQOOP-3300) Implement JDBC and Kerberos tools for HiveServer2 support
[
https://issues.apache.org/jira/browse/SQOOP-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411449#comment-16411449
]
Boglarka Egyed edited comment on SQOOP-3300 at 3/23/18 6:09 PM:
Unfortunately one of the newly added unit tests (TestKerberosAuthenticator) is
failing in the Apache Jenkins while passing locally with an OutOfMemory
exception:
{noformat}
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 50.61 sec
[junit] Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
[junit] at java.util.Arrays.copyOf(Arrays.java:2367)
[junit] at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
[junit] at
java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
[junit] at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
[junit] at java.lang.StringBuffer.append(StringBuffer.java:237)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.PlainJUnitResultFormatter.endTestSuite(PlainJUnitResultFormatter.java:141)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.fireEndTestSuite(JUnitTestRunner.java:731)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:553)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
{noformat}
As this is not a functional defect in Sqoop I am resolving this ticket but
[~vasas] I would like to ask you to open another sub-task under SQOOP-3299 to
increase the max memory size for JUnit and submit a patch for it as soon as
possible. Also, please close the related Review Request.
Thanks,
Bogi
was (Author: boglarkaegyed):
Unfortunately one of the newly added unit tests (TestKerberosAuthenticator) is
failing in the Apache Jenkins while passing locally with an OutOfMemory
exception:
{noformat}
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 50.61 sec
[junit] Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
[junit] at java.util.Arrays.copyOf(Arrays.java:2367)
[junit] at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
[junit] at
java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
[junit] at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
[junit] at java.lang.StringBuffer.append(StringBuffer.java:237)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.PlainJUnitResultFormatter.endTestSuite(PlainJUnitResultFormatter.java:141)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.fireEndTestSuite(JUnitTestRunner.java:731)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:553)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
{noformat}
As this is not a functional defect in Sqoop I am resolving this ticket but
[~vasas] I would like to ask you to open another sub-task under SQOOP-3299 to
increase the Java heap space for this test and submit a patch for it as soon as
possible. Also, please close the related Review Request.
Thanks,
Bogi
> Implement JDBC and Kerberos tools for HiveServer2 support
> -
>
> Key: SQOOP-3300
> URL: https://issues.apache.org/jira/browse/SQOOP-3300
> Project: Sqoop
> Issue Type: Sub-task
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: SQOOP-3300.patch
>
>
> The idea of the Sqoop HS2 support is to connect to HS2 using JDBC and execute
> the Hive commands on this connection. Sqoop should also support Kerberos
> authentication when building this JDBC connection.
> The goal of this JIRA is to implement the necessary classes for building JDBC
> connections and authenticating with Kerberos.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Review Request 66067: SQOOP-3052: Introduce gradle-based build for Sqoop to make it more developer friendly / open
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66067/#review199896 --- Hi Anna, Awesome, thanks for the quick update. I have tried running the third party tests today, it was successful, however there is a small problem with the cleanup. If I run ./gradlew test first and then the third party tests then org.apache.sqoop.manager.sqlserver.SQLServerHiveImportTest.testHiveImportAsParquetWhenTableExistsWithIncompatibleSchema fails. This test case succeeds if I run the third party tests only after a ./gradlew clean. I think the problem is that SQLServerHiveImportTest extends HiveImportTest which does not do a proper Hive metastore database cleanup, so a table which has to be created by the test already exists. Anyway, changing the Hive metastore DB to an in-memory DB solves the problem so I think can incorporate it into your patch (https://github.com/szvasas/sqoop/blob/gradletest2/testdata/hcatalog/conf/hive-site.xml#L37). I am not sure why but I did not experience this issue when running the tests with ant. An interesting thing I have noticed is that when I try to execute one test case Gradle seems to start up lots of JVMs even after the test has successfully finished. ./gradlew test --tests org.apache.sqoop.hive.TestHiveImport :test Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; support was removed in 8.0 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0 objc[10180]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined. Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; support was removed in 8.0 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0 objc[10183]: Class JavaLaunchHelper is implemented in both ... ... ... ... org.apache.sqoop.hive.TestHiveImport > testDecimalMapColumnHive STARTED org.apache.sqoop.hive.TestHiveImport > testDecimalMapColumnHive PASSED org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelims STARTED org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelims PASSED org.apache.sqoop.hive.TestHiveImport > testGenerateOnly STARTED org.apache.sqoop.hive.TestHiveImport > testGenerateOnly PASSED org.apache.sqoop.hive.TestHiveImport > testHiveExitFails STARTED org.apache.sqoop.hive.TestHiveImport > testHiveExitFails PASSED org.apache.sqoop.hive.TestHiveImport > testDate STARTED org.apache.sqoop.hive.TestHiveImport > testDate PASSED org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelimsReplacement STARTED org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelimsReplacement PASSED org.apache.sqoop.hive.TestHiveImport > testCustomDelimiters STARTED org.apache.sqoop.hive.TestHiveImport > testCustomDelimiters PASSED org.apache.sqoop.hive.TestHiveImport > testHiveDropAndReplaceOptionValidation STARTED org.apache.sqoop.hive.TestHiveImport > testHiveDropAndReplaceOptionValidation PASSED org.apache.sqoop.hive.TestHiveImport > testCreateOverwriteHiveImport STARTED org.apache.sqoop.hive.TestHiveImport > testCreateOverwriteHiveImport PASSED org.apache.sqoop.hive.TestHiveImport > testHiveImportAsParquetWhenTableExistsWithIncompatibleSchema STARTED org.apache.sqoop.hive.TestHiveImport > testHiveImportAsParquetWhenTableExistsWithIncompatibleSchema PASSED org.apache.sqoop.hive.TestHiveImport > testCreateHiveImportAsParquet STARTED org.apache.sqoop.hive.TestHiveImport > testCreateHiveImportAsParquet PASSED org.apache.sqoop.hive.TestHiveImport > testCreateOnlyHiveImport STARTED org.apache.sqoop.hive.TestHiveImport > testCreateOnlyHiveImport PASSED org.apache.sqoop.hive.TestHiveImport > testAppendHiveImportAsParquet STARTED org.apache.sqoop.hive.TestHiveImport > testAppendHiveImportAsParquet PASSED org.apache.sqoop.hive.TestHiveImport > testNormalHiveImport STARTED org.apache.sqoop.hive.TestHiveImport > testNormalHiveImport PASSED org.apache.sqoop.hive.TestHiveImport > testNormalHiveImportAsParquet STARTED org.apache.sqoop.hive.TestHiveImport > testNormalHiveImportAsParquet PASSED org.apache.sqoop.hive.TestHiveImport > testImportWithBadPartitionKey STARTED org.apache.sqoop.hive.TestHiveImport > testImportWithBadPartitionKey PASSED org.apache.sqoop.hive.TestHiveImport > testCreateOverwriteHiveImportAsParquet STARTED org.apache.sqoop.hive.TestHiveImport > testCreateOverwriteHiveImportAsParquet PASSED org.apache.sqoop.hive.TestHiveImport > testImportHiveWithPartitions STARTED org.apache.sqoop.hive.TestHiveImport > testImportHiveWithPartitions PASSED org.apache.sqoop.hive.TestHiveImport > testNumeric
[jira] [Comment Edited] (SQOOP-3300) Implement JDBC and Kerberos tools for HiveServer2 support
[
https://issues.apache.org/jira/browse/SQOOP-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411449#comment-16411449
]
Boglarka Egyed edited comment on SQOOP-3300 at 3/23/18 2:23 PM:
Unfortunately one of the newly added unit tests (TestKerberosAuthenticator) is
failing in the Apache Jenkins while passing locally with an OutOfMemory
exception:
{noformat}
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 50.61 sec
[junit] Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
[junit] at java.util.Arrays.copyOf(Arrays.java:2367)
[junit] at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
[junit] at
java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
[junit] at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
[junit] at java.lang.StringBuffer.append(StringBuffer.java:237)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.PlainJUnitResultFormatter.endTestSuite(PlainJUnitResultFormatter.java:141)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.fireEndTestSuite(JUnitTestRunner.java:731)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:553)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
{noformat}
As this is not a functional defect in Sqoop I am resolving this ticket but
[~vasas] I would like to ask you to open another sub-task under SQOOP-3299 to
increase the Java heap space for this test and submit a patch for it as soon as
possible. Also, please close the related Review Request.
Thanks,
Bogi
was (Author: boglarkaegyed):
Unfortunately one of the newly added unit tests (TestKerberosAuthenticator)is
failing in the Apache Jenkins while passing locally with an OutOfMemory
exception:
{noformat}
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 50.61 sec
[junit] Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
[junit] at java.util.Arrays.copyOf(Arrays.java:2367)
[junit] at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
[junit] at
java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
[junit] at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
[junit] at java.lang.StringBuffer.append(StringBuffer.java:237)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.PlainJUnitResultFormatter.endTestSuite(PlainJUnitResultFormatter.java:141)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.fireEndTestSuite(JUnitTestRunner.java:731)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:553)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
{noformat}
As this is not a functional defect in Sqoop I am resolving this ticket but
[~vasas] I would like to ask you to open another sub-task under SQOOP-3299 to
increase the Java heap space for this test and submit a patch for it as soon as
possible. Also, please close the related Review Request.
Thanks,
Bogi
> Implement JDBC and Kerberos tools for HiveServer2 support
> -
>
> Key: SQOOP-3300
> URL: https://issues.apache.org/jira/browse/SQOOP-3300
> Project: Sqoop
> Issue Type: Sub-task
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: SQOOP-3300.patch
>
>
> The idea of the Sqoop HS2 support is to connect to HS2 using JDBC and execute
> the Hive commands on this connection. Sqoop should also support Kerberos
> authentication when building this JDBC connection.
> The goal of this JIRA is to implement the necessary classes for building JDBC
> connections and authenticating with Kerberos.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (SQOOP-3300) Implement JDBC and Kerberos tools for HiveServer2 support
[
https://issues.apache.org/jira/browse/SQOOP-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411449#comment-16411449
]
Boglarka Egyed edited comment on SQOOP-3300 at 3/23/18 2:23 PM:
Unfortunately one of the newly added unit tests (TestKerberosAuthenticator)is
failing in the Apache Jenkins while passing locally with an OutOfMemory
exception:
{noformat}
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 50.61 sec
[junit] Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
[junit] at java.util.Arrays.copyOf(Arrays.java:2367)
[junit] at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
[junit] at
java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
[junit] at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
[junit] at java.lang.StringBuffer.append(StringBuffer.java:237)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.PlainJUnitResultFormatter.endTestSuite(PlainJUnitResultFormatter.java:141)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.fireEndTestSuite(JUnitTestRunner.java:731)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:553)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
{noformat}
As this is not a functional defect in Sqoop I am resolving this ticket but
[~vasas] I would like to ask you to open another sub-task under SQOOP-3299 to
increase the Java heap space for this test and submit a patch for it as soon as
possible. Also, please close the related Review Request.
Thanks,
Bogi
was (Author: boglarkaegyed):
Unfortunately one of the newly added unit tests is failing in the Apache
Jenkins while passing locally with an OutOfMemory exception:
{noformat}
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 50.61 sec
[junit] Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
[junit] at java.util.Arrays.copyOf(Arrays.java:2367)
[junit] at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
[junit] at
java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
[junit] at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
[junit] at java.lang.StringBuffer.append(StringBuffer.java:237)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.PlainJUnitResultFormatter.endTestSuite(PlainJUnitResultFormatter.java:141)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.fireEndTestSuite(JUnitTestRunner.java:731)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:553)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
{noformat}
As this is not a functional defect in Sqoop I am resolving this ticket but
[~vasas] I would like to ask you to open another sub-task under SQOOP-3299 to
increase the Java heap space for this test and submit a patch for it as soon as
possible. Also, please close the related Review Request.
Thanks,
Bogi
> Implement JDBC and Kerberos tools for HiveServer2 support
> -
>
> Key: SQOOP-3300
> URL: https://issues.apache.org/jira/browse/SQOOP-3300
> Project: Sqoop
> Issue Type: Sub-task
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: SQOOP-3300.patch
>
>
> The idea of the Sqoop HS2 support is to connect to HS2 using JDBC and execute
> the Hive commands on this connection. Sqoop should also support Kerberos
> authentication when building this JDBC connection.
> The goal of this JIRA is to implement the necessary classes for building JDBC
> connections and authenticating with Kerberos.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (SQOOP-3300) Implement JDBC and Kerberos tools for HiveServer2 support
[
https://issues.apache.org/jira/browse/SQOOP-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411449#comment-16411449
]
Boglarka Egyed edited comment on SQOOP-3300 at 3/23/18 2:16 PM:
Unfortunately one of the newly added unit tests is failing in the Apache
Jenkins while passing locally with an OutOfMemory exception:
{noformat}
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 50.61 sec
[junit] Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
[junit] at java.util.Arrays.copyOf(Arrays.java:2367)
[junit] at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
[junit] at
java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
[junit] at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
[junit] at java.lang.StringBuffer.append(StringBuffer.java:237)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.PlainJUnitResultFormatter.endTestSuite(PlainJUnitResultFormatter.java:141)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.fireEndTestSuite(JUnitTestRunner.java:731)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:553)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
{noformat}
As this is not a functional defect in Sqoop I am resolving this ticket but
[~vasas] I would like to ask you to open another sub-task under SQOOP-3299 to
increase the Java heap space for this test and submit a patch for it as soon as
possible. Also, please close the related Review Request.
Thanks,
Bogi
was (Author: boglarkaegyed):
Unfortunately one of the newly added unit tests is failing in the Apache
Jenkins while passing locally with an OutOfMemory exception:
{noformat}
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 50.61 sec
[junit] Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
[junit] at java.util.Arrays.copyOf(Arrays.java:2367)
[junit] at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
[junit] at
java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
[junit] at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
[junit] at java.lang.StringBuffer.append(StringBuffer.java:237)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.PlainJUnitResultFormatter.endTestSuite(PlainJUnitResultFormatter.java:141)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.fireEndTestSuite(JUnitTestRunner.java:731)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:553)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
{noformat}
As this is not a functional defect in Sqoop I am resolving this ticket but
[~vasas] I would like to ask you to open another sub-task under SQOOP-3299 to
increase the Java heap space for this test and submit a patch for it as soon as
possible.
Thanks,
Bogi
> Implement JDBC and Kerberos tools for HiveServer2 support
> -
>
> Key: SQOOP-3300
> URL: https://issues.apache.org/jira/browse/SQOOP-3300
> Project: Sqoop
> Issue Type: Sub-task
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3300.patch
>
>
> The idea of the Sqoop HS2 support is to connect to HS2 using JDBC and execute
> the Hive commands on this connection. Sqoop should also support Kerberos
> authentication when building this JDBC connection.
> The goal of this JIRA is to implement the necessary classes for building JDBC
> connections and authenticating with Kerberos.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SQOOP-3300) Implement JDBC and Kerberos tools for HiveServer2 support
[
https://issues.apache.org/jira/browse/SQOOP-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411449#comment-16411449
]
Boglarka Egyed commented on SQOOP-3300:
---
Unfortunately one of the newly added unit tests is failing in the Apache
Jenkins while passing locally with an OutOfMemory exception:
{noformat}
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 50.61 sec
[junit] Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
[junit] at java.util.Arrays.copyOf(Arrays.java:2367)
[junit] at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
[junit] at
java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
[junit] at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
[junit] at java.lang.StringBuffer.append(StringBuffer.java:237)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.PlainJUnitResultFormatter.endTestSuite(PlainJUnitResultFormatter.java:141)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.fireEndTestSuite(JUnitTestRunner.java:731)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:553)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)
[junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
{noformat}
As this is not a functional defect in Sqoop I am resolving this ticket but
[~vasas] I would like to ask you to open another sub-task under SQOOP-3299 to
increase the Java heap space for this test and submit a patch for it as soon as
possible.
Thanks,
Bogi
> Implement JDBC and Kerberos tools for HiveServer2 support
> -
>
> Key: SQOOP-3300
> URL: https://issues.apache.org/jira/browse/SQOOP-3300
> Project: Sqoop
> Issue Type: Sub-task
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3300.patch
>
>
> The idea of the Sqoop HS2 support is to connect to HS2 using JDBC and execute
> the Hive commands on this connection. Sqoop should also support Kerberos
> authentication when building this JDBC connection.
> The goal of this JIRA is to implement the necessary classes for building JDBC
> connections and authenticating with Kerberos.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SQOOP-3300) Implement JDBC and Kerberos tools for HiveServer2 support
[ https://issues.apache.org/jira/browse/SQOOP-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411390#comment-16411390 ] Hudson commented on SQOOP-3300: --- FAILURE: Integrated in Jenkins build Sqoop-hadoop200 #1152 (See [https://builds.apache.org/job/Sqoop-hadoop200/1152/]) SQOOP-3300: Implement JDBC and Kerberos tools for HiveServer2 support (bogi: [https://git-wip-us.apache.org/repos/asf?p=sqoop.git=commit=d67bb816ce35cdf59a3504845a0bc5639f690dd6]) * (add) src/test/org/apache/sqoop/db/TestDriverManagerJdbcConnectionFactory.java * (add) src/java/org/apache/sqoop/authentication/KerberosAuthenticator.java * (add) src/java/org/apache/sqoop/db/JdbcConnectionFactory.java * (add) src/test/org/apache/sqoop/db/decorator/TestKerberizedConnectionFactoryDecorator.java * (edit) src/test/org/apache/sqoop/infrastructure/kerberos/MiniKdcInfrastructureRule.java * (add) src/java/org/apache/sqoop/db/decorator/JdbcConnectionFactoryDecorator.java * (add) src/java/org/apache/sqoop/db/DriverManagerJdbcConnectionFactory.java * (add) src/java/org/apache/sqoop/db/decorator/KerberizedConnectionFactoryDecorator.java * (add) src/test/org/apache/sqoop/authentication/TestKerberosAuthenticator.java * (edit) src/test/org/apache/sqoop/hbase/HBaseTestCase.java > Implement JDBC and Kerberos tools for HiveServer2 support > - > > Key: SQOOP-3300 > URL: https://issues.apache.org/jira/browse/SQOOP-3300 > Project: Sqoop > Issue Type: Sub-task >Affects Versions: 1.5.0 >Reporter: Szabolcs Vasas >Assignee: Szabolcs Vasas >Priority: Major > Attachments: SQOOP-3300.patch > > > The idea of the Sqoop HS2 support is to connect to HS2 using JDBC and execute > the Hive commands on this connection. Sqoop should also support Kerberos > authentication when building this JDBC connection. > The goal of this JIRA is to implement the necessary classes for building JDBC > connections and authenticating with Kerberos. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SQOOP-3302) Add missing license header to HBaseKerberizedConnectivityTest and TestSQLServerDBRecordReader
[ https://issues.apache.org/jira/browse/SQOOP-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szabolcs Vasas updated SQOOP-3302: -- Attachment: SQOOP-3302.patch > Add missing license header to HBaseKerberizedConnectivityTest and > TestSQLServerDBRecordReader > - > > Key: SQOOP-3302 > URL: https://issues.apache.org/jira/browse/SQOOP-3302 > Project: Sqoop > Issue Type: Task >Affects Versions: 1.5.0 >Reporter: Szabolcs Vasas >Assignee: Szabolcs Vasas >Priority: Major > Attachments: SQOOP-3302.patch > > > HBaseKerberizedConnectivityTest and TestSQLServerDBRecordReader do not have > the Apache license header which should be fixed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Review Request 66247: Add missing license header to HBaseKerberizedConnectivityTest and TestSQLServerDBRecordReader
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66247/ --- Review request for Sqoop. Bugs: SQOOP-3302 https://issues.apache.org/jira/browse/SQOOP-3302 Repository: sqoop-trunk Description --- HBaseKerberizedConnectivityTest and TestSQLServerDBRecordReader do not have the Apache license header which should be fixed. Diffs - src/test/org/apache/sqoop/hbase/HBaseKerberizedConnectivityTest.java 73a22474468af405224385e016870751189d62ea src/test/org/apache/sqoop/mapreduce/db/TestSQLServerDBRecordReader.java 8b49c8dda19cc2bd1b6444d3a23d8227dcef9fcf Diff: https://reviews.apache.org/r/66247/diff/1/ Testing --- Compiled the project. Thanks, Szabolcs Vasas
[jira] [Commented] (SQOOP-3042) Sqoop does not clear compile directory under /tmp/sqoop-/compile automatically
[
https://issues.apache.org/jira/browse/SQOOP-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411372#comment-16411372
]
Eric Lin commented on SQOOP-3042:
-
Hi [~dkozlowski],
The patch has not been reviewed yet, I will try to update the patch with latest
trunk code and push for review again.
> Sqoop does not clear compile directory under /tmp/sqoop-/compile
> automatically
>
>
> Key: SQOOP-3042
> URL: https://issues.apache.org/jira/browse/SQOOP-3042
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Critical
> Labels: patch
> Attachments: SQOOP-3042.1.patch, SQOOP-3042.2.patch,
> SQOOP-3042.4.patch
>
>
> After running sqoop, all the temp files generated by ClassWriter are left
> behind on disk, so anyone can check those JAVA files to see the schema of
> those tables that Sqoop has been interacting with. By default, the directory
> is under /tmp/sqoop-/compile.
> In class org.apache.sqoop.SqoopOptions, function getNonceJarDir(), I can see
> that we did add "deleteOnExit" on the temp dir:
> {code}
> for (int attempts = 0; attempts < MAX_DIR_CREATE_ATTEMPTS; attempts++) {
> hashDir = new File(baseDir, RandomHash.generateMD5String());
> while (hashDir.exists()) {
> hashDir = new File(baseDir, RandomHash.generateMD5String());
> }
> if (hashDir.mkdirs()) {
> // We created the directory. Use it.
> // If this directory is not actually filled with files, delete it
> // when the JVM quits.
> hashDir.deleteOnExit();
> break;
> }
> }
> {code}
> However, I believe it failed to delete due to directory is not empty.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SQOOP-3303) Fix warnings during Sqoop compilation
[
https://issues.apache.org/jira/browse/SQOOP-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szabolcs Vasas updated SQOOP-3303:
--
Attachment: SQOOP-3303.patch
> Fix warnings during Sqoop compilation
> -
>
> Key: SQOOP-3303
> URL: https://issues.apache.org/jira/browse/SQOOP-3303
> Project: Sqoop
> Issue Type: Task
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3303.patch
>
>
> Ant prints the following warning during every Sqoop compilation:
> {code:java}
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestAvroExport.java:477:
> warning: non-varargs call of varargs method with inexact argument type for
> last parameter;
> createAvroFile(0, TOTAL_RECORDS, null);
> ^
> cast to ColumnGenerator for a varargs call
> cast to ColumnGenerator[] for a non-varargs call and to suppress this
> warning
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestAvroExport.java:492:
> warning: non-varargs call of varargs method with inexact argument type for
> last parameter;
> createAvroFile(0, TOTAL_RECORDS, null);
> ^
> cast to ColumnGenerator for a varargs call
> cast to ColumnGenerator[] for a non-varargs call and to suppress this
> warning
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestParquetExport.java:422:
> warning: non-varargs call of varargs method with inexact argument type for
> last parameter;
> createParquetFile(0, TOTAL_RECORDS, null);
> ^
> cast to ColumnGenerator for a varargs call
> cast to ColumnGenerator[] for a non-varargs call and to suppress this
> warning
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestParquetExport.java:435:
> warning: non-varargs call of varargs method with inexact argument type for
> last parameter;
> createParquetFile(0, TOTAL_RECORDS, null);
> ^
> cast to ColumnGenerator for a varargs call
> cast to ColumnGenerator[] for a non-varargs call and to suppress this
> warning
> {code}
> It is kind of annoying and would be very easy to fix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Review Request 66246: Fix warnings during Sqoop compilation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66246/ --- Review request for Sqoop. Bugs: SQOOP-3303 https://issues.apache.org/jira/browse/SQOOP-3303 Repository: sqoop-trunk Description --- Ant prints the following warning during every Sqoop compilation: /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestAvroExport.java:477: warning: non-varargs call of varargs method with inexact argument type for last parameter; createAvroFile(0, TOTAL_RECORDS, null); ^ cast to ColumnGenerator for a varargs call cast to ColumnGenerator[] for a non-varargs call and to suppress this warning /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestAvroExport.java:492: warning: non-varargs call of varargs method with inexact argument type for last parameter; createAvroFile(0, TOTAL_RECORDS, null); ^ cast to ColumnGenerator for a varargs call cast to ColumnGenerator[] for a non-varargs call and to suppress this warning /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestParquetExport.java:422: warning: non-varargs call of varargs method with inexact argument type for last parameter; createParquetFile(0, TOTAL_RECORDS, null); ^ cast to ColumnGenerator for a varargs call cast to ColumnGenerator[] for a non-varargs call and to suppress this warning /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestParquetExport.java:435: warning: non-varargs call of varargs method with inexact argument type for last parameter; createParquetFile(0, TOTAL_RECORDS, null); ^ cast to ColumnGenerator for a varargs call cast to ColumnGenerator[] for a non-varargs call and to suppress this warning It is kind of annoying and would be very easy to fix. Diffs - src/test/org/apache/sqoop/TestAvroExport.java d1f10549f8c13c1433e8cb624d1bd97bb998fdf6 src/test/org/apache/sqoop/TestParquetExport.java fdaa5d810dc3562723bd397b72206ed36c889d36 Diff: https://reviews.apache.org/r/66246/diff/1/ Testing --- ant test Thanks, Szabolcs Vasas
[jira] [Updated] (SQOOP-3303) Fix warnings during Sqoop compilation
[
https://issues.apache.org/jira/browse/SQOOP-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szabolcs Vasas updated SQOOP-3303:
--
Summary: Fix warnings during Sqoop compilation (was: Fix warnings during
Sqoop compilaton)
> Fix warnings during Sqoop compilation
> -
>
> Key: SQOOP-3303
> URL: https://issues.apache.org/jira/browse/SQOOP-3303
> Project: Sqoop
> Issue Type: Task
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
>
> Ant prints the following warning during every Sqoop compilation:
> {code:java}
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestAvroExport.java:477:
> warning: non-varargs call of varargs method with inexact argument type for
> last parameter;
> createAvroFile(0, TOTAL_RECORDS, null);
> ^
> cast to ColumnGenerator for a varargs call
> cast to ColumnGenerator[] for a non-varargs call and to suppress this
> warning
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestAvroExport.java:492:
> warning: non-varargs call of varargs method with inexact argument type for
> last parameter;
> createAvroFile(0, TOTAL_RECORDS, null);
> ^
> cast to ColumnGenerator for a varargs call
> cast to ColumnGenerator[] for a non-varargs call and to suppress this
> warning
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestParquetExport.java:422:
> warning: non-varargs call of varargs method with inexact argument type for
> last parameter;
> createParquetFile(0, TOTAL_RECORDS, null);
> ^
> cast to ColumnGenerator for a varargs call
> cast to ColumnGenerator[] for a non-varargs call and to suppress this
> warning
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestParquetExport.java:435:
> warning: non-varargs call of varargs method with inexact argument type for
> last parameter;
> createParquetFile(0, TOTAL_RECORDS, null);
> ^
> cast to ColumnGenerator for a varargs call
> cast to ColumnGenerator[] for a non-varargs call and to suppress this
> warning
> {code}
> It is kind of annoying and would be very easy to fix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (SQOOP-3303) Fix warnings during Sqoop compilaton
[
https://issues.apache.org/jira/browse/SQOOP-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szabolcs Vasas reassigned SQOOP-3303:
-
Assignee: Szabolcs Vasas
> Fix warnings during Sqoop compilaton
>
>
> Key: SQOOP-3303
> URL: https://issues.apache.org/jira/browse/SQOOP-3303
> Project: Sqoop
> Issue Type: Task
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
>
> Ant prints the following warning during every Sqoop compilation:
> {code:java}
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestAvroExport.java:477:
> warning: non-varargs call of varargs method with inexact argument type for
> last parameter;
> createAvroFile(0, TOTAL_RECORDS, null);
> ^
> cast to ColumnGenerator for a varargs call
> cast to ColumnGenerator[] for a non-varargs call and to suppress this
> warning
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestAvroExport.java:492:
> warning: non-varargs call of varargs method with inexact argument type for
> last parameter;
> createAvroFile(0, TOTAL_RECORDS, null);
> ^
> cast to ColumnGenerator for a varargs call
> cast to ColumnGenerator[] for a non-varargs call and to suppress this
> warning
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestParquetExport.java:422:
> warning: non-varargs call of varargs method with inexact argument type for
> last parameter;
> createParquetFile(0, TOTAL_RECORDS, null);
> ^
> cast to ColumnGenerator for a varargs call
> cast to ColumnGenerator[] for a non-varargs call and to suppress this
> warning
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestParquetExport.java:435:
> warning: non-varargs call of varargs method with inexact argument type for
> last parameter;
> createParquetFile(0, TOTAL_RECORDS, null);
> ^
> cast to ColumnGenerator for a varargs call
> cast to ColumnGenerator[] for a non-varargs call and to suppress this
> warning
> {code}
> It is kind of annoying and would be very easy to fix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (SQOOP-3303) Fix warnings during Sqoop compilaton
Szabolcs Vasas created SQOOP-3303:
-
Summary: Fix warnings during Sqoop compilaton
Key: SQOOP-3303
URL: https://issues.apache.org/jira/browse/SQOOP-3303
Project: Sqoop
Issue Type: Task
Affects Versions: 1.5.0
Reporter: Szabolcs Vasas
Ant prints the following warning during every Sqoop compilation:
{code:java}
/Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestAvroExport.java:477:
warning: non-varargs call of varargs method with inexact argument type for
last parameter;
createAvroFile(0, TOTAL_RECORDS, null);
^
cast to ColumnGenerator for a varargs call
cast to ColumnGenerator[] for a non-varargs call and to suppress this warning
/Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestAvroExport.java:492:
warning: non-varargs call of varargs method with inexact argument type for
last parameter;
createAvroFile(0, TOTAL_RECORDS, null);
^
cast to ColumnGenerator for a varargs call
cast to ColumnGenerator[] for a non-varargs call and to suppress this warning
/Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestParquetExport.java:422:
warning: non-varargs call of varargs method with inexact argument type for
last parameter;
createParquetFile(0, TOTAL_RECORDS, null);
^
cast to ColumnGenerator for a varargs call
cast to ColumnGenerator[] for a non-varargs call and to suppress this warning
/Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestParquetExport.java:435:
warning: non-varargs call of varargs method with inexact argument type for
last parameter;
createParquetFile(0, TOTAL_RECORDS, null);
^
cast to ColumnGenerator for a varargs call
cast to ColumnGenerator[] for a non-varargs call and to suppress this warning
{code}
It is kind of annoying and would be very easy to fix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (SQOOP-3302) Add missing license header to HBaseKerberizedConnectivityTest and TestSQLServerDBRecordReader
[ https://issues.apache.org/jira/browse/SQOOP-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szabolcs Vasas reassigned SQOOP-3302: - Assignee: Szabolcs Vasas > Add missing license header to HBaseKerberizedConnectivityTest and > TestSQLServerDBRecordReader > - > > Key: SQOOP-3302 > URL: https://issues.apache.org/jira/browse/SQOOP-3302 > Project: Sqoop > Issue Type: Task >Affects Versions: 1.5.0 >Reporter: Szabolcs Vasas >Assignee: Szabolcs Vasas >Priority: Major > > HBaseKerberizedConnectivityTest and TestSQLServerDBRecordReader do not have > the Apache license header which should be fixed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SQOOP-3300) Implement JDBC and Kerberos tools for HiveServer2 support
[ https://issues.apache.org/jira/browse/SQOOP-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411351#comment-16411351 ] ASF subversion and git services commented on SQOOP-3300: Commit d67bb816ce35cdf59a3504845a0bc5639f690dd6 in sqoop's branch refs/heads/trunk from [~BoglarkaEgyed] [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=d67bb81 ] SQOOP-3300: Implement JDBC and Kerberos tools for HiveServer2 support (Szabolcs Vasas via Boglarka Egyed) > Implement JDBC and Kerberos tools for HiveServer2 support > - > > Key: SQOOP-3300 > URL: https://issues.apache.org/jira/browse/SQOOP-3300 > Project: Sqoop > Issue Type: Sub-task >Affects Versions: 1.5.0 >Reporter: Szabolcs Vasas >Assignee: Szabolcs Vasas >Priority: Major > Attachments: SQOOP-3300.patch > > > The idea of the Sqoop HS2 support is to connect to HS2 using JDBC and execute > the Hive commands on this connection. Sqoop should also support Kerberos > authentication when building this JDBC connection. > The goal of this JIRA is to implement the necessary classes for building JDBC > connections and authenticating with Kerberos. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SQOOP-3302) Add missing license header to HBaseKerberizedConnectivityTest and TestSQLServerDBRecordReader
Szabolcs Vasas created SQOOP-3302: - Summary: Add missing license header to HBaseKerberizedConnectivityTest and TestSQLServerDBRecordReader Key: SQOOP-3302 URL: https://issues.apache.org/jira/browse/SQOOP-3302 Project: Sqoop Issue Type: Task Affects Versions: 1.5.0 Reporter: Szabolcs Vasas HBaseKerberizedConnectivityTest and TestSQLServerDBRecordReader do not have the Apache license header which should be fixed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 66195: Implement JDBC and Kerberos tools for HiveServer2 support
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66195/#review199880 --- Ship it! Ship It! - daniel voros On March 21, 2018, 12:48 p.m., Szabolcs Vasas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/66195/ > --- > > (Updated March 21, 2018, 12:48 p.m.) > > > Review request for Sqoop. > > > Bugs: SQOOP-3300 > https://issues.apache.org/jira/browse/SQOOP-3300 > > > Repository: sqoop-trunk > > > Description > --- > > The idea of the Sqoop HS2 support is to connect to HS2 using JDBC and execute > the Hive commands on this connection. Sqoop should also support Kerberos > authentication when building this JDBC connection. > > The goal of this JIRA is to implement the necessary classes for building JDBC > connections and authenticating with Kerberos. > > > Diffs > - > > src/java/org/apache/sqoop/authentication/KerberosAuthenticator.java > PRE-CREATION > src/java/org/apache/sqoop/db/DriverManagerJdbcConnectionFactory.java > PRE-CREATION > src/java/org/apache/sqoop/db/JdbcConnectionFactory.java PRE-CREATION > src/java/org/apache/sqoop/db/decorator/JdbcConnectionFactoryDecorator.java > PRE-CREATION > > src/java/org/apache/sqoop/db/decorator/KerberizedConnectionFactoryDecorator.java > PRE-CREATION > src/test/org/apache/sqoop/authentication/TestKerberosAuthenticator.java > PRE-CREATION > src/test/org/apache/sqoop/db/TestDriverManagerJdbcConnectionFactory.java > PRE-CREATION > > src/test/org/apache/sqoop/db/decorator/TestKerberizedConnectionFactoryDecorator.java > PRE-CREATION > src/test/org/apache/sqoop/hbase/HBaseTestCase.java > f96b6587ff3756aa5a696df8b7fc12ef0b0f > > src/test/org/apache/sqoop/infrastructure/kerberos/MiniKdcInfrastructureRule.java > a704d0b07282e54e7c19d7a6725d6d026d037073 > > > Diff: https://reviews.apache.org/r/66195/diff/1/ > > > Testing > --- > > Executed unit and third party tests. > > > Thanks, > > Szabolcs Vasas > >
Re: Review Request 66195: Implement JDBC and Kerberos tools for HiveServer2 support
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66195/#review199878 --- Ship it! Ship It! - Boglarka Egyed On March 21, 2018, 12:48 p.m., Szabolcs Vasas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/66195/ > --- > > (Updated March 21, 2018, 12:48 p.m.) > > > Review request for Sqoop. > > > Bugs: SQOOP-3300 > https://issues.apache.org/jira/browse/SQOOP-3300 > > > Repository: sqoop-trunk > > > Description > --- > > The idea of the Sqoop HS2 support is to connect to HS2 using JDBC and execute > the Hive commands on this connection. Sqoop should also support Kerberos > authentication when building this JDBC connection. > > The goal of this JIRA is to implement the necessary classes for building JDBC > connections and authenticating with Kerberos. > > > Diffs > - > > src/java/org/apache/sqoop/authentication/KerberosAuthenticator.java > PRE-CREATION > src/java/org/apache/sqoop/db/DriverManagerJdbcConnectionFactory.java > PRE-CREATION > src/java/org/apache/sqoop/db/JdbcConnectionFactory.java PRE-CREATION > src/java/org/apache/sqoop/db/decorator/JdbcConnectionFactoryDecorator.java > PRE-CREATION > > src/java/org/apache/sqoop/db/decorator/KerberizedConnectionFactoryDecorator.java > PRE-CREATION > src/test/org/apache/sqoop/authentication/TestKerberosAuthenticator.java > PRE-CREATION > src/test/org/apache/sqoop/db/TestDriverManagerJdbcConnectionFactory.java > PRE-CREATION > > src/test/org/apache/sqoop/db/decorator/TestKerberizedConnectionFactoryDecorator.java > PRE-CREATION > src/test/org/apache/sqoop/hbase/HBaseTestCase.java > f96b6587ff3756aa5a696df8b7fc12ef0b0f > > src/test/org/apache/sqoop/infrastructure/kerberos/MiniKdcInfrastructureRule.java > a704d0b07282e54e7c19d7a6725d6d026d037073 > > > Diff: https://reviews.apache.org/r/66195/diff/1/ > > > Testing > --- > > Executed unit and third party tests. > > > Thanks, > > Szabolcs Vasas > >
Re: Review Request 66195: Implement JDBC and Kerberos tools for HiveServer2 support
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66195/#review199877 --- Ship it! Ship It! - Fero Szabo On March 21, 2018, 12:48 p.m., Szabolcs Vasas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/66195/ > --- > > (Updated March 21, 2018, 12:48 p.m.) > > > Review request for Sqoop. > > > Bugs: SQOOP-3300 > https://issues.apache.org/jira/browse/SQOOP-3300 > > > Repository: sqoop-trunk > > > Description > --- > > The idea of the Sqoop HS2 support is to connect to HS2 using JDBC and execute > the Hive commands on this connection. Sqoop should also support Kerberos > authentication when building this JDBC connection. > > The goal of this JIRA is to implement the necessary classes for building JDBC > connections and authenticating with Kerberos. > > > Diffs > - > > src/java/org/apache/sqoop/authentication/KerberosAuthenticator.java > PRE-CREATION > src/java/org/apache/sqoop/db/DriverManagerJdbcConnectionFactory.java > PRE-CREATION > src/java/org/apache/sqoop/db/JdbcConnectionFactory.java PRE-CREATION > src/java/org/apache/sqoop/db/decorator/JdbcConnectionFactoryDecorator.java > PRE-CREATION > > src/java/org/apache/sqoop/db/decorator/KerberizedConnectionFactoryDecorator.java > PRE-CREATION > src/test/org/apache/sqoop/authentication/TestKerberosAuthenticator.java > PRE-CREATION > src/test/org/apache/sqoop/db/TestDriverManagerJdbcConnectionFactory.java > PRE-CREATION > > src/test/org/apache/sqoop/db/decorator/TestKerberizedConnectionFactoryDecorator.java > PRE-CREATION > src/test/org/apache/sqoop/hbase/HBaseTestCase.java > f96b6587ff3756aa5a696df8b7fc12ef0b0f > > src/test/org/apache/sqoop/infrastructure/kerberos/MiniKdcInfrastructureRule.java > a704d0b07282e54e7c19d7a6725d6d026d037073 > > > Diff: https://reviews.apache.org/r/66195/diff/1/ > > > Testing > --- > > Executed unit and third party tests. > > > Thanks, > > Szabolcs Vasas > >
Re: Review Request 66195: Implement JDBC and Kerberos tools for HiveServer2 support
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66195/#review199876 --- Ship it! Ship It! - Ferenc Szabo On March 21, 2018, 12:48 p.m., Szabolcs Vasas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/66195/ > --- > > (Updated March 21, 2018, 12:48 p.m.) > > > Review request for Sqoop. > > > Bugs: SQOOP-3300 > https://issues.apache.org/jira/browse/SQOOP-3300 > > > Repository: sqoop-trunk > > > Description > --- > > The idea of the Sqoop HS2 support is to connect to HS2 using JDBC and execute > the Hive commands on this connection. Sqoop should also support Kerberos > authentication when building this JDBC connection. > > The goal of this JIRA is to implement the necessary classes for building JDBC > connections and authenticating with Kerberos. > > > Diffs > - > > src/java/org/apache/sqoop/authentication/KerberosAuthenticator.java > PRE-CREATION > src/java/org/apache/sqoop/db/DriverManagerJdbcConnectionFactory.java > PRE-CREATION > src/java/org/apache/sqoop/db/JdbcConnectionFactory.java PRE-CREATION > src/java/org/apache/sqoop/db/decorator/JdbcConnectionFactoryDecorator.java > PRE-CREATION > > src/java/org/apache/sqoop/db/decorator/KerberizedConnectionFactoryDecorator.java > PRE-CREATION > src/test/org/apache/sqoop/authentication/TestKerberosAuthenticator.java > PRE-CREATION > src/test/org/apache/sqoop/db/TestDriverManagerJdbcConnectionFactory.java > PRE-CREATION > > src/test/org/apache/sqoop/db/decorator/TestKerberizedConnectionFactoryDecorator.java > PRE-CREATION > src/test/org/apache/sqoop/hbase/HBaseTestCase.java > f96b6587ff3756aa5a696df8b7fc12ef0b0f > > src/test/org/apache/sqoop/infrastructure/kerberos/MiniKdcInfrastructureRule.java > a704d0b07282e54e7c19d7a6725d6d026d037073 > > > Diff: https://reviews.apache.org/r/66195/diff/1/ > > > Testing > --- > > Executed unit and third party tests. > > > Thanks, > > Szabolcs Vasas > >
[jira] [Commented] (SQOOP-3224) Mainframe FTP transfer should have an option to use binary mode for transfer
[
https://issues.apache.org/jira/browse/SQOOP-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411299#comment-16411299
]
ASF GitHub Bot commented on SQOOP-3224:
---
Github user christeoh commented on a diff in the pull request:
https://github.com/apache/sqoop/pull/44#discussion_r176717474
--- Diff: src/java/org/apache/sqoop/util/MainframeFTPClientUtils.java ---
@@ -207,8 +208,18 @@ public static FTPClient getFTPConnection(Configuration
conf)
throw new IOException("Could not login to server " + server
+ ":" + ftp.getReplyString());
}
- // set ASCII transfer mode
- ftp.setFileType(FTP.ASCII_FILE_TYPE);
+ // set transfer mode
+ String transferMode =
conf.get(MainframeConfiguration.MAINFRAME_FTP_TRANSFER_MODE);
--- End diff --
I'm not sure how you would refactor it. I can give it a try and see if it
is what you're looking for.
> Mainframe FTP transfer should have an option to use binary mode for transfer
>
>
> Key: SQOOP-3224
> URL: https://issues.apache.org/jira/browse/SQOOP-3224
> Project: Sqoop
> Issue Type: Improvement
>Reporter: Chris Teoh
>Assignee: Chris Teoh
>Priority: Minor
>
> Currently the mainframe FTP module is hard coded to use ascii transfer mode.
> Propose a mainframe module flag to be able to change modes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Review Request 66067: SQOOP-3052: Introduce gradle-based build for Sqoop to make it more developer friendly / open
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66067/ --- (Updated March 23, 2018, 10:28 a.m.) Review request for Sqoop. Changes --- Apologies for the barrage of changes - fixed messed up diff Bugs: Sqoop-3052 https://issues.apache.org/jira/browse/Sqoop-3052 Repository: sqoop-trunk Description --- SQOOP-3052: Introduce gradle based build for Sqoop to make it more developer friendly / open Diffs (updated) - .gitignore 68cbe28 COMPILING.txt 3b82250 build.gradle PRE-CREATION buildSrc/customUnixStartScript.txt PRE-CREATION buildSrc/customWindowsStartScript.txt PRE-CREATION buildSrc/sqoop-package.gradle PRE-CREATION buildSrc/sqoop-version-gen.gradle PRE-CREATION config/checkstyle/checkstyle-java-header.txt PRE-CREATION config/checkstyle/checkstyle-noframes.xsl PRE-CREATION config/checkstyle/checkstyle.xml PRE-CREATION gradle.properties PRE-CREATION gradle/wrapper/gradle-wrapper.jar PRE-CREATION gradle/wrapper/gradle-wrapper.properties PRE-CREATION gradlew PRE-CREATION gradlew.bat PRE-CREATION settings.gradle PRE-CREATION Diff: https://reviews.apache.org/r/66067/diff/6/ Changes: https://reviews.apache.org/r/66067/diff/5-6/ Testing --- ran all new tasks, except for internal maven publishing Notes: - To try it out you can call ./gradlew tasks --all to see all the tasks and compare them to current tasks/artifacts. - Replaced cobertura with jacoco, as it's easier/cleaner to configure, easier to combine all test results into a single report. - Generated pom.xml now has correct dependencies/versions - Script generation is currently hardcoded and not based on sqoop help, as previously - though added the possiblity of hooking it in later Thanks, Anna Szonyi
Re: Review Request 66067: SQOOP-3052: Introduce gradle-based build for Sqoop to make it more developer friendly / open
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66067/ --- (Updated March 23, 2018, 10:19 a.m.) Review request for Sqoop. Changes --- Fixed diff vs. trunk Bugs: Sqoop-3052 https://issues.apache.org/jira/browse/Sqoop-3052 Repository: sqoop-trunk Description --- SQOOP-3052: Introduce gradle based build for Sqoop to make it more developer friendly / open Diffs (updated) - .gitignore 68cbe28 COMPILING.txt 3b82250 src/docs/user/import.txt e91a5a8 Diff: https://reviews.apache.org/r/66067/diff/5/ Changes: https://reviews.apache.org/r/66067/diff/4-5/ Testing --- ran all new tasks, except for internal maven publishing Notes: - To try it out you can call ./gradlew tasks --all to see all the tasks and compare them to current tasks/artifacts. - Replaced cobertura with jacoco, as it's easier/cleaner to configure, easier to combine all test results into a single report. - Generated pom.xml now has correct dependencies/versions - Script generation is currently hardcoded and not based on sqoop help, as previously - though added the possiblity of hooking it in later Thanks, Anna Szonyi
Re: Review Request 66067: SQOOP-3052: Introduce gradle-based build for Sqoop to make it more developer friendly / open
> On March 23, 2018, 9:39 a.m., Szabolcs Vasas wrote: > > Hi Anna, > > > > Thank you for replying so quickly! > > The changes look good in RB but I could no apply it in my repo, I get the > > following error: > > > > error: patch failed: COMPILING.txt:403 > > error: COMPILING.txt: patch does not apply > > error: cannot apply binary patch to 'gradle/wrapper/gradle-wrapper.jar' > > without full index line > > error: gradle/wrapper/gradle-wrapper.jar: patch does not apply Apologies, please try again. I ran the diff with diff-index --binary and it applies for me. - Anna --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66067/#review199871 --- On March 23, 2018, 10:05 a.m., Anna Szonyi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/66067/ > --- > > (Updated March 23, 2018, 10:05 a.m.) > > > Review request for Sqoop. > > > Bugs: Sqoop-3052 > https://issues.apache.org/jira/browse/Sqoop-3052 > > > Repository: sqoop-trunk > > > Description > --- > > SQOOP-3052: Introduce gradle based build for Sqoop to make it more developer > friendly / open > > > Diffs > - > > .gitignore 68cbe28 > COMPILING.txt 86be509 > build.gradle PRE-CREATION > buildSrc/customUnixStartScript.txt PRE-CREATION > buildSrc/customWindowsStartScript.txt PRE-CREATION > buildSrc/sqoop-package.gradle PRE-CREATION > buildSrc/sqoop-version-gen.gradle PRE-CREATION > config/checkstyle/checkstyle-java-header.txt PRE-CREATION > config/checkstyle/checkstyle-noframes.xsl PRE-CREATION > config/checkstyle/checkstyle.xml PRE-CREATION > gradle.properties PRE-CREATION > gradle/wrapper/gradle-wrapper.jar PRE-CREATION > gradle/wrapper/gradle-wrapper.properties PRE-CREATION > gradlew PRE-CREATION > gradlew.bat PRE-CREATION > settings.gradle PRE-CREATION > > > Diff: https://reviews.apache.org/r/66067/diff/4/ > > > Testing > --- > > ran all new tasks, except for internal maven publishing > > Notes: > - To try it out you can call ./gradlew tasks --all to see all the tasks and > compare them to current tasks/artifacts. > - Replaced cobertura with jacoco, as it's easier/cleaner to configure, easier > to combine all test results into a single report. > - Generated pom.xml now has correct dependencies/versions > - Script generation is currently hardcoded and not based on sqoop help, as > previously - though added the possiblity of hooking it in later > > > Thanks, > > Anna Szonyi > >
Re: Review Request 66067: SQOOP-3052: Introduce gradle-based build for Sqoop to make it more developer friendly / open
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66067/ --- (Updated March 23, 2018, 10:05 a.m.) Review request for Sqoop. Changes --- ran the diff with diff-index --binary Bugs: Sqoop-3052 https://issues.apache.org/jira/browse/Sqoop-3052 Repository: sqoop-trunk Description --- SQOOP-3052: Introduce gradle based build for Sqoop to make it more developer friendly / open Diffs (updated) - .gitignore 68cbe28 COMPILING.txt 86be509 build.gradle PRE-CREATION buildSrc/customUnixStartScript.txt PRE-CREATION buildSrc/customWindowsStartScript.txt PRE-CREATION buildSrc/sqoop-package.gradle PRE-CREATION buildSrc/sqoop-version-gen.gradle PRE-CREATION config/checkstyle/checkstyle-java-header.txt PRE-CREATION config/checkstyle/checkstyle-noframes.xsl PRE-CREATION config/checkstyle/checkstyle.xml PRE-CREATION gradle.properties PRE-CREATION gradle/wrapper/gradle-wrapper.jar PRE-CREATION gradle/wrapper/gradle-wrapper.properties PRE-CREATION gradlew PRE-CREATION gradlew.bat PRE-CREATION settings.gradle PRE-CREATION Diff: https://reviews.apache.org/r/66067/diff/4/ Changes: https://reviews.apache.org/r/66067/diff/3-4/ Testing --- ran all new tasks, except for internal maven publishing Notes: - To try it out you can call ./gradlew tasks --all to see all the tasks and compare them to current tasks/artifacts. - Replaced cobertura with jacoco, as it's easier/cleaner to configure, easier to combine all test results into a single report. - Generated pom.xml now has correct dependencies/versions - Script generation is currently hardcoded and not based on sqoop help, as previously - though added the possiblity of hooking it in later Thanks, Anna Szonyi
Re: Review Request 66067: SQOOP-3052: Introduce gradle-based build for Sqoop to make it more developer friendly / open
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66067/#review199871 --- Hi Anna, Thank you for replying so quickly! The changes look good in RB but I could no apply it in my repo, I get the following error: error: patch failed: COMPILING.txt:403 error: COMPILING.txt: patch does not apply error: cannot apply binary patch to 'gradle/wrapper/gradle-wrapper.jar' without full index line error: gradle/wrapper/gradle-wrapper.jar: patch does not apply - Szabolcs Vasas On March 22, 2018, 8:53 p.m., Anna Szonyi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/66067/ > --- > > (Updated March 22, 2018, 8:53 p.m.) > > > Review request for Sqoop. > > > Bugs: Sqoop-3052 > https://issues.apache.org/jira/browse/Sqoop-3052 > > > Repository: sqoop-trunk > > > Description > --- > > SQOOP-3052: Introduce gradle based build for Sqoop to make it more developer > friendly / open > > > Diffs > - > > .gitignore 68cbe28 > COMPILING.txt 86be509 > build.gradle PRE-CREATION > buildSrc/customUnixStartScript.txt PRE-CREATION > buildSrc/customWindowsStartScript.txt PRE-CREATION > buildSrc/sqoop-package.gradle PRE-CREATION > buildSrc/sqoop-version-gen.gradle PRE-CREATION > config/checkstyle/checkstyle-java-header.txt PRE-CREATION > config/checkstyle/checkstyle-noframes.xsl PRE-CREATION > config/checkstyle/checkstyle.xml PRE-CREATION > gradle.properties PRE-CREATION > gradle/wrapper/gradle-wrapper.jar PRE-CREATION > gradle/wrapper/gradle-wrapper.properties PRE-CREATION > gradlew PRE-CREATION > gradlew.bat PRE-CREATION > settings.gradle PRE-CREATION > > > Diff: https://reviews.apache.org/r/66067/diff/3/ > > > Testing > --- > > ran all new tasks, except for internal maven publishing > > Notes: > - To try it out you can call ./gradlew tasks --all to see all the tasks and > compare them to current tasks/artifacts. > - Replaced cobertura with jacoco, as it's easier/cleaner to configure, easier > to combine all test results into a single report. > - Generated pom.xml now has correct dependencies/versions > - Script generation is currently hardcoded and not based on sqoop help, as > previously - though added the possiblity of hooking it in later > > > Thanks, > > Anna Szonyi > >
