[jira] [Created] (SQOOP-3173) support DB2 xml data type when sqoop import with parquet

2017-04-13 Thread Ying Cao (JIRA) 
Ying Cao created SQOOP-3173:
---

 Summary: support DB2 xml data type when sqoop import with parquet
 Key: SQOOP-3173
 URL: https://issues.apache.org/jira/browse/SQOOP-3173
 Project: Sqoop
  Issue Type: Bug
  Components: connectors
Affects Versions: 1.4.6
 Environment: RedHat6.4+Sqoop 1.4.6 +Hadoop 2.7.3
Reporter: Ying Cao
Priority: Minor
 Fix For: 1.4.7


When sqoop run import  job with parquet from db2, the xml data type failed with 
:
17/01/16 00:48:39 ERROR tool.ImportTool: Imported Failed: Cannot convert SQL 
type 

Sqoop using Kite SDK to revolve parquet dataset,  and it requires an Avro 
schema to represent the data structure, during create avro schema sqoop can not 
find corresponding type mapping of Types.OTHER.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (SQOOP-3169) Evaluate and fix SQLServer Manual tests

2017-04-13 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/SQOOP-3169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967359#comment-15967359
 ] 

Hudson commented on SQOOP-3169:
---

SUCCESS: Integrated in Jenkins build Sqoop-hadoop200 #1107 (See 
[https://builds.apache.org/job/Sqoop-hadoop200/1107/])
SQOOP-3169: Evaluate and fix SQLServer Manual tests (annaszonyi: 
[https://git-wip-us.apache.org/repos/asf?p=sqoop.git=commit=0ca73d4e71bf4724cd7dd15faa108e6ee56ee121])
* (edit) 
src/test/org/apache/sqoop/manager/sqlserver/SQLServerWhereManualTest.java
* (edit) build.xml
* (edit) 
src/test/org/apache/sqoop/manager/sqlserver/SQLServerMultiMapsManualTest.java
* (edit) 
src/test/org/apache/sqoop/manager/sqlserver/SQLServerDatatypeImportDelimitedFileManualTest.java
* (edit) src/test/com/cloudera/sqoop/hive/TestHiveImport.java
* (edit) 
src/test/com/cloudera/sqoop/manager/SQLServerManagerExportManualTest.java
* (edit) 
src/test/org/apache/sqoop/manager/sqlserver/SQLServerDatatypeExportSequenceFileManualTest.java
* (edit) 
src/test/org/apache/sqoop/manager/sqlserver/SQLServerHiveImportManualTest.java
* (edit) 
src/test/org/apache/sqoop/manager/sqlserver/SQLServerDatatypeExportDelimitedFileManualTest.java
* (edit) 
src/test/org/apache/sqoop/manager/sqlserver/SQLServerSplitByManualTest.java
* (edit) 
src/test/org/apache/sqoop/manager/sqlserver/SQLServerDatatypeImportSequenceFileManualTest.java
* (edit) src/test/org/apache/sqoop/manager/sqlserver/ManagerCompatExport.java
* (edit) 
src/test/org/apache/sqoop/manager/sqlserver/SQLServerQueryManualTest.java
* (edit) 
src/test/org/apache/sqoop/manager/sqlserver/SQLServerManagerManualTest.java
* (edit) 
src/test/org/apache/sqoop/manager/sqlserver/SQLServerParseMethodsManualTest.java
* (edit) 
src/test/com/cloudera/sqoop/manager/SQLServerManagerImportManualTest.java
* (edit) src/test/org/apache/sqoop/manager/sqlserver/MSSQLTestUtils.java
* (edit) 
src/test/org/apache/sqoop/manager/sqlserver/SQLServerMultiColsManualTest.java


> Evaluate and fix SQLServer Manual tests
> ---
>
> Key: SQOOP-3169
> URL: https://issues.apache.org/jira/browse/SQOOP-3169
> Project: Sqoop
>  Issue Type: Improvement
>  Components: test
>Affects Versions: 1.4.6
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
> Attachments: SQOOP-3169.patch
>
>
> SQL Server Manual tests seem to be somewhat abandoned as there are several 
> failing test cases which don't bring any value to the quality of Sqoop. I 
> took a look at them and found some improvement areas besides the failing test 
> cases. Fixing and improving these tests would make easier to execute them in 
> case of SQL Server related changes which would help to keep Sqoop more robust.
> The following 14 test classes are affected by this ticket:
> * com.cloudera.sqoop.manager
> ** SQLServerManagerExportManualTest
> ** SQLServerManagerImportManualTest
> * org.apache.sqoop.manager.sqlserver
> ** SQLServerDatatypeExportDelimitedFileManualTest
> ** SQLServerDatatypeExportSequenceFileManualTest
> ** SQLServerDatatypeImportDelimitedFileManualTest
> ** SQLServerDatatypeImportSequenceFileManualTest
> ** SQLServerHiveImportManualTest
> ** SQLServerManagerManualTest
> ** SQLServerMultiColsManualTest
> ** SQLServerMultiMapsManualTest
> ** SQLServerParseMethodsManualTest
> ** SQLServerQueryManualTest
> ** SQLServerSplitByManualTest
> ** SQLServerWhereManualTest
> Scope of this ticket is only fixing the test cases and also make them unified 
> from the DB connect string and credentials point of view. However, an 
> extensive refactor could be made in these test classes, which is addressed in 
> another ticket, see SQOOP-3170



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (SQOOP-3169) Evaluate and fix SQLServer Manual tests

2017-04-13 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/SQOOP-3169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967346#comment-15967346
 ] 

ASF subversion and git services commented on SQOOP-3169:


Commit 0ca73d4e71bf4724cd7dd15faa108e6ee56ee121 in sqoop's branch 
refs/heads/trunk from [~szonyi]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=0ca73d4 ]

SQOOP-3169: Evaluate and fix SQLServer Manual tests

(Boglarka Egyed via Anna Szonyi)


> Evaluate and fix SQLServer Manual tests
> ---
>
> Key: SQOOP-3169
> URL: https://issues.apache.org/jira/browse/SQOOP-3169
> Project: Sqoop
>  Issue Type: Improvement
>  Components: test
>Affects Versions: 1.4.6
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
> Attachments: SQOOP-3169.patch
>
>
> SQL Server Manual tests seem to be somewhat abandoned as there are several 
> failing test cases which don't bring any value to the quality of Sqoop. I 
> took a look at them and found some improvement areas besides the failing test 
> cases. Fixing and improving these tests would make easier to execute them in 
> case of SQL Server related changes which would help to keep Sqoop more robust.
> The following 14 test classes are affected by this ticket:
> * com.cloudera.sqoop.manager
> ** SQLServerManagerExportManualTest
> ** SQLServerManagerImportManualTest
> * org.apache.sqoop.manager.sqlserver
> ** SQLServerDatatypeExportDelimitedFileManualTest
> ** SQLServerDatatypeExportSequenceFileManualTest
> ** SQLServerDatatypeImportDelimitedFileManualTest
> ** SQLServerDatatypeImportSequenceFileManualTest
> ** SQLServerHiveImportManualTest
> ** SQLServerManagerManualTest
> ** SQLServerMultiColsManualTest
> ** SQLServerMultiMapsManualTest
> ** SQLServerParseMethodsManualTest
> ** SQLServerQueryManualTest
> ** SQLServerSplitByManualTest
> ** SQLServerWhereManualTest
> Scope of this ticket is only fixing the test cases and also make them unified 
> from the DB connect string and credentials point of view. However, an 
> extensive refactor could be made in these test classes, which is addressed in 
> another ticket, see SQOOP-3170



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (SQOOP-3172) SQOOP - Broken Pipe Error in the Sqoop export

2017-04-13 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/SQOOP-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967277#comment-15967277
 ] 

ASF GitHub Bot commented on SQOOP-3172:
---

Github user ebogi commented on the issue:

https://github.com/apache/sqoop/pull/35
  
Hi,
Sqoop project currently does not accept pull requests. To contribute, you 
will need to generate patch and upload it to the JIRA. See instructions here: 
https://cwiki.apache.org/confluence/display/SQOOP/How+to+Contribute
If you need more help, feel free to reach out to the community at 
dev@sqoop.apache.org


> SQOOP - Broken Pipe Error in the Sqoop export
> -
>
> Key: SQOOP-3172
> URL: https://issues.apache.org/jira/browse/SQOOP-3172
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.4.5
>Reporter: Akshay
>  Labels: sqoop, testing
> Fix For: 1.4.5
>
>
> Error: java.io.IOException: Broken pipe
> at java.io.FileOutputStream.writeBytes(Native Method)
> at java.io.FileOutputStream.write(FileOutputStream.java:326)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
> at 
> org.apache.sqoop.mapreduce.MySQLExportMapper.closeExportHandles(MySQLExportMapper.java:259)
> at 
> org.apache.sqoop.mapreduce.MySQLExportMapper.run(MySQLExportMapper.java:250)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:796)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] sqoop issue #35: SQOOP-3172:checking whether the object is stale or not

2017-04-13 Thread ebogi 
Github user ebogi commented on the issue:

https://github.com/apache/sqoop/pull/35
  
Hi,
Sqoop project currently does not accept pull requests. To contribute, you 
will need to generate patch and upload it to the JIRA. See instructions here: 
https://cwiki.apache.org/confluence/display/SQOOP/How+to+Contribute
If you need more help, feel free to reach out to the community at 
dev@sqoop.apache.org


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Comment Edited] (SQOOP-3172) SQOOP - Broken Pipe Error in the Sqoop export

2017-04-13 Thread Akshay (JIRA) 

[ 
https://issues.apache.org/jira/browse/SQOOP-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967254#comment-15967254
 ] 

Akshay edited comment on SQOOP-3172 at 4/13/17 7:52 AM:


I am releasing a patch
 for version 1.4.5 .  Please review the pull request and merge changes



was (Author: shetaksroc):
I am releasing a path for this version.  Please review the pull request

> SQOOP - Broken Pipe Error in the Sqoop export
> -
>
> Key: SQOOP-3172
> URL: https://issues.apache.org/jira/browse/SQOOP-3172
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.4.5
>Reporter: Akshay
>  Labels: sqoop, testing
> Fix For: 1.4.5
>
>
> Error: java.io.IOException: Broken pipe
> at java.io.FileOutputStream.writeBytes(Native Method)
> at java.io.FileOutputStream.write(FileOutputStream.java:326)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
> at 
> org.apache.sqoop.mapreduce.MySQLExportMapper.closeExportHandles(MySQLExportMapper.java:259)
> at 
> org.apache.sqoop.mapreduce.MySQLExportMapper.run(MySQLExportMapper.java:250)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:796)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (SQOOP-3172) SQOOP - Broken Pipe Error in the Sqoop export

2017-04-13 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/SQOOP-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967249#comment-15967249
 ] 

ASF GitHub Bot commented on SQOOP-3172:
---

GitHub user shetaksroc opened a pull request:

https://github.com/apache/sqoop/pull/35

SQOOP-3172:checking whether the object is stale or not



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shetaksroc/sqoop fix_state_connection

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/sqoop/pull/35.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #35


commit b26615cc69885e96fd100e7451e0f5657cd5616b
Author: akshayshet 
Date:   2017-04-13T07:43:09Z

SQOOP-3172:checking whether the object is stale or not




> SQOOP - Broken Pipe Error in the Sqoop export
> -
>
> Key: SQOOP-3172
> URL: https://issues.apache.org/jira/browse/SQOOP-3172
> Project: Sqoop
>  Issue Type: Bug
>Reporter: Akshay
>  Labels: sqoop, testing
>
> Error: java.io.IOException: Broken pipe
> at java.io.FileOutputStream.writeBytes(Native Method)
> at java.io.FileOutputStream.write(FileOutputStream.java:326)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
> at 
> org.apache.sqoop.mapreduce.MySQLExportMapper.closeExportHandles(MySQLExportMapper.java:259)
> at 
> org.apache.sqoop.mapreduce.MySQLExportMapper.run(MySQLExportMapper.java:250)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:796)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] sqoop pull request #35: SQOOP-3172:checking whether the object is stale or n...

2017-04-13 Thread shetaksroc 
GitHub user shetaksroc opened a pull request:

https://github.com/apache/sqoop/pull/35

SQOOP-3172:checking whether the object is stale or not



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shetaksroc/sqoop fix_state_connection

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/sqoop/pull/35.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #35


commit b26615cc69885e96fd100e7451e0f5657cd5616b
Author: akshayshet 
Date:   2017-04-13T07:43:09Z

SQOOP-3172:checking whether the object is stale or not




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---