[jira] [Commented] (SQOOP-3487) Remove unintentionally added rowkeys (via PUT in ToStringPutTransformer.java) during HBase (incremental) import

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302545#comment-17302545
 ] 

ASF subversion and git services commented on SQOOP-3487:


Commit f8beae32a067d72bf9ed6e903b041ad347ca5491 in sqoop's branch 
refs/heads/trunk from 游侠
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=f8beae3 ]

SQOOP-3487: Remove unintentionally added rowkeys (via PUT in 
ToStringPutTransformer.java) during HBase (incremental) import

(Zhou Hong via Attila Szabo)

> Remove unintentionally added rowkeys (via PUT in ToStringPutTransformer.java) 
> during HBase (incremental) import
> ---
>
> Key: SQOOP-3487
> URL: https://issues.apache.org/jira/browse/SQOOP-3487
> Project: Sqoop
>  Issue Type: Improvement
>  Components: hbase-integration
>Affects Versions: no-release
> Environment: sqoop-version:1.4.7-cdh6.3.2
>Reporter: hong
>Priority: Major
>  Labels: hbase
> Fix For: no-release
>
> Attachments: 
> 0001-SQOOP-3487-Add-PUT-repeatedly-when-importing-to-HBas.patch
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Repeated add of PUT cause the import into HBase to slow down



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (SQOOP-3462) Sqoop ant build fails due to outdated maven repo URLs

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023645#comment-17023645
 ] 

ASF subversion and git services commented on SQOOP-3462:


Commit a9f051c5af31a8e70be1e3c2684dc351e4311243 in sqoop's branch 
refs/heads/trunk from Istvan Toth
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=a9f051c ]

SQOOP-3462: Sqoop ant build fails due to outdated maven repo URLs

(Istvan Toth via Szabolcs Vasas)


> Sqoop ant build fails due to outdated maven repo URLs
> -
>
> Key: SQOOP-3462
> URL: https://issues.apache.org/jira/browse/SQOOP-3462
> Project: Sqoop
>  Issue Type: Bug
>  Components: build
>Reporter: Istvan Toth
>Priority: Critical
> Attachments: SQOOP-3462.v1.patch
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Sqoop can no longer be built with ant, as the maven central repos no longer 
> support HTTP.
> {{[ivy:resolve] SERVER ERROR: HTTPS Required 
> url=http://repo1.maven.org/maven2/org/apache/avro/avro/1.8.1/avro-1.8.1.pom}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (SQOOP-3428) Fix the CI

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952788#comment-16952788
 ] 

ASF subversion and git services commented on SQOOP-3428:


Commit 875e431af15c514a6c3dbf30e3afefdf640c66f9 in sqoop's branch 
refs/heads/trunk from Fokko Driesprong
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=875e431 ]

SQOOP-3428: Try to fix the CI (#85)



> Fix the CI
> --
>
> Key: SQOOP-3428
> URL: https://issues.apache.org/jira/browse/SQOOP-3428
> Project: Sqoop
>  Issue Type: Improvement
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Currently, the CI is broken because the Oracle 11 XE Dockerimage isn't 
> available anymore.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (SQOOP-3441) Prepare Sqoop for Java 11 support

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862068#comment-16862068
 ] 

ASF subversion and git services commented on SQOOP-3441:


Commit cee4ab15c729e213e622f8f91674a480aca6bd49 in sqoop's branch 
refs/heads/trunk from Fero Szabo
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=cee4ab1 ]

SQOOP-3441: Prepare Sqoop for Java 11 support

(Fero Szabo via Boglarka Egyed)

> Prepare Sqoop for Java 11 support
> -
>
> Key: SQOOP-3441
> URL: https://issues.apache.org/jira/browse/SQOOP-3441
> Project: Sqoop
>  Issue Type: Improvement
>Reporter: Fero Szabo
>Assignee: Fero Szabo
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> A couple of code changes will be required in order for Sqoop to work with 
> Java11 and we'll also have to bump a couple of dependencies and the gradle 
> version. 
> I'm not sure what's required for ant, that is to be figured out in a separate 
> Jira, if we keep the ant build.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3438) Sqoop Import with create hcatalog table for ORC will not work with Hive3 as the table created would be a ACID table and transactional

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856496#comment-16856496
 ] 

ASF subversion and git services commented on SQOOP-3438:


Commit be260e3096a6a2710e661c7fe33f6b402ce66476 in sqoop's branch 
refs/heads/trunk from Denes Bodo
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=be260e3 ]

SQOOP-3438: Sqoop Import with create hcatalog table for ORC will not work with 
Hive3 as the table created would be a ACID table and transactional

(Denes Bodo via Boglarka Egyed)


> Sqoop Import with create hcatalog table for ORC will not work with Hive3 as 
> the table created would be a ACID table and transactional
> -
>
> Key: SQOOP-3438
> URL: https://issues.apache.org/jira/browse/SQOOP-3438
> Project: Sqoop
>  Issue Type: Improvement
>  Components: hive-integration
>Affects Versions: 1.4.7
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> PROBLEM: Running a sqoop import command with the option 
> --create-hcatalog-table will not work due to the following reasons
> When create-hcatalog-table is used it creates the table as a Managed ACID 
> table.
> HCatalog does not support transactional or bucketing table
> So customer who need to create a ORC based table cannot use sqoop to create a 
> ORC based table which means their existing code where if in case they use 
> sqoop to create these tables would fail.
> The current workaround is a two step process
> 1. Create the ORC table in hive with the keyword external and set 
> transactional to false
> 2. Then use the sqoop command to load the data into the orc table.
> The request is to add in an extra argument in the sqoop command line to 
> specify that the table is external (example: --hcatalog-external-table )so we 
> can use the option --hcatalog-storage-stanza "stored as orc tblproperties 
> (\"transactional\"=\"false\")".
> 
> Thank you [~mbalakrishnan] for your findings. This ticket is created based on 
> your work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3423) Let user pass password to connect Hive when it set to LDAP authentication

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16833840#comment-16833840
 ] 

ASF subversion and git services commented on SQOOP-3423:


Commit dfeb14534ff0c57ebe27625aa9c79e37b4e12558 in sqoop's branch 
refs/heads/trunk from Denes Bodo
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=dfeb145 ]

SQOOP-3423: Let user pass password to connect Hive when it set to LDAP 
authentication

(Denes Bodo via Boglarka Egyed)

> Let user pass password to connect Hive when it set to LDAP authentication
> -
>
> Key: SQOOP-3423
> URL: https://issues.apache.org/jira/browse/SQOOP-3423
> Project: Sqoop
>  Issue Type: Improvement
>  Components: hive-integration
>Affects Versions: 1.4.7
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Major
> Attachments: SQOOP-3423-001.patch
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> If HiveServer2 is using password based authentication, additional 
> username/password information has to be provided to be able to connect to it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3134) --class-name should override default Avro schema name

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810872#comment-16810872
 ] 

ASF subversion and git services commented on SQOOP-3134:


Commit 1a04d2007d7e4e111d9cde9efbe3485547075fc6 in sqoop's branch 
refs/heads/trunk from daniel voros
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=1a04d20 ]

SQOOP-3134: --class-name should override default Avro schema name



> --class-name should override default Avro schema name
> -
>
> Key: SQOOP-3134
> URL: https://issues.apache.org/jira/browse/SQOOP-3134
> Project: Sqoop
>  Issue Type: Improvement
>Reporter: Markus Kemper
>Assignee: Daniel Voros
>Priority: Major
> Attachments: SQOOP-3134.1.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Please consider adding an option to configure the Avro schema output file 
> name that is created with Sqoop (import + --as-avrodatafile), example cases 
> below.
> {noformat}
> #
> # STEP 01 - Create Data
> #
> export MYCONN=jdbc:mysql://mysql.cloudera.com:3306/db_coe
> export MYUSER=sqoop
> export MYPSWD=cloudera
> sqoop list-tables --connect $MYCONN --username $MYUSER --password $MYPSWD
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query 
> "drop table t1"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query 
> "create table t1 (c1 int, c2 date, c3 varchar(10))"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query 
> "insert into t1 values (1, current_date, 'some data')"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query 
> "select * from t1"
> -
> | c1  | c2 | c3 | 
> -
> | 1   | 2017-02-13 | some data  | 
> -
> #
> # STEP 02 - Import + --table + --as-avrodatafile
> #
> sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table 
> t1 --target-dir /user/root/t1 --delete-target-dir --num-mappers 1 
> --as-avrodatafile 
> ls -l ./*
> Output:
> 17/02/13 12:14:52 INFO mapreduce.ImportJobBase: Transferred 413 bytes in 
> 20.6988 seconds (19.9529 bytes/sec)
> 17/02/13 12:14:52 INFO mapreduce.ImportJobBase: Retrieved 1 records.
> 
> -rw-r--r-- 1 root root   492 Feb 13 12:14 ./t1.avsc < want option to 
> configure this file name
> -rw-r--r-- 1 root root 12462 Feb 13 12:14 ./t1.java
> #
> # STEP 03 - Import + --query + --as-avrodatafile
> #
> sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --query 
> "select * from t1 where \$CONDITIONS" --split-by c1 --target-dir 
> /user/root/t1 --delete-target-dir --num-mappers 1 --as-avrodatafile 
> ls -l ./*
> Output:
> 17/02/13 12:16:58 INFO mapreduce.ImportJobBase: Transferred 448 bytes in 
> 25.2757 seconds (17.7245 bytes/sec)
> 17/02/13 12:16:58 INFO mapreduce.ImportJobBase: Retrieved 1 records.
> ~
> -rw-r--r-- 1 root root   527 Feb 13 12:16 ./AutoGeneratedSchema.avsc < 
> want option to configure this file name
> -rw-r--r-- 1 root root 12590 Feb 13 12:16 ./QueryResult.java
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3435) SqoopOptions missed a null check

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808479#comment-16808479
 ] 

ASF subversion and git services commented on SQOOP-3435:


Commit e90e244396ecffeb332633c89e641357adcf8eda in sqoop's branch 
refs/heads/trunk from Tak Lon (Stephen) Wu
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=e90e244 ]

SQOOP-3435: Avoid NullPointerException due to different JSONObject library in 
classpath



> SqoopOptions missed a null check
> 
>
> Key: SQOOP-3435
> URL: https://issues.apache.org/jira/browse/SQOOP-3435
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.4.7, 1.5.0
>Reporter: Tak Lon (Stephen) Wu
>Priority: Major
> Attachments: SQOOP-3435.trunk.001.patch, SQOOP-3435.trunk.002.patch
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> In line of 
> [SqoopOptions.java#L785|https://github.com/apache/sqoop/blob/branch-1.4.7/src/java/org/apache/sqoop/SqoopOptions.java#L785]
>  with
> {\{SqoopJsonUtil.getJsonStringforMap((Map) f.get(this))); }}
> Above line should check NULL pointer like in line of 
> [SqoopOptions.java#L778|https://github.com/apache/sqoop/blob/branch-1.4.7/src/java/org/apache/sqoop/SqoopOptions.java#L778]
>  which has
> {{f.get(this) == null ? "null" : f.get(this).toString()}}
> Please see the stacktrace below when running command:
> {{sqoop job --create myjob -- import --connect jdbc:mysql://localhost/db 
> --username root --table employee --m 1}}
> {code:java}
> 19/02/02 01:09:21 ERROR sqoop.Sqoop: Got exception running Sqoop: 
> java.lang.NullPointerException
> java.lang.NullPointerException
> at org.json.JSONObject.(JSONObject.java:144)
> at 
> org.apache.sqoop.util.SqoopJsonUtil.getJsonStringforMap(SqoopJsonUtil.java:43)
> at org.apache.sqoop.SqoopOptions.writeProperties(SqoopOptions.java:785)
> at 
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.createInternal(HsqldbJobStorage.java:399)
> at 
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.create(HsqldbJobStorage.java:379)
> at org.apache.sqoop.tool.JobTool.createJob(JobTool.java:181)
> at org.apache.sqoop.tool.JobTool.run(JobTool.java:294)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
> {code}
> The above {{NullPointerException}} is due to use the of 
> [{{com.tdunning}}|https://github.com/tdunning/open-json/blob/rc1.8/src/main/java/org/json/JSONObject.java#L141-L155]
>  as part of the HIVE libs (if one is reusing the {{HADOOP_CLASSPATH}}) in the 
> classpath. but I think we can better have a checker of {{null}} in 
> {{SqoopJsonUtil.getJsonStringforMap(Map map)}} before calling 
> {{JSONObject pathPartMap = new JSONObject(map);}}
> Reporting this bug and the right behavior need to be decided by the assignee. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3427) Add Travis badge to the Readme

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794863#comment-16794863
 ] 

ASF subversion and git services commented on SQOOP-3427:


Commit 0216f7fbb7b71f610d02dfa95dafb2c81acbc690 in sqoop's branch 
refs/heads/trunk from Fokko Driesprong
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=0216f7f ]

SQOOP-3427: Add Travis badge to the Readme




> Add Travis badge to the Readme
> --
>
> Key: SQOOP-3427
> URL: https://issues.apache.org/jira/browse/SQOOP-3427
> Project: Sqoop
>  Issue Type: Improvement
>  Components: docs
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> We love badges



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3429) Bump Hadoop to 2.9.2

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791846#comment-16791846
 ] 

ASF subversion and git services commented on SQOOP-3429:


Commit 3007648c0801c0ef2ad3ef1eb9e3b8681f1c2e02 in sqoop's branch 
refs/heads/trunk from Fokko Driesprong
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=3007648 ]

SQOOP-3429: Bump Hadoop to 2.9.2




> Bump Hadoop to 2.9.2
> 
>
> Key: SQOOP-3429
> URL: https://issues.apache.org/jira/browse/SQOOP-3429
> Project: Sqoop
>  Issue Type: Improvement
>Affects Versions: 1.4.7
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.5.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> I would like to bump Sqoop to Hadoop 2.9.3



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3430) Remove the old maven pom

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791556#comment-16791556
 ] 

ASF subversion and git services commented on SQOOP-3430:


Commit 279f84c53db93c47845f6be7124774b1f5e40c76 in sqoop's branch 
refs/heads/trunk from Fokko Driesprong
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=279f84c ]

SQOOP-3430: Remove the old maven pom



> Remove the old maven pom
> 
>
> Key: SQOOP-3430
> URL: https://issues.apache.org/jira/browse/SQOOP-3430
> Project: Sqoop
>  Issue Type: Improvement
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> I think both ant and gradle are enough :)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3428) Remove the old Maven pom

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787972#comment-16787972
 ] 

ASF subversion and git services commented on SQOOP-3428:


Commit 20fe120706039adbf7e58985f93b2020f688abc0 in sqoop's branch 
refs/heads/trunk from Fokko Driesprong
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=20fe120 ]

SQOOP-3428: Fix the CI

(Fokko Driesprong via Fero Szabo)


> Remove the old Maven pom
> 
>
> Key: SQOOP-3428
> URL: https://issues.apache.org/jira/browse/SQOOP-3428
> Project: Sqoop
>  Issue Type: Improvement
>Reporter: Fokko Driesprong
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> I think both ant and gradle are enough :-)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3420) Invalid ERROR message initiates false alarms

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758425#comment-16758425
 ] 

ASF subversion and git services commented on SQOOP-3420:


Commit 5ab5190304d0e13d1c5eaa56d1f486f3106529d3 in sqoop's branch 
refs/heads/trunk from Denes Bodo
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=5ab5190 ]

SQOOP-3420: Invalid ERROR message initiates false alarms

(Denes Bodo via Fero Szabo)

> Invalid ERROR message initiates false alarms
> 
>
> Key: SQOOP-3420
> URL: https://issues.apache.org/jira/browse/SQOOP-3420
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.4.7
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
>  Labels: usability
> Attachments: SQOOP-3420_001.patch
>
>
> In SQOOP-3042, a debug message was refactored to be error instead means false 
> alarms in customer log analyser. After understanding the functionality it is 
> recommended to use info level message instead of error in case when 
> ImportTool is unable to backup generated .java file.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3237) Mainframe FTP transfer option to insert custom FTP commands prior to transfer

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720251#comment-16720251
 ] 

ASF subversion and git services commented on SQOOP-3237:


Commit 4a22691f45d7d66157ff6dfaa8fca5581e0a8955 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=4a22691 ]

SQOOP-3237: Mainframe FTP transfer option to insert custom FTP commands prior 
to transfer

(Chris Teoh via Szabolcs Vasas)


> Mainframe FTP transfer option to insert custom FTP commands prior to transfer
> -
>
> Key: SQOOP-3237
> URL: https://issues.apache.org/jira/browse/SQOOP-3237
> Project: Sqoop
>  Issue Type: Improvement
>Reporter: Chris Teoh
>Assignee: Chris Teoh
>Priority: Minor
>
> To get some data included in the FTP transfer, some extra FTP commands need 
> to be added prior to transfer. A sample interaction below:-
> ftp> binary
> 200 Representation type is Image
> ftp> {color:red}quote SITE RDW{color}
> 200 SITE command was accepted
> ftp> {color:red}quote SITE RDW READTAPEFORMAT=V{color}
> 200 SITE command was accepted
> ftp>
> Proposed approach: --ftpcmds {csv list of commands}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3418) Document decimal support in Hive external import into parquet files

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719052#comment-16719052
 ] 

ASF subversion and git services commented on SQOOP-3418:


Commit b94a0bd948a5f51b9fd708dece5653130f1239ec in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=b94a0bd ]

SQOOP-3418: Document decimal support in Hive external import into parquet files

(Fero Szabo via Szabolcs Vasas)

> Document decimal support in Hive external import into parquet files
> ---
>
> Key: SQOOP-3418
> URL: https://issues.apache.org/jira/browse/SQOOP-3418
> Project: Sqoop
>  Issue Type: Task
>Reporter: Fero Szabo
>Assignee: Fero Szabo
>Priority: Major
> Fix For: 1.5.0, 3.0.0
>
>
> Remember to note the limitations in Hive i.e. the max scale and precision is 
> 38 and how it behaves in edge cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3396) Add parquet numeric support for Parquet in Hive import

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712782#comment-16712782
 ] 

ASF subversion and git services commented on SQOOP-3396:


Commit a50394977bcdec8ae2274618b3a5c9e7e6a1082b in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=a503949 ]

SQOOP-3396: Add parquet numeric support for Parquet in Hive import

(Fero Szabo via Szabolcs Vasas)

This closes #60


> Add parquet numeric support for Parquet in Hive import
> --
>
> Key: SQOOP-3396
> URL: https://issues.apache.org/jira/browse/SQOOP-3396
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Fero Szabo
>Assignee: Fero Szabo
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3417) Execute Oracle XE tests on Travis CI

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708770#comment-16708770
 ] 

ASF subversion and git services commented on SQOOP-3417:


Commit 302674d96b18bae3c5283d16603afb985b892795 in sqoop's branch 
refs/heads/trunk from [~fero]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=302674d ]

SQOOP-3417: Execute Oracle XE tests on Travis CI

(Szabolcs Vasas via Fero Szabo)

This closes #65


> Execute Oracle XE tests on Travis CI
> 
>
> Key: SQOOP-3417
> URL: https://issues.apache.org/jira/browse/SQOOP-3417
> Project: Sqoop
>  Issue Type: Test
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
>
> The task is to enable the Travis CI to execute Oracle XE tests too 
> automatically.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3393) TestNetezzaExternalTableExportMapper hangs

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706984#comment-16706984
 ] 

ASF subversion and git services commented on SQOOP-3393:


Commit 3c1fb870e20bf2d0316a7eb0aa524e437499ab05 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=3c1fb87 ]

SQOOP-3393: TestNetezzaExternalTableExportMapper hangs

(Daniel Voros via Szabolcs Vasas)

This closes #63


> TestNetezzaExternalTableExportMapper hangs
> --
>
> Key: SQOOP-3393
> URL: https://issues.apache.org/jira/browse/SQOOP-3393
> Project: Sqoop
>  Issue Type: Bug
>  Components: test
>Affects Versions: 1.5.0, 3.0.0
>Reporter: Daniel Voros
>Assignee: Daniel Voros
>Priority: Major
> Fix For: 1.5.0, 3.0.0
>
>
> Introduced in SQOOP-3378, spotted by [~vasas].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3415) Fix gradle test+build when clean applied as the first command + warning issue fixes

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706973#comment-16706973
 ] 

ASF subversion and git services commented on SQOOP-3415:


Commit 122d1c0b27c0f5e4532f7a23ce016a5c5756321a in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=122d1c0 ]

SQOOP-3415: Fix gradle test+build when clean applied as the first command + 
warning issue fixes

(Attila Szabo via Szabolcs Vasas)

This closes #62


> Fix gradle test+build when clean applied as the first command + warning issue 
> fixes
> ---
>
> Key: SQOOP-3415
> URL: https://issues.apache.org/jira/browse/SQOOP-3415
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.5.0
>Reporter: Attila Szabo
>Assignee: Attila Szabo
>Priority: Major
> Fix For: 1.5.0
>
>
> If the user wants to build like the following command:
> gradlew clean unittest
> the gradle process ends up in an exception and the whole process left there 
> hanging forever. The root cause of this is the following:
> tasks.withType runs in the configuration part of the build, where we ensure 
> the neccessary directories exist.
> after that clean is executed and all of the dirs got deleted.
> Proposed fix:
> Apply directory creation as the first step of test tasks.
> on the top:
> there are some missing options b/c of Junit annotation processors, and also 
> Xlint information are swallowed currently. We aim to fix these things as well



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3414) Introduce a Gradle build parameter to set the ignoreTestFailures of the test tasks

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16701632#comment-16701632
 ] 

ASF subversion and git services commented on SQOOP-3414:


Commit 6a159ed282c1816452a46dbc7acc2a56b5d2dc46 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=6a159ed ]

SQOOP-3414: Introduce a Gradle build parameter to set the ignoreTestFailures of 
the test tasks

This closes #59

(Szabolcs Vasas via Boglarka Egyed)


> Introduce a Gradle build parameter to set the ignoreTestFailures of the test 
> tasks
> --
>
> Key: SQOOP-3414
> URL: https://issues.apache.org/jira/browse/SQOOP-3414
> Project: Sqoop
>  Issue Type: Test
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: test_with_ignoreTestFailures=true.txt, 
> test_without_ignoreTestFailures.txt
>
>
> The 
> [ignoreFailures|https://docs.gradle.org/current/dsl/org.gradle.api.tasks.testing.Test.html#org.gradle.api.tasks.testing.Test:ignoreFailures]
>  parameter of the Gradle test tasks is set to false which means that if a 
> Gradle test task fails the gradle
> process returns with non-zero. In some CI tools (e.g. Jenkins) this will make 
> the status of the job red and not yellow
> which usually means some more serious issue than a test failure.
> I would like to introduce a parameter to be able set this parameter of the 
> test tasks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3413) TestMainframeManager does not restore the inner state of AccumuloUtil

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16700613#comment-16700613
 ] 

ASF subversion and git services commented on SQOOP-3413:


Commit f8fe691e21aadaf51cc51905d94b3dc22a7f818c in sqoop's branch 
refs/heads/trunk from [~fero]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=f8fe691 ]

SQOOP-3413: TestMainframeManager does not restore the inner state of 
AccumuloUtil

(Szabolcs Vasas via Fero Szabo)

This closes #52


> TestMainframeManager does not restore the inner state of AccumuloUtil
> -
>
> Key: SQOOP-3413
> URL: https://issues.apache.org/jira/browse/SQOOP-3413
> Project: Sqoop
>  Issue Type: Test
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
>
> org.apache.sqoop.manager.TestMainframeManager#testImportTableNoAccumuloJarPresent
>  sets the testingMode field of AccumuloUtil to true but it does not restores 
> it so Accumulo tests will fail if they are executed after 
> TestMainframeManager.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3411) PostgresMetaConnectIncrementalImportTest fails if metastore tables are absent from the database

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16698784#comment-16698784
 ] 

ASF subversion and git services commented on SQOOP-3411:


Commit 696187bb16ae31ca2aa78be93c342604cc1290d1 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=696187b ]

SQOOP-3411: PostgresMetaConnectIncrementalImportTest fails if metastore tables 
are absent from the database

This closes #51

(Szabolcs Vasas via Boglarka Egyed)


> PostgresMetaConnectIncrementalImportTest fails if metastore tables are absent 
> from the database
> ---
>
> Key: SQOOP-3411
> URL: https://issues.apache.org/jira/browse/SQOOP-3411
> Project: Sqoop
>  Issue Type: Test
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3410) Test S3 import with fs.s3a.security.credential.provider.path

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697189#comment-16697189
 ] 

ASF subversion and git services commented on SQOOP-3410:


Commit 08eb5bdc405c22a25cf680390bbd438f6513199c in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=08eb5bd ]

SQOOP-3410: Test S3 import with fs.s3a.security.credential.provider.path

(Boglarka Egyed via Szabolcs Vasas)


> Test S3 import with fs.s3a.security.credential.provider.path
> 
>
> Key: SQOOP-3410
> URL: https://issues.apache.org/jira/browse/SQOOP-3410
> Project: Sqoop
>  Issue Type: Improvement
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Major
> Attachments: SQOOP-3410.patch
>
>
> Based on 
> [https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html#Configure_the_hadoop.security.credential.provider.path_property]
>  property fs.s3a.security.credential.provider.path can also be used for 
> passing the location of the credential store. This should be also tested and 
> documented.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3289) Add .travis.yml

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697180#comment-16697180
 ] 

ASF subversion and git services commented on SQOOP-3289:


Commit 2bf6f3ccd0b59c9ff29d56a32b25f3a659dcfe19 in sqoop's branch 
refs/heads/trunk from [~fero]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=2bf6f3c ]

SQOOP-3289: Add .travis.yml

(Szabolcs Vasas via Fero Szabo)


> Add .travis.yml
> ---
>
> Key: SQOOP-3289
> URL: https://issues.apache.org/jira/browse/SQOOP-3289
> Project: Sqoop
>  Issue Type: Sub-task
>  Components: build
>Affects Versions: 1.4.7
>Reporter: Daniel Voros
>Assignee: Szabolcs Vasas
>Priority: Minor
> Fix For: 1.5.0, 3.0.0
>
> Attachments: SQOOP-3289.patch
>
>
> Adding a .travis.yml would enable running builds/tests on travis-ci.org. 
> Currently if you wish to use Travis for testing your changes, you have to 
> manually add a .travis.yml to your branch. Having it committed to trunk would 
> save us this extra step.
> I currently have an example 
> [{{.travis.yml}}|https://github.com/dvoros/sqoop/blob/93a4c06c1a3da1fd5305c99e379484507797b3eb/.travis.yml]
>  on my travis branch running unit tests for every commit and every pull 
> request: https://travis-ci.org/dvoros/sqoop/builds
> Later we could add the build status to the project readme as well, see: 
> https://github.com/dvoros/sqoop/tree/travis
> Also, an example of a pull request: https://github.com/dvoros/sqoop/pull/1



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3289) Add .travis.yml

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697179#comment-16697179
 ] 

ASF subversion and git services commented on SQOOP-3289:


Commit cbc39c3bfa04001a411fda456429e686220ecbba in sqoop's branch 
refs/heads/trunk from [~fero]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=cbc39c3 ]

Revert "SQOOP-3289: Add .travis.yml"

This reverts commit 83a18e195111adb9f906401b0c030666378bae69.

Reverting because of accidental inclusion of .cache file.


> Add .travis.yml
> ---
>
> Key: SQOOP-3289
> URL: https://issues.apache.org/jira/browse/SQOOP-3289
> Project: Sqoop
>  Issue Type: Sub-task
>  Components: build
>Affects Versions: 1.4.7
>Reporter: Daniel Voros
>Assignee: Szabolcs Vasas
>Priority: Minor
> Fix For: 1.5.0, 3.0.0
>
> Attachments: SQOOP-3289.patch
>
>
> Adding a .travis.yml would enable running builds/tests on travis-ci.org. 
> Currently if you wish to use Travis for testing your changes, you have to 
> manually add a .travis.yml to your branch. Having it committed to trunk would 
> save us this extra step.
> I currently have an example 
> [{{.travis.yml}}|https://github.com/dvoros/sqoop/blob/93a4c06c1a3da1fd5305c99e379484507797b3eb/.travis.yml]
>  on my travis branch running unit tests for every commit and every pull 
> request: https://travis-ci.org/dvoros/sqoop/builds
> Later we could add the build status to the project readme as well, see: 
> https://github.com/dvoros/sqoop/tree/travis
> Also, an example of a pull request: https://github.com/dvoros/sqoop/pull/1



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3289) Add .travis.yml

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697056#comment-16697056
 ] 

ASF subversion and git services commented on SQOOP-3289:


Commit 83a18e195111adb9f906401b0c030666378bae69 in sqoop's branch 
refs/heads/trunk from [~fero]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=83a18e1 ]

SQOOP-3289: Add .travis.yml

(Szabolcs Vasas via Fero Szabo)


> Add .travis.yml
> ---
>
> Key: SQOOP-3289
> URL: https://issues.apache.org/jira/browse/SQOOP-3289
> Project: Sqoop
>  Issue Type: Sub-task
>  Components: build
>Affects Versions: 1.4.7
>Reporter: Daniel Voros
>Assignee: Szabolcs Vasas
>Priority: Minor
> Fix For: 1.5.0, 3.0.0
>
> Attachments: SQOOP-3289.patch
>
>
> Adding a .travis.yml would enable running builds/tests on travis-ci.org. 
> Currently if you wish to use Travis for testing your changes, you have to 
> manually add a .travis.yml to your branch. Having it committed to trunk would 
> save us this extra step.
> I currently have an example 
> [{{.travis.yml}}|https://github.com/dvoros/sqoop/blob/93a4c06c1a3da1fd5305c99e379484507797b3eb/.travis.yml]
>  on my travis branch running unit tests for every commit and every pull 
> request: https://travis-ci.org/dvoros/sqoop/builds
> Later we could add the build status to the project readme as well, see: 
> https://github.com/dvoros/sqoop/tree/travis
> Also, an example of a pull request: https://github.com/dvoros/sqoop/pull/1



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3409) Fix temporary rootdir clean up in Sqoop-S3 tests

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696026#comment-16696026
 ] 

ASF subversion and git services commented on SQOOP-3409:


Commit 264cddbecf6d99a793600885e4c0ded72158695c in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=264cddb ]

SQOOP-3409: Fix temporary rootdir clean up in Sqoop-S3 tests

(Boglarka Egyed via Szabolcs Vasas)


> Fix temporary rootdir clean up in Sqoop-S3 tests
> 
>
> Key: SQOOP-3409
> URL: https://issues.apache.org/jira/browse/SQOOP-3409
> Project: Sqoop
>  Issue Type: Task
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Major
> Attachments: SQOOP-3409.patch
>
>
> Temporary root directory clean up doesn't work as expected, many generated 
> temprootdirs are being kept in the used bucket after test runs. Clean up 
> logic should be checked and fixed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3408) Introduce a Gradle build parameter to set the default forkEvery value for the tests

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696016#comment-16696016
 ] 

ASF subversion and git services commented on SQOOP-3408:


Commit 74252163f0455747ed2a99eb6c2c9d0e48f3dedc in sqoop's branch 
refs/heads/trunk from [~fero]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=7425216 ]

SQOOP-3408 Introduce a Gradle build parameter to set the default forkEvery 
value for the tests

(Szabolcs Vasas via Fero Szabo)


> Introduce a Gradle build parameter to set the default forkEvery value for the 
> tests
> ---
>
> Key: SQOOP-3408
> URL: https://issues.apache.org/jira/browse/SQOOP-3408
> Project: Sqoop
>  Issue Type: Test
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3408.patch
>
>
> The [forkEvery 
> parameter|https://docs.gradle.org/current/dsl/org.gradle.api.tasks.testing.Test.html#org.gradle.api.tasks.testing.Test:forkEvery]
>  of the Gradle test tasks is currently set to 0 which means that all of the 
> tests run in a single JVM (the only exception is the kerberizedTest task 
> which requires a new JVM for every test class).
> The benefit of this setup is that the test tasks finish much faster since the 
> JVM creation is a slow operation. However the Sqoop test framework seems to 
> consume/leak too much memory which can lead to an OutOfMemoryError during the 
> build if there is not enough memory on the machine running the tests.
> The goal of this JIRA is to introduce a new parameter to the Gradle build 
> which can be used to set the default forkEvery parameter and thus prevent the 
> JVM running out of memory.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3405) Refactor: break up Parameterized tests on a per database basis

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695958#comment-16695958
 ] 

ASF subversion and git services commented on SQOOP-3405:


Commit 0a7407613b12a4bb25e737506ef0f091d3a7dae1 in sqoop's branch 
refs/heads/trunk from root
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=0a74076 ]

SQOOP-3405: Refactor: break up Parameterized tests on a per database basis

(Fero Szabo via Szabolcs Vasas)


> Refactor: break up Parameterized tests on a per database basis
> --
>
> Key: SQOOP-3405
> URL: https://issues.apache.org/jira/browse/SQOOP-3405
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Fero Szabo
>Assignee: Fero Szabo
>Priority: Major
>
> Follow the example of the abstract class SavedJobsTestBase and it's 
> subclasses!
> We need this to be able to add test categories (so for Travis integration) as 
> well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3406) Sqoop should not try to execute test category interfaces as tests with Ant

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694732#comment-16694732
 ] 

ASF subversion and git services commented on SQOOP-3406:


Commit 65b51b959eefd1e3eef540f570e048877f5a4026 in sqoop's branch 
refs/heads/trunk from [~fero]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=65b51b9 ]

SQOOP-3406 Sqoop should not try to execute test category interfaces as tests 
with Ant

(Szabolcs Vasas via Fero Szabo)


> Sqoop should not try to execute test category interfaces as tests with Ant
> --
>
> Key: SQOOP-3406
> URL: https://issues.apache.org/jira/browse/SQOOP-3406
> Project: Sqoop
>  Issue Type: Test
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3406.patch, SQOOP-3406.patch
>
>
> When Ant third party test suite is being run Ant tries to execute the test 
> category interfaces too because they end with the 'Test' postfix.
> These "tests" obviously fail so we need to make sure that Ant does not 
> execute them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3407) Introduce methods instead of TEMP_BASE_DIR and LOCAL_WAREHOUSE_DIR static fields

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694644#comment-16694644
 ] 

ASF subversion and git services commented on SQOOP-3407:


Commit 28896c8df1d151da29b3697c08fe5e81da4f3e94 in sqoop's branch 
refs/heads/trunk from [~fero]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=28896c8 ]

SQOOP-3407 Introduce methods instead of TEMP_BASE_DIR and LOCAL_WAREHOUSE_DIR 
static fields

(Szabolcs Vasas via Fero Szabo)


> Introduce methods instead of TEMP_BASE_DIR and LOCAL_WAREHOUSE_DIR static 
> fields
> 
>
> Key: SQOOP-3407
> URL: https://issues.apache.org/jira/browse/SQOOP-3407
> Project: Sqoop
>  Issue Type: Test
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3407.patch
>
>
> BaseSqoopTestCase.TEMP_BASE_DIR and BaseSqoopTestCase.LOCAL_WAREHOUSE_DIR are 
> public static fields which get initialized once at the JVM startup and store 
> the paths for the test temp and warehouse directories.
> The problem is that HBase test cases change the value of the test.build.data 
> system property which can cause tests using these static fields to fail.
> Since we do not own the code in HBase which changes the system property we 
> need to turn these static fields into methods which evaluate the 
> test.build.data system property every time they invoked which will make sure 
> that the invoking tests will be successful.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3404) Categorize all tests in the project

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691478#comment-16691478
 ] 

ASF subversion and git services commented on SQOOP-3404:


Commit bb9c2dd85b0cac84503e69906a05c76d6a0413e1 in sqoop's branch 
refs/heads/trunk from [~fero]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=bb9c2dd ]

SQOOP-3404: Categorize all tests in the project

(Szabolcs Vasas via Fero Szabo)


> Categorize all tests in the project
> ---
>
> Key: SQOOP-3404
> URL: https://issues.apache.org/jira/browse/SQOOP-3404
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: SQOOP-3404.patch
>
>
> SQOOP-3104 has introduced test categories but while it was under review many 
> other patches with new test cases were committed.
> The task is to make sure that all of the new tests are properly categorized 
> and the test tasks are cleaned up.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3104) Create test categories instead of test suites and naming conventions

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686725#comment-16686725
 ] 

ASF subversion and git services commented on SQOOP-3104:


Commit d58e5f106ba932879112d7b69997301e442335f1 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=d58e5f1 ]

SQOOP-3104: Create test categories instead of test suites and naming conventions

(Nguyen Truong via Szabolcs Vasas)


> Create test categories instead of test suites and naming conventions
> 
>
> Key: SQOOP-3104
> URL: https://issues.apache.org/jira/browse/SQOOP-3104
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Anna Szonyi
>Assignee: Nguyen Truong
>Priority: Major
>
> We are currently unsing test naming conventions to differentiate between 
> ManualTests, Unit tests and 3rd party tests.
> Instead of this, we should use junit4 categories, so it's clear by looking at 
> a test where it fits. This would also remove the reliance on the test class 
> name.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3382) Add parquet numeric support for Parquet in hdfs import

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686325#comment-16686325
 ] 

ASF subversion and git services commented on SQOOP-3382:


Commit 00a02dec2f7507f813ee4899096c470ba1112a9e in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=00a02de ]

SQOOP-3382: Add parquet numeric support for Parquet in hdfs import

(Fero Szabo via Szabolcs Vasas)


> Add parquet numeric support for Parquet in hdfs import
> --
>
> Key: SQOOP-3382
> URL: https://issues.apache.org/jira/browse/SQOOP-3382
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Fero Szabo
>Assignee: Fero Szabo
>Priority: Major
> Fix For: 3.0.0
>
>
> The current Avro numeric tests are suitable to be used as Parquet tests, with 
> very minor modifications, as parquet can be written with the same input and 
> nearly the same args. Since we are writing Parquet with it's Avro support, it 
> would be good to cover this code with the same, or similar tests (including 
> the edge cases related to padding, missing scale and precision cases).
> Differences are:
>  * the expected output, since stored in a parquet file is different.
>  * input arguements



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3403) Sqoop2: Add Fero Szabo to committer list in our pom file

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681278#comment-16681278
 ] 

ASF subversion and git services commented on SQOOP-3403:


Commit 8382ffb9ce74e5a7ac37dac010eeebf034a6d5fb in sqoop's branch 
refs/heads/sqoop2 from [~fero]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=8382ffb ]

SQOOP-3403: Sqoop2: Add Fero Szabo to committer list in our pom file

(Fero Szabo)


> Sqoop2: Add Fero Szabo to committer list in our pom file
> 
>
> Key: SQOOP-3403
> URL: https://issues.apache.org/jira/browse/SQOOP-3403
> Project: Sqoop
>  Issue Type: Task
>Affects Versions: 1.99.8
>Reporter: Boglarka Egyed
>Assignee: Fero Szabo
>Priority: Major
>
> Now that [~fero] is committer we should update our committer list in the root 
> pom.xml file:



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-2949) SQL Syntax error when split-by column is of character type and min or max value has single quote inside it

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670288#comment-16670288
 ] 

ASF subversion and git services commented on SQOOP-2949:


Commit 6dd6a4fc863074d570919b183fdf5c20e86c5e0b in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=6dd6a4f ]

SQOOP-2949: SQL Syntax error when split-by column is of character type and min 
or max value has single quote inside it

(Fero Szabo via Szabolcs Vasas)


> SQL Syntax error when split-by column is of character type and min or max 
> value has single quote inside it
> --
>
> Key: SQOOP-2949
> URL: https://issues.apache.org/jira/browse/SQOOP-2949
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.4.6
> Environment: Sqoop 1.4.6
> Run on Hadoop 2.6.0
> On Ubuntu
>Reporter: Gireesh Puthumana
>Assignee: Gireesh Puthumana
>Priority: Major
>
> Did a sqoop import from mysql table "emp", with split-by column "ename", 
> which is a varchar(100) type.
> +Used below command:+
> sqoop import --connect jdbc:mysql://localhost/testdb --username root 
> --password * --table emp --m 2 --target-dir /sqoopTest/5 --split-by ename;
> +Ename has following records:+
> | ename   |
> | gireesh |
> | aavesh  |
> | shiva'  |
> | jamir   |
> | balu|
> | santosh |
> | sameer  |
> Min value is "aavesh" and max value is "shiva'" (please note the single quote 
> inside max value).
> When run, it tried to execute below query in mapper 2 and failed:
> SELECT `ename`, `eid`, `deptid` FROM `emp` AS `emp` WHERE ( `ename` >= 
> 'jd聯聭聪G耀' ) AND ( `ename` <= 'shiva'' )
> +Stack trace:+
> {quote}
> 2016-06-05 16:54:06,749 ERROR [main] 
> org.apache.sqoop.mapreduce.db.DBRecordReader: Top level exception: 
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error 
> in your SQL syntax; check the manual that corresponds to your MySQL server 
> version for the right syntax to use near ''shiva'' )' at line 1
>   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>   at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>   at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
>   at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
>   at com.mysql.jdbc.Util.getInstance(Util.java:387)
>   at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:942)
>   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3966)
>   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3902)
>   at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2526)
>   at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673)
>   at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2549)
>   at 
> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861)
>   at 
> com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1962)
>   at 
> org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
>   at 
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
>   at 
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
>   at 
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>   at 
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
>   at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>   at 
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
>   at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>   at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>   at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> {quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3395) Document Hadoop CredentialProvider usage in case of import into S3

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16663799#comment-16663799
 ] 

ASF subversion and git services commented on SQOOP-3395:


Commit c2211d6118197f2ee2800e0e6cd6c0a8879f0d0a in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=c2211d6 ]

SQOOP-3395: Document Hadoop CredentialProvider usage in case of import into S3

(Boglarka Egyed via Szabolcs Vasas)


> Document Hadoop CredentialProvider usage in case of import into S3
> --
>
> Key: SQOOP-3395
> URL: https://issues.apache.org/jira/browse/SQOOP-3395
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: SQOOP-3395.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3399) TestS3ImportWithHadoopCredProvider fails if credential generator command is not provided

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16663331#comment-16663331
 ] 

ASF subversion and git services commented on SQOOP-3399:


Commit 7f61ae21e39a45b2b5e0107c4165bc4a255a27eb in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=7f61ae2 ]

SQOOP-3399: TestS3ImportWithHadoopCredProvider fails if credential generator 
command is not provided

(Szabolcs Vasas)


> TestS3ImportWithHadoopCredProvider fails if credential generator command is 
> not provided
> 
>
> Key: SQOOP-3399
> URL: https://issues.apache.org/jira/browse/SQOOP-3399
> Project: Sqoop
>  Issue Type: Test
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3399.patch
>
>
> BeforeClass method of TestS3ImportWithHadoopCredProvider should not throw 
> NullPointerException when the credential generator command is not provided 
> since it fails the test with Gradle.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-1905) add --schema option for import-all-tables and list-tables against db2

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-1905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16663327#comment-16663327
 ] 

ASF subversion and git services commented on SQOOP-1905:


Commit 7ab99f41cf75da22e523c132cb782bb20102b348 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=7ab99f4 ]

SQOOP-3355: Document SQOOP-1905 DB2 --schema option

(Fero Szabo via Szabolcs Vasas)


> add --schema option for import-all-tables and list-tables against db2
> -
>
> Key: SQOOP-1905
> URL: https://issues.apache.org/jira/browse/SQOOP-1905
> Project: Sqoop
>  Issue Type: Improvement
>  Components: connectors
>Affects Versions: 1.4.5
> Environment: RedHat6.4+Sqoop1.4.5+Hadoop2.4.1+Hive0.13.0+DB2 10.5
>Reporter: xieshiju
>Assignee: Ying Cao
>Priority: Major
>  Labels: features
> Fix For: 1.4.7
>
> Attachments: SQOOP-1905.patch, SQOOP-1905.patch, SQOOP-1905.patch
>
>   Original Estimate: 504h
>  Remaining Estimate: 504h
>
> For Sqoop import-all-tables, and list-tables function, we can use --schema 
> option to qualify the table name.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3355) Document SQOOP-1905 DB2 --schema option

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16663326#comment-16663326
 ] 

ASF subversion and git services commented on SQOOP-3355:


Commit 7ab99f41cf75da22e523c132cb782bb20102b348 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=7ab99f4 ]

SQOOP-3355: Document SQOOP-1905 DB2 --schema option

(Fero Szabo via Szabolcs Vasas)


> Document SQOOP-1905 DB2 --schema option
> ---
>
> Key: SQOOP-3355
> URL: https://issues.apache.org/jira/browse/SQOOP-3355
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Fero Szabo
>Assignee: Fero Szabo
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3398) Tests using HiveMiniCluster can be unstable on some platforms

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662410#comment-16662410
 ] 

ASF subversion and git services commented on SQOOP-3398:


Commit f6455c86486e48dacd67885fb2b1d29aaed32c67 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=f6455c8 ]

SQOOP-3398: Tests using HiveMiniCluster can be unstable on some platforms

(Szabolcs Vasas via Boglarka Egyed)


> Tests using HiveMiniCluster can be unstable on some platforms
> -
>
> Key: SQOOP-3398
> URL: https://issues.apache.org/jira/browse/SQOOP-3398
> Project: Sqoop
>  Issue Type: Test
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3398.patch
>
>
> Since the last Hive upgrade TestHiveMiniCluster fails on some platforms 
> because and older version of the ASM library is picked up.
> The task is to exclude the older ASM library in ivy and gradle to make sure 
> the test passes on all platforms.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3390) Document S3Guard usage with Sqoop

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662380#comment-16662380
 ] 

ASF subversion and git services commented on SQOOP-3390:


Commit 9e328a53e1740ca1ed85861311281f8ea5846ecf in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=9e328a5 ]

SQOOP-3390: Document S3Guard usage with Sqoop

(Boglarka Egyed via Szabolcs Vasas)


> Document S3Guard usage with Sqoop
> -
>
> Key: SQOOP-3390
> URL: https://issues.apache.org/jira/browse/SQOOP-3390
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Major
> Attachments: SQOOP-3390.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3394) External Hive table tests should use unique external dir names

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656495#comment-16656495
 ] 

ASF subversion and git services commented on SQOOP-3394:


Commit 15097756c7fc694ec86d003843f40a793bcd9633 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=1509775 ]

SQOOP-3394: External Hive table tests should use unique external dir names

(Boglarka Egyed via Szabolcs Vasas)


> External Hive table tests should use unique external dir names
> --
>
> Key: SQOOP-3394
> URL: https://issues.apache.org/jira/browse/SQOOP-3394
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Major
> Attachments: SQOOP-3394.patch, SQOOP-3394.patch
>
>
> Current external Hive table tests on S3 uses the same external directory name 
> in every unit test cases which can cause problems during running them in an 
> automated environment. These names should be unique in every test cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3391) Test storing AWS credentials in Hadoop CredentialProvider during import

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16655267#comment-16655267
 ] 

ASF subversion and git services commented on SQOOP-3391:


Commit 1f7b0bf841f9d1cbafb58062d0bb0b6f19312874 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=1f7b0bf ]

SQOOP-3391: Test storing AWS credentials in Hadoop CredentialProvider during 
import

(Boglarka Egyed)


> Test storing AWS credentials in Hadoop CredentialProvider during import
> ---
>
> Key: SQOOP-3391
> URL: https://issues.apache.org/jira/browse/SQOOP-3391
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Major
> Attachments: SQOOP-3391.patch, SQOOP-3391.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3381) Upgrade the Parquet library from 1.6.0 to 1.9.0

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653172#comment-16653172
 ] 

ASF subversion and git services commented on SQOOP-3381:


Commit 5dd8c8aad1c7732754fae190eb5424371ed6fef4 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=5dd8c8a ]

SQOOP-3381: Upgrade the Parquet library from 1.6.0 to 1.9.0

(Fero Szabo via Szabolcs Vasas)


> Upgrade the Parquet library from 1.6.0 to 1.9.0
> ---
>
> Key: SQOOP-3381
> URL: https://issues.apache.org/jira/browse/SQOOP-3381
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Fero Szabo
>Assignee: Fero Szabo
>Priority: Major
> Fix For: 3.0.0
>
>
> As we will need to register a data supplier in the fix for parquet decimal 
> support, we will need a version that contains PARQUET-243.
> We need to upgrade the Parquet library to a version that contains this fix 
> and is compatible with Hadoop. Most probably, the newest version will be 
> adequate. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3384) Document import into external Hive table backed by S3

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650210#comment-16650210
 ] 

ASF subversion and git services commented on SQOOP-3384:


Commit c329f360dd08ef3b9bd82897fcd611e7431d32c8 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=c329f36 ]

SQOOP-3384: Document import into external Hive table backed by S3

(Boglarka Egyed via Szabolcs Vasas)


> Document import into external Hive table backed by S3
> -
>
> Key: SQOOP-3384
> URL: https://issues.apache.org/jira/browse/SQOOP-3384
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Major
> Attachments: SQOOP-3384.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3383) Disable FileSystem static cache in S3 tests

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611686#comment-16611686
 ] 

ASF subversion and git services commented on SQOOP-3383:


Commit 932822aa8fdddeb0aff6445d4f585a599ccb0084 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=932822a ]

SQOOP-3383: Disable FileSystem static cache in S3 tests

(Boglarka Egyed via Szabolcs Vasas)


> Disable FileSystem static cache in S3 tests
> ---
>
> Key: SQOOP-3383
> URL: https://issues.apache.org/jira/browse/SQOOP-3383
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Minor
> Attachments: SQOOP-3383.patch, SQOOP-3383.patch
>
>
> FileSystem has a static cache meaning when the authentication happens in the 
> org.apache.sqoop.testutil.S3TestUtils#setS3CredentialsInHadoopConf method 
> Sqoop import will get the same FileSystem object from the cache thus its 
> authentication via the \{{-Dfs.s3a.access.key}} and -{{Dfs.s3a.secret.key}} 
> properties is effectless. See 
> org.apache.hadoop.fs.FileSystem#get(java.net.URI, 
> org.apache.hadoop.conf.Configuration). 
> This static cache should be disabled (by settin {{fs.s3a.impl.disable.cache}} 
> to true) in the setup phase of the S3 tests to make sure Sqoop relies on the 
> S3 credentials set via the -D properties.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3380) parquet-configurator-implementation is not recognized as an option

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611481#comment-16611481
 ] 

ASF subversion and git services commented on SQOOP-3380:


Commit b37e0c6053b92f2d7a56026e7fde5637b7f70588 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=b37e0c6 ]

SQOOP-3380: parquet-configurator-implementation is not recognized as an option

(Szabolcs Vasas)


> parquet-configurator-implementation is not recognized as an option
> --
>
> Key: SQOOP-3380
> URL: https://issues.apache.org/jira/browse/SQOOP-3380
> Project: Sqoop
>  Issue Type: Bug
>Reporter: Fero Szabo
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3380.patch
>
>
> The parquet-configurator-implementation option was added to Sqoop with 
> SQOOP-3329: Remove Kite dependency from the Sqoop project, but the command 
> line parser doesn't recognize it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3375) HiveMiniCluster does not restore hive-site.xml location

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16601956#comment-16601956
 ] 

ASF subversion and git services commented on SQOOP-3375:


Commit c814e58348308b05b215db427412cd6c0b21333e in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=c814e58 ]

SQOOP-3375: HiveMiniCluster does not restore hive-site.xml location

(Szabolcs Vasas via Boglarka Egyed)


> HiveMiniCluster does not restore hive-site.xml location
> ---
>
> Key: SQOOP-3375
> URL: https://issues.apache.org/jira/browse/SQOOP-3375
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3375.patch
>
>
> HiveMiniCluster sets the hive-site.xml location using 
> org.apache.hadoop.hive.conf.HiveConf#setHiveSiteLocation static method during 
> startup but it does not restore the original location during shutdown.
> This makes HCatalogImportTest and HCatalogExportTest fail if they are ran in 
> the same JVM after any test using HiveMiniCluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3373) Document simple and incremental import into S3

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598478#comment-16598478
 ] 

ASF subversion and git services commented on SQOOP-3373:


Commit 35556651e2eabc768f765ffd0db3379727c78276 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=3555665 ]

SQOOP-3373: Document simple and incremental import into S3

(Boglarka Egyed)


> Document simple and incremental import into S3
> --
>
> Key: SQOOP-3373
> URL: https://issues.apache.org/jira/browse/SQOOP-3373
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Major
> Attachments: SQOOP-3373.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3368) Add fail-fast scenarios to S3 incremental import use cases without --temporary-rootdir option

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598369#comment-16598369
 ] 

ASF subversion and git services commented on SQOOP-3368:


Commit 6fa45a95a41f80778338cd2a2bb13418b2c37376 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=6fa45a9 ]

SQOOP-3368: Add fail-fast scenarios to S3 incremental import use cases without 
--temporary-rootdir option

(Boglarka Egyed)


> Add fail-fast scenarios to S3 incremental import use cases without 
> --temporary-rootdir option
> -
>
> Key: SQOOP-3368
> URL: https://issues.apache.org/jira/browse/SQOOP-3368
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Major
> Attachments: SQOOP-3368.patch, SQOOP-3368.patch, SQOOP-3368.patch
>
>
> The current implementation of Sqoop handles HDFS as a default filesystem, 
> i.e. it creates temporary directories on HDFS in case of incremental append 
> or merge imports. To make these incremental import use cases work with S3 the 
> user needs to set the {{--temporary-rootdir}} to an S3 location properly.
> There should be fail-fast scenarios without the {{--temporary-rootdir}} 
> option.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3367) Improve third party tests to be able to execute them in a single JVM

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594974#comment-16594974
 ] 

ASF subversion and git services commented on SQOOP-3367:


Commit 327aec8bf1b7a40936503a9c5a94b759caf16e11 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=327aec8 ]

SQOOP-3367: Improve third party tests to be able to execute them in a single JVM

(Szabolcs Vasas via Boglarka Egyed)


> Improve third party tests to be able to execute them in a single JVM
> 
>
> Key: SQOOP-3367
> URL: https://issues.apache.org/jira/browse/SQOOP-3367
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3367.patch
>
>
> The goal of this JIRA is to improve the third party tests to be able to 
> execute them in a single JVM. See the parent JIRA for the details.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3363) Test incremental import with S3

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594899#comment-16594899
 ] 

ASF subversion and git services commented on SQOOP-3363:


Commit 816146df567cf46a972d16873c46595f778b136c in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=816146d ]

SQOOP-3363: Test incremental import with S3

(Boglarka Egyed via Szabolcs Vasas)


> Test incremental import with S3
> ---
>
> Key: SQOOP-3363
> URL: https://issues.apache.org/jira/browse/SQOOP-3363
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Major
> Attachments: SQOOP-3363.patch, SQOOP-3363.patch, SQOOP-3363.patch, 
> SQOOP-3363.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3371) Fix tests using HiveMiniCluster

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593442#comment-16593442
 ] 

ASF subversion and git services commented on SQOOP-3371:


Commit 1f73341070c56bda9c22820b2fa1cb7449239d32 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=1f73341 ]

SQOOP-3371: Fix tests using HiveMiniCluster

(Szabolcs Vasas via Boglarka Egyed)


> Fix tests using HiveMiniCluster
> ---
>
> Key: SQOOP-3371
> URL: https://issues.apache.org/jira/browse/SQOOP-3371
> Project: Sqoop
>  Issue Type: Test
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3371.patch
>
>
> It seems that SQOOP-3360 broke our tests which use HiveMiniCluster because 
> org.apache.calcite is not present in the hive-exec:core JAR but this 
> dependency seems to be needed by these tests.
> I am not sure why our Jenkins job did not catch the issue earlier but I get 
> consistent failures when I run these tests with a clean ivy cache and ant so 
> the dependency issue needs to be fixed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3366) Improve unit tests to be able to execute them in a single JVM

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591449#comment-16591449
 ] 

ASF subversion and git services commented on SQOOP-3366:


Commit f0215808447f40bcddd50b42f48a765d55094273 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=f021580 ]

SQOOP-3366: Improve unit tests to be able to execute them in a single JVM

(Nguyen Truong via Szabolcs Vasas)


> Improve unit tests to be able to execute them in a single JVM
> -
>
> Key: SQOOP-3366
> URL: https://issues.apache.org/jira/browse/SQOOP-3366
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Nguyen Truong
>Priority: Major
> Attachments: SQOOP-3366.patch
>
>
> The goal of this JIRA is to improve the unit tests to be able to execute them 
> in a single JVM. See the parent JIRA for the details.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3224) Mainframe FTP transfer should have an option to use binary mode for transfer

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590359#comment-16590359
 ] 

ASF subversion and git services commented on SQOOP-3224:


Commit 0d6c455e5bbfc092d4a90f352eb262f347758132 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=0d6c455 ]

SQOOP-3224: Mainframe FTP transfer should have an option to use binary mode for 
transfer

(Chris Teoh via Szabolcs Vasas)


> Mainframe FTP transfer should have an option to use binary mode for transfer
> 
>
> Key: SQOOP-3224
> URL: https://issues.apache.org/jira/browse/SQOOP-3224
> Project: Sqoop
>  Issue Type: Improvement
>Reporter: Chris Teoh
>Assignee: Chris Teoh
>Priority: Minor
>
> Currently the mainframe FTP module is hard coded to use ascii transfer mode. 
> Propose a mainframe module flag to be able to change modes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3364) Upgrade Gradle version to 4.9

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16588920#comment-16588920
 ] 

ASF subversion and git services commented on SQOOP-3364:


Commit f5eda1e208fba9058c250dd1b142a97eb118ced9 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=f5eda1e ]

SQOOP-3364: Upgrade Gradle version to 4.9

(Szabolcs Vasas)


> Upgrade Gradle version to 4.9
> -
>
> Key: SQOOP-3364
> URL: https://issues.apache.org/jira/browse/SQOOP-3364
> Project: Sqoop
>  Issue Type: Improvement
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3364.patch
>
>
> Sqoop uses Gradle 3.5.1 currently which is a pretty old version, let's 
> upgrade it to the newest 4.9 version.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3362) Fix toString() methods of OraOopOracleDataChunk

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582306#comment-16582306
 ] 

ASF subversion and git services commented on SQOOP-3362:


Commit 005e5d6af82679b64a9683366aa5ae67e216f18d in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=005e5d6 ]

SQOOP-3362: Fix toString() methods of OraOopOracleDataChunk

(Nguyen Truong via Szabolcs Vasas)


> Fix toString() methods of OraOopOracleDataChunk
> ---
>
> Key: SQOOP-3362
> URL: https://issues.apache.org/jira/browse/SQOOP-3362
> Project: Sqoop
>  Issue Type: Bug
>Reporter: Nguyen Truong
>Assignee: Nguyen Truong
>Priority: Major
>
> The method is currently returning the hash of data chunk object. It would be 
> nice to show the information of the class's variables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3359) Add LOG message for git hash

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576217#comment-16576217
 ] 

ASF subversion and git services commented on SQOOP-3359:


Commit 96593b1a9b8168954f1d5b13f802f2f3ee5beab8 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=96593b1 ]

SQOOP-3359: Add LOG message for git hash

(Nguyen Truong via Szabolcs Vasas)


> Add LOG message for git hash
> 
>
> Key: SQOOP-3359
> URL: https://issues.apache.org/jira/browse/SQOOP-3359
> Project: Sqoop
>  Issue Type: Improvement
>Reporter: Nguyen Truong
>Assignee: Nguyen Truong
>Priority: Minor
> Attachments: SQOOP-3359.patch
>
>
> Besides the version, it would be nice to also know the git hash of the 
> running Sqoop.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3348) Add hadoop-aws dependency and S3 credential generator logic for tests

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576211#comment-16576211
 ] 

ASF subversion and git services commented on SQOOP-3348:


Commit 282d41ee376d67c229dc8e4d775432ba732d9b5f in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=282d41e ]

SQOOP-3348: Add hadoop-aws dependency and S3 credential generator logic for 
tests

(Boglarka Egyed via Szabolcs Vasas)


> Add hadoop-aws dependency and S3 credential generator logic for tests
> -
>
> Key: SQOOP-3348
> URL: https://issues.apache.org/jira/browse/SQOOP-3348
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Major
> Attachments: SQOOP-3348.patch
>
>
> Task is to add {{hadoop-aws}} dependency to Sqoop and implement an S3 
> credential generator logic to enable automated testing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3360) Fix hive-exec dependency issues in Gradle

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575936#comment-16575936
 ] 

ASF subversion and git services commented on SQOOP-3360:


Commit ec18e6f47ebe4308495052afb4a2fd6f9ad0102d in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=ec18e6f ]

SQOOP-3360: Fix hive-exec dependency issues in Gradle

(Szabolcs Vasas via Boglarka Egyed)


> Fix hive-exec dependency issues in Gradle
> -
>
> Key: SQOOP-3360
> URL: https://issues.apache.org/jira/browse/SQOOP-3360
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3360.patch
>
>
> It turned out during making SQOOP-3345 work with Gradle that there are issues 
> with the dependency handling which cause problems with Gradle. Root cause of 
> the problem seems to be the hive-exec dependency which also contains its own 
> dependencies pulling in incorrect libs and versions runtime. This has to be 
> cleaned up via better dependency management.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3357) Change MainframeImportTool to refer to MainframeManager class directly

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574987#comment-16574987
 ] 

ASF subversion and git services commented on SQOOP-3357:


Commit c3d916ab20ca02c9d6b85bb51d6c31eb31aaff88 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=c3d916a ]

SQOOP-3357: Change MainframeImportTool to refer to MainframeManager class 
directly

(Nguyen Truong via Szabolcs Vasas)


> Change MainframeImportTool to refer to MainframeManager class directly
> --
>
> Key: SQOOP-3357
> URL: https://issues.apache.org/jira/browse/SQOOP-3357
> Project: Sqoop
>  Issue Type: Improvement
>Affects Versions: 1.4.7
>Reporter: Nguyen Truong
>Assignee: Nguyen Truong
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: SQOOP-3357.patch
>
>
> Currently MainframeImportTool refers to the MainframeManager with a string 
> which can create a problem in refactoring. It would be beneficial to change 
> the string to a MainframeManager class reference.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3349) Remove Kite dependency from the Gradle dependencies

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559831#comment-16559831
 ] 

ASF subversion and git services commented on SQOOP-3349:


Commit b56b1767d2615abba03f07bab28a9161e58bb6f6 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=b56b176 ]

SQOOP-3349: Remove Kite dependency from the Gradle dependencies

(Szabolcs Vasas via Boglarka Egyed)


> Remove Kite dependency from the Gradle dependencies
> ---
>
> Key: SQOOP-3349
> URL: https://issues.apache.org/jira/browse/SQOOP-3349
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3349.patch
>
>
> Since Sqoop can be built using Gradle as well we need to make sure that the 
> Kite dependency is removed from sqoop-package.gradle too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3350) Fix tests which use warehouse-dir as target-dir

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559481#comment-16559481
 ] 

ASF subversion and git services commented on SQOOP-3350:


Commit 1213b7725a65e561d8db071141e04944c85d1f94 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=1213b77 ]

SQOOP-3350: Fix tests which use warehouse-dir as target-dir

(Szabolcs Vasas via Boglarka Egyed)


> Fix tests which use warehouse-dir as target-dir
> ---
>
> Key: SQOOP-3350
> URL: https://issues.apache.org/jira/browse/SQOOP-3350
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3350.patch
>
>
> Some Sqoop tests use the value of the 
> org.apache.sqoop.testutil.BaseSqoopTestCase#getWarehouseDir as target-dir:
> {code:java}
> args.add("--target-dir");
> args.add(getWarehouseDir());
> {code}
> This leads to an error when the warehouse directory exists which can happen 
> if a previous test does not clean up properly.
> The issue was found when executing the tests with Gradle since it probably 
> executes the tests in a different order.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3353) Sqoop should not check incremental constraints for HBase imports

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559436#comment-16559436
 ] 

ASF subversion and git services commented on SQOOP-3353:


Commit a06f2f3a8305b7a58b35e27f1fa6aea820dda5aa in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=a06f2f3 ]

SQOOP-3353: Sqoop should not check incremental constraints for HBase imports

(Szabolcs Vasas via Boglarka Egyed)


> Sqoop should not check incremental constraints for HBase imports
> 
>
> Key: SQOOP-3353
> URL: https://issues.apache.org/jira/browse/SQOOP-3353
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3353.patch
>
>
> ImportTool#initIncrementalConstraints method is invoked for every import in 
> Sqoop and it can throw the following error even for HBase imports:
> {code:java}
> --merge-key or --append is required when using --incremental lastmodified and 
> the output directory exists.{code}
> The task is to fix the validation not to throw an exception when importing 
> into an HBase table since in that case it does not matter if the table 
> directory exists on HDFS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3352) Bump java target version to 1.8

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555804#comment-16555804
 ] 

ASF subversion and git services commented on SQOOP-3352:


Commit eefb7a0f3e28956fc51346b91f8a9337c5be9aca in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=eefb7a0 ]

SQOOP-3352: Bump java target version to 1.8

(Boglarka Egyed via Szabolcs Vasas)


> Bump java target version to 1.8
> ---
>
> Key: SQOOP-3352
> URL: https://issues.apache.org/jira/browse/SQOOP-3352
> Project: Sqoop
>  Issue Type: Improvement
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: SQOOP-3352.patch, SQOOP-3352.patch, SQOOP-3352.patch
>
>
> We should move to Java 8 as a minimum requirement.
> Java 7 is EOL'ed for more than 3 years now: 
> [http://www.oracle.com/technetwork/java/eol-135779.html]
> Many Apache projects are adopting Java 8 as a minimum requirement, for 
> instance:
>  * Hadoop 3: HADOOP-11858
>  * Hbase 2: HBASE-15624
>  * Flume 1.8: FLUME-2945
> This change affects the Ant and the Gradle build too.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3052) Introduce Gradle based build for Sqoop to make it more developer friendly / open

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552731#comment-16552731
 ] 

ASF subversion and git services commented on SQOOP-3052:


Commit b148d54000da8d718f95917f8bf805860a6cbfe6 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=b148d54 ]

SQOOP-3052: Introduce Gradle based build for Sqoop to make it more developer 
friendly / open

(Anna Szonyi via Szabolcs Vasas)


> Introduce Gradle based build for Sqoop to make it more developer friendly / 
> open
> 
>
> Key: SQOOP-3052
> URL: https://issues.apache.org/jira/browse/SQOOP-3052
> Project: Sqoop
>  Issue Type: Improvement
>Reporter: Attila Szabo
>Assignee: Anna Szonyi
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: SQOOP-3052.patch
>
>
> The current trunk version can only be build with Ant/Ivy combination, which 
> has some painful limitations (resolve is slow / needs to be tweaked to use 
> only caches, the current profile / variable based settings are not working in 
> IDEs out of the box, the current solution does not download the related 
> sources, etc.)
> It would be nice to provide a solution, which would give the possibility for 
> the developers to choose between the nowadays well used build infrsturctures 
> (e.g. Maven, Gradle, etc.). For this solution it would be also essential to 
> keep the different build files (if there is more then one) synchronized 
> easily, and the configuration wouldn't diverege by time. Test execution has 
> to be solved also, and should cover all the available test cases.
> In this scenario:
> If we can provide one good working solution is much better, then provide 
> three different ones which become out of sync easily. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3347) Make verify() more generic in AvroTestUtils

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550852#comment-16550852
 ] 

ASF subversion and git services commented on SQOOP-3347:


Commit 6c6963abe8f0513a19bf61a8c9055563ff245b1c in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=6c6963a ]

SQOOP-3347: Make verify() more generic in AvroTestUtils

(Boglarka Egyed via Szabolcs Vasas)


> Make verify() more generic in AvroTestUtils
> ---
>
> Key: SQOOP-3347
> URL: https://issues.apache.org/jira/browse/SQOOP-3347
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Major
> Attachments: SQOOP-3347.patch
>
>
> AvroTestUtils is a util class for Avro related tests however its verify() 
> method contains a decimal conversion logic which should be extracted and used 
> only in relevant test cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3346) Upgrade Hadoop version to 2.8.0

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550532#comment-16550532
 ] 

ASF subversion and git services commented on SQOOP-3346:


Commit b3e941be050d615f6aae5b4fafb9e0754c8c6a72 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=b3e941b ]

SQOOP-3346: Upgrade Hadoop version to 2.8.0

(Boglarka Egyed via Szabolcs Vasas)


> Upgrade Hadoop version to 2.8.0
> ---
>
> Key: SQOOP-3346
> URL: https://issues.apache.org/jira/browse/SQOOP-3346
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Major
> Attachments: SQOOP-3346.patch
>
>
> Support for AWS temporary credentials has been introduced in Hadoop 2.8.0 
> based on HADOOP-12537 and it would make more sense to test and support this 
> capability too with Sqoop.
> There is [SQOOP-3305|https://reviews.apache.org/r/66300/bugs/SQOOP-3305/] 
> being open for upgrading Hadoop to 3.0.0 however it has several issues 
> described in [https://reviews.apache.org/r/66300/] currently thus I would 
> like to proceed with an "intermediate" upgrade to 2.8.0 to enable development 
> on S3 front. [~dvoros] are you OK with this?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3329) Remove Kite dependency from the Sqoop project

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550345#comment-16550345
 ] 

ASF subversion and git services commented on SQOOP-3329:


Commit 739bbce48593a82575435f1cc48ca7ebd48537c9 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=739bbce ]

SQOOP-3329: Remove Kite dependency from the Sqoop project

(Szabolcs Vasas via Boglarka Egyed)


> Remove Kite dependency from the Sqoop project
> -
>
> Key: SQOOP-3329
> URL: https://issues.apache.org/jira/browse/SQOOP-3329
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3329.patch
>
>
> Now that we have an alternative solution for reading and writing Parquet 
> files we can remove the Kite dependency and the classes using Kite from the 
> Sqoop project.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3338) Document Parquet support

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545256#comment-16545256
 ] 

ASF subversion and git services commented on SQOOP-3338:


Commit 17461e91db01bf67663caf0fb35e8920128c1aba in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=17461e9 ]

SQOOP-3338: Document Parquet support

(Szabolcs Vasas via Boglarka Egyed)


> Document Parquet support
> 
>
> Key: SQOOP-3338
> URL: https://issues.apache.org/jira/browse/SQOOP-3338
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3338.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3335) Add Hive support to the new Parquet writing implementation

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545098#comment-16545098
 ] 

ASF subversion and git services commented on SQOOP-3335:


Commit e639053251b65f943481666a62ed137bfef15b76 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=e639053 ]

SQOOP-3335: Add Hive support to the new Parquet writing implementation

(Szabolcs Vasas via Boglarka Egyed)


> Add Hive support to the new Parquet writing implementation
> --
>
> Key: SQOOP-3335
> URL: https://issues.apache.org/jira/browse/SQOOP-3335
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.7
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: SQOOP-3335.patch
>
>
> SQOOP-3328 adds a new Parquet reading and writing implementation to Sqoop it 
> does not add support to Hive Parquet imports. The task of this Jira is to add 
> this missing functionality.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3042) Sqoop does not clear compile directory under /tmp/sqoop-/compile automatically

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541770#comment-16541770
 ] 

ASF subversion and git services commented on SQOOP-3042:


Commit 0cfbf56713f7574568ea3754f6854e82f5540954 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=0cfbf56 ]

SQOOP-3042: Sqoop does not clear compile directory under 
/tmp/sqoop-/compile automatically

(Eric Lin via Szabolcs Vasas)


> Sqoop does not clear compile directory under /tmp/sqoop-/compile 
> automatically
> 
>
> Key: SQOOP-3042
> URL: https://issues.apache.org/jira/browse/SQOOP-3042
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Critical
>  Labels: patch
> Attachments: SQOOP-3042.1.patch, SQOOP-3042.2.patch, 
> SQOOP-3042.4.patch, SQOOP-3042.5.patch, SQOOP-3042.6.patch, 
> SQOOP-3042.7.patch, SQOOP-3042.9.patch
>
>
> After running sqoop, all the temp files generated by ClassWriter are left 
> behind on disk, so anyone can check those JAVA files to see the schema of 
> those tables that Sqoop has been interacting with. By default, the directory 
> is under /tmp/sqoop-/compile.
> In class org.apache.sqoop.SqoopOptions, function getNonceJarDir(), I can see 
> that we did add "deleteOnExit" on the temp dir:
> {code}
> for (int attempts = 0; attempts < MAX_DIR_CREATE_ATTEMPTS; attempts++) {
>   hashDir = new File(baseDir, RandomHash.generateMD5String());
>   while (hashDir.exists()) {
> hashDir = new File(baseDir, RandomHash.generateMD5String());
>   }
>   if (hashDir.mkdirs()) {
> // We created the directory. Use it.
> // If this directory is not actually filled with files, delete it
> // when the JVM quits.
> hashDir.deleteOnExit();
> break;
>   }
> }
> {code}
> However, I believe it failed to delete due to directory is not empty.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3331) Add Mainframe FTP integration test for GDG dataset.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16513911#comment-16513911
 ] 

ASF subversion and git services commented on SQOOP-3331:


Commit b2643d5094da405acfc695d1da22a54d32aa74ed in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=b2643d5 ]

SQOOP-3331: Add Mainframe FTP integration test for GDG dataset.

(Chris Teoh via Szabolcs Vasas)


> Add Mainframe FTP integration test for GDG dataset.
> ---
>
> Key: SQOOP-3331
> URL: https://issues.apache.org/jira/browse/SQOOP-3331
> Project: Sqoop
>  Issue Type: Test
>Reporter: Chris Teoh
>Assignee: Chris Teoh
>Priority: Minor
>
> Current mainframe import functionality doesn't have integration tests, only 
> unit tests. Adding this test improves validation of this functionality.
> Dockerfile and accompanying contents at 
> https://github.com/christeoh/zos-ftpmock/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3319) Extract code using Kite into separate classes

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493230#comment-16493230
 ] 

ASF subversion and git services commented on SQOOP-3319:


Commit 3233db8e1c481e38c538f4caaf55bcbc0c11f208 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=3233db8 ]

SQOOP-3319: Extract code using Kite into separate classes

(Szabolcs Vasas via Boglarka Egyed)


> Extract code using Kite into separate classes
> -
>
> Key: SQOOP-3319
> URL: https://issues.apache.org/jira/browse/SQOOP-3319
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3319.patch, SQOOP-3319.patch
>
>
> Kite Dataset API is used in many places in the code to read/write Parquet 
> files and configure MR jobs.
> The goal of this JIRA is to introduce an implementation agnostic interface 
> for Parquet reading/writing and extract the code using Kite Dataset API into 
> separate classes implementing this interface. The benefit of this refactoring 
> is that it enables us introducing a new Parquet reading/writing 
> implementation which does not use Kite but plugs in easily.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3082) Sqoop import fails after TCP connection reset if split by datetime column

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480406#comment-16480406
 ] 

ASF subversion and git services commented on SQOOP-3082:


Commit ad7d046ef1d68b46aacdf6e124056f5901e95e5c in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=ad7d046 ]

SQOOP-3082: Sqoop import fails after TCP connection reset if split by datetime 
column

(Sergey Svynarchuk via Szabolcs Vasas)


> Sqoop import fails after TCP connection reset if split by datetime column
> -
>
> Key: SQOOP-3082
> URL: https://issues.apache.org/jira/browse/SQOOP-3082
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Sergey Svynarchuk
>Priority: Major
> Attachments: SQOOP-3082-1.patch, SQOOP-3082.patch
>
>
> If sqoop-to-mssqlserver connection reset, the whole command fails with 
> "Connection reset with com.microsoft.sqlserver.jdbc.SQLServerException: 
> Incorrect syntax near '00'" . On reestablishing connection, Sqoop tries to 
> resume import from the last record that was successfully read by :
> {code}
> 2016-12-10 15:18:54,523 INFO [main] 
> org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: select * from 
> test.dbo.test1 WITH (nolock) where Date >= '2015-01-10' and Date <= 
> '2016-11-24' and ( Date > 2015-09-18 00:00:00.0 ) AND ( Date < '2015-09-23 
> 11:48:00.0' ) 
> {code}
> Not quoted 2015-09-18 00:00:00.0 in SQL.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3324) Document SQOOP-816: Sqoop add support for external Hive tables

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479116#comment-16479116
 ] 

ASF subversion and git services commented on SQOOP-3324:


Commit 6c0b3201280f3c7c4c56530909fbb2977e58542e in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=6c0b320 ]

SQOOP-3324: Document SQOOP-816: Sqoop add support for external Hive tables

(Fero Szabo via Szabolcs Vasas)


> Document SQOOP-816: Sqoop add support for external Hive tables
> --
>
> Key: SQOOP-3324
> URL: https://issues.apache.org/jira/browse/SQOOP-3324
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Fero Szabo
>Assignee: Fero Szabo
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-816) Scoop and support for external Hive tables

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479117#comment-16479117
 ] 

ASF subversion and git services commented on SQOOP-816:
---

Commit 6c0b3201280f3c7c4c56530909fbb2977e58542e in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=6c0b320 ]

SQOOP-3324: Document SQOOP-816: Sqoop add support for external Hive tables

(Fero Szabo via Szabolcs Vasas)


> Scoop and support for external Hive tables
> --
>
> Key: SQOOP-816
> URL: https://issues.apache.org/jira/browse/SQOOP-816
> Project: Sqoop
>  Issue Type: Improvement
>  Components: hive-integration
>Reporter: Santosh Achhra
>Assignee: Chris Teoh
>Priority: Minor
>  Labels: External, Hive,, Scoop,, Tables, newbie
> Fix For: 1.4.7
>
>
> Sqoop is not supporting HIVE external tables at the moment. Any imports using 
> scoop creates a managed table, in real world scenario it is very important to 
> have EXTERNAL tables. As of now we have to execute ALTER statement to change 
> table properties to make the the table as external table which is not a big 
> deal but it would nice have an option in scoop to specify type of table which 
> is required



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3322) Version differences between ivy configurations

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470713#comment-16470713
 ] 

ASF subversion and git services commented on SQOOP-3322:


Commit 2ca85527fd9f927add9127f91f3f3ef0c98fed6e in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=2ca8552 ]

SQOOP-3322: Version differences between ivy configurations

(Daniel Voros via Boglarka Egyed)


> Version differences between ivy configurations
> --
>
> Key: SQOOP-3322
> URL: https://issues.apache.org/jira/browse/SQOOP-3322
> Project: Sqoop
>  Issue Type: Bug
>  Components: build
>Affects Versions: 1.4.7
>Reporter: Daniel Voros
>Assignee: Daniel Voros
>Priority: Minor
> Fix For: 3.0.0
>
>
> We have multiple ivy configurations defined in ivy.xml.
>  - The {{redist}} configuration is used to select the artifacts that need to 
> be distributed with Sqoop in its tar.gz.
>  - The {{common}} configuration is used to set the classpath during 
> compilation (also refered to as 'hadoop classpath')
>  -  The {{test}} configuration is used to set the classpath during junit 
> execution. It extends the {{common}} config.
> Some artifacts end up having different versions between these three 
> configurations, which means we're using different versions during 
> compilation/testing/runtime.
> Differences:
> ||Artifact||redist||common (compilation)||test||
> |commons-pool|not in redist|1.5.4|*1.6*|
> |commons-codec|1.4|1.9|*1.9*|
> |commons-io|1.4|2.4|*2.4*|
> |commons-logging|1.1.1|1.2|*1.2*|
> |slf4j-api|1.6.1|1.7.7|*1.7.7*|
> I'd suggest using the version *in bold* in all three configurations to use 
> the latest versions.
> To achieve this we should exclude these artifacts from the transitive 
> dependencies and define them explicitly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3321) Fix TestHiveImport failing on Jenkins and Linux

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470573#comment-16470573
 ] 

ASF subversion and git services commented on SQOOP-3321:


Commit 03a895c5881b6dc9f748480b7d0562377d3cc978 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=03a895c ]

SQOOP-3321: Fix TestHiveImport failing on Jenkins and Linux

(Daniel Voros via Boglarka Egyed)


> Fix TestHiveImport failing on Jenkins and Linux
> ---
>
> Key: SQOOP-3321
> URL: https://issues.apache.org/jira/browse/SQOOP-3321
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.4.7
>Reporter: Boglarka Egyed
>Assignee: Daniel Voros
>Priority: Major
> Attachments: TEST-org.apache.sqoop.hive.TestHiveImport.txt
>
>
> org.apache.sqoop.hive.TestHiveImport is failing since 
> [SQOOP-3318|https://reviews.apache.org/r/66761/bugs/SQOOP-3318/] has been 
> committed. This test seem to be failing only in the Jenkins environment as it 
> pass on several local machines. There can be some difference in the 
> filesystem which may cause this issue, it shall be investigated. I am 
> attaching the log from a failed run.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3318) Remove Kite dependency from test cases

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456113#comment-16456113
 ] 

ASF subversion and git services commented on SQOOP-3318:


Commit 11c83f68386add243762929ecf7f6f25a99efbf4 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=11c83f6 ]

SQOOP-3318: Remove Kite dependency from test cases

(Szabolcs Vasas via Boglarka Egyed)


> Remove Kite dependency from test cases
> --
>
> Key: SQOOP-3318
> URL: https://issues.apache.org/jira/browse/SQOOP-3318
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3318.patch
>
>
> Some Sqoop tests use Kite to create test data and verify test results.
> Since we want to remove the Kite dependency from Sqoop we should rewrite 
> these test cases not to use Kite anymore.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3309) Implement HiveServer2 client

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16442549#comment-16442549
 ] 

ASF subversion and git services commented on SQOOP-3309:


Commit 72c5cd717e3fad6d5f5a3a2b3d185ffbacd876cf in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=72c5cd7 ]

SQOOP-3309: Implement HiveServer2 client

(Szabolcs Vasas via Boglarka Egyed)


> Implement HiveServer2 client
> 
>
> Key: SQOOP-3309
> URL: https://issues.apache.org/jira/browse/SQOOP-3309
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3309.patch
>
>
> This JIRA covers the implementation of the client for HiveServer2 and its 
> integration into the classes which use HiveImport.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-2567) SQOOP import for Oracle fails with invalid precision/scale for decimal

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16442171#comment-16442171
 ] 

ASF subversion and git services commented on SQOOP-2567:


Commit 44ac3012f19e479dd8ceb487bf029f0bc09c393b in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=44ac301 ]

SQOOP-2567: SQOOP import for Oracle fails with invalid precision/scale for 
decimal

(Fero Szabo via Szabolcs Vasas)


> SQOOP import for Oracle fails with invalid precision/scale for decimal
> --
>
> Key: SQOOP-2567
> URL: https://issues.apache.org/jira/browse/SQOOP-2567
> Project: Sqoop
>  Issue Type: Bug
>  Components: connectors
>Affects Versions: 1.4.5
> Environment: CDH5.3
>Reporter: Suresh Deoda
>Assignee: Fero Szabo
>Priority: Major
>  Labels: AVRO, ORACLE
>
> Sqoop import fails creating avrodata file from the oracle source with decimal 
> data. If the table in oracle is defined as say,
> Col1  as Decimal(12,11) , but if some data has few less digits in scale then 
> it fails with the error as,
> Error: org.apache.avro.file.DataFileWriter$AppendWriteException: 
> org.apache.avro.AvroTypeException: Cannot encode decimal with scale 10 as 
> scale 11
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:296)
> at 
> org.apache.sqoop.mapreduce.AvroOutputFormat$1.write(AvroOutputFormat.java:112)
> at 
> org.apache.sqoop.mapreduce.AvroOutputFormat$1.write(AvroOutputFormat.java:108)
> at 
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:655)
> at 
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at 
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> at 
> org.apache.sqoop.mapreduce.AvroImportMapper.map(AvroImportMapper.java:73)
> at 
> org.apache.sqoop.mapreduce.AvroImportMapper.map(AvroImportMapper.java:39)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> at 
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> Caused by: org.apache.avro.AvroTypeException: Cannot encode decimal with 
> scale 10 as scale 11
> at 
> org.apache.avro.Conversions$DecimalConversion.toBytes(Conversions.java:68)
> at 
> org.apache.avro.Conversions$DecimalConversion.toBytes(Conversions.java:39)
> at 
> org.apache.avro.generic.GenericDatumWriter.convert(GenericDatumWriter.java:90)
> at 
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:70)
> at 
> org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:143)
> at 
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:112)
> at 
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at 
> org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:143)
> at 
> org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
> at 
> org.apache.avro.reflect.ReflectDatumWriter.writeField(ReflectDatumWriter.java:175)
> also, when we dont have precision defined in Oracle ( which it takes default 
> (38,0) i guess) it gives error as ,
> ERROR tool.ImportTool: Imported Failed: Invalid decimal precision: 0 (must be 
> positive)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3216) Expanded Metastore support for MySql, Oracle, Postgresql, MSSql, and DB2

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431947#comment-16431947
 ] 

ASF subversion and git services commented on SQOOP-3216:


Commit af7a594d987ece6c1990be950c48d94bbab8271f in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=af7a594 ]

SQOOP-3301: Document SQOOP-3216 - metastore related change

(Fero Szabo via Szabolcs Vasas)


> Expanded Metastore support for MySql, Oracle, Postgresql, MSSql, and DB2
> 
>
> Key: SQOOP-3216
> URL: https://issues.apache.org/jira/browse/SQOOP-3216
> Project: Sqoop
>  Issue Type: New Feature
>  Components: metastore
>Reporter: Zach Berkowitz
>Assignee: Zach Berkowitz
>Priority: Minor
> Fix For: 1.5.0
>
> Attachments: SQOOP-3216-2.patch, SQOOP-3216-3.patch, 
> SQOOP-3216-4.patch, SQOOP-3216.patch
>
>
> Reconfigured HsqldbJobStorage class to support MySql, Oracle, Postgresql, 
> MSSql, and DB2 databases in addition to Hsqldb, renamed HsqldbJobStorage 
> GenericJobStorage. This new class also serves the function of 
> AutoHsqldbStorage, which has been removed.
> Two new commands, --meta-username and --meta-password have been added to 
> connect to metastore databases that require a username and password. 
> Added an enum class JdbcDrivers that holds Jdbc connection information.
> Added two testClasses, MetaConnectIncrementalImportTest and JobToolTest, and 
> modified TestSavedJobs (now SavedJobsTest) to test with all supported 
> database services.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3301) Document SQOOP-3216 - metastore related change

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431946#comment-16431946
 ] 

ASF subversion and git services commented on SQOOP-3301:


Commit af7a594d987ece6c1990be950c48d94bbab8271f in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=af7a594 ]

SQOOP-3301: Document SQOOP-3216 - metastore related change

(Fero Szabo via Szabolcs Vasas)


> Document SQOOP-3216 - metastore related change
> --
>
> Key: SQOOP-3301
> URL: https://issues.apache.org/jira/browse/SQOOP-3301
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Fero Szabo
>Assignee: Fero Szabo
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3307) Don't create HTML during Ivy report

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419126#comment-16419126
 ] 

ASF subversion and git services commented on SQOOP-3307:


Commit c146b3f94e937a8c03b6ae85a60800d1c62bdc94 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=c146b3f ]

SQOOP-3307: Don't create HTML during Ivy report

(Daniel Voros via Boglarka Egyed)


> Don't create HTML during Ivy report
> ---
>
> Key: SQOOP-3307
> URL: https://issues.apache.org/jira/browse/SQOOP-3307
> Project: Sqoop
>  Issue Type: Task
>Affects Versions: 1.4.7
>Reporter: Daniel Voros
>Assignee: Daniel Voros
>Priority: Minor
> Fix For: 1.5.0
>
>
> {{ant clean report}} invokes the [ivy:report 
> |https://ant.apache.org/ivy/history/2.1.0/use/report.html] task and creates 
> both HTML and GraphML reports.
> Creation of the HTML reports takes ~7 minutes and results in a ~700MB html 
> that's hard to make use of, while the GraphML reporting is fast and is easier 
> to read.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3308) Mock ConnManager field in TestTableDefWriter

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419060#comment-16419060
 ] 

ASF subversion and git services commented on SQOOP-3308:


Commit d2c40366a7458b0f4804a382403bb40ac72fbb4f in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=d2c4036 ]

SQOOP-3308: Mock ConnManager field in TestTableDefWriter

(Szabolcs Vasas via Boglarka Egyed)


> Mock ConnManager field in TestTableDefWriter
> 
>
> Key: SQOOP-3308
> URL: https://issues.apache.org/jira/browse/SQOOP-3308
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3308.patch, SQOOP-3308.patch, SQOOP-3308.patch, 
> SQOOP-3308.patch
>
>
> TableDefWriter has a dependency on ConnManager to retrieve the names and the 
> types of the table. It also introduces a field called _externalColTypes_ for 
> testing purposes and TestTableDefWriter uses this field to inject the test 
> table column names and types instead of mocking the ConnManager field.
> This setup makes it harder to add test cases to TestTableDefWriter and not a 
> good practice so it should be fixed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3303) Fix warnings during Sqoop compilation

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413073#comment-16413073
 ] 

ASF subversion and git services commented on SQOOP-3303:


Commit 7186b9d654c54fbe575e273c0a182307d3d48893 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=7186b9d ]

SQOOP-3303: Fix warnings during Sqoop compilation

(Szabolcs Vasas via Boglarka Egyed)


> Fix warnings during Sqoop compilation
> -
>
> Key: SQOOP-3303
> URL: https://issues.apache.org/jira/browse/SQOOP-3303
> Project: Sqoop
>  Issue Type: Task
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3303.patch
>
>
> Ant prints the following warning during every Sqoop compilation:
> {code:java}
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestAvroExport.java:477:
>  warning: non-varargs call of varargs method with inexact argument type for 
> last parameter;
>     createAvroFile(0, TOTAL_RECORDS, null);
>                                      ^
>   cast to ColumnGenerator for a varargs call
>   cast to ColumnGenerator[] for a non-varargs call and to suppress this 
> warning
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestAvroExport.java:492:
>  warning: non-varargs call of varargs method with inexact argument type for 
> last parameter;
>     createAvroFile(0, TOTAL_RECORDS, null);
>                                      ^
>   cast to ColumnGenerator for a varargs call
>   cast to ColumnGenerator[] for a non-varargs call and to suppress this 
> warning
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestParquetExport.java:422:
>  warning: non-varargs call of varargs method with inexact argument type for 
> last parameter;
>     createParquetFile(0, TOTAL_RECORDS, null);
>                                         ^
>   cast to ColumnGenerator for a varargs call
>   cast to ColumnGenerator[] for a non-varargs call and to suppress this 
> warning
> /Users/szabolcsvasas/Documents/workspace/sqoop/sqoop-vasas-gradle/sqoop/src/test/org/apache/sqoop/TestParquetExport.java:435:
>  warning: non-varargs call of varargs method with inexact argument type for 
> last parameter;
>     createParquetFile(0, TOTAL_RECORDS, null);
>                                         ^
>   cast to ColumnGenerator for a varargs call
>   cast to ColumnGenerator[] for a non-varargs call and to suppress this 
> warning
> {code}
> It is kind of annoying and would be very easy to fix.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3302) Add missing license header to HBaseKerberizedConnectivityTest and TestSQLServerDBRecordReader

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413062#comment-16413062
 ] 

ASF subversion and git services commented on SQOOP-3302:


Commit a69e716d1ebece53036a242ad88ebd1f52404b38 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=a69e716 ]

SQOOP-3302: Add missing license header to HBaseKerberizedConnectivityTest and 
TestSQLServerDBRecordReader

(Szabolcs Vasas via Boglarka Egyed)


> Add missing license header to HBaseKerberizedConnectivityTest and 
> TestSQLServerDBRecordReader
> -
>
> Key: SQOOP-3302
> URL: https://issues.apache.org/jira/browse/SQOOP-3302
> Project: Sqoop
>  Issue Type: Task
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3302.patch
>
>
> HBaseKerberizedConnectivityTest and TestSQLServerDBRecordReader do not have 
> the Apache license header which should be fixed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3304) Increase the maximum memory for JUnit tests

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413041#comment-16413041
 ] 

ASF subversion and git services commented on SQOOP-3304:


Commit 5de9603cd2dd74c1210ff69d9a55f24417f83513 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=5de9603 ]

SQOOP-3304: Increase the maximum memory for JUnit tests

(Szabolcs Vasas via Boglarka Egyed)


> Increase the maximum memory for JUnit tests
> ---
>
> Key: SQOOP-3304
> URL: https://issues.apache.org/jira/browse/SQOOP-3304
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3304.patch
>
>
> TestKerberosAuthenticator committed in SQOOP-3300 fails in our Jenkins build 
> with an OutOfMemoryError:
> {code:java}
> [junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
> [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 50.61 sec
> [junit] Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> [junit] at java.util.Arrays.copyOf(Arrays.java:2367)
> [junit] at 
> java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
> [junit] at 
> java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
> [junit] at 
> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
> [junit] at java.lang.StringBuffer.append(StringBuffer.java:237)
> [junit] at 
> org.apache.tools.ant.taskdefs.optional.junit.PlainJUnitResultFormatter.endTestSuite(PlainJUnitResultFormatter.java:141)
> [junit] at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.fireEndTestSuite(JUnitTestRunner.java:731)
> [junit] at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:553)
> [junit] at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
> [junit] at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)
> [junit] Running org.apache.sqoop.authentication.TestKerberosAuthenticator
> [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
> {code}
> As this test case worked fine in our local environments increasing the 
> maximum memory available for JUnit tests should solve the problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3300) Implement JDBC and Kerberos tools for HiveServer2 support

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411351#comment-16411351
 ] 

ASF subversion and git services commented on SQOOP-3300:


Commit d67bb816ce35cdf59a3504845a0bc5639f690dd6 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=d67bb81 ]

SQOOP-3300: Implement JDBC and Kerberos tools for HiveServer2 support

(Szabolcs Vasas via Boglarka Egyed)


> Implement JDBC and Kerberos tools for HiveServer2 support
> -
>
> Key: SQOOP-3300
> URL: https://issues.apache.org/jira/browse/SQOOP-3300
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
>Priority: Major
> Attachments: SQOOP-3300.patch
>
>
> The idea of the Sqoop HS2 support is to connect to HS2 using JDBC and execute 
> the Hive commands on this connection. Sqoop should also support Kerberos 
> authentication when building this JDBC connection.
> The goal of this JIRA is to implement the necessary classes for building JDBC 
> connections and authenticating with Kerberos.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3293) Document SQOOP-2976

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408035#comment-16408035
 ] 

ASF subversion and git services commented on SQOOP-3293:


Commit d57f9fb06b55650adc75cd1972df0024d7e4dba1 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=d57f9fb ]

SQOOP-3293: Document SQOOP-2976

(Fero Szabo by Szabolcs Vasas)


> Document SQOOP-2976
> ---
>
> Key: SQOOP-3293
> URL: https://issues.apache.org/jira/browse/SQOOP-3293
> Project: Sqoop
>  Issue Type: Sub-task
>Reporter: Fero Szabo
>Assignee: Fero Szabo
>Priority: Major
> Attachments: SQOOP-3292.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-2976) Flag to expand decimal values to fit AVRO schema

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408036#comment-16408036
 ] 

ASF subversion and git services commented on SQOOP-2976:


Commit d57f9fb06b55650adc75cd1972df0024d7e4dba1 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=d57f9fb ]

SQOOP-3293: Document SQOOP-2976

(Fero Szabo by Szabolcs Vasas)


> Flag to expand decimal values to fit AVRO schema
> 
>
> Key: SQOOP-2976
> URL: https://issues.apache.org/jira/browse/SQOOP-2976
> Project: Sqoop
>  Issue Type: Improvement
>Affects Versions: 1.4.6
>Reporter: Thomas Scott
>Assignee: Fero Szabo
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: SQOOP-2976.patch, SQOOP-2976.patch
>
>
> As per https://issues.apache.org/jira/browse/AVRO-1864 when importing from 
> Oracle (or any other database that truncates decimals) Sqoop jobs can fail 
> because the scale of the decimal produced by the database does not match the 
> scale in the AVRO file.
> For instance if the value 3.15 is produced by Oracle and the AVRO decimal 
> scale  is 3 (this can happen even if the Oracle column is defined with scale 
> of 3) then the job will fail. 
> Can we have a flag (--pad-decimals) that pads incoming values with zeros to 
> fit the AVRO schema (e.g. 3.15 becomes 3.150).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3288) Incremental import's upper bound ignores session time zone in Oracle

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374132#comment-16374132
 ] 

ASF subversion and git services commented on SQOOP-3288:


Commit a7f5e0d298ffbf8e674bd35ee10f2accc1da5453 in sqoop's branch 
refs/heads/trunk from [~maugli]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=a7f5e0d ]

SQOOP-3288: Changing OracleManager to use CURRENT_TIMESTAMP instead of
SYSDATE in case of incremental import to make the process session time
aware

(Daniel Voros via Attila Szabo)


> Incremental import's upper bound ignores session time zone in Oracle
> 
>
> Key: SQOOP-3288
> URL: https://issues.apache.org/jira/browse/SQOOP-3288
> Project: Sqoop
>  Issue Type: Bug
>  Components: connectors/oracle
>Affects Versions: 1.4.7
>Reporter: Daniel Voros
>Assignee: Daniel Voros
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: SQOOP-3288.1.patch
>
>
> At the moment we're using [{{SELECT SYSDATE FROM 
> dual}}|https://github.com/apache/sqoop/blob/3153c3610da7e5db388bfb14f3681d308e9e89c6/src/java/org/apache/sqoop/manager/OracleManager.java#L652]
>  when getting current time from Oracle.
> SYSDATE returns the underlying operating system's current time, while 
> CURRENT_TIMESTAMP uses the session time zone. This could lead to problems 
> during incremental imports *when Oracle's time zone is different from the OS*.
> Consider the following scenario when Oracle is configured to {{+0:00}}, while 
> the OS is {{+5:00}}:
> ||Oracle time||OS time||Event||
> |2:00|7:00|{{sqoop import --last-value 1:00 ...}} => imports {{[1:00, 7:00)}}|
> |2:30|7:30|{{update ... set last_updated = current_timestamp ...}} => set to 
> {{2:30}} *Won't be imported!*|
> |3:00|8:00|{{sqoop import --last-value 7:00 ...}} => imports {{[7:00, 8:00)}}|
> This way records updated within 5 hours after the last sqoop import won't get 
> imported.
> Please note, that the example above assumes, that the user/administrator 
> who's updating the Oracle table will use the current session time of Oracle 
> when setting the "last updated" column of the table.
> I think the solution is to use CURRENT_TIMESTAMP instead of SYSDATE. Other 
> connection managers, like MySQL or PostgreSQL are using that as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3267) Incremental import to HBase deletes only last version of column

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372849#comment-16372849
 ] 

ASF subversion and git services commented on SQOOP-3267:


Commit 69463f0b3ed3af28581202ef59079b9df7bc0bad in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=69463f0 ]

SQOOP-3267: Incremental import to HBase deletes only last version of column

(Daniel Voros by Szabolcs Vasas)


> Incremental import to HBase deletes only last version of column
> ---
>
> Key: SQOOP-3267
> URL: https://issues.apache.org/jira/browse/SQOOP-3267
> Project: Sqoop
>  Issue Type: Bug
>  Components: hbase-integration
>Affects Versions: 1.4.7
>Reporter: Daniel Voros
>Assignee: Daniel Voros
>Priority: Major
> Attachments: SQOOP-3267.1.patch, SQOOP-3267.2.patch
>
>
> Deletes are supported since SQOOP-3149, but we're only deleting the last 
> version of a column when the corresponding cell was set to NULL in the source 
> table.
> This can lead to unexpected and misleading results if the row has been 
> transferred multiple times, which can easily happen if it's being modified on 
> the source side.
> Also SQOOP-3149 is using a new Put command for every column instead of a 
> single Put per row as before. This could probably lead to a performance drop 
> for wide tables (for which HBase is otherwise usually recommended).
> [~jilani], [~anna.szonyi] could you please comment on what you think would be 
> the expected behavior here?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3283) MySQL thirdparty tests hang if there's no USER environment variable

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364527#comment-16364527
 ] 

ASF subversion and git services commented on SQOOP-3283:


Commit 3153c3610da7e5db388bfb14f3681d308e9e89c6 in sqoop's branch 
refs/heads/trunk from [~maugli]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=3153c36 ]

SQOOP-3283: Fixing MySQL 3rd party test hanging issue by getting username
from System env/property instead of depending on whoami and Process#waitFor

(Daniel Voros by Attila Szabo)


> MySQL thirdparty tests hang if there's no USER environment variable
> ---
>
> Key: SQOOP-3283
> URL: https://issues.apache.org/jira/browse/SQOOP-3283
> Project: Sqoop
>  Issue Type: Bug
>  Components: connectors/mysql, test
>Affects Versions: 1.4.7
>Reporter: Daniel Voros
>Assignee: Daniel Voros
>Priority: Minor
> Attachments: SQOOP-3283.1.patch, SQOOP-3283.2.patch
>
>
> {{org.apache.sqoop.manager.mysql.MySQLTestUtils#getCurrentUser()}} executes 
> {{whoami}} in a subprocess if there's no USER environment variable (happened 
> to me while running tests from Docker). However, it waits for the Process 
> variable to become null, that never happens:
> {code:java}
> // wait for whoami to exit.
> while (p != null) {
>   try {
> int ret = p.waitFor();
> if (0 != ret) {
>   LOG.error("whoami exited with error status " + ret);
>   // suppress original return value from this method.
>   return null;
> }
>   } catch (InterruptedException ie) {
> continue; // loop around.
>   }
> }
> {code}
> We could get rid of the while loop since {{Process#waitFor()}} blocks while 
> it completes.
> Note, that it's easy to workaround the issue by setting the USER environment 
> variable when running the tests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-2976) Flag to expand decimal values to fit AVRO schema

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364091#comment-16364091
 ] 

ASF subversion and git services commented on SQOOP-2976:


Commit f7b460b3f57c1bc81e2e0a1e8c28a331729f4213 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=f7b460b ]

SQOOP-2976: Flag to expand decimal values to fit AVRO schema

(Ferenc Szabo via Boglarka Egyed)


> Flag to expand decimal values to fit AVRO schema
> 
>
> Key: SQOOP-2976
> URL: https://issues.apache.org/jira/browse/SQOOP-2976
> Project: Sqoop
>  Issue Type: Improvement
>Affects Versions: 1.4.6
>Reporter: Thomas Scott
>Assignee: Ferenc Szabo
>Priority: Major
> Attachments: SQOOP-2976.patch, SQOOP-2976.patch
>
>
> As per https://issues.apache.org/jira/browse/AVRO-1864 when importing from 
> Oracle (or any other database that truncates decimals) Sqoop jobs can fail 
> because the scale of the decimal produced by the database does not match the 
> scale in the AVRO file.
> For instance if the value 3.15 is produced by Oracle and the AVRO decimal 
> scale  is 3 (this can happen even if the Oracle column is defined with scale 
> of 3) then the job will fail. 
> Can we have a flag (--pad-decimals) that pads incoming values with zeros to 
> fit the AVRO schema (e.g. 3.15 becomes 3.150).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SQOOP-3275) HBase test cases should start mini DFS cluster as well

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316601#comment-16316601
 ] 

ASF subversion and git services commented on SQOOP-3275:


Commit bf8b8a6acd9f69c01c1bc8e42a2070f5cc596ccd in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=bf8b8a6 ]

SQOOP-3275: HBase test cases should start mini DFS cluster as well

(Szabolcs Vasas via Boglarka Egyed)


> HBase test cases should start mini DFS cluster as well
> --
>
> Key: SQOOP-3275
> URL: https://issues.apache.org/jira/browse/SQOOP-3275
> Project: Sqoop
>  Issue Type: Test
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
> Attachments: SQOOP-3275.patch
>
>
> The HBase test cases in Sqoop start mini HBase and mini Zookeeper clusters 
> but they do not start mini DFS cluster.
> The recommended way to use the mini HBase cluster is to use the mini DFS 
> cluster too it would be good if Sqoop should start it as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (SQOOP-3255) Sqoop ignores metastore properties defined in sqoop-site.xml

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313302#comment-16313302
 ] 

ASF subversion and git services commented on SQOOP-3255:


Commit 010042dd47296e0132944cba146c077754b15195 in sqoop's branch 
refs/heads/trunk from [~BoglarkaEgyed]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=010042d ]

SQOOP-3255: Sqoop ignores metastore properties defined in sqoop-site.xml

(Szabolcs Vasas via Boglarka Egyed)


> Sqoop ignores metastore properties defined in sqoop-site.xml
> 
>
> Key: SQOOP-3255
> URL: https://issues.apache.org/jira/browse/SQOOP-3255
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.5.0
>Reporter: Szabolcs Vasas
>Assignee: Szabolcs Vasas
> Attachments: SQOOP-3255.patch, SQOOP-3255.patch, SQOOP-3255.patch
>
>
> Sqoop ignores the following configuration parameters defined in 
> sqoop-site.xml:
> sqoop.metastore.client.autoconnect.url
> sqoop.metastore.client.autoconnect.username
> sqoop.metastore.client.autoconnect.password
> These parameters are ignored even if they are specified with -D option for 
> Sqoop.
> This bug is introduced in SQOOP-3216 the task is to restore the original 
> behavior and continue supporting these parameters.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (SQOOP-3241) ImportAllTablesTool uses the same SqoopOptions object for every table import

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313175#comment-16313175
 ] 

ASF subversion and git services commented on SQOOP-3241:


Commit 7ef6a8013a99738e38a41b20f1f0edccc927a9b8 in sqoop's branch 
refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=7ef6a80 ]

SQOOP-3241: ImportAllTablesTool uses the same SqoopOptions object for every 
table import

(Ferenc Szabo via Szabolcs Vasas)


> ImportAllTablesTool uses the same SqoopOptions object for every table import
> 
>
> Key: SQOOP-3241
> URL: https://issues.apache.org/jira/browse/SQOOP-3241
> Project: Sqoop
>  Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Szabolcs Vasas
>Assignee: Ferenc Szabo
> Attachments: SQOOP-3241.patch
>
>
> ImportAllTablesTool queries the list of tables from the database and invokes 
> ImportTool#importTable method for each table.
> The problem is that it passes the same SqoopOptions object in every 
> invocation and since SqoopOptions is not immutable this can lead to issues.
> For example in case of Parquet imports the CodeGenTool#generateORM method 
> modifies the className field of the SqoopOptions object which is then remains 
> the same for all the subsequent table imports and can cause job failures.
> One solution could be to create a new SqoopOptions object with the same field 
> values for every ImportTool#importTable invocation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)