Re: Review Request 58600: add --schema option for import-all-tables and list-tables against db2

2017-06-05 Thread Ying Cao 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58600/
---

(Updated 六月 5, 2017, 3:01 p.m.)


Review request for Sqoop, Abraham Elmahrek, Boglarka Egyed, Jarek Cecho, Attila 
Szabo, and Szabolcs Vasas.


Changes
---

Fix base on Bogi's comments


Bugs: SQOOP-1905
https://issues.apache.org/jira/browse/SQOOP-1905


Repository: sqoop-trunk


Description
---

SQOOP-1905 : add --schema option for import-all-tables and list-tables against 
db2


Diffs (updated)
-

  src/java/org/apache/sqoop/manager/Db2Manager.java 52ab05ef 
  src/java/org/apache/sqoop/manager/GenericJdbcManager.java 2113a5f5 
  
src/test/org/apache/sqoop/manager/db2/DB2ImportAllTableWithSchemaManualTest.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/58600/diff/4/

Changes: https://reviews.apache.org/r/58600/diff/3-4/


Testing
---

Mannual UT  is passed


Thanks,

Ying Cao

Need help in Sqoop2

2017-06-05 Thread aman singh 
Hi All,

I am facing some issues while importing data from multiple tables of Oracle
and Postgres.
Whenever I start the job, it results to error after going in the running
state. Can anyone help me in this.
PFA the job and error details.
Many thanks.

Best regards,
Amanpreet Singh
sqoop:000> show job --jid 307
1 job(s) to show:
Job with id 307 and name pgjob (Enabled: true, Created by root at 5/29/17 10:25 
PM, Updated by root at 5/29/17 11:49 PM)
Using link id 699 and Connector id 4
  From database configuration
Schema name:
Table name:
Table SQL statement: select 
dg.testing.newdata.id,dg.testing.newdata.creditcard,dg.testing.testdata.phone 
from dg.testing.newdata,dg.testing.testdata where ${CONDITIONS}
Table column names:
Partition column name: id
Null value allowed for the partition column:
Boundary query:
  Throttling resources
Extractors: 1
Loaders: 1
  ToJob configuration
Override null value:
Null value:
Output format: TEXT_FILE
Compression format: NONE
Custom compression format:
Output directory: /sqoop
2017-05-29 23:50:07,013 WARN [main] 
org.apache.hadoop.metrics2.impl.MetricsConfig: Cannot locate configuration: 
tried hadoop-metrics2-maptask.properties,hadoop-metrics2.properties
2017-05-29 23:50:07,275 INFO [main] 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 
10 second(s).
2017-05-29 23:50:07,275 INFO [main] 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system 
started
2017-05-29 23:50:07,328 INFO [main] org.apache.hadoop.mapred.YarnChild: 
Executing with tokens:
2017-05-29 23:50:07,329 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: 
mapreduce.job, Service: job_1494932549864_0004, Ident: 
(org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@4351786b)
2017-05-29 23:50:07,695 INFO [main] org.apache.hadoop.mapred.YarnChild: 
Sleeping for 0ms before retrying again. Got null now.
2017-05-29 23:50:08,571 INFO [main] org.apache.hadoop.mapred.YarnChild: 
mapreduce.cluster.local.dir for child: 
/var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/sqoop2/appcache/application_1494932549864_0004
2017-05-29 23:50:10,448 INFO [main] 
org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. 
Instead, use dfs.metrics.session-id
2017-05-29 23:50:11,478 INFO [main] org.apache.hadoop.mapred.Task:  Using 
ResourceCalculatorProcessTree : [ ]
2017-05-29 23:50:12,010 INFO [main] org.apache.hadoop.mapred.MapTask: 
Processing split: org.apache.sqoop.job.mr.SqoopSplit@79c8173a
2017-05-29 23:50:12,747 INFO [main] org.apache.hadoop.mapred.MapTask: (EQUATOR) 
0 kvi 26214396(104857584)
2017-05-29 23:50:12,747 INFO [main] org.apache.hadoop.mapred.MapTask: 
mapreduce.task.io.sort.mb: 100
2017-05-29 23:50:12,747 INFO [main] org.apache.hadoop.mapred.MapTask: soft 
limit at 83886080
2017-05-29 23:50:12,747 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart 
= 0; bufvoid = 104857600
2017-05-29 23:50:12,747 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 
26214396; length = 6553600
2017-05-29 23:50:12,773 INFO [main] org.apache.hadoop.mapred.MapTask: Map 
output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-05-29 23:50:13,019 INFO [main] org.apache.sqoop.job.mr.SqoopMapper: 
Starting progress service
2017-05-29 23:50:13,025 INFO [main] org.apache.sqoop.job.mr.SqoopMapper: 
Running extractor class org.apache.sqoop.connector.jdbc.GenericJdbcExtractor
2017-05-29 23:50:13,342 INFO [main] 
org.apache.sqoop.connector.jdbc.GenericJdbcExtractor: Using query: select 
dg.testing.newdata.id,dg.testing.newdata.creditcard,dg.testing.testdata.phone 
from dg.testing.newdata,dg.testing.testdata where 1 <= id AND id <= 10
2017-05-29 23:50:13,402 ERROR [main] 
org.apache.sqoop.connector.jdbc.GenericJdbcExecutor: Caught SQLException:
org.postgresql.util.PSQLException: ERROR: column reference "id" is ambiguous
  Position: 141
at 
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2102)
at 
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1835)
at 
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:500)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:374)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:254)
at 
org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.executeQuery(GenericJdbcExecutor.java:67)
at 
org.apache.sqoop.connector.jdbc.GenericJdbcExtractor.extract(GenericJdbcExtractor.java:58)
at 
org.apache.sqoop.connector.jdbc.GenericJdbcExtractor.extract(GenericJdbcExtractor.java:38)
at org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:95)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)

Re: Review Request 58600: add --schema option for import-all-tables and list-tables against db2

2017-06-05 Thread Boglarka Egyed 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58600/#review176905
---


Fix it, then Ship it!




Hi Angela,

Build and ant clean test were successful with your patch.

I have left one last finding, please correct it before committing this change.

Thanks for your contribution!

Cheers,
Bogi


src/test/org/apache/sqoop/manager/db2/DB2ImportAllTableWithSchemaManual.java
Lines 76 (patched)


Please use ManualTest at the end so that the related ant task for manual 
tests will find it.


- Boglarka Egyed


On June 4, 2017, 4:38 p.m., Ying Cao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58600/
> ---
> 
> (Updated June 4, 2017, 4:38 p.m.)
> 
> 
> Review request for Sqoop, Abraham Elmahrek, Boglarka Egyed, Jarek Cecho, 
> Attila Szabo, and Szabolcs Vasas.
> 
> 
> Bugs: SQOOP-1905
> https://issues.apache.org/jira/browse/SQOOP-1905
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> ---
> 
> SQOOP-1905 : add --schema option for import-all-tables and list-tables 
> against db2
> 
> 
> Diffs
> -
> 
>   src/java/org/apache/sqoop/manager/Db2Manager.java 52ab05ef 
>   src/java/org/apache/sqoop/manager/GenericJdbcManager.java 2113a5f5 
>   
> src/test/org/apache/sqoop/manager/db2/DB2ImportAllTableWithSchemaManual.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/58600/diff/3/
> 
> 
> Testing
> ---
> 
> Mannual UT  is passed
> 
> 
> Thanks,
> 
> Ying Cao
> 
>

Re: Review Request 59676: [SQOOP-2922] Add --hive-database documentation to user guide

2017-06-05 Thread Boglarka Egyed 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59676/#review176904
---


Ship it!




Ship It!

- Boglarka Egyed


On May 31, 2017, 1:23 a.m., Andy Stadtler wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59676/
> ---
> 
> (Updated May 31, 2017, 1:23 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-2922
> https://issues.apache.org/jira/browse/SQOOP-2922
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> ---
> 
> SQOOP-912 added the --hive-database feature but it has not been added to the 
> user-guide documentation.
> 
> 
> Diffs
> -
> 
>   src/docs/user/hive-args.txt 441f54e 
> 
> 
> Diff: https://reviews.apache.org/r/59676/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andy Stadtler
> 
>