Ya sure, please have a look on below commands :-
bin/sqoop job -- export --connect jdbc:postgresql://localhost/dbname
--export-dir /data/data.2012-09-08-00.csv --staging-table daily_tmp
--clear-staging-table --verbose --table daily --username abc --password
abc --input-fields-terminated-by '^A'
Also attaching the output of job. But below lines explains everything :
12/09/14 04:13:52 INFO mapreduce.ExportJobBase: Transferred 396.1099 MB in
237.2008 seconds (1.6699 MB/sec)
12/09/14 04:13:52 INFO mapreduce.ExportJobBase: Exported 4071315 records.
12/09/14 04:13:52 INFO mapreduce.ExportJobBase: Starting to migrate data
from staging table to destination.
12/09/14 04:14:29 INFO manager.SqlManager: Migrated 5321315 records from
daily_tmp to daily
Total Records in CSV : 4071315 , Records Inserted : 52321315 ( due to fail
& rerun map tasks )
12/09/14 04:11:57 INFO mapred.JobClient: map 79% reduce 0%
12/09/14 04:12:00 INFO mapred.JobClient: map 80% reduce 0% ( Now map fails
)
12/09/14 04:12:01 INFO mapred.JobClient: map 75% reduce 0%
12/09/14 04:12:12 INFO mapred.JobClient: map 76% reduce 0%
Please let me know if other info is reqd.
Thanks
On Thu, Sep 13, 2012 at 10:12 PM, Kathleen Ting <[email protected]> wrote:
> Hi Adarsh, can you re-run with the --verbose option enabled? Also,
> please paste in the entire Sqoop command used.
>
> Thanks, Kathleen
>
> On Thu, Sep 13, 2012 at 7:53 AM, Adarsh Sharma <[email protected]>
> wrote:
> > Hi all,
> >
> > I am using sqoop-1.4.2 with cloudera hadoop and doing some tesing. We
> need
> > to export some tables from CSV's in HDFS.
> > As sqoop provides a mechanism of staging tables to write data in main
> tables
> > only if all maps are succeeded.
> >
> > While executing a sqoop job on hadoop , suppose a map fails & hadoop
> > reattempt the map to re-run and finish after 3 attempts, it results in
> > duplicate records in staging table and the job finished but data
> inserted is
> > higher than in CSV's. Below is the output :
> >
> > 12/09/13 14:46:55 INFO mapreduce.ExportJobBase: Exported 4071315 records.
> > 12/09/13 14:46:55 INFO mapreduce.ExportJobBase: Starting to migrate data
> > from staging table to destination.
> > 12/09/13 14:47:29 INFO manager.SqlManager: Migrated 5391315 records from
> > table1_tmp to table
> >
> > Is this is a bug in Sqoop and is there any fix or patch for it. Please
> let
> > me know.
> >
> >
> > Thanks
>
adarsh@1002:~/sqoop-1.4.2.bin__hadoop-0.20$ bin/sqoop j export --connect
jdbc:postgresql://localhost/dbname --export-dir /data/data.2012-09-08-00.csv
--staging-table daily_tmp --clear-staging-table --verbose --table daily
--username abc --password abc --input-fields-terminated-by '^A'
12/09/14 04:09:53 DEBUG tool.BaseSqoopTool: Enabled debug logging.
12/09/14 04:09:53 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
12/09/14 04:09:53 DEBUG sqoop.ConnFactory: Loaded manager factory:
com.cloudera.sqoop.manager.DefaultManagerFactory
12/09/14 04:09:53 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
com.cloudera.sqoop.manager.DefaultManagerFactory
12/09/14 04:09:53 DEBUG manager.DefaultManagerFactory: Trying with scheme:
jdbc:postgresql:
12/09/14 04:09:53 INFO manager.SqlManager: Using default fetchSize of 1000
12/09/14 04:09:53 DEBUG sqoop.ConnFactory: Instantiated ConnManager
org.apache.sqoop.manager.PostgresqlManager@b6e39f
12/09/14 04:09:53 INFO tool.CodeGenTool: Beginning code generation
12/09/14 04:09:53 DEBUG manager.SqlManager: No connection paramenters
specified. Using regular API for making connection.
12/09/14 04:09:53 DEBUG manager.SqlManager: Using fetchSize for next query: 1000
12/09/14 04:09:53 INFO manager.SqlManager: Executing SQL statement: SELECT t.*
FROM "daily" AS t LIMIT 1
12/09/14 04:09:53 DEBUG orm.ClassWriter: selected columns:
12/09/14 04:09:53 DEBUG orm.ClassWriter: a
12/09/14 04:09:53 DEBUG orm.ClassWriter: b
12/09/14 04:09:53 DEBUG orm.ClassWriter: total_cost
12/09/14 04:09:53 DEBUG orm.ClassWriter: c
12/09/14 04:09:53 DEBUG orm.ClassWriter: tpid
12/09/14 04:09:53 DEBUG orm.ClassWriter: daily
12/09/14 04:09:53 DEBUG orm.ClassWriter: e
12/09/14 04:09:53 DEBUG orm.ClassWriter: Writing source file:
/tmp/sqoop-adarsh/compile/70af23436d0ed0f9d7f1b6bb713f1227/daily.java
12/09/14 04:09:53 DEBUG orm.ClassWriter: Table name: daily
12/09/14 04:09:53 DEBUG orm.ClassWriter: Columns: a:12,b:8, total_cost:8, c:4,
tpid:12, daily:-5, e:12,
12/09/14 04:09:53 DEBUG orm.ClassWriter: sourceFilename is daily.java
12/09/14 04:09:53 DEBUG orm.CompilationManager: Found existing
/tmp/sqoop-adarsh/compile/70af23436d0ed0f9d7f1b6bb713f1227/
12/09/14 04:09:53 INFO orm.CompilationManager: HADOOP_HOME is
/usr/lib/hadoop-0.20
12/09/14 04:09:53 INFO orm.CompilationManager: Found hadoop core jar at:
/usr/lib/hadoop-0.20/hadoop-0.20.2-cdh3u3-core.jar
12/09/14 04:09:53 DEBUG orm.CompilationManager: Adding source file:
/tmp/sqoop-adarsh/compile/70af23436d0ed0f9d7f1b6bb713f1227/daily.java
12/09/14 04:09:53 DEBUG orm.CompilationManager: Invoking javac with args:
12/09/14 04:09:53 DEBUG orm.CompilationManager: -sourcepath
12/09/14 04:09:53 DEBUG orm.CompilationManager:
/tmp/sqoop-adarsh/compile/70af23436d0ed0f9d7f1b6bb713f1227/
12/09/14 04:09:53 DEBUG orm.CompilationManager: -d
12/09/14 04:09:53 DEBUG orm.CompilationManager:
/tmp/sqoop-adarsh/compile/70af23436d0ed0f9d7f1b6bb713f1227/
12/09/14 04:09:53 DEBUG orm.CompilationManager: -classpath
12/09/14 04:09:53 DEBUG orm.CompilationManager:
/usr/lib/hadoop/conf/:/usr/lib/jvm/java-6-sun-1.6.0.24/lib/tools.jar:/usr/lib/hadoop-0.20:/usr/lib/hadoop-0.20/hadoop-core-0.20.2-cdh3u3.jar:/usr/lib/hadoop-0.20/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop-0.20/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20/lib/commons-daemon-1.0.1.jar:/usr/lib/hadoop-0.20/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-0.20/lib/commons-lang-2.4.jar:/usr/lib/hadoop-0.20/lib/commons-logging-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-logging-api-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-net-1.4.1.jar:/usr/lib/hadoop-0.20/lib/core-3.1.1.jar:/usr/lib/hadoop-0.20/lib/guava-r09-jarjar.jar:/usr/lib/hadoop-0.20/lib/hadoop-capacity-scheduler-0.20.2-cdh3u3.jar:/usr/lib/hadoop-0.20/lib/hadoop-fairscheduler-0.20.2-cdh3u3.jar:/usr/lib/hadoop-0.20/lib/hadoop-lzo-0.4.10.jar:/usr/lib/hadoop-0.20/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20/lib/jackson-core-asl-1.5.2.jar:/usr/lib/hadoop-0.20/lib/jackson-mapper-asl-1.5.2.jar:/usr/lib/hadoop-0.20/lib/jasper-compiler-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jasper-runtime-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20/lib/jetty-6.1.26.cloudera.1.jar:/usr/lib/hadoop-0.20/lib/jetty-servlet-tester-6.1.26.cloudera.1.jar:/usr/lib/hadoop-0.20/lib/jetty-util-6.1.26.cloudera.1.jar:/usr/lib/hadoop-0.20/lib/jsch-0.1.42.jar:/usr/lib/hadoop-0.20/lib/junit-4.5.jar:/usr/lib/hadoop-0.20/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20/lib/log4j-1.2.15.jar:/usr/lib/hadoop-0.20/lib/mockito-all-1.8.2.jar:/usr/lib/hadoop-0.20/lib/oro-2.0.8.jar:/usr/lib/hadoop-0.20/lib/servlet-api-2.5-20081211.jar:/usr/lib/hadoop-0.20/lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hadoop-0.20/lib/slf4j-api-1.4.3.jar:/usr/lib/hadoop-0.20/lib/slf4j-log4j12-1.4.3.jar:/usr/lib/hadoop-0.20/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20/lib/jsp-2.1/jsp-2.1.jar:/usr/lib/hadoop-0.20/lib/jsp-2.1/jsp-api-2.1.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../conf:/etc/zookeeper::/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../lib/ant-contrib-1.0b3.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../lib/ant-eclipse-1.0-jvm1.2.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../lib/avro-1.5.3.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../lib/avro-ipc-1.5.3.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../lib/avro-mapred-1.5.3.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../lib/commons-io-1.4.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../lib/hsqldb-1.8.0.10.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../lib/jackson-core-asl-1.7.3.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../lib/jackson-mapper-asl-1.7.3.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../lib/jopt-simple-3.2.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../lib/paranamer-2.3.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../lib/postgresql-9.1-902.jdbc3.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../lib/snappy-java-1.0.3.2.jar:/usr/lib/hbase/bin/../conf:/usr/lib/jvm/java-6-sun-1.6.0.24/lib/tools.jar:/usr/lib/hbase/bin/..:/usr/lib/hbase/bin/../hbase-0.90.4-cdh3u3.jar:/usr/lib/hbase/bin/../hbase-0.90.4-cdh3u3-tests.jar:/usr/lib/hbase/bin/../lib/activation-1.1.jar:/usr/lib/hbase/bin/../lib/asm-3.1.jar:/usr/lib/hbase/bin/../lib/avro-1.5.4.jar:/usr/lib/hbase/bin/../lib/avro-ipc-1.5.4.jar:/usr/lib/hbase/bin/../lib/commons-cli-1.2.jar:/usr/lib/hbase/bin/../lib/commons-codec-1.4.jar:/usr/lib/hbase/bin/../lib/commons-el-1.0.jar:/usr/lib/hbase/bin/../lib/commons-httpclient-3.1.jar:/usr/lib/hbase/bin/../lib/commons-lang-2.5.jar:/usr/lib/hbase/bin/../lib/commons-logging-1.1.1.jar:/usr/lib/hbase/bin/../lib/commons-net-1.4.1.jar:/usr/lib/hbase/bin/../lib/core-3.1.1.jar:/usr/lib/hbase/bin/../lib/guava-r06.jar:/usr/lib/hbase/bin/../lib/guava-r09-jarjar.jar:/usr/lib/hbase/bin/../lib/hadoop-core.jar:/usr/lib/hbase/bin/../lib/jackson-core-asl-1.5.2.jar:/usr/lib/hbase/bin/../lib/jackson-jaxrs-1.5.5.jar:/usr/lib/hbase/bin/../lib/jackson-mapper-asl-1.5.2.jar:/usr/lib/hbase/bin/../lib/jackson-xc-1.5.5.jar:/usr/lib/hbase/bin/../lib/jamon-runtime-2.3.1.jar:/usr/lib/hbase/bin/../lib/jasper-compiler-5.5.23.jar:/usr/lib/hbase/bin/../lib/jasper-runtime-5.5.23.jar:/usr/lib/hbase/bin/../lib/jaxb-api-2.1.jar:/usr/lib/hbase/bin/../lib/jaxb-impl-2.1.12.jar:/usr/lib/hbase/bin/../lib/jersey-core-1.4.jar:/usr/lib/hbase/bin/../lib/jersey-json-1.4.jar:/usr/lib/hbase/bin/../lib/jersey-server-1.4.jar:/usr/lib/hbase/bin/../lib/jettison-1.1.jar:/usr/lib/hbase/bin/../lib/jetty-6.1.26.jar:/usr/lib/hbase/bin/../lib/jetty-util-6.1.26.jar:/usr/lib/hbase/bin/../lib/jruby-complete-1.6.0.jar:/usr/lib/hbase/bin/../lib/jsp-2.1-6.1.14.jar:/usr/lib/hbase/bin/../lib/jsp-api-2.1-6.1.14.jar:/usr/lib/hbase/bin/../lib/jsp-api-2.1.jar:/usr/lib/hbase/bin/../lib/jsr311-api-1.1.1.jar:/usr/lib/hbase/bin/../lib/log4j-1.2.16.jar:/usr/lib/hbase/bin/../lib/netty-3.2.4.Final.jar:/usr/lib/hbase/bin/../lib/protobuf-java-2.3.0.jar:/usr/lib/hbase/bin/../lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hbase/bin/../lib/servlet-api-2.5.jar:/usr/lib/hbase/bin/../lib/slf4j-api-1.5.8.jar:/usr/lib/hbase/bin/../lib/slf4j-log4j12-1.5.8.jar:/usr/lib/hbase/bin/../lib/snappy-java-1.0.3.2.jar:/usr/lib/hbase/bin/../lib/stax-api-1.0.1.jar:/usr/lib/hbase/bin/../lib/thrift-0.2.0.jar:/usr/lib/hbase/bin/../lib/velocity-1.5.jar:/usr/lib/hbase/bin/../lib/xmlenc-0.52.jar:/usr/lib/hbase/bin/../lib/zookeeper.jar:/usr/lib/hadoop/conf/:/usr/lib/hadoop-0.20/hadoop-core-0.20.2-cdh3u3.jar:/usr/lib/hadoop-0.20/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop-0.20/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20/lib/commons-daemon-1.0.1.jar:/usr/lib/hadoop-0.20/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-0.20/lib/commons-lang-2.4.jar:/usr/lib/hadoop-0.20/lib/commons-logging-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-logging-api-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-net-1.4.1.jar:/usr/lib/hadoop-0.20/lib/core-3.1.1.jar:/usr/lib/hadoop-0.20/lib/guava-r09-jarjar.jar:/usr/lib/hadoop-0.20/lib/hadoop-capacity-scheduler-0.20.2-cdh3u3.jar:/usr/lib/hadoop-0.20/lib/hadoop-fairscheduler-0.20.2-cdh3u3.jar:/usr/lib/hadoop-0.20/lib/hadoop-lzo-0.4.10.jar:/usr/lib/hadoop-0.20/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20/lib/jackson-core-asl-1.5.2.jar:/usr/lib/hadoop-0.20/lib/jackson-mapper-asl-1.5.2.jar:/usr/lib/hadoop-0.20/lib/jasper-compiler-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jasper-runtime-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20/lib/jetty-6.1.26.cloudera.1.jar:/usr/lib/hadoop-0.20/lib/jetty-servlet-tester-6.1.26.cloudera.1.jar:/usr/lib/hadoop-0.20/lib/jetty-util-6.1.26.cloudera.1.jar:/usr/lib/hadoop-0.20/lib/jsch-0.1.42.jar:/usr/lib/hadoop-0.20/lib/junit-4.5.jar:/usr/lib/hadoop-0.20/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20/lib/log4j-1.2.15.jar:/usr/lib/hadoop-0.20/lib/mockito-all-1.8.2.jar:/usr/lib/hadoop-0.20/lib/oro-2.0.8.jar:/usr/lib/hadoop-0.20/lib/servlet-api-2.5-20081211.jar:/usr/lib/hadoop-0.20/lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hadoop-0.20/lib/slf4j-api-1.4.3.jar:/usr/lib/hadoop-0.20/lib/slf4j-log4j12-1.4.3.jar:/usr/lib/hadoop-0.20/lib/xmlenc-0.52.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../sqoop-1.4.2.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/bin/../sqoop-test-1.4.2.jar::/usr/lib/hadoop-0.20/hadoop-0.20.2-cdh3u3-core.jar:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/sqoop-1.4.2.jar
Note: /tmp/sqoop-adarsh/compile/70af23436d0ed0f9d7f1b6bb713f1227/daily.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
12/09/14 04:09:54 DEBUG orm.CompilationManager: Could not rename
/tmp/sqoop-adarsh/compile/70af23436d0ed0f9d7f1b6bb713f1227/daily.java to
/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/./daily.java
java.io.IOException: Destination
'/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/./daily.java' already exists
at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:1811)
at
org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:227)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:64)
at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:97)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
12/09/14 04:09:54 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-adarsh/compile/70af23436d0ed0f9d7f1b6bb713f1227/daily.jar
12/09/14 04:09:54 DEBUG orm.CompilationManager: Scanning for .class files in
directory: /tmp/sqoop-adarsh/compile/70af23436d0ed0f9d7f1b6bb713f1227
12/09/14 04:09:54 DEBUG orm.CompilationManager: Got classfile:
/tmp/sqoop-adarsh/compile/70af23436d0ed0f9d7f1b6bb713f1227/daily.class ->
daily.class
12/09/14 04:09:54 DEBUG orm.CompilationManager: Finished writing jar file
/tmp/sqoop-adarsh/compile/70af23436d0ed0f9d7f1b6bb713f1227/daily.jar
12/09/14 04:09:54 INFO mapreduce.ExportJobBase: Data will be staged in the
table: daily_tmp
12/09/14 04:09:54 INFO mapreduce.ExportJobBase: Beginning export of daily
12/09/14 04:09:54 INFO manager.SqlManager: Deleted 0 records from daily_tmp
12/09/14 04:09:54 INFO security.UserGroupInformation: JAAS Configuration
already set up for Hadoop, not re-installing.
12/09/14 04:09:54 DEBUG mapreduce.JobBase: Using InputFormat: class
org.apache.sqoop.mapreduce.ExportInputFormat
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/sqoop-1.4.2.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/lib/postgresql-9.1-902.jdbc3.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/sqoop-1.4.2.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/sqoop-1.4.2.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/lib/jopt-simple-3.2.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/lib/ant-eclipse-1.0-jvm1.2.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/lib/paranamer-2.3.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/lib/commons-io-1.4.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/lib/jackson-mapper-asl-1.7.3.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/lib/avro-ipc-1.5.3.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/lib/avro-1.5.3.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/lib/ant-contrib-1.0b3.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/lib/hsqldb-1.8.0.10.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/lib/avro-mapred-1.5.3.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/lib/postgresql-9.1-902.jdbc3.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/lib/jackson-core-asl-1.7.3.jar
12/09/14 04:09:55 DEBUG mapreduce.JobBase: Adding to job classpath:
file:/home/adarsh/sqoop-1.4.2.bin__hadoop-0.20/lib/snappy-java-1.0.3.2.jar
12/09/14 04:09:56 INFO input.FileInputFormat: Total input paths to process : 1
12/09/14 04:09:56 DEBUG mapreduce.ExportInputFormat: Target numMapTasks=4
12/09/14 04:09:56 DEBUG mapreduce.ExportInputFormat: Total input bytes=414957434
12/09/14 04:09:56 DEBUG mapreduce.ExportInputFormat: maxSplitSize=103739358
12/09/14 04:09:56 INFO input.FileInputFormat: Total input paths to process : 1
12/09/14 04:09:56 DEBUG mapreduce.ExportInputFormat: Generated splits:
12/09/14 04:09:56 DEBUG mapreduce.ExportInputFormat:
Paths:/data/data.2012-09-08-00.csv:0+134217728 Locations:localhost:;
12/09/14 04:09:56 DEBUG mapreduce.ExportInputFormat:
Paths:/data/data.2012-09-08-00.csv:134217728+134217728 Locations:localhost:;
12/09/14 04:09:56 DEBUG mapreduce.ExportInputFormat:
Paths:/data/data.2012-09-08-00.csv:268435456+134217728 Locations:10.1.2.3.4:;
12/09/14 04:09:56 INFO mapred.JobClient: Running job: job_201209050808_49604
12/09/14 04:09:57 INFO mapred.JobClient: map 0% reduce 0%
12/09/14 04:10:16 INFO mapred.JobClient: map 14% reduce 0%
12/09/14 04:10:19 INFO mapred.JobClient: map 23% reduce 0%
12/09/14 04:10:22 INFO mapred.JobClient: map 31% reduce 0%
12/09/14 04:10:23 INFO mapred.JobClient: map 32% reduce 0%
12/09/14 04:10:25 INFO mapred.JobClient: map 33% reduce 0%
12/09/14 04:10:26 INFO mapred.JobClient: map 34% reduce 0%
12/09/14 04:10:29 INFO mapred.JobClient: map 36% reduce 0%
12/09/14 04:10:31 INFO mapred.JobClient: map 37% reduce 0%
12/09/14 04:10:32 INFO mapred.JobClient: map 38% reduce 0%
12/09/14 04:10:34 INFO mapred.JobClient: map 39% reduce 0%
12/09/14 04:10:35 INFO mapred.JobClient: map 40% reduce 0%
12/09/14 04:10:37 INFO mapred.JobClient: map 41% reduce 0%
12/09/14 04:10:38 INFO mapred.JobClient: map 42% reduce 0%
12/09/14 04:10:41 INFO mapred.JobClient: map 44% reduce 0%
12/09/14 04:10:44 INFO mapred.JobClient: map 46% reduce 0%
12/09/14 04:10:47 INFO mapred.JobClient: map 48% reduce 0%
12/09/14 04:10:50 INFO mapred.JobClient: map 50% reduce 0%
12/09/14 04:10:53 INFO mapred.JobClient: map 52% reduce 0%
12/09/14 04:10:56 INFO mapred.JobClient: map 55% reduce 0%
12/09/14 04:10:59 INFO mapred.JobClient: map 57% reduce 0%
12/09/14 04:11:02 INFO mapred.JobClient: map 59% reduce 0%
12/09/14 04:11:05 INFO mapred.JobClient: map 61% reduce 0%
12/09/14 04:11:08 INFO mapred.JobClient: map 63% reduce 0%
12/09/14 04:11:11 INFO mapred.JobClient: map 65% reduce 0%
12/09/14 04:11:14 INFO mapred.JobClient: map 67% reduce 0%
12/09/14 04:11:17 INFO mapred.JobClient: map 68% reduce 0%
12/09/14 04:11:18 INFO mapred.JobClient: map 69% reduce 0%
12/09/14 04:11:20 INFO mapred.JobClient: map 70% reduce 0%
12/09/14 04:11:21 INFO mapred.JobClient: map 71% reduce 0%
12/09/14 04:11:23 INFO mapred.JobClient: map 72% reduce 0%
12/09/14 04:11:24 INFO mapred.JobClient: map 74% reduce 0%
12/09/14 04:11:27 INFO mapred.JobClient: map 59% reduce 0%
12/09/14 04:11:29 INFO mapred.JobClient: map 60% reduce 0%
12/09/14 04:11:30 INFO mapred.JobClient: map 61% reduce 0%
12/09/14 04:11:33 INFO mapred.JobClient: map 62% reduce 0%
12/09/14 04:11:35 INFO mapred.JobClient: map 63% reduce 0%
12/09/14 04:11:36 INFO mapred.JobClient: map 65% reduce 0%
12/09/14 04:11:39 INFO mapred.JobClient: map 67% reduce 0%
12/09/14 04:11:42 INFO mapred.JobClient: map 69% reduce 0%
12/09/14 04:11:44 INFO mapred.JobClient: map 70% reduce 0%
12/09/14 04:11:45 INFO mapred.JobClient: map 71% reduce 0%
12/09/14 04:11:48 INFO mapred.JobClient: map 73% reduce 0%
12/09/14 04:11:51 INFO mapred.JobClient: map 75% reduce 0%
12/09/14 04:11:54 INFO mapred.JobClient: map 77% reduce 0%
12/09/14 04:11:57 INFO mapred.JobClient: map 79% reduce 0%
12/09/14 04:12:00 INFO mapred.JobClient: map 80% reduce 0%
12/09/14 04:12:01 INFO mapred.JobClient: map 75% reduce 0%
12/09/14 04:12:12 INFO mapred.JobClient: map 76% reduce 0%
12/09/14 04:12:15 INFO mapred.JobClient: map 77% reduce 0%
12/09/14 04:12:21 INFO mapred.JobClient: map 78% reduce 0%
12/09/14 04:12:24 INFO mapred.JobClient: map 79% reduce 0%
12/09/14 04:12:30 INFO mapred.JobClient: map 80% reduce 0%
12/09/14 04:12:33 INFO mapred.JobClient: map 81% reduce 0%
12/09/14 04:12:36 INFO mapred.JobClient: map 82% reduce 0%
12/09/14 04:12:42 INFO mapred.JobClient: map 83% reduce 0%
12/09/14 04:12:45 INFO mapred.JobClient: map 84% reduce 0%
12/09/14 04:12:51 INFO mapred.JobClient: map 85% reduce 0%
12/09/14 04:12:54 INFO mapred.JobClient: map 86% reduce 0%
12/09/14 04:13:01 INFO mapred.JobClient: map 87% reduce 0%
12/09/14 04:13:04 INFO mapred.JobClient: map 88% reduce 0%
12/09/14 04:13:07 INFO mapred.JobClient: map 89% reduce 0%
12/09/14 04:13:13 INFO mapred.JobClient: map 90% reduce 0%
12/09/14 04:13:16 INFO mapred.JobClient: map 91% reduce 0%
12/09/14 04:13:22 INFO mapred.JobClient: map 92% reduce 0%
12/09/14 04:13:25 INFO mapred.JobClient: map 93% reduce 0%
12/09/14 04:13:28 INFO mapred.JobClient: map 94% reduce 0%
12/09/14 04:13:34 INFO mapred.JobClient: map 95% reduce 0%
12/09/14 04:13:37 INFO mapred.JobClient: map 96% reduce 0%
12/09/14 04:13:43 INFO mapred.JobClient: map 97% reduce 0%
12/09/14 04:13:46 INFO mapred.JobClient: map 98% reduce 0%
12/09/14 04:13:49 INFO mapred.JobClient: map 99% reduce 0%
12/09/14 04:13:52 INFO mapred.JobClient: map 100% reduce 0%
12/09/14 04:13:52 INFO mapred.JobClient: Job complete: job_201209050808_49604
12/09/14 04:13:52 INFO mapred.JobClient: Counters: 16
12/09/14 04:13:52 INFO mapred.JobClient: Job Counters
12/09/14 04:13:52 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=346917
12/09/14 04:13:52 INFO mapred.JobClient: Total time spent by all reduces
waiting after reserving slots (ms)=0
12/09/14 04:13:52 INFO mapred.JobClient: Total time spent by all maps
waiting after reserving slots (ms)=0
12/09/14 04:13:52 INFO mapred.JobClient: Rack-local map tasks=6
12/09/14 04:13:52 INFO mapred.JobClient: Launched map tasks=6
12/09/14 04:13:52 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=1483
12/09/14 04:13:52 INFO mapred.JobClient: FileSystemCounters
12/09/14 04:13:52 INFO mapred.JobClient: HDFS_BYTES_READ=415351349
12/09/14 04:13:52 INFO mapred.JobClient: FILE_BYTES_WRITTEN=243012
12/09/14 04:13:52 INFO mapred.JobClient: Map-Reduce Framework
12/09/14 04:13:52 INFO mapred.JobClient: Map input records=4071315
12/09/14 04:13:52 INFO mapred.JobClient: Physical memory (bytes)
snapshot=2018009088
12/09/14 04:13:52 INFO mapred.JobClient: Spilled Records=0
12/09/14 04:13:52 INFO mapred.JobClient: CPU time spent (ms)=240740
12/09/14 04:13:52 INFO mapred.JobClient: Total committed heap usage
(bytes)=1919221760
12/09/14 04:13:52 INFO mapred.JobClient: Virtual memory (bytes)
snapshot=5843054592
12/09/14 04:13:52 INFO mapred.JobClient: Map output records=4071315
12/09/14 04:13:52 INFO mapred.JobClient: SPLIT_RAW_BYTES=672
12/09/14 04:13:52 INFO mapreduce.ExportJobBase: Transferred 396.1099 MB in
237.2008 seconds (1.6699 MB/sec)
12/09/14 04:13:52 INFO mapreduce.ExportJobBase: Exported 4071315 records.
12/09/14 04:13:52 INFO mapreduce.ExportJobBase: Starting to migrate data from
staging table to destination.
12/09/14 04:14:29 INFO manager.SqlManager: Migrated 5321315 records from
daily_tmp to daily
adarsh@gs1002:~/sqoop-1.4.2.bin__hadoop-0.20$
