Hi Han, would you mind sharing entire Sqoop log generated with parameter --verbose as an attachment? It might be also beneficial to share the map task log.
Oozie is not using bash nor any other shell to execute Sqoop, so you should
remove any escaping introduced especially for shell. Looking at your example, I
think that it should be something like:
... --innput-fields-terminated-by | --input-null-string \\N ...
Jarcec
On Tue, May 21, 2013 at 05:56:16AM -0700, Han Sen Tey wrote:
> Greeting experts,
>
> I have a table "CHRYSLER_Anametrix_Aggregated" in Hive and in MySQL.
> Similarly I can find this table in hdfs --
> /user/hive/warehouse/chrysler_anametrix_aggregated.
>
>
> I can successfully execute the following SQOOP command from namenode CLI -
>
> sqoop export --verbose --connect
> jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table
> CHRYSLER_Anametrix_Aggregated --update-key
> VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession
> --update-mode allowinsert --export-dir
> /user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1
> --input-fields-terminated-by '|' --input-null-string '\\N'
> --input-null-non-string '\\N'
>
> However, when I want to run the above as part of Oozie workflow action ...
>
>
> <action name="sqoop-node">
> <sqoop xmlns="uri:oozie:sqoop-action:0.2">
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
>
> <configuration>
> <property>
> <name>sqoop.connection.factories</name>
>
> <value>com.cloudera.sqoop.manager.DefaultManagerFactory</value>
> </property>
> <property>
>
> <name>mapred.job.queue.name</name>
> <value>${queueName}</value>
> </property>
> <property>
> <name>mapred.compress.map.output</name>
> <value>true</value>
> </property>
>
> <property>
>
> <name>oozie.service.WorkflowAppService.system.libpath</name>
> <value>${nameNode}/user/oozie/share/lib/sqoop</value>
> </property>
> </configuration>
>
> <command>export --verbose --connect
> jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table
> CHRYSLER_Anametrix_Aggregated --update-key
> VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession
> --update-mode allowinsert --export-dir
> /user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1
> --input-fields-terminated-by '|' --input-null-string '\\N'
> --input-null-non-string '\\N'</command>
> </sqoop>
> <ok to="end"/>
> <error to="fail"/>
> </action>
>
> I get the following error message. I have
> "mysql-connector-java.5.1.24-bin.jar" shared library -
> /user/oozie/share/lib/sqoop.
>
> Hadoop version - 2.00 - CDH4.2.0
> Sqoop version - 1.4.2. - CDH4.2.0
>
>
> Can you experts please guide me ... a rookie to Hadoop and its projects.
>
> 2013-05-21 00:48:35,135 WARN mapreduce.Counters: Group
> org.apache.hadoop.mapred.Task$Counter is deprecated. Use
> org.apache.hadoop.mapreduce.TaskCounter instead 2013-05-21 00:48:37,394 WARN
> org.apache.hadoop.conf.Configuration: session.id is deprecated. Instead, use
> dfs.metrics.session-id 2013-05-21 00:48:37,408 INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with
> processName=MAP, sessionId= 2013-05-21 00:48:38,042 INFO
> org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0 2013-05-21
> 00:48:38,058 INFO org.apache.hadoop.mapred.Task: Using
> ResourceCalculatorPlugin :
> org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5dce1bea 2013-05-21
> 00:48:38,423 INFO org.apache.hadoop.mapred.MapTask: Processing split:
> hdfs://ip-10-147-191-48.ec2.internal:8020/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/input/dummy.txt:0+5
> 2013-05-21 00:48:38,453 WARN mapreduce.Counters: Counter name MAP_INPUT_BYTES
> is deprecated. Use FileInputFormatCounters as group name and BYTES_READ as
> counter name instead 2013-05-21 00:48:38,459 INFO
> org.apache.hadoop.mapred.MapTask: numReduceTasks: 0 2013-05-21 00:48:39,743
> WARN org.apache.sqoop.tool.SqoopTool: $SQOOP_CONF_DIR has not been set in the
> environment. Cannot check for additional configuration. 2013-05-21
> 00:48:40,032 DEBUG org.apache.sqoop.tool.BaseSqoopTool: Enabled debug
> logging. 2013-05-21 00:48:40,036 WARN org.apache.sqoop.SqoopOptions:
> Character argument '|' has multiple characters; only the first will be used.
> 2013-05-21 00:48:40,084 DEBUG org.apache.sqoop.ConnFactory:
> sqoop.connection.factories is set; ignoring managers.d 2013-05-21
> 00:48:40,139 DEBUG org.apache.sqoop.ConnFactory: Loaded manager factory:
> com.cloudera.sqoop.manager.DefaultManagerFactory 2013-05-21 00:48:40,139
> DEBUG org.apache.sqoop.ConnFactory: Trying ManagerFactory:
> com.cloudera.sqoop.manager.DefaultManagerFactory 2013-05-21 00:48:40,140
> DEBUG org.apache.sqoop.manager.DefaultManagerFactory: Trying with scheme:
> jdbc:mysql: 2013-05-21 00:48:40,183 INFO
> org.apache.sqoop.manager.MySQLManager: Preparing to use a MySQL streaming
> resultset. 2013-05-21 00:48:40,183 DEBUG org.apache.sqoop.ConnFactory:
> Instantiated ConnManager org.apache.sqoop.manager.MySQLManager@795e0c2b
> 2013-05-21 00:48:40,184 INFO org.apache.sqoop.tool.CodeGenTool: Beginning
> code generation 2013-05-21 00:48:40,233 DEBUG
> org.apache.sqoop.manager.SqlManager: No connection paramenters specified.
> Using regular API for making connection. 2013-05-21 00:48:40,911 DEBUG
> org.apache.sqoop.manager.SqlManager: Using fetchSize for next query:
> -2147483648 2013-05-21 00:48:40,911 INFO org.apache.sqoop.manager.SqlManager:
> Executing SQL statement: SELECT t.* FROM `CHRYSLER_Anametrix_Aggregated` AS t
> LIMIT 1 2013-05-21 00:48:40,930 DEBUG org.apache.sqoop.orm.ClassWriter:
> selected columns: 2013-05-21 00:48:40,931 DEBUG
> org.apache.sqoop.orm.ClassWriter: VisitorID 2013-05-21 00:48:40,931 DEBUG
> org.apache.sqoop.orm.ClassWriter: VisitorDay 2013-05-21 00:48:40,931 DEBUG
> org.apache.sqoop.orm.ClassWriter: VisitorMonth 2013-05-21 00:48:40,931
> DEBUG org.apache.sqoop.orm.ClassWriter: VisitorYear 2013-05-21 00:48:40,931
> DEBUG org.apache.sqoop.orm.ClassWriter: VisitorSession 2013-05-21
> 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: Brand 2013-05-21
> 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: DeviceType 2013-05-21
> 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: DeviceBrowser
> 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:
> TrafficSource 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:
> SiteID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:
> BuyID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:
> AdvertisementID 2013-05-21 00:48:40,931 DEBUG
> org.apache.sqoop.orm.ClassWriter:
> PlacementID 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:
> CreativeID 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:
> PaidKeyword 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:
> Model200 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:
> Model200Conv 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:
> Model300 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:
> Model300TownCountry 2013-05-21 00:48:40,934 DEBUG
> org.apache.sqoop.orm.ClassWriter: BMOLineup 2013-05-21 00:48:40,935 DEBUG
> org.apache.sqoop.orm.ClassWriter: BMOModels 2013-05-21 00:48:40,935 DEBUG
> org.apache.sqoop.orm.ClassWriter: BMOColors 2013-05-21 00:48:40,935 DEBUG
> org.apache.sqoop.orm.ClassWriter: BMOOptions 2013-05-21 00:48:40,935 DEBUG
> org.apache.sqoop.orm.ClassWriter: BMOSummary 2013-05-21 00:48:40,935 DEBUG
> org.apache.sqoop.orm.ClassWriter: BrochureLineup 2013-05-21
> 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureForm
> 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:
> BrochureMailLink 2013-05-21 00:48:40,935 DEBUG
> org.apache.sqoop.orm.ClassWriter: BrochureDownloadLink 2013-05-21
> 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: PreDL 2013-05-21
> 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DealerLocate
> 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DLHours
> 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DLServices
> 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:
> DLDealerSite 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:
> PreSNI 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:
> SearchNewInventory 2013-05-21 00:48:40,936 DEBUG
> org.apache.sqoop.orm.ClassWriter: SNIVIN 2013-05-21 00:48:40,936 DEBUG
> org.apache.sqoop.orm.ClassWriter: SNIDetails 2013-05-21 00:48:40,936 DEBUG
> org.apache.sqoop.orm.ClassWriter: SNIDealer 2013-05-21 00:48:40,936 DEBUG
> org.apache.sqoop.orm.ClassWriter: TestDriveForm 2013-05-21 00:48:40,936
> DEBUG org.apache.sqoop.orm.ClassWriter: TestDriveComplete 2013-05-21
> 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: Inventives 2013-05-21
> 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PreCal 2013-05-21
> 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PaymentCalculator
> 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PreGAQ
> 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: GetAQuote
> 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: LandingPage
> 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: Models
> 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:
> PhotosGallery 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:
> Interior 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:
> Exterior 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:
> Performance 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter:
> Safety 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter:
> Specifications 2013-05-21 00:48:40,938 DEBUG
> org.apache.sqoop.orm.ClassWriter: ModelHighlights 2013-05-21 00:48:40,938
> DEBUG org.apache.sqoop.orm.ClassWriter: Dimensions 2013-05-21 00:48:40,938
> DEBUG org.apache.sqoop.orm.ClassWriter: Warranty 2013-05-21 00:48:40,965
> DEBUG org.apache.sqoop.manager.SqlManager: Using fetchSize for next query:
> -2147483648 2013-05-21 00:48:40,965 INFO org.apache.sqoop.manager.SqlManager:
> Executing SQL statement: SELECT t.* FROM `CHRYSLER_Anametrix_Aggregated` AS t
> LIMIT 1 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter:
> Writing source file:
> /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.java
> 2013-05-21 00:48:40,990 DEBUG
> org.apache.sqoop.orm.ClassWriter: Table name: CHRYSLER_Anametrix_Aggregated
> 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter: Columns:
> VisitorID:12, VisitorDay:12, VisitorMonth:12, VisitorYear:12,
> VisitorSession:4, Brand:12, DeviceType:12, DeviceBrowser:12,
> TrafficSource:12, SiteID:12, BuyID:12, AdvertisementID:12, PlacementID:12,
> CreativeID:12, PaidKeyword:12, Model200:4, Model200Conv:4, Model300:4,
> Model300TownCountry:4, BMOLineup:4, BMOModels:4, BMOColors:4, BMOOptions:4,
> BMOSummary:4, BrochureLineup:4, BrochureForm:4, BrochureMailLink:4,
> BrochureDownloadLink:4, PreDL:4, DealerLocate:4, DLHours:4, DLServices:4,
> DLDealerSite:4, PreSNI:4, SearchNewInventory:4, SNIVIN:4, SNIDetails:4,
> SNIDealer:4, TestDriveForm:4, TestDriveComplete:4, Inventives:4, PreCal:4,
> PaymentCalculator:4, PreGAQ:4, GetAQuote:4, LandingPage:4, Models:4,
> PhotosGallery:4, Interior:4, Exterior:4, Performance:4, Safety:4,
> Specifications:4, ModelHighlights:4,
> Dimensions:4, Warranty:4, 2013-05-21 00:48:40,990 DEBUG
> org.apache.sqoop.orm.ClassWriter: sourceFilename is
> CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:40,991 DEBUG
> org.apache.sqoop.orm.CompilationManager: Found existing
> /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21
> 00:48:40,992 INFO org.apache.sqoop.orm.CompilationManager:
> $HADOOP_MAPRED_HOME is not set 2013-05-21 00:48:41,098 DEBUG
> org.apache.sqoop.orm.CompilationManager: Adding source file:
> /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.java
> 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager:
> Invoking javac with args: 2013-05-21 00:48:41,098 DEBUG
> org.apache.sqoop.orm.CompilationManager: -sourcepath 2013-05-21
> 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager:
> /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21
> 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: -d
> 2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager:
> /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21
> 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager: -classpath
> 2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager:
>
> /run/cloudera-scm-agent/process/76-mapreduce-TASKTRACKER:/usr/lib/jvm/j2sdk1.6-oracle/lib/tools.jar:/usr/lib/hadoop-0.20-mapreduce:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/activation-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-1.7.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-compiler-1.7.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-configuration-1.6.jar:/usr/lib/hadoop-
> 0.20-mapreduce/lib/commons-digester-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-lang-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-math-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-net-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hadoop-fairscheduler-2.0.0-mr1-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/j
> asper-runtime-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-core-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-json-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-server-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jettison-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jline-0.9.94.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsch-0.1.42.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-0.20-mapreduce/lib/junit-4.8.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-0.20-map
> reduce/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-0.20-mapreduce/lib/servlet-api-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/stax-api-1.0.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20-mapreduce/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-api-2.1.jar:/usr/share/cmf/lib/plugins/navigator-plugin-4.5.1-shaded.jar:/usr/share/cmf/lib/plugins/event-publish-4.5.1-shaded.jar:/usr/share/cmf/lib/plugins/tt-instrumentation-4.5.1.jar:/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.2.jar:
> /usr/lib/hadoop-hdfs/lib/jersey-core-1.8.jar:/usr/lib/hadoop-hdfs/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.1.jar:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.5.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.8.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/
> usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.2.0-tests.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/avro-1.7.3.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jersey-core-1.8.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/jsr305-1.3.9.jar:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-io-2.1.jar:/usr/lib/hadoop/lib/jline-0.9.94.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/commons-c
> onfiguration-1.6.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop/lib/commons-math-2.1.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/stax-api-1.0.1.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/commons-lang-2.5.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/junit-4.8.2.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hado
> op/lib/hue-plugins-2.2.0-cdh4.2.0.jar:/usr/lib/hadoop/lib/kfs-0.3.jar:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/jersey-server-1.8.jar:/usr/lib/hadoop/lib/jersey-json-1.8.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-auth-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.2.0-tests.jar:/usr/lib/hadoop/hadoop-annotations-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-common.jar:/mnt/mapred/local/taskTracker/distcache/-3359360379488511723_1277082233_1150217957/ip-10-147-191-48.ec2.internal/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/sqoop-launcher.jar:/mnt/mapred/local/taskTracker/distcache/-3926870669093555941_811939984_1868217431/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoo
> p/avro-ipc-1.7.3-tests.jar:/mnt/mapred/local/taskTracker/distcache/4298483340499219542_2138859044_1868217453/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/avro-ipc-1.7.3.jar:/mnt/mapred/local/taskTracker/distcache/-1975491928978283532_-1185784129_1868217475/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/avro-mapred-1.7.3.jar:/mnt/mapred/local/taskTracker/distcache/8448357694523829347_-1491317642_1868217500/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-beanutils-1.7.0.jar:/mnt/mapred/local/taskTracker/distcache/-334079317866666788_-232552041_1868217523/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-beanutils-core-1.8.0.jar:/mnt/mapred/local/taskTracker/distcache/-644448586073609072_-372575484_1868217547/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-configuration-1.6.jar:/mnt/mapred/local/taskTracker/distcache/246054495199880993_357051411_1868217589/ip-10-147-191-48.ec2.internal/
> user/oozie/share/lib/sqoop/commons-digester-1.8.jar:/mnt/mapred/local/taskTracker/distcache/-4144140590487253933_766503300_1868217614/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-io-2.1.jar:/mnt/mapred/local/taskTracker/distcache/5753546228468760323_-553874526_1868217665/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/guava-11.0.2.jar:/mnt/mapred/local/taskTracker/distcache/8662866019753074519_-1003108238_1868217924/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/hbase-0.94.2-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-4840377369495417802_845918870_1868217954/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/high-scale-lib-1.1.1.jar:/mnt/mapred/local/taskTracker/distcache/-142623586016294918_445946554_1868217996/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/hsqldb-1.8.0.7.jar:/mnt/mapred/local/taskTracker/distcache/5142583573507929188_-105788043_1868218025/ip-10-147-191-48.ec2.internal/us
> er/oozie/share/lib/sqoop/httpclient-4.1.3.jar:/mnt/mapred/local/taskTracker/distcache/-6413369533723682352_-180904991_1868218051/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/httpcore-4.1.3.jar:/mnt/mapred/local/taskTracker/distcache/3446550291828223401_633399508_1868218088/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/jsr305-1.3.9.jar:/mnt/mapred/local/taskTracker/distcache/-6492962697058022370_-616322509_1868218133/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/libthrift-0.9.0.jar:/mnt/mapred/local/taskTracker/distcache/7049151581973813525_2017354882_1868218156/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/metrics-core-2.1.2.jar:/mnt/mapred/local/taskTracker/distcache/623583253106481693_846556431_1142287025/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/mysql-connector-java-5.1.24-bin.jar:/mnt/mapred/local/taskTracker/distcache/-7982675208196719520_-1207761111_1868218197/ip-10-147-191-48.ec2.interna
> l/user/oozie/share/lib/sqoop/netty-3.4.0.Final.jar:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-412267918101123238_-517736257_1868215515/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/oozie/json-simple-1.1.jar:/mnt/mapred/local/taskTracker/ubuntu/jobcache/job_201305131705_0181/attempt_201305131705_0181_m_000000_0/work:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
> 2013-05-21 00:48:48,206 INFO org.apache.sqoop.orm.CompilationManager:
> Writing jar file:
> /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.jar
> 2013-05-21 00:48:48,210 DEBUG
> org.apache.sqoop.orm.CompilationManager: Scanning for .class files in
> directory: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca
> 2013-05-21 00:48:48,221 DEBUG org.apache.sqoop.orm.CompilationManager: Got
> classfile:
> /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.class
> -> CHRYSLER_Anametrix_Aggregated.class 2013-05-21 00:48:48,236 DEBUG
> org.apache.sqoop.orm.CompilationManager: Finished writing jar file
> /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.jar
> 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: MySQL
> Connector upsert functionality is using INSERT ON 2013-05-21 00:48:48,243
> WARN org.apache.sqoop.manager.MySQLManager: DUPLICATE KEY UPDATE clause that
> relies on table's unique key. 2013-05-21 00:48:48,243 WARN
> org.apache.sqoop.manager.MySQLManager: Insert/update distinction is therefore
> independent on column 2013-05-21 00:48:48,243 WARN
> org.apache.sqoop.manager.MySQLManager: names specified in --update-key
> parameter. Please see MySQL 2013-05-21 00:48:48,245 WARN
> org.apache.sqoop.manager.MySQLManager: documentation for additional
> limitations. 2013-05-21 00:48:48,290 INFO
> org.apache.sqoop.mapreduce.ExportJobBase: Beginning export of
> CHRYSLER_Anametrix_Aggregated 2013-05-21 00:48:48,319 DEBUG
> org.apache.sqoop.mapreduce.JobBase: Using InputFormat: class
> org.apache.sqoop.mapreduce.ExportInputFormat 2013-05-21 00:48:48,575 DEBUG
> org.apache.sqoop.mapreduce.JobBase: Adding to job classpath:
> file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
> 2013-05-21 00:48:48,581 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to
> job classpath:
>
> file:/mnt/mapred/local/taskTracker/distcache/623583253106481693_846556431_1142287025/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/mysql-connector-java-5.1.24-bin.jar
> 2013-05-21 00:48:48,585 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to
> job classpath:
> file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
> 2013-05-21 00:48:48,588 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to
> job classpath:
> file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
> 2013-05-21 00:48:48,588 WARN org.apache.sqoop.mapreduce.JobBase: SQOOP_HOME
> is unset. May not be able to find all job dependencies. 2013-05-21
> 00:48:48,739 WARN org.apache.hadoop.mapred.JobClient: Use
> GenericOptionsParser for parsing the arguments. Applications
> should implement Tool for the same. 2013-05-21 00:48:49,058 INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to
> process : 1 2013-05-21 00:48:49,059 DEBUG
> org.apache.sqoop.mapreduce.ExportInputFormat: Target numMapTasks=1 2013-05-21
> 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Total input
> bytes=8187428 2013-05-21 00:48:49,059 DEBUG
> org.apache.sqoop.mapreduce.ExportInputFormat: maxSplitSize=8187428 2013-05-21
> 00:48:49,063 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat:
> Total input paths to process : 1 2013-05-21 00:48:49,073 DEBUG
> org.apache.sqoop.mapreduce.ExportInputFormat: Generated splits: 2013-05-21
> 00:48:49,073 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat:
> Paths:/user/hive/warehouse/chrysler_anametrix_aggregated/part-AG-00000-1.txt:0+8187428
> Locations:ip-10-147-191-48.ec2.internal:; 2013-05-21 00:48:49,403 INFO
> org.apache.hadoop.mapred.JobClient: Running job: job_201305131705_0182
> 2013-05-21 00:48:50,417 INFO org.apache.hadoop.mapred.JobClient: map 0%
> reduce 0% 2013-05-21 00:49:06,478 INFO org.apache.hadoop.mapred.JobClient:
> Task Id : attempt_201305131705_0182_m_000000_0, Status : FAILED 2013-05-21
> 00:49:16,703 INFO org.apache.hadoop.mapred.JobClient: Task Id :
> attempt_201305131705_0182_m_000000_1, Status : FAILED 2013-05-21 00:49:25,861
> INFO org.apache.hadoop.mapred.JobClient: Task Id :
> attempt_201305131705_0182_m_000000_2, Status : FAILED 2013-05-21 00:49:42,012
> INFO org.apache.hadoop.mapred.JobClient: Job complete: job_201305131705_0182
> 2013-05-21 00:49:42,015 INFO org.apache.hadoop.mapred.JobClient: Counters: 7
> 2013-05-21 00:49:42,020 INFO org.apache.hadoop.mapred.JobClient: Job
> Counters 2013-05-21 00:49:42,023 INFO org.apache.hadoop.mapred.JobClient:
> Failed map tasks=1 2013-05-21 00:49:42,023 INFO
> org.apache.hadoop.mapred.JobClient: Launched map tasks=4 2013-05-21
> 00:49:42,023 INFO
> org.apache.hadoop.mapred.JobClient: Data-local map tasks=4 2013-05-21
> 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by
> all maps in occupied slots (ms)=50187 2013-05-21 00:49:42,024 INFO
> org.apache.hadoop.mapred.JobClient: Total time spent by all reduces in
> occupied slots (ms)=0 2013-05-21 00:49:42,024 INFO
> org.apache.hadoop.mapred.JobClient: Total time spent by all maps waiting
> after reserving slots (ms)=0 2013-05-21 00:49:42,024 INFO
> org.apache.hadoop.mapred.JobClient: Total time spent by all reduces
> waiting after reserving slots (ms)=0 2013-05-21 00:49:42,032 WARN
> mapreduce.Counters: Group FileSystemCounters is deprecated. Use
> org.apache.hadoop.mapreduce.FileSystemCounter instead 2013-05-21 00:49:42,033
> INFO org.apache.sqoop.mapreduce.ExportJobBase: Transferred 0 bytes in 53.4336
> seconds (0 bytes/sec) 2013-05-21 00:49:42,035 WARN mapreduce.Counters: Group
> org.apache.hadoop.mapred.Task$Counter is deprecated.
> Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-05-21 00:49:42,035
> INFO org.apache.sqoop.mapreduce.ExportJobBase: Exported 0 records. 2013-05-21
> 00:49:42,035 ERROR org.apache.sqoop.tool.ExportTool: Error during export:
> Export job failed! 2013-05-21 00:49:42,063 INFO
> org.apache.hadoop.mapred.Task: Task:attempt_201305131705_0181_m_000000_0 is
> done. And is in the process of commiting 2013-05-21 00:49:43,124 INFO
> org.apache.hadoop.mapred.Task: Task attempt_201305131705_0181_m_000000_0 is
> allowed to commit now 2013-05-21 00:49:43,292 INFO
> org.apache.hadoop.mapred.FileOutputCommitter: Saved output of task
> 'attempt_201305131705_0181_m_000000_0' to
> hdfs://ip-10-147-191-48.ec2.internal:8020/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/output
> 2013-05-21 00:49:43,300 INFO org.apache.hadoop.mapred.Task: Task
> 'attempt_201305131705_0181_m_000000_0' done. 2013-05-21 00:49:43,309 INFO
> org.apache.hadoop.mapred.TaskLogsTruncater:
> Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
>
> Regards,
> Andy Tey
> Student !!
signature.asc
Description: Digital signature
