I see, I'm glad that you were able to investigate the issue and resolve it on your end. I don't see a need to file a JIRA at the moment.
Happy New Year! Jarcec On Mon, Dec 31, 2012 at 11:03:56AM -0800, Mason wrote: > hi Jarcec, > > I dug deeper and it looks like a configuration error on my end > resulted in Hive throwing an error, which caused Sqoop to silently > abort the import, before updating incremental.last.value. I looked > around the source, but my Java is too rusty to figure out why Sqoop > didn't report failure after that error. > > Still want a JIRA? > > Logs are below. > > ### Sqoop CLI output ### > ### ...a bunch of prior output... ### > 12/12/27 10:57:38 INFO mapreduce.ImportJobBase: Transferred 9.1538 MB > in 49.4693 seconds (189.4804 KB/sec) > 12/12/27 10:57:38 INFO mapreduce.ImportJobBase: Retrieved 41969 records. > 12/12/27 10:57:38 INFO util.AppendUtils: Appending to directory dataset > 12/12/27 10:57:39 INFO manager.SqlManager: Executing SQL statement: > SELECT t.* FROM `dataset` AS t LIMIT 1 > 12/12/27 10:57:40 WARN hive.TableDefWriter: Column created_at had to > be cast to a less precise type in Hive > 12/12/27 10:57:40 WARN hive.TableDefWriter: Column updated_at had to > be cast to a less precise type in Hive > 12/12/27 10:57:40 INFO hive.HiveImport: Removing temporary files from > import process: hdfs://localhost:9000/user/mason/dataset/_logs > 12/12/27 10:57:40 INFO hive.HiveImport: Loading uploaded data into Hive > WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. > Please use org.apache.hadoop.log.metrics.EventCounter in all the > log4j.properties files. > Logging initialized using configuration in > jar:file:/usr/local/Cellar/sqoop/1.4.2/libexec/lib/hive-common-0.9.0.jar!/hive-log4j.properties > Hive history file=/tmp/mason/hive_job_log_mason_201212271057_2014979494.txt > OK > Time taken: 4.503 seconds > Loading data to table default.dataset > OK > Time taken: 0.656 seconds > ### end of output ### > > ### Hive error log ### > 2012-12-27 10:57:40,138 WARN conf.HiveConf > (HiveConf.java:<clinit>(70)) - hive-site.xml not found on CLASSPATH > 2012-12-27 10:57:41,635 ERROR DataNucleus.Plugin > (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires > "org.eclipse.core.resources" but it cannot be resolved. > 2012-12-27 10:57:41,635 ERROR DataNucleus.Plugin > (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires > "org.eclipse.core.resources" but it cannot be resolved. > 2012-12-27 10:57:41,636 ERROR DataNucleus.Plugin > (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires > "org.eclipse.core.runtime" but it cannot be resolved. > 2012-12-27 10:57:41,636 ERROR DataNucleus.Plugin > (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires > "org.eclipse.core.runtime" but it cannot be resolved. > 2012-12-27 10:57:41,637 ERROR DataNucleus.Plugin > (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires > "org.eclipse.text" but it cannot be resolved. > 2012-12-27 10:57:41,637 ERROR DataNucleus.Plugin > (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires > "org.eclipse.text" but it cannot be resolved. > 2012-12-27 10:57:46,030 ERROR tool.ImportTool > (ImportTool.java:run(484)) - Encountered IOException running import > job: java.io.IOException: Exception thrown in Hive > at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:335) > at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:226) > at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:415) > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476) > at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:228) > at org.apache.sqoop.tool.JobTool.run(JobTool.java:283) > at org.apache.sqoop.Sqoop.run(Sqoop.java:145) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) > at org.apache.sqoop.Sqoop.main(Sqoop.java:238) > at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:308) > ... 12 more > Caused by: ExitSecurityException > at > org.apache.sqoop.util.SubprocessSecurityManager.checkExit(SubprocessSecurityManager.java:83) > at java.lang.Runtime.exit(Runtime.java:105) > at java.lang.System.exit(System.java:960) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:558) > ... 17 more > ### end of stack trace ### > > On Fri, Dec 28, 2012 at 10:42 AM, Jarek Jarcec Cecho <[email protected]> > wrote: > > Hi Mason, > > that seems as a bug to me. Would you mind opening a new JIRA [1] for that? > > > > Jarcec > > > > Links: > > 1: https://issues.apache.org/jira/browse/SQOOP > > > > On Thu, Dec 27, 2012 at 12:46:31PM -0800, Mason wrote: > >> I'm using hive 0.9.0, hadoop 1.1.1, and sqoop 1.4.2. > >> > >> When I create a saved job to do an incremental import from a MySQL > >> server and import into Hive using "--hive-import", it appears that the > >> data is correctly imported, but that the incremental.last.value of the > >> job is not updated, so re-executing the job just imports the same > >> data. > >> > >> If I create a job that's identical in all respects, but leave off > >> "--hive-import", it finishes with "INFO tool.ImportTool: Saving > >> incremental import state to the metastore", as I'd expect, and does in > >> fact update the incremental.last.value. > >> > >> Is this behavior a bug? Or does Sqoop just not support incremental > >> imports into Hive? > >> > >> -Mason
signature.asc
Description: Digital signature
