Greg, In Sqoop 1.4.4, I've an issue when I import data from RDBMS to Hive too.
I use the following solution : 1) Import RDBMS to file with Sqoop into an HDFS directory 2) LOAD DATA statement to fill your Hive table. Good luck ! -- Laurent HATIER - Consultant Big Data & Business Intelligence chez CapGemini fr.linkedin.com/pub/laurent-hatier/25/36b/a86/ <http://fr.linkedin.com/pub/laurent-h/25/36b/a86/> 2014-12-02 22:25 GMT+01:00 Greg Sharek <[email protected]>: > Hi all, > > I’m attempting to create a workflow which uses sqoop to import data from > my postgres db into a hive table. My sqoop configuration runs fine outside > of Oozie. > > My main question: Does oozie support sqoop importing directly into hive? > I see some comments claiming it doesn’t, but I haven’t found anything > definitive in the Oozie docs. > > For the curious, here’s what I’m experiencing. I’d be happy to provide > more details if they would help. > > When I run my Oozie workflow, I see the following: > > * Sqoop is able to import data into HDFS under the directory specified > by —target-dir > * When Sqoop attempts to copy this to the Hive table, it gets a > permission error as it’s attempting to write to a table as user ‘yarn’ as > opposed to the executing user. The table is owned by the executing user > and yarn doesn’t have write permissions. > > 29743 [Thread-31] INFO org.apache.sqoop.hive.HiveImport - Authorization > failed:java.security.AccessControlException: action WRITE not permitted on > path hdfs:// > cloudfeeds-visual-n02.test.ord1.ci.rackspace.net:8020/apps/hive/warehouse/entries_staging_test_ord_2014_12_01 > for user yarn. Use SHOW GRANT to get more details. > 30089 [main] ERROR org.apache.sqoop.tool.ImportTool - Encountered > IOException running import job: java.io.IOException: Hive exited with > status 147 > at > org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:385) > at > org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:335) > at > org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:239) > at > org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:425) > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:506) > at org.apache.sqoop.Sqoop.run(Sqoop.java:147) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) > at org.apache.sqoop.Sqoop.main(Sqoop.java:240) > at > org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:206) > at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:174) > at > org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:38) > at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:45) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:225) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > > > > Thanks for any pointers, > > Greg > >
