Thanks for the pointers, Laurent. Works! Greg
On 12/5/14, 3:55 AM, "Laurent H" <[email protected]> wrote: >Greg, > >In Sqoop 1.4.4, I've an issue when I import data from RDBMS to Hive too. > >I use the following solution : > >1) Import RDBMS to file with Sqoop into an HDFS directory >2) LOAD DATA statement to fill your Hive table. > >Good luck ! > > >-- >Laurent HATIER - Consultant Big Data & Business Intelligence chez >CapGemini >fr.linkedin.com/pub/laurent-hatier/25/36b/a86/ ><http://fr.linkedin.com/pub/laurent-h/25/36b/a86/> > >2014-12-02 22:25 GMT+01:00 Greg Sharek <[email protected]>: > >> Hi all, >> >> I¹m attempting to create a workflow which uses sqoop to import data from >> my postgres db into a hive table. My sqoop configuration runs fine >>outside >> of Oozie. >> >> My main question: Does oozie support sqoop importing directly into >>hive? >> I see some comments claiming it doesn¹t, but I haven¹t found anything >> definitive in the Oozie docs. >> >> For the curious, here¹s what I¹m experiencing. I¹d be happy to provide >> more details if they would help. >> >> When I run my Oozie workflow, I see the following: >> >> * Sqoop is able to import data into HDFS under the directory >>specified >> by ‹target-dir >> * When Sqoop attempts to copy this to the Hive table, it gets a >> permission error as it¹s attempting to write to a table as user Œyarn¹ >>as >> opposed to the executing user. The table is owned by the executing user >> and yarn doesn¹t have write permissions. >> >> 29743 [Thread-31] INFO org.apache.sqoop.hive.HiveImport - >>Authorization >> failed:java.security.AccessControlException: action WRITE not permitted >>on >> path hdfs:// >> >>cloudfeeds-visual-n02.test.ord1.ci.rackspace.net:8020/apps/hive/warehouse >>/entries_staging_test_ord_2014_12_01 >> for user yarn. Use SHOW GRANT to get more details. >> 30089 [main] ERROR org.apache.sqoop.tool.ImportTool - Encountered >> IOException running import job: java.io.IOException: Hive exited with >> status 147 >> at >> >>org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.jav >>a:385) >> at >> org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:335) >> at >> org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:239) >> at >> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:425) >> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:506) >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240) >> at >> org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:206) >> at >>org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:174) >> at >> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:38) >> at >>org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:45) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> >>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java >>:57) >> at >> >>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI >>mpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> >>org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:225 >>) >> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) >> at >>org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) >> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:415) >> at >> >>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation >>.java:1594) >> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) >> >> >> >> Thanks for any pointers, >> >> Greg >> >>
