yes, we can do that way but it adds one more layer and also requires some extra time.
Another thing is i want to schedule my hive queries once data is loaded in hive tables. So now we are creating external table (to hive) in hdfs and loading files through cron job and using some custom logic in oozie to check data is uploaded in hdfs or not before it runs hive queries. So just wanted to check for other ways to do that. Thanks Shreehari On Wed, Feb 13, 2013 at 1:34 PM, Mohammad Islam <[email protected]> wrote: > > > Hi Shreehari, > I think there is no way of executing this at this point. > You are right, Oozie needs the file to be in hdfs. > Is there a way of uploading the files in hdfs and then execute with LOAD > command giving the hdfs path as the source. > > Regards, > Mohammad > > > ________________________________ > From: Shreehari Padaki <[email protected]> > To: Harsh J <[email protected]> > Cc: [email protected] > Sent: Tuesday, February 12, 2013 11:53 PM > Subject: Re: Oozie - accessing files from local file system > > Hi Harsh, > > We have tried giving absolute path but it didn't work. It always try to > look for the path in hdfs. > > So what is the way to schedule a job to copy files from local file system > to hdfs or hive? is this possible through oozie? > > > Thanks > Shreehari > > On Wed, Feb 13, 2013 at 11:13 AM, Harsh J <[email protected]> wrote: > > > Hi Shreehari, > > > > I'd recommend using a list since my limited free time does not permit > > me to make sure to reply to every single email sent to me personally. > > Using a community mailing list also lets you get thoughts of several > > experienced folks out there, than just one person :) > > > > I'd recommend sending the question to the Oozie lists, > > [email protected]. I've added the list to this response. You can > > subscribe to these lists via instructions at > > http://oozie.apache.org/mail-lists.html. > > > > From a brief read, however, your problem is that Oozie may not > > understand relative paths, given the user/location it runs that > > command as. Try passing an absolute path, such as > > "/user/shreehari/downloads/test/" for example, and verify? > > > > On Wed, Feb 13, 2013 at 10:59 AM, <[email protected]> wrote: > > > Hi Harsha, > > > > > > We are planning to use Oozie as work flow scheduler with Hadoop and > Hive. > > > > > > But the problem we are facing is we are unable to access files stored > in > > local file system to upload it in hive through Oozie. > > > > > > example: LOAD DATA LOCAL INPATH 'downloads/test/' INTO TABLE TEST; > > > > > > if i execute this in hive then it works fine, but same thing if run it > > through Oozie it throws error saying the path 'downloads/test/' as > invalid > > path. > > > > > > As i understood Oozie run over hdfs, so it is expecting files in hdfs. > > > > > > I am new to hadoop and oozie. Is there anyway we can achieve this > > through Oozie? > > > > > > is there any better work flow scheduler we can use? > > > > > > Thanks > > > Shreehari > > > > > > > > -- > > Harsh J > > >
