I have also faced similar issue. Would appreciate a solution for this issue.
Thanks & Regards, Harish.T.K On Wed, Feb 13, 2013 at 1:39 AM, Mohammad Islam <[email protected]> wrote: > For data checking , did you consider coordinator in place of custom logic? > > Do you see any issue with external table in this respect? > > Regards, > Mohammad > > > ________________________________ > From: Shreehari Padaki <[email protected]> > To: [email protected]; Mohammad Islam <[email protected]> > Cc: Harsh J <[email protected]> > Sent: Wednesday, February 13, 2013 12:13 AM > Subject: Re: Oozie - accessing files from local file system > > yes, we can do that way but it adds one more layer and also requires some > extra time. > > Another thing is i want to schedule my hive queries once data is loaded in > hive tables. So now we are creating external table (to hive) in hdfs and > loading files through cron job and using some custom logic in oozie to > check data is uploaded in hdfs or not before it runs hive queries. > > So just wanted to check for other ways to do that. > > Thanks > Shreehari > > > On Wed, Feb 13, 2013 at 1:34 PM, Mohammad Islam <[email protected]> > wrote: > > > > > > > Hi Shreehari, > > I think there is no way of executing this at this point. > > You are right, Oozie needs the file to be in hdfs. > > Is there a way of uploading the files in hdfs and then execute with LOAD > > command giving the hdfs path as the source. > > > > Regards, > > Mohammad > > > > > > ________________________________ > > From: Shreehari Padaki <[email protected]> > > To: Harsh J <[email protected]> > > Cc: [email protected] > > Sent: Tuesday, February 12, 2013 11:53 PM > > Subject: Re: Oozie - accessing files from local file system > > > > Hi Harsh, > > > > We have tried giving absolute path but it didn't work. It always try to > > look for the path in hdfs. > > > > So what is the way to schedule a job to copy files from local file system > > to hdfs or hive? is this possible through oozie? > > > > > > Thanks > > Shreehari > > > > On Wed, Feb 13, 2013 at 11:13 AM, Harsh J <[email protected]> wrote: > > > > > Hi Shreehari, > > > > > > I'd recommend using a list since my limited free time does not permit > > > me to make sure to reply to every single email sent to me personally. > > > Using a community mailing list also lets you get thoughts of several > > > experienced folks out there, than just one person :) > > > > > > I'd recommend sending the question to the Oozie lists, > > > [email protected]. I've added the list to this response. You can > > > subscribe to these lists via instructions at > > > http://oozie.apache.org/mail-lists.html. > > > > > > From a brief read, however, your problem is that Oozie may not > > > understand relative paths, given the user/location it runs that > > > command as. Try passing an absolute path, such as > > > "/user/shreehari/downloads/test/" for example, and verify? > > > > > > On Wed, Feb 13, 2013 at 10:59 AM, <[email protected]> wrote: > > > > Hi Harsha, > > > > > > > > We are planning to use Oozie as work flow scheduler with Hadoop and > > Hive. > > > > > > > > But the problem we are facing is we are unable to access files stored > > in > > > local file system to upload it in hive through Oozie. > > > > > > > > example: LOAD DATA LOCAL INPATH 'downloads/test/' INTO TABLE TEST; > > > > > > > > if i execute this in hive then it works fine, but same thing if run > it > > > through Oozie it throws error saying the path 'downloads/test/' as > > invalid > > > path. > > > > > > > > As i understood Oozie run over hdfs, so it is expecting files in > hdfs. > > > > > > > > I am new to hadoop and oozie. Is there anyway we can achieve this > > > through Oozie? > > > > > > > > is there any better work flow scheduler we can use? > > > > > > > > Thanks > > > > Shreehari > > > > > > > > > > > > -- > > > Harsh J > > > > > >
