Thanks for your reply guys , i am looking for open source do we have any ??
On Tue, Jul 23, 2013 at 12:53 PM, Devaraj k <[email protected]> wrote: > Hi Balamurali,**** > > ** ** > > As per my knowledge, there is nothing in the hadoop which does exactly as > per your requirement.**** > > ** ** > > You can write mapreduce jobs according to your functionality and submit > hourly/daily/weekly or monthly . And then you can aggregate the results.** > ** > > ** ** > > If you want some help regarding Hive, you can ask the same in Hive mailing > list.**** > > ** ** > > ** ** > > ** ** > > Thanks**** > > Devaraj k**** > > ** ** > > *From:* Balamurali [mailto:[email protected]] > *Sent:* 23 July 2013 12:42 > *To:* user > *Subject:* Re: Copy data from Mainframe to HDFS**** > > ** ** > > Hi,**** > > > I configured hadoop-1.0.3, hbase-0.92.1 and hive-0.10.0 . > Created table in HBase.Inserted records.Processing the data using Hive.*** > * > > I have to show a graph with some points ( 7 - 7 days or 12 for one > year).In one day records may include 1000 - lacks.I need to show average of > these 1000 - lacks records.is there any built in haddop mechanism to > process these records fast.**** > > Also I need to run a hive query or job (when we run a hive query actually > a job is submitting) in every 1 hour.Is there a scheduling mechanism in > hadoop to handle thsese > > **** > > Please reply.**** > > Balamurali**** > > ** ** > > On Tue, Jul 23, 2013 at 12:24 PM, Mohammad Tariq <[email protected]> > wrote:**** > > Hello Sandeep,**** > > ** ** > > You don't have to convert the data in order to copy it into the HDFS. But > you might have to think about the MR processing of these files because of > the format of these files.**** > > ** ** > > You could probably make use of Sqoop <http://sqoop.apache.org/>.**** > > ** ** > > I also came across DMX-H a few days ago while browsing. I don't know > anything about the licensing and how good it is. Just thought of sharing it > with you. You can visit their > page<http://www.syncsort.com/en/Data-Integration/Home>to see more. They also > provide a VM(includes CDH) to get started quickly. > **** > > > **** > > Warm Regards,**** > > Tariq**** > > cloudfront.blogspot.com**** > > ** ** > > On Tue, Jul 23, 2013 at 11:54 AM, Sandeep Nemuri <[email protected]> > wrote:**** > > Hi , **** > > ** ** > > "How to copy datasets from Mainframe to HDFS directly? I know that we can > NDM files to Linux box and then we can use simple put command to copy data > to HDFS. But, how to copy data directly from mainframe to HDFS? I have > PS, PDS and VSAM datasets to copy to HDFS for analysis using MapReduce. ** > ** > > ** ** > > Also, Do we need to convert data from EBCDIC to ASCII before copy? "**** > > ** ** > > -- **** > > --Regards**** > > Sandeep Nemuri**** > > ** ** > > ** ** > -- --Regards Sandeep Nemuri
