Hi, We have parsers examples at: https://github.com/DataTorrent/examples/tree/master/tutorials/parser (The csv parser example will be available in couple of days) Check if this can help you.
-Priyanka On Thu, Aug 11, 2016 at 7:05 PM, Mukkamula, Suryavamshivardhan (CWM-NR) < [email protected]> wrote: > Thanks Priyanka, I would try with caching in my upcoming project. > > > > Do you have any sample that can be helpful to me? > > > > Regards, > > Surya Vamshi > > > > *From:* Priyanka Gugale [mailto:[email protected]] > *Sent:* 2016, August, 11 12:53 AM > *To:* [email protected] > *Subject:* Re: Information Needed > > > > IMO, it's okay to keep the files in HDFS, at startup you can load them in > memory which will be one time activity in application lifetime. Considering > they are format/config files, it won't be difficult to cache it in memory > of your current operator (no need to caching operator separately). > > > > -Priyanka > > > > On Wed, Aug 10, 2016 at 7:06 PM, Mukkamula, Suryavamshivardhan (CWM-NR) < > [email protected]> wrote: > > Hello Team, > > > > I have a use case where I am trying to read number of input files with > different formats(may be 100 source systems) and I have to parse each > record. > > > > To achieve this we need to have metadata(configuration files) of the input > file structures. Can you please let me know that, having this metadata on > HDFS is good or having the metadata in some sql/oracle database is good? > > > > Assume that I don’t have caching operator for now, I need to get the > configuration file for each source and I have to parse them at run time. To > do this going to HDFS is good or going to database is good? > > > > Regards, > > Surya Vamshi > > > > _______________________________________________________________________ > > If you received this email in error, please advise the sender (by return > email or otherwise) immediately. You have consented to receive the attached > electronically at the above-noted email address; please retain a copy of > this confirmation for future reference. > > Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur > immédiatement, par retour de courriel ou par un autre moyen. Vous avez > accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à > l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de > cette confirmation pour les fins de reference future. > > > > _______________________________________________________________________ > > If you received this email in error, please advise the sender (by return > email or otherwise) immediately. You have consented to receive the attached > electronically at the above-noted email address; please retain a copy of > this confirmation for future reference. > > Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur > immédiatement, par retour de courriel ou par un autre moyen. Vous avez > accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à > l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de > cette confirmation pour les fins de reference future. > >
