@Surya On a side note, I have seen multiple emails with subject line "Information needed". I urge you to use more informative subject line for your subsequent posts.
This would help future users to look through the mailing list archive. ~ Yogi On 11 August 2016 at 10:23, Priyanka Gugale <[email protected]> wrote: > IMO, it's okay to keep the files in HDFS, at startup you can load them in > memory which will be one time activity in application lifetime. Considering > they are format/config files, it won't be difficult to cache it in memory > of your current operator (no need to caching operator separately). > > -Priyanka > > On Wed, Aug 10, 2016 at 7:06 PM, Mukkamula, Suryavamshivardhan (CWM-NR) < > [email protected]> wrote: > >> Hello Team, >> >> I have a use case where I am trying to read number of input files with >> different formats(may be 100 source systems) and I have to parse each >> record. >> >> To achieve this we need to have metadata(configuration files) of the >> input file structures. Can you please let me know that, having this >> metadata on HDFS is good or having the metadata in some sql/oracle database >> is good? >> >> Assume that I don’t have caching operator for now, I need to get the >> configuration file for each source and I have to parse them at run time. To >> do this going to HDFS is good or going to database is good? >> >> Regards, >> Surya Vamshi >> >> >> _______________________________________________________________________ >> >> If you received this email in error, please advise the sender (by return >> email or otherwise) immediately. You have consented to receive the attached >> electronically at the above-noted email address; please retain a copy of >> this confirmation for future reference. >> >> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur >> immédiatement, par retour de courriel ou par un autre moyen. Vous avez >> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à >> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de >> cette confirmation pour les fins de reference future. >> >> >
