2011/7/16 jagaran das <[email protected]>
> Hi, > > Due to requirements in our current production CDH3 cluster we need to copy > around 11520 small size files (Total Size 12 GB) to the cluster for one > application. > Like this we have 20 applications that would run in parallel > > So one set would have 11520 files of total size 12 GB > Like this we would have 15 sets in parallel, > > We have a total SLA for the pipeline from copy to pig aggregation to copy > to local and sql load is 15 mins. > > Have you tried to use HARs? -- Best regards, Vitalii Tymchyshyn
