Regarding containers not launching

What should I be looking for if my 24-node cluster in not launching enough containers? only 40/288 cores are used and 87GB/700GB is memory is used. Yarn.nodemanager memory/core conf look good. And so do container memory/core conf. Thanks Nishchay Malhotra

RE: HDFS latency and bandwidth/speed

Thank you very much Anu, This is very useful Manuel From: Anu Engineer [mailto:aengin...@hortonworks.com] Sent: Tuesday, January 30, 2018 12:25 PM To: Manuel Sopena Ballesteros; user@hadoop.apache.org Subject: Re: HDFS latency and bandwidth/speed Hi Manuel, Depending on your use case: There

Re: HDFS latency and bandwidth/speed

Hi Manuel, Depending on your use case: There are several tools. Unfortunately, most of them need some familiarity with HDFS. Here is a quick set of links that google returns. https://hadoop.apache.org/docs/r2.8.0/hadoop-project-dist/hadoop-common/Benchmarking.html An old blog, but most of

HDFS latency and bandwidth/speed

Hi all, I am going to start working on HDFS, how could I test HDFS latency and speed? Is there an ioping command and or hdpram or fio I can use in HDFS? Thank you very much Manuel Sopena Ballesteros | Big data Engineer Garvan Institute of Medical Research The Kinghorn Cancer Centre, 370

Re: performance about writing data to HDFS

Hello, Here is an example. You can set an initial low replication like this code does:

performance about writing data to HDFS

Hi community: I have a question about the performance of writing to HDFS. I've learned that When We write data to HDFS using the interface provided by HDFS such as 'FileSystem.create', our client will block until all the blocks and their replications are done. This will cause efficiency

Re: Anyone has tried accessing TDE using HDFS Java APIs

Hi Ajay Did you get any chance to look into this. Thanks Regards Prav On Fri, Jan 26, 2018 at 8:48 AM, praveenesh kumar wrote: > Hi Ajay > > We are using HDP 2.5.5 with HDFS 2.7.1.2.5 > > Thanks > Prav > > On Thu, Jan 25, 2018 at 5:47 PM, Ajay Kumar