Thanks Andrew If I'm correct e.s. will first fill up the first disk, and so on if you specify multiple data paths vs scattering over them like hdfs.. correct? You overcome this by giving easch e.s. instanve a data path on a sepparate disk
Thanks Regards Ronny On 5 Sep 2014 02:30, "Andrew Selden" <[email protected]> wrote: > > I have not tried this but my initial thoughts would be > > - Set ES_HEAP_SIZE = 30 GB, give Hadoop an appropriate amount, leave the > rest for the OS cache. > - Set the filesystem paths where ES and Hadoop store data to separate > physical disk(s). You don’t want them contending for bandwidth. > - You don’t have to use RAID for ES, you can use multiple data paths if > you have multiple disks. > - At this size, many people choose to run multiple instances of ES on a > single physical. Give each instance 30GB and point to different disks. > > A > > On Sep 4, 2014, at 11:32 AM, Ronny Vaningh <[email protected]> wrote: > > Hi > > > I have some beefy boxes with 512 Gb ram and I would like to co-locate > yarn/hadoop with elasticsearch > > Does anyone have experience in doing the same ? > How did you split the resources (memory/disk) across both functions ? > > Hdfs like jbod while E.S. like raid10 > > > Thank > > > Reg > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CACA5U5mW8zn0BLWryeePdR%2B0yiBGhd_9pQWVy-Uufj3H-oeQhA%40mail.gmail.com > <https://groups.google.com/d/msgid/elasticsearch/CACA5U5mW8zn0BLWryeePdR%2B0yiBGhd_9pQWVy-Uufj3H-oeQhA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/0A8CF809-7F38-40B2-AE23-508F1477B823%40elasticsearch.com > <https://groups.google.com/d/msgid/elasticsearch/0A8CF809-7F38-40B2-AE23-508F1477B823%40elasticsearch.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACA5U5%3DUD_HuLL_DEM4J-kKL330pvnex5QLjwJkF-f_ac1Sehg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
