I've run into this with large shuffles - I assumed that there was
contention between the shuffle output files and the JVM for memory.
Whenever we start getting these fetch failures, it corresponds with high
load on the machines the blocks are being fetched from, and in some cases
complete unrespons
I think this is exactly what packer is for. See e.g.
http://www.packer.io/intro/getting-started/build-image.html
On a related note, the current AMI for hvm systems (e.g. m3.*, r3.*) has a
bad package for httpd, whcih causes ganglia not to start. For some reason I
can't get access to the raw AMI to
fter next week. We are
> > planning
> > > on
> > > >>> taking our client work around hive/spark, plus taking over the
> bigtop
> > > >>> automation work to modernize and get that fit for human consumption
> > > outside
> > &
