Don't try this on anything but an experimental filesystem. If you can
simply find the places where HDFS calls the CRC algorithm and replace
them with zeros, then you should be able to get a reasonable benchmark.
Doug
Bryan Duxbury wrote:
I'm willing to give this a shot. Let me just be sure I understand what
I'd have to do: if I make it stop computing CRCs altogether, I need to
make changes in the datanode as well, right? To stop checking validity
of CRCs? Will this break anything interesting and unexpected?
On Oct 6, 2008, at 4:58 PM, Doug Cutting wrote:
Bryan Duxbury wrote:
I am profiling with YourKit on random reducers. I'm also running on
HDFS, so I don't know how one would go about disabling CRCs.
Hack the CRC-computing code to fill things with zeros?
Doug
