Re: [ccache] ccache cache in RAM -- bypassing file cache?
We did some more experiments and found that it did indeed perform better with more OS file cache. > it would be fairly insane if there was. That was basically my conclusion after reading Justin's reply. Thanks. On Jun 28, 2011, at 8:49 AM, Martin Pool wrote: I think Justin is right that there should be no copying; it would be fairly insane if there was. I would speculate that the cause is nothing to do with the cache, but rather just that on the larger machine, more of the source tree and the header files used by it can be kept in memory. You could try with tmpfs instead to see if it's different. Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] ccache cache in RAM -- bypassing file cache?
I think Justin is right that there should be no copying; it would be fairly insane if there was. I would speculate that the cause is nothing to do with the cache, but rather just that on the larger machine, more of the source tree and the header files used by it can be kept in memory. You could try with tmpfs instead to see if it's different. Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] ccache cache in RAM -- bypassing file cache?
I'm not sure that ramfs copies data into the buffer cache. According to [1], a ramfs mount *is* the buffer cache. [1] http://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt On Fri, Jun 24, 2011 at 2:40 PM, David Coppit wrote: > We're trying to debug a performance difference of 12% speedup on a machine > with 12GB versus a 41% speedup on a similar machine with 48GB of RAM. > > We've profiled the build, and believe it to have performance degradation if > it has less than 10GB of ram. (5% slower builds with 10GB of system RAM, 10% > slower builds with 9GB of system RAM.) > > Our ccache cache is in memory, on a ramfs mounted this way: sudo mount -t > ramfs -o size=1G,mode=0700 ramfs /mnt/ramfs > > Our cache size after the initial warming run is 1GB. > > Here's what I'm thinking is happening: The OS is reading from the ramdisk > and then caching the files in memory like it normally would. This is > creating file cache contention with other file I/O in the build, offsetting > our performance gain. > > For the machine with plenty of spare RAM, there is no file cache contention > in the OS, and we see much better perf. Does this sound reasonable? > > I don't know if there's a way to prevent the OS from caching files on the > ramfs mount, except to modify ccache's open() calls to include O_DIRECT. If > I go that route, do I need to just modify the calls in hash.c, manifest.c, > util.c, and gzlib.c? > ___ > ccache mailing list > ccache@lists.samba.org > https://lists.samba.org/mailman/listinfo/ccache > ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
[ccache] ccache cache in RAM -- bypassing file cache?
We're trying to debug a performance difference of 12% speedup on a machine with 12GB versus a 41% speedup on a similar machine with 48GB of RAM. We've profiled the build, and believe it to have performance degradation if it has less than 10GB of ram. (5% slower builds with 10GB of system RAM, 10% slower builds with 9GB of system RAM.) Our ccache cache is in memory, on a ramfs mounted this way: sudo mount -t ramfs -o size=1G,mode=0700 ramfs /mnt/ramfs Our cache size after the initial warming run is 1GB. Here's what I'm thinking is happening: The OS is reading from the ramdisk and then caching the files in memory like it normally would. This is creating file cache contention with other file I/O in the build, offsetting our performance gain. For the machine with plenty of spare RAM, there is no file cache contention in the OS, and we see much better perf. Does this sound reasonable? I don't know if there's a way to prevent the OS from caching files on the ramfs mount, except to modify ccache's open() calls to include O_DIRECT. If I go that route, do I need to just modify the calls in hash.c, manifest.c, util.c, and gzlib.c? ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
