Re: hackage-server memory requirements

2011-10-26 Thread Max Bolingbroke
On 25 October 2011 23:18, Bas van Dijk v.dijk@gmail.com wrote: What kind[1] of heap profile would be most useful. (In my experience -hy or -hd are the most informative). Either of those sound reasonable. I'd like to try do this out myself, but I can't mirror Hackage without hitting the

Re: hackage-server memory requirements

2011-10-26 Thread Bas van Dijk
On 26 October 2011 00:18, Bas van Dijk v.dijk@gmail.com wrote: I could run a sync on my 4GB laptop in the background tomorrow and do a heap profile. I do expect to kill it half way since I also have to get other stuff done. I did a quick heap profile using -hy:

Re: hackage-server memory requirements

2011-10-26 Thread Max Bolingbroke
On 26 October 2011 13:46, Bas van Dijk v.dijk@gmail.com wrote: According to the profile most space is used by ARR_WORDS (which is the internal name for a ByteArray# if I remember correctly). Interesting. There are a lot of ByteStrings in use in the server, so candidates for a leak might be:

Re: hackage-server memory requirements

2011-10-26 Thread Bas van Dijk
On 26 October 2011 19:36, Max Bolingbroke batterseapo...@hotmail.com wrote: So this is almost pure speculation, but perhaps adding (BS.length pkgStr `seq`) just before the liftIO on Mirror.hs:122 would reduce the memory usage significantly. Worth a try? I made the change but after syncing for

hackage-server memory requirements

2011-10-25 Thread Bas van Dijk
Hello, What are the typical memory requirements to run hackage-server (as a mirror of the current hackage). The reason I ask is that I tried mirroring hackage on my 4GB desktop machine. I let it mirror overnight but when I came back in the morning the machine was completely unresponsive. It

Re: hackage-server memory requirements

2011-10-25 Thread Bas van Dijk
On 25 October 2011 22:53, Max Bolingbroke batterseapo...@hotmail.com wrote: Do you still see 4GB usage if you restart the server? No, after restarting it was less but, if I remember correctly, still over 700MB. Another thing to do would be to heap profile the server.. it would be very