Re: [fossil-users] What is maximum limit size of a .fsl file ?
Hi, stephan Thanks for replying to my question again. I may split up source tree into a few branches, but What is biggest size a .fsl should limit to ? So I can operate within the safe boundary. Also I hope the warning of oversized .fsl should come out of the fossil commands somehow. T.J. Yang Date: Thu, 7 Jan 2010 16:18:25 +0100 From: sgb...@googlemail.com To: fossil-users@lists.fossil-scm.org Subject: Re: [fossil-users] What is maximum limit size of a .fsl file ? On Thu, Jan 7, 2010 at 3:39 PM, T.J. Yang wrote: http://www.fossil-scm.org/index.html/tktview?name=b074a7588b You might be hitting a RAM limit here, as opposed to a filesystem or sqlite3/fossil limit. Fossil performs many operations in memory (e.g. diffs). Internally it uses unsigned int for its size type. If your platform has a 32-bit integer (that isn't clear based on your bug report), then fossil will fail to diff large objects. Also, since the diffs are done in RAM, large objects (approx. 2^31 bytes) will fail to diff properly because two copies are needed to perform a diff. It is a limit of 32-bit platforms that no single process can address more than 4GB of memory, but in practice that limit is a bit lower (3.6-3.8GB is the most i've seen successfully allocated to a single 32-bit process). In my opinion, your particular use case is a bit out of scope for fossil. Every system has limits, and your use case would appear to push fossil past its limits. Even if that given repository will work in a 64-bit environment (though i'm not sure that it will), the repository itself would then be unusable on 32-bit machines. -- - stephan beal http://wanderinghorse.net/home/stephan/ _ Hotmail: Trusted email with powerful SPAM protection. http://clk.atdmt.com/GBL/go/196390707/direct/01/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] What is maximum limit size of a .fsl file ?
On Thu, Jan 7, 2010 at 4:32 PM, T.J. Yang tj_y...@hotmail.com wrote: I may split up source tree into a few branches, but What is biggest size a .fsl should limit to ? _in theory_, the largest item which can portably be committed is somewhere around 1.7GB. That number comes from: a) max memory space for 32-bit platforms = ~4GB. In my experience, 3.6-3.8GB is the max. b) fossil does its diffs in memory, meaning 2 copies (plus the delta) are in memory. That said, it is theoretically possible to create fossil repos on 64-bit platforms which will not work on 32-bit platforms if individual artifacts are too big. The file format itself is endian/bitness-neutral, but the limits of 32 bits can still be hit. For example, if an artifact is 1.95GB in size, diffing that artifact is very likely to fail on 32-bit systems because fossil probably won't be able to malloc() two contiguous chunks of 1.95GB each. Also I hope the warning of oversized .fsl should come out of the fossil commands somehow. i can't personally say how well suited sqlite3 is for 4GB resp. 32bit, but Richard can certainly enlighten us there. IMO, though, anything more than a few hundred MB is out of scope for a fossil repository. i wouldn't even bother to put such large repos in source control - i'd just put the parts which have to be versioned AND will change often into source control, and store the rest as tar files or in a separate tree. -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] What is maximum limit size of a .fsl file ?
Note that blobs are limited to 1G, so artifacts are limited to 1G compressed, even on a 64-bit processor with 8GB of memory (I tried). Gé On 01/07/2010 08:15 AM, Stephan Beal wrote: On Thu, Jan 7, 2010 at 4:32 PM, T.J. Yang tj_y...@hotmail.com mailto:tj_y...@hotmail.com wrote: I may split up source tree into a few branches, but What is biggest size a .fsl should limit to ? _in theory_, the largest item which can portably be committed is somewhere around 1.7GB. That number comes from: a) max memory space for 32-bit platforms = ~4GB. In my experience, 3.6-3.8GB is the max. b) fossil does its diffs in memory, meaning 2 copies (plus the delta) are in memory. That said, it is theoretically possible to create fossil repos on 64-bit platforms which will not work on 32-bit platforms if individual artifacts are too big. The file format itself is endian/bitness-neutral, but the limits of 32 bits can still be hit. For example, if an artifact is 1.95GB in size, diffing that artifact is very likely to fail on 32-bit systems because fossil probably won't be able to malloc() two contiguous chunks of 1.95GB each. Also I hope the warning of oversized .fsl should come out of the fossil commands somehow. i can't personally say how well suited sqlite3 is for 4GB resp. 32bit, but Richard can certainly enlighten us there. IMO, though, anything more than a few hundred MB is out of scope for a fossil repository. i wouldn't even bother to put such large repos in source control - i'd just put the parts which have to be versioned AND will change often into source control, and store the rest as tar files or in a separate tree. -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users -- Gé Weijers email: g...@weijers.org mailto:g...@weijers.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] What is maximum limit size of a .fsl file ?
2010/1/8 Stephan Beal sgb...@googlemail.com _in theory_, the largest item which can portably be committed is somewhere around 1.7GB. That number comes from: a) max memory space for 32-bit platforms = ~4GB. In my experience, 3.6-3.8GB is the max. b) fossil does its diffs in memory, meaning 2 copies (plus the delta) are in memory. The overwhelming number of modern 32-bit systems are actually 36-bit in address space. You have to turn on appropriate options to use that mode, of course. For Windows that means running a server edition. For Linux that means turning on certain kernel options when compiling (or using a server kernel for those distros that support this). I am, for example, happily using my full 4GB on a 32-bit version of Ubuntu. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
