David/Przemek, Thanks for discussing this, if even in passing. I too, like David, could not find documentation as to locking schemes either, and therefore read "DB_DBFLOCK_XHB64" as being specific to a 64 bit executable - and as such never even tried it. Upon trying it, however, my problems disappeared on large data file manipulations - no more problems in either the write or read portions of my executables/application.
Thanks for jogging my assumptions, and again, great release. F -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of David Arturo Macias Corona Sent: Sunday, January 17, 2010 7:40 AM To: [email protected] Subject: [Harbour] Re: OT: file size Przemek: This is becoming more interesting :-) First thanks for your help >In the past I was answering for such questions over 10 times. >You may find these messages in the Harbour and xHarbour mailing >list archives. A short description I added also to tests/xhb-diff.txt I read xhb-diff.txt and checked it About "over 10 times": I have seen before your info about "locking schemes" and so on, but in that cases was not relevant because does not exist a direct need and we usually do not go deep in those messages if we are not involved For example, nobody have response for my recent messages about hbcurl, hbcairo, hbqt because are considered as irrelevant in this moment Surprisely, except for source code, there are not reference of DB_DBFLOCK_VFP, "locking scheme" in doc files and even changelog file My mail file for Harbour messages reached 105 Mb file size and no one editor in OS/2 can open it ( EPM, E, TEDIT, JEdit ) :-) So I used many hours to split file in many files until it can open under 50 Mb: harbour0 21 Mb, 2006 to 2007 harbour08 26 Mb, 2008 harbour09a 31 Mb, 2009 first six months harbour 27 Mb, 2009 rest six months as we can see, 2009 was plenty of messages ... and surprisely too, in messages there are not too much info about dbf file sizes, "locking schemes". Last and detailed message was around 30 Oct 2009 In source code we find info as this: /* LOCK SCHEMES */ #define DB_DBFLOCK_DEFAULT 0 #define DB_DBFLOCK_CLIP 1 #define DB_DBFLOCK_CL53 2 #define DB_DBFLOCK_VFP 3 #define DB_DBFLOCK_CL53EXT 4 #define DB_DBFLOCK_XHB64 5 /* DBF locking schemes */ #define DBF_LOCKPOS_CLIP 1000000000UL #define DBF_LOCKPOS_CL53 1000000000UL #define DBF_LOCKPOS_VFP 0x40000000UL #define DBF_LOCKPOS_VFPX 0x7ffffffeUL #define DBF_LOCKPOS_CL53EXT 4000000000UL #define DBF_LOCKPOS_XHB64 HB_LL( 0x7FFFFFFF00000001 ) which are difficult to understand for a user without too much knowledge For example, what is CL53EXT, VFPX ? For me was hard to find information about this case Perhaps info is there somewhere but I do not found it, so I suggest you to expand xhb-diff.txt or create a different one to clarify: - DBF file size limit ( currently brief ) - "locking schemes" so users can be oriented to file info in place to repeat in messages >I do not know ADS details so I do not it's limitations. >For sure it does not support all Harbour extenssions so they are >smaller and for sure there is nothing in ADS what exceed Harbour >any limits for DBFs and related files (memos and indexes) so >migration to ADS without changing the format to ADT+ADM+ADI can >only reduce the maximum supported file size or in some cases leave >them on the same level. If someone claims sth different then he >simply does not know what he's talking about and you can ignore him. >If you need bigger files then you have to switch to different format. >Each program which uses OS files is limited by OS API. The exact >limitations can be found in the OS documentation and they depends >on the operating system internals (OS kernel) and used file system >(FS). >Additionally please remember that locking schemes can seriously reduce >maximum file size. On some systems it's not possible to read from >locked area. In such case you have to switch all your applications >accessing the files to locking scheme which do not cause such >limitations, i.e. use Harbour 64bit locking. [...] >They use non POSIX system i.e. DOS or WINDOWS with VFP locking scheme >in which the RLOCK and FLOCK are is at range 0x6E6B27FF : 0x7FFFFFFE. >In such systems read/write access from locked region is blocked by >OS if client is not lock owner so the above error appear when other >station [R|F]locked some records in DBF file and user program wants >to read records which are located in the above range. That's all. >It's expected behavior. They should read about locking schemes and >switch to other locking scheme which do not cause such limitation. [...] >And this is result of file corruption, i.e. due to wrong CPs or server/ >client computer crash and FS corruption. I have more info provided by users They are using my (x)Harbour systems with ADS 6.2x and VFP systems with unknown connection type to ADS As you know ADS have two "Locking Modes": - Advantage Proprietary Locking - Advantage Compatibility Locking with Xbase Files and I do not know which are technical details of each one, and if "compatibility mode" is based in VFP type or if 2 Gb size is involved I checked and my systems are using "Proprietary mode" and "rights checkings" by default but I do not know what is using VFP Along years I did not found detailed info in ADS docs about technical facts of both "locking modes", so I respect to use Proprietary I do not know what is "range 0x6E6B27FF : 0x7FFFFFFE" :-) It is related to 2 Gb ? First info of users was about failure reaching 2 Gb, but maybe what is happening is related to "locking scheme" and not DBF file size Until now we confirmed it is not a file size problem and next tests must be for "locking schemes" So I have to check what ways they are using to connect ADS-VFP and exactly what kind of errors are happening, including memo corruption perhaps for a different memo block size used >I've never heard about such limits in code I created. All limits >are result of used file structures and external limitations like >used OS or FS. Harbour does not introduce any own limits. Anyhow >if you will find sth then I'm very interesting. For sure it will >be fixed immediately. I am sure of it too, thanks :-) David Macias _______________________________________________ Harbour mailing list (attachment size limit: 40KB) [email protected] http://lists.harbour-project.org/mailman/listinfo/harbour _______________________________________________ Harbour mailing list (attachment size limit: 40KB) [email protected] http://lists.harbour-project.org/mailman/listinfo/harbour
