Re: [sqlite] VFS memory leak : During lock / unlock operations
* Dennis Cote: > The OP said they were measuring an excess of lock calls. That would > imply that SQLite is locking files it has already locked. Is that > possible with the POSIX APIs? SQLite uses fcntl-based locks, which keep a separate lock for each byte in a file (or, more precisely, any byte that you can address with off_t). You can lock and unlock ranges, and if the sizes do not match (which isn't a requirement), the number of calls do not need to match. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] VFS memory leak : During lock / unlock operations
Dennis Cote <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] wrote: > > > > There are no lock leaks. But on the other hand, there is not > > a one-to-one mapping of lock to unlock calls. unlock is called > > more often than lock and there are often attempts to unlock files > > that have never been locked, irrc. This is harmless on windows > > and unix. > > > > Richard, > > The OP said they were measuring an excess of lock calls. That would > imply that SQLite is locking files it has already locked. Is that > possible with the POSIX APIs? > That might also be possible. It has been a while since I looked into this, but I think SQLite might be acquiring (for example) multiple read locks then releasing them all with a single unlock. -- D. Richard Hipp <[EMAIL PROTECTED]> ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] VFS memory leak : During lock / unlock operations
[EMAIL PROTECTED] wrote: > > There are no lock leaks. But on the other hand, there is not > a one-to-one mapping of lock to unlock calls. unlock is called > more often than lock and there are often attempts to unlock files > that have never been locked, irrc. This is harmless on windows > and unix. > Richard, The OP said they were measuring an excess of lock calls. That would imply that SQLite is locking files it has already locked. Is that possible with the POSIX APIs? Dennis Cote ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] VFS memory leak : During lock / unlock operations
<[EMAIL PROTECTED]> wrote: > Hi.. > >we are using sqlite3.3.4 with Integrity OS. we are facing a problem > where in the VFS memory is getting exhausted due >to large lock/unlock calls made by sqlite. Integrity support team > said that, for each file lock call made by sqlite, a definite amount >of memory is allocated, this memory is released only after the unlock > or when the file is closed. And they are claiming that >number of file unlock calls are not same as the number of file lock > calls. they have put traces and identified that for 1000 lock calls >there are only 950 unlock calls, which is a shortage of 50 unlock > calls. This will leak considerable amount of memory if the >system is left for long hours, with continuous sqlite operations > being made. > > can any one throw some light on this problem, is there any known > issue like this. Any information on this will be very helpful The os_unix.c backend to SQLite makes no attempt to match lock/unlock calls, because posix does not requires such. If you are running on an operating system that does require matching lock/unlock calls, you will probably need to modify the os_unix.c layer in order for it to work properly on your system. > > thanks > murthy > > Please do not print this email unless it is absolutely necessary. Spread > environmental awareness. > > The information contained in this electronic message and any attachments to > this message are intended for the exclusive use of the addressee(s) and may > contain proprietary, confidential or privileged information. If you are not > the intended recipient, you should not disseminate, distribute or copy this > e-mail. Please notify the sender immediately and destroy all copies of this > message and any attachments. > > WARNING: Computer viruses can be transmitted via email. The recipient should > check this email and any attachments for the presence of viruses. The company > accepts no liability for any damage caused by any virus transmitted by this > email. > > www.wipro.com > ___ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > . ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] VFS memory leak : During lock / unlock operations
Brad House <[EMAIL PROTECTED]> wrote: > >> they have put traces and identified that for 1000 lock calls > >> there are only 950 unlock calls, which is a shortage of 50 unlock > >> calls. > > The os_unix.c backend to SQLite makes no attempt to match lock/unlock > > calls, because posix does not requires such. If you are running > > on an operating system that does require matching lock/unlock calls, > > you will probably need to modify the os_unix.c layer in order for it > > to work properly on your system. > > I'm not sure what you're getting at here. Are you saying that it's > possible that some mutexes will _never_ be unlocked? In what circumstance > does that occur? Is there a case where a mutex could be destroyed while > a lock is held? I know my linux 'man pthread_mutex_destroy' says > "Attempting to destroy a locked mutex results in undefined behavior". > > Under what circumstances does this "lock leakage" occur exactly? There are no lock leaks. But on the other hand, there is not a one-to-one mapping of lock to unlock calls. unlock is called more often than lock and there are often attempts to unlock files that have never been locked, irrc. This is harmless on windows and unix. -- D. Richard Hipp <[EMAIL PROTECTED]> ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] VFS memory leak : During lock / unlock operations
>> they have put traces and identified that for 1000 lock calls >> there are only 950 unlock calls, which is a shortage of 50 unlock >> calls. > The os_unix.c backend to SQLite makes no attempt to match lock/unlock > calls, because posix does not requires such. If you are running > on an operating system that does require matching lock/unlock calls, > you will probably need to modify the os_unix.c layer in order for it > to work properly on your system. I'm not sure what you're getting at here. Are you saying that it's possible that some mutexes will _never_ be unlocked? In what circumstance does that occur? Is there a case where a mutex could be destroyed while a lock is held? I know my linux 'man pthread_mutex_destroy' says "Attempting to destroy a locked mutex results in undefined behavior". Under what circumstances does this "lock leakage" occur exactly? -Brad ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] VFS memory leak : During lock / unlock operations
Hi, We are working on different operating system other than unix and windows. We are using the os_win.c for porting, we have ported the code by making some changes with respect to the FAT32 file system apis in the code. We have not tested much, my worry is whether we also need to take care of such in the os_win.c In our OS also. Also, is this the OS dependent? Best Regards, A. Sreedhar. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Friday, February 08, 2008 7:24 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] VFS memory leak : During lock / unlock operations <[EMAIL PROTECTED]> wrote: > Hi.. > >we are using sqlite3.3.4 with Integrity OS. we are facing a problem > where in the VFS memory is getting exhausted due >to large lock/unlock calls made by sqlite. Integrity support team > said that, for each file lock call made by sqlite, a definite amount >of memory is allocated, this memory is released only after the > unlock or when the file is closed. And they are claiming that >number of file unlock calls are not same as the number of file lock > calls. they have put traces and identified that for 1000 lock calls >there are only 950 unlock calls, which is a shortage of 50 unlock > calls. This will leak considerable amount of memory if the >system is left for long hours, with continuous sqlite operations > being made. > > can any one throw some light on this problem, is there any known > issue like this. Any information on this will be very helpful The os_unix.c backend to SQLite makes no attempt to match lock/unlock calls, because posix does not requires such. If you are running on an operating system that does require matching lock/unlock calls, you will probably need to modify the os_unix.c layer in order for it to work properly on your system. > > thanks > murthy > > Please do not print this email unless it is absolutely necessary. Spread environmental awareness. > > The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. > > WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. > > www.wipro.com > ___ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > . ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] VFS memory leak : During lock / unlock operations
From: Murthy Girigowda (WT01 - TES-Access Networks) Sent: Friday, February 08, 2008 12:25 PM To: 'sqlite-users@sqlite.org' Subject: VFS memory leak : During lock / unlock operations Hi.. we are using sqlite3.3.4 with Integrity OS. we are facing a problem where in the VFS memory is getting exhausted due to large lock/unlock calls made by sqlite. Integrity support team said that, for each file lock call made by sqlite, a definite amount of memory is allocated, this memory is released only after the unlock or when the file is closed. And they are claiming that number of file unlock calls are not same as the number of file lock calls. they have put traces and identified that for 1000 lock calls there are only 950 unlock calls, which is a shortage of 50 unlock calls. This will leak considerable amount of memory if the system is left for long hours, with continuous sqlite operations being made. can any one throw some light on this problem, is there any known issue like this. Any information on this will be very helpful thanks murthy Please do not print this email unless it is absolutely necessary. Spread environmental awareness. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] VFS memory leak : During lock / unlock operations
Hi.. we are using sqlite3.3.4 with Integrity OS. we are facing a problem where in the VFS memory is getting exhausted due to large lock/unlock calls made by sqlite. Integrity support team said that, for each file lock call made by sqlite, a definite amount of memory is allocated, this memory is released only after the unlock or when the file is closed. And they are claiming that number of file unlock calls are not same as the number of file lock calls. they have put traces and identified that for 1000 lock calls there are only 950 unlock calls, which is a shortage of 50 unlock calls. This will leak considerable amount of memory if the system is left for long hours, with continuous sqlite operations being made. can any one throw some light on this problem, is there any known issue like this. Any information on this will be very helpful thanks murthy Please do not print this email unless it is absolutely necessary. Spread environmental awareness. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
