Re: [courier-users] Corrupted Pythonfilter greylist_NotPassed database
Gordon Messmer pisze: Paweł Tęcza wrote: Paweł Tęcza pisze: What Python module for PostgreSQL support do you use? ... Gordon uses python-pgsql module by Cristian Gafton. You can download it from following URL: http://people.rpath.com/~gafton/pgsql/ Sorry to leave you hanging like that, Paweł. You're correct about the python module, also available here: http://pypi.python.org/pypi/python-pgsql/ Hi Gordon, Thanks a lot for the reply! It's good to know you read my posts :) At some point in the future, I'll probably add support for the psycopg2 module, used by Zope and Django for PostgreSQL support. PostgreSQL is a terrific DB, but it could seriously use a well supported standard Python module. Yuck. Fortunately Debian has good package with psycopg2, so I don't need to debianize it :) Let me know if you run into issues with SQL support. I still consider it experimental. It was implemented to enable SMTP server clustering, but I haven't used it anywhere in production. OK, I'll write here about noticed problems. I'm going to put it into action soon. Have a nice day, P. -- SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada. The future of the web can't happen without you. Join us at MIX09 to help pave the way to the Next Web now. Learn more and register at http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Corrupted Pythonfilter greylist_NotPassed database
Paweł Tęcza pisze: Gordon Messmer pisze: At some point in the future, I'll probably add support for the psycopg2 module, used by Zope and Django for PostgreSQL support. PostgreSQL is a terrific DB, but it could seriously use a well supported standard Python module. Yuck. Fortunately Debian has good package with psycopg2, so I don't need to debianize it :) Hello again, Is it very hard to add support for the psycopg2 module right now? I wrote that I want to debianize real python-pgsql module at Debian Python mailing list [1]. I heard in response that I could try to use psycopg2 module instead of python-pgsql. I did it, but unfortunately it seems that these modules haven't compatible API. Let me know if you run into issues with SQL support. I still consider it experimental. It was implemented to enable SMTP server clustering, but I haven't used it anywhere in production. OK, I'll write here about noticed problems. I'm going to put it into action soon. I'm running your Pythonfilter on one of our front-ends. It's working about 0.5h and haven't seen any issues :) pythonfilter= select count(*) from greylist_notpassed; count --- 57 (1 row) pythonfilter= select count(*) from greylist_passed; count --- 2 (1 row) Cheers, P. [1] http://lists.debian.org/debian-python/2008/12/msg00016.html -- SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada. The future of the web can't happen without you. Join us at MIX09 to help pave the way to the Next Web now. Learn more and register at http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Corrupted Pythonfilter greylist_NotPassed database
Paweł Tęcza wrote: Is it very hard to add support for the psycopg2 module right now? Probably not. I haven't checked on which variant of the DB-API it supports. Changes from the existing pgsql support will be very minor. At worst, the connect() function will take a different format argument, and the select queries will use a different character to indicate parameter replacement. -- SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada. The future of the web can't happen without you. Join us at MIX09 to help pave the way to the Next Web now. Learn more and register at http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Corrupted Pythonfilter greylist_NotPassed database
Paweł Tęcza wrote: Paweł Tęcza pisze: What Python module for PostgreSQL support do you use? ... Gordon uses python-pgsql module by Cristian Gafton. You can download it from following URL: http://people.rpath.com/~gafton/pgsql/ Sorry to leave you hanging like that, Paweł. You're correct about the python module, also available here: http://pypi.python.org/pypi/python-pgsql/ At some point in the future, I'll probably add support for the psycopg2 module, used by Zope and Django for PostgreSQL support. PostgreSQL is a terrific DB, but it could seriously use a well supported standard Python module. Yuck. Please note that Debian/Ubuntu package named python-pgsql is *different* module! The debian/copyright file says that it was created by Billy G. Allie and is available at http://pypgsql.sourceforge.net/ Good to know. In Red Hat/Fedora, the python-pgsql package is the correct one. Let me know if you run into issues with SQL support. I still consider it experimental. It was implemented to enable SMTP server clustering, but I haven't used it anywhere in production. -- SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada. The future of the web can't happen without you. Join us at MIX09 to help pave the way to the Next Web now. Learn more and register at http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Corrupted Pythonfilter greylist_NotPassed database
Paweł Tęcza pisze: Paweł Tęcza pisze: Paweł Tęcza pisze: [...] I've read about PostgreSQL support for Pythonfilter, but supposedly it's experimental. Also I don't have PostgreSQL database right now. Hello Folks, Does someone here use Pythonfilter with PostgreSQL? Gordon, What Python module for PostgreSQL support do you use? This is an information for other Courier users who are trying Pythonfilter with PostgreSQL support like me: Gordon uses python-pgsql module by Cristian Gafton. You can download it from following URL: http://people.rpath.com/~gafton/pgsql/ Please note that Debian/Ubuntu package named python-pgsql is *different* module! The debian/copyright file says that it was created by Billy G. Allie and is available at http://pypgsql.sourceforge.net/ The name of the package is very confusing, but it's old package, so I don't believe that its maintainer will change name of his package. Cheers, P. -- SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada. The future of the web can't happen without you. Join us at MIX09 to help pave the way to the Next Web now. Learn more and register at http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Corrupted Pythonfilter greylist_NotPassed database
Paweł Tęcza pisze: Paweł Tęcza pisze: [...] I've read about PostgreSQL support for Pythonfilter, but supposedly it's experimental. Also I don't have PostgreSQL database right now. Hello Folks, Does someone here use Pythonfilter with PostgreSQL? Gordon, What Python module for PostgreSQL support do you use? I've installed PostgreSQL 8.3 server successfully, but I still can't start your Pythonfilter, because of the following error: Dec 15 13:29:34 xxx courierfilter: File /usr/lib/python2.5/site-packages/pythonfilter/TtlDb.py, line 59, in __init__ Dec 15 13:29:34 xxx courierfilter: import pgsql Dec 15 13:29:34 xxx courierfilter: ImportError: No module named pgsql I have installed Ubuntu package named python-pgsql, but it doesn't contain pgsql module. It includes only PgSQL module. There are the same modules? Sorry for stupid questions for you, but I don't have any Python experiences. Have a nice day, P. -- SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada. The future of the web can't happen without you. Join us at MIX09 to help pave the way to the Next Web now. Learn more and register at http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
[courier-users] Corrupted Pythonfilter greylist_NotPassed database
Hello Gordon, Yesterday I installed your Pythonfilter for my users to improve fighting spam using greylisting module. It has been working well for a few hours. Unfortunately after that time the greylist_NotPassed database has been corrupted: Dec 11 18:37:29 xxx courierfilter: Uncaught exception in greylist doFilter function: class 'bsddb.db.DBRunRecoveryError':(-30975, 'DB_RUNRECOVERY: Fatal error, run database recovery -- PANIC: fatal region error detected; run recovery') Dec 11 18:37:29 xxx courierfilter: File /etc/courier/filters/active/pythonfilter, line 199, in processMessage Dec 11 18:37:29 xxx courierfilter: replyCode = i_filter[1](bodyFile, controlFileList) Dec 11 18:37:29 xxx courierfilter: File /usr/lib/python2.5/site-packages/pythonfilter/greylist.py, line 160, in doFilter Dec 11 18:37:29 xxx courierfilter: _sendersNotPassed.unlock() Dec 11 18:37:29 xxx courierfilter: File /usr/lib/python2.5/site-packages/pythonfilter/TtlDb.py, line 196, in unlock Dec 11 18:37:29 xxx courierfilter: self.db.sync() Dec 11 18:37:29 xxx courierfilter: File bsddb/__init__.py, line 293, in sync Dec 11 18:37:29 xxx courierfilter: return _DeadlockWrap(self.db.sync) In the Courier log file also I can see a lot of error messages like below: Dec 12 11:00:06 xxx courierfilter: Uncaught exception in greylist doFilter function: type 'exceptions.KeyError':'3d89ef72e9bbb1e4d677350b748b1832' Dec 12 11:00:06 xxx courierfilter: File /etc/courier/filters/active/pythonfilter, line 199, in processMessage Dec 12 11:00:06 xxx courierfilter: replyCode = i_filter[1](bodyFile, controlFileList) Dec 12 11:00:06 xxx courierfilter: File /usr/lib/python2.5/site-packages/pythonfilter/greylist.py, line 106, in doFilter Dec 12 11:00:06 xxx courierfilter: _sendersNotPassed.purge() Dec 12 11:00:06 xxx courierfilter: File /usr/lib/python2.5/site-packages/pythonfilter/TtlDb.py, line 214, in purge Dec 12 11:00:06 xxx courierfilter: if float(self.db[key]) minVal: Dec 12 11:00:06 xxx courierfilter: File bsddb/__init__.py, line 223, in __getitem__ Dec 12 11:00:06 xxx courierfilter: return _DeadlockWrap(lambda: self.db[key]) # self.db[key] Dec 12 11:00:06 xxx courierfilter: File bsddb/dbutils.py, line 62, in DeadlockWrap Dec 12 11:00:06 xxx courierfilter: File bsddb/__init__.py, line 223, in lambda Dec 12 11:00:06 xxx courierfilter: return _DeadlockWrap(lambda: self.db[key]) # self.db[key] I know that the greylist_NotPassed file is broken, because I checked it using `db4.6_verify` utility (Debian/Ubuntu db4.6-utils package): # db4.6_verify greylist_Passed # db4.6_verify greylist_NotPassed db4.6_verify: Page 20: hash page has bad prev_pgno db4.6_verify: Page 11: first page in hash bucket 9 has a prev_pgno db4.6_verify: Page 14: non-empty page in unused hash bucket 12 db4.6_verify: greylist_NotPassed: DB_VERIFY_BAD: Database verification failed I've fixed it using `db4.6_dump` and `db4.6_load` tools (the same package): # db4.6_dump greylist_NotPassed greylist_NotPassed.dump # rm -f greylist_NotPassed # db4.6_load greylist_NotPassed greylist_NotPassed.dump # db4.6_verify greylist_NotPassed It seems to be OK now, but I'm affaid to enable Pythonfilter again :) Here more details about my mail system. It includes a few virtual OpenVZ front-ends. My users Maildirs are shared between front-ends using NFS, so the greylist_* files was also shared via NFS. Do you think it matters? I've read about PostgreSQL support for Pythonfilter, but supposedly it's experimental. Also I don't have PostgreSQL database right now. What's your opinion about using NFS to sharing the greylist_* files? Do you think that PostgreSQL support is ready to use in stable and production environment? My best regards, Pawel -- SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada. The future of the web can't happen without you. Join us at MIX09 to help pave the way to the Next Web now. Learn more and register at http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
