Re: [ZODB-Dev] TypeError: unbound method beforeCompletion() must be called with Synchronizer instance as first argument (got Transaction instance instead)

2011-07-26 Thread Wichert Akkerman
On 07/26/2011 01:47 PM, Stéphane Klein wrote:
 Hi,

 I've this in my source code :

 from zope.interface import implements
 from transaction.interfaces import ISynchronizer

 class Synchronizer(object):
   implements(ISynchronizer)

   def beforeCompletion(self, transaction):
   print(beforeCompletion)

   def afterCompletion(self, transaction):
   print(afterCompletion, transaction)

   def newTransaction(self, transaction):
   print(newTransaction, transaction)

 ...

 transaction.manager.registerSynch(Synchronizer)

I bet this works:

transaction.manager.registerSynch(Synchronizer())


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] [RFC] ZEO: Allow non-packaged products

2011-04-29 Thread Wichert Akkerman
On 4/29/11 15:24 , Vincent Pelletier wrote:
 Hi.

 I need ZEO to be able to find non-packaged products for conflict resolution
 purposes. As ZEO AFAIK doesn't support this I gave it a quick try. I reached
 the works for me state, that I now would like to get feedback on.

 Basically, I transposed Zope's products config option to ZEO.

Out of curiosity, why do you need this? Zope itself is transitioning 
away from the products-magic, so I'm surprised you need a similar feature.

Wichert.
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] 3.10.0b6 traceback

2010-09-10 Thread Wichert Akkerman
I just got the following traceback from a system running 3.10.0b6:

Traceback (most recent call last):
   File 
/srv/software/buildout/eggs/WebError-0.10.2-py2.6.egg/weberror/errormiddleware.py,
 
line 162, in __call__
 app_iter = self.application(environ, sr_checker)
   File 
/srv/software/buildout/eggs/repoze.retry-0.9.4-py2.6.egg/repoze/retry/__init__.py,
 
line 88, in __call__
 app_iter = self.application(environ, replace_start_response)
   File 
/srv/software/buildout/eggs/repoze.tm2-1.0a5-py2.6.egg/repoze/tm/__init__.py, 
line 23, in __call__
 result = self.application(environ, save_status_and_headers)
   File 
/srv/software/buildout/eggs/repoze.filesafe-1.0-py2.6.egg/repoze/filesafe/__init__.py,
 
line 31, in __call__
 return self.app(environ, start_response)
   File 
/srv/software/buildout/eggs/Beaker-1.5.4-py2.6.egg/beaker/middleware.py, 
line 152, in __call__
 return self.wrap_app(environ, session_start_response)
   File 
/srv/software/buildout/eggs/repoze.bfg-1.3a12-py2.6.egg/repoze/bfg/router.py, 
line 91, in __call__
 root = root_factory(request)
   File /srv/software/buildout/src/s4u.site/s4u/site/run.py, line 111, 
in get_root
 return appmaker(conn.root())
   File 
/srv/software/buildout/eggs/ZODB3-3.10.0b6-py2.6-linux-x86_64.egg/ZODB/Connection.py,
 
line 359, in root
 return RootConvenience(self.get(z64))
   File 
/srv/software/buildout/eggs/ZODB3-3.10.0b6-py2.6-linux-x86_64.egg/ZODB/Connection.py,
 
line 257, in get
 self._cache.new_ghost(oid, obj)
AssertionError: The given oid is already in the cache

is that likely to be a ZODB bug, or a bug in our application?

Wichert.

-- 
Wichert Akkerman wich...@wiggy.net   It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] 3.10.0b6 traceback

2010-09-10 Thread Wichert Akkerman
And another one, not sure if it is related:

Traceback (most recent call last):
   File 
/srv/software/buildout/eggs/WebError-0.10.2-py2.6.egg/weberror/errormiddleware.py,
 
line 162, in __call__
 app_iter = self.application(environ, sr_checker)
   File 
/srv/software/buildout/eggs/repoze.retry-0.9.4-py2.6.egg/repoze/retry/__init__.py,
 
line 88, in __call__
 app_iter = self.application(environ, replace_start_response)
   File 
/srv/software/buildout/eggs/repoze.tm2-1.0a5-py2.6.egg/repoze/tm/__init__.py, 
line 23, in __call__
 result = self.application(environ, save_status_and_headers)
   File 
/srv/software/buildout/eggs/repoze.filesafe-1.0-py2.6.egg/repoze/filesafe/__init__.py,
 
line 31, in __call__
 return self.app(environ, start_response)
   File 
/srv/software/buildout/eggs/Beaker-1.5.4-py2.6.egg/beaker/middleware.py, 
line 152, in __call__
 return self.wrap_app(environ, session_start_response)
   File 
/srv/software/buildout/eggs/repoze.bfg-1.3a12-py2.6.egg/repoze/bfg/router.py, 
line 91, in __call__
 root = root_factory(request)
   File /srv/software/buildout/src/s4u.site/s4u/site/run.py, line 111, 
in get_root
 return appmaker(conn.root())
   File 
/srv/software/buildout/src/s4u.site/s4u/site/content/__init__.py, line 
27, in appmaker
 return zodb_root[cms_root]
   File /usr/lib/python2.6/UserDict.py, line 18, in __getitem__
 if key in self.data:
   File 
/srv/software/buildout/eggs/ZODB3-3.10.0b6-py2.6-linux-x86_64.egg/persistent/mapping.py,
 
line 30, in __get__
 return self.func(inst)
   File 
/srv/software/buildout/eggs/ZODB3-3.10.0b6-py2.6-linux-x86_64.egg/persistent/mapping.py,
 
line 99, in data
 data = self.__dict__.pop('_container')
KeyError: 'pop(): dictionary is empty'


On 2010-9-10 08:40, Wichert Akkerman wrote:
 I just got the following traceback from a system running 3.10.0b6:

 Traceback (most recent call last):
 File
 /srv/software/buildout/eggs/WebError-0.10.2-py2.6.egg/weberror/errormiddleware.py,
 line 162, in __call__
   app_iter = self.application(environ, sr_checker)
 File
 /srv/software/buildout/eggs/repoze.retry-0.9.4-py2.6.egg/repoze/retry/__init__.py,
 line 88, in __call__
   app_iter = self.application(environ, replace_start_response)
 File
 /srv/software/buildout/eggs/repoze.tm2-1.0a5-py2.6.egg/repoze/tm/__init__.py,
 line 23, in __call__
   result = self.application(environ, save_status_and_headers)
 File
 /srv/software/buildout/eggs/repoze.filesafe-1.0-py2.6.egg/repoze/filesafe/__init__.py,
 line 31, in __call__
   return self.app(environ, start_response)
 File
 /srv/software/buildout/eggs/Beaker-1.5.4-py2.6.egg/beaker/middleware.py,
 line 152, in __call__
   return self.wrap_app(environ, session_start_response)
 File
 /srv/software/buildout/eggs/repoze.bfg-1.3a12-py2.6.egg/repoze/bfg/router.py,
 line 91, in __call__
   root = root_factory(request)
 File /srv/software/buildout/src/s4u.site/s4u/site/run.py, line 111,
 in get_root
   return appmaker(conn.root())
 File
 /srv/software/buildout/eggs/ZODB3-3.10.0b6-py2.6-linux-x86_64.egg/ZODB/Connection.py,
 line 359, in root
   return RootConvenience(self.get(z64))
 File
 /srv/software/buildout/eggs/ZODB3-3.10.0b6-py2.6-linux-x86_64.egg/ZODB/Connection.py,
 line 257, in get
   self._cache.new_ghost(oid, obj)
 AssertionError: The given oid is already in the cache

 is that likely to be a ZODB bug, or a bug in our application?

 Wichert.



-- 
Wichert Akkerman wich...@wiggy.net   It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Problem with handling of data managers that join transactions after savepoints

2010-05-11 Thread Wichert Akkerman
On 5/11/10 19:41 , Chris Withers wrote:
 Jim Fulton wrote:
 I plan to implement A soon if there are no objections.

 Unless someone somehow convinced me to do D, I'll also add an
 assertion in the Transaction.join method to raise an error if a
 data manager joins more than once.
 Option A sounds sensible. It also means I won't have to change
 anything in the zope.sqlalchemy data manager.

 Very cool. I was hoping non-ZODB-data-manager authors
 were paying attention. :)

 If anyone knows of any other, I would appreciate someone forwarding
 this thread to them.

 zope.sendmail and MaildropHost have data managers.
 I've seen some file-based things that was a data manager and a few
 others in people's BFG stacks, maybe they can pipe up and/or let the
 others of those wsgi components know.

I wrote repoze.filesafe a while ago to do transaction-aware creation of 
files, which uses a datamanager. It never tries to join a transaction 
more than once, so should be fine.

Wichert.
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Using zodb and blobs

2010-04-14 Thread Wichert Akkerman
On 4/14/10 08:24 , Christian Theune wrote:

 I'm pretty sure it's not. IIRC fsync is defined by POSIX and absolutely
 requires the implementor to flush data physically to disk ensuring its
 persistency. If that doesn't hold true then all transactions are borked.

That was the problem with fsync on Linux: it effectively flushed all 
pending filesystem work, not just that for your current filehandle. That 
was needed to satisfy ordering constraints for the filesystem. And even 
though the result might be a lie since disks or other bits of hardwire 
can lie to you. It is generally better to use fdatasync() instead of 
fsync(), but you could still end up waiting much longer than you would 
expect.

Wichert.
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] redirect burps on unicode URLs

2010-02-25 Thread Wichert Akkerman
wrong list I suspect? :)


On 2/25/10 14:56 , Adam GROSZER wrote:
 Hello,

 Looks like zope.publisher burps on unicode URL which contain non-ascii
 chars. This is from a KGS 3.4 application, but looking at the source
 it still seems to have the same problems.

 opinions?

 ...
  self.request.response.redirect(url)
File 
 d:\home\.buildout\eggs\zope.publisher-3.4.6-py2.5.egg\zope\publisher\browser.py,
  line
 729, in redirect
  return super(BrowserResponse, self).redirect(location, status)
File 
 d:\home\.buildout\eggs\zope.publisher-3.4.6-py2.5.egg\zope\publisher\http.py,
  line 882,
 in redirect
  self.setHeader('Location', location)
File 
 d:\home\.buildout\eggs\zope.publisher-3.4.6-py2.5.egg\zope\publisher\http.py,
  line 676,
 in setHeader
  value = str(value)
 UnicodeEncodeError: 'ascii' codec can't encode character u'\xd6' in position 
 71: ordinal not in
 range(128)


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Data.fs size grows non-stop

2009-12-10 Thread Wichert Akkerman
On 2009-12-10 10:06, Pedro Ferreira wrote:
 The situation is simple: we have events (objects) that have to be
 indexed by date. We currently use an OOBTree with the days as keys and
 lists of objects as values. The objective is to avoid rewriting these
 lists each time something changes, since we can have tens of thousands
 of events in the same day. Some kind of btree-based list or set would
 work, I guess.

Why not store the events on some BTree or similar structure with a 
simple unique integer id, and use a separate BTree to map dates to these 
events? You can even use zc.catalog or similar tools to create that 
mapping - you would get a lot of query options for free with those as well.

Wichert.

-- 
Wichert Akkerman wich...@wiggy.net   It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Data.fs size grows non-stop

2009-12-07 Thread Wichert Akkerman
On 2009-12-7 17:34, Jim Fulton wrote:
 On Mon, Dec 7, 2009 at 11:00 AM, Alan Runyanruny...@gmail.com  wrote:
 ...
 A design pattern for RDBMS is to have 2 pools. READ pool and WRITE pool.
 Often the READ pool comes from some replica and WRITE is to the master.
 I'm unsure this pattern would work for ZODB.  I know Malthe was thinking
 about this but unsure if he had anything concrete.

 Would both pools be used by a single process?

 From what I gather on high-load sites it is common to use one pool for 
queries, and a second pool for updates. Both are used in the same 
process(es)/threads.

Wichert.

-- 
Wichert Akkerman wich...@wiggy.net   It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] [OT] NoSQL

2009-11-14 Thread Wichert Akkerman
On 11/13/09 21:33 , Shane Hathaway wrote:
 I've been studying how to build an enormous database based on what I
 know.  There are an incredible number of distributed databases these
 days, but all of them concern me in one way or another.

Can you share some of those concerns with us? I'ld be interested to hear 
what kind of problems you see.

Wichert.
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Zeo cache files : how to estimate the space?

2009-08-11 Thread Wichert Akkerman
On 8/11/09 15:29 , Dominique Arnoult wrote:
 Hello,
 i deal with a problem of /tmp. I have a 1 Go partition and cannot easily
 change it. I run a Zeo cluster of 7 clients. Since the lsof command, I
 can see 7 process and each process has 4 files opened of 100 Mb max.
 That corresponds of the cache-size directive of my zopecore.conf. But, I
 don't undertstand why there are 4 files and not only one by zeo client.
 Any can help me?

Each zserver thread has its own cache.

Wichert.
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] error installing ZODB 3.9.0a12 with Python 2.6 on Windows

2009-04-03 Thread Wichert Akkerman
On 4/3/09 1:41 PM, Chris Withers wrote:
 buildout said easy_install barfed:

 Installing zeoinstance.
 Getting distribution for 'ZODB3'.
 error: Setup script exited with error: None
 An error occured when trying to install ZODB3 3.9.0a12.Look above this
 message f
 or any errors thatwere output by easy_install.
 While:
 Installing zeoinstance.
 Getting distribution for 'ZODB3'.
 Error: Couldn't install: ZODB3 3.9.0a12

 Not exactly the most helpful error message in the world ;-)

 Any ideas?


buildout hides all compile errors unless you run it with -v (or -vv).

Wichert.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] create unique container keys

2009-04-02 Thread Wichert Akkerman
On 4/2/09 4:36 PM, Christian Theune wrote:
 ZODB has autoincrement support for one type: OIDs.

 The problem of autoincrement is that this needs to be handled outside
 the scope of transactions.

 In a distributed fashion this seems rather hard to do in comparison to
 just buying into conflicts.


I suspect it might be faster than the penalty of retrying a request on 
conflict if used properly.

W.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Can I use RelStorage with ZODB3.9.0

2009-03-05 Thread Wichert Akkerman
On 3/6/09 6:27 AM, eastxing wrote:
 Hi all,

 About one month ago, I asked a question about 'ZODB pack' and got 
 suggestions to update to new ZODB version.
 It took me one month to update my site from Plone2.5.5(with 
 Zope2.9.6-final,ZODB3.6.2) to Plone3.1.7(with Zope2.10.7,ZODB3.7.3).
 Then I update zasync(an schedule durable task framework used by Zope2) 
 to use its second-generation replacer -- 'zc.async', 'zc.async' needs 
 ZODB3.9.0, then I went further to update to use Zope2.11.7 and 
 ZODB3.9.0a12, so far so good.

Please note that Zope 2.11.7 is not supported for Plone 3.1.7. You 
should be able to use ZODB 3.9 with Zope 2.10.x though.

Wichert.

-- 
Wichert Akkermanwich...@wiggy.netIt is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] SVN woes.

2008-11-11 Thread Wichert Akkerman

On 11/11/08 4:10 PM, Izak Burger wrote:

Jim Fulton wrote:
   

I'm going to restore svn from a backup and see where that leaves us.
I'm going to disable svn access while I work on this.
 


Good luck :-) I know a little something about the hard work involved in
recovering subversion repos, in the last year we had TWO cases of
corrupted revisions in an svn repo we were managing. It had something to
do with concurrent commits and the fact that we were running
apache-mpm-worker (instead of prefork).
   


FWIW, the setup we have for svn.plone.org may be useful for others as 
well: we have two servers (svn.plone.org and svn-mirror.plone.org) with 
a svnsync setup to keep the mirror (almost) realtime in sync with the 
main server. There is an LDAP database which is replicated between the 
two servers as well which we use for authentication. This setup allows 
us to swich svn.plone.org to the other server in minutes if it dies, 
without any loss of data.


Wichert.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] PyTables Storage

2008-11-01 Thread Wichert Akkerman
On 11/1/08 1:17 PM, David Pratt wrote:
 Hi guys.

 repoze has package is useful for solving  the transaction management
 issue.

 http://svn.repoze.org/repoze.tm2


repoze.tm2 is a WSGI middleware wrapper around the transaction package 
that was split out from ZODB. It does not add magically add transaction 
support to a database - it merely coordinates transaction handling 
between multiple already transaction-aware entitities.

Wichert.
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] [Checkins] SVN: ZODB/trunk/src/persistent/wref.py Merged the rst formatting to the trunk

2008-10-13 Thread Wichert Akkerman
Christophe Combelles wrote:
 Zvezdan Petkovic a écrit :
 On Oct 12, 2008, at 5:51 AM, Christophe Combelles wrote:

 Log message for revision 92067:
  Merged the rst formatting to the trunk
 ...

 Here's an example. We'll start by creating a persistent object and
 -a refernce to it:
 +a reference to it::

 - import persistent.list
 - import ZODB.tests.util
 - ob = persistent.list.PersistentList()
 - ref = WeakRef(ob)
 - ref() is ob
 -True
 + import persistent.list
 + import ZODB.tests.util
 + ob = persistent.list.PersistentList()
 + ref = WeakRef(ob)
 + ref() is ob
 +True
 Why have these changes been made?
 This is now an invalid reST.

 The reST specification clearly states the difference between a literal  
 block (started with ::) and a doctest block.

 Please see the examples in the section Doctest blocks of the  
 specification at

 http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html

 At the bottom of that section is also stated.

  Indentation is not required for doctest blocks.

 Yes, they can be indented, but there is absolutely no need to change a  
 perfectly valid reST only to introduce a whitespace change in SVN and  
 a wrong literal block use.

 Can we, please, refrain from such changes in the future unless we are  
 absolutely certain that the changes are better than the original?

 This should be reverted.
 
 
 The only reason I did this is for readability in the APIDOC.

I would suggest that the better change is to fix apidoc.

Wichert.


-- 
Wichert Akkerman [EMAIL PROTECTED]It is simple to make things.
http://www.wiggy.net/   It is hard to make things simple.


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] [Checkins] SVN: ZODB/trunk/src/persistent/wref.pyMerged the rst formatting to the trunk

2008-10-13 Thread Wichert Akkerman
Christophe Combelles wrote:
 On Mon, 13 Oct 2008 08:32:52 +0200, Wichert Akkerman [EMAIL PROTECTED]
 wrote:
 Christophe Combelles wrote:
 Zvezdan Petkovic a écrit :
 On Oct 12, 2008, at 5:51 AM, Christophe Combelles wrote:

 Log message for revision 92067:
  Merged the rst formatting to the trunk
 ...

 Here's an example. We'll start by creating a persistent object and
 -a refernce to it:
 +a reference to it::

 - import persistent.list
 - import ZODB.tests.util
 - ob = persistent.list.PersistentList()
 - ref = WeakRef(ob)
 - ref() is ob
 -True
 + import persistent.list
 + import ZODB.tests.util
 + ob = persistent.list.PersistentList()
 + ref = WeakRef(ob)
 + ref() is ob
 +True
 Why have these changes been made?
 This is now an invalid reST.

 The reST specification clearly states the difference between a literal 
 block (started with ::) and a doctest block.

 Please see the examples in the section Doctest blocks of the  
 specification at

 http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html

 At the bottom of that section is also stated.

Indentation is not required for doctest blocks.

 Yes, they can be indented, but there is absolutely no need to change a 
 perfectly valid reST only to introduce a whitespace change in SVN and  
 a wrong literal block use.

 Can we, please, refrain from such changes in the future unless we are  
 absolutely certain that the changes are better than the original?

 This should be reverted.

 The only reason I did this is for readability in the APIDOC.
 I would suggest that the better change is to fix apidoc.
 
 
 I've finally found that it is still OK in the apidoc if I revert all
 indenting, but I only keep the __docformat__=restructuredtext
 Are you ok to keep just this change?

No objections here.

Wichert.
-- 
Wichert Akkerman [EMAIL PROTECTED]It is simple to make things.
http://www.wiggy.net/   It is hard to make things simple.


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Broken instances after refactoring in ZODB

2008-10-04 Thread Wichert Akkerman

Adam wrote:

On Fri, 2008-10-03 at 14:24 -0600, Shane Hathaway wrote:
   

Marius Gedminas wrote:
 

   1. create modules in their old locations that import all the classes
  from the new locations (and do nothing else).
   2. write a script that opens the DB, iterates over all persistent
  objects in it, touches every single one of them (obj._p_activate()
  followed by obj._p_changed = True) and commits the transaction.
   

Marius said it correctly, but I'd like to emphasize that it's not enough
to touch just the objects whose class changed, since persistent
references encode the module and class name.  It's best to just touch
everything.  You will have to use multiple transactions to touch
everything in a large database.
 


Thanks for that, guys, I've not used a mailing list like this before so
unsure how to respond.

If ZODB stores the Package.Module.Class name in the pickle would it be
possible for me to simply rename them in the binary file?

My confusion here is that I've globally imported everything from the
packages into the current namespace of my main module. ZODB shouldn't be
aware I've moved the modules since for all intents and purposes to
Python, they are still there.
   


It doesn't matter where you import it from or to - python uses the 
location of the actual implementation and ZODB uses that. If you move 
your implementation to another place you have to either update all 
objects in the ZODB or add module aliases.


Wichert.

--
Wichert Akkerman[EMAIL PROTECTED]It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] ZODB 3.8.1b8 tests do not run on OSX

2008-09-22 Thread Wichert Akkerman
When I try to run the tests for ZODB 3.8.1b8 using python2.4 on OSX 10.4 
nothing happens:

[snowflake;..mp/wichert/3.8.1b8]-118 bin/test
Running zope.testing.testrunner.layer.UnitTests tests:
   Set up zope.testing.testrunner.layer.UnitTests in 0.000 seconds.

and then nothing. If I hit ctrl-c I get this backtrace:

Running zope.testing.testrunner.layer.UnitTests tests:
   Set up zope.testing.testrunner.layer.UnitTests in 0.000 seconds.
^CTraceback (most recent call last):
   File /private/tmp/wichert/3.8.1b8/bin/test, line 22, in ?
 zope.testing.testrunner.run([
   File 
/Users/wichert/Library/eggs/tmpIjLqaT/zope.testing-3.6.0-py2.4.egg/zope/testing/testrunner/__init__.py,
 
line 33, in run
   File 
/Users/wichert/Library/eggs/tmpIjLqaT/zope.testing-3.6.0-py2.4.egg/zope/testing/testrunner/runner.py,
 
line 141, in run
   File 
/Users/wichert/Library/eggs/tmpIjLqaT/zope.testing-3.6.0-py2.4.egg/zope/testing/testrunner/runner.py,
 
line 216, in run_tests
   File 
/Users/wichert/Library/eggs/tmpIjLqaT/zope.testing-3.6.0-py2.4.egg/zope/testing/testrunner/runner.py,
 
line 365, in run_layer
   File 
/Users/wichert/Library/eggs/tmpIjLqaT/zope.testing-3.6.0-py2.4.egg/zope/testing/testrunner/runner.py,
 
line 301, in run_tests
   File 
/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/unittest.py,
 
line 281, in __call__
 return self.run(*args, **kwds)
   File 
/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/unittest.py,
 
line 260, in run
 testMethod()
   File 
/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/doctest.py,
 
line 2151, in runTest
 failures, tries = runner.run(
   File 
/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/doctest.py,
 
line 1381, in run
 return self.__run(test, compileflags, out)
   File 
/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/doctest.py,
 
line 1248, in __run
 compileflags, 1) in test.globs
   File doctest zeo-fan-out.test[48], line 1, in ?
   File /private/tmp/wichert/3.8.1b8/src/ZEO/tests/forker.py, line 
198, in shutdown_zeo_server
 ack = s.recv(1024)
KeyboardInterrupt

Wichert.

-- 
Wichert Akkerman[EMAIL PROTECTED]It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] what could cause this kind of cache corruption?

2008-08-02 Thread Wichert Akkerman

Jim Fulton wrote:


On Aug 1, 2008, at 11:13 AM, Wichert Akkerman wrote:


Jim Fulton wrote:


On Jul 31, 2008, at 1:53 PM, Chris Withers wrote:
What I'd *really* like is a stable zodb release with Christian's 
patches for zeoraid and Shane's patches for RelStorage that then 
feeds through into a stable release of Zope 2.



I'm not familiar with Christians patches. If they are bug fixes, 
they could make it into 3.8.  I plan to include Shane's patches in 
3.9. I'm not sure when there will be a stable release of 3.9.


Do you know where there will be a new 3.8 stable release?


3.8 is stable now.  We could make a 3.8.1 any time.  I think there are 
probably going to need to be fixes to blob packing code, which could 
go into 3.8.1 or 3.8.2, depending on timing.


Did you mean 3.9? If so, I have no idea. It's not a very high priority 
to me.


I meant 3.8. I'ld like to use both the ZEO authentication fix and your 
caching fixes.


Wichert.

--
Wichert Akkerman[EMAIL PROTECTED]It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] RelStorage: Clearing temp_store in replication-friendly way

2008-07-24 Thread Wichert Akkerman

Stephan Richter wrote:

On Thursday 24 July 2008, Shane Hathaway wrote:
   
P.S.: I am quite excited about the memcached support. Does it just  
work? I.e. can I run my ZODB in RAM now? ;-)
   

I expect the new memcache support to be safe for everyone to use, but we
still require the main database to be connected at all times, since
memcache provides no ACID properties by itself.  We unfortunately can't
do obvious things like cache the current transaction ID for an object,
since that would break MVCC.  What we do cache is:

1. The current tid, given an oid and the transaction ID that is active
for the current connection.

2. The pickle given an oid and tid.

These should both help read-heavy databases, but might be detrimental
for write-heavy databases.  My performance test suite, which writes a
lot, produced slightly *worse* results with memcache enabled.
 


We should get Brian Aker into this discussion, since is one of the main
architects of MySQL and one of the founders of memcached.

As Shane knows, I have some access to Brian these days, so we could have an
online meeting talking about it.
   


Well, if I can make a suggestion :)

From what I understand a new memcache client library was written for 
MySQL (libmemcache is quite horrible). Python bindings for that library 
would improve many things, including this :)


Wichert.

--
Wichert Akkerman[EMAIL PROTECTED]It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Re: Ruby/Smalltalk OODB

2008-06-02 Thread Wichert Akkerman

Martijn Faassen wrote:

Hi there,

Christian Theune wrote:

this might be interesting to ZODB users and developers:
http://rss.slashdot.org/~r/Slashdot/slashdot/~3/302177093/article.pl


What I find interesting is that Python has had such a thing for about 
a decade (the ZODB), and a mostly vaporware announcement in the Ruby 
world makes such a splash.


That's because most things Zope are not very good with PR.

The future is already here folks. So, what's the status of the 
zodb.zope.org project to actually promote it better? It's easy to know 
what to write on the homepage, just go to the Ruby buzz, translate the 
hype in terms of the ZODB, tone it down some, and add that it's been 
battle-tested for a decade.


I assume you mean http://new.zope.org/projects/zodb ? It's there, 
waiting for the rest of that site to be finished.


Wichert.


--
Wichert Akkerman[EMAIL PROTECTED]It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Re: Ruby/Smalltalk OODB

2008-06-02 Thread Wichert Akkerman

Tim Cook wrote:

On Mon, 2008-06-02 at 15:45 +0200, Martijn Faassen wrote:

   

Lots documentation on the ZODB is available, scattered around the web.
Let's gather it into one place for starters.

So, please, someone actually do something about this rotten situation?
Finally? Please? Please?

Regards,

Martijn

P.S. This is not a complaint to all the people who are doing something.
We just need a bit of effort from some more people to get this done.
 


I'm feeling some guilt pangs here.  :-)

I benefit from Zope and the ZODB especially.  I know that I am a little
better at PR than I am at the deep technical aspects (not very good) so
I will initially gather (I already have lots of bookmarks) all the info
I can find and start building something coherent out of it. Can I have a
few minutes of attention to emails now and then from experts?
   


Of course. Tech people are good at finding faults in documentation, but 
not good at writing documentation.



I am NOT good at the flashy layout and graphics stuff that attracts the
attention of people.  I will needs some help in that respect.  But I can
organize information into something that new-comers can learn from.
   


We already have a complete design with proper layouts on new.zope.org. 
We just need more content.


Wichert.


--
Wichert Akkerman[EMAIL PROTECTED]It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] RelStorage and BLOBs

2008-05-07 Thread Wichert Akkerman

Christian Theune wrote:

On Tue, May 06, 2008 at 01:47:45PM -0600, Shane Hathaway wrote:
  

David Durham, Jr. wrote:


My questions is whether or not RelStorage supports BLOBs.  My
situation is that I have a Plone site with user uploaded images and
other files, that should presumably be stored as BLOBs with
RelStorage.  I'm open to placing BLOBs outside of RelStorage if that's
reasonably accomplished.
  
I think an NFS mount is still a better place for BLOBs than a relational  
database.  If we put BLOBs in a relational database, we'd have to  
carefully dance with inter-thread connections and simulated file handles  
that may stay open for a long time.  With NFS those concerns are already  
taken care of.


The normal way to set up BLOB storage in zope.conf is to create a  
blobstorage wrapper around a storage configuration.  That might just  
work with RelStorage, but I haven't tried it yet.



At leasts its a scenario that is intended to work. I'd be happy about feedback
of people who use it.
  


For some of us a the main reason for using RelStorage is to be able to 
satisfy the 'all data must be stored in our oracle cluster' requirement 
from enterprise customers. That means we can not use NFS mounts to store 
blobs.


Wichert.

--
Wichert Akkerman [EMAIL PROTECTED]   It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] ZEO authentication change backported to 3.7 and 3.8

2008-05-03 Thread Wichert Akkerman
Since nobody objected I have backported the ZEO authentication patch 
from trunk to the 3.7 and 3.8 branches. In the meantime someone also 
filed a bugreport for this issue 
(https://bugs.launchpad.net/zodb/+bug/220856) which is fixed by this change.


Wichert.

--
Wichert Akkerman [EMAIL PROTECTED]   It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZEO authentication change backported to 3.7 and 3.8

2008-05-03 Thread Wichert Akkerman

Jim Fulton wrote:
I don't see any mention of this under 3.8.1 in the NEWS.txt on the 
3.8.1 branch.


Please add a news item.


Sorry about that, fixed.

Wichert.

--
Wichert Akkerman [EMAIL PROTECTED]   It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Completing ZEO authentication support

2008-04-24 Thread Wichert Akkerman

Shane Hathaway wrote:

Jim Fulton wrote:

On Apr 23, 2008, at 11:39 AM, Shane Hathaway wrote:


Andreas Jung wrote:
--On 23. April 2008 09:20:00 -0600 Shane Hathaway 
[EMAIL PROTECTED]

wrote:
AFAIK, Jim is working on major ZODB changes for ZODB 3.9.  Could  
we move
those major changes into ZODB 3.10 and create ZODB 3.9 with only  
modest

feature enhancements, then use ZODB 3.9 with Zope 2.11?

Zope 2.11 will ship with ZODB 3.8 - not with 3.9.
I am suggesting we change that decision and simultaneously make 
ZODB  3.9 only slightly different from ZODB 3.8.


The ZODB trunk already has some significant changes from 3.8.


I'm sorry, I must have expressed the idea unclearly.  Here is what I 
am suggesting:


1. Declare that the ZODB trunk is the future ZODB 3.10 rather than 
ZODB 3.9.


2. Create a ZODB 3.9 branch as an exact copy of ZODB 3.8.

3. Check in only a few minor patches to the new ZODB 3.9 branch.

4. Include the new, less-different ZODB 3.9 in Zope 2.11.

That way, Zope 2.11 can include some minor enhancements to ZODB that 
we wouldn't be able to include if we shipped Zope 2.11 with ZODB 3.8 
(since ZODB 3.8 is already released).


I think the onle proposed changes that would be in this new 3.9 are your 
invalidation patch and my authentication patch, is that correct? 
Considering the interest there appears to be in RelStorage that may make 
a lot of sense - currently we need to monkey-patch ZODB to use 
RelStorage and I'ld rather not have to do that.


Now I'ld love to have a release from current trunk as well to be able to 
have an official release with blob support, but I understand that that 
may not be realistic short term.


Wichert.

--
Wichert Akkerman [EMAIL PROTECTED]   It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Completing ZEO authentication support

2008-04-24 Thread Wichert Akkerman

Shane Hathaway wrote:

Wichert Akkerman wrote:
I think the onle proposed changes that would be in this new 3.9 are your 


Scratch the proposal--I suggested it when I didn't realize Zope 2.11b1 
was already out.


invalidation patch and my authentication patch, is that correct? 
Considering the interest there appears to be in RelStorage that may 
make a lot of sense - currently we need to monkey-patch ZODB to use 
RelStorage and I'ld rather not have to do that.


Minor correction: the patch is a normal patch rather than a monkey 
patch.  I don't think a monkey patch is appropriate for this.


Right, I misremembered. Right now for deployments have to patching the 
ZODB shipped with Zope to be able to use RelStorage. I thought we do 
that using a monkey patch, but we appear to be doing that using a 
buildout recipe instead.


Wichert.

--
Wichert Akkerman [EMAIL PROTECTED]   It is simple to make things.
http://www.wiggy.net/  It is hard to make things simple.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Completing ZEO authentication support

2008-04-23 Thread Wichert Akkerman

Shane Hathaway wrote:

Wichert Akkerman wrote:
I would like to get that fix in for Zope 2.11, which will use ZODB 
3.8. Are there any objections to changing this there as well? This is 
a bit of a grey area between a bugfix and completing an unfinished 
feature, and I'm do not know what the rules for ZODB development are.


I have a similar request: I'd like the invalidation polling patch 
(needed by RelStorage) to be included in Zope 2.11.  Is there some way 
that can be done?


AFAIK, Jim is working on major ZODB changes for ZODB 3.9.  Could we move 
those major changes into ZODB 3.10 and create ZODB 3.9 with only modest 
feature enhancements, then use ZODB 3.9 with Zope 2.11?


Zope 2.11 needs a few changes to work with ZODB 3.9 as well: ZODB.Mount was 
removed, which TemporaryFolder is still using.


Wichert.

--
Wichert Akkerman [EMAIL PROTECTED]It is simple to make things.
http://www.wiggy.net/   It is hard to make things simple.

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev