Re: [ZODB-Dev] grab zodb dict keys by search term
Hi Tamer,
It depends on what you're trying to achieve. BTrees should not be mutated
whilst iterating over them, so depending on how you have structured your
data, you might do something like:
for key in [x for x in BTree.keys() if x.startswith('aa')]:
# Do stuff
Or perhaps something else like:
for key in list(BTree.keys()):
if key['subkey'] == 5:
yield BTree[key]
...Or some other pattern depending on your use case.
The key here is that the ZODB really allows you to cut out the middle
man where previously you might store data in an RDBMS and then do
processing/transformation of that data in Python after extracting it with
SQL (and then re-storing it again afterwards), instead now you can just
directly process the data as if it was in Python objects waiting the
entire time.
Hope this helps.
regards,
Darryl Dixon
Winterhouse Consulting Ltd
Hi people!
I am working a lot with IOB and OOB Trees as well with PersistentDict
and PersistentList to store my Data in ZODB.
I want to ask if there is a way to grab data from a dataset by search
time.
I know only the SQL way:
select lower(COL1) from table1 where COL1 LIKE 'aa%'
so usually I want to get the entryies only that starts with 'aa'
do I have to iterate the entire dict with an interator or are there
builtin functions that could accomplish those tasks for me ?!
I would kindly thank you
Tamer
___
For more information about ZODB, see http://zodb.org/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
___
For more information about ZODB, see http://zodb.org/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Implications of eliminating use of ZOID_SEQ in Oracle
Hi, We have found some extremely undesirable behaviour when using RelStorage with Oracle RAC in the interaction of ZOID_SEQ with various Oracle tuning parameters such as the CACHE value and ORDER/NOORDER. What we are considering is dropping use of ZOID_SEQ entirely and simply reverting to SELECT MAX(zoid) FROM object_state. Because the existing code optimistically pre-allocates OIDs anyway based on the last value of ZOID_SEQ, we see this as being a huge win in eliminating calls to the database. Because Oracle automatically generates an index when PRIMARY KEY is set, the calls to MAX(zoid) are very fast when we have tested them. Questions: * What downsides can the audience here anticipate? * What are the chances that a change like this could make it in to the core RelStorage distribution? * Any other considerations that anyone can comment on? regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsultign.com ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZeoServer, multiple storages and open file handles
On Wed, Oct 24, 2012 at 12:33 AM, Tim Godfrey t...@obsidian.com.au wrote: Hi Jim Do you have any idea as to why people recommend against many storages under a single Zeo? Seriously? Go back and read the thread. Seriously mate. The why is very vague. There is nothing vague about the first reply to your question: ZEO does not handle each storage in a separate thread, so you're underusing multiple CPUs if you have them. Multiple ZEO servers would also spread the open sockets among them. There could still be a leak, but it'd take longer to exhaust file descriptors. Yes but this is not really strictly accurate. Because it is primarily IO bound, it is certainly possible to do more than 1 CPU worth of work (which Jim notes in a followup email). A useful thread that talks through some of the issues (as they were in 2006, anyway): http://www.gossamer-threads.com/lists/zope/users/201868?do=post_view_threaded#201868 From my point of view, the biggest reason that I have seen *in practice* for splitting off storages into their own separate ZEO was the issue noted here: http://www.gossamer-threads.com/lists/zope/users/201929?do=post_view_threaded#201929 Where Dieter talks about the vote phase on large transactions preventing other requests from running. I have seen this happen in practice (with 2GB transactions) and other requests were, without doubt, prevented from being serviced during this phase. I hope this helps, regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] zeopack error
On Thu, Feb 9, 2012 at 11:32 AM, Marius Gedminas mar...@gedmin.as wrote: On Thu, Feb 09, 2012 at 02:26:10AM +0100, Kaweh Kazemi wrote: Thanks a lot for the investigation, Marius. That was awesome. Seems to me, I need to upgrade to Python 2.7 and be done with the problem then. :/ I wonder, though, if ZODB ought to use the 'noload' method of cPickle.Unpickler, which is undocumented and broken on 2.6 (and gone on 3.x, AFAIU). noload exists for ZODB. We ZC (DC at the time) wrote cPickle because ZODB needed a highly-optimized pickler. There are a number of decisions we made specifically with ZODB in mind. I don't think the Python 3 version of cPickle reflects these priorities. I strongly suspect we'll end up having our own cPickle in the long run. It would be wonderful if ZODB settled on an internal version of cPickle that didn't use recursion so we could Pickler.dump() OFS.File objects bigger than 32mb :-) Cheers, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Transaction notes
Hi Jeff, I'd like to add something to our application framework code so that when page messages are displayed to the user (simple messages like 'saved changes' or 'deleted ') that they are also added to the transaction description (using the 'note' method on transactions). [...snip...] My question is: Will using transaction 'note' in a non-writing transaction cause a record to be written out when using FileStorage and/or ZEO? In short, no. Longer answer: I'm assuming by 'non-writing transaction' you are meaning a transaction which has been abort() or doom() -ed. If you mean instead an 'empty' transaction that has no actual changed data, but which still calls 'commit()'; then the answer is still 'no' and the reason is simple - against which database would the note be written? Cheers, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RelStorage - high ConflictError rate in newly converted DB
Hi Shane, On 11/07/2011 03:15 PM, Darryl Dixon - Winterhouse Consulting wrote: Hi All, We have just converted a site with a handful of editors over from ZEO to RelStorage on Oracle and and the rate of ConflictErrors has gone through the roof (5 an hour average continuously for days). [...snip...] Currently we are using a very basic RelStorage configuration without any additional bits like memcache. I have a few thoughts: - You aren't using poll-interval, are you? Make sure that is disabled. - When you switched from ZEO to Oracle without turning on memcache, you lost the ZEO cache and didn't replace it with anything, so now loading objects takes longer, which may lead to more conflicts. I would try turning on memcache. - Do you have multiple storages / databases? Make sure each uses a different Oracle lock ID. We are not using the poll-interval, so clear on that one. I wasn't clear that memcache would assist in this situation - I will see about enabling it in this environment, but there are a few questions I need a better understanding for with this around what exactly are the characteristics of memcache integration: * What is the situation with using multiple memcache instances (eg, one per machine - will this lead to cache incoherency - inconsistency between instances, etc? * What is the failure scenario for a memcache instance dying - is it handled gracefully by RelStorage or will there be hanging/blocking/tracebacks until it returns? * Will this potentially exacerbate the ancient ZEO-Client-Race condition that occurs with load-balancing when data changes? (To be clear, I am referring to when one Zope instance updates/commits some data, and then the client browser makes a new request which is directed to a second Zope instance, which has not yet received an invalidation message for the changed objects, and it returns, eg, a 404 to the client). Any guidance on the above once again very gratefully received. With regards the the commit lock - yes, we use multiple storages and each uses a different commit-lock-id. Thanks heaps for your help, regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] RelStorage - high ConflictError rate in newly converted DB
Hi All, We have just converted a site with a handful of editors over from ZEO to RelStorage on Oracle and and the rate of ConflictErrors has gone through the roof (5 an hour average continuously for days). This seems very surprising to me. What options/changes should we look at configuring to lower this back down? We used to use ZEO with an invalidation-queue-size of 5000 and a client-cache of 1000. Currently we are using a very basic RelStorage configuration without any additional bits like memcache. Any guidance gratefully appreciated. regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RelStorage with Oracle and separation of rights
Hi Shane, On 10/30/2011 07:05 PM, Darryl Dixon - Winterhouse Consulting wrote: Hi All, Part of the setup of our Oracle RelStorage environment involves the DBAs wanting to separate ownership of the schema from the rights to actually use the schema. In other words, user A owns all the tables etc that RelStorage creates, but then when it comes to actually making use of them, the application (Zope) uses user B for the connection. [...snip...] Is there any straightforward way to resolve this? I think you're saying your DBAs want to be in charge of all DDL (Data Definition Language like CREATE TABLE) while apps can only use DML (Data Manipulation Language like SELECT, INSERT, UPDATE, and DELETE). Setting the create-schema option to false should do the trick. I wonder if you'll run into problems with OID management, since some DDL is required there, but other than that and creating the schema, everything RelStorage does with Oracle is (or should be) DML. BTW, I wonder why your DBAs bother with a shadow schema. They can simply revoke the permission to use specific DDL instead. Thanks for the informative reply. We seem to have got this working with the setup described (everything linked via synonyms and a different user for the Zope connection from the one that owns the objects in Oracle). The primary changes made were: 1) create-schema false in zope.conf 2) A set of GRANTs in Oracle for user B to access the synonyms which were more restrictive than full ownership (I'll talk to the DBAs tomorrow and send the full list) 3) TRUNCATE TABLE changed to DELETE FROM (see below) The last change was to solve an intractable problem with the packing code - the Oracle scriptrunner uses 'TRUNCATE TABLE' when packing, but in Oracle, one cannot TRUNCATE a synonym, even with complete permissions to the source object. Instead I altered the code to use DELETE FROM which seems to perform adequately for our needs. Everything now seems to be working successfully. Thanks all (especially Shane) for a great product and the assistance at this last hurdle. regards, Darryl Dixon Winterhouse Consulting Ltd ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] RelStorage with Oracle and separation of rights
Hi All, Part of the setup of our Oracle RelStorage environment involves the DBAs wanting to separate ownership of the schema from the rights to actually use the schema. In other words, user A owns all the tables etc that RelStorage creates, but then when it comes to actually making use of them, the application (Zope) uses user B for the connection. Apparently this is a standard Oracle risk-mitigation strategy and the DBAs are quite firm on this requirement. The way they achieve this is by creating what I would call a 'shadow schema' for user B that consists of a big pile of synonyms and grants pointing back at the 'real' user A schema. This doesn't work with parts of RelStorage. For example, it seems that RelStorage will unilaterally try to 'CREATE TABLE pack_lock' even though that table already exists. When this code runs as user A, Oracle seems to treat this as a no-op and continues on its merry way. When this code runs as user B, Oracle throws a fit and complains that cx_Oracle.DatabaseError: ORA-00955: name is already used by an existing object. Because presumably for user B, 'pack_lock' already exists, but it is a *synonym* not an actual, for-real table. I suspect that other such situations may arise involving, eg Indexes or Sequences. Is there any straightforward way to resolve this? regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] RelStorage - zodbconvert script and the --clear flag
Hi Folks, Just had a quick query from my friendly local DBA; he wanted to know why --clear was using DELETE rather than TRUNCATE; his comments were along the lines of: * TRUNCATE creates no UNDO * TRUNCATE cleans out the indexes nicer Is there any real downside or gotcha I should be aware of to following this suggestion in this instance? regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RelStorage - zodbconvert script and the --clear flag
Just had a quick query from my friendly local DBA; he wanted to know why --clear was using DELETE rather than TRUNCATE; his comments were along the lines of: * TRUNCATE creates no UNDO * TRUNCATE cleans out the indexes nicer Is there any real downside or gotcha I should be aware of to following this suggestion in this instance? To be clear: This is RelStorage 1.5.0 running against Oracle RAC 10g. I tried changing the line in question (relstorage/adapters/schema.py line 895) to TRUNCATE TABLE %s % table, and the following error is generated: ORA-02266: unique/primary keys in table referenced by enabled foreign keys So I guess without a bunch of work it will not be possible. regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] RelStorage: Error Installing Plone Site on Oracle RAC
Hi Folks, We have a Plone (3.3.2) infrastructure currently residing on ZEO (ZODB). We are looking to swap some of the databases to a RelStorage backed DB instead (Oracle RAC). We have been running the RelStorage (1.4.1) connection to the RAC without error for a while now for storing some metadata and other data structures (general Zope stuff, not Plone specific). We have added a new schema to our Oracle RAC database and mount point inside Zope, attempted to add a Plone site, and get the following error. If everything else remains the same, but the ZODB Mount in the zope.conf gets pointed back to a regular ZEO/ZODB, it works fine: 2010-12-03T10:31:14 ERROR Zope.SiteErrorLog 1291325474.290.650785606429 http://web-cms-edit-node3.dev.internal:8080/nu-gov-companies-www-preview/manage_addProduct/CMFPlone/addPl oneSite Traceback (innermost last): Module ZPublisher.Publish, line 119, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 42, in call_object Module Products.CMFPlone.factory, line 86, in addPloneSite Module Products.GenericSetup.tool, line 390, in runAllImportStepsFromProfile - __traceback_info__: profile-Products.CMFPlone:plone Module Products.GenericSetup.tool, line 1179, in _runImportStepsFromContext Module Products.GenericSetup.tool, line 1090, in _doRunImportStep - __traceback_info__: content Module Products.CMFCore.exportimport.content, line 37, in importSiteStructure Module Products.CMFCore.exportimport.content, line 166, in import_ Module Products.CMFCore.exportimport.content, line 185, in _makeInstance Module Products.CMFCore.TypesTool, line 716, in constructContent Module Products.CMFCore.TypesTool, line 278, in constructInstance Module Products.CMFCore.TypesTool, line 290, in _finishConstruction Module Products.CMFCore.CMFCatalogAware, line 148, in notifyWorkflowCreated Module Products.CMFCore.WorkflowTool, line 292, in notifyCreated Module Products.CMFCore.WorkflowTool, line 642, in _reindexWorkflowVariables Module Products.Archetypes.CatalogMultiplex, line 66, in reindexObjectSecurity Module Products.ZCatalog.CatalogBrains, line 52, in _unrestrictedGetObject Module OFS.Traversable, line 175, in unrestrictedTraverse - __traceback_info__: (['Members', 'site'], '') IndexError: string index out of range Any thoughts on why RelStorage usage might cause this? regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RelStorage: Error Installing Plone Site on Oracle RAC
[...snip...] We have added a new schema to our Oracle RAC database and mount point inside Zope, attempted to add a Plone site, and get the following error. If everything else remains the same, but the ZODB Mount in the zope.conf gets pointed back to a regular ZEO/ZODB, it works fine: 2010-12-03T10:31:14 ERROR Zope.SiteErrorLog 1291325474.290.650785606429 http://web-cms-edit-node3.dev.internal:8080/nu-gov-companies-www-preview/manage_addProduct/CMFPlone/addPl oneSite Traceback (innermost last): Module ZPublisher.Publish, line 119, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 42, in call_object Module Products.CMFPlone.factory, line 86, in addPloneSite Module Products.GenericSetup.tool, line 390, in runAllImportStepsFromProfile - __traceback_info__: profile-Products.CMFPlone:plone Module Products.GenericSetup.tool, line 1179, in _runImportStepsFromContext Module Products.GenericSetup.tool, line 1090, in _doRunImportStep - __traceback_info__: content Module Products.CMFCore.exportimport.content, line 37, in importSiteStructure Module Products.CMFCore.exportimport.content, line 166, in import_ Module Products.CMFCore.exportimport.content, line 185, in _makeInstance Module Products.CMFCore.TypesTool, line 716, in constructContent Module Products.CMFCore.TypesTool, line 278, in constructInstance Module Products.CMFCore.TypesTool, line 290, in _finishConstruction Module Products.CMFCore.CMFCatalogAware, line 148, in notifyWorkflowCreated Module Products.CMFCore.WorkflowTool, line 292, in notifyCreated Module Products.CMFCore.WorkflowTool, line 642, in _reindexWorkflowVariables Module Products.Archetypes.CatalogMultiplex, line 66, in reindexObjectSecurity Module Products.ZCatalog.CatalogBrains, line 52, in _unrestrictedGetObject Module OFS.Traversable, line 175, in unrestrictedTraverse - __traceback_info__: (['Members', 'site'], '') IndexError: string index out of range Any thoughts on why RelStorage usage might cause this? So, it seems that this is a Heisenbug - I instrumented OFS.Traversable to catch the error, log some stuff, and re-raise... And the site add then worked without any exception being thrown, no logging appear, etc :-( The bug otherwise was replicated twice prior to the code change. That is somewhat worrisome. Anyone who can shed some light? regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] zodb monitor port / tailing a .fs
On 14/10/2010 02:40, Darryl Dixon - Winterhouse Consulting wrote: ...and fsdump appears to have no way of saying show me the last few transactions in this 35GB file :-( Yeah it's pretty verbose. I usually just pump the output to a text file ...yeah, but a 35GB zodb is going to result in, what, a 120? 200? GB text file. I don't think I actually have that much disk space lying around on these servers, let alone how long fsdump would take to produce it ;-) No, definitely not. The fsdump does not dump textual representations of complete objects, but rather, a summary of each transaction including when it began, what offset into the db file it lives at, and what OIDs including class information were included in that transaction. The text output is much smaller than the actual DB. The class information is what might give you the best clue as to what caused the transaction - if it turns out to be a class(es) that you only use for a specific project, or in a specific content object etc, then that may be a solid hint for the trigger. regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] zodb monitor port / tailing a .fs
On 13/10/2010 15:23, Jim Fulton wrote: You can connect to the monitor port in 3.9 and earlier, if the monitor port is configured. In 3.10, the monitor server is replaced by a ZEO client method, server_status. This tells you much the same information that's in the log messages. Okay, monitor port up and running now. I see commits listed when I'm not expecting any. Do we have any kind of tail -f /some/filestorage.fs yet? (or have we always had such a tool) to see what the last few transactions in the underlying file storage look like in a human-readable form? fsdump.py gets you pretty close (ZODB/scripts/fsdump.py). Between that and the Undo log for the DB inside Zope, you might be able to figure it out... regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] zodb monitor port / tailing a .fs
On 13/10/2010 15:23, Jim Fulton wrote: You can connect to the monitor port in 3.9 and earlier, if the monitor port is configured. In 3.10, the monitor server is replaced by a ZEO client method, server_status. This tells you much the same information that's in the log messages. Okay, monitor port up and running now. I see commits listed when I'm not expecting any. Do we have any kind of tail -f /some/filestorage.fs yet? (or have we always had such a tool) to see what the last few transactions in the underlying file storage look like in a human-readable form? fsdump.py gets you pretty close (ZODB/scripts/fsdump.py). Between that and the Undo log for the DB inside Zope, you might be able to figure it out... Of course, there is also the fstail.py script in the same directory, but it has a different level of detail than fsdump, and at least in my version (ZODB 3.7.3) it has a typo bug where Main() calls Main(path, ntxn) instead of main(path, nxtn) (note capitalisation). regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] zodb monitor port / tailing a .fs
On 14/10/2010 01:28, Darryl Dixon - Winterhouse Consulting wrote: Do we have any kind of tail -f /some/filestorage.fs yet? (or have we always had such a tool) to see what the last few transactions in the underlying file storage look like in a human-readable form? fsdump.py gets you pretty close (ZODB/scripts/fsdump.py). Between that and the Undo log for the DB inside Zope, you might be able to figure it out... ..well, the Undo log shows no entries ;-) ...and fsdump appears to have no way of saying show me the last few transactions in this 35GB file :-( Yeah it's pretty verbose. I usually just pump the output to a text file somewhere and then you can use all the usual tools to slice'n'dice it afterwards. Each transaction is easy to find in the output, so it's not too hard to jump to the end of the file and search back upwards. D ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] RelStorage on Oracle RAC (doing now)
Folks, We have a client with an existing Oracle RAC infrastructure (4 node RAC). We are going to using RelStorage + cx_Oracle to connect to this from RedHat Linux servers, running Plone 3.3.x. This note is to say we're here and to provide some technical notes on our experience for the benefit of other implementers. First, we have created a buildout recipe which knows how to install the necessary oracle instantclient libraries as instructed by the cx_Oracle package - hopefully we will pypi-ify this recipe shortly. It sets things up such that the cx_Oracle egg can then be built and installed by buildout simply by specifying eggs = cx_Oracle. Second, the recipe exports the necessary values for ORACLE_HOME and LD_LIBRARY_PATH in its options so that other buildout parts (eg, [instance]) can take these and put them in to zope.conf's environment section. Third, it accepts tnsnames entries and puts them in to an appropriate, instance-specific tnsnames.ora file ready for Zope to find and use. From that point, all that remains is for the user to specify the usual RelStorage zope.conf options (%import relstorage, etc) and use the oracle tag with user/password/dsn. This works. We are going to be testing RAC failover etc shortly to gauge the failure characteristics of the client connector library. Our expectation is that it should probably be seamless. One (aesthetic?) problem is that the string for the relstorage_util package is hard-coded to SYS; eg, in relstorage.adapters.locker.OracleLocker.hold_commit_lock is the string sys.relstorage_util.request_lock. Our DBA finds this distasteful and would prefer to keep the relstorage_util package in the Zope schema that he has created to partition this stuff off. This all works fine, except that we must currently manually hotfix locker.py to remove 'sys.' from the front of that string. Options for handling this differently might be: 1) Hardcode the string as relstorage_util.request_lock and advise users/provide the necessary SQL to add a synonym into their schema for sys.relstorage_util 2) Make this a configurable parameter inside the oracle zope.conf tags that would override the default string of 'sys.relstorage_util'; along the lines of 'utilpackage relstorage_util' or 'utilpackage sys.relstorage_util' 3) ??? Better database/oracle people than me can comment here :) Aside from that everything has been relatively smooth. We are going to be running this up on both 32bit and 64bit Linux clients. It is a shame that the Oracle client libraries downloads are hidden behind both a login/registration mechanism and a hokey javascript 'Accept this licence' scheme, which prevents them from being automatically fetched and installed a-la-buildout. We are keeping the necessary copies on our internal buildout index server to make deployment to our various environments smooth here. We were surprised to discover that with cx_Oracle and the oracle-instantclient libraries, that connection passwords must be supplied in lower case in zope.conf, regardless of whether upper case was used when specifying the password initially. I will keep the community posted on some basic performance metrics and failover tests as we perform them. Footnote - the SQL the we used to initialise the zope user and create the relstorage_util package is: --8--[cut] As SYS execute: CREATE USER zope IDENTIFIED BY DEFAULT TABLESPACE XXtablespaceXX TEMPORARY TABLESPACE sys_temp QUOTA UNLIMITED ON XXtablespaceXX QUOTA UNLIMITED ON XXindexXX GRANT CONNECT, RESOURCE, CREATE TABLE, CREATE SEQUENCE TO zope; GRANT EXECUTE ON DBMS_LOCK TO zope; As ZOPE execute: CREATE OR REPLACE PACKAGE relstorage_util AS FUNCTION request_lock(id IN NUMBER, timeout IN NUMBER) RETURN NUMBER; END relstorage_util; / CREATE OR REPLACE PACKAGE BODY relstorage_util AS FUNCTION request_lock(id IN NUMBER, timeout IN NUMBER) RETURN NUMBER IS BEGIN RETURN DBMS_LOCK.REQUEST( id = id, lockmode = DBMS_LOCK.X_MODE, timeout = timeout, release_on_commit = TRUE); END request_lock; END relstorage_util; --8--[cut] many regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
