[Zope-dev] Re: ZSQL Methods in Z Classes

2001-05-21 Thread Shai Berger


Hi Guys,

Last week, I wrote:
 
 Hi Guys,
 
 I wanted to use ZSQL Methods in Z Classes to share code between different
 parts of a large project. The different parts need to use similar, but
 separate, database namespaces, but I still wanted to use the same methods.
 
[...]
 
 I tried to bypass this limitation, by defining a forwarding database adapter.
 The idea was that this adapter's connection string would point to a resource
 (property or method) which will specify the actual connection. The idea was
 that in a ZClass, I could have this be a pythonscript which will look into
 an instance property sheet, thereby allowing different instances to use
 different connections.
 
 This seemed to work very nicely at first: After the ZClass with the forwarding
 adapter and pythonscript was installed into a folder, all its methods worked
 like magic. I was very happy.
 
 Then we restarted Zope, and ever since, it seems like the Adapter is being
 called by the ZSQL Methods without any acquisiotion context.
 
Then we shut down Zope, and started it again from the command line, and everything
seems to work OK.

Except... that still,

 ZSQL Methods in the same class as the adapter act funny -- you
 can set it as their connection, but in the source they say connection not found,
 and you can't test them in the class context (even when they don't rely on instance
 resources).
 

And besides, my excellent adapter seems to have a problem:

The idea behind it is very simple. Most of the adapter code is taken from ZGadfly,
but the database implementation -- i.e. the connection between adapter and Python
binding -- is replaced with the function,

def DB(self, connection_finder):
try: db_finder=getattr(self, connection_finder)
except AttributeError:
raise AttributeError, (
The database connection finder em%s/em cannot be found. % (
connection_finder))

# db_finder may be a string or a method which returns a string
if hasattr(db_finder,'__call__'):
db_name = db_finder()
else:
db_name = db_finder


# Now find the actual db
try: db = getattr(self, db_name)
except AttributeError:
raise AttributeError, (
The database connection em%s/em 
(indicated by em%s/em) cannot be found. % (
db_name, connection_finder))

return db()

The function DB is called whenever there is need to fetch a new database connection.
It finds the name of the connection (db_name) and calls it, which is a lot like the
way ZSQL Methods themselves find their connections.

The problem is that when I try this with the proxied connection being a Z Oracle 
Connection, ZSQL methods in the ZClass get a different connection (at the Oracle
level) from those outside it; in my case, this leads to very unpleasant database 
locks. It looks to me like some bad case of acquisition illness, but I can't quite 
get a grip on it.

About ZSQL Methods in ZClasses:

Searching through the archives, I found that Lindell Alderman tried to do, essentially,
the same thing (use same set of ZSQL Methods on different connections thru ZClasses), 
and asked about it in zope-dev, exactly a year ago; there was no answer.
(http://zope.nipltd.com/public/lists/dev-archive.nsf/0a8715d5f3c7b6a3802568c1006328f7/
098f2c0745ca099c802568e6007f2f31?OpenDocumentHighlight=2,ZClass)

Is there a non-ZClass way to do this?

Thanks again, and sorry for my impatience -- I really didn't want to share Lindell's
fate of being unanswered, so I figured at least I could reply...

Have fun,
Shai.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] ZSQL Methods in Z Classes

2001-05-16 Thread Shai Berger

Hi Guys,

I wanted to use ZSQL Methods in Z Classes to share code between different
parts of a large project. The different parts need to use similar, but
separate, database namespaces, but I still wanted to use the same methods.

Now, ZSQL Methods treat their Database connection ID, for some reason,
on a par with their source -- it is not a parameter. You are not supposed
to run one ZSQL Method on two connections. (One can try to acquire, restricting
yourself to the same ID name in different folders; I think this will not work,
and the ZSQL methods will get to their original DB connection first).

I tried to bypass this limitation, by defining a forwarding database adapter.
The idea was that this adapter's connection string would point to a resource
(property or method) which will specify the actual connection. The idea was
that in a ZClass, I could have this be a pythonscript which will look into
an instance property sheet, thereby allowing different instances to use
different connections.

This seemed to work very nicely at first: After the ZClass with the forwarding
adapter and pythonscript was installed into a folder, all its methods worked
like magic. I was very happy.

Then we restarted Zope, and ever since, it seems like the Adapter is being
called by the ZSQL Methods without any acquisiotion context.

Could you help show me what went wrong?

BTW: I intended to make the forwarding DA a product, and to ship with it 
a base ZClass for ZSQL methods-using ZClasses, which I wanted to call ZSQL Class.
But even before the current problems, I was halted by the fact that if you
define a database connection in a ZClass, ZSQL Methods in derived classes cannot
use it.

In fact, even ZSQL Methods in the same class as the adapter act funny -- you
can set it as their connection, but in the source they say connection not found,
and you can't test them in the class context (even when they don't rely on instance
resources).

Hope this is informative enough, and that others have thought of these problems,

Shai.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope] A little sqltest patch: op=ne multiple

2001-01-31 Thread Shai Berger

Hi guys,

The following trivial patch lets dtml-sqltest make sense
of multiple inequality tests, that is: Currently

dtml-sqltest name op="ne" multiple

has the funny behavior of rendering

nameval

when passed a single value, and

name in (val1,val2)

when a list is passed. With this fix, the latter case becomes

name not in (val1,val2)

(It might be considered a good idea to take care of the other
ops too, but that would be non-trivial ;-)

Have fun,
Shai.

*** sqltest.py  Wed Jan 31 19:42:15 2001
--- old_sqltest.py  Tue May 30 18:46:55 2000
***
*** 222,229 
  
  if len(vs)  1:
  vs=join(map(str,vs),', ')
- if (self.op == comparison_operators['ne']):
- return "%s not in (%s)" % (self.column,vs)
  return "%s in (%s)" % (self.column,vs)
  return "%s %s %s" % (self.column, self.op, vs[0])
  
--- 222,227 

___
Zope maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )




[Zope] Zope gets stuck

2001-01-25 Thread Shai Berger

Hi guys,

We've repeatedly encountered a weird situation where Zope gets
stuck. Then, when we kill one of the Zope processes, it is released
and answers requests.

Has anybody encountered anything like this? I know this is a little
short on information, but we are still trying to figure more ourselves.

The stats: 

Zope version: Zope 2.2.5 (binary release, python 1.5.2, linux2-x86) 
Python version: 1.5.2 (#10, Dec 6 1999, 12:16:27) [GCC 2.7.2.3] 
System Platform: linux2 

We're using Oracle and ZOracleDA too, but this time they seem not
to be the culprits...

Thanks,
Shai.

___
Zope maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )




RE: [Zope] hasRole bug or feature in 2.2.?

2001-01-16 Thread Shai Berger


Dieter writes:

   If I can't safely assume any of the above, would I be better off using a
   session product to track a user after log in so I can determine their roles
   from an unprotected document?  Any other ways?
 If the session product uses cookies, you will have a situation
 similar to cookie based authentication. Otherwise, you
 will need to shift the session id often between query string
 and hidden variable which is a bit tedious.

One relatively less tedious way to do this would be to put the session
id in the URL rather than in forms; you can do this by using an access
rule along the lines of:

dtml-let stack="REQUEST['TraversalRequestNameStack']"
 dtml-if "(stack and stack[0][:6]!='manage')"
  (assuming the next component of the path is the session id,
   remove it from the traversal path)
  dtml-call "REQUEST.set('session_id',stack.pop())"
  (somehow make sure that the session_id is valid and revive the
   session object. With SQLSession, may be done by
  dtml-call "Session(session_id)"
  (path in the next line is supposed to be replaced by something
   which, preferebly dynamically, retrieves the path traversed up
   to this point. This is needed so the session id shows up in
   URLs generated down the tree)
  dtml-call "REQUEST.setVirtualRoot(path+'/'+session_id)
 /dtml-if
/dtml-let

What this does is translate a url of the form
http://server.com/123456/real/path
to http://server.com/real/path for resource-search purposes, while
making sure that all calls to absolute_url() return urls of the form
http://server.com/123456/... . This means that as long as you rely
on absolute_url rather than relative links, you're essentially done.

This ignores session initialization first time, but I hope the
general idea of how to do this is enough. I don't have a live
example at my fingertips, but something a lot like this was done
here some time ago.

Have fun,
Shai.

___
Zope maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )




Re: [Zope] REPOST: dtml-with doesn't work

2001-01-14 Thread Shai Berger


Philip has already made it clear that this was not his problem, but
still,

Dieter Maurer wrote:

 Just a remark:
 
   SQL methods do *NOT* look at the DTML namespace *AT ALL*,
   just at REQUEST (or the expliciitly passed keyword arguments).
 
   Thus, "dtml-with", "dtml-let" and friends are all ineffective
   with respect to ZSQL methods.
 

My experience is somewhat different: ZSQL methods in general *DO*
look at the namespace just like normal DTML methods, with two 
exceptions:

1) The major one, is that Dieter's remark does describe (almost) 
correctly the behavior of dtml-sqlvar and dtml-sqltest. These two 
tags only look at a very limited local namespace, initialized with the
REQUEST and passed arguments; but even this namespace may be modified,
*INSIDE* the ZSQL method, with dtml-in and dtml-let. However, 
other tags behave quite normally -- which means that a lot of the
problems can be bypassed by thoughtful use of dtml-var, especially
using the sql_quote modifier (BTW, it is by virtue of this that you
can call other methods -- even other ZSQL methods -- from within a
ZSQL method).

2) Changing REQUEST does *not* update the local namespace, that is,
within a ZSQL method,

dtml-call "REQUEST.set('some_var', 17)"
dtml-var some_var

doesn't work; but

dtml-var "REQUEST['some_var']"

Does.

Hope this helps,

Shai.

___
Zope maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )




[Zope] DCOracle bug?

2001-01-14 Thread Shai Berger


Hi guys,

As I have written before, we are using DCOracle 1.3.2 / Oracle 8.0.5
on a Linux/i386. We need some LONG RAW columns, and we hold them
in a special table long_data, with two columns, long_id (int) and
long_data (long raw). Other tables where long data is needed hold
references to this table.

We have recently encountered a peculiar behavior: In Zope, if we issue
a query which selects many rows with one long column each (essentially,

select * from long_data

), it works; if the query selects more than one column, but only one 
row (essentially,

select a.*,b.*
from long_data a, long_data b
where a.long_id=17 and b.long_id=34

), it also works; but if we select multiple rows with multiple long 
columns (as in

select a.*,b.*
from long_data a, long_data b

) we get an Oracle error indicating a buffer is not allocated to 
receive the data.

Has anyone seen anything like this before?

Thanks,
Shai.

___
Zope maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )




[Zope] Raw post data

2001-01-14 Thread Shai Berger

Hi guys,

For several reasons, I would like to have direct access to raw posted 
data in my dtml-code (an analogue of QUERY_STRING). Can anyone please
show me the light?

Thanks,
Shai.

___
Zope maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )




[Zope] Re: Leaking Oracle connections processes

2001-01-08 Thread Shai Berger



Dieter Maurer wrote:
 
 Shai Berger writes:
   ... excess Oracle processes and zombies ...
 I have checked our installation and can confirm that
 both problems have gone.
 
 We do not yet know what caused the excess Oracle processes.
 After an increase of Zope's ZODB cache and an upgrade
 to the newest DCOracle, the problem simply went away.
 
Do you have a DCOracle version later than 1.3.2?
We use 1.3.2; we have increased the cache from 400 (the default) to
1000, to no avail. Our ZODB size, last I checked, was ~40M. I 
remember you mentioning that yours was less then 10M, I think; how
much cache do you use?

 We do know what caused our zombie processes:
 a bug in the Oracle Java runtime library:
 they had added a "NOWAIT" flag to their "wait" calls.
 This ensured that a "wait" call does not release
 the Zombie child but keeps it in its state.
 As a consequence,
 each process started from Java became a Zombie
 process.

Zope starts Oracle processes from Java??? All the zombies we see are
Zope-spawned... a Zope zombie-collection problem has been reported
independently of Oracle.

 As Oracle reported, the problem was introduced in 8.1.6
 and should be fixed in 8.1.7.
 
...which confuses me, because we use 8.0.5.

Thanks,
Shai.

___
Zope maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )




[Zope] Re: Leaking Oracle connections processes

2001-01-07 Thread Shai Berger



Shai Berger wrote:

 I have checked a little before the disaster, and it seems
 that not all those processes actually have connections; the problem
 may be unrelated to open connections, but just to processes.
 
A little progress: We just had it happen again, and checked to see how
many live processes and how many zombies were involved. pstree showed
108 Zope-spawned Oracle processes, but ps showed only 55 zombies.

I assume, therefore, that the problem is not because of zombies; there
were over 50 live processes accessing oracle.

The accumulation of zombies is a problem in itself, of course, but I
understand it is also a known problem. And from what I see, it is a
separate problem.

Our impression here -- based on not much more than gut feeling -- is
that the problem is related to abandoned transactions, where an 
exception is raised.

Thanks, Dieter, for your earlier response -- we are trying a larger
cache now, and we'll see how things go.

Have fun,
Shai.

___
Zope maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )




[Zope] Leaking Oracle connections processes

2001-01-03 Thread Shai Berger


Hi guys,

We are trying to build a site with Zope and Oracle, and we see again
and again a situation where Oracle connections and processes are
being leaked: the more we work, the more there are, until we reach
Oracle's limit for number of processes. At this point, the Connection
objects close on their own, and you can't even open SQL*Plus until
Zope is closed or restarted.

We use Oracle's default limit on the number of processes, which is 50.
We have about three users (developers), two Oracle connections in the
application (one for SqlSessions and one for other data), and although
we use some devious multithreading (some of our transactions call an
external method which essentially URLRetrieve()s a URL from our own
Zope server in a separate thread), 50 open connections should be more 
than enough. I have checked a little before the disaster, and it seems
that not all those processes actually have connections; the problem
may be unrelated to open connections, but just to processes.

We do one fishy thing that may also be related: We keep some long raw
data; we keep it all in one table, and other tables which need long
columns hold pointers (indices) into this table. To put data into this
table from Zope, we use an external method SaveLONG, which uses the 
database connection to access the DB (in order to use the 
transactionality of the whole thing). We use it in ZSQL methods as 
follows:


INSERT INTO items (
field, 
long_field_ptr
) VALUES (
dtml-sqlvar value type=string
dtml-var "SaveLONG(DB_CONNECTION, long_value)"
)


SaveLONG saves the long value in the long data table and returns
its index in the table.
The source for SaveLONG is this. It relies on a DB trigger to
populate the LONG_ID field from a sequence.


long_table_name = 'LONG_DATA'
long_column_name = 'LONG_DATA'
long_seq_name = 'SEQ_LONG_DATA'
index_column_name = 'LONG_ID'

save_command = '''
INSERT INTO %s (%s)
VALUES (:content)
'''% (long_table_name,long_column_name)

get_key_command = 'SELECT %s.currval FROM dual' % long_seq_name

#
# Put a long raw on the DB
#
def SaveLONG(self,connection, content):
connection = connection._v_database_connection.db # the real DB
connection object
connection.execute(save_command, content=dbi.dbiRaw(content))
connection.execute(get_key_command)
return connection.fetchone()[0]


We're using Zope 2.2.4 with ZOracleDA 2.1.0 and DCOracle 1.3.2
with Oracle 8.0.5 on an intel-linux (RH).

Can anyone see anything wrong with what we're doing? Is anyone else
experiencing this kind of problems?

Thanks in advance,
Shai

___
Zope maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )