Re: [Zope] XML-RPC within ZOPE
Jan-Ole Esleben wrote at 2005-12-11 19:10 +0100: Is it at all impossible to use XML-RPC _within_ a ZOPE architecture? In principle yes. Be careful however: it is easy to introduce deadlocks! When during request processing you call back into the same Zope via XML-RPC, the calling out request will not complete until the XML-RPC returns a result (this always holds for calls, XML-RPC or other, to an external or internal server). Zope has a limited amount of workers (the default is 4) able to execute requests. If there are no free workers, requests have to wait for one. Now imagine that as many requests arrive as there are workers and each of them wants to perform an XML-RPC against the same Zope. Then you have a deadlock: none of the XML-RPC requests will finish, because there are no free workers. An no worker will ever become free again, because each of them waits for its XML-RPC to finish. Therefore, you should directly call internal resources (rather than use XML-RPC). -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] XML-RPC within ZOPE
The problem is that it isn't my design, and the design is part of a contract. I can't really do anything about that (the major problem is that there will be SOAP components that might even need to be transparently integrated - which completely excludes ZEO). I tried doing my own commits, but it doesn't work, it just bombs earlier (no matter where I do them). I really don't know why, my only guess is that something about the transaction management breaks terribly when you go back and forth between objects without them being able to track it. Only I cannot figure out what. It is not possible to do _really_ explicit transaction management with ZODB, is it? Ole 2005/12/12, Alan Milligan [EMAIL PROTECTED]: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jan-Ole Esleben wrote: OK, the following is the case: The setup is for functions dispatching either locally or via XML-RPC. At the moment, the mechanism I use is to detect locality is flawed and thus doesn't work well. So I have local XML-RPC calls where I could have global ones. But. 1. I need that possibility for testing the XML-RPC setup. I cannot set up multiple instances for every cheap test. At least I shouldn't have to. The very important reason, however, is: 2. This is a generic call dispatcher. It is used, for example, for object name resolution in a widely deployed system. So, what could happen and couldn't happen locally is this: Zope isn't CORBA. ZEO is the correct way to scale/cluster Zope's. 1. Server1 dispatcher calls, via XML-RPC, Server2. 2. Server2 dispatcher calls Server1, which does something locally (same communication management object as the dispatcher, maybe another local call) and then returns This is the same problem spread out over two servers, where locality isn't possible anymore. The call from Server1 to Server2 must be made via XML-RPC and I cannot exclude Server2 calling Server1 back. If you really must architect your solution this way, then you are going to have to do your own transaction management. Your 'local' calls need to do explicit commit() calls and you're going to have to reread 'tainted' objects. Again, it will take you half an hour to modify your design instead of half a year to get all the bugs out of your implementation. Alan -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.6 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFDnN2ACfroLk4EZpkRAkQ7AJ4rTWpmldRq1GoPGIV8r0CT7+VPJwCZAbKB 5PBuLd5sJgh7QZe1ey8lZ9g= =3TrG -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] XML-RPC within ZOPE
Jan-Ole Esleben wrote: The problem is that it isn't my design, and the design is part of a contract. I can't really do anything about that (the major problem is that there will be SOAP components that might even need to be transparently integrated - which completely excludes ZEO). That depends on your thinking... I tried doing my own commits, but it doesn't work, it just bombs earlier (no matter where I do them). You;re not giving us a lot of info here, but I think you need to read the code in lib/python/ZPublisher in depth if you really need to play at this level... I really don't know why, my only guess is that something about the transaction management breaks terribly when you go back and forth between objects without them being able to track it. Only I cannot figure out what. Well, you could start by showing us the errors you're getting... It is not possible to do _really_ explicit transaction management with ZODB, is it? Of course it is. But you're talking about Zope here, not ZODB. Zope manages transactions so users don't often have to worry about them. If you want to manage transactions within Zope, the nyou have to understadn and co-operate with the transaction management that Zope does... Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [ZOPE] XML-RPC within ZOPE
Chris: the errors I get are always ConflictErrors without any usable tracebacks. I would like to give more information, but it really is very complex when you go a little deeper and would probably not be useful. I'll try to describe it a little better anyway: I have an Object that offers an interface for dispatching methods. It does so by deciding wether the method's object is on the local server or not, and depending on that, calls it via XML-RPC or via the ZOPE context (self). A remote ZOPE server cannot be part of a ZEO backend because there need to be several (hundred) self sustained servers that can act on their own when the connection to the others fails (which is likely rather than possible). But, as I just realised - in an internet context, isn't it possible that something like this could happen without even being part of a design? One ZOPE server's object, say, LibraryManager, calls an object in another ZOPE server, say UniversityLibrary, via XML-RPC (because it doesn't even know it's ZOPE). It asks it for a book. The University Library doesn't have it, but it has a service that looks in all the libraries it knows and returns any found occurences. Thus, at some point during that call, it asks the first server's LibraryManager about the book - again, not even knowing that it is a ZOPE server, via XML-RPC. Should this work, and where does it differ from what I'm doing? The problem is that it isn't my design, and the design is part of a contract. I can't really do anything about that (the major problem is that there will be SOAP components that might even need to be transparently integrated - which completely excludes ZEO). Why do you think that? ZEO is simply a single point of persistence for a group of Zope Application Servers. But what if I don't know which components are ZOPE, and components can be transparently exchanged? Ole ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] XML-RPC within ZOPE
Hi! Is it at all impossible to use XML-RPC _within_ a ZOPE architecture? What I mean is: I have a nested call structure: 1. Python script calls a method from a ZODB object 2. That method calls a Python module function 3. That module function dispatches an XML-RPC call to the same ZODB object called in step 1 4. This last call, which is rather complex, returns, and all the calls leading up to it return, sometimes doing some minor work When I use the same code _without_ any XML-RPC (the Zope server's self instead of the XML-RPC server object for that server), it works just fine. When I use XML-RPC, I get a ConflictError during the final return (from step 1), and I get it if I commit manually beforehand, so it must be the commit triggering it. The tricky thing is: nothing in my ZODB changes. I have several mutable _arguments_ that are passed on, but they are not default arguments and they don't get written to the ZODB. Is what I'm trying to do impossible for some reason? Or can I just make the whole call take place outside any transactions? Thanks in advance for any pointers, I'm really a little desparate at this point. Ole ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] XML-RPC within ZOPE
It really *is* in your interests to ensure a single request completes all your functional requirements ;) That's not possible unfortunately. The design of the system is rather complex, and unalterable in this respect. However, it is not true that it is usually possible to fulfil this requirement. What of programs running on an operating system that call other programs via that operating system? Given that that is a requirement that needs to be met, do I have any options? Ole ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] XML-RPC within ZOPE
OK, the following is the case: The setup is for functions dispatching either locally or via XML-RPC. At the moment, the mechanism I use is to detect locality is flawed and thus doesn't work well. So I have local XML-RPC calls where I could have global ones. But. 1. I need that possibility for testing the XML-RPC setup. I cannot set up multiple instances for every cheap test. At least I shouldn't have to. The very important reason, however, is: 2. This is a generic call dispatcher. It is used, for example, for object name resolution in a widely deployed system. So, what could happen and couldn't happen locally is this: 1. Server1 dispatcher calls, via XML-RPC, Server2. 2. Server2 dispatcher calls Server1, which does something locally (same communication management object as the dispatcher, maybe another local call) and then returns This is the same problem spread out over two servers, where locality isn't possible anymore. The call from Server1 to Server2 must be made via XML-RPC and I cannot exclude Server2 calling Server1 back. Does this make it a little more of a real problem? Ole 2005/12/12, Alan Milligan [EMAIL PROTECTED]: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jan-Ole Esleben wrote: It really *is* in your interests to ensure a single request completes all your functional requirements ;) That's not possible unfortunately. The design of the system is rather complex, and unalterable in this respect. However, it is not true that it is usually possible to fulfil this requirement. What of programs running on an operating system that call other programs via that operating system? I don't believe you. You have a client application calling your Zope server's bla() function. It can do this either by HTTP or XML-RPC by choosing the appropriate Content-Type. You've chosen to implement this interface by making other XML-RPC calls to the same server to complete the task. This is both unnecessary and bad. Unnecessary because you're doing an RPC when you've got all you need in local memory. This is bad because you're introducing the additional overhead of setting up said RPC. Worse, you're now responsible for your own transaction management because you've split the task across multiple ZODB connections. You need to think about the *correct* way to write your bla() function. In doing this, you want to be able to take advantage of the automatic rollback facilities of Zope by keeping all of this in one transaction, which will also avoid your initial problem of conflict errors. Alan -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.6 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFDnNZcCfroLk4EZpkRArTDAJ96TGwDy047D09jkmRNbGyD7e+3QgCg0aNZ lLo0znXi4qvXr5j4SseH2k8= =hu4V -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
