Re: [viff-devel] Confusing behaviour?
Thanks for the clarification. I erroneously assumed that blocking stuff was ok in Twisted as long as it happens before any communication. But it makes good sense that one should avoid this anywhere in a Twisted program. Regards, Thomas 2009/3/26 Martin Geisler : > Thomas P Jakobsen writes: > >> By the way, the same thing seem sto happen if the protocol, insted of >> asking the user to press enter, does some local computations like >> quering a database, sleeping, computing primes or the like. [...] > > Right -- callback functions *cannot block*! They must always returns > very quickly and use some asynchronous mechanism to signal when a > long-running operations is finished. See this FAQ: > > http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#HowdoIuseDeferredstomakemyblockingcodenon-blocking > > Twisted has a database abstraction layer that does this for databases: > > http://twistedmatrix.com/projects/core/documentation/howto/rdbms.html > > And a threadpool for more general tasks: > > http://twistedmatrix.com/projects/core/documentation/howto/threading.html > > For non-blocking access to stdin and stdout there is the > twisted.internet.stdio module, see: > > http://twistedmatrix.com/trac/browser/trunk/doc/core/examples/stdiodemo.py > > This means that your example should look similar to this: > > > > -- > Martin Geisler > > VIFF (Virtual Ideal Functionality Framework) brings easy and efficient > SMPC (Secure Multiparty Computation) to Python. See: http://viff.dk/. > > ___ > viff-devel mailing list (http://viff.dk/) > viff-devel@viff.dk > http://lists.viff.dk/listinfo.cgi/viff-devel-viff.dk > > ___ viff-devel mailing list (http://viff.dk/) viff-devel@viff.dk http://lists.viff.dk/listinfo.cgi/viff-devel-viff.dk
Re: [viff-devel] Confusing behaviour?
Thomas P Jakobsen writes:
> By the way, the same thing seem sto happen if the protocol, insted of
> asking the user to press enter, does some local computations like
> quering a database, sleeping, computing primes or the like. [...]
Right -- callback functions *cannot block*! They must always returns
very quickly and use some asynchronous mechanism to signal when a
long-running operations is finished. See this FAQ:
http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#HowdoIuseDeferredstomakemyblockingcodenon-blocking
Twisted has a database abstraction layer that does this for databases:
http://twistedmatrix.com/projects/core/documentation/howto/rdbms.html
And a threadpool for more general tasks:
http://twistedmatrix.com/projects/core/documentation/howto/threading.html
For non-blocking access to stdin and stdout there is the
twisted.internet.stdio module, see:
http://twistedmatrix.com/trac/browser/trunk/doc/core/examples/stdiodemo.py
This means that your example should look similar to this:
#!/usr/bin/python
import sys
From optparse import OptionParser
From twisted.internet import reactor, stdio, defer
From twisted.protocols import basic
From viff.field import GF
From viff.runtime import create_runtime, gather_shares, Share
From viff.comparison import Toft05Runtime
From viff.config import load_config
class SomeProtocol(object):
def __init__(self, runtime, io):
self.runtime = runtime
# First do some local stuff.
io.sendLine("Press Enter!")
line = io.readLine()
line.addCallback(self.go)
def go(self, line):
# Now use VIFF.
Zp = GF(30916444023318367583)
a = Share(self.runtime, Zp, value=Zp(0))
b = Share(self.runtime, Zp, value=Zp(1))
c = a > b
c.addCallback(lambda _: self.runtime.shutdown())
class WaitForLine(basic.LineReceiver):
from os import linesep as delimiter
deferred = None
def lineReceived(self, line):
self.sendLine("* received: %s" % line)
if self.deferred:
d, self.deferred = self.deferred, None
d.callback(line)
else:
self.sendLine("** no deferred available!")
def readLine(self):
self.deferred = defer.Deferred()
return self.deferred
if __name__ == "__main__":
io = WaitForLine()
stdio.StandardIO(io)
# Set up VIFF.
parser = OptionParser()
Toft05Runtime.add_options(parser)
options, args = parser.parse_args()
id, players = load_config(args[0])
pre_runtime = create_runtime(id, players, 1, options, Toft05Runtime)
pre_runtime.addCallback(SomeProtocol, io)
reactor.run()
--
Martin Geisler
VIFF (Virtual Ideal Functionality Framework) brings easy and efficient
SMPC (Secure Multiparty Computation) to Python. See: http://viff.dk/.
pgpcdjr8YHEdg.pgp
Description: PGP signature
___
viff-devel mailing list (http://viff.dk/)
viff-devel@viff.dk
http://lists.viff.dk/listinfo.cgi/viff-devel-viff.dk
Re: [viff-devel] Confusing behaviour?
Thanks for the reply. > However, your problem probably could be fixed by synchronizing before > blocking: > > sync = runtime.synchronize() > sync.schedule_callback(blocking_code()) Yes, I was thinking the same thing. Another way of avoiding the problem is to place initial local computations outside the VIFF protocol. But I don't think its intuitive and it will probably confuse future VIFF users too if it's not fixed or at least documented somewhere. Regards, Thomas ___ viff-devel mailing list (http://viff.dk/) viff-devel@viff.dk http://lists.viff.dk/listinfo.cgi/viff-devel-viff.dk
Re: [viff-devel] Confusing behaviour?
I think the problem is the following: Every players sends its id on a new connection, and the connection is considered to be set up when this id arrives. So it may occur that a player has received the ids from all other players and wants to send its id to other player(s). But nothing is sent before all callbacks are executed. Since the player considers all connections to made, also the main callback, which blocks, is executed. Therefore, the problem is somehow related to what I'm working on because in a two-threaded solution the Twisted reactor is (nearly) never blocked. However, your problem probably could be fixed by synchronizing before blocking: sync = runtime.synchronize() sync.schedule_callback(blocking_code()) Thomas P Jakobsen wrote: Hi all, When I execute the attached VIFF protocol on three servers I would expect all three to ask me to press enter. When all three servers have done that, I would expect the computation of c to start and that the servers will eventually finish. I've run the protocol several times on Linux and Windows. What happens is this: Sometimes it works out as expected, but at other times only two of the servers will ask the user to press enter. In some cases, the third server will ask its user to press enter as soon as one of the two other servers presses enter. At other times, the third server will first wake up and ask its user to press enter when both of the two first servers have pressed enter. I find this behaviour confusing (..twisted?) and wonder whether it is a bug or a feature? If it's a bug, could it be related to the issue discussed in the thread "Mystery of the quadratic running time solved?". I must admit that I haven't followed that thread in detail... By the way, the same thing seem sto happen if the protocol, insted of asking the user to press enter, does some local computations like quering a database, sleeping, computing primes or the like. Sometimes, one of the servers will sit still until one or both of the other servers have finished their local jobs. This was how I initially stumbled across the problem. Best regards, Thomas ___ viff-devel mailing list (http://viff.dk/) viff-devel@viff.dk http://lists.viff.dk/listinfo.cgi/viff-devel-viff.dk
Re: [viff-devel] Confusing behaviour?
I have experienced the same problem/issue. -- Janus Dam Nielsen R&D Scientist Alexandra Instituttet janus.niel...@alexandra.dk On 23/03/2009, at 15.42, Thomas P Jakobsen wrote: Hi all, When I execute the attached VIFF protocol on three servers I would expect all three to ask me to press enter. When all three servers have done that, I would expect the computation of c to start and that the servers will eventually finish. I've run the protocol several times on Linux and Windows. What happens is this: Sometimes it works out as expected, but at other times only two of the servers will ask the user to press enter. In some cases, the third server will ask its user to press enter as soon as one of the two other servers presses enter. At other times, the third server will first wake up and ask its user to press enter when both of the two first servers have pressed enter. I find this behaviour confusing (..twisted?) and wonder whether it is a bug or a feature? If it's a bug, could it be related to the issue discussed in the thread "Mystery of the quadratic running time solved?". I must admit that I haven't followed that thread in detail... By the way, the same thing seem sto happen if the protocol, insted of asking the user to press enter, does some local computations like quering a database, sleeping, computing primes or the like. Sometimes, one of the servers will sit still until one or both of the other servers have finished their local jobs. This was how I initially stumbled across the problem. Best regards, Thomas ___ viff-devel mailing list (http://viff.dk/) viff-devel@viff.dk http://lists.viff.dk/listinfo.cgi/viff-devel-viff.dk ___ viff-devel mailing list (http://viff.dk/) viff-devel@viff.dk http://lists.viff.dk/listinfo.cgi/viff-devel-viff.dk
[viff-devel] Confusing behaviour?
Hi all, When I execute the attached VIFF protocol on three servers I would expect all three to ask me to press enter. When all three servers have done that, I would expect the computation of c to start and that the servers will eventually finish. I've run the protocol several times on Linux and Windows. What happens is this: Sometimes it works out as expected, but at other times only two of the servers will ask the user to press enter. In some cases, the third server will ask its user to press enter as soon as one of the two other servers presses enter. At other times, the third server will first wake up and ask its user to press enter when both of the two first servers have pressed enter. I find this behaviour confusing (..twisted?) and wonder whether it is a bug or a feature? If it's a bug, could it be related to the issue discussed in the thread "Mystery of the quadratic running time solved?". I must admit that I haven't followed that thread in detail... By the way, the same thing seem sto happen if the protocol, insted of asking the user to press enter, does some local computations like quering a database, sleeping, computing primes or the like. Sometimes, one of the servers will sit still until one or both of the other servers have finished their local jobs. This was how I initially stumbled across the problem. Best regards, Thomas example.py Description: Binary data ___ viff-devel mailing list (http://viff.dk/) viff-devel@viff.dk http://lists.viff.dk/listinfo.cgi/viff-devel-viff.dk
