Re: [Twisted-Python] How to get caller’s IP address in Perspective Broker server
Am Montag, 16. Januar 2017, 22:50:36 CET schrieb Glyph Lefkowitz: > This is untested but hopefully it will be close enough to put you on your > way. Make sense? if you plan to use the portal/realm/avatar things later on, you can use avatar.source() -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] [Twisted] #7598: Port twisted.spread.pb to Python3
Am Dienstag, 2. August 2016, 10:45:32 CEST schrieb Wolfgang Rohdewald: > Am Donnerstag, 28. Juli 2016, 11:27:15 CEST schrieb Twisted: > > #7598: Port twisted.spread.pb to Python3 > > did you test with client on Python2 and server on Python3 and vice versa? > > Not a bug report, but anyway ... This was client with Python3, > server with Python2 I have now restricted testing to using the same version of Python on both sides. That works nicely with my application. Hope this is released soon! -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] [Twisted] #7598: Port twisted.spread.pb to Python3
Am Dienstag, 2. August 2016, 03:39:58 CEST schrieb Glyph Lefkowitz:
> Probably we should up-convert from bytes automatically in the places
> where we know python will be using the values as identifiers, including
> callRemote and the keys in a __dict__.
Attached is some grep for my port showing where I had co convert.
grep -ri -e '\(network\|native\)'
> This is unfortunately very tricky to know though, if we support
> serializing both bytes and strings :-\.
Why? The wire protocol does not change. PY3 bytes are already
compatible, and PY3 strings are serialized as unicode objects.
--
Wolfgangjelly.py:from twisted.python.compat import nativeString, networkString
jelly.py:classname = nativeString(classname)
jelly.py:sxp.extend([networkString(qual(inst.__class__)),
jellier.jelly(state)])
jelly.py:networkString(qual(self.__class__)),
jelly.py:return [function_atom,
networkString(pickle.whichmodule(obj, obj.__name__))
jelly.py:return [datetime_atom, networkString('%s %s %s %s %s
%s %s' % (
jelly.py:return [time_atom, networkString('%s %s %s %s' %
(obj.hour, obj.minute,
jelly.py:return [date_atom, networkString('%s %s %s' %
(obj.year, obj.month, obj.day))]
jelly.py:return [timedelta_atom, networkString('%s %s %s' %
(obj.days, obj.seconds,
jelly.py:jelType = nativeString(obj[0])
jelly.py:moduleName = nativeString(moduleName)
jelly.py:qualName = nativeString(rest[0])
jelly.py:im_name = nativeString(rest[0])
jelly.py:typeName = nativeString(typeName)
flavors.py:from twisted.python.compat import _PY3, nativeString, networkString
flavors.py:kw = nativeStringDict(broker.unserialize(kw))
flavors.py:method = getattr(self, "remote_%s" % nativeString(message),
None)
flavors.py:raise NoSuchMethod("No such method: remote_%s" %
(nativeString(message),))
flavors.py:method = getattr(self.object, "view_%s" %
nativeString(message))
flavors.py:t = networkString(self.getTypeToCopyFor(p))
flavors.py:l.extend([networkString(type_), luid, jstate])
flavors.py:def nativeStringDict(networkDict, encoding='ascii'):
flavors.py:Convert them to the native string type.
flavors.py:@param networkDict: Any C{dict}. Keys must be C{bytes} or
C{unicode}.
flavors.py:@return: The passed dict, but all keys converted with
nativeString().
flavors.py:return dict((nativeString(key, encoding), value)
flavors.py:for key, value in networkDict.items())
flavors.py:self.__dict__ = nativeStringDict(state)
flavors.py:method = getattr(self, "observe_%s" % nativeString(message))
banana.py:from twisted.python.compat import _PY3, networkChar
banana.py:stream(networkChar(0))
banana.py:stream(networkChar(integer & 0x7f))
banana.py:LIST = networkChar(0x80)
banana.py:INT = networkChar(0x81)
banana.py:STRING = networkChar(0x82)
banana.py:NEG = networkChar(0x83)
banana.py:FLOAT= networkChar(0x84)
banana.py:LONGINT = networkChar(0x85)
banana.py:LONGNEG = networkChar(0x86)
banana.py:VOCAB= networkChar(0x87)
banana.py:typebyte = networkChar(buffer[pos])
pb.py:from twisted.python.compat import xrange, networkString, nativeString
pb.py:from twisted.python.compat import nativeString, _PY3
pb.py:from twisted.python.compat import unicode, networkChar
pb.py:This method is called when a network message is received.
pb.py:This method is called when a network message is received.
pb.py:method = getattr(self, "perspective_%s" % nativeString(message))
pb.py:state['value'] = nativeString(self.value, encoding='utf-8') #
Exception instance
pb.py:file.write(nativeString(self.traceback, encoding='utf-8'))
pb.py:nativeString(str(self.type)),
pb.py:nativeString(self.value, encoding='utf-8'))
pb.py:methodName = "proto_%s" % nativeString(command)
pb.py:objectID = networkString(objectID)
pb.py:self.sendCall(prefix+message_atom, requestID, objectID,
networkString(message), answerRequired, netArgs, netKw)
pb.py:message = nativeString(message)
pb.py:objectID = nativeString(objectID)
pb.py:#unserFail.value =
unserFail.type((nativeString(unserFail.value, encoding='utf-8')))
pb.py:crap = crap + networkChar(random.randint(65,90))
pb.py:Alternatively, any object providing or adaptable to L{IPBRoot} can be
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] [Twisted] #7598: Port twisted.spread.pb to Python3
Am Donnerstag, 28. Juli 2016, 11:27:15 CEST schrieb Twisted: > #7598: Port twisted.spread.pb to Python3 did you test with client on Python2 and server on Python3 and vice versa? Not a bug report, but anyway ... This was client with Python3, server with Python2 The same works with my old port to Python3: git clone https://github.com/wrohdewald/twisted.git git checkout spread-py3-7598 My version uses helpers in remoteMessageReceived: kw = nativeStringDict(broker.unserialize(kw)) method = getattr(self, "remote_%s" % nativeString(message), None) If you need a minimal example and an official bug report - that would take some more time. Maybe in a week or so. Peer will receive following PB traceback: Unhandled Error Traceback (most recent call last): File "/home/wr/src/kajongg/src/twisted/spread/banana.py", line 173, in gotItem self.callExpressionReceived(item) File "/home/wr/src/kajongg/src/twisted/spread/banana.py", line 136, in callExpressionReceived self.expressionReceived(obj) File "/home/wr/src/kajongg/src/twisted/spread/pb.py", line 575, in expressionReceived method(*sexp[1:]) File "/home/wr/src/kajongg/src/twisted/spread/pb.py", line 896, in proto_message self._recvMessage(self.localObjectForID, requestID, objectID, message, answerRequired, netArgs, netKw) --- --- File "/home/wr/src/kajongg/src/twisted/spread/pb.py", line 913, in _recvMessage netResult = object.remoteMessageReceived(self, message, netArgs, netKw) File "/home/wr/src/kajongg/src/twisted/spread/flavors.py", line 120, in remoteMessageReceived state = method(*args, **kw) builtins.TypeError: remote_move() keywords must be strings -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Porting PB to python3
Am Freitag, 10. Juni 2016, 12:39:20 CEST schrieb Craig Rodrigues: > I hope you haven't lost interest in this, and are still willing to push > forward. Maybe in autumn. But I rather hope somebody else would continue with this. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Need clarification on reviews for Python 3 fixes for Twisted
Am Samstag, 4. Juni 2016, 04:41:37 schrieb Craig Rodrigues: > On Fri, May 27, 2016 at 6:14 AM, Wolfgang Rohdewald < > wolfgang@rohdewald.de> wrote: > > > Am Freitag, 27. Mai 2016, 08:13:10 schrieb Itamar Turner-Trauring: > > > > This would have been helpful when I tried to port PB to python3. > > Instead, that port is now bitrotting. > > > > Can you point me to your attempt to port PB to python3? > Glyph and cyli have been merging a lot of my simple python3 fixes, > so now I am motivated to do more. :) > > -- > Craig You are welcome: https://github.com/wrohdewald/twisted/commits/spread-py3-7598/twisted -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Need clarification on reviews for Python 3 fixes for Twisted
Am Freitag, 27. Mai 2016, 08:13:10 schrieb Itamar Turner-Trauring: > On 05/27/2016 04:19 AM, Craig Rodrigues wrote: > > However, Adi has mentioned that in this document: > > http://twistedmatrix.com/trac/wiki/Plan/Python3, > > the strategy of submitting incremental Python3 fixes is not mentioned. > > Before doing any further reviews, Adi would like clarification that > > these types of reviews/patches are OK for submission and review. > > > > Are they OK? Would it be possible extend the Plan/Python3 document to > > accept incremental Python3 fixes > > as long as: > > > > * adheres to Twisted coding standards > > * works on Python 2.7 > > * passes existing tests > > * comes with new tests if functionality is changed that is not > > currently being tested > > > > My experience working with Python3 on other projects, is that incremental > > fixes is easier to review and get working, rather than an all or > > nothing approach. > > Some Python3 porting such as bytes/string/unicode or Python C API > > changes are very hard, > > while print vs. print() are very easy. Holding up the easy changes, > > until every hard change > > is also done is quite hard, and slows things down. > > I think they're fine to accept insofar as: > > 1. There is strong ongoing momentum for the port now, so these changes > makes porting module-by-module easier and won't just bitrot. > 2. They're doing one particular incompatibility at a time, rather than > "here's an assortment of random changes to a module that may or may not > port that module fully, who knows." > > I don't think they are sufficient to port a module (someone needs to > read the code and think a bit, usually), but they will make it easier to > do so, so they definitely are worth continuing. > > -Itamar This would have been helpful when I tried to port PB to python3. Instead, that port is now bitrotting. I did try hard to deliver simple changes (like print()) before tackling harder problems but not much of all that went into the source code. Interest in PB does not seem very high. Anyway there still is the public git fork (I did mention it here at that time) - if anybody would like to integrate that. Not me - for the foreseeable future. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Porting twisted.spread.banana to py3
Am Freitag, 3. Juli 2015, 23:00:45 schrieb Jørn Lomax: I have found another issue I'm wondering a little how to handle. There are places in the bana_test file that sys.maxint is used. This no longer exists in python3, so how should it be handled. Should we make an artificial maxint to use for python3. I might want to note that it is not used for comparison, but to create an integer over a given size: i5:~/tmp/twisted/twisted/test (spread-py3-7598) git show a0d35fa commit a0d35fa912b055bd65fb76b597de1d64675711d8 Author: Wolfgang Rohdewald wolfg...@rohdewald.de Date: Sun Nov 2 18:50:54 2014 +0100 twisted.test.test_banana: PY3 has no sys.maxint diff --git a/twisted/test/test_banana.py b/twisted/test/test_banana.py index e0504d0..c43bdc8 100644 --- a/twisted/test/test_banana.py +++ b/twisted/test/test_banana.py @@ -455,10 +455,11 @@ class BananaTestCase(BananaTestBase): Test feeding the data byte per byte to the receiver. Normally data is not split. +maxint = sys.maxsize if _PY3 else sys.maxint foo = [1, 2, [3, 4], [30.5, 40.2], 5, [bsix, bseven, [beight, 9]], [10], # TODO: currently the C implementation's a bit buggy... - sys.maxint * 3l, sys.maxint * 2l, sys.maxint * long(-2)] + maxint * long(3), maxint * long(2), maxint * long(-2)] self.enc.sendEncoded(foo) self.feed(self.io.getvalue()) assert self.result == foo, %s!=%s % (repr(self.result), repr(foo)) -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Porting twisted.spread.banana to py3
Am Mittwoch, 1. Juli 2015, 11:33:58 schrieb Jørn Lomax: I'm a gsoc student working on the buildbot project, porting it to py3. Part of this is looking at porting the dependencies buildbot has in twisted and try to port them. twisted.spread is the most used twisted dependency, so I have started working on this. I would much appreciate seeing spread ported. Last year I actually already did (all tests passed) but then I did not find the time to make separate tickets to be reviewed. Actually porting spread is easier than getting the source into something acceptable by the core developers. But I published what I got, see my mail on this list from Nov 3 2014: there now is https://github.com/wrohdewald/twisted.git branch spread-py3-7598 If I would continue with this now, I would probably first rebase that branch onto master (not sure if the twisted project still uses svn or if the migration to git has already been done). You could also look for tickets opened by myself (wolfgang61), most of them should be about porting spread. I started working on banana.py and I have encountered a question. In dataRecieved (banana.py:174). A chunk is added to the buffer. On python3, how should the distinction be made between string and bytes. Buffer seems to be a string, the chunk seems to be bytes, and sometimes opposite. I believe that chunk is a banana-encoded string, so there should be no need to treat it as bytes. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] t.p.filepath and Unicode
Am Sonntag, 8. März 2015, 12:30:38 schrieben Sie: Some months ago someone outlined his idea, I believe in the mailing list in a thread I participated in. should have tried to find it before posting. That was Glyph On Sep 9, 2014, at 08:01 with the subject Re: [Twisted-Python] Python3: should paths be bytes or str? -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] t.p.filepath and Unicode
Am Sonntag, 8. März 2015, 10:03:03 schrieb Adi Roiban: Hi, I am trying to help porting Twisted on Py3 and for me the fact that FilePath requires all paths to be bytes is a constant pain. To reduce the Unicode pain when working with string my principle is to do all internal operations in Unicode and then convert to bytes only at input and output. Right now I am trying to port t.p.moduled and when using the current implementation of FilePath I have to constantly encode/decode path. I would prefer not to consider the FilePath an input/output level. Does anyone find this useful and plan to use FilePath with bytes in another Python3 project? What do you say if we will also have an UnicodeFilePath class which only accept Unicode as path input and only outputs Unicode? It can start with simple path traversal/directory creation/file creation just to have t.p.moduled (and trial) ported. Thanks! Some months ago someone outlined his idea, I believe in the mailing list in a thread I participated in. I do not remember who it was. IIRC, the plan was that FilePath accepts all types of string. It keeps a 1:1 copy of that string and has attributes returning the variant that the caller needs, converting on the fly. There also is bug report 7166. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] porting twisted.spread to Python3
there now is https://github.com/wrohdewald/twisted.git branch spread-py3-7598 Not all is finished, especially utf-8 wherever Python 3 allows it, pbfailure, and handling of wired exceptions. what should I do next to start getting this into trunk? Maybe - open a ticket for porting banana - create a branch with all commits for that - do a pull request I do not want to create one big patch because I think it is helpful to have more smaller commits. But I probably should wait with all that until ticket 7664 is merged. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] spread: differences between remote_ and perspective_ needing an explanation
Hi, while trying to isolate a bug in my app, I found some strange differences between the implementations of remote_* and perspective_*. 1. in pb.Broker.serialize(), those attributes are set and restored: jellyMethod, jellyArgs, jellyKw. Within twisted Those attributes are not used anywhere. I grepped, and I ran the test suite without those. Is there some code outside of twisted using this? What for? Or is there a need for avoiding garbage collection? (which would need some explanation). My theory is that it was forgotten to remove them when doing svn commit 1448: Before, they were actually used in _NetJellier._jelly_instance, but no more after this commit. 2. Those implementations differ: remote_*: return broker.serialize(state, self.perspective) perspective_*: return broker.serialize(state, self, method, args, kw) view_*: return broker.serialize(state, self.perspective, method, args, kw) observe_*: return broker.serialize(state, None, method, args, kw) Can you please explain why remote_* is implemented differently? This goes back to before the first import to svn, so I cannot browse the history. 3. They all raise AttributeError for undefined method except remote_* which raises MethodNotFound. Both are not documented in the API. Could this be unified to MethodNotFound? My theory is that svn commit 13473 was incomplete. Commit message was: Raise NoSuchMethod instead of AttributeError when call is made to remote_ method that doesn't exist -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] spread: differences between remote_ and perspective_ needing an explanation
Am Samstag, 11. Oktober 2014, 23:30:56 schrieb Wolfgang Rohdewald: 2. Those implementations differ: remote_*: return broker.serialize(state, self.perspective) perspective_*: return broker.serialize(state, self, method, args, kw) view_*: return broker.serialize(state, self.perspective, method, args, kw) observe_*: return broker.serialize(state, None, method, args, kw) Can you please explain why remote_* is implemented differently? This goes back to before the first import to svn, so I cannot browse the history. I was unclear, what I want to know is only why args, kw are not passed for remote_*. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] my branches merged by you (itamar)
Thanks for fixing my mistakes! Based on your fixes I started a personal checklist, see below. Is that list correct so far? (it will not include things I already did right) I noticed that you also changed 'x' to b'x' in some places. I did not do that yet because it is not needed for those tickets, I wanted to keep them small and focussed. Instead I am planning tickets for just that: (after minimizing the number of places to be fixed) twisted.spread.banana and tests: literal strings become bytes where appropriate twisted.spread.jelly and tests: define *_atom consistently for all atoms, change them to bytes twisted.spread.pb and tests: literal strings become bytes where appropriate The current checklist: Line length max. 79 *.rst doc: One line per sentence, no length limit fully qualified names in doc and news, twisted.* no __ in Twisted. Everything in a test case is private anyway. if possible __execute__ (not __define__) only one assert per test document changed API like raising more or different exceptions use new print() formatting syntax Document and isolate test code using the private API of the module to be tested. test everything with PY2.6 too did twistedchecker add new warnings? news: most frequent form: Qualified now does ... news about spread belong into twisted/topfiles doc: not pb but PB Epydoc docstring ends with point break docstring lines not sooner than needed avoid the word “test” in test docstrings do not use the “I am” form in docstrings test docstring: describe what happens, including exceptions doc: avoid mentioning the private API -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] my branches merged by you (exarkun)
That should have been exarkun, not itamar. Sorry for that. Am Sonntag, 5. Oktober 2014, 11:59:40 schrieb Wolfgang Rohdewald: Thanks for fixing my mistakes! Based on your fixes I started a personal checklist, see below. Is that list correct so far? (it will not include things I already did right) I noticed that you also changed 'x' to b'x' in some places. I did not do that yet because it is not needed for those tickets, I wanted to keep them small and focussed. Instead I am planning tickets for just that: (after minimizing the number of places to be fixed) twisted.spread.banana and tests: literal strings become bytes where appropriate twisted.spread.jelly and tests: define *_atom consistently for all atoms, change them to bytes twisted.spread.pb and tests: literal strings become bytes where appropriate The current checklist: Line length max. 79 *.rst doc: One line per sentence, no length limit fully qualified names in doc and news, twisted.* no __ in Twisted. Everything in a test case is private anyway. if possible __execute__ (not __define__) only one assert per test document changed API like raising more or different exceptions use new print() formatting syntax Document and isolate test code using the private API of the module to be tested. test everything with PY2.6 too did twistedchecker add new warnings? news: most frequent form: Qualified now does ... news about spread belong into twisted/topfiles doc: not pb but PB Epydoc docstring ends with point break docstring lines not sooner than needed avoid the word “test” in test docstrings do not use the “I am” form in docstrings test docstring: describe what happens, including exceptions doc: avoid mentioning the private API -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] porting twisted.spread to Python3
Am Sonntag, 5. Oktober 2014, 13:48:12 schrieb exar...@twistedmatrix.com: On 3 Oct, 11:09 pm, wolfgang@rohdewald.de wrote: I now have a local git branch with about 70 commits, always rebased onto current trunk. It makes me sad to learn you're carrying so many patches to Twisted. It might be useful to the project if you could share why this development was easier to do outside the tree rather than contributing it to the project as you developed it. This is just my personal style of development. A bit chaotic. Sometimes trial and error. And really a lot of git rebase -i. Whenever I find some necessary or helpful change that could be done before porting, put that at the top of the patch list and readjust everything. The methodic part comes last: Look at what I changed, rethink why that is really needed, and look for similar places I overlooked. Then generate tickets for things that seem ready. BTW what about Ticket 7628, news extension port? I could soon start feeding porting tickets but if this extension is useful, I guess it should be applied first. I think that before the Twisted project wants to call a particular module ported, we want it to have test coverage that can run on Python 3. I was afraid you'd say that. https://twistedmatrix.com/trac/wiki/Plan/Python3 does not explicitly say so, maybe this should be changed. I can do that, but then I have no write permission for the wiki. I'll let other people volunteer projects they know of - but I strongly suspect there are few or no such applications. That makes me wonder if I really should have used PB at all. But now I do and have no plans to change that. Again, it seems unfortunate that you have all of this work ... somewhere. Somewhere only you (as far as I can tell) can see it. Somewhere only you can test it. Somewhere only you can work on it and get it contributed to Twisted. I believe I saw some mailing list posts where many years ago somebody said he has spread running with PY3 but as it seems nothing of that got into trunk. Be assured that I want to avoid that. What is the preferred place for Twisted public repositories? No svn please, only git. But first I want to do some more cleaning and reshuffling, I cannot really do that anymore with commits already pushed to a public repository. Maybe 2 or 3 weeks. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] my branches merged by you (exarkun)
Am Sonntag, 5. Oktober 2014, 14:20:35 schrieb exar...@twistedmatrix.com: I noticed that you also changed 'x' to b'x' in some places. I did not do that yet because it is not needed for those tickets, I wanted to keep them small and focussed. I think I only did this where new uses of 'x' was being introduced to represent a bytes object. Then I would not have mentioned this. Three such changes in svn r43220. b'remote' and two banana encoded strings. I knowingly left it at 'remote' because I wanted to change that everywhere together. My goal is for new code being added to be as correct as possible so that it doesn't increase the burden of making these fixes later. The time when the code is being first introduced is the time when it's easiest to make sure these things are correct. I fully agree. Excluding cases where you just re-use an existing string in one more place because then you have both 'remote' and b'remote' in the source until you change them all (for that you have to grep everything anyway). On the other hand, not adapting existing code to changed guidelines right when they are changed always generates a lot of such inconsistencies, so this one is really peanuts. (I know - more manpower is always needed). This is actually a small problem for me because I rather copy the style of existing code instead of reading boring guidelines. if possible __execute__ (not __define__) only one assert per test Hmm. Yes? I'm not sure. What do you have in mind here? svn r43207 Split up the test with multiple assertions into several tests There you unfolded the loop. If the real reason was the different constructions of the type names (like __builtin__.type), I would have expected a different commit text, and of course you could have left the loop, like for obj, name in ((type, '__builtin__.typ'), ...) So I assumed that you really want as few assertions executed in one test as possible. I cannot think of any other reason than those two. BTW your commit epytext fixes most but not all. There are still a few ``xxx`` left. What markup language did you have in mind there? Document and isolate test code using the private API of the module to be tested. I'm also unsure about this one. Cases like svn r43221 Hoist the use of this private API to a single location in a helper function. which explains why we need to use the private API. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] porting twisted.spread to Python3
Am Sonntag, 5. Oktober 2014, 19:58:29 schrieb exar...@twistedmatrix.com: For example, following the plan, I would have expected to see perhaps two patches to port the banana module: one to add the missing test coverage and one to make any and all changes necessary to get the test suite to pass on Python 3. Impossible. test_jelly imports pb, test_pb imports jelly. But I can of course reduce the number of tickets related to porting and put more than one patchfile into them. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] porting twisted.spread to Python3
I now have a local git branch with about 70 commits, always
rebased onto current trunk.
Those test suites pass with Python2 and Python3:
test_banana, test_jelly, test_pb, test_pbfailure, test_persisted
I also have a real app using PB (kajongg, a KDE game). Running
the server and the client with any combination of Python 2 and
Python 3 works but more testing is needed there. Especially
transferring failures with non-ascii test, so I need to inject
random errors.
spread.banana is now fully covered by tests (was at 85% before).
There is still a lot of code in twisted.spread which has no
tests yet.
I do not think I want to add missing tests for everything,
only for those regions where I do changes or where it is not
too difficult. And of course if an application triggers a bug
yet uncovered by tests.
Can you point me to other PB applications I could use for testing?
Preferrably ones that are already mostly compatible with PY3
themselves, but that is not a _must_, I would try porting them
unless it turns out to be too much work. And preferrably ones
developed by people with a non-ascii native language. I do not
yet trust Failures and logging with non-ascii.
Below is the current list of commits. Everything you see
there is still much in flux. This includes shuffling code
between commits, combining and splitting commits, changing order.
I want to open one ticket per commit. Simple things hopefully
not needing much discussion first.
Estimating one week per review/commit to trunk, this process
will take more than one year. Often I can only add a new ticket
after the previous one has been integrated because they
depend on each other.
So if it really seems to take that long, at some point I might
just decide to put what I have to github and not care anymore.
Which of course means I might have to distribute a patched
twisted version with my application. I really want to leave
Python2 behind, and only twisted is in the way. Latest versions
of KDE do not have Python2 bindings anymore (KDE Frameworks 5).
This log of course includes things being in tickets up for review.
git log --pretty='format:%s' --reverse trunk..pb3
spread.banana: selecting an unsupported dialect now raises an Exception
spread.banana now raises BananaError when receiving pb messages without pb
being the selected dialect
spread.banana: specific error message about unicode objects
spread.banana: test coverage goes up from 85% to 100%
test.test_banana: eliminate duplicate code
twisted.python.reflect.safe_str on Python 3 converts utf-8 encoded bytes to
clean str instead of b'a'
test.test_{failure,banana,persisted,reflect}: consistently name all test
methods test_*
test.test_jelly: add tests for yet untested code
spread.jelly: support the sets module only for Python 2
twisted.spread: Assume that Python always has unicode support enabled
Ticket 7653. spread.jelly: deprecate unjellying instances
spread.jelly can now transfer objects with new style classes
cred.credentials: implements-implementer
cred.checkers: implements-implementer
test.test_pb: implements-implementer
persisted.styles: add new tests
persisted.styles.unpickleMethod: remove dead code
spread.flavors: fix import paths, making them work with Python 3 too
spread: Python 3: add from __future__
do not use argument tuple unpacking anymore, not available in Python 3
spread.publish and test.test_pb: replace file by open
test.test_pb: use a constant for BRAINS!
python.compat: get_im_func, get_im_self, get_im_class and use them in spread
persisted.styles: simplify unpickleMethod
persisted.styles: Port to Python 3
python.compat: for PY3, define long=int and xrange=range
spread.banana: replace StringIO.StringIO by io.BytesIO
python.compat.networkChar: New, and use it in spread.banana
python.compat.networkString: improve Exception message
spread.banana: change literals from to b where appropriate
test.test_banana: fix a wrong assertion message
test.test_banana: use itertools.chain
test.test_banana: Python 3 does not know type long
spread: Python 3 does not have sys.maxint
spread.banana and test: Complete port to Python 3: handling of data
spread.jelly: Unjellying instance is not supported with Python 3
spread.jelly: unify usage of the types module
spread.jelly: consistently use _atom identifiers
spread.jelly: complete port to Python 3
spread.pb, test.test_pbfailure: new syntax for except
spread.pb: do not use argument tuple unpacking anymore, not available in Python
3
test.test_jelly now passes all tests with Python 3
cred.credentials.updateHash accepts unicode
add ported modules to python/dist3.py
spread.flavors: consistently use _atom identifiers
spread.flavors: Python 3: eliminate __cmp__. Only support __eq__ and __ne__
spread: Use networkString/nativeString where appropriate
spread.flavors: switch atoms from str to b
test.test_pb: improve test_concurrentLogin: use a non-ascii password
spread.pb: eliminate __cmp__, using FancyEqMixin
test.test_pb: add comment to test_tooManyRefs:
Re: [Twisted-Python] twistedchecker TypeError: modifiedBlankLines() takes exactly 6 arguments (7 given)
Am Montag, 29. September 2014, 00:07:33 schrieb Glyph: This week I will try to get https://github.com/twisted/twistedchecker/pull/71/files merged, which requires exactly pep8 1.5.1 Ubuntu 14.04 installs 1.4.6 I have the current 1.5.7 After downgrading to 1.4.6, twistedchecker works correctly. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] twistedchecker TypeError: modifiedBlankLines() takes exactly 6 arguments (7 given)
I am getting this traceback on about every source file like:
i5:~/ssdsrc/alt/Twisted/twisted/test (pb3) twistedchecker reflect_helper_ZDE.py
* Module reflect_helper_ZDE
W9001: 1,0: Missing copyright header
W9002: 1,0: Missing a reference to test module in header
C0103: 1,0: Invalid name reflect_helper_ZDE for type module (should match
(([a-z_][a-z0-9_]*))$)
W9208: 1,0: Missing docstring
Traceback (most recent call last):
File /usr/local/bin/twistedchecker, line 5, in module
pkg_resources.run_script('TwistedChecker==0.2.0', 'twistedchecker')
File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 528, in
run_script
self.require(requires)[0].run_script(script_name, ns)
File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 1394, in
run_script
execfile(script_filename, namespace, namespace)
File
/usr/lib/python2.7/dist-packages/TwistedChecker-0.2.0-py2.7.egg/EGG-INFO/scripts/twistedchecker,
line 10, in module
Runner().run(sys.argv[1:])
File
/usr/lib/python2.7/dist-packages/TwistedChecker-0.2.0-py2.7.egg/twistedchecker/core/runner.py,
line 325, in run
self.linter.check(args)
File
/usr/lib/python2.7/dist-packages/pylint-0.26.0-py2.7.egg/pylint/lint.py, line
542, in check
self.check_astng_module(astng, walker, rawcheckers)
File
/usr/lib/python2.7/dist-packages/pylint-0.26.0-py2.7.egg/pylint/lint.py, line
615, in check_astng_module
walker.walk(astng)
File
/usr/lib/python2.7/dist-packages/pylint-0.26.0-py2.7.egg/pylint/utils.py,
line 553, in walk
cb(astng)
File
/usr/lib/python2.7/dist-packages/TwistedChecker-0.2.0-py2.7.egg/twistedchecker/checkers/pep8format.py,
line 179, in visit_module
self._runPEP8Checker(node.file)
File
/usr/lib/python2.7/dist-packages/TwistedChecker-0.2.0-py2.7.egg/twistedchecker/checkers/pep8format.py,
line 188, in _runPEP8Checker
recorder = PEP8WarningRecorder(file)
File
/usr/lib/python2.7/dist-packages/TwistedChecker-0.2.0-py2.7.egg/twistedchecker/checkers/pep8format.py,
line 49, in __init__
self.run()
File
/usr/lib/python2.7/dist-packages/TwistedChecker-0.2.0-py2.7.egg/twistedchecker/checkers/pep8format.py,
line 78, in run
pep8.Checker.check_all(self)
File /usr/lib/python2.7/dist-packages/pep8-1.5.7-py2.7.egg/pep8.py, line
1445, in check_all
self.check_logical()
File /usr/lib/python2.7/dist-packages/pep8-1.5.7-py2.7.egg/pep8.py, line
1338, in check_logical
for offset, text in self.run_check(check, argument_names) or ():
File /usr/lib/python2.7/dist-packages/pep8-1.5.7-py2.7.egg/pep8.py, line
1278, in run_check
return check(*arguments)
TypeError: modifiedBlankLines() takes exactly 6 arguments (7 given)
--
Wolfgang
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Python3: should paths be bytes or str?
Am Montag, 8. September 2014, 23:01:50 schrieb Glyph: This implies a parallel structure might be possible for FilePath: if you pass its constructor bytes, you get a BytesFilePath; if you pass it text, you get a TextFilePath. You can't mix the two, and once you've chosen a path you can't choose a different one. this sounds good. After the port of pb to PY3 is done, I might have a look at it, it probably should be done before trying to port modules.py and trial to PY3. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] jelly and instances
doc says:
Instance: s is an instance of UserString.UserString, with a __dict__
{'data': 'hello'}::
[\UserString.UserString\, ['dictionary', ['data', 'hello']]]
There is no test for instances. To be precise, no test ever enters
jelly._unjelly_instance()
I also cannot find any place in jelly.py where an Instance would
be jellied. There is
instance_atom = 'instance'
but instance_atom is not used anywhere.
Is this outdated code? If not, could you please give me a code
example using this?
--
Wolfgang
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Python3: should paths be bytes or str?
Am Montag, 8. September 2014, 02:14:10 schrieb exar...@twistedmatrix.com:
On 01:26 am, wolfgang@rohdewald.de wrote:
The porting guide says
No byte paths in sys.path.
What porting guide is that?
https://twistedmatrix.com/trac/wiki/Plan/Python3
see the reviewer check list
it's also not clear it's possible for Twisted to try to make any other
choice (at some point Twisted has to interoperate with the path-related
APIs in Python itself - `sys.path`, for example).
Am Montag, 8. September 2014, 02:14:10 schrieb exar...@twistedmatrix.com:
If paths are being represented using unicode somewhere and you want to
use them with FilePath then you have to encode them (or you have to add
unicode path support to FilePath and let FilePath encode them).
I always thought module names must be ascii-only but now found PEP3131
So we should do the same for twisted.python.modules as in those other places
grepped below.
And add that assert isinstance(path, bytes) for PY3 in FilePath.
And maybe this should go into the above check list? I have no edit rights
in the Wiki.
BUT – I will stop trying to port python/modules.py, the usage of the
same strings for both module names and file paths is too much interwoven,
I do not want to touch that.
My feeling is that file names should all be unicode, converting them only
where needed. But then I am not an expert about this.
Next problem - PEP3131. See separate mail.
grep -r __file__ | grep encode
web/test/test_webclient.py:serverPEM =
FilePath(test.__file__.encode(utf-8)).sibling(b'server.pem')
test/ssl_helpers.py:certPath = nativeString(FilePath(__file__.encode(utf-8)
test/test_setup.py:if not
FilePath(twisted.__file__.encode('utf-8')).sibling(b'topfiles').child(b'setup.py').exists():
python/test/test_deprecate.py:
self.assertEqual(FilePath(module.__file__.encode(utf-8)),
internet/test/test_gireactor.py:path =
FilePath(__file__.encode(utf-8)).sibling(
--
Wolfgang
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] PEP3131: non-ascii identifiers
This does not seem to be supported by Python yet.
Should that be enabled at all?
If one process with PY3 sends such identifiers to
a separate process with PY2, that will fail. I am not
sure if that would be a problem, whoever uses this must
make sure PY3 is used everywhere.
If this should be forbidden, I will add a test to
test_pb for this. And of course somebody should document that
somewhere. There more PEP3131 is used, the more users will
fall into this trap.
If this should be enabled (which I think is not difficult,
at least for pb):
At least the patch below will be needed (only for PY3),
maybe it is already sufficient. Given that nativeString
and networkString are always used (done that for pb).
networkString may then return bytes with the high bit set
But since networkString is called in many places I want to ask and
make sure that it may really be changed this way.
https://twistedmatrix.com/documents/14.0.0/core/specifications/banana.html
does not speak against it, so I wonder why networkString has that limitation
to 7bit.
concrete banana-encoded example, from modified test_pb: (the method name is
getSimpleä)
test_pb still passes with patched nativeString/networkString (but I
only have one test for this so far, test_refcount).
b'\x07\x80\x07\x82message\x01\x81\x03\x82foo\x0b\x82getSimple\xc3\xa4\x01\x81\x01\x80\x05\x82tuple\x01\x80\n\x82dictionary'
diff --git twisted/python/compat.py twisted/python/compat.py
index 6f76c39..6919cf6 100644
--- twisted/python/compat.py
+++ twisted/python/compat.py
@@ -348,10 +348,9 @@ def nativeString(s):
raise TypeError(%r is neither bytes nor unicode % s)
if _PY3:
if isinstance(s, bytes):
-return s.decode(ascii)
+return s.decode(utf-8)
else:
-# Ensure we're limited to ASCII subset:
-s.encode(ascii)
+return s
else:
if isinstance(s, unicode):
return s.encode(ascii)
@@ -428,7 +427,7 @@ if _PY3:
def networkString(s):
if not isinstance(s, unicode):
raise TypeError(Can only convert text to bytes on Python 3, I got
%r % (s,))
-return s.encode('ascii')
+return s.encode('utf-8')
def networkChar(integer):
--
Wolfgang
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] PEP3131: non-ascii identifiers
Am Montag, 8. September 2014, 10:54:44 schrieb exar...@twistedmatrix.com: Can you point out the specific places where you think PB needs to start using UTF-8 instead of ASCII? Those are the places that need to be fixed, not the underlying porting helpers they happen to use. My question is whether PEP3131 should be supported or not. After all, it is an integral part of Python3. I have no opinion about this, I just want to point it out as a potential problem. So you say it should not be supported, and I have no problem with that. Just to be clear about it: In Python3 you can legally write def ÜnicödeMethod€Name(self): or see https://mail.python.org/pipermail/python-3000/2007-June/008172.html for a nice Russian example. but PB cannot transfer those identifiers. Which is not what the user would expect. This should be clearly communicated somewhere in the docs, IMHO. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] PEP3131: non-ascii identifiers
Am Montag, 8. September 2014, 12:04:46 schrieb exar...@twistedmatrix.com: PB supports unicode perfectly well and has for many years. This is why I asked which specific part of PB has a problem. PB transfers names of methods and classes as bytes, not as unicode. Which is logical since PY2 does not support unicode identifiers, and bytes is already a native PY2 string. Unicode is only used for content. It not yet always clear to me what is content and what is a formal string like method names or the *_atom strings which must be bytes, this needs more testing. I guess I should patch banana.py such that it dumps all it encodes or decodes into one file, so I can compare output from PY2/PY3 tests. I was assuming that suddenly transferring method names as unicode would really be a break of wire protocol stability, or do you think otherwise? If you think this is acceptable, I will check if the existing twisted code can handle getting those as unicode without source code changes. Not sure. Just tested this with Python2.6, and I am surprised that it works: getattr(A,u'x') unbound method A.x Supporting PEP3131 would only introduce a backward-incompatibility. Of course you are right that this is not part of porting. Right now I have a long list of small unsorted git commits, I will have to do a lot of reshuffling and cleaning before I will ask you how to get it into the official codebase. Not all of those commits are strictly porting, some just clean the code, making the porting commits simpler. A failing unit test or a minimal example (http://sscce.org/) would communicate this most clearly, but perhaps you can just mention a specific API and give an incomplete example of how it will fail when it runs up against the changes defined by PEP 3131. see my first mail in this thread: take test_pb.py, rename getSimple to getSimpleä, run the test. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Python3: should paths be bytes or str?
The porting guide says
No byte paths in sys.path.
I am not sure what this means, I would assume that
file paths should always be native strings. Or does it mean
that sys.path must only contain ascii bytes?
doc for FilePath says
On both Python 2 and Python 3, paths can only be bytes.
and svn commit 35410 by itamarst
changed the doc for some path functions in python/filepath.py
from str to bytes
but not all of them:
fgrep 'type path:' filepath.py
@type path: L{str}
@type path: L{str}
@type path: L{bytes}
@type path: L{bytes}
@type path: L{bytes}
I stumbled upon this while trying to find out how much work it might be
to make bin/trial run with python3
admin/run-python3-tests already passes for all twisted.spread related
tests but I still need to clean up a lot.
after adding an assert to FilePath.__init__, python3 bin/trial ... gives
File /home/wr/ssdsrc/Twisted/twisted/scripts/trial.py, line 601, in run
config.parseOptions()
File /home/wr/ssdsrc/Twisted/twisted/python/usage.py, line 277, in
parseOptions
self.postOptions()
File /home/wr/ssdsrc/Twisted/twisted/scripts/trial.py, line 472, in
postOptions
_BasicOptions.postOptions(self)
File /home/wr/ssdsrc/Twisted/twisted/scripts/trial.py, line 382, in
postOptions
self['reporter'] = self._loadReporterByName(self['reporter'])
File /home/wr/ssdsrc/Twisted/twisted/scripts/trial.py, line 369, in
_loadReporterByName
for p in plugin.getPlugins(itrial.IReporter):
File /home/wr/ssdsrc/Twisted/twisted/plugin.py, line 209, in getPlugins
allDropins = getCache(package)
File /home/wr/ssdsrc/Twisted/twisted/plugin.py, line 134, in getCache
mod = getModule(module.__name__)
File /home/wr/ssdsrc/Twisted/twisted/python/modules.py, line 781, in
getModule
return theSystemPath[moduleName]
File /home/wr/ssdsrc/Twisted/twisted/python/modules.py, line 702, in
__getitem__
self._findEntryPathString(moduleObject)),
File /home/wr/ssdsrc/Twisted/twisted/python/modules.py, line 627, in
_findEntryPathString
if _isPackagePath(FilePath(topPackageObj.__file__)):
File /home/wr/ssdsrc/Twisted/twisted/python/filepath.py, line 664, in
__init__
assert isinstance(path, bytes), 'path must be bytes: %r' % (path,)
AssertionError: path must be bytes:
'/home/wr/ssdsrc/Twisted/twisted/__init__.py'
--
Wolfgang
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] what is a non-class class?
I still do not like non-class class because this is a contradiction in itself. Anyway a non-class class is a class whose objects may not get a method attached. So far so good, but that was obvious. Am Dienstag, 2. September 2014, 16:29:41 schrieb Glyph Lefkowitz: Except I think it might be broken in the face of new-style classes; ClassType is the old-style class type, 'type' is the new one, so, that should probably be fixed That is my problem. How do I know if the object is of a non-class class? As you say, current code does not handle new style classes. I cannot check if the object is of type type because int is also of type type. An int certainly should not get a method attached. Then we could just as well remove this check. That is why I proposed to instead exclude a list of basic types int, float, list, dict, set and so on. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] what is a non-class class?
Am Mittwoch, 3. September 2014, 00:29:59 schrieb Glyph: That is my problem. How do I know if the object is of a non-class class? isinstance(something, (types.ClassType, type)). but that will make it possible to attach a method even to type int Python2: isinstance(int, type) True isinstance(object, type) True isinstance(object(), type) False Sure but the test only gets the class and it should certainly not instantiate it. Meanwhile I believe it is best to simply remove the test since python itself will reject most: a=int a.x=5 Traceback (most recent call last): File stdin, line 1, in module TypeError: can't set attributes of built-in/extension type 'int' -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] jelly with new-style classes: method references are not resolved
Hi, while trying to port jelly to Python 3 (banana tests already pass with Python 3), I found a bug in jelly. It has already been reported 3 years ago with ticket 4935. And there is a fix attached to that ticket. This fix resolves the problem for me. Why has this fix never been included? It is necessary for porting to Python 3. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] jelly with new-style classes: method references are not resolved
Am Montag, 1. September 2014, 11:57:22 schrieb exar...@twistedmatrix.com: Why has this fix never been included? It is necessary for porting to Python 3. Perhaps because the ticket was never given the review keyword - so no one ever noticed that there was a potential fix attached to it. So I should formally be able to review this since I am not the author. What I can say for certain is that the fix looks OK, and porting to Python3 depends on it. Could somebody else please review this ticket 4935? -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted throws error for more or less no reason
Am Samstag, 30. August 2014, 21:28:19 schrieb Stefan Hesse:
I hope it is ok when I post the link to stackoverflow, where I already
explained the problem.
File /usr/lib/python2.7/dist-packages/twisted/web/http.py, line 1001, in
write
value = networkString('%s' % (value,))
File /usr/lib/python2.7/dist-packages/twisted/python/compat.py, line 364,
in networkString
raise TypeError(Can only pass-through bytes on Python 2)
The source code:
if not isinstance(s, str):
raise TypeError(Can only pass-through bytes on Python 2)
# Ensure we're limited to ASCII subset:
s.decode('ascii')
You could check if what you get here is not a str but unicode. You
could change the error message to include the type and value of s
--
Wolfgang
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Maximum number of SSL connections to the server
Am Montag, 13. Januar 2014, 18:58:01 schrieb Sumanth Puram: Now it fails after 1010 successful connections which limit did you raise? Per process or systemwide? 1010 sound like you have a limit of 1024 somewhere -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] twisted/python3/ubuntu 12.10
it seems that Ubuntu 12.10 will only preinstall Python3, no Python2.7. So programs using twisted (including my own) will not come preinstalled. how is progress going with twisted and Python3? -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] twisted/python3/ubuntu 12.10
Am Donnerstag, 14. Juni 2012, 14:46:45 schrieb Vladimir Perić: As far as Ubuntu itself is concerned, I wasn't aware that Twisted is available from the default Ubuntu CD. you are right, it is not. I thought all of kdegames is preinstalled but actually only kpat is there. Sorry for the noise. -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] unhandled error in example pb_client.py
this is twisted 11.0.0 on kubuntu 11.10 if I call cd /usr/share/doc/python-twisted-words/examples/pb_client.py python pb_client.py fdfdasfdfdas.com 1000 a b c I am getting an unhandled error. Same for servers not answering or rejecting. how would I have to modify pb_client if I want to catch and handle all errors? my real-world problem is more complicated: In case of error I show a modal Qt4 message box. This gives me interactions between twisted and qt4reactor leading to infinite recursion. I can fix that for DNS lookup failure and not answering servers by temporarily disabling qt4reactor, but for rejecting servers I cannot get rid of the unhandled error. But before tackling that, I want to understand how I can really catch and handle all errors. when that works I will have one more problem: I my case I have class myClient(pb.PBClientFactory): def clientConnectionFailed(self, connector, reason): connection from client to server failed print 'getErrorMessage:', repr(reason.getErrorMessage()) which gives me DNS lookup failed: address u'fdksfsfa0s0f.com' not found: [Errno -5] Zu diesem Hostnamen geh\xc3\xb6rt keine Adresse. so it is translated to German. But pb_client.py returns english messages if I insert and use myClient. $LANG is the same for both: de_DE.UTF-8 so why does pb_client not translate the message? -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] CopyableFailure fails
Hi, with twisted 11.0.0 and python 2.7, my error fails here: File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux- x86_64.egg/twisted/spread/pb.py, line 407, in getStateToCopy state['value'] = str(self.value) # Exception instance exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\u201e' in position 10: ordinal not in range(128) this bug hides my own bug, giving me no useful backtrace. simple fix: --- x 2012-02-06 21:05:33.907228573 +0100 +++ pb.py 2012-02-06 21:05:36.923228461 +0100 @@ -404,7 +404,7 @@ if isinstance(self.value, failure.Failure): state['value'] = failure2Copyable(self.value, self.unsafeTracebacks) else: -state['value'] = str(self.value) # Exception instance +state['value'] = repr(self.value) # Exception instance if isinstance(self.type, str): state['type'] = self.type else: -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] safe_str is not safe
after patching CopyableFailure as written,
the stack trace now ends with the following lines. The correct
exception message is still not shown. It contains unicode chars
which reflect.py/safe_str cannot handle.
File /hdd/pub/src/gitgames/kajongg/src/scoringengine.py, line 170, in
initRuleset
raise Exception(m18n('ruleset %1 not found', self.name))
exceptions.Exception: Exception instance at 0x30e8b48 with str error:
'Traceback (most recent call last):\n File /usr/local/lib/python2.7/dist-
packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/python/reflect.py, line
546, in _safeFormat\nreturn formatter(o)\nUnicodeEncodeError: \'ascii\'
codec can\'t encode character u\'\\u201e\' in position 10: ordinal not in
range(128)\n'
Unhandled error in Deferred:
after replacing safe_str by safe_repr in twisted/python/failure.py,
I finally get a meaningful printout:
File /hdd/pub/src/gitgames/kajongg/src/scoringengine.py, line 170, in
initRuleset
raise Exception(m18n('ruleset %1 not found', self.name))
exceptions.Exception: Exception(u'Regelsatz
\u201e55a8a17e29a1a087cc8df9cedb5590cd\u201c nicht gefunden',)
Unhandled error in Deferred:
of course a correct encoding would be even better...
my patch:
--- failure.org 2012-02-06 22:22:46.395065076 +0100
+++ failure.py 2012-02-06 22:22:51.259064816 +0100
@@ -472,7 +472,7 @@
Get a string of the exception which caused this Failure.
if isinstance(self.value, Failure):
return self.value.getErrorMessage()
-return reflect.safe_str(self.value)
+return reflect.safe_repr(self.value)
def getBriefTraceback(self):
io = StringIO()
@@ -517,7 +517,7 @@
w(%s: %s: %s\n % (
hasFrames,
reflect.safe_str(self.type),
-reflect.safe_str(self.value)))
+reflect.safe_repr(self.value)))
else:
w( 'Traceback (most recent call last):\n')
@@ -542,7 +542,7 @@
w(self.type + \n)
else:
w(%s: %s\n % (reflect.qual(self.type),
-reflect.safe_str(self.value)))
+reflect.safe_repr(self.value)))
# chaining
if isinstance(self.value, Failure):
# TODO: indentation for chained failures?
--
Wolfgang
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] CopyableFailure fails
Am Montag, 6. Februar 2012, 23:12:29 schrieb exar...@twistedmatrix.com: On 08:07 pm, wolfg...@rohdewald.de wrote: Hi, with twisted 11.0.0 and python 2.7, my error fails here: Hello, Please provide a minimal example reproducing the problem so we can discuss it. See http://sscce.org/ for more details. I was hoping it would be self-evident. Trimming this to a short example could be rather much work. I hope the full backtrace will help. anyway I overlooked the fact that two backtraces are printed where the first on gives me my own calling stack. But the second one (UnicodeEncodeError) should not happen IMHO, and it does not with my patch. Unhandled Error Traceback (most recent call last): File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/spread/pb.py, line 1348, in remote_respond d = self.portal.login(self, mind, IPerspective) File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/cred/portal.py, line 116, in login ).addCallback(self.realm.requestAvatar, mind, *interfaces File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py, line 297, in addCallback callbackKeywords=kw) File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py, line 286, in addCallbacks self._runCallbacks() --- exception caught here --- File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py, line 542, in _runCallbacks current.result = callback(current.result, *args, **kw) File /hdd/pub/src/gitgames/kajongg/src/server.py, line 956, in requestAvatar avatar.attached(mind) File /hdd/pub/src/gitgames/kajongg/src/server.py, line 908, in attached self.server.login(self) File /hdd/pub/src/gitgames/kajongg/src/server.py, line 717, in login self.loadSuspendedTables(user) File /hdd/pub/src/gitgames/kajongg/src/server.py, line 889, in loadSuspendedTables table = Table(self, None, Ruleset.cached(ruleset, used=True), playOpen, autoPlay=False, seed=seed) File /hdd/pub/src/gitgames/kajongg/src/scoringengine.py, line 77, in cached result = Ruleset(name, used) File /hdd/pub/src/gitgames/kajongg/src/scoringengine.py, line 110, in __init__ self.initRuleset() File /hdd/pub/src/gitgames/kajongg/src/scoringengine.py, line 167, in initRuleset raise Exception(u'just one unicode char:ä') exceptions.Exception: Exception(u'just one unicode char:\xe4',) fail: twisted.spread.pb.CopyableFailure type 'exceptions.Exception' Unhandled error in Deferred: Unhandled Error Traceback (most recent call last): File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/spread/pb.py, line 515, in expressionReceived method(*sexp[1:]) File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/spread/pb.py, line 827, in proto_message self._recvMessage(self.localObjectForID, requestID, objectID, message, answerRequired, netArgs, netKw) File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/spread/pb.py, line 865, in _recvMessage callbackArgs=args, errbackArgs=args) File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py, line 286, in addCallbacks self._runCallbacks() --- exception caught here --- File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py, line 542, in _runCallbacks current.result = callback(current.result, *args, **kw) File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/spread/pb.py, line 896, in _sendFailureOrError self._sendFailure(fail, requestID) File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/spread/pb.py, line 905, in _sendFailure self._sendError(fail, requestID) File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/spread/pb.py, line 921, in _sendError self.sendCall(error, requestID, self.serialize(fail)) File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/spread/pb.py, line 764, in serialize return jelly(object, self.security, None, self) File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/spread/jelly.py, line 1122, in jelly return _Jellier(taster, persistentStore, invoker).jelly(object) File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/spread/jelly.py, line 475, in jelly return obj.jellyFor(self) File /usr/local/lib/python2.7/dist-packages/Twisted-11.0.0-py2.7-linux-x86_64.egg/twisted/spread/flavors.py, line 299, in jellyFor state = self.getStateToCopyFor(p) File
[Twisted-Python] Kajongg - a new game base on PyQt and twisted
I would like to announce Kajongg - it is a Mah Jongg game (not yet another solitaire but the real game for four players). It is written 100% in python and regular expressions. The GUI is done with PyQt4, and the network part is written using twisted. Dear twisted people - you have a list of projects using twisted - if you want to add Kajongg, please go ahead! http://www.kde-apps.org/content/show.php/kajongg?content=103206 source code: svn co svn://anonsvn.kde.org/home/kde/trunk/playground/games/kajongg -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] order of remote requests
On Tuesday 29 December 2009, Andrew Bennetts wrote: Wolfgang Rohdewald wrote: when I send two remote calls A and B over the network, in that order, using the twisted perspective broker callRemote method, is it granted that A is processed first on the other side even if B arrives first? Or should I only send B after I got an answer to A? It depends on the server, and what exactly you want to happen. Perspective broker can have many concurrent remote calls on the same connection, and depending on how the server is implemented it might process those sequentially in the order they arrive, or concurrently, or even in an arbitrary order. So if call A really must be completed before starting B, then the client must not make call B until it knows A has finished. I should have given more details, sorry. I am writing both the server and the client with twisted pb. The server (a game server) sends messages to four player clients. Say the game server sends messages A and B to a player client. Is it granted that the player client always processes A first even if the internet routes the messages differently and B arrives first? -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] order of remote requests
On Tuesday 29 December 2009, David Ripton wrote: TCP guarantees in-order message delivery. If you send A before B over the same TCP connection, and A and B reach the client, then the client will receive A before B. Of course - now I wonder what made me think otherwise. Thank you for your explanation! -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] pb, viewable, avatars: The example chatserver is buggy
I tried this example: http://twistedmatrix.com/projects/core/documentation/howto/pb-cred.html#auto18 (scripts chatserver.py and chatclient.py) but they do not work with python 2.6.4 (twisted 8.2.0). Before the client can do much, the server function User.detached is called resulting in an exception later when User.send is called but self.remote is None. I can fix this by changing chatclient.py: def connected(self, perspective): self.perspective = perspective ... and continue as in the original script it seems that otherwise the perspective is destroyed too early. also the documentation should perhaps mention that the client can only connect to the server once because the server code does not remove a user from her group when she detaches. Is there any more complete example anywhere with better error handling? -- Wolfgang ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
