Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys

[M.-A. Lemburg]

Delaney, Timothy (Tim) wrote:
> M.-A. Lemburg wrote:
> 
>> Perhaps we ought to add an exception to the dict lookup mechanism
>> and continue to silence UnicodeErrors ?!
> 
> I'd definitely consider a UnicodeError to be an indication that two
> objects are not equal. 

Not really: Python expects all strings to be ASCII whenever they
meet Unicode strings and have to be converted to Unicode. If a string
is not ASCII and thus causes the exception, there's not a lot you
can say, since you don't know the encoding of the string. All you
know is that it's not ASCII. Instead of guessing, Python then raises
an exception to let the programmer decide.

> At the very least, in the context of a dictionary
> lookup.

The point here is that a typical user won't expect any comparisons
to be made when dealing with dictionaries, simply because the fact
that you do need to make comparisons is an implementation detail.

So in this particular case silencing the exception might be the
more user friendly way of dealing with the problem.

That said, the problem still lingers in that dictionary, so it may
bite you in some other context, e.g. when iterating over the list
of keys.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 04 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys

[Giovanni Bajo]

M.-A. Lemburg <[EMAIL PROTECTED]> wrote:

>> At the very least, in the context of a dictionary
>> lookup.
>
> The point here is that a typical user won't expect any comparisons
> to be made when dealing with dictionaries, simply because the fact
> that you do need to make comparisons is an implementation detail.


I'd also weight in the fact that this is a change of behaviour since the
introduction of unicode in 2.0, and it's break real-world applications (as
demonstrated by this thread). I think Python ought to revert to the previous
behaviour and, in case there's interest, the discussion moved to the py3k list.

Giovanni Bajo

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers)

[Greg Ewing]

James Y Knight wrote:

> And it does in C because C doesn't have arbitrary size integers, so  if 
> round returned integers, round(1e+308) couldn't work.

Also, in C you can use the result directly in an int
context without complaint. In Python these days, that
is usually disallowed.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys

[Ralf Schmitt]

Jean-Paul Calderone wrote:
> 
> I like the exception that 2.5 raises.  I only wish it raised by default
> when using 'ascii' and u'ascii' as keys in the same dictionary. ;)  Oh,
> and that str and unicode did not hash like they do.  ;)

No problem:

 >>> import sys
 >>> reload(sys)

 >>> sys.setdefaultencoding("base64")
 >>> "a"==u"a"
Traceback (most recent call last):
   File "", line 1, in 
   File "/exp/lib/python2.5/encodings/base64_codec.py", line 42, in 
base64_decode
 output = base64.decodestring(input)
   File "/exp/lib/python2.5/base64.py", line 321, in decodestring
 return binascii.a2b_base64(s)
binascii.Error: Incorrect padding
 >>> "a"=="a"
True
 >>> d={u"a":1, "a":1}
Traceback (most recent call last):
   File "", line 1, in 
   File "/exp/lib/python2.5/encodings/base64_codec.py", line 42, in 
base64_decode
 output = base64.decodestring(input)
   File "/exp/lib/python2.5/base64.py", line 321, in decodestring
 return binascii.a2b_base64(s)
binascii.Error: Incorrect padding

Maybe this is all just a matter of choosing the right defaultencoding ? :)


BTW, python 2.4 also suppresses this exception (when instantiating the 
dictionary)

Does python 2.4 catch any exception when comparing keys (which are not 
basestrings) in dictionaries?


- Ralf

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.5 status

[M.-A. Lemburg]

Neal Norwitz wrote:
> I believe all 3 outstanding issues (and solutions!) could use some
> more discussion.  All bugs/patches blocking release are set to
> priority 9.
> 
> http://python.org/sf/1530559 - struct rejecting floats (patch pending)
> 
> http://mail.python.org/pipermail/python-dev/2006-July/067774.html
> Problem with __index__ (patch pending)
> 
> http://mail.python.org/pipermail/python-dev/2006-August/067926.html
> str/unicode dict keys can raise an exception

I'd like to add this issue:

http://mail.python.org/pipermail/python-dev/2006-August/067869.html
distutils version string should become static again

This is part of a longer discussion we can have after 2.6
final is out, but for now, I would like to request that the
change

http://mail.python.org/pipermail/python-checkins/2006-July/055139.html

be backed out again before c1 gets released.

In the above patch, the distutils version was made to dynamically
depend on the Python version running the code, effectively
making it impossible to query the true distutils version that
is being used, bypassing the whole purpose of having static
version in the code.

I don't really see manually bumping the version number every
now and then to be such a great issue and have also volunteered
to take care of that in the future.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 04 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys

[M.-A. Lemburg]

Ralf Schmitt wrote:
> Does python 2.4 catch any exception when comparing keys (which are not 
> basestrings) in dictionaries?

Yes. It does so for all equality compares that need to be done
as part of the hash collision algorithm (not only w/r to strings
and Unicode, but in general).

This was changed in 2.5, which now reports the exception.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 04 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys

[Greg Ewing]

M.-A. Lemburg wrote:

> If a string
> is not ASCII and thus causes the exception, there's not a lot you
> can say, since you don't know the encoding of the string.

That's one way of looking at it.

Another is that any string containing chars > 127 is not
text at all, but binary data, in which case it's not equal
to *any* unicode string -- just like bytes objects will
not be equal to strings in Py3k.

> All you
> know is that it's not ASCII. Instead of guessing, Python then raises
> an exception to let the programmer decide.

There's no disputing that an exception should be raised
if the string *must* be interpretable as characters in
order to continue. But that's not true here if you allow
for the interpretation that they're simply objects of
different (duck) type and therefore unequal.

--
Greg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys

[Ralf Schmitt]

M.-A. Lemburg wrote:
> Ralf Schmitt wrote:
>> Does python 2.4 catch any exception when comparing keys (which are not 
>> basestrings) in dictionaries?
> 
> Yes. It does so for all equality compares that need to be done
> as part of the hash collision algorithm (not only w/r to strings
> and Unicode, but in general).
> 
> This was changed in 2.5, which now reports the exception.
> 

So, this thread isn't about "unicode hell" at all.
I guess this change will break lots of code (or will reveal lots of 
broken code...as it did in my case actually).

- Ralf

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.5 status

[Georg Brandl]

Neal Norwitz wrote:
> Things are getting better, but we still have some really important
> outstanding issues.  PLEASE CONTINUE TESTING AS MUCH AS POSSIBLE.
> Also, it would be great to use as many tools as possible to find bugs
> and improve quality.  It would be especially nice to run Purify on
> Windows.
> 
> I've updated PEP 356 with the current status.  The current schedule is:
> 
> rc 1:August 18, 2006 [planned]
> final:   September 12, 2006 [planned]
> 
> This somewhat strange schedule is to accommodate availability of
> people cutting the release.  A branch will be made when the release
> candidate is done.  Don't try to rush bug fixes in.  Let's try to keep
> things stable.  Keep the doc fixes coming.

> http://python.org/sf/1523610 - PyArg_ParseTupleAndKeywords
> potential core dump

This one's almost fixed if we can decide what to do with "levels".
I wrote some time ago:

"""
With respect to this bug (which is about stack issues in Python/getargs.c
involving misuse of the "levels" array), I think that we can drop the
"levels" thing completely. It's only there to tell the user which exact item
passed as part of a tuple argument cannot be accepted (and only if that
function is implemented in C code). As tuple arguments
are very rare "argument x" should be enough to tell the user that
something's wrong with that tuple.
"""

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys

[Michael Hudson]

"M.-A. Lemburg" <[EMAIL PROTECTED]> writes:

> The point here is that a typical user won't expect any comparisons
> to be made when dealing with dictionaries, simply because the fact
> that you do need to make comparisons is an implementation detail.

Of course looking things up in a dictionary involves comparisons!  How
could it not?

> So in this particular case silencing the exception might be the
> more user friendly way of dealing with the problem.

Please, no.

> That said, the problem still lingers in that dictionary, so it may
> bite you in some other context, e.g. when iterating over the list
> of keys.

For this reason, and others.

Cheers,
mwh

-- 
   web in my head get it out get it out
-- from Twisted.Quotes
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys

[M.-A. Lemburg]

Greg Ewing wrote:
> M.-A. Lemburg wrote:
> 
>> If a string
>> is not ASCII and thus causes the exception, there's not a lot you
>> can say, since you don't know the encoding of the string.
> 
> That's one way of looking at it.
> 
> Another is that any string containing chars > 127 is not
> text at all, but binary data, in which case it's not equal
> to *any* unicode string -- just like bytes objects will
> not be equal to strings in Py3k.
> 
>> All you
>> know is that it's not ASCII. Instead of guessing, Python then raises
>> an exception to let the programmer decide.
> 
> There's no disputing that an exception should be raised
> if the string *must* be interpretable as characters in
> order to continue. But that's not true here if you allow
> for the interpretation that they're simply objects of
> different (duck) type and therefore unequal.

Hmm, given that interpretation, 1 == 1.0 would have to be
False.

Note that you do have to interpret the string as characters
if you compare it to Unicode and there's nothing wrong with
that.

What's making this particular case interesting is that
the comparison is hidden in the dictionary implementation
and only triggers if you get a hash collision, which makes
the whole issue appear to be happening randomly.

This whole thread aside: it's never recommended to mix strings
and Unicode, unless you really have to.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 04 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys

[M.-A. Lemburg]

Ralf Schmitt wrote:
> M.-A. Lemburg wrote:
>> Ralf Schmitt wrote:
>>> Does python 2.4 catch any exception when comparing keys (which are not 
>>> basestrings) in dictionaries?
>> Yes. It does so for all equality compares that need to be done
>> as part of the hash collision algorithm (not only w/r to strings
>> and Unicode, but in general).
>>
>> This was changed in 2.5, which now reports the exception.
>>
> 
> So, this thread isn't about "unicode hell" at all.

For some reason people always think of hell when dealing with Unicode.
Instead, the should think of hell when dealing with strings.

> I guess this change will break lots of code (or will reveal lots of 
> broken code...as it did in my case actually).

I don't think it will break a lot of code that wasn't already
broken.

Let's see how many reports we get for 2.5b3 and then decide.

If things turn out bad, we might silence the UnicodeError
and instead issue a warning everytime this situation occurs.
In 2.6 we'd then revert to raising an exception.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 04 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys

[Bob Ippolito]

On Aug 3, 2006, at 9:34 PM, Josiah Carlson wrote:

>
> Bob Ippolito <[EMAIL PROTECTED]> wrote:
>> On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote:
>>
>>> M.-A. Lemburg wrote:
>>>
 Perhaps we ought to add an exception to the dict lookup mechanism
 and continue to silence UnicodeErrors ?!
>>>
>>> Seems to be that comparison of unicode and non-unicode
>>> strings for equality shouldn't raise exceptions in the
>>> first place.
>>
>> Seems like a slightly better idea than having dictionaries suppress
>> exceptions. Still not ideal though because sticking non-ASCII strings
>> that are supposed to be text and unicode in the same data structures
>> is *probably* still an error.
>
> If/when 'python -U -c "import test.testall"' runs without unexpected
> error (I doubt it will happen prior to the "all strings are unicode"
> conversion), then I think that we can say that there aren't any
> use-cases for strings and unicode being in the same dictionary.
>
> As an alternate idea, rather than attempting to .decode('ascii') when
> strings and unicode compare, why not .decode('latin-1')?  We lose the
> unicode decoding error, but "the right thing" happens (in my opinion)
> when u'\xa1' and '\xa1' compare.

Well, in this case it would cause different behavior if u'\xa1' and  
'\xa1' compared equal. It'd just be an even more subtle error.

-bob

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode hell/mixing str and unicode as dictionarykeys

[Kristján V . Jónsson]

The "string" isn´t necessarily text, so selecting latin-1 doesn´t help  (in 
fact, what happens is that the current default encoding is used, in his case 
this was ascii).  What if it is image data?  What if you are using a dict to 
implement a singleton set for arbitrary objects?

The point is that if the comparison operator raises an exception, the two 
objects are likely to be dissimilar.  We could even define that behaviour.  
Propagating the exception means that you can't have objects as keys in a 
dictionary that raise an exception when compared.  This goes over and beyond 
any unicode vs. string question.

If the propagation of the exception was a concious change for debugging 
purposes, why not make that somehow optional?  A flag on the dict object?  Or 
special lookup mehtods for that?

Cheers,
Kristján

> -Original Message-
> From: [EMAIL PROTECTED] 
> [mailto:[EMAIL PROTECTED] 
> On Behalf Of Josiah Carlson
> Sent: 4. ágúst 2006 04:34
> To: Bob Ippolito; python-dev@python.org
> Subject: Re: [Python-Dev] unicode hell/mixing str and unicode 
> as dictionarykeys
> 
> 
> Bob Ippolito <[EMAIL PROTECTED]> wrote:
> > On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote:
> > 
> > > M.-A. Lemburg wrote:
> > >
> > >> Perhaps we ought to add an exception to the dict lookup 
> mechanism 
> > >> and continue to silence UnicodeErrors ?!
> > >
> > > Seems to be that comparison of unicode and non-unicode 
> strings for 
> > > equality shouldn't raise exceptions in the first place.
> > 
> > Seems like a slightly better idea than having dictionaries suppress 
> > exceptions. Still not ideal though because sticking 
> non-ASCII strings 
> > that are supposed to be text and unicode in the same data 
> structures 
> > is *probably* still an error.
> 
> If/when 'python -U -c "import test.testall"' runs without 
> unexpected error (I doubt it will happen prior to the "all 
> strings are unicode"
> conversion), then I think that we can say that there aren't 
> any use-cases for strings and unicode being in the same dictionary.
> 
> As an alternate idea, rather than attempting to 
> .decode('ascii') when strings and unicode compare, why not 
> .decode('latin-1')?  We lose the unicode decoding error, but 
> "the right thing" happens (in my opinion) when u'\xa1' and 
> '\xa1' compare.
> 
>  - Josiah
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/kristjan%40c
cpgames.com
> 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[Michael Chermside]

I'm changing the subject line because I want to convince everyone that
the problem being discussed in the "unicode hell" thread has nothing
to do with unicode and strings. It's all about dicts.

I have not observed real breakage in my own code, but I will present
a sample of made-up-but-completely-reasonable code that works in
2.4, fails in 2.5, and arguably ought to work fine. I think we should
restore the behavior of dicts that when they compare keys for
equality they suppress exceptions (treating the objects as unequal),
or at LEAST retain the behavior for one more release making it a
warning this time.

Here is my sample code:

--- problem_with_dicts.py --
# A sample program to demonstrate that the proposed behavior
# of dicts in Python 2.5 generates bugs. This is not code from
# an actual program, but it is completely reasonable.

# First import from
# http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/413486
# the only 5-star recipe in the Python Cookbook for implementing
# enums.
import cookbookenum

# Then set up some reasonable enums. We'll say we're writing
# a program for dealing with dates.
DaysOfWeek = cookbookenum.Enum(
 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun')
Months = cookbookenum.Enum(
 'Jan','Feb','Mar','Apr','May','Jun','Jul','Aug',
 'Sep','Oct','Nov','Dec')

# Let's say we also do some translations. Here is a
# useful dictionary:
translations = {}
# which we populate with values
translations[ DaysOfWeek.Mon ] = {
 'en': 'Monday',
 'es': 'Lunes',
 'fr': 'Lundi',
 }
# and assume we do the other days
translations[ Months.Jan ] = {
 'en': 'January',
 'es': 'Enero',
 'fr': 'Janvier',
 }
# and assume we do the other months

# ...then later in the code we could do things like this:
language = 'en'
dayOfWeek = DaysOfWeek.Mon
month = Months.Jan
dayOfMonth = 3
print '%s, %s %s' % (
 translations[dayOfWeek][language],
 translations[month][language],
 dayOfMonth)

# this works in 2.4 but fails in 2.5
- end problem_with_dicts.py 

Please reconsider.

-- Michael Chermside

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys

[Christopher Armstrong]

On 8/4/06, Ralf Schmitt <[EMAIL PROTECTED]> wrote:
Jean-Paul Calderone wrote:>> I like the exception that 2.5 raises.  I only wish it raised by default> when using 'ascii' and u'ascii' as keys in the same dictionary. ;)  Oh,> and that str and unicode did not hash like they do.  ;)
No problem: >>> import sys >>> reload(sys) >>> sys.setdefaultencoding("base64") >>> "a"==u"a"
Traceback (most recent call last):...binascii.Error: Incorrect paddingMaybe this is all just a matter of choosing the right defaultencoding ? :)Doing this is amazingly stupid. I can't believe how often I hear this suggestion. Apparently the fact that you have to "reload(sys)" to do it isn't warning enough?
-- Christopher ArmstrongInternational Man of Twisteryhttp://radix.twistedmatrix.com/http://twistedmatrix.com/
http://canonical.com/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys

[Ralf Schmitt]

Christopher Armstrong wrote:
> On 8/4/06, *Ralf Schmitt* <[EMAIL PROTECTED] > 
> wrote:
> 
> 
> Maybe this is all just a matter of choosing the right
> defaultencoding ? :)
> 
> 
> 
> Doing this is amazingly stupid. I can't believe how often I hear this 
> suggestion. Apparently the fact that you have to "reload(sys)" to do it 
> isn't warning enough?

Ahh, the twisted people, friendly as I know them. Did you actually 
notice the smiley?

- Ralf


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[Michael Hudson]

Michael Chermside <[EMAIL PROTECTED]> writes:

> I'm changing the subject line because I want to convince everyone that
> the problem being discussed in the "unicode hell" thread has nothing
> to do with unicode and strings. It's all about dicts.

I'd say it's more to do with __eq__.  It's a strange __eq__ method
that raises an Exception, IMHO.

Please do realize that the motivation for this change was hours and
hours of tortous debugging caused by a buggy __eq__ method making keys
"impossibly" seem to not be in dictionaries.

> I have not observed real breakage in my own code, but I will present
> a sample of made-up-but-completely-reasonable code that works in
> 2.4, fails in 2.5, and arguably ought to work fine. I think we should
> restore the behavior of dicts that when they compare keys for
> equality they suppress exceptions (treating the objects as unequal),
> or at LEAST retain the behavior for one more release making it a
> warning this time.

Please no.  Here's just one piece of evidence that the 2.4 semantics
are pretty silly too:

>>> d = {u'\xe0':1, '\xe0\:2}
  File "", line 1
d = {u'\xe0':1, '\xe0\:2}
^
SyntaxError: EOL while scanning single-quoted string
>>> d = {u'\xe0':1, '\xe0':2}
>>> '\xe0' in d
True
>>> '\xe0' in d.keys()
Traceback (most recent call last):
  File "", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe0 in position 0: ordinal 
not in range(128)

Cheers,
mwh

-- 
  same software, different verbosity settings (this one goes to
  eleven) -- the effbot on the martellibot
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[Terry Reedy]

"Michael Hudson" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> Michael Chermside <[EMAIL PROTECTED]> writes:
>
>> I'm changing the subject line because I want to convince everyone that
>> the problem being discussed in the "unicode hell" thread has nothing
>> to do with unicode and strings. It's all about dicts.
>
> I'd say it's more to do with __eq__.  It's a strange __eq__ method
> that raises an Exception, IMHO.

I agree; a == b should always work, certainly unless explicitly programmed 
otherwise in Python for a particular class.  So I think the proper solution 
is fix the buggy __eq__ method to return False instead.  If a byte string 
does not have a default (ascii) text interpretation, then it obviously is 
not equal to any particular unicode text.

The fundamental axiom of sets and hence of dict keys is that any 
object/value either is or is not a member (at any given time for 'mutable' 
set collections).  This requires that testing an object for possible 
membership by equality give a clean True or False answer.

> Please do realize that the motivation for this change was hours and
> hours of tortous debugging caused by a buggy __eq__ method making keys
> "impossibly" seem to not be in dictionaries.

So why not fix the buggy __eq__ method?

>> 2.4, fails in 2.5, and arguably ought to work fine. I think we should
>> restore the behavior of dicts that when they compare keys for
>> equality they suppress exceptions (treating the objects as unequal),
>> or at LEAST retain the behavior for one more release making it a
>> warning this time.
>
> Please no.  Here's just one piece of evidence that the 2.4 semantics
> are pretty silly too:

We mostly agreed half a decode ago that 1/2 should be .5 instead of 0, but 
to avoid breaking code, have (or Guido has) refrained from yet making the 
change the default.  To me, the same principle applies here at least as 
strongly.

Terry Jan Reedy



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[M.-A. Lemburg]

Terry Reedy wrote:
> "Michael Hudson" <[EMAIL PROTECTED]> wrote in message 
> news:[EMAIL PROTECTED]
>> Michael Chermside <[EMAIL PROTECTED]> writes:
>>
>>> I'm changing the subject line because I want to convince everyone that
>>> the problem being discussed in the "unicode hell" thread has nothing
>>> to do with unicode and strings. It's all about dicts.
>> I'd say it's more to do with __eq__.  It's a strange __eq__ method
>> that raises an Exception, IMHO.
> 
> I agree; a == b should always work, certainly unless explicitly programmed 
> otherwise in Python for a particular class. 

... which this is.

> So I think the proper solution 
> is fix the buggy __eq__ method to return False instead.  If a byte string 
> does not have a default (ascii) text interpretation, then it obviously is 
> not equal to any particular unicode text.
> 
> The fundamental axiom of sets and hence of dict keys is that any 
> object/value either is or is not a member (at any given time for 'mutable' 
> set collections).  This requires that testing an object for possible 
> membership by equality give a clean True or False answer.
> 
>> Please do realize that the motivation for this change was hours and
>> hours of tortous debugging caused by a buggy __eq__ method making keys
>> "impossibly" seem to not be in dictionaries.
> 
> So why not fix the buggy __eq__ method?

Because it's not buggy.

Python just doesn't know the encoding of the 8-bit string, so can't
make any assumptions on it. As result, it raises an exception to inform
the programmer.

It is well possible that the string uses an encoding where the
Unicode string is indeed the equal to the string, assuming this
encoding, e.g.

>>> s = 'trärää'
>>> u = u'trärää'
>>> s == u
Traceback (most recent call last):
  File "", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 2:
ordinal not in range(128)
>>> hash(s)
673683206
>>> hash(u)
673683206

Here, the encoding that creates the match is Latin-1.

>>> 2.4, fails in 2.5, and arguably ought to work fine. I think we should
>>> restore the behavior of dicts that when they compare keys for
>>> equality they suppress exceptions (treating the objects as unequal),
>>> or at LEAST retain the behavior for one more release making it a
>>> warning this time.
>> Please no.  Here's just one piece of evidence that the 2.4 semantics
>> are pretty silly too:
> 
> We mostly agreed half a decode ago that 1/2 should be .5 instead of 0, but 
> to avoid breaking code, have (or Guido has) refrained from yet making the 
> change the default.  To me, the same principle applies here at least as 
> strongly.

I think that's a different category of semantic change: the integer
division change will cause applications to return wrong data (if not
fixed properly). The exception will just let the application refuse
to continue.

How about generating a warning instead and then go for the exception
in 2.6 ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 04 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[Barry Warsaw]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Aug 4, 2006, at 11:43 AM, M.-A. Lemburg wrote:

> How about generating a warning instead and then go for the exception
> in 2.6 ?

 From the perspective of wanting to avoid blog entries in 2007  
railing against our gratuitous breakages in Python 2.5 but also  
wanting to avoid perpetuating broken code, this solution seems the  
most reasonable.

- -Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.3 (Darwin)

iQCVAwUBRNNu8HEjvBPtnXfVAQL9MQP/SCuLPFwS0m5vWJ3+i2iVZVCg21eXKQte
R8zoTngSx7unxfn5WQ7Bi8l9Ai1SkmZ7z0mOr6UbtRXmxM9+HwSFvpYpWazuaC4R
AylYA1ZbfsnzplHZW/TPxZKZJG++qWK2+mIcdHa9MS6OoBb583BQ4oXN8gs6tT2P
9VoUL5OW4Gw=
=IPeG
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python 2.5b3 and AIX 4.3 - It Works

[Michael Kent]

Because of a requirement to remain compatible with AIX 4.3, I have been forced
to stay with Python 2.3, because while 2.4 would compile under AIX 4.3, it would
segfault immediately when run.

I'm happy to report that Python 2.5b3 compiles and runs fine under AIX 4.3, and
passes most of its test suite.  However, here are a few test failures.  I
realize AIX 4.3 is long obsolete, so is there any interest on the list for me to
report which tests failed and how?

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] uuid module - byte order issue

[Ka-Ping Yee]

On Thu, 3 Aug 2006, Oren Tirosh wrote:
> The UUID module uses network byte order, regardless of the platform
> byte order. On little-endian platforms like Windows the ".bytes"
> property of UUID objects is not compatible with the memory layout

RFC 4122 says:

In the absence of explicit application or presentation protocol
specification to the contrary, a UUID is encoded as a 128-bit
object, as follows:

The fields are encoded as 16 octets, with the sizes and order of
the fields defined above, and with each field encoded with the
Most Significant Byte first (known as network byte order).

> Ka-Ping Yee writes* that the Windows UUID generation calls are not RFC
> 4122 compliant and have an illegal version field.
[...]
> If the correct byte order is used the UUIDs generated by Windows XP
> are valid version 4 UUIDs:

I see.  I stand corrected, then.  My interpretation of RFC 4122 would
be that the uuid.py module currently completely implements RFC 4122;
while Windows XP can't be said to violate RFC 4122, supporting this
alternative byte order would be an optional feature.

> The bytes property and bytes argument to the constructor should use
> the platform byte order.

I disagree; i think the main interface to uuid.py should behave
consistently independent of the platform, and should conform to the
default encoding in RFC 4122.  However, we could certainly consider
adding a 'bytes_le' field to support the functionality you describe.

I'll ask Anthony whether adding such a field would be permitted at
this stage of the release process.

> There is another issue with version 1 uuid generation:
> >>> len(set(uuid.uuid1() for i in range(1000)))
> 992
>
> The problem is that the random clock_seq field is only 14 bits long.
> If enough UUIDs are generated within the same system clock tick there
> will be collisions.

The clock field of the UUID has enough resolution to avoid collisions;
the problem you're describing is a limitation of the platform's clock,
not the UUID module.  It doesn't happen on Mac OS X, for example.

> Suggested solution: use the high-resolution of the
> time field (100ns) to generate a monotonically increasing timestamp
> that advances at least by 1 for each call, when time.time() returns
> the same value on subsequent calls.

That sounds like a fine solution to me.  I'll look into it.


-- ?!ng
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.5b3 and AIX 4.3 - It Works

[Neal Norwitz]

There is at least one outstanding bug report for test_mmap failing on
AIX IIRC.  Possibly another for test_resource.  Please review bug
reports and file new ones/update old ones with the current status.
Unless if you provide patches, they probably won't be fixed though.
No one has access to AIX AFAIK.

n
--

On 8/4/06, Michael Kent <[EMAIL PROTECTED]> wrote:
> Because of a requirement to remain compatible with AIX 4.3, I have been forced
> to stay with Python 2.3, because while 2.4 would compile under AIX 4.3, it 
> would
> segfault immediately when run.
>
> I'm happy to report that Python 2.5b3 compiles and runs fine under AIX 4.3, 
> and
> passes most of its test suite.  However, here are a few test failures.  I
> realize AIX 4.3 is long obsolete, so is there any interest on the list for me 
> to
> report which tests failed and how?
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[Michael Chermside]

Marc-Andre Lemburg writes:

> How about generating a warning instead and then go for the exception
> in 2.6 ?

Agreed. Michael Hudson's explanation convinced me.

-- Michael Chermside

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[Josiah Carlson]

There's one problem with generating a warning for 2.5, and that is the
same problem as generating a warning for possible packages that lack an
__init__.py; users may start to get a bunch of warnings, and be unaware
of how to suppress them.

All in all though, I'm +0 on the warning, and +1 on it not raising an
exception in 2.5 .

 - Josiah

Barry Warsaw <[EMAIL PROTECTED]> wrote:
> 
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> On Aug 4, 2006, at 11:43 AM, M.-A. Lemburg wrote:
> 
> > How about generating a warning instead and then go for the exception
> > in 2.6 ?
> 
>  From the perspective of wanting to avoid blog entries in 2007  
> railing against our gratuitous breakages in Python 2.5 but also  
> wanting to avoid perpetuating broken code, this solution seems the  
> most reasonable.
> 
> - -Barry
> 
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.3 (Darwin)
> 
> iQCVAwUBRNNu8HEjvBPtnXfVAQL9MQP/SCuLPFwS0m5vWJ3+i2iVZVCg21eXKQte
> R8zoTngSx7unxfn5WQ7Bi8l9Ai1SkmZ7z0mOr6UbtRXmxM9+HwSFvpYpWazuaC4R
> AylYA1ZbfsnzplHZW/TPxZKZJG++qWK2+mIcdHa9MS6OoBb583BQ4oXN8gs6tT2P
> 9VoUL5OW4Gw=
> =IPeG
> -END PGP SIGNATURE-
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jcarlson%40uci.edu

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[Jean-Paul Calderone]

On Fri, 04 Aug 2006 11:23:10 -0700, Josiah Carlson <[EMAIL PROTECTED]> wrote:
>
>There's one problem with generating a warning for 2.5, and that is the
>same problem as generating a warning for possible packages that lack an
>__init__.py; users may start to get a bunch of warnings, and be unaware
>of how to suppress them.
>
>All in all though, I'm +0 on the warning, and +1 on it not raising an
>exception in 2.5 .
>

Um.  This warning would indicate a bug in the code which will lead to
actual misbehavior in a future version of Python.  The __init__.py
warning would have indicated a deployment configuration which didn't
actually cause any misbehavior.

They aren't the same case at all, unless you think that all warnings
should be classed this way (a position I do not think is completely
unreasonable, but since you singled out the package warning by way of
comparison, I assume this is not the argument you are trying to make).

Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[Paul Colomiets]

Hi!

Terry Reedy wrote:
> The fundamental axiom of sets and hence of dict keys is that any 
> object/value either is or is not a member (at any given time for 'mutable' 
> set collections).  This requires that testing an object for possible 
> membership by equality give a clean True or False answer.
>   
Yes this makes sense. But returning to dictionaries for python newbies, 
it will be strange why this
 >>> d = { u'abc': 1, u'ab\xe8': 2}
 >>> d['abc']
 >1
works as expected, but this
 >>> d['ab\xe8']
raises an exception.

Another good argument pronounced by M.-A. Lemburg:
> What's making this particular case interesting is that
> the comparison is hidden in the dictionary implementation
> and only triggers if you get a hash collision, which makes
> the whole issue appear to be happening randomly.
>
> This whole thread aside: it's never recommended to mix strings
> and Unicode, unless you really have to.
...
 >How about generating a warning instead and then go for the exception
 >in 2.6 ?

Well it's not recomended to mix strings and unicode in the dictionaries 
but if we mix for example integer and float we have the same thing. It 
doesn't raise exception but still it is not expected behavior for me:
 >>> d = { 1.0: 10, 2.0: 20 }
then if i somewhere later do:
 >>> d[1] = 100
 >>> d[2] = 200
to have here all floats in d.keys(). May be this is not a best example. 
So if you generate a warning, it should be generated every time when 
there are keys of different types inserted into dict. May be python 
should check type of the key after collision and before testing for 
equality? So the 1 and 1.0 is different as u'a' and 'a' also different. 
It even can give some perfomance overhead I think.

--
Regards,
  Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken ...

[Nick Maclaren]

Michael Hudson <[EMAIL PROTECTED]> wrote:
> 
> I'd say it's more to do with __eq__.  It's a strange __eq__ method
> that raises an Exception, IMHO.

Not entirely.  Any type that supports invalid values (e.g. IEEE 754)
and is safe against losing the invalid state by accident needs to
raise an exception on A == B.  IEEE 754 is not safe.

> Please do realize that the motivation for this change was hours and
> hours of tortous debugging caused by a buggy __eq__ method making keys
> "impossibly" seem to not be in dictionaries.

Quite.  Been there - been caught by that.  It is a catastrophic (but
very common) misdesign to conflate failure and the answer "no".
There is a fundamental flaw of that nature in card-based banking,
that I pointed out was insoluble to the Grid people, and then got
caught by just a month later!


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  [EMAIL PROTECTED]
Tel.:  +44 1223 334761Fax:  +44 1223 334679
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[Giovanni Bajo]

Paul Colomiets <[EMAIL PROTECTED]> wrote:

> Well it's not recomended to mix strings and unicode in the
> dictionaries
> but if we mix for example integer and float we have the same thing. It
> doesn't raise exception but still it is not expected behavior for me:
>  >>> d = { 1.0: 10, 2.0: 20 }
> then if i somewhere later do:
>  >>> d[1] = 100
>  >>> d[2] = 200
> to have here all floats in d.keys(). May be this is not a best
> example.

There is a strong difference. Python is moving towards unifying number types in
a way (see the true division issue): the idea is that, all in all, user
shouldn't really care what type a number is, as long as he knows it's a number.
On the other hand, unicode and str are going to diverge more and more.

Giovanni Bajo

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[Bob Ippolito]

On Aug 4, 2006, at 12:51 PM, Giovanni Bajo wrote:

> Paul Colomiets <[EMAIL PROTECTED]> wrote:
>
>> Well it's not recomended to mix strings and unicode in the
>> dictionaries
>> but if we mix for example integer and float we have the same  
>> thing. It
>> doesn't raise exception but still it is not expected behavior for me:
> d = { 1.0: 10, 2.0: 20 }
>> then if i somewhere later do:
> d[1] = 100
> d[2] = 200
>> to have here all floats in d.keys(). May be this is not a best
>> example.
>
> There is a strong difference. Python is moving towards unifying  
> number types in
> a way (see the true division issue): the idea is that, all in all,  
> user
> shouldn't really care what type a number is, as long as he knows  
> it's a number.
> On the other hand, unicode and str are going to diverge more and more.

Well, not really. True division makes int/int return float instead of  
an int. You really do have to care if you have an int or a float most  
of the time, they're very different semantically.

Unicode and str are eventually going to be the same thing (str would  
ideally end up becoming a synonym of unicode). The difference being  
that there will be some other type to contain bytes.

-bob

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[Paul Colomiets]

Giovanni Bajo wrote:
> Paul Colomiets <[EMAIL PROTECTED]> wrote:
>
>   
>> Well it's not recomended to mix strings and unicode in the
>> dictionaries
>> but if we mix for example integer and float we have the same thing. It
>> doesn't raise exception but still it is not expected behavior for me:
>>  >>> d = { 1.0: 10, 2.0: 20 }
>> then if i somewhere later do:
>>  >>> d[1] = 100
>>  >>> d[2] = 200
>> to have here all floats in d.keys(). May be this is not a best
>> example.
>> 
>
> There is a strong difference. Python is moving towards unifying number types 
> in
> a way (see the true division issue): the idea is that, all in all, user
> shouldn't really care what type a number is, as long as he knows it's a 
> number.
> On the other hand, unicode and str are going to diverge more and more.
>
> Giovanni Bajo
>
>   
It makes sense, but consider this example:

 >>> from decimal import Decimal
 >>> d = {}
 >>> d[Decimal(0)] = 1
 >>> d[0] = 2
 >>> d[Decimal("0.5")] = 3
 >>> d[0.5]  = 4
 >>> d.keys()
[Decimal("0"), 0.5, Decimal("0.5")]

I expect d.keys() to have 2 or 4 keys but don't 3, it's confusing. Don't 
you think that someday line "d[0.5] = 4" will raise exception? Or at 
least that it should raise if mixing str and unicode raises?

--
Regards,
  Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] More tracker demos online

[David Ascher]

> ""Martin v. Löwis"" <
[EMAIL PROTECTED]> wrote in message> news:[EMAIL PROTECTED]>> > Currently, we have two running tracker demos online:> >> > Roundup:> > 
http://efod.se/python-tracker/> >> > Jira:> > http://jira.python.atlassian.com/secure/Dashboard.jspa
Is anyone looking at the Google Code Hosting tracker, just for fun? =)  (code.google.com/hosting, although performance seems to be an issue for now)--david 

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[Ron Adam]

M.-A. Lemburg wrote:
> Terry Reedy wrote:
>> "Michael Hudson" <[EMAIL PROTECTED]> wrote in message 
>> news:[EMAIL PROTECTED]
>>> Michael Chermside <[EMAIL PROTECTED]> writes:
>>>
 I'm changing the subject line because I want to convince everyone that
 the problem being discussed in the "unicode hell" thread has nothing
 to do with unicode and strings. It's all about dicts.
>>> I'd say it's more to do with __eq__.  It's a strange __eq__ method
>>> that raises an Exception, IMHO.
>> I agree; a == b should always work, certainly unless explicitly programmed 
>> otherwise in Python for a particular class. 
> 
> ... which this is.
> 
>> So I think the proper solution 
>> is fix the buggy __eq__ method to return False instead.  If a byte string 
>> does not have a default (ascii) text interpretation, then it obviously is 
>> not equal to any particular unicode text.
>>
>> The fundamental axiom of sets and hence of dict keys is that any 
>> object/value either is or is not a member (at any given time for 'mutable' 
>> set collections).  This requires that testing an object for possible 
>> membership by equality give a clean True or False answer.
>>
>>> Please do realize that the motivation for this change was hours and
>>> hours of tortous debugging caused by a buggy __eq__ method making keys
>>> "impossibly" seem to not be in dictionaries.
>> So why not fix the buggy __eq__ method?
> 
> Because it's not buggy.
> 
> Python just doesn't know the encoding of the 8-bit string, so can't
> make any assumptions on it. As result, it raises an exception to inform
> the programmer.
> 
> It is well possible that the string uses an encoding where the
> Unicode string is indeed the equal to the string, assuming this
> encoding, e.g.

Isn't this a case where it should be up to the programmer to make sure 
the comparison makes sense in the context it is being used.  That is, if 
I'm comparing two different forms of data, it's up to me to convert them 
explicitly to the same form before comparing them?

In the case of comparing an 8 bit string and unicode, I would think they 
are always unequal.  But changing that now would probably (?) break way 
too much. (but it may also uncover quite a few potential or even real 
bugs as well.) ;-)

Cheers,
Ron









___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[Josiah Carlson]

Jean-Paul Calderone <[EMAIL PROTECTED]> wrote:
> On Fri, 04 Aug 2006 11:23:10 -0700, Josiah Carlson <[EMAIL PROTECTED]> wrote:
> >There's one problem with generating a warning for 2.5, and that is the
> >same problem as generating a warning for possible packages that lack an
> >__init__.py; users may start to get a bunch of warnings, and be unaware
> >of how to suppress them.
> >
> >All in all though, I'm +0 on the warning, and +1 on it not raising an
> >exception in 2.5 .
> 
> Um.  This warning would indicate a bug in the code which will lead to
> actual misbehavior in a future version of Python.  The __init__.py
> warning would have indicated a deployment configuration which didn't
> actually cause any misbehavior.
> 
> They aren't the same case at all, unless you think that all warnings
> should be classed this way (a position I do not think is completely
> unreasonable, but since you singled out the package warning by way of
> comparison, I assume this is not the argument you are trying to make).

I see both as being a potential cause for a large number of warning
messages to people starting to use Python 2.5 (from 2.3 or 2.4) .

 - Josiah

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] need an SSH key removed

[Brett Cannon]

My [EMAIL PROTECTED] SSH key should be removed since my internship is now over.-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] need an SSH key removed

[Tim Peters]

[Brett Cannon]
> My [EMAIL PROTECTED] SSH key should be removed since my internship is now
> over.

Thank you for being conscientious.  While it feared death, your key
didn't complain about being deleted, and right before it vanished I
saw the most astonishing look of profound peace passing over its bits.
 You should rest assured tonight that yYv6r9zcrb and ZORyPRLc5Sf and
all the other little key segments have gone on to a better repository.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] need an SSH key removed

[Aahz]

On Fri, Aug 04, 2006, Tim Peters wrote:
> [Brett Cannon]
>>
>> My [EMAIL PROTECTED] SSH key should be removed since my internship is now
>> over.
> 
> Thank you for being conscientious.  While it feared death, your key
> didn't complain about being deleted, and right before it vanished I
> saw the most astonishing look of profound peace passing over its bits.
> You should rest assured tonight that yYv6r9zcrb and ZORyPRLc5Sf and
> all the other little key segments have gone on to a better repository.

That means it's now an ex-key?
-- 
Aahz ([EMAIL PROTECTED])   <*> http://www.pythoncraft.com/

C'mon, you knew *someone* was gonna say that
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com