Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
Dave Cinege d...@cinege.com wrote:
You will notice that the code is disgusting simple. However I have
found that this has completely changed the way I program in python.
I've re-written some exiting programs using Thesaurus, and often
relized 15-30% code reduction. Additionally I find the new code much
easier to read.
And here's the same code written without your class but maintaining as
far as possible the same structure. I find my version far easier to read
then your's with all your spurious 'g.' 'L.' prefixes.
-
#!python2.7
from textwrap import dedent
class Blob(object): pass
prog = Blob()
prog.VERSION = '1.0'# But isn't this so much cleaner?
prog.NAME = 'Thesaurus'
class TestClass:
no = 'Class'
way = 'this'
def main ():
tc = TestClass()
l = ['Some', 'objects']
# Here's how you should create output without a fight.
print dedent('''\
When programing python without {prog.NAME}, it is very
easy to access your {l[1]}.
{l[0]} people might say {prog.NAME} has no
{tc.no}.''').format(prog=prog, l=l, tc=tc)
if hasattr(prog, 'VERSION'):
print 'But I challenge them to write code {tc.way} clean
without it!'.format(**locals())
if __name__ == '__main__':
main()
-
--
Duncan Booth http://kupuguy.blogspot.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
在 2013年1月10日星期四UTC+8下午7时34分23秒，Duncan Booth写道：
Dave Cinege d...@cinege.com wrote:
You will notice that the code is disgusting simple. However I have
found that this has completely changed the way I program in python.
I've re-written some exiting programs using Thesaurus, and often
relized 15-30% code reduction. Additionally I find the new code much
easier to read.
And here's the same code written without your class but maintaining as
far as possible the same structure. I find my version far easier to read
then your's with all your spurious 'g.' 'L.' prefixes.
-
#!python2.7
from textwrap import dedent
class Blob(object): pass
prog = Blob()
prog.VERSION = '1.0' # But isn't this so much cleaner?
prog.NAME = 'Thesaurus'
class TestClass:
no = 'Class'
way = 'this'
def main ():
tc = TestClass()
l = ['Some', 'objects']
# Here's how you should create output without a fight.
print dedent('''\
When programing python without {prog.NAME}, it is very
easy to access your {l[1]}.
{l[0]} people might say {prog.NAME} has no
{tc.no}.''').format(prog=prog, l=l, tc=tc)
if hasattr(prog, 'VERSION'):
print 'But I challenge them to write code {tc.way} clean
without it!'.format(**locals())
if __name__ == '__main__':
main()
-
--
Duncan Booth http://kupuguy.blogspot.com
An object can accquire new properties and methods
in the run time without the limitations from
the class definition of the object which belongs to.
This is a true OOP language.
--
http://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
Thesaurus: A different way to call a dictionary.
Thesaurus is a new a dictionary subclass which allows calling keys as
if they are class attributes and will search through nested objects
recursively when __getitem__ is called.
You will notice that the code is disgusting simple. However I have found that
this has completely changed the way I program in python. I've re-written some
exiting programs using Thesaurus, and often realized 15-30% code reduction.
Additionally I find the new code much easier to read.
If you find yourself programing with nested dictionaries often, fighting to
generate output or command lines for external programs, or wish you had
a dictionary that could act (sort of) like a class, Thesaurus may be for you.
#!/usr/bin/env python
thesaurus.py 2012-09-13
Copyright (c) 2012 Dave Cinege
Licence: python, Copyright notice may not be altered.
Thesaurus: A different way to call a dictionary.
Thesaurus is a new a dictionary subclass which allows calling keys as
if they are class attributes and will search through nested objects
recursivly when __getitem__ is called.
You will notice that the code is disgusting simple. However I have found that
this has completely changed the way I program in python. I've re-written some
exiting programs using Thesaurus, and often relized 15-30% code reduction.
Additionally I find the new code much easier to read.
If you find yourself programing with nested dictionaries often, fighting to
generate output or command lines for external programs, or wish you had
a dictionary that could act (sort of) like a class, Thesaurus may be for you.
By example:
---
#!/usr/bin/env python
import thesaurus
thes = thesaurus.Thesaurus
# I like to create a master global object called 'g'.
# No more 'global' statements for me.
g = thes()
g.prog = thes()
g['prog']['VERSION'] = '1.0' # I can do this like a normal nested dictionary
g.prog.VERSION = '1.0' # But isn't this so much cleaner?
g.prog.NAME = 'Thesaurus'
class TestClass:
no = 'Class'
way = 'this'
def main ():
L = thes() # Some local varibles
L.tc = TestClass() # We can also recurse to class attributes
L.l = list() # And recurse to indices too!
L.l.append('Some')
L.l.append('objects')
g.L = L # Easily make these locals global
# Here's the real magic. Creating output without a fight.
print '''
When programing python using %(prog.NAME)s, it is very
easy to access your %(L.l.1)s.
%(L.l.0)s people might say %(prog.NAME)s has no %(L.tc.no)s.
'''.replace('\t','')[1:-1] % g
del g.L # Clean locals up out of global space
# If I was using a storage class, I must use hasattr() or an ugly eval.
# But we can still use a str for the key name and 'in' like on any
# regular dictionary.
if 'VERSION' in g.prog:
print 'But I challenge them to write code %(tc.way)s clean without it!' % L
if __name__ == '__main__':
main()
---
__VERSION__ = 20120913
class Thesaurus (dict):
def __getattr__(self, name):
return self.__getitem__(name)
def __setattr__(self, name, value):
return dict.__setitem__(self, name, value)
def __delattr__(self, name):
return dict.__delitem__(self, name)
def __getitem__(self, key):
if '.' not in key:
return dict.__getitem__(self, key)
l = key.split('.')
if isinstance(l[0], (dict, Thesaurus)):
a = self.data
else:
a = self
for i in range(len(l)): # Walk keys recursivly
try:
a = a[l[i]] # Attempt key
except:
try:
a = getattr(a, l[i]) # Attempt attr
except:
try:
a = a[int(l[i])] # Attempt indice
except:
raise KeyError(key + ' [%s]' % i)
return a
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
Did you intend to give anyone permission to use the code? I see only a copyright notice, but no permissions. -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tue, Jan 8, 2013 at 12:16 PM, Neal Becker ndbeck...@gmail.com wrote: Did you intend to give anyone permission to use the code? I see only a copyright notice, but no permissions. It also says Licence: python, Copyright notice may not be altered. Which suggests to me that the intent is that it be licensed under the PSF License, although I wouldn't want to be the one testing that inference in a courtroom. Dave, this announcement looks identical to the one you posted a couple months ago, and it also seems to include the earlier version of the source. -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Wed, 12 Dec 2012 17:44:24 +1100 Chris Angelico ros...@gmail.com wrote: On Wed, Dec 12, 2012 at 5:34 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: and you consider this not productive, not worth my time. Code review with you must be *all* sorts of fun. He never asked for code review :) I think by posting it he sort of did. He should probably grow a thicker skin before he does so again though. -- D'Arcy J.M. Cain da...@druid.net | Democracy is three wolves http://www.druid.net/darcy/| and a sheep voting on +1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner. IM: da...@vex.net -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Wednesday 12 December 2012 05:25:11 D'Arcy J.M. Cain wrote: As a 16yr OSS vet I know that for every 1 person that that actually creates something there will always be 2000 people to bitch about it. My skin isn't thin, I just don't give a shit to listen to anyone one that doesn't get it. The point to Thesaurus for those that want to pay attention: The concept in these ~25 lines of code have changed the way I program Python and reduced existing functionally identical code up to 30%...and I like the code better. If that doesn't peak your interest, then move on...nothing here for you to see. If you feel it needs to be expanded/corrected, do it and share it. If you can do it better, re-implement it. That's why I sent it to the mailing list. On Wed, 12 Dec 2012 17:44:24 +1100 Chris Angelico ros...@gmail.com wrote: On Wed, Dec 12, 2012 at 5:34 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: and you consider this not productive, not worth my time. Code review with you must be *all* sorts of fun. He never asked for code review :) I think by posting it he sort of did. He should probably grow a thicker skin before he does so again though. -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tuesday 11 December 2012 01:41:38 Ian Kelly wrote: I have a few critiques on the code. First, you might want to use __getattribute__ instead of __getattr__. Otherwise you'll end up File /home/dcinege/src/thesaurus/thesaurus.py, line 84, in __getattribute__ return self.__getitem__(name) RuntimeError: maximum recursion depth exceeded while calling a Python object This takes me into the same hell as when I was trying to implement this as a class. Someone else would have to take the time to do this. __getattr__ doing what I expect it to do, for what i do. Dave -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Wed, Dec 12, 2012 at 12:20 PM, Dave Cinege d...@cinege.com wrote:
On Tuesday 11 December 2012 01:41:38 Ian Kelly wrote:
I have a few critiques on the code. First, you might want to use
__getattribute__ instead of __getattr__. Otherwise you'll end up
File /home/dcinege/src/thesaurus/thesaurus.py, line 84, in
__getattribute__
return self.__getitem__(name)
RuntimeError: maximum recursion depth exceeded while calling a Python object
This takes me into the same hell as when I was trying to implement this as a
class. Someone else would have to take the time to do this. __getattr__ doing
what I expect it to do, for what i do.
def __getattribute__(self, name):
if name.startswith('__') and name.endswith('__'):
return super(Thesaurus, self).__getattribute__(name)
return self.__getitem__(name)
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Wednesday 12 December 2012 15:42:36 Ian Kelly wrote:
def __getattribute__(self, name):
if name.startswith('__') and name.endswith('__'):
return super(Thesaurus, self).__getattribute__(name)
return self.__getitem__(name)
Ian,
Tested, and works as you advertised.
Isn't super() depreciated? I've replaced it with this:
-return super(Thesaurus, self).__getattribute__(name)
+return dict.__getattribute__(self, name)
Aside from a more palatable result in the python shell for otherwise bad
code...does this get me anything else? Is it really worth the performance hit
of 2 string comparisons for every getattribute call?
I also just reviewed all your original comments again:
In general: The recursion code is nothing special to me and I finally settled
on the nested try's as something simple that 'just works good for now'.
I think the concept of what I'm doing here is the big idea.
Should the idea of implementing what Thesaurus does in mainline python ever
happen, those 10 lines of code will likely spark a 3 month jihad about how to
properly do in python which up until now hasn't been something you do in
python.
To me for i in range(len(l)) seems like simpler, faster, tighter code for this
now. It's duly noted that enumerate() is more python and I'm an old fart that
still thinks too much in state machine. I've add except Exception per your
advise.
What I'd really like to hear is that someone reading was curious enough to
convert some existing code and their joy or displeasure with the experience.
Dave
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Wed, Dec 12, 2012 at 3:20 PM, Dave Cinege d...@cinege.com wrote:
On Wednesday 12 December 2012 15:42:36 Ian Kelly wrote:
def __getattribute__(self, name):
if name.startswith('__') and name.endswith('__'):
return super(Thesaurus, self).__getattribute__(name)
return self.__getitem__(name)
Ian,
Tested, and works as you advertised.
Isn't super() depreciated? I've replaced it with this:
-return super(Thesaurus, self).__getattribute__(name)
+return dict.__getattribute__(self, name)
It's not deprecated. Some people consider it harmful, and others
disagree. I was once in the former camp but have shifted somewhat
toward the latter.
Aside from a more palatable result in the python shell for otherwise bad
code...does this get me anything else? Is it really worth the performance hit
of 2 string comparisons for every getattribute call?
It could affect real code, not just interactive code. Any time you
unthinkingly choose an attribute name that happens to be the name of a
dict method (e.g. 'items', which otherwise seems like a rather
innocent variable name), that's a potential bug. Depending on how
that attribute is subsequently accessed, the bug might not even be
noticed immediately.
The performance hit compared to the __getattr__ version on my system
is about 1.3 microseconds per call, as measured by timeit, or around
40%. For comparison, the performance hit of using the __getattr__
version versus just using a global variable is about 1.7 microseconds
per call, or around 4000%. For my own use, I don't consider that
substantial enough to worry about, as I'm not in the business of
writing code that would be making hundreds of thousands of accesses
per second.
Should the idea of implementing what Thesaurus does in mainline python ever
happen, those 10 lines of code will likely spark a 3 month jihad about how to
properly do in python which up until now hasn't been something you do in
python.
The basic idea of proxying attribute access on a dict to key lookup is
pretty common, actually. It likely won't ever make it into the
standard library because 1) there's no clear agreement on what it
should look like; 2) it's easy to roll your own; and 3) it looks too
much like JavaScript. That last probably isn't valid; attribute
proxying is annoying and cumbersome when it automatically happens on
every single object in the language; it's much more manageable when
you have a single type like Thesaurus that you can use only in the
instances where you actually want it.
To me for i in range(len(l)) seems like simpler, faster, tighter code for this
now. It's duly noted that enumerate() is more python and I'm an old fart that
still thinks too much in state machine. I've add except Exception per your
advise.
Your intuition about what seems faster can lead you astray. Using Python 2.7:
timerD = timeit.Timer('for i in range(len(seq)): x = seq[i]', 'seq =
range(5)')
timerE = timeit.Timer('for i, x in enumerate(seq): pass', 'seq = range(5)')
min(timerD.repeat(3))
0.8711640725291545
min(timerE.repeat(3))
0.7172601545726138
Of course, that's running each loop a million times, so the difference
here really is pretty negligible.
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On 12/12/2012 18:13, Dave Cinege wrote: On Wednesday 12 December 2012 05:25:11 D'Arcy J.M. Cain wrote: As a 16yr OSS vet I know that for every 1 person that that actually creates something there will always be 2000 people to bitch about it. My skin isn't thin, I just don't give a shit to listen to anyone one that doesn't get it. The point to Thesaurus for those that want to pay attention: The concept in these ~25 lines of code have changed the way I program Python and reduced existing functionally identical code up to 30%...and I like the code better. If that doesn't peak your interest, then move on...nothing here for you to see. Please don't place responses like this. The Python community prides itself on tolerance. If you don't wish to follow that recommendation please go to an alternative site. -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Wed, 12 Dec 2012 23:56:24 +, Mark Lawrence wrote: Please don't place responses like this. The Python community prides itself on tolerance. If you don't wish to follow that recommendation please go to an alternative site. Well, I must say, I think that you've just won an award for Most Cognitive Dissonance Exhibited In The Shortest Time. I'm not sure how you can yell at somebody that Google users should be dead in one post, and then nine minutes later say this. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Wed, 12 Dec 2012 17:20:53 -0500, Dave Cinege wrote:
Isn't super() depreciated?
Heavens no. super() is the recommended way to do inheritance, and the
*only* way to correctly do multiple inheritance[1]. What makes you think
that it has been deprecated?
[...]
To me for i in range(len(l)) seems like simpler, faster, tighter code
for this now.
It's not. It is more complex, slower, less direct code. Python is not C,
or Fortran, or whatever low-level language you cut your teeth on and get
your intuitions from.
[steve@ando ~]$ python -m timeit -s L = list('abcdefghij') for i in
range(len(L)):
c = L[i]
pass
100 loops, best of 3: 1.67 usec per loop
[steve@ando ~]$ python -m timeit -s L = list('abcdefghij') for i,c in
enumerate(L):
pass
100 loops, best of 3: 1.39 usec per loop
That's only a small performance speedup, but the real advantages are:
* the version with enumerate is much more general: it works with
data structures where the length is expensive to calculate, lazy
data streams where the length is impossible to know up front,
and infinite data streams;
* the version with enumerate makes the intent more clear: since we
care about looping over the items, we should iterate over the
items directly, not over their indices;
* it is more readable and maintainable: both loop variables (the
index and the item) are defined in the same place, the start of
the for-loop, instead of one in the header and one in the body.
What I'd really like to hear is that someone reading was curious enough
to convert some existing code and their joy or displeasure with the
experience.
I don't have any code with nested dicts where this would make a
difference. If I had such code, I would be looking to redesign it so that
I could avoided the nested dicts, not find a palliative. The Zen of
Python is a useful guide to general design principles:
py import this
The Zen of Python, by Tim Peters
[...]
Flat is better than nested.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
Your Thesaurus class violates too many of these principles for it to be
of interest to me. Having spent a good hour or so playing around with it
in the interactive interpreter, it is too hard for me to reason about
what it is doing (especially since your description of what it does is
actively misleading), and too hard to predict under what circumstances it
will fail.
Short code is not necessarily simple code, and I find your class too
magical and complicated to be interested in using it in production code
as it stands now.
[1] Well, technically there's another way: one might reimplement the
functionality of super() in your own code, and avoid using super() while
having all the usual joys of reinventing the wheel.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Thu, Dec 13, 2012 at 11:30 AM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Wed, 12 Dec 2012 17:20:53 -0500, Dave Cinege wrote: To me for i in range(len(l)) seems like simpler, faster, tighter code for this now. * the version with enumerate makes the intent more clear: since we care about looping over the items, we should iterate over the items directly, not over their indices; To add to this: Using enumerate gives the possibility (don't know if any current interpreter takes advantage or not, but a future one certainly could) that the enumerate() call could be optimized out. Yes, it's theoretically possible that someone could redefine enumerate, which would be broken by such an optimization. But all it'd take is some kind of directive-based optimization and voila, safe performance improvements. Example code used: def foo(x): for i,val in enumerate(x): print(x[%d] = %s%(i,str(val))) def foo(x): for i in range(len(x)): val=x[i] print(x[%d] = %s%(i,str(val))) A simple look at dis.dis() for the above two functions disproves the faster. Steven has already disproven the simpler and tighter. (I would like, though, to see a key-providing iteration as a standard feature. Looking at dis.dis() for the above examples and also at a simple iteration over a dictionary's .items(), I'm seeing what looks like a lot of effort to deal with the fact that iterators return a stream of items, rather than a stream of keys and values. But it's probably not worth changing now.) ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Wednesday 12 December 2012 20:14:08 Chris Angelico wrote: Ok enough already, I'll use the frigging thing! Be happy I'm at least still not coding for python 1.5. To add to this: Using enumerate gives the possibility (don't know if any current interpreter takes advantage or not, but a future one certainly could) that the enumerate() call could be optimized out. Yes, it's theoretically possible that someone could redefine enumerate, which would be broken by such an optimization. But all it'd take is some kind of directive-based optimization and voila, safe performance improvements. -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On 12/12/2012 7:30 PM, Steven D'Aprano wrote: On Wed, 12 Dec 2012 17:20:53 -0500, Dave Cinege wrote: Isn't super() depreciated? Heavens no. super() is the recommended way to do inheritance, and the *only* way to correctly do multiple inheritance[1]. Indeed. Rather than super() being deprecated, it was made easier to use in 3.x by being special cased during compilation. Notice the difference of signatures: 2.7: super(type[, object-or-type]) 3.3: super([type[, object-or-type]]) The zero argument form only works inside a class definition, as the compiler fills in the necessary details to correctly retrieve the class being defined, as well as accessing the current instance for ordinary methods. [1] Well, technically there's another way: one might reimplement the functionality of super() in your own code, and avoid using super() while having all the usual joys of reinventing the wheel. This deeper integration means that it could not be completely reimplemented in Python ;-). -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes - v20121212
Version 20121212
#!/usr/bin/env python
thesaurus.py 2012-12-12
Copyright (c) 2012 Dave Cinege
Licence: PSF Licence, Copyright notice may not be altered.
Thesaurus: A different way to call a dictionary.
Thesaurus is a new a dictionary subclass which allows calling keys as
if they are class attributes and will search through nested objects
recursively when __getitem__ is called.
You will notice that the code is disgusting simple. However I have found that
this has completely changed the way I program in Python. I've re-written some
exiting programs using Thesaurus, and often realized 15-30% code reduction.
Additionally I find the new code much easier to read.
If you find yourself programing with nested dictionaries often, fighting to
generate output or command lines for external programs, or wish you had
a dictionary that could act (sort of) like a class, Thesaurus may be for you.
By example:
---
#!/usr/bin/env python
import thesaurus
thes = thesaurus.Thesaurus
# I like to create a master global object called 'g'.
# No more 'global' statements for me.
g = thes()
g.prog = thes()
g['prog']['VERSION'] = '1.0' # I can do this like a normal nested dictionary
g.prog.VERSION = '1.0' # But isn't this so much cleaner?
g.prog.NAME = 'Thesaurus'
class TestClass:
no = 'Class'
way = 'this'
def main ():
L = thes() # Some local varibles
L.tc = TestClass() # We can also recurse to class attributes
L.l = list() # And recurse to indices too!
L.l.append('Some')
L.l.append('objects')
g.L = L # Easily make these locals global
# Here's the real magic. Creating output without a fight.
print '''
When programing python using %(prog.NAME)s, it is very
easy to access your %(L.l.1)s.
%(L.l.0)s people might say %(prog.NAME)s has no %(L.tc.no)s.
'''.replace('\t','')[1:-1] % g
del g.L # Clean locals up out of global space
# If I was using a storage class, I must use hasattr() or an ugly eval.
# But we can still use a str for the key name and 'in' like on any
# regular dictionary.
if 'VERSION' in g.prog:
print 'But I challenge them to write code %(tc.way)s clean without it!' % L
if __name__ == '__main__':
main()
---
__VERSION__ = 20121212
class Thesaurus (dict):
def __getattribute__(self, name):
if name.startswith('__') and name.endswith('__'):
return dict.__getattribute__(self, name)
return self.__getitem__(name)
def __setattr__(self, name, value):
return dict.__setitem__(self, name, value)
def __delattr__(self, name):
return dict.__delitem__(self, name)
def __getitem__(self, key):
if '.' not in key:
return dict.__getitem__(self, key)
a = self
l = key.split('.')
for i,v in enumerate(l): # Walk keys recursivly
try:
a = a[v] # Attempt key
except Exception:
try:
a = getattr(a, v) # Attempt attr
except Exception:
try:
a = a[int(v)] # Attempt indice
except Exception:
raise KeyError('%(v)s (%(key)s[%(i)s])' % locals() )
return a
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Monday 10 December 2012 23:08:24 Jason Friedman wrote: 2) Posting your code at ActiveState.com. If someone wants to please do. I'm back to being completely overloaded with normal work now. The v20121212 release based on the last few days comments is as far as I will go with this now. Dave -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Mon, 10 Dec 2012 22:48:50 -0500, Dave Cinege wrote:
Thesaurus: A different way to call a dictionary.
Is this intended as a ready-for-production class?
Thesaurus is a new a dictionary subclass which allows calling keys as if
they are class attributes and will search through nested objects
recursively when __getitem__ is called.
If only that were true...
py d = Thesaurus()
py d['spam'] = {}
py d['spam']['ham'] = 'cheese' # key access works
py d.spam.ham # but attribute access doesn't
Traceback (most recent call last):
File stdin, line 1, in module
AttributeError: 'dict' object has no attribute 'ham'
You will notice that the code is disgusting simple. However I have found
that this has completely changed the way I program in python.
By making it less robust and more prone to errors?
py d = Thesaurus()
py d.copy = some value
py assert d.copy == some value
Traceback (most recent call last):
File stdin, line 1, in module
AssertionError
Operations which silently fail or do the wrong thing are not a good
thing. Scarily, this isn't even consistent:
py %(copy)s % d
'some access'
py d.copy
built-in method copy of Thesaurus object at 0xb717a65c
Some further comments about your code:
class Thesaurus (dict):
[...]
def __getitem__(self, key):
if '.' not in key:
return dict.__getitem__(self, key)
l = key.split('.')
if isinstance(l[0], (dict, Thesaurus)):
a = self.data
Testing for isinstance (dict, Thesaurus) is redundant, since Thesaurus is
a subclass of dict.
More importantly, there are no circumstances where l[0] will be a dict.
Since l is created by splitting a string, l[0] must be a string and this
clause is dead code.
Which is good, because self.data is not defined anywhere, so if you ever
did reach this branch, your code would fail.
else:
a = self
for i in range(len(l)): # Walk keys recursivly
This is usually better written as:
for subkey in l:
# look up subkey
# or if all else fails
raise KeyError(subkey)
KeyError('spam.ham.cheese [1]') is misleading, since it implies to me
that looking up spam.ham.cheese succeeded, and the failed lookup was on
1. I would expect either of these:
KeyError('spam.ham')
KeyErroor('ham')
with a slight preference for the first.
Further issues with your code:
try:
a = a[l[i]] # Attempt key
except:
Bare excepts like this are not good. At best they are lazy and sloppy,
good only for throw-away code. At worst, they are hide bugs. In this
case, they can hide bugs:
py class X(object):
... @property
... def test(self):
... return 1/0 # oops a bug
...
py d = Thesaurus(a=X())
py d.a.test # Gives the correct exception for the error.
Traceback (most recent call last):
File stdin, line 1, in module
File stdin, line 4, in test
ZeroDivisionError: float division
py
py %(a.test)s % d # Lies about the problem.
Traceback (most recent call last):
File stdin, line 1, in module
File stdin, line 29, in __getitem__
KeyError: 'a.test [1]'
One more comment:
# I like to create a master global object called 'g'.
# No more 'global' statements for me.
g = thes()
This is not a good thing. Encouraging the use of globals is a bad thing.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
- Original Message - Thesaurus: A different way to call a dictionary. Thesaurus is a new a dictionary subclass which allows calling keys as if they are class attributes and will search through nested objects recursively when __getitem__ is called. You will notice that the code is disgusting simple. However I have found that this has completely changed the way I program in python. I've re-written some exiting programs using Thesaurus, and often realized 15-30% code reduction. Additionally I find the new code much easier to read. If you find yourself programing with nested dictionaries often, fighting to generate output or command lines for external programs, or wish you had a dictionary that could act (sort of) like a class, Thesaurus may be for you. -- http://mail.python.org/mailman/listinfo/python-list did you know you can write class TestClass: no = 'Class' way = 'this' tc = TestClass() print Something you find clean, and I'm not sure why, this is way %(way)s and this is no %(no)s % vars(tc). JM -- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tuesday 11 December 2012 01:41:38 Ian Kelly wrote: running into bugs like this: thes = Thesaurus() thes.update = 'now' thes.update built-in method update of Thesaurus object at 0x01DB30C8 I've noticed this but it's mostly pointless, as meaningful code does work. (Also you stepped on the existing 'update()' dictionary method.) import thesaurus t = thesaurus.Thesaurus() t.s = 'now' t.s + ' this' 'now this' Works perfect. However I'll take a look at __getattribute__ as from what you say it's more proper. Second, in __getitem__ you start a loop with for i in range(len(l)):, and then you use i as an index into l several times. It would be cleaner and more Pythonic to do for i, part in enumerate(l):, and then you can replace every occurrence of l[i] My python is still 'old school' due to being stuck on old versions for in production embedded system python applications. It's not clear to me what the isinstance call here is meant to be testing for. It's used to determine if it's the root instance of the recursive string because self.data, not self must be used to access that. Can you offer a better way? The prior statements require key to be a string. If key is a string, then by construction l[0] is also a string. So it seems to me that the isinstance check here will always be False. OK, try and remove it and then get back to me. :-) In any case, the key splitting here seems to be done primarily to support the use of formatting placeholders like %(L.l.1)s in the examples. I want to point out that this use case is already well supported (I might even say better supported since it cleanly distinguishes index elements from attributes with syntax) by the Thesaurus recursion works with modules like Template, in addition to allowing easy hierarchical organization of (global) data. It's not about string formatting. It's about organizing data and recursive retrieval of that data. Lastly, you have several bare except clauses in the code. Bare Not going to get into religion. There is no 'correct' way to do this code at all because it's not established normal python. I left it simple and it suits my needs. I've converted several production commercial utilities and applications to use Thesaurus, and have seen nothing but benefit. If anyone needs more to this, have at it, I'm busy Dave -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tuesday 11 December 2012 03:12:19 Steven D'Aprano wrote:
Is this intended as a ready-for-production class?
For me, yes. In production code.
py d = Thesaurus()
py d['spam'] = {}
Maybe because spam is type dict instead of type thes???
import thesaurus
thes = thesaurus.Thesaurus
t = thes()
t.spam = thes()
t.spam.ham = 'cheese'
print t.spam.ham
cheese
print t['spam'].ham
cheese
print t['spam']['ham']
cheese
'%(spam.ham)s' % t
'cheese'
Works for me!
Remainder of your post, not productive, not worth my time.
Dave
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tue, Dec 11, 2012 at 1:57 PM, Dave Cinege d...@linkscape.net wrote: On Tuesday 11 December 2012 01:41:38 Ian Kelly wrote: Second, in __getitem__ you start a loop with for i in range(len(l)):, and then you use i as an index into l several times. It would be cleaner and more Pythonic to do for i, part in enumerate(l):, and then you can replace every occurrence of l[i] My python is still 'old school' due to being stuck on old versions for in production embedded system python applications. Just out of curiosity, how old are we talking? enumerate was added in Python 2.3, which is nearly 10 years old. Prior to 2.2 I don't think it was even possible to subclass dict, which would make your Thesaurus implementation unusable, so are these systems running Python 2.2? It's not clear to me what the isinstance call here is meant to be testing for. It's used to determine if it's the root instance of the recursive string because self.data, not self must be used to access that. Can you offer a better way? The prior statements require key to be a string. If key is a string, then by construction l[0] is also a string. So it seems to me that the isinstance check here will always be False. OK, try and remove it and then get back to me. :-) Okay. I replaced this code: if isinstance(l[0], (dict, Thesaurus)): a = self.data else: a = self with: a = self and then I ran the examples, and the output was unchanged. As Steven pointed out, I don't see how that first branch could succeed anyway, since self.data is never defined. -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tue, Dec 11, 2012 at 2:53 PM, Ian Kelly ian.g.ke...@gmail.com wrote: and then I ran the examples, and the output was unchanged. As Steven pointed out, I don't see how that first branch could succeed anyway, since self.data is never defined. It occurs to me that the UserDict class does have a data attribute. Perhaps there was another version of Thesaurus that inherited from UserDict instead of dict, and so the access to .data is a holdover from legacy code? -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tuesday 11 December 2012 16:53:12 Ian Kelly wrote:
Just out of curiosity, how old are we talking? enumerate was added in
Python 2.3, which is nearly 10 years old. Prior to 2.2 I don't think
it was even possible to subclass dict, which would make your Thesaurus
implementation unusable, so are these systems running Python 2.2?
I'm finally beyond 2.2 and getting rid of 2.4 soon. Just started using 2.6 5
months ago.
Thesaurus initially came about from me doing this:
class Global:
pass
g = Global()
As a way to organize/consolidate global vars and eliminate the global
statement.
After a brain fart one day I expanded this to some simple recursion and felt I
was onto something as my entire life changed with how easy it now was to build
output strings.
As you noted it was not possible to subclass dict, so I first tried with a
class, and you run into recursion hell with __setatrib__ to which i think
there is no fix.
I then made a UserDict version. Then when I moved mostly to 2.6 I could do a
proper dict subclass.
So I believe you're actually correct here
if isinstance(l[0], (dict, Thesaurus)):
a = self.data
Looks like an artifact from my UserDict version and was needed. :-(
class UserDict:
def __init__(self, dict=None, **kwargs):
self.data = {}
Thanks for this. You'll see from the version number I wrote this 3 months ago
so it's not 100% fresh in my mind. I'm releasing it now because a python coder
I contracted to pick up some slack for me saw this and went ape at how much he
liked it...and that prompted me to finally get it out into the wild.
As for in depth discussion and enhancement to this, I lack the time.
Dave
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On 11Dec2012 15:57, Dave Cinege d...@linkscape.net wrote: | On Tuesday 11 December 2012 01:41:38 Ian Kelly wrote: | running into bugs like this: | thes = Thesaurus() | thes.update = 'now' | thes.update | | built-in method update of Thesaurus object at 0x01DB30C8 | | I've noticed this but it's mostly pointless, as meaningful code does work. | (Also you stepped on the existing 'update()' dictionary method.) I think that was a deliberate choice of name by Ian. I've got a class like Thesaurus that subclasses dict and maps attributes to dictionary elements (with a few special purpose tweaks I could go into if anyone cares). I made a deliberate decision to only map UPPERCASE attributes to dict keys to avoid exactly the kind of conflict above, because: thes.update = 'now' must either trash the dict.update method _or_ fail to present .update as 'now'. Both have their downsides. So at the cost of shoutier but still effective code I accepted only .UPPERCASE attribute names as mapping to keys. This compromise also makes subclassing much easier, because the subclasser is free to use conventional lowercase attribute names. Cheers, -- Cameron Simpson c...@zip.com.au Thousands at his bidding speed, And post o'er land and ocean without rest - Milton -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tuesday 11 December 2012 17:39:12 Dave Cinege wrote: My memory is getting jogged more why did some things: raise KeyError(key + ' [%s]' % i) I did this to specificly give you the indice that failed recursion but provide the entire key name as it was provided to __getitem__ So if: g.cfg.host.cpu fails recursion on 'host' you will see: g.cfg.host.cpu [2] I know my code sent g.cfg.host.cpu. I know host failed. if It was g.cfg.host.host, I'd know which host failed. Makes sense to me. Works for me. Sure there are other ways to do it. -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On 12/11/2012 05:39 PM, Dave Cinege wrote: On Tuesday 11 December 2012 16:53:12 Ian Kelly wrote: Just out of curiosity, how old are we talking? enumerate was added in Python 2.3, which is nearly 10 years old. Prior to 2.2 I don't think it was even possible to subclass dict, which would make your Thesaurus implementation unusable, so are these systems running Python 2.2? I'm finally beyond 2.2 and getting rid of 2.4 soon. Just started using 2.6 5 months ago. Thesaurus initially came about from me doing this: class Global: pass g = Global() As a way to organize/consolidate global vars and eliminate the global statement. I think that's the key issue here. I find that when code is well structured, you pretty much never have a need for global statements, anyway. By the way, the Thesaurus class reminds me of using the old recipe called 'Bunch': http://code.activestate.com/recipes/52308-the-simple-but-handy-collector-of-a-bunch-of-named/ like this: b = Bunch(x=1) b.stuff = Bunch(y=2) b.stuff.y 2 I've also seen an answer on StackOverflow that uses automatic recursive 'Bunching': http://stackoverflow.com/questions/1123000/does-python-have-anonymous-classes I've seen another variation of recursive bunching, I think it was by Alex Martelli on StackOverflow, but I can't find it now, I believe it used defaultdict as part of it.. This approach can be handy sometimes but has drawbacks, as others have pointed out. I think the issue is that it's not a solution for avoiding globals, which is not a problem in need of solution, but this can be a quick and dirty way to organize a few levels of dicts/Bunches and usually people come up with a custom variation on these recipes that suit their program. -- Lark's Tongue Guide to Python: http://lightbird.net/larks/ -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On 12/11/2012 07:53 PM, Mitya Sirenef wrote: By the way, the Thesaurus class reminds me of using the old recipe called 'Bunch': http://code.activestate.com/recipes/52308-the-simple-but-handy-collector-of-a-bunch-of-named/ like this: b = Bunch(x=1) b.stuff = Bunch(y=2) b.stuff.y 2 Sorry, this was meant to be: b = Bunch(x=1) b.stuff = Bunch(y=2) b.stuff.y 2 -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tue, 11 Dec 2012 16:08:34 -0500, Dave Cinege wrote:
On Tuesday 11 December 2012 03:12:19 Steven D'Aprano wrote:
Is this intended as a ready-for-production class?
For me, yes. In production code.
py d = Thesaurus()
py d['spam'] = {}
Maybe because spam is type dict instead of type thes???
Well duh. By the way, it's a dirty trick to cut out all context to try to
make me seem like an idiot.
In context, you stated that Thesaurus will search through nested objects
recursively when __getitem__ is called, but in fact that is not true. It
does not do what you state it does.
Remainder of your post, not productive, not worth my time.
Oh well I'm just hurt now. *sobs*
So, let's see now... I identified that your Thesaurus code:
* will fail silently;
* contains dead code that is never used;
* contains redundant code that is pointless;
* hides errors in the caller's code;
and you consider this not productive, not worth my time. Code review
with you must be *all* sorts of fun.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tue, 11 Dec 2012 16:08:34 -0500, Dave Cinege wrote:
On Tuesday 11 December 2012 03:12:19 Steven D'Aprano wrote:
Is this intended as a ready-for-production class?
For me, yes. In production code.
py d = Thesaurus()
py d['spam'] = {}
Maybe because spam is type dict instead of type thes???
Well duh. By the way, it's a dirty trick to cut out all context to try to
make me seem like an idiot.
In context, you stated that Thesaurus will search through nested objects
recursively when __getitem__ is called, but in fact that is not true. It
does not do what you state it does.
Remainder of your post, not productive, not worth my time.
Oh well I'm just hurt now. *sobs*
So, let's see now... I identified that your Thesaurus code:
* will fail silently;
* contains dead code that is never used;
* contains redundant code that is pointless;
* hides errors in the caller's code;
and you consider this not productive, not worth my time. Code review
with you must be *all* sorts of fun.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Wed, Dec 12, 2012 at 5:34 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: So, let's see now... I identified that your Thesaurus code: * will fail silently; * contains dead code that is never used; * contains redundant code that is pointless; * hides errors in the caller's code; and you consider this not productive, not worth my time. Code review with you must be *all* sorts of fun. He never asked for code review :) ChrisA -- http://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
Thesaurus: A different way to call a dictionary.
Thesaurus is a new a dictionary subclass which allows calling keys as
if they are class attributes and will search through nested objects
recursively when __getitem__ is called.
You will notice that the code is disgusting simple. However I have found that
this has completely changed the way I program in python. I've re-written some
exiting programs using Thesaurus, and often realized 15-30% code reduction.
Additionally I find the new code much easier to read.
If you find yourself programing with nested dictionaries often, fighting to
generate output or command lines for external programs, or wish you had
a dictionary that could act (sort of) like a class, Thesaurus may be for you.
#!/usr/bin/env python
thesaurus.py 2012-09-13
Copyright (c) 2012 Dave Cinege
Licence: python, Copyright notice may not be altered.
Thesaurus: A different way to call a dictionary.
Thesaurus is a new a dictionary subclass which allows calling keys as
if they are class attributes and will search through nested objects
recursivly when __getitem__ is called.
You will notice that the code is disgusting simple. However I have found that
this has completely changed the way I program in python. I've re-written some
exiting programs using Thesaurus, and often relized 15-30% code reduction.
Additionally I find the new code much easier to read.
If you find yourself programing with nested dictionaries often, fighting to
generate output or command lines for external programs, or wish you had
a dictionary that could act (sort of) like a class, Thesaurus may be for you.
By example:
---
#!/usr/bin/env python
import thesaurus
thes = thesaurus.Thesaurus
# I like to create a master global object called 'g'.
# No more 'global' statements for me.
g = thes()
g.prog = thes()
g['prog']['VERSION'] = '1.0' # I can do this like a normal nested dictionary
g.prog.VERSION = '1.0' # But isn't this so much cleaner?
g.prog.NAME = 'Thesaurus'
class TestClass:
no = 'Class'
way = 'this'
def main ():
L = thes() # Some local varibles
L.tc = TestClass() # We can also recurse to class attributes
L.l = list() # And recurse to indices too!
L.l.append('Some')
L.l.append('objects')
g.L = L # Easily make these locals global
# Here's the real magic. Creating output without a fight.
print '''
When programing python using %(prog.NAME)s, it is very
easy to access your %(L.l.1)s.
%(L.l.0)s people might say %(prog.NAME)s has no %(L.tc.no)s.
'''.replace('\t','')[1:-1] % g
del g.L # Clean locals up out of global space
# If I was using a storage class, I must use hasattr() or an ugly eval.
# But we can still use a str for the key name and 'in' like on any
# regular dictionary.
if 'VERSION' in g.prog:
print 'But I challenge them to write code %(tc.way)s clean without it!' % L
if __name__ == '__main__':
main()
---
__VERSION__ = 20120913
class Thesaurus (dict):
def __getattr__(self, name):
return self.__getitem__(name)
def __setattr__(self, name, value):
return dict.__setitem__(self, name, value)
def __delattr__(self, name):
return dict.__delitem__(self, name)
def __getitem__(self, key):
if '.' not in key:
return dict.__getitem__(self, key)
l = key.split('.')
if isinstance(l[0], (dict, Thesaurus)):
a = self.data
else:
a = self
for i in range(len(l)): # Walk keys recursivly
try:
a = a[l[i]] # Attempt key
except:
try:
a = getattr(a, l[i]) # Attempt attr
except:
try:
a = a[int(l[i])] # Attempt indice
except:
raise KeyError(key + ' [%s]' % i)
return a
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
Thesaurus is a new a dictionary subclass which allows calling keys as if they are class attributes and will search through nested objects recursively when __getitem__ is called. Good stuff. You might consider: 1) Licensing under an OSI-approved license (http://opensource.org/licenses/index.html). 2) Posting your code at ActiveState.com. -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Mon, Dec 10, 2012 at 8:48 PM, Dave Cinege d...@cinege.com wrote:
Thesaurus: A different way to call a dictionary.
Thesaurus is a new a dictionary subclass which allows calling keys as
if they are class attributes and will search through nested objects
recursively when __getitem__ is called.
You will notice that the code is disgusting simple. However I have found that
this has completely changed the way I program in python. I've re-written some
exiting programs using Thesaurus, and often realized 15-30% code reduction.
Additionally I find the new code much easier to read.
If you find yourself programing with nested dictionaries often, fighting to
generate output or command lines for external programs, or wish you had
a dictionary that could act (sort of) like a class, Thesaurus may be for you.
I have a few critiques on the code. First, you might want to use
__getattribute__ instead of __getattr__. Otherwise you'll end up
running into bugs like this:
thes = Thesaurus()
thes.update = 'now'
thes.update
built-in method update of Thesaurus object at 0x01DB30C8
Hey, where'd my data go? The answer is that it is in the Thesaurus:
thes['update']
42
But it's not visible as an attribute because it is shadowed by the
dict methods. Using __getattribute__ instead of __getattr__ would
mean that those non-special methods simply wouldn't be visible at all.
Second, in __getitem__ you start a loop with for i in
range(len(l)):, and then you use i as an index into l several times.
It would be cleaner and more Pythonic to do for i, part in
enumerate(l):, and then you can replace every occurrence of l[i]
with part (or whatever you want to call that variable).
Third, also in __getitem__ you have this code:
if '.' not in key:
return dict.__getitem__(self, key)
l = key.split('.')
if isinstance(l[0], (dict, Thesaurus)):
a = self.data
else:
a = self
It's not clear to me what the isinstance call here is meant to be
testing for. The prior statements require key to be a string. If key
is a string, then by construction l[0] is also a string. So it seems
to me that the isinstance check here will always be False.
In any case, the key splitting here seems to be done primarily to
support the use of formatting placeholders like %(L.l.1)s in the
examples. I want to point out that this use case is already well
supported (I might even say better supported since it cleanly
distinguishes index elements from attributes with syntax) by the
str.format style of string formatting:
L = {'l': ['zero', 'one']}
There should be {L[l][1]}-- and preferably only {L[l][1]} --obvious way to
do it..format(L=L)
'There should be one-- and preferably only one --obvious way to do it.'
Lastly, you have several bare except clauses in the code. Bare
excepts are almost always incorrect. I appreciate that it's not easy
to predict exactly what exceptions might turn up here (although I
posit that for all of these, subsets of (TypeError, KeyError,
AttributeError, IndexError) are sufficient), but at the very minimum
you should specify except Exception, so that you're not
inadvertently catching things like SystemExit and KeyboardInterrupt.
Cheers and hope this is helpful,
Ian
--
http://mail.python.org/mailman/listinfo/python-list
