Re: Dunder variables
Frank Millman wrote: > Hi all > > I have read that one should not call dunder methods in application code. > > Does the same apply to dunder variables? I am thinking of the instance > attribute __dict__, which allows access to the contents of the instance. > > I only want to read from __dict__, not update it. Is this frowned upon? Why would you care whether it is "frowned upon" if it's the best way to achieve something useful? So the real question is /what/ you are trying to do and what your options are. PS: vars(a) is the non-dunder equivalent of a.__dict__. Both have the same limitations, like failing with __slots__ and not seeing properties and class attributes. -- https://mail.python.org/mailman/listinfo/python-list
Re: Dunder variables
On Tue, 09 Jan 2018 09:55:49 +0200, Frank Millman wrote: > Hi all > > I have read that one should not call dunder methods in application code. "Should not" rather than "must not", but that's correct. In general, calling a dunder method directly is *not* the same as the operation you probably want. For example, `x + y` may not be the same as `x.__add__(y)` or `y.__radd__(x)`. Better to use `operator.add(x, y)` instead. > Does the same apply to dunder variables? I am thinking of the instance > attribute __dict__, which allows access to the contents of the instance. > > I only want to read from __dict__, not update it. Is this frowned upon? If you are trying to look up obj.attribute where you don't know the name of the attribute until runtime, the right way is to use getattr: name = 'spam' # this is unknown until runtime value = getattr(obj, name) # like obj.spam If you want to avoid inheritance and avoid looking up attributes on the class or any superclasses, but only look it up on the instance, then the nice way is to use vars(obj): vars(obj)['attribute'] But be aware that not all objects even have a __dict__ -- not even all those with data attributes. -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: Dunder variables
"Peter Otten" wrote in message news:p31v3m$pji$1...@blaine.gmane.org... Frank Millman wrote: > Hi all > > I have read that one should not call dunder methods in application code. > > Does the same apply to dunder variables? I am thinking of the instance > attribute __dict__, which allows access to the contents of the instance. > > I only want to read from __dict__, not update it. Is this frowned upon? Why would you care whether it is "frowned upon" if it's the best way to achieve something useful? So the real question is /what/ you are trying to do and what your options are. Here is a brief history of how I have got to where I am now. I have a class call Context containing only data, not methods. Instances are passed around a lot in my application, with various methods accessing various attributes. I wanted to allow various parts of my app to 'piggy back' on this by adding their own attributes at certain points, to be read back at various other points. My first attempt was to add a new attribute to Context called 'vars', which was an empty dictionary. Any method could add to it or read from it, but I would basically ignore it. This worked, but it was awkward. My entire app revolves around passing passing units of data around, and my address mechanism is always two-part - container name, attribute name. This introduced a third element - container name, attribute name, key name. To tidy this up, I changed it to allow other parts of the app to store attributes directly into Context. To protect my 'core' attributes, I made them read-only, using @property. This all works quite well. Now I have a situation where various processes are 'long-running', and I need to persist the data to disk so that it can be recovered in the event of a crash. I want to use Json to store the data as a dictionary. However, I have no idea which additional attributes have been added by other parts of the application. My solution is to use __dict__ to get at the data. If there are any other options, I will be interested to hear them. Thanks Frank -- https://mail.python.org/mailman/listinfo/python-list
Re: Dunder variables
"Frank Millman" wrote in message news:p321rb$9ct$1...@blaine.gmane.org... "Peter Otten" wrote in message news:p31v3m$pji$1...@blaine.gmane.org... Frank Millman wrote: > Hi all > > I have read that one should not call dunder methods in application code. > > Does the same apply to dunder variables? I am thinking of the instance > attribute __dict__, which allows access to the contents of the instance. > > I only want to read from __dict__, not update it. Is this frowned upon? Why would you care whether it is "frowned upon" if it's the best way to achieve something useful? So the real question is /what/ you are trying to do and what your options are. Here is a brief history of how I have got to where I am now. [...] My solution is to use __dict__ to get at the data. If there are any other options, I will be interested to hear them. Now that I have read yours and Steve's replies properly, I realise that you were telling me the answer all along. I just have to use vars(...) and it gives me the information I am looking for. Thank you Frank -- https://mail.python.org/mailman/listinfo/python-list
Re: Dunder variables
On Tue, 09 Jan 2018 11:28:03 +0200, Frank Millman wrote: > I have a class call Context containing only data, not methods. Instances > are passed around a lot in my application, with various methods > accessing various attributes. That contradicts itself... your Context class has data, but no methods, and yet it has methods accessing various attributes? If you have a class with only data, and you access the attributes via the instance's __dict__, why not use an ordinary dict? Alternatively, use a SimpleNamespace: py> from types import SimpleNamespace py> bag = SimpleNamespace() py> bag.spam = 1 py> bag.eggs = 2 py> bag namespace(eggs=2, spam=1) > I wanted to allow various parts of my app to 'piggy back' on this by > adding their own attributes at certain points, to be read back at > various other points. [...] > To tidy this up, I changed it to allow other parts of the app to store > attributes directly into Context. To protect my 'core' attributes, I > made them read-only, using @property. This all works quite well. Except it is no longer a class with data and no methods. > Now I have a situation where various processes are 'long-running', and I > need to persist the data to disk so that it can be recovered in the > event of a crash. I want to use Json to store the data as a dictionary. > However, I have no idea which additional attributes have been added by > other parts of the application. Does it have to be JSON? If this is just for your own use, pickle will probably do what you want: py> import pickle py> class Foo: ... pass ... py> x = Foo() py> x.spam = 1 py> s = pickle.dumps(x) py> y = pickle.loads(s) py> y.spam 1 (Warning: pickle is not secure if arbitrary, untrusted users have write- access to the pickle files.) -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: Dunder variables
"Steven D'Aprano" wrote in message news:p32g4v$v88$2...@blaine.gmane.org... On Tue, 09 Jan 2018 11:28:03 +0200, Frank Millman wrote: > I have a class call Context containing only data, not methods. Instances > are passed around a lot in my application, with various methods > accessing various attributes. That contradicts itself... your Context class has data, but no methods, and yet it has methods accessing various attributes? Maybe I was not clear. The Context instance is passed as an argument to many methods. The methods can access the attributes of the instance. The instance has no methods of its own. If you have a class with only data, and you access the attributes via the instance's __dict__, why not use an ordinary dict? I don’t access them *via* the __dict__, I access them via normal object [dot] attribute syntax. Alternatively, use a SimpleNamespace: I did play with that for a while, but AFAIK it does not allow you to define read-only attributes. > I wanted to allow various parts of my app to 'piggy back' on this by > adding their own attributes at certain points, to be read back at > various other points. [...] > To tidy this up, I changed it to allow other parts of the app to store > attributes directly into Context. To protect my 'core' attributes, I > made them read-only, using @property. This all works quite well. Except it is no longer a class with data and no methods. If you are referring to the @property methods, I think this is a bit pedantic. > Now I have a situation where various processes are 'long-running', and I > need to persist the data to disk so that it can be recovered in the > event of a crash. I want to use Json to store the data as a dictionary. > However, I have no idea which additional attributes have been added by > other parts of the application. Does it have to be JSON? If this is just for your own use, pickle will probably do what you want: I thought of that. But some of the attributes are things like database connections and context managers. Are they pickleable? What I do at the moment is make a copy of the attributes, using vars(...).copy(), delete the ones I do not want, and save the rest. Frank -- https://mail.python.org/mailman/listinfo/python-list
Tips or strategies to understanding how CPython works under the hood
Hey all, Been subscribed to this thread for a while but haven't contributed much. One of my ultimate goals this year is to get under the hood of CPython and get a decent understanding of mechanics Guido and the rest of you wonderful people have designed and implemented. I've been programming in python for nearly 10 years now and while I can write a mean Python script, I've been becoming more and more interested in low level operations or complex C programs so I thought I could spread my love of both to make a difference for me and others. So besides just grabbing a chunk of CPython source code and digesting it, I was wondering if those of you have read and understood the source code, do you have any tips or good starting points? Robert -- https://mail.python.org/mailman/listinfo/python-list
Re: Dunder variables
On 01/09/2018 07:30 AM, Steven D'Aprano wrote: > If you have a class with only data, and you access the attributes via the > instance's __dict__, why not use an ordinary dict? Or even subclass dict. class MyClass(dict): VAR = 5 m = MyClass() m['newvar'] = "Something" I do this and wrap things like __getitem__, __call__, __del__, etc. with my own methods. For example... def __getitem__(self, key): # return self.attribute if given "_attribute_" mapping if key[0] == '_' and key[-1] == '_' and key[1] != '_': k = key[1:-1] if hasattr(self, k): return getattr(self, k) return dict.__getitem__(self, key) So, in the above example... >>>print("newvar = '%(newvar)s', VAR = '%(_VAR_)s'" % m newvar = 'Something', VAR = '5' -- D'Arcy J.M. Cain Vybe Networks Inc. http://www.VybeNetworks.com/ IM:da...@vex.net VoIP: sip:da...@vybenetworks.com -- https://mail.python.org/mailman/listinfo/python-list
[RELEASE] Python 3.7.0a4 is now available for testing
Python 3.7.0a4 is the last of four planned alpha releases of Python 3.7, the next feature release of Python. During the alpha phase, Python 3.7 remains under heavy development: additional features will be added and existing features may be modified or deleted. Please keep in mind that this is a preview release and its use is not recommended for production environments. The next preview release, 3.7.0b1, is planned for 2018-01-29. You can find Python 3.7.0a4 and more information here: https://www.python.org/downloads/release/python-370a4/ -- Ned Deily n...@python.org -- [] -- https://mail.python.org/mailman/listinfo/python-list
Re: Tips or strategies to understanding how CPython works under the hood
On Wed, Jan 10, 2018 at 2:21 AM, Robert O'Shea wrote: > Hey all, > > Been subscribed to this thread for a while but haven't contributed much. > One of my ultimate goals this year is to get under the hood of CPython and > get a decent understanding of mechanics Guido and the rest of you wonderful > people have designed and implemented. > > I've been programming in python for nearly 10 years now and while I can > write a mean Python script, I've been becoming more and more interested in > low level operations or complex C programs so I thought I could spread my > love of both to make a difference for me and others. > > So besides just grabbing a chunk of CPython source code and digesting it, I > was wondering if those of you have read and understood the source code, do > you have any tips or good starting points? Cool! Let's see. The first thing I'd do is to explore the CPython byte code. Use the 'dis' module to examine the compiled version of a function, and then look at the source code for dis.py (and the things it imports, like opcode.py) to get a handle on what's happening in that byte code. CPython is a stack-based interpreter, which means it loads values onto an (invisible) internal stack, processes values at the top of the stack, and removes them when it's done. Once you've gotten a handle on the bytecode, I'd next dive into one particular core data type. Pick one of dict, list, str (note that, for hysterical raisins, it's called "unicode" in the source), int (for similar hysterical raisins, it's called "long"), etc. In the CPython source code, there's an Objects/ directory, Explore the functionality of that one object type, keeping in mind the interpreter's stack. Get an understanding of the different things you can do with it at the low level; some of them will be the same as you're used to from the high level, but some won't (for instance, Python code is never aware of a call to list_resize). Especially, read all the comments; the top few pages of dictobject.c are pretty much entirely English, and give a lot of insight into how Python avoids potential pitfalls in dict behaviour. >From there, it's all up to you! Don't hesitate to ask questions about stuff you see. Tinkering is strongly encouraged! Oh, one thing to keep an eye on is error handling. You might discover something because there's code to raise an exception if something happens... like raising ValueError("generator already executing"), which I found highly amusing. (I cannot imagine ANY sane code that would ever trigger that error!) Have fun with it! ChrisA not a CPython source code expert by any means, but has followed the above steps and greatly enjoyed the process -- https://mail.python.org/mailman/listinfo/python-list
Re: Tips or strategies to understanding how CPython works under the hood
On 9 January 2018 at 16:18, Chris Angelico wrote: > On Wed, Jan 10, 2018 at 2:21 AM, Robert O'Shea > wrote: >> Hey all, >> >> Been subscribed to this thread for a while but haven't contributed much. >> One of my ultimate goals this year is to get under the hood of CPython and >> get a decent understanding of mechanics Guido and the rest of you wonderful >> people have designed and implemented. >> >> I've been programming in python for nearly 10 years now and while I can >> write a mean Python script, I've been becoming more and more interested in >> low level operations or complex C programs so I thought I could spread my >> love of both to make a difference for me and others. >> >> So besides just grabbing a chunk of CPython source code and digesting it, I >> was wondering if those of you have read and understood the source code, do >> you have any tips or good starting points? > > Cool! Let's see. > > The first thing I'd do is to explore the CPython byte code. Use the > 'dis' module to examine the compiled version of a function, and then > look at the source code for dis.py (and the things it imports, like > opcode.py) to get a handle on what's happening in that byte code. > CPython is a stack-based interpreter, which means it loads values onto > an (invisible) internal stack, processes values at the top of the > stack, and removes them when it's done. > > Once you've gotten a handle on the bytecode, I'd next dive into one > particular core data type. Pick one of dict, list, str (note that, for > hysterical raisins, it's called "unicode" in the source), int (for > similar hysterical raisins, it's called "long"), etc. In the CPython > source code, there's an Objects/ directory, Explore the functionality > of that one object type, keeping in mind the interpreter's stack. Get > an understanding of the different things you can do with it at the low > level; some of them will be the same as you're used to from the high > level, but some won't (for instance, Python code is never aware of a > call to list_resize). Especially, read all the comments; the top few > pages of dictobject.c are pretty much entirely English, and give a lot > of insight into how Python avoids potential pitfalls in dict > behaviour. > > From there, it's all up to you! Don't hesitate to ask questions about > stuff you see. Tinkering is strongly encouraged! > > Oh, one thing to keep an eye on is error handling. You might discover > something because there's code to raise an exception if something > happens... like raising ValueError("generator already executing"), > which I found highly amusing. (I cannot imagine ANY sane code that > would ever trigger that error!) > > Have fun with it! In addition to Chris' suggestions, it would probably be good to look at the documentation - the "Extending and Embedding" and "Python/C API" manuals, although focused more on people writing C code to interface with Python, nevertheless include a lot of information about how the C code that implements Python works. And a lot of the core data types (as well as anything in the standard library) are written using the same C API as 3rd party extensions use, so the documentation will give you a lot of help with those sections of the code. Paul -- https://mail.python.org/mailman/listinfo/python-list
Re: Tips or strategies to understanding how CPython works under the hood
Chris Angelico wrote: CPython is a stack-based interpreter, which means it loads values onto an (invisible) internal stack, processes values at the top of the stack, and removes them when it's done. Is this similar to how Lua operates too? -- https://mail.python.org/mailman/listinfo/python-list
Re: Tips or strategies to understanding how CPython works under the hood
On Wed, Jan 10, 2018 at 6:20 AM, ElChino wrote: > Chris Angelico wrote: > >> CPython is a stack-based interpreter, which means it loads values onto >> an (invisible) internal stack, processes values at the top of the >> stack, and removes them when it's done. > > > Is this similar to how Lua operates too? I don't know (haven't looked into Lua's implementation at all), but it would be highly likely. Stack-based interpreters are a pest to actually program against, as it's too easy to mess something up, but they're beautifully easy to implement; so it's very common to compile to a stack-based bytecode. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
python server socket file transfer
how much client can i handel whit this code what the amount of client that i can handel the size of the file is 716 kb import socket from threading import Thread from SocketServer import ThreadingMixIn ## TCP_IP = '' TCP_PORT = 3156 BUFFER_SIZE = 1024 class ClientThread(Thread): def __init__(self,ip,port,sock): Thread.__init__(self) self.ip = ip self.port = port self.sock = sock print " New thread started for "+ip+":"+str(port) def run(self): filename='System.exe' f = open(filename,'rb') while True: l = f.read(BUFFER_SIZE) while (l): self.sock.send(l) #print('Sent ',repr(l)) l = f.read(BUFFER_SIZE) if not l: f.close() self.sock.close() break tcpsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) tcpsock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) tcpsock.bind((TCP_IP, TCP_PORT)) threads = [] while True: tcpsock.listen(1) print "Waiting for incoming connections..." (conn, (ip,port)) = tcpsock.accept() print 'Got connection from ', (ip,port) newthread = ClientThread(ip,port,conn) newthread.start() threads.append(newthread) for t in threads: t.join() and the client side is import socket TCP_IP = '' TCP_PORT = 3156 BUFFER_SIZE = 1024 try: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((TCP_IP, TCP_PORT)) with open('Sys.exe', 'wb') as f: print 'file opened' while True: #print('receiving data...') data = s.recv(BUFFER_SIZE) if not data: f.close() print 'file close()' break # write data to a file f.write(data) except: print "problem" print('Successfully get the file') s.close() print('connection closed') -- https://mail.python.org/mailman/listinfo/python-list
Re: Tips or strategies to understanding how CPython works under the hood
ElChino writes: > Chris Angelico wrote: > >> CPython is a stack-based interpreter, which means it loads values onto >> an (invisible) internal stack, processes values at the top of the >> stack, and removes them when it's done. > > Is this similar to how Lua operates too? No. Lua uses a register-based (virtual) machine. See https://www.lua.org/doc/jucs05.pdf I think Lua was the first language in widespread use to move to a register-based machine. -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
CSV file edition
Dear: With what function could you separate the numerical part of each column? MarketTime Price Type Type=0 MarketTime=11:18:26.549 Price=112.8300 Type=0 MarketTime=11:18:28.792 Price=112.8300 Type=0 MarketTime=11:18:28.792 Price=112.8400 Type=0 MarketTime=11:18:28.792 Price=112.8300 Type=0 MarketTime=11:18:45.798 Price=112.8500 Type=0 MarketTime=11:18:45.799 Price=112.8500 Type=0 MarketTime=11:18:45.880 Price=112.8400 I would need to filter only the numeric part of all the columns. -- https://mail.python.org/mailman/listinfo/python-list
Re: CSV file edition
Manuel Rincon writes: [...] > Type=0 MarketTime=11:18:26.549 Price=112.8300 > Type=0 MarketTime=11:18:28.792 Price=112.8300 [...] > > I would need to filter only the numeric part of all the columns. I assume that by "numeric" you mean the value after Price= line.split()[2].split('=')[1] line.rsplit('=',1)[1] line.rpartition('=')[2] line[line.rfind('=')+1:] I prefer the last one. You can also use regexps, but it would be overkill if your lines are that simple. If your lines are really that simple, use line[39:] (check that 39 is correct). Your data look like fixed-width fields. -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: Tips or strategies to understanding how CPython works under the hood
On 09-Jan 15:21, Robert O'Shea wrote: > > Been subscribed to this thread for a while but haven't contributed much. +1. I'm a lurker too. > So besides just grabbing a chunk of CPython source code and digesting it, I > was wondering if those of you have read and understood the source code, do > you have any tips or good starting points? > There are a ton of videos on Youtube that talk about Python internals. I liked https://www.youtube.com/playlist?list=PLzV58Zm8FuBL6OAv1Yu6AwXZrnsFbbR0S quite a bit. Even though I knew a good portion of the material, there was still a lot of new stuff. -- david stanek web: https://dstanek.com twitter: https://twitter.com/dstanek -- https://mail.python.org/mailman/listinfo/python-list
Re: Tips or strategies to understanding how CPython works under the hood
On Tuesday, January 9, 2018 at 3:22:30 PM UTC, Robert O'Shea wrote: > Hey all, > > Been subscribed to this thread for a while but haven't contributed much. > One of my ultimate goals this year is to get under the hood of CPython and > get a decent understanding of mechanics Guido and the rest of you wonderful > people have designed and implemented. > > I've been programming in python for nearly 10 years now and while I can > write a mean Python script, I've been becoming more and more interested in > low level operations or complex C programs so I thought I could spread my > love of both to make a difference for me and others. > > So besides just grabbing a chunk of CPython source code and digesting it, I > was wondering if those of you have read and understood the source code, do > you have any tips or good starting points? > > Robert Hopefully you'll find this http://pgbovine.net/cpython-internals.htm of interest. -- Kindest regards. Mark Lawrence. -- https://mail.python.org/mailman/listinfo/python-list
Re: How to create "transitional" package?
On 09/01/18 05:09, INADA Naoki wrote: > Hi, all. > > Yesterday, I released msgpack-0.5, which was msgpack-python. > Both packages provide "msgpack" python package. > > I used msgpack in early days, but easy_install crawling website > and download msgpack-1.0.0.tar.gz, which is msgpack for C instead > of Python package I upload to PyPI. > So I renamed msgpack-python but I really dislike it. > > Now pip doesn't such silly crawling so I decided to rename back > to msgpack. > To ease transition, I follow what uritemplate.py [1] did (which was > migrated to uritemplate). > > I released msgpack-python 0.5. It is empty package. > But it's metadata contains `install_requires=["msgpack>=0.5"]`. > > Sadly, this breaks upgrading from old version via `pip install -U > msgpack-python`. > It does: > > * Install msgpack-0.5 (overwrite msgpack-python 0.4.7) > * Uninstall msgpack-python 0.4.7 (removes msgapck 0.5!) > * Install msgpack-python 0.5 (empty!) Here's a thought: if you distribute a source package for msgpack-python-0.5, your setup.py is run during installation; you can do whatever you want. You could check, at msgpack-python installation time, whether msgpack is still installed correctly - and if it's not, reinstall. Doing that correctly (and without somehow confusing or angering pip) might, of course, be easier said than done. > > I found uritemplate.py has same issue. Maybe pip's behavior was > changed after migration of uritemplate.py to uritemplate. > > Now what can I do for smooth transition? > I don't want to back to msgpack-python again. > > [1] https://pypi.python.org/pypi/uritemplate.py > > Regards, > > INADA Naoki > -- https://mail.python.org/mailman/listinfo/python-list
Re: Dunder variables
On Tue, 09 Jan 2018 16:14:27 +0200, Frank Millman wrote: > Maybe I was not clear. The Context instance is passed as an argument to > many methods. The methods can access the attributes of the instance. The > instance has no methods of its own. Ah, I see, I misunderstood. [...] >> Alternatively, use a SimpleNamespace: > > I did play with that for a while, but AFAIK it does not allow you to > define read-only attributes. Not directly, no, but you can subclass: py> class Spam(SimpleNamespace): ... @property ... def eggs(self): ... return "foo" ... py> bag = Spam() py> bag.eggs 'foo' py> bag.eggs = 1 Traceback (most recent call last): File "", line 1, in AttributeError: can't set attribute However, at the point you are subclassing and adding multiple properties, I'm not sure there's any advantage over just defining a class from scratch. While it is true that SimpleNamespace gives you a nice repr, the properties don't show up in that repr, and I assume it is those read-only attributes that you most care about. -- Steve -- https://mail.python.org/mailman/listinfo/python-list
pyplot: change the number of x labels (from 6)
Hi all. I want to have dates as major ticks labels of X axis. This fragment of code works fine except that I need more dates to appear instead the 6 I am getting. The number of dates in dtsd is for ex. 262. Thanks for any help. BTW, I got most of this code from some site on the internet. ... fig=plt.figure(figsize=(12,9)) gs=gridspec.GridSpec(2,1,height_ratios=[4,1]) ... ax0=plt.subplot(gs[0]) ... plt.xlim(-0.1,dts[-1]+0.1) dtsd=pd.to_datetime(ds.getIndexes()) def format_date(x,__pos=None): thisind=np.clip(int(x+0.5),0,dtslen-1) return dtsd[thisind].strftime('%Y-%m-%d') ax0.xaxis.set_major_formatter(ticker.FuncFormatter(format_date)) fig.autofmt_xdate() ... -- https://mail.python.org/mailman/listinfo/python-list
Re: python server socket file transfer
bingbong3...@gmail.com writes: > how much client can i handel whit this code what the amount of client that i > can handel > the size of the file is 716 kb > ... > self.sock.send(l) Please read the documentation for *send* in the "socket" module: it tells you that "send" (in contrast to "sendall") is *not* guarantied to send the complete *l* (there is no guarantee how much is sent); the return value of "send" tells you how many bytes have been sent. -- https://mail.python.org/mailman/listinfo/python-list