Guppy-PE/Heapy 0.1.11
I am happy to announce Guppy-PE 0.1.11 Guppy-PE is a library and programming environment for Python, currently providing in particular the Heapy subsystem, which supports object and heap memory sizing, profiling and debugging. It also includes a prototypical specification language, the Guppy Specification Language (GSL), which can be used to formally specify aspects of Python programs and generate tests and documentation from a common source. The main news in this release: Contains notes and check that this package is only for Python2. A version for Python3 is recommended and is available here: https://github.com/zhuyifei1999/guppy3 License: MIT The project homepage is on GitHub: https://svenil.github.io/guppy-pe/ Enjoy, Sverker Nilsson -- Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Support the Python Software Foundation: http://www.python.org/psf/donations/
Guppy-PE/Heapy 0.1.10
I am happy to announce Guppy-PE 0.1.10 Guppy-PE is a library and programming environment for Python, currently providing in particular the Heapy subsystem, which supports object and heap memory sizing, profiling and debugging. It also includes a prototypical specification language, the Guppy Specification Language (GSL), which can be used to formally specify aspects of Python programs and generate tests and documentation from a common source. The main news in this release: o Support for Python 2.7 However, there are still some problems with the tests that I haven't been able to sort out. o Some minor bug fixes License: MIT Guppy-PE 0.1.10 is available in source tarball format on the Python Package Index (a.k.a. Cheeseshop): http://pypi.python.org/pypi/guppy/0.1.10 The project homepage is on Sourceforge: http://guppy-pe.sourceforge.net Enjoy, Sverker Nilsson -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Guppy-PE/Heapy 0.1.10
I am happy to announce Guppy-PE 0.1.10 Guppy-PE is a library and programming environment for Python, currently providing in particular the Heapy subsystem, which supports object and heap memory sizing, profiling and debugging. It also includes a prototypical specification language, the Guppy Specification Language (GSL), which can be used to formally specify aspects of Python programs and generate tests and documentation from a common source. The main news in this release: o Support for Python 2.7 However, there are still some problems with the tests that I haven't been able to sort out. o Some minor bug fixes License: MIT Guppy-PE 0.1.10 is available in source tarball format on the Python Package Index (a.k.a. Cheeseshop): http://pypi.python.org/pypi/guppy/0.1.10 The project homepage is on Sourceforge: http://guppy-pe.sourceforge.net Enjoy, Sverker Nilsson-- http://mail.python.org/mailman/listinfo/python-list
Python assignments
Dear friends, This is Sverker from Sweden. You probably know me better as the guy who made Guppy/Heapy: http://guppy-pe.sf.net I am currently in the process of preparing version 0.1.10 with support for Python 2.7. I will let you know when it is updated. For those who don’t know, I work as a consultant and I am looking for new assignments and I really wish to work with Heapy/Python instead of regular programming in eg Java or C++. Therefore I was wondering if any of you know any companies in the States or anywhere else that need my services. Further information about who I am and what I can do you can find on my website: http://sncs.se I appreciate all advice and information you can provide me with. So long my friends, Sverker -- http://mail.python.org/mailman/listinfo/python-list
Re: [Guppy-pe-list] An iteration idiom (Was: Re: loading files containing multiple dumps)
On Wed, 2009-09-09 at 13:47 +0100, Chris Withers wrote: Sverker Nilsson wrote: As the enclosing class or frame is deallocated, so is its attribute h itself. Right, but as long as the h hangs around, it hangs on to all the memory it's used to build its stats, right? This caused me problems in my most recent use of guppy... If you just use heap(), and only want total memory not relative to a reference point, you can just use hpy() directly. So rather than: CASE 1: h=hpy() h.heap().dump(...) #other code, the data internal to h is still around h.heap().dump(...) you'd do: CASE 2: hpy().heap().dump(...) #other code. No data from Heapy is hanging around hpy().heap().dump(...) The difference is that in case 1, the second call to heap() could reuse the internal data in h, whereas in case 2, it would have to be recreated which would take longer time. (The data would be such things as the dictionary owner map.) However, if you measure memory relative to a reference point, you would have to keep h around, as in case 1. [snip] Do you mean we should actually _remove_ features to create a new standalone system? Absolutely, why provide more than is used or needed? How should we understand this? Should we have to support 2 or more systems depending on what functionality you happen to need? Or do you mean most functionality is actually _never_ used by _anybody_ (and will not be in the future)? That would be quite gross wouldn't it. I'd be hard pressed to support several versions just for the sake of some of them would have only the most common methods used in certain situations. That's would be like to create an additional Python dialect that contained say only the 10 % functionality that is used 90 % of the time. Quite naturally this is not done anytime soon. Even though one could perhaps argue it would be easier to use for children etc, the extra work to support this has not been deemed meaningful. You are free to wrap functions as you find suitable; a minimal wrapper module could be just like this: # Module heapyheap from guppy import hpy h=hpy() heap=heap() I don't follow this.. did you mean heap = h.heap()? Actually I meant heap=h.heap If so, isn't that using all the gubbinz in Use, etc, anyway? Depends on what you mean with 'using', but I would say no. Less minor rant: this applies to most things to do with heapy... Having __repr__ return the same as __str__ and having that be a long lump of text is rather annoying. If you really must, make __str__ return the big lump of text but have __repr__ return a simple, short, item containing the class, the id, and maybe the number of contained objects... I thought it was cool to not have to use print but get the result directly at the prompt. That's fine, that's what __str__ is for. __repr__ should be short. No, it's the other way around: __repr__ is used when evaluating directly at the prompt. The docs give the idea: http://docs.python.org/reference/datamodel.html?highlight=__repr__#object.__repr__ I believe you big strings would be classed as informal and so would be computed by __str__. Informal or not, they contain the information I thought was most useful and are created by __str__, but also with __repr__ because that is used when evaluated at the prompt. According to the doc you linked to above, __repr__ should preferably be a Python expression that could be used to recreate it. I think this has been discussed and criticized before and in general there is no way to create such an expression. For example, for the result of h.heap(), there is no expression that can recreate it later (since the heap changes) and the object returned is just an IdentitySet, which doesn't know how it was created. It also gives as an alternative, If this is not possible, a string of the form ...some useful description... should be returned The __repr__ I use don't have the enclosing , granted, maybe I missed this or it wasn't in the docs in 2005 or I didn't think it was important (still don't) but was that really what the complain was about? The docs also say that it is important that the representation is information-rich and unambiguous. I thought it was more useful to actually get information of what was contained in the object directly at the prompt, than try to show how to recreate it which wasn't possible anyway. [snip] The index (__getitem__) method was available so I used it to take the subset of the i'ths row in the partition defined by its equivalence order. That should have another name... I don't know what a partition or equivalence order are in the contexts you're using them, but I do know that hijacking __getitem__ for this is wrong. Opinions may differ, I'd say one can in principle never 'know' if such a thing is 'right' or 'wrong', but that gets us into philosophical territory. Anyway... To get a tutorial provided by someone who did not seem to share your
Re: [Guppy-pe-list] An iteration idiom (Was: Re: loading files containing multiple dumps)
On Mon, 2009-09-07 at 16:53 +0100, Chris Withers wrote: Sverker Nilsson wrote: I hope the new loadall method as I wrote about before will resolve this. def loadall(self,f): ''' Generates all objects from an open file f or a file named f''' if isinstance(f,basestring): f=open(f) while True: yield self.load(f) It would be great if load either returned just one result ever, or properly implemented the iterator protocol, rather than half implementing it... Agreed, this is arguably a bug or at least a misfeature, as also Raymond Hettinger remarked, it is not normal for a normal function to raise StopIteration. But I don't think I would want to risk breaking someone's code just for this when we could just add a new method. Should we call it loadall? It is a generator so it doesn't really load all immedietally, just lazily. Maybe call it iload? Or redefine load, but that might break existing code so would not be good. loadall works for me, iload doesn't. Or we could have an option to hpy() to redefine load() as loadall(), but I think it is cleaner (and easier) to just define a new method... Settled then? :-) Minor rant, why do I have to instantiate a class 'guppy.heapy.Use._GLUECLAMP_' to do anything with heapy? Why doesn't heapy just expose load, dump, etc? Basically, the need for the h=hpy() idiom is to avoid any global variables. Eh? What's h then? (And h will reference whatever globals you were worried about, surely?) h is what you make it to be in the context you create it; you can make it either a global variable, a local variable, or an object attribute. Interactively, I guess one tends to have it as a global variable, yes. But it is a global variable you created and responds for yourself, and there are no other global variables behind the scene but the ones you create yourself (also possibly the results of heap() etc as you store them in your environment). If making test programs, I would not use global variables but instead would tend to have h as a class attribute in a test class, eg as in UnitTest. It could also be a local variable in a test function. As the enclosing class or frame is deallocated, so is its attribute h itself. There should be nothing that stays allocated in other modules after one test (class) is done (other than some loaded modules themselves, but I am talking about more severe data that can be hundreds of megabytes or more). Heapy uses some rather big internal data structures, to cache such things as dict ownership. I didn't want to have all those things in global variables. What about attributes of a class instance of some sort then? They are already attributes of an instance: hpy() is a convenience factory method that creates a top level instance for this purpose. the other objects you created. Also, it allows for several parallel invocations of Heapy. When is that helpful? For example, the setref() method sets a reference point somewhere in h. Further calls to heap() would report only objects allocated after that call. But you could use a new hpy() instance to see all objects again. Multiple threads come to mind, where each thread would have its own hpy() object. (Thread safety may still be a problem but at least it should be improved by not sharing the hpy() structures.) Even in the absence of multiple threads, you might have an outer invocation of hpy() that is used for global analysis, with its specific options, setref()'s etc, and inner invocations that make some local analysis perhaps in a single method. However, I am aware of the extra initial overhead to do h=hpy(). I discussed this in my thesis. Section 4.7.8 Why not importing Use directly? page 36, http://guppy-pe.sourceforge.net/heapy-thesis.pdf I'm afraid, while I'd love to, I don't have the time to read a thesis... But it is (an important) part of the documentation. For example it contains the rationale and an introduction to the main categories such as Sets, Kinds and EquivalenceRelations, and some usecases for example how to seal a memory leak in a windowing program. I'm afraid, while I'd love to, I don't have the time to duplicate the thesis here...;-) Try sunglasses:) (Well, I am aware of this, it was a research/experimental system and could have some refactoring :-) I would suggest creating a minimal system that allows you to do heap() and then let other people build what they need from there. Simple is *always* better... Do you mean we should actually _remove_ features to create a new standalone system? I don't think that'd be meaningful. You don't need to use anything else than heap() if you don't want to. You are free to wrap functions as you find suitable; a minimal wrapper module could be just like this: # Module heapyheap from guppy import hpy h=hpy() heap=heap() Should we add some such module? In the thesis I discussed this already and argued
Guppy-PE/Heapy 0.1.9 released
I am happy to announce Guppy-PE 0.1.9 Guppy-PE is a library and programming environment for Python, currently providing in particular the Heapy subsystem, which supports object and heap memory sizing, profiling and debugging. It also includes a prototypical specification language, the Guppy Specification Language (GSL), which can be used to formally specify aspects of Python programs and generate tests and documentation from a common source. The main news in this release: o A patch by Chad Austin to compile with Visual Studio 2003 and Python 2.5 C compilers. I think this may fix similar problems with other Microsoft compilers. o Interactive help system, providing a .doc attribute that can be used to get info on available attributes in text form and also by invoking a browser. o New documentation file, heapy_tutorial.html. It contains for now an example of how to get started and use the interactive help. o Bug fix wrt pop for mutable bitset. (Not used in heapy, only in sets/test.py) o guppy.hpy().test() now includes the set-specific tests. o Remote monitor does not longer require readline library being installed. License: MIT Guppy-PE 0.1.9 is available in source tarball format on the Python Package Index (a.k.a. Cheeseshop): http://pypi.python.org/pypi/guppy/0.1.9 The project homepage is on Sourceforge: http://guppy-pe.sourceforge.net Enjoy, Sverker Nilsson Expertise in Linux, embedded systems, image processing, C, Python... http://sncs.se -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Guppy-PE/Heapy 0.1.9 released
I am happy to announce Guppy-PE 0.1.9 Guppy-PE is a library and programming environment for Python, currently providing in particular the Heapy subsystem, which supports object and heap memory sizing, profiling and debugging. It also includes a prototypical specification language, the Guppy Specification Language (GSL), which can be used to formally specify aspects of Python programs and generate tests and documentation from a common source. The main news in this release: o A patch by Chad Austin to compile with Visual Studio 2003 and Python 2.5 C compilers. I think this may fix similar problems with other Microsoft compilers. o Interactive help system, providing a .doc attribute that can be used to get info on available attributes in text form and also by invoking a browser. o New documentation file, heapy_tutorial.html. It contains for now an example of how to get started and use the interactive help. o Bug fix wrt pop for mutable bitset. (Not used in heapy, only in sets/test.py) o guppy.hpy().test() now includes the set-specific tests. o Remote monitor does not longer require readline library being installed. License: MIT Guppy-PE 0.1.9 is available in source tarball format on the Python Package Index (a.k.a. Cheeseshop): http://pypi.python.org/pypi/guppy/0.1.9 The project homepage is on Sourceforge: http://guppy-pe.sourceforge.net Enjoy, Sverker Nilsson Expertise in Linux, embedded systems, image processing, C, Python... http://sncs.se -- http://mail.python.org/mailman/listinfo/python-list
How to print a sorted list as a multi-column table
Hi all, I would like to ask about opinions about the best way to format sorted tables of items for interactive use. I have begun to add interactive help to Guppy/Heapy (http://guppy-pe.sourceforge.net) because it lacks the usual ways for introspection (see for example http://www.pkgcore.org/trac/pkgcore/doc/dev-notes/heapy.rst) which is due to Heapy's use of dynamic things like __getattr__ and to some extent properties. There seems to be no (good or otherwise) way to hook into the standard help() (?) (At least not before 2.6 where I recall seeing a __dir__ magic method may be introduced, but I want it to be backwards compatible to at least 2.4, preferably to 2.3.) So I am about to roll my own help system. It will be based on having the API objects supplying a standardized '.help' attribute which can be used interactively, and will limit the output it prints to say 15 or 20 rows at a time. That's for a background. Now what I was stumbling on which pumped up enough adrenalin to make me begin to write this --- Why are tables formatted like the following, when sorted? (Both in linux eg ls, ftp help, and in Python help() when listing (eg) modules)) (1) a g m s b h n t c i o u d j p v e k q f l r Wouldn't it be more natural to just sort them like this: (2) a b c d e f g h i j k l m n o p q r s t u v What's the rationale for the choice of (1)? In a pager, if you want to limit the number of lines output at a time, then yo'd see with (1) (if artifically limiting output to 2 lines): a g m s b h n t So to see the f item you would have to scroll down all the way. The number of times you would have to scroll down is in completely unrelated to the item's position in sort order. That seems to defeat the purpose of sorting in the first place. It feels strange, to me at least. Anybody had the same feeling? Well, what's the rationale (if any) for using the layout (1)? Wouldn't layout (2) be better? And/or would it be confusing / non-pythonic / non-unlixonic if a program used the (2) layout instead? Have a (:nice:) weekend :-) Sverker -- http://mail.python.org/mailman/listinfo/python-list
Re: py3k s***s
On Apr 17, 12:02 am, Carl Banks [EMAIL PROTECTED] wrote: On Apr 16, 12:40 pm, Aaron Watters [EMAIL PROTECTED] wrote: On Apr 16, 12:27 pm, Rhamphoryncus [EMAIL PROTECTED] wrote: On Apr 16, 6:56 am, Aaron Watters [EMAIL PROTECTED] wrote: I don't get it. It ain't broke. Don't fix it. So how would you have done the old-style class to new-style class transition? I'd ignore it. I never understood it and never had any need for it anyway. New-style classes and metaclasses were a complicated solution to an unimportant problem in my opinion. And also a fiendish way to make code inscrutible -- which I thought was more of a Perl thing than a Python thing, or should be. I must be missing some of the deeper issues here. Please educate me. The deeper issue is that you're benefiting from these unimportant changes even if you never use them yourself. Carl Banks That just seems a BIT categorical for a statement. Who is 'you'? I don't see I benefit from any important or unimportant features in py3k. External libraries I rely on, I can benefit from --- But it would take SOME while to get those libraries ported to py3k, if ever. And I have been benefiting from Python in general, so far. Thanks, community. But now... I'll probably stop posting here for now, I may stop other things too. Just my 2c. Sverker -- http://mail.python.org/mailman/listinfo/python-list
Re: py3k s***s
On Apr 17, 12:02 am, Carl Banks [EMAIL PROTECTED] wrote: On Apr 16, 12:40 pm, Aaron Watters [EMAIL PROTECTED] wrote: On Apr 16, 12:27 pm, Rhamphoryncus [EMAIL PROTECTED] wrote: On Apr 16, 6:56 am, Aaron Watters [EMAIL PROTECTED] wrote: I don't get it. It ain't broke. Don't fix it. So how would you have done the old-style class to new-style class transition? I'd ignore it. I never understood it and never had any need for it anyway. New-style classes and metaclasses were a complicated solution to an unimportant problem in my opinion. And also a fiendish way to make code inscrutible -- which I thought was more of a Perl thing than a Python thing, or should be. I must be missing some of the deeper issues here. Please educate me. The deeper issue is that you're benefiting from these unimportant changes even if you never use them yourself. Carl Banks That just seems a BIT categorical for a statement. Who is 'you'? I don't see I benefit from any important or unimportant features in py3k. External libraries I rely on, I can benefit from --- But it would take SOME while to get those libraries ported to py3k, if ever. And I have been benefiting from Python in general, so far. Thanks, community. But now... I'll probably stop posting here for now, I may stop other things too. Just my 2c. Sverker -- http://mail.python.org/mailman/listinfo/python-list
Re: py3k s***s
On Apr 18, 12:59 am, Diez B. Roggisch [EMAIL PROTECTED] wrote: And I have been benefiting from Python in general, so far. Thanks, community. But now... I'll probably stop posting here for now, I may stop other things too. Just my 2c. You know what I was just wondering about? All these C-written cross-platform libraries (which Python users benefit from, most probably including evven you) that run on different unixes windows, which are a much greater diversity to handle than the not-even-yet-settled differences between Py3K 2.x. How the heck do they do that? Oh, and these dreaded 64 bit processors, possibly with multi-cores, which need changes in code as well, to be utilized to their power. But then, these guys most probably don't whine about diversity and constant change, and cry out useless threats to people who probably can't care less. Fare well, if you must. But getting mad over something which impact you can't even judge right now is childish. Nothing else. Diez Some whine. Some just don't care. Why not whine? -- http://mail.python.org/mailman/listinfo/python-list
Re: py3k s***s
On Apr 18, 1:27 am, John Machin [EMAIL PROTECTED] wrote: Diez B. Roggisch wrote: And I have been benefiting from Python in general, so far. Thanks, community. But now... I'll probably stop posting here for now, I may stop other things too. Just my 2c. You know what I was just wondering about? All these C-written cross-platform libraries (which Python users benefit from, most probably including evven you) that run on different unixes windows, which are a much greater diversity to handle than the not-even-yet-settled differences between Py3K 2.x. How the heck do they do that? Oh, and these dreaded 64 bit processors, possibly with multi-cores, which need changes in code as well, to be utilized to their power. But then, these guys most probably don't whine about diversity and constant change, and cry out useless threats to people who probably can't care less. Fare well, if you must. But getting mad over something which impact you can't even judge right now is childish. Nothing else. At the start of this thread I was pondering the possible meaning of the verb to sverk. Thanks for the exposition, Diez. Yah, and I am getting all the more depressed. Thanks? -- http://mail.python.org/mailman/listinfo/python-list
Re: py3k s***s
Thanks for your well-formulated article Providing the Python infrastructure with my program doesn't apply since I am providing a program/library that is intended to be general. So it doesn't help. All that py3k does to me, it seems, is some extra work. To be frank, no innovation. Just changes, no progress. And yes, I am pd. Somebody compared it with MS stuff. Yes. Nothing personal, I appreciate Your comment. And all others. Sverker On Apr 15, 7:09 pm, Donn Cave [EMAIL PROTECTED] wrote: In article [EMAIL PROTECTED], Sverker Nilsson [EMAIL PROTECTED] wrote: No one forces me, but sooner or later they will want a Python 3.0 and then a 3.1 whatever. I don't want that fuzz. As about the C versions, I am not that worried. What's your point? I just like want to write a program that will stay working. And maybe I can go on with something else hopefully than just compatibility fixes. They take some work afterall. It seems hard with Python. Esp. 2 - 3 Welcome to the world of Python. There was a period of relative stability in the '90s, culminating with version 1.5.2, which just happens to be about the time that people started taking Python seriously. It turns out that this stability was only due to lack of resources, though. I think most of us are working under two imperatives here that really boil down to the same thing: we want a programming language that lets us focus on the goal of the program. On one hand, that means things like automatic memory management that are brought to us by newer languages (i.e., less than 30 years old.) On the other hand it means stability that allows our deployed code to go on working without constant intervention, which usually we find in languages that have become utterly boring and out of fashion (i.e., more than 30 years old.) It's hard to find a good compromise between these two, in an interpreted language. I don't know what the current party line may be on this matter, but some years back it was that you should consider the interpreter part of your application. That is, each application should deploy with its own dedicated Python interpreter, complete with libraries and everything. This naturally relieves some of the maintenance issues, since at least you can upgrade on your own schedule, but of course it has its costs too. Anyone who might be thinking about using Python for an application should seriously think about this. Donn Cave, [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
py3k s***s
do i dare to open a thread about this? come on you braver men we are at least not bought by g***le but why? others have said it so many times i think :- but why? a few syntactic 'cleanups' for the cost of a huge rewrite of all the code that have been builtup from all the beginning when the once great Python came along and people began to use it and write code for it. Like all that code would have to be rewritten. blaah. and i have perhaps been drinking but i have been p**d all week since i began look into this:-( -- http://mail.python.org/mailman/listinfo/python-list
Re: py3k s***s
On Apr 15, 1:34 am, Steve Holden [EMAIL PROTECTED] wrote: Sverker Nilsson wrote: do i dare to open a thread about this? come on you braver men we are at least not bought by g***le but why? others have said it so many times i think :- but why? a few syntactic 'cleanups' for the cost of a huge rewrite of I have sobered up abit, it doesnt matter much I tried out py3k on my project, http://guppy-pe.sf.net And i have looked into py3k also at the list and read quite a bit about it. no fun, to me at least Try it yourself, all contributions are welcome of course Sverker all the code that have been builtup from all the beginning when the once great Python came along and people began to use it and write code for it. Like all that code would have to be rewritten. blaah. and i have perhaps been drinking but i have been p**d all week since i began look into this:-( Perhaps you should sober up and look at the reality of Python 3, which has deliberately avoided a complete rewrite. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: py3k s***s
On Apr 15, 2:58 am, ajaksu [EMAIL PROTECTED] wrote: On Apr 14, 8:10 pm, Sverker Nilsson [EMAIL PROTECTED] wrote: do i dare to open a thread about this? Yeah, you sure do! come on you braver men Yeah! we are at least not bought by g***le Hell no! but why? others have said it so many times i think Huh?! :- ?! Whatever! but why? a few syntactic 'cleanups' for the cost of a huge rewrite of all the code that have been builtup from all the beginning when the once great Python came along and people began to use it and write code for it. Like all that code would have to be rewritten. blaah. Yeah! Woo-hoo! Wait... What? No, no, you got it all wrong. Python developers are being extra-careful and doing a lot of hard work to keep things sane, allow easy migration, etc. and i have perhaps been drinking but i have been p**d all week since i began look into this:-( Ah, OK, calm down and look again. Things are way better than you think, but there is a lot of FUD going on too, so focus on serious, reliable reports. Cheers, Daniel I was serious! It's just from my right brain half. I take it you are ironic, I appreciate it, hope we will hear from some Director. doesnt matter perhapss I said what I said, now I am just trollin or trying How do one troll best? I know but I can also sell you Rolexes for a good price... S-- And what should I say? Perhaps that the debate on the the py3k list is I cant find the word because i am swedish, but it is just that most people or some are so concerned about they having 'commit- privileges' that the don't dare say anything contoversial that would go against mr. GVR. so this is a much freer forum.Not that I am more intelligent than GVR but we can at least debate, though I doubt he has the time:-) So, regrds to all and hope you guys want to continut the debate... Sverker -- http://mail.python.org/mailman/listinfo/python-list
Re: py3k s***s
On Apr 15, 2:58 am, ajaksu [EMAIL PROTECTED] wrote: On Apr 14, 8:10 pm, Sverker Nilsson [EMAIL PROTECTED] wrote: do i dare to open a thread about this? Yeah, you sure do! come on you braver men Yeah! we are at least not bought by g***le Hell no! but why? others have said it so many times i think Huh?! :- ?! Whatever! but why? a few syntactic 'cleanups' for the cost of a huge rewrite of all the code that have been builtup from all the beginning when the once great Python came along and people began to use it and write code for it. Like all that code would have to be rewritten. blaah. Yeah! Woo-hoo! Wait... What? No, no, you got it all wrong. Python developers are being extra-careful and doing a lot of hard work to keep things sane, allow easy migration, etc. and i have perhaps been drinking but i have been p**d all week since i began look into this:-( Ah, OK, calm down and look again. Things are way better than you think, but there is a lot of FUD going on too, so focus on serious, reliable reports. Cheers, Daniel What serious reports? -- http://mail.python.org/mailman/listinfo/python-list
Re: py3k s***s
On Apr 15, 3:50 am, Gabriel Genellina [EMAIL PROTECTED] wrote: En Mon, 14 Apr 2008 22:02:38 -0300, Sverker Nilsson [EMAIL PROTECTED] escribió: I tried out py3k on my project,http://guppy-pe.sf.net And what happened? I've seen that your project already supports Python 2.6 so the migration path to 3.0 should be easy. -- Gabriel Genellina 2.6 was no big deal, It was an annoyance that they had to make 'as' a reserved word. Annoyances were also with 2.4, and 2.5. No big problems, I could make guppy backwards compatible to 2.3. But that seems not to be possible with Python 3.x ... it is a MUCH bigger change. And it would require a fork of the code bases, in C, Guido has written tha or to sprinkle with #ifdefs. Would not happen soon for me. It takes some work anyways. Do you volunteer, Guido van Rossum? :-) It's not exactly easy. Perhaps not very hard anyways. But think of 1000's of such projects. How many do you think there are? I think many. How many do yo think care? I think few. When it has been the fuzz with versions before, then I could have the same code still work with older versions. But now it seems I have to fork TWO codes. It's becoming too much. Think of the time you could write a program in C or even C++ and then it'll work. How do you think eg writers of bash or other unix utilities come along. Do they have to rewrite their code each year? No, it stays. And they can be happy about that, and go on to other things. Why should I have to think about staying compatible with the newest fancy Python all the time? NO -- but the answer may be, they don't care, though the others (C/C++, as they rely on) do. :-( Sverker -- http://mail.python.org/mailman/listinfo/python-list
Re: py3k s***s
No one forces me, but sooner or later they will want a Python 3.0 and then a 3.1 whatever. I don't want that fuzz. As about the C versions, I am not that worried. What's your point? I just like want to write a program that will stay working. And maybe I can go on with something else hopefully than just compatibility fixes. They take some work afterall. It seems hard with Python. Esp. 2 - 3 Sverker On Apr 15, 5:41 am, Gabriel Genellina [EMAIL PROTECTED] wrote: En Mon, 14 Apr 2008 23:38:56 -0300, Sverker Nilsson [EMAIL PROTECTED] escribió: On Apr 15, 3:50 am, Gabriel Genellina [EMAIL PROTECTED] wrote: En Mon, 14 Apr 2008 22:02:38 -0300, Sverker Nilsson [EMAIL PROTECTED] escribió: I tried out py3k on my project,http://guppy-pe.sf.net And what happened? I've seen that your project already supports Python 2.6 so the migration path to 3.0 should be easy. 2.6 was no big deal, It was an annoyance that they had to make 'as' a reserved word. Annoyances were also with 2.4, and 2.5. No big problems, I could make guppy backwards compatible to 2.3. But that seems not to be possible with Python 3.x ... it is a MUCH bigger change. And it would require a fork of the code bases, in C, Guido has written tha or to sprinkle with #ifdefs. Would not happen soon for me. It takes some work anyways. Do you volunteer, Guido van Rossum? :-) It's not exactly easy. Perhaps not very hard anyways. But think of 1000's of such projects. How many do you think there are? I think many. How many do yo think care? I think few. When it has been the fuzz with versions before, then I could have the same code still work with older versions. But now it seems I have to fork TWO codes. It's becoming too much. Think of the time you could write a program in C or even C++ and then it'll work. How do you think eg writers of bash or other unix utilities come along. Do they have to rewrite their code each year? No, it stays. And they can be happy about that, and go on to other things. Why should I have to think about staying compatible with the newest fancy Python all the time? NO -- but the answer may be, they don't care, though the others (C/C++, as they rely on) do. :-( You can stay with Python 2.6 and not support 3.0; nobody will force you to use it. And nobody will come and wipe out your Python installation, be it 2.6, 2.1 or whatever. And if you still enjoy using Python 1.5, please keep using it - it won't disappear the day after 3.0 becomes available. Regarding the C language: yes, souce code *had* to be modified for newer versions of the language and/or compiler. See by example, the new restrict keyword in C99, or the boolean names. The C guys are much more concerned about backwards compatibility than Python, but they can't guarantee that (at risk of freezing the language). The 3.0 incompatibilities are all justified, anyway, and Python is changing (as a language) much more than C - and that's a good thing. There is a strategy to migrate from 2.x to 3.0, including the 2to3 tool. Have you used it? -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Guppy-PE / Heapy 0.1.8
I am happy to announce Guppy-PE 0.1.8 Guppy-PE is a library and programming environment for Python, currently providing in particular the Heapy subsystem, which supports object and heap memory sizing, profiling and debugging. It also includes a prototypical specification language, the Guppy Specification Language (GSL), which can be used to formally specify aspects of Python programs and generate tests and documentation from a common source. The current version is updated to work in 64-bit mode and with Python-2.6, and still works in 32-bit mode and with Python 2.3, 2.4 and 2.5. No other major changes have been made from Guppy 0.1.6. In the nearest future, I plan to add interactive help and more examples to the documentation, perhaps a tutorial. Look out for this in the svn HEAD according to instructions at the project home page, if you want the latest version. I will try to make a new release in perhaps a month or so. License: MIT Guppy-PE 0.1.8 is available in source tarball format on the Python Package Index (a.k.a. Cheeseshop): http://pypi.python.org/pypi/guppy/0.1.8 The project homepage is on Sourceforge: http://guppy-pe.sourceforge.net Enjoy, Sverker -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
Guppy-PE / Heapy 0.1.8
I am happy to announce Guppy-PE 0.1.8 Guppy-PE is a library and programming environment for Python, currently providing in particular the Heapy subsystem, which supports object and heap memory sizing, profiling and debugging. It also includes a prototypical specification language, the Guppy Specification Language (GSL), which can be used to formally specify aspects of Python programs and generate tests and documentation from a common source. The current version is updated to work in 64-bit mode and with Python-2.6, and still works in 32-bit mode and with Python 2.3, 2.4 and 2.5. No other major changes have been made from Guppy 0.1.6. In the nearest future, I plan to add interactive help and more examples to the documentation, perhaps a tutorial. Look out for this in the svn HEAD according to instructions at the project home page, if you want the latest version. I will try to make a new release in perhaps a month or so. License: MIT Guppy-PE 0.1.8 is available in source tarball format on the Python Package Index (a.k.a. Cheeseshop): http://pypi.python.org/pypi/guppy/0.1.8 The project homepage is on Sourceforge: http://guppy-pe.sourceforge.net Enjoy, Sverker -- http://mail.python.org/mailman/listinfo/python-list
Guppy-PE 0.1.5 released
I would like to announce Guppy-PE 0.1.5 Guppy-PE is a library and programming environment for Python, currently providing in particular the Heapy subsystem, which supports object and heap memory sizing, profiling and debugging. It also includes a prototypical specification language, the Guppy Specificaion Language (GSL), which can be used to formally specify aspects of Python programs and generate tests and documentation from a common source. The current version is updated to work with Python 2.5 (and can still be used with 2.3 and 2.4). It also has some feature improvements, such as some shorthand attributes, and a clearer representation of shortest paths. Bugs have been fixed wrt especially remote monitoring and making the HTML generated by GSL more standards-compliant. For update details, see the changelog included with the source code. License: MIT For more information and to download the Guppy-PE source code, see: http://guppy-pe.sourceforge.net My master thesis, Heapy: A Memory Profiler and Debugger for Python has now been approved and is available here: http://guppy-pe.sourceforge.net/heapy-thesis.pdf Best regards, Sverker Nilsson -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
Guppy-PE 0.1.5 released
I would like to announce Guppy-PE 0.1.5 Guppy-PE is a library and programming environment for Python, currently providing in particular the Heapy subsystem, which supports object and heap memory sizing, profiling and debugging. It also includes a prototypical specification language, the Guppy Specificaion Language (GSL), which can be used to formally specify aspects of Python programs and generate tests and documentation from a common source. The current version is updated to work with Python 2.5 (and can still be used with 2.3 and 2.4). It also has some feature improvements, such as some shorthand attributes, and a clearer representation of shortest paths. Bugs have been fixed wrt especially remote monitoring and making the HTML generated by GSL more standards-compliant. For update details, see the changelog included with the source code. License: MIT For more information and to download the Guppy-PE source code, see: http://guppy-pe.sourceforge.net My master thesis, Heapy: A Memory Profiler and Debugger for Python has now been approved and is available here: http://guppy-pe.sourceforge.net/heapy-thesis.pdf Best regards, Sverker Nilsson -- http://mail.python.org/mailman/listinfo/python-list
Re: memory usage of a specific function
Stephen Kellett wrote: In message [EMAIL PROTECTED], Sverker Nilsson [EMAIL PROTECTED] writes [Note that actually it was Hermann Maier that wrote the following but as quoted, it may look like it was I that wrote it.] i need to find out the memory usage of a specific function that i use in my program. this function does some recursive calculations and i want my program to display the amount of memory the function used to calculate a specific value. Python Memory Validator. Run your program to completion. Switch to the hotspots tab. Search for your function. All memory used in that function will be shown in the tree (with the effective callstack) underneath that function node in the tree. http://www.softwareverify.com I can't try it out easily since it doesn't seem to support Linux. Also looking at the home page I couldn't find any description of what it was actually doing. The info links didn't work, they showed blank pages with my browser (Netscape 3.01.) So if you would like to explain here, it would be helpful. Especially, it isn't clear to me what this means (requoting): All memory used in that function Since Python has a reference-counting allocator, and various kinds of objects use specialized allocation pools, it seems unclear and implementation defined what is meant with 'all memory used'. I could think of some different things this could mean. 1. The total memory allocated, regardless of whether it is released again. If it is to mean really all the memory, it should include memory allocated by special allocators such as the allocator for ints. In this case, the following example function def f(): x = 0 while x 100: x += 1 would have allocated (at least) 100 objects, one for each iteration since int's are immutable and one is allocated each time x is assigned to (although each one is also deallocated.) So according to the meaning (1.), the 'total memory used' would be 100 * the size of ints. [ Though with some alternative interpreter implementation it might be an entirely different value such as 0 bytes since it may not need to allocate every integer in the heap. Would 'allocating' on the stack count? In a CPU register?] Is something like this what would be reported by your system? 2. The second alternative meaning would be the memory allocated in the function, MINUS the memory deallocated in the function. Is this what is reported by your system? If the case is according to the 2nd alternative, 'memory used in' f would be 0 bytes since all memory allocated is deallocated, and for a function like the following def g(): return range(1000) it would be the size allocated for the list of length 1000 + any int objects allocated to fill the list. Given the above function g, its memory usage could be tested using heapy and the following function. def test(g): from guppy import hpy hp=hpy() hp.setrelheap() hp.setrelheap() # Has to be done twice after the first import (XXX) x=g() print hp.heap() test(g) Partition of a set of 902 objects. Total size = 15364 bytes. Index Count % Size % Cumulative % Kind (class / dict of class) 0900 10010800 70 10800 70 int 1 1 0 4128 27 14928 97 list 2 1 0 436 3 15364 100 types.FrameType [ Why it shows an object of type FrameType is something of a mystery, I'm not sure of the best way to get rid of it.] I guess I could add a function somewhat like test(g) to Heapy, depending on what common use cases seem to require. On the other hand, test for memory usage according to alternative (1) would be harder to add, and it is somewhat undefined what it means. And it is perhaps not so useful as the 2nd alternative? Sverker Project home page: http://guppy-pe.sourceforge.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Compiling Guppy-PE extension modules
Claudio Grondi [EMAIL PROTECTED] wrote: I don't know if it applies here, but in this context the extern keyword comes to my mind. [snip extracts from Microsoft docs] Perhaps. But I suspect it isn't that simple since ... I'd think even if I don't use the extern keyword the compiler should be able to figure it out. (As a service to the programmer.) And it shouldn't be very difficult to do. I'll probably not try this out for a while, but maybe you can check it yourself or ask someone that knows C. Regards, Sverker -- http://mail.python.org/mailman/listinfo/python-list
Re: Compiling Guppy-PE extension modules
Claudio Grondi [EMAIL PROTECTED] wrote:
but the problem with sets.c remains:
C:\VisualC++NET2003\Vc7\bin\cl.exe /c /nologo /Ox /MD /W3 /G7 /GX
/DNDEBUG -IE:\Python24\include -IE:\Python24\PC /Tcsrc/sets/sets.c
/Fobuild\temp.win32-2.4\Re
lease\src/sets/sets.obj
sets.c
src\sets\sets.c(68) : error C2099: initializer is not a constant
src\sets\sets.c(68) : warning C4047: 'initializing' : 'PyTypeObject *'
differs in levels of indirection from 'NyHeapDef_SizeGetter'
src\sets\sets.c(69) : error C2099: initializer is not a constant
src\sets\sets.c(69) : warning C4028: formal parameter 1 different from
declaration
src\sets\sets.c(70) : error C2099: initializer is not a constant
src\sets\sets.c(70) : warning C4047: 'initializing' : 'PyTypeObject *'
differs in levels of indirection from 'int (__cdecl *)(PyObject *)'
src\sets\sets.c(70) : warning C4028: formal parameter 1 different from
declaration
src\sets\sets.c(70) : warning C4028: formal parameter 1 different from
declaration
error: command 'E:\VisualC++NET2003\Vc7\bin\cl.exe' failed with exit status
2
The MSDN help gives a simple example of code raising compiler error C2099:
// C2099.c
int j;
int *p;
j = *p; // C2099, *p is not a constant
The example code shows to me, that there is a good reason compiler
generates an error in that case.
j = *p; leads to an assignment of a random (and therefore maybe leading to a
non deterministic crash of the executable during runtime) value to a
variable, what can't be intended in a deterministic program.
This is confusing, because the problem of the example code is not that
*p is not constant but rather that p is undefined aka uninitialized.
Humm..
Compiling the following program:
#include stdio.h
int main(void) {
int j;
int *p;
j = *p;
printf(%d\n,j); /* Makes sure the optimizer doesnt remove the
previous code */
return 0;
}
with
gcc -O3 -Wall main.c
gives me:
main.c: In function `main':
main.c:4: warning: `p' might be used uninitialized in this function
(And I get a Segmentation fault from running the executable.)
The -Wall flag enables, umm, as it is saying 'all' warnings
though perhaps not really all, and this flag + others is used
by the compilation command generated by distutils when
building with gcc.
I don't see any warnings when building Guppy.
So there should be no case (as easy to discover) as that in the
example.
So I am confused.
I was beginning to wonder if we were talking about the same file/code.
This code is from sets.c lines 66..71
static NyHeapDef nysets_heapdefs[] = {
{0, NyMutBitSet_Type, (NyHeapDef_SizeGetter) mutbitset_indisize},
{0, NyCplBitSet_Type, 0, cplbitset_traverse},
{0, NyNodeSet_Type, nodeset_indisize, nodeset_traverse,
nodeset_relate},
{0}
};
I can't see how there can be problems with initializers not being
constants here unless, perhaps if the compiler has a problem since
those functions (mutbitset_indisize etc) and the types that are used
as initializers are in separately compiled files. But gcc -Wall
-pedantic handles it and doesn't warn. I don't have the ANSI standard
but I have the 2nd edition of The C Programming Language by Kernighan
and Ritchie (1988, 'Based on Draft-Proposed ANSI C'), and it is saying
- quotes -
A8.7 Initialization
...
All the expressions in the initializer for a static object or array
must be constant expressions as described in A7.19
...
A7.19 Constant Expressions
Syntactically, a constant expression is an expression restricted to a
subset of operators...
...
More latitude is permitted for the constant expressions of
initializers... Initializers must evaluate either to a constant or to
the address of a previously declared external or static object plus or
minus a constant.
- end quotes -
If this means the Microsoft .NET compiler is perhaps not compliant
with standard/ANSI C then since it was from 2003 maybe they have come
out with an update now that fixes this problem. I know, it may not
really help you because it may cost money, I don't know how long one
can get updates without paying. Alternatively, I think GCC is
available for Microsoft Windows although I don't know how it handles
the .NET architecture.
For me, if the case is according to hypothesis, and I would have to
rewrite the code to put all the functions and type objects in a single
file, it would be.. umm.. painful. (Though I could get around
it with #include trickery but it is still painful.)
Sverker
--
http://mail.python.org/mailman/listinfo/python-list
Re: Compiling Guppy-PE extension modules
Claudio Grondi [EMAIL PROTECTED] wrote: Beside the problem with the multiline strings in sets.c I was getting also: src\sets\sets.c(70) : error C2099: initializer is not a constant src\sets\sets.c(71) : error C2099: initializer is not a constant src\sets\sets.c(71) : warning C4028: formal parameter 1 different from declaration src\sets\sets.c(72) : error C2099: initializer is not a constant src\sets\sets.c(72) : warning C4047: 'initializing' : 'PyTypeObject *' differs in levels of indirection from 'int (__cdecl *)(PyObject *)' src\sets\sets.c(72) : warning C4028: formal parameter 1 different from declaration error: command 'E:\VisualC++NET2003\Vc7\bin\cl.exe' failed with exit status 2 Some of those messages are misleading, confused me at least. Seems like a bug in the error reporting part of the compiler. Anyway, the code was not ANSI and what I got around that place was: sets.c:69: warning: ANSI forbids initialization between function pointer and `void *' sets.c:71: warning: ANSI forbids initialization between function pointer and `void *' There were a bunch of other problems too. I have made a new version now, 0.1.1 . It compiles cleanly with gcc -pedantic . When looking in the code I became worried that there may be other problems with Microsoft Windows since there is some magic with DL_IMPORT that may be necessary and that I think I added most of the time but I may have missed some. I have no easy way to test with MSWindows currently and it will probably take a while before I get around to do it. The new version is on the home page: http://guppy-pe.sourceforge.net Regards, Sverker -- http://mail.python.org/mailman/listinfo/python-list
Compiling Guppy-PE extension modules
I have been informed that Guppy-PE (http://guppy-pe.sourceforge.net) has failed to compile its extension modules with a Microsoft .NET 2003 compiler under Windows 2000. [To the person who informed me about this in an email per 27 Nov: Thanks for your message. I couldn't reply to you because your email address bounced my mail.] One of the problems, seems to be string constants in the C source that contain newlines. I am using GCC on Linux so, I missed this with the standard warning options. Compiling with -pedantic reveals 'a lot' of places where this is a problem. I could fix this but it takes some work and it makes the source code less readable so I was wondering ... Is this a common problem? Or maybe it is just the compiler version mentioned that doesn't handle it? Does someone know of some switch to enable the Microsoft compiler to accept strings with newlines? If so, the setup could perhaps be made to pass this switch to the compiler when building under Windows. Or else,.. somebody knows about a script to convert the files to the format the Microsoft compiler wants? I guess the build compilation could then be setup to pipe them through this or otherwise convert the files as needed. Otherwise, I guess I'll just have to hand-hack around this problem, write a conversion script, or whatever. I will see. Sorry for any inconvenience so far. Regards, Sverker Nilsson PS. I know it's not ANSI-correct but why do we have to work to make our source codes less clear? -- http://mail.python.org/mailman/listinfo/python-list
