Re: ANN: GMPY 1.10 alpha with support for Python 3
GMPY 1.10 beta is now available. This version fixes an issue where very large objects would be cached for reuse instead of being freed. Source code and Windows installers may be found at http://code.google.com/p/gmpy/downloads/list casevh -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: GMPY 1.10 alpha with support for Python 3
On Jul 19, 5:05 pm, casevh cas...@gmail.com wrote: GMPY 1.10 beta is now available. This version fixes an issue where very large objects would be cached for reuse instead of being freed. Excellent! That explains the funny memory usage graph. Source code and Windows installers may be found athttp://code.google.com/p/gmpy/downloads/list Test results: import collatz_functions as cf # imports gmpy # the cf.Type12MH function does the equivalent # of the pure Python version of the function # using gmpy import time #3.1 gmpy 1.10 beta # t0 = time.time() n=10 for k in range(1,n): for i in range(1,n-2): print((str(cf.gmpy.numdigits(cf.Type12MH(k,i))).zfill(n)),end=' ') print() print() t1 = time.time() print(t1-t0) # 3.1 pure Python # ##t0 = time.time() ##n=10 ##for k in range(1,n): ## for i in range(1,n-2): ##print((str(cf.gmpy.numdigits( \ ## 2**(6*((i-1)*9**(k-1)+(9**(k-1)-1)//2+1)-1)-1 \ ## )).zfill(n)),end=' ') ## print() ##print() ##t1 = time.time() ## ##print(t1-t0) ## 3.1 gmpy 1.10 alpha ## ##02 04 06 07 09 11 13 ##09 25 42 58 74 91 000107 ##74 000221 000367 000513 000659 000806 000952 ##000659 001976 003293 004610 005926 007243 008560 ##005926 01 029627 041477 053328 065178 077028 ##053328 159981 266634 373287 479940 586593 693246 ##479940 0001439818 0002399696 0003359574 0004319453 0005279331 0006239209 ##0004319453 0012958355 0021597258 0030236161 0038875064 0047513967 0056152869 ##0038875064 0116625189 0194375315 0272125440 0349875565 0427625691 0505375816 ## ##3.3225422 ## funny memory usage display, different between 2.6 3.1, but funny regardless, ## memory remains allocated (~700 MB) when program halts ## 3.1 pure Python ## ##02 04 06 07 09 11 13 ##09 25 42 58 74 91 000107 ##74 000221 000367 000513 000659 000806 000952 ##000659 001976 003293 004610 005926 007243 008560 ##005926 01 029627 041477 053328 065178 077028 ##053328 159981 266634 373287 479940 586593 693246 ##479940 0001439818 0002399696 0003359574 0004319453 0005279331 0006239209 ##0004319453 0012958355 0021597258 0030236161 0038875064 0047513967 0056152869 ##0038875064 0116625189 0194375315 0272125440 0349875565 0427625691 0505375816 ## ##338.83217 ## memory usage seems normal, runs slow enough to observe memory being allocated ## and freed and remains freed when program halts ## 2.6 pure Python ## ##02 04 06 07 09 11 13 ##09 25 42 58 74 91 000107 ##74 000221 000367 000513 000659 000806 000952 ##000659 001976 003293 004610 005926 007243 008560 ##005926 01 029627 041477 053328 065178 077028 ##053328 159981 266634 373287 479940 586593 693246 ##479940 0001439818 0002399696 0003359574 0004319453 0005279331 0006239209 ##0004319453 0012958355 0021597258 0030236161 0038875064 0047513967 0056152869 ##aborted after ONE FULL WEEK of processing (only 8 of 9 rows completed) ## Something funny happened here. By the time I ran this test, there ## were 14 orphaned copies of pythonw.exe present, at least one of which ## was still running along with the real pair executing the program. Yet ## the Performance only registered ~50%. Normally, this would be 100% if ## two programs were running. Pure Python is a LOT slower than gmpy, ## but not THAT slow. These orphaned processes may be a Python 3.1 issue. ##=== ## 3.1 gmpy 1.10 beta ## ## 02 04 06 07 09 11 13 ## 09 25 42 58 74 91 000107 ## 74 000221 000367 000513 000659 000806 000952 ## 000659 001976 003293 004610 005926 007243 008560 ## 005926 01 029627 041477 053328 065178 077028 ## 053328 159981 266634 373287 479940 586593 693246 ## 479940 0001439818 0002399696 0003359574 0004319453 0005279331 0006239209 ## 0004319453 0012958355 0021597258 0030236161 0038875064 0047513967 0056152869 ## 0038875064 0116625189 0194375315 0272125440 0349875565 0427625691 0505375816 ## ## 2.4178283 ## memory usage now appears normal, no more allocated memory when program halts ## and it's so fast all we see on Performance graph is a small blip, much better ## than the 700-1400 MB
Re: ANN: GMPY 1.10 alpha with support for Python 3
On Jul 7, 12:47 am, Mensanator mensana...@aol.com wrote: On Jul 7, 12:16 am, casevh cas...@gmail.com wrote: I discovered a serious bug with comparisons and have posted alpha2 which fixes that bug and adds Unicode support for Python 2.x casevh Damn! I was just congatulating myself for pulling off a hat trick (there had been no point in downloading 3.x without gmpy so I have been putting it off): - installing Python 3.1 - installing gmpy 1.10 - converting my Collatz Function library to 3.1 syntax And it all worked smoothly, just had to add parentheses to my print statements, change xrange to range and all my / to // (the library is exclusively integer). I had gmpy running in my library on 3.1 in about 10 minutes. So I'll have to re-do the gmpy install. Shouldn't be any big deal. I started doing some real world tests. Generally, things look good (nothing crashes, timing looks not bad) but I'm getting some funny results on one of my tests, so I'll report back when I have more information. As I said, I was getting funny results from one of my tests. It seemed to work ok, but timing varied from 2 to 88 seconds, which seemed odd. The other tests were generally consistent for a given environment (cpu speed, OS, Python version, gmpy version). At some point I watched the memory usage profile from Windows. On the same machine I have both Python 2.6 and 3.1 installed, with the appropriate gmpy 1.10 version loaded for each. In Python 2.6, it looks like this: memory usage profile Python 2.6 gmpy 1.1 Vista /--\ /---\ .RESTART SHELL / .\/ \ /.\ . . . . start of RUN start of RUN The first start of RUN is the first time the test is run (from IDLE). That change in usage represents about 700 MB (I'm testing really BIG numbers, up to 500 million digits). The memory remains allocated after the program terminates (the flat plateau). When I run a second time, we see the allocation dip, then climb back up to the plateau, so it appears that the allocation never climbs above 1.1 GB. Finally, doing a RESTART SHELL seems to completely free the allocated memory. I assume this is normal behaviour. With Python 3.1, it get this profile: memory usage profile Python 3.1 gmpy 1.1 Vista /- / | // \--\ .RESTART SHELL / . \ / .\___ . . . . start of RUNstart of RUN Here, at the start of the second RUN, it appears that new memory is allocated BEFORE the previous is cleared. Is this a quirk in the way 3.1 behaves? Here, the peak usage climbs to 1.8 GB which I think causes VM thrashing accounting for the increased execution times. My guess is that gmpy is provoking, but not causing this behaviour. The actual test is: t0 = time.time() n=10 for k in range(1,n): for i in range(1,n-2): print((str(cf.gmpy.numdigits(cf.Type12MH(k,i))).zfill(n)),end=' ') print() print() t1 = time.time() The library function Type12MH is: def Type12MH(k,i): Find ith, kth Generation Type [1,2] Mersenne Hailstone using the closed form equation Type12MH(k,i) k: generation i: member of generation returns Hailstone (a) ONE = gmpy.mpz(1) TWO = gmpy.mpz(2) SIX = gmpy.mpz(6) NIN = gmpy.mpz(9) if (k1) or (i1): return 0 i = gmpy.mpz(i) k = gmpy.mpz(k) # a = (i-1)*9**(k-1) + (9**(k-1) - 1)//2 + 1 # return 2**(6*a - 1) - 1 a = (i-ONE)*NIN**(k-ONE) + (NIN**(k-ONE) - ONE)//TWO + ONE return TWO**(SIX*a - ONE) - ONE ## Sample runs ## ## Test 1 - create numbers up to 500 million digits ## ## 02 04 06 07 09 11 13 ## 09 25 42 58 74 91 000107 ## 74 000221 000367 000513 000659 000806 000952 ## 000659 001976 003293 004610 005926 007243 008560 ## 005926 01 029627 041477 053328 065178 077028 ## 053328 159981 266634 373287 479940 586593 693246 ## 479940 0001439818 0002399696 0003359574 0004319453 0005279331 0006239209 ## 0004319453 0012958355 0021597258 0030236161 0038875064 0047513967 0056152869 ## 0038875064 0116625189 0194375315 0272125440 0349875565 0427625691 0505375816 ## ## 15.546038 ## RESTART ## ## 02 04 06 07 09 11 13 ## 09 25 42 58 74 91 000107 ## 74 000221 000367 000513 000659 000806 000952 ## 000659 001976 003293 004610 005926 007243
Re: ANN: GMPY 1.10 alpha with support for Python 3
On Jul 8, 5:03 pm, Mensanator mensana...@aol.com wrote: On Jul 7, 12:47 am, Mensanator mensana...@aol.com wrote: On Jul 7, 12:16 am, casevh cas...@gmail.com wrote: I discovered a serious bug with comparisons and have posted alpha2 which fixes that bug and adds Unicode support for Python 2.x casevh Damn! I was just congatulating myself for pulling off a hat trick (there had been no point in downloading 3.x without gmpy so I have been putting it off): - installing Python 3.1 - installing gmpy 1.10 - converting my Collatz Function library to 3.1 syntax And it all worked smoothly, just had to add parentheses to my print statements, change xrange to range and all my / to // (the library is exclusively integer). I had gmpy running in my library on 3.1 in about 10 minutes. So I'll have to re-do the gmpy install. Shouldn't be any big deal. I started doing some real world tests. Generally, things look good (nothing crashes, timing looks not bad) but I'm getting some funny results on one of my tests, so I'll report back when I have more information. As I said, I was getting funny results from one of my tests. It seemed to work ok, but timing varied from 2 to 88 seconds, which seemed odd. The other tests were generally consistent for a given environment (cpu speed, OS, Python version, gmpy version). At some point I watched the memory usage profile from Windows. On the same machine I have both Python 2.6 and 3.1 installed, with the appropriate gmpy 1.10 version loaded for each. In Python 2.6, it looks like this: memory usage profile Python 2.6 gmpy 1.1 Vista /--\ /---\ .RESTART SHELL / .\/ \ / . \ . . . . start of RUN start of RUN The first start of RUN is the first time the test is run (from IDLE). That change in usage represents about 700 MB (I'm testing really BIG numbers, up to 500 million digits). The memory remains allocated after the program terminates (the flat plateau). When I run a second time, we see the allocation dip, then climb back up to the plateau, so it appears that the allocation never climbs above 1.1 GB. Finally, doing a RESTART SHELL seems to completely free the allocated memory. I assume this is normal behaviour. With Python 3.1, it get this profile: memory usage profile Python 3.1 gmpy 1.1 Vista /- / | // \--\ .RESTART SHELL / . \ / . \___ . . . . start of RUN start of RUN Here, at the start of the second RUN, it appears that new memory is allocated BEFORE the previous is cleared. Is this a quirk in the way 3.1 behaves? Here, the peak usage climbs to 1.8 GB which I think causes VM thrashing accounting for the increased execution times. Hmmm. It looks like memory is not being release properly. I don't see that behavior under Linux. The running time is a very consistent 1.35 seconds. I'm traveling at the moment so it will be at least a week before I can test under Windows. Thanks for the report. I'll try to see if I can figure out what is going on. casevh My guess is that gmpy is provoking, but not causing this behaviour. The actual test is: t0 = time.time() n=10 for k in range(1,n): for i in range(1,n-2): print((str(cf.gmpy.numdigits(cf.Type12MH(k,i))).zfill(n)),end=' ') print() print() t1 = time.time() The library function Type12MH is: def Type12MH(k,i): Find ith, kth Generation Type [1,2] Mersenne Hailstone using the closed form equation Type12MH(k,i) k: generation i: member of generation returns Hailstone (a) ONE = gmpy.mpz(1) TWO = gmpy.mpz(2) SIX = gmpy.mpz(6) NIN = gmpy.mpz(9) if (k1) or (i1): return 0 i = gmpy.mpz(i) k = gmpy.mpz(k) # a = (i-1)*9**(k-1) + (9**(k-1) - 1)//2 + 1 # return 2**(6*a - 1) - 1 a = (i-ONE)*NIN**(k-ONE) + (NIN**(k-ONE) - ONE)//TWO + ONE return TWO**(SIX*a - ONE) - ONE ## Sample runs ## ## Test 1 - create numbers up to 500 million digits ## ## 02 04 06 07 09 11 13 ## 09 25 42 58 74 91 000107 ## 74 000221 000367 000513 000659 000806 000952 ## 000659 001976 003293 004610 005926 007243 008560 ## 005926 01 029627 041477 053328 065178 077028 ## 053328 159981 266634 373287 479940 586593 693246 ## 479940 0001439818 0002399696 0003359574 0004319453 0005279331 0006239209 ## 0004319453 0012958355 0021597258 0030236161 0038875064 0047513967 0056152869 ##
ANN: GMPY 1.10 alpha with support for Python 3
An alpha release of GMPY that supports Python 2 and 3 is available. GMPY is a wrapper for the GMP multiple-precision arithmetic library. The MPIR multiple-precision arithmetic library is also supported. GMPY is available for download from http://code.google.com/p/gmpy/ Support for Python 3 required many changes to the logic used to convert between different numerical types. The result type of some combinations has changed. For example, 'mpz' + 'float' now returns an 'mpf' instead of a 'float'. See the file changes.txt for more information. In addition to support for Python 3, there are several other changes and bug fixes: - Bug fixes in mpz.binary() and mpq.binary(). - Unicode strings are accepted as input on Python 2. (Known bug: works for mpz, fails for mpq and mpf) - The overhead for calling GMPY routines has been reduced. If one operand in a small integer, it is not converted to mpz. - 'mpf' and 'mpq' now support % and divmod. Comments on provided binaries The 32-bit Windows installers were compiled using MPIR 1.2.1 and will automatically recognize the CPU type and use code optimized for that CPU. Please test with your applications and report any issues found! casevh -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
ANN: GMPY 1.10 alpha with support for Python 3
An alpha release of GMPY that supports Python 2 and 3 is available. GMPY is a wrapper for the GMP multiple-precision arithmetic library. The MPIR multiple-precision arithmetic library is also supported. GMPY is available for download from http://code.google.com/p/gmpy/ Support for Python 3 required many changes to the logic used to convert between different numerical types. The result type of some combinations has changed. For example, 'mpz' + 'float' now returns an 'mpf' instead of a 'float'. See the file changes.txt for more information. In addition to support for Python 3, there are several other changes and bug fixes: - Bug fixes in mpz.binary() and mpq.binary(). - Unicode strings are accepted as input on Python 2. (Known bug: works for mpz, fails for mpq and mpf) - The overhead for calling GMPY routines has been reduced. If one operand in a small integer, it is not converted to mpz. - 'mpf' and 'mpq' now support % and divmod. Comments on provided binaries The 32-bit Windows installers were compiled using MPIR 1.2.1 and will automatically recognize the CPU type and use code optimized for that CPU. Please test with your applications and report any issues found! casevh -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: GMPY 1.10 alpha with support for Python 3
I discovered a serious bug with comparisons and have posted alpha2 which fixes that bug and adds Unicode support for Python 2.x casevh -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: GMPY 1.10 alpha with support for Python 3
On Jul 7, 12:16�am, casevh cas...@gmail.com wrote: I discovered a serious bug with comparisons and have posted alpha2 which fixes that bug and adds Unicode support for Python 2.x casevh Damn! I was just congatulating myself for pulling off a hat trick (there had been no point in downloading 3.x without gmpy so I have been putting it off): - installing Python 3.1 - installing gmpy 1.10 - converting my Collatz Function library to 3.1 syntax And it all worked smoothly, just had to add parentheses to my print statements, change xrange to range and all my / to // (the library is exclusively integer). I had gmpy running in my library on 3.1 in about 10 minutes. So I'll have to re-do the gmpy install. Shouldn't be any big deal. I started doing some real world tests. Generally, things look good (nothing crashes, timing looks not bad) but I'm getting some funny results on one of my tests, so I'll report back when I have more information. -- http://mail.python.org/mailman/listinfo/python-list
