Re: Lies in education [was Re: The "loop and a half"]
On 10/5/17, Steve D'Apranowrote: > The A and E in the word "are" are not vowels, since they are silent. Interesting! :) Is then R (half?) silent in word "Brazil"? -- https://mail.python.org/mailman/listinfo/python-list
Re: Creating a Dictionary
On 10/5/17, Chris Angelicowrote: > On Thu, Oct 5, 2017 at 12:24 PM, Stefan Ram wrote: >> One might wish to implement a small language with these commands: >> >> F - move forward >> B - move backward >> L - larger stepsize >> S - smaller stepsize >> >> . One could start with the following pseudocode for a dictionary: >> >> { 'F': lambda: myturtle.forward( s ), >> 'B': lambda: myturtle.backward( s ), >> 'L': lambda: global s; s *= 2, >> 'S': lambda: global s; s /= 2 } >> >> . But lambda-expressions cannot contain statements. >> >> Any other suggestions? > > There are a few options here. One is to make use of a simple > "collector decorator": > > commands = {} > def cmd(f): > commands[f.__name__.upper()] = f > return f > > @cmd > def f(): > myturtle.forward( s ) > > @cmd > def b(): > myturtle.backward( s ) > > @cmd > def l(): > global siz > size *= 2 > > @cmd > def s(): > global siz > size //= 2 > > (Also untested, but the pattern is one I've used many times.) > > Another is to deploy the whole thing as a class, with no globals: > > class Turtle: > def __init__(self): > self.size = ... > self.turtle = ... > def cmd_F(self): > self.turtle.forward(self.size) > def cmd_B(self): > self.turtle.backward(self.size) > def cmd_L(self): > self.size *= 2 > def cmd_S(self): > self.size //= 2 > > Note that I've added "cmd_" in front of the names, which means you can > safely getattr(turtle, "cmd_" + letter) and be confident you won't > accidentally grab something that isn't a command. > > (Note also that I used // for division. If your size is actually a > float, then change those back to single slashes.) > > Either of these patterns would work fairly well. There are others, > too, but I'd be inclined to use one of these. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list size = 2*size if size else 1 -- https://mail.python.org/mailman/listinfo/python-list
Re: Line terminators in Python?
On 9/29/17, Rob Gaddiwrote: > On 09/29/2017 10:54 AM, Stefan Ram wrote: >>In some languages, printing »'\n'«, the Unicode code point 10, >>will have the effect of printing a line terminator, which might >>mean that the output device actually receives »\r\n«. >> >>The line terminator ostensibly depends on the operating >>system, but actually it depends on the output system. E.g., >>under one operating system the console might accept another >>set of line separators than an editor. (Under Windows, >>»wordpad« accepts »\n«, while »notepad« requires »\r\n«.) >> >>What is the recommended way to terminate a line written with >>Python? Is it »\n« or something else? For example, in Java, >>in some cases, one should terminate the line with the value >>of »java.lang.System.lineSeparator()« which might or might >>not be equal to the value of »"\n"«. >> >>Does it possibly depend on the entity being written to, which >>might be >> >>- the Python console, >>- the IDLE console, >>- the operating system console or >>- a text file? >> > > As everyone else has said; for general purpose (any of your cases) use > you should always just use '\n' and it automagically works. The only > time I've ever needed to explicitly worry about '\r' is communicating > over sockets or serial ports to devices. And in those cases you need to > stuff them with bytes rather than str anyhow, so you're already down in > the gory bits. > > -- > Rob Gaddi, Highland Technology -- www.highlandtechnology.com There is also https://docs.python.org/3/library/csv.html#csv.Dialect.lineterminator which (I think) come from RFC4180 (see http://www.rfc-archive.org/getrfc.php?rfc=4180 search CRLF) http://www.rfc-archive.org/getrfc.php?rfc=2046 could be also interesting in this case. -- https://mail.python.org/mailman/listinfo/python-list
Re: How to share class relationship representations?
On 9/23/17, Stephan Houben <stephan...@gmail.com.invalid> wrote: > Op 2017-09-22, Pavol Lisy schreef <pavol.l...@gmail.com>: >> On 9/19/17, leam hall <leamh...@gmail.com> wrote: >>> I'm working on designing the classes, sub-classes, and relationships in >>> my >>> code. What is a good visual way to represent it so it can be stored in >>> git >>> and shared on the list without large images or attachments? >>> >>> Thanks! >>> >>> Leam >> >> https://stackoverflow.com/questions/29586520/can-one-get-hierarchical-graphs-from-networkx-with-python-3#29597209 > > For direct inclusion in source code, what about plain text with > Unicode box drawing characters? > > ┏━━┓ > ┃object┃ > ┗━━━┳━━┛ > ┃ > ┏━━┻━━┓ > ┃float┃ > ┗━┛ I am not big fan of UML. I don't really think it is helping. I think that it often just enhance maintenance cost. But if you really need or want it could be good to have some tool to simplify work. And you could probably look at -> http://www.plantuml.com/plantuml To trying something like diagram above you could put there next text and submit: @startuml object object object float object int object -- float object -- int @enduml As I said I don't like UML so I don't propose this :) But if you need "git-able" (text representation) of class diagrams which could be used to generate quite nice images then maybe this tool could be useful. -- https://mail.python.org/mailman/listinfo/python-list
Re: what is happening in panda "where" clause
On 9/22/17, Peter Otten <__pete...@web.de> wrote: > Exposito, Pedro (RIS-MDW) wrote: > >> This code does a "where" clause on a panda data frame... >> >> Code: >> import pandas as pd; >> col_names = ['Name', 'Age', 'Weight', "Education"]; >> # create panda dataframe >> x = pd.read_csv('test.dat', sep='|', header=None, names = col_names); >> # apply "where" condition >> z = x[ (x['Age'] == 55) ] >> # prints row WHERE age == 55 >> print (z); >> >> What is happening in this statement: >> z = x[ (x['Age'] == 55) ] >> >> Thanks, > > Let's take it apart into individual steps: > > Make up example data: > import pandas as pd x = pd.DataFrame([["Jim", 44], ["Sue", 55], ["Alice", 66]], > columns=["Name", "Age"]) x > Name Age > 0Jim 44 > 1Sue 55 > 2 Alice 66 > > Have a look at the inner expression: > x["Age"] == 55 > 0False > 1 True > 2False > > So this is a basically vector of boolean values. If you want more details: > in numpy operations involving a a scalar and an array work via > "broadcasting". In pure Python you would write something similar as > [v == 55 for v in x["Age"]] > [False, True, False] > > Use the result as an index: > x[[False, True, True]] > Name Age > 1Sue 55 > 2 Alice 66 > > [2 rows x 2 columns] > > This is again in line with numpy arrays -- if you pass an array of boolean > values as an index the values in the True positions are selected. In pure > Python you could achieve that with > index = [v == 55 for v in x["Age"]] index > [False, True, False] [v for b, v in zip(index, x["Age"]) if b] > [55] You could also write: >>> x[index] x[index] Name Age 1 Sue 55 As Peter told, understanding numpy or "numpy like" behavior behind curtain could be key to understanding what is happening. Look at this: >>> def half_age(a): >>> return a/2 we could use this function in where() -> >>> x[half_age(x.Age)<30] Name Age 0 Jim 44 1 Sue 55 You could think that any function could be used, but it is not true. a/2 (where a is numpy array) is defined. (I am not sure that it is really numpy under the hood or if it will be in future pandas versions but it seems so). But what if we have more universal function? >>> def first_char(a): >>> return a[0].lower() >>> x[first_char(x.Name)=='a'] ... ERROR ... (first_char could not work with pandas serie as a argument. It means pandas Series doesn't have lower() function) But you could create index like Peter described. It could be simpler to remember if you are using pandas not often: >>> x[[first_char(i)=='a' for i in x.Name]] Name Age 2 Alice 66 It is not (IMHO) best solution because you are creating list in memory (which could be big) . Unfortunately my version of pandas (0.20.3) doesn't support generator >>> x[(first_char(i)=='a' for i in x.Name)] ... ERROR... But you could apply more complex function to Series! >>> x[x.Name.apply(first_char)=='a'] Name Age 2 Alice 66 x.Name.apply(first_char) is Series where first_char is applied on each value. And serie (and numpy.array) understand == operator (and return value is Series. Which is acceptable as where function parameter) Although I am not sure that actual version of pandas doesn't create whole Series in memory :) I expect it is more optimized (and could be even more in future). Be aware that I am not expert and I am using pandas only very seldom just for my little statistics! -- https://mail.python.org/mailman/listinfo/python-list
Re: How to share class relationship representations?
On 9/19/17, leam hallwrote: > I'm working on designing the classes, sub-classes, and relationships in my > code. What is a good visual way to represent it so it can be stored in git > and shared on the list without large images or attachments? > > Thanks! > > Leam https://stackoverflow.com/questions/29586520/can-one-get-hierarchical-graphs-from-networkx-with-python-3#29597209 there are probably more ideas useful for you. Maybe this one could be inspiring too: import networkx as nx g=nx.DiGraph() g.add_edges_from([('class 1','class 2'), ('class 1','class 3'), ('class 1','class 4'), ('class 2','class 5'), ('class 2','class 6'), ('class 2','class 7'), ('class 3','class 8'), ('class 3','class 9'), ('class 4','class 10'), ('class 5','class 11'), ('class 5','class 12'), ('class 6','class 13')]) p=nx.drawing.nx_pydot.to_pydot(g) p.write_png('example.png') And you could make function which from list of classes create list of edges! :) In git you don't need to store images just list of classes and script to make images (documentation). -- https://mail.python.org/mailman/listinfo/python-list
Re: Convert pandas series to string and datetime object
On 9/21/17, Peter Otten <__pete...@web.de> wrote: > zljubi...@gmail.com wrote: > >> I have sliced the pandas dataframe >> >> end_date = df[-1:]['end'] >> >> type(end_date) >> Out[4]: pandas.core.series.Series >> >> end_date >> Out[3]: >> 48173 2017-09-20 04:47:59 >> Name: end, dtype: datetime64[ns] >> >> 1. How to get rid of index value 48173 and get only "2017-09-20 > 04:47:59" >> string? I have to call REST API with "2017-09-20 04:47:59" as a >> parameter, >> so I have to get string from pandas datetime64 series. >> 2. How to get rid of index value 48173 and get only datetime object >> [something like datetime.datetime.strptime('2017-09-20 04:47:59', >> '%Y-%m-%d %H:%M:%S')]. Later I will have to check if '2017-09-20 >> 04:47:59' >> < datetime.datetime(2017,1,9) >> >> How to do these conversions? > > After a web search and some trial and error: > d = pd.DataFrame([[1, datetime.datetime.now()], [2, > datetime.datetime.now()]], columns=["whatever", "end"]) d >whateverend > 0 1 2017-09-21 22:36:52.342757 > 1 2 2017-09-21 22:36:52.349973 > > [2 rows x 2 columns] d["end"].astype(datetime.datetime).values[-1] > datetime.datetime(2017, 9, 21, 22, 36, 52, 349973) Or: >>> d.iloc[-1]['end'].strftime('%Y-%m-%d %H:%M:%S') '2017-09-22 09:03:40' PS. pandas is one of reasons why python is so popular these days. But "there is only milion way how to do it" (and other unpythonic issues) I see there every time I am looking at it. :) -- https://mail.python.org/mailman/listinfo/python-list
Re: iPython ? magic
On 9/21/17, Steve D'Apranowrote: > In the iPython interactive interpreter, obj? prints information about the > given > object. For example: > > > In [11]: None? > Type: NoneType > Base Class: > String Form:None > Namespace: Python builtin > Docstring: > > > Does anyone know that the Namespace field is supposed to show? I can't get > it to > display anything except either "Python builtin" or "Interactive". I discovered that there are 3 default namespaces -> https://github.com/ipython/ipython/blob/79921f6fe380f57cf353d76615e4fd8472c83118/IPython/core/interactiveshell.py#L1405 But you could somehow change or add other namespaces (maybe ipdb could add local namespace to evaluate local variables). You could check thist code: import IPython ip = IPython.core.getipython.get_ipython() print(ip.user_ns) # this is in 'Interactive' namespace print(ip.user_global_ns)# this is in 'Interactive (global)' namespace # next line show how somewhat somewhere could change namespace where object names are searched ip._inspect("pinfo", "None", [("my namespace", {"None":None})]) -- https://mail.python.org/mailman/listinfo/python-list
Re: Old Man Yells At Cloud
On 9/20/17, Steve D'Aprano <steve+pyt...@pearwood.info> wrote: > On Wed, 20 Sep 2017 02:55 pm, Pavol Lisy wrote: Thanks Steve, I agree with most of your mail and really appreciate interesting reading! :) > (a) "you save one character (two keystrokes)"; and First I have to admit that I forgot space! But if we like to be pedantic (what I am not sure) then we compare vs <shift+( and shif+)> so 1 vs 4 , right? So not two keystrokes but three? > print ', '.join(str(obj) for obj in (foo, bar, baz)) # 52 chars > # and builds up a potentially large string ahead of time > print(foo, bar, baz, sep=', ') # 30 chars > # and DOESN'T build up a large string ahead of time > # and is self-explanatory I am totally on your side here! Just forgive me next: print(f"{foo}, {bar}, {baz}") # 29 :P >> 3. "for sake of saving one character"? Could we really simplify >> motivation of opposite side to just this? (what about breaking old >> code?) > > I don't know what the motivation of the "print should be a statement!" side > is, > because they never say why it was better. (If they have said, I have never > seen > it.) I am not sure if links below could bring a little light to this, but probably it could be interesting: Julia has print and println functions but has also printf macro. About motivation why not function you could read here -> https://stackoverflow.com/questions/19783030/in-julia-why-is-printf-a-macro-instead-of-a-function#19784718 and update of view is IMHO worth to read too -> https://github.com/JuliaLang/julia/pull/11941#issuecomment-129966921 -- https://mail.python.org/mailman/listinfo/python-list
Re: Old Man Yells At Cloud
On 9/19/17, Steve D'Apranowrote: [...] > The point is, we all make the occasional silly error. Doesn't mean we should > cripple our functions and fill the language with special cases like the > print > statement to avoid such rare errors. If print had always been a function, > and > someone suggested making it a statement, who would be willing to accept all > those disadvantages for the sake of saving one character? I am not going to support print as a statement in this discussion! But what you are describing is some kind of alternative history which did not happen. Question was not about crippling function to statement. (Which function?) What I mean is that if we like to convince people that it was good decision then we need to use proper arguments. For example some others which seems like not so proper to me: 1. "learn-unlearn" argument could be viewed differently from opposite camp (they need to unlearn statement) 2. "print foo" save more than one character ("print foo ," more than two characters) 3. "for sake of saving one character"? Could we really simplify motivation of opposite side to just this? (what about breaking old code?) BTW if python would only bring "from __cleverness__ import print_function" how many people would accept your reasons and use it? And how many would rewrite old code? How many people would say something like: "Oh it is cool! Now I could rewrite my old code and monkey patch print!" ? -- https://mail.python.org/mailman/listinfo/python-list
Re: People choosing Python 3
On 9/11/17, Thomas Jollanswrote: > On 2017-09-10 09:05, INADA Naoki wrote: >> I saw encouraging tweet from Kenneth Reitz. >> >> https://twitter.com/kennethreitz/status/902028601893294081/photo/1 >> >> On Heroku, most people choose Python 3! >> I know, it's because Python 3 is the default Python on Heroku. >> >> I can't wait Python 3 is the default Python of Red Hat, and "python" >> command means Python 3 on Debian and Ubuntu. > > https://www.python.org/dev/peps/pep-0394/ > > Debian follows PEP 394, which recommends that "python" point to python2, > and I don't see that changing any time soon (certainly not before RHEL > includes python3 by default. Which part of third party ecosystem surrounding Python 3 is not (and could not be any time soon) sufficiently mature? -- https://mail.python.org/mailman/listinfo/python-list
Re: A question on modification of a list via a function invocation
On 9/8/17, Gregory Ewingwrote: > Steve D'Aprano wrote: >> A harder question is, what if you take a random number from the Integers? >> How >> many digits will it have in (say) base 10? I don't have a good answer to >> that. >> I think it may be ill-defined. > > I think the answer is that on average it has infinitely many > digits -- despite every actual integer only having finitely > many digits! > > We can prove this by contradiction. Suppose the answer were > some finite number N. There are only finitely many integers > with N or fewer digits, but there are infinitely many with > more than N digits, so including them in the average must > make it bigger than N. So N cannot be finite. Sorry that my english is so poor that I could only draft ideas. :/ I think that it probably depends on distribution. Think something like: def numbers(e=0.999): ''' random numbers from integers ''' while 1: r = random.random() yield int(1/(1-r)**e) and see: https://www.wolframalpha.com/input/?i=area+between+y+%3D+1%2Fx%5E0.999+and+y+%3D+0+between+x+%3D+0+and+1 and unbounded (for e==1) -> https://www.wolframalpha.com/input/?i=area+between+y+%3D+1%2Fx+and+y+%3D+0+between+x+%3D+0+and+1 # if somebody likes to test hipothesis -> def avg(N=1000): return sum(itertools.islice(numbers(), 0,N,1))/N -- https://mail.python.org/mailman/listinfo/python-list
Re: Case-insensitive string equality
On 9/3/17, Steve D'Apranowrote: > On Sun, 3 Sep 2017 05:17 pm, Stephan Houben wrote: > >> Generally speaking, the more you learn about case normalization, >> the more attractive case sensitivity looks > > Just because something is hard doesn't mean its not worth doing. > > And just because you can't please all the people all the time doesn't mean > its > not worthwhile. I was thinking about compare where false positivity is acceptable (and well defined property). For example if somebody has case sensitive FS and wants to publish files and avoid name collision on any case insensitive FS then compare with false positive equals could be useful. Then backward compatibility problem could be (theoretically) simplified to enhancing equivalence classes in future. I mean something like -> equal = lambda a, b: any(f(a) == f(b) for f in C)# where C is enhanceble list of compare equals functions Could you think that such equivalence relation could solve problems which you describe in first mail in this thread? And if trying to "solve" unicode problem why not? -> a ≐ b a ⋵ L -- https://mail.python.org/mailman/listinfo/python-list
Re: Case-insensitive string equality
On 9/2/17 at 4:21, Steve D'Apranowrote: > If regular case-sensitive string comparisons don't support the locale, why > should case-insensitive comparisons be required to? I think that Chris answered very good before: On 9/2/17 at 2:53 AM, Chris Angelico wrote: > On Sat, Sep 2, 2017 at 10:31 AM, Steve D'Aprano > But code is often *wrong* due to backward compatibility concerns. Then you > have to > decide whether, for a brand new API, it's better to "do the same as > the regex module" or to "do what the Unicode consortium says". But problem is that if somebody like to have stable API it has to be changed to "do what the Unicode consortium said (at X.Y. )" :/ Maybe it is simpler to write intelligent linter to catch wrong comparisions? -- https://mail.python.org/mailman/listinfo/python-list
Re: If you are running 32-bit 3.6 on Windows, please test this
On 9/2/17, eryk sunwrote: > On Fri, Sep 1, 2017 at 3:23 AM, Peter Otten <__pete...@web.de> wrote: >> >> I think you have to specify the types yourself: >> > import ctypes > libm = ctypes.cdll.LoadLibrary("libm.so") > libm.sqrt(42) >> 0 > libm.sqrt.argtypes = [ctypes.c_double] > libm.sqrt.restype = ctypes.c_double > libm.sqrt(42) >> 6.48074069840786 > > On POSIX systems, use ctypes.util.find_library('m'), which, for > example, resolves to "libm.so.6" in Ubuntu Linux 16.04. On the same > system, "libm.so" is an ld script that's used by the compile-time > linker. > > $ cat /usr/lib/x86_64-linux-gnu/libm.so > /* GNU ld script > */ > OUTPUT_FORMAT(elf64-x86-64) > GROUP ( /lib/x86_64-linux-gnu/libm.so.6 > AS_NEEDED ( > /usr/lib/x86_64-linux-gnu/libmvec_nonshared.a > /lib/x86_64-linux-gnu/libmvec.so.1 ) ) > > The runtime linker doesn't know how to handle this script. > > >>> ctypes.CDLL('libm.so') > Traceback (most recent call last): > File "", line 1, in > File "/usr/lib/python3.5/ctypes/__init__.py", > line 347, in __init__ > self._handle = _dlopen(self._name, mode) > OSError: /usr/lib/x86_64-linux-gnu/libm.so: invalid ELF header Peter and Eryk thanks very much! :) BTW julia on ubuntu 16.04 could do it -> julia> result = ccall((:sqrt, "libm"), Cdouble, (Cdouble,), 1.3) 1.140175425099138 but I am afraid that it probably uses some llvm's magic so it could not help to solve this problem in ctypes... -- https://mail.python.org/mailman/listinfo/python-list
Re: If you are running 32-bit 3.6 on Windows, please test this
On 8/31/17, 20/20 Lab <l...@2020fresno.com> wrote: > > > On 08/31/2017 01:53 AM, Pavol Lisy wrote: [...] > Valid point, fired up a windows 10 machine and worked as well. > > Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:14:34) [MSC v.1900 32 bit > (Intel)] on win32 > Type "copyright", "credits" or "license()" for more information. > >>> import math > >>> math.sqrt(1.3) > 1.140175425099138 > >>> > > This machine does not have the creators update yet. So there's that. > -- > https://mail.python.org/mailman/listinfo/python-list Thx! :) Could somebody help me? I was trying to call sqrt using ctypes from msvcrt but I am not succesful: import ctypes msc = ctypes.windll.msvcrt def msqrt(arg): s = ctypes.create_string_buffer(100) d = ctypes.c_longdouble(arg) msc.sprintf(s, b'arg = %g', d) print(s.value.decode()) r = msc.sqrt(d) msc.sprintf(s, b'sqrt = %g', r) # r is int so this format is wrong I just like to show my intention print(s.value.decode()) print("r = ", r, r.__class__) >>> msqrt(1.3) arg = 1.3 sqrt = 0 r = 0 >>> msqrt(-1) arg = -1 sqrt = 4.00144e-320 # because wrong format in sprintf r = 8099 And -> >>> msc.sqrt.restype ctypes.c_long >>> msc.sqrt.argtypes is None True How to do it properly? Isn't ctypes broken? -- https://mail.python.org/mailman/listinfo/python-list
Re: Case-insensitive string equality
On 8/31/17, Steve D'Apranowrote: >> Additionally: a proper "case insensitive comparison" should almost >> certainly start with a Unicode normalization. But should it be NFC/NFD >> or NFKC/NFKD? IMO that's a good reason to leave it in the hands of the >> application. > > Normalisation is orthogonal to comparisons and searches. Python doesn't > automatically normalise strings, as people have pointed out a bazillion > times > in the past, and it happily compares > > 'ö' LATIN SMALL LETTER O WITH DIAERESIS > > 'ö' LATIN SMALL LETTER O + COMBINING DIAERESIS > > > as unequal. I don't propose to change that just so that we can get 'a' > equals 'A' :-) Locale-dependent Case Mappings. The principal example of a case mapping that depends on the locale is Turkish, where U+0131 “ı” latin small letter dotless i maps to U+0049 “I” latin capital letter i and U+0069 “i” latin small letter i maps to U+0130 “İ” latin capital letter i with dot above. (source: http://www.unicode.org/versions/Unicode10.0.0/ch05.pdf) So 'SIKISIN'.casefold() could be dangerous -> https://translate.google.com/#tr/en/sikisin%0As%C4%B1k%C4%B1s%C4%B1n (although I am not sure if this story is true -> https://www.theinquirer.net/inquirer/news/1017243/cellphone-localisation-glitch ) -- https://mail.python.org/mailman/listinfo/python-list
Re: If you are running 32-bit 3.6 on Windows, please test this
On 8/31/17, Terry Reedywrote: > On 8/30/2017 1:35 PM, Terry Reedy wrote: >> https://stackoverflow.com/questions/45965545/math-sqrt-domain-error-when-square-rooting-a-positive-number >> >> >> >> reports the following: >> - >> Microsoft Windows [Version 10.0.16251.1002] >> (c) 2017 Microsoft Corporation. All rights reserved. >> >> C:\Users\Adam>python >> Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:14:34) [MSC v.1900 32 bit >> (Intel)] on win32 >> Type "help", "copyright", "credits" or "license" for more information. >> >>> import math >> >>> math.sqrt(1.3) >> Traceback (most recent call last): >> File "", line 1, in >> ValueError: math domain error >> >>> >> >> I upgraded from version 3.6.1 to 3.6.2 to try to resolve the issue and >> restarted my computer but it is still occurring. Some numbers are >> working (1.2, 1.4) and some others are also not working (1.128). >> >> >> Neither installed 64 bit 3.6.2 nor my repository 3.6 32-bit debug build >> reproduce this. If anyone has the python.org 32bit 3.6.1/2 releases >> installed on Windows, please test and report. > > Three people have reported that math.sqrt(1.3) works in 32 bit Python on > 64-bit Windows and no one otherwise. I reported back on SO that the > problem is likely local. Thanks for the responses. Problem is reported on win10 and I see just 2 tests on win7 (third is maybe Terry's but on SO I don't see win version). If I am not wrong (with analyze source code) sqrt is calling function from "libm" which is some kind of msvcrt.dll on windows... (see https://github.com/python/cpython/blob/a0ce375e10b50f7606cb86b072fed7d8cd574fe7/Modules/mathmodule.c#L1183 and https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Lib/ctypes/util.py#L34 ) And with "MSC v. 1900 ..." it seems that "alternative approaches" (see here https://bugs.python.org/issue23606 ) are used. So I would be cautious. PS. BTW on my ubuntu I got this: from ctypes import cdll print(cdll.LoadLibrary("libcrypt.so")) print(cdll.LoadLibrary("libm.so")) ... OSError: /usr/lib/x86_64-linux-gnu/libm.so: invalid ELF header (same with distro's python3, python2 and anaconda 3.6.2) So this test -> https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Lib/ctypes/util.py#L328 has to crash on some environments (see for example: https://github.com/scipy/scipy/pull/5416/files ). And it seems like some test failures are just ignored... -- https://mail.python.org/mailman/listinfo/python-list
Re: doctest random output?
On 8/28/17, Leam Hallwrote: > On 08/28/2017 11:40 AM, Dennis Lee Bieber wrote: > > ... a bunch of good stuff ... > > I'm (re-)learning python and just trying make sure my function works. > Not at the statistical or cryptographic level. :) > > Thanks! > > Leam > -- > https://mail.python.org/mailman/listinfo/python-list > I am not sure what is best practice but I would use sys.exit to propagate failure (for better scripting possibility). For example: if __name__ == "__main__": import doctest import sys sys.exit(doctest.testmod()[0]) But maybe I am wrong and non zero exit status is just for errors in code? --- If you don't need something at scientific level (which is hard, see: https://www.random.org/analysis/ ) you could probably use fact that random sequences are "hard" to compress. For example something like this could help -> >>> import zlib >>> A = [my_thing() for i in range(100)] >>> 50 < len(zlib.compress(bytes(A))) < 70 True But be careful!! Other randint parameters would need other limit values! # for randint(1,6) you have distribution of lengths like this collections.Counter(len(zlib.compress(bytes(random.randint(1,6) for i in range(100 for j in range(10)) Counter({55: 1, 56: 46, 57: 834, 58: 7349, 59: 31035, 60: 42884, 61: 16434, 62: 1397, 63: 20}) # but for randint(1,16) you have distribution like this! collections.Counter(len(zlib.compress(bytes(random.randint(1,16) for i in range(100 for j in range(10)) Counter({71: 4, 72: 412, 73: 11291, 74: 27392, 75: 28293, 76: 29103, 77: 3296, 78: 209}) So maybe it help you, maybe not :) -- https://mail.python.org/mailman/listinfo/python-list
Re: Quiz: Difference between implicit and explicit inheritence
On 8/28/17, Steven D'Apranowrote: > In Python 3, what's the difference between these two classes? > # implicitly inherit from object > class Spam: > ... > > # explicitly inherit from object > class Spam(object): > ... > > If you sense a trick question, you're right :-) > object = int # this could be trick ... class Spam(object): ... dis.dis show difference too, next line is just for "class Spam(object):" version: LOAD_GLOBAL 0 (object) So nor rebinding object nor rebinding __builtin__.object affect "class Spam:" version. -- https://mail.python.org/mailman/listinfo/python-list
Re: Proposed new syntax
On 8/21/17, Chris Angelicowrote: > On Mon, Aug 21, 2017 at 12:19 PM, MRAB wrote: >> On 2017-08-21 03:00, Steve D'Aprano wrote: >>> >>> On Fri, 18 Aug 2017 04:55 pm, Marko Rauhamaa wrote: >>> Is a Python implementation allowed to parallelize or otherwise reorder the evaluation loop? >>> >>> >>> No. >>> >> [snip] >> >> Well, I suppose an implementation _could_ parallelise, or whatever, >> _provided that_ it gave the same result. > > In other words, it's allowed to parallelise, just as long as > everything happens sequentially. With arbitrary expressions, one of > them could affect another easily. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > import multiprocessing def square(param): ret = param[0] * param[0] param[1].put("%d is done" % param[0]) return ret pool = multiprocessing.Pool(processes=3) m = multiprocessing.Manager() q = m.Queue() squares = pool.map(square, ((i, q) for i in range(10))) print(squares) # ordered print([q.get() for i in range(q.qsize())]) # unordered [0, 1, 4, 9, 16, 25, 36, 49, 64, 81] ['0 is done', '2 is done', '1 is done', '3 is done', '4 is done', '5 is done', '7 is done', '6 is done', '8 is done', '9 is done'] You could collect result sequentially (ordered) but calculate it parallel (unordered). -- https://mail.python.org/mailman/listinfo/python-list
Re: Dataframe iterating question : 3 ways of calling a row and column
On 8/21/17, zach.sm...@orthofi.comwrote: > I wouldn't say I'm a Python noob, but I wouldn't say I'm a Python expert > either. I work in data science and use Pandas Dataframes a lot. My question > is regarding the difference in calling out a specific row, column > combination in a dataframe. > > I see 3 ways of doing this: > (1) df.loc[row_ind, column_ind] > (2) df.column_ind.loc[row_ind] > (3) df[column_ind].loc[row_ind] > > where column_ind is the column name & row_ind is the named row index/row > name in the dataframe. > > Can anyone enlighten me as to the differences between the above 3 methods of > getting to the same cell in the dataframe? > Are there speed differences? > Is it simply a preference thing? > Is there a PEP8 preferred way of doing this? > Are there specific disadvantages to any of the methods? > > Thanks in advance. > Zach First of all I am not expert in pandas or python either. I write just a few thoughts... I don't think PEP-8 is about it. df.column_id is calling __getattr__ where (after some checks) df[column_id] is returned. So it is slower than df[column_id]. But if you are doing things in REPL you could probably be happy to use tab completion to get column name. Or if you are writing paper in (for example) jupyter notebook readability of your formulas could count more than speed! BTW. there are more ways to do it (and I could miss many others) -> (4) df.column_id[row_ind] (5) df.get_value(row_ind, column_ind) (6) df.ix[row_ind, column_ind] interestingly doc - http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.ix.html doesn't say it is deprecated ( http://pandas.pydata.org/pandas-docs/stable/whatsnew.html#deprecate-ix ) Just quick stupid test (python 3.6.2, IPython 6.1.0, pandas 0.20.3) gave me this results (sorted by speed): df = pd.DataFrame(data=[[1, 2, 3], [4, 5, 6]], columns=['A', 'B', 'C']) %timeit df.ix[1,0] # this is deprecated! 188 µs ± 6.55 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each) %timeit df.A.loc[1] 46.2 µs ± 908 ns per loop (mean ± std. dev. of 7 runs, 1 loops each) %timeit df['A'].loc[1] 42.6 µs ± 2 µs per loop (mean ± std. dev. of 7 runs, 1 loops each) aa = df.A %timeit aa[1] 16.6 µs ± 519 ns per loop (mean ± std. dev. of 7 runs, 1 loops each) %timeit df.iloc[1,0] 14.5 µs ± 92.2 ns per loop (mean ± std. dev. of 7 runs, 10 loops each) %timeit df.loc[1,'A'] 13.8 µs ± 251 ns per loop (mean ± std. dev. of 7 runs, 10 loops each) %timeit df.ix[1,'A'] # this is deprecated! 8.68 µs ± 107 ns per loop (mean ± std. dev. of 7 runs, 10 loops each) %timeit df.get_value(1, 'A') 3.51 µs ± 133 ns per loop (mean ± std. dev. of 7 runs, 10 loops each) But I want to add that thinking about speed of getting one value could be thinking in wrong direction because using built in functions could be better. I just test my stupid benchmark (and gave quite opposite result :P) -> %timeit sum(df.sum()) 150 µs ± 2.35 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each) def tst(): summa = 0 for i in range(len(df)): for j in df.columns: summa += df.get_value(i, j) return summa %timeit tst() 37.6 µs ± 1.1 µs per loop (mean ± std. dev. of 7 runs, 1 loops each) But with just a little bigger data frame it is 10x faster using built in function! -> df = pd.DataFrame(data=[[1+3*i, 2+3*i, 3+3*i] for i in range(100)], columns=['A', 'B', 'C']) def tst(): summa = 0 for i in range(len(df)): for j in df.columns: summa += df.get_value(i, j) return summa %timeit tst() 1.67 ms ± 68.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) %timeit sum(df.sum()) 151 µs ± 2.01 µs per loop (mean ± std. dev. of 7 runs, 1 loops each) -- https://mail.python.org/mailman/listinfo/python-list
Re: Proposed new syntax
On 8/17/17, Marko Rauhamaa <ma...@pacujo.net> wrote: > Pavol Lisy <pavol.l...@gmail.com>: > >> On 8/17/17, Gregory Ewing <greg.ew...@canterbury.ac.nz> wrote: >>> I don't agree that the word "for" necessarily implies proceduralness. >> >> With this logic (I humbly think that) word "while" neither >> necessarilly implies proceduralness. > > I don't see the logic. > > Here's a random page from an algebra textbook: > >A sequence {a[n]} is called *bounded* if there is some real number b, >a *bound*, such that |a[n]| ≤ b for all n. > > >Z = {(a) ∈ ℝ**∞ | a[n] = 0 for all but finitely many n } > > On another one: > ><χ[i], χ[j]> = 0 if i ≠ j, and <χ[i], χ[j]> = 1 for each i > > > And: > >For every value a of the variable x, there are at most n points of S >whose x-coordinate is a. > > > Marko > -- > https://mail.python.org/mailman/listinfo/python-list I meant while "implemented" like this (N is set of natural numbers) -> { n ∈ N | condition(m) for all m <= n } or more correctly: { n ∈ N | condition(m) for all m ∈ { o ∈ N | o <= n } } -- https://mail.python.org/mailman/listinfo/python-list
Re: Proposed new syntax
On 8/17/17, Gregory Ewingwrote: > Steve D'Aprano wrote: >> If he wanted declarative semantics, why didn't he argue for declarative >> syntax >> like "select...where", instead of choosing procedural syntax which matches >> the >> actual procedural semantics given? > > I don't agree that the word "for" necessarily implies proceduralness. With this logic (I humbly think that) word "while" neither necessarilly implies proceduralness. -- https://mail.python.org/mailman/listinfo/python-list
Re: Cross-language comparison: function map and similar
On 8/16/17, Steve D'Apranowrote: > Over in another thread, we've been talking about comprehensions and their > similarities and differences from the functional map() operation. > > Reminder: > > map(chr, [65, 66, 67, 68]) > > will return ['A', 'B', 'C']. > > My questions for those who know languages apart from Python: > > Are there language implementations which evaluate the result of map() (or > its > equivalent) in some order other than the obvious left-to-right first-to-last > sequential order? Is that order guaranteed by the language, or is it an > implementation detail? Is it guaranteed in python? Or future version could implement map with something like subscriptability "propagation"? >>>range(1_000_000_000_000_000_000)[-1] >>> map(lambda a:a+1,range(1_000_000_000_000_000_000))[-1] TypeError: 'map' object is not subscriptable -- https://mail.python.org/mailman/listinfo/python-list
Re: Falsey Enums
On 7/28/17, Steve D'Apranowrote: > On Fri, 28 Jul 2017 05:52 pm, Ethan Furman wrote: > >> class X(Enum): >> Falsey = 0 >> Truthy = 1 >> Fakey = 2 >> def __bool__(self): >> return bool(self.value) > > Thanks Ethan. BTW bool at enum seems to be expensive: %timeit 7 if x else 0 850 ns ± 12.9 ns per loop (mean ± std. dev. of 7 runs, 100 loops each) %timeit 7 if x.value else 0 479 ns ± 4.38 ns per loop (mean ± std. dev. of 7 runs, 100 loops each) %timeit 7 if x!=X.Falsey else 0 213 ns ± 10.6 ns per loop (mean ± std. dev. of 7 runs, 100 loops each) > Like Ben, I'm surprised that's not the default behaviour. Me too. I was trying to find some example which is not completely semantically wrong and where backward compatibility is important. Maybe something like this? -> class Color(Enum): black = 0 # this could equal to some external library constant value for black white = 0xff color = user_choice() if not color: # color is None color = random.choice(list(Color)) -- https://mail.python.org/mailman/listinfo/python-list
Re: Write this accumuator in a functional style
On 7/14/17, Steve D'Apranowrote: > On Fri, 14 Jul 2017 09:06 am, Ned Batchelder wrote: > >> Steve's summary is qualitatively right, but a little off on the >> quantitative >> details. Lists don't resize to 2*N, they resize to ~1.125*N: >> >> new_allocated = (size_t)newsize + (newsize >> 3) + (newsize < 9 ? 3 : >> 6); >> >> (https://github.com/python/cpython/blob/master/Objects/listobject.c#L49-L58) > > Ah, thanks for the correction. I was going off vague memories of long-ago > discussion (perhaps even as long ago as Python 1.5!) when Tim Peters (I > think > it was) described how list overallocation worked. You could remember it from sets: return set_table_resize(so, so->used>5 ? so->used*2 : so->used*4); (https://github.com/python/cpython/blob/master/Objects/setobject.c#L239) -- https://mail.python.org/mailman/listinfo/python-list
Re: Write this accumuator in a functional style
On 7/13/17, Steve D'Aprano <steve+pyt...@pearwood.info> wrote: > On Fri, 14 Jul 2017 12:59 am, Pavol Lisy wrote: > >> On 7/13/17, Steve D'Aprano <steve+pyt...@pearwood.info> wrote: >> >>> [1] Actually, CPython's lists initially quadruple the size of the array, >>> up >>> to a >>> certain point, and then switch to doubling. This ensures that small lists >>> have >>> even fewer expensive resizes, at the cost of wasting a bit more memory, >>> but >>> its >>> only a small array so who cares? >> >> IMHO problem is doubling size for huge lists. > > If your list is so huge that a factor of two makes a difference, then you > should > be using a different data structure. Something that doesn't live in memory > all > at once. And by huge, I mean hundreds of millions or billions of items. > > But under normal circumstances, a factor of two is nothing. > > (Unless you're working on an embedded device, where memory is really > constrained. But then you shouldn't be using CPython. Try MicroPython.) > > It's not 1970 where 64K is more memory than anyone can imagine using and we > have > to count every byte of memory. Its 2017 and memory is cheap, we can afford > to > use some memory to speed up our algorithms. Maybe I don't remember it well but we have thread on python-list (or python-ideas) where somebody had problem with memory after update to python3.5 which was following with this issue -> https://bugs.python.org/issue29949 He has enough memory (maybe 64GB or 128GB? my brain memory has limits too) but his models was very slow after upgrade python. (due to swapping) >> Or waste big memory for huge frozensets. I mean resize it to 2*N if >> its size is just N+1. > > Frozensets aren't mutable, so they never resize. Yes so it seems to be easy to allocate just precisely amount of memory. But they are created by using same quadrupling/doubling machinery as you can see: from sys import getsizeof print( [getsizeof(frozenset(range(n))) for n in range(20)] ) [224, 224, 224, 224, 224, 736, 736, 736, 736, 736, 736, 736, 736, 736, 736, 736, 736, 736, 736, 736] I was thinking that this could be good starting point to try to make some patch. Is there any tutorial for dummy beginner how to start to be good at cpython patchwork? :) -- https://mail.python.org/mailman/listinfo/python-list
Re: Write this accumuator in a functional style
On 7/13/17, Steve D'Apranowrote: > [1] Actually, CPython's lists initially quadruple the size of the array, up > to a > certain point, and then switch to doubling. This ensures that small lists > have > even fewer expensive resizes, at the cost of wasting a bit more memory, but > its > only a small array so who cares? IMHO problem is doubling size for huge lists. Or waste big memory for huge frozensets. I mean resize it to 2*N if its size is just N+1. -- https://mail.python.org/mailman/listinfo/python-list
Re: Compiling Python 3.6.1 on macOS 10.12.5
On 7/10/17, Nigel Palmerwrote: > Hi > > I am trying to compile Python 3.6.1 on macOS 10.12.5 with xcode 8.8.3 using > the instructions at > https://docs.python.org/devguide/setup.html#build-dependencies but I am > getting the error > > ./python.exe -E -S -m sysconfig --generate-posix-vars ;\ > if test $? -ne 0 ; then \ > echo "generate-posix-vars failed" ; \ > rm -f ./pybuilddir.txt ; \ > exit 1 ; \ > fi > /bin/sh: line 1: 96973 Killed: 9 ./python.exe -E -S -m > sysconfig --generate-posix-vars > generate-posix-vars failed > make: *** [pybuilddir.txt] Error 1 > > When I manually run that command using dbll I get: > > lldb ./python.exe -- -E -S -m sysconfig --generate-posix-vars > (lldb) target create "./python.exe" > Current executable set to './python.exe' (x86_64). > (lldb) settings set -- target.run-args "-E" "-S" "-m" "sysconfig" > "--generate-posix-vars" > (lldb) r > Process 96978 launched: './python.exe' (x86_64) > Could not find platform dependent libraries > Consider setting $PYTHONHOME to [:] > Process 96978 exited with status = 0 (0x) > (lldb) > > > The commands I ran to configure and build python are: > brew install openssl xz > CPPFLAGS="-I$(brew --prefix openssl)/include" LDFLAGS="-L$(brew --prefix > openssl)/lib" ./configure --prefix=`pwd`/../build > make > > Any ideas on what I am doing wrong? > > Many Thanks > Nigel > -- > https://mail.python.org/mailman/listinfo/python-list > Something like this is working? python -c "from distutils.sysconfig import get_config_var as gv; px = 'prefix' ;epx = 'exec_prefix' ;LIBDEST = 'LIBDEST' ;LD = 'LIBDIR' ;print(f'LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:{gv(LD)} PYTHONPATH=\$PYTHONPATH:{gv(LIBDEST)} PYTHONHOME=\$PYTHONHOME:{gv(px)}:{gv(epx)} ')" or maybe python.exe in your case -> python.exe -c "from distutils.sysconfig import get_config_var as gv; px = 'prefix' ;epx = 'exec_prefix' ;LIBDEST = 'LIBDEST' ;LD = 'LIBDIR' ;print(f'LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:{gv(LD)} PYTHONPATH=\$PYTHONPATH:{gv(LIBDEST)} PYTHONHOME=\$PYTHONHOME:{gv(px)}:{gv(epx)} ')" -- https://mail.python.org/mailman/listinfo/python-list
Re: Write this accumuator in a functional style
On 7/11/17, Steven D'Apranowrote: > I have a colleague who is allergic to mutating data structures. Yeah, I > know, he needs to just HTFU but I thought I'd humour him. > > Suppose I have an iterator that yields named tuples: > > Parrot(colour='blue', species='Norwegian', status='tired and shagged out') > > and I want to collect them by colour: > > accumulator = {'blue': [], 'green': [], 'red': []} > for parrot in parrots: > accumulator[parrot.colour].append(parrot) > > > That's pretty compact and understandable, but it require mutating a bunch > of pre-allocated lists inside an accumulator. Can we re-write this in a > functional style? > > The obvious answer is "put it inside a function, then pretend it works by > magic" but my colleague's reply to that is "Yes, but I'll know that its > actually doing mutation inside the function". > > > Help me humour my colleague. This seems to be philosophical question to me: How to (or "what colleague could accept" or "what do you mean by") collect elements if not adding them to lists? -- https://mail.python.org/mailman/listinfo/python-list
Re: ezdxf type of spline
It seems to be: http://pythonhosted.org/ezdxf/entities.html?highlight=spline#Spline -> https://knowledge.autodesk.com/support/autocad/learn-explore/caas/CloudHelp/cloudhelp/2015/ENU/AutoCAD-Core/files/GUID-58316136-30EB-499C-ACAD-31D0C653B2B2-htm.html -> https://en.wikipedia.org/wiki/Non-uniform_rational_B-spline On 7/10/17, amka1...@gmail.comwrote: > Hi, > > Can someone says please to me which kind are the splines of the ezdxf python > module ? Is it bezier curves ? > > Thanks, > > dylan > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Check Python version from inside script? Run Pythons script in v2 compatibility mode?
On 7/7/17, Steve D'Apranowrote: > import sys > if sys.version_info >= (3,): # the comma is important > print("version 3") But be careful inside script! It could live long enough to see python4 :) -- https://mail.python.org/mailman/listinfo/python-list
Re: How to write raw strings to Python
On 7/5/17, Binary Boywrote: > On Wed, 05 Jul 2017 20:37:38 +0300, Jussi Piitulainen wrote: >> Sam Chats writes: >> >> > On Wednesday, July 5, 2017 at 9:09:18 PM UTC+5:30, Grant Edwards wrote: >> >> On 2017-07-05, Sam Chats wrote: >> >> >> >> > I want to write, say, 'hello\tworld' as-is to a file, but doing >> >> > f.write('hello\tworld') makes the file look like: >> >> [...] >> >> > How can I fix this? >> >> >> >> That depends on what you mean by "as-is". >> >> >> >> Seriously. >> >> >> >> Do you want the single quotes in the file? Do you want the backslash >> >> and 't' character in the file? >> >> >> >> When you post a question like this it helps immensely to provide an >> >> example of the output you desire. >> > >> > I would add to add the following couple lines to a file: >> > >> > for i in range(5): >> > print('Hello\tWorld') >> > >> > Consider the leading whitespace to be a tab. >> >> import sys >> >> lines = r''' >> for line in range(5): >> print('hello\tworld') >> ''' >> >> print(lines.strip()) >> >> sys.stdout.write(lines.strip()) >> sys.stdout.write('\n') > > Thanks! But will this work if I already have a string through a string > variable, rather than using it directly linke you did (by declaring the > lines variable)? > And, will this work while writing to files? > > Sam If I understand you well then no. >>> a = '%s' % 'a\tb' # we have string with tab (similar as we expect from >>> NNTP server?) >>> print(a) # this is not what you like to have in file a b >>> print(repr(a)) # maybe this is conversion you need 'a\tb' >>> print(repr(a)[1:-1]) # or maybe this a\tb -- https://mail.python.org/mailman/listinfo/python-list
Re: how to make this situation return this result?
On 7/1/17, Ho Yeung Leewrote: > just want to compare tuples like index (0,1), (0,2), (1,2) without > duplicate > such as (2,0), (1,0) etc > > > On Saturday, July 1, 2017 at 7:00:17 PM UTC+8, Peter Otten wrote: >> Ho Yeung Lee wrote: >> >> > finally i searched dict.values()[index] solved this >> >> That doesn't look like a good solution to anything -- including "this", >> whatever it may be ;) >> >> If you make an effort to better explain your problem in plain english >> rather >> than with code examples you are likely tho get better answers. > > -- > https://mail.python.org/mailman/listinfo/python-list > [(i, j) for j in range(3) for i in range(j)] # is this good for you? -- https://mail.python.org/mailman/listinfo/python-list
Re: DJANGO cannot import name _compare_digest
On 7/1/17, Thomas Jollans <t...@tjol.eu> wrote: > On 30/06/17 13:32, Pavol Lisy wrote: >> [snip] >> >> python 3.6.1 works as I expected >> >>>>> import logging as operator >>>>> from operator import _compare_digest as compare_digest >> Traceback (most recent call last): >> File "", line 1, in >> ImportError: cannot import name '_compare_digest' >> > > All you're seeing here is that operator._compare_digest doesn't exist in > python3. You are right. I am sorry that I overlook this. -- https://mail.python.org/mailman/listinfo/python-list
Re: DJANGO cannot import name _compare_digest
On 6/28/17, Xristos Xristoouwrote: > i dont have 'operator.py' in my system only 'test_operator.py' and > 'fix_operator.py' > > if : > import sys > print sys.modules['operator'] > > i get this : > > that say me > > how to rename it? > -- > https://mail.python.org/mailman/listinfo/python-list Something interesting here! I just tried to simulate your problems: python 2.7.12 >>> import logging as operator # I want to mask operator module >>> from operator import _compare_digest as compare_digest # WHAT? >>> operator.__name__ # OK 'logging' >>> operator._compare_digest # OK Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute '_compare_digest' >>> compare_digest.__name__ # ? '_compare_digest' >>> compare_digest.__class__ # ? >>> compare_digest.__module__ # ? 'operator' >>> operator.log.__module__ # OK 'logging' python 3.6.1 works as I expected >>> import logging as operator >>> from operator import _compare_digest as compare_digest Traceback (most recent call last): File "", line 1, in ImportError: cannot import name '_compare_digest' -- https://mail.python.org/mailman/listinfo/python-list
Re: Converting epoch to string in format yyyy-mm-dd, or maybe it is not necessary
(I am not very familiar with panda too!) In case of his data he needs to set unit to 's' df['dt'] = pd.to_datetime(df.epoch,unit='s') It return utc time from epoch, so maybe this is what he could use -> df['dt'] = df.epoch.apply(time.ctime) or something like (if he needs strings) -> df['dt'] = df.epoch.apply(lambda a:time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(a)) df['dt'] = df.epoch.apply(lambda a:time.strftime('%Y-%m-%d %H:%M:%S', time.gmtime(a)) # if utc is desired On 6/15/17, Andre Müllerwrote: > I'm not familar with pandas. > > If you look on stackoverfolow you'll find this solution: > > df.epoch = pd.to_datetime(df.epoch) > https://stackoverflow.com/questions/17134716/convert-dataframe-column-type-from-string-to-datetime > > But in this case, it's not a plain string, then it's a datetime object. > > Greetings > Andre > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: ensurepip
$ python3 -m ensurepip /usr/bin/python3: No module named ensurepip But maybe this help to understand: $ python -m ensurepip ensurepip is disabled in Debian/Ubuntu for the system python. Python modules For the system python are usually handled by dpkg and apt-get. apt-get install python- Install the python-pip package to use pip itself. Using pip together with the system python might have unexpected results for any system installed module, so use it on your own risk, or make sure to only use it in virtual environments. On 6/14/17, Steven D'Apranowrote: > ensurepip is added in Python 3.4: > > https://docs.python.org/3/library/ensurepip.html > > > But: > > root@runes:~# python3.4 -m ensurepip > /usr/bin/python3.4: No module named ensurepip > > > > Any clues on what is going on or how to diagnose this? > > > > > -- > Steve > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Transitioning from Linux to Windows
If I understand well then you like to have simple quick solution without need to learn much. And that you don't need to support big traffic... Maybe cherrypy ( http://cherrypy.org/ ) is what you like to look at. Probably this is good enough on your ubuntu: sudo apt-get install python3-cherrypy3 You could simply run: python3 cherry_test.py # source code bellow and open web page http://localhost:8080/ in your browser. cherry_test.py import string import os import os.path import cherrypy from cherrypy.lib import static localDir = os.path.dirname(__file__) absDir = os.path.join(os.getcwd(), localDir) class StringGenerator(object): @cherrypy.expose def index(self): return """ Give it now! """ @cherrypy.expose def generate(self, filename='test.txt'): with open(filename, 'w') as f: f.write('test') # generate simple output file path = os.path.join(absDir, filename) return static.serve_file(path, 'application/x-download', 'attachment', os.path.basename(path)) if __name__ == '__main__': cherrypy.quickstart(StringGenerator()) ### Simple isn't it? But don't forget to think about security! :) On 6/3/17, chitt...@uah.eduwrote: > I am looking for suggestions, ideas. > > I have developed python (3.6.x, 2.7.x) scripts that run well as a user on an > ubuntu/16.04 system - the scripts look for files, parses the files, > assembles an output for the user. > > I first cd into a particular directory on the system (where I know the files > exist) and run the script - the final result is in my working directory. > > What I am now trying to do is ... figure out the least painful way to have > other users do the same - BUT sitting in front of a Windows desktop (7 or > 10). > > Ideally, I would like to set up the user on their Windows 7/10 system so > that they can "login" to the ubuntu system (say putty) - change working > directory (to where desired) - run the script (on the ubuntu system) - and > scp the file back to the windows desktop. > > ("porting" the ubuntu script to anaconda(3) on the windows desktop IS > possible (but it has not been as easy as I had hoped!) (django and programs > like that do seem to provide a "GUI" to have python scripts run on > ubuntu/systems - but the setup looks mysterious/complicated (to me anyway)) > > I stumbled onto "paramiko" - is that a possible answer? > > Any suggestion/ideas would be greatly appreciated! > > (I am perfectly willing to stick to 3.6.x, unless there is a clean/easy > solution using 2.7.x) > > krishnan > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Python DB API - commit() v. execute("commit transaction")?
On 5/31/17, Jon Ribbens <jon+use...@unequivocal.eu> wrote: > On 2017-05-31, Pavol Lisy <pavol.l...@gmail.com> wrote: >> But althoug return from execute is undefined ( >> https://www.python.org/dev/peps/pep-0249/#id16 ), you could iterate >> over cursor ( https://www.python.org/dev/peps/pep-0249/#iter ) > > ... which is also optional. Thanks, I missed that. BTW these optional extensions were added to PEP 15 years ago ( https://github.com/python/peps/commit/cdb77a562cccd2da35bc1e795cf54a8c1916a08d ). Aren't they de facto standard now? -- https://mail.python.org/mailman/listinfo/python-list
Re: Python DB API - commit() v. execute("commit transaction")?
On 5/31/17, Peter Otten <__pete...@web.de> wrote: > Jon Ribbens wrote: > >> On 2017-05-31, Skip Montanarowrote: >>> I'm kind of stuck with the database API I have. ("Love the child you >>> have, not the one you wish you had?") Given that I have the choice to >>> execute those three statements to bound a transaction, is there any >>> reason not to use them instead of >>> >>> (conn or cur).execute("begin transaction") >>> conn.commit() >>> conn.rollback() >> >> I very much doubt it. >> >>> I must say this discussion has been enlightening for me. I'm not a >>> database guy, so for better or worse, my view of how to interact with >>> relational databases has always been colored by the Python database >>> adapters. >> >> Indeed. This is where I was coming from too, and when I moved beyond >> that point I got very confused because it turned out that Python had >> seriously misled me. >> >>> cur1 = conn.cursor() >>> cur2 = conn.cursor() >>> >>> for row in cur1.execute("select something ..."): >>> tidbit = ... pick something of interest out of the row ... >>> for new_row in cur2.execute("select something else involving the >>> tidbit ..."): >>> ... process new_row ... >> >> Well, you can't do that, because you can't iterate over an execute(). > > You can, if the database is sqlite3. cx_Oracle too. But althoug return from execute is undefined ( https://www.python.org/dev/peps/pep-0249/#id16 ), you could iterate over cursor ( https://www.python.org/dev/peps/pep-0249/#iter ) >> You would do: >> >> cur.execute("SELECT ...") >> for row1 in cur.fetchall(): >> cur.execute("SELECT ...") >> for row2 in cur.fetchall(): >> >> >> and as far as the database is concerned, the query is over and done >> with as soon as you call fetchall(), so there is no possible overlap >> between the different queries. > > It's easy to simulate an iterable cursor with > > iter(cursor.fetchone, None) > > and then two cursors instead of one start making sense again. fetchall could be disaster for memory! -- https://mail.python.org/mailman/listinfo/python-list
Re: Python not able to find package but it is installed
On 5/31/17, Dennis Lee Bieberwrote: > On Tue, 30 May 2017 19:27:48 + (UTC), Mahmood Naderan via Python-list > declaimed the following: > >># trying /usr/lib/python2.6/site-packages/openpyxl.so >># trying /usr/lib/python2.6/site-packages/openpyxlmodule.so >># trying /usr/lib/python2.6/site-packages/openpyxl.py >># trying /usr/lib/python2.6/site-packages/openpyxl.pyc >>Traceback (most recent call last): >>File "", line 1, in >>ImportError: No module named openpyxl >> >> >> >> >>But >> >>$ find /opt -name openpyxl >>/opt/rocks/lib/python2.6/site-packages/openpyxl > > > Which tells us that the running python is not configured to LOOK in > /opt/rocks/... > > That means you need to modify the pythonpath environment to add your > directory... In case there are two python environment it could create mess. Maybe problem started by adding /opt/rocks/bin into PATH manually? If it is true I rather remove it, then and then . -- https://mail.python.org/mailman/listinfo/python-list
Re: Python not able to find package but it is installed
What is output from $ pip ---version $ easy_install --version ? Maybe you have two pythons and only one pip? $ find /opt -name python $ find /usr -name python $ find /opt -name pip $ find /usr -name pip On 5/30/17, Mahmood Naderan via Python-listwrote: > Well yes. It looks in other folders > import openpyxl > # trying openpyxl.so > # trying openpyxlmodule.so > # trying openpyxl.py > # trying openpyxl.pyc > # trying /usr/lib64/python2.6/openpyxl.so > # trying /usr/lib64/python2.6/openpyxlmodule.so > # trying /usr/lib64/python2.6/openpyxl.py > # trying /usr/lib64/python2.6/openpyxl.pyc > # trying /usr/lib64/python2.6/plat-linux2/openpyxl.so > # trying /usr/lib64/python2.6/plat-linux2/openpyxlmodule.so > # trying /usr/lib64/python2.6/plat-linux2/openpyxl.py > # trying /usr/lib64/python2.6/plat-linux2/openpyxl.pyc > # trying /usr/lib64/python2.6/lib-dynload/openpyxl.so > # trying /usr/lib64/python2.6/lib-dynload/openpyxlmodule.so > # trying /usr/lib64/python2.6/lib-dynload/openpyxl.py > # trying /usr/lib64/python2.6/lib-dynload/openpyxl.pyc > # trying /usr/lib64/python2.6/site-packages/openpyxl.so > # trying /usr/lib64/python2.6/site-packages/openpyxlmodule.so > # trying /usr/lib64/python2.6/site-packages/openpyxl.py > # trying /usr/lib64/python2.6/site-packages/openpyxl.pyc > # trying /usr/lib64/python2.6/site-packages/gtk-2.0/openpyxl.so > # trying /usr/lib64/python2.6/site-packages/gtk-2.0/openpyxlmodule.so > # trying /usr/lib64/python2.6/site-packages/gtk-2.0/openpyxl.py > # trying /usr/lib64/python2.6/site-packages/gtk-2.0/openpyxl.pyc > # trying /usr/lib/python2.6/site-packages/openpyxl.so > # trying /usr/lib/python2.6/site-packages/openpyxlmodule.so > # trying /usr/lib/python2.6/site-packages/openpyxl.py > # trying /usr/lib/python2.6/site-packages/openpyxl.pyc > Traceback (most recent call last): > File "", line 1, in > ImportError: No module named openpyxl > > > > > But > > $ find /opt -name openpyxl > /opt/rocks/lib/python2.6/site-packages/openpyxl > > > > > Regards, > Mahmood > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: How to install Python package from source on Windows
On 5/15/17, Chris Angelicowrote: > Perhaps a future version of gcc will be implemented in Python. Would > you then say that Python is beholden to no one? You would still need > to have a C compiler installed in order to compile extension modules. > It's too big to include with every single Python installation. So > you'd have the same situation even then. Don't forget that 640 kB was too big too! ;) (and compiler as service is also possibility) PL. -- https://mail.python.org/mailman/listinfo/python-list
Re: help with an error msg please
On 5/14/17, Pavol Lisy <pavol.l...@gmail.com> wrote: > On 5/14/17, Charles T. Smith <cts.private.ya...@gmail.com> wrote: >> I'm stumped by this: > >> $ PYTHONPATH= python except >> Traceback (most recent call last): >> File "except", line 7, in >> except getopt.error, msg: >> AttributeError: 'module' object has no attribute 'error' >> >> >> The program is: >> >> $ cat except >> #!/usr/bin/env python >> >> import getopt >> >> try: >> opts, args = getopt.getopt (sys.argv[1:], "t:") >> except getopt.error, msg: >> raise "Usage: some other way", msg >> >> what am I doing wrong? > > Did you create getopt.py in your working directory? If so then try to > rename it. > > PL. And remove getopt.pyc! PL. -- https://mail.python.org/mailman/listinfo/python-list
Re: help with an error msg please
On 5/14/17, Charles T. Smithwrote: > I'm stumped by this: > $ PYTHONPATH= python except > Traceback (most recent call last): > File "except", line 7, in > except getopt.error, msg: > AttributeError: 'module' object has no attribute 'error' > > > The program is: > > $ cat except > #!/usr/bin/env python > > import getopt > > try: > opts, args = getopt.getopt (sys.argv[1:], "t:") > except getopt.error, msg: > raise "Usage: some other way", msg > > what am I doing wrong? Did you create getopt.py in your working directory? If so then try to rename it. PL. -- https://mail.python.org/mailman/listinfo/python-list
Re: Out of memory while reading excel file
On 5/11/17, Peter Otten <__pete...@web.de> wrote: > Mahmood Naderan via Python-list wrote: > >> Excuse me, I changed >> >> csv.writer(outstream) >> >> to >> >> csv.writer(outstream, delimiter =' ') >> >> >> It puts space between cells and omits "" around some content. > > If your data doesn't contain any spaces that's fine. Otherwise you need a > way to distinguish between space as a delimiter and space inside a field, e. > > g. by escaping it: > w = csv.writer(sys.stdout, delimiter=" ", quoting=csv.QUOTE_NONE, > escapechar="\\") w.writerow(["a", "b c"]) > a b\ c > 8 > >> However, >> between two lines there is a new empty line. In other word, the first >> line >> is the first row of excel file. The second line is empty ("\n") and the >> third line is the second row of the excel file. >> >> Any thought? > > In text mode Windows translates "\n" to b"\r\n" in the file. Python allows > you to override that: > help(open) > Help on built-in function open in module io: > > open(...) > open(file, mode='r', buffering=-1, encoding=None, > errors=None, newline=None, closefd=True, opener=None) -> file > object > > > > newline controls how universal newlines works (it only applies to text > mode). It can be None, '', '\n', '\r', and '\r\n'. It works as > follows: > > > > * On output, if newline is None, any '\n' characters written are > translated to the system default line separator, os.linesep. If > newline is '' or '\n', no translation takes place. If newline is any > of the other legal values, any '\n' characters written are translated > to the given string. > > So you need to specify newlines: > > with open(dest, "w", newline="") as outstream: > ... > But lineterminator parameter ( https://docs.python.org/3.6/library/csv.html#csv.Dialect.lineterminator ) is by default \r\n on linux too! b = io.StringIO() w = csv.writer(b) w.writerows([["a", "b c"], ['a', 'b,c']]) b.getvalue() # 'a,b c\r\na,"b,c"\r\n' b = io.StringIO() w = csv.writer(b, lineterminator='\n') w.writerows([["a", "b c"], ['a', 'b,c']]) b.getvalue() # 'a,b c\na,"b,c"\n' PL. -- https://mail.python.org/mailman/listinfo/python-list
Re: import docx error
On 5/11/17, Grant Edwardswrote: > On Wed, May 10, 2017 at 05:14:22PM -0700, somebody wrote: >> I have downloaded Anaconda to Cinnamon Mint 18.1 64 bit where Python >> 3.6 exists. >> >> It will not start up. > > The anaconda that I know about is the RedHat installer program (which > was originally written in Python, BTW), but I'm guessing that's not > what you're asking about. > >> My naive question is: When I go to pypi for example, am I to download >> packages into python or Mint? > > I don't understand the question: python is a language, Mint is a Linux > OS distro. If you can't use your distro's package manager to install > the package you're looking for (see below), then here is how you > install a package from pypi: > > https://packaging.python.org/installing/#installing-from-pypi Under linux you could use anaconda python and distro's python side by side. If you use default installation process you could get anaconda probably in $HOME/anaconda directory. If you don't change .bashrc then you could start anaconda's virtual environment by: source $HOME/anaconda/bin/activate ~/anaconda/ After this command pip will install packages into anaconda's python environment. (without this command it install into distro's python environment) So answer to "somebody's" question is probably that it depends on. Under ubuntu 16.04 which could be similar to Mint I got this: python -V # by default my python is distro's python Python 2.7.12 source $HOME/anaconda3/bin/activate anaconda3 (anaconda3) xyz:~$ python -V # now I could use nice features like f-strings from python 3.6 :) Python 3.6.1 :: Anaconda custom (64-bit) PL. -- https://mail.python.org/mailman/listinfo/python-list
Re: import docx error
On 5/10/17, Grant Edwardswrote: > On 2017-05-10, RRS1 via Python-list wrote: > >> I am very new to Python, have only done simple things >>>print("hello >> world") type things. I've really been looking forward to using Python. I >> bought Two books, downloaded Python 3.6.1 (32 & 64) and each time I try >> this: >> >> > import docx >> >> I get errors. >> >> Traceback (most recent call last): >> File "", line 1 in >> ModuleNotFoundError: No module named docx > > You need to install the docx module: > > https://pypi.python.org/pypi/docx > https://pypi.python.org/pypi I am afraid https://pypi.python.org/pypi/python-docx could be what he needs. Using anaconda it could be better to do: conda install python-docx # but this doesnt work or conda install docx # but this doesnt work too Anaconda has channels. For example cjs14 channel includes docx. But unfortunately it is only for python2 :( conda install -c cjs14 python-docx UnsatisfiableError: The following specifications were found to be in conflict: - python 3.6* - python-docx -> python 2.7* -> openssl 1.0.1* PL. -- https://mail.python.org/mailman/listinfo/python-list
[issue30024] Treat `import a.b.c as m` as `m = sys.modules['a.b.c']`
Pavol Lisy added the comment: Maybe I am wrong but don't we get another weird behavior? import sys.path # this is error now ModuleNotFoundError: No module named 'sys.path'; 'sys' is not a package So we probably expect same behavior here: import sys.path as foo But if we just replace LOAD_ATTR with IMPORT_FROM in the compiler then wouldn't it works like: from sys import path as foo ? -- nosy: +Pavol Lisy ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30024> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Program uses twice as much memory in Python 3.6 than in Python 3.5
On 3/30/17, INADA Naokiwrote: > Maybe, this commit make this regression. > > https://github.com/python/cpython/commit/4897300276d870f99459c82b937f0ac22450f0b6 > > Old: > minused = (so->used + other->used)*2 (L619) > > New: > minused = so->used + other->used (L620) > minused = (minused > 5) ? minused * 2 : minused * 4; (L293) > > So size of small set is doubled. > > $ /usr/bin/python3 > Python 3.5.2+ (default, Sep 22 2016, 12:18:14) > [GCC 6.2.0 20160927] on linux > Type "help", "copyright", "credits" or "license" for more information. import sys s = set(range(10)) sys.getsizeof(frozenset(s)) > 736 > > $ python3 > Python 3.6.0 (default, Dec 30 2016, 20:49:54) > [GCC 6.2.0 20161005] on linux > Type "help", "copyright", "credits" or "license" for more information. import sys s = set(range(10)) sys.getsizeof(frozenset(s)) > 1248 > > > > On Fri, Mar 31, 2017 at 2:34 AM, INADA Naoki > wrote: >> I reproduced the issue. >> This is very usual, memory usage issue. Slashing is just a result of >> large memory usage. >> >> After 1st pass of optimization, RAM usage is 20GB+ on Python 3.5 and >> 30GB on Python 3.6. >> And Python 3.6 starts slashing in 2nd optimization pass. >> >> I enabled tracemalloc while 1st pass. Results is end of this mail. >> It seems frozenset() cause regression, but I'm not sure yet. >> I don't know what is contents of frozenset yet. (I know almost >> nothing about this application). >> >> Jan, do you know about what this is? >> Could you make script which just runs `transitive_closure(edges)` with >> edges similar to >> `log_reduction.py spaun`? >> >> I'll dig into it later, maybe next week. >> >> --- >> Python 3.6.1 >> 1191896 memory blocks: 22086104.2 KiB >> File >> "/home/inada-n/work/gosmann-frontiers2017/gosmann_frontiers2017/optimized/optimizer.py", >> line 85 >> reachables[vertex] = frozenset(reachables[vertex]) >> File >> "/home/inada-n/work/gosmann-frontiers2017/gosmann_frontiers2017/optimized/optimizer.py", >> line 410 >> self.dependents = transitive_closure(self.dg.forward) >> 602986 memory blocks: 51819.1 KiB >> File "", line 14 >> File >> "/home/inada-n/work/gosmann-frontiers2017/gosmann_frontiers2017/optimized/optimizer.py", >> line 634 >> first_view=None, v_offset=0, v_size=0, v_base=None) >> >> Python 3.5.3 >> 1166804 memory blocks: 6407.0 KiB >> File >> "/home/inada-n/work/gosmann-frontiers2017/gosmann_frontiers2017/optimized/optimizer.py", >> line 85 >> reachables[vertex] = frozenset(reachables[vertex]) >> File >> "/home/inada-n/work/gosmann-frontiers2017/gosmann_frontiers2017/optimized/optimizer.py", >> line 410 >> self.dependents = transitive_closure(self.dg.forward) >> 602989 memory blocks: 51819.3 KiB >> File "", line 14 >> File >> "/home/inada-n/work/gosmann-frontiers2017/gosmann_frontiers2017/optimized/optimizer.py", >> line 634 >> first_view=None, v_offset=0, v_size=0, v_base=None) > -- > https://mail.python.org/mailman/listinfo/python-list > Interesting. 1. using set_table_resize with growing factor 2 or 4 doesn't seem to be optimal for constructing (immutable) frozenset from set. 2. growth factor 2 could be too big too ( https://en.wikipedia.org/wiki/Dynamic_array#Growth_factor ,https://github.com/python/cpython/blob/80ec8364f15857c405ef0ecb1e758c8fc6b332f7/Objects/listobject.c#L58 ) PL. -- https://mail.python.org/mailman/listinfo/python-list
Re: Program uses twice as much memory in Python 3.6 than in Python 3.5
On 3/29/17, Jan Gosmannwrote: > On 28 Mar 2017, at 14:21, INADA Naoki wrote: > >> On Wed, Mar 29, 2017 at 12:29 AM, Jan Gosmann >> wrote: >> >> I suppose smaller and faster benchmark is better to others looking for >> it. >> I already stopped the azure instance. >> [...] >> There are no maxrss difference in "smaller existing examples"? >> [...] >> I want to investigate RAM usage, without any swapping. > > Running further trials indicate that the problem actually is related to > swapping. If I reduce the model size in the benchmark slightly so that > everything fits into the main memory, the problem disappears. Only when > the memory usage exceeds the 32GB that I have, Python 3.6 will acquire > way more memory (from the swap) than Python 3.5. > > Jan > -- > https://mail.python.org/mailman/listinfo/python-list Could you add table comparing time benchmarks when memory is bigger? (if your hypothesis is true and memory measurement tools are right than time difference has to be huge) Did you compare "pip list" results? There could be more differences in your environments (not only python version). For example different numpy versions or some missing packages could change game. I tried to search "except.*ImportError" in your repository, but I am not sure that it could change it significantly... ( https://github.com/ctn-archive/gosmann-frontiers2017/search?utf8=%E2%9C%93=ImportError= This one seems suspitious - sparse matrix class could be game changer from scipy.sparse import bsr_matrix assert bsr_matrix except (ValueError, ImportError): return False ) This one doesn't seems suspicious to me (but who knows?): try: import faulthandler faulthandler.enable() except: pass PL. -- https://mail.python.org/mailman/listinfo/python-list
Re: pandas dataframe, find duplicates and add suffix
On 3/28/17, zljubi...@gmail.comwrote: > In dataframe > > import pandas as pd > > data = {'model': ['first', 'first', 'second', 'second', 'second', 'third', > 'third'], > 'dtime': ['2017-01-01_112233', '2017-01-01_112234', > '2017-01-01_112234', '2017-01-01_112234', '2017-01-01_112234', > '2017-01-01_112235', '2017-01-01_112235'], > } > df = pd.DataFrame(data, index = ['a.jpg', 'b.jpg', 'c.jpg', 'd.jpg', > 'e.jpg', 'f.jpg', 'g.jpg'], columns=['model', 'dtime']) > > print(df.head(10)) > > model dtime > a.jpg first 2017-01-01_112233 > b.jpg first 2017-01-01_112234 > c.jpg second 2017-01-01_112234 > d.jpg second 2017-01-01_112234 > e.jpg second 2017-01-01_112234 > f.jpg third 2017-01-01_112235 > g.jpg third 2017-01-01_112235 > > within model, there are duplicate dtime values. > For example, rows d and e are duplicates of the c row. > Row g is duplicate of the f row. > > For each duplicate (within model) I would like to add suffix (starting from > 1) to the dtime value. Something like this: > > model dtime > a.jpg first 2017-01-01_112233 > b.jpg first 2017-01-01_112234 > c.jpg second 2017-01-01_112234 > d.jpg second 2017-01-01_112234-1 > e.jpg second 2017-01-01_112234-2 > f.jpg third 2017-01-01_112235 > g.jpg third 2017-01-01_112235-1 > > How to do that? > -- > https://mail.python.org/mailman/listinfo/python-list > I am not expert, just played a little... This one could work: gb = df.groupby([df.model, df.dtime]) df.dtime = df.dtime + gb.cumcount().apply(lambda a:str(-a) if a else '') this one is probably more readable: df.dtime = df.dtime + [str(-a) if a else '' for a in gb.cumcount()] I don't know which one is better in memory consumption and/or speed. This small dataframe gave me: %timeit -r 5 df.dtime + gb.cumcount().apply(lambda a:str(-a) if a else '') 1000 loops, best of 5: 387 µs per loop %timeit -r 5 df.dtime + [str(-a) if a else '' for a in gb.cumcount()] 1000 loops, best of 5: 324 µs per loop PL. -- https://mail.python.org/mailman/listinfo/python-list
Re: cross python version randomness
On 3/21/17, Kev Dwyerwrote: > Robin Becker wrote: > >> Is there a way to get the same sequences of random numbers in python 2.7 >> and python >= 3.3? >> >> I notice that this simple script produces different values in python 2.7 >> and >=3.3 >> >> C:\code\hg-repos\reportlab>cat s.py >> import sys, random >> print(sys.version) >> random.seed(103) >> for i in range(5): >> print(i, random.randint(10,25)) >> >> C:\code\hg-repos\reportlab>\python27\python s.py >> 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:53:40) [MSC v.1500 64 bit >> (AMD64)] 0 25 >> 1 17 >> 2 21 >> 3 21 >> 4 13 >> >> C:\code\hg-repos\reportlab>\python33\python s.py >> 3.3.5 (v3.3.5:62cf4e77f785, Mar 9 2014, 10:35:05) [MSC v.1600 64 bit >> (AMD64)] 0 24 >> 1 16 >> 2 12 >> 3 13 >> 4 22 >> >> However, when I use random.random() all seems to be the same so this >> script C:\code\hg-repos\reportlab>cat u.py >> import sys, random >> print(sys.version) >> random.seed(103) >> for i in range(5): >> print(i, random.random()) >> >> seems to be fine >> >> >> C:\code\hg-repos\reportlab>\python27\python u.py >> 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:53:40) [MSC v.1500 64 bit >> (AMD64)] (0, 0.9790501200727744) >> (1, 0.45629827629184827) >> (2, 0.7188470341002364) >> (3, 0.7348862425853395) >> (4, 0.21490166849706338) >> >> C:\code\hg-repos\reportlab>\python33\python u.py >> 3.3.5 (v3.3.5:62cf4e77f785, Mar 9 2014, 10:35:05) [MSC v.1600 64 bit >> (AMD64)] 0 0.9790501200727744 >> 1 0.45629827629184827 >> 2 0.7188470341002364 >> 3 0.7348862425853395 >> 4 0.21490166849706338 >> >> presumably randint is doing something different to get its values. > > > The docs [https://docs.python.org/3/library/random.html#random.randrange] > for randrange have this note: > > Changed in version 3.2: randrange() is more sophisticated about producing > equally distributed values. Formerly it used a style like int(random()*n) > which could produce slightly uneven distributions. > > Maybe that's the explanation? Unfortunately I don't have an install of > 3.0/1 to test against. > > -- > https://mail.python.org/mailman/listinfo/python-list > This is inspiring Robin! :) import sys, random print(sys.version) def randint(a, b): return int(random.random()*(b-a+1))+a random.seed(103) for i in range(5): print(i, randint(10,25)) 3.6.0 |Anaconda custom (64-bit)| (default, Dec 23 2016, 12:22:00) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] 0 25 1 17 2 21 3 21 4 13 Output seems to be same as in your 2.7 version :) So if you are sure that sequences of random.random are same then you could use it. You could also save a sequence to file and use it repeatedly. (if it is not performance problem for you) PL. -- https://mail.python.org/mailman/listinfo/python-list
About linters and PEP-8
On 3/19/17, Chris Angelico <ros...@gmail.com> wrote: > On Sun, Mar 19, 2017 at 4:53 PM, Pavol Lisy <pavol.l...@gmail.com> wrote: >> In case of spaces there is not discrepancy. pep8 and similar linters >> could work fine without incompatible configurations. > I don't like the pep8 tool; its name implies far more authority than > it actually has. But if you look at the rules Python 3 applies, or > Python 2 with "-tt", you should easily be able to comply. There was discussion on python-idea about duplicity in dict definition. {"a": "foo", "a": "bar"} Conclusion was that it is not an error and anybody could use linters for finding it. For example pylint could do it. Modern editors could do linting parallel to editing and many people use it. I think - I couldn't like everything what pep8 or pylint or any others do. But it is some kind of responsibility to write codes, which do not produce plenty of warnings. (because they could hide warning like duplicity in dict) And I think that significance of linters could grow as machine learning will evolve. So if there is something wrong about PEP-8 maybe there is some work for us to improve it. PL. -- https://mail.python.org/mailman/listinfo/python-list
Re: Who are the "spacists"?
On 3/19/17, Pavol Lisy <pavol.l...@gmail.com> wrote: > On 3/18/17, Nathan Ernst <nathan.er...@gmail.com> wrote: > PS. I am "spacist" :P but I feel this a little more aggressive than is > necessary too: > >> Feel free to start your own discussion forum for your new programming >> language that forbids spaces for indentation. That language will never >> be Python, so please don't ask us to discuss it here. > > I think there could be enough time to deprecate now and forbid tabs > (or spaces) as indentation in python4. > > That doesn't mean I propose that! I only think that this transition > could be technically possible so discussion is not necessary useless. Sorry here I forgot to mention that citation above is from Ben Finney's mail and not from Nathan's! Anyway. I just made a simple and primitive analyze: re.findall('^ '...) and re.findall('^\t'...) for more than 100k py-files discoverable by find / -name "*.py" on my (ubuntu 16.04) computer. Anybody could see that it incorrectly finds lines in multiline strings (and has maybe much more flaws) but it could be probably still interesting. there are: 127 837 files where at least one line starts with space 2 273 files where at least one line starts with tab 1 192 files where are lines which starts with space and lines which starts with tab and there are: 35 144 893 lines which starts with space 428 160 lines which starts with tab PL. -- https://mail.python.org/mailman/listinfo/python-list
Re: Who are the "spacists"?
On 3/18/17, Nathan Ernstwrote: > My issue with using spaces instead of tabs, is that, as mentioned earlier > in the thread, everyone has their own preferences on indentation. I've > worked on teams where different developers used 2, 3 & 4 spaces as > indentation. Obviously, if you're using spaces, several of the members will > be unhappy. > > Tabs have the benefit that most editors used by developers allow the > adjustment of the width of a tab. If you use tabs, everyone can be happy > with the visual presentation of their code. > > My rule of thumb: tabs for indentation, spaces for alignment (i.e. trying > to line up anything after a non-whitespace character on a single line). > > Regards, > Nathan Everyone has their own preferences how to display tabs (2, 3, 4 spaces?). How could alignment look in case of mixed tabs and spaces? Everybody will be happy? In case of spaces there is not discrepancy. pep8 and similar linters could work fine without incompatible configurations. PL. PS. I am "spacist" :P but I feel this a little more aggressive than is necessary too: > Feel free to start your own discussion forum for your new programming > language that forbids spaces for indentation. That language will never > be Python, so please don't ask us to discuss it here. I think there could be enough time to deprecate now and forbid tabs (or spaces) as indentation in python4. That doesn't mean I propose that! I only think that this transition could be technically possible so discussion is not necessary useless. -- https://mail.python.org/mailman/listinfo/python-list
Re: python-libxdo?
On 2/28/17, Jimwrote: simplified: > from xdo import Xdo > xdo = Xdo() > win = xdo.search_windows(winname = 'Mozilla Firefox') > File "/home/jfb/EVs/env/lib/python3.5/site-packages/xdo/__init__.py" > TypeError: bytes or integer address expected instead of str instance Example on github seems to use python2 where bytes and str are same type. Because you have python3, you need to write -> win = xdo.search_windows(winname = b'Mozilla Firefox') PS. this reminds me "Talk Python to Me with GvR" thread on this list. :) How to minimize frustration? Could not have ctypes.Structure something like encoding attribute? -- https://mail.python.org/mailman/listinfo/python-list
Re: panda, column access
On 2/27/17, Andrew Zymanwrote: > Hello, > i'm trying to understand what is the difference between the below code. > And how do i access the column's data without hardcoding the column name in > such a way that i'll be able to convert it's values into the list of > strings? > > I appreciate your help. > > Short version: > colnames='hostname' I think you meant: colnames = ['hostname'] > data = pandas.read_csv(input_file, names=colnames) > list_data = data.hostname.tolist() or list_data = data['hostname'].tolist() > returns the list like - ['hostname','hostname1'] - exactly what i want. > but neither of the below versions work as expected (returning the list of > strings): > list_data = data[0].tolist() -> errors with "return > self._getitem_column(key)" > list_data = data[colnames].values.tolist() -> the list_data is : [ > ['hostname'],['hostname1'...] > list_data = data.tolist() -> errors with "AttributeError: 'DataFrame' object > has no attribute 'tolist'" What do you want is probably: >>> data[colnames[0]].tolist() or maybe if you don't want to remember colnames: >>> data[data.columns[0]].tolist() Be careful! Type of data[x] depend on what type is x. >>> type(data['hostname']) # -> pandas.core.series.Series >>> type(data[['hostname']]) # -> pandas.core.frame.DataFrame -- https://mail.python.org/mailman/listinfo/python-list
Re: Spyder 3.1.3 update
On 2/24/17, Pablo Lozanowrote: > Good day, > > I installed the Spyder 3.6 IDE from the Anaconda package and at the start k > to the IDE I get the Spyder Update saying Spyder 3.1.3 is available. Did it > incorrectly install Anaconda? As far as I know Spyder 3.1.3 is rather old > and the latest is 3.6 Spyder is spyder and python is python. If you look at ipython console in your spyder window than you probably see something like: Python 3.6.0 |Anaconda custom (64-bit)| (default, Dec 23 2016, 12:22:00) ... 3.1.3 is latest version of spyder (and not very old (2017-02-20)): https://github.com/spyder-ide/spyder/blob/master/CHANGELOG.md -- https://mail.python.org/mailman/listinfo/python-list
Re: Namedtuples problem
On 2/23/17, Peter Otten <__pete...@web.de> wrote: > Peter Otten wrote: > >> Functions to the rescue: > > On second thought this was still more code than necessary. If we are talking about less code than necessary then probably we could use something from python's ecosystem... >>> import pandas as pd # really useful package >>> import io # I simlulate file in csv with header (like Deborah has) >>> data = """\ what,location foo,here foo,there bar,eberywhere""" >>> df = pd.read_csv(io.StringIO(data)) # reading csv into DataFrame is very >>> simple >>> print(df[df.what == 'foo']) # data where what == foo what location 0 foo here 1 foothere >>> list(df[df.what=='foo'].location.values) # list of values ['here', 'there'] >>> list(df.itertuples()) # if we want to use namedtuples [Pandas(Index=0, what='foo', location='here'), Pandas(Index=1, what='foo', location='there'), Pandas(Index=2, what='bar', location='eberywhere')] etc. -- https://mail.python.org/mailman/listinfo/python-list
Re: python decorator
On 2/22/17, Steve D'Apranowrote: > On Wed, 22 Feb 2017 08:47 pm, Cecil Westerhof wrote: > >> On Wednesday 22 Feb 2017 08:49 CET, Argentinian Black ops lll wrote: >> >>> *** SOLVED *** >> >> It would be nice if you shared the solution. > > I believe Cameron's post contains the bones of a solution. > > Here's my untested solution. > > > def func_cache(cache): > # Create a decorator that uses cache. > def decorate(function): > @functools.wraps(function) > def wrapper(*args): > try: > result = cache[args] > except KeyError: > result = function(*args) > cache[args] = result > except TypeError: > result = function(*args) > return result > return wrapper > return decorate > > > @func_cache({}): > def something(x): > ... Maybe this technique could be reusable (and maybe part of functools?) With this decorator: def wrap_args(decorator): def decor_out(*args, **kwargs): def decor_in(func): return decorator(func, *args, **kwargs) return decor_in return decor_out Alfredo needs to change only (*) next 2 lines: def fun_cache(function): memo = {} to: @wrap_args def fun_cache(function, cache): memo = cache (*) - Steve's improvements (for example using functools.wraps) are good to consider as well! :) (but maybe catching TypeError could more problems hide than solve) -- https://mail.python.org/mailman/listinfo/python-list
Re: Python application launcher (for Python code)
On 2/21/17, Deborah Swansonwrote: > Chris Angelico wrote, on February 21, 2017 7:30 AM >> >> On Wed, Feb 22, 2017 at 2:16 AM, Deborah Swanson >> wrote: >> > Really? We used software called Powershell a couple decades ago in >> > the 90s, as an improvement on the DOS box. I didn't like it much and > I >> > was going through software like candy those days. Maybe that version > >> > disappeared. It may have re-emerged in an improved form a decade >> > later. >> >> With a name like that, there've almost certainly been >> multiple things using it. The PowerShell that comes with >> modern Windowses came out in 2006, and IMO it's less of a >> shell language and more of a scripting language. And if I >> want to use a scripting language for program flow control, >> I'll use Python or Pike, fankyouwerrymuch. >> >> ChrisA > > Yes, the name "Powershell" would have an appeal to a certain kind of > would-be "power user", if anyone uses that term anymore. It makes sense > that the idea would keep cropping up. > > And my sentiments exactly on scripting languages other than straight > Python, which is why I was originally asking if anyone knew of an > application launcher, written in Python for launching Python code. > > -- > https://mail.python.org/mailman/listinfo/python-list Maybe try to look at https://github.com/spyder-ide/spyder . It could satisfy your expectations. You can edit, run, debug, statically analyze python codes and more. And it is open source written in python... -- https://mail.python.org/mailman/listinfo/python-list
Re: Create ordering of points
On 2/15/17, spiess.benjamin--- via Python-listwrote: > Hello !:) > I've got a problem which I would really like to solve. > I got a cloud of points (in the simplest example its a 2-dimensional cloud > of points). > First, I want to set one of the points as the initial (or middle) point. > Starting from there, the next points with the closest distance to the > initial points are to be found. Afterwards, the closest points to these > points are to be written out. These points shall be connected like it is > depicted in this figure > http://fs5.directupload.net/images/170215/b835zixm.jpg > > With this approach, the "fastest" path from every point to the defined > initial point are specified by this pattern. > > Does somebody know a good technique for this problem? or can even give a > hint to a existing python procedure? > I would be really grateful! Something like https://networkx.github.io/documentation/networkx-1.10/reference/algorithms.shortest_paths.html could help? Or maybe this https://networkx.github.io/documentation/networkx-1.10/reference/generated/networkx.algorithms.mst.minimum_spanning_tree.html?highlight=minimum_spanning_tree ? PL. -- https://mail.python.org/mailman/listinfo/python-list
Re: Coding issue with XML for word document
On 2/7/17, Steven D'Apranowrote: > On Mon, 06 Feb 2017 17:00:25 -0800, accessnewbie wrote: [...] >> But when I try to bold, it bombs. It does not seem to like either of >> these (+ myText + "") (separately or together) appended on the >> end. > > How are you trying to bold? If you're programming in a word processor, > and you apply bold to your code, that won't work. > > What do you mean, "bombs"? Do you literally mean you are running a > classic Macintosh from the 1990s and you are getting a fatal System Error > in a "Bomb" dialog box? > > Otherwise, please be more precise: do you get a Blue Screen of Death? A > kernel panic? Does the Python interpreter segfault? Or do you get an > ordinary Python traceback? > > If it is an ordinary Python traceback, don't keep it a secret -- tell us > what it says. COPY AND PASTE the traceback, in full, not just the last > line. I am afraid he means that it create mishmash in word document... Dear accessnewbie I propose divide problem to smaller subproblems. For example make python code as simple as possible (avoid python part if you could! As Deborah wrote it is very probably not problem in python) And try to explicitly try to write example text where is bold part. #myBlurb = "My Favorite Cars - My favorite cars are available at " + myText + "" myBlurb = "My Favorite Cars - My favorite cars are available at my garage " if it is still wrong try to make more and more similar text to: myBlurb = "My Favorite Cars - My favorite cars are available at my garage" I mean for example remove "" part etc. Second - This is probably not your problem (unless you have special char in your example myList) but if you could have xml spec chars in your input then python could help. See example -> >>> from xml.sax.saxutils import escape >>> escape("< & >") ' ' myBlurb = "My Favorite Cars - My favorite cars are available at " + escape(myText) + "" # this is safer than your code! I hope you understand why it is useful. :) -- https://mail.python.org/mailman/listinfo/python-list
Re: best way to ensure './' is at beginning of sys.path?
On 2/6/17, Chris Angelicowrote: > On Mon, Feb 6, 2017 at 1:19 PM, Steve D'Aprano > wrote: [...] >> How about graphic and video designers? Just how well does hg cope with >> gigabytes of video data? > > I've no idea, but I know that git can handle large amounts of data. > (There are a couple of extensions that make it easier.) But there could be problem with "keeping eye" part of your premise. ("If you commit everything to git and keep an eye on your diffs before you push, the encryption would have to be _extremely_ sneaky"). -- https://mail.python.org/mailman/listinfo/python-list