Re: Fatal Python error: PyCOND_WAIT(gil_cond) failed

[dieter]

keepplearningpython  writes:

> I am running ipython3 on Unix and constantly see this crash -
> It happens when i try to issue commands on the ipython interactive shell.
>
> I have tried to set the PYTHONDIR to /var/tmp/ in case there was an issue 
> accessing the default location of the history file. However, this makes no 
> difference.
>
> I can run a command a few times successfully after which I hit this error.
> ...
> Any suggestions on what might be going on?
>
>
>
>
> Fatal Python error: PyCOND_WAIT(gil_cond) failed

Apparently, some thread should wait for the GIL
(Python's Global Interpreter Lock) but instead of waiting,
the operation results in an error.

This looks like an important bug in GIL handling -- likely
caused by some C extension, likely executing Python code
without first acquiring the GIL.

It might be difficult to locate the error: likely C level
debugging will be necessary. Should my hypothesis be true,
a change in C code would be necessary to fix the problem.


I recommend to check for known problems related to GIL handling
for all C implemented components of your application.
If you are lucky, your problem is known and has been fixed
in a different version. Then upgrading would be all you need to do.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list comprehension return a list and sum over in loop

[KK Sasa]

Sorry, i should say I'm using pythonxy, maybe it imports other things. 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python Script to convert firewall rules

[Jason Friedman]

>
>
> Thanks for the reply. Yes I can make the all possible keywords/values for
> both formate. But after that what gonna be the logic to convert one format
> to other format. Like to convert one line below are the keywords:
>
> set interface ethernet2/5 ip 10.17.10.1/24 (format 1)
> set interfaces ge-0/0/0 unit 0 family inet address 10.17.10.1/24 (format
> 2)
>
> (set, interface, ip) = (set, interfaces, family inet address)
>
> But some values are variable and should ask the user to convert manually
> like ethernet2/5 equal to ge-0/0/0 or ge-0/0/1 or ge-0/0/2
>
> And some values keep as it is like 10.17.10.1/24
>
> Also then format 2 can be converted int o format 3 (as below) for more
> readability of format 2. This is just optional.
>
> interfaces {
> ge-2/0/5 {
> unit 0 {
> family inet {
> address 10.17.10.1/24;
> }
> }
> }
> }
>
>
Note that the practice on this list is to put your response after the
(edited) portion of the previous posts.

Are you willing to learn some Python, if someone gets you started?
Would it be helpful if someone provided Python code to convert this:

set interfaces ge-0/0/0 unit 0 family inet address 10.17.10.1/24

to this:

interfaces {
ge-2/0/5 {
unit 0 {
family inet {
address 10.17.10.1/24;
}
}
}
}

?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: encrypt the http request url on the local machine

[Michael Torrie]

On 12/12/2014 08:53 AM, iMath wrote:
> After some retinking on my question ,I found what I am really want is
> not let  any other guys using packet analyzer software know the
> server name (host name) my software is sending data to .
> 
> so I translate the host name to IP address format, it somewhat hides
> the host name. Another question is : for common sites, is the IP
> address under the host name hardly  changing ?

I don't see what that will accomplish.  Of what value is the hostname vs
the IP address?  Many servers have proper reverse entries in DNS to map
IP addresses back to host names.

Furthermore a packet trace can show exactly what the HTTP request was,
and that doesn't matter whether you do IP address or hostname.

As for whether the IP address changes often, depends on the server.
Sometimes some web sites use a cluster of servers and each name lookup
will give you a different IP address (round robin lookup).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: is_ as method or property?

[Mateusz Loskot]

On 12 December 2014 at 22:14, Chris Angelico  wrote:
> On Sat, Dec 13, 2014 at 8:03 AM, Mateusz Loskot  wrote:
>> On 12 December 2014 at 12:26, Chris Angelico  wrote:
>>> On Fri, Dec 12, 2014 at 10:21 PM, Mateusz Loskot  wrote:
 I've got several cases which are not obvious to me.
 For instance, class Foo has a boolean attribute, read-write,
 which I see a couple of realisations for possible:

>>>
>>> 0) Attribute only.
>>>
>>> class Foo:
>>> pass
>>>
>>> Foo().default = True
>>>
>>> Unless you need something more than this, go with this style. Much simpler.
>>
>>
>> Agreed, in general, but if the question was related to such simple case,
>> I wouldn't bother asking it.
>>
>> I mentioned, setting the new value involves more changes to Foo()
>> instance, so i's not possible to capture it with just an assignment.
>> Hence, the discussion between property vs method.
>
> It's still worth mentioning, for completeness.

Of course, very good point.

> Plenty of people come from C++

That's me.

> or Java and simply don't realize that direct attribute access
> makes fine sense. Your statement here talks about a read-write boolean
> attribute, so starting off with the simplest way of doing that - an
> actual attribute - emphasizes the commonness of this.

Yes, I see your point now.

> So, your subsequent design decisions are based around the fact that
> you cannot simply assign to .default and have it do the right thing.
> Are you even able to set default to False?

Right. The particular case (with is_default and make_default members)
I'm discussing is similar to this:

class Window:
   def close(self):
  # operate window so it gets closed

   @property
   def is_closed(self)
  # query state of window

> For instance, you might
> have a system where there must always be one "default", and so you
> can't make this one not-default, but you could make a different one
> the default, which will un-default-ify this one as a side effect. In
> that case, a "make_default()" method makes the most sense. But if you
> can independently and cheaply set any instance's default flag, you
> could have the actual attribute, and then a set_default() method,
> which updates it. Plenty of Python classes recommend not arbitrarily
> altering certain attributes.

Yes, makes sense. Thanks for brainstorming those ideas deeper.

Best regards,
-- 
Mateusz  Loskot, http://mateusz.loskot.net
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: is_ as method or property?

[Chris Angelico]

On Sat, Dec 13, 2014 at 8:03 AM, Mateusz Loskot  wrote:
> On 12 December 2014 at 12:26, Chris Angelico  wrote:
>> On Fri, Dec 12, 2014 at 10:21 PM, Mateusz Loskot  wrote:
>>> I've got several cases which are not obvious to me.
>>> For instance, class Foo has a boolean attribute, read-write,
>>> which I see a couple of realisations for possible:
>>>
>>
>> 0) Attribute only.
>>
>> class Foo:
>> pass
>>
>> Foo().default = True
>>
>> Unless you need something more than this, go with this style. Much simpler.
>
>
> Agreed, in general, but if the question was related to such simple case,
> I wouldn't bother asking it.
>
> I mentioned, setting the new value involves more changes to Foo()
> instance, so i's not possible to capture it with just an assignment.
> Hence, the discussion between property vs method.

It's still worth mentioning, for completeness. Plenty of people come
from C++ or Java and simply don't realize that direct attribute access
makes fine sense. Your statement here talks about a read-write boolean
attribute, so starting off with the simplest way of doing that - an
actual attribute - emphasizes the commonness of this.

So, your subsequent design decisions are based around the fact that
you cannot simply assign to .default and have it do the right thing.
Are you even able to set default to False? For instance, you might
have a system where there must always be one "default", and so you
can't make this one not-default, but you could make a different one
the default, which will un-default-ify this one as a side effect. In
that case, a "make_default()" method makes the most sense. But if you
can independently and cheaply set any instance's default flag, you
could have the actual attribute, and then a set_default() method,
which updates it. Plenty of Python classes recommend not arbitrarily
altering certain attributes.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: is_ as method or property?

[Mateusz Loskot]

On 12 December 2014 at 12:26, Chris Angelico  wrote:
> On Fri, Dec 12, 2014 at 10:21 PM, Mateusz Loskot  wrote:
>> I've got several cases which are not obvious to me.
>> For instance, class Foo has a boolean attribute, read-write,
>> which I see a couple of realisations for possible:
>>
>
> 0) Attribute only.
>
> class Foo:
> pass
>
> Foo().default = True
>
> Unless you need something more than this, go with this style. Much simpler.


Agreed, in general, but if the question was related to such simple case,
I wouldn't bother asking it.

I mentioned, setting the new value involves more changes to Foo()
instance, so i's not possible to capture it with just an assignment.
Hence, the discussion between property vs method.

Best regards,
-- 
Mateusz  Loskot, http://mateusz.loskot.net
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make subprocess run for 60 sec?

[Akira Li]

Dan Stromberg  writes:

> On Fri, Dec 12, 2014 at 12:48 AM, Robert Clove  wrote:
>> Hi All,
>>
>> I have the following python script that runs.
>> I want is to run the subprocess to run for 60 sec and then send the SIGINT
>> signal to subprocess and write the output in file.
>>
>> #!/usr/bin/python
>> import os
>> import subprocess
>> PIPE = subprocess.PIPE
>> import signal
>> import time
>>
>> def handler(signum, frame):
>> pass
>>
>> signal.signal(signal.SIGALRM, handler)
>> signal.alarm(60)
>> command = "strace -c ./server"
>> os.chdir("/root/Desktop/")
>> p = subprocess.Popen(command, stdout=PIPE, stderr=PIPE)
>> time.sleep(60)
>> p.send_signal(signal.SIGINT)
>> signal.alarm(0)
>> print p.communicate()[1]
>
> Perhaps try something like:
>
> #!/usr/bin/python
>
> #import os
> import subprocess
> PIPE = subprocess.PIPE
> import signal
> import time
>
> #def handler(signum, frame):
> #pass
>
> with open('output.txt', 'w') as out_file:
> #signal.signal(signal.SIGALRM, handler)
> #signal.alarm(60)
> #command = "strace -c ./server"
> command = "./test-script"
> #os.chdir("/root/Desktop/")
> p = subprocess.Popen(command, stdout=out_file, stderr=out_file, 
> shell=True)
> time.sleep(5)
> p.send_signal(signal.SIGINT)
> #signal.alarm(0)
> #print p.communicate()[1]
>
> with open('output.txt', 'r') as in_file:
> output = in_file.read()
>
> print(output)

You should probably call p.wait() after p.send_signal(), to wait until
the child process exits (possibly properly flushing its stdout buffer). 
It might make the output.txt content less garbled.

>
> BTW, killing an active strace may leave strace's subprocess stuck in
> an unkillable state.
>
> Also note that this will sleep for n seconds whether the subprocess
> takes that long or not.

The answer on StackOverflow [1] shows how to avoid waiting n seconds if the
subprocess finishes sooner.

[1] 
http://stackoverflow.com/questions/27443480/how-to-make-subprocess-run-for-60-sec

> If you just want a program that does this, and it doesn't have to be
> in Python, you might try
> http://stromberg.dnsalias.org/~strombrg/maxtime.html
> It's in C, and is the product of considerable real-world use.  It
> exits almost immediately after its subprocess exits, FWIW.


--
Akira

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Python-Dev] Python 2.x and 3.x use survey, 2014 edition

[Barry Warsaw]

On Dec 12, 2014, at 08:07 PM, Petr Viktorin wrote:

>If anyone is wondering why their favorite Linux distribution is stuck with
>Python 2 – well, I can only speak for Fedora, but nowadays most of what's
>left are CPython bindings.  No pylint --py3k or 2to3 will help there...

It's true that some of these are tough.  I tried and failed a few times to
port Xapian to Python 3.  The issue was opened upstream 6 years ago and it's
still unresolved: http://trac.xapian.org/ticket/346

OTOH, I ported dbus-python to Python 3 and that worked out much better; we've
had solid Python 3 bindings for several years now, which allowed us to port
many important Debian/Ubuntu tools to Python 3 and more importantly, do all
our new work in Python 3.  With other big toolkits like GObject introspection
working on Python 3, there's a lot you can do.

IME, if the underlying model is string/bytes clean, then the C extension port
can sometimes be easier than pure-Python, thanks to cpp games.  D-Bus's model
is pretty clean, Xapian I found to be not so much (it doesn't help that Xapian
is C++ ;).

We're actually not terribly far from switching Debian and Ubuntu's default
to Python 3.  On Debian, the big blocker is the BTS code (which uses SOAP) and
on Ubuntu it's the launchpadlib stack.  I hope to find time after Jessie to
work on the former, and before 16.04 LTS to work on the latter.

Not that I disagree that there's a long tail of code that would still benefit
a significant population if it got ported to Python 3.  By far Python 3 is a
better language, with a better stdlib, so the work is worth it.

Cheers,
-Barry
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Python-Dev] Python 2.x and 3.x use survey, 2014 edition

[Matthew Ruffalo]

On 12/11/2014 09:48 PM, Terry Reedy wrote:
> A possible reason: one is developing an app expected to be released
> fall 2015 after the 3.5 release and the app depends on something new
> in 3.5.  I must admit though that I cannot think of any such thing now
> for 3.5.  For 3.3 there was the new unicode, which was first committed
> about a year before release.  For 3.4, there was asyncio, but it was
> not committed until beta1, and was hardly usable then.
>
> So it was a defensible position.  Anyone who would check 3.5 could
> just as well check 3.4 and have most of the same impact on the summary.
The NumPy and SciPy developers who are implementing support for PEP 465
('@' as a dedicated matrix multiplication operator) will likely want to
test against a development version of 3.5 at some point. Using 3.5 isn't
strictly required, though, since supporting the @ operator is mostly a
matter of implementing the new __matmul__ methods and one can test those
by calling them directly instead of using the new operator.

MMR...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Run Programming ?????

[sohcahtoa82]

> On Friday, December 12, 2014, William Ray Wing  wrote:
> 
> 
> 
> 
> On Dec 12, 2014, at 8:03 AM, Chris Warrick  wrote:
> 
> 
> 
> 
> On Dec 12, 2014 1:40 PM, "Delgado Motto"  wrote:
> 
> >
> 
> > I travel alot, if not just interested in things of pocketable portability, 
> > and was curious if you can tell me if Python can be LEARNED from beginner 
> > on an IOS device ( with interest of being able to test my code, possibly 
> > even if a free website is capable of reviewing scripts ) but if not then I 
> > prefer if you can suggest a language that can be used from such a machine. 
> > My ultimate goal is to be able to create web pages and internet bots 
> > capable of searching specific things for me, simply to save me time in my 
> > day as little as crawling Youtube for a song that fails to be uploaded or 
> > other related examples. Please advise me. Thanks.  
> 
> > --
> 
> > https://mail.python.org/mailman/listinfo/python-list
> 
> >
> Get a real computer. An iOS device won't work, unless you care to buy a vps, 
> use ssh and can stand the onscreen keyboard. It's easier to buy a notebook.
> -- 
> 
> Chris Warrick 
> 
> Sent from my Galaxy S3.
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 
> 
> Second the motion.  Apple's sandbox policies on iOS devices mean that while 
> you can run Python on them (there are several versions available), the 
> sandbox  pretty much guarantees that at some point you will need a library 
> you can't import, and you won't be able to test your web pages as you go.
> 
> 
> A MacBook Air is within a fraction of being as portable as an iPad, and can 
> easily do everything you want.  If you are currently traveling with an iPad, 
> you _might_ even discover you prefer traveling with the MacBook.
> 
> 
> -Bill

On Friday, December 12, 2014 7:44:12 AM UTC-8, Delgado Motto wrote:
> I was specifically talking POCKETABLE devices so Phablet or Telephone 
> preferably, simply hopeful as smaller machines continue to become more 
> capable, but I expected as much of this being a problem. Thanks. 
> 

Its not a matter of processing power.  The Motorola Droid I got five years ago 
could certainly run a Python app if someone wrote an interpreter for it, and I 
imagine someone already has.

The problem is actually writing code.  Typing on a touch-screen keyboard is 
already difficult enough.  Dealing with frequently having to type special 
characters (parentheses, brackets, colons, etc.) and numbers would make it 
extremely tedious to the point of being painful.  Top that off with having a 
tiny screen so you can only see around 5 lines of code at a time and being 
unable to have a decent debugging UI, and you're really looking at a truly 
miserable time.

The real challenge won't be the software engineering, it'll be fighting with an 
absolutely piss-poor interface for doing it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Python-Dev] Python 2.x and 3.x use survey, 2014 edition

[Petr Viktorin]

Also keep in mind that not all Python libraries are on PyPI.
For non-Python projects with Python bindings (think video players,
OpenCV, systemd, Samba), distribution via PyPI doesn't make much
sense. And since the Python bindings are usually second-class
citizens, the porting doesn't have a high priority.

If anyone is wondering why their favorite Linux distribution is stuck
with Python 2 – well, I can only speak for Fedora, but nowadays most
of what's left are CPython bindings.
No pylint --py3k or 2to3 will help there...


On Fri, Dec 12, 2014 at 7:24 PM, Mark Roberts  wrote:
> So, I'm more than aware of how to write Python 2/3 compatible code. I've
> ported 10-20 libraries to Python 3 and write Python 2/3 compatible code at
> work. I'm also aware of how much writing 2/3 compatible code makes me hate
> Python as a language. It'll be a happy day when one of the two languages
> dies so that I never have to write code like that again. However, my point
> was that just because the core libraries by usage are *starting* to roll out
> Python 3 support doesn't mean that things are "easy" or "convenient" yet.
> There are too many libraries in the long tail which fulfill semi-common
> purposes and haven't been moved over yet. Yeah, sure, they haven't been
> updated in years... but neither has the language they're built on.
>
> I suppose what I'm saying is that the long tail of libraries is far more
> valuable than it seems the Python3 zealots are giving it credit for. Please
> don't claim it's "easy" to move over just because merely most of the top 20
> libraries have been moved over. :-/
>
> -Mark
>
> On Thu, Dec 11, 2014 at 12:14 PM, Dan Stromberg  wrote:
>>
>> On Thu, Dec 11, 2014 at 11:35 AM, Mark Roberts  wrote:
>> > I disagree. I know there's a huge focus on The Big Libraries (and
>> > wholesale
>> > migration is all but impossible without them), but the long tail of
>> > libraries is still incredibly important. It's like saying that migrating
>> > the
>> > top 10 Perl libraries to Perl 6 would allow people to completely ignore
>> > all
>> > of CPAN. It just doesn't make sense.
>>
>> Things in the Python 2.x vs 3.x world aren't that bad.
>>
>> See:
>> https://python3wos.appspot.com/ and
>> https://wiki.python.org/moin/PortingPythonToPy3k
>> http://stromberg.dnsalias.org/~strombrg/Intro-to-Python/ (writing code
>> to run on 2.x and 3.x)
>>
>> I believe just about everything I've written over the last few years
>> either ran on 2.x and 3.x unmodified, or ran on 3.x alone.  If you go
>> the former route, you don't need to wait for your libraries to be
>> updated.
>>
>> I usually run pylint twice for my projects (after each change, prior
>> to checkin), once with a 2.x interpreter, and once with a 3.x
>> interpreter (using
>> http://stromberg.dnsalias.org/svn/this-pylint/trunk/this-pylint) , but
>> I gather pylint has the option of running on a 2.x interpreter and
>> warning about anything that wouldn't work on 3.x.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Python-Dev] Python 2.x and 3.x use survey, 2014 edition

[Mark Roberts]

So, I'm more than aware of how to write Python 2/3 compatible code. I've
ported 10-20 libraries to Python 3 and write Python 2/3 compatible code at
work. I'm also aware of how much writing 2/3 compatible code makes me hate
Python as a language. It'll be a happy day when one of the two languages
dies so that I never have to write code like that again. However, my point
was that just because the core libraries by usage are *starting* to roll
out Python 3 support doesn't mean that things are "easy" or "convenient"
yet. There are too many libraries in the long tail which fulfill
semi-common purposes and haven't been moved over yet. Yeah, sure, they
haven't been updated in years... but neither has the language they're built
on.

I suppose what I'm saying is that the long tail of libraries is far more
valuable than it seems the Python3 zealots are giving it credit for. Please
don't claim it's "easy" to move over just because merely most of the top 20
libraries have been moved over. :-/

-Mark

On Thu, Dec 11, 2014 at 12:14 PM, Dan Stromberg  wrote:

> On Thu, Dec 11, 2014 at 11:35 AM, Mark Roberts  wrote:
> > I disagree. I know there's a huge focus on The Big Libraries (and
> wholesale
> > migration is all but impossible without them), but the long tail of
> > libraries is still incredibly important. It's like saying that migrating
> the
> > top 10 Perl libraries to Perl 6 would allow people to completely ignore
> all
> > of CPAN. It just doesn't make sense.
>
> Things in the Python 2.x vs 3.x world aren't that bad.
>
> See:
> https://python3wos.appspot.com/ and
> https://wiki.python.org/moin/PortingPythonToPy3k
> http://stromberg.dnsalias.org/~strombrg/Intro-to-Python/ (writing code
> to run on 2.x and 3.x)
>
> I believe just about everything I've written over the last few years
> either ran on 2.x and 3.x unmodified, or ran on 3.x alone.  If you go
> the former route, you don't need to wait for your libraries to be
> updated.
>
> I usually run pylint twice for my projects (after each change, prior
> to checkin), once with a 2.x interpreter, and once with a 3.x
> interpreter (using
> http://stromberg.dnsalias.org/svn/this-pylint/trunk/this-pylint) , but
> I gather pylint has the option of running on a 2.x interpreter and
> warning about anything that wouldn't work on 3.x.
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make subprocess run for 60 sec?

[Dan Stromberg]

On Fri, Dec 12, 2014 at 12:48 AM, Robert Clove  wrote:
> Hi All,
>
> I have the following python script that runs.
> I want is to run the subprocess to run for 60 sec and then send the SIGINT
> signal to subprocess and write the output in file.
>
> #!/usr/bin/python
> import os
> import subprocess
> PIPE = subprocess.PIPE
> import signal
> import time
>
> def handler(signum, frame):
> pass
>
> signal.signal(signal.SIGALRM, handler)
> signal.alarm(60)
> command = "strace -c ./server"
> os.chdir("/root/Desktop/")
> p = subprocess.Popen(command, stdout=PIPE, stderr=PIPE)
> time.sleep(60)
> p.send_signal(signal.SIGINT)
> signal.alarm(0)
> print p.communicate()[1]

Perhaps try something like:

#!/usr/bin/python

#import os
import subprocess
PIPE = subprocess.PIPE
import signal
import time

#def handler(signum, frame):
#pass

with open('output.txt', 'w') as out_file:
#signal.signal(signal.SIGALRM, handler)
#signal.alarm(60)
#command = "strace -c ./server"
command = "./test-script"
#os.chdir("/root/Desktop/")
p = subprocess.Popen(command, stdout=out_file, stderr=out_file, shell=True)
time.sleep(5)
p.send_signal(signal.SIGINT)
#signal.alarm(0)
#print p.communicate()[1]

with open('output.txt', 'r') as in_file:
output = in_file.read()

print(output)


BTW, killing an active strace may leave strace's subprocess stuck in
an unkillable state.

Also note that this will sleep for n seconds whether the subprocess
takes that long or not.

If you just want a program that does this, and it doesn't have to be
in Python, you might try
http://stromberg.dnsalias.org/~strombrg/maxtime.html
It's in C, and is the product of considerable real-world use.  It
exits almost immediately after its subprocess exits, FWIW.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list comprehension return a list and sum over in loop

[Oscar Benjamin]

On 12 December 2014 at 06:22, KK Sasa  wrote:
> Hi there,
>
> The list comprehension is results = [d2(t[k]) for k in xrange(1000)], where 
> d2 is a function returning a list, say [x1,x2,x3,x4] for one example. So 
> "results" is a list consisting of 1000 lists, each of length four. Here, what 
> I want to get is the sum of 1000 lists, and then the result is a list of 
> length four. Is there any efficient way to do this? Because I found it is 
> slow in my case. I tried sum(d2(t[k]) for k in xrange(1000)), but it returned 
> error: TypeError: unsupported operand type(s) for +: 'int' and 'list'. Thanks.

Use numpy.sum:

>>> import numpy as np
>>> a = [[1, 1, 1, 1], [2, 2, 2, 2], [3, 3, 3, 3]]
>>> np.sum(a, 0)
array([6, 6, 6, 6])
>>> np.sum(a, 1)
array([ 4,  8, 12])

The second argument to numpy.sum is the dimension along which you want
to sum e.g. rows or columns.


Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list comprehension return a list and sum over in loop

[Peter Otten]

KK Sasa wrote:

> Peter Otten於 2014年12月12日星期五UTC+8下午8時32分55秒寫道：
>> Jussi Piitulainen wrote:
>> 
>> > KK Sasa writes:
>> > 
>> >> def p(x,t,point,z,obs):
>> >> d = x[0]
>> >> tau = [0]+[x[1:point]]
>> >> a = x[point:len(x)]
>> >> at = sum(i*j for i, j in zip(a, t))
>> >> nu = [exp(z[k]*(at-d)-sum(tau[k])) for k in xrange(point)]
>> >> de = sum(nu, axis=0)
>> >> probability = [nu[k]/de for k in xrange(point)]
>> >> return probability[obs]
>> > 
>> > I must be blind, but this looks like computing a whole probability
>> > distribution and then throwing almost all of it away.
>> > 
>> > Can't this just return nu[obs]/de?
>> > 
>> > The expression for tau also seems weird to me. Isn't it equivalent to
>> > [0, x[1:point]], a two-element list with the second element a list?
>> > How can sum(tau[k]) work at all then, for any k > 1?
>> 
>> Also, after adding
>> 
>> from numpy.random import seed
>> 
>> to the code I run into the next problem:
>> 
>> Traceback (most recent call last):
>>   File "hessian.py", line 34, in 
>> re = [d2(x,t[k],2,z,1) for k in range_people]
>>   File "/home/petto/.local/lib/python2.7/site-packages/ad/__init__.py",
>>   line
>> 1114, in hess
>> return func(xa, *args).hessian([xa])
>>   File "hessian.py", line 26, in p
>> nu = [exp(z[k]*(at-d)-sum(tau[k])) for k in xrange(point)]
>> TypeError: 'int' object is not iterable
>> 
>> OP: Optimizations only make sense when you can start from a known-good
>> base. You should sort out the logic of your problem (we probably can't
>> help you with that), then fix the bugs in your code and only then revisit
>> the "speed issue".
> 
> I have no idea why you added "from numpy.random import seed" to this
> syntax. Even I added that, my python didn't complain any bugs inside.
> Maybe your ad package was not be installed correctly?

It may be a version issue. I'm using

>>> import ad, numpy
>>> ad.__version__
'1.2.2'
>>> numpy.__version__
'1.9.1'

Or are you using an environment that does some imports automatically?


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: encrypt the http request url on the local machine

[Chris Angelico]

On Sat, Dec 13, 2014 at 2:53 AM, iMath  wrote:
> After some retinking on my question ,I found what I am really want is not let 
>  any other guys using packet analyzer software know the server name (host 
> name) my software is sending data to .
>
> so I translate the host name to IP address format, it somewhat hides the host 
> name.
> Another question is : for common sites, is the IP address under the host name 
> hardly  changing ?

It depends entirely on the site's admins. Usually, you don't want to
do this, in case the IP changes. Plus, you run into a catch-22: if you
use the IP without ever using the host name, you depend on the server
responding with what you want in the absence of a host header
(basically, you depend on the server running just one web site), but
if that's the case, then concealing the host name is almost completely
useless, because anyone else can talk to the same server without a
host name. In fact, for big web sites, chances are the reverse DNS for
that IP will at least point to a related domain, even if not to the
exact web site you're accessing (for instance, www.wikipedia.org ->
198.35.26.96 -> text-lb.ulsfo.wikimedia.org). Basically, this either
will stop you from accessing what you want, or won't conceal anything.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: encrypt the http request url on the local machine

[iMath]

在 2014年12月9日星期二UTC+8下午2时58分36秒，iMath写道：
> my software on the local machine needs to send http request to a specific web 
> server , is there any way to protect the http request url from being found by 
> Packet analyzer software like Wireshark and fiddler. The sever is not mine, 
> so I can do nothing in the server .
> 
> It would be better to show some code, I am an absolutely newbie in encryption 
> .

After some retinking on my question ,I found what I am really want is not let  
any other guys using packet analyzer software know the server name (host name) 
my software is sending data to .

so I translate the host name to IP address format, it somewhat hides the host 
name.
Another question is : for common sites, is the IP address under the host name 
hardly  changing ?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Call for speakers for the first PyCon Belarus. Python-announce

[serge Guelton]

On Fri, Dec 12, 2014 at 04:32:26PM +0300, Alina Dolgikh wrote:
> Hello, dear community!
> 
> I represent Belarusian Python community. We have regular monthly meet-ups
> for 70-100 persons and we are going to develop further.
> 
> We are planning to make the first Belarusian PyCon on the 31st of January
> and looking for speakers.
> 
> We will be glad to meet at our event speakers, which are experienced in
> public talks (links for videos of public talks or for the other conferences
> web-pages are preferable) and can speak at

> - CI, pypy, C-extensions, python performance

I there,

I am the core developer of the Pythran[0] compiler, a Python to C++
compiler that focuses on the numpy package optimization, and tries hard
to turn high-level Numpy code into low-level, vectorized, parallel C++
code. I presented an early version at Scipy 2013[1], this should give
you a decent idea of my yet to improve speaker skills :-/

Any way, I would be happy to speak about it at PyCon Belarusian!

If you're intereseted/curious, I can provide a more detailed description
;-)

See ya,

Serge

[0] http://pythonhosted.org/pythran/
[1] https://www.youtube.com/watch?v=KT5-uGEpnGw
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Run Programming ?????

[Chris Angelico]

On Sat, Dec 13, 2014 at 2:43 AM, Delgado Motto  wrote:
> I was specifically talking POCKETABLE devices so Phablet or Telephone
> preferably, simply hopeful as smaller machines continue to become more
> capable, but I expected as much of this being a problem. Thanks.

Those usually are terrible for any sort of programming. Typing on a
touch-screen - especially a tiny one - is tedious and error-prone, and
as mentioned, you often can't get a decent environment set up. Get
some real hardware, preferably with a free OS on it (Linux being the
most popular, but not the only, option), and carry around something
that sits on your lap instead of in your pocket, but can actually do
what you need it to. I've been carrying around a 14" or 15" laptop for
years, now, and I happily sit all the way up back of a bus, coding
away on an actual keyboard. (I recommend about that size,
incidentally. Smaller ones tend to feel cramped, larger ones have
trouble fitting in between other passengers. 14-15 inch is about
right.)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Run Programming ?????

[Delgado Motto]

I was specifically talking POCKETABLE devices so Phablet or Telephone
preferably, simply hopeful as smaller machines continue to become more
capable, but I expected as much of this being a problem. Thanks.



On Friday, December 12, 2014, William Ray Wing  wrote:

>
> On Dec 12, 2014, at 8:03 AM, Chris Warrick  > wrote:
>
>
> On Dec 12, 2014 1:40 PM, "Delgado Motto"  > wrote:
> >
> > I travel alot, if not just interested in things of pocketable
> portability, and was curious if you can tell me if Python can be LEARNED
> from beginner on an IOS device ( with interest of being able to test my
> code, possibly even if a free website is capable of reviewing scripts ) but
> if not then I prefer if you can suggest a language that can be used from
> such a machine. My ultimate goal is to be able to create web pages and
> internet bots capable of searching specific things for me, simply to save
> me time in my day as little as crawling Youtube for a song that fails to be
> uploaded or other related examples. Please advise me. Thanks.
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> >
>
> Get a real computer. An iOS device won't work, unless you care to buy a
> vps, use ssh and can stand the onscreen keyboard. It's easier to buy a
> notebook.
>
> --
> Chris Warrick 
> Sent from my Galaxy S3.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>
> Second the motion.  Apple’s sandbox policies on iOS devices mean that
> while you can run Python on them (there are several versions available),
> the sandbox  pretty much guarantees that at some point you will need a
> library you can’t import, and you won’t be able to test your web pages as
> you go.
>
> A MacBook Air is within a fraction of being as portable as an iPad, and
> can easily do everything you want.  If you are currently traveling with an
> iPad, you _might_ even discover you prefer traveling with the MacBook.
>
> -Bill
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Run Programming ?????

[Chris Angelico]

On Sat, Dec 13, 2014 at 2:03 AM, William Ray Wing  wrote:
> A MacBook Air is within a fraction of being as portable as an iPad, and can
> easily do everything you want.  If you are currently traveling with an iPad,
> you _might_ even discover you prefer traveling with the MacBook.

Or get some hardware that you can actually put a free operating system
on. I like the IBM/Lenovo laptops for their reliability.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Run Programming ?????

[William Ray Wing]

> On Dec 12, 2014, at 8:03 AM, Chris Warrick  wrote:
> 
> 
> On Dec 12, 2014 1:40 PM, "Delgado Motto"  > wrote:
> >
> > I travel alot, if not just interested in things of pocketable portability, 
> > and was curious if you can tell me if Python can be LEARNED from beginner 
> > on an IOS device ( with interest of being able to test my code, possibly 
> > even if a free website is capable of reviewing scripts ) but if not then I 
> > prefer if you can suggest a language that can be used from such a machine. 
> > My ultimate goal is to be able to create web pages and internet bots 
> > capable of searching specific things for me, simply to save me time in my 
> > day as little as crawling Youtube for a song that fails to be uploaded or 
> > other related examples. Please advise me. Thanks.  
> > --
> > https://mail.python.org/mailman/listinfo/python-list 
> > 
> >
> 
> Get a real computer. An iOS device won't work, unless you care to buy a vps, 
> use ssh and can stand the onscreen keyboard. It's easier to buy a notebook.
> 
> -- 
> Chris Warrick >
> Sent from my Galaxy S3.
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list

Second the motion.  Apple’s sandbox policies on iOS devices mean that while you 
can run Python on them (there are several versions available), the sandbox  
pretty much guarantees that at some point you will need a library you can’t 
import, and you won’t be able to test your web pages as you go.

A MacBook Air is within a fraction of being as portable as an iPad, and can 
easily do everything you want.  If you are currently traveling with an iPad, 
you _might_ even discover you prefer traveling with the MacBook.

-Bill-- 
https://mail.python.org/mailman/listinfo/python-list

Re: beautifulsoup VS lxml

[iMath]

在 2014年12月12日星期五UTC+8上午10时19分56秒，Michael Torrie写道：
> On 12/11/2014 07:02 PM, iMath wrote:
> > 
> > which is more easy and elegant for pulling data  out of HTML?
> 
> Beautiful Soup is specialized for HTML parsing, and it can deal with
> badly formed HTML, but if I recall correctly BeautifulSoup can use the
> lxml engine under the hood, so maybe it's the way to go for you, is it
> gives you the most flexibility.  It certainly has a good API that's easy
> to use for data scraping.  Try it and see if it's acceptable.

tried it, very elegant and Pythonic.
thank you very much !!!
-- 
https://mail.python.org/mailman/listinfo/python-list

Python REST API Wrapper framework?

[Alec Taylor]

It's not overly difficult to build a wrapper of someones RESTfull HTTP
API using something like: urllib2 or requests.

However, there is still a decent amount of generic boilerplate required.

Are there any decent frameworks around which reduce the amount of
boilerplate required to consume 3rd-party APIs?



# Research results (thus far)
So far my research has come up with:

###[Finsh](https://github.com/jaimegildesagredo/finch)
>Asynchronous RESTful API consumer for Python. Finch is focused on remove all 
>of the boilerplate related to consuming http based APIs and provide a high 
>level abstraction to develop API clients.

###[Hammock](https://github.com/kadirpekel/hammock)
>Hammock is a fun module lets you deal with rest APIs by converting them into 
>dead simple programmatic APIs.

###[Tortilla](https://github.com/redodo/tortilla)
>Wrapping web APIs made easy. Tortilla uses a bit of magic to wrap APIs. 
>Whenever you get or call an attribute of a wrapper, the URL is appended by 
>that attribute's name or method parameter.



Would be great to get more suggestions + scientific anecdotes =)

Thanks
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list comprehension return a list and sum over in loop

[KK Sasa]

Peter Otten於 2014年12月12日星期五UTC+8下午8時32分55秒寫道：
> Jussi Piitulainen wrote:
> 
> > KK Sasa writes:
> > 
> >> def p(x,t,point,z,obs):
> >> d = x[0]
> >> tau = [0]+[x[1:point]]
> >> a = x[point:len(x)]
> >> at = sum(i*j for i, j in zip(a, t))
> >> nu = [exp(z[k]*(at-d)-sum(tau[k])) for k in xrange(point)]
> >> de = sum(nu, axis=0)
> >> probability = [nu[k]/de for k in xrange(point)]
> >> return probability[obs]
> > 
> > I must be blind, but this looks like computing a whole probability
> > distribution and then throwing almost all of it away.
> > 
> > Can't this just return nu[obs]/de?
> > 
> > The expression for tau also seems weird to me. Isn't it equivalent to
> > [0, x[1:point]], a two-element list with the second element a list?
> > How can sum(tau[k]) work at all then, for any k > 1?
> 
> Also, after adding 
> 
> from numpy.random import seed
> 
> to the code I run into the next problem:
> 
> Traceback (most recent call last):
>   File "hessian.py", line 34, in 
> re = [d2(x,t[k],2,z,1) for k in range_people]
>   File "/home/petto/.local/lib/python2.7/site-packages/ad/__init__.py", line 
> 1114, in hess
> return func(xa, *args).hessian([xa])
>   File "hessian.py", line 26, in p
> nu = [exp(z[k]*(at-d)-sum(tau[k])) for k in xrange(point)]
> TypeError: 'int' object is not iterable
> 
> OP: Optimizations only make sense when you can start from a known-good base. 
> You should sort out the logic of your problem (we probably can't help you 
> with that), then fix the bugs in your code and only then revisit the "speed 
> issue".

I have no idea why you added "from numpy.random import seed" to this syntax. 
Even I added that, my python didn't complain any bugs inside. Maybe your ad 
package was not be installed correctly?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list comprehension return a list and sum over in loop

[KK Sasa]

Jussi Piitulainen於 2014年12月12日星期五UTC+8下午7時12分39秒寫道：
> KK Sasa writes:
> 
> > def p(x,t,point,z,obs):
> > d = x[0]
> > tau = [0]+[x[1:point]] 
> > a = x[point:len(x)]
> > at = sum(i*j for i, j in zip(a, t))
> > nu = [exp(z[k]*(at-d)-sum(tau[k])) for k in xrange(point)]
> > de = sum(nu, axis=0)
> > probability = [nu[k]/de for k in xrange(point)]
> > return probability[obs]
> 
> I must be blind, but this looks like computing a whole probability
> distribution and then throwing almost all of it away.
> 
> Can't this just return nu[obs]/de?
> 
> The expression for tau also seems weird to me. Isn't it equivalent to
> [0, x[1:point]], a two-element list with the second element a list?
> How can sum(tau[k]) work at all then, for any k > 1?

This is just a probability of binary response. Not continuous one. Tau is a 
parameter and just have two in this case.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Run Programming ?????

[Chris Warrick]

On Dec 12, 2014 1:40 PM, "Delgado Motto"  wrote:
>
> I travel alot, if not just interested in things of pocketable
portability, and was curious if you can tell me if Python can be LEARNED
from beginner on an IOS device ( with interest of being able to test my
code, possibly even if a free website is capable of reviewing scripts ) but
if not then I prefer if you can suggest a language that can be used from
such a machine. My ultimate goal is to be able to create web pages and
internet bots capable of searching specific things for me, simply to save
me time in my day as little as crawling Youtube for a song that fails to be
uploaded or other related examples. Please advise me. Thanks.
> --
> https://mail.python.org/mailman/listinfo/python-list
>

Get a real computer. An iOS device won't work, unless you care to buy a
vps, use ssh and can stand the onscreen keyboard. It's easier to buy a
notebook.

-- 
Chris Warrick 
Sent from my Galaxy S3.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [newbie] how to make program suggest to install missing modules

[Chris Warrick]

On Dec 12, 2014 11:56 AM, "hugocoolens"  wrote:
>
> On Monday, December 8, 2014 9:00:13 PM UTC+1, sohca...@gmail.com wrote:
> > On Monday, December 8, 2014 10:46:47 AM UTC-8, Jean-Michel Pichavant
wrote:
> > > - Original Message -
> > > > From: sohcahto...@gmail.com
> > > > try:
> > > > import someModule
> > > > except ImportError:
> > > > print "Module is missing"
> > > > # handle it!
> > > >
> > > > Just make sure to attempt to import it again after making the call
to
> > > > pip to install it.
> > >
> > > Note that ImportError may be raised for other reasons than a missing
module.
> > >
> > > Check https://docs.python.org/2/library/imp.html and the
imp.find_module, it could be a safer way to check for a missing module.
> > >
> > > JM
> > >
> > >
> > > -- IMPORTANT NOTICE:
> > >
> > > The contents of this email and any attachments are confidential and
may also be privileged. If you are not the intended recipient, please
notify the sender immediately and do not disclose the contents to any other
person, use it for any purpose, or store or copy the information in any
medium. Thank you.
> > Good point.
> > Of course, imp.find_module ALSO throws ImportError if the module can't
be found, but at least in that case, you'd know the exact cause.
>
> Thanks for the suggestions, you can see here below what I came up with.
> All suggestions/corrections welcome:
>
> #!/usr/bin/env python
> import imp
> import os
> import sys
> try:
> imp.find_module('rtlsdr')
> except ImportError:
> print('Module rtlsdr is missing')
> print("I'll try to install it")
> os.system('sudo pip install pyrtlsdr')
> try:
> imp.find_module('rtlsdr')
> except ImportError:
> sys.exit('Sorry could not install module rtlsdr, contact your
> local Python-guru')
> import rtlsdr
> print('Module rtlsdr succesfully imported')
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list

This is bad practice. The user should install it themselves: sudo pip may
interfere with virtualenv and OS package managers. Just tell the user what
to install and call it a day.

-- 
Chris Warrick 
Sent from my Galaxy S3.
-- 
https://mail.python.org/mailman/listinfo/python-list

Run Programming ?????

[Delgado Motto]

I travel alot, if not just interested in things of pocketable portability,
and was curious if you can tell me if Python can be LEARNED from beginner
on an IOS device ( with interest of being able to test my code, possibly
even if a free website is capable of reviewing scripts ) but if not then I
prefer if you can suggest a language that can be used from such a machine.
My ultimate goal is to be able to create web pages and internet bots
capable of searching specific things for me, simply to save me time in my
day as little as crawling Youtube for a song that fails to be uploaded or
other related examples. Please advise me. Thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list comprehension return a list and sum over in loop

[Peter Otten]

Jussi Piitulainen wrote:

> KK Sasa writes:
> 
>> def p(x,t,point,z,obs):
>> d = x[0]
>> tau = [0]+[x[1:point]]
>> a = x[point:len(x)]
>> at = sum(i*j for i, j in zip(a, t))
>> nu = [exp(z[k]*(at-d)-sum(tau[k])) for k in xrange(point)]
>> de = sum(nu, axis=0)
>> probability = [nu[k]/de for k in xrange(point)]
>> return probability[obs]
> 
> I must be blind, but this looks like computing a whole probability
> distribution and then throwing almost all of it away.
> 
> Can't this just return nu[obs]/de?
> 
> The expression for tau also seems weird to me. Isn't it equivalent to
> [0, x[1:point]], a two-element list with the second element a list?
> How can sum(tau[k]) work at all then, for any k > 1?

Also, after adding 

from numpy.random import seed

to the code I run into the next problem:

Traceback (most recent call last):
  File "hessian.py", line 34, in 
re = [d2(x,t[k],2,z,1) for k in range_people]
  File "/home/petto/.local/lib/python2.7/site-packages/ad/__init__.py", line 
1114, in hess
return func(xa, *args).hessian([xa])
  File "hessian.py", line 26, in p
nu = [exp(z[k]*(at-d)-sum(tau[k])) for k in xrange(point)]
TypeError: 'int' object is not iterable

OP: Optimizations only make sense when you can start from a known-good base. 
You should sort out the logic of your problem (we probably can't help you 
with that), then fix the bugs in your code and only then revisit the "speed 
issue". 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: is_ as method or property?

[Chris Angelico]

On Fri, Dec 12, 2014 at 10:21 PM, Mateusz Loskot  wrote:
> I've got several cases which are not obvious to me.
> For instance, class Foo has a boolean attribute, read-write,
> which I see a couple of realisations for possible:
>

0) Attribute only.

class Foo:
pass

Foo().default = True

Unless you need something more than this, go with this style. Much simpler.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: is_ as method or property?

[Mateusz Loskot]

On 11 December 2014 at 19:20, Chris Angelico  wrote:
> On Fri, Dec 12, 2014 at 4:34 AM, Mateusz Loskot  wrote:
>> If a class member function simply tests something and
>> returns a b::oolean call it
>>
>> def is_():
>>  pass
>>
>> like 'is_even'.
>>
>> Should I define it as a classic method
>>
>> def is_even(self):
>> pass
>>
>> or as a property
>>
>> @property
>> def is_even(self):
>> pass
>
> A property should be used if what you're creating is "virtually an
> attribute". If it would make sense to have an attribute is_even, then
> a property is_even makes sense. If in doubt, go with a method.

Thanks for the advise.

I've got several cases which are not obvious to me.
For instance, class Foo has a boolean attribute, read-write,
which I see a couple of realisations for possible:

1) properties only
class Foo:
  @property
  def is_default(self):
pass

  @is_default.setter
  def is_default(self, current):
pass

2) property + method mix
class Foo:
  @property
  def is_default(self):
pass

  def make_default(self, current):
pass

3) methods only
class Foo:
  def is_default(self):
pass

  def make_default(self, current):
pass

>From one angle, that is is_default is known and does not need to be
calculated upon every query, the option 1) fits well.
It is aligned with Ethan's suggestion in the other post.

>From other point, let's assume updating it takes a little bit of changes of
state of Foo instance, then perhaps methods-only option 3) fits best.

Looks like "if in doubt, go with a method" is the safest bet.
'Statistics' from Python codebase also seem to suggest that.

Best regards,
-- 
Mateusz  Loskot, http://mateusz.loskot.net
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list comprehension return a list and sum over in loop

[Jussi Piitulainen]

KK Sasa writes:

> def p(x,t,point,z,obs):
> d = x[0]
> tau = [0]+[x[1:point]] 
> a = x[point:len(x)]
> at = sum(i*j for i, j in zip(a, t))
> nu = [exp(z[k]*(at-d)-sum(tau[k])) for k in xrange(point)]
> de = sum(nu, axis=0)
> probability = [nu[k]/de for k in xrange(point)]
> return probability[obs]

I must be blind, but this looks like computing a whole probability
distribution and then throwing almost all of it away.

Can't this just return nu[obs]/de?

The expression for tau also seems weird to me. Isn't it equivalent to
[0, x[1:point]], a two-element list with the second element a list?
How can sum(tau[k]) work at all then, for any k > 1?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [newbie] how to make program suggest to install missing modules

[hugocoolens]

On Monday, December 8, 2014 9:00:13 PM UTC+1, sohca...@gmail.com wrote:
> On Monday, December 8, 2014 10:46:47 AM UTC-8, Jean-Michel Pichavant wrote:
> > - Original Message -
> > > From: sohcahto...@gmail.com
> > > try:
> > > import someModule
> > > except ImportError:
> > > print "Module is missing"
> > > # handle it!
> > > 
> > > Just make sure to attempt to import it again after making the call to
> > > pip to install it.
> > 
> > Note that ImportError may be raised for other reasons than a missing module.
> > 
> > Check https://docs.python.org/2/library/imp.html and the imp.find_module, 
> > it could be a safer way to check for a missing module.
> > 
> > JM
> > 
> > 
> > -- IMPORTANT NOTICE: 
> > 
> > The contents of this email and any attachments are confidential and may 
> > also be privileged. If you are not the intended recipient, please notify 
> > the sender immediately and do not disclose the contents to any other 
> > person, use it for any purpose, or store or copy the information in any 
> > medium. Thank you.
> Good point.
> Of course, imp.find_module ALSO throws ImportError if the module can't be 
> found, but at least in that case, you'd know the exact cause.

Thanks for the suggestions, you can see here below what I came up with.
All suggestions/corrections welcome:

#!/usr/bin/env python
import imp
import os
import sys
try:
imp.find_module('rtlsdr')
except ImportError:
print('Module rtlsdr is missing')
print("I'll try to install it")
os.system('sudo pip install pyrtlsdr')
try:
imp.find_module('rtlsdr')
except ImportError:
sys.exit('Sorry could not install module rtlsdr, contact your
local Python-guru')
import rtlsdr
print('Module rtlsdr succesfully imported')



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Extension of while syntax

[Marko Rauhamaa]

c...@isbd.net:

> Marko Rauhamaa  wrote:
>> Chris Angelico :
>> 
>> > You could deduplicate it by shifting the condition:
>> >
>> > while True:
>> > value = get_some_value()
>> > if value not in undesired_values: break
>> >
>> > But I'm not sure how common this idiom actually is.
>> 
>> Extremely common, and not only in Python.
>> 
> It's the classic C 'for' loop.

It's the classic:

for (;;) {
...
if (...)
break;
...
}


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list comprehension return a list and sum over in loop

[Steven D'Aprano]

KK Sasa wrote:

> Hi there,
> 
> The list comprehension is results = [d2(t[k]) for k in xrange(1000)],
> where d2 is a function returning a list, say [x1,x2,x3,x4] for one
> example. So "results" is a list consisting of 1000 lists, each of length
> four. Here, what I want to get is the sum of 1000 lists, and then the
> result is a list of length four. Is there any efficient way to do this?
> Because I found it is slow in my case. I tried sum(d2(t[k]) for k in
> xrange(1000)), but it returned error: TypeError: unsupported operand
> type(s) for +: 'int' and 'list'. Thanks.

That's because sum() defaults to adding with a default value of 0:

py> sum([[1, 2], [3, 4]], 0)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unsupported operand type(s) for +: 'int' and 'list'
py> sum([[1, 2], [3, 4]], [])
[1, 2, 3, 4]


But don't do that! It will be slow.


I don't completely understand your requirements. You should show some
typical data, and the expected result. The business with the xrange and
t[k] and even d2() is probably irrelevant. The important part is summing
the lists. That could mean either of these two things:

results = [ [1, 2, 3, 4], 
[5, 6, 7, 8],
[9, 10, 11, 12]]

sum(results)
--> gives [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]

The way to do this efficiently is:

results = []
for sublist in [d2(t[k]) for k in xrange(1000)]:
results.extend(sublist)


Or perhaps you mean this:


results = [ [1, 2, 3, 4], 
[5, 6, 7, 8],
[9, 10, 11, 12]]

sum(results)
--> gives [15, 18, 21, 24]


I expect that the fastest, most efficient way to do this will be with the
third-party library numpy. But in pure Python, you can do this:

def add(alist, blist):
if len(blist) != len(alist): raise ValueError
for i, x in blist:
alist[i] += x


results = [0]*4  # Like [0,0,0,0]
for sublist in [d2(t[k]) for k in xrange(1000)]:
add(results, sublist)





-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list comprehension return a list and sum over in loop

[KK Sasa]

Peter Otten於 2014年12月12日星期五UTC+8下午5時13分58秒寫道：
> KK Sasa wrote:
> 
> > Mark Lawrence於 2014年12月12日星期五UTC+8下午3時17分43秒寫道：
> >> On 12/12/2014 06:22, KK Sasa wrote:
> >> > Hi there,
> >> >
> >> > The list comprehension is results = [d2(t[k]) for k in xrange(1000)],
> >> > where d2 is a function returning a list, say [x1,x2,x3,x4] for one
> >> > example. So "results" is a list consisting of 1000 lists, each of
> >> > length four. Here, what I want to get is the sum of 1000 lists, and
> >> > then the result is a list of length four. Is there any efficient way to
> >> > do this? Because I found it is slow in my case. I tried sum(d2(t[k])
> >> > for k in xrange(1000)), but it returned error: TypeError: unsupported
> >> > operand type(s) for +: 'int' and 'list'. Thanks.
> >> >
> >> 
> >> I think you need something like this
> >> http://stackoverflow.com/questions/19339/a-transpose-unzip-function-in-python-inverse-of-zip
> >> 
> >> I'll let you add the finishing touches if I'm correct :)
> >> 
> >> --
> >> My fellow Pythonistas, ask not what our language can do for you, ask
> >> what you can do for our language.
> >> 
> >> Mark Lawrence
> > 
> > Hi Mark and Yotam,
> >   Thanks for kind reply. I think I didn't make my problem clear enough.
> >   The slow part is "[d2(t[k]) for k in xrange(1000)]". In addition, I
> >   don't need to construct a list of 1000 lists inside, but my aim is to
> >   get the sum of all "d2(t[k])". I wonder if there is any method to sum up
> >   efficiently.
> 
> If that is slow the culprit is probably the d2() function. If so 
> 
> results = [0] * 4
> for k in xrange(1000):
> for i, v in enumerate(d2(t[k])):
> results[i] += v
> 
> won't help. Can you tell us what's inside d2()?

Thanks for reply, Christian and Peter. Actually, the d2() is the Hessian 
function of a simple function (derived by using "ad" package, 
http://pythonhosted.org//ad/). The package hasn't supported Numpy array so far. 
And I am still not how to take a look inside d2(). Because I have to use d2() 
in a heavy way for iterative algorithm, the efficiency is a issue in my case. 
Following is an example.

import scipy
from scipy import stats
import numpy
from ad import adnumber
from ad.admath import *
from ad import jacobian
from ad import gh  # the gradient and hessian functions generator
from ad import *
import time
people = 1000
range_people = xrange(people)
dim0 = 2; mean0 = [0,0]; cov0 = [[1,0],[0,1]]
seed([1])
t = stats.multivariate_normal.rvs(mean0,cov0,people)
t = t.reshape(people,dim0)
t = t.tolist() # back to list
x = [0, 0, 1, 2]
point = 2
def p(x,t,point,z,obs):
d = x[0]
tau = [0]+[x[1:point]] 
a = x[point:len(x)]
at = sum(i*j for i, j in zip(a, t))
nu = [exp(z[k]*(at-d)-sum(tau[k])) for k in xrange(point)]
de = sum(nu, axis=0)
probability = [nu[k]/de for k in xrange(point)]
return probability[obs]

d1, d2 = gh(p)
tStart = time.time()
z = range(point)
re = [d2(x,t[k],2,z,1) for k in range_people]
tEnd = time.time()
print "It cost %f sec" % (tEnd - tStart)
-- 
https://mail.python.org/mailman/listinfo/python-list

encoding name mappings in codecs.py with email/charset.py

[Stefanos Karasavvidis]

I've hit a wall with mailman which seems to be caused by pyhon's character
encoding names.

I've narrowed the problem down to the email/charset.py file. Basically the
following happens:

given an encoding name as 'iso-8859-X' it is transformed to 'iso8859-X'
(without the first dash). This happens with python 2.7, but not with python
3.4. Now Microsoft Exchange doesn't like the form without the dash, and
bounces the emails from Mailman. And Mailman doesn't work with python 3.x.

This transformation is done in charset.py with the following line
input_charset = codecs.lookup(input_charset).name

The following code example demonstrates the issue
   from email.charset import Charset
   charset = Charset('iso-8859-7')
   print(str(charset))

In python 2.7, iso8859-7 is printed. In python 3.4 iso-8859-7 is printed.

I tried to find the location of these mappings in the codecs.py file, but
it seems that it uses some internal mapping I couldn't find. And I'm not
100% sure that this is not OS related.

So the question basically is if there is a way to change the name mappings
this codecs file does.

My environment is Ubuntu 14.04
python2.7 --version
Python 2.7.6

python3.4 --version
Python 3.4.0

-- 
==
Stefanos Karasavvidis,  Electronic & Computer Engineer, M.Sc.
e-mail: s...@isc.tuc.gr, Tel.: (+30) 2821037508, Fax: (+30)
2821037520
Technical University of Crete, Campus, Building A1
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Extension of while syntax

[cl]

Marko Rauhamaa  wrote:
> Chris Angelico :
> 
> > You could deduplicate it by shifting the condition:
> >
> > while True:
> > value = get_some_value()
> > if value not in undesired_values: break
> >
> > But I'm not sure how common this idiom actually is.
> 
> Extremely common, and not only in Python.
> 
It's the classic C 'for' loop.

'for' in C is essentially a while with an initialiser for the loop
variable.

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list comprehension return a list and sum over in loop

[Peter Otten]

KK Sasa wrote:

> Mark Lawrence於 2014年12月12日星期五UTC+8下午3時17分43秒寫道：
>> On 12/12/2014 06:22, KK Sasa wrote:
>> > Hi there,
>> >
>> > The list comprehension is results = [d2(t[k]) for k in xrange(1000)],
>> > where d2 is a function returning a list, say [x1,x2,x3,x4] for one
>> > example. So "results" is a list consisting of 1000 lists, each of
>> > length four. Here, what I want to get is the sum of 1000 lists, and
>> > then the result is a list of length four. Is there any efficient way to
>> > do this? Because I found it is slow in my case. I tried sum(d2(t[k])
>> > for k in xrange(1000)), but it returned error: TypeError: unsupported
>> > operand type(s) for +: 'int' and 'list'. Thanks.
>> >
>> 
>> I think you need something like this
>> http://stackoverflow.com/questions/19339/a-transpose-unzip-function-in-python-inverse-of-zip
>> 
>> I'll let you add the finishing touches if I'm correct :)
>> 
>> --
>> My fellow Pythonistas, ask not what our language can do for you, ask
>> what you can do for our language.
>> 
>> Mark Lawrence
> 
> Hi Mark and Yotam,
>   Thanks for kind reply. I think I didn't make my problem clear enough.
>   The slow part is "[d2(t[k]) for k in xrange(1000)]". In addition, I
>   don't need to construct a list of 1000 lists inside, but my aim is to
>   get the sum of all "d2(t[k])". I wonder if there is any method to sum up
>   efficiently.

If that is slow the culprit is probably the d2() function. If so 

results = [0] * 4
for k in xrange(1000):
for i, v in enumerate(d2(t[k])):
results[i] += v

won't help. Can you tell us what's inside d2()?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list comprehension return a list and sum over in loop

[Christian Gollwitzer]

Am 12.12.14 09:30, schrieb KK Sasa:

Mark Lawrence於 2014年12月12日星期五UTC+8下午3時17分43秒寫道：
Hi Mark and Yotam, Thanks for kind reply. I think I didn't make my
problem clear enough. The slow part is "[d2(t[k]) for k in
xrange(1000)]". In addition, I don't need to construct a list of 1000
lists inside, but my aim is to get the sum of all "d2(t[k])". I
wonder if there is any method to sum up efficiently.


Not sure I understand what you need, but it seems that NumPy would be a 
more efficient method. numpy.sum can sum elements along every dimensions 
of a higher-dimensional matrix, and in general NumPy stores the elements 
in native doubles instead of Python lists, which improves both 
performance and memory usage


Christian

--
https://mail.python.org/mailman/listinfo/python-list

How to make subprocess run for 60 sec?

[Robert Clove]

Hi All,

I have the following python script that runs.
I want is to run the subprocess to run for 60 sec and then send the SIGINT
signal to subprocess and write the output in file.

#!/usr/bin/python
import os
import subprocess
PIPE = subprocess.PIPE
import signal
import time

def handler(signum, frame):
pass

signal.signal(signal.SIGALRM, handler)
signal.alarm(60)
command = "strace -c ./server"
os.chdir("/root/Desktop/")
p = subprocess.Popen(command, stdout=PIPE, stderr=PIPE)
time.sleep(60)
p.send_signal(signal.SIGINT)
signal.alarm(0)
print p.communicate()[1]


Regards
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PyQt: user interface freezed when using concurrent.futures.ThreadPoolExecutor

[Marko Rauhamaa]

Chris Angelico :

> And I don't remember how Java did things, except that I struggled to
> find basic fundamental primitives like semaphores, and had to use
> synchronized functions/objects instead.

Java now has a diverse set of synchornization facilities, but the
builtin object synchronization produces excellent idioms and is usually
preferred (by me). Java also has rigorously defined its multithreaded
data model (the Happens-Before Relation).

Java's threads have two major problems:

 * The classic I/O forced you to dedicate a thread for each
   communication context (connection) because there was no multiplexing
   facility and because the sockets were blocking (and buffered IIRC).
   The thread proliferation caused serious scalability issues.
   Latter-day Java's NIO framework addresses this shortcoming to a great
   degree, but even that is a surprisingly tricky beast to program
   for -- it suffers from builtin race conditions.

 * There is no way to interrupt a thread -- except in Solaris! You can
   mark a thread for interruption and there is an associated exception
   but they are not guaranteed to be provided by JVM.

And then there's the inherent problems of thread programming:

 * Deadlocks.

 * Missing synchronization.

which in practice are just too hard for mortals. I've seen it. I've been
complicit.

Now, when it comes to Python, asyncio is its answer to Java's NIO. It
seeks to provide a cross-platform Way to Life, Universe and Everything.
A commendable objective. Unfortunately, I find the coroutine approach
artificial and unintuitive in practice. Select.epoll(EPOLLET) plus a
timer implementation easily beats it (as long as you can limit yourself
to linux).


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list comprehension return a list and sum over in loop

[KK Sasa]

Mark Lawrence於 2014年12月12日星期五UTC+8下午3時17分43秒寫道：
> On 12/12/2014 06:22, KK Sasa wrote:
> > Hi there,
> >
> > The list comprehension is results = [d2(t[k]) for k in xrange(1000)], where 
> > d2 is a function returning a list, say [x1,x2,x3,x4] for one example. So 
> > "results" is a list consisting of 1000 lists, each of length four. Here, 
> > what I want to get is the sum of 1000 lists, and then the result is a list 
> > of length four. Is there any efficient way to do this? Because I found it 
> > is slow in my case. I tried sum(d2(t[k]) for k in xrange(1000)), but it 
> > returned error: TypeError: unsupported operand type(s) for +: 'int' and 
> > 'list'. Thanks.
> >
> 
> I think you need something like this 
> http://stackoverflow.com/questions/19339/a-transpose-unzip-function-in-python-inverse-of-zip
> 
> I'll let you add the finishing touches if I'm correct :)
> 
> -- 
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
> 
> Mark Lawrence

Hi Mark and Yotam,
  Thanks for kind reply. I think I didn't make my problem clear enough. The 
slow part is "[d2(t[k]) for k in xrange(1000)]". In addition, I don't need to 
construct a list of 1000 lists inside, but my aim is to get the sum of all 
"d2(t[k])". I wonder if there is any method to sum up efficiently.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Extension of while syntax

[Marko Rauhamaa]

Chris Angelico :

> On Fri, Dec 12, 2014 at 6:10 PM, Marko Rauhamaa  wrote:
>> Chris Angelico :
>>
>>> You could deduplicate it by shifting the condition:
>>>
>>> while True:
>>> value = get_some_value()
>>> if value not in undesired_values: break
>>>
>>> But I'm not sure how common this idiom actually is.
>>
>> Extremely common, and not only in Python.
>
> Something like it is certainly common, but to justify dedicated
> syntax, the pure form has to be so amazingly common as to merit it.

You already showed the perfect dedicated syntax.

The variations of the while-True-...-break idiom are limitless. There's
little need to "optimize" it further.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Extension of while syntax

[Chris Angelico]

On Fri, Dec 12, 2014 at 7:00 PM, Mark Lawrence  wrote:
> It won't happen as different format loops have been discussed and rejected
> umpteen times over the last 20 odd years, mainly because the code can be
> restructured using break as others have already pointed out.  Unless of
> course you fork Python, joining others working on variants such as Python
> 2.8 or RickedPython :)

RickRolledPython, the variant in which errors become warnings because
it's never gonna give you up.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Extension of while syntax

[Mark Lawrence]

On 12/12/2014 02:21, Nelson Crosby wrote:

I was thinking a bit about the following pattern:

value = get_some_value()
while value in undesired_values:
 value = get_some_value()

I've always hated code that looks like this. Partly due to the repetition, but 
partly also due to the fact that without being able to immediately recognise 
this pattern, it isn't very readable.

Python already has one-line syntaxes (e.g. list comprehensions), I was 
wondering what people thought about a similar thing with while. It might look 
something like:

value = get_some_value() while value in undesired_values()

Perhaps not this exact syntax though, as the parser might try to do `value = 
(get_some_value() while...)` instead of `(value = get_some_value) while...`.

Other languages have features which allow something to look slightly less like 
this pattern, e.g. Java:

SomeType value;
while ((/* The assignment */ value = getSomeValue()) /* Compare */ == 
undesired_value) {}

Granted, this isn't exactly tidy, but it's a little more DRY and, IMO, 
preferable.

What are other's thoughts on this?



It won't happen as different format loops have been discussed and 
rejected umpteen times over the last 20 odd years, mainly because the 
code can be restructured using break as others have already pointed out. 
 Unless of course you fork Python, joining others working on variants 
such as Python 2.8 or RickedPython :)


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list