Re: pip3 : command not found

[Ben Finney]

Jon Ribbens  writes:

> He […] lied about me "not arguing in good faith"

I find you to be not arguing in good faith; if you consistently engage
someone in a manner that requires response, that's not consistent with
also “wondering when they'd shut up”.

> So yes, I view him bowing out of the conversation as an unmitigated
> blessing.

You could have disengaged at any time, so I don't find your statement
believable.

-- 
 \   “Firmness in decision is often merely a form of stupidity. It |
  `\indicates an inability to think the same thing out twice.” |
_o__)—Henry L. Mencken |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pip3 : command not found

[Jon Ribbens]

On 2016-11-06, Ben Finney  wrote:
> Jon Ribbens  writes:
>> On 2016-11-06, Steve D'Aprano  wrote:
>> > *plonk*
>>
>> Thank feck for that, I was beginning to think he'd never shut up.
>
> Really? I didn't see a single message from Steven that wasn't solicited
> by an assertion from you that needed response.
>
> If your concept of “won't shut up” includes “responds to me when I
> demand he acknowledge my position is correct”, then it seems clear the
> problem has a different solution.

He consistently misunderstood almost everything I said (which is
a pattern of behaviour he seems to have, i.e. heroically
misunderstanding people so he can argue with them), lied about
what I had said, lied about what he had said, lied about me "not
arguing in good faith", was rude and insulting, and studiously avoided
the actual interesting and useful topic in favour of a stupid and
pointless flamewar about his misunderstanding of my original question.

He clearly didn't know the answer to the question nor did he have any
interest in dicussing it in a civilised manner with a view to
discovering the answer, he just wants an argument. So yes, I view him
bowing out of the conversation as an unmitigated blessing.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pip3 : command not found

[Jon Ribbens]

On 2016-11-06, Chris Angelico  wrote:
> On Sun, Nov 6, 2016 at 3:03 PM, Jon Ribbens  wrote:
>> I don't suppose anyone else more constructive and informed actually
>> knows the answer to my rather simple question of how Python knows
>> it's in a venv? ;-)
>
> Two ways.
>
> 1) Normally, you 'activate' the venv by sourcing a script into your
> shell. This modifies $PATH, $PYTHONHOME, and I think a couple of other
> environment variables.

It sets VIRTUAL_ENV, adds the virtualenv's bin directory to PATH,
and *unsets* PYTHONHOME if set. That's it (modulo storing old values
of variables and updating PS1 which is presumably just cosmetic).

> 2) If Python notices that its executable comes from a venv, it uses it.

Yes. My question is *how does it notice*?
-- 
https://mail.python.org/mailman/listinfo/python-list

Delete h2 until you reach the next h2 in beautifulsoup

[rosefox911]

Considering the following html:

cool stuff  hizz  

and the following list:

ignore_list = ['example','lalala']

My goal is, while going through the HTML using Beautifulsoup, I find a h2 that 
has an ID that is in my list (ignore_list) I should delete all the ul and lis 
under it until I find another h2. I would then check if the next h2 was in my 
ignore list, if it is, delete all the ul and lis until I reach the next h2 (or 
if there are no h2s left, delete the ul and lis under the current one and 
stop). 

How I see the process going: you read all the h2s from up to down in the DOM. 
If the id for any of those is in the ignore_list, then delete all the ul and li 
under the h2 until you reach the NEXT h2. If there is no h2, then delete the ul 
and LI then stop.

Here is the full HMTL I am trying to work with: http://pastebin.com/Z3ev9c8N

I am trying to delete all the UL and lis after "See_also"How would I accomplish 
this in Python?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pip3 : command not found

[Ben Finney]

Jon Ribbens  writes:

> On 2016-11-06, Steve D'Aprano  wrote:
> > *plonk*
>
> Thank feck for that, I was beginning to think he'd never shut up.

Really? I didn't see a single message from Steven that wasn't solicited
by an assertion from you that needed response.

If your concept of “won't shut up” includes “responds to me when I
demand he acknowledge my position is correct”, then it seems clear the
problem has a different solution.

-- 
 \   “Prayer must never be answered: if it is, it ceases to be |
  `\   prayer and becomes correspondence.” —Oscar Wilde, _The Epigrams |
_o__)of Oscar Wilde_, 1952 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue23262] webbrowser module broken with Firefox 36+

[Oleg Broytman]

Oleg Broytman added the comment:

> I'm not sure that we can break the compatibility with old browser

I agree with this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: pip3 : command not found

[Chris Angelico]

On Sun, Nov 6, 2016 at 3:03 PM, Jon Ribbens  wrote:
> I don't suppose anyone else more constructive and informed actually
> knows the answer to my rather simple question of how Python knows
> it's in a venv? ;-)

Two ways.

1) Normally, you 'activate' the venv by sourcing a script into your
shell. This modifies $PATH, $PYTHONHOME, and I think a couple of other
environment variables.

2) If Python notices that its executable comes from a venv, it uses it.

(I wasn't aware of #2 until this thread.)

Generally, you type "python" or "python3" and the shell searches
$PATH. If you have a venv active, it'll find the one there before it
finds the system one.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pip3 : command not found

[Jon Ribbens]

On 2016-11-06, Steve D'Aprano  wrote:
> *plonk*

Thank feck for that, I was beginning to think he'd never shut up.

I don't suppose anyone else more constructive and informed actually
knows the answer to my rather simple question of how Python knows
it's in a venv? ;-)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: distributed development methodology

[Rustom Mody]

On Saturday, October 29, 2016 at 1:20:25 PM UTC+5:30, Paul Rubin wrote:
> Adam Jensen  writes:
> > So what are some of the more successful distributed. multi-platform,
> > development models?
> 
> Use an orchestration program to keep the systems in sync: I use ansible
> (ansible.com) which is written in Python and fairly simple once you get
> used to it, but there are lots of other programs in that space and
> everyone has their own preferences.
> 
> If stuff doesn't change too often, you could also use Docker or whatever
> the current hotness is to make a container image and deploy it to your
> fleet.

Been seeing things like this of late:
https://thehftguy.wordpress.com/2016/11/01/docker-in-production-an-history-of-failure/
[Disclaimer: No direct experience myself one way or other]

LXC and others compared: 
https://www.flockport.com/alternatives-to-docker-and-lxc/
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue23218] Modernize the IDLE Find/Replace/Find in Files dialogs

[Al Sweigart]

Al Sweigart added the comment:

*Bump* Just wanted to bring attention to this issue. We could keep "Regular 
expression" instead of "Regex" for the label (Sublime Text and other editors 
have "Regular expression")

I think Mark's patch would be better over mine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28617] Why isn't "in" called a comparison operation?

[Raymond Hettinger]

Raymond Hettinger added the comment:

newpatch.diff looks fine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28524] Set default argument of logging.disable() to logging.CRITICAL

[Al Sweigart]

Al Sweigart added the comment:

Setting up different configurations for dev/prod is a bit more complicated than 
I'd like for most projects. I'd instead just call 
logging.disable(logging.CRITICAL).

The entire point of this is just for the convenience of being able to disable 
logging messages by calling logging.disable() instead of 
logging.disable(logging.CRITICAL).

It's a two-line change, backwards compatible, and (imo) a sensible default. You 
call logging.disable() expecting it to disable logging. You might want to 
disable a lower level, but as the Google search shows, most people just want to 
disable all logging period.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: [Theory] How to speed up python code execution / pypy vs GPU

[Ben Bacarisse]

Steve D'Aprano  writes:

> On Sun, 6 Nov 2016 08:17 am, Ben Bacarisse wrote:
>
>> Steve D'Aprano  writes:
>
>>> Here's the same program in Objective C:
>>>
>>> --- cut ---
>>>
>>> #import 
>>>
>>> int main (int argc, const char * argv[])
>>> {
>>> NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
>>> NSLog (@"Hello, World!");
>>> [pool drain];
>>> return 0;
>>> }
>>>
>>> --- cut ---
>>>
>>> Which would you rather write?
>> 
>> That's a rather odd comparison.  Why not
>> 
>>   #import 
>> 
>>   int main()
>>   {
>> printf("Hello world\n");
>> return 0;
>>   }
>
> Because that's not Objective-C? (This is not a rhetorical question.)

Yes, I think so.

> I'm not an Objective-C expert, but to my eye, that doesn't look like
> Objective-C. It looks like plain old regular C.

C has no #import directive.


-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Theory] How to speed up python code execution / pypy vs GPU

[Steve D'Aprano]

On Sun, 6 Nov 2016 09:17 am, Mr. Wrobel wrote:

> However the most important is second part of my question.
> 
> What do you think about using GPU processing or pypy?

I don't have any experience with GPU processing. I expect that it will be
useful for somethings, but for number-crushing and numeric work, I am
concerned that GPUs rarely provide correctly rounded IEEE-754 maths. That
means that they are accurate enough for games where a few visual glitches
don't matter, but they risk being inaccurate for serious work.

I fear that doing numeric work in GPUs will be returning to the 1970s, when
every computer was incompatible with every other computer, and it was
almost impossible to write cross-platform, correct, accurate numeric code.

As far as PyPy goes, it is a proven optimizing Python JIT compiler that can
speed-up long-running code very well. It is not suitable for short scripts
or programmers that run quickly. It takes time for the JIT to warm up and
start showing optimizations.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pip3 : command not found

[Steve D'Aprano]

On Sun, 6 Nov 2016 07:55 am, Jon Ribbens wrote:

>> The implication is that the answer to your question is Yes, you can run
>> Python in the context of a virtualenv by just invoking that virtualenv's
>> local Python without running 'activate' first.
> 
> So you were wrong earlier when you said you couldn't do that?

I'm happy to admit it when I have made a mistake, but this isn't one of
those times. I didn't say it couldn't be done, I initially said:

"Possibly you do have to run `activate` first, I don't know"

but you already know that, since you can read just as well as I can.



> You're still wrong no matter how many times you repeat yourself.
> I've understood him, you have understood neither him nor me.
[...]
> You're spending an awful lot of effort being pointlessly argumentative
> about who has or hasn't understood what while avoiding addressing the
> actual perfectly simple question - if running Python directly from a
> venv path is equivalent to running 'activate' then Python, how does it
> know that it's in a venv?

And now you change your story for the second time.

As I said, I'm happy to admit when I am wrong. I was wrong to think you were
discussing this in good faith. You're clearly not: you keep changing your
story, keep insisting that the question you have asked is different from
what you previously wrong.

First you asked about symlinks, misunderstanding what Ben wrote; then you
denied you asked that and insisted you really asked about activate (despite
not having even mentioned activate); and now you're inventing a new
question, "how does it know it is in a venv?".

Life is too short to waste time in a pointless discussion with trolls who
argue in bad faith, as you are doing. Welcome to my kill-file.

*plonk*





-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Theory] How to speed up python code execution / pypy vs GPU

[Steve D'Aprano]

On Sun, 6 Nov 2016 08:17 am, Ben Bacarisse wrote:

> Steve D'Aprano  writes:

>> Here's the same program in Objective C:
>>
>> --- cut ---
>>
>> #import 
>>
>> int main (int argc, const char * argv[])
>> {
>> NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
>> NSLog (@"Hello, World!");
>> [pool drain];
>> return 0;
>> }
>>
>> --- cut ---
>>
>> Which would you rather write?
> 
> That's a rather odd comparison.  Why not
> 
>   #import 
> 
>   int main()
>   {
> printf("Hello world\n");
> return 0;
>   }


Because that's not Objective-C? (This is not a rhetorical question.)

I'm not an Objective-C expert, but to my eye, that doesn't look like
Objective-C. It looks like plain old regular C.

Here's where I stole the code from:

https://www.binpress.com/tutorial/objectivec-lesson-1-hello-world/41

and its not too dissimilar from the versions here:

http://rosettacode.org/wiki/Hello_world/Text#Objective-C


> ?  It's decades since I wrote any Objective-C (and then not much) but I
> think this is the closest comparison.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pip3 : command not found

[Ben Finney]

Steve D'Aprano  writes:

> Oh, and ironically, you'll see that Ben anticipated and answered your
> question about activate. Here's the link again, to save you scrolling up:
>
> https://mail.python.org/pipermail/python-list/2016-October/715994.html
>
> where he says:
>
> If you already have a specific environment active and know that
> ‘python3’ is the correct Python interpreter from that environment,
> you can omit the explicit path.

That's what I said, and Steven is reading its meaning correctly.

-- 
 \   “You can be a theist, and you can be a skeptic. But if you're |
  `\  both, you're not very good at one of them.” —Dave Silverman, |
_o__)   2011-11-19 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Theory] How to speed up python code execution / pypy vs GPU

[Ben Bacarisse]

"Mr. Wrobel"  writes:

> ... However the most important is second part of my question.
>
> What do you think about using GPU processing or pypy?

Sorry, I don't have enough experience of them to offer any useful advice.

-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue28618] Decorate hot functions using __attribute__((hot)) to optimize Python

[STINNER Victor]

STINNER Victor added the comment:

Antoine Pitrou added the comment:
>> Do you mean comparison between current Python with PGO and patched
>> Python without PGO?
>
> Yes.

Oh ok, sure. I will try to run these 2 benchmarks.

>>> Ubuntu 14.04 is old, and I don't think this is something we should worry 
>>> about.
>>
>> Well, it's a practical issue for me to run benchmarks for speed.python.org.
>
> Why isn't the OS updated on that machine?

I am not sure that I want to use PGO compilation to run benchmarks.
Last time I checked, I noticed performance differences between two
compilations. PGO compilation doesn't seem 100% deterministic.

Maybe PGO compilation is fine when you build Python to create a Linux
package. But to get reliable benchmarks, I'm not sure that it's a good
idea.

I'm still running benchmarks on Python recompiled many times using
different compiler options, to measure the impact of the compiler
options (especially LTO and/or PGO) on the benchmark stability.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: [Theory] How to speed up python code execution / pypy vs GPU

[Mr. Wrobel]

W dniu 05.11.2016 o 22:17, Ben Bacarisse pisze:

Steve D'Aprano  writes:


On Sun, 6 Nov 2016 04:10 am, Mr. Wrobel wrote:


Hi,

Some skeptics asked my why there is a reason to use Python against of
any other "not interpreted" languages, like objective-C.


Here's the "Hello World" program in Python:

--- cut ---

print("Hello World")

--- cut ---



Here's the same program in Objective C:

--- cut ---

#import 

int main (int argc, const char * argv[])
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSLog (@"Hello, World!");
[pool drain];
return 0;
}

--- cut ---

Which would you rather write?


That's a rather odd comparison.  Why not

  #import 

  int main()
  {
printf("Hello world\n");
return 0;
  }

?  It's decades since I wrote any Objective-C (and then not much) but I
think this is the closest comparison.


Wel, indeed. However the most important is second part of my question.

What do you think about using GPU processing or pypy?
--
https://mail.python.org/mailman/listinfo/python-list

[issue28621] Refactor duplicate code calculating digit's bit length

[SilentGhost]

Changes by SilentGhost :


--
nosy: +mark.dickinson
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: [Theory] How to speed up python code execution / pypy vs GPU

[Ben Bacarisse]

Steve D'Aprano  writes:

> On Sun, 6 Nov 2016 04:10 am, Mr. Wrobel wrote:
>
>> Hi,
>> 
>> Some skeptics asked my why there is a reason to use Python against of
>> any other "not interpreted" languages, like objective-C.
>
> Here's the "Hello World" program in Python:
>
> --- cut ---
>
> print("Hello World")
>
> --- cut ---
>
>
>
> Here's the same program in Objective C:
>
> --- cut ---
>
> #import 
>
> int main (int argc, const char * argv[])
> {
> NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
> NSLog (@"Hello, World!");
> [pool drain];
> return 0;
> }
>
> --- cut ---
>
> Which would you rather write?

That's a rather odd comparison.  Why not

  #import 

  int main()
  {
printf("Hello world\n");
return 0;
  }

?  It's decades since I wrote any Objective-C (and then not much) but I
think this is the closest comparison.

-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pip3 : command not found

[Jon Ribbens]

On 2016-11-05, Steve D'Aprano  wrote:
> On Sun, 6 Nov 2016 02:55 am, Jon Ribbens wrote:
>> I'm afraid I can only suggest that you try re-reading the subthread
>> again until you manage to understand it. It wasn't really that
>> complicated but you seem to have confused yourself greatly.
>
> Are you serious?

Yes. You haven't understood the thread, and I don't know how to
explain it any simpler, so I'm afraid all I can suggest is that
you re-read what was already written.

> I can only repeat yet again: you have not understood Ben.

You're still wrong no matter how many times you repeat yourself.
I've understood him, you have understood neither him nor me.
You've got hung up on the pip3 issue which is blinding you to
the other implications of Ben's answer.

Threads can and usually do diverge to a greater or lesser degree. Just
because a post is a follow-up in a thread doesn't mean that post is
entirely and solely about the original topic of that thread. Try
dropping your preconceptions and reading the sub-thread again from the
post before my first post without making false assumptions and while
bearing in mind that I am asking a question that is tangential to the
original question rather than directly descended from it.

> The implication is that the answer to your question is Yes, you can run
> Python in the context of a virtualenv by just invoking that virtualenv's
> local Python without running 'activate' first.

So you were wrong earlier when you said you couldn't do that?

You're spending an awful lot of effort being pointlessly argumentative
about who has or hasn't understood what while avoiding addressing the
actual perfectly simple question - if running Python directly from a
venv path is equivalent to running 'activate' then Python, how does it
know that it's in a venv? (Yes, I know it can sort-of work out where
it's run from using argv[0] but how does it know it's *in a venv*?)
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue28618] Decorate hot functions using __attribute__((hot)) to optimize Python

[Antoine Pitrou]

Antoine Pitrou added the comment:

Le 05/11/2016 à 16:37, STINNER Victor a écrit :
> 
> Antoine Pitrou added the comment:
>> Can you compare against a PGO build?
> 
> Do you mean comparison between current Python with PGO and patched
> Python without PGO?

Yes.

>> Ubuntu 14.04 is old, and I don't think this is something we should worry 
>> about.
> 
> Well, it's a practical issue for me to run benchmarks for speed.python.org.

Why isn't the OS updated on that machine?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26935] android: test_os fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

New patch. Thanks for the review Martin!

--
Added file: http://bugs.python.org/file45368/test_urandom_fd_reopened_3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28621] Refactor duplicate code calculating digit's bit length

[Adrian Wielgosik]

Changes by Adrian Wielgosik :


--
keywords: +patch
Added file: http://bugs.python.org/file45367/bit_length.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28621] Refactor duplicate code calculating digit's bit length

[Adrian Wielgosik]

New submission from Adrian Wielgosik:

The attached patch uses an existing function bits_in_digit() in two other 
functions:

- in long_bit_length() - it already had identical logic
- in _PyLong_NumBits() - it used a naive, slower way of calculating bit length, 
so as an added bonus the patch speeds it up a bit. It's visible in float-long 
comparison microbenchmark:

$ ./old -m timeit "1 == 1.0"
500 loops, best of 5: 55 nsec per loop

$ ./new -m timeit "1 == 1.0"
500 loops, best of 5: 52.3 nsec per loop

$ ./old -m timeit "12345678 == 12345678.0"
500 loops, best of 5: 70.4 nsec per loop

$ ./new -m timeit "12345678 == 12345678.0"
500 loops, best of 5: 53.5 nsec per loop

--
components: Interpreter Core
messages: 280123
nosy: Adrian Wielgosik
priority: normal
severity: normal
status: open
title: Refactor duplicate code calculating digit's bit length
type: performance
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: First security bug related to f-strings

[eryk sun]

On Sat, Nov 5, 2016 at 6:50 PM, Irmen de Jong  wrote:
> Perhaps. But in those cases you could just leave things on the default.
> If you choose to run the interpreter with eval (and exec) disabled, you 
> should be aware
> that you'll break tools like that. But for other situations (web server etc) 
> it could
> still be useful? I do agree that not being able to use namedtuple (and 
> perhaps other
> things from the stdlib) is a problem then.

Breaking importlib at startup is not an option. An application would
need to import everything before disabling exec.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue28563] Arbitrary code execution in gettext.c2py

[Xiang Zhang]

Xiang Zhang added the comment:

Christian, I think our patches are quite similar in function. They only allow 
limited tokens. 

> I consider it a superior solution and a fix for more generic attacks

Mine now still allows **. But it can be easily fixed.

But both our patches still translate a C expression to Python and still suffer 
from nested ternary operator and different semantics between C and Python, e.g. 
(2==2==2 as Serhiy notes). :-( I plan to try a simple parser.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28542] document cross compilation

[Xavier de Gaye]

Xavier de Gaye added the comment:

Yes, I was not aware of DESTDIR. The 2.7 README contains also useful 
information on the build process, for example about some use cases of the 
'*shared*' entries in the Setup files (for when setup.py cannot detect/build an 
extension module) or about the '*static' entries (to be able to profile 
extension modules). It also explains Setup.local whereas the only information 
on Setup.local in the Python3 documentation, is restricted to 
https://docs.python.org/3/extending/extending.html#compilation-and-linkage :(

There is also a lot of noise in the 2.7 README, this possibly explains the 
trimming. Maybe the cross-build section in the 3.6 README could be a link to a 
new build section in the documentation ?

> --prefix etc are already mentioned in “./configure --help”

The documentation on sys.prefix and sys.exec_prefix gives details about where 
go which files which is useful when trying to figure out where to copy the 
files on the target after a cross-build.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: [Theory] How to speed up python code execution / pypy vs GPU

[Steve D'Aprano]

On Sun, 6 Nov 2016 04:10 am, Mr. Wrobel wrote:

> Hi,
> 
> Some skeptics asked my why there is a reason to use Python against of
> any other "not interpreted" languages, like objective-C.

Here's the "Hello World" program in Python:

--- cut ---

print("Hello World")

--- cut ---



Here's the same program in Objective C:

--- cut ---

#import 

int main (int argc, const char * argv[])
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSLog (@"Hello, World!");
[pool drain];
return 0;
}

--- cut ---


Which would you rather write?





-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pip3 : command not found

[Chris Angelico]

On Sun, Nov 6, 2016 at 3:53 AM, Steve D'Aprano
 wrote:
> [1] Technically, the application being run may invoke different behaviour
> depending on the name it was invoked by. Some editors do that, e.g.
> the "joe" editor. But I don't believe Python does anything like this.
>
> http://joe-editor.sourceforge.net/

And quite a few other examples, too - some versions of bash will act
in a more sh-compatible way if invoked as /bin/sh, thus allowing one
to be a link to the other.

Here's how I'm probing Python for this behaviour:

1) Brand new venv

rosuav@sikorsky:~/tmp$ python3 -m venv env

2) With venv active:

(env) rosuav@sikorsky:~/tmp$ python3
Python 3.7.0a0 (default:72e64fc8746b+, Oct 28 2016, 12:35:28)
[GCC 6.2.0 20161010] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; sys.executable
'/home/rosuav/tmp/env/bin/python3'
>>> sys.path
['', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7',
'/usr/local/lib/python3.7/lib-dynload',
'/home/rosuav/tmp/env/lib/python3.7/site-packages']

3) With venv not active:

rosuav@sikorsky:~$ python3
Python 3.7.0a0 (default:72e64fc8746b+, Oct 28 2016, 12:35:28)
[GCC 6.2.0 20161010] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; sys.executable
'/usr/local/bin/python3'
>>> sys.path
['', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7',
'/usr/local/lib/python3.7/lib-dynload',
'/home/rosuav/.local/lib/python3.7/site-packages',
'/usr/local/lib/python3.7/site-packages']

4) Proof that it really is a symlink:

(env) rosuav@sikorsky:~/tmp$ ll env/bin
total 32
-rw-r--r-- 1 rosuav rosuav 2134 Nov  6 05:57 activate
-rw-r--r-- 1 rosuav rosuav 1250 Nov  6 05:57 activate.csh
-rw-r--r-- 1 rosuav rosuav 2414 Nov  6 05:57 activate.fish
-rwxr-xr-x 1 rosuav rosuav  249 Nov  6 05:57 easy_install
-rwxr-xr-x 1 rosuav rosuav  249 Nov  6 05:57 easy_install-3.7
-rwxr-xr-x 1 rosuav rosuav  221 Nov  6 05:57 pip
-rwxr-xr-x 1 rosuav rosuav  221 Nov  6 05:57 pip3
-rwxr-xr-x 1 rosuav rosuav  221 Nov  6 05:57 pip3.7
lrwxrwxrwx 1 rosuav rosuav7 Nov  6 05:57 python -> python3
lrwxrwxrwx 1 rosuav rosuav   22 Nov  6 05:57 python3 -> /usr/local/bin/python3

5) Moment of truth:

rosuav@sikorsky:~$ tmp/env/bin/python3
Python 3.7.0a0 (default:72e64fc8746b+, Oct 28 2016, 12:35:28)
[GCC 6.2.0 20161010] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; sys.executable
'/home/rosuav/tmp/env/bin/python3'
>>> sys.path
['', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7',
'/usr/local/lib/python3.7/lib-dynload',
'/home/rosuav/tmp/env/lib/python3.7/site-packages']


So it looks like Python actually *does* take notice of the executable
path. This is presumably to allow shebangs to be created the easy way
("#!" + sys.executable) and to always represent the correct Python,
even if you had a venv active.

Now, that said, everything about the previous thread *here* was still
true. People weren't talking about symlinks. But this does mean that
(at least on recent Pythons) running Python from within a venv will
implicitly activate it. (And since the pip* files in that directory
use exactly that shebang, this is also true of running
"tmp/env/bin/pip install spam".)

I'll leave it to someone else to test the older 'virtualenv' module,
as I don't have it installed here.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: First security bug related to f-strings

[Irmen de Jong]

On 5-11-2016 19:08, eryk sun wrote:
> On Sat, Nov 5, 2016 at 5:33 PM, Irmen de Jong  wrote:
>> I think perhaps we should have a command line option / environment variable 
>> to be able
>> to disable 'eval' altogether
> 
> I don't think that's practical. exec and eval are commonly used by
> shells and IDEs such as IDLE and IPython. In the standard library,
> importlib and namedtuple are two important users of exec. Just try
> `import builtins; del builtins.exec, builtins.eval`.
> 

Perhaps. But in those cases you could just leave things on the default.
If you choose to run the interpreter with eval (and exec) disabled, you should 
be aware
that you'll break tools like that. But for other situations (web server etc) it 
could
still be useful? I do agree that not being able to use namedtuple (and perhaps 
other
things from the stdlib) is a problem then.


It was just a thought

Irmen

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue28563] Arbitrary code execution in gettext.c2py

[Christian Heimes]

Christian Heimes added the comment:

Argh, sorry. I meant to write "The gettext module might be vulnerable to more 
than f-string attacks.".

May I suggest that you have a look at my old patch? It uses an AST visitor to 
inspect the AST of a gettext plural expression. It allows only a limited set of 
AST types as well as limited amount of expressions. I consider it a superior 
solution and a fix for more generic attacks.

I haven't tested my patch with f-strings yet. It either refuses f-strings 
FormattedValue already or can be easily modified to reject it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: First security bug related to f-strings

[eryk sun]

On Sat, Nov 5, 2016 at 5:33 PM, Irmen de Jong  wrote:
> I think perhaps we should have a command line option / environment variable 
> to be able
> to disable 'eval' altogether

I don't think that's practical. exec and eval are commonly used by
shells and IDEs such as IDLE and IPython. In the standard library,
importlib and namedtuple are two important users of exec. Just try
`import builtins; del builtins.exec, builtins.eval`.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue28563] Arbitrary code execution in gettext.c2py

[Carl Ekerot]

Carl Ekerot added the comment:

> The gettext module might be vulnerable to f-string attacks

It is. See the example in the first comment:

   gettext.c2py('f"{os.system(\'sh\')}"')(0)

This vulnerability seems to be solved in Xiang's patch. The DoS aspect is 
interesting though, since there's no constraints against malicious use of the 
power-operator, say "9**9**9**..**9". This too would be solved by implementing 
a parser with only simple arithmetics.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1647489] zero-length match confuses re.finditer()

[Mark Lawrence]

Changes by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Christian Heimes]

Christian Heimes added the comment:

The gettext module might be vulnerable to f-string attacks, too. Also see 
#18317.

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: First security bug related to f-strings

[Irmen de Jong]

On 5-11-2016 18:12, Steve D'Aprano wrote:
> Well, that didn't take very long at all.
> 
> Here's the first security bug which is related to the new (and badly
> misnamed) f-string feature:
> 
> http://bugs.python.org/issue28563


I think perhaps we should have a command line option / environment variable to 
be able
to disable 'eval' altogether

Irmen

-- 
https://mail.python.org/mailman/listinfo/python-list

SCons 2.5.1 Released

[Bill Deegan]

Available at:

https://sourceforge.net/projects/scons/files/latest/download?source=files

Changelog:

 SCons - a software construction tool

Change Log

RELEASE 2.5.1 - Mon, 03 Nov 2016 13:37:42 -0400

  From William Deegan:
- Add scons-configure-cache.py to packaging. It was omitted

  From Alexey Klimkin:
- Use memoization to optimize PATH evaluation across all dependencies
per
  node. (PR #345)


RELEASE 2.5.0 - Mon, 09 Apr 2016 11:27:42 -0700

  From Dirk Baechle:
- Removed a lot of compatibility methods and workarounds
  for Python versions < 2.7, in order to prepare the work
  towards a combined 2.7/3.x version. (PR #284)
  Also fixed the default arguments for the print_tree and
  render_tree methods. (PR #284, too)

  From William Blevins:
- Added support for cross-language dependency scanning;
  SCons now respects scanner keys for implicit dependencies.
  - Notes for SCons users with heterogeneous systems.
- May find new (previously missed) dependencies.
- May cause rebuild after upgrade due to dependency changes.
- May find new dependency errors (EG. cycles).
  - Discovered in some of the SCons QT tests.
- Resolved missing cross-language dependencies for
  SWIG bindings (fixes #2264).
- Corrected typo in User Guide for Scanner keyword. (PR #2959)
- Install builder interacts with scanner found in SCANNERS differently.
  - Previous: Install builder recursively scanned implicit dependencies
for scanners from SCANNER, but not for built-in (default) scanners.
  - Current: Install builder will not scan for implicit dependencies via
either scanner source. This optimizes some Install builder behavior
and brings orthogonality to Install builder scanning behavior.

  From William Deegan:
- Add better messaging when two environments have
  different actions for the same target (Bug #2024)
- Fix issue only with MSVC and Always build where targets
  marked AlwaysBuild wouldn't make it into CHANGED_SOURCES
  and thus yield an empty compile command line. (Bug #2622)
- Fix posix platform escaping logic to properly handle paths
  with parens in them "()".  (Bug #2225)

  From Jakub Pola:
- Intel Compiler 2016 (Linux/Mac) update for tool directories.

  From Adarsh Sanjeev:
- Fix for issue #2494: Added string support for Chmod function.

  From Tom Tanner:
- change cache to use 2 character subdirectories, rather than one
character,
  so as not to give huge directories for large caches, a situation which
  causes issues for NFS.
  For existing caches, you will need to run the scons-configure-cache.py
  script to update them to the new format. You will get a warning every
time
  you build until you co this.
- Fix a bunch of unit tests on windows
-- 
https://mail.python.org/mailman/listinfo/python-list

First security bug related to f-strings

[Steve D'Aprano]

Well, that didn't take very long at all.

Here's the first security bug which is related to the new (and badly
misnamed) f-string feature:

http://bugs.python.org/issue28563

Note what I'm not saying: I'm not saying that the bug is *caused* by
f-strings. It is not. The bug is actually caused by the inappropriate use
of eval. But the worrying thing here is:


Bonus: With the new string interpolation in Python 3.7, exploiting 
gettext.c2py becomes trivial:

   gettext.c2py('f"{os.system(\'sh\')}"')(0)

The tokenizer will recognize the entire format-string as just a string,
thus bypassing the security checks.


Yay for hiding arbitrary code evaluation inside something that pretends to
be a string!

My guess is that there is probably more code out there in the wild which is
vulnerable to code injection attacks thanks to the use of eval or exec, but
its been too hard to exploit up until now. But with f-strings, chances are
that they too will be trivially exploitable.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

[Theory] How to speed up python code execution / pypy vs GPU

[Mr. Wrobel]

Hi,

Some skeptics asked my why there is a reason to use Python against of 
any other "not interpreted" languages, like objective-C. As my 
explanation, I have answered that there is a a lot of useful APIs, 
language is modern, has advanced objective architecture, and what is the 
most important - it is  dynamic and support is simply great.


However the same skeptics told my that, ok we believe that it is true, 
however the code execution is much slower than any other compiled language.


I must tell you that is the reason I started to dig into internet and 
searching some methods to speed up python's code.


1. What I have found is modified python interpreter - pypy  - 
http://pypy.org that does not require any different approach to develop 
your code.


2. And: Gpu based computing powered by Nvidia (NumbaPro compiler): 
https://developer.nvidia.com/how-to-cuda-python


Do you have any experience in that areas, and maybe some benchmarks 
showing advantage of using these technologies?


Cheers,
Marcin
--
https://mail.python.org/mailman/listinfo/python-list

Re: pip3 : command not found

[Steve D'Aprano]

On Sun, 6 Nov 2016 02:55 am, Jon Ribbens wrote:

> I'm afraid I can only suggest that you try re-reading the subthread
> again until you manage to understand it. It wasn't really that
> complicated but you seem to have confused yourself greatly.

Are you serious?

Okay. Here's the start of the thread:

https://mail.python.org/pipermail/python-list/2016-October/715993.html

where the OP asks for help because running "pip3" causes a "command not
found" error. Just as the subject line says.

Here's Ben's reply:

https://mail.python.org/pipermail/python-list/2016-October/715994.html

where he clearly says not to rely on the "pip3" command, as it may not
exist, or may not be in the environment you expect, but to specify
precisely which Python interpreter you want to run:

[quote]
Instead, you should invoke the exact Python interpreter you want


And here's your response:

https://mail.python.org/pipermail/python-list/2016-October/716005.html

where you quote those exact same words from Ben and ask what difference it
makes as it is a symbolic link. You'll note that there is not one word
about "activate" in your post, just as I said. Instead you question why Ben
thinks there's a difference between running the sym link and running the
direct link to the executable, even though that's not what Ben said.

I can only repeat yet again: you have not understood Ben. He's not stating
that there is a difference between:

/direct/path/to/python3.x

and 

/symbolic/link/to/direct/path/to/python3.x


You are right to state that they will be exactly the same[1], but wrong to
imagine that Ben said that they were different. That's not what Ben said.
He stated that there is a difference between:

/direct/path/to/python3.x -m pip ...

which specifies the precise Python environment you want to run, and:

pip3 ... 

which may not exist at all, or if it exists it may not be on the PATH, or if
it is on the PATH it may not install into the Python environment that you
expect.


Oh, and ironically, you'll see that Ben anticipated and answered your
question about activate. Here's the link again, to save you scrolling up:

https://mail.python.org/pipermail/python-list/2016-October/715994.html

where he says:

If you already have a specific environment active and know that
‘python3’ is the correct Python interpreter from that environment,
you can omit the explicit path.


The implication is that the answer to your question is Yes, you can run
Python in the context of a virtualenv by just invoking that virtualenv's
local Python without running 'activate' first. Is Ben correct? The
virtualenv documentation confirms that Ben is right:

If you directly run a script or the python interpreter from the
virtualenv’s bin/ directory (e.g. path/to/ENV/bin/pip or 
/path/to/ENV/bin/python-script.py) there’s no need for activation.

https://virtualenv.pypa.io/en/stable/userguide/#activate-script






[1] Technically, the application being run may invoke different behaviour
depending on the name it was invoked by. Some editors do that, e.g.
the "joe" editor. But I don't believe Python does anything like this.

http://joe-editor.sourceforge.net/

-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue28610] Provide PDB hook to customize how to find source files

[Xavier de Gaye]

Xavier de Gaye added the comment:

This patch is an attempt at allowing the source debugging of scripts executed 
by the Python exec() function. It misses tests and documentation.

You may use it using the idiom given in the following example to avoid stepping 
into the pdb code on the first invocation of pdb.exec_script() (see the 
exec_script() doc string):

import sys

def main():
foo = 123
s = """if 1:
x = foo
x = 555
"""
exec_script(s)

if __name__ == '__main__':
if '--debug' in sys.argv[1:]:
import pdb
exec_script = pdb.exec_script
pdb.Pdb(skip=['pdb']).set_trace()
else:
exec_script = exec

main()

--
components:  -Demos and Tools
keywords: +patch
stage:  -> needs patch
versions: +Python 3.7 -Python 3.5
Added file: http://bugs.python.org/file45366/debug_script.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28618] Decorate hot functions using __attribute__((hot)) to optimize Python

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> Moreover, I like the idea of getting a fast(er) Python even when no
advanced optimization techniques like LTO or PGO is used.

Seconded.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: pip3 : command not found

[Jon Ribbens]

On 2016-11-05, Steve D'Aprano  wrote:
> Your implied question here:
>
>> Maybe he meant what you are saying, I don't know, but 
>> it isn't what he wrote. He clearly implied that you can run Python
>> in the context of a virtualenv by just invoking that virtualenv's
>> local Python without running 'activate' first. I'm curious as to
>> whether this is true or not (how virtualenvs work seems to be very
>> opaque).
>
> is a separate issue from the symbolic link question. Possibly you do have to
> run `activate` first, I don't know, but either way this was not part of
> your earlier question. You said nothing about `activate` in your earlier
> post, and this is the first time you have mentioned it.

I'm afraid I can only suggest that you try re-reading the subthread
again until you manage to understand it. It wasn't really that
complicated but you seem to have confused yourself greatly.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue28618] Decorate hot functions using __attribute__((hot)) to optimize Python

[STINNER Victor]

STINNER Victor added the comment:

Antoine Pitrou added the comment:
> Can you compare against a PGO build?

Do you mean comparison between current Python with PGO and patched
Python without PGO?

The hot attribute is ignored by GCC when PGO compilation is used.

> Ubuntu 14.04 is old, and I don't think this is something we should worry 
> about.

Well, it's a practical issue for me to run benchmarks for speed.python.org.

Moreover, I like the idea of getting a fast(er) Python even when no
advanced optimization techniques like LTO or PGO is used. At least,
it's common to build quickly Python using "./configure && make" for a
quick benchmark.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28620] Build Memory Leak

[R. David Murray]

Changes by R. David Murray :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> memory leaks in pgen build step abort build with address 
sanitizer enabled

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1647489] zero-length match confuses re.finditer()

[irdb]

Changes by irdb :


--
nosy: +irdb

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28620] Build Memory Leak

[Honor]

Honor added the comment:

Hmmm, Ok. Thanks a lot.

On Sat, Nov 5, 2016 at 4:50 PM, Xiang Zhang  wrote:

>
> Xiang Zhang added the comment:
>
> This seems a same problem as in #27780.
>
> --
> nosy: +xiang.zhang
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28620] Build Memory Leak

[Xiang Zhang]

Xiang Zhang added the comment:

This seems a same problem as in #27780.

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Xiang Zhang]

Xiang Zhang added the comment:

> '1?2:3?4:5' -> '(2 if 1 else 3)?4:5' -> '(4 if (2 if 1 else 3) else 5'

This is not right. It's right associative so it should be

1?2:(3?4:5) -> 1?2:(4 if 3 else 5) -> 2 if 1 else (4 if 3 else 5)

> It would be nice to make c2py() working with any expressions, but if this is 
> too hard, this can be left for other issue.

Agree. But I am interested in trying.

> gettext_c2py.patch itself LGTM for fixing security issue, but tests are 
> needed.

It gets drawbacks so I don't include tests. I'll add in next try.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28620] Build Memory Leak

[Honor]

New submission from Honor:

Hi,

I am compiling python from source code with clang compiler.

as follows result:

==5284==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 11776 byte(s) in 8 object(s) allocated from:
#0 0x49ccbe (/home/y/Downloads/Python-3.5.2/Parser/pgen+0x49ccbe)
#1 0x4c86ca (/home/y/Downloads/Python-3.5.2/Parser/pgen+0x4c86ca)

Indirect leak of 2000 byte(s) in 3 object(s) allocated from:
#0 0x49ccbe (/home/y/Downloads/Python-3.5.2/Parser/pgen+0x49ccbe)
#1 0x4c86ca (/home/y/Downloads/Python-3.5.2/Parser/pgen+0x4c86ca)

Indirect leak of 898 byte(s) in 86 object(s) allocated from:
#0 0x49c9cb (/home/y/Downloads/Python-3.5.2/Parser/pgen+0x49c9cb)
#1 0x2ad0d5405679 (/lib/x86_64-linux-gnu/libc.so.6+0x89679)

Indirect leak of 520 byte(s) in 1 object(s) allocated from:
#0 0x49c9cb (/home/y/Downloads/Python-3.5.2/Parser/pgen+0x49c9cb)
#1 0x4cb549 (/home/y/Downloads/Python-3.5.2/Parser/pgen+0x4cb549)

Indirect leak of 178 byte(s) in 33 object(s) allocated from:
#0 0x49c9cb (/home/y/Downloads/Python-3.5.2/Parser/pgen+0x49c9cb)
#1 0x4c14d4 (/home/y/Downloads/Python-3.5.2/Parser/pgen+0x4c14d4)

SUMMARY: AddressSanitizer: 15372 byte(s) leaked in 131 allocation(s).

Python version 3.5.2
Operating System: Linux y 3.13.0-24-generic  14.04 ubuntu
gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3)

--
messages: 280111
nosy: Stone
priority: normal
severity: normal
status: open
title: Build Memory Leak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: mentor training python Romania with certification

[neha . agrawal1428]

On Monday, September 5, 2016 at 3:33:55 PM UTC+5:30, neha.agr...@gmail.com 
wrote:
> On Tuesday, August 16, 2016 at 6:45:04 PM UTC+5:30, blue wrote:
> > Hi. 
> > 
> > I'm from Romania.
> > I need to update my skils under python.
> > I need to find one mentor ( class, training ) to obtain one certified under 
> > python language.
> > 
> > Cătălin George Feștilă
> 
> Hi,
> 
> You might find Python course offered by edureka to be useful
> http://www.edureka.co/python
> 
> Check it out!
> 
> Regards
> Neha

Hi,

Please check out the below link for a demo on Edureka's Python course
http://unbouncepages.com/python-training-3/

Regards
Neha
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Django Application Model Reference

[Chris Angelico]

On Thu, Nov 3, 2016 at 4:44 AM, Dreyton Scott  wrote:
> On Wednesday, November 2, 2016 at 1:40:35 PM UTC-4, Dreyton Scott wrote:
>> Hello. I am currently creating a notification django application that will 
>> need to be able to "hook" into another django application. What my app needs 
>> is a way to retrieve all model classes in the connected application. Is 
>> there a way to do this?
>
> In a way such that my app (a) is connected to app (b). Have (a) retrieve all 
> model class instances in the connected app (b). Is that possible?

You may have to elaborate on "connected" here. Is it able to import
the code of the other app, or is it restricted to HTTP requests and
responses? If the latter, there's no significance to them both being
written in Django; you'll have to create an explicit endpoint that
enumerates the models, probably sending the info as a JSON object.
(Caveat: I don't know Django, but I do know HTTP.)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue19569] Use __attribute__((deprecated)) to warn usage of deprecated functions and macros

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Proposed patch marks most deprecated functions. Code is rewritten for using 
non-deprecated functions if possible. Unfortunately some deprecated function 
still are used in the code and can't be easier replaced. They are left not 
marked.

* PyEval_ReleaseLock() is used in Python/pystate.c. It can't be replaced with 
PyEval_ReleaseThread() since the latter don't accept NULL.

* Py_UNICODE (currently an alias of wchar_t) is used in a number of deprecated 
functions and bridges between deprecated and new APIs. Maybe it can be just 
replaced with wchar_t.

* Macros PyUnicode_GET_SIZE, PyUnicode_GET_DATA_SIZE, PyUnicode_AS_UNICODE, 
PyUnicode_AS_DATA, functions PyUnicode_AsUnicode and PyUnicode_AsUnicodeAndSize 
are used in a number of places. They can't be easily replaced with wchar-based 
functions since they return a borrowed reference to cached representation.

* PyUnicode_FromUnicode, PyUnicode_EncodeDecimal and 
PyUnicode_TransformDecimalToASCII are used only in Modules/_testcapimodule.c. I 
think we should write tests for modern APIs and eliminate tests for deprecated 
APIs. Or temporary silence compiler warning in test functions.

* _PyUnicode_ToLowercase, _PyUnicode_ToUppercase and corresponding public 
macros Py_UNICODE_TOLOWER and Py_UNICODE_TOUPPER are used in Modules/_sre.c 
(this is a bug in regex implementation). The problem is that more modern 
functions _PyUnicode_ToLowerFull and _PyUnicode_ToUpperFull is a private API.

All these cases needs separate issues.

--
keywords: +patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file45365/mark-deprecated-functions.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Pre-pep discussion material: in-place equivalents to map and filter

[Chris Angelico]

On Sat, Nov 5, 2016 at 9:50 PM, Arthur Havlicek
 wrote:
> Pick 10 programmers for hire and count how many know how to write a
> decorator. If you have specified you needed python specialists, you may
> have 3-4. If not, you are lucky to find even one.

By "write a decorator", I presume you mean implement the decorator
function, because anyone who's used Flask will have used "@app.route",
for instance.

But here's the thing. For everyone who writes a decorator function,
there could be dozens who use it. How many people *on this mailing
list* know how to implement namedtuple, vs how how many know how to
use it? How many know how to properly implement a cache, vs how many
can slap "@lru_cache()" on top of a function? For the most utterly
extreme example possible, how many people in the world know how to
encrypt data securely, vs how many know how to put "https:" into their
browsers? When you look at "programmers for hire", you get a small
number of brilliant experts with huge amounts of experience, and the
rest are split between mid-level people looking for a new job, and
entry-level people straight out of college. (Depending on your
environment, the stats could easily be skewed either direction.) Most
entry-level programmers are not going to have experience with writing
decorator functions - but they don't need it. (Not that it's that
hard. They're functions that accept and return functions, that's all.)

> This is easy but time-consuming, I could roll my implementation and
> showcase a few benchs. I am very confident that is the case. I would have
> bet that I would need to do it at some point, but I wanted to poll opinions
> a bit beforehand.
>
> About your bench: I don't really know why you are surprised the for loop is
> slower. That's the result of comprehension being native while for loop
> isn't. That does not mean writing to a copy would save time for exchange of
> memory. In my opinion, the fact that we will win something is guaranteed
> because we save a copy call and do the exact same operation. There is
> nothing like cache magic optimization that could happen because the mapping
> needs to read the first list anyway. Nor we need a temporary buffer to
> cache operations since they are independent. Really, I am ready to make a
> serious bet.

Okay! *Do* make that bet. Gamble the most precious currency there is:
developer time. How much are you prepared to bet? Two hours? Ten
hours? Spend that much time writing the implementation and
benchmarking it. If you're confident that this really is such a win,
the time won't be lost - you'll end up with a viable patch for
inclusion. If you lose the bet, well, that's what betting is like.
Sometimes you lose.

I may sound cynical and critical from the above ("show me"), but in
reality, I would love to see the results of your benchmark. Improving
performance is a good thing. I just want to know that it's actually
going to happen.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue28619] [Patch] Stop using inet_ntoa() when possible.

[Ed Schouten]

New submission from Ed Schouten:

Modern C code should use inet_ntop()/inet_pton() as opposed to 
inet_addr()/inet_aton()/inet_ntoa().

Though the former functions may typically act as drop-in replacements for the 
latter, the inet_addr()/inet_aton() functions still have the advantage over 
inet_pton() of allowing you to parse IPv4 addresses that don't use the dotted 
quad notation (e.g. '0x0a01' for 10.0.0.1).

There is no difference between inet_ntop() and inet_ntoa(), as they both always 
print the address in dotted quad form. inet_ntop() does have the advantage of 
being thread-safe, as inet_ntoa() uses internal storage for the return value. 
In other words, we'd better not use inet_ntoa() at all.

Attached is a patch for Python's socketmodule that changes the existing call to 
inet_ntoa() to use inet_ntop() when available. This has the advantage of fixing 
the build on CloudABI 
(https://mail.python.org/pipermail/python-dev/2016-July/145708.html), which 
intentionally omits any APIs that are thread-unsafe.

--
components: Extension Modules
files: patch-inet_ntoa.diff
keywords: patch
messages: 280109
nosy: EdSchouten
priority: normal
severity: normal
status: open
title: [Patch] Stop using inet_ntoa() when possible.
versions: Python 3.7
Added file: http://bugs.python.org/file45364/patch-inet_ntoa.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: pip3 : command not found

[Steve D'Aprano]

On Sat, 5 Nov 2016 08:45 pm, Jon Ribbens wrote:

> On 2016-10-31, Steve D'Aprano  wrote:
>> On Mon, 31 Oct 2016 07:21 pm, Jon Ribbens wrote:
>>> On 2016-10-31, Ben Finney  wrote:
 Instead, you should invoke the exact Python interpreter you want – and,
 by extension, the Python environment into which you want packages
 installed.

 $ /foo/bar/virtualenv/bin/python3 -m pip install LoremIpsum
>>> 
>>> I'm slightly curious about that. /foo/bar/virtualenv/bin/python3
>>> will just be a symbolic link to /usr/bin/python3, so how does
>>> invoking the intepreter that way make any difference?
>>
>> It doesn't. If you read the rest of Ben's post, or for that matter the
>> subject line of this thread, you will see he is comparing:
>>
>> path/to/python3 -m pip install LoremIpsum
>>
>> against:
>>
>> pip3 install LoremIpsum
> 
> No, if you read the rest of Ben's post you will see that that is not
> what he wrote. 

Excuse me, that is what he wrote. As you yourself quoted.

The OP said he tried to call the "pip3" command, and got an error that the
command was not found. Ben replied:

"There is no guarantee that a command named ‘pip3’ will be installed."

and then stated:

"Instead, you should invoke the exact Python interpreter you want"

The fact that (and now I'm quoting *you*, not Ben) 

/foo/bar/virtualenv/bin/python3 will just be a symbolic link
to /usr/bin/python3

is irrelevant. It doesn't matter whether you call python3 via the actual
executable file, or the symbolic link. But Ben never suggested that it
would make a difference: he was contrasting calling the actual Python
interpreter wanted (Python 3, via a virtualenv) with calling a command pip3
(which does not actually exist on the OP's system).


Your implied question here:

> Maybe he meant what you are saying, I don't know, but 
> it isn't what he wrote. He clearly implied that you can run Python
> in the context of a virtualenv by just invoking that virtualenv's
> local Python without running 'activate' first. I'm curious as to
> whether this is true or not (how virtualenvs work seems to be very
> opaque).

is a separate issue from the symbolic link question. Possibly you do have to
run `activate` first, I don't know, but either way this was not part of
your earlier question. You said nothing about `activate` in your earlier
post, and this is the first time you have mentioned it.

The bottom line is, I couldn't have answered your question about `activate`
because you hadn't asked it yet; you are correct that there's no difference
between calling Python via the executable and via a symlink; but Ben never
said that there was such a difference.


This thread reminds me of a scene from the Marx Bros movie "At The Circus".
Alas, Google has let me down as far as the exact quote, but it goes
something like this:

Groucho Marx, accidentally letting the big secret slip: 
"The elephants will be here soon."

Society dame Margaret Dumont, taken aback:
"Elephants? What elephants?"

Groucho:
"Nobody said anything about elephants!"

[shakes head disgustedly]

"Elephants! At your age!"


I'll let you decide whether I'm Groucho or Dumont.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pre-pep discussion material: in-place equivalents to map and filter

[Arthur Havlicek]

2016-11-05 9:42 GMT+01:00 Steve D'Aprano :

>
> I don't know who you are quoting there. It is considered impolite to quote
> people without giving attribution, and makes it harder to respond.
>

My bad. I was unaware of that. This was quoted from Ned Batchelder's mali.


2016-11-05 9:42 GMT+01:00 Steve D'Aprano :

> I trust that isn't actually the case, but by describing decorators and
> lambda as too fancy to use, you're giving a good impression of somebody who
> doesn't know what they're doing.
>

I was a bit humorous here, making fun about the fact I do not write
anything sophisticated.

However, the fact that we do avoid them is entirely true ! We do so because
these concepts are foreign to the average developer (lambda being a bit
apart: it's easily understood, but functional constructs may not, so its a
result of us avoiding functional constructs as well).

Pick 10 programmers for hire and count how many know how to write a
decorator. If you have specified you needed python specialists, you may
have 3-4. If not, you are lucky to find even one. And where I live we don't
have the luxury of finding competent Python experts by kicking a tree. In
our open space, we are 6 devs (+ one ex-dev being my manager), only 2 had
previous Python experience: me and the CEO. And the other 4 are doing fine
! Locking ourselves in Python-specific semantics, over time, will make us
loose precious dev time. The best code is the one everyone can understand.
That is why I point this as "complex, non-obvious", and my manager wouldn't
have so kind words (but he is a Java fan, so he doesn't count.)


2016-11-05 9:42 GMT+01:00 Steve D'Aprano :

> - you would have to demonstrate a good reason to think that this new
> feature
> will actually be more efficient and faster
>

This is easy but time-consuming, I could roll my implementation and
showcase a few benchs. I am very confident that is the case. I would have
bet that I would need to do it at some point, but I wanted to poll opinions
a bit beforehand.


2016-11-05 9:42 GMT+01:00 Steve D'Aprano :

> In other words, in my opinion:
>
> - you would need to prove that there is a widespread need for in-place
> modification of lists, where the lists are big enough that using a
> temporary list is not practical
>

However this is tricky, I may very well be an exception.

About your bench: I don't really know why you are surprised the for loop is
slower. That's the result of comprehension being native while for loop
isn't. That does not mean writing to a copy would save time for exchange of
memory. In my opinion, the fact that we will win something is guaranteed
because we save a copy call and do the exact same operation. There is
nothing like cache magic optimization that could happen because the mapping
needs to read the first list anyway. Nor we need a temporary buffer to
cache operations since they are independent. Really, I am ready to make a
serious bet.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue28618] Decorate hot functions using __attribute__((hot)) to optimize Python

[Antoine Pitrou]

Antoine Pitrou added the comment:

Can you compare against a PGO build? Ubuntu 14.04 is old, and I don't think 
this is something we should worry about.

Overall I think this manual approach is really the wrong way to look at it. 
Compilers can do better than us.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Carl Ekerot]

Carl Ekerot added the comment:

Verified gettext.c2py with gettext_c2py.patch applied agains the plural forms 
actually used in localization, listed over at 
http://docs.translatehouse.org/projects/localization-guide/en/latest/l10n/pluralforms.html.
 I tested all of the none-trivial forms, and from what I can tell they generate 
valid syntax and are correct.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: pip3 : command not found

[Jon Ribbens]

On 2016-10-31, Steve D'Aprano  wrote:
> On Mon, 31 Oct 2016 07:21 pm, Jon Ribbens wrote:
>> On 2016-10-31, Ben Finney  wrote:
>>> Instead, you should invoke the exact Python interpreter you want – and,
>>> by extension, the Python environment into which you want packages
>>> installed.
>>>
>>> $ /foo/bar/virtualenv/bin/python3 -m pip install LoremIpsum
>> 
>> I'm slightly curious about that. /foo/bar/virtualenv/bin/python3
>> will just be a symbolic link to /usr/bin/python3, so how does
>> invoking the intepreter that way make any difference?
>
> It doesn't. If you read the rest of Ben's post, or for that matter the
> subject line of this thread, you will see he is comparing:
>
> path/to/python3 -m pip install LoremIpsum
>
> against:
>
> pip3 install LoremIpsum

No, if you read the rest of Ben's post you will see that that is not
what he wrote. Maybe he meant what you are saying, I don't know, but
it isn't what he wrote. He clearly implied that you can run Python
in the context of a virtualenv by just invoking that virtualenv's
local Python without running 'activate' first. I'm curious as to
whether this is true or not (how virtualenvs work seems to be very
opaque).
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue28618] Decorate hot functions using __attribute__((hot)) to optimize Python

[STINNER Victor]

STINNER Victor added the comment:

Oh, I forgot to mention that I compiled Python with "./configure -C". The 
purpose of the patch is to optimize Python when LTO and/or PGO compilation are 
not explicitly used.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28618] Decorate hot functions using __attribute__((hot)) to optimize Python

[STINNER Victor]

STINNER Victor added the comment:

I ran benchmarks. Globally, it seems like the impact of the patch is positive. 
regex_v8 and call_simple are slower, but these benchmarks are microbenchmarks 
impacted by low level stuff like CPU L1 cache. Well, my patch was supposed to 
optimize CPython for call_simple :-/ I should maybe investigate a little bit 
more.


Performance comparison (performance 0.3.2):

haypo@smithers$ python3 -m perf compare_to orig.json hot.json -G
Slower (6):
- regex_v8: 40.6 ms +- 5.7 ms -> 47.1 ms +- 0.3 ms: 1.16x slower
- call_simple: 12.6 ms +- 0.2 ms -> 13.2 ms +- 1.3 ms: 1.05x slower
- regex_effbot: 4.58 ms +- 0.07 ms -> 4.70 ms +- 0.05 ms: 1.03x slower
- sympy_integrate: 43.4 ms +- 0.3 ms -> 44.0 ms +- 0.2 ms: 1.01x slower
- nqueens: 239 ms +- 2 ms -> 241 ms +- 1 ms: 1.01x slower
- scimark_fft: 674 ms +- 12 ms -> 680 ms +- 75 ms: 1.01x slower

Faster (32):
- scimark_monte_carlo: 255 ms +- 4 ms -> 234 ms +- 7 ms: 1.09x faster
- chameleon: 28.4 ms +- 3.1 ms -> 27.0 ms +- 0.4 ms: 1.05x faster
- scimark_sor: 488 ms +- 27 ms -> 467 ms +- 10 ms: 1.05x faster
- sqlite_synth: 9.16 us +- 1.03 us -> 8.82 us +- 0.23 us: 1.04x faster
- scimark_lu: 485 ms +- 20 ms -> 469 ms +- 14 ms: 1.03x faster
- xml_etree_process: 226 ms +- 30 ms -> 219 ms +- 4 ms: 1.03x faster
- logging_simple: 29.7 us +- 0.4 us -> 28.9 us +- 0.3 us: 1.03x faster
- pickle_list: 7.99 us +- 0.88 us -> 7.78 us +- 0.05 us: 1.03x faster
- raytrace: 1.26 sec +- 0.08 sec -> 1.23 sec +- 0.01 sec: 1.03x faster
- sympy_expand: 995 ms +- 31 ms -> 971 ms +- 35 ms: 1.03x faster
- deltablue: 17.0 ms +- 0.1 ms -> 16.6 ms +- 0.2 ms: 1.02x faster
- call_method_slots: 16.0 ms +- 0.1 ms -> 15.6 ms +- 0.2 ms: 1.02x faster
- fannkuch: 983 ms +- 12 ms -> 962 ms +- 29 ms: 1.02x faster
- pickle_pure_python: 1.25 ms +- 0.14 ms -> 1.22 ms +- 0.01 ms: 1.02x faster
- logging_format: 34.0 us +- 0.3 us -> 33.4 us +- 1.5 us: 1.02x faster
- xml_etree_parse: 274 ms +- 9 ms -> 270 ms +- 5 ms: 1.02x faster
- sympy_str: 441 ms +- 3 ms -> 433 ms +- 3 ms: 1.02x faster
- genshi_text: 87.6 ms +- 9.2 ms -> 86.0 ms +- 1.4 ms: 1.02x faster
- genshi_xml: 187 ms +- 17 ms -> 184 ms +- 1 ms: 1.02x faster
- django_template: 376 ms +- 4 ms -> 370 ms +- 2 ms: 1.02x faster
- json_dumps: 27.1 ms +- 0.4 ms -> 26.7 ms +- 0.4 ms: 1.02x faster
- sqlalchemy_declarative: 295 ms +- 3 ms -> 291 ms +- 3 ms: 1.01x faster
- call_method_unknown: 18.1 ms +- 0.1 ms -> 17.8 ms +- 0.1 ms: 1.01x faster
- nbody: 218 ms +- 4 ms -> 216 ms +- 2 ms: 1.01x faster
- regex_dna: 250 ms +- 24 ms -> 247 ms +- 2 ms: 1.01x faster
- go: 573 ms +- 2 ms -> 566 ms +- 3 ms: 1.01x faster
- richards: 173 ms +- 4 ms -> 171 ms +- 4 ms: 1.01x faster
- python_startup: 24.6 ms +- 0.1 ms -> 24.5 ms +- 0.1 ms: 1.00x faster
- regex_compile: 404 ms +- 6 ms -> 403 ms +- 5 ms: 1.00x faster
- dulwich_log: 143 ms +- 11 ms -> 143 ms +- 1 ms: 1.00x faster
- pidigits: 290 ms +- 1 ms -> 289 ms +- 0 ms: 1.00x faster
- pickle_dict: 58.3 us +- 6.5 us -> 58.3 us +- 0.7 us: 1.00x faster

Benchmark hidden because not significant (26): 2to3, call_method, chaos, 
crypto_pyaes, float, hexiom, html5lib, json_loads, logging_silent, mako, 
meteor_contest, pathlib, pickle, python_startup_no_site, 
scimark_sparse_mat_mult, spectral_norm, sqlalchemy_imperative, sympy_sum, 
telco, tornado_http, unpack_sequence, unpickle, unpickle_list, 
unpickle_pure_python, xml_etree_generate, xml_etree_iterparse

--

More readable output, only display differences >= 5%:

haypo@smithers$ python3 -m perf compare_to orig.json hot.json -G --min-speed=5
Slower (1):
- regex_v8: 40.6 ms +- 5.7 ms -> 47.1 ms +- 0.3 ms: 1.16x slower

Faster (2):
- scimark_monte_carlo: 255 ms +- 4 ms -> 234 ms +- 7 ms: 1.09x faster
- chameleon: 28.4 ms +- 3.1 ms -> 27.0 ms +- 0.4 ms: 1.05x faster

Benchmark hidden because not significant (61): 2to3, call_method, 
call_method_slots, call_method_unknown, call_simple, chaos, crypto_pyaes, 
deltablue, django_template, dulwich_log, fannkuch, float, genshi_text, 
genshi_xml, go, hexiom, html5lib, json_dumps, json_loads, logging_format, 
logging_silent, logging_simple, mako, meteor_contest, nbody, nqueens, pathlib, 
pickle, pickle_dict, pickle_list, pickle_pure_python, pidigits, python_startup, 
python_startup_no_site, raytrace, regex_compile, regex_dna, regex_effbot, 
richards, scimark_fft, scimark_lu, scimark_sor, scimark_sparse_mat_mult, 
spectral_norm, sqlalchemy_declarative, sqlalchemy_imperative, sqlite_synth, 
sympy_expand, sympy_integrate, sympy_str, sympy_sum, telco, tornado_http, 
unpack_sequence, unpickle, unpickle_list, unpickle_pure_python, 
xml_etree_generate, xml_etree_iterparse, xml_etree_parse, xml_etree_process

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 

Re: Pre-pep discussion material: in-place equivalents to map and filter

[Steve D'Aprano]

I've been giving your proposal a bit more thought, and while I can't say I'm
really keep on the idea, I have warmed slightly to it.


On Fri, 4 Nov 2016 07:29 am, Arthur Havlicek wrote:

> I understand that, the cost of change is such that it's very unlikely
> something like this ever goes into Python, but I feel like the interest of
> the proposition is being underestimated here, that's why I'm going to
> argue a few points and give a bit more context as needed.
> 
>> While mapping and filtering are common operations, I'm not sure mapping
>> and filtering and then reassigning back to the original sequence is
>> especially common.
> 
> It depends of your context. On the last 3 months, I stumbled across this
> at least 3 times,

I don't know who you are quoting there. It is considered impolite to quote
people without giving attribution, and makes it harder to respond. But for
what it's worth, I agree with this person.

In my code, it is quite common for me to map back to the same *variable*,
for example I might write something like this:

def process(alist):
alist = [float(x) for x in alist]
...


But that is not the same as *in-place* modification of the list. I would
very rarely modify the list in place, but if I did, I would write it like
this:

alist[:] = [float(x) for x in alist]


In my experience, wanting to modify a list in-place without creating a
temporary version first is unusual. And *needing* to do so (rather than
doing so as premature optimization) is even rarer.

That's my experience. But if you can demonstrate the need for in-place
modifications, that changes the situation somewhat.


> which is 3 times more than I used a lambda or a
> metaclass or a decorator or other fancy language feature that we simply
> avoid whenever possible.

You aren't helping your case by describing "lambda or ... decorator" as
fancy language features to be avoided.

The impression that gives (hopefully this is the wrong impression!) is that
you are a barely competent Python developer, fearful and suspicious of some
rather simple, powerful and widely-used features, stuck in an ancient 1980s
programming paradigm.

I trust that isn't actually the case, but by describing decorators and
lambda as too fancy to use, you're giving a good impression of somebody who
doesn't know what they're doing.


[...]
> The reason I'm being especially impacted by this is because I am
> maintainer of a decent-sized Python application (~50-100K lines of code)
> that extensively uses lists and dictionaries. We value "low level" data
> manipulation and efficiency a lot more than complex, non-obvious
> constructs.

And what do you consider "complex, non-obvious"?


[...]
> Like most Python programmers, I'm not in the case of needing a performance
> boost very bad, but that does not mean I disregard performance entirely.
> The need of performance is not so binary that it either don't matter at
> all or is enough to motivate a rewrite.

Indeed.

But you're arguing for a new feature on the basis that it will boost
performance, without giving any reason to think that it actually will lead
to a performance improvement!

I know that there is a chicken-and-egg problem here. Nobody wants to spend
time writing a new feature if there's no possibly chance for it to be
accepted, but when your sole argument for a new feature is that it will be
more efficient (in both memory and time!), then you need to prove that is
the case!

In other words, in my opinion:

- you would need to prove that there is a widespread need for in-place
modification of lists, where the lists are big enough that using a
temporary list is not practical;

- you would have to demonstrate a good reason to think that this new feature
will actually be more efficient and faster

before this feature would be seriously considered for addition to the
language.

You might think that it is obvious that in-place modification MUST be faster
since it avoids making a temporary copy, but that's not obviously true at
all! Not for a high-level language like Python. Its often the case that
algorithms can exchange space for time (use more memory to save time, or
use more time to save memory). It may be that this is one of the times.

Even when doing (nearly) everything in pure Python, making a new list and
then doing a slice assignment is virtually as fast as writing directly to
the original list:

py> from timeit import Timer
py> setup = "data = list(range(1))"
py> t1 = Timer("""for i, a in enumerate(data):
... data[i] = a+1
... """, setup=setup)
py> 
py> t2 = Timer("""tmp = [None]*len(data)
... for i, a in enumerate(data):
... tmp[i] = a+1
... data[:] = tmp
... """, setup=setup)
py>
py> min(t1.repeat(number=1, repeat=7))
36.63875983003527
py>
py> min(t2.repeat(number=1, repeat=7))
37.70047474466264


Taking the plain Python for-loop, modifying the list directly in place, we
see that making a temporary list and then copying it over the original list
is just 3% 

[issue28563] Arbitrary code execution in gettext.c2py

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> How does it work for '1?2:3?4:5'?

'1?2:3?4:5' -> '(2 if 1 else 3)?4:5' -> '(4 if (2 if 1 else 3) else 5'

But there are other problems. Precedence of some operators is different in C 
and Python. Chained comparison in Python cause different result that in C (e.g. 
'2 == 2 == 2'). Seems there is no other way besides a simple parser.

gettext_c2py.patch itself LGTM for fixing security issue, but tests are needed. 
It would be nice to make c2py() working with any expressions, but if this is 
too hard, this can be left for other issue. I'm going to commit a variant of 
gettext_c2py.patch before 3.6 release if there will be not better patches.

--
priority: high -> deferred blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Xiang Zhang]

Xiang Zhang added the comment:

> gettext.c2py("n()")(lambda: os.system("sh"))
> gettext.c2py("1()")(0)

Empty parentheses should be disallowed. Function calls are not allowed in 
plural expression. And non-integer argument should be disallowed either, just 
as Serhiy's example shows.

> What if make repeated replacements with regular expression 
> r'([^?:]*?)\?([^?:]*?):([^?:]*)'?

How does it work for '1?2:3?4:5'? :-( I am considering a parser.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Call a shell command from Python

[Bob Martin]

in 767198 20161104 142132 Thomas 'PointedEars' Lahn  wrote:
>Ben Finney wrote:
>
>> Note that ‘sudo’ is specifically designed to be invoked interactively,
>
>Nonsense.
>
>> seeking to verify that the current user has credentials to run the
>> command.
>
>NOPASSWD is not the default in sudoers(5),

It is on the Raspberry Pi

 but conclude from that what you
>concluded is a bit far-fetched, to say the least.
-- 
https://mail.python.org/mailman/listinfo/python-list

How to pass C++ function pointer argument in embedded python environment?

[xzldyx]

Hi,All,
background:I have a python api like this,
   def a(arg1,arg2,progress_callback = None)
obviously argument progress_callback is a callback function.In python,I can 
define a function like this:
   def pro_call(prog_arg1,prog_arg2):
   #do something with arg1 & arg2
and I just need call a function like follow and it worked
   a(arg1,arg2,progress_callback = pro_call)


question: I don't know how to do like the former exam in embedded python 
environment.


thank you for your reading ,thanks for any response!
-- 
https://mail.python.org/mailman/listinfo/python-list