[issue31415] Add -X option to show import time

[miss-islington]

miss-islington  added the comment:


New changeset 73943ce7d31595d152dc01bf0008b37c802c0d3b by Miss Islington (bot) 
in branch '3.9':
bpo-31415: importtime was made by Inada Naoki (GH-31875)
https://github.com/python/cpython/commit/73943ce7d31595d152dc01bf0008b37c802c0d3b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[miss-islington]

miss-islington  added the comment:


New changeset 9f1587e8d3f18729a0982774c6c4409472cbd9e3 by Miss Islington (bot) 
in branch '3.10':
bpo-31415: importtime was made by Inada Naoki (GH-31875)
https://github.com/python/cpython/commit/9f1587e8d3f18729a0982774c6c4409472cbd9e3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[miss-islington]

Change by miss-islington :


--
pull_requests: +29982
pull_request: https://github.com/python/cpython/pull/31884

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 12.0 -> 13.0
pull_requests: +29981
pull_request: https://github.com/python/cpython/pull/31883

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset 29624e769c5c3c1e59c6acc8b69383ead53e8a9f by Victor Stinner in 
branch 'main':
bpo-31415: importtime was made by Inada Naoki (GH-31875)
https://github.com/python/cpython/commit/29624e769c5c3c1e59c6acc8b69383ead53e8a9f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29973
pull_request: https://github.com/python/cpython/pull/31875

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Nico Schlömer]

Nico Schlömer  added the comment:

I just updated tuna [1] to support import time profiles as well.
Install with
```
pip install tuna
```
and use with
```
python -X importprofile yourfile.py 2> import.log
tuna import.log
```
See screenshot for example output.

Cheers,
Nico

[1] https://github.com/nschloe/tuna

--
nosy: +nschloe
Added file: https://bugs.python.org/file47663/sc.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[STINNER Victor]

STINNER Victor  added the comment:

FYI I'm working on the Py_Main() function in bpo-32030. My latest PR (PR 4412) 
moves the code to parse -X importtime and PYTHONPROFILEIMPORTTIME into 
Py_Main(), to group all functions parsing the command line arguments and 
environment variables.

With this change, the "importtime" option is now checked before the very first 
"Python" function call.

The new code doesn't use the sys module anymore to parse -X options, and so 
break the dependency between "importtime", the import machinery and the sys 
module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

I think there is nothing left to do for this issue, so I'm closing it.

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 088929cf62fa22c06f6a44e25915abce9048a545 by Serhiy Storchaka in 
branch 'master':
bpo-31415: Improve error handling and caching of the importtime option. (#4138)
https://github.com/python/cpython/commit/088929cf62fa22c06f6a44e25915abce9048a545


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Nov 4, 2017, at 14:29, Terry J. Reedy  wrote:

> The importtime output is sent to stderr.  Should this be documented?

Yes. :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Terry J. Reedy]

Terry J. Reedy  added the comment:

> with environment variable you get more information.

For timing package import time, such as for idlelib.pyshell, rather than bare 
python startup time, the extra information is just a bit more noise to ignore.

The importtime output is sent to stderr.  Should this be documented?  It is 
important because displaying results immediately on command prompt rather than 
redirecting to a file adds about .07 seconds to the total time reported for 
pyshell, whether I use debug or normal binaries.  The relative variation 
between repeated runs (about 1%) also seems less with redirection to a file.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

Okay, given the non-propagation and Windows issues.  I'm -1 on removing -X, +1 
on fixing the negative cache.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Antoine Pitrou]

Antoine Pitrou  added the comment:

(and Terry is right that environment variables can be significantly more 
annoying on Windows)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Antoine Pitrou]

Antoine Pitrou  added the comment:

One benefit (or drawback, depending on how you see it) of the -X option is that 
it wouldn't propagate to child processes spawned by multiprocessing.  See 
_args_from_interpreter_flags() in Lib/subprocess.py.

More generally, the idea that we should remove a command line option because 
there's an environment variable sounds dubious to me.  Should we do the same 
for all command-line options?  The faulthandler module can be enabled either 
using the `PYTHONFAULTHANDLER` environment variable or by specifying `-X 
faulthandler` on the command line.

The only serious reason I can imagine would be that the -X option has a 
significant maintenance cost.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[INADA Naoki]

INADA Naoki  added the comment:

I'm +1 to remove -X option.
Supporting both form makes code ugly and benefit is too small.
And +1 to negative cache too.   It's simple for environment varianle.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Nov 3, 2017, at 14:23, Terry J. Reedy  wrote:
> 
> Does it set the EV for the entire session (which one likely would not want), 
> or just the one command (which has no Windows equivalent that I know of)?  
> Please leave the easily remembered and typed option.

On *nix, this sets the environment variable for just this process.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Nov 3, 2017, at 14:41, Serhiy Storchaka  wrote:
> 
> Note that with environment variable you get more information.

Fun!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

On Windows you can create a 2-line bat-file that sets the environment variable 
and runs Python.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Note that with environment variable you get more information.

$ ./python -X importtime -c pass
import time: self [us] | cumulative | imported package
import time:88 | 88 | _codecs
import time:   789 |876 |   codecs
import time:   602 |602 |   encodings.aliases
import time:   809 |   2287 | encodings
...

$ PYTHONPROFILEIMPORTTIME=1 ./python -c pass
import time: self [us] | cumulative | imported package
import time:  3133 |   3133 | _frozen_importlib_external
import time:   371 |371 | zipimport
import time:   327 |327 | _codecs
import time:  2656 |   2982 |   codecs
import time:  1821 |   1821 |   encodings.aliases
import time:  2604 |   7406 | encodings
...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Terry J. Reedy]

Terry J. Reedy  added the comment:

> `PYTHONPROFILEIMPORTTIME=x python` isn’t *too* onerous

It does not work on Windows.

C:\Users\Terry>PYTHONPROFILEIMPORTTIME=x python
'PYTHONPROFILEIMPORTTIME' is not recognized as an internal or external command, 
operable program or batch file.

Does it set the EV for the entire session (which one likely would not want), or 
just the one command (which has no Windows equivalent that I know of)?  Please 
leave the easily remembered and typed option.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Nov 3, 2017, at 11:05, Serhiy Storchaka  wrote:
> 
> But now, after adding the environment variable, do we still need the -X 
> option? From a user side I don't see much difference between specifying an 
> option and an environment variable, but the code for handling the environment 
> variable is much simpler.

It’s a good question.  I guess in the limited amount of time I’ve used the 
feature so far, I find it kind of nice to be able to type `python -X 
importtime` when doing general import profiling.  The use case for the 
environment variable is more compelling IMHO which is why I really wanted to 
add it.  I suppose typing `PYTHONPROFILEIMPORTTIME=x python` isn’t *too* 
onerous, even if it is kind of a weird long mashed together word.

I suppose I’m -0 on removing the -X option, and +0 on adding the negative cache.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> I didn't think it's worth enough because import will be much slower than one 
> dict lookup.

I agree. The main value of my patch is handling possible (but very unlike) 
errors. Implementing negative value caching adds not many lines to the existing 
code and additional error handling.

But now, after adding the environment variable, do we still need the -X option? 
From a user side I don't see much difference between specifying an option and 
an environment variable, but the code for handling the environment variable is 
much simpler.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Nov 2, 2017, at 22:50, INADA Naoki  wrote:
> 
> When adding environment variable option, it should be documented in
> man page and `python -h`.

I thought about that, but the problem is that none of the -X options are 
documented in the —help output.  So do we only document the ones (e.g. 
importtime) that have an environment variable version, or do we now have to 
document them all?

I opted to document PYTHONPROFILEIMPORTTIME only in the cmdline documentation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[INADA Naoki]

INADA Naoki  added the comment:

When adding environment variable option, it should be documented in
man page and `python -h`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:


New changeset 700d2e4755921d6c339ff20dacecde1aea64de34 by Barry Warsaw in 
branch 'master':
bpo-31415: Support PYTHONPROFILEIMPORTTIME envvar equivalent to -X importtime 
(#4240)
https://github.com/python/cpython/commit/700d2e4755921d6c339ff20dacecde1aea64de34


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

I like this a lot and while the issue is still open, I'm piggybacking my PR 
4240 on this one.

I have a use case for enabling this feature via environment variable, so PR 
4240 adds PYTHONPROFILEIMPORTTIME which also enables this feature.

We build our tools using pex, which hardcodes the shebang at the top of the zip 
file.  This isn't easily changed, and clearly we wouldn't want to enable -X 
importtime everywhere.  When we have startup performance issues, I want to be 
able to tell people (or set it up in our CI) to set the environment variable 
and collect the data for analysis.  This is much easier than tweaking the 
shebang or invoking the zip with an explicit command.  Also, there are 
potential gotchas with loading too much up on the shebang line (IIRC, there are 
line length limits and/or multiple option limits for shebangs, e.g. if we 
wanted to have both -I and -X).

I haven't fixed the initialization check problem that serhiy.storchaka points 
out, since that's addressed by PR 4138.  But maybe I should fold that change 
into mine.

--
nosy: +barry
stage: patch review -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
pull_requests: +4205
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[INADA Naoki]

INADA Naoki  added the comment:

Xoptions is not environment variable.

Some modules are imported before xoptions are parsed.
So negative cache for xoptions is little more complex than
negative cache for environment variable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[STINNER Victor]

STINNER Victor  added the comment:

> I didn't think it's worth enough because import will be much slower than one 
> dict lookup.

I don't care much of performance. For the consistency with other environment 
variables, I like to only read the environment variable once ("at startup"), 
and not do it each time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[INADA Naoki]

INADA Naoki  added the comment:

Does it worth enough?
I didn't think it's worth enough because import will be much slower than one 
dict lookup.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The importtime option is cached only if it is set. If it is not set (the common 
case), it is not cached.

PR 4138 makes it be cached in the common case and improves errors handling.

--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +4101

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[STINNER Victor]

STINNER Victor  added the comment:

Ok, the feature was implemented, the Windows clock resolution was fixed as 
well. It's time to close this issue.

If you still have concerns about import time, see open discussions on the 
python-dev mailing list, or open new issues ;-)

Thanks again Naoki INADA for this nice and useful feature!

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[STINNER Victor]

STINNER Victor  added the comment:

> Why global _PyTime_GetWinPerfCounterWithInfo() is needed at all? It seems to 
> me that _PyTime_GetPerfCounterWithInfo() can be used instead. Smaller API is 
> better.

I chose to expose _PyTime_GetWinPerfCounterWithInfo() to make my change as 
small as possible.

I didn't want to break the old time.clock() clock by mistake.

By the way, time.clock() is deprecated since Python 3.3. It's maybe time to 
drop it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Why global _PyTime_GetWinPerfCounterWithInfo() is needed at all? It seems to me 
that _PyTime_GetPerfCounterWithInfo() can be used instead. Smaller API is 
better.

_PyTime_GetPerfCounter() is a simple wrapper around 
_PyTime_GetPerfCounterWithInfo() and is used only in one place. Getting rid of 
it will simplify C API too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +3930

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +3914

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[STINNER Victor]

STINNER Victor  added the comment:


New changeset a997c7b434631f51e00191acea2ba6097691e859 by Victor Stinner in 
branch 'master':
bpo-31415: Add _PyTime_GetPerfCounter() and use it for -X importtime (#3936)
https://github.com/python/cpython/commit/a997c7b434631f51e00191acea2ba6097691e859


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +3910
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[STINNER Victor]

STINNER Victor  added the comment:

> Could we change PyTime_GetMonotonicClock implementation to use it on Windows?

No. See the PEP 418 for the rationale.

But we could add a _PyTime_GetPerfCounter() to Python/pytime.c.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[INADA Naoki]

INADA Naoki  added the comment:

https://github.com/python/cpython/blob/27c623c845dd6e4b8e1782666ca3a956636da266/Modules/timemodule.c#L80-L112

time module has perf_counter implementation based on QueryPerformanceCounter().

Could we change PyTime_GetMonotonicClock implementation to use it on Windows?

I don't want to so much code to import.c.  But if pytime.c has nice clock
for fine grained monotonic performance counter, I want to use it in
import.c.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Terry J. Reedy]

Terry J. Reedy  added the comment:

This is pretty useless on Windows because you are using a clock with 15-16 
milli (not micro) second resolution on Windows.  Before Victor introduced 
time.perf_counter, with decent behavior on all systems, timeit used different 
time module functions on Windows and *nix because the two different functions 
had such different behavior on the two systems.  I believe it used time.clock 
on Windows (1 second resolution of *nix?) and time.time otherwise (.016 sec 
resolution on Windows).  

Time.monotonic seems to be based on time.time as it has the same low resolution 
on Windows.  Run something like

for i in range(1000): time.monotonic()

and you will see identical times except for 1 or 2 jumps of .016.  So the C 
equivalent or time.clock or time.perf_counter is needed on Windows.  Victor can 
advise as he knows this better than me.

An example of nearly useless output, with 15-16 millisecond resolution.

f:\dev\3x>python -X importtime -c "import idlelib.pyshell"
Running Debug|Win32 interpreter...
import time: self [us] | cumulative | imported package
import time: 0 |  0 | _codecs
import time: 0 |  0 |   codecs
import time: 0 |  0 |   encodings.aliases
import time: 15000 |  15000 | encodings
import time: 0 |  0 | encodings.utf_8
import time: 0 |  0 | _signal
import time: 0 |  0 | encodings.latin_1
import time: 0 |  0 | _weakrefset
import time: 0 |  0 |   abc
import time: 0 |  0 | io
import time: 0 |  0 |   _stat
import time: 0 |  0 | stat
import time: 0 |  0 |   genericpath
import time: 0 |  0 | ntpath
import time: 0 |  0 | _collections_abc
import time: 16000 |  16000 |   os
import time: 0 |  0 |   _sitebuiltins
import time: 0 |  0 |   sitecustomize
import time: 0 |  0 |   usercustomize
import time: 16000 |  32000 | site
import time: 0 |  0 |   idlelib
import time: 0 |  0 |   types
import time: 0 |  0 |   _collections
import time: 0 |  0 | enum
import time: 16000 |  16000 | _tkinter
import time: 0 |  0 | tkinter.constants
import time: 0 |  0 | _sre
import time: 0 |  0 |   sre_constants
import time: 0 |  0 | sre_parse
import time: 0 |  0 |   sre_compile
import time: 0 |  0 | _functools
import time: 0 |  0 | _operator
import time: 0 |  0 |   operator
import time: 0 |  0 |   keyword
import time: 0 |  0 | _heapq
import time: 0 |  0 |   heapq
import time: 0 |  0 |   itertools
import time: 15000 |  15000 |   reprlib
import time: 0 |  15000 | collections
import time: 0 |  15000 |   functools
import time: 0 |  0 |   _locale
import time: 0 |  0 |   copyreg
import time: 16000 |  31000 | re
import time: 0 |  47000 |   tkinter
import time: 0 |  0 | tkinter.commondialog
import time: 16000 |  16000 |   tkinter.messagebox
import time: 0 |  0 |   token
import time: 0 |  0 | tokenize
import time: 0 |  0 |   linecache
import time: 0 |  0 | traceback
import time: 15000 |  15000 |   __future__
import time: 0 |  15000 | codeop
import time: 0 |  15000 |   code
import time: 0 |  0 |   time
import time: 0 |  0 |   signal
import time: 0 |  0 |   warnings
import time: 0 |  0 |   errno
import time: 0 |  0 |   threading
import time: 0 |  0 |   msvcrt
import time: 0 |  0 |   _winapi
import time: 16000 |  16000 | subprocess
import time: 16000 |  32000 |   platform
import time: 0 |  0 | _socket
import time: 0 |  0 |   collections.abc
import time: 0 |  0 |   math
import time: 0 |  0 |   select
import time: 0 |  0 | selectors
import time: 15000 |  15000 |   socket
import time: 16000 |  16000 |   configparser
import time: 0 |  0 |   tkinter.font
import time: 0 |  0 |   _bootlocale
import time: 0 |  0 |   encodings.cp1252
import time: 31000 |  47000 | idlelib.config
import time: 0 

[issue31415] Add -X option to show import time

[STINNER Victor]

STINNER Victor  added the comment:

Thank you Naoki for this nice enhancement! Tooling always help to take smart 
decisions on optimizations.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[INADA Naoki]

INADA Naoki  added the comment:


New changeset 1a87de7fcfa3c19f08e29047337c350b4a32b259 by INADA Naoki in branch 
'master':
bpo-31415: Add `-X importtime` option (GH-3490)
https://github.com/python/cpython/commit/1a87de7fcfa3c19f08e29047337c350b4a32b259


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[INADA Naoki]

Change by INADA Naoki :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Łukasz Langa]

Łukasz Langa  added the comment:

Related: https://bugs.python.org/issue31574

--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[STINNER Victor]

STINNER Victor added the comment:

> The output in PR 3765 partially duplicates the output of the -v option.

Right. But the idea is to be able to use "grep 'import time:'" to only extract 
importtime logs. "-v" logs different kind of informations.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The output in PR 3765 partially duplicates the output of the -v option.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +3750

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[INADA Naoki]

INADA Naoki added the comment:

how can I make it more machine readable?

On 2017年9月25日(月) 19:39 Christian Heimes  wrote:

>
> Christian Heimes added the comment:
>
> +1 for an option to profile import time.
>
> I have a minor complain: The output format is not machine-friendly, which
> makes it harder to perform automatic analysis. I created #31574 / PR3749 to
> add dtrace probes.
>
> --
> nosy: +christian.heimes
>
> ___
> Python tracker 
> 
> ___
>
-- 
Inada Naoki 

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Christian Heimes]

Christian Heimes added the comment:

+1 for an option to profile import time.

I have a minor complain: The output format is not machine-friendly, which makes 
it harder to perform automatic analysis. I created #31574 / PR3749 to add 
dtrace probes.

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[INADA Naoki]

INADA Naoki added the comment:

I don't have any good idea for it.

I usually run `python -X importtime -c 'import functools'` several times to 
ensure pyc files are created and it is cached by OS.

On Linux, `strace -c` and `time` command can be used.
But I don't know cross platform way to do it.

$ strace -c python -c 'import asyncio'
% time seconds  usecs/call callserrors syscall
-- --- --- - - 
  0.000.00   0   244   read
  0.000.00   0   160 2 open
  0.000.00   0   161   close
  0.000.00   0   52746 stat
  0.000.00   0   266   fstat
  0.000.00   016   lstat
  0.000.00   0   114 6 lseek
  0.000.00   0   109   mmap

$ /usr/bin/time python -c 'import asyncio'
0.12user 0.00system 0:00.13elapsed 98%CPU (0avgtext+0avgdata 17004maxresident)k
0inputs+0outputs (0major+2613minor)pagefaults 0swaps

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Raymond Hettinger]

Raymond Hettinger added the comment:

Naoki, is it possible to separate out how much of this is I/O vs CPU time?

If the I/O dominates, optimizations tend toward zipimports, faster drives, disk 
caching etc.  If the CPU time is dominant, different techniques are used (lazy 
evaluation, splitting-out parts of a package, etc).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Guido van Rossum]

Guido van Rossum added the comment:

@rhettinger
> Guido, is this something you want?

I think it is useful given how much debate there has been around startup time 
in various contexts (not just pure interpreter startup time but also startup 
time when using specific libraries, packages or applications).

> Historically, we've shown a lot of restraint when it comes to adding
> command-line options.

But this is hardly a new command-line option; -X already exists. I think it's 
reasonable to add it here.

> Also, I'm not sure we want to induce people to start moving their imports
> inside function calls. [...]

That's a big leap from collecting the data to suggesting a specific solution. I 
don't think that making it harder to get the data really is the right approach 
to discouraging a debatable practice (plus, there are many situations where the 
practice *is* useful -- just as there are many situations where it's 
counterproductive). Consenting adults.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Raymond Hettinger]

Raymond Hettinger added the comment:

Guido, is this something you want?

Historically, we've shown a lot of restraint when it comes to adding 
command-line options.  

Also, I'm not sure we want to induce people to start moving their imports 
inside function calls.  Just because we sometimes adopt constipated programming 
practices in the standard library doesn't mean we think everyone should do it.  
(This is doubly true because sometimes the savings is a false savings if the 
import eventually occurs downstream and because there is currently work being 
done on lazy imports that would make the technique irrelevant).

--
assignee:  -> gvanrossum
nosy: +gvanrossum, rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[INADA Naoki]

INADA Naoki added the comment:

> Actually it is easy. You need just one global integer accumulator and a local 
> variable for keeping a temporary copy of it.

You're right!  I updated my pull request.
See current output: 
https://gist.github.com/methane/185d75a3c8da762d85317dd95918a623

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> Stack management is difficult than inc/dec integer in multi threading case.

Actually it is easy. You need just one global integer accumulator and a local 
variable for keeping a temporary copy of it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[INADA Naoki]

INADA Naoki added the comment:

> If this proposition be accepted I would want to see not only cumulated times, 
> but also pure times, without time of nested imports.

It can be calculated by script.
I don't want to add more complexity to import.c.

> I guess this feature doesn't work correctly with threading.

Yes, it should be documented.
And that's reason why I don't want to support time without nested import.
Stack management is difficult than inc/dec integer in multi threading case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Brett Cannon]

Brett Cannon added the comment:

Ah, OK. Then I wouldn't put it in square brackets as it seems to suggest it's 
somehow disconnected from the number.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

If this proposition be accepted I would want to see not only cumulated times, 
but also pure times, without time of nested imports.

I guess this feature doesn't work correctly with threading.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[INADA Naoki]

INADA Naoki added the comment:

It means μs (micro seconds), in ASCII.

On 2017年9月20日(水) 3:35 Brett Cannon  wrote:

>
> Brett Cannon added the comment:
>
> What does the "[us]" mean?
>
> --
> nosy: +brett.cannon
>
> ___
> Python tracker 
> 
> ___
>
-- 
Inada Naoki 

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Brett Cannon]

Brett Cannon added the comment:

What does the "[us]" mean?

--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[INADA Naoki]

Changes by INADA Naoki :


--
keywords: +patch
pull_requests: +3483
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[INADA Naoki]

New submission from INADA Naoki:

I used my local patch to profile import time.  I think it's useful for
3rd party authors.
For example:

./python -Ximportprofile -c 'import traceback'
- _codecs 71 [us]
  - codecs 677 [us]
  - encodings.aliases 412 [us]
- encodings 1688 [us]
- encodings.utf_8 250 [us]
- _signal 120 [us]
- encodings.latin_1 281 [us]
- _weakrefset 301 [us]
  - abc 593 [us]
- io 1032 [us]
  - _locale 100 [us]
- _bootlocale 262 [us]
- errno 165 [us]
  - _stat 73 [us]
- stat 270 [us]
  - genericpath 166 [us]
- posixpath 456 [us]
- _collections_abc 1798 [us]
  - os 3366 [us]
  - _sitebuiltins 221 [us]
  - sitecustomize 408 [us]
  - usercustomize 168 [us]
- site 5019 [us]
  - _operator 104 [us]
- operator 873 [us]
- keyword 177 [us]
  - _heapq 188 [us]
- heapq 399 [us]
- itertools 140 [us]
- reprlib 224 [us]
- _collections 89 [us]
  - collections 3009 [us]
  - _functools 76 [us]
- collections.abc 267 [us]
  - types 592 [us]
  - weakref 499 [us]
- functools 1726 [us]
- enum 894 [us]
  - _sre 97 [us]
- sre_constants 428 [us]
  - sre_parse 849 [us]
- sre_compile 1267 [us]
- copyreg 200 [us]
  - re 3035 [us]
  - token 222 [us]
- tokenize 4660 [us]
  - linecache 6592 [us]
- traceback 10108 [us]

--
components: Interpreter Core
messages: 301858
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: Add -X option to show import time
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com