This Change has been withdrawn and replaced with
https://fedoraproject.org/wiki/Changes/PythonNoSemanticInterpositionSpeedup
Discussion is at
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/ONEOQ4XWRL7IUNTQA7YFSFTNHXY5MJS4/
--
Ben Cotton
He / Him / His
Fedora
* John Reiser:
Thinking aloud: does anyone ever use symbol overriding for anything
other than glibc?
>
>>> Yes. It is particularly useful for "spear fishing" debugging of lower-level
>>> interfaces in large, complex multi-process applications.
>
>> That only seems to need shallow
Thinking aloud: does anyone ever use symbol overriding for anything
other than glibc?
Yes. It is particularly useful for "spear fishing" debugging of lower-level
interfaces in large, complex multi-process applications.
That only seems to need shallow interposition, though. In most cases,
* John Reiser:
> On 2019-11-15 at 14:51 UTC, David Malcolm wrote:
>
>> Thinking aloud: does anyone ever use symbol overriding for anything
>> other than glibc?
>
> Yes. It is particularly useful for "spear fishing" debugging of lower-level
> interfaces in large, complex multi-process
On 2019-11-15 at 14:51 UTC, David Malcolm wrote:
Thinking aloud: does anyone ever use symbol overriding for anything
other than glibc?
Yes. It is particularly useful for "spear fishing" debugging of lower-level
interfaces in large, complex multi-process applications. By some means
you
* David Malcolm:
> What would it do to distro-wide performance if
> -fno-semantic-interposition
> were added to the default rpm build flags, (and glibc added -fsemantic-
> interposition to override this)?
glibc already does the equivalent of -fno-semantic-interposition
manually. We even have
On Fri, 2019-11-15 at 16:28 +0100, Miro Hrončok wrote:
> On 15. 11. 19 16:20, Vít Ondruch wrote:
> > Dne 15. 11. 19 v 15:51 David Malcolm napsal(a):
> > > On Fri, 2019-11-15 at 12:31 +, Daniel P. Berrangé wrote:
> > > > On Fri, Nov 15, 2019 at 01:23:09PM +0100, Miroslav Suchý wrote:
> > > > >
On 14. 11. 19 14:47, Miro Hrončok wrote:
On 05. 11. 19 16:03, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/PythonStaticSpeedup
== Summary ==
Python 3 traditionally in Fedora was built with a shared library
libpython3.?.so and the final binary was dynamically linked against
that
On Fri, Nov 15, 2019 at 09:51:45AM -0500, David Malcolm wrote:
> On Fri, 2019-11-15 at 12:31 +, Daniel P. Berrangé wrote:
> > On Fri, Nov 15, 2019 at 01:23:09PM +0100, Miroslav Suchý wrote:
> > > Dne 15. 11. 19 v 10:21 Victor Stinner napsal(a):
> > > > I'm not sure if we need a Fedora change
On 15. 11. 19 16:20, Vít Ondruch wrote:
Dne 15. 11. 19 v 15:51 David Malcolm napsal(a):
On Fri, 2019-11-15 at 12:31 +, Daniel P. Berrangé wrote:
On Fri, Nov 15, 2019 at 01:23:09PM +0100, Miroslav Suchý wrote:
Dne 15. 11. 19 v 10:21 Victor Stinner napsal(a):
I'm not sure if we need a
Dne 15. 11. 19 v 15:51 David Malcolm napsal(a):
> On Fri, 2019-11-15 at 12:31 +, Daniel P. Berrangé wrote:
>> On Fri, Nov 15, 2019 at 01:23:09PM +0100, Miroslav Suchý wrote:
>>> Dne 15. 11. 19 v 10:21 Victor Stinner napsal(a):
I'm not sure if we need a Fedora change just for a compiler
On Fri, 2019-11-15 at 12:31 +, Daniel P. Berrangé wrote:
> On Fri, Nov 15, 2019 at 01:23:09PM +0100, Miroslav Suchý wrote:
> > Dne 15. 11. 19 v 10:21 Victor Stinner napsal(a):
> > > I'm not sure if we need a Fedora change just for a compiler flag.
> > > Again, the only drawback is that we will
On 15. 11. 19 13:24, Aleksandra Fedorova wrote:
Hi
On Fri, Nov 15, 2019 at 10:22 AM Victor Stinner wrote:
Hi Jan,
With the helper of Florian Weimer and Charalampos Stratakis, we also agreed to
test this flag in priority. I understood that it disables the LD_PRELOAD
feature: it's no longer
- Original Message -
> From: "Victor Stinner"
> To: devel@lists.fedoraproject.org
> Sent: Friday, November 15, 2019 10:21:44 AM
> Subject: Re: Fedora 32 System-Wide Change proposal: Build Python 3 to
> statically link with libpython3.8.a for better
> performance
>
> Hi Jan,
>
> With
On Fri, Nov 15, 2019 at 01:23:09PM +0100, Miroslav Suchý wrote:
> Dne 15. 11. 19 v 10:21 Victor Stinner napsal(a):
> > I'm not sure if we need a Fedora change just for a compiler flag. Again,
> > the only drawback is that we will no longer be able to override a symbol
> > using LD_PRELOAD.
Hi
On Fri, Nov 15, 2019 at 10:22 AM Victor Stinner wrote:
>
> Hi Jan,
>
> With the helper of Florian Weimer and Charalampos Stratakis, we also agreed
> to test this flag in priority. I understood that it disables the LD_PRELOAD
> feature: it's no longer possible to override symbols in
Dne 15. 11. 19 v 10:21 Victor Stinner napsal(a):
I'm not sure if we need a Fedora change just for a compiler flag. Again, the
only drawback is that we will no longer be able to override a symbol using
LD_PRELOAD. Honestly, I never did that. I don't see any use case for that. But
I used
Hi Jan,
With the helper of Florian Weimer and Charalampos Stratakis, we also agreed to
test this flag in priority. I understood that it disables the LD_PRELOAD
feature: it's no longer possible to override symbols in libpython with
LD_PRELOAD. Thanks to that, the compiler can avoid PLT
On 05. 11. 19 16:03, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/PythonStaticSpeedup
== Summary ==
Python 3 traditionally in Fedora was built with a shared library
libpython3.?.so and the final binary was dynamically linked against
that shared library. This change is about creating
On 14. 11. 19 2:48, Gordon Messmer wrote:
On 11/12/19 2:21 PM, John M. Harris Jr wrote:
However, I believe there's a third option here. It could be as simple as
providing a python3-static in addition, and NOT using `alternatives`.
Is that an option, though? From the discussion, I was under
On 11/12/19 2:21 PM, John M. Harris Jr wrote:
However, I believe there's a third option here. It could be as simple as
providing a python3-static in addition, and NOT using `alternatives`.
Is that an option, though? From the discussion, I was under the
impression that static vs dynamic
Dne 13. 11. 19 v 0:21 Kevin Kofler napsal(a):
But the wasted space will be even more, because now you have libpython, the
dynamic python3 linked against it, AND the python3-static binary. So it does
not address the issue at all.
+1
"Requires: /path/to/my/python3" is no go. Because no
On Tuesday, November 12, 2019 4:21:03 PM MST Kevin Kofler wrote:
> But the wasted space will be even more, because now you have libpython, the
> dynamic python3 linked against it, AND the python3-static binary. So it
> does not address the issue at all.
Yes, that would be the case if something
John M. Harris Jr wrote:
> However, I believe there's a third option here. It could be as simple as
> providing a python3-static in addition, and NOT using `alternatives`. This
> way, packages and scripts that actually need the performance improvements
> can directly call python3-static, and
On 12. 11. 19 23:21, John M. Harris Jr wrote:
If that software was to be packaged, in this case, you'd simply:
Requires: python3
change the shebang to /path/to/my/python3
I am strongly against any proposals that involve /path/to/my/python3.
However, I believe there's a third option here. It
On Tuesday, November 12, 2019 6:18:00 AM MST Miro Hrončok wrote:
> On 12. 11. 19 14:00, Miroslav Suchý wrote:
>
> > Dne 05. 11. 19 v 16:03 Ben Cotton napsal(a):
> >
> >> == Summary ==
> >> Python 3 traditionally in Fedora was built with a shared library
> >> libpython3.?.so and the final binary
> Will python still be PIE? Or will you disable hardening and build it as
> a position-dependent binary?
Yes, the python ELF binary still uses PIE (Position Independent Executable). I
checked the patched package:
$ file /usr/bin/python3.8
/usr/bin/python3.8: ELF 64-bit LSB pie executable,
On 12. 11. 19 14:18, Miro Hrončok wrote:
On 12. 11. 19 14:00, Miroslav Suchý wrote:
Dne 05. 11. 19 v 16:03 Ben Cotton napsal(a):
== Summary ==
Python 3 traditionally in Fedora was built with a shared library
libpython3.?.so and the final binary was dynamically linked against
that shared
On 12. 11. 19 14:00, Miroslav Suchý wrote:
Dne 05. 11. 19 v 16:03 Ben Cotton napsal(a):
== Summary ==
Python 3 traditionally in Fedora was built with a shared library
libpython3.?.so and the final binary was dynamically linked against
that shared library. This change is about creating the
Dne 05. 11. 19 v 16:03 Ben Cotton napsal(a):
== Summary ==
Python 3 traditionally in Fedora was built with a shared library
libpython3.?.so and the final binary was dynamically linked against
that shared library. This change is about creating the static library
and linking the final python3
On Mon, Nov 11, 2019 at 5:23 AM Florian Weimer wrote:
>
> * John M. Harris, Jr.:
>
> > Anyone that has ever worked with CD images understands that every megabyte
> > counts.
>
> It's clearly not a priority for Fedora. It wouldn't be too difficult to
> replace glibc-all-langpacks with
* John M. Harris, Jr.:
> Anyone that has ever worked with CD images understands that every megabyte
> counts.
It's clearly not a priority for Fedora. It wouldn't be too difficult to
replace glibc-all-langpacks with glibc-locale-source in the installer,
going from 26 MiB to less than 5 MiB
Dne 08. 11. 19 v 20:40 Fabio Valentini napsal(a):
> On Fri, Nov 8, 2019 at 6:01 PM Charalampos Stratakis
> wrote:
>>
>>
>> - Original Message -
>>> From: "Vít Ondruch"
>>> Cc: devel@lists.fedoraproject.org
>>> Sent: Friday, November 8, 2019 10:01:47 AM
>>> Subject: Re: Fedora 32
On 11/5/19, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/PythonStaticSpeedup
== Summary ==
Python 3 traditionally in Fedora was built with a shared library
libpython3.?.so and the final binary was dynamically linked against
that shared library. This change is about creating the
On Sunday, November 10, 2019, Kevin Kofler wrote:
> Frantisek Zatloukal wrote:
> > How is statically linked libpython hack? It's just a different way to do
> > it, isn't it?
>
> It means you are shipping 2 copies of the Python interpreter, one
> statically
> linked into the python3 binary and
Frantisek Zatloukal wrote:
> How is statically linked libpython hack? It's just a different way to do
> it, isn't it?
It means you are shipping 2 copies of the Python interpreter, one statically
linked into the python3 binary and one as a shared library. This is much
less elegant than shipping
* Ben Cotton:
> https://fedoraproject.org/wiki/Changes/PythonStaticSpeedup
>
> == Summary ==
> Python 3 traditionally in Fedora was built with a shared library
> libpython3.?.so and the final binary was dynamically linked against
> that shared library. This change is about creating the static
On Sat, Nov 9, 2019 at 8:31 AM Kevin Kofler wrote:
> Sorry, but I'm with Vít there. If Python is running into toolchain
> limitations, the goal should be to work on improving the toolchain, not to
> add a hack with side effects (bloat, compatibility issues) to the Python
> package, a hack with
Charalampos Stratakis wrote:
>> From: "Vít Ondruch"
>> 1) Apparently, there is some work which needs to be done on the
>> toolchain. Applying workarounds just hides the issues and we won't move
>> forward ever.
>
> I think it's more reasonable to do a small SPEC change in Python to
> achieve a
On 08. 11. 19 20:40, Fabio Valentini wrote:
(Side note: I wouldn't even have objected to this being a
Self-Contained Change, since it basically only affects one package -
albeit an important one (python3)
Doing this as a system wide change was my decision, Harris (Charalampos) wanted
to do
On Friday, November 8, 2019 3:20:33 PM MST Daniel Walsh wrote:
> On 11/8/19 5:16 PM, John M. Harris Jr wrote:
>
> > On Tuesday, November 5, 2019 12:09:55 PM MST Martin Kolman wrote:
> >
> >> On Tue, 2019-11-05 at 19:41 +0100, Kevin Kofler wrote:
> >>
> >>
> >>
> Python 3 traditionally in
On 11/8/19 5:16 PM, John M. Harris Jr wrote:
> On Tuesday, November 5, 2019 12:09:55 PM MST Martin Kolman wrote:
>> On Tue, 2019-11-05 at 19:41 +0100, Kevin Kofler wrote:
>>
Python 3 traditionally in Fedora was built with a shared library
libpython3.?.so and the final binary was
On Tuesday, November 5, 2019 12:09:55 PM MST Martin Kolman wrote:
> On Tue, 2019-11-05 at 19:41 +0100, Kevin Kofler wrote:
>
> > > Python 3 traditionally in Fedora was built with a shared library
> > > libpython3.?.so and the final binary was dynamically linked against
> > > that shared library.
On Fri, Nov 8, 2019 at 6:01 PM Charalampos Stratakis
wrote:
>
>
>
> - Original Message -
> > From: "Vít Ondruch"
> > Cc: devel@lists.fedoraproject.org
> > Sent: Friday, November 8, 2019 10:01:47 AM
> > Subject: Re: Fedora 32 System-Wide Change proposal: Build Python 3 to
> > statically
- Original Message -
> From: "Vít Ondruch"
> Cc: devel@lists.fedoraproject.org
> Sent: Friday, November 8, 2019 10:01:47 AM
> Subject: Re: Fedora 32 System-Wide Change proposal: Build Python 3 to
> statically link with libpython3.8.a for better
> performance
>
>
> Dne 07. 11. 19 v
Dne 07. 11. 19 v 23:08 Florian Weimer napsal(a):
> * Vít Ondruch:
>
>> Dne 07. 11. 19 v 16:05 Tom Hughes napsal(a):
>>> On 07/11/2019 14:59, Victor Stinner wrote:
>>>
I cannot explain why PLT is needed when a libpython function calls a
libpython function.
>>> Because an exported symbol
On Thu, 07 Nov 2019 22:36:44 +0100, Jan Kratochvil wrote:
> On Thu, 07 Nov 2019 15:59:41 +0100, Victor Stinner wrote:
> > I cannot explain why inlining cannot be done more often in libpython.
> >
> > I cannot explain why PLT is needed when a libpython function calls a
> > libpython function.
>
Chris Adams wrote:
> Alternately, is there some way to reduce the overhead of the dynamic
> library (that could help multiple languages)?
-fno-semantic-interposition
Can this please be tried on the dynamically linked Python?
Kevin Kofler
___
* Vít Ondruch:
> Dne 07. 11. 19 v 16:05 Tom Hughes napsal(a):
>> On 07/11/2019 14:59, Victor Stinner wrote:
>>
>>> I cannot explain why PLT is needed when a libpython function calls a
>>> libpython function.
>>
>> Because an exported symbol in an ELF shared library is subject to
>> potential
On Thu, 07 Nov 2019 15:59:41 +0100, Victor Stinner wrote:
> I cannot explain why inlining cannot be done more often in libpython.
>
> I cannot explain why PLT is needed when a libpython function calls a
> libpython function.
Could you re-run the benchmark with shared library but with
Once upon a time, Miro Hrončok said:
> If we build things statically with libraries, it's a can full of worms.
> What needs to be said about this change that we don't staticaly link
> against different libraries, we just build CPython source into one
> "fat" executable instead of splitting it
On 07. 11. 19 17:15, Vít Ondruch wrote:
Dne 07. 11. 19 v 16:05 Tom Hughes napsal(a):
On 07/11/2019 14:59, Victor Stinner wrote:
I cannot explain why PLT is needed when a libpython function calls a
libpython function.
Because an exported symbol in an ELF shared library is subject to
On Thu, Nov 07, 2019 at 05:15:18PM +0100, Vít Ondruch wrote:
> This sounds like the whole system could be 25% faster if we link statically.
Yeah, that's the advantage of static linking. This brings us stuff
like statically linked distibutions - https://sta.li/faq/
Generally advantages of
Dne 07. 11. 19 v 16:05 Tom Hughes napsal(a):
> On 07/11/2019 14:59, Victor Stinner wrote:
>
>> I cannot explain why PLT is needed when a libpython function calls a
>> libpython function.
>
> Because an exported symbol in an ELF shared library is subject to
> potential interposition using
On 07/11/2019 14:59, Victor Stinner wrote:
I cannot explain why PLT is needed when a libpython function calls a libpython
function.
Because an exported symbol in an ELF shared library is subject to
potential interposition using LD_PRELOAD so the calls need to go
through the PLT to be
> Where are these number coming from?
There are pyperformance results:
https://fedoraproject.org/wiki/Changes/PythonStaticSpeedup#Benefit_to_Fedora
It's the official benchmark suite to measure the Python performance on
speed.python.org.
I ran the benchmarks on my laptop using CPU isolation
On Wed, 6 Nov 2019 at 05:50, Vít Ondruch wrote:
> Dne 05. 11. 19 v 16:03 Ben Cotton napsal(a):
> > When we compile the python3 package on Fedora (prior to this change),
> > we create the libpython3.?.so shared library and the final python3
> > binary (/usr/bin/python3) is dynamically linked
On 06. 11. 19 17:44, Alexander Bokovoy wrote:
If you'd be able to help us removing this linking dependency, that would
be great.
We would. However we'd only invest the time and energy into it if this change is
accepted, not before that. IF samba and or freeipa breaks, that would be Fedora
32
On ke, 06 marras 2019, Miro Hrončok wrote:
On 06. 11. 19 11:41, Alexander Bokovoy wrote:
Python extension modules embedding Python and linking to libpython
- needs to be evaluated case by case
- changes to cmake/autotools are needed
- changes in code might be necessary as well
- if not
On 06. 11. 19 11:41, Alexander Bokovoy wrote:
Python extension modules embedding Python and linking to libpython
- needs to be evaluated case by case
- changes to cmake/autotools are needed
- changes in code might be necessary as well
- if not changed, might misbehave
- Python Maint will
On Wed, Nov 06, 2019 at 12:12:54PM +0100, Jan Kratochvil wrote:
> On Wed, 06 Nov 2019 11:49:18 +0100, Vít Ondruch wrote:
> > > we can achieve a
> > > performance gain of 5% to 27% depending on the workload.
> >
> > Where are these number coming from? And what is the reason for the
> > performance
On Wed, 06 Nov 2019 11:49:18 +0100, Vít Ondruch wrote:
> > we can achieve a
> > performance gain of 5% to 27% depending on the workload.
>
> Where are these number coming from? And what is the reason for the
> performance hit for dynamically linked Python?
Yes, it looks suspicious. -fPIC was a
Dne 05. 11. 19 v 16:03 Ben Cotton napsal(a):
> https://fedoraproject.org/wiki/Changes/PythonStaticSpeedup
>
> == Summary ==
> Python 3 traditionally in Fedora was built with a shared library
> libpython3.?.so and the final binary was dynamically linked against
> that shared library. This change
On ke, 06 marras 2019, Miro Hrončok wrote:
On 06. 11. 19 0:26, Michael Cronenworth wrote:
On 11/5/19 4:59 PM, Kevin Kofler wrote:
… and Calamares …
... and Domoticz (Fedora), and Kodi (RPMFusion)...
Will this be as simple as a BR change in the spec or will
application patches be necessary?
On 06. 11. 19 0:26, Michael Cronenworth wrote:
On 11/5/19 4:59 PM, Kevin Kofler wrote:
… and Calamares …
... and Domoticz (Fedora), and Kodi (RPMFusion)...
Will this be as simple as a BR change in the spec or will application patches be
necessary?
Not for most cases. See this list:
On 11/5/19 4:59 PM, Kevin Kofler wrote:
… and Calamares …
... and Domoticz (Fedora), and Kodi (RPMFusion)...
Will this be as simple as a BR change in the spec or will application patches be
necessary?
___
devel mailing list --
Miro Hrončok wrote:
> Only applications embedding Python need to link to libpython. That is what
> software like krita or blender
… and Calamares …
> are most likely doing.
Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
On 05. 11. 19 21:11, Neal Gompa wrote:
We need a way to autogenerate the the Python language ABI dependency
then. So far, no solution has been presented, and that needs to be
fixed before this can be implemented. Without that and no library
dependency, we have no way of knowing what to rebuild.
On Tue, Nov 5, 2019 at 2:17 PM Miro Hrončok wrote:
>
> On 05. 11. 19 19:41, Kevin Kofler wrote:
> >> Python 3 traditionally in Fedora was built with a shared library
> >> libpython3.?.so and the final binary was dynamically linked against
> >> that shared library. This change is about creating
On 05. 11. 19 19:41, Kevin Kofler wrote:
Python 3 traditionally in Fedora was built with a shared library
libpython3.?.so and the final binary was dynamically linked against
that shared library. This change is about creating the static library
and linking the final python3 binary against it,
I
On Tue, 2019-11-05 at 19:41 +0100, Kevin Kofler wrote:
> > Python 3 traditionally in Fedora was built with a shared library
> > libpython3.?.so and the final binary was dynamically linked against
> > that shared library. This change is about creating the static library
> > and linking the final
> Python 3 traditionally in Fedora was built with a shared library
> libpython3.?.so and the final binary was dynamically linked against
> that shared library. This change is about creating the static library
> and linking the final python3 binary against it,
I oppose this change, because this is
https://fedoraproject.org/wiki/Changes/PythonStaticSpeedup
== Summary ==
Python 3 traditionally in Fedora was built with a shared library
libpython3.?.so and the final binary was dynamically linked against
that shared library. This change is about creating the static library
and linking the final
https://fedoraproject.org/wiki/Changes/PythonStaticSpeedup
== Summary ==
Python 3 traditionally in Fedora was built with a shared library
libpython3.?.so and the final binary was dynamically linked against
that shared library. This change is about creating the static library
and linking the final
74 matches
Mail list logo
