Re: [Rpm-maint] [rpm] initial embedded python interpreter support (@rpm5.org derived) (#25)

[Vít Ondruch]

Dne 28.11.2015 v 14:37 Neal Gompa napsal(a):
> On Thu, Nov 26, 2015 at 5:20 AM, Florian Festi  >wrote:
>
> On 11/19/2015 04:09 PM, Vít Ondruch wrote:
> > What is the usecase for this? Isn't this just feature bloat?
>
> I kinda agree that this looks like feature bloat. This patch set
> needs a
> very good justification to go in upstream. The overall trend is to
> rather keep minimal installs smaller as this is a big issue for
> containers and VMs. Python is one of the candidates to be thrown
> out at
> some point.
>
> Also embedding a particular version of Python into rpm itself
> makes this
> Python version part of the spec syntax with all kind of possibly ugly
> implications.
>
> It also seems to not bring many benefits except from saving the
> interpreter starting time (which can be significant if there are many
> Python scriptlets). But the trend here is also to rather getting
> rid of
> more of the scriptlets using the new file trigger feature (We'll
> see how
> this works out).
>
> Having access to the transaction objects of the running rpm instance
> could give such justification as it can hardly done in any other way.
> But I really, really doubt that we wan to go down that route.
> Especially
> as it gives a whole new way of how scriptlets can break a transaction.
>
> As the patch set is not interfering with many places in the rpm
> code it
> should be not too hard to maintain the changes outside of the upstream
> repository for distributions with (still) rely on this feature.
>
> Florian
>
>
> ​So the main justification for the Python stuff is less about
> scriptlets and more about being able to have templates for spec files.
> In OpenMandriva, for instance, the distro-release package[0]​ includes
> a Python script that is used by the main spec file at build-time (not
> run-time!) to construct the complete spec customized for a particular
> distribution (as there's actually at least three different
> distributions hosted and built through OpenMandriva's ABF system).
> There's no really nice way to do this with the existing RPM feature set.
>
> The ways I've seen this done in the absence of this feature (and I
> have seen it attempted a few times before) are pretty terrible. It's
> usually a combination of macros, bconds, and a whole bunch of other
> crazy things to make it work, and it's an unmanageable mess.

So what are the other ways? Could you share them with us?

And why not use LUA?

Vít

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm] initial embedded python interpreter support (@rpm5.org derived) (#25)

[Neal Gompa]

On Mon, Nov 30, 2015 at 4:22 AM, Vít Ondruch  wrote:
> Dne 28.11.2015 v 14:37 Neal Gompa napsal(a):
>
> On Thu, Nov 26, 2015 at 5:20 AM, Florian Festi  wrote:
>>
>> On 11/19/2015 04:09 PM, Vít Ondruch wrote:
>> > What is the usecase for this? Isn't this just feature bloat?
>>
>> I kinda agree that this looks like feature bloat. This patch set needs a
>> very good justification to go in upstream. The overall trend is to
>> rather keep minimal installs smaller as this is a big issue for
>> containers and VMs. Python is one of the candidates to be thrown out at
>> some point.
>>
>> Also embedding a particular version of Python into rpm itself makes this
>> Python version part of the spec syntax with all kind of possibly ugly
>> implications.
>>
>> It also seems to not bring many benefits except from saving the
>> interpreter starting time (which can be significant if there are many
>> Python scriptlets). But the trend here is also to rather getting rid of
>> more of the scriptlets using the new file trigger feature (We'll see how
>> this works out).
>>
>> Having access to the transaction objects of the running rpm instance
>> could give such justification as it can hardly done in any other way.
>> But I really, really doubt that we wan to go down that route. Especially
>> as it gives a whole new way of how scriptlets can break a transaction.
>>
>> As the patch set is not interfering with many places in the rpm code it
>> should be not too hard to maintain the changes outside of the upstream
>> repository for distributions with (still) rely on this feature.
>>
>> Florian
>>
>
> So the main justification for the Python stuff is less about scriptlets and
> more about being able to have templates for spec files. In OpenMandriva, for
> instance, the distro-release package[0] includes a Python script that is
> used by the main spec file at build-time (not run-time!) to construct the
> complete spec customized for a particular distribution (as there's actually
> at least three different distributions hosted and built through
> OpenMandriva's ABF system). There's no really nice way to do this with the
> existing RPM feature set.
>
> The ways I've seen this done in the absence of this feature (and I have seen
> it attempted a few times before) are pretty terrible. It's usually a
> combination of macros, bconds, and a whole bunch of other crazy things to
> make it work, and it's an unmanageable mess.
>
>
> So what are the other ways? Could you share them with us?
>
> And why not use LUA?
>
> Vít

The other ways involved writing fairly complex macros, doing some
shell-scripting things, and using conditionals/bconds to switch around
values. It's generally quite messy and difficult to follow/maintain. I
don't know if I have an example of it anymore, as the other Mandriva
derivative (Mageia) greatly simplified theirs because they don't care
to support all of that. Since OpenMandriva is designed as a base to
build multiple distributions (ROSA Desktop, OpenMandriva Lx, etc.)
from, this particular requirement remains.

I do not know if it can be done with Lua in the same manner that it is
done in Python in OpenMandriva. Nothing I've read suggests it is
possible to import Lua scripts and use functions in those scripts
within the spec. I'm also not too familiar with Lua as a language,
either.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm] initial embedded python interpreter support (@rpm5.org derived) (#25)

[Neal Gompa]

On Thu, Nov 26, 2015 at 5:20 AM, Florian Festi  wrote:

> On 11/19/2015 04:09 PM, Vít Ondruch wrote:
> > What is the usecase for this? Isn't this just feature bloat?
>
> I kinda agree that this looks like feature bloat. This patch set needs a
> very good justification to go in upstream. The overall trend is to
> rather keep minimal installs smaller as this is a big issue for
> containers and VMs. Python is one of the candidates to be thrown out at
> some point.
>
> Also embedding a particular version of Python into rpm itself makes this
> Python version part of the spec syntax with all kind of possibly ugly
> implications.
>
> It also seems to not bring many benefits except from saving the
> interpreter starting time (which can be significant if there are many
> Python scriptlets). But the trend here is also to rather getting rid of
> more of the scriptlets using the new file trigger feature (We'll see how
> this works out).
>
> Having access to the transaction objects of the running rpm instance
> could give such justification as it can hardly done in any other way.
> But I really, really doubt that we wan to go down that route. Especially
> as it gives a whole new way of how scriptlets can break a transaction.
>
> As the patch set is not interfering with many places in the rpm code it
> should be not too hard to maintain the changes outside of the upstream
> repository for distributions with (still) rely on this feature.
>
> Florian
>
>
​So the main justification for the Python stuff is less about scriptlets
and more about being able to have templates for spec files. In
OpenMandriva, for instance, the distro-release package[0]​ includes a
Python script that is used by the main spec file at build-time (not
run-time!) to construct the complete spec customized for a particular
distribution (as there's actually at least three different distributions
hosted and built through OpenMandriva's ABF system). There's no really nice
way to do this with the existing RPM feature set.

The ways I've seen this done in the absence of this feature (and I have
seen it attempted a few times before) are pretty terrible. It's usually a
combination of macros, bconds, and a whole bunch of other crazy things to
make it work, and it's an unmanageable mess.

Most of the usage of this feature is centered about the build-time
availability rather than the run-time one (in fact, the run-time
functionality is horrifically broken in rpm5, from what I can tell).
However, *I* don't want this code to go in without *everything* working,
including scriptlets.

Perhaps if it was actually possible to maintain as an actual RPM plugin, it
could be maintained as such. I believe Per Øyvind Karlsen is trying to make
it as non-invasive as possible so that it *could* be as much like a plugin
as possible. However, maintaining it as a plugin isn't really possible
because it *does* amend the spec syntax.

This feature would also likely be a candidate for being disabled by default
rather than enabled, since it's not particularly useful for anyone unless
they actually have a goal of making it easier to create spec templates
where you can tweak a few things and customize it that way. Not a lot of
distributions want to go out of their way to make it easy to do that.

[0]: https://abf.io/openmandriva/distro-release


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm] initial embedded python interpreter support (@rpm5.org derived) (#25)

[proyvind]

I'm not surprised by yum segfaulting, separate python interpreters outside of 
rpm running in the same address space isn't really a scenario taken into 
consideration so far..
As only usage within macros has really been supported and seen real world usage 
so far, getting scriptlet support in place and sanitizied will require a bit 
more work.
Will get around to it...

---
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/25#issuecomment-160292840___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm] initial embedded python interpreter support (@rpm5.org derived) (#25)

[ニール・ゴンパ]

So, I gave it another go, and while the scriptlets seem to work now, 
interesting issues can come up.

It causes segfaults in Yum and DNF, and I suspect anything that uses bindings 
to talk to RPM.

---
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/25#issuecomment-160121507___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm] initial embedded python interpreter support (@rpm5.org derived) (#25)

[Florian Festi]

On 11/19/2015 04:09 PM, Vít Ondruch wrote:
> What is the usecase for this? Isn't this just feature bloat?

I kinda agree that this looks like feature bloat. This patch set needs a
very good justification to go in upstream. The overall trend is to
rather keep minimal installs smaller as this is a big issue for
containers and VMs. Python is one of the candidates to be thrown out at
some point.

Also embedding a particular version of Python into rpm itself makes this
Python version part of the spec syntax with all kind of possibly ugly
implications.

It also seems to not bring many benefits except from saving the
interpreter starting time (which can be significant if there are many
Python scriptlets). But the trend here is also to rather getting rid of
more of the scriptlets using the new file trigger feature (We'll see how
this works out).

Having access to the transaction objects of the running rpm instance
could give such justification as it can hardly done in any other way.
But I really, really doubt that we wan to go down that route. Especially
as it gives a whole new way of how scriptlets can break a transaction.

As the patch set is not interfering with many places in the rpm code it
should be not too hard to maintain the changes outside of the upstream
repository for distributions with (still) rely on this feature.

Florian

-- 

Red Hat GmbH, http://www.de.redhat.com/ Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael
O'Neill, Charles Peters
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint