Re: flit

[Nick Coghlan]

On 17 November 2017 at 11:55, Toshio Kuratomi  wrote:
>
> On Thu, Nov 16, 2017 at 5:37 PM, Nick Coghlan  wrote:
> > So the two possible approaches are:
> >
> > * traditional sdist: "setup.py build", "setup.py install"
> > * Current wheel macros: "setup.py bdist_wheel", "pip install "
> >
> > If we tweak the %py_build_wheel macro to use `pip wheel` [1] rather than
> > calling `setup.py bdist_wheel` directly, then even the wheel build macro
> > would be usable without a setup.py shim (once pip itself fully supports PEP
> > 517/518)
> >
>
> I'm not sure what you're saying.  I must be further outside of the
> packaging loop than I thought.
>
> The two questions that I need to know the answer to to make sure we're
> even vaguley on the same page are:
>
> * Is the rpm Source: line still going to be an sdist?
> * Are the files that are in the built rpm going to be the same as now?

Yes, those are both unchanged.

The issue is specifically with the example in
https://fedoraproject.org/wiki/Packaging:Python#Example_common_spec_file,
which only shows the %pyX_build and %pyX_install macros, which assume
that the right way to install a Python project inside the buildroot is
to run:

"setup.py build" in the %build step
"setup.py install" in the %install step

That works as long as the project either uses setup.py natively (i.e.
distutils/setuptools projects), or provides a backwards compatibility
shim, but won't work consistently for newer projects that rely on
pyproject.toml and the static build dependency declarations from PEPs
517 & 518.

Switching to a wheel based build doesn't change either the starting
point (which is still an sdist) or the end point (which is still a
policy compliant RPM), it changes the internal interface between the
build step and the install step from being a distutils build directory
to a wheel archive:

"pip wheel" (or "setup.py bdist_wheel") in the %build step
"pip install" in the %install step

At a policy management level, I think it makes sense to separate the
"these are the policy decisions you *must* abide by" guidelines (which
are the domain of FPC) from the "Here are the helper macros that we
provide to make it easier to abide by those guidelines". The
distinction is that you can still pass a package review without using
the helper macros, but you'll still want to use them in most cases
simply because they make ongoing package maintenance easier (since the
helper macros will adjust automatically to rebases and policy changes,
while handcrafted spec files might not).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: flit

[Toshio Kuratomi]

On Thu, Nov 16, 2017 at 5:37 PM, Nick Coghlan  wrote:
> On 17 November 2017 at 01:51, Toshio Kuratomi  wrote:
>>
>> On Nov 16, 2017 4:59 AM, "Nick Coghlan"  wrote:
>
>
>>
>> Rather than emphasising the absence of setup.py, I'd emphasise the use of
>> wheel files:
>>
>>
>> * "Defining an RPM based on a wheel build process"
>> * "Defining an RPM based on a setup.py file"
>>
>>
>> I would not emphasize the use of wheel files as they are not source and
>> from flit's documentation, it doesn't appear that wheels are even central to
>> it (contrast how much wheel is mentioned in  its documentation compared to
>> pyproject.tom).  Instead I would emphasize flit itself as the build tool
>> which we're using to transform the source into the files on our systems.  If
>> there's ever an alternative to flit which builds with wheels as part of that
>> process we'll need new guidelines based on that so using wheel as the prime
>> keyword that people associate with this build process instead of flit is not
>> future proof either.
>
>
> It's not just flit - it's enscons and other PEP 517/518 backends, whereby
> the only thing we know for sure about the sdist in the long term is that
> "pip wheel" (and other PEP 517 frontends) will be able to convert it into a
> Python wheel in the %build phase, which can then be unpacked by the wheel
> installation macro in the %install phase.
>
> So the two possible approaches are:
>
> * traditional sdist: "setup.py build", "setup.py install"
> * Current wheel macros: "setup.py bdist_wheel", "pip install "
>
> If we tweak the %py_build_wheel macro to use `pip wheel` [1] rather than
> calling `setup.py bdist_wheel` directly, then even the wheel build macro
> would be usable without a setup.py shim (once pip itself fully supports PEP
> 517/518)
>
I'm not sure what you're saying.  I must be further outside of the
packaging loop than I thought.

The two questions that I need to know the answer to to make sure we're
even vaguley on the same page are:

* Is the rpm Source: line still going to be an sdist?
* Are the files that are in the built rpm going to be the same as now?

-Toshio
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: flit

[Nick Coghlan]

On 17 November 2017 at 04:50, Jason L Tibbitts III 
wrote:

> Note also that the guidelines are in the wiki for convenience, but
> there have been at least two efforts to move them to a "better" format.
> And I would truly love to have a companion set of documents which aren't
> guidelines but which expand on them and document things like this.
> Whether that ends up being maintained as part of the guidelines or
> separately hasn't even been discussed yet, but at least you'll be able
> to send a PR in pagure to get something updated.
>

OK, it sounds like putting something into
https://fedora-python.readthedocs.io/en/latest/ will actually be the best
way to go, then.

That way, the update to the guidelines page can just be a link to a page
maintained in https://github.com/fedora-python/fedora-python

Cheers,
Nick.

P.S. The latter page is mainly in GitHub because I couldn't figure out how
to get Pagure's RTD integration to work, but I also see some benefits in
lowering barriers to contribution for Python folks that already have GitHub
accounts, but don't have Fedora accounts yet.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: flit

[Jason L Tibbitts III]

> "TK" == Toshio Kuratomi  writes:

TK> However, you should check with tibbs/FPC about
TK> whether the definitions/list of macros are an altogether dated
TK> concept.

I think it's reasonable to document macros which are going to need to
use.  Python packaging just isn't sufficiently regularized (or simple)
that people can avoid having to come into more "intimate" contact with
the macros.

That documentation doesn't really have to be in the guidelines page, of
course, and we could certainly move them out to a separate document.
(We recommend but don't document %autosetup, for example.  There's
plenty of stuff you have to search for, which is unfortunate.)
Currently we have them in a list that's collapsed by default so at least
they don't take up too much space on the page.

Note also that the guidelines are in the wiki for convenience, but
there have been at least two efforts to move them to a "better" format.
And I would truly love to have a companion set of documents which aren't
guidelines but which expand on them and document things like this.
Whether that ends up being maintained as part of the guidelines or
separately hasn't even been discussed yet, but at least you'll be able
to send a PR in pagure to get something updated.

 - J<
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: flit

[Toshio Kuratomi]

On Nov 16, 2017 4:59 AM, "Nick Coghlan"  wrote:

On 16 November 2017 at 22:33, Miro Hrončok  wrote:

> Adding the link makes sense to me. Adding all the macros definition to the
> wiki does not make sense to me, but form different reasons. I think that
> having %py3_build_egg and %py3_install_egg there is just not necessary.
> Since there are more files at [0] I'd just add that link.
>
> [0] https://src.fedoraproject.org/rpms/python-rpm-macros/tree/master


Even though it's just a new informational link, I'm guessing we still need
to file an FPC ticket for that?


I think that macros we care about should be on that page, not just at that
link.  However, you should check with tibbs/FPC about whether the
definitions/list of macros are an altogether dated concept.   When I was
there we used those because spec files are just shell scripts.  For a
sysadmin to become a packager, they just needed to understand a few
concepts about the structure of the spec file but otherwise they just put
their steps from the command line into the spec.

Defining the macros on the page allowed those users to see how the macro
replaced some steps of their manual procedure.

However, that may no longer be the audience of the guidelines.  It may be
that they're now targeting programmers or packagers who don't have the same
intimate relationship to the command line as sysadmin and have a greater
comfort learning a domain specific language to do a job.

In that case, perhaps the entire concept of enumerating the macros is
unneeded for that target audience.  Instead, simply introducing the macros
when they're used in the guidelines is enough.  Talk to tibbs/FPC about
what their thoughts are.


>  * if so, should we add a new section of the guidelines? something like
>"Packaging setup.py-less projects"?
>

Rather than emphasising the absence of setup.py, I'd emphasise the use of
wheel files:

* "Defining an RPM based on a wheel build process"
* "Defining an RPM based on a setup.py file"


I would not emphasize the use of wheel files as they are not source and
from flit's documentation, it doesn't appear that wheels are even central
to it (contrast how much wheel is mentioned in  its documentation compared
to pyproject.tom).  Instead I would emphasize flit itself as the build tool
which we're using to transform the source into the files on our systems.
If there's ever an alternative to flit which builds with wheels as part of
that process we'll need new guidelines based on that so using wheel as the
prime keyword that people associate with this build process instead of flit
is not future proof either.

-Toshio
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: flit

[Charalampos Stratakis]

- Original Message - 

> From: "Nick Coghlan" 
> To: "Fedora Python SIG" 
> Sent: Thursday, November 16, 2017 1:59:24 PM
> Subject: Re: flit

> Even though it's just a new informational link, I'm guessing we still need to
> file an FPC ticket for that?

That is correct, ideally with a draft page of the proposed changes.

Regards,

Charalampos Stratakis
Software Engineer
Python Maintenance Team, Red Hat
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: flit

[Nick Coghlan]

On 16 November 2017 at 22:33, Miro Hrončok  wrote:

> Adding the link makes sense to me. Adding all the macros definition to the
> wiki does not make sense to me, but form different reasons. I think that
> having %py3_build_egg and %py3_install_egg there is just not necessary.
> Since there are more files at [0] I'd just add that link.
>
> [0] https://src.fedoraproject.org/rpms/python-rpm-macros/tree/master


Even though it's just a new informational link, I'm guessing we still need
to file an FPC ticket for that?


> About flit:
>
>  * is this way of creating Python distributions getting more and
>more popular?
>

It is - Thomas Kluyver (flit's creator) is actually driving a lot of work
to get pip's pluggable build system support working properly.

The key differences between it and setuptools are:

1. It's designed specifically for pure Python projects
2. It assumes it can get the list of files to package from git

Those two assumptions sweep away a *lot* of the complexity in setuptools,
and hence make flit far more beginner friendly.


>  * if so, should we add a new section of the guidelines? something like
>"Packaging setup.py-less projects"?
>

Rather than emphasising the absence of setup.py, I'd emphasise the use of
wheel files:

* "Defining an RPM based on a wheel build process"
* "Defining an RPM based on a setup.py file"

The latter would just be a renamed
https://fedoraproject.org/wiki/Packaging:Python#Example_common_spec_file -
that is the setup.py based build process, but it isn't currently obvious
that `pyX_build` and `pyX_install` assume the use of a setup.py file.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: flit

[Miro Hrončok]

On 16.11.2017 08:17, Nick Coghlan wrote:
On 16 November 2017 at 16:51, Elliott Sales de Andrade 
> wrote:


On 16 November 2017 at 01:31, Nick Coghlan > wrote:

However, if flit is now adding its own shim implcitly, then the
answer would just be "Yes".

There's no need for a shim; basically use flit to build a wheel and
then use the wheel installation macros instead.


Wait, we have wheel installation macros?

/me goes and checks 
https://src.fedoraproject.org/rpms/python-rpm-macros/blob/master/f/macros.python


Huh, so we do. Somehow I managed to miss that :)

How would folks feel about adding a link to the dist-git macro file to 
the macros section in 
https://fedoraproject.org/wiki/Packaging:Python#Macros ?


I'm not currently inclined to add the information directly to the wiki 
page as:


- I don't like wikis in general as a docs management tool
- the wiki only shows one version, but the RPM with the default macros 
is branched along with the rest of Fedora


Adding the link makes sense to me. Adding all the macros definition to 
the wiki does not make sense to me, but form different reasons. I think 
that having %py3_build_egg and %py3_install_egg there is just not 
necessary. Since there are more files at [0] I'd just add that link.


[0] https://src.fedoraproject.org/rpms/python-rpm-macros/tree/master


About flit:

 * is this way of creating Python distributions getting more and
   more popular?
 * if so, should we add a new section of the guidelines? something like
   "Packaging setup.py-less projects"?

We could potentially add something to 
https://fedora-python.readthedocs.io/ about Fedora's RPM packaging tools 
for Python projects, but that would be significantly more work than just 
linking to dist-git.


Cheers,
Nick.

--
Nick Coghlan   | ncogh...@gmail.com    |   
Brisbane, Australia



___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org



--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org