Re: [gentoo-dev] [PATCHES] python-r1.eclass: any-of dep API support

[Alex Turbov]

When it'll be possible to start to use it?

On Sat, May 20, 2017 at 8:30 PM, Michał Górny  wrote:

> Hi, everyone.
>
> Here's a set of patches inspired by the recent Sphinx dependency
> discussion. They make python-r1 (and therefore distutils-r1) capable
> of any-of dependency logic similar to the one used in python-any-r1.
>
> The basic goal is relatively simple -- to improve handling of pure
> build-time dependencies in the eclass. It solves two common problems:
>
> a. dependencies on packages that support only a subset of PYTHON_COMPAT,
>
> b. dependencies that need to be implementation-bound between themselves
>(e.g. Sphinx plugins).
>
> The new API improves both of those cases significantly. For the former,
> we no longer force user to select additional targets via REQUIRED_USE --
> instead, we just any-of dependencies + python_check_deps() to select
> implementation independently of whether it is enabled or not.
>
> For the latter, we no longer have to force all targets of the package
> on all the involved dependencies. Again, using any-of dep
> and appropriate python_check_deps() we can enforce a single (any)
> target throughout all the packages and use it.
>
> The first three patches do some code refactoring that makes the change
> easier and possibly improves maintainability of the code. The next two
> patches add support for python_check_deps() and python_gen_any_dep()
> respectively. The last two patches provide examples for both use cases
> mentioned.
>
> Please review.
>
> --
> Best regards,
> Michał Górny
>
>
>

Re: [gentoo-dev] Should Sphinx really depends on PYTHON_COMPAT/PYTHON_USEDEP for `dev-python/*` ebuilds?

[Alex Turbov]

As for me I'm doing few Python projects and as I said before I prefer to
have (real) offline docs, cuz often visit places far from "civilization"
and where 150Kib/s considered as pretty fast Internet connection. Also I'm
very patient on keeping my Gentoo system under control and minimized
(eliminating unnecessary dependencies and files). I could help with adding
patches and bug reports for packages I use.

On Thu, May 18, 2017 at 12:10 PM, Michał Górny  wrote:

> On śro, 2017-05-17 at 21:44 -0700, Daniel Campbell wrote:
> > On Sat, May 13, 2017 at 09:32:46AM +0200, Michał Górny wrote:
> > > On pią, 2017-05-12 at 17:42 -0700, Daniel Campbell wrote:
> > > > On 05/11/2017 12:51 AM, Michał Górny wrote:
> > > > > In fact, I'm personally leaning towards not building docs at all
> > > > > in ebuilds. It's practically a wasted effort since most of the time
> > > > > users read docs online anyway.
> > > >
> > > > I believe that's a little myopic; a user (or even developer) may not
> > > > have Internet access all the time, or may not have it in their
> primary
> > > > development environment. Having a copy of the docs locally (the
> entire
> > > > point of USE="doc") is super valuable to have when you're away from
> the
> > > > network. I'm sure I'm not alone as one of the people who uses the
> flag
> > > > and appreciates the work that goes into making sure said flag works.
> > > >
> > > > Sure, we could yank out every single USE="doc", but then we lose a
> nice
> > > > feature of the tree and users are back to either (a) trawling the
> Web to
> > > > find the project site, then hope they have docs in a separate
> download,
> > > > or (b) we end up with foo+1 packages, one extra for any package that
> has
> > > > documentation. Neither are particularly good solutions; Debian has
> done
> > > > the latter and it results in a huge number of packages for little
> gain.
> > >
> > > The Python team mostly focuses on providing packages for dependencies
> of
> > > other Gentoo packages, not direct Python development. We do not have
> > > the manpower to go above that.
> > >
> > > --
> > > Best regards,
> > > Michał Górny
> >
> > Ah, well that at least explains why you're not interested in it.
> > Dependency management alone can be tough; I've not noticed any Python
> > issues, so it seems like you guys do well. :) If you don't mind me
> > asking, what would it take to solve the USE="doc" issue to the Python
> > team's standard? I have some personal interest in Python and wouldn't
> > mind adding 'doc' support for Python packages that users request docs
> > for.
> >
> > Maybe others are willing to join me on this. Is that something we can
> > make happen without getting in anyone's hair?
> >
>
> For a start, it'd be nice to figure all the stuff out in detail,
> and document it -- when USEDEP is needed, not needed, when we need
> something else (like the plugin case). Once that is done, it's just
> a matter of checking and fixing existing packages, and being patient
> with devs doing the same mistakes again ;-).
>
> --
> Best regards,
> Michał Górny
>

Re: [gentoo-dev] Should Sphinx really depends on PYTHON_COMPAT/PYTHON_USEDEP for `dev-python/*` ebuilds?

[Alex Turbov]

On Thu, May 11, 2017 at 2:51 PM, Michał Górny  wrote:

> Hi,
>
> Few janitorial notes for a start:
>
> 1. please fix your line wrapping since your messages are wrapped twice
> now, and it's really hard to read with single words on every second
> line;
>

sorry, I don't understand what are you talking about... probably some
problem with your email client (or whatever you use).
I'm using gmail's web UI and see no double wraps...


>
> 2. hardcore Python topics belong on gentoo-python@ but I guess we'll
> continue here,
>

I don't see this ML here: https://gentoo.org/get-involved/mailing-lists/ so
I decided to use `gentoo-dev`


>
> 3. please keep your messages brief. The first three paragraphs tell
> a thing that could be told in one sentence.
>

I've got no idea what message format is "usual" in this ML... from my
experience talking to various "tech support" and bug trackers ppl usually
asking a lot of stupid questions if I wrote just "one sentence"...


>
> You can't use python_targets directly since it will break when the old
> implementations are disabled (and also make it PITA for others to add
> new impls).
>

Ok, what I can use instead?


>
>
> Long story short, it's not worth the effort.
>
> Yes, most of the time people specify PYTHON_USEDEP on sphinx needlessly.
>  There are two other major cases when you need it though:
>
> 1. things like autointerface that interface with packages' code,
>

what are you talking about? (
https://pypi.python.org/pypi/repoze.sphinx.autointerface/ ??)


>
> 2. and packages calling sphinx via 'python /usr/bin/sphinx ...' (i.e.
> requiring impl match between python in use and sphinx).
>

do you mean they are doing it from ebuild?


>
> In fact, I'm personally leaning towards not building docs at all
> in ebuilds. It's practically a wasted effort since most of the time
> users read docs online anyway.
>

unfortunately I'm travelling a lot and really often in places where
Internet connection is far from good. it is why I like to have offline docs
for some packages. moreover I really hate when some docs are not really
offline and want to load google fonts or JS :(


>
> However, tracking the other uses down and figuring them is not worth
> the effort. In the end, someone will probably add it back thinking
> someone must've missed it. It's too hard to get it right.
>

I didn't get what are you talking about...


> Building Sphinx with less implementations than its reverse dependencies
> is a corner case. It's not really worth spending hours making sure
> depends are 100% strictly correct. The more important goal is to have
> things working reliably, and overspecified deps are reliable, i.e.
> packages won't fail to build because of them.
>
>
Ok, seems I've got your point of view, but can't agree w/ it... Well, I
would fight alone w/ it


> --
> Best regards,
> Michał Górny
>

[gentoo-dev] Should Sphinx really depends on PYTHON_COMPAT/PYTHON_USEDEP for `dev-python/*` ebuilds?

[Alex Turbov]

Hi Gentoo devs,

I'd like to discuss one thing I've concernted for a long time already.
Recently I've done some `dev-python` ebuild. And as most Python packages
it also uses `dev-python/sphinx` to produce documentation. I want to note
that my primary (and default) Python version is 3.5 and I have some 2.7
packages
(ebuilds) only because they required by something else (really important to
me).
or when it is dependency of some other Python2-only package.

Unfortunately most of `dev-python/*` packages, to produce docs from RST
files,
have `DEPEND` on `dev-python/sphinx[${PYTHON_USEDEP}]`! However, being a
standalone application (just processing a bunch of RST files to HTMLs),
`/usr/bin/sphinx-build` **do not** actually depeds on `PYTHON_COMPAT` of
the ebuild it used inside! It just needs some/any interpreter it (Sphinx)
was "build" with!

So, nowadays Gentoo has some reeally weired IMO behaviour: to build docs for
some package from `dev-python/*` with `PYTHON_TARGETS` set to (for example)
2.7 and
3.5 Python versions, I have to build Sphinx also **for all enabled Python
versions**
of that package, meanwhile **only one** is needed (and really used)! This
is my
real concern: why I have to waste my disk+time to build `dev-python/*`
packages
with `PYTHON_TARGETS` I'll never use? Some of that packages are actually
dependencies
of other Python2-only packages. In other words, to build docs using Python
**3**
version/build of Sphinx, which is work pretty fine already, for Python2
enabled package,
I have to install a lot of Python-2 packages, dependencies of Sphinx to be
built w/ Python 2,
and enable `python2_7` for packages I've already have as Python3-only,
which in turn could
bring even more strange dependencies into scope.

Thus generally specking, Sphinx dependencies have no relations to `DEPEND`
of particular
`dev-python/*` ebuilds! So, in simple case there is should be enough to
specify

DEPEND=( doc? ( dev-python/sphinx ) )

for that ebuilds. In some rare cases (like
https://bugs.gentoo.org/show_bug.cgi?id=618162)
Sphinx could use some extensions (plugins) and they also have no any
relation to `PYTHON_COMPAT`
of particular `dev-python/*` ebuild! That plugins to work need just the
same `PYTHON_TARGETS`
as used to build Sphinx. Unfortunately I can't find appropriate helper
function(s) in any
currently present Python reelated eclasses (or am I miss smth?), so I used
the following
dependency spec:

DEPEND=( doc?
|| (
(
dev-python/sphinx[python_targets_python2_7]
# NOTE This packages provide extensions for Sphinx
dev-python/rst-linker[python_targets_python2_7]
dev-python/jaraco-packaging[python_targets_python2_7]
)
(
dev-python/sphinx[python_targets_python3_5]
dev-python/rst-linker[python_targets_python3_5]
dev-python/jaraco-packaging[python_targets_python3_5]
)
(
dev-python/sphinx[python_targets_python3_6]
dev-python/rst-linker[python_targets_python3_6]
dev-python/jaraco-packaging[python_targets_python3_6]
)
)
  )

So, my questions are:

0. am I missed smth? (and there are some other cases, I don't know about)
1. am I missed smth? (and there are some helper functions exist in eclasses
to expess that kind
   of dependencies)
2. I think it would be nice to have some support for Sphinx in eclasses to
simplify ebuilds writing
   (if #1 is false)

Ideas/comments/opinions are really welcome...