Re: [Distutils] pipenv best practices?

[Nick Coghlan]

On 16 January 2018 at 19:47, Paul Moore  wrote:
> I think that if the pipenv docs had some better guidance on what use
> cases it was intended to cover (and what it wasn't, in relation to the
> broader range of pip+virtualenv use cases) that would help people
> better understand its place in the ecosystem.

That's fair, and making the PyPUG tutorial specifically about
*application* dependency management was an initial step in that
direction - for the library development use case, you're generally
going to step out of pipenv's world as soon as you try to run your
tests across multiple versions.

The basic usage constraint is that pipenv expects you to control your
target Python version, and it expects you to have exactly one - to go
beyond that (as is needed for multi-version library support), you'll
still need to drop down to the lower level tooling, either skipping
the use of pipenv entirely, or else by using `pipfile lock
--requirements` to shift levels.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pipenv best practices?

[Paul Moore]

On 16 January 2018 at 08:08, Chris Withers  wrote:
>> That doesn't fall under pipenv's purview. Think of pipenv as trying to
>> make venv + pip easier to work with. Since you don't use pip to express
>> dependencies for your library then you shouldn't with pipenv either. Or put
>> another way, think of your pipfile as just a different format for a
>> requirements.txt file, not a replacement for flit or setuptools.
>
>
> Well, kinda, pipenv is shaping up to be the "replacement" for
> pip+virtualenv, where the latter becomes just an implementation detail.
> Would be great if it had a good story for that use case :-)

I think the big issue is that it's not clear what use cases pipenv is
*trying* to address. Your description of it being a "replacement" for
pip+virtualenv is more or less what I originally assumed it was
intended to be. But I tripped over this a while back, when I found it
very difficult to use pipenv on a project where I needed to:

1. Reference an unreleased version of a project from github
2. Have one of its dependencies be satisfied from a locally (within
the project directory) stored copy of a wheel, as there was no
suitable wheel on PyPI.

I can't recall the details - they are in various issues on the pipenv
tracker - but it was clear that I was trying to do something that
wasn't considered "expected use" of pipenv, and I was getting very
frustrated by the workarounds I needed to use to support my
requirements. The guys on the pipenv tracker were very helpful, but
when you're hitting design limitations, you can't expect quick fixes,
so I ended up going back to pew + requrements.txt.

I think that if the pipenv docs had some better guidance on what use
cases it was intended to cover (and what it wasn't, in relation to the
broader range of pip+virtualenv use cases) that would help people
better understand its place in the ecosystem.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pipenv best practices?

[Chris Withers]

On 16/01/2018 04:24, Nick Coghlan wrote:

Anyway, we genuinely don't have a clear answer to this question, so
I've posted a meta-issue on the pipenv tracker to decide how we want
to handle it: https://github.com/pypa/pipenv/issues/1305


Thanks!


Continuing on the deployment theme: I'm used to being able to put
/path/to/some/ve/bin/some_script in crontabs and the like, where
some_script comes form a setuptools entry point.
What's the canonical way of doing this with pipenv?



"pipenv --venv" gives the path for the current project, so what I tend
to do is run "ln -s $(pipenv --venv) .venv" from the directory
containing Pipfile and Pipfile.lock in order to create a deterministic
path name.


Hmm, that feels hacky. It doesn't feel like there's consensus on this 
issue, and there's enough dupes coming in that it feels a shame that 
https://github.com/pypa/pipenv/issues/1049 has been closed.


I actually want both use cases in different scenarios. For dev, I 
typically want one venv per python version (how do I do that?) somewhere 
central. For deployment, I want a ./.venv or ./ve right next to the Pipfile.



Last up, how should pipenv be used for library development? (ie: where
dependencies and minimal constraints are expressed in setup.py and where
you want to test against as many python versions and library
combinations as you support).



pipenv does cover this case to some degree - the trick is that it
involves thinking of your library's *test suite* as the application
you want to deploy. (Hence the reference to "development and testing
environments" at the end of the intro to
https://packaging.python.org/tutorials/managing-dependencies/)


As an aside: I find the doc split difficult. Even as a relatively 
experience python dev, knowing that I wanted to figure out what pipenv 
does and how to use it, I started at 
http://pipenv.readthedocs.io/en/latest/ which has no links to the 
tutorial(s).



For that case, you don't put *any* of your dependencies from
`setup.py` into `Pipfile`, you just run "pipenv install -e src", which
will give you a Pipfile entry like:

"" = {editable = true, path = "src"}

I personally then edit that entry to replace the commit hash with the
actual project name


I'm afraid I don't know enough about pipenv internals to understand the 
implications of the above, or why you might want to change the commit 
has to the project name, and indeed, what the implications of that might be.



All the testing and linting dependencies then go under [dev-packages].


Where can I find out more about [dev-packages]?


The part you can't fully rely on yet is the lock file, since that aims
to lock down the version of Python as well, not just the versions of
the Python level dependencies. So if you're testing across multiple
versions (e.g. with tox),


...or a .travis.yml axis for CI, or just separate venvs for local dev, 
in my case...



the best current approach is to use "pipenv
lock --requirements" to export just the library dependencies from
Pipfile.lock. 


...but what do I do with Pipfile.lock then? .gitigore?

cheers,

Chris
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pipenv best practices?

[Chris Withers]

On 15/01/2018 19:46, Brett Cannon wrote:


But I happen to know the answer to your pipenv questions, Chris, so I'll 
answer them here.


Continuing here then :-)


So, with pipenv, what files do we version control for a project? both
Pipfile and Pipfile.lock?


Yes.


great, thanks! How does Pipfile.lock work in the context of a project 
which may be installed on multiple operating systems with different 
final package requirements?



Hopefully one I missed from the docs: with the correct files source
controlled, how do I reproduce the environment on another machine?

pipenv install --ignore-pipfile


That's a surprising spelling. Why does Pipfile need to be ignored? 
Surely Pipfile.lock will be consulted if present and used for the 
explicit requirements?



Last up, how should pipenv be used for library development? (ie: where
dependencies and minimal constraints are expressed in setup.py and where
you want to test against as many python versions and library
combinations as you support).


That doesn't fall under pipenv's purview. Think of pipenv as trying to 
make venv + pip easier to work with. Since you don't use pip to express 
dependencies for your library then you shouldn't with pipenv either. Or 
put another way, think of your pipfile as just a different format for a 
requirements.txt file, not a replacement for flit or setuptools.


Well, kinda, pipenv is shaping up to be the "replacement" for 
pip+virtualenv, where the latter becomes just an implementation detail. 
Would be great if it had a good story for that use case :-)


cheers,

Chris
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pipenv best practices?

[Nick Coghlan]

On 16 January 2018 at 14:24, Nick Coghlan  wrote:
> On 16 January 2018 at 05:46, Brett Cannon  wrote:
>> On Mon, 15 Jan 2018 at 00:33 Chris Withers  wrote:
>>> Continuing on the deployment theme: I'm used to being able to put
>>> /path/to/some/ve/bin/some_script in crontabs and the like, where
>>> some_script comes form a setuptools entry point.
>>> What's the canonical way of doing this with pipenv?
>>
>> Beats me. I think there's a command to get back to the venv that pipenv
>> created on your behalf.
>
> "pipenv --venv" gives the path for the current project, so what I tend
> to do is run "ls -s $(pipenv --venv) .venv" from the directory
> containing Pipfile and Pipfile.lock in order to create a deterministic
> path name.

Oops, that should have been "ln -s $(pipenv --venv) .venv".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pipenv best practices?

[Nick Coghlan]

On 16 January 2018 at 05:46, Brett Cannon  wrote:
> On Mon, 15 Jan 2018 at 00:33 Chris Withers  wrote:
>>
>> Hi All,
>>
>> Couldn't find an obviously better discussion forum than this for pipenv
>> stuff, but please point me to where the rest of the discussions are
>> happening...
>
>
> Not here from what I can tell. :) Probably your best bet is to ask on the
> pipenv GitHub project and file an issue so the pipenv docs can give you
> guidance. I know for me personally it's getting a bit hazy between this list
> and pypa-dev where stuff should go (I'm personally starting to shift towards
> pypa-dev and considering this actually for distutils proper).

Personally, I don't think mailing lists in general are a great medium
for asking usage and support questions.

Anyway, we genuinely don't have a clear answer to this question, so
I've posted a meta-issue on the pipenv tracker to decide how we want
to handle it: https://github.com/pypa/pipenv/issues/1305

>> Continuing on the deployment theme: I'm used to being able to put
>> /path/to/some/ve/bin/some_script in crontabs and the like, where
>> some_script comes form a setuptools entry point.
>> What's the canonical way of doing this with pipenv?
>
> Beats me. I think there's a command to get back to the venv that pipenv
> created on your behalf.

"pipenv --venv" gives the path for the current project, so what I tend
to do is run "ls -s $(pipenv --venv) .venv" from the directory
containing Pipfile and Pipfile.lock in order to create a deterministic
path name.

>> Last up, how should pipenv be used for library development? (ie: where
>> dependencies and minimal constraints are expressed in setup.py and where
>> you want to test against as many python versions and library
>> combinations as you support).
>
> That doesn't fall under pipenv's purview. Think of pipenv as trying to make
> venv + pip easier to work with. Since you don't use pip to express
> dependencies for your library then you shouldn't with pipenv either. Or put
> another way, think of your pipfile as just a different format for a
> requirements.txt file, not a replacement for flit or setuptools.

pipenv does cover this case to some degree - the trick is that it
involves thinking of your library's *test suite* as the application
you want to deploy. (Hence the reference to "development and testing
environments" at the end of the intro to
https://packaging.python.org/tutorials/managing-dependencies/)

For that case, you don't put *any* of your dependencies from
`setup.py` into `Pipfile`, you just run "pipenv install -e src", which
will give you a Pipfile entry like:

   "" = {editable = true, path = "src"}

I personally then edit that entry to replace the commit hash with the
actual project name

All the testing and linting dependencies then go under [dev-packages].

The part you can't fully rely on yet is the lock file, since that aims
to lock down the version of Python as well, not just the versions of
the Python level dependencies. So if you're testing across multiple
versions (e.g. with tox), the best current approach is to use "pipenv
lock --requirements" to export just the library dependencies from
Pipfile.lock. That's an area that could definitely use some UX
improvement, but it isn't clear to me at this point what that would
actually look like (short of adding a notion of named "install
profiles" to the lock file format, which could then be aligned with
tox test environments).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pipenv best practices?

[Brett Cannon]

On Mon, 15 Jan 2018 at 00:33 Chris Withers  wrote:

> Hi All,
>
> Couldn't find an obviously better discussion forum than this for pipenv
> stuff, but please point me to where the rest of the discussions are
> happening...
>

Not here from what I can tell. :) Probably your best bet is to ask on the
pipenv GitHub project and file an issue so the pipenv docs can give you
guidance. I know for me personally it's getting a bit hazy between this
list and pypa-dev where stuff should go (I'm personally starting to shift
towards pypa-dev and considering this actually for distutils proper).

But I happen to know the answer to your pipenv questions, Chris, so I'll
answer them here.


>
> So, with pipenv, what files do we version control for a project? both
> Pipfile and Pipfile.lock?
>

Yes.


> Hopefully one I missed from the docs: with the correct files source
> controlled, how do I reproduce the environment on another machine?
>

pipenv install --ignore-pipfile



>
> Continuing on the deployment theme: I'm used to being able to put
> /path/to/some/ve/bin/some_script in crontabs and the like, where
> some_script comes form a setuptools entry point.
> What's the canonical way of doing this with pipenv?
>

Beats me. I think there's a command to get back to the venv that pipenv
created on your behalf.


>
> Last up, how should pipenv be used for library development? (ie: where
> dependencies and minimal constraints are expressed in setup.py and where
> you want to test against as many python versions and library
> combinations as you support).
>

That doesn't fall under pipenv's purview. Think of pipenv as trying to make
venv + pip easier to work with. Since you don't use pip to express
dependencies for your library then you shouldn't with pipenv either. Or put
another way, think of your pipfile as just a different format for a
requirements.txt file, not a replacement for flit or setuptools.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig