Jira (PUP-11681) Package provider "pip3" fails to target pip from virtual env on first run

[Josh Cooper (Jira)]

Title: Message Title

Josh Cooper commented on PUP-11681   Re: Package provider "pip3" fails to target pip from virtual env on first run Dirk Heinrichs the problem is that the prefetch command for the pip3 provider executes before all of the virtual environments are created. So you will want to create all of the environments first before managing pip3 packages like: # setup all of the virtual environments$venvs.each |$venv| {     $venv_path = "${venv_base_path}/${venv['name']}/bin"     $pip = "${venv_path}/pip3"       exec { "setup_venv_${venv['name']}":       command => "python3 -m venv --upgrade ${venv_base_path}/${venv['name']}",       path    => $facts['path'],       unless  => "test -L ${venv_path}/python3",     }   } # manage packages within each environment   $venvs.each |$venv| {     $venv_path = "${venv_base_path}/${venv['name']}/bin"     $pip = "${venv_path}/pip3"       $venv['modules'].each |$module| {       notice("module ${module['name']}")         $mod_ver = if $module['version'] {         $module['version']       }       else {         latest       }         package { "mod_${venv['name']}_${module['name']}":         name     => $module['name'],         ensure   => $mod_ver,         provider => 'pip3',         command  => $pip,         require  => Exec["setup_venv_${venv['name']}"],       }     }   } Also I think you want to specify the command as $pip = "${venv_path}/pip3" so that you don't mix the "pip" command with the "pip3" provider. I can't install cffi on my system, but if I change the hiera data to "celery", then the manifest applies successfully: --- python::venvs: - name: foo modules: - name: celery - name: bar modules: - name: celery - name: baz modules: - name: celery version: 4.4.7 Debug: /Stage[main]/Python/Package[python3-pip]/require: require to Package[python3] Debug: /Stage[main]/Python/Package[python3-venv]/require: require to Package[python3] Debug: /Stage[main]/Python/Package[mod_foo_celery]/require: require to Exec[setup_venv_foo] Debug: /Stage[main]/Python/Package[mod_bar_celery]/require: require to Exec[setup_venv_bar] Debug: /Stage[main]/Python/Package[mod_baz_celery]/require: require to Exec[setup_venv_baz] Debug: Prefetching apt resources for package Debug: Executing '/usr/bin/dpkg-query -W --showformat '${Status} ${Package} ${Version}\n'' Debug: Executing: '/usr/bin/apt-mark showmanual' Debug: Executing: '/usr/bin/apt-cache policy python3' Debug: Executing: '/usr/bin/apt-cache policy python3-pip' Debug: Executing: '/usr/bin/apt-cache policy python3-venv' Notice: /Stage[main]/Python/File[venv_base]/ensure: created Debug: /Stage[main]/Python/File[venv_base]: The container Class[Python] will propagate my refresh event Debug: Exec[setup_venv_foo](provider=posix): Executing check 'test -L /opt/venvs/foo/bin/python3' Debug: Executing: 'test -L /opt/venvs/foo/bin/python3' Debug: Exec[setup_venv_foo](provider=posix): Executing 'python3 -m venv --upgrade /opt/venvs/foo' Debug: Executing: 'python3 -m venv --upgrade /opt/venvs/foo' Notice: /Stage[main]/Python/Exec[setup_venv_foo]/returns: executed successfully Debug: /Stage[main]/Python/Exec[setup_venv_foo]: The container Class[Python] will propagate my refresh event Debug: Exec[setup_venv_bar](provider=posix): Executing check 'test -L /opt/venvs/bar/bin/python3' Debug: Executing: 'test -L /opt/venvs/bar/bin/python3' Debug: Exec[setup_venv_bar](provider=posix): Executing 'python3 -m venv --upgrade /opt/venvs/bar' Debug: Executing: 'python3 -m venv --upgrade /opt/venvs/bar' Notice: /Stage[main]/Python/Exec[setup_venv_bar]/returns: executed successfully Debug: /Stage[main]/Python/Exec[setup_venv_bar]: The container Class[Python] will propagate my refresh event Debug: Exec[setup_venv_baz](provider=posix): Executing check 'test -L /opt/venvs/baz/bin/python3' Debug: Executing: 'test -L /opt/venvs/baz/bin/python3' Debug: Exec[setup_venv_baz](provider=posix): Executing 'python3 -m venv --upgrade /opt/venvs/baz' Debug: Executing: 'python3 -m venv --upgrade /opt/venvs/baz' Notice: /Stage[main]/Python/Exec[setup_venv_baz]/returns: executed successfully Debug: /Stage[main]/Python/Exec[setup_venv_baz]: The container Class[Python] will propagate my refresh event Debug: Prefetching pip3 resources for package Debug: Executing '/opt/venvs/foo/bin/pip3 --version' Debug: Executing '/opt/venvs/foo/bin/pip3 freeze --all' Debug: Executing '/opt/venvs/bar/bin/pip3 --version' Debug: Executing '/opt/venvs/bar/bin/pip3 freeze --all' Debug: Executing '/opt/venvs/baz/bin/pip3 --version' Debug: Executing '/opt/venvs/baz/bin/pip3 freeze --all' Debug: Executing '/opt/venvs/foo/bin/pip3 --version' Debug: Executing '/opt/venvs/foo/bin/pip3 freeze --all' Debug: Executing: '/opt/venvs/foo/bin/pip3 install -q --upgrade celery' Notice: /Stage[main]/Python/Package[mod_foo_celery]/ensure: created Debug: /Package[mod_foo_celery]: The container Class[Python] will propagate my refresh event Debug: Executing '/opt/venvs/bar/bin/pip3 --version' Debug: Executing '/opt/venvs/bar/bin/pip3 freeze --all' Debug: Executing: '/opt/venvs/bar/bin/pip3 install -q --upgrade celery' Notice: /Stage[main]/Python/Package[mod_bar_celery]/ensure: created Debug: /Package[mod_bar_celery]: The container Class[Python] will propagate my refresh event Debug: Executing '/opt/venvs/baz/bin/pip3 --version' Debug: Executing '/opt/venvs/baz/bin/pip3 freeze --all' Debug: Executing: '/opt/venvs/baz/bin/pip3 install -q celery==4.4.7' Notice: /Stage[main]/Python/Package[mod_baz_celery]/ensure: created Debug: /Package[mod_baz_celery]: The container Class[Python] will propagate my refresh event   Add Comment   This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.475636.1669388388000.72393.1673478900072%40Atlassian.JIRA.