Re: [vpp-dev] [csit-dev] "vpp-make-test-docs-verify-master" job failure

2017-08-08 Thread Luke, Chris 
I'm not making any request; I'm merely positing ways forward to avoid the below 
issue, assuming this is in fact what the issue is.

Chris.

> -Original Message-
> From: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> [mailto:ksek...@cisco.com]
> Sent: Tuesday, August 08, 2017 11:47 AM
> To: Dave Barach (dbarach) ; Luke, Chris
> ; csit-...@lists.fd.io
> Cc: vpp-dev@lists.fd.io
> Subject: RE: [csit-dev] [vpp-dev] "vpp-make-test-docs-verify-master" job
> failure
> 
> Are you suggesting we put all the dependent python packages as part of the
> vpp source code tree then?
> Or is this request aimed at the test box maintainers?
> 
> Thanks,
> Klement
> 
> Quoting Luke, Chris (2017-08-08 17:04:57)
> > Sure, I know what pip is.
> >
> > My contention is that virtualenv should not be going to the network; in the
> old days it would dip into the system packages for anything the base venv
> needed. It's a really poor and insecure assumption that unfettered internet
> access is universally available. In this case, I suspect wherever the job runs
> needs an HTTP proxy to fit that assumption.
> >
> > So either pre-stage the packages you need, and tell pip where to find
> > them, or provide a proxy. :)
> >
> > Chris.
> >
> >
> > > -Original Message-
> > > From: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> > > [mailto:ksek...@cisco.com]
> > > Sent: Tuesday, August 08, 2017 10:51 AM
> > > To: Dave Barach (dbarach) ; Luke, Chris
> > > ; csit-...@lists.fd.io
> > > Cc: vpp-dev@lists.fd.io
> > > Subject: RE: [csit-dev] [vpp-dev] "vpp-make-test-docs-verify-master"
> > > job failure
> > >
> > > Think of pip (used by virtualenv) like apt-get.  Python maintainers
> > > are innocent in this case. When adding the virtualenv feature to
> > > test framework, it never occured to me that somebody might be doing
> > > vpp development on an offline machine. Laptop which was online and
> > > later went offline doesn't count, because virtualenv uses cached
> > > versions for subsequent installations afaik.
> > >
> > > If this is a requirement, we'll need to think of a way to ship the
> > > python packages which virtualenv requires - e.g. setuptools - ourselves.
> > > But if we do that, we also need to have all the other packages which
> > > the test framework uses (e.g. scapy) somewhere in-tree(?).
> > >
> > > The packages currently used by test framework (apart from the
> > > implicit
> > > requirements) are:
> > >
> > > scapy==2.3.3 pexpect subprocess32 cffi
> > > git+https://github.com/klement/py-lispnetworking@setup
> > >
> > > the last one is a fork of py-lispnetworking, which is used by some
> > > LISP tests. I forked it and added an installation script, which was
> > > missing in the original repo so that we can install it in the same way as
> other packages.
> > >
> > > I have a strong feeling that this isn't something which we want to do...
> > >
> > > Klement
> > >
> > > Quoting Luke, Chris (2017-08-08 16:31:31)
> > > > I've seen precisely this issue with Python's virtualenv whenever
> > > > the host
> > > doesn't have internet access while the venv is being created; I work
> > > around it using an HTTP proxy, though requiring internet access just
> > > to create a venv seems like a particularly braindead decision by the
> Python maintainers.
> > > >
> > > > Chris.
> > > >
> > > > > -Original Message-
> > > > > From: csit-dev-boun...@lists.fd.io
> > > > > [mailto:csit-dev-boun...@lists.fd.io] On Behalf Of Klement
> > > > > Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> > > > > Sent: Tuesday, August 08, 2017 8:23 AM
> > > > > To: Dave Barach (dbarach) ;
> > > > > csit-...@lists.fd.io
> > > > > Cc: vpp-dev@lists.fd.io
> > > > > Subject: Re: [csit-dev] [vpp-dev] "vpp-make-test-docs-verify-master"
> > > > > job failure
> > > > >
> > > > > Looks to me like a bug in the urllib3 python library while
> > > > > reporting some kind of connection failure.. never seen it before..
> > > > >
> > > > > Klement
> > > > >
> > > > > Quoting Dave Barach (dbarach) (2017-08-08 14:05:16)
> > > > > >Please see [1]https://gerrit.fd.io/r/#/c/7885,
> > > > > >[2]https://jenkins.fd.io/job/vpp-make-test-docs-verify-
> > > > > master/2814/console.
> > > > > >
> > > > > >
> > > > > >
> > > > > >Any idea what this is all about?
> > > > > >
> > > > > >
> > > > > >
> > > > > >Thanks… Dave
> > > > > >
> > > > > >
> > > > > >
> > > > > > References
> > > > > >
> > > > > >Visible links
> > > > > >1. https://gerrit.fd.io/r/#/c/7885
> > > > > >2.
> > > > > > https://jenkins.fd.io/job/vpp-make-test-docs-verify-master/281
> > > > > > 4/co
> > > > > > nsol
> > > > > > e
> > > > > ___
> > > > > csit-dev mailing list
> > > > > csit-...@lists.fd.io
> > > > > https://lists.fd.io/mailman/listinfo/csit-dev

Re: [vpp-dev] [csit-dev] "vpp-make-test-docs-verify-master" job failure

2017-08-08 Thread Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco) 
Are you suggesting we put all the dependent python packages as part of
the vpp source code tree then?
Or is this request aimed at the test box maintainers?

Thanks,
Klement

Quoting Luke, Chris (2017-08-08 17:04:57)
> Sure, I know what pip is.
> 
> My contention is that virtualenv should not be going to the network; in the 
> old days it would dip into the system packages for anything the base venv 
> needed. It's a really poor and insecure assumption that unfettered internet 
> access is universally available. In this case, I suspect wherever the job 
> runs needs an HTTP proxy to fit that assumption.
> 
> So either pre-stage the packages you need, and tell pip where to find them, 
> or provide a proxy. :)
> 
> Chris.
> 
> 
> > -Original Message-
> > From: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> > [mailto:ksek...@cisco.com]
> > Sent: Tuesday, August 08, 2017 10:51 AM
> > To: Dave Barach (dbarach) ; Luke, Chris
> > ; csit-...@lists.fd.io
> > Cc: vpp-dev@lists.fd.io
> > Subject: RE: [csit-dev] [vpp-dev] "vpp-make-test-docs-verify-master" job
> > failure
> > 
> > Think of pip (used by virtualenv) like apt-get.  Python maintainers are
> > innocent in this case. When adding the virtualenv feature to test framework,
> > it never occured to me that somebody might be doing vpp development on
> > an offline machine. Laptop which was online and later went offline doesn't
> > count, because virtualenv uses cached versions for subsequent installations
> > afaik.
> > 
> > If this is a requirement, we'll need to think of a way to ship the python
> > packages which virtualenv requires - e.g. setuptools - ourselves.
> > But if we do that, we also need to have all the other packages which the 
> > test
> > framework uses (e.g. scapy) somewhere in-tree(?).
> > 
> > The packages currently used by test framework (apart from the implicit
> > requirements) are:
> > 
> > scapy==2.3.3 pexpect subprocess32 cffi
> > git+https://github.com/klement/py-lispnetworking@setup
> > 
> > the last one is a fork of py-lispnetworking, which is used by some LISP 
> > tests. I
> > forked it and added an installation script, which was missing in the 
> > original
> > repo so that we can install it in the same way as other packages.
> > 
> > I have a strong feeling that this isn't something which we want to do...
> > 
> > Klement
> > 
> > Quoting Luke, Chris (2017-08-08 16:31:31)
> > > I've seen precisely this issue with Python's virtualenv whenever the host
> > doesn't have internet access while the venv is being created; I work around 
> > it
> > using an HTTP proxy, though requiring internet access just to create a venv
> > seems like a particularly braindead decision by the Python maintainers.
> > >
> > > Chris.
> > >
> > > > -Original Message-
> > > > From: csit-dev-boun...@lists.fd.io
> > > > [mailto:csit-dev-boun...@lists.fd.io] On Behalf Of Klement Sekera -X
> > > > (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> > > > Sent: Tuesday, August 08, 2017 8:23 AM
> > > > To: Dave Barach (dbarach) ; csit-...@lists.fd.io
> > > > Cc: vpp-dev@lists.fd.io
> > > > Subject: Re: [csit-dev] [vpp-dev] "vpp-make-test-docs-verify-master"
> > > > job failure
> > > >
> > > > Looks to me like a bug in the urllib3 python library while reporting
> > > > some kind of connection failure.. never seen it before..
> > > >
> > > > Klement
> > > >
> > > > Quoting Dave Barach (dbarach) (2017-08-08 14:05:16)
> > > > >Please see [1]https://gerrit.fd.io/r/#/c/7885,
> > > > >[2]https://jenkins.fd.io/job/vpp-make-test-docs-verify-
> > > > master/2814/console.
> > > > >
> > > > >
> > > > >
> > > > >Any idea what this is all about?
> > > > >
> > > > >
> > > > >
> > > > >Thanks… Dave
> > > > >
> > > > >
> > > > >
> > > > > References
> > > > >
> > > > >Visible links
> > > > >1. https://gerrit.fd.io/r/#/c/7885
> > > > >2.
> > > > > https://jenkins.fd.io/job/vpp-make-test-docs-verify-master/2814/co
> > > > > nsol
> > > > > e
> > > > ___
> > > > csit-dev mailing list
> > > > csit-...@lists.fd.io
> > > > https://lists.fd.io/mailman/listinfo/csit-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [csit-dev] "vpp-make-test-docs-verify-master" job failure

2017-08-08 Thread Luke, Chris 
Sure, I know what pip is.

My contention is that virtualenv should not be going to the network; in the old 
days it would dip into the system packages for anything the base venv needed. 
It's a really poor and insecure assumption that unfettered internet access is 
universally available. In this case, I suspect wherever the job runs needs an 
HTTP proxy to fit that assumption.

So either pre-stage the packages you need, and tell pip where to find them, or 
provide a proxy. :)

Chris.


> -Original Message-
> From: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> [mailto:ksek...@cisco.com]
> Sent: Tuesday, August 08, 2017 10:51 AM
> To: Dave Barach (dbarach) ; Luke, Chris
> ; csit-...@lists.fd.io
> Cc: vpp-dev@lists.fd.io
> Subject: RE: [csit-dev] [vpp-dev] "vpp-make-test-docs-verify-master" job
> failure
> 
> Think of pip (used by virtualenv) like apt-get.  Python maintainers are
> innocent in this case. When adding the virtualenv feature to test framework,
> it never occured to me that somebody might be doing vpp development on
> an offline machine. Laptop which was online and later went offline doesn't
> count, because virtualenv uses cached versions for subsequent installations
> afaik.
> 
> If this is a requirement, we'll need to think of a way to ship the python
> packages which virtualenv requires - e.g. setuptools - ourselves.
> But if we do that, we also need to have all the other packages which the test
> framework uses (e.g. scapy) somewhere in-tree(?).
> 
> The packages currently used by test framework (apart from the implicit
> requirements) are:
> 
> scapy==2.3.3 pexpect subprocess32 cffi
> git+https://github.com/klement/py-lispnetworking@setup
> 
> the last one is a fork of py-lispnetworking, which is used by some LISP 
> tests. I
> forked it and added an installation script, which was missing in the original
> repo so that we can install it in the same way as other packages.
> 
> I have a strong feeling that this isn't something which we want to do...
> 
> Klement
> 
> Quoting Luke, Chris (2017-08-08 16:31:31)
> > I've seen precisely this issue with Python's virtualenv whenever the host
> doesn't have internet access while the venv is being created; I work around it
> using an HTTP proxy, though requiring internet access just to create a venv
> seems like a particularly braindead decision by the Python maintainers.
> >
> > Chris.
> >
> > > -Original Message-
> > > From: csit-dev-boun...@lists.fd.io
> > > [mailto:csit-dev-boun...@lists.fd.io] On Behalf Of Klement Sekera -X
> > > (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> > > Sent: Tuesday, August 08, 2017 8:23 AM
> > > To: Dave Barach (dbarach) ; csit-...@lists.fd.io
> > > Cc: vpp-dev@lists.fd.io
> > > Subject: Re: [csit-dev] [vpp-dev] "vpp-make-test-docs-verify-master"
> > > job failure
> > >
> > > Looks to me like a bug in the urllib3 python library while reporting
> > > some kind of connection failure.. never seen it before..
> > >
> > > Klement
> > >
> > > Quoting Dave Barach (dbarach) (2017-08-08 14:05:16)
> > > >Please see [1]https://gerrit.fd.io/r/#/c/7885,
> > > >[2]https://jenkins.fd.io/job/vpp-make-test-docs-verify-
> > > master/2814/console.
> > > >
> > > >
> > > >
> > > >Any idea what this is all about?
> > > >
> > > >
> > > >
> > > >Thanks… Dave
> > > >
> > > >
> > > >
> > > > References
> > > >
> > > >Visible links
> > > >1. https://gerrit.fd.io/r/#/c/7885
> > > >2.
> > > > https://jenkins.fd.io/job/vpp-make-test-docs-verify-master/2814/co
> > > > nsol
> > > > e
> > > ___
> > > csit-dev mailing list
> > > csit-...@lists.fd.io
> > > https://lists.fd.io/mailman/listinfo/csit-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [csit-dev] "vpp-make-test-docs-verify-master" job failure

2017-08-08 Thread Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco) 
Think of pip (used by virtualenv) like apt-get.  Python maintainers
are innocent in this case. When adding the virtualenv feature to test
framework, it never occured to me that somebody might be doing vpp development
on an offline machine. Laptop which was online and later went offline doesn't
count, because virtualenv uses cached versions for subsequent installations
afaik.

If this is a requirement, we'll need to think of a way to ship the python
packages which virtualenv requires - e.g. setuptools - ourselves.
But if we do that, we also need to have all the other packages which the
test framework uses (e.g. scapy) somewhere in-tree(?).

The packages currently used by test framework (apart from the implicit
requirements) are:

scapy==2.3.3 pexpect subprocess32 cffi
git+https://github.com/klement/py-lispnetworking@setup

the last one is a fork of py-lispnetworking, which is used by some LISP
tests. I forked it and added an installation script, which was missing
in the original repo so that we can install it in the same way as other
packages.

I have a strong feeling that this isn't something which we want to do...

Klement

Quoting Luke, Chris (2017-08-08 16:31:31)
> I've seen precisely this issue with Python's virtualenv whenever the host 
> doesn't have internet access while the venv is being created; I work around 
> it using an HTTP proxy, though requiring internet access just to create a 
> venv seems like a particularly braindead decision by the Python maintainers.
> 
> Chris.
> 
> > -Original Message-
> > From: csit-dev-boun...@lists.fd.io [mailto:csit-dev-boun...@lists.fd.io] On
> > Behalf Of Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> > Sent: Tuesday, August 08, 2017 8:23 AM
> > To: Dave Barach (dbarach) ; csit-...@lists.fd.io
> > Cc: vpp-dev@lists.fd.io
> > Subject: Re: [csit-dev] [vpp-dev] "vpp-make-test-docs-verify-master" job
> > failure
> > 
> > Looks to me like a bug in the urllib3 python library while reporting some 
> > kind
> > of connection failure.. never seen it before..
> > 
> > Klement
> > 
> > Quoting Dave Barach (dbarach) (2017-08-08 14:05:16)
> > >Please see [1]https://gerrit.fd.io/r/#/c/7885,
> > >[2]https://jenkins.fd.io/job/vpp-make-test-docs-verify-
> > master/2814/console.
> > >
> > >
> > >
> > >Any idea what this is all about?
> > >
> > >
> > >
> > >Thanks… Dave
> > >
> > >
> > >
> > > References
> > >
> > >Visible links
> > >1. https://gerrit.fd.io/r/#/c/7885
> > >2.
> > > https://jenkins.fd.io/job/vpp-make-test-docs-verify-master/2814/consol
> > > e
> > ___
> > csit-dev mailing list
> > csit-...@lists.fd.io
> > https://lists.fd.io/mailman/listinfo/csit-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [csit-dev] "vpp-make-test-docs-verify-master" job failure

2017-08-08 Thread Luke, Chris 
I've seen precisely this issue with Python's virtualenv whenever the host 
doesn't have internet access while the venv is being created; I work around it 
using an HTTP proxy, though requiring internet access just to create a venv 
seems like a particularly braindead decision by the Python maintainers.

Chris.

> -Original Message-
> From: csit-dev-boun...@lists.fd.io [mailto:csit-dev-boun...@lists.fd.io] On
> Behalf Of Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> Sent: Tuesday, August 08, 2017 8:23 AM
> To: Dave Barach (dbarach) ; csit-...@lists.fd.io
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [csit-dev] [vpp-dev] "vpp-make-test-docs-verify-master" job
> failure
> 
> Looks to me like a bug in the urllib3 python library while reporting some kind
> of connection failure.. never seen it before..
> 
> Klement
> 
> Quoting Dave Barach (dbarach) (2017-08-08 14:05:16)
> >Please see [1]https://gerrit.fd.io/r/#/c/7885,
> >[2]https://jenkins.fd.io/job/vpp-make-test-docs-verify-
> master/2814/console.
> >
> >
> >
> >Any idea what this is all about?
> >
> >
> >
> >Thanks… Dave
> >
> >
> >
> > References
> >
> >Visible links
> >1. https://gerrit.fd.io/r/#/c/7885
> >2.
> > https://jenkins.fd.io/job/vpp-make-test-docs-verify-master/2814/consol
> > e
> ___
> csit-dev mailing list
> csit-...@lists.fd.io
> https://lists.fd.io/mailman/listinfo/csit-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev