Hi Iwase-San, On Tue, 8 Nov 2016 14:39:45 +0900 Iwase Yusuke <[email protected]> wrote:
> Hi akuma-San, > > > On 2016年11月08日 12:58, fumihiko kakuma wrote: > > Hi Iwase-san, > > > > On Tue, 8 Nov 2016 09:51:18 +0900 > > Iwase Yusuke <[email protected]> wrote: > > > >> Hi Kakuma-San, > >> > >> Thank you for your reply! > >> > >> > >> On 2016年11月07日 21:50, fumihiko kakuma wrote: > >>> Hi Iwase-san, > >>> > >>> I am sorry for the delay in my reply. > >>> > >>> On Mon, 7 Nov 2016 16:08:09 +0900 > >>> Iwase Yusuke <[email protected]> wrote: > >>> > >>>> Hi Kakuma-San, > >>>> > >>>> Could you check my answer as following? > >>>> This patch will fix the Travis-CI build error of Ryu upstream. > >>>> > >>> > >>> I also was checking it and I found that it is caused by the fact that > >>> tinyrpc is not installed. > >> > >> Yes, the embedded tinyrpc has been deleted in the latest Ryu, > >> but docker_base.py does not install it yet. > >> > >> > >>> I answered in inline. > >>> > >>>> Thanks, > >>>> Iwase > >>>> > >>>> > >>>> On 2016年11月06日 14:28, Iwase Yusuke wrote: > >>>>> Hi, Kakuma-San, > >>>>> > >>>>> > >>>>> On 2016年11月04日 06:52, fumihiko kakuma wrote: > >>>>>> Hi Iwase-san, > >>>>>> > >>>>>> On Wed, 2 Nov 2016 16:51:35 +0900 > >>>>>> IWASE Yusuke <[email protected]> wrote: > >>>>>> > >>>>>>> Signed-off-by: IWASE Yusuke <[email protected]> > >>>>>>> --- > >>>>>>> ryu/tests/integrated/common/docker_base.py | 23 > >>>>>>> +++++++++++++---------- > >>>>>>> 1 file changed, 13 insertions(+), 10 deletions(-) > >>>>>>> > >>>>>>> diff --git a/ryu/tests/integrated/common/docker_base.py > >>>>>>> b/ryu/tests/integrated/common/docker_base.py > >>>>>>> index 13fe223..fd3ceef 100644 > >>>>>>> --- a/ryu/tests/integrated/common/docker_base.py > >>>>>>> +++ b/ryu/tests/integrated/common/docker_base.py > >>>>>>> @@ -196,25 +196,28 @@ class DockerImage(object): > >>>>>>> def create_ryu(self, tagname='ryu', image=None, > >>>>>>> check_exist=False): > >>>>>>> if check_exist and self.exist(tagname): > >>>>>>> return tagname > >>>>>>> - workdir = TEST_BASE_DIR + '/' + tagname > >>>>>>> + workdir = '%s/%s' % (TEST_BASE_DIR, tagname) > >>>>>>> workdir_ctn = '/root/osrg/ryu' > >>>>>>> - pkges = 'telnet tcpdump iproute2 ' > >>>>>>> - pkges += 'gcc python-pip python-dev libffi-dev libssl-dev' > >>>>>>> + pkges = 'telnet tcpdump iproute2' > >>>>>>> + pkges += ' gcc python-dev libffi-dev libssl-dev' > >>>>>>> + pkges += ' libxml2-dev libxslt1-dev zlib1g-dev' > >>>>>>> if image: > >>>>>>> use_image = image > >>>>>>> else: > >>>>>>> use_image = self.baseimage > >>>>>>> c = CmdBuffer() > >>>>>>> - c << 'FROM ' + use_image > >>>>>>> + c << 'FROM %s' % use_image > >>>>>>> c << 'RUN apt-get update' > >>>>>>> - c << 'RUN apt-get install -qy --no-install-recommends ' + > >>>>>>> pkges > >>>>>>> - c << 'RUN pip install -U six paramiko msgpack-rpc-python' > >>>>>>> - c << 'ADD ryu ' + workdir_ctn > >>>>>>> - install = 'RUN cd %s && pip install -r tools/pip-requires ' > >>>>>>> % workdir_ctn > >>>>>>> - install += '&& python setup.py install' > >>>>>>> + c << 'RUN apt-get install -qy --no-install-recommends %s' % > >>>>>>> pkges > >>>>>>> + c << 'ADD ryu %s' % workdir_ctn > >>>>>>> + install = 'RUN cd %s' % workdir_ctn > >>>>>>> + install += ' && pip install' > >>>>>>> + install += ' -r tools/pip-requires' > >>>>>>> + install += ' -r tools/test-requires' > >>>>>> > >>>>>> Why do we need the test-requires to run a ryu app on a container? > >>>>>> I think that we may need a pip-requires which some packages are added > >>>>>> or > >>>>>> an another required file. > >>>>> > >>>>> test-requires also includes the optional requirements (e.g. paramiko, > >>>>> tinyrpc) > >>>>> to run BGPSpeaker and OF-Config, for example. > >>>>> > >>>>> http://ryu.readthedocs.io/en/latest/getting_started.html#optional-requirements > >>>>> > >>>>> So, when we want to test the other feature on osrg/ryu based Docker > >>>>> container, > >>>>> we must have installed them, I guess. > >>> > >>> Yes, but I didn't like to install test tool(mock, nose and etc). > >> > >> I agree with you. > >> I don't prefer to installing unnecessary packages too. > >> > >> Well, I would like to separate test-requires into two files("test > >> requirements" and "optional requirements"). > >> > > > > Oh, it's greate! > > > >> # Removes "optional requirements" from test-requires. > >> $ cat tools/test-requires > >> coverage > >> mock > >> nose > >> pep8 > >> pylint > >> formencode > >> > >> # Create a new file for "optional requirements". > >> $ cat tools/optional-requires > >> lxml; platform_python_implementation != 'PyPy' # OF-Config > >> lxml==3.4.0; platform_python_implementation == 'PyPy' > >> cryptography==1.5 > >> paramiko # NETCONF, BGP speaker > >> tinyrpc # RPC > >> ncclient # OF-Config > >> > >> > >>> > >>>>> I prefer to separating the optional requirements from "test-requires", > >>>>> though... > >>>>> > >>> > >>> But I am ok in using test-requires if it makes many works. > >>> > >>>>>> > >>>>>> And the above Dockerfile seems to faile in a case not to use > >>>>>> osrg/ryu image. But we may not need that case. > >>>>> > >>>>> You mean the above does not have "python-pip", for example? > >>> > >>> > >>> Yes. And I get the following error while I build ryu image file. > >>> > >>> $ cat /tmp/ctn_docker/bgp/ryu/Dockerfile > >>> FROM ubuntu:14.04.5 > >>> RUN apt-get update > >>> RUN apt-get install -qy --no-install-recommends telnet tcpdump iproute2 > >>> gcc python-dev libffi-dev libssl-dev libxml2-dev libxslt1-dev zlib1g-dev > >>> python-pip > >>> ADD ryu /root/osrg/ryu > >>> RUN cd /root/osrg/ryu && pip install -r tools/pip-requires -r > >>> tools/test-requires && python setup.py install > >>> $ sudo docker build -t ryu /tmp/ctn_docker/bgp/ryu > >>> Sending build context to Docker daemon 179.3 MB > >>> Step 1 : FROM ubuntu:14.04.5 > >>> ---> f2d8ce9fa988 > >>> Step 2 : RUN apt-get update > >>> ---> Running in 90d7f0bb0ef3 > >>> Ign http://archive.ubuntu.com trusty InRelease > >>> Get:1 http://archive.ubuntu.com trusty-updates InRelease [65.9 kB] > >>> > >>> ... > >>> > >>> Removing intermediate container 384b0764eeaf > >>> Step 4 : ADD ryu /root/osrg/ryu > >>> ---> ee57cfffe5d9 > >>> Removing intermediate container 1adb83f316e9 > >>> Step 5 : RUN cd /root/osrg/ryu && pip install -r tools/pip-requires -r > >>> tools/test-requires && python setup.py install > >>> ---> Running in 9de973780048 > >>> Exception: > >>> Traceback (most recent call last): > >>> File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, > >>> in main > >>> status = self.run(options, args) > >>> File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line > >>> 262, in run > >>> for req in parse_requirements(filename, finder=finder, > >>> options=options, session=session): > >>> File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1632, in > >>> parse_requirements > >>> req = InstallRequirement.from_line(line, comes_from, > >>> prereleases=getattr(options, "pre", None)) > >>> File "/usr/lib/python2.7/dist-packages/pip/req.py", line 173, in > >>> from_line > >>> return cls(req, comes_from, url=url, prereleases=prereleases) > >>> File "/usr/lib/python2.7/dist-packages/pip/req.py", line 71, in __init__ > >>> req = pkg_resources.Requirement.parse(req) > >>> File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2667, in > >>> parse > >>> reqs = list(parse_requirements(s)) > >>> File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2605, in > >>> parse_requirements > >>> line, p, specs = scan_list(VERSION,LINE_END,line,p,(1,2),"version > >>> spec") > >>> File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2573, in > >>> scan_list > >>> raise ValueError("Expected "+item_name+" in",line,"at",line[p:]) > >>> ValueError: ('Expected version spec in', "lxml; > >>> platform_python_implementation != 'PyPy'", 'at', "; > >>> platform_python_implementation != 'PyPy'") > >>> > >>> Storing debug log for failure in /root/.pip/pip.log > >>> The command '/bin/sh -c cd /root/osrg/ryu && pip install -r > >>> tools/pip-requires -r tools/test-requires && python setup.py install' > >>> returned a non-zero code: 2 > >>> $ > >> > >> The above error show the pip version is too old, I guess. > >> The packaged python-pip on Ubuntu 14.04 might be old to use > >> the environment marker in the requirements file. > >> https://www.python.org/dev/peps/pep-0345/#environment-markers > >> > > > > Thank you for your explanation. > > > >> > >>> > >>>>> Well, is there any case or request for using the other base image? > >>>>> If any, I guess we can use the Ubuntu based image only... > >>>>> > >>> > >>> In the default DockerImage class uses ubuntu 14.04.5. > >>> I think that there may be a case try multiple ubuntu version and > >>> the latest ubuntu version and use debian. > >>> But I don't know whether we try those cases. > >>> > >>> BTW how about having a separate method like the following? > >>> > >>> def create_ryu(self, tagname='ryu', image=None, check_exist=False): > >>> ... > >>> > >>> def create_ryu_osrg(self, tagname='ryu', image='osrg/ryu', > >>> check_exist=False): > >>> ... > >> > >> Hummm... I don't want to separate method for the specific case... > >> > > > > I don't like that. But I thought that in case osrg/ryu and create from > > the begining we may need a different procedure. > > If we have the optional requirements file, we will not need separate > > method. > > > >> With Ubuntu 16.04, the packaged python-pip version is enough to > >> use the environment marker, so this problem will not occur. > >> But, if we should consider the multiple Ubuntu version, > >> I will update my patch to include python-pip. > > > > Pehaps when the latest ubuntu(for example 18.04) is released, we > > will face the same problem. > > What exactly "the same problem" means? I thought a case that we use an another feature of pip which is supported by the only latest ubuntu version and if use get-pip.py, it works withou depending on a version of image file. But it will be my over-thinking. > > Well, if on Ubuntu 18.04, the packaged pip should be the latest or > stable at the time of its release, we will never face this problem, > I guess. > > BTW, with my v2 patch, we can remove the environment marker in the > new requirement files and be free from this problem. > It is good if the environment marke is unnecessary or change a default image file name. $ git diff ryu/tests/integrated/common/docker_base.py diff --git a/ryu/tests/integrated/common/docker_base.py b/ryu/tests/integrated/common/docker_ba index 13fe223..f95e544 100644 --- a/ryu/tests/integrated/common/docker_base.py +++ b/ryu/tests/integrated/common/docker_base.py @@ -145,7 +145,7 @@ class Command(object): class DockerImage(object): - def __init__(self, baseimage='ubuntu:14.04.5'): + def __init__(self, baseimage='ubuntu:16.04'): self.baseimage = baseimage self.cmd = Command() $ > Thanks, > Iwase > > > > I think that we will can use get-pip.py > > > > # apt-get install curl > > # curl -k -f --retry 6 -o get-pip.py https://bootstrap.pypa.io/get-pip.py > > # python get-pip.py > > # pip --version > > pip 9.0.1 from /usr/local/lib/python2.7/dist-packages (python 2.7) > > # > > > >> > >> Thanks, > >> Iwase > >> > >> > >>> > >>> > >>> Thank you, > >>> fumihiko kakuma > >>> > >>> > >>>>> Thanks, > >>>>> Iwase > >>>>> > >>>>>> > >>>>>>> + install += ' && python setup.py install' > >>>>>>> c << install > >>>>>>> > >>>>>>> - self.cmd.sudo('rm -rf ' + workdir) > >>>>>>> + self.cmd.sudo('rm -rf %s' % workdir) > >>>>>>> self.cmd.execute('mkdir -p ' + workdir) > >>>>>>> self.cmd.execute("echo '%s' > %s/Dockerfile" % (str(c), > >>>>>>> workdir)) > >>>>>>> self.cmd.execute('cp -r ../ryu %s/' % workdir) > >>>>>>> -- > >>>>>>> 2.7.4 > >>>>>>> > >>>>>>> > >>>>>>> ------------------------------------------------------------------------------ > >>>>>>> Developer Access Program for Intel Xeon Phi Processors > >>>>>>> Access to Intel Xeon Phi processor-based developer platforms. > >>>>>>> With one year of Intel Parallel Studio XE. > >>>>>>> Training and support from Colfax. > >>>>>>> Order your platform today. http://sdm.link/xeonphi > >>>>>>> _______________________________________________ > >>>>>>> Ryu-devel mailing list > >>>>>>> [email protected] > >>>>>>> https://lists.sourceforge.net/lists/listinfo/ryu-devel > >>>>>> > >>> > > -- fumihiko kakuma <[email protected]> ------------------------------------------------------------------------------ Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi _______________________________________________ Ryu-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ryu-devel
