Re: [DISCUSS] Python thin client development approach.
Ivan, if Petr can`t help here, how can we change it out own ? May be we need some help from pmc chair or someone else ? > >> >>Igniters. Almost half of a year passed since this thread begun. We >>released 0.4.0, we adopted travis-ci and use it as primary source for >>test results but nothing changed in TC >> >>1. We use deprecated pytest-runner (even here >>( https://pypi.org/project/pytest-runner ) you can see deprecation >>notice and advice to use tox instead) in TC. >> This dependency is absolutely unnecessary and cause a bunch of >>problems when pyignite is installed as source from pypi.org local >>mirror >> We just need at least tox in linux agents on TC. >>2. We build release 0.4.0 manually on my and Igor Sapego's laptops. >>There are no release build on TC, despite the fact that all needed >>scripts are included in git repo and comprehensive instruction is >>available. >>For running these scripts, we just need docker available on linux >>agents and some additional packages on windows agents. >> >>I, Igor and Petr Ivanov had personal talk about this few months ago >>but nothing changed. >> >>пт, 22 янв. 2021 г. в 16:12, Ivan Daschinsky < ivanda...@gmail.com >: >>> >>> Igor, I've never talked about complete removal of TC builds. >>> I just suggested to add TC jobs for different python versions and use >>> travis heavily. >>> >>> Currently, we have done most of the tasks, except of run TC tests on >>> different python's versions. >>> >>> >>> >>> пт, 22 янв. 2021 г. в 15:16, Igor Sapego < isap...@apache.org >: Ivan, Though I generally agree with the approach you've suggested, I can see a problem here. Since we now have a separate repos for thin clients, for some features we may need to introduce changes to Ignite and python-thin repos in a single ticket and we should have an ability to run tests on with changes on both python client and server nodes. Current TC suites provide such ability, Travis does not. So I believe, it's too soon to abandon TC for thin clients, at least until we could solve the issue I've described. Best Regards, Igor On Fri, Dec 25, 2020 at 1:49 PM Nikolay Izhikov < nizhi...@apache.org > wrote: > Hello, Ivan. > > I’m +1 for your proposal. > > > 25 дек. 2020 г., в 13:14, Ivan Daschinsky < ivanda...@gmail.com > > написал(а): > > > > Hi folks! > > > > Since we already have a separate repo for thin-clients [1], [2] > > I'd like to propose some improvements in development process/ > > > > 1. We should simplify and automate unit tests run for different versions > of > > python > > 2. We should add travis integration per commit and pr. Tests could be run > > against latest dockered image of ignite > > 3. There should be ability to run tests against multiple pythons on TC > > 4. For thin client development process, travis should be the first > option. > > TC suite should be used only to check that compatibility is not broken > > and when new functionality is developed (rare case). > > > > I've prepared fix [3], you can see successful builds for travis. It uses > > tox [5], the most common tool to run tests in multiple environments. > > There are few environments set up in tox.ini -- with and without docker, > > with or without ssl, etc. This helped a lot > > to setup travis CI build (you can see in commits list of PR) and simplify > > run tests for developers. Also docker-compose was introduced > > to help python thin client developers. > > > > But I need some assistance for TC: > > 1. There is outdated python setuptools on TC agents, so tests cannot be > run > > with updated pytest etc. > > 2. Multiple pythons should be installed on TC agents (via > > https://github.com/pyenv/pyenv ), latest minor versions > > for 3.6, 3.7 and 3.8 > > 3. After that, TC job should be changed to utilize tox > > > > WDYT about this initiative? > > > > > > [1] -- https://issues.apache.org/jira/browse/IGNITE-13767 > > [2] -- https://github.com/apache/ignite-python-thin-client > > [3] -- https://issues.apache.org/jira/browse/IGNITE-13903 > > [4] -- > https://github.com/apache/ignite-python-thin-client/pull/1/commits > > [5] -- https://tox.readthedocs.io/en/latest/ > > > > -- > > Sincerely yours, Ivan Daschinskiy > > >>> >>> >>> >>> -- >>> Sincerely yours, Ivan Daschinskiy > > > >
Re: [DISCUSS] Python thin client development approach.
Igniters. Almost half of a year passed since this thread begun. We released 0.4.0, we adopted travis-ci and use it as primary source for test results but nothing changed in TC 1. We use deprecated pytest-runner (even here (https://pypi.org/project/pytest-runner) you can see deprecation notice and advice to use tox instead) in TC. This dependency is absolutely unnecessary and cause a bunch of problems when pyignite is installed as source from pypi.org local mirror We just need at least tox in linux agents on TC. 2. We build release 0.4.0 manually on my and Igor Sapego's laptops. There are no release build on TC, despite the fact that all needed scripts are included in git repo and comprehensive instruction is available. For running these scripts, we just need docker available on linux agents and some additional packages on windows agents. I, Igor and Petr Ivanov had personal talk about this few months ago but nothing changed. пт, 22 янв. 2021 г. в 16:12, Ivan Daschinsky : > > Igor, I've never talked about complete removal of TC builds. > I just suggested to add TC jobs for different python versions and use travis > heavily. > > Currently, we have done most of the tasks, except of run TC tests on > different python's versions. > > > > пт, 22 янв. 2021 г. в 15:16, Igor Sapego : >> >> Ivan, >> >> Though I generally agree with the approach you've suggested, I can see >> a problem here. Since we now have a separate repos for thin clients, for >> some features we may need to introduce changes to Ignite and python-thin >> repos in a single ticket and we should have an ability to run tests on with >> changes on both python client and server nodes. Current TC suites provide >> such ability, Travis does not. So I believe, it's too soon to abandon TC >> for thin >> clients, at least until we could solve the issue I've described. >> >> Best Regards, >> Igor >> >> >> On Fri, Dec 25, 2020 at 1:49 PM Nikolay Izhikov wrote: >> >> > Hello, Ivan. >> > >> > I’m +1 for your proposal. >> > >> > > 25 дек. 2020 г., в 13:14, Ivan Daschinsky >> > написал(а): >> > > >> > > Hi folks! >> > > >> > > Since we already have a separate repo for thin-clients [1], [2] >> > > I'd like to propose some improvements in development process/ >> > > >> > > 1. We should simplify and automate unit tests run for different versions >> > of >> > > python >> > > 2. We should add travis integration per commit and pr. Tests could be run >> > > against latest dockered image of ignite >> > > 3. There should be ability to run tests against multiple pythons on TC >> > > 4. For thin client development process, travis should be the first >> > option. >> > > TC suite should be used only to check that compatibility is not broken >> > > and when new functionality is developed (rare case). >> > > >> > > I've prepared fix [3], you can see successful builds for travis. It uses >> > > tox [5], the most common tool to run tests in multiple environments. >> > > There are few environments set up in tox.ini -- with and without docker, >> > > with or without ssl, etc. This helped a lot >> > > to setup travis CI build (you can see in commits list of PR) and simplify >> > > run tests for developers. Also docker-compose was introduced >> > > to help python thin client developers. >> > > >> > > But I need some assistance for TC: >> > > 1. There is outdated python setuptools on TC agents, so tests cannot be >> > run >> > > with updated pytest etc. >> > > 2. Multiple pythons should be installed on TC agents (via >> > > https://github.com/pyenv/pyenv), latest minor versions >> > > for 3.6, 3.7 and 3.8 >> > > 3. After that, TC job should be changed to utilize tox >> > > >> > > WDYT about this initiative? >> > > >> > > >> > > [1] -- https://issues.apache.org/jira/browse/IGNITE-13767 >> > > [2] -- https://github.com/apache/ignite-python-thin-client >> > > [3] -- https://issues.apache.org/jira/browse/IGNITE-13903 >> > > [4] -- >> > https://github.com/apache/ignite-python-thin-client/pull/1/commits >> > > [5] -- https://tox.readthedocs.io/en/latest/ >> > > >> > > -- >> > > Sincerely yours, Ivan Daschinskiy >> > >> > > > > > -- > Sincerely yours, Ivan Daschinskiy -- Sincerely yours, Ivan Daschinskiy
Re: [DISCUSS] Python thin client development approach.
Igor, I've never talked about complete removal of TC builds. I just suggested to add TC jobs for different python versions and use travis heavily. Currently, we have done most of the tasks, except of run TC tests on different python's versions. пт, 22 янв. 2021 г. в 15:16, Igor Sapego : > Ivan, > > Though I generally agree with the approach you've suggested, I can see > a problem here. Since we now have a separate repos for thin clients, for > some features we may need to introduce changes to Ignite and python-thin > repos in a single ticket and we should have an ability to run tests on with > changes on both python client and server nodes. Current TC suites provide > such ability, Travis does not. So I believe, it's too soon to abandon TC > for thin > clients, at least until we could solve the issue I've described. > > Best Regards, > Igor > > > On Fri, Dec 25, 2020 at 1:49 PM Nikolay Izhikov > wrote: > > > Hello, Ivan. > > > > I’m +1 for your proposal. > > > > > 25 дек. 2020 г., в 13:14, Ivan Daschinsky > > написал(а): > > > > > > Hi folks! > > > > > > Since we already have a separate repo for thin-clients [1], [2] > > > I'd like to propose some improvements in development process/ > > > > > > 1. We should simplify and automate unit tests run for different > versions > > of > > > python > > > 2. We should add travis integration per commit and pr. Tests could be > run > > > against latest dockered image of ignite > > > 3. There should be ability to run tests against multiple pythons on TC > > > 4. For thin client development process, travis should be the first > > option. > > > TC suite should be used only to check that compatibility is not broken > > > and when new functionality is developed (rare case). > > > > > > I've prepared fix [3], you can see successful builds for travis. It > uses > > > tox [5], the most common tool to run tests in multiple environments. > > > There are few environments set up in tox.ini -- with and without > docker, > > > with or without ssl, etc. This helped a lot > > > to setup travis CI build (you can see in commits list of PR) and > simplify > > > run tests for developers. Also docker-compose was introduced > > > to help python thin client developers. > > > > > > But I need some assistance for TC: > > > 1. There is outdated python setuptools on TC agents, so tests cannot be > > run > > > with updated pytest etc. > > > 2. Multiple pythons should be installed on TC agents (via > > > https://github.com/pyenv/pyenv), latest minor versions > > > for 3.6, 3.7 and 3.8 > > > 3. After that, TC job should be changed to utilize tox > > > > > > WDYT about this initiative? > > > > > > > > > [1] -- https://issues.apache.org/jira/browse/IGNITE-13767 > > > [2] -- https://github.com/apache/ignite-python-thin-client > > > [3] -- https://issues.apache.org/jira/browse/IGNITE-13903 > > > [4] -- > > https://github.com/apache/ignite-python-thin-client/pull/1/commits > > > [5] -- https://tox.readthedocs.io/en/latest/ > > > > > > -- > > > Sincerely yours, Ivan Daschinskiy > > > > > -- Sincerely yours, Ivan Daschinskiy
Re: [DISCUSS] Python thin client development approach.
Ivan, Though I generally agree with the approach you've suggested, I can see a problem here. Since we now have a separate repos for thin clients, for some features we may need to introduce changes to Ignite and python-thin repos in a single ticket and we should have an ability to run tests on with changes on both python client and server nodes. Current TC suites provide such ability, Travis does not. So I believe, it's too soon to abandon TC for thin clients, at least until we could solve the issue I've described. Best Regards, Igor On Fri, Dec 25, 2020 at 1:49 PM Nikolay Izhikov wrote: > Hello, Ivan. > > I’m +1 for your proposal. > > > 25 дек. 2020 г., в 13:14, Ivan Daschinsky > написал(а): > > > > Hi folks! > > > > Since we already have a separate repo for thin-clients [1], [2] > > I'd like to propose some improvements in development process/ > > > > 1. We should simplify and automate unit tests run for different versions > of > > python > > 2. We should add travis integration per commit and pr. Tests could be run > > against latest dockered image of ignite > > 3. There should be ability to run tests against multiple pythons on TC > > 4. For thin client development process, travis should be the first > option. > > TC suite should be used only to check that compatibility is not broken > > and when new functionality is developed (rare case). > > > > I've prepared fix [3], you can see successful builds for travis. It uses > > tox [5], the most common tool to run tests in multiple environments. > > There are few environments set up in tox.ini -- with and without docker, > > with or without ssl, etc. This helped a lot > > to setup travis CI build (you can see in commits list of PR) and simplify > > run tests for developers. Also docker-compose was introduced > > to help python thin client developers. > > > > But I need some assistance for TC: > > 1. There is outdated python setuptools on TC agents, so tests cannot be > run > > with updated pytest etc. > > 2. Multiple pythons should be installed on TC agents (via > > https://github.com/pyenv/pyenv), latest minor versions > > for 3.6, 3.7 and 3.8 > > 3. After that, TC job should be changed to utilize tox > > > > WDYT about this initiative? > > > > > > [1] -- https://issues.apache.org/jira/browse/IGNITE-13767 > > [2] -- https://github.com/apache/ignite-python-thin-client > > [3] -- https://issues.apache.org/jira/browse/IGNITE-13903 > > [4] -- > https://github.com/apache/ignite-python-thin-client/pull/1/commits > > [5] -- https://tox.readthedocs.io/en/latest/ > > > > -- > > Sincerely yours, Ivan Daschinskiy > >
Re: [DISCUSS] Python thin client development approach.
Hello, Ivan. I’m +1 for your proposal. > 25 дек. 2020 г., в 13:14, Ivan Daschinsky написал(а): > > Hi folks! > > Since we already have a separate repo for thin-clients [1], [2] > I'd like to propose some improvements in development process/ > > 1. We should simplify and automate unit tests run for different versions of > python > 2. We should add travis integration per commit and pr. Tests could be run > against latest dockered image of ignite > 3. There should be ability to run tests against multiple pythons on TC > 4. For thin client development process, travis should be the first option. > TC suite should be used only to check that compatibility is not broken > and when new functionality is developed (rare case). > > I've prepared fix [3], you can see successful builds for travis. It uses > tox [5], the most common tool to run tests in multiple environments. > There are few environments set up in tox.ini -- with and without docker, > with or without ssl, etc. This helped a lot > to setup travis CI build (you can see in commits list of PR) and simplify > run tests for developers. Also docker-compose was introduced > to help python thin client developers. > > But I need some assistance for TC: > 1. There is outdated python setuptools on TC agents, so tests cannot be run > with updated pytest etc. > 2. Multiple pythons should be installed on TC agents (via > https://github.com/pyenv/pyenv), latest minor versions > for 3.6, 3.7 and 3.8 > 3. After that, TC job should be changed to utilize tox > > WDYT about this initiative? > > > [1] -- https://issues.apache.org/jira/browse/IGNITE-13767 > [2] -- https://github.com/apache/ignite-python-thin-client > [3] -- https://issues.apache.org/jira/browse/IGNITE-13903 > [4] -- https://github.com/apache/ignite-python-thin-client/pull/1/commits > [5] -- https://tox.readthedocs.io/en/latest/ > > -- > Sincerely yours, Ivan Daschinskiy
[DISCUSS] Python thin client development approach.
Hi folks! Since we already have a separate repo for thin-clients [1], [2] I'd like to propose some improvements in development process/ 1. We should simplify and automate unit tests run for different versions of python 2. We should add travis integration per commit and pr. Tests could be run against latest dockered image of ignite 3. There should be ability to run tests against multiple pythons on TC 4. For thin client development process, travis should be the first option. TC suite should be used only to check that compatibility is not broken and when new functionality is developed (rare case). I've prepared fix [3], you can see successful builds for travis. It uses tox [5], the most common tool to run tests in multiple environments. There are few environments set up in tox.ini -- with and without docker, with or without ssl, etc. This helped a lot to setup travis CI build (you can see in commits list of PR) and simplify run tests for developers. Also docker-compose was introduced to help python thin client developers. But I need some assistance for TC: 1. There is outdated python setuptools on TC agents, so tests cannot be run with updated pytest etc. 2. Multiple pythons should be installed on TC agents (via https://github.com/pyenv/pyenv), latest minor versions for 3.6, 3.7 and 3.8 3. After that, TC job should be changed to utilize tox WDYT about this initiative? [1] -- https://issues.apache.org/jira/browse/IGNITE-13767 [2] -- https://github.com/apache/ignite-python-thin-client [3] -- https://issues.apache.org/jira/browse/IGNITE-13903 [4] -- https://github.com/apache/ignite-python-thin-client/pull/1/commits [5] -- https://tox.readthedocs.io/en/latest/ -- Sincerely yours, Ivan Daschinskiy
