conda <https://conda.io/docs/index.html> installs a complete local python, R or other infrastructure separate from the system installation, there is no global installation component. this is an example of a high performance pipeline combining many components routinely installed on academic clusters: https://github.com/chapmanb/bcbio-nextgen. i think this is their solution to the problem you describe by completely controlling the necessary installations locally, since especially python with versions 2 and 3 being routinely used, seems inherently incapable on it's own of keeping track of how it gets installed.
as far as left-over compiled python bits, as a last ditch measure i eliminate the opencog related files and directories in /usr/local/* and delete the build folders and redo cmake and 'make install'. is there anywhere else leftover python files can accumulate? On Thursday, January 25, 2018 at 2:08:21 PM UTC-5, linas wrote: > > Based on my experience with link-grammar, any system (conda?) which > installs packages both locally and globally, on a per-app deployment, makes > the problem much much worse. There, we have an Apple user who had an > endless, bottomless litany of crazy failures we the developers never saw > before. During diagnosis, we discovered that his system contained 5 > different versions of python installed in 5 different locations, and > during a typical compile, header files from one version would be linked to > libraries from a different (incompatible) version. We told him to nuke > everything. Easier said than done. > > As soon as you have multiple versions of crap on a system, you will get > crashes, compile errors, test failures, etc that are hard-to-debug, or > even very hard, almost impossible to debug. It is a recipie for > guaranteed disaster. Don't go there. > > In general, on a linux system, avoid and do not use: pip, easy_install, > virtualenv, conda -- they are the root cause of these failures. Just use > apt-get. Everything that you need is on apt-get already, and using pip or > conda is gauranteeed to install a second copy of things that is not > compatible, and *WILL* with 100% certainty, cause crazy-making failures. > > --linas > > On Thu, Jan 25, 2018 at 2:07 AM, Michael Duncan <[email protected] > <javascript:>> wrote: > >> ed, someone in japan made some debian packages and set up an apt repo >> last august, see threads here >> <https://groups.google.com/forum/#!topic/opencog/AQtyUiffyK4> and here >> <https://groups.google.com/forum/#!topic/opencog/GfIIDgH0sww>. i don't >> know their current state. >> >> i've often had different test failures than jenkins on ubuntu 14.04 and >> 16.04: currently i'm getting 1 passing and 1 failing test different for >> both the atomspace and opencog builds. >> here's an interesting clue: i had some folders installed by pip called >> without sudo for some packages (also installed globally) that i cleared out >> in case that was the problem. this caused the PythonEvalUTest fail in >> addition to the CythonGuile test fail for the atomspace repo. in opencog, >> my ShellUTest passes but PLNRulesUTest fails. >> >> i think this mess could be fixed with dedicated python installation like >> conda. opensource bioinformatics pipelines that use python do this to >> avoid all the inter-machine variation that inevitably seems to accumulate >> in python installations. i've given up on 14.04, it's too much of a pain >> keeping up as the packages get out-dated. >> >> good luck with the package work, it will really help make opencog more >> accessible to all the folks getting inspired by the singularity net ! >> >> >> On Saturday, January 20, 2018 at 9:30:59 AM UTC-5, Ed Guy wrote: >>> >>> Mike, >>> >>> The tests which fail are very dependent on the precise packages >>> installed. >>> I started to address the unicode issues ( at a deb package level) and >>> broke the 1404 builds. >>> I made the mistake of not putting the docker config files in a repo, so >>> I'm going to have to grok the log files >>> to find the root cause. (fortunately, the logs are in a repo) I'd >>> eventually like to get debian/ubuntu packages >>> out of this process. >>> >>> /ed @ cen.ai >>> >>> On Thursday, January 18, 2018 at 4:29:05 PM UTC-5, linas wrote: >>>> >>>> Mike, >>>> >>>> in general, open bug reports. Also please try to get the iCog labs guys >>>> to fix them. For the CythonGuile crash, you can run the test by hand, >>>> like >>>> so: >>>> >>>> cd to/where/atomspace/is >>>> export PYTHONPATH=./build/opencog/cython >>>> nosetests -vs tests/cython/guile >>>> >>>> Actually, before you do that, make sure you remove all the various >>>> *.pyc files, and the __pycache__ cache directories in tests/cython/guile >>>> and elsewhere. Python tends to compute caches, write them in random >>>> directories, forget that it created those caches, not regenerate them and >>>> then crazy-making. I assume this is fixed in python3... also make sure >>>> you did a `sudo make install` so that you are not picking up old, stale >>>> junk from /usrlocal .. that should be enough to make the test pass. >>>> >>>> --linas >>>> >>>> >>>> On Thu, Jan 18, 2018 at 1:03 PM, Michael Duncan <[email protected]> >>>> wrote: >>>> >>>>> >>>>> thanks, ed! i think i'm ahead of jenkins now but for 16.04 i'm getting >>>>> atomspace$ git branch -v >>>>> * master 9c1ad80 Revise python dictionary printing >>>>> .. >>>>> The following tests FAILED: >>>>> 118 - CythonGuile (SEGFAULT) >>>>> >>>>> and >>>>> opencog$ git branch -v >>>>> * master 9e89aa1 Merge pull request #2979 from >>>>> ngeiswei/optimize-xpattern-miner >>>>> ... >>>>> The following tests FAILED: >>>>> 2 - AtomSpacePublisherModuleUTest (OTHER_FAULT) >>>>> 6 - AtomOcTreeUTest (SEGFAULT) >>>>> 7 - TimeSpaceAtomUTest (SEGFAULT) >>>>> 8 - AnaphoraTest (Failed) >>>>> 12 - PLNRulesUTest (Failed) >>>>> 21 - RestApiTest (Failed) >>>>> >>>>> with cogutil and moses passing 100% >>>>> >>>>> On Tuesday, January 16, 2018 at 10:59:20 PM UTC-5, Ed Guy wrote: >>>>>> >>>>>> Michael, >>>>>> >>>>>> >>>>>> >>>>>> If you click on the module name, e.g., ‘OpenCOG_U1404_AtomSpace >>>>>> <https://github.com/edguy3/CenStatus/blob/master/jobs/OpenCOG_U1404_AtomSpace.log>’, >>>>>> >>>>>> it should bring you to the full log file. (except right now while a new >>>>>> build is running ) >>>>>> >>>>>> But to make it easier, I’ve added the last two dozen or so problems >>>>>> into the report page. >>>>>> >>>>>> >>>>>> >>>>>> *OpenCOG_U1604_AtomSpace* >>>>>> >>>>>> 41/111 Test #41: BasicSCMUTest ....................***Failed 0.17 >>>>>> sec >>>>>> >>>>>> 111/111 Test #111: CythonGuile ......................***Exception: >>>>>> SegFault 0.21 sec >>>>>> >>>>>> >>>>>> >>>>>> *OpenCOG_U1604_Opencog* >>>>>> >>>>>> 2/25 Test #2: AtomSpacePublisherModuleUTest ....***Exception: Other >>>>>> 5.14 sec >>>>>> >>>>>> 6/25 Test #6: AtomOcTreeUTest ..................***Exception: >>>>>> SegFault 0.07 sec >>>>>> >>>>>> 7/25 Test #7: TimeSpaceAtomUTest ...............***Exception: >>>>>> SegFault 0.07 sec >>>>>> >>>>>> 8/25 Test #8: AnaphoraTest .....................***Failed 1.22 sec >>>>>> >>>>>> 12/25 Test #12: OpenPsiRulesUTest ................***Failed 0.49 >>>>>> sec >>>>>> >>>>>> 14/25 Test #14: OpenPsiSCMUTest ..................***Failed 8.84 >>>>>> sec >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *Linas, * >>>>>> >>>>>> >>>>>> >>>>>> I’ll look at the UTF-8 config; perhaps a regional default is giving a >>>>>> different configuration. >>>>>> >>>>>> >>>>>> >>>>>> Regarding the 118 vs 111 tests. Yes, postgres was not installed. >>>>>> I’ll add it at next opportunity. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> I’ll be in Miami for the rest of the week – If anyone is at the BTC, >>>>>> drop me a line for a face-to-face. >>>>>> >>>>>> >>>>>> >>>>>> /ed >>>>>> >>>>>> [email protected] >>>>>> >>>>>> >>>>>> >>>>>> On Tuesday, January 16, 2018 at 8:49:57 AM UTC-5, Ed Guy wrote: >>>>>>> >>>>>>> >>>>>>> Team: >>>>>>> >>>>>>> What is the current expectation for passing test percentages on the >>>>>>> included unit tests? >>>>>>> >>>>>>> I'm getting these results on Ubuntu 14.04 (U1404) and Ubuntu 16.04 >>>>>>> (U1604) >>>>>>> : >>>>>>> >>>>>>> >>>>>>> OpenCOG_U1404_AtomSpace >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/jobs/OpenCOG_U1404_AtomSpace.log> >>>>>>> [image: >>>>>>> Status] >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/images/TESTFAIL.svg> >>>>>>> 98% >>>>>>> tests passed, 2 tests failed out of 111 >>>>>>> >>>>>>> >>>>>>> >>>>>>> OpenCOG_U1404_CogUtil >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/jobs/OpenCOG_U1404_CogUtil.log> >>>>>>> [image: >>>>>>> Status] >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/images/TESTPASS.svg> >>>>>>> 100% >>>>>>> tests passed, 0 tests failed out of 12 >>>>>>> >>>>>>> >>>>>>> >>>>>>> OpenCOG_U1404__Core_Build >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/jobs/OpenCOG_U1404__Core_Build.log> >>>>>>> [image: >>>>>>> Status] >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/images/BUILDPASS.svg> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> OpenCOG_U1404_Moses >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/jobs/OpenCOG_U1404_Moses.log> >>>>>>> [image: >>>>>>> Status] >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/images/TESTPASS.svg> >>>>>>> 100% >>>>>>> tests passed, 0 tests failed out of 35 >>>>>>> >>>>>>> >>>>>>> >>>>>>> OpenCOG_U1404_Opencog >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/jobs/OpenCOG_U1404_Opencog.log> >>>>>>> [image: >>>>>>> Status] >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/images/TESTFAIL.svg> >>>>>>> 76% >>>>>>> tests passed, 6 tests failed out of 25 >>>>>>> >>>>>>> >>>>>>> >>>>>>> OpenCOG_U1604_AtomSpace >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/jobs/OpenCOG_U1604_AtomSpace.log> >>>>>>> [image: >>>>>>> Status] >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/images/TESTFAIL.svg> >>>>>>> 98% >>>>>>> tests passed, 2 tests failed out of 111 >>>>>>> >>>>>>> >>>>>>> >>>>>>> OpenCOG_U1604_CogUtil >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/jobs/OpenCOG_U1604_CogUtil.log> >>>>>>> [image: >>>>>>> Status] >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/images/TESTPASS.svg> >>>>>>> 100% >>>>>>> tests passed, 0 tests failed out of 12 >>>>>>> >>>>>>> >>>>>>> >>>>>>> OpenCOG_U1604__Core_Build >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/jobs/OpenCOG_U1604__Core_Build.log> >>>>>>> [image: >>>>>>> Status] >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/images/BUILDPASS.svg> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> OpenCOG_U1604_Moses >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/jobs/OpenCOG_U1604_Moses.log> >>>>>>> [image: >>>>>>> Status] >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/images/TESTPASS.svg> >>>>>>> 100% >>>>>>> tests passed, 0 tests failed out of 35 >>>>>>> >>>>>>> >>>>>>> >>>>>>> OpenCOG_U1604_Opencog >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/jobs/OpenCOG_U1604_Opencog.log> >>>>>>> [image: >>>>>>> Status] >>>>>>> <https://github.com/edguy3/CenStatus/blob/master/images/TESTFAIL.svg> >>>>>>> 72% >>>>>>> tests passed, 7 tests failed out of 25 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> ( from https://github.com/edguy3/CenStatus ) >>>>>>> >>>>>>> >>>>>>> Thanks in advance, >>>>>>> >>>>>>> /ed >>>>>>> >>>>>>> >>>>>>> ===================== Edward T Guy, III, Ph.D. Cen.ai Chief >>>>>>> Technology Officer +1 908 400 0094 >>>>>>> >>>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "opencog" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at https://groups.google.com/group/opencog. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/opencog/aadddfd2-3ed7-45b1-bed0-5e512ea90275%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/opencog/aadddfd2-3ed7-45b1-bed0-5e512ea90275%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>>> >>>> -- >>>> *"The problem is not that artificial intelligence will get too smart >>>> and take over the world," computer scientist Pedro Domingos writes, "the >>>> problem is that it's too stupid and already has." * >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "opencog" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> Visit this group at https://groups.google.com/group/opencog. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/opencog/b2964bdd-f8f3-499f-a40b-36fa08900ec8%40googlegroups.com >> >> <https://groups.google.com/d/msgid/opencog/b2964bdd-f8f3-499f-a40b-36fa08900ec8%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > cassette tapes - analog TV - film cameras - you > -- You received this message because you are subscribed to the Google Groups "opencog" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/opencog. To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/e200f85c-b5c3-44ae-8bac-4051d5ef8457%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
