Thanks Waldek! You can compile sklearn without openmp, which I believe would get rid of the multiprocessing part.
As an alternative I guess I could try and use the weka machine learning library which is in Java. Mike Sent from my iPhone > On Aug 28, 2019, at 11:24 AM, Waldek Kozaczuk <[email protected]> wrote: > > Hi, > > Given you are the second person asking about it, I started implementing it > last night given it is not that difficult. But it would still take a bit of > time to make it correct and covered with unit tests. > > However, I peeked more at the details of > https://github.com/cloudius-systems/osv/issues/1025 and realized scikit-learn > uses the multiprocessing module > (https://docs.python.org/3.4/library/multiprocessing.html?highlight=process) > which allows multiprocessing based on forking new processes vs > multithreading. OSv supports pthreads very well even on multiple cores (SMP) > but being unikernel it does NOT and never intended to support fork()/spawn(). > That being said I suspect that even if implemented name semaphores you will > hit the fork() obstacle down the road. > > Based on this doc - > https://scipy.github.io/old-wiki/pages/ParallelProgramming, and > https://scipy.github.io/old-wiki/pages/Cookbook/Multithreading.html you can > use SciPy with multithreading which should work on OSv. > > There is also GraalVM Python which is supposed to run python much faster and > is focused on SciPy but it is still experimental - > https://www.graalvm.org/docs/reference-manual/languages/python/. I have had a > lot of luck running native images (JVM code AOT-compiled to machine code) on > OSv. > > Regards, > Waldek > >> On Tuesday, August 27, 2019 at 10:21:22 PM UTC-4, Michael De Lucia wrote: >> Waldek, >> >> >> >> I finally got sklearn installed and got to the same issue you pointed out >> described here - https://github.com/cloudius-systems/osv/issues/1025 >> >> >> >> Has this been implemented yet? >> >> >> >> Thanks, >> >> >> >> Mike >> >> >> >> From: [email protected] <[email protected]> On Behalf Of Waldek >> Kozaczuk >> Sent: Tuesday, August 27, 2019 11:40 AM >> To: OSv Development <[email protected]> >> Subject: Re: [osv-dev] Re: Python 3 Scikitlearn >> >> >> >> It looks like you have one of the libraries, >> libopenblasp-r0-2ecf47d5.3.7.dev.so depends on, missing. >> >> >> >> You can use: >> >> >> >> ./scripts/manifest_from_host.sh -l >> /lib/python3.7/numpy/.libs/libopenblasp-r0-2ecf47d5.3.7.dev.so >> >> >> >> to see which libraries have to be added. >> >> >> On Tuesday, August 27, 2019 at 11:15:51 AM UTC-4, Michael De Lucia wrote: >> >> Hi Waldek, >> >> >> >> No that does not seem to be the problem. I started with just trying to see >> with numpy first, since that is a dependency. Here is what I did. I used >> your docker image build on Fedora29, which does a git and pulls down the OSv >> repo. Then I built the python3x osv-app in osv-apps/python3x/ on the github >> repo. >> >> >> >> I have python 3.7 installed on the fedora29 image and did pip3 install numpy >> to get that installed in my python 3.7 environment. I modified the GET file >> under the python3x directory to include the numpy module. >> >> >> >> Next I ran ./scripts/build image=python3x and the build completed >> successfully. >> >> >> >> Then I ran ./scripts/run.py –api -e “/python3 -c \”import numpy; >> print(numpy.__version__)\”” >> >> >> >> That is when it failed in an unresolved symbol for blas_memory_alloc >> >> >> >> [root@8ec8090052ab osv]# ./scripts/run.py --api -e "/python3 -c \"import >> numpy; print(numpy.__version__)\"" >> >> OSv v0.53.0-74-gef56fde7 >> >> eth0: 192.168.122.15 >> >> Booted up in 611.56 ms >> >> /lib/python3.7/numpy/.libs/libopenblasp-r0-2ecf47d5.3.7.dev.so: failed >> looking up symbol blas_memory_alloc >> >> >> >> [backtrace] >> >> 0x0000000040356209 <elf::object::symbol(unsigned int, bool)+969> >> >> 0x00000000403562cf <elf::object::resolve_pltgot(unsigned int)+127> >> >> 0x0000000040356494 <elf_resolve_pltgot+52> >> >> 0x000000004039b82f <???+1077524527> >> >> 0x000020000077ff6f <???+7864175> >> >> 0x0000000000000003 <???+3> >> >> >> >> Any ideas on how to resolve this? >> >> >> >> Thanks, >> >> >> >> Mike >> >> >> >> From: [email protected] <[email protected]> On Behalf Of Waldek >> Kozaczuk >> Sent: Monday, August 26, 2019 6:03 PM >> To: OSv Development <[email protected]> >> Subject: Re: [osv-dev] Re: Python 3 Scikitlearn >> >> >> >> Is it possibly the same issue as described here - >> https://github.com/cloudius-systems/osv/issues/1025? >> >> On Wednesday, August 21, 2019 at 10:08:15 PM UTC-4, Michael De Lucia wrote: >> >> I Waldek, >> >> >> >> I will get the exact command and stack trace to you on Friday. >> >> >> >> Thanks, >> >> >> >> Mike >> >> Sent from my iPhone >> >> >> On Aug 21, 2019, at 9:39 PM, Waldek Kozaczuk <[email protected]> wrote: >> >> Hi, >> >> >> >> Can you provide exact build command you used to build the image? Also, can >> you provide the exact stack trace so we know which symbol is was unresolved? >> >> >> >> Thanks, >> >> Waldek >> >> On Wednesday, August 21, 2019 at 9:19:36 PM UTC-4, [email protected] wrote: >> >> Hi, >> >> >> >> I tried to use the OSv app for python 3 support. I built the image with the >> system installed python 3 version. I also installed scikit learn and added >> that to the modules to keep from the site packages. Scikit also relies on >> numpy and scipy. However, I got it to import and build an image. But when I >> tried to run it, failed with an unresolved symbol. I have a centos system >> and was building within the docker container fedora 29 template. Any >> suggestions or thoughts on why this would fail? >> >> >> >> Thanks, >> >> >> >> Mike >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "OSv Development" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/osv-dev/V9fch2WFX7o/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/osv-dev/61cb76c8-63f4-490c-b040-5634148c3930%40googlegroups.com. >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "OSv Development" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/osv-dev/V9fch2WFX7o/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/osv-dev/dba8eead-7d65-4f86-bef0-98eb80925d6d%40googlegroups.com. >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "OSv Development" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/osv-dev/V9fch2WFX7o/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/osv-dev/e1bdaf15-e259-4bad-8beb-15f81c4506f8%40googlegroups.com. >> > > -- > You received this message because you are subscribed to a topic in the Google > Groups "OSv Development" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/osv-dev/V9fch2WFX7o/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/osv-dev/d7902b92-c058-4fcb-8db9-bcdaf0152b8a%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/A110F8DD-4AB2-4E74-9CF0-4B113F894050%40gmail.com.
