Hi, Given you are the second person asking about it, I started implementing it last night given it is not that difficult. But it would still take a bit of time to make it correct and covered with unit tests.
However, I peeked more at the details of https://github.com/cloudius-systems/osv/issues/1025 and realized scikit-learn uses the multiprocessing module (https://docs.python.org/3.4/library/multiprocessing.html?highlight=process) which allows multiprocessing based on forking new processes vs multithreading. OSv supports pthreads very well even on multiple cores (SMP) but being unikernel it *does NOT and never intended to support fork()/spawn()*. That being said I suspect that even if implemented name semaphores you will hit the fork() obstacle down the road. Based on this doc - https://scipy.github.io/old-wiki/pages/ParallelProgramming, and https://scipy.github.io/old-wiki/pages/Cookbook/Multithreading.html you can use SciPy with multithreading which should work on OSv. There is also GraalVM Python which is supposed to run python much faster and is focused on SciPy but it is still experimental - https://www.graalvm.org/docs/reference-manual/languages/python/. I have had a lot of luck running native images (JVM code AOT-compiled to machine code) on OSv. Regards, Waldek On Tuesday, August 27, 2019 at 10:21:22 PM UTC-4, Michael De Lucia wrote: > > Waldek, > > > > I finally got sklearn installed and got to the same issue you pointed out > described here - https://github.com/cloudius-systems/osv/issues/1025 > > > > Has this been implemented yet? > > > > Thanks, > > > > Mike > > > > *From:* [email protected] <javascript:> <[email protected] > <javascript:>> *On Behalf Of *Waldek Kozaczuk > *Sent:* Tuesday, August 27, 2019 11:40 AM > *To:* OSv Development <[email protected] <javascript:>> > *Subject:* Re: [osv-dev] Re: Python 3 Scikitlearn > > > > It looks like you have one of the libraries, > libopenblasp-r0-2ecf47d5.3.7.dev.so > <http://www.google.com/url?q=http%3A%2F%2Flibopenblasp-r0-2ecf47d5.3.7.dev.so&sa=D&sntz=1&usg=AFQjCNEzhQZhSHkxBkhVADvoaOnGZdTRaQ> > depends > on, missing. > > > > You can use: > > > > ./scripts/manifest_from_host.sh -l /lib/python3.7/numpy/.libs/ > libopenblasp-r0-2ecf47d5.3.7.dev.so > <http://www.google.com/url?q=http%3A%2F%2Flibopenblasp-r0-2ecf47d5.3.7.dev.so&sa=D&sntz=1&usg=AFQjCNEzhQZhSHkxBkhVADvoaOnGZdTRaQ> > > > > to see which libraries have to be added. > > > On Tuesday, August 27, 2019 at 11:15:51 AM UTC-4, Michael De Lucia wrote: > > Hi Waldek, > > > > No that does not seem to be the problem. I started with just trying to see > with numpy first, since that is a dependency. Here is what I did. I used > your docker image build on Fedora29, which does a git and pulls down the > OSv repo. Then I built the python3x osv-app in osv-apps/python3x/ on the > github repo. > > > > I have python 3.7 installed on the fedora29 image and did pip3 install > numpy to get that installed in my python 3.7 environment. I modified the > GET file under the python3x directory to include the numpy module. > > > > Next I ran ./scripts/build image=python3x and the build completed > successfully. > > > > Then I ran ./scripts/run.py –api -e “/python3 -c \”import numpy; > print(numpy.__version__)\”” > > > > That is when it failed in an unresolved symbol for blas_memory_alloc > > > > [root@8ec8090052ab osv]# ./scripts/run.py --api -e "/python3 -c \"import > numpy; print(numpy.__version__)\"" > > OSv v0.53.0-74-gef56fde7 > > eth0: 192.168.122.15 > > Booted up in 611.56 ms > > /lib/python3.7/numpy/.libs/libopenblasp-r0-2ecf47d5.3.7.dev.so > <http://www.google.com/url?q=http%3A%2F%2Flibopenblasp-r0-2ecf47d5.3.7.dev.so&sa=D&sntz=1&usg=AFQjCNEzhQZhSHkxBkhVADvoaOnGZdTRaQ>: > > failed looking up symbol blas_memory_alloc > > > > [backtrace] > > 0x0000000040356209 <elf::object::symbol(unsigned int, bool)+969> > > 0x00000000403562cf <elf::object::resolve_pltgot(unsigned int)+127> > > 0x0000000040356494 <elf_resolve_pltgot+52> > > 0x000000004039b82f <???+1077524527> > > 0x000020000077ff6f <???+7864175> > > 0x0000000000000003 <???+3> > > > > Any ideas on how to resolve this? > > > > Thanks, > > > > Mike > > > > *From:* [email protected] <[email protected]> *On Behalf Of > *Waldek > Kozaczuk > *Sent:* Monday, August 26, 2019 6:03 PM > *To:* OSv Development <[email protected]> > *Subject:* Re: [osv-dev] Re: Python 3 Scikitlearn > > > > Is it possibly the same issue as described here - > https://github.com/cloudius-systems/osv/issues/1025? > > On Wednesday, August 21, 2019 at 10:08:15 PM UTC-4, Michael De Lucia wrote: > > I Waldek, > > > > I will get the exact command and stack trace to you on Friday. > > > > Thanks, > > > > Mike > > Sent from my iPhone > > > On Aug 21, 2019, at 9:39 PM, Waldek Kozaczuk <[email protected]> wrote: > > Hi, > > > > Can you provide exact build command you used to build the image? Also, can > you provide the exact stack trace so we know which symbol is was unresolved? > > > > Thanks, > > Waldek > > On Wednesday, August 21, 2019 at 9:19:36 PM UTC-4, [email protected] > wrote: > > Hi, > > > > I tried to use the OSv app for python 3 support. I built the image with > the system installed python 3 version. I also installed scikit learn and > added that to the modules to keep from the site packages. Scikit also > relies on numpy and scipy. However, I got it to import and build an image. > But when I tried to run it, failed with an unresolved symbol. I have a > centos system and was building within the docker container fedora 29 > template. Any suggestions or thoughts on why this would fail? > > > > Thanks, > > > > Mike > > -- > You received this message because you are subscribed to a topic in the > Google Groups "OSv Development" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/osv-dev/V9fch2WFX7o/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/osv-dev/61cb76c8-63f4-490c-b040-5634148c3930%40googlegroups.com > > <https://groups.google.com/d/msgid/osv-dev/61cb76c8-63f4-490c-b040-5634148c3930%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to a topic in the > Google Groups "OSv Development" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/osv-dev/V9fch2WFX7o/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/osv-dev/dba8eead-7d65-4f86-bef0-98eb80925d6d%40googlegroups.com > > <https://groups.google.com/d/msgid/osv-dev/dba8eead-7d65-4f86-bef0-98eb80925d6d%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to a topic in the > Google Groups "OSv Development" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/osv-dev/V9fch2WFX7o/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected] <javascript:>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/osv-dev/e1bdaf15-e259-4bad-8beb-15f81c4506f8%40googlegroups.com > > <https://groups.google.com/d/msgid/osv-dev/e1bdaf15-e259-4bad-8beb-15f81c4506f8%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/d7902b92-c058-4fcb-8db9-bcdaf0152b8a%40googlegroups.com.
