Hi Waldek, Any update on this issue? I implemented a extremely horrible sem_open and stubbed close and unlink, but I'm getting a mmap fault somewhere in LogisticRegression fit ( https://github.com/scikit-learn/scikit-learn/blob/1495f6924/sklearn/linear_model/logistic.py#L1466 ).
0x000000004046537b <osv::generate_signal(siginfo&, exception_frame*)+59> 0x00000000404653ea <osv::handle_mmap_fault(unsigned long, int, exception_frame*)+26> 0x000000004032f509 <mmu::vm_fault(unsigned long, exception_frame*)+185> 0x0000000040393a36 <page_fault+166> 0x0000000040392876 <???+1077487734> 0x0000100000c11e5f <???+12656223> On Wednesday, August 28, 2019 at 10:24:42 AM UTC-5, Waldek Kozaczuk wrote: > > Hi, > > Given you are the second person asking about it, I started implementing it > last night given it is not that difficult. But it would still take a bit of > time to make it correct and covered with unit tests. > > However, I peeked more at the details of > https://github.com/cloudius-systems/osv/issues/1025 and realized > scikit-learn uses the multiprocessing module ( > https://docs.python.org/3.4/library/multiprocessing.html?highlight=process) > which allows multiprocessing based on forking new processes vs > multithreading. OSv supports pthreads very well even on multiple cores > (SMP) but being unikernel it *does NOT and never intended to support > fork()/spawn()*. That being said I suspect that even if implemented name > semaphores you will hit the fork() obstacle down the road. > > Based on this doc - > https://scipy.github.io/old-wiki/pages/ParallelProgramming, and > https://scipy.github.io/old-wiki/pages/Cookbook/Multithreading.html you > can use SciPy with multithreading which should work on OSv. > > There is also GraalVM Python which is supposed to run python much faster > and is focused on SciPy but it is still experimental - > https://www.graalvm.org/docs/reference-manual/languages/python/. I have > had a lot of luck running native images (JVM code AOT-compiled to machine > code) on OSv. > > Regards, > Waldek > > On Tuesday, August 27, 2019 at 10:21:22 PM UTC-4, Michael De Lucia wrote: >> >> Waldek, >> >> >> >> I finally got sklearn installed and got to the same issue you pointed out >> described here - https://github.com/cloudius-systems/osv/issues/1025 >> >> >> >> Has this been implemented yet? >> >> >> >> Thanks, >> >> >> >> Mike >> >> >> >> *From:* [email protected] <[email protected]> *On Behalf Of >> *Waldek >> Kozaczuk >> *Sent:* Tuesday, August 27, 2019 11:40 AM >> *To:* OSv Development <[email protected]> >> *Subject:* Re: [osv-dev] Re: Python 3 Scikitlearn >> >> >> >> It looks like you have one of the libraries, >> libopenblasp-r0-2ecf47d5.3.7.dev.so >> <http://www.google.com/url?q=http%3A%2F%2Flibopenblasp-r0-2ecf47d5.3.7.dev.so&sa=D&sntz=1&usg=AFQjCNEzhQZhSHkxBkhVADvoaOnGZdTRaQ> >> depends >> on, missing. >> >> >> >> You can use: >> >> >> >> ./scripts/manifest_from_host.sh -l /lib/python3.7/numpy/.libs/ >> libopenblasp-r0-2ecf47d5.3.7.dev.so >> <http://www.google.com/url?q=http%3A%2F%2Flibopenblasp-r0-2ecf47d5.3.7.dev.so&sa=D&sntz=1&usg=AFQjCNEzhQZhSHkxBkhVADvoaOnGZdTRaQ> >> >> >> >> to see which libraries have to be added. >> >> >> On Tuesday, August 27, 2019 at 11:15:51 AM UTC-4, Michael De Lucia wrote: >> >> Hi Waldek, >> >> >> >> No that does not seem to be the problem. I started with just trying to >> see with numpy first, since that is a dependency. Here is what I did. I >> used your docker image build on Fedora29, which does a git and pulls down >> the OSv repo. Then I built the python3x osv-app in osv-apps/python3x/ on >> the github repo. >> >> >> >> I have python 3.7 installed on the fedora29 image and did pip3 install >> numpy to get that installed in my python 3.7 environment. I modified the >> GET file under the python3x directory to include the numpy module. >> >> >> >> Next I ran ./scripts/build image=python3x and the build completed >> successfully. >> >> >> >> Then I ran ./scripts/run.py –api -e “/python3 -c \”import numpy; >> print(numpy.__version__)\”” >> >> >> >> That is when it failed in an unresolved symbol for blas_memory_alloc >> >> >> >> [root@8ec8090052ab osv]# ./scripts/run.py --api -e "/python3 -c \"import >> numpy; print(numpy.__version__)\"" >> >> OSv v0.53.0-74-gef56fde7 >> >> eth0: 192.168.122.15 >> >> Booted up in 611.56 ms >> >> /lib/python3.7/numpy/.libs/libopenblasp-r0-2ecf47d5.3.7.dev.so >> <http://www.google.com/url?q=http%3A%2F%2Flibopenblasp-r0-2ecf47d5.3.7.dev.so&sa=D&sntz=1&usg=AFQjCNEzhQZhSHkxBkhVADvoaOnGZdTRaQ>: >> >> failed looking up symbol blas_memory_alloc >> >> >> >> [backtrace] >> >> 0x0000000040356209 <elf::object::symbol(unsigned int, bool)+969> >> >> 0x00000000403562cf <elf::object::resolve_pltgot(unsigned int)+127> >> >> 0x0000000040356494 <elf_resolve_pltgot+52> >> >> 0x000000004039b82f <???+1077524527> >> >> 0x000020000077ff6f <???+7864175> >> >> 0x0000000000000003 <???+3> >> >> >> >> Any ideas on how to resolve this? >> >> >> >> Thanks, >> >> >> >> Mike >> >> >> >> *From:* [email protected] <[email protected]> *On Behalf Of >> *Waldek >> Kozaczuk >> *Sent:* Monday, August 26, 2019 6:03 PM >> *To:* OSv Development <[email protected]> >> *Subject:* Re: [osv-dev] Re: Python 3 Scikitlearn >> >> >> >> Is it possibly the same issue as described here - >> https://github.com/cloudius-systems/osv/issues/1025? >> >> On Wednesday, August 21, 2019 at 10:08:15 PM UTC-4, Michael De Lucia >> wrote: >> >> I Waldek, >> >> >> >> I will get the exact command and stack trace to you on Friday. >> >> >> >> Thanks, >> >> >> >> Mike >> >> Sent from my iPhone >> >> >> On Aug 21, 2019, at 9:39 PM, Waldek Kozaczuk <[email protected]> wrote: >> >> Hi, >> >> >> >> Can you provide exact build command you used to build the image? Also, >> can you provide the exact stack trace so we know which symbol is was >> unresolved? >> >> >> >> Thanks, >> >> Waldek >> >> On Wednesday, August 21, 2019 at 9:19:36 PM UTC-4, [email protected] >> wrote: >> >> Hi, >> >> >> >> I tried to use the OSv app for python 3 support. I built the image with >> the system installed python 3 version. I also installed scikit learn and >> added that to the modules to keep from the site packages. Scikit also >> relies on numpy and scipy. However, I got it to import and build an image. >> But when I tried to run it, failed with an unresolved symbol. I have a >> centos system and was building within the docker container fedora 29 >> template. Any suggestions or thoughts on why this would fail? >> >> >> >> Thanks, >> >> >> >> Mike >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "OSv Development" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/osv-dev/V9fch2WFX7o/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/osv-dev/61cb76c8-63f4-490c-b040-5634148c3930%40googlegroups.com >> >> <https://groups.google.com/d/msgid/osv-dev/61cb76c8-63f4-490c-b040-5634148c3930%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "OSv Development" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/osv-dev/V9fch2WFX7o/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/osv-dev/dba8eead-7d65-4f86-bef0-98eb80925d6d%40googlegroups.com >> >> <https://groups.google.com/d/msgid/osv-dev/dba8eead-7d65-4f86-bef0-98eb80925d6d%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "OSv Development" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/osv-dev/V9fch2WFX7o/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/osv-dev/e1bdaf15-e259-4bad-8beb-15f81c4506f8%40googlegroups.com >> >> <https://groups.google.com/d/msgid/osv-dev/e1bdaf15-e259-4bad-8beb-15f81c4506f8%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/bbd783d5-4254-42ec-8df9-261f099bc1ad%40googlegroups.com.
