Hi Kaj, Well, your virtualenv looks correct. The packages are the correct ones. I assume all is good with the virtualenv. I noticed the following on your side that you have this constructor error I was talking about - did you edit the castro.py file that I listed in my previous thread or are you using the casper-astro/mlib_devel (casper-astro-soak-test branch which contains my castro.py edit fix)? Please send me a more complete screen capture from the time you run Matlab to when the error occurs. Also send me your startsg.local file, so that I can investigate.
I assume you did the following under "mlib_devel": 0) Make sure you are using my latest castro.py changes, as explained above and previous email thread - "5) You will need to edit line 32 of castro.py (located in the same folder as mlib_devel/jasper_library), so that it is "c = yaml.load(fh, Loader=yaml.Loader) . Refer to https://github.com/yaml/pyyaml/issues/266 <https://github.com/yaml/pyyaml/issues/266>for more info on this version change issue. I tested the build with my original working virtualenv and the newly generated virtualenv and it works for both. Don't worry, I will be committing this change once I have done a bit more investigation - maybe adding versions in requirements.txt is not a bad idea for future support". 1) Under "mlib_devel" ./startsg to set your environment and run Matlab. 2) You opened the slx file 3) You ran "jasper_frontend" in the Matlab command window. 4) You opened another terminal 5) In the new terminal under "mlib_devel" you typed "source startsg startsg.local". This sets up the environment variables using your new terminal. 6) You cut and paste the last line out of the matlab command window and you pasted it into the new terminal and then build. I think your issue according to your screen capture is that you don't have the castro.py mod I made. Please check and let me know. In terms of providing virtual environments. I think what we need to do is provide VMs without Matlab and Vivado licensing, so the users can just start. A few years back we investigated containers, which are much smaller hard disk wise. Alas, we had issues getting the container to work on other machines. VMs are bulky, but may be the best answer. We will discuss during the next CASPER meeting. Kaj, I think you are close! Hang in there :). Kind regards, Adam Isaacson South African Radio Astronomy Observatory (SARAO) Hardware Manager Cell: (+27) 825639602 Tel: (+27) 215067300 email: aisaac...@ska.ac.za On Tue, Feb 2, 2021 at 8:00 PM Kaj Wiik <kaj.w...@utu.fi> wrote: > Hi all, > > An idea: what if you shipped a working python venv as tgz with the > toolflow? > > Cheers, > Kaj > > On 02/02/2021 18:33, Kaj Wiik wrote: > > Hi Adam, > > > > The screen capture of ../casper-venv/lib64/python3.5/site-packages is > attached. > > > > When I opened the test_snap.slx, I got several > > "Warning: did not properly cleanup after previous model termination" > > errors. Maybe there are some leftovers in hidden dirs from the previous > incarnation of the Ubuntu system (homedir is not destroyed). I of course > removed all visible files and dirs... I think this is not related to the > problem below. > > > > I then started venv in another terminal, "cd ~test", from there "source > ../mlib_devel/startsg" and ran > > " > > /home/kjwiik/casper-venv/bin/python > /home/kjwiik/mlib_devel/jasper_library/exec_flow.py -m > /home/kjwiik/mlib_devel/jasper_library/test_models/test_snap.slx > --middleware --backend --software > > " > > I got unfortunately an error message and a number of deprecation > warnings, screen capture is attached. > > > > Thanks, > > Kaj > > > > On 2/1/21 6:29 PM, Adam Isaacson wrote: > >> Hi Kaj, > >> > >> Please can you go to your virtual environment directory: > ../casper-venv/libpython3.5/site-packages and do a "ls -la". It will list > all the packages installed. There should be colorlog, lxml, numpy, odict, > pip, pkg_resources,pyaml,setuptools and yaml python packages. Please can > you send me a screen capture, thanks? > >> > >> It looks like your pyyaml did eventually install correctly though at > the end - let's assume for now all is good with your virtual environment. > You may be able to continue trying to build your slx file in the toolflow > now. Try the next step. > >> > >> 1) > https://casper-toolflow.readthedocs.io/en/latest/src/Configuring-the-Toolflow.html > >> 2) > https://casper-toolflow.readthedocs.io/en/latest/src/Running-the-Toolflow.html > >> > >> Let me know what you get. > >> > >> Kind regards, > >> > >> Adam Isaacson > >> South African Radio Astronomy Observatory (SARAO) > >> Hardware Manager > >> Cell: (+27) 825639602 > >> Tel: (+27) 215067300 > >> email: aisaac...@ska.ac.za <mailto:aisaac...@ska.ac.za> > >> > >> > >> > >> > >> > >> On Mon, Feb 1, 2021 at 6:13 PM Kaj Wiik <kjw...@utu.fi <mailto: > kjw...@utu.fi>> wrote: > >> > >> Sorry, I sent a wrong screenshot, here's a correct one... > >> https://seafile.utu.fi/f/50b186da6c404a7db4ee > >> > >> Kaj > >> > >> On 2/1/21 6:05 PM, Kaj Wiik wrote: > >> > Hi all! > >> > > >> > Many thnks for looking into this, I have followed the emails over > >> the > >> > week but couldn't do any testing because of other commitments. > >> > > >> > I tried to follow the suggested procedure accurately but still > >> got an > >> > error. Well, in addition to that I ran update and had to 'apt-get > >> > install python3-venv' to get venv working. > >> > > >> > The error I got is 'Running setup.py bdist_wheel for PyYAML ... > >> error', > >> > see the attached screenshot for the full log (here's a link to > >> the log: > >> > https://seafile.utu.fi/f/a464c7711d6e441c8898/?dl=1). > >> > > >> > As I am not a python user (Julia, C and Perl only...), I have no > >> idea > >> > what wheels it is trying to turn ;-). > >> > > >> > Any ideas? > >> > > >> > Thanks again, > >> > Kaj > >> > > >> > > >> > On 1/29/21 12:42 PM, Adam Isaacson wrote: > >> >> Okay, so it turns out that creating the virtual environment for > >> python > >> >> using "virtualen -p python3 <name_of_env>" does not work. This > >> is what > >> >> you need to do in order to create a successful virtual > >> environment and > >> >> get your designs to build. I tested it on a brand new virtual > >> >> environment and it works. Thanks to Clifford van Wyk (Peralex), > >> Morag > >> >> Brown and Jack Hickish for their assistance. The docs will > >> definitely > >> >> need to be updated - we will add this to the agenda for the > CASPER > >> >> meeting. > >> >> > >> >> The following needs to be done if you want to generate a proper > >> >> virtual environment that will build in the toolflow - Kaj, I > >> recommend > >> >> the below way: > >> >> > >> >> 1) Create the virtual environment: "python3 -m venv > >> <some-venv-name>". > >> >> The old way of using virtualenv -p python 3 <some-venv-name> > just > >> >> doesn't work. Thanks to Jack and Morag for pointing this out to > me. > >> >> 2) Activate the environment: "source > <some-venv-name>/bin/activate" > >> >> 3) Go to the mlib_devel directory and edit the requirements.txt > >> file. > >> >> It should have "numpy<1.19" (thanks for your sleuth work, > >> Jack!). Now > >> >> save the file. > >> >> 4) Go to mlib_devel directory and type exactly: "pip install -r > >> >> requirements.txt". This will install without error or issues. if > >> you > >> >> check the site-packages in the virtual environment you will see > >> what I > >> >> mean - all the python packages will be installed properly in > your > >> >> virtual environment. > >> >> 5) You will need to edit line 32 of castro.py (located in the > same > >> >> folder as mlib_devel/jasper_library), so that it is "c = > >> yaml.load(fh, > >> >> Loader=yaml.Loader) . Refer to > >> >> https://github.com/yaml/pyyaml/issues/266 > >> >> <https://github.com/yaml/pyyaml/issues/266>for more info on > this > >> >> version change issue. I tested the build with my original > working > >> >> virtualenv and the newly generated virtualenv and it works for > >> both. > >> >> Don't worry, I will be committing this change once I have done a > >> bit > >> >> more investigation - maybe adding versions in requirements.txt > >> is not > >> >> a bad idea for future support. > >> >> > >> >> You can now start your matlab session and build your designs :). > >> Hooray! > >> >> > >> > > >> > >> -- You received this message because you are subscribed to the > Google > >> Groups "casper@lists.berkeley.edu > >> <mailto:casper@lists.berkeley.edu>" group. > >> To unsubscribe from this group and stop receiving emails from it, > >> send an email to casper+unsubscr...@lists.berkeley.edu > >> <mailto:casper%2bunsubscr...@lists.berkeley.edu>. > >> To view this discussion on the web visit > >> > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/4f852e11-4725-c8e8-03fe-d84add6a4198%40utu.fi > . > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups "casper@lists.berkeley.edu" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an email to casper+unsubscr...@lists.berkeley.edu <mailto: > casper+unsubscr...@lists.berkeley.edu>. > >> To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CADTJ%3DnHacLga2eCWexYa5RChnq-pJ3muqWb_rQx%3Dd3W3Jk-dkw%40mail.gmail.com > < > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CADTJ%3DnHacLga2eCWexYa5RChnq-pJ3muqWb_rQx%3Dd3W3Jk-dkw%40mail.gmail.com?utm_medium=email&utm_source=footer > >. > > > > -- > You received this message because you are subscribed to the Google Groups " > casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/813c8f59-3174-dc55-d285-c58b64dc3b20%40utu.fi > . > -- You received this message because you are subscribed to the Google Groups "casper@lists.berkeley.edu" group. To unsubscribe from this group and stop receiving emails from it, send an email to casper+unsubscr...@lists.berkeley.edu. To view this discussion on the web visit https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CADTJ%3DnH16om9%2BZ%2BSDic8LdWZ9_WPK_H6xNY1Zd-de74LcT6A3w%40mail.gmail.com.
