Hi David, Yes, it was a bit of a rabbit hole exercise, but we (between Kaj and Peralex) worked our way through it and now debugging other issues :).
You raise a valid point. There is no reason we shouldn't port to Ubuntu 18.04LTS or higher. It makes sense with what Kaj and Peralex are experiencing. Please note, my team (SARAO) is baselining the SKARAB work at Ubuntu 16.04LTS. We are coming to the end of the MeerKAT development for the F and X engines. I do believe there are those that have got the toolflow to work on Ubuntu 18.04LTS. I know Wesley New has struggled to get the SKARAB toolflow to work using Ubuntu 18.04LTS. I haven't tried it on my side yet. My team's focus this year is to update the toolflow to work with Vitis and target those Alveo boards. This will definitely involve moving to a newer version of Ubuntu. For now that is likely to be Ubuntu 18.04LTS, Vitis 2020.2 and Vivado 2020.2. I think this is something we should discuss at the next CASPER meeting on the 11th Feb. We should allocate a resource for this, but SARAO is focusing on other areas for now. Thanks for your email :). Kind regards, Adam Isaacson South African Radio Astronomy Observatory (SARAO) Hardware Manager Cell: (+27) 825639602 Tel: (+27) 215067300 email: aisaac...@ska.ac.za On Thu, Jan 28, 2021 at 12:09 AM David MacMahon <dav...@berkeley.edu> wrote: > Wow, thanks for these truly awesome forensics, Adam! It sounds like you > went down a rabbit hole and lived to tell the tale. I'm sure many of us > will benefit from these details. > > Sorry if this is a FAQ, but what are the prospects for moving the tool > flow beyond Ubuntu 16.04? That release is really starting to show its age, > especially since its stock Python3 is only 3.5. (Though to be fair, Python > is at least partly to blame due to its poor track record on version > compatibility. :P) > > Cheers, > Dave > > On Jan 27, 2021, at 13:00, Adam Isaacson <aisaac...@ska.ac.za> wrote: > > Hi Kaj, > > I am including the CASPER community in this email thread as it applies to > everyone. > > Interesting, so I have run into another person with the same virtualenv > install issue that you encountered as shown in red below. I have been > helping him debug too on a new machine and I am pleased to say that his > virtualenv is now working. It seems like the new Ubuntu 16.04LTS installs > come with a newer version of virtualenv which is not compatible with python > 3.5 - fun, fun, fun. This definitely applies to all Casperites who are > installing the toolflow on their new machines. > > kjwiik@casperx:~/work$ virtualenv -p python3 casper_venv > Traceback (most recent call last): > File "/usr/local/bin/virtualenv", line 7, in <module> > from virtualenv.__main__ import run_with_catch > File > "/home/kjwiik/.local/lib/python3.5/site-packages/virtualenv/__init__.py", > line 3, in <module> > from .run import cli_run, session_via_cli > File > "/home/kjwiik/.local/lib/python3.5/site-packages/virtualenv/run/__init__.py", > line 13, in <module> > from .plugin.activators import ActivationSelector > File > "/home/kjwiik/.local/lib/python3.5/site-packages/virtualenv/run/plugin/activators.py", > line 6, in <module> > from .base import ComponentBuilder > File > "/home/kjwiik/.local/lib/python3.5/site-packages/virtualenv/run/plugin/base.py", > line 9, in <module> > from importlib_metadata import entry_points > File > "/home/kjwiik/.local/lib/python3.5/site-packages/importlib_metadata/__init__.py", > line 88 > dist: Optional['Distribution'] = None > ^ > SyntaxError: invalid syntax > kjwiik@casperx:~/work$ > > I want you to check the following versions for me: > > 1) virtualenv -> type "virtualenv --version" at the prompt. You should get > 16.7.5. If not then you will need to uninstall the virtualenv by typing: > "pip3 uninstall virtualenv" and then we will need to install the 16.7.5 > version. To install an exact version then type: "pip3 install > virtualenv==16.7.5". The use of "sudo" with "-H" arguments may or may not > be needed. The installs will guide you. > > 2) What is your Ubuntu 16.04LTS version? -> type "lsb_release -a". I am > using Ubuntu 16.04.7 LTS, Xenial. > > 3) What is your pip3 version? -> type "pip3 --version". I am using pip > 8.1.1. > > 4) What is your python version? -> type: "python --version". I am using > python version 2.7.12 > > 5) What is your python3 version? -> type: "python3 --version". I am using > python3 version 3.5.2. > > Please make sure all these versions are the same before continuing. I > suspect that the latest Ubuntu 16.04LTS installs have made some upgrades > that are incompatible. > > Once these versions are as above then try the following step to create the > virtual env: > > 1) Type: "virtualenv -p python3 <name_of_virtual_env>". This should create > a folder with the same name as "name_of_virtual_env". There should be no > issues with the install. It should complete without issues. > 2) To activate the virtual environment -> type: "source > <path_to_folder>/<name_of_virtual_env>/bin/activate > 3) Once activated then you can deactivate it by typing "deactivate". > 4) If the virtual environment is activated then if you type "python" you > should get version 3.5.2 > 5) If the virtual environment is deactivated then if you type "python" you > should get 2.7.12. > > I think if you can achieve this then you are ready to install the toolflow > by following these steps: > > 1) > https://casper-toolflow.readthedocs.io/en/latest/src/Installing-the-Toolflow.html# > 2) > https://casper-toolflow.readthedocs.io/en/latest/src/How-to-install-Matlab.html > 3) > https://casper-toolflow.readthedocs.io/en/latest/src/How-to-install-Xilinx-Vivado.html > 4) > https://casper-toolflow.readthedocs.io/en/latest/src/Configuring-the-Toolflow.html > 5) > https://casper-toolflow.readthedocs.io/en/latest/src/Running-the-Toolflow.html > 6) https://casper-toolflow.readthedocs.io/projects/casperfpga/en/latest/ (This > is how to install casperfpga) > > @Jonathon Kocz <jxk...@gmail.com> : we will need to update the docs to > reflect this new information. I will wait until both parties have working > systems before updating the ReadtheDocs. One of the parties is using a > proxy and has some additional information on how to install with a proxy > that will be useful for other casperites. > > I hope this helps! > > Kind regards, > > Adam Isaacson > South African Radio Astronomy Observatory (SARAO) > Hardware Manager > Cell: (+27) 825639602 > Tel: (+27) 215067300 > email: aisaac...@ska.ac.za > > > > On Sun, Jan 24, 2021 at 8:09 PM Adam Isaacson <aisaac...@ska.ac.za> wrote: > >> Hi Kaj, >> >> Are you running a python virtual environment using python 3.5? Once you >> start the virtual environment and type python you should see version 3.5.2 >> or something like that. It doesn't look like it. Here is some documentation >> that should be helpful - make sure you follow these steps: >> >> 1) How to configure the toollflow: >> https://casper-toolflow.readthedocs.io/en/latest/src/Configuring-the-Toolflow.html >> 2) Running the toolflow: >> https://casper-toolflow.readthedocs.io/en/latest/src/Running-the-Toolflow.html >> >> Does your startsg.local file look like this: >> >> export XILINX_PATH=/opt/Xilinx/Vivado/2019.1export >> MATLAB_PATH=/usr/local/MATLAB/R2018aexport PLATFORM=lin64export >> JASPER_BACKEND=vivado >> # over-ride the MATLAB libexpat version with the OS's one.# Using >> LD_PRELOAD=${LD_PRELOAD}:"..." rather than just LD_PRELOAD="..."# ensures >> that we preserve any other settings already configuredexport >> LD_PRELOAD=${LD_PRELOAD}:"/usr/lib/x86_64-linux-gnu/libexpat.so" >> # Activate a custom python environment on loadexport >> CASPER_PYTHON_VENV_ON_START=/home/user/work/casper_venv >> >> >> You must run everything from the virtual environment in order to work. >> The last line in the startsg.local will automatically start your virtual >> environment session, but you can also do it manually e.g. source >> ~/venv/bin/activate will give you the following "(venv) >> aisaacson@adam-cm:~/work/git_work/ska-sa/2019_1/mlib_devel$". >> The red (venv) tells you that you are in the virtual python environment. >> Remember you also need to install requirements.txt while the virtual >> environment is running. >> >> Then all you really need to do is run "./startsg" at the prompt and it >> should open up Matlab where you can build your slx file by typing in >> "jasper" or "jasper_frontend. if you use "jasper_frontend" then it will >> just compile sysgen and the simulink frontend. You can then open another >> terminal session and under "mlib_devel" type "source startsg >> startsg.local". This will make sure everything is in your path. You can >> then copy and paste the exec_flow.py command from the Matlab command window >> and then it should build the middle end and backend. Remember to start the >> virtual environment for this terminal window as well. This should all be in >> the above docs, but if you are still struggling then please send us a step >> by step terminal screen capture, so we can see if you are doing everything >> correctly, thanks. >> >> I hope this helps! >> >> Kind regards, >> >> Adam Isaacson >> South African Radio Astronomy Observatory (SARAO) >> Hardware Manager >> Cell: (+27) 825639602 >> Tel: (+27) 215067300 >> email: aisaac...@ska.ac.za >> >> >> >> On Sun, Jan 24, 2021 at 7:15 PM Kaj Wiik <kjw...@utu.fi> wrote: >> >>> Hi! >>> >>> A followup... >>> >>> I was able to start (but not run, see below...) the toolflow with the >>> following modifications: >>> >>> - to run jasper_frontend, I had to install python-yaml. I tried first to >>> >>> add python3-yaml but it seems that the toolflow is mixing python2 and >>> python3? >>> >>> - I added 'export LM_LICENSE_FILE=/opt/Xilinx/Xilinx.lic' and >>> 'export MLIB_DEVEL_PATH=~/mlib_devel' to startsg.local >>> >>> When matlab gave a command line and I run it, I got the following error >>> message that I do not (yet!) understand: >>> >>> kjwiik@casperx:~/tmp$ source ../mlib_devel/startsg.local >>> kjwiik@casperx:~/tmp$ /usr/bin/python >>> /home/kjwiik/mlib_devel/jasper_library/exec_flow.py -m >>> /home/kjwiik/mlib_devel/jasper_library/test_models/test_zcu111.slx >>> --middleware --backend --software >>> Starting compile >>> Starting Toolflow! >>> Frontend is simulink >>> Setting compile directory: >>> /home/kjwiik/mlib_devel/jasper_library/test_models/test_zcu111 >>> /home/kjwiik/mlib_devel/jasper_library/platforms/zcu111.yaml >>> {'backend_target': 'vivado', 'name': 'zcu111', 'fpga': >>> 'xczu28dr-ffvg1517-2-e', 'mmbus_architecture': 'AXI4-Lite', 'sources': >>> [], 'cfgbvs': 'GND', 'provides': ['sys_clk'], 'config_voltage': 1.8, >>> 'pins': {'clk_100_p': {'loc': 'AM15', 'iostd': 'LVDS'}, 'clk_100_n': >>> {'loc': 'AN15', 'iostd': 'LVDS'}, 'led': {'loc': ['AR13', 'AP13', >>> 'AR16', 'AP16', 'AP15', 'AN16', 'AN17', 'AV15'], 'iostd': 'LVCMOS18'}}, >>> 'mmbus_base_address': 2684354560, 'mmbus_address_alignment': 4, >>> 'manufacturer': 'Xilinx', 'constraints': []} >>> sw_reg >>> Traceback (most recent call last): >>> File "/home/kjwiik/mlib_devel/jasper_library/exec_flow.py", line 197, >>> >>> in <module> >>> tf.gen_periph_objs() >>> File "/home/kjwiik/mlib_devel/jasper_library/toolflow.py", line 363, >>> in gen_periph_objs >>> self.peripherals[pk], self.plat)) >>> File >>> "/home/kjwiik/mlib_devel/jasper_library/yellow_blocks/yellow_block.py", >>> line 65, in make_block >>> return cls(blk,platform,hdl_root=hdl_root) >>> File >>> "/home/kjwiik/mlib_devel/jasper_library/yellow_blocks/yellow_block.py", >>> line 92, in __init__ >>> self.hdl_root = os.getenv('HDL_ROOT').rstrip('/') #: The base >>> directory from which source file's locations are specified >>> AttributeError: 'NoneType' object has no attribute 'rstrip' >>> ----------------------------------------------------------------------- >>> >>> Could this be again a python version problem? >>> >>> Thanks, >>> Kaj >>> >>> On 1/22/21 6:50 PM, Kaj Wiik wrote: >>> > Hi! >>> > >>> > I tried a bit different route: I installed python3-numpy and >>> > python3-setuptools-git Ubuntu packages first and commented out numpy >>> > from requirements.txt. At least the installation went fine using these >>> >>> > workarounds. >>> > >>> > About ZCU111 Slack channel, I am very interested (and my colleague >>> > derek.mc...@utu.fi also), could you please send us an invitation? >>> > >>> > Many thanks for your kind help! >>> > >>> > Kaj >>> > >>> > On 1/20/21 10:22 PM, Dan Werthimer wrote: >>> >> >>> >> hi kaj, >>> >> >>> >> regarding your interest in ZCU111: >>> >> there's a casper slack channel on RFSOC that you might find useful. >>> >> wei liu recently developed a casper ADC yellow block for the ZCU111. >>> >> >>> >> best wishes, >>> >> >>> >> dan >>> >> >>> >> >>> >> >>> >> Dan Werthimer >>> >> Marilyn and Watson Alberts Chair >>> >> Astronomy Dept and Space Sciences Lab >>> >> University of California, Berkeley >>> >> >>> >> >>> >> On Wed, Jan 20, 2021 at 12:17 PM Kaj Wiik <kaj.w...@utu.fi >>> >> <mailto:kaj.w...@utu.fi>> wrote: >>> >> >>> >> >>> >> >>> >> On 20/01/2021 18:49, Jack Hickish wrote: >>> >> >>> >> > I've been using Ubuntu 18.04 LTS without issues, at least with >>> >> the boards which use Vivado 2019. I'm not sure what versions of >>> >> python the OS came with, but I'm currently running the toolflow >>> in a >>> >> python 3.6.9 virtual env. >>> >> >>> >> I first tried with Ubuntu 18.04 LTS but I got stuck with a problem >>> >> with Matlab and some system library versions. >>> >> >>> >> I think I'll change the requirements.txt file to read "numpy<1.19" >>> >> and try with that first. >>> >> >>> >> I am very interested of all information and experiences with >>> Casper >>> >> on ZCU111. Thanks to Adam for pointing out of a mlib_devel ZCU111 >>> >> branch, I haven't noticed that! >>> >> >>> >> Thanks! >>> >> >>> >> Kaj >>> >> >>> >> -- You received this message because you are subscribed to the >>> >>> >> Google >>> >> Groups "casper@lists.berkeley.edu >>> >> <mailto:casper@lists.berkeley.edu>" group. >>> >> To unsubscribe from this group and stop receiving emails from it, >>> >> send an email to casper+unsubscr...@lists.berkeley.edu >>> >> <mailto:casper%2bunsubscr...@lists.berkeley.edu>. >>> >> To view this discussion on the web visit >>> >> >>> >> >>> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/020a6f84-ded9-49cc-2d4a-00a766e743be%40utu.fi >>> . >>> >> >>> >> >>> >> -- >>> >> You received this message because you are subscribed to the Google >>> >> Groups "casper@lists.berkeley.edu" group. >>> >> To unsubscribe from this group and stop receiving emails from it, send >>> >>> >> an email to casper+unsubscr...@lists.berkeley.edu >>> >> <mailto:casper+unsubscr...@lists.berkeley.edu>. >>> >> To view this discussion on the web visit >>> >> >>> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAGHS_vHQ1bTha%3DFjCe0sGddd3Gt6GrYkh8dgz_eKDWXebf87iQ%40mail.gmail.com >>> >>> >> < >>> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAGHS_vHQ1bTha%3DFjCe0sGddd3Gt6GrYkh8dgz_eKDWXebf87iQ%40mail.gmail.com?utm_medium=email&utm_source=footer >>> >. >>> >> >>> > >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "casper@lists.berkeley.edu" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to casper+unsubscr...@lists.berkeley.edu. >>> To view this discussion on the web visit >>> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/335be9c2-39ca-d9f9-fb8f-960076bfffa1%40utu.fi >>> . >>> >> > -- > You received this message because you are subscribed to the Google Groups " > casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CADTJ%3DnGgmuxdVex74bZPBOn0kpwxMO4rPrUyGcu_PPHvgvU8XA%40mail.gmail.com > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CADTJ%3DnGgmuxdVex74bZPBOn0kpwxMO4rPrUyGcu_PPHvgvU8XA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > > -- > You received this message because you are subscribed to the Google Groups " > casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/34A86FAB-52C2-4923-83A3-B14BD83A8F2A%40berkeley.edu > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/34A86FAB-52C2-4923-83A3-B14BD83A8F2A%40berkeley.edu?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "casper@lists.berkeley.edu" group. To unsubscribe from this group and stop receiving emails from it, send an email to casper+unsubscr...@lists.berkeley.edu. To view this discussion on the web visit https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CADTJ%3DnFFsXT--%3DnkCGMJdOgaWXBner4dB%2B%2Bp%2BbSj7evXALeUMA%40mail.gmail.com.
