I just realized I turned off my computer since I'll be traveling, and my test run of sst can't finish if the computer is off :-P. I have not completed a run of your example command, but it starts and I have no reason to believe it won't finish. I would still suggest running it yourself (with my changes) to verify that things are fixed. I need to wake up to go to the airport in 4 hours, so I won't be able to do anything non-trivial with this until Friday at the earliest.
Gabe On Thu, Dec 16, 2021 at 3:29 AM Gabe Black <[email protected]> wrote: > https://gem5-review.googlesource.com/c/public/gem5/+/54325/1 > > On Wed, Dec 15, 2021 at 11:12 PM Gabe Black <[email protected]> wrote: > >> There will be a couple more patches coming, since the systemc subsystem >> has problems when python is already running when static initializers run. I >> had been building without systemc enabled to simplify things earlier, but I >> need to get it running too. That should be a fairly quick fix. >> >> Gabe >> >> On Wed, Dec 15, 2021 at 10:40 PM Hoa Nguyen <[email protected]> >> wrote: >> >>> Hi Gabe, >>> >>> Thanks for uploading the changes. Typically it'll take less than an hour >>> for RISC-V. >>> >>> Regards, >>> Hoa Nguyen >>> >>> On Wed, Dec 15, 2021 at 10:38 PM Gabe Black <[email protected]> >>> wrote: >>> >>>> I have it running, although it's taking a while. How long should it >>>> take to finish? Should it finish immediately, 15 minutes, a few hours, >>>> tomorrow...? I'll upload my changes in the mean time. >>>> >>>> Gabe >>>> >>>> On Wed, Dec 15, 2021 at 9:13 PM Gabe Black <[email protected]> >>>> wrote: >>>> >>>>> Ok, I did/am doing a little more looking, and part of the problem >>>>> seems to be that some of the python blobs executed with executePython >>>>> assume prior blobs have done imports for them already. That was where the >>>>> modules or their contents not being found was coming from, and then after >>>>> that something about the error reporting was blowing up, but that wasn't >>>>> the primary problem. >>>>> >>>>> With that fixed, it's still upset, but now it's because "fatal: Need >>>>> to instantiate Root() before calling instantiate()". Also, gem5 is >>>>> printing >>>>> the help blob which seems wrong, so maybe the command line arguments >>>>> aren't >>>>> making it through like they should? It could be that no config script is >>>>> run, and that's why m5.instantiate() doesn't work. >>>>> >>>>> It's also a little weird that there's an ordering dependency between >>>>> the execPythonCommands invocations, considering the same set of those >>>>> should still be happening in the same order. I suppose I'm not explicitly >>>>> importing the m5 module into __main__.__dict__ to run m5.main(), so that >>>>> may be where that's coming from. >>>>> >>>>> Anyway, I'm still chopping away at this, but I think it's pretty close. >>>>> >>>>> Gabe >>>>> >>>>> On Wed, Dec 15, 2021 at 8:29 PM Gabe Black <[email protected]> >>>>> wrote: >>>>> >>>>>> Also, I have to say, while I'm grateful that there was a push to get >>>>>> the SCons changes reviewed for the release, we wouldn't be in this crunch >>>>>> if they had been reviewed a month ago. It would be best not to let things >>>>>> bunch up and then make a herculean push right at the deadline, where >>>>>> problems have a very limited time to get resolved. >>>>>> >>>>>> Gabe >>>>>> >>>>>> On Wed, Dec 15, 2021 at 4:38 PM Gabe Black <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> That's actually very helpful, since I was pretty sure there was a >>>>>>> memory problem but couldn't figure out where. I think there's a good >>>>>>> chance >>>>>>> this will be an easy fix. >>>>>>> >>>>>>> Gabe >>>>>>> >>>>>>> On Wed, Dec 15, 2021, 8:59 AM Jason Lowe-Power <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Ok, I was able to get a debugger to work and I dug in a bit more... >>>>>>>> >>>>>>>> The problem is that the new code does not initialize the >>>>>>>> __main__.py module. The original SST code had the following line: >>>>>>>> `pythonMain = PyImport_AddModule(PyCC("__main__"));` >>>>>>>> >>>>>>>> This was executed before `execPythonCommands`. >>>>>>>> >>>>>>>> In `execPythonCommands` we use the `pythonMain` variable. However, >>>>>>>> in the *new* code, that variable is NULL (or unitialized, actually). >>>>>>>> >>>>>>>> As far as I can tell, all of the new python changes didn't come >>>>>>>> with any documentation on how you are supposed to interact with Python >>>>>>>> from >>>>>>>> gem5. So, there's no good way for us to update this code. >>>>>>>> >>>>>>>> Gabe, if you can update the `execPythonCommands` function to work >>>>>>>> however you expected it to, that would be great. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jason >>>>>>>> >>>>>>>> On Wed, Dec 15, 2021 at 10:45 AM Jason Lowe-Power < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Here's my understanding of where we are on this: >>>>>>>>> >>>>>>>>> As of >>>>>>>>> https://gem5.googlesource.com/public/gem5/+/cf7ce21848ea4aeee28737823e6e768f9a14ceaf >>>>>>>>> SST was working. (Committed Dec. 7th) >>>>>>>>> >>>>>>>>> Then, this large relation chain was pushed: >>>>>>>>> https://gem5-review.googlesource.com/c/public/gem5/+/49425 ending >>>>>>>>> with >>>>>>>>> https://gem5.googlesource.com/public/gem5/+/4c1422e3ba780a5f152426e214b4be3d45b751e4. >>>>>>>>> This relation chain broke SST. This was committed on Dec. 8th. >>>>>>>>> >>>>>>>>> We have two options: Either we fix this bug ASAP (as in within 6 >>>>>>>>> days so we can release on 12/22) or we revert the changesets that were >>>>>>>>> committed on the 8th. >>>>>>>>> >>>>>>>>> SST is a *must have* feature of gem5 21.2. It is quite frustrating >>>>>>>>> to see it broken after all of the hard work that Hoa and Giacomo put >>>>>>>>> into >>>>>>>>> it. >>>>>>>>> >>>>>>>>> Let me know if I'm missing something. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Jason >>>>>>>>> >>>>>>>>> On Sat, Dec 11, 2021 at 10:35 AM Gabe Black <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> https://gem5-review.googlesource.com/c/public/gem5/+/54006 >>>>>>>>>> >>>>>>>>>> At least one problem I ran into was that the order of static >>>>>>>>>> initializers was not determinstic, so the structures which said what >>>>>>>>>> embedded python modules existed might be statically constructed >>>>>>>>>> after the >>>>>>>>>> thing that consumes and sets them up. That could work by coincidence >>>>>>>>>> if the >>>>>>>>>> order is favorable, but it wasn't for me and so lots of things >>>>>>>>>> didn't exist >>>>>>>>>> as far as the python was concerned. >>>>>>>>>> >>>>>>>>>> To be clear, there is some mysterious deeper issue here which is >>>>>>>>>> probably separate from that. As far as I can tell running gem5 >>>>>>>>>> itself is >>>>>>>>>> working fine, but when running as a library in sst there is some >>>>>>>>>> sort of >>>>>>>>>> memory corruption or premature garbage collection or something, and >>>>>>>>>> it >>>>>>>>>> leads to a segfault or incorrect behavior (failure to import modules, >>>>>>>>>> failure to find things inside them, etc). Things were even more >>>>>>>>>> tempermental until I added what feels like an unnecessary increment >>>>>>>>>> of the >>>>>>>>>> reference count for the modules in the new pybind_init.hh, which >>>>>>>>>> seemed to >>>>>>>>>> tame things in gem5 proper at least. I'm too tired to keep looking >>>>>>>>>> for it >>>>>>>>>> right now, but hopefully this will give you a good leg up. >>>>>>>>>> >>>>>>>>>> Gabe >>>>>>>>>> >>>>>>>>>> On Sat, Dec 11, 2021 at 8:25 AM Jason Lowe-Power < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Everything was working as of last week-ish, if I remember >>>>>>>>>>> correctly. We were booting both Arm and RISCV full system with gem5 >>>>>>>>>>> cores >>>>>>>>>>> and SST caches/memory. >>>>>>>>>>> >>>>>>>>>>> Cheers, >>>>>>>>>>> Jason >>>>>>>>>>> >>>>>>>>>>> On Sat, Dec 11, 2021, 8:23 AM Gabe Black <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Thanks, that was really helpful. I've fixed the compilation >>>>>>>>>>>> problems, and then a few more that were hiding behind that, >>>>>>>>>>>> including a few >>>>>>>>>>>> ones that were intrinsic to the way sst was initializing the gem5 >>>>>>>>>>>> library >>>>>>>>>>>> (with dlopen). Was this working to begin with and all this >>>>>>>>>>>> breakage is new, >>>>>>>>>>>> or was it just that the compilation broke and there were these >>>>>>>>>>>> other >>>>>>>>>>>> problems already? I want to make sure I don't leave things in a >>>>>>>>>>>> worse state >>>>>>>>>>>> than when I got here, but I also don't want to go chasing down >>>>>>>>>>>> things that >>>>>>>>>>>> weren't working to begin with. >>>>>>>>>>>> >>>>>>>>>>>> I'm going to upload what I have so far, and hopefully someone >>>>>>>>>>>> else can take a look at it too. The current problem seems to be >>>>>>>>>>>> that the >>>>>>>>>>>> modules in _m5 aren't working right, either because they've been >>>>>>>>>>>> garbage >>>>>>>>>>>> collected, or because they're not being set up properly, or... >>>>>>>>>>>> >>>>>>>>>>>> Gabe >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Dec 10, 2021 at 8:17 PM Bobby Bruce <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hey gabe. >>>>>>>>>>>>> >>>>>>>>>>>>> No idea if this is the _best_ solution to your problem, but my >>>>>>>>>>>>> solution would be to rebuild the image with this installed. >>>>>>>>>>>>> Modify the >>>>>>>>>>>>> `util/dockerfiles/sst-11.1.0/Dockerfile` to the environment you >>>>>>>>>>>>> want. Then >>>>>>>>>>>>> run `docker build -t <some name here> >>>>>>>>>>>>> util/dockerfiles/sst-11.1.0` to build >>>>>>>>>>>>> an image with the name "<some name here>". >>>>>>>>>>>>> >>>>>>>>>>>>> Then you can execute `docker run -u $UID:$GID --volume >>>>>>>>>>>>> $(pwd):$(pwd) -w $(pwd) -it <same name here>` within the gem5 >>>>>>>>>>>>> directory to >>>>>>>>>>>>> spin up and enter a container from the image you just built. I >>>>>>>>>>>>> think you'll >>>>>>>>>>>>> be able to do what you want inside the container. >>>>>>>>>>>>> >>>>>>>>>>>>> Kind regards, >>>>>>>>>>>>> Bobby >>>>>>>>>>>>> -- >>>>>>>>>>>>> Dr. Bobby R. Bruce >>>>>>>>>>>>> Room 3050, >>>>>>>>>>>>> Kemper Hall, UC Davis >>>>>>>>>>>>> Davis, >>>>>>>>>>>>> CA, 95616 >>>>>>>>>>>>> >>>>>>>>>>>>> web: https://www.bobbybruce.net >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Dec 10, 2021 at 7:24 PM Gabe Black < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> > >>>>>>>>>>>>> > Dumb question: I'm trying to run gdb inside this container >>>>>>>>>>>>> on the sst thing. How do I do that? It's not installed in the >>>>>>>>>>>>> container >>>>>>>>>>>>> now, and I can't (easily) figure out how to get it installed. I >>>>>>>>>>>>> can tell >>>>>>>>>>>>> docker to install it, but then it seems to throw that away as >>>>>>>>>>>>> soon as the >>>>>>>>>>>>> command ends. >>>>>>>>>>>>> > >>>>>>>>>>>>> > Gabe >>>>>>>>>>>>> > >>>>>>>>>>>>> > On Fri, Dec 10, 2021 at 5:09 PM Bobby Bruce < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> Thanks Gabe, >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> This is very much appreciated. I'm going to create the >>>>>>>>>>>>> release staging once a couple more things get in. Feel free to >>>>>>>>>>>>> push any >>>>>>>>>>>>> patches related to these bugs to the release staging branch. >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> If there is an order of priority I'd say the bug affecting >>>>>>>>>>>>> SST is of higher importance than that affecting the Weeklies (as >>>>>>>>>>>>> far as I >>>>>>>>>>>>> can see the latter is hard to trigger). That being said, we'll >>>>>>>>>>>>> apply both >>>>>>>>>>>>> to the new release one way or another. >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> -- >>>>>>>>>>>>> >> Dr. Bobby R. Bruce >>>>>>>>>>>>> >> Room 3050, >>>>>>>>>>>>> >> Kemper Hall, UC Davis >>>>>>>>>>>>> >> Davis, >>>>>>>>>>>>> >> CA, 95616 >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> web: https://www.bobbybruce.net >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> On Fri, Dec 10, 2021 at 5:02 PM Gabe Black < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>> >>>>>>>>>>>>> >>> Hi Bobby, not yet, I meant to look into this for the last >>>>>>>>>>>>> couple days but kept running out of time. I'm sitting down to >>>>>>>>>>>>> work on it >>>>>>>>>>>>> right now. >>>>>>>>>>>>> >>> >>>>>>>>>>>>> >>> Gabe >>>>>>>>>>>>> >>> >>>>>>>>>>>>> >>> On Fri, Dec 10, 2021 at 1:21 PM Bobby Bruce < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> >>>> Hey Gabe, >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> >>>> Is there any update on this? >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> >>>> Kind regards, >>>>>>>>>>>>> >>>> Bobby >>>>>>>>>>>>> >>>> -- >>>>>>>>>>>>> >>>> Dr. Bobby R. Bruce >>>>>>>>>>>>> >>>> Room 3050, >>>>>>>>>>>>> >>>> Kemper Hall, UC Davis >>>>>>>>>>>>> >>>> Davis, >>>>>>>>>>>>> >>>> CA, 95616 >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> >>>> web: https://www.bobbybruce.net >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> >>>> On Wed, Dec 8, 2021 at 5:51 PM Hoa Nguyen via gem5-dev < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> >>>>> Hi Gabe, >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> >>>>> I have more details about this. In this use case, SST >>>>>>>>>>>>> initialized the >>>>>>>>>>>>> >>>>> Python environment before adding the "gem5 object". This >>>>>>>>>>>>> gem5 object >>>>>>>>>>>>> >>>>> will add more Python stuff from gem5 to the environment. >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> >>>>> The function that does that is initPython() >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> https://gem5.googlesource.com/public/gem5/+/refs/heads/develop/ext/sst/gem5.cc#415 >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> >>>>> The following commands will pull the docker image for >>>>>>>>>>>>> SST testing >>>>>>>>>>>>> >>>>> purposes (note that host_gem5_root and guest_gem5_root >>>>>>>>>>>>> must be >>>>>>>>>>>>> >>>>> specified), >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> >>>>> ``` >>>>>>>>>>>>> >>>>> docker run -u $UID:$GID --volume >>>>>>>>>>>>> "${host_gem5_root}":"${guest_gem5_root}" -w \ >>>>>>>>>>>>> >>>>> "${guest_gem5_root}" --rm gcr.io/gem5-test/sst-env \ >>>>>>>>>>>>> >>>>> bash -c "\ >>>>>>>>>>>>> >>>>> scons build/RISCV/libgem5_opt.so -j${nproc} >>>>>>>>>>>>> --without-tcmalloc; \ >>>>>>>>>>>>> >>>>> cd ext/sst; \ >>>>>>>>>>>>> >>>>> make clean; make; \ >>>>>>>>>>>>> >>>>> sst --add-lib-path=./ sst/example.py; >>>>>>>>>>>>> >>>>> ``` >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> >>>>> We appreciate your help! >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> >>>>> Regards, >>>>>>>>>>>>> >>>>> Hoa Nguyen >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> >>>>> On 12/8/21, Jason Lowe-Power <[email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>> > Hey Gabe, >>>>>>>>>>>>> >>>>> > >>>>>>>>>>>>> >>>>> > This change breaks the SST integration. In the SST >>>>>>>>>>>>> integration python is >>>>>>>>>>>>> >>>>> > initialized from the SST module, not from init.cc >>>>>>>>>>>>> (this is because SST has >>>>>>>>>>>>> >>>>> > their own python interpreter). >>>>>>>>>>>>> >>>>> > >>>>>>>>>>>>> >>>>> > We would appreciate some help in fixing this. Hoa and >>>>>>>>>>>>> Giacomo can give you >>>>>>>>>>>>> >>>>> > an example that's breaking to help you fix it, I >>>>>>>>>>>>> believe. >>>>>>>>>>>>> >>>>> > >>>>>>>>>>>>> >>>>> > >>>>>>>>>>>>> https://gem5-review.googlesource.com/c/public/gem5/+/49413 >>>>>>>>>>>>> >>>>> > >>>>>>>>>>>>> >>>>> > There's strong interest in having the SST integration >>>>>>>>>>>>> working in this >>>>>>>>>>>>> >>>>> > current release. We've spent a lot of time figuring >>>>>>>>>>>>> out all of the >>>>>>>>>>>>> >>>>> > intricacies and would appreciate any help you can >>>>>>>>>>>>> provide in these last few >>>>>>>>>>>>> >>>>> > days before the release! >>>>>>>>>>>>> >>>>> > >>>>>>>>>>>>> >>>>> > Thanks! >>>>>>>>>>>>> >>>>> > >>>>>>>>>>>>> >>>>> > Jason >>>>>>>>>>>>> >>>>> > >>>>>>>>>>>>> >>>>> _______________________________________________ >>>>>>>>>>>>> >>>>> gem5-dev mailing list -- [email protected] >>>>>>>>>>>>> >>>>> To unsubscribe send an email to [email protected] >>>>>>>>>>>>> >>>>> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s >>>>>>>>>>>>> >>>>>>>>>>>>
_______________________________________________ gem5-dev mailing list -- [email protected] To unsubscribe send an email to [email protected] %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
