Ok, I was able to get a debugger to work and I dug in a bit more...
The problem is that the new code does not initialize the __main__.py
module. The original SST code had the following line:
`pythonMain = PyImport_AddModule(PyCC("__main__"));`
This was executed before `execPythonCommands`.
In `execPythonCommands` we use the `pythonMain` variable. However, in the
*new* code, that variable is NULL (or unitialized, actually).
As far as I can tell, all of the new python changes didn't come with any
documentation on how you are supposed to interact with Python from gem5.
So, there's no good way for us to update this code.
Gabe, if you can update the `execPythonCommands` function to work however
you expected it to, that would be great.
Thanks,
Jason
On Wed, Dec 15, 2021 at 10:45 AM Jason Lowe-Power <[email protected]>
wrote:
> Here's my understanding of where we are on this:
>
> As of
> https://gem5.googlesource.com/public/gem5/+/cf7ce21848ea4aeee28737823e6e768f9a14ceaf
> SST was working. (Committed Dec. 7th)
>
> Then, this large relation chain was pushed:
> https://gem5-review.googlesource.com/c/public/gem5/+/49425 ending with
> https://gem5.googlesource.com/public/gem5/+/4c1422e3ba780a5f152426e214b4be3d45b751e4.
> This relation chain broke SST. This was committed on Dec. 8th.
>
> We have two options: Either we fix this bug ASAP (as in within 6 days so
> we can release on 12/22) or we revert the changesets that were committed on
> the 8th.
>
> SST is a *must have* feature of gem5 21.2. It is quite frustrating to see
> it broken after all of the hard work that Hoa and Giacomo put into it.
>
> Let me know if I'm missing something.
>
> Thanks,
> Jason
>
> On Sat, Dec 11, 2021 at 10:35 AM Gabe Black <[email protected]> wrote:
>
>> https://gem5-review.googlesource.com/c/public/gem5/+/54006
>>
>> At least one problem I ran into was that the order of static initializers
>> was not determinstic, so the structures which said what embedded python
>> modules existed might be statically constructed after the thing that
>> consumes and sets them up. That could work by coincidence if the order is
>> favorable, but it wasn't for me and so lots of things didn't exist as far
>> as the python was concerned.
>>
>> To be clear, there is some mysterious deeper issue here which is probably
>> separate from that. As far as I can tell running gem5 itself is working
>> fine, but when running as a library in sst there is some sort of memory
>> corruption or premature garbage collection or something, and it leads to a
>> segfault or incorrect behavior (failure to import modules, failure to find
>> things inside them, etc). Things were even more tempermental until I added
>> what feels like an unnecessary increment of the reference count for the
>> modules in the new pybind_init.hh, which seemed to tame things in gem5
>> proper at least. I'm too tired to keep looking for it right now, but
>> hopefully this will give you a good leg up.
>>
>> Gabe
>>
>> On Sat, Dec 11, 2021 at 8:25 AM Jason Lowe-Power <[email protected]>
>> wrote:
>>
>>> Everything was working as of last week-ish, if I remember correctly. We
>>> were booting both Arm and RISCV full system with gem5 cores and SST
>>> caches/memory.
>>>
>>> Cheers,
>>> Jason
>>>
>>> On Sat, Dec 11, 2021, 8:23 AM Gabe Black <[email protected]> wrote:
>>>
>>>> Thanks, that was really helpful. I've fixed the compilation problems,
>>>> and then a few more that were hiding behind that, including a few ones that
>>>> were intrinsic to the way sst was initializing the gem5 library (with
>>>> dlopen). Was this working to begin with and all this breakage is new, or
>>>> was it just that the compilation broke and there were these other problems
>>>> already? I want to make sure I don't leave things in a worse state than
>>>> when I got here, but I also don't want to go chasing down things that
>>>> weren't working to begin with.
>>>>
>>>> I'm going to upload what I have so far, and hopefully someone else can
>>>> take a look at it too. The current problem seems to be that the modules in
>>>> _m5 aren't working right, either because they've been garbage collected, or
>>>> because they're not being set up properly, or...
>>>>
>>>> Gabe
>>>>
>>>> On Fri, Dec 10, 2021 at 8:17 PM Bobby Bruce <[email protected]> wrote:
>>>>
>>>>> Hey gabe.
>>>>>
>>>>> No idea if this is the _best_ solution to your problem, but my
>>>>> solution would be to rebuild the image with this installed. Modify the
>>>>> `util/dockerfiles/sst-11.1.0/Dockerfile` to the environment you want. Then
>>>>> run `docker build -t <some name here> util/dockerfiles/sst-11.1.0` to
>>>>> build
>>>>> an image with the name "<some name here>".
>>>>>
>>>>> Then you can execute `docker run -u $UID:$GID --volume $(pwd):$(pwd)
>>>>> -w $(pwd) -it <same name here>` within the gem5 directory to spin up and
>>>>> enter a container from the image you just built. I think you'll be able to
>>>>> do what you want inside the container.
>>>>>
>>>>> Kind regards,
>>>>> Bobby
>>>>> --
>>>>> Dr. Bobby R. Bruce
>>>>> Room 3050,
>>>>> Kemper Hall, UC Davis
>>>>> Davis,
>>>>> CA, 95616
>>>>>
>>>>> web: https://www.bobbybruce.net
>>>>>
>>>>>
>>>>> On Fri, Dec 10, 2021 at 7:24 PM Gabe Black <[email protected]>
>>>>> wrote:
>>>>> >
>>>>> > Dumb question: I'm trying to run gdb inside this container on the
>>>>> sst thing. How do I do that? It's not installed in the container now, and
>>>>> I
>>>>> can't (easily) figure out how to get it installed. I can tell docker to
>>>>> install it, but then it seems to throw that away as soon as the command
>>>>> ends.
>>>>> >
>>>>> > Gabe
>>>>> >
>>>>> > On Fri, Dec 10, 2021 at 5:09 PM Bobby Bruce <[email protected]>
>>>>> wrote:
>>>>> >>
>>>>> >> Thanks Gabe,
>>>>> >>
>>>>> >> This is very much appreciated. I'm going to create the release
>>>>> staging once a couple more things get in. Feel free to push any patches
>>>>> related to these bugs to the release staging branch.
>>>>> >>
>>>>> >> If there is an order of priority I'd say the bug affecting SST is
>>>>> of higher importance than that affecting the Weeklies (as far as I can see
>>>>> the latter is hard to trigger). That being said, we'll apply both to the
>>>>> new release one way or another.
>>>>> >>
>>>>> >> --
>>>>> >> Dr. Bobby R. Bruce
>>>>> >> Room 3050,
>>>>> >> Kemper Hall, UC Davis
>>>>> >> Davis,
>>>>> >> CA, 95616
>>>>> >>
>>>>> >> web: https://www.bobbybruce.net
>>>>> >>
>>>>> >>
>>>>> >> On Fri, Dec 10, 2021 at 5:02 PM Gabe Black <[email protected]>
>>>>> wrote:
>>>>> >>>
>>>>> >>> Hi Bobby, not yet, I meant to look into this for the last couple
>>>>> days but kept running out of time. I'm sitting down to work on it right
>>>>> now.
>>>>> >>>
>>>>> >>> Gabe
>>>>> >>>
>>>>> >>> On Fri, Dec 10, 2021 at 1:21 PM Bobby Bruce <[email protected]>
>>>>> wrote:
>>>>> >>>>
>>>>> >>>> Hey Gabe,
>>>>> >>>>
>>>>> >>>> Is there any update on this?
>>>>> >>>>
>>>>> >>>> Kind regards,
>>>>> >>>> Bobby
>>>>> >>>> --
>>>>> >>>> Dr. Bobby R. Bruce
>>>>> >>>> Room 3050,
>>>>> >>>> Kemper Hall, UC Davis
>>>>> >>>> Davis,
>>>>> >>>> CA, 95616
>>>>> >>>>
>>>>> >>>> web: https://www.bobbybruce.net
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> On Wed, Dec 8, 2021 at 5:51 PM Hoa Nguyen via gem5-dev <
>>>>> [email protected]> wrote:
>>>>> >>>>>
>>>>> >>>>> Hi Gabe,
>>>>> >>>>>
>>>>> >>>>> I have more details about this. In this use case, SST
>>>>> initialized the
>>>>> >>>>> Python environment before adding the "gem5 object". This gem5
>>>>> object
>>>>> >>>>> will add more Python stuff from gem5 to the environment.
>>>>> >>>>>
>>>>> >>>>> The function that does that is initPython()
>>>>> >>>>>
>>>>> https://gem5.googlesource.com/public/gem5/+/refs/heads/develop/ext/sst/gem5.cc#415
>>>>> >>>>>
>>>>> >>>>> The following commands will pull the docker image for SST testing
>>>>> >>>>> purposes (note that host_gem5_root and guest_gem5_root must be
>>>>> >>>>> specified),
>>>>> >>>>>
>>>>> >>>>> ```
>>>>> >>>>> docker run -u $UID:$GID --volume
>>>>> "${host_gem5_root}":"${guest_gem5_root}" -w \
>>>>> >>>>> "${guest_gem5_root}" --rm gcr.io/gem5-test/sst-env \
>>>>> >>>>> bash -c "\
>>>>> >>>>> scons build/RISCV/libgem5_opt.so -j${nproc} --without-tcmalloc; \
>>>>> >>>>> cd ext/sst; \
>>>>> >>>>> make clean; make; \
>>>>> >>>>> sst --add-lib-path=./ sst/example.py;
>>>>> >>>>> ```
>>>>> >>>>>
>>>>> >>>>> We appreciate your help!
>>>>> >>>>>
>>>>> >>>>> Regards,
>>>>> >>>>> Hoa Nguyen
>>>>> >>>>>
>>>>> >>>>> On 12/8/21, Jason Lowe-Power <[email protected]> wrote:
>>>>> >>>>> > Hey Gabe,
>>>>> >>>>> >
>>>>> >>>>> > This change breaks the SST integration. In the SST integration
>>>>> python is
>>>>> >>>>> > initialized from the SST module, not from init.cc (this is
>>>>> because SST has
>>>>> >>>>> > their own python interpreter).
>>>>> >>>>> >
>>>>> >>>>> > We would appreciate some help in fixing this. Hoa and Giacomo
>>>>> can give you
>>>>> >>>>> > an example that's breaking to help you fix it, I believe.
>>>>> >>>>> >
>>>>> >>>>> > https://gem5-review.googlesource.com/c/public/gem5/+/49413
>>>>> >>>>> >
>>>>> >>>>> > There's strong interest in having the SST integration working
>>>>> in this
>>>>> >>>>> > current release. We've spent a lot of time figuring out all of
>>>>> the
>>>>> >>>>> > intricacies and would appreciate any help you can provide in
>>>>> these last few
>>>>> >>>>> > days before the release!
>>>>> >>>>> >
>>>>> >>>>> > Thanks!
>>>>> >>>>> >
>>>>> >>>>> > Jason
>>>>> >>>>> >
>>>>> >>>>> _______________________________________________
>>>>> >>>>> gem5-dev mailing list -- [email protected]
>>>>> >>>>> To unsubscribe send an email to [email protected]
>>>>> >>>>> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
>>>>>
>>>>
_______________________________________________
gem5-dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
