Re: Initial draft of GSOC proposal - Offloading to a separate process on the same host.

[Soumya Ranjan via Gcc]

Thank you Martin!
I've taken your advice into account and I've uploaded my proposal.

On Sat, Mar 30, 2024 at 1:49 PM Martin Jambor  wrote:

> Hello,
>
> On Wed, Mar 27 2024, Soumya Ranjan wrote:
> > Hello!
> > Thanks for your response Martin!
> > Sorry for the late response, I've been researching the project, going
> over
> > the source code and preparing the proposal. After a lot of thought, I've
> > decided to go with the "Offloading to a separate process on the same
> host"
> > project, mostly because I feel like I've reasonable background on this
> > project, as I've worked on OpenMP, GPU Programming and have done
> coursework
> > on compilers and operating systems. Yes, I am no longer a student. I
> > recently graduated from the University of California, Irvine with a
> > master's degree in Computer Engineering (About 3 months back) and I've
> > recently joined Qualcomm as a firmware engineer. I realized that I have a
> > lot of free time, that I mostly spend playing video games, and I've
> always
> > wanted to get into open source development, so I thought this would be a
> > good opportunity, given how much I use gcc for everything.
> >
>
> This sounds great.
>
> First, please note that timing of the GSoC contributor application
> deadline (on the upcoming Tuesday) is a bit unfortunate because of
> Easter, many involved mentors have a long weekend (public holiday on
> Friday or Monday or, like me, both).  So please even if you do not
> receive any feedback, make sure to apply - and don't leave it until the
> last day.  IIUC a proposal can be always updated later.
>
> I'll have to admit that I read your proposal only quickly and it makes
> sense.  I'd just like to point out that the VGPU part is really a second
> (though perhaps much larger and interesting) part of the project, the
> first part would be to simulate a CPU-like accelerator with a separate
> memory.  But most of this work would be necessary for VGPU part too.
> What is more, the VGPU part is likely to be hard, so if your time
> constraints allow it and doing both is your goal, I'd suggest to apply
> for an 350-hour (large) project.
>
> I'll see if I can cough out any more feedback in time but as I wrote
> above, generally it is good and don't wait - t least not with the
> initial application.
>
> Good luck!
>
> Martin
>
>
> > Why specifically this project -
> > OpenMP's support for offloading to physical GPUs broadens the horizon for
> > high-performance computing applications, the complexity of setting up
> such
> > environments and the lack of adequate tooling for development and
> debugging
> > can hinder productivity. The VGPU project directly addresses these
> > challenges by providing a developer-friendly offloading target that
> > emulates GPU execution on the host CPU, bridging the existing tooling gap
> > and significantly enhancing developer productivity in the realm of
> parallel
> > computing.
> >
> > Anyway, getting into the details of the project, from my understanding,
> the
> > goals are -
> > 1) To implement a virtual GPU (VGPU) environment that mirrors physical
> GPU
> > architecture including support for different levels of parallelism (warp,
> > thread block, etc.).
> > 2) To enable the VGPU to serve as an offload target within the
> LLVM/OpenMP
> > framework. This includes adding a host-ISA offloaded code generation mode
> > that allows the compilation of OpenMP applications using GPU-specific
> paths
> > and runtimes, facilitating a more accurate emulation of GPU environments.
> > 3) To implement a plugin for libgomp that communicates with the libgomp
> > offloading machinery to manage the execution of offloaded code in a new
> > process, simulating the behavior of actual GPU devices.
> > 4) To optimize the VGPU to ensure that OpenMP applications executed on it
> > incur minimal performance overhead compared to native host execution,
> > thereby making it a viable option for development and testing purposes.
> >
> > Here's a rough timeline (Based on the timeline on the gsoc website) -
> > Pre-coding (Until May 27) -
> > 1) Setting up a development environment including LLVM/OpenMP and
> necessary
> > debugging tools.
> > 2) Conducting thorough literature review on existing GPU simulation
> > techniques and OpenMP offloading mechanisms.
> >
> > Week 1-3: Initial Infrastructure
> > 1) Design VGPU architecture (simulate gpu parallel execution models
> (warps,
> > blocks) and memory hierarchy (global, shared, private))
> > 2) Implement the core vgpu infrastructure, like basic memory management.
> >
> > Week 4-6: Integration with LLVM/OpenMP and Host-ISA Offload Mode
> > 1) Develop LLVM IR generation for VGPU target, thereby ensuring openMP
> > directives can be compiled into vgpu-compatible code.
> > 2) Add a new mode in the LLVM/OpenMP framework for generating offloaded
> > code specifically for the VGPU target.
> > 3) Get simple openMP applications to compile and execute on the VGPU.
> >
> > By Midterm evaluation, 

Re: Initial draft of GSOC proposal - Offloading to a separate process on the same host.

[Martin Jambor]

Hello,

On Wed, Mar 27 2024, Soumya Ranjan wrote:
> Hello!
> Thanks for your response Martin!
> Sorry for the late response, I've been researching the project, going over
> the source code and preparing the proposal. After a lot of thought, I've
> decided to go with the "Offloading to a separate process on the same host"
> project, mostly because I feel like I've reasonable background on this
> project, as I've worked on OpenMP, GPU Programming and have done coursework
> on compilers and operating systems. Yes, I am no longer a student. I
> recently graduated from the University of California, Irvine with a
> master's degree in Computer Engineering (About 3 months back) and I've
> recently joined Qualcomm as a firmware engineer. I realized that I have a
> lot of free time, that I mostly spend playing video games, and I've always
> wanted to get into open source development, so I thought this would be a
> good opportunity, given how much I use gcc for everything.
>

This sounds great.

First, please note that timing of the GSoC contributor application
deadline (on the upcoming Tuesday) is a bit unfortunate because of
Easter, many involved mentors have a long weekend (public holiday on
Friday or Monday or, like me, both).  So please even if you do not
receive any feedback, make sure to apply - and don't leave it until the
last day.  IIUC a proposal can be always updated later.

I'll have to admit that I read your proposal only quickly and it makes
sense.  I'd just like to point out that the VGPU part is really a second
(though perhaps much larger and interesting) part of the project, the
first part would be to simulate a CPU-like accelerator with a separate
memory.  But most of this work would be necessary for VGPU part too.
What is more, the VGPU part is likely to be hard, so if your time
constraints allow it and doing both is your goal, I'd suggest to apply
for an 350-hour (large) project.

I'll see if I can cough out any more feedback in time but as I wrote
above, generally it is good and don't wait - t least not with the
initial application.

Good luck!

Martin


> Why specifically this project -
> OpenMP's support for offloading to physical GPUs broadens the horizon for
> high-performance computing applications, the complexity of setting up such
> environments and the lack of adequate tooling for development and debugging
> can hinder productivity. The VGPU project directly addresses these
> challenges by providing a developer-friendly offloading target that
> emulates GPU execution on the host CPU, bridging the existing tooling gap
> and significantly enhancing developer productivity in the realm of parallel
> computing.
>
> Anyway, getting into the details of the project, from my understanding, the
> goals are -
> 1) To implement a virtual GPU (VGPU) environment that mirrors physical GPU
> architecture including support for different levels of parallelism (warp,
> thread block, etc.).
> 2) To enable the VGPU to serve as an offload target within the LLVM/OpenMP
> framework. This includes adding a host-ISA offloaded code generation mode
> that allows the compilation of OpenMP applications using GPU-specific paths
> and runtimes, facilitating a more accurate emulation of GPU environments.
> 3) To implement a plugin for libgomp that communicates with the libgomp
> offloading machinery to manage the execution of offloaded code in a new
> process, simulating the behavior of actual GPU devices.
> 4) To optimize the VGPU to ensure that OpenMP applications executed on it
> incur minimal performance overhead compared to native host execution,
> thereby making it a viable option for development and testing purposes.
>
> Here's a rough timeline (Based on the timeline on the gsoc website) -
> Pre-coding (Until May 27) -
> 1) Setting up a development environment including LLVM/OpenMP and necessary
> debugging tools.
> 2) Conducting thorough literature review on existing GPU simulation
> techniques and OpenMP offloading mechanisms.
>
> Week 1-3: Initial Infrastructure
> 1) Design VGPU architecture (simulate gpu parallel execution models (warps,
> blocks) and memory hierarchy (global, shared, private))
> 2) Implement the core vgpu infrastructure, like basic memory management.
>
> Week 4-6: Integration with LLVM/OpenMP and Host-ISA Offload Mode
> 1) Develop LLVM IR generation for VGPU target, thereby ensuring openMP
> directives can be compiled into vgpu-compatible code.
> 2) Add a new mode in the LLVM/OpenMP framework for generating offloaded
> code specifically for the VGPU target.
> 3) Get simple openMP applications to compile and execute on the VGPU.
>
> By Midterm evaluation, hopefully should have basic openmp applications
> offloaded on the VGPU.
>
> Week 7-9: Extending functionality and Implementing libgomp Plugin
> 1) Extend VGPU to support more functionality like loops, sections, parallel
> blocks.
> 2) Implement a plug-in for libgomp that interfaces with its offloading
> machinery.
> 3) Maybe look to 

Initial draft of GSOC proposal - Offloading to a separate process on the same host.

[Soumya Ranjan via Gcc]

Hello!
Thanks for your response Martin!
Sorry for the late response, I've been researching the project, going over
the source code and preparing the proposal. After a lot of thought, I've
decided to go with the "Offloading to a separate process on the same host"
project, mostly because I feel like I've reasonable background on this
project, as I've worked on OpenMP, GPU Programming and have done coursework
on compilers and operating systems. Yes, I am no longer a student. I
recently graduated from the University of California, Irvine with a
master's degree in Computer Engineering (About 3 months back) and I've
recently joined Qualcomm as a firmware engineer. I realized that I have a
lot of free time, that I mostly spend playing video games, and I've always
wanted to get into open source development, so I thought this would be a
good opportunity, given how much I use gcc for everything.

Why specifically this project -
OpenMP's support for offloading to physical GPUs broadens the horizon for
high-performance computing applications, the complexity of setting up such
environments and the lack of adequate tooling for development and debugging
can hinder productivity. The VGPU project directly addresses these
challenges by providing a developer-friendly offloading target that
emulates GPU execution on the host CPU, bridging the existing tooling gap
and significantly enhancing developer productivity in the realm of parallel
computing.

Anyway, getting into the details of the project, from my understanding, the
goals are -
1) To implement a virtual GPU (VGPU) environment that mirrors physical GPU
architecture including support for different levels of parallelism (warp,
thread block, etc.).
2) To enable the VGPU to serve as an offload target within the LLVM/OpenMP
framework. This includes adding a host-ISA offloaded code generation mode
that allows the compilation of OpenMP applications using GPU-specific paths
and runtimes, facilitating a more accurate emulation of GPU environments.
3) To implement a plugin for libgomp that communicates with the libgomp
offloading machinery to manage the execution of offloaded code in a new
process, simulating the behavior of actual GPU devices.
4) To optimize the VGPU to ensure that OpenMP applications executed on it
incur minimal performance overhead compared to native host execution,
thereby making it a viable option for development and testing purposes.

Here's a rough timeline (Based on the timeline on the gsoc website) -
Pre-coding (Until May 27) -
1) Setting up a development environment including LLVM/OpenMP and necessary
debugging tools.
2) Conducting thorough literature review on existing GPU simulation
techniques and OpenMP offloading mechanisms.

Week 1-3: Initial Infrastructure
1) Design VGPU architecture (simulate gpu parallel execution models (warps,
blocks) and memory hierarchy (global, shared, private))
2) Implement the core vgpu infrastructure, like basic memory management.

Week 4-6: Integration with LLVM/OpenMP and Host-ISA Offload Mode
1) Develop LLVM IR generation for VGPU target, thereby ensuring openMP
directives can be compiled into vgpu-compatible code.
2) Add a new mode in the LLVM/OpenMP framework for generating offloaded
code specifically for the VGPU target.
3) Get simple openMP applications to compile and execute on the VGPU.

By Midterm evaluation, hopefully should have basic openmp applications
offloaded on the VGPU.

Week 7-9: Extending functionality and Implementing libgomp Plugin
1) Extend VGPU to support more functionality like loops, sections, parallel
blocks.
2) Implement a plug-in for libgomp that interfaces with its offloading
machinery.
3) Maybe look to integrate with debugging tools, so users can step through
offloaded regions and profile code.

Week 10-12: Evaluation and Final Submission
1) Benchmark against physical GPU's to evaluate the VGPU's performance.
2) Prepare a final project report documenting the development process,
challenges, results and future work.

I know this is a pretty high-level description, but I will try my best to
stick to this. This submission is mainly to go over the content. I would
appreciate any feedback I can get, and will make sure to submit a more
detailed description on my final submission. Awaiting your feedback.
Thanks,
Soumya Ranjan

GSoC Proposal

[Abhigyan Kashyap via Gcc]

Re: GSoC proposal for extending static analyzer

[David Malcolm via Gcc]

On Fri, 2022-04-15 at 22:36 +0530, Mir Immad wrote:
> I've updated the link on the repo --  
> https://mirimmad.github.io/zeta-lang.
> 
> > You don't give many specifics in your personal decription.  One thing
> > I'm not seeing is a sense of how proficient you are in various
> > programming languages.  In particular, how is your C and C++?  How
> > familiar are you with the debugger?  Looking at your github, you seem
> > to have relevant experience in compilers, which is great, but all
> > your
> > code appears to be with "managed" languages such as Ruby, Java, and
> > Python  [and Zeta :)].
> 
>  I'm pretty comfortable with both C and C++ and manual mem management .
> Unfortunately, I don't have any project in C/C++ to show. Maybe I can
> use
> the time before May 20 to write something in cpp?

Perhaps write some low-level code in C and/or C++ that uses file-
descriptors?  That would help you gain familiarity with both C/C++, and
with the problem domain.  (in terms of what does good code look like,
and also, what are the ways in which a programmer can make mistakes
when using a particular API).


> About the debugger, I'm okay-ish with it. In fact, it was due to the
> debugger (and your blog on it) that I was initially able to walk
> through
> the codebase.

(nods)

> 
> 
> > That said, I got
> > the sense from your previous emails that you're not very familiar
> > with
> > the APIs, and that you chose them because that was the suggestion I
> > had
> > made on the wiki page.
> 
> thats right.
> 
> 
> >  Obviously it's something you can learn on the
> >  way, but it would be better to accurately identify which areas
> > you're
> > going need to learn along the way, and the timetable and scope should
> > reflect that.
> 
> If I understand the statement correctly; currently I'm thinking of
> extending the support for open() (for creating the fd), write/read (for
> working on the fd) and close(). This is quite analogous to what we have
> in
> sm-file. Please let me know how do you want the analyzer to be extended
> and
> if you expect support for any other FD APIs  too as I understand there
> are
> many other APIs for creating and working wiith FDs?
> 
> Thank you.

There are a lot of them; see e.g.:
  https://en.wikipedia.org/wiki/File_descriptor
Perhaps it's worth considering implementing some kind of attribute for
function-decls to specify what their behavior is with regards to file
descriptors, rather than hardcoding the expected behaviors
individually?  (not sure, perhaps one part of the project will be to
catalog the different expected preconditions and behaviors of API
entrypoints that work with file descriptors, in that I expect that they
will fall into patterns).

BTW, I'm about to go on a week-long trip, and will be away from the
computer during that time, so I probably won't be able to reply further
before the application deadline.

Hope this is helpful
Dave

> 
> 
> 
> On Fri, Apr 15, 2022 at 9:35 PM David Malcolm 
> wrote:
> 
> > On Fri, 2022-04-15 at 19:58 +0530, Mir Immad wrote:
> > > I've submitted a proposal for extending the static analyzer to
> > > support
> > > posix fd APIs on GSoC website. Here is the Google docs link (gdocs
> > > <
> > > 
> > https://docs.google.com/document/d/188zxPUsuYcF-uGVYL_G1s2RVtHhJSZeQ4sha40H7374/edit?usp=sharing
> > > > ).
> > > 
> > > 
> > > Please take a look and let me know what you think.
> > > 
> > > Thank you.
> > 
> > Thanks.
> > 
> > FWIW, I'm getting an error when trying the URL given in your github
> > repo: http://mirimmad.me/
> > but https://mirimmad.github.io/ seems to work  - but it's almost
> > empty.
> > 
> > You don't give many specifics in your personal decription.  One thing
> > I'm not seeing is a sense of how proficient you are in various
> > programming languages.  In particular, how is your C and C++?  How
> > familiar are you with the debugger?  Looking at your github, you seem
> > to have relevant experience in compilers, which is great, but all
> > your
> > code appears to be with "managed" languages such as Ruby, Java, and
> > Python  [and Zeta :)].
> > 
> > Also, the proposal is to extend the analyzer to cover a specific
> > domain: various POSIX APIs.  Can you please give a sense of your
> > level
> > of expertise with these APIs?  I was pleased at your initiative in
> > trying to reuse the existing code to work with them.  That said, I
> > got
> > the sense from your previous emails that you're not very familiar
> > with
> > the APIs, and that you chose them because that was the suggestion I
> > had
> > made on the wiki page.  Obviously it's something you can learn on the
> > way, but it would be better to accurately identify which areas you're
> > going need to learn along the way, and the timetable and scope should
> > reflect that.
> > 
> > Hope this is constructive
> > Dave
> > 
> > 
> > 

Re: GSoC proposal for extending static analyzer

[Mir Immad via Gcc]

I've updated the link on the repo -- https://mirimmad.github.io/zeta-lang.

> You don't give many specifics in your personal decription.  One thing
> I'm not seeing is a sense of how proficient you are in various
> programming languages.  In particular, how is your C and C++?  How
> familiar are you with the debugger?  Looking at your github, you seem
> to have relevant experience in compilers, which is great, but all your
> code appears to be with "managed" languages such as Ruby, Java, and
> Python  [and Zeta :)].

 I'm pretty comfortable with both C and C++ and manual mem management .
Unfortunately, I don't have any project in C/C++ to show. Maybe I can use
the time before May 20 to write something in cpp?
About the debugger, I'm okay-ish with it. In fact, it was due to the
debugger (and your blog on it) that I was initially able to walk through
the codebase.


> That said, I got
> the sense from your previous emails that you're not very familiar with
> the APIs, and that you chose them because that was the suggestion I had
> made on the wiki page.

thats right.


>  Obviously it's something you can learn on the
>  way, but it would be better to accurately identify which areas you're
> going need to learn along the way, and the timetable and scope should
> reflect that.

If I understand the statement correctly; currently I'm thinking of
extending the support for open() (for creating the fd), write/read (for
working on the fd) and close(). This is quite analogous to what we have in
sm-file. Please let me know how do you want the analyzer to be extended and
if you expect support for any other FD APIs  too as I understand there are
many other APIs for creating and working wiith FDs?

Thank you.



On Fri, Apr 15, 2022 at 9:35 PM David Malcolm  wrote:

> On Fri, 2022-04-15 at 19:58 +0530, Mir Immad wrote:
> > I've submitted a proposal for extending the static analyzer to support
> > posix fd APIs on GSoC website. Here is the Google docs link (gdocs
> > <
> >
> https://docs.google.com/document/d/188zxPUsuYcF-uGVYL_G1s2RVtHhJSZeQ4sha40H7374/edit?usp=sharing
> > >).
> >
> >
> > Please take a look and let me know what you think.
> >
> > Thank you.
>
> Thanks.
>
> FWIW, I'm getting an error when trying the URL given in your github
> repo: http://mirimmad.me/
> but https://mirimmad.github.io/ seems to work  - but it's almost empty.
>
> You don't give many specifics in your personal decription.  One thing
> I'm not seeing is a sense of how proficient you are in various
> programming languages.  In particular, how is your C and C++?  How
> familiar are you with the debugger?  Looking at your github, you seem
> to have relevant experience in compilers, which is great, but all your
> code appears to be with "managed" languages such as Ruby, Java, and
> Python  [and Zeta :)].
>
> Also, the proposal is to extend the analyzer to cover a specific
> domain: various POSIX APIs.  Can you please give a sense of your level
> of expertise with these APIs?  I was pleased at your initiative in
> trying to reuse the existing code to work with them.  That said, I got
> the sense from your previous emails that you're not very familiar with
> the APIs, and that you chose them because that was the suggestion I had
> made on the wiki page.  Obviously it's something you can learn on the
> way, but it would be better to accurately identify which areas you're
> going need to learn along the way, and the timetable and scope should
> reflect that.
>
> Hope this is constructive
> Dave
>
>
>

Re: GSoC proposal for extending static analyzer

[David Malcolm via Gcc]

On Fri, 2022-04-15 at 19:58 +0530, Mir Immad wrote:
> I've submitted a proposal for extending the static analyzer to support
> posix fd APIs on GSoC website. Here is the Google docs link (gdocs
> <
> https://docs.google.com/document/d/188zxPUsuYcF-uGVYL_G1s2RVtHhJSZeQ4sha40H7374/edit?usp=sharing
> >).
> 
> 
> Please take a look and let me know what you think.
> 
> Thank you.

Thanks.

FWIW, I'm getting an error when trying the URL given in your github
repo: http://mirimmad.me/
but https://mirimmad.github.io/ seems to work  - but it's almost empty.

You don't give many specifics in your personal decription.  One thing
I'm not seeing is a sense of how proficient you are in various
programming languages.  In particular, how is your C and C++?  How
familiar are you with the debugger?  Looking at your github, you seem
to have relevant experience in compilers, which is great, but all your
code appears to be with "managed" languages such as Ruby, Java, and
Python  [and Zeta :)].

Also, the proposal is to extend the analyzer to cover a specific
domain: various POSIX APIs.  Can you please give a sense of your level
of expertise with these APIs?  I was pleased at your initiative in
trying to reuse the existing code to work with them.  That said, I got
the sense from your previous emails that you're not very familiar with
the APIs, and that you chose them because that was the suggestion I had
made on the wiki page.  Obviously it's something you can learn on the
way, but it would be better to accurately identify which areas you're
going need to learn along the way, and the timetable and scope should
reflect that.

Hope this is constructive
Dave

GSoC proposal for extending static analyzer

[Mir Immad via Gcc]

I've submitted a proposal for extending the static analyzer to support
posix fd APIs on GSoC website. Here is the Google docs link (gdocs
).


Please take a look and let me know what you think.

Thank you.

GSoC proposal - LTO incremental linking

[Carlos Eduardo Climaco Barbosa via Gcc]

Hi! My name is Carlos Barbosa, and I would like to participate in this
year's GSoC with GCC.
I'm a brazilian undergraduate student and I've been passionate about free
software ever since I first heard of it. I'm most interested in OS level
programming, language development, optimization and, of course, compiler
development.
I'm actually only starting to contribute to free software code now, but I
have been active in other ways at local FOSS communities for a couple of
years now.

About the GSoC proposal, the LTO now re-processes everything from scratch,
for any changes in the code, no matter how small. So, the project idea is
to make it work incrementally, avoiding unnecessary re-processing.
In order to do so, I thought about focusing the most at WPA, first
verifying and mapping still valid parts of the old ELF code, then building
the new optimization_summary with them and with new code. Then, if LTRANS
can't determine which files have been modified, this "diff" might be passed
in the IPA to it, so only the modified files get re-optimized. During
LTRANS, would using an analogous approach, trying to make local
optimizations incremental too, be interesting?
Do you have any suggestions about how to improve this approach idea?

I apologize already if I misunderstood something in the docs and/or didn't
make sense at some point.

Regards,

Carlos Barbosa

GSoC proposal - LTO incremental linking optimization

[Carlos Eduardo Climaco Barbosa via Gcc]

Hi! My name is Carlos Barbosa, and I would like to participate in this
year's GSoC with GCC.
I'm a brazilian undergraduate student and I've been passionate about free
software ever since I first heard of it. I'm most interested in OS level
programming, language development, optimization and, of course, compiler
development.
I'm actually only starting to contribute to free software code now, but I
have been active in other ways at local FOSS communities for a couple of
years now.

About the GSoC proposal, the LTO now re-processes everything from scratch,
for any changes in the code, no matter how small. So, the project idea is
to make it work incrementally, avoiding unnecessary re-processing.
In order to do so, I thought about focusing the most at WPA, first
verifying and mapping still valid parts of the old ELF code, then building
the new optimization_summary with them and with new code. Then, if LTRANS
can't determine which files have been modified, this "diff" might be passed
in the IPA to it, so only the modified files get re-optimized. During
LTRANS, would using an analogous approach, trying to make local
optimizations incremental too, be interesting?
Do you have any suggestions about how to improve this approach idea?

I apologize already if I misunderstood something in the docs and/or didn't
make sense at some point.

Regards,

Carlos Barbosa

My GSoC proposal for the Rust frontend

[PKU via Gcc]

It’s about improving compiler dumps for the Rust frontend. Full proposal here: 
https://docs.google.com/document/d/1gyAOM-f3RyZh3HVpjmIMDSuQ5gBscal71sFY6XUMcHI/edit#

Yizhe

Re: GSOC Proposal

[nick]

On 2019-04-08 9:42 a.m., Richard Biener wrote:
> On Mon, 8 Apr 2019, nick wrote:
> 
>>
>>
>> On 2019-04-08 3:29 a.m., Richard Biener wrote:
>>> On Sun, 7 Apr 2019, nick wrote:
>>>


 On 2019-04-07 5:31 a.m., Richard Biener wrote:
> On April 5, 2019 6:11:15 PM GMT+02:00, nick  wrote:
>>
>>
>> On 2019-04-05 6:25 a.m., Richard Biener wrote:
>>> On Wed, 3 Apr 2019, nick wrote:
>>>


 On 2019-04-03 7:30 a.m., Richard Biener wrote:
> On Mon, 1 Apr 2019, nick wrote:
>
>>
>>
>> On 2019-04-01 9:47 a.m., Richard Biener wrote:
>>> On Mon, 1 Apr 2019, nick wrote:
>>>
 Well I'm talking about the shared roots of this garbage
>> collector core state 
 data structure or just struct ggc_root_tab.

 But also this seems that this to be no longer shared globally if
>> I'm not mistaken 
 or this:
 static vec extra_root_vec;

 Not sure after reading the code which is a bigger deal through
>> so I wrote
 my proposal not just asking which is a better issue for not
>> being thread
 safe. Sorry about that.

 As for the second question injection seems to not be the issue
>> or outside
 callers but just internal so phase 3 or step 3 would now be:
 Find internal callers or users of x where x is one of the above
>> rather
 than injecting outside callers. Which answers my second question
>> about
 external callers being a issue still.

 Let me know which  of the two is a better issue:
 1. struct ggc_root_tabs being shared
 2.static vec extra_root_vec; as a shared
>> heap or
 vector of root nodes for each type of allocation

 and I will gladly rewrite my proposal sections for that
 as needs to be reedited.
>>>
>>> I don't think working on the garbage collector as a separate
>>> GSoC project is useful at this point.  Doing locking around
>>> allocation seems like a good short-term solution and if that
>>> turns out to be a performance issue for the threaded part
>>> using per-thread freelists is likely an easy to deploy
>>> solution.
>>>
>>> Richard.
>>>
>> I agree but we were discussing this:
>> Or maybe a project to be more
>> explicit about regions of the code that assume that the garbage-
>> collector can't run within them?[3] (since the GC is state that
>> would
>> be shared by the threads).
>
> The process of collecting garbage is not the only issue (and that
> very issue is easiest mitigated by collecting only at specific
> points - which is what we do - and have those be serializing
>> points).
> The main issue is the underlying memory allocator (GCC uses memory
> that is garbage collected plus regular heap memory).
>
>> In addition I moved my paper back to our discussion about garbage
>> collector
>> state with outside callers.Seems we really need to do something
>> about
>> my wording as the idea of my project in a nutshell was to figure
>> out how to mark shared state by callers and inject it into the
>> garbage collector letting it known that the state was not shared
>> between
>> threads or shared. Seems that was on the GSoc page and in our
>> discussions the issue
>> is marking outside code for shared state. If that's correct then
>> my
>> wording of outside callers is incorrect it should have been shared
>> state between threads on outside callers to the garbage collector.
>> If the state is that in your wording above then great as I
>> understand
>> where we are going and will gladly change my wording.
>
> I'm still not sure what you are shooting at, the above sentences do
> not make any sense to me.
>
>> Also freelists don't work here as the state is shared at the
>> caller's 
>> end which would need two major issues:
>> 1. Locking on nodes of the 
>> freelists when two threads allocate at the same thing which can be
>> a 
>> problem if the shared state is shared a lot
>> 2. Locking allocation with 
>> large numbers of callers can starve threads
>
> First of all allocating memory from the GC pool is not the main
> work of GIMPLE passes so simply serializing at allocation time
>> might
> work out.  Second free lists of course do work.  What you'd do is
> have a fast path in allocation using a thread-local "free list"
> which you can allocate from without 

Re: GSOC Proposal

[Richard Biener]

On Mon, 8 Apr 2019, nick wrote:

> 
> 
> On 2019-04-08 3:29 a.m., Richard Biener wrote:
> > On Sun, 7 Apr 2019, nick wrote:
> > 
> >>
> >>
> >> On 2019-04-07 5:31 a.m., Richard Biener wrote:
> >>> On April 5, 2019 6:11:15 PM GMT+02:00, nick  wrote:
> 
> 
>  On 2019-04-05 6:25 a.m., Richard Biener wrote:
> > On Wed, 3 Apr 2019, nick wrote:
> >
> >>
> >>
> >> On 2019-04-03 7:30 a.m., Richard Biener wrote:
> >>> On Mon, 1 Apr 2019, nick wrote:
> >>>
> 
> 
>  On 2019-04-01 9:47 a.m., Richard Biener wrote:
> > On Mon, 1 Apr 2019, nick wrote:
> >
> >> Well I'm talking about the shared roots of this garbage
>  collector core state 
> >> data structure or just struct ggc_root_tab.
> >>
> >> But also this seems that this to be no longer shared globally if
>  I'm not mistaken 
> >> or this:
> >> static vec extra_root_vec;
> >>
> >> Not sure after reading the code which is a bigger deal through
>  so I wrote
> >> my proposal not just asking which is a better issue for not
>  being thread
> >> safe. Sorry about that.
> >>
> >> As for the second question injection seems to not be the issue
>  or outside
> >> callers but just internal so phase 3 or step 3 would now be:
> >> Find internal callers or users of x where x is one of the above
>  rather
> >> than injecting outside callers. Which answers my second question
>  about
> >> external callers being a issue still.
> >>
> >> Let me know which  of the two is a better issue:
> >> 1. struct ggc_root_tabs being shared
> >> 2.static vec extra_root_vec; as a shared
>  heap or
> >> vector of root nodes for each type of allocation
> >>
> >> and I will gladly rewrite my proposal sections for that
> >> as needs to be reedited.
> >
> > I don't think working on the garbage collector as a separate
> > GSoC project is useful at this point.  Doing locking around
> > allocation seems like a good short-term solution and if that
> > turns out to be a performance issue for the threaded part
> > using per-thread freelists is likely an easy to deploy
> > solution.
> >
> > Richard.
> >
>  I agree but we were discussing this:
>  Or maybe a project to be more
>  explicit about regions of the code that assume that the garbage-
>  collector can't run within them?[3] (since the GC is state that
>  would
>  be shared by the threads).
> >>>
> >>> The process of collecting garbage is not the only issue (and that
> >>> very issue is easiest mitigated by collecting only at specific
> >>> points - which is what we do - and have those be serializing
>  points).
> >>> The main issue is the underlying memory allocator (GCC uses memory
> >>> that is garbage collected plus regular heap memory).
> >>>
>  In addition I moved my paper back to our discussion about garbage
>  collector
>  state with outside callers.Seems we really need to do something
>  about
>  my wording as the idea of my project in a nutshell was to figure
>  out how to mark shared state by callers and inject it into the
>  garbage collector letting it known that the state was not shared
>  between
>  threads or shared. Seems that was on the GSoc page and in our
>  discussions the issue
>  is marking outside code for shared state. If that's correct then
>  my
>  wording of outside callers is incorrect it should have been shared
>  state between threads on outside callers to the garbage collector.
>  If the state is that in your wording above then great as I
>  understand
>  where we are going and will gladly change my wording.
> >>>
> >>> I'm still not sure what you are shooting at, the above sentences do
> >>> not make any sense to me.
> >>>
>  Also freelists don't work here as the state is shared at the
>  caller's 
>  end which would need two major issues:
>  1. Locking on nodes of the 
>  freelists when two threads allocate at the same thing which can be
>  a 
>  problem if the shared state is shared a lot
>  2. Locking allocation with 
>  large numbers of callers can starve threads
> >>>
> >>> First of all allocating memory from the GC pool is not the main
> >>> work of GIMPLE passes so simply serializing at allocation time
>  might
> >>> work out.  Second free lists of course do work.  What you'd do is
> >>> have a fast path in allocation using a thread-local "free list"
> >>> which you can allocate from without taking any lock.  Maybe I
>  should
> >>> 

Re: GSOC Proposal

[nick]

On 2019-04-08 3:29 a.m., Richard Biener wrote:
> On Sun, 7 Apr 2019, nick wrote:
> 
>>
>>
>> On 2019-04-07 5:31 a.m., Richard Biener wrote:
>>> On April 5, 2019 6:11:15 PM GMT+02:00, nick  wrote:


 On 2019-04-05 6:25 a.m., Richard Biener wrote:
> On Wed, 3 Apr 2019, nick wrote:
>
>>
>>
>> On 2019-04-03 7:30 a.m., Richard Biener wrote:
>>> On Mon, 1 Apr 2019, nick wrote:
>>>


 On 2019-04-01 9:47 a.m., Richard Biener wrote:
> On Mon, 1 Apr 2019, nick wrote:
>
>> Well I'm talking about the shared roots of this garbage
 collector core state 
>> data structure or just struct ggc_root_tab.
>>
>> But also this seems that this to be no longer shared globally if
 I'm not mistaken 
>> or this:
>> static vec extra_root_vec;
>>
>> Not sure after reading the code which is a bigger deal through
 so I wrote
>> my proposal not just asking which is a better issue for not
 being thread
>> safe. Sorry about that.
>>
>> As for the second question injection seems to not be the issue
 or outside
>> callers but just internal so phase 3 or step 3 would now be:
>> Find internal callers or users of x where x is one of the above
 rather
>> than injecting outside callers. Which answers my second question
 about
>> external callers being a issue still.
>>
>> Let me know which  of the two is a better issue:
>> 1. struct ggc_root_tabs being shared
>> 2.static vec extra_root_vec; as a shared
 heap or
>> vector of root nodes for each type of allocation
>>
>> and I will gladly rewrite my proposal sections for that
>> as needs to be reedited.
>
> I don't think working on the garbage collector as a separate
> GSoC project is useful at this point.  Doing locking around
> allocation seems like a good short-term solution and if that
> turns out to be a performance issue for the threaded part
> using per-thread freelists is likely an easy to deploy
> solution.
>
> Richard.
>
 I agree but we were discussing this:
 Or maybe a project to be more
 explicit about regions of the code that assume that the garbage-
 collector can't run within them?[3] (since the GC is state that
 would
 be shared by the threads).
>>>
>>> The process of collecting garbage is not the only issue (and that
>>> very issue is easiest mitigated by collecting only at specific
>>> points - which is what we do - and have those be serializing
 points).
>>> The main issue is the underlying memory allocator (GCC uses memory
>>> that is garbage collected plus regular heap memory).
>>>
 In addition I moved my paper back to our discussion about garbage
 collector
 state with outside callers.Seems we really need to do something
 about
 my wording as the idea of my project in a nutshell was to figure
 out how to mark shared state by callers and inject it into the
 garbage collector letting it known that the state was not shared
 between
 threads or shared. Seems that was on the GSoc page and in our
 discussions the issue
 is marking outside code for shared state. If that's correct then
 my
 wording of outside callers is incorrect it should have been shared
 state between threads on outside callers to the garbage collector.
 If the state is that in your wording above then great as I
 understand
 where we are going and will gladly change my wording.
>>>
>>> I'm still not sure what you are shooting at, the above sentences do
>>> not make any sense to me.
>>>
 Also freelists don't work here as the state is shared at the
 caller's 
 end which would need two major issues:
 1. Locking on nodes of the 
 freelists when two threads allocate at the same thing which can be
 a 
 problem if the shared state is shared a lot
 2. Locking allocation with 
 large numbers of callers can starve threads
>>>
>>> First of all allocating memory from the GC pool is not the main
>>> work of GIMPLE passes so simply serializing at allocation time
 might
>>> work out.  Second free lists of course do work.  What you'd do is
>>> have a fast path in allocation using a thread-local "free list"
>>> which you can allocate from without taking any lock.  Maybe I
 should
>>> explain "free list" since that term doesn't make too much sense in
>>> a garbage collector world.  What I'd do is when a client thread
>>> asks for memory of size N allocate M objects of that size but put
>>> M - 1 on the client thread local "free list" to be 

Re: GSOC Proposal

[Richard Biener]

On Sun, 7 Apr 2019, nick wrote:

> 
> 
> On 2019-04-07 5:31 a.m., Richard Biener wrote:
> > On April 5, 2019 6:11:15 PM GMT+02:00, nick  wrote:
> >>
> >>
> >> On 2019-04-05 6:25 a.m., Richard Biener wrote:
> >>> On Wed, 3 Apr 2019, nick wrote:
> >>>
> 
> 
>  On 2019-04-03 7:30 a.m., Richard Biener wrote:
> > On Mon, 1 Apr 2019, nick wrote:
> >
> >>
> >>
> >> On 2019-04-01 9:47 a.m., Richard Biener wrote:
> >>> On Mon, 1 Apr 2019, nick wrote:
> >>>
>  Well I'm talking about the shared roots of this garbage
> >> collector core state 
>  data structure or just struct ggc_root_tab.
> 
>  But also this seems that this to be no longer shared globally if
> >> I'm not mistaken 
>  or this:
>  static vec extra_root_vec;
> 
>  Not sure after reading the code which is a bigger deal through
> >> so I wrote
>  my proposal not just asking which is a better issue for not
> >> being thread
>  safe. Sorry about that.
> 
>  As for the second question injection seems to not be the issue
> >> or outside
>  callers but just internal so phase 3 or step 3 would now be:
>  Find internal callers or users of x where x is one of the above
> >> rather
>  than injecting outside callers. Which answers my second question
> >> about
>  external callers being a issue still.
> 
>  Let me know which  of the two is a better issue:
>  1. struct ggc_root_tabs being shared
>  2.static vec extra_root_vec; as a shared
> >> heap or
>  vector of root nodes for each type of allocation
> 
>  and I will gladly rewrite my proposal sections for that
>  as needs to be reedited.
> >>>
> >>> I don't think working on the garbage collector as a separate
> >>> GSoC project is useful at this point.  Doing locking around
> >>> allocation seems like a good short-term solution and if that
> >>> turns out to be a performance issue for the threaded part
> >>> using per-thread freelists is likely an easy to deploy
> >>> solution.
> >>>
> >>> Richard.
> >>>
> >> I agree but we were discussing this:
> >> Or maybe a project to be more
> >> explicit about regions of the code that assume that the garbage-
> >> collector can't run within them?[3] (since the GC is state that
> >> would
> >> be shared by the threads).
> >
> > The process of collecting garbage is not the only issue (and that
> > very issue is easiest mitigated by collecting only at specific
> > points - which is what we do - and have those be serializing
> >> points).
> > The main issue is the underlying memory allocator (GCC uses memory
> > that is garbage collected plus regular heap memory).
> >
> >> In addition I moved my paper back to our discussion about garbage
> >> collector
> >> state with outside callers.Seems we really need to do something
> >> about
> >> my wording as the idea of my project in a nutshell was to figure
> >> out how to mark shared state by callers and inject it into the
> >> garbage collector letting it known that the state was not shared
> >> between
> >> threads or shared. Seems that was on the GSoc page and in our
> >> discussions the issue
> >> is marking outside code for shared state. If that's correct then
> >> my
> >> wording of outside callers is incorrect it should have been shared
> >> state between threads on outside callers to the garbage collector.
> >> If the state is that in your wording above then great as I
> >> understand
> >> where we are going and will gladly change my wording.
> >
> > I'm still not sure what you are shooting at, the above sentences do
> > not make any sense to me.
> >
> >> Also freelists don't work here as the state is shared at the
> >> caller's 
> >> end which would need two major issues:
> >> 1. Locking on nodes of the 
> >> freelists when two threads allocate at the same thing which can be
> >> a 
> >> problem if the shared state is shared a lot
> >> 2. Locking allocation with 
> >> large numbers of callers can starve threads
> >
> > First of all allocating memory from the GC pool is not the main
> > work of GIMPLE passes so simply serializing at allocation time
> >> might
> > work out.  Second free lists of course do work.  What you'd do is
> > have a fast path in allocation using a thread-local "free list"
> > which you can allocate from without taking any lock.  Maybe I
> >> should
> > explain "free list" since that term doesn't make too much sense in
> > a garbage collector world.  What I'd do is when a client thread
> > asks for memory of size N allocate M objects of that size but put
> > M - 1 on the client thread local "free list" to be allocated
> >> lock-free
> > from for the next M - 1 

Re: GSOC Proposal

[nick]

On 2019-04-07 5:31 a.m., Richard Biener wrote:
> On April 5, 2019 6:11:15 PM GMT+02:00, nick  wrote:
>>
>>
>> On 2019-04-05 6:25 a.m., Richard Biener wrote:
>>> On Wed, 3 Apr 2019, nick wrote:
>>>


 On 2019-04-03 7:30 a.m., Richard Biener wrote:
> On Mon, 1 Apr 2019, nick wrote:
>
>>
>>
>> On 2019-04-01 9:47 a.m., Richard Biener wrote:
>>> On Mon, 1 Apr 2019, nick wrote:
>>>
 Well I'm talking about the shared roots of this garbage
>> collector core state 
 data structure or just struct ggc_root_tab.

 But also this seems that this to be no longer shared globally if
>> I'm not mistaken 
 or this:
 static vec extra_root_vec;

 Not sure after reading the code which is a bigger deal through
>> so I wrote
 my proposal not just asking which is a better issue for not
>> being thread
 safe. Sorry about that.

 As for the second question injection seems to not be the issue
>> or outside
 callers but just internal so phase 3 or step 3 would now be:
 Find internal callers or users of x where x is one of the above
>> rather
 than injecting outside callers. Which answers my second question
>> about
 external callers being a issue still.

 Let me know which  of the two is a better issue:
 1. struct ggc_root_tabs being shared
 2.static vec extra_root_vec; as a shared
>> heap or
 vector of root nodes for each type of allocation

 and I will gladly rewrite my proposal sections for that
 as needs to be reedited.
>>>
>>> I don't think working on the garbage collector as a separate
>>> GSoC project is useful at this point.  Doing locking around
>>> allocation seems like a good short-term solution and if that
>>> turns out to be a performance issue for the threaded part
>>> using per-thread freelists is likely an easy to deploy
>>> solution.
>>>
>>> Richard.
>>>
>> I agree but we were discussing this:
>> Or maybe a project to be more
>> explicit about regions of the code that assume that the garbage-
>> collector can't run within them?[3] (since the GC is state that
>> would
>> be shared by the threads).
>
> The process of collecting garbage is not the only issue (and that
> very issue is easiest mitigated by collecting only at specific
> points - which is what we do - and have those be serializing
>> points).
> The main issue is the underlying memory allocator (GCC uses memory
> that is garbage collected plus regular heap memory).
>
>> In addition I moved my paper back to our discussion about garbage
>> collector
>> state with outside callers.Seems we really need to do something
>> about
>> my wording as the idea of my project in a nutshell was to figure
>> out how to mark shared state by callers and inject it into the
>> garbage collector letting it known that the state was not shared
>> between
>> threads or shared. Seems that was on the GSoc page and in our
>> discussions the issue
>> is marking outside code for shared state. If that's correct then
>> my
>> wording of outside callers is incorrect it should have been shared
>> state between threads on outside callers to the garbage collector.
>> If the state is that in your wording above then great as I
>> understand
>> where we are going and will gladly change my wording.
>
> I'm still not sure what you are shooting at, the above sentences do
> not make any sense to me.
>
>> Also freelists don't work here as the state is shared at the
>> caller's 
>> end which would need two major issues:
>> 1. Locking on nodes of the 
>> freelists when two threads allocate at the same thing which can be
>> a 
>> problem if the shared state is shared a lot
>> 2. Locking allocation with 
>> large numbers of callers can starve threads
>
> First of all allocating memory from the GC pool is not the main
> work of GIMPLE passes so simply serializing at allocation time
>> might
> work out.  Second free lists of course do work.  What you'd do is
> have a fast path in allocation using a thread-local "free list"
> which you can allocate from without taking any lock.  Maybe I
>> should
> explain "free list" since that term doesn't make too much sense in
> a garbage collector world.  What I'd do is when a client thread
> asks for memory of size N allocate M objects of that size but put
> M - 1 on the client thread local "free list" to be allocated
>> lock-free
> from for the next M - 1 calls.  Note that garbage collected memory
> objects are only handed out in fixed chunks (powers of two plus
> a few special sizes) so you'd have one "free list" per chunk size
> per thread.
>
> The collection itself (mark & sweep) would be fully 

Re: GSOC Proposal

[Richard Biener]

On April 5, 2019 6:11:15 PM GMT+02:00, nick  wrote:
>
>
>On 2019-04-05 6:25 a.m., Richard Biener wrote:
>> On Wed, 3 Apr 2019, nick wrote:
>> 
>>>
>>>
>>> On 2019-04-03 7:30 a.m., Richard Biener wrote:
 On Mon, 1 Apr 2019, nick wrote:

>
>
> On 2019-04-01 9:47 a.m., Richard Biener wrote:
>> On Mon, 1 Apr 2019, nick wrote:
>>
>>> Well I'm talking about the shared roots of this garbage
>collector core state 
>>> data structure or just struct ggc_root_tab.
>>>
>>> But also this seems that this to be no longer shared globally if
>I'm not mistaken 
>>> or this:
>>> static vec extra_root_vec;
>>>
>>> Not sure after reading the code which is a bigger deal through
>so I wrote
>>> my proposal not just asking which is a better issue for not
>being thread
>>> safe. Sorry about that.
>>>
>>> As for the second question injection seems to not be the issue
>or outside
>>> callers but just internal so phase 3 or step 3 would now be:
>>> Find internal callers or users of x where x is one of the above
>rather
>>> than injecting outside callers. Which answers my second question
>about
>>> external callers being a issue still.
>>>
>>> Let me know which  of the two is a better issue:
>>> 1. struct ggc_root_tabs being shared
>>> 2.static vec extra_root_vec; as a shared
>heap or
>>> vector of root nodes for each type of allocation
>>>
>>> and I will gladly rewrite my proposal sections for that
>>> as needs to be reedited.
>>
>> I don't think working on the garbage collector as a separate
>> GSoC project is useful at this point.  Doing locking around
>> allocation seems like a good short-term solution and if that
>> turns out to be a performance issue for the threaded part
>> using per-thread freelists is likely an easy to deploy
>> solution.
>>
>> Richard.
>>
> I agree but we were discussing this:
> Or maybe a project to be more
> explicit about regions of the code that assume that the garbage-
> collector can't run within them?[3] (since the GC is state that
>would
> be shared by the threads).

 The process of collecting garbage is not the only issue (and that
 very issue is easiest mitigated by collecting only at specific
 points - which is what we do - and have those be serializing
>points).
 The main issue is the underlying memory allocator (GCC uses memory
 that is garbage collected plus regular heap memory).

> In addition I moved my paper back to our discussion about garbage
>collector
> state with outside callers.Seems we really need to do something
>about
> my wording as the idea of my project in a nutshell was to figure
> out how to mark shared state by callers and inject it into the
> garbage collector letting it known that the state was not shared
>between
> threads or shared. Seems that was on the GSoc page and in our
>discussions the issue
> is marking outside code for shared state. If that's correct then
>my
> wording of outside callers is incorrect it should have been shared
> state between threads on outside callers to the garbage collector.
> If the state is that in your wording above then great as I
>understand
> where we are going and will gladly change my wording.

 I'm still not sure what you are shooting at, the above sentences do
 not make any sense to me.

> Also freelists don't work here as the state is shared at the
>caller's 
> end which would need two major issues:
> 1. Locking on nodes of the 
> freelists when two threads allocate at the same thing which can be
>a 
> problem if the shared state is shared a lot
> 2. Locking allocation with 
> large numbers of callers can starve threads

 First of all allocating memory from the GC pool is not the main
 work of GIMPLE passes so simply serializing at allocation time
>might
 work out.  Second free lists of course do work.  What you'd do is
 have a fast path in allocation using a thread-local "free list"
 which you can allocate from without taking any lock.  Maybe I
>should
 explain "free list" since that term doesn't make too much sense in
 a garbage collector world.  What I'd do is when a client thread
 asks for memory of size N allocate M objects of that size but put
 M - 1 on the client thread local "free list" to be allocated
>lock-free
 from for the next M - 1 calls.  Note that garbage collected memory
 objects are only handed out in fixed chunks (powers of two plus
 a few special sizes) so you'd have one "free list" per chunk size
 per thread.

 The collection itself (mark & sweep) would be fully serialized
>still
 (and not return to any threads local "free list").

 ggc_free'd objects _might_ go to the threads "free list"s (yeah, we
 _do_ have ggc_free ...).

 As 

Re: GSOC Proposal

[nick]

On 2019-04-05 6:25 a.m., Richard Biener wrote:
> On Wed, 3 Apr 2019, nick wrote:
> 
>>
>>
>> On 2019-04-03 7:30 a.m., Richard Biener wrote:
>>> On Mon, 1 Apr 2019, nick wrote:
>>>


 On 2019-04-01 9:47 a.m., Richard Biener wrote:
> On Mon, 1 Apr 2019, nick wrote:
>
>> Well I'm talking about the shared roots of this garbage collector core 
>> state 
>> data structure or just struct ggc_root_tab.
>>
>> But also this seems that this to be no longer shared globally if I'm not 
>> mistaken 
>> or this:
>> static vec extra_root_vec;
>>
>> Not sure after reading the code which is a bigger deal through so I wrote
>> my proposal not just asking which is a better issue for not being thread
>> safe. Sorry about that.
>>
>> As for the second question injection seems to not be the issue or outside
>> callers but just internal so phase 3 or step 3 would now be:
>> Find internal callers or users of x where x is one of the above rather
>> than injecting outside callers. Which answers my second question about
>> external callers being a issue still.
>>
>> Let me know which  of the two is a better issue:
>> 1. struct ggc_root_tabs being shared
>> 2.static vec extra_root_vec; as a shared heap or
>> vector of root nodes for each type of allocation
>>
>> and I will gladly rewrite my proposal sections for that
>> as needs to be reedited.
>
> I don't think working on the garbage collector as a separate
> GSoC project is useful at this point.  Doing locking around
> allocation seems like a good short-term solution and if that
> turns out to be a performance issue for the threaded part
> using per-thread freelists is likely an easy to deploy
> solution.
>
> Richard.
>
 I agree but we were discussing this:
 Or maybe a project to be more
 explicit about regions of the code that assume that the garbage-
 collector can't run within them?[3] (since the GC is state that would
 be shared by the threads).
>>>
>>> The process of collecting garbage is not the only issue (and that
>>> very issue is easiest mitigated by collecting only at specific
>>> points - which is what we do - and have those be serializing points).
>>> The main issue is the underlying memory allocator (GCC uses memory
>>> that is garbage collected plus regular heap memory).
>>>
 In addition I moved my paper back to our discussion about garbage collector
 state with outside callers.Seems we really need to do something about
 my wording as the idea of my project in a nutshell was to figure
 out how to mark shared state by callers and inject it into the
 garbage collector letting it known that the state was not shared between
 threads or shared. Seems that was on the GSoc page and in our discussions 
 the issue
 is marking outside code for shared state. If that's correct then my
 wording of outside callers is incorrect it should have been shared
 state between threads on outside callers to the garbage collector.
 If the state is that in your wording above then great as I understand
 where we are going and will gladly change my wording.
>>>
>>> I'm still not sure what you are shooting at, the above sentences do
>>> not make any sense to me.
>>>
 Also freelists don't work here as the state is shared at the caller's 
 end which would need two major issues:
 1. Locking on nodes of the 
 freelists when two threads allocate at the same thing which can be a 
 problem if the shared state is shared a lot
 2. Locking allocation with 
 large numbers of callers can starve threads
>>>
>>> First of all allocating memory from the GC pool is not the main
>>> work of GIMPLE passes so simply serializing at allocation time might
>>> work out.  Second free lists of course do work.  What you'd do is
>>> have a fast path in allocation using a thread-local "free list"
>>> which you can allocate from without taking any lock.  Maybe I should
>>> explain "free list" since that term doesn't make too much sense in
>>> a garbage collector world.  What I'd do is when a client thread
>>> asks for memory of size N allocate M objects of that size but put
>>> M - 1 on the client thread local "free list" to be allocated lock-free
>>> from for the next M - 1 calls.  Note that garbage collected memory
>>> objects are only handed out in fixed chunks (powers of two plus
>>> a few special sizes) so you'd have one "free list" per chunk size
>>> per thread.
>>>
>>> The collection itself (mark & sweep) would be fully serialized still
>>> (and not return to any threads local "free list").
>>>
>>> ggc_free'd objects _might_ go to the threads "free list"s (yeah, we
>>> _do_ have ggc_free ...).
>>>
>>> As said, I don't see GC or the memory allocator as sth interesting
>>> to work on for parallelization until the basic setup works and it
>>> proves to be a 

Re: GSOC Proposal

[Richard Biener]

On Wed, 3 Apr 2019, nick wrote:

> 
> 
> On 2019-04-03 7:30 a.m., Richard Biener wrote:
> > On Mon, 1 Apr 2019, nick wrote:
> > 
> >>
> >>
> >> On 2019-04-01 9:47 a.m., Richard Biener wrote:
> >>> On Mon, 1 Apr 2019, nick wrote:
> >>>
>  Well I'm talking about the shared roots of this garbage collector core 
>  state 
>  data structure or just struct ggc_root_tab.
> 
>  But also this seems that this to be no longer shared globally if I'm not 
>  mistaken 
>  or this:
>  static vec extra_root_vec;
> 
>  Not sure after reading the code which is a bigger deal through so I wrote
>  my proposal not just asking which is a better issue for not being thread
>  safe. Sorry about that.
> 
>  As for the second question injection seems to not be the issue or outside
>  callers but just internal so phase 3 or step 3 would now be:
>  Find internal callers or users of x where x is one of the above rather
>  than injecting outside callers. Which answers my second question about
>  external callers being a issue still.
> 
>  Let me know which  of the two is a better issue:
>  1. struct ggc_root_tabs being shared
>  2.static vec extra_root_vec; as a shared heap or
>  vector of root nodes for each type of allocation
> 
>  and I will gladly rewrite my proposal sections for that
>  as needs to be reedited.
> >>>
> >>> I don't think working on the garbage collector as a separate
> >>> GSoC project is useful at this point.  Doing locking around
> >>> allocation seems like a good short-term solution and if that
> >>> turns out to be a performance issue for the threaded part
> >>> using per-thread freelists is likely an easy to deploy
> >>> solution.
> >>>
> >>> Richard.
> >>>
> >> I agree but we were discussing this:
> >> Or maybe a project to be more
> >> explicit about regions of the code that assume that the garbage-
> >> collector can't run within them?[3] (since the GC is state that would
> >> be shared by the threads).
> > 
> > The process of collecting garbage is not the only issue (and that
> > very issue is easiest mitigated by collecting only at specific
> > points - which is what we do - and have those be serializing points).
> > The main issue is the underlying memory allocator (GCC uses memory
> > that is garbage collected plus regular heap memory).
> > 
> >> In addition I moved my paper back to our discussion about garbage collector
> >> state with outside callers.Seems we really need to do something about
> >> my wording as the idea of my project in a nutshell was to figure
> >> out how to mark shared state by callers and inject it into the
> >> garbage collector letting it known that the state was not shared between
> >> threads or shared. Seems that was on the GSoc page and in our discussions 
> >> the issue
> >> is marking outside code for shared state. If that's correct then my
> >> wording of outside callers is incorrect it should have been shared
> >> state between threads on outside callers to the garbage collector.
> >> If the state is that in your wording above then great as I understand
> >> where we are going and will gladly change my wording.
> > 
> > I'm still not sure what you are shooting at, the above sentences do
> > not make any sense to me.
> > 
> >> Also freelists don't work here as the state is shared at the caller's 
> >> end which would need two major issues:
> >> 1. Locking on nodes of the 
> >> freelists when two threads allocate at the same thing which can be a 
> >> problem if the shared state is shared a lot
> >> 2. Locking allocation with 
> >> large numbers of callers can starve threads
> > 
> > First of all allocating memory from the GC pool is not the main
> > work of GIMPLE passes so simply serializing at allocation time might
> > work out.  Second free lists of course do work.  What you'd do is
> > have a fast path in allocation using a thread-local "free list"
> > which you can allocate from without taking any lock.  Maybe I should
> > explain "free list" since that term doesn't make too much sense in
> > a garbage collector world.  What I'd do is when a client thread
> > asks for memory of size N allocate M objects of that size but put
> > M - 1 on the client thread local "free list" to be allocated lock-free
> > from for the next M - 1 calls.  Note that garbage collected memory
> > objects are only handed out in fixed chunks (powers of two plus
> > a few special sizes) so you'd have one "free list" per chunk size
> > per thread.
> > 
> > The collection itself (mark & sweep) would be fully serialized still
> > (and not return to any threads local "free list").
> > 
> > ggc_free'd objects _might_ go to the threads "free list"s (yeah, we
> > _do_ have ggc_free ...).
> > 
> > As said, I don't see GC or the memory allocator as sth interesting
> > to work on for parallelization until the basic setup works and it
> > proves to be a bottleneck.
> > 
> >> Seems that working on the 

GSOC Proposal on GENERIC level issues with threads

[nick]

Richard,

This is the link:
https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit

Seems that the function finalize_compilation_unit is a issue as it's the final
function before the GIMPLE level. It seems to have lots of issues related to
shared state if I'm understanding it correctly. In my proposal I'm assuming
that it's the final GENERIC pass and should be fixed, this is due to even
if the GIMPLE level is fixed we can only pass in shared state there which 
is still not a complete threaded pipeline through all three layers of 
GENERIC,GIMPLE and RTA. We would be bottle necked here and that seems to
be a issue after reading the code.


Let me know if this makes more sense to you as a proposal and feel free to
ask questions if something doesn't make sense,

Nick

Re: GSOC Proposal

[nick]

On 2019-04-03 7:30 a.m., Richard Biener wrote:
> On Mon, 1 Apr 2019, nick wrote:
> 
>>
>>
>> On 2019-04-01 9:47 a.m., Richard Biener wrote:
>>> On Mon, 1 Apr 2019, nick wrote:
>>>
 Well I'm talking about the shared roots of this garbage collector core 
 state 
 data structure or just struct ggc_root_tab.

 But also this seems that this to be no longer shared globally if I'm not 
 mistaken 
 or this:
 static vec extra_root_vec;

 Not sure after reading the code which is a bigger deal through so I wrote
 my proposal not just asking which is a better issue for not being thread
 safe. Sorry about that.

 As for the second question injection seems to not be the issue or outside
 callers but just internal so phase 3 or step 3 would now be:
 Find internal callers or users of x where x is one of the above rather
 than injecting outside callers. Which answers my second question about
 external callers being a issue still.

 Let me know which  of the two is a better issue:
 1. struct ggc_root_tabs being shared
 2.static vec extra_root_vec; as a shared heap or
 vector of root nodes for each type of allocation

 and I will gladly rewrite my proposal sections for that
 as needs to be reedited.
>>>
>>> I don't think working on the garbage collector as a separate
>>> GSoC project is useful at this point.  Doing locking around
>>> allocation seems like a good short-term solution and if that
>>> turns out to be a performance issue for the threaded part
>>> using per-thread freelists is likely an easy to deploy
>>> solution.
>>>
>>> Richard.
>>>
>> I agree but we were discussing this:
>> Or maybe a project to be more
>> explicit about regions of the code that assume that the garbage-
>> collector can't run within them?[3] (since the GC is state that would
>> be shared by the threads).
> 
> The process of collecting garbage is not the only issue (and that
> very issue is easiest mitigated by collecting only at specific
> points - which is what we do - and have those be serializing points).
> The main issue is the underlying memory allocator (GCC uses memory
> that is garbage collected plus regular heap memory).
> 
>> In addition I moved my paper back to our discussion about garbage collector
>> state with outside callers.Seems we really need to do something about
>> my wording as the idea of my project in a nutshell was to figure
>> out how to mark shared state by callers and inject it into the
>> garbage collector letting it known that the state was not shared between
>> threads or shared. Seems that was on the GSoc page and in our discussions 
>> the issue
>> is marking outside code for shared state. If that's correct then my
>> wording of outside callers is incorrect it should have been shared
>> state between threads on outside callers to the garbage collector.
>> If the state is that in your wording above then great as I understand
>> where we are going and will gladly change my wording.
> 
> I'm still not sure what you are shooting at, the above sentences do
> not make any sense to me.
> 
>> Also freelists don't work here as the state is shared at the caller's 
>> end which would need two major issues:
>> 1. Locking on nodes of the 
>> freelists when two threads allocate at the same thing which can be a 
>> problem if the shared state is shared a lot
>> 2. Locking allocation with 
>> large numbers of callers can starve threads
> 
> First of all allocating memory from the GC pool is not the main
> work of GIMPLE passes so simply serializing at allocation time might
> work out.  Second free lists of course do work.  What you'd do is
> have a fast path in allocation using a thread-local "free list"
> which you can allocate from without taking any lock.  Maybe I should
> explain "free list" since that term doesn't make too much sense in
> a garbage collector world.  What I'd do is when a client thread
> asks for memory of size N allocate M objects of that size but put
> M - 1 on the client thread local "free list" to be allocated lock-free
> from for the next M - 1 calls.  Note that garbage collected memory
> objects are only handed out in fixed chunks (powers of two plus
> a few special sizes) so you'd have one "free list" per chunk size
> per thread.
> 
> The collection itself (mark & sweep) would be fully serialized still
> (and not return to any threads local "free list").
> 
> ggc_free'd objects _might_ go to the threads "free list"s (yeah, we
> _do_ have ggc_free ...).
> 
> As said, I don't see GC or the memory allocator as sth interesting
> to work on for parallelization until the basic setup works and it
> proves to be a bottleneck.
> 
>> Seems that working on the garbage collector itself isn't the issue but 
>> the callers as I just figured out as related to your state idea. Let me 
>> know if that's correct and if the wording change I mentioned is fine 
>> with you as that's the state it seems that needs to 

Re: GSOC Proposal

[Richard Biener]

On Mon, 1 Apr 2019, nick wrote:

> 
> 
> On 2019-04-01 9:47 a.m., Richard Biener wrote:
> > On Mon, 1 Apr 2019, nick wrote:
> > 
> >> Well I'm talking about the shared roots of this garbage collector core 
> >> state 
> >> data structure or just struct ggc_root_tab.
> >>
> >> But also this seems that this to be no longer shared globally if I'm not 
> >> mistaken 
> >> or this:
> >> static vec extra_root_vec;
> >>
> >> Not sure after reading the code which is a bigger deal through so I wrote
> >> my proposal not just asking which is a better issue for not being thread
> >> safe. Sorry about that.
> >>
> >> As for the second question injection seems to not be the issue or outside
> >> callers but just internal so phase 3 or step 3 would now be:
> >> Find internal callers or users of x where x is one of the above rather
> >> than injecting outside callers. Which answers my second question about
> >> external callers being a issue still.
> >>
> >> Let me know which  of the two is a better issue:
> >> 1. struct ggc_root_tabs being shared
> >> 2.static vec extra_root_vec; as a shared heap or
> >> vector of root nodes for each type of allocation
> >>
> >> and I will gladly rewrite my proposal sections for that
> >> as needs to be reedited.
> > 
> > I don't think working on the garbage collector as a separate
> > GSoC project is useful at this point.  Doing locking around
> > allocation seems like a good short-term solution and if that
> > turns out to be a performance issue for the threaded part
> > using per-thread freelists is likely an easy to deploy
> > solution.
> > 
> > Richard.
> > 
> I agree but we were discussing this:
> Or maybe a project to be more
> explicit about regions of the code that assume that the garbage-
> collector can't run within them?[3] (since the GC is state that would
> be shared by the threads).

The process of collecting garbage is not the only issue (and that
very issue is easiest mitigated by collecting only at specific
points - which is what we do - and have those be serializing points).
The main issue is the underlying memory allocator (GCC uses memory
that is garbage collected plus regular heap memory).

> In addition I moved my paper back to our discussion about garbage collector
> state with outside callers.Seems we really need to do something about
> my wording as the idea of my project in a nutshell was to figure
> out how to mark shared state by callers and inject it into the
> garbage collector letting it known that the state was not shared between
> threads or shared. Seems that was on the GSoc page and in our discussions the 
> issue
> is marking outside code for shared state. If that's correct then my
> wording of outside callers is incorrect it should have been shared
> state between threads on outside callers to the garbage collector.
> If the state is that in your wording above then great as I understand
> where we are going and will gladly change my wording.

I'm still not sure what you are shooting at, the above sentences do
not make any sense to me.

> Also freelists don't work here as the state is shared at the caller's 
> end which would need two major issues:
> 1. Locking on nodes of the 
> freelists when two threads allocate at the same thing which can be a 
> problem if the shared state is shared a lot
> 2. Locking allocation with 
> large numbers of callers can starve threads

First of all allocating memory from the GC pool is not the main
work of GIMPLE passes so simply serializing at allocation time might
work out.  Second free lists of course do work.  What you'd do is
have a fast path in allocation using a thread-local "free list"
which you can allocate from without taking any lock.  Maybe I should
explain "free list" since that term doesn't make too much sense in
a garbage collector world.  What I'd do is when a client thread
asks for memory of size N allocate M objects of that size but put
M - 1 on the client thread local "free list" to be allocated lock-free
from for the next M - 1 calls.  Note that garbage collected memory
objects are only handed out in fixed chunks (powers of two plus
a few special sizes) so you'd have one "free list" per chunk size
per thread.

The collection itself (mark & sweep) would be fully serialized still
(and not return to any threads local "free list").

ggc_free'd objects _might_ go to the threads "free list"s (yeah, we
_do_ have ggc_free ...).

As said, I don't see GC or the memory allocator as sth interesting
to work on for parallelization until the basic setup works and it
proves to be a bottleneck.

> Seems that working on the garbage collector itself isn't the issue but 
> the callers as I just figured out as related to your state idea. Let me 
> know if that's correct and if the wording change I mentioned is fine 
> with you as that's the state it seems that needs to be changed.
> Nick 

Richard.

-- 
Richard Biener 
SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany;
GF: Felix Imendörffer, Mary Higgins, 

Re: GSOC Proposal

[nick]

On 2019-04-01 9:47 a.m., Richard Biener wrote:
> On Mon, 1 Apr 2019, nick wrote:
> 
>>
>>
>> On 2019-04-01 5:56 a.m., Richard Biener wrote:
>>> On Fri, 29 Mar 2019, nick wrote:
>>>


 On 2019-03-29 10:28 a.m., nick wrote:
>
>
> On 2019-03-29 5:08 a.m., Richard Biener wrote:
>> On Thu, 28 Mar 2019, nick wrote:
>>
>>>
>>>
>>> On 2019-03-28 4:59 a.m., Richard Biener wrote:
 On Wed, Mar 27, 2019 at 6:31 PM nick  wrote:
>
> Greetings All,
>
> I've already done most of the work required for signing up for GSoC
> as of last year i.e. reading getting started, being signed up legally
> for contributions.
>
> My only real concern would be the proposal which I started writing 
> here:
> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing
>
> The biography and success section I'm fine with my bigger concern 
> would be the project and roadmap
> section. The roadmap is there and I will go into more detail about it 
> in the projects section as
> need be. Just wanted to known if the roadmap is detailed enough or 
> can I just write out a few
> paragraphs discussing it in the Projects Section.

 I'm not sure I understand either the problem analysis nor the project
 goal parts.  What
 shared state with respect to garbage collection are you talking about?

 Richard.

>>> I just fixed it. Seems we were discussing RTL itself. I edited it to 
>>> reflect those changes. Let me know if it's unclear or you would 
>>> actually 
>>> like me to discuss some changes that may occur in the RTL layer itself.
>>>
>>>
>>> I'm glad to be more exact if that's better but seems your confusion was 
>>> just what layer we were touching.
>>
>> Let me just throw in some knowledge here.  The issue with RTL
>> is that we currently can only have a single function in this
>> intermediate language state since a function in RTL has some
>> state in global variables that would differ if it were another
>> function.  We can have multiple functions in GIMPLE intermediate
>> language state since all such state is in a function-specific
>> data structure (struct function).  The hard thing about moving
>> all this "global" state of RTL into the same place is that
>> there's global state in the various backends (and there's
>> already a struct funtion 'machine' part for such state, so there's
>> hope the issue isn't as big as it could be) and that some of
>> the global state is big and only changes very rarely.
>> That said, I'm not sure if anybody knows the full details here.
>>
>> So as far as I understand you'd like to tackle this as project
>> with the goal to be able to have multiple functions in RTL
>> state.
>>
>> That's laudable but IMHO also quite ambitious for a GSoC
>> project.  It's also an area I am not very familiar with so
>> I opt out of being a mentor for this project.
>>
> While I'm aware of three areas where the shared state is an issue
> currently:
> 1, Compiler's Proper
> 2. The expand_functions 
> 3. RTL
> 4.Garbage Collector
>
> Or maybe a project to be more
> explicit about regions of the code that assume that the garbage-
> collector can't run within them?[3] (since the GC is state that would
> be shared by the threads).
>
> This is what we were discussing previously and I wrote my proposal for
> that. You however seem confused about what parts of the garbage collector
> would be touched. That's fine with me, however seems you want be to
> be more exact about which part  is touched.
>
> My questions would be as it's changed back to the garbage collector 
> project:
> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit
>
> 1. Your confusion about which part of the garbage collector is touched 
> doesn't
> really make sense s it's for the whole garbage collector as related to 
> shared
> state?
> 2. Injection was my code here in phase 3 for the callers of the new 
> functions or
> macros, perhaps this is not needed as the work with the garbage collector 
> is enough?
> 3. Am I not understanding this project as I thought I was in the proposal 
> I wrote?
>
> Seems your more confusing my wording probably so I'm going to suggest one 
> of 
> two things here:
> a) I'm going to allow you to make comments with what's confusing you and
> it needs that's the issue here more than anything else so I sent you 
> a link and please comment where you are having issues with this not
> be clear for you:
> Or maybe a project to be more
> 

Re: GSOC Proposal

[Richard Biener]

On Mon, 1 Apr 2019, nick wrote:

> 
> 
> On 2019-04-01 5:56 a.m., Richard Biener wrote:
> > On Fri, 29 Mar 2019, nick wrote:
> > 
> >>
> >>
> >> On 2019-03-29 10:28 a.m., nick wrote:
> >>>
> >>>
> >>> On 2019-03-29 5:08 a.m., Richard Biener wrote:
>  On Thu, 28 Mar 2019, nick wrote:
> 
> >
> >
> > On 2019-03-28 4:59 a.m., Richard Biener wrote:
> >> On Wed, Mar 27, 2019 at 6:31 PM nick  wrote:
> >>>
> >>> Greetings All,
> >>>
> >>> I've already done most of the work required for signing up for GSoC
> >>> as of last year i.e. reading getting started, being signed up legally
> >>> for contributions.
> >>>
> >>> My only real concern would be the proposal which I started writing 
> >>> here:
> >>> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing
> >>>
> >>> The biography and success section I'm fine with my bigger concern 
> >>> would be the project and roadmap
> >>> section. The roadmap is there and I will go into more detail about it 
> >>> in the projects section as
> >>> need be. Just wanted to known if the roadmap is detailed enough or 
> >>> can I just write out a few
> >>> paragraphs discussing it in the Projects Section.
> >>
> >> I'm not sure I understand either the problem analysis nor the project
> >> goal parts.  What
> >> shared state with respect to garbage collection are you talking about?
> >>
> >> Richard.
> >>
> > I just fixed it. Seems we were discussing RTL itself. I edited it to 
> > reflect those changes. Let me know if it's unclear or you would 
> > actually 
> > like me to discuss some changes that may occur in the RTL layer itself.
> >
> >
> > I'm glad to be more exact if that's better but seems your confusion was 
> > just what layer we were touching.
> 
>  Let me just throw in some knowledge here.  The issue with RTL
>  is that we currently can only have a single function in this
>  intermediate language state since a function in RTL has some
>  state in global variables that would differ if it were another
>  function.  We can have multiple functions in GIMPLE intermediate
>  language state since all such state is in a function-specific
>  data structure (struct function).  The hard thing about moving
>  all this "global" state of RTL into the same place is that
>  there's global state in the various backends (and there's
>  already a struct funtion 'machine' part for such state, so there's
>  hope the issue isn't as big as it could be) and that some of
>  the global state is big and only changes very rarely.
>  That said, I'm not sure if anybody knows the full details here.
> 
>  So as far as I understand you'd like to tackle this as project
>  with the goal to be able to have multiple functions in RTL
>  state.
> 
>  That's laudable but IMHO also quite ambitious for a GSoC
>  project.  It's also an area I am not very familiar with so
>  I opt out of being a mentor for this project.
> 
> >>> While I'm aware of three areas where the shared state is an issue
> >>> currently:
> >>> 1, Compiler's Proper
> >>> 2. The expand_functions 
> >>> 3. RTL
> >>> 4.Garbage Collector
> >>>
> >>> Or maybe a project to be more
> >>> explicit about regions of the code that assume that the garbage-
> >>> collector can't run within them?[3] (since the GC is state that would
> >>> be shared by the threads).
> >>>
> >>> This is what we were discussing previously and I wrote my proposal for
> >>> that. You however seem confused about what parts of the garbage collector
> >>> would be touched. That's fine with me, however seems you want be to
> >>> be more exact about which part  is touched.
> >>>
> >>> My questions would be as it's changed back to the garbage collector 
> >>> project:
> >>> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit
> >>>
> >>> 1. Your confusion about which part of the garbage collector is touched 
> >>> doesn't
> >>> really make sense s it's for the whole garbage collector as related to 
> >>> shared
> >>> state?
> >>> 2. Injection was my code here in phase 3 for the callers of the new 
> >>> functions or
> >>> macros, perhaps this is not needed as the work with the garbage collector 
> >>> is enough?
> >>> 3. Am I not understanding this project as I thought I was in the proposal 
> >>> I wrote?
> >>>
> >>> Seems your more confusing my wording probably so I'm going to suggest one 
> >>> of 
> >>> two things here:
> >>> a) I'm going to allow you to make comments with what's confusing you and
> >>> it needs that's the issue here more than anything else so I sent you 
> >>> a link and please comment where you are having issues with this not
> >>> be clear for you:
> >>> Or maybe a project to be more
> >>> explicit about regions of the code that assume that 

Re: GSOC Proposal

[nick]

On 2019-04-01 5:56 a.m., Richard Biener wrote:
> On Fri, 29 Mar 2019, nick wrote:
> 
>>
>>
>> On 2019-03-29 10:28 a.m., nick wrote:
>>>
>>>
>>> On 2019-03-29 5:08 a.m., Richard Biener wrote:
 On Thu, 28 Mar 2019, nick wrote:

>
>
> On 2019-03-28 4:59 a.m., Richard Biener wrote:
>> On Wed, Mar 27, 2019 at 6:31 PM nick  wrote:
>>>
>>> Greetings All,
>>>
>>> I've already done most of the work required for signing up for GSoC
>>> as of last year i.e. reading getting started, being signed up legally
>>> for contributions.
>>>
>>> My only real concern would be the proposal which I started writing here:
>>> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing
>>>
>>> The biography and success section I'm fine with my bigger concern would 
>>> be the project and roadmap
>>> section. The roadmap is there and I will go into more detail about it 
>>> in the projects section as
>>> need be. Just wanted to known if the roadmap is detailed enough or can 
>>> I just write out a few
>>> paragraphs discussing it in the Projects Section.
>>
>> I'm not sure I understand either the problem analysis nor the project
>> goal parts.  What
>> shared state with respect to garbage collection are you talking about?
>>
>> Richard.
>>
> I just fixed it. Seems we were discussing RTL itself. I edited it to 
> reflect those changes. Let me know if it's unclear or you would actually 
> like me to discuss some changes that may occur in the RTL layer itself.
>
>
> I'm glad to be more exact if that's better but seems your confusion was 
> just what layer we were touching.

 Let me just throw in some knowledge here.  The issue with RTL
 is that we currently can only have a single function in this
 intermediate language state since a function in RTL has some
 state in global variables that would differ if it were another
 function.  We can have multiple functions in GIMPLE intermediate
 language state since all such state is in a function-specific
 data structure (struct function).  The hard thing about moving
 all this "global" state of RTL into the same place is that
 there's global state in the various backends (and there's
 already a struct funtion 'machine' part for such state, so there's
 hope the issue isn't as big as it could be) and that some of
 the global state is big and only changes very rarely.
 That said, I'm not sure if anybody knows the full details here.

 So as far as I understand you'd like to tackle this as project
 with the goal to be able to have multiple functions in RTL
 state.

 That's laudable but IMHO also quite ambitious for a GSoC
 project.  It's also an area I am not very familiar with so
 I opt out of being a mentor for this project.

>>> While I'm aware of three areas where the shared state is an issue
>>> currently:
>>> 1, Compiler's Proper
>>> 2. The expand_functions 
>>> 3. RTL
>>> 4.Garbage Collector
>>>
>>> Or maybe a project to be more
>>> explicit about regions of the code that assume that the garbage-
>>> collector can't run within them?[3] (since the GC is state that would
>>> be shared by the threads).
>>>
>>> This is what we were discussing previously and I wrote my proposal for
>>> that. You however seem confused about what parts of the garbage collector
>>> would be touched. That's fine with me, however seems you want be to
>>> be more exact about which part  is touched.
>>>
>>> My questions would be as it's changed back to the garbage collector project:
>>> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit
>>>
>>> 1. Your confusion about which part of the garbage collector is touched 
>>> doesn't
>>> really make sense s it's for the whole garbage collector as related to 
>>> shared
>>> state?
>>> 2. Injection was my code here in phase 3 for the callers of the new 
>>> functions or
>>> macros, perhaps this is not needed as the work with the garbage collector 
>>> is enough?
>>> 3. Am I not understanding this project as I thought I was in the proposal I 
>>> wrote?
>>>
>>> Seems your more confusing my wording probably so I'm going to suggest one 
>>> of 
>>> two things here:
>>> a) I'm going to allow you to make comments with what's confusing you and
>>> it needs that's the issue here more than anything else so I sent you 
>>> a link and please comment where you are having issues with this not
>>> be clear for you:
>>> Or maybe a project to be more
>>> explicit about regions of the code that assume that the garbage-
>>> collector can't run within them?[3] (since the GC is state that would
>>> be shared by the threads).
>>> as that's the actual project
>>>
>>> b) Just comment here about the wording that's an issue for you or
>>> where you want more exact wording
>>>
>>> Sorry and 

Re: GSOC Proposal

[Nathan Sidwell]

On 4/1/19 1:24 AM, Eric Gallager wrote:

On 3/29/19, nick  wrote:



Seems your right touching the complete garbage collector is too much. I'm
just looking at the users of the garbage collector and it seems one of the
major ones is pre compiled headers.



The thing about pre-compiled headers is that I seem to remember some
GCC devs saying they wanted to rip out pre-compiled headers completely
once the C++ modules branch is merged, so I'm not sure if it's worth
putting that much work into something that might be removed soon,
anyways... I'm pretty sure Nathan Sidwell is the main person working
on the C++ modules branch, so I'm cc-ing him.


The use of the GC machinery for PCH is needed by the front ends and is 
orthogonal to that for GC during code generation.


nathan

--
Nathan Sidwell

Re: GSOC Proposal

[Richard Biener]

On Fri, 29 Mar 2019, nick wrote:

> 
> 
> On 2019-03-29 10:28 a.m., nick wrote:
> > 
> > 
> > On 2019-03-29 5:08 a.m., Richard Biener wrote:
> >> On Thu, 28 Mar 2019, nick wrote:
> >>
> >>>
> >>>
> >>> On 2019-03-28 4:59 a.m., Richard Biener wrote:
>  On Wed, Mar 27, 2019 at 6:31 PM nick  wrote:
> >
> > Greetings All,
> >
> > I've already done most of the work required for signing up for GSoC
> > as of last year i.e. reading getting started, being signed up legally
> > for contributions.
> >
> > My only real concern would be the proposal which I started writing here:
> > https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing
> >
> > The biography and success section I'm fine with my bigger concern would 
> > be the project and roadmap
> > section. The roadmap is there and I will go into more detail about it 
> > in the projects section as
> > need be. Just wanted to known if the roadmap is detailed enough or can 
> > I just write out a few
> > paragraphs discussing it in the Projects Section.
> 
>  I'm not sure I understand either the problem analysis nor the project
>  goal parts.  What
>  shared state with respect to garbage collection are you talking about?
> 
>  Richard.
> 
> >>> I just fixed it. Seems we were discussing RTL itself. I edited it to 
> >>> reflect those changes. Let me know if it's unclear or you would actually 
> >>> like me to discuss some changes that may occur in the RTL layer itself.
> >>>
> >>>
> >>> I'm glad to be more exact if that's better but seems your confusion was 
> >>> just what layer we were touching.
> >>
> >> Let me just throw in some knowledge here.  The issue with RTL
> >> is that we currently can only have a single function in this
> >> intermediate language state since a function in RTL has some
> >> state in global variables that would differ if it were another
> >> function.  We can have multiple functions in GIMPLE intermediate
> >> language state since all such state is in a function-specific
> >> data structure (struct function).  The hard thing about moving
> >> all this "global" state of RTL into the same place is that
> >> there's global state in the various backends (and there's
> >> already a struct funtion 'machine' part for such state, so there's
> >> hope the issue isn't as big as it could be) and that some of
> >> the global state is big and only changes very rarely.
> >> That said, I'm not sure if anybody knows the full details here.
> >>
> >> So as far as I understand you'd like to tackle this as project
> >> with the goal to be able to have multiple functions in RTL
> >> state.
> >>
> >> That's laudable but IMHO also quite ambitious for a GSoC
> >> project.  It's also an area I am not very familiar with so
> >> I opt out of being a mentor for this project.
> >>
> > While I'm aware of three areas where the shared state is an issue
> > currently:
> > 1, Compiler's Proper
> > 2. The expand_functions 
> > 3. RTL
> > 4.Garbage Collector
> > 
> > Or maybe a project to be more
> > explicit about regions of the code that assume that the garbage-
> > collector can't run within them?[3] (since the GC is state that would
> > be shared by the threads).
> > 
> > This is what we were discussing previously and I wrote my proposal for
> > that. You however seem confused about what parts of the garbage collector
> > would be touched. That's fine with me, however seems you want be to
> > be more exact about which part  is touched.
> > 
> > My questions would be as it's changed back to the garbage collector project:
> > https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit
> > 
> > 1. Your confusion about which part of the garbage collector is touched 
> > doesn't
> > really make sense s it's for the whole garbage collector as related to 
> > shared
> > state?
> > 2. Injection was my code here in phase 3 for the callers of the new 
> > functions or
> > macros, perhaps this is not needed as the work with the garbage collector 
> > is enough?
> > 3. Am I not understanding this project as I thought I was in the proposal I 
> > wrote?
> > 
> > Seems your more confusing my wording probably so I'm going to suggest one 
> > of 
> > two things here:
> > a) I'm going to allow you to make comments with what's confusing you and
> > it needs that's the issue here more than anything else so I sent you 
> > a link and please comment where you are having issues with this not
> > be clear for you:
> > Or maybe a project to be more
> > explicit about regions of the code that assume that the garbage-
> > collector can't run within them?[3] (since the GC is state that would
> > be shared by the threads).
> > as that's the actual project
> > 
> > b) Just comment here about the wording that's an issue for you or
> > where you want more exact wording
> > 
> > Sorry and hopefully this is helps you understand where I'm 

Re: GSOC Proposal

[Eric Gallager]

On 3/29/19, nick  wrote:
>
>
> On 2019-03-29 10:28 a.m., nick wrote:
>>
>>
>> On 2019-03-29 5:08 a.m., Richard Biener wrote:
>>> On Thu, 28 Mar 2019, nick wrote:
>>>


 On 2019-03-28 4:59 a.m., Richard Biener wrote:
> On Wed, Mar 27, 2019 at 6:31 PM nick  wrote:
>>
>> Greetings All,
>>
>> I've already done most of the work required for signing up for GSoC
>> as of last year i.e. reading getting started, being signed up legally
>> for contributions.
>>
>> My only real concern would be the proposal which I started writing
>> here:
>> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing
>>
>> The biography and success section I'm fine with my bigger concern
>> would be the project and roadmap
>> section. The roadmap is there and I will go into more detail about it
>> in the projects section as
>> need be. Just wanted to known if the roadmap is detailed enough or can
>> I just write out a few
>> paragraphs discussing it in the Projects Section.
>
> I'm not sure I understand either the problem analysis nor the project
> goal parts.  What
> shared state with respect to garbage collection are you talking about?
>
> Richard.
>
 I just fixed it. Seems we were discussing RTL itself. I edited it to
 reflect those changes. Let me know if it's unclear or you would actually

 like me to discuss some changes that may occur in the RTL layer itself.


 I'm glad to be more exact if that's better but seems your confusion was

 just what layer we were touching.
>>>
>>> Let me just throw in some knowledge here.  The issue with RTL
>>> is that we currently can only have a single function in this
>>> intermediate language state since a function in RTL has some
>>> state in global variables that would differ if it were another
>>> function.  We can have multiple functions in GIMPLE intermediate
>>> language state since all such state is in a function-specific
>>> data structure (struct function).  The hard thing about moving
>>> all this "global" state of RTL into the same place is that
>>> there's global state in the various backends (and there's
>>> already a struct funtion 'machine' part for such state, so there's
>>> hope the issue isn't as big as it could be) and that some of
>>> the global state is big and only changes very rarely.
>>> That said, I'm not sure if anybody knows the full details here.
>>>
>>> So as far as I understand you'd like to tackle this as project
>>> with the goal to be able to have multiple functions in RTL
>>> state.
>>>
>>> That's laudable but IMHO also quite ambitious for a GSoC
>>> project.  It's also an area I am not very familiar with so
>>> I opt out of being a mentor for this project.
>>>
>> While I'm aware of three areas where the shared state is an issue
>> currently:
>> 1, Compiler's Proper
>> 2. The expand_functions
>> 3. RTL
>> 4.Garbage Collector
>>
>> Or maybe a project to be more
>> explicit about regions of the code that assume that the garbage-
>> collector can't run within them?[3] (since the GC is state that would
>> be shared by the threads).
>>
>> This is what we were discussing previously and I wrote my proposal for
>> that. You however seem confused about what parts of the garbage collector
>> would be touched. That's fine with me, however seems you want be to
>> be more exact about which part  is touched.
>>
>> My questions would be as it's changed back to the garbage collector
>> project:
>> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit
>>
>> 1. Your confusion about which part of the garbage collector is touched
>> doesn't
>> really make sense s it's for the whole garbage collector as related to
>> shared
>> state?
>> 2. Injection was my code here in phase 3 for the callers of the new
>> functions or
>> macros, perhaps this is not needed as the work with the garbage collector
>> is enough?
>> 3. Am I not understanding this project as I thought I was in the proposal
>> I wrote?
>>
>> Seems your more confusing my wording probably so I'm going to suggest one
>> of
>> two things here:
>> a) I'm going to allow you to make comments with what's confusing you and
>> it needs that's the issue here more than anything else so I sent you
>> a link and please comment where you are having issues with this not
>> be clear for you:
>> Or maybe a project to be more
>> explicit about regions of the code that assume that the garbage-
>> collector can't run within them?[3] (since the GC is state that would
>> be shared by the threads).
>> as that's the actual project
>>
>> b) Just comment here about the wording that's an issue for you or
>> where you want more exact wording
>>
>> Sorry and hopefully this is helps you understand where I'm going,
>> Nick
>>
>>> Richard.
>>>
 Nick
>> Any other comments are welcome as well as I write it there,
>> Nick

Re: GSOC Proposal

[nick]

On 2019-03-29 10:28 a.m., nick wrote:
> 
> 
> On 2019-03-29 5:08 a.m., Richard Biener wrote:
>> On Thu, 28 Mar 2019, nick wrote:
>>
>>>
>>>
>>> On 2019-03-28 4:59 a.m., Richard Biener wrote:
 On Wed, Mar 27, 2019 at 6:31 PM nick  wrote:
>
> Greetings All,
>
> I've already done most of the work required for signing up for GSoC
> as of last year i.e. reading getting started, being signed up legally
> for contributions.
>
> My only real concern would be the proposal which I started writing here:
> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing
>
> The biography and success section I'm fine with my bigger concern would 
> be the project and roadmap
> section. The roadmap is there and I will go into more detail about it in 
> the projects section as
> need be. Just wanted to known if the roadmap is detailed enough or can I 
> just write out a few
> paragraphs discussing it in the Projects Section.

 I'm not sure I understand either the problem analysis nor the project
 goal parts.  What
 shared state with respect to garbage collection are you talking about?

 Richard.

>>> I just fixed it. Seems we were discussing RTL itself. I edited it to 
>>> reflect those changes. Let me know if it's unclear or you would actually 
>>> like me to discuss some changes that may occur in the RTL layer itself.
>>>
>>>
>>> I'm glad to be more exact if that's better but seems your confusion was 
>>> just what layer we were touching.
>>
>> Let me just throw in some knowledge here.  The issue with RTL
>> is that we currently can only have a single function in this
>> intermediate language state since a function in RTL has some
>> state in global variables that would differ if it were another
>> function.  We can have multiple functions in GIMPLE intermediate
>> language state since all such state is in a function-specific
>> data structure (struct function).  The hard thing about moving
>> all this "global" state of RTL into the same place is that
>> there's global state in the various backends (and there's
>> already a struct funtion 'machine' part for such state, so there's
>> hope the issue isn't as big as it could be) and that some of
>> the global state is big and only changes very rarely.
>> That said, I'm not sure if anybody knows the full details here.
>>
>> So as far as I understand you'd like to tackle this as project
>> with the goal to be able to have multiple functions in RTL
>> state.
>>
>> That's laudable but IMHO also quite ambitious for a GSoC
>> project.  It's also an area I am not very familiar with so
>> I opt out of being a mentor for this project.
>>
> While I'm aware of three areas where the shared state is an issue
> currently:
> 1, Compiler's Proper
> 2. The expand_functions 
> 3. RTL
> 4.Garbage Collector
> 
> Or maybe a project to be more
> explicit about regions of the code that assume that the garbage-
> collector can't run within them?[3] (since the GC is state that would
> be shared by the threads).
> 
> This is what we were discussing previously and I wrote my proposal for
> that. You however seem confused about what parts of the garbage collector
> would be touched. That's fine with me, however seems you want be to
> be more exact about which part  is touched.
> 
> My questions would be as it's changed back to the garbage collector project:
> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit
> 
> 1. Your confusion about which part of the garbage collector is touched doesn't
> really make sense s it's for the whole garbage collector as related to shared
> state?
> 2. Injection was my code here in phase 3 for the callers of the new functions 
> or
> macros, perhaps this is not needed as the work with the garbage collector is 
> enough?
> 3. Am I not understanding this project as I thought I was in the proposal I 
> wrote?
> 
> Seems your more confusing my wording probably so I'm going to suggest one of 
> two things here:
> a) I'm going to allow you to make comments with what's confusing you and
> it needs that's the issue here more than anything else so I sent you 
> a link and please comment where you are having issues with this not
> be clear for you:
> Or maybe a project to be more
> explicit about regions of the code that assume that the garbage-
> collector can't run within them?[3] (since the GC is state that would
> be shared by the threads).
> as that's the actual project
> 
> b) Just comment here about the wording that's an issue for you or
> where you want more exact wording
> 
> Sorry and hopefully this is helps you understand where I'm going,
> Nick
> 
>> Richard.
>>
>>> Nick
> Any other comments are welcome as well as I write it there,
> Nick
>>>

Richard,

Seems your right touching the complete garbage collector is too much. I'm
just looking at the users of the garbage collector and it seems 

Re: GSOC Proposal

[nick]

On 2019-03-29 5:08 a.m., Richard Biener wrote:
> On Thu, 28 Mar 2019, nick wrote:
> 
>>
>>
>> On 2019-03-28 4:59 a.m., Richard Biener wrote:
>>> On Wed, Mar 27, 2019 at 6:31 PM nick  wrote:

 Greetings All,

 I've already done most of the work required for signing up for GSoC
 as of last year i.e. reading getting started, being signed up legally
 for contributions.

 My only real concern would be the proposal which I started writing here:
 https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing

 The biography and success section I'm fine with my bigger concern would be 
 the project and roadmap
 section. The roadmap is there and I will go into more detail about it in 
 the projects section as
 need be. Just wanted to known if the roadmap is detailed enough or can I 
 just write out a few
 paragraphs discussing it in the Projects Section.
>>>
>>> I'm not sure I understand either the problem analysis nor the project
>>> goal parts.  What
>>> shared state with respect to garbage collection are you talking about?
>>>
>>> Richard.
>>>
>> I just fixed it. Seems we were discussing RTL itself. I edited it to 
>> reflect those changes. Let me know if it's unclear or you would actually 
>> like me to discuss some changes that may occur in the RTL layer itself.
>>
>>
>> I'm glad to be more exact if that's better but seems your confusion was 
>> just what layer we were touching.
> 
> Let me just throw in some knowledge here.  The issue with RTL
> is that we currently can only have a single function in this
> intermediate language state since a function in RTL has some
> state in global variables that would differ if it were another
> function.  We can have multiple functions in GIMPLE intermediate
> language state since all such state is in a function-specific
> data structure (struct function).  The hard thing about moving
> all this "global" state of RTL into the same place is that
> there's global state in the various backends (and there's
> already a struct funtion 'machine' part for such state, so there's
> hope the issue isn't as big as it could be) and that some of
> the global state is big and only changes very rarely.
> That said, I'm not sure if anybody knows the full details here.
> 
> So as far as I understand you'd like to tackle this as project
> with the goal to be able to have multiple functions in RTL
> state.
> 
> That's laudable but IMHO also quite ambitious for a GSoC
> project.  It's also an area I am not very familiar with so
> I opt out of being a mentor for this project.
> 
While I'm aware of three areas where the shared state is an issue
currently:
1, Compiler's Proper
2. The expand_functions 
3. RTL
4.Garbage Collector

Or maybe a project to be more
explicit about regions of the code that assume that the garbage-
collector can't run within them?[3] (since the GC is state that would
be shared by the threads).

This is what we were discussing previously and I wrote my proposal for
that. You however seem confused about what parts of the garbage collector
would be touched. That's fine with me, however seems you want be to
be more exact about which part  is touched.

My questions would be as it's changed back to the garbage collector project:
https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit

1. Your confusion about which part of the garbage collector is touched doesn't
really make sense s it's for the whole garbage collector as related to shared
state?
2. Injection was my code here in phase 3 for the callers of the new functions or
macros, perhaps this is not needed as the work with the garbage collector is 
enough?
3. Am I not understanding this project as I thought I was in the proposal I 
wrote?

Seems your more confusing my wording probably so I'm going to suggest one of 
two things here:
a) I'm going to allow you to make comments with what's confusing you and
it needs that's the issue here more than anything else so I sent you 
a link and please comment where you are having issues with this not
be clear for you:
Or maybe a project to be more
explicit about regions of the code that assume that the garbage-
collector can't run within them?[3] (since the GC is state that would
be shared by the threads).
as that's the actual project

b) Just comment here about the wording that's an issue for you or
where you want more exact wording

Sorry and hopefully this is helps you understand where I'm going,
Nick

> Richard.
> 
>> Nick
 Any other comments are welcome as well as I write it there,
 Nick
>>
> 

Re: GSOC Proposal

[Richard Biener]

On Thu, 28 Mar 2019, nick wrote:

> 
> 
> On 2019-03-28 4:59 a.m., Richard Biener wrote:
> > On Wed, Mar 27, 2019 at 6:31 PM nick  wrote:
> >>
> >> Greetings All,
> >>
> >> I've already done most of the work required for signing up for GSoC
> >> as of last year i.e. reading getting started, being signed up legally
> >> for contributions.
> >>
> >> My only real concern would be the proposal which I started writing here:
> >> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing
> >>
> >> The biography and success section I'm fine with my bigger concern would be 
> >> the project and roadmap
> >> section. The roadmap is there and I will go into more detail about it in 
> >> the projects section as
> >> need be. Just wanted to known if the roadmap is detailed enough or can I 
> >> just write out a few
> >> paragraphs discussing it in the Projects Section.
> > 
> > I'm not sure I understand either the problem analysis nor the project
> > goal parts.  What
> > shared state with respect to garbage collection are you talking about?
> > 
> > Richard.
> > 
> I just fixed it. Seems we were discussing RTL itself. I edited it to 
> reflect those changes. Let me know if it's unclear or you would actually 
> like me to discuss some changes that may occur in the RTL layer itself.
> 
> 
> I'm glad to be more exact if that's better but seems your confusion was 
> just what layer we were touching.

Let me just throw in some knowledge here.  The issue with RTL
is that we currently can only have a single function in this
intermediate language state since a function in RTL has some
state in global variables that would differ if it were another
function.  We can have multiple functions in GIMPLE intermediate
language state since all such state is in a function-specific
data structure (struct function).  The hard thing about moving
all this "global" state of RTL into the same place is that
there's global state in the various backends (and there's
already a struct funtion 'machine' part for such state, so there's
hope the issue isn't as big as it could be) and that some of
the global state is big and only changes very rarely.
That said, I'm not sure if anybody knows the full details here.

So as far as I understand you'd like to tackle this as project
with the goal to be able to have multiple functions in RTL
state.

That's laudable but IMHO also quite ambitious for a GSoC
project.  It's also an area I am not very familiar with so
I opt out of being a mentor for this project.

Richard.

> Nick
> >> Any other comments are welcome as well as I write it there,
> >> Nick
> 

-- 
Richard Biener 
SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany;
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah; HRB 21284 (AG Nürnberg)

Re: GSOC Proposal

[nick]

On 2019-03-28 4:59 a.m., Richard Biener wrote:
> On Wed, Mar 27, 2019 at 6:31 PM nick  wrote:
>>
>> Greetings All,
>>
>> I've already done most of the work required for signing up for GSoC
>> as of last year i.e. reading getting started, being signed up legally
>> for contributions.
>>
>> My only real concern would be the proposal which I started writing here:
>> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing
>>
>> The biography and success section I'm fine with my bigger concern would be 
>> the project and roadmap
>> section. The roadmap is there and I will go into more detail about it in the 
>> projects section as
>> need be. Just wanted to known if the roadmap is detailed enough or can I 
>> just write out a few
>> paragraphs discussing it in the Projects Section.
> 
> I'm not sure I understand either the problem analysis nor the project
> goal parts.  What
> shared state with respect to garbage collection are you talking about?
> 
> Richard.
> 
I just fixed it. Seems we were discussing RTL itself. I edited it to reflect 
those
changes. Let me know if it's unclear or you would actually like me to discuss
some changes that may occur in the RTL layer itself.


I'm glad to be more exact if that's better but seems your confusion was just 
what
layer we were touching.

Nick
>> Any other comments are welcome as well as I write it there,
>> Nick

Re: GSOC Proposal

[Richard Biener]

On Wed, Mar 27, 2019 at 6:31 PM nick  wrote:
>
> Greetings All,
>
> I've already done most of the work required for signing up for GSoC
> as of last year i.e. reading getting started, being signed up legally
> for contributions.
>
> My only real concern would be the proposal which I started writing here:
> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing
>
> The biography and success section I'm fine with my bigger concern would be 
> the project and roadmap
> section. The roadmap is there and I will go into more detail about it in the 
> projects section as
> need be. Just wanted to known if the roadmap is detailed enough or can I just 
> write out a few
> paragraphs discussing it in the Projects Section.

I'm not sure I understand either the problem analysis nor the project
goal parts.  What
shared state with respect to garbage collection are you talking about?

Richard.

> Any other comments are welcome as well as I write it there,
> Nick

GSOC Proposal

[nick]

Greetings All,

I've already done most of the work required for signing up for GSoC
as of last year i.e. reading getting started, being signed up legally
for contributions.

My only real concern would be the proposal which I started writing here:
https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing

The biography and success section I'm fine with my bigger concern would be the 
project and roadmap
section. The roadmap is there and I will go into more detail about it in the 
projects section as
need be. Just wanted to known if the roadmap is detailed enough or can I just 
write out a few
paragraphs discussing it in the Projects Section.

Any other comments are welcome as well as I write it there,
Nick

Re: GSOC proposal

[Martin Jambor]

Hello Ismael,

On Wed, Mar 21 2018, Ismael El Houas Ghouddana wrote:
> Dear Mr./Mrs,
>
> First of all, I really appreciate your time and attention. I am Ismael El
> Houas an aerospace engineer student with knowledge of Google Cloud Platform
> and I want to express my interest in working on your project.

I am sorry to reply only now, mainly because of traveling, I was not
reading my email in the second half of last week.  

>
> Secondly, I want to ask if I am still at a time to apply to this project,
> unfortunately, I was not aware of GSOC until yesterday. In the case, I am
> still able to apply for it, I will make the proposal as soon as possible.

Strictly speaking, the deadline is tomorrow, as decided by the GSoC
organizers.  If you have been working on a proposal despite not hearing
from us, we would sill like to see it and encourage you to submit it
before the deadline.  If you haven't, it is really getting rather late,
unless you have a very clear idea of what you want to do (in that case
we should still try!).

My apologies again for missing you message, I hope GSoC works out for
you one way or another.

Martin

GSOC proposal

[Ismael El Houas Ghouddana]

Dear Mr./Mrs,

First of all, I really appreciate your time and attention. I am Ismael El
Houas an aerospace engineer student with knowledge of Google Cloud Platform
and I want to express my interest in working on your project.

Secondly, I want to ask if I am still at a time to apply to this project,
unfortunately, I was not aware of GSOC until yesterday. In the case, I am
still able to apply for it, I will make the proposal as soon as possible.

Finally, many thanks for your attention.

Yours faithfully,

Ismael El Houas

Re: GSoC Proposal

[Benjamin De Kosnik]

 I have been told that the Project - Implement regular expressions in
 c++ mentored by Sir Benjamin De Kosnik is not completed and is
 available for this year GSoC project also by the the Mentor.

Sorry, there still appears to be some confusion here.

I am not mentoring GSOC this year. Here is some helpful information for
people who want to propose projects:
http://gcc.gnu.org/ml/gcc/2013-03/msg00082.html

-benjamin

RE: GSoC proposal: Provide optimizations feedback through post-compilation messages

[Thibault Raffaillac]

Quite lengthy but very interesting mail! It took me a while to formulate a 
proper reply :)

 Feedback can be scarce, but don't let that stop you from submitting a
 proposal.
 Either way, can you keep me informed about any progress? I might wish to help
 though that would probably be later in the cycle (got a lot queued up for
 the comming months).

Submitted :) The reviews are not too positive yet, my biggest efforts go into
making my plan clear. If any progress, help will be very appreciable indeed.

 Great that's exactly what I'm aiming at:) It's not just presenting the
 results of static analysis in real-time, as I actually dislike most
 kinds of it like finding memory leaks, to me that seems like an attempt
 to make the computer do what it's really bad at (understanding the
 code). I just want to give the programmer the fullest picture of the
 situation but at the same time make it so it doesn't become noise that
 interferes. More or less you can say the goal is To provide feedback
 that allows the user to extend his understanding of the program. That
 mostly means giving access to all the information that can be
 unambiguously concluded from the code by the computer. To what degree
 we carry it and how much the compiler is involved is only a question of
 practicality and performance.

I quite agree for the most part, still there is a subtle nuance on which I want
to argue: Do we really help the programmer by offering all the valuable
information that is possible to infer? Ten years from now, would he/she be a
better programmer if we had not let him/her strive to simulate the program in
mind, or code a portion in assembly and finally learn about machine
architecture?

My point is to avoid creating an interface that assists of helps the
programmer, as he/she might become dependent on it. This is just helping in the
short term, and the only person who ever learns something is the one who
actually creates the compiler. If a statement could sum my view, it would be
that the user improves through his/her use of the interface (here the
feedback messages).

How does it make a difference in practice? I want to minimize the information 
given :)
The reason I want to introduce feedback messages is that this particular
information (the inner workings of compilers) is very hard to find in practice.
I want to give a slight help to put the user on the rails, nothing more.

 Perfect! However, how to do that so that it actually works seems a bit
 complex. The first (practically unsolvable) issue is what actually
 constitutes better code, as given two pieces one may be faster in some
 cirtumstances while the other in different. But as I understand that's
 not really what we're trying to tell the user, rather we want him to
 explore for himself what's possible and what are the results and why
 they are the way they are? I'm guessing this will unfortunately (or
 fortunately) require him to actually see and undestand the intermediate
 code, see how it changes after different optimizations, and see the
 output assembly. Personally I really need/want that;) Though my end
 target is a bit more to broaden the abstraction when programming
 (both up and down), so not to just show what's happening with the code
 but also allow the programmer to interact with it on that lower level.
 LLVM seems like the perfect fit for that but I've got some gripes with
 it, and that is still far away in the future.

Excellent! Letting the user explore by himself sounds great, and seing the
output assembly/IR besides is indeed a must. I like the idea that compilation
is a cooperation between programmer and machine (as far as the programmer is
inclined to help of course). It would also be nice to see compilation be split
at Value range propagation, as one could verify it is properly computed, before
proceeding into optimizations.

 Unfortunately I only saw 36m of it as it broke and seeking doesn't work
 on vimeo for me, so I'll watch the rest later. To me it touches on some
 of the right issues/concepts but in slightly the wrong way, and it
 completely ignores some issues.

Agreed. (Only the first half of the video is relevant for the programming
prototype)

Thibault

Re: GSoC proposal: Provide optimizations feedback through post-compilation messages

[Tomasz Borowik]

On Mon, 2 Apr 2012 19:57:20 +
Thibault Raffaillac t...@kth.se wrote:

 Bump!
 
 Let me renew my interest in contributing through GSoC with post-compilation
 feedback (This was not an early april joke). Do you think it could lead to an
 acceptable GSoC proposal? (mentor interested?)

Feedback can be scarce, but don't let that stop you from submitting a
proposal.
Either way, can you keep me informed about any progress? I might wish to help
though that would probably be later in the cycle (got a lot queued up for
the comming months).

 @Tomasz:
 On the interaction side I totally agree that communication between compiler 
 and
 programmer is scarce (and there is room for improvement). Focusing too soon on
 the editor would overlook the vast users needs though, as:
 _ some users do not use an IDE (and will kindly refuse);
 _ some users do not need more communication, as they already know what GCC can
   and cannot do;
 _ some users do not want more communication, as they have other business to
   focus on;

Sure, I'm one of the people who don't use an IDE as it causes more
issues than it solves for me. This isn't meant for everyone the same
way anything else isn't, it just can't;p Still looking at it, other
languages, different IDEs, I'd say my way of tackling the issues is
more usable and useful than most other, and could easily see wider
adoption. Btw my experience is mostly in low-level kernel/driver
programming, 2/3d graphics, games.

 I think the editor being split from the compiler is good thing. There still
 exist tools to expose static analysis data from the compiler (and choose the
 editor to visualize it with), but fundamentally they are assisting him/her
 rather than helping him/her improve. Instead of gathering loads of data on the
 optimizations/analysis performed, and filtering it for visualization by the
 user, we could relate the optimization technique used so that the user truly
 knows what GCC is capable of (instead of guessing by observation).

Great that's exactly what I'm aiming at:) It's not just presenting the
results of static analysis in real-time, as I actually dislike most
kinds of it like finding memory leaks, to me that seems like an attempt
to make the computer do what it's really bad at (understanding the
code). I just want to give the programmer the fullest picture of the
situation but at the same time make it so it doesn't become noise that
interferes. More or less you can say the goal is To provide feedback
that allows the user to extend his understanding of the program. That
mostly means giving access to all the information that can be
unambiguously concluded from the code by the computer. To what degree
we carry it and how much the compiler is involved is only a question of
practicality and performance.

 My proposal is thus not to be confused with a static analysis visualization:
 the programmer learns what techniques are implemented in GCC (or in compilers
 in general), how to write code that is more easily compiled, and can further
 browse the Intwawaernet for detailed theory on the techniques involved.

Perfect! However, how to do that so that it actually works seems a bit
complex. The first (practically unsolvable) issue is what actually
constitutes better code, as given two pieces one may be faster in some
cirtumstances while the other in different. But as I understand that's
not really what we're trying to tell the user, rather we want him to
explore for himself what's possible and what are the results and why
they are the way they are? I'm guessing this will unfortunately (or
fortunately) require him to actually see and undestand the intermediate
code, see how it changes after different optimizations, and see the
output assembly. Personally I really need/want that;) Though my end
target is a bit more to broaden the abstraction when programming
(both up and down), so not to just show what's happening with the code
but also allow the programmer to interact with it on that lower level.
LLVM seems like the perfect fit for that but I've got some gripes with
it, and that is still far away in the future.

 The point on the possible-optimizations-which-could-be-enabled-if-specific-
 -constraint-is-lifted is particularly interesting, but is also extremely risky
 if the compiler makes a stupid remark on a constraint which can obviously
 (for the programmer) not be lifted. If ever, I would introduce it with a LOT 
 of
 care.

Yes and no. First of all I don't necessarily mean for the
compiler/editor to suggest anything to the programmer, rather if the
programmer asks just say what's physically possible, and not what's
right, since if the compiler could do that it would just perform the
optimization. Furthermore the situation with my source code is that I
can probably make all this in such a form that it is actually usable
and useful which seems to me close to impossible with normal languages.
I can also with almost no effort store within the source code the
dialogue between

Re: GSoC proposal: Provide optimizations feedback through post-compilation messages

[Thibault Raffaillac]

Bump!

Let me renew my interest in contributing through GSoC with post-compilation
feedback (This was not an early april joke). Do you think it could lead to an
acceptable GSoC proposal? (mentor interested?)

@Tomasz:
On the interaction side I totally agree that communication between compiler and
programmer is scarce (and there is room for improvement). Focusing too soon on
the editor would overlook the vast users needs though, as:
_ some users do not use an IDE (and will kindly refuse);
_ some users do not need more communication, as they already know what GCC can
  and cannot do;
_ some users do not want more communication, as they have other business to
  focus on;

I think the editor being split from the compiler is good thing. There still
exist tools to expose static analysis data from the compiler (and choose the
editor to visualize it with), but fundamentally they are assisting him/her
rather than helping him/her improve. Instead of gathering loads of data on the
optimizations/analysis performed, and filtering it for visualization by the
user, we could relate the optimization technique used so that the user truly
knows what GCC is capable of (instead of guessing by observation).

My proposal is thus not to be confused with a static analysis visualization:
the programmer learns what techniques are implemented in GCC (or in compilers
in general), how to write code that is more easily compiled, and can further
browse the Internet for detailed theory on the techniques involved.

The point on the possible-optimizations-which-could-be-enabled-if-specific-
-constraint-is-lifted is particularly interesting, but is also extremely risky
if the compiler makes a stupid remark on a constraint which can obviously
(for the programmer) not be lifted. If ever, I would introduce it with a LOT of
care.

Thibault
ps: As for an editor with real-time feedback on static analysis and more, I am
100% with you :) (and there are some promising prototypes, like in this talk:
http://vimeo.com/36579366)

 Hello all,
 
 My name is Thibault Raffaillac, CS degree student at Kungliga Tekniska 
 Högskolan,
 Stockholm, Sweden (in double-degree partnership with Ecole Centrale Marseille,
 France).
 GCC currently provides no concise way to inform the user whether it applied an
 expected optimization (ie, it understood the code). As a result, some will 
 do
 premature optimizations when they do not trust the compiler, and some others
 will create overly convoluted code with blind belief in the compiler. This is
 especially relevant for users non-initiated to the internals of GCC.
 The project I would like to propose is a feedback for the optimizations
 performed by GCC. To avoid binding users to the compiler, I would focus on 
 some
 very standard optimizations across vendors, or for some specific yet nice
 features I would indicate their specificity to GCC/an architecture.
 
 The feedback would be triggered when compilation is successful, and display a
 couple of different messages each time it is run:
 gcc --feedback test.c
 test.c:xx:x: info: All operands being constant, constant folding was applied 
 to assign '2560' to 'a'
 test.c:xx:x: info: GCC could not fold constants here because...
 test.c:xx:x: info: As integers are stored in binary format, strength 
 reduction was applied to replace '* 8' by ' 3'
 test.c:xx:x: info: Basic block vectorization was applied to pack the 3 
 independent additions into a single SIMD instruction
 test.c:xx:x: info: GCC implements unordered_map as open-addressed hash 
 tables, with double hashing probing
 
 As a difference with the internal verbose messages, here they would form a 
 set,
 and the system would remember those already displayed and decrease their
 frequency of occurence between compilations. All messages would explain what
 triggered them, cite the optimization name, and describe the consequence.
 
 As for the work plan, it would consist in:
 _ Enumerating all possible messages in the messages set.
 _ Implementing a function receiving feedback from each optimization unit and
   choosing whether to display it: info_printf(enum INFO_INDEX, const char*, 
 ...);
 _ Write a formatting guide for adding messages in the set.
 
 My academic background includes compiler construction, C programming and 
 Human-
 Computer Interactions. I am very much interested in the usability of compilers
 (on which I am currently carrying my degree thesis -
 http://www.csc.kth.se/~traf/traf-sketch.pdf) and thus would be glad to
 contribute to GCC.
 
 If this can be of interest, suggestions are welcome!
 
 Best regards,
 Thibault (http://www.csc.kth.se/~traf/)

Re: GSoC proposal: Provide optimizations feedback through post-compilation messages

[Tomasz Borowik]

On Tue, 27 Mar 2012 22:33:39 +
Thibault Raffaillac t...@kth.se wrote:

 Hello all,
 
 My name is Thibault Raffaillac, CS degree student at Kungliga Tekniska 
 Högskolan,
 Stockholm, Sweden (in double-degree partnership with Ecole Centrale Marseille,
 France).
 GCC currently provides no concise way to inform the user whether it applied an
 expected optimization (ie, it understood the code). As a result, some will 
 do
 premature optimizations when they do not trust the compiler, and some others
 will create overly convoluted code with blind belief in the compiler. This is
 especially relevant for users non-initiated to the internals of GCC.
 The project I would like to propose is a feedback for the optimizations
 performed by GCC. To avoid binding users to the compiler, I would focus on 
 some
 very standard optimizations across vendors, or for some specific yet nice
 features I would indicate their specificity to GCC/an architecture.
 
 The feedback would be triggered when compilation is successful, and display a
 couple of different messages each time it is run:
 gcc --feedback test.c
 test.c:xx:x: info: All operands being constant, constant folding was applied 
 to assign '2560' to 'a'
 test.c:xx:x: info: GCC could not fold constants here because...
 test.c:xx:x: info: As integers are stored in binary format, strength 
 reduction was applied to replace '* 8' by ' 3'
 test.c:xx:x: info: Basic block vectorization was applied to pack the 3 
 independent additions into a single SIMD instruction
 test.c:xx:x: info: GCC implements unordered_map as open-addressed hash 
 tables, with double hashing probing
 
 As a difference with the internal verbose messages, here they would form a 
 set,
 and the system would remember those already displayed and decrease their
 frequency of occurence between compilations. All messages would explain what
 triggered them, cite the optimization name, and describe the consequence.
 
 As for the work plan, it would consist in:
 _ Enumerating all possible messages in the messages set.
 _ Implementing a function receiving feedback from each optimization unit and
   choosing whether to display it: info_printf(enum INFO_INDEX, const char*, 
 ...);
 _ Write a formatting guide for adding messages in the set.
 
 My academic background includes compiler construction, C programming and 
 Human-
 Computer Interactions. I am very much interested in the usability of compilers
 (on which I am currently carrying my degree thesis -
 http://www.csc.kth.se/~traf/traf-sketch.pdf) and thus would be glad to
 contribute to GCC.
 
 If this can be of interest, suggestions are welcome!
 
 Best regards,
 Thibault (http://www.csc.kth.se/~traf/)
 

Hi Thibault,

I completely agree, and it's actually a part of what I'm targeting in the long 
term, so I think we might be able to join forces. I'm also thinking of a gsoc 
project though in different areas (there's an email in the list about them on 
19.03), so maybe we could do separate parts that combine into something even 
more awesome;)

I think a huge part of the issue is in the medium of communication between the 
programmer and compiler. I'm targeting an environment where the source code 
editor practically becomes the compiler's front-end. My project allows 
extremely dynamic presentation of the source code, so I can e.g.
 - easily inform the programmer about anything in an unobtrusive manner within 
the code, 
 - give him different perspectives of the same code,
 - allow him to give precise and detailed information to the compiler about 
possible code optimizations without making the code unreadable.

The first two points may seem already solved by eclipse, xcode or whatever 
other gigantic ide, but I'm talking about a much larger scale of feedback 
presented instantly like: ex/implicit and inferred typing info, constant folds, 
dead code, unfolded loops, data flow, vector operations, tree view of 
expressions.

The first issue is that for any non trivial amount of code you'll end up with 
thousands of messages 90% of which are probably not very interesting (similarly 
to warnings in a certain style of objective programming in C). As long as the 
output is not interleaved with the code at the right place and the delay from 
writing to getting feedback is too long, the feature will loose much of its 
usefullness. Though don't misunderstand me, I think it's still better to have 
the info in any form than not.

The last point is probably the more important, as there often is a large amount 
of optimizations that cannot be done due to for example pointer aliasing rules, 
but the programmer knows that the optimization is safe. I can easily add 
literally hundreds of markers like this expression is volatile, the result 
of this function call will not change within this loop, these two pointers 
don't alias and it wouldn't obfuscate the code as much as with normal 
languages. Furthermore my editor can easily list only the meaningful options 
for a given 

GSoC proposal: Provide optimizations feedback through post-compilation messages

[Thibault Raffaillac]

Hello all,

My name is Thibault Raffaillac, CS degree student at Kungliga Tekniska 
Högskolan,
Stockholm, Sweden (in double-degree partnership with Ecole Centrale Marseille,
France).
GCC currently provides no concise way to inform the user whether it applied an
expected optimization (ie, it understood the code). As a result, some will do
premature optimizations when they do not trust the compiler, and some others
will create overly convoluted code with blind belief in the compiler. This is
especially relevant for users non-initiated to the internals of GCC.
The project I would like to propose is a feedback for the optimizations
performed by GCC. To avoid binding users to the compiler, I would focus on some
very standard optimizations across vendors, or for some specific yet nice
features I would indicate their specificity to GCC/an architecture.

The feedback would be triggered when compilation is successful, and display a
couple of different messages each time it is run:
gcc --feedback test.c
test.c:xx:x: info: All operands being constant, constant folding was applied to 
assign '2560' to 'a'
test.c:xx:x: info: GCC could not fold constants here because...
test.c:xx:x: info: As integers are stored in binary format, strength reduction 
was applied to replace '* 8' by ' 3'
test.c:xx:x: info: Basic block vectorization was applied to pack the 3 
independent additions into a single SIMD instruction
test.c:xx:x: info: GCC implements unordered_map as open-addressed hash tables, 
with double hashing probing

As a difference with the internal verbose messages, here they would form a set,
and the system would remember those already displayed and decrease their
frequency of occurence between compilations. All messages would explain what
triggered them, cite the optimization name, and describe the consequence.

As for the work plan, it would consist in:
_ Enumerating all possible messages in the messages set.
_ Implementing a function receiving feedback from each optimization unit and
  choosing whether to display it: info_printf(enum INFO_INDEX, const char*, 
...);
_ Write a formatting guide for adding messages in the set.

My academic background includes compiler construction, C programming and Human-
Computer Interactions. I am very much interested in the usability of compilers
(on which I am currently carrying my degree thesis -
http://www.csc.kth.se/~traf/traf-sketch.pdf) and thus would be glad to
contribute to GCC.

If this can be of interest, suggestions are welcome!

Best regards,
Thibault (http://www.csc.kth.se/~traf/)