[go-nuts] Re: Can/should the SSA optimizer cross package boundaries?
In the gc compiler inlining occurs early on in the parse and import phase, it’s is independent of the ssa passes later. Gccgo may handle this differently. Can the gc compiler handle 10 functions in a package. I’m going to go with yes, but you’ll probably need an enormous amount of memory. The linker time will probably be extreme. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Can/should the SSA optimizer cross package boundaries?
And thanks!
On Sunday, November 12, 2017 at 11:40:51 AM UTC-5, Petar Maymounkov wrote:
>
> Inline:
>
> On Saturday, November 11, 2017 at 2:52:30 PM UTC-5, Dave Cheney wrote:
>>
>> Hi,
>>
>> Thanks for following up here.
>>
>>
>>
>> On Sunday, 12 November 2017 03:35:27 UTC+11, Petar Maymounkov wrote:
>>>
>>> Consider a chain of functions that call each other:
>>>
>>>
>>> func f1(x X) { ... f2(y) ... }
>>> func f2(y Y) { ... f3(z) ... }
>>> and so on.
>>>
>>>
>>> Assume also that their arguments and return values are static Go types
>>> (no interfaces).
>>>
>>>
>>> Generally, such a chain of statically-typed invocations will fall within
>>> the domain of the SSA optimizer and will be rewritten (theoretically)
>>> optimally.
>>>
>>
>> What do these functions do? Can how show some working examples? What do
>> you mean by rewritten?
>>
>
>
> We have a system that code-generates a Go simulator for large
> microprocessor circuits (> 1M gates and subsystems).
> Both hardware gates and subsystems are represented as Go functions with
> statically-typed arguments and return values.
> Subsystems use/call each other (the recursion).
>
> As is, if we want to benefit from code optimization, we need to
> code-generate the whole micro-processor in one package.
> This produces millions of functions per package.
>
>
>>
>>
>>>
>>> The issue arises when the invocation chain is recursive, e.g.
>>>
>>>
>>> func f1(x X) { ... f2(y) ... }
>>> func f2(y Y) { ... f3(z) ... }
>>> func f3(z Z) { ... f1(x) ... }
>>>
>>>
>>> and the user desires to implement f1 and f3 in different packages.
>>>
>>>
>>> This is not possible due to the design of the packaging system (as the
>>> packages of f1 and f3 would have to import each other).
>>>
>>> Consequently, large amounts of recursive code cannot be spread across
>>> packages.
>>>
>>>
>>> This situation has arisen in practice for us at a pretty large scale
>>> (many/long recursive chains).
>>>
>>>
>>> I am wondering a couple of things:
>>>
>>> (a) Is there any technical consideration prohibiting large-scale SSA,
>>>
>>
>> There are lots of phases in the compiler, SSA occurs relatively late in
>> the process, and the one that's probably more relevant here is inlining.
>> Inlining does work across package boundaries, exported functions from one
>> package aren't supposed to be treated differently from functions in
>> another, although there could always be a bug. This is where a practical
>> example of what problem you're facing would be really helpful.
>>
>
> SSA as an algorithmic technology applies to chains of static calls
> regardless of packaging considerations. (Inlining is a special case of SSA.)
> Whether Go/LLVM implement/apply SSA to its full potential is something I
> don't know. Hence the question.
>
>
>>
>>
>>> (b) Is there any technical consideration prohibiting go packages from
>>> importing each other.
>>>
>>
>> The prohibition on import loops exists because it would significantly
>> complicate the type checking of the compiler and the package import/export
>> mechanism. it's also just not a good idea from a program design point of
>> view as circular imports often denote high coupling which limit isolation
>> and effective code reuse.
>>
>
> I personally disagree with all considerations stated. But to keep my
> question practical:
>
> I am simply wondering whether the Go compiler is designed to handle
> packages with ~10M static functions,
> which invoke each other in cyclical ways.
>
>
>>
>>
>>>
>>>
>>> Thank you.
>>>
>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Can/should the SSA optimizer cross package boundaries?
Inline:
On Saturday, November 11, 2017 at 2:52:30 PM UTC-5, Dave Cheney wrote:
>
> Hi,
>
> Thanks for following up here.
>
>
>
> On Sunday, 12 November 2017 03:35:27 UTC+11, Petar Maymounkov wrote:
>>
>> Consider a chain of functions that call each other:
>>
>>
>> func f1(x X) { ... f2(y) ... }
>> func f2(y Y) { ... f3(z) ... }
>> and so on.
>>
>>
>> Assume also that their arguments and return values are static Go types
>> (no interfaces).
>>
>>
>> Generally, such a chain of statically-typed invocations will fall within
>> the domain of the SSA optimizer and will be rewritten (theoretically)
>> optimally.
>>
>
> What do these functions do? Can how show some working examples? What do
> you mean by rewritten?
>
We have a system that code-generates a Go simulator for large
microprocessor circuits (> 1M gates and subsystems).
Both hardware gates and subsystems are represented as Go functions with
statically-typed arguments and return values.
Subsystems use/call each other (the recursion).
As is, if we want to benefit from code optimization, we need to
code-generate the whole micro-processor in one package.
This produces millions of functions per package.
>
>
>>
>> The issue arises when the invocation chain is recursive, e.g.
>>
>>
>> func f1(x X) { ... f2(y) ... }
>> func f2(y Y) { ... f3(z) ... }
>> func f3(z Z) { ... f1(x) ... }
>>
>>
>> and the user desires to implement f1 and f3 in different packages.
>>
>>
>> This is not possible due to the design of the packaging system (as the
>> packages of f1 and f3 would have to import each other).
>>
>> Consequently, large amounts of recursive code cannot be spread across
>> packages.
>>
>>
>> This situation has arisen in practice for us at a pretty large scale
>> (many/long recursive chains).
>>
>>
>> I am wondering a couple of things:
>>
>> (a) Is there any technical consideration prohibiting large-scale SSA,
>>
>
> There are lots of phases in the compiler, SSA occurs relatively late in
> the process, and the one that's probably more relevant here is inlining.
> Inlining does work across package boundaries, exported functions from one
> package aren't supposed to be treated differently from functions in
> another, although there could always be a bug. This is where a practical
> example of what problem you're facing would be really helpful.
>
SSA as an algorithmic technology applies to chains of static calls
regardless of packaging considerations. (Inlining is a special case of SSA.)
Whether Go/LLVM implement/apply SSA to its full potential is something I
don't know. Hence the question.
>
>
>> (b) Is there any technical consideration prohibiting go packages from
>> importing each other.
>>
>
> The prohibition on import loops exists because it would significantly
> complicate the type checking of the compiler and the package import/export
> mechanism. it's also just not a good idea from a program design point of
> view as circular imports often denote high coupling which limit isolation
> and effective code reuse.
>
I personally disagree with all considerations stated. But to keep my
question practical:
I am simply wondering whether the Go compiler is designed to handle
packages with ~10M static functions,
which invoke each other in cyclical ways.
>
>
>>
>>
>> Thank you.
>>
>>
>>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Can/should the SSA optimizer cross package boundaries?
Hi,
Thanks for following up here.
On Sunday, 12 November 2017 03:35:27 UTC+11, Petar Maymounkov wrote:
>
> Consider a chain of functions that call each other:
>
>
> func f1(x X) { ... f2(y) ... }
> func f2(y Y) { ... f3(z) ... }
> and so on.
>
>
> Assume also that their arguments and return values are static Go types (no
> interfaces).
>
>
> Generally, such a chain of statically-typed invocations will fall within
> the domain of the SSA optimizer and will be rewritten (theoretically)
> optimally.
>
What do these functions do? Can how show some working examples? What do you
mean by rewritten?
>
> The issue arises when the invocation chain is recursive, e.g.
>
>
> func f1(x X) { ... f2(y) ... }
> func f2(y Y) { ... f3(z) ... }
> func f3(z Z) { ... f1(x) ... }
>
>
> and the user desires to implement f1 and f3 in different packages.
>
>
> This is not possible due to the design of the packaging system (as the
> packages of f1 and f3 would have to import each other).
>
> Consequently, large amounts of recursive code cannot be spread across
> packages.
>
>
> This situation has arisen in practice for us at a pretty large scale
> (many/long recursive chains).
>
>
> I am wondering a couple of things:
>
> (a) Is there any technical consideration prohibiting large-scale SSA,
>
There are lots of phases in the compiler, SSA occurs relatively late in the
process, and the one that's probably more relevant here is inlining.
Inlining does work across package boundaries, exported functions from one
package aren't supposed to be treated differently from functions in
another, although there could always be a bug. This is where a practical
example of what problem you're facing would be really helpful.
> (b) Is there any technical consideration prohibiting go packages from
> importing each other.
>
The prohibition on import loops exists because it would significantly
complicate the type checking of the compiler and the package import/export
mechanism. it's also just not a good idea from a program design point of
view as circular imports often denote high coupling which limit isolation
and effective code reuse.
>
>
> Thank you.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
