Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Lluís Vilanova]

Claudio Fontana writes:

> Clearly late to the party,
> On 08.04.2016 22:14, Paolo Bonzini wrote:
>> 
>> On 08/04/2016 15:15, Markus Armbruster wrote:
 On the other hand, minimal usage of templates instead of some of the
 preprocessor gunk we have would be a very good thing IMNSHO.  I am
 referring to the multiply included header files and to the macros with
 type arguments (mostly QOM casts).
 
 I don't think we need more C++ than that, but using templates as
 basically a type-safe preprocessor would improve QEMU a little bit.
 More rarely, lambdas could replace some preprocessor magic too, but
 that's C11 and not many compilers support them.
 
 But I won't weep if people say no because we have a lot other
 low-hanging fruit to make QEMU better (especially the header file
>>> 
>>> "No!"  (Hey, you asked for it)
>>> 
>>> Back to serious.  As Peter Maydell said, "if we move away from C I'd
>>> rather it to be a language that's nicer than C rather than one that's
>>> uglier and larger and still retains all of C's flaws."
>> 
>> Sure, except that one plan is feasible now and can be done in small
>> steps; the other is not feasible now (for example Rust is not even
>> packaged in Fedora) and entails pretty much a rewrite of the whole code
>> base.

> I skip my personal take on these specific issue, but one very practical 
> concern I have with a potential move to C++,
> if I understand these proposals correctly, is with the speed of compilation 
> and the binary size, and also QEMU startup time.

> I noticed while developing for OSv that compilation with C++ and templates was
> very slow compared with C, so certain scripts around git I am using to build 
> and
> check code from scratch after each patch took a long time to complete.

> Do you know what these refactoring proposals' impact on compilation speed, 
> binary size and startup time would be?

For templates, binary size and startup time should be exactly the same (plus
whatever fixed costs C++ might have, if any at all). Compilation times really
depend on the complexity of the templates, but I think that a straight
conversion should not noticeably increase compilation times; the use of
templates that QEMU would have is pretty straightforward (not any complex
interrelation between different templated classes with their own templated
methods), and we'd be replacing the preprocessing of a file multiple times
(e.g., include softmmu_template.h 8 times) for templated functions in the
implementation file (no need to see templates on the headers).

In the case of objects, binary/memory size should be marginally lower with C++
(in some cases, the C++ compiler can elide the "parent_class" pointer that QEMU
adds on every object class). On startup time, I'm not aware of any costs added
by C++ objects, but I might be wrong (aren't vtable pointers prepared at
compilation time?). As for compilation time, I really don't know what important
effects classes can have.


Cheers,
  Lluis

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Claudio Fontana]

Clearly late to the party,

On 08.04.2016 22:14, Paolo Bonzini wrote:
> 
> On 08/04/2016 15:15, Markus Armbruster wrote:
>>> On the other hand, minimal usage of templates instead of some of the
>>> preprocessor gunk we have would be a very good thing IMNSHO.  I am
>>> referring to the multiply included header files and to the macros with
>>> type arguments (mostly QOM casts).
>>>
>>> I don't think we need more C++ than that, but using templates as
>>> basically a type-safe preprocessor would improve QEMU a little bit.
>>> More rarely, lambdas could replace some preprocessor magic too, but
>>> that's C11 and not many compilers support them.
>>>
>>> But I won't weep if people say no because we have a lot other
>>> low-hanging fruit to make QEMU better (especially the header file
>>
>> "No!"  (Hey, you asked for it)
>>
>> Back to serious.  As Peter Maydell said, "if we move away from C I'd
>> rather it to be a language that's nicer than C rather than one that's
>> uglier and larger and still retains all of C's flaws."
> 
> Sure, except that one plan is feasible now and can be done in small
> steps; the other is not feasible now (for example Rust is not even
> packaged in Fedora) and entails pretty much a rewrite of the whole code
> base.

I skip my personal take on these specific issue, but one very practical concern 
I have with a potential move to C++,
if I understand these proposals correctly, is with the speed of compilation and 
the binary size, and also QEMU startup time.

I noticed while developing for OSv that compilation with C++ and templates was 
very slow compared with C, so certain scripts around git I am using to build 
and check code from scratch after each patch took a long time to complete.

Do you know what these refactoring proposals' impact on compilation speed, 
binary size and startup time would be?

Thank you,

Claudio


> 
>> People sometimes propose to defang C++ by subsetting and/or coding
>> conventions.  I'll take that seriously when I see the tool that
>> rigorously checks adherence to subset / convention.
> 
> The problem with subsetting and conventions is that they always come
> with exceptions, besides the fact that no one writes the tool.
> 
> So I prefer common sense :) and common sense says that a million-line
> codebase that mixes procedural, home-grown OO and language OO is going
> to stink from ten miles.  And maintainers sit at most two feet from the
> screen.
> 
> Really even just -Wc++-compat would be a nice improvement so we enjoy a
> little more type safety.  IMHO, C++'s biggest tax is that Coccinelle
> does not like it.
> 
>>> cleanups that Markus started and I want to conclude very early in 2.7).
>>
>> Speaking of which: the plan was you post yours for 2.7, and then I can
>> build on top (assuming there's useful work left), right?
> 
> I have rebased my stuff already, but I'm going to disappear in about a
> week and for the first week or two after the 2.6 release (depending on
> whether it slips or not).  Also, I will travel most of next week.  So I
> either I'll post it soonish or it will have to wait a little.
> 
> Anyhow, there's definitely useful work left from your last two patches,
> even more so after Veronia Bahaa cleaned up qemu-common.h substantially
> so there may be more pointless inclusions of it and fewer headers that
> really need it.
> 
> Paolo
> 

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Paolo Bonzini]

On 08/04/2016 15:15, Markus Armbruster wrote:
> > On the other hand, minimal usage of templates instead of some of the
> > preprocessor gunk we have would be a very good thing IMNSHO.  I am
> > referring to the multiply included header files and to the macros with
> > type arguments (mostly QOM casts).
> >
> > I don't think we need more C++ than that, but using templates as
> > basically a type-safe preprocessor would improve QEMU a little bit.
> > More rarely, lambdas could replace some preprocessor magic too, but
> > that's C11 and not many compilers support them.
> >
> > But I won't weep if people say no because we have a lot other
> > low-hanging fruit to make QEMU better (especially the header file
> 
> "No!"  (Hey, you asked for it)
> 
> Back to serious.  As Peter Maydell said, "if we move away from C I'd
> rather it to be a language that's nicer than C rather than one that's
> uglier and larger and still retains all of C's flaws."

Sure, except that one plan is feasible now and can be done in small
steps; the other is not feasible now (for example Rust is not even
packaged in Fedora) and entails pretty much a rewrite of the whole code
base.

> People sometimes propose to defang C++ by subsetting and/or coding
> conventions.  I'll take that seriously when I see the tool that
> rigorously checks adherence to subset / convention.

The problem with subsetting and conventions is that they always come
with exceptions, besides the fact that no one writes the tool.

So I prefer common sense :) and common sense says that a million-line
codebase that mixes procedural, home-grown OO and language OO is going
to stink from ten miles.  And maintainers sit at most two feet from the
screen.

Really even just -Wc++-compat would be a nice improvement so we enjoy a
little more type safety.  IMHO, C++'s biggest tax is that Coccinelle
does not like it.

> > cleanups that Markus started and I want to conclude very early in 2.7).
>
> Speaking of which: the plan was you post yours for 2.7, and then I can
> build on top (assuming there's useful work left), right?

I have rebased my stuff already, but I'm going to disappear in about a
week and for the first week or two after the 2.6 release (depending on
whether it slips or not).  Also, I will travel most of next week.  So I
either I'll post it soonish or it will have to wait a little.

Anyhow, there's definitely useful work left from your last two patches,
even more so after Veronia Bahaa cleaned up qemu-common.h substantially
so there may be more pointless inclusions of it and fewer headers that
really need it.

Paolo

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Markus Armbruster]

Paolo Bonzini  writes:

> On 07/04/2016 16:49, Peter Maydell wrote:
>> > QOM to C++ classes
>> I suspect if you looked at this you'd find that the QOM semantics
>> for various things don't map onto C++ (ie that we have more runtime
>> flexibility than C++ does).
>
> True, but you don't have to use it. :)  If your code is static, one
> could imagine bindings to C++ that eliminate some of the boilerplate.
> Don't look at me, though.
>
> On the other hand, minimal usage of templates instead of some of the
> preprocessor gunk we have would be a very good thing IMNSHO.  I am
> referring to the multiply included header files and to the macros with
> type arguments (mostly QOM casts).
>
> I don't think we need more C++ than that, but using templates as
> basically a type-safe preprocessor would improve QEMU a little bit.
> More rarely, lambdas could replace some preprocessor magic too, but
> that's C11 and not many compilers support them.
>
> But I won't weep if people say no because we have a lot other
> low-hanging fruit to make QEMU better (especially the header file

"No!"  (Hey, you asked for it)

Back to serious.  As Peter Maydell said, "if we move away from C I'd
rather it to be a language that's nicer than C rather than one that's
uglier and larger and still retains all of C's flaws."

People sometimes propose to defang C++ by subsetting and/or coding
conventions.  I'll take that seriously when I see the tool that
rigorously checks adherence to subset / convention.  We're bad enough at
getting everybody write half-decent C, and making the problem harder is
unlikely to help.

> cleanups that Markus started and I want to conclude very early in 2.7).

Speaking of which: the plan was you post yours for 2.7, and then I can
build on top (assuming there's useful work left), right?

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Paolo Bonzini]

On 07/04/2016 16:49, Peter Maydell wrote:
> > QOM to C++ classes
> I suspect if you looked at this you'd find that the QOM semantics
> for various things don't map onto C++ (ie that we have more runtime
> flexibility than C++ does).

True, but you don't have to use it. :)  If your code is static, one
could imagine bindings to C++ that eliminate some of the boilerplate.
Don't look at me, though.

On the other hand, minimal usage of templates instead of some of the
preprocessor gunk we have would be a very good thing IMNSHO.  I am
referring to the multiply included header files and to the macros with
type arguments (mostly QOM casts).

I don't think we need more C++ than that, but using templates as
basically a type-safe preprocessor would improve QEMU a little bit.
More rarely, lambdas could replace some preprocessor magic too, but
that's C11 and not many compilers support them.

But I won't weep if people say no because we have a lot other
low-hanging fruit to make QEMU better (especially the header file
cleanups that Markus started and I want to conclude very early in 2.7).

Paolo

> This is just vaguely remembered from
> discussions back when we first added QOM though, I have no specific
> detail and might have misremembered.

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Peter Maydell]

On 3 April 2016 at 14:05, Lluís Vilanova  wrote:
> QOM to C++ classes

I suspect if you looked at this you'd find that the QOM semantics
for various things don't map onto C++ (ie that we have more runtime
flexibility than C++ does). This is just vaguely remembered from
discussions back when we first added QOM though, I have no specific
detail and might have misremembered.

thanks
-- PMM

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Markus Armbruster]

Lluís Vilanova  writes:

> Markus Armbruster writes:
>
>> Peter Maydell  writes:
>> [...]
>>> if we move away from C I'd rather
>>> it to be a language that's nicer than C rather than one that's
>>> uglier and larger and still retains all of C's flaws.
>
>> Seconded strongly.
>
> Just curious. Do we agree though that moving selected pieces like the code
> templates with macros to C++ templated functions, or QOM to C++ classes would 
> be
> an improvement over the current C code without any dramatic changes in the
> codebase? (I don't think more than that is needed)

I don't think we do.

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Lluís Vilanova]

Markus Armbruster writes:

> Peter Maydell  writes:
> [...]
>> if we move away from C I'd rather
>> it to be a language that's nicer than C rather than one that's
>> uglier and larger and still retains all of C's flaws.

> Seconded strongly.

Just curious. Do we agree though that moving selected pieces like the code
templates with macros to C++ templated functions, or QOM to C++ classes would be
an improvement over the current C code without any dramatic changes in the
codebase? (I don't think more than that is needed)


Cheers,
  Lluis

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Lluís Vilanova]

KONRAD Frederic writes:

> Hi,
> Le 09/03/2016 16:52, Richard Henderson a écrit :
>> On 03/09/2016 09:38 AM, Lluís Vilanova wrote:
>>> Hi,
>>> 
>>> NOTE: I won't be throwing patches anytime soon, I just want to know if
>>> there's
>>> interest in this for the future.
>>> 
>>> While adding events for tracing guest instructions, I've found that the
>>> per-target "gen_intermediate_code()" function is very similar but not 
>>> exactly
>>> the same for each of the targets. This makes architecture-agnostic features
>>> harder to maintain across targets, specially when it comes to their relative
>>> order.
>>> 
>>> So, would it be worth it if I generalized part of that code into an
>>> architecture-agnostic function that calls into target-specific hooks 
>>> wherever
>>> it
>>> needs extending? There are many ways to do it that we can discuss later.
>> 
>> It's worth talking about, since I do believe it would make long-term
>> maintenance across the targets easier.
>> 
>> These "target-specific hooks" probably ought not be "hooks" in the
>> traditional sense of attaching them to CPUState.  I'd be more comfortable 
>> with
>> a refactoring that used include files -- maybe .h or maybe .inc.c.  If we do
>> the normal sort of hook, then we've got to either expose DisasContext in
>> places we shouldn't, or dynamically allocate it.  Neither seems particularly
>> appealing.
>> 
>> 
>> r~
>> 

> On the other side I think attaching them to CPUState would make heterogenous
> system emulation easier?

Yes, that's something to take into account too. But for this I think that
"gen_intermediate_code()" would suffice.

Still, abstracting it into the level of KVM vs. TCG would be even more
interesting, since we could switch between backends, or even have different CPUs
using different backends (e.g., trace a TCG vCPU while another runs at full
speed using KVM). For example, there could be a virtual "exec()" method,
although reality is not that straightforward since each KVM vCPU runs on a
different thread, while in TCG this is not necessarily so (even with MTTCG).


Cheers,
  Lluis

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Paolo Bonzini]

On 14/03/2016 13:23, KONRAD Frederic wrote:
>> These "target-specific hooks" probably ought not be "hooks" in the
>> traditional sense of attaching them to CPUState.  I'd be more
>> comfortable with a refactoring that used include files -- maybe .h or
>> maybe .inc.c.  If we do the normal sort of hook, then we've got to
>> either expose DisasContext in places we shouldn't, or dynamically
>> allocate it.  Neither seems particularly appealing.
> 
> On the other side I think attaching them to CPUState would make
> heterogenous system emulation easier?

It depends on whether they're going to be called from generic code.  As
long as they are only called from other CPU-specific bits, adding them
to CPUState isn't necessary.

Paolo

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[KONRAD Frederic]

Hi,

Le 09/03/2016 16:52, Richard Henderson a écrit :

On 03/09/2016 09:38 AM, Lluís Vilanova wrote:

Hi,

NOTE: I won't be throwing patches anytime soon, I just want to know 
if there's

   interest in this for the future.

While adding events for tracing guest instructions, I've found that the
per-target "gen_intermediate_code()" function is very similar but not 
exactly
the same for each of the targets. This makes architecture-agnostic 
features
harder to maintain across targets, specially when it comes to their 
relative

order.

So, would it be worth it if I generalized part of that code into an
architecture-agnostic function that calls into target-specific hooks 
wherever it

needs extending? There are many ways to do it that we can discuss later.


It's worth talking about, since I do believe it would make long-term 
maintenance across the targets easier.


These "target-specific hooks" probably ought not be "hooks" in the
traditional sense of attaching them to CPUState.  I'd be more 
comfortable with a refactoring that used include files -- maybe .h or 
maybe .inc.c.  If we do the normal sort of hook, then we've got to 
either expose DisasContext in places we shouldn't, or dynamically 
allocate it.  Neither seems particularly appealing.



r~



On the other side I think attaching them to CPUState would make 
heterogenous system emulation easier?


Fred

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Lluís Vilanova]

Markus Armbruster writes:

> Peter Maydell  writes:
> [...]
>> if we move away from C I'd rather
>> it to be a language that's nicer than C rather than one that's
>> uglier and larger and still retains all of C's flaws.

> Seconded strongly.

I don't feel like that about C++, but then I maybe had a different experience
with it (specially with the latest standard additions).


Cheers,
  Lluis

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Markus Armbruster]

Peter Maydell  writes:

[...]
>   if we move away from C I'd rather
> it to be a language that's nicer than C rather than one that's
> uglier and larger and still retains all of C's flaws.

Seconded strongly.

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Peter Maydell]

On 13 March 2016 at 13:16, Lluís Vilanova  wrote:
> Peter Maydell writes:
>> I would be more interested in a proposal to move parts of QEMU
>> to Rust, or just about anything else except C++...
>
> QEMU is pretty low-level, so I'm not sure other languages will fit the bill as
> good, and for the parts relevant to QEMU you have just as much control of
> low-level details as with C (having a very close syntax also helps
> transition).
>
> But I'm curious, what'd be the advantage of rust? Cross-language
> bindings are usually expensive, and require some duplication for
> defining structures across them (maybe it's not the case for rust).

It's a systems programming language that's not insanely huge
and designed by continuous accretion of features, that's all.
(More positively, it has the usual nice features of newer languages
such as not letting you write code that's exploitable by a
malicious guest via buffer overflows.) But I'm not so much
trying to advocate for Rust (which I have not investigated at all)
as expressing an opinion that if we move away from C I'd rather
it to be a language that's nicer than C rather than one that's
uglier and larger and still retains all of C's flaws.

thanks
-- PMM

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Lluís Vilanova]

Peter Maydell writes:

> On 10 March 2016 at 05:29, Lluís Vilanova  wrote:
>> Richard Henderson writes:
>>> Alternately... can we broach the subject of C++?  Honestly, it
>>> seems we work too hard sometimes to re-implement templates and
>>> classes in C.
>> 
>> Whooo, I'd really *love* to switch to C++ just for templates and
>> classes... But last time this was discussed, the idea wasn't met
>> with much joy :)

> I would be more interested in a proposal to move parts of QEMU
> to Rust, or just about anything else except C++...

QEMU is pretty low-level, so I'm not sure other languages will fit the bill as
good, and for the parts relevant to QEMU you have just as much control of
low-level details as with C (having a very close syntax also helps
transition).

But I'm curious, what'd be the advantage of rust? Cross-language bindings are
usually expensive, and require some duplication for defining structures across
them (maybe it's not the case for rust).


Lluis

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Peter Maydell]

On 10 March 2016 at 05:29, Lluís Vilanova  wrote:
> Richard Henderson writes:
>> Alternately... can we broach the subject of C++?  Honestly, it
>> seems we work too hard sometimes to re-implement templates and
>> classes in C.
>
> Whooo, I'd really *love* to switch to C++ just for templates and
> classes... But last time this was discussed, the idea wasn't met
> with much joy :)

I would be more interested in a proposal to move parts of QEMU
to Rust, or just about anything else except C++...

thanks
-- PMM

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Lluís Vilanova]

Richard Henderson writes:

> On 03/09/2016 01:16 PM, Lluís Vilanova wrote:
>> Richard Henderson writes:
>> 
>>> On 03/09/2016 09:38 AM, Lluís Vilanova wrote:
 Hi,
 
 NOTE: I won't be throwing patches anytime soon, I just want to know if 
 there's
 interest in this for the future.
 
 While adding events for tracing guest instructions, I've found that the
 per-target "gen_intermediate_code()" function is very similar but not 
 exactly
 the same for each of the targets. This makes architecture-agnostic features
 harder to maintain across targets, specially when it comes to their 
 relative
 order.
 
 So, would it be worth it if I generalized part of that code into an
 architecture-agnostic function that calls into target-specific hooks 
 wherever it
 needs extending? There are many ways to do it that we can discuss later.
>> 
>>> It's worth talking about, since I do believe it would make long-term 
>>> maintenance
>>> across the targets easier.
>> 
>>> These "target-specific hooks" probably ought not be "hooks" in the
>>> traditional sense of attaching them to CPUState.  I'd be more comfortable 
>>> with a
>>> refactoring that used include files -- maybe .h or maybe .inc.c.  If we do 
>>> the
>>> normal sort of hook, then we've got to either expose DisasContext in places 
>>> we
>>> shouldn't, or dynamically allocate it.  Neither seems particularly 
>>> appealing.
>> 
>> Great. I pondered about using QOM vs "template code", and I'm leaning towards
>> the latter:
>> 
>> * translate.c:
>> 
>> struct DisasContextArch {
>> DisasContext common;
>> };
>> 
>> // implement "hooks"
>> 
>> void gen_intermediate_code(CPUArchState *env, TranslationBlock *tb)
>> {
>> DisasContextArch dc;
>> gen_intermediate_code_template(get_cpu(env), , tb);

> Pointer down into "common" here...

>> }
>> 
>> * translate-template.h:
>> 
>> struct DisasContext { /* ... */ };
>> 
>> void gen_intermediate_code_template(CPUState *cpu, DisasContext *dc,
>> TranslationBlock *tb)
>> {
>> // init dc
>> // arch-specific init dc hook
>> 
>> // gen_icount, etc.
>> while (true) {
>> // generic processing code with calls to hooks

> ... means you have to upcast to DisasContextArch here, or in the hooks 
> themselves.

Yup, that was the intention. Hooks should then use "container_of()" to get their
part.


>> }
>> }
>> 
>> While initially simpler, the "template code" still feels a little dirty to
>> me. With QOM, you could implement a DisasContextClass that provides the
>> per-target hook pointers, which can be globally allocated once per target and
>> pointed to by the DisasContext allocated in stack.
>> 
>> I'm not sure about what you mean by exposing DisasContext in places it 
>> shouldn't
>> be, though.

> You either split DisasContextArch / DisasContext in "odd" ways, and then have 
> to
> cast back and forth, or you have to expose the whole of DisasContextArch. It's
> the latter that I considered inappropriate.

Well, moving whatever is used in the generic code into DisasContext doesn't
sound too odd as a first solution.


> Alternately... can we broach the subject of C++?  Honestly, it seems we work 
> too
> hard sometimes to re-implement templates and classes in C.

Whooo, I'd really *love* to switch to C++ just for templates and classes... But
last time this was discussed, the idea wasn't met with much joy :)


Cheers,
  Lluis

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Richard Henderson]

On 03/09/2016 01:16 PM, Lluís Vilanova wrote:

Richard Henderson writes:


On 03/09/2016 09:38 AM, Lluís Vilanova wrote:

Hi,

NOTE: I won't be throwing patches anytime soon, I just want to know if there's
interest in this for the future.

While adding events for tracing guest instructions, I've found that the
per-target "gen_intermediate_code()" function is very similar but not exactly
the same for each of the targets. This makes architecture-agnostic features
harder to maintain across targets, specially when it comes to their relative
order.

So, would it be worth it if I generalized part of that code into an
architecture-agnostic function that calls into target-specific hooks wherever it
needs extending? There are many ways to do it that we can discuss later.



It's worth talking about, since I do believe it would make long-term maintenance
across the targets easier.



These "target-specific hooks" probably ought not be "hooks" in the
traditional sense of attaching them to CPUState.  I'd be more comfortable with a
refactoring that used include files -- maybe .h or maybe .inc.c.  If we do the
normal sort of hook, then we've got to either expose DisasContext in places we
shouldn't, or dynamically allocate it.  Neither seems particularly appealing.


Great. I pondered about using QOM vs "template code", and I'm leaning towards
the latter:

* translate.c:

   struct DisasContextArch {
   DisasContext common;
   };

   // implement "hooks"

   void gen_intermediate_code(CPUArchState *env, TranslationBlock *tb)
   {
  DisasContextArch dc;
  gen_intermediate_code_template(get_cpu(env), , tb);


Pointer down into "common" here...


   }

* translate-template.h:

   struct DisasContext { /* ... */ };

   void gen_intermediate_code_template(CPUState *cpu, DisasContext *dc,
   TranslationBlock *tb)
   {
   // init dc
   // arch-specific init dc hook

   // gen_icount, etc.
   while (true) {
   // generic processing code with calls to hooks


... means you have to upcast to DisasContextArch here, or in the hooks 
themselves.


   }
   }

While initially simpler, the "template code" still feels a little dirty to
me. With QOM, you could implement a DisasContextClass that provides the
per-target hook pointers, which can be globally allocated once per target and
pointed to by the DisasContext allocated in stack.

I'm not sure about what you mean by exposing DisasContext in places it shouldn't
be, though.


You either split DisasContextArch / DisasContext in "odd" ways, and then have 
to cast back and forth, or you have to expose the whole of DisasContextArch. 
It's the latter that I considered inappropriate.


Alternately... can we broach the subject of C++?  Honestly, it seems we work 
too hard sometimes to re-implement templates and classes in C.



r~

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Lluís Vilanova]

Richard Henderson writes:

> On 03/09/2016 09:38 AM, Lluís Vilanova wrote:
>> Hi,
>> 
>> NOTE: I won't be throwing patches anytime soon, I just want to know if 
>> there's
>> interest in this for the future.
>> 
>> While adding events for tracing guest instructions, I've found that the
>> per-target "gen_intermediate_code()" function is very similar but not exactly
>> the same for each of the targets. This makes architecture-agnostic features
>> harder to maintain across targets, specially when it comes to their relative
>> order.
>> 
>> So, would it be worth it if I generalized part of that code into an
>> architecture-agnostic function that calls into target-specific hooks 
>> wherever it
>> needs extending? There are many ways to do it that we can discuss later.

> It's worth talking about, since I do believe it would make long-term 
> maintenance
> across the targets easier.

> These "target-specific hooks" probably ought not be "hooks" in the
> traditional sense of attaching them to CPUState.  I'd be more comfortable 
> with a
> refactoring that used include files -- maybe .h or maybe .inc.c.  If we do the
> normal sort of hook, then we've got to either expose DisasContext in places we
> shouldn't, or dynamically allocate it.  Neither seems particularly appealing.

Great. I pondered about using QOM vs "template code", and I'm leaning towards
the latter:

* translate.c:

  struct DisasContextArch {
  DisasContext common;
  };

  // implement "hooks"

  void gen_intermediate_code(CPUArchState *env, TranslationBlock *tb)
  {
 DisasContextArch dc;
 gen_intermediate_code_template(get_cpu(env), , tb);
  }

* translate-template.h:

  struct DisasContext { /* ... */ };

  void gen_intermediate_code_template(CPUState *cpu, DisasContext *dc,
  TranslationBlock *tb)
  {
  // init dc
  // arch-specific init dc hook

  // gen_icount, etc.
  while (true) {
  // generic processing code with calls to hooks
  }
  }

While initially simpler, the "template code" still feels a little dirty to
me. With QOM, you could implement a DisasContextClass that provides the
per-target hook pointers, which can be globally allocated once per target and
pointed to by the DisasContext allocated in stack.

I'm not sure about what you mean by exposing DisasContext in places it shouldn't
be, though.


Cheers,
  Lluis

Re: [Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Richard Henderson]

On 03/09/2016 09:38 AM, Lluís Vilanova wrote:

Hi,

NOTE: I won't be throwing patches anytime soon, I just want to know if there's
   interest in this for the future.

While adding events for tracing guest instructions, I've found that the
per-target "gen_intermediate_code()" function is very similar but not exactly
the same for each of the targets. This makes architecture-agnostic features
harder to maintain across targets, specially when it comes to their relative
order.

So, would it be worth it if I generalized part of that code into an
architecture-agnostic function that calls into target-specific hooks wherever it
needs extending? There are many ways to do it that we can discuss later.


It's worth talking about, since I do believe it would make long-term 
maintenance across the targets easier.


These "target-specific hooks" probably ought not be "hooks" in the
traditional sense of attaching them to CPUState.  I'd be more comfortable with 
a refactoring that used include files -- maybe .h or maybe .inc.c.  If we do 
the normal sort of hook, then we've got to either expose DisasContext in places 
we shouldn't, or dynamically allocate it.  Neither seems particularly appealing.



r~

[Qemu-devel] [RFC] [tcg] Idea on refactoring target code generation loop (gen_intermediate_code)

[Lluís Vilanova]

Hi,

NOTE: I won't be throwing patches anytime soon, I just want to know if there's
  interest in this for the future.

While adding events for tracing guest instructions, I've found that the
per-target "gen_intermediate_code()" function is very similar but not exactly
the same for each of the targets. This makes architecture-agnostic features
harder to maintain across targets, specially when it comes to their relative
order.

So, would it be worth it if I generalized part of that code into an
architecture-agnostic function that calls into target-specific hooks wherever it
needs extending? There are many ways to do it that we can discuss later.


Cheers,
  Lluis