Re: [fpc-devel] Class field reordering
On 14 Jul 2012, at 08:00, Martin Schreiber wrote: On Saturday 14 July 2012 07:50:58 Martin Schreiber wrote: On Saturday 14 July 2012 01:44:39 Jonas Maebe wrote: I've implemented an optimization that reorders the instance fields of Delphi-style classes (and only of Delphi-style classes) to minimise memory gaps caused by alignment differences and odd sizes. The effect is the same as when you would change the order of the fields in the source code to achieve this effect. I sometimes need cracker classes in MSEide and MSEgui to fix bugs and make extensinsions for FCL classes in order to access private fields (examples TDataset, TParam, TField, TFiler, TReader, TWriter, TComponent). Is it guaranteed that the local definitions have the same layout as the definition in the FCL unit? If you you use the same packrecords setting, yes (I doubt any such fcl units contain packrecords directives though, in which case they'll use the default). Hmm, up to now I listed in the cracker classes fields up to the last private field I need to access so the cracker classes would not brake by changing or adding successive fields in the original classes. I assume now it is necessary to always list all fields of the original class. If would you want to use that optimization, then yes. Working around the type system of the language is however generally asking for trouble, regardless of what reason you do it for. Jonas___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Class field reordering
On Sat, 14 Jul 2012, Jonas Maebe wrote: On 14 Jul 2012, at 08:00, Martin Schreiber wrote: On Saturday 14 July 2012 07:50:58 Martin Schreiber wrote: On Saturday 14 July 2012 01:44:39 Jonas Maebe wrote: I've implemented an optimization that reorders the instance fields of Delphi-style classes (and only of Delphi-style classes) to minimise memory gaps caused by alignment differences and odd sizes. The effect is the same as when you would change the order of the fields in the source code to achieve this effect. I sometimes need cracker classes in MSEide and MSEgui to fix bugs and make extensinsions for FCL classes in order to access private fields (examples TDataset, TParam, TField, TFiler, TReader, TWriter, TComponent). Is it guaranteed that the local definitions have the same layout as the definition in the FCL unit? If you you use the same packrecords setting, yes (I doubt any such fcl units contain packrecords directives though, in which case they'll use the default). Hmm, up to now I listed in the cracker classes fields up to the last private field I need to access so the cracker classes would not brake by changing or adding successive fields in the original classes. I assume now it is necessary to always list all fields of the original class. If would you want to use that optimization, then yes. Working around the type system of the language is however generally asking for trouble, regardless of what reason you do it for. Indeed. Maybe we can help by making some methods/properties protected ? Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Class field reordering
On Saturday 14 July 2012 14:26:32 Michael Van Canneyt wrote: If would you want to use that optimization, then yes. Working around the type system of the language is however generally asking for trouble, regardless of what reason you do it for. Indeed. Maybe we can help by making some methods/properties protected ? You are talking seriously? If so I'll provide a list of private FCL fields I need to access in MSEide+MSEgui. Thanks Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Class field reordering
On Saturday 14 July 2012 14:07:30 Jonas Maebe wrote: Hmm, up to now I listed in the cracker classes fields up to the last private field I need to access so the cracker classes would not brake by changing or adding successive fields in the original classes. I assume now it is necessary to always list all fields of the original class. If would you want to use that optimization, then yes. Working around the type system of the language is however generally asking for trouble, regardless of what reason you do it for. I do not necessarily want to use field order optimization but if the FPC RTL is compiled with it I need to compile my cracker classes with optimization too. Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Class field reordering
Am 14.07.2012 07:45 schrieb Martin Schreiber mse00...@gmail.com: On Saturday 14 July 2012 01:44:39 Jonas Maebe wrote: Hi, I've implemented an optimization that reorders the instance fields of Delphi-style classes (and only of Delphi-style classes) to minimise memory gaps caused by alignment differences and odd sizes. The effect is the same as when you would change the order of the fields in the source code to achieve this effect. I sometimes need cracker classes in MSEide and MSEgui to fix bugs and make extensinsions for FCL classes in order to access private fields (examples TDataset, TParam, TField, TFiler, TReader, TWriter, TComponent). Is it guaranteed that the local definitions have the same layout as the definition in the FCL unit? Do you have current examples where you fixed bugs? Maybe these can be fixed at the source which might be better than making them protected (though this can still be done for non-bug fields where you need access to certain members) Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Class field reordering
Am 14.07.2012 01:44, schrieb Jonas Maebe: Hi, I've implemented an optimization that reorders the instance fields of Delphi-style classes (and only of Delphi-style classes) to minimise memory gaps caused by alignment differences and odd sizes. The effect is the same as when you would change the order of the fields in the source code to achieve this effect. In general, I think this should be safe since unlike records and TP-style objects, a Delphi class is normally never blockwritten to disk or so. There is a switch to disable this transformation, but I'm wondering whether anyone sees a problem with enabling it by default when -O2 or higher is used. The only drawback I see that people might sort fields for better cache usage but this is rarely the case imo. So if the optimization can be turned off, it should be fine. It works fine with at least the compiler and Lazarus without any ill effects, saving a small amount of memory for both (about 2.5MB on 87MB for the Darwin/x86-64 compiler compiling itslf, and 2.5MB on 62.7MB for a Carbon/i386 Lazarus right after startup and loading some source files). Jonas PS: a similar reordering could be performed for local variables, although the implementation would be quite different. Additionally, different kinds of reordering, e.g. targeting cache improvements by putting fields/local variables often used together next to each other, could also be implemented in the future. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Class field reordering
On 14.07.12 01:44, Jonas Maebe wrote:
I've implemented an optimization that reorders the instance fields of
Delphi-style classes (and only of Delphi-style classes) to minimise
memory gaps caused by alignment differences and odd sizes. The effect is
the same as when you would change the order of the fields in the source
code to achieve this effect.
In general, I think this should be safe since unlike records and
TP-style objects, a Delphi class is normally never blockwritten to disk
or so. There is a switch to disable this transformation, but I'm
wondering whether anyone sees a problem with enabling it by default when
-O2 or higher is used.
As this might cause an overall gain by better cache utilization i would
keep it on by default. But I think it would be best that it can be
disabled per class-definition. If somebody tries to do manual cache
optimization it will usually only be for a couple of classes. Not the
whole project.
So basically, something like
class MyClass = class
{$OPTIMIZATION OFF}
private
FHeavilyUsedValue: Boolean;
FAlsoHeavilyUsedValue: DWord;
{$OPTIMIZATION DEFAULT}
NotOftenUsed: Byte;
end;
should leave the ordering of the first values alone, while optimizing
the rest. Or at least disable reordering for the whole class.
Nico
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Class field reordering
On 14 Jul 2012, at 18:02, Martin Schreiber wrote: I do not necessarily want to use field order optimization but if the FPC RTL is compiled with it I need to compile my cracker classes with optimization too. That's indeed true. Jonas___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Class field reordering
On 14 Jul 2012, at 20:29, Nico Erfurth wrote:
So basically, something like
class MyClass = class
{$OPTIMIZATION OFF}
private
FHeavilyUsedValue: Boolean;
FAlsoHeavilyUsedValue: DWord;
{$OPTIMIZATION DEFAULT}
NotOftenUsed: Byte;
end;
should leave the ordering of the first values alone, while optimizing
the rest. Or at least disable reordering for the whole class.
Only the setting active at the end of the class declaration (before the end)
will have effect. The setting can be changed via {$optimization orderfields}
and {$optimization noorderfields}, just like all other optimization settings.
Jonas___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Class field reordering
On 14 Jul 2012, at 20:52, Jonas Maebe wrote: On 14 Jul 2012, at 18:02, Martin Schreiber wrote: I do not necessarily want to use field order optimization but if the FPC RTL is compiled with it I need to compile my cracker classes with optimization too. That's indeed true. That may actually lead to quite some troubles: if someone recompiles the RTL without optimizations, then your class crackers also have to change that setting. And there's no way to detect how the RTL (or in general: units containing classes you are cracking) has been compiled in your source code. Adding support for something like that is definitely not a road I want to go down -- even a switch to simply treat all private fields as protected would be less bad (not that I would consider such a switch desirable in any way; it's like adding a switch to allow calling functions only declared in the implementation of another unit). On the other hand, regardless of whether or not we would make that transformation the default for the release units, you will always get problems if someone recompiles the FPC units using the opposite setting. So that's mainly an argument against including this feature altogether in the compiler, at least without additional hacks to make class crackers a sort of supported feature one way or another (which I would be quite reluctant to do). This demonstrates a real pain of low level languages: there's almost always at least one way in which someone has figured out a way to break the language, killing a lot of possible optimizations and error checking you would normally be able to do without any problem, unless you want to go through the trouble of annoying at least some users (or, in cases like this one, adding extra complexity to turn that breakage into a supported feature). I once had a discussion with a person who worked on IBM's C compiler, and he said they couldn't even reorder local variables that do not fit in a register because certain large commercial code bases they had to be able to compile correctly explicitly relied on buffer overflows from one local variable into the next according to the source declaration order (and annoying those users was not an option). I guess there are probably no such FPC-compiled programs yet (*), so maybe now is the time to add that before we also get stuck with that particular piece of ballast. Jonas (*) except for a few in the FPC testsuite whose purpose is to verify that certain code in fact does not cause a buffer overflow; but those can be easily changed to use all variables of the same size to prevent reordering___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Class field reordering
Am 14.07.2012 21:48 schrieb Jonas Maebe jonas.ma...@elis.ugent.be: I guess there are probably no such FPC-compiled programs yet (*), so maybe now is the time to add that before we also get stuck with that particular piece of ballast. Somehow I have the feeling that we all know what one of your upcoming commits will be :P Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
