Re: [webkit-dev] global new/delete operator in WebKit

[Alexey Proskuryakov]

On 18.02.2010, at 6:59, İsmail Dönmez wrote:

Recently I came across some memory corruption issues and turns out  
that WebKit uses std::stable_sort which uses the operator  
new(nothrow) . I am not sure if there is a useful replacement but  
its good to know about this.


We definitely need a replacement - another problematic place is  
JSArray::sort, which should be a stable sort, but isn't on platforms  
that don't provide mergesort.


- WBR, Alexey Proskuryakov

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] global new/delete operator in WebKit

[İsmail Dönmez]

Hi Stephan;

On Thu, Feb 18, 2010 at 5:30 PM, Stephan Assmus  wrote:

> Hi,
>
> On 2010-02-18 at 15:59:35 [+0100], İsmail Dönmez 
> wrote:
> > Picking up and old thread I know...
> >
> > On Wed, Jan 13, 2010 at 7:19 PM, Darin Adler  wrote:
> >
> > > Yong Li also asked about standard library functions calling new and
> > > delete,
> > > specifically STL. I believe we have been avoiding calling these
> functions
> > > in
> > > WebKit, but I may be mistaken.
> > >
> >
> > Recently I came across some memory corruption issues and turns out that
> > WebKit uses std::stable_sort which uses the operator new(nothrow) . I am
> not
> > sure if there is a useful replacement but its good to know about this.
>
> Can you elaborate on this? Were these the cause for your memory corruption?
> If yes, where and how did you solve it?
>
>
For my own WinCE port I override new & delete globally via a special memory
pool. I override all 8 signatures of new & delete so its supposed to work
fine. But...

std::stable_sort calls operator new(nothrow) which somehow does not pick up
my replacement but it does use my delete replacement. So I end up deleting
memory I didn't allocate which as expected crashes.

I am still debugging why STL would not use my replacement function, but
meanwhile beware about this :-)

Regards,
İsmail
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] global new/delete operator in WebKit

[Stephan Assmus]

Hi,

On 2010-02-18 at 15:59:35 [+0100], İsmail Dönmez  wrote:
> Picking up and old thread I know...
> 
> On Wed, Jan 13, 2010 at 7:19 PM, Darin Adler  wrote:
> 
> > Yong Li also asked about standard library functions calling new and 
> > delete,
> > specifically STL. I believe we have been avoiding calling these functions 
> > in
> > WebKit, but I may be mistaken.
> >
> 
> Recently I came across some memory corruption issues and turns out that
> WebKit uses std::stable_sort which uses the operator new(nothrow) . I am not
> sure if there is a useful replacement but its good to know about this.

Can you elaborate on this? Were these the cause for your memory corruption? 
If yes, where and how did you solve it?

Best regards,
-Stephan
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] global new/delete operator in WebKit

[İsmail Dönmez]

Hi;

Picking up and old thread I know...

On Wed, Jan 13, 2010 at 7:19 PM, Darin Adler  wrote:

> Yong Li also asked about standard library functions calling new and delete,
> specifically STL. I believe we have been avoiding calling these functions in
> WebKit, but I may be mistaken.
>

Recently I came across some memory corruption issues and turns out that
WebKit uses std::stable_sort which uses the operator new(nothrow) . I am not
sure if there is a useful replacement but its good to know about this.

Regards,
İsmail
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] global new/delete operator in WebKit

[Darin Adler]

On Jan 13, 2010, at 6:25 PM, Yong Li wrote:

> Sorry for my English. "does" should be "will".

I think your question is unclear. The WebKit project will not “ban” use of a 
global new/delete operator.

-- Darin

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] global new/delete operator in WebKit

[Yong Li]

2010/1/13 Zoltan Horvath 

>
> First, Darin thanks for the summary!
>
> On Wednesday 13 January 2010, at 20:23, Yong Li wrote:
> > Darin, thank you very much for writing this for my questions. I have
> > another question for "new int[10]", but I guess the answer will be "using
> > Vector instead".
>
> At the most of the cases WebKit doesn't use arrays like this.
>

I'm told that fastNewArray is for this.

>
> > I like the idea that webkit should allow using a custom memory allocator,
> > but I think for most platforms/compilers, overloading global new/delete
> > operators is much better than deriving all classes from FastAllocBase.
> I've
> > seen even the simple utility class "Noncopyable" is derived from
> > FastAllocBase, and the follwing code:
> > class A: Noncopyable
> > must be changed to:
> > class A: public Noncopyable
> > in order to make "new A" accessible.
>
> As Darin said, there are platforms where you can not orverload global
> new/delete, because it causes problems.
>

A silly question: will this abnormal problem be fixed some day?

Yes, you have to inherit it publicly, fortunately, the publicly inheriting
> doesn't cause performance or other loss at this cases.
>
> > As webkit provides the option of using custom allocator with
> FastAllocBase
> > for some special platform/compiler, why does it ban the option of using
> > global new/delete operators on other platforms/compilers? At least, when
> > USE_SYSTEM_MALLOC=1, FastAllocBase should just be empty or redirect to
> > global new/delete in my opinion.
> >
> > -Yong
>
> We don't ban it yet. :-) What kind of platforms do mean?
>

Sorry for my English. "does" should be "will". I just think overloading
global new/delete operators is much cleaner than deriving all classes from a
root class and using fastNew<> and fastNewArray<>, unless the platform
doesn't support it, though :)


> Zoltan
>
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] global new/delete operator in WebKit

[Zoltan Horvath]

First, Darin thanks for the summary!

On Wednesday 13 January 2010, at 20:23, Yong Li wrote:
> Darin, thank you very much for writing this for my questions. I have
> another question for "new int[10]", but I guess the answer will be "using
> Vector instead".

At the most of the cases WebKit doesn't use arrays like this.

> I like the idea that webkit should allow using a custom memory allocator,
> but I think for most platforms/compilers, overloading global new/delete
> operators is much better than deriving all classes from FastAllocBase. I've
> seen even the simple utility class "Noncopyable" is derived from
> FastAllocBase, and the follwing code:
> class A: Noncopyable
> must be changed to:
> class A: public Noncopyable
> in order to make "new A" accessible.

As Darin said, there are platforms where you can not orverload global 
new/delete, because it causes problems.

Yes, you have to inherit it publicly, fortunately, the publicly inheriting 
doesn't cause performance or other loss at this cases.

> As webkit provides the option of using custom allocator with FastAllocBase
> for some special platform/compiler, why does it ban the option of using
> global new/delete operators on other platforms/compilers? At least, when
> USE_SYSTEM_MALLOC=1, FastAllocBase should just be empty or redirect to
> global new/delete in my opinion.
>
> -Yong

We don't ban it yet. :-) What kind of platforms do mean?

Zoltan


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] global new/delete operator in WebKit

[Yong Li]

Darin, thank you very much for writing this for my questions. I have another
question for "new int[10]", but I guess the answer will be "using Vector
instead".

I like the idea that webkit should allow using a custom memory allocator,
but I think for most platforms/compilers, overloading global new/delete
operators is much better than deriving all classes from FastAllocBase. I've
seen even the simple utility class "Noncopyable" is derived from
FastAllocBase, and the follwing code:
class A: Noncopyable
must be changed to:
class A: public Noncopyable
in order to make "new A" accessible.

As webkit provides the option of using custom allocator with FastAllocBase
for some special platform/compiler, why does it ban the option of using
global new/delete operators on other platforms/compilers? At least, when
USE_SYSTEM_MALLOC=1, FastAllocBase should just be empty or redirect to
global new/delete in my opinion.

-Yong

2010/1/13 Darin Adler 

> In the discussion in bug 32831, Yong Li asked me why we are planning, in
> the future, to remove the global new/delete operator in the Mac OS X WebKit.
>
> I decided it was worth answering here.
>
> Why does WebKit override operator new/delete? Historically on Mac OS X we
> got faster performance by using a custom malloc in the WebKit framework. At
> some future date we may discover that we no longer get a performance boost
> from this, and change this on Mac OS X.
>
> Ports to other platforms should not assume they need to use a custom
> allocator. It’s not a given that the FastMalloc code will be faster than
> using the platform malloc/free directly. But it was true in the first WebKit
> platform, Mac OS X, at the time.
>
> On Mac OS X, WebKit overrides the global operator new and delete to
> accomplish this. That was a big time saver — it made all new and delete use
> the faster allocator without code changes to all the classes. This
> particular technique works well for frameworks on Mac OS X. The inlined new
> and delete operator functions are marked “private extern”, which means they
> are visible inside the WebKit umbrella framework, but not seen by programs
> loading the framework. As long as we don’t have any new/delete allocations
> that cross the boundary between WebKit and the host application or other
> frameworks, this works fine. The new/delete invocations inside WebKit use
> the global new/delete defined inside the framework and the new/delete
> invocations outside WebKit are not affected.
>
> On some other platforms, though, this creates problems. An operator new and
> delete defined inside the WebKit library may not even work, depending on how
> C++ binds the symbols. The “private extern” extension may not work at all.
> Or the API for that platform might involve objects where the new is done
> inside the framework and the delete is done outside the framework. A simple
> solution would be to not override the global new/delete operator on those
> platforms.
>
> But some contributors want to use a custom allocator even though the global
> new/delete operator technique does not work on the platform they care about.
> So they began the massive task of adding code so that all allocations in
> WebKit were done through the custom allocator without relying on global
> new/delete. This project, see <
> https://bugs.webkit.org/show_bug.cgi?id=20422> began in 2008. The
> check-ins to put it into practice began las April <
> http://trac.webkit.org/changeset/42344>. We’ve still got a ways to go to
> get it deployed throughout the WebKit code.
>
> To keep this code maintained properly, I propose that no platform rely on
> overriding the global new/delete operator inside WebKit once the work is
> complete. Instead I think we can use global new/delete operator as a
> debugging technique to make sure people remember to use FastAllocBase and
> fastNew/Delete rather than invoking the built-in C++ new/delete by accident.
> This has no direct benefit for the Mac OS X version of WebKit, but I believe
> it’s helpful for the community to use that platform to support the others
> that wish to do this.
>
> Yong Li also asked about standard library functions calling new and delete,
> specifically STL. I believe we have been avoiding calling these functions in
> WebKit, but I may be mistaken.
>
> I haven’t discussed this tradeoff in detail with others at Apple, so I’m
> not absolutely certain what we’ll do in the Mac OS X version.
>
> If you have any questions or concerns, please bring them up here.
>
>-- Darin
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

[webkit-dev] global new/delete operator in WebKit

[Darin Adler]

In the discussion in bug 32831, Yong Li asked me why we are planning, in the 
future, to remove the global new/delete operator in the Mac OS X WebKit.

I decided it was worth answering here.

Why does WebKit override operator new/delete? Historically on Mac OS X we got 
faster performance by using a custom malloc in the WebKit framework. At some 
future date we may discover that we no longer get a performance boost from 
this, and change this on Mac OS X.

Ports to other platforms should not assume they need to use a custom allocator. 
It’s not a given that the FastMalloc code will be faster than using the 
platform malloc/free directly. But it was true in the first WebKit platform, 
Mac OS X, at the time.

On Mac OS X, WebKit overrides the global operator new and delete to accomplish 
this. That was a big time saver — it made all new and delete use the faster 
allocator without code changes to all the classes. This particular technique 
works well for frameworks on Mac OS X. The inlined new and delete operator 
functions are marked “private extern”, which means they are visible inside the 
WebKit umbrella framework, but not seen by programs loading the framework. As 
long as we don’t have any new/delete allocations that cross the boundary 
between WebKit and the host application or other frameworks, this works fine. 
The new/delete invocations inside WebKit use the global new/delete defined 
inside the framework and the new/delete invocations outside WebKit are not 
affected.

On some other platforms, though, this creates problems. An operator new and 
delete defined inside the WebKit library may not even work, depending on how 
C++ binds the symbols. The “private extern” extension may not work at all. Or 
the API for that platform might involve objects where the new is done inside 
the framework and the delete is done outside the framework. A simple solution 
would be to not override the global new/delete operator on those platforms.

But some contributors want to use a custom allocator even though the global 
new/delete operator technique does not work on the platform they care about. So 
they began the massive task of adding code so that all allocations in WebKit 
were done through the custom allocator without relying on global new/delete. 
This project, see  began in 
2008. The check-ins to put it into practice began las April 
. We’ve still got a ways to go to get 
it deployed throughout the WebKit code.

To keep this code maintained properly, I propose that no platform rely on 
overriding the global new/delete operator inside WebKit once the work is 
complete. Instead I think we can use global new/delete operator as a debugging 
technique to make sure people remember to use FastAllocBase and fastNew/Delete 
rather than invoking the built-in C++ new/delete by accident. This has no 
direct benefit for the Mac OS X version of WebKit, but I believe it’s helpful 
for the community to use that platform to support the others that wish to do 
this.

Yong Li also asked about standard library functions calling new and delete, 
specifically STL. I believe we have been avoiding calling these functions in 
WebKit, but I may be mistaken.

I haven’t discussed this tradeoff in detail with others at Apple, so I’m not 
absolutely certain what we’ll do in the Mac OS X version.

If you have any questions or concerns, please bring them up here.

-- Darin

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev