Re: [boost] Re: what happened to allocators in boost?
Gregory Colvin wrote: > * document whether and how a library allocates memory The Throws clause is a pretty good estimate for "whether". :-) ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: what happened to allocators in boost?
Gregory Colvin <[EMAIL PROTECTED]> writes: > * use the standard parameterization mechanisms (Allocator) when >choosing to parameterize I'm not sure about this one. std::allocator are a mess, and almost everyone knows it. They were designed for containers, and they only barely work there, IMO. Remember the weasel-wording about unequal allocators and the assumptions implementations are allowed to make about allocator's nested types? I really don't want to be in the position of forcing library authors to match users' expectations about the way their particular implementation's allocators work. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Gregory Colvin <[EMAIL PROTECTED]> wrote: > > At the least, we could add the following bullet > >* Discussion of memory management strategy. > > to http://www.boost.org/more/lib_guide.htm#Documentation I think it is a great start. > I'm reluctant to say very much more at this point, as my opinions > may not be suited to the needs of the many different libraries in > Boost. But I think it's generally reasonable to: > * not allocate memory unless it's really necessary $0.02 > * use the standard mechanisms (::operator new or std::allocator) >when it is necessary Boost already has boost::allocator. IMO other boost libaries should consider using boost::allocator instead of ::new and std::allocator. > * use custom allocation mechanisms only when there is a clear >advantage to doing so $.02 > * parameterize only when there is a clear advantage to doing so I'd modify it to * Consider parametrization if your library is to be available for embedded or non-traditional platfroms. > * use the standard parameterization mechanisms (Allocator) when >choosing to parameterize I'd add to it * Follow boost::allocator specification for allocator parameters > * use custom parameterization mechanisms only when there is a >clear advantage to doing so $.02 > * document whether and how a library allocates memory $.02 Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
On Friday, Aug 29, 2003, at 15:14 America/Denver, E. Gladyshev wrote: ... People are adopted to follow similar requirements for STL allocators anyway. I guess they can be recommended to all boost authors who wants to make memory management data types. Perhaps they can be added to the "Guidlines" section http://www.boost.org/more/lib_guide.htm#Guidelines At the least, we could add the following bullet * Discussion of memory management strategy. to http://www.boost.org/more/lib_guide.htm#Documentation I'm reluctant to say very much more at this point, as my opinions may not be suited to the needs of the many different libraries in Boost. But I think it's generally reasonable to: * not allocate memory unless it's really necessary * use the standard mechanisms (::operator new or std::allocator) when it is necessary * use custom allocation mechanisms only when there is a clear advantage to doing so * parameterize only when there is a clear advantage to doing so * use the standard parameterization mechanisms (Allocator) when choosing to parameterize * use custom parameterization mechanisms only when there is a clear advantage to doing so * document whether and how a library allocates memory ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: what happened to allocators in boost?
"E. Gladyshev" <[EMAIL PROTECTED]> writes: > I guess they can be recommended to all boost > authors who wants to make memory management > data types. > > Perhaps they can be added to the "Guidlines" section > http://www.boost.org/more/lib_guide.htm#Guidelines *What* can be recommended? -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Gregory Colvin <[EMAIL PROTECTED]> wrote: > On Friday, Aug 29, 2003, at 13:57 America/Denver, Gregory Colvin wrote: > > On Friday, Aug 29, 2003, at 13:34 America/Denver, E. Gladyshev wrote: > >> ... > >> All I am trying to say is that shared_ptr doesn't specify > >> any requirements on its Deleter parameter. > > > > Bullshit: > > Please excuse my rude language. Don't worry about that. Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Gregory Colvin <[EMAIL PROTECTED]> wrote: > Bullshit: > >template shared_ptr(Y * p, D d); > >Requirements: p must be convertible to T *. D must be >CopyConstructible. The copy constructor and destructor of D >must not throw. The expression d(p) must be well-formed, must >not invoke undefined behavior, and must not throw exceptions. > >Effects: Constructs a shared_ptr that owns the pointer p and >the deleter d. > >Postconditions: use_count() == 1 && get() == p . > >Throws: std::bad_alloc or an implementation-defined exception >when a resource other than memory could not be obtained. > >Exception safety: If an exception is thrown, d(p) is called. > >Notes: When the time comes to delete the object pointed to by p, >the stored copy of d is invoked with the stored copy of p as an >argument. > > http://www.boost.org/libs/smart_ptr/shared_ptr.htm#constructors Well said. :) Sorry about that. I looked at the "Common Requirements" section and description of shared_ptr::reset(); - template void reset(Y * p, D d); Effects: Equivalent to shared_ptr(p, d).swap(*this). - Somehow I missed the constructor docs. Sorry again. Anyway like I said before several times it is not a big deal in my opinion. People are adopted to follow similar requirements for STL allocators anyway. I guess they can be recommended to all boost authors who wants to make memory management data types. Perhaps they can be added to the "Guidlines" section http://www.boost.org/more/lib_guide.htm#Guidelines Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
On Friday, Aug 29, 2003, at 13:57 America/Denver, Gregory Colvin wrote: On Friday, Aug 29, 2003, at 13:34 America/Denver, E. Gladyshev wrote: ... All I am trying to say is that shared_ptr doesn't specify any requirements on its Deleter parameter. Bullshit: Please excuse my rude language. ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: what happened to allocators in boost?
"E. Gladyshev" <[EMAIL PROTECTED]> writes: > --- Gregory Colvin <[EMAIL PROTECTED]> wrote: > >> The C++ standard requires that a copy of an allocator is equivalent >> to the original. > >> Right. If your allocators can't be copied safely then you have a >> problem. Peter's approach is one way to fix the problem. But I >> don't see that shared_ptr has a problem. > > All I am trying to say is that shared_ptr doesn't specify > any requirements on its Deleter parameter. Not so: "Requirements: p must be convertible to T *. D must be CopyConstructible. The copy constructor and destructor of D must not throw. The expression d(p) must be well-formed, must not invoke undefined behavior, and must not throw exceptions." -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
On Friday, Aug 29, 2003, at 13:34 America/Denver, E. Gladyshev wrote: ... All I am trying to say is that shared_ptr doesn't specify any requirements on its Deleter parameter. Bullshit: template shared_ptr(Y * p, D d); Requirements: p must be convertible to T *. D must be CopyConstructible. The copy constructor and destructor of D must not throw. The expression d(p) must be well-formed, must not invoke undefined behavior, and must not throw exceptions. Effects: Constructs a shared_ptr that owns the pointer p and the deleter d. Postconditions: use_count() == 1 && get() == p . Throws: std::bad_alloc or an implementation-defined exception when a resource other than memory could not be obtained. Exception safety: If an exception is thrown, d(p) is called. Notes: When the time comes to delete the object pointed to by p, the stored copy of d is invoked with the stored copy of p as an argument. http://www.boost.org/libs/smart_ptr/shared_ptr.htm#constructors ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Gregory Colvin <[EMAIL PROTECTED]> wrote: > The C++ standard requires that a copy of an allocator is equivalent > to the original. > Right. If your allocators can't be copied safely then you have a > problem. Peter's approach is one way to fix the problem. But I > don't see that shared_ptr has a problem. All I am trying to say is that shared_ptr doesn't specify any requirements on its Deleter parameter. shared_ptr's Deleter is not a C++ standard's allocator. shared_ptr *does* have a problem because it doesn't set any standards for the Deleter type like C++ does for Allocator. It is just one example of the inconsistent memory management concept in boost that should be sorted out eventually. Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
On Friday, Aug 29, 2003, at 12:33 America/Denver, E. Gladyshev wrote: --- Gregory Colvin <[EMAIL PROTECTED]> wrote: [...] It's still not obvious to me. But I suspect I have yet to understand your example. Perhaps Peter can help me here. In his sample solution before, in this thread, he addresses this problem nicely by using static functions that take references to allocator instances. In his solution, shared_ptr doesn't create an internal Allocator copy. The C++ standard requires that a copy of an allocator is equivalent to the original. Instead he binds a static function pointer to an Allocator instance and uses the pointer as Deleter. My sample stateful allocator will work just fine in Peter's code. Right. If your allocators can't be copied safely then you have a problem. Peter's approach is one way to fix the problem. But I don't see that shared_ptr has a problem. ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Gregory Colvin <[EMAIL PROTECTED]> wrote: [...] > It's still not obvious to me. But I suspect I have yet to understand > your example. Perhaps Peter can help me here. In his sample solution before, in this thread, he addresses this problem nicely by using static functions that take references to allocator instances. In his solution, shared_ptr doesn't create an internal Allocator copy. Instead he binds a static function pointer to an Allocator instance and uses the pointer as Deleter. My sample stateful allocator will work just fine in Peter's code. Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
On Friday, Aug 29, 2003, at 10:48 America/Denver, E. Gladyshev wrote: --- Gregory Colvin <[EMAIL PROTECTED]> wrote: [...] Does it make sense? Not to me. Sounds like a very broken allocator design. If I assume that I going to have a full control over my allocator instances (not a very unusual assumption), there is nothing broken here. Whether it is broken or not should be discussed in a specific context. Anyway, my point was that the shared_ptr( Data* p, Deleter ) has a *potential* problem that was not obvious even to to some people here. (it may not be obivous to other developers). It's still not obvious to me. But I suspect I have yet to understand your example. Like I said, I don't think that it is a big deal as soon as we state a set of requirements for boost "deleters"/allocators. (STL standard has). The "Common Requirements" section in the shared_ptr description doesn't seem to have them. The whole point of a shared_ptr is to invoke its deleter when the last shared_ptr to an object goes away. If intervening allocations cause the deleter to be invalid then either the deleter is broken or the code that did the allocations is broken. ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Gregory Colvin <[EMAIL PROTECTED]> wrote: [...] > > Does it make sense? > > Not to me. Sounds like a very broken allocator design. > If I assume that I going to have a full control over my allocator instances (not a very unusual assumption), there is nothing broken here. Whether it is broken or not should be discussed in a specific context. Anyway, my point was that the shared_ptr( Data* p, Deleter ) has a *potential* problem that was not obvious even to to some people here. (it may not be obivous to other developers). Like I said, I don't think that it is a big deal as soon as we state a set of requirements for boost "deleters"/allocators. (STL standard has). The "Common Requirements" section in the shared_ptr description doesn't seem to have them. Perhaps the established requirements will be used by other libraries as well. BTW. I am also concerned about the boost::signals library. It seems like this library could be very usefull for embedded (real-time?) development but it hides std::allocator. If we cannot come up with memory management policies for boost, perhaps we can define a set of no-so-strict guidlines for boost developers. Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
On Friday, Aug 29, 2003, at 00:52 America/Denver, E. Gladyshev wrote:
--- Gregory Colvin <[EMAIL PROTECTED]> wrote:
On Thursday, Aug 28, 2003, at 23:48 America/Denver, E. Gladyshev
wrote:
*pseudo-code*
template< typename T >
sturct my_allocator
{
my_heap_control _heap;
T* create()
{
return _heap.create();
}
void operator()( T* d )
{
_heap.destroy(d);
}
};
Now if I do something like the following code (that looks perfecly
fine).
f()
{
my_allocator a;
{
shared_ptr s( a.create(), a );
...
int* n = a.create();
a(n);
}
}
I got a problem, because by the time when 's' deletes my data
the heap state is changed while 's' still has an old copy
of the heap state.
* If we state that boost allocators must be implemented
like statless policies not like real data types,
this problem is not a big deal.
I am not understanding the above at all, maybe because I don't
know what you mean by "heap control block" or "stateless policies"
or "real data types".
I called "stateless policies" data types that define some actions
(policies) that never change the state of any of this data
type instances.
The "heap control block" is a data type that manages
the state of a memory heap. For instance it can keep
a list/directory of pointers to free block in the heap.
No imagine the the my_allocator has a list of pointers
to free memory blocks in some memory heap.
template< typename T >
sturct my_allocator
{
char** _freeblocks[100];
};
When you write
shared_ptr s( a.create(), a );
's' will create an internal copy of 'a'.
The internal copy will have a copy of _freeblocks[100];
Now we create a new object, that changes the state of _freeblocks[100]
int* n = a.create();
Next, 's' goes out scope and deallocates its object using
its own copy of _freeblocks[100] but this copy is already bad.
It doesn't represent the current configuration of free blocks
in the heap.
Does it make sense?
Not to me. Sounds like a very broken allocator design.
___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
Daryle Walker wrote: > On Wednesday, August 27, 2003, at 2:21 PM, Peter Dimov wrote: > >> I say that the primary motivation to customize memory management >> details is that the default memory manager is slow. > > Just because you dislike the (so-called-by-you) primary motivation, > you want to ban all other motivations? Motivations cannot be banned. The point is that the correct way to deal with the primary motivation is to pressure implementors until it mostly goes away. Adding a way for users to circumvent the default allocator takes off pressure from implementors. There is an inherent conflict here. Obviously a portion of the community has a valid need to replace the default allocator. What is not obvious is that the rest of the community loses in the end. Throughout this thread I have tried to highlight the non-obvious part. This does not mean that I can't see the obvious. ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: what happened to allocators in boost?
On Wednesday, August 27, 2003, at 2:21 PM, Peter Dimov wrote: Daryle Walker wrote: That's an invalid argument, since the STL containers give you a default value for the allocator argument, which makes containers work out of the box, and the default is "for free." I'm afraid it is not. The allocator parameter has a cost. It (a) prevents some very useful optimizations, (b) implementations that take it seriously (i.e. respect ::pointer, ::construct etc) make me suffer a measurable abstraction penalty. I wasn't talking about abstraction "costs," my "for free" phrase was a counter to your (sarcastic?) statement about library vendors providing a 90% solution and then charging for the remaining 10%. Bugs that have no workarounds are always fixed first. :-) You're making a worse assumption here. Who says that the need for a different allocator class is because the default one has-bugs/is-slow/plain-old-sucks? I say that the primary motivation to customize memory management details is that the default memory manager is slow. Just because you dislike the (so-called-by-you) primary motivation, you want to ban all other motivations? The standard allocators have a somewhat different story. They were originally intended to encapsulate the memory model, not the memory management details; this was in the near/far days. The original STL did its own memory management (free lists) independent of the allocator argument. Someone that has actually been present at the relevant meetings can fill the details why the allocators are now what they are (I'm tempted to use Alexander Terekhov's favorite characterization here :-)). There can be an orthogonal need for a different allocator class, which no improvement on the default one can fix. Oh yes, there "can be". And there "can be" a need for something else, too. Drop the sarcasm. I think some of the embedded-system guys around here gave examples. I'll give another one. Some operating systems have "evolved" several memory APIs. (I'm not making this up, look at pre-X Mac OS.) The C++ library provider maps the C++ memory APIs to one of the system memory APIs. You could wrap calls for the other APIs with new allocator classes. And before you counter with "why need the other memory calls at all, just substitute the main memory API," there are other APIs can only accept memory from a certain memory API. In this case, the Java user is 100% screwed since they can't change the memory manager at all. [...] My point exactly. This is why the Java memory manager beats C++'s memory manager in the long run, because the user can't fix it. As I said, performance isn't the only reason for making a new allocator. The vendor may make performance improvements, but probably not orthogonal features. (Are you proposing that the vendor should provide a "Swiss-army-knife" solution every time a suggestion is made.) There is also the latency concern. This is true even if (or especially if) your problem with the default allocator is performance. You can't always make the user with performance concerns wait until "the vendor feels like updating." Daryle ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Gregory Colvin <[EMAIL PROTECTED]> wrote:
> On Thursday, Aug 28, 2003, at 23:48 America/Denver, E. Gladyshev wrote:
> > *pseudo-code*
> >
> > template< typename T >
> > sturct my_allocator
> > {
> >my_heap_control _heap;
> >
> >T* create()
> >{
> > return _heap.create();
> >}
> >void operator()( T* d )
> >{
> > _heap.destroy(d);
> >}
> > };
> >
> > Now if I do something like the following code (that looks perfecly
> > fine).
> >
> > f()
> > {
> > my_allocator a;
> >
> > {
> > shared_ptr s( a.create(), a );
> > ...
> > int* n = a.create();
> > a(n);
> > }
> > }
> >
> > I got a problem, because by the time when 's' deletes my data
> > the heap state is changed while 's' still has an old copy
> > of the heap state.
> >
> > * If we state that boost allocators must be implemented
> > like statless policies not like real data types,
> > this problem is not a big deal.
>
> I am not understanding the above at all, maybe because I don't
> know what you mean by "heap control block" or "stateless policies"
> or "real data types".
I called "stateless policies" data types that define some actions
(policies) that never change the state of any of this data
type instances.
The "heap control block" is a data type that manages
the state of a memory heap. For instance it can keep
a list/directory of pointers to free block in the heap.
No imagine the the my_allocator has a list of pointers
to free memory blocks in some memory heap.
template< typename T >
sturct my_allocator
{
char** _freeblocks[100];
};
When you write
shared_ptr s( a.create(), a );
's' will create an internal copy of 'a'.
The internal copy will have a copy of _freeblocks[100];
Now we create a new object, that changes the state of _freeblocks[100]
int* n = a.create();
Next, 's' goes out scope and deallocates its object using
its own copy of _freeblocks[100] but this copy is already bad.
It doesn't represent the current configuration of free blocks
in the heap.
Does it make sense?
Eugene
__
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com
___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
On Thursday, Aug 28, 2003, at 23:48 America/Denver, E. Gladyshev wrote:
--- Gregory Colvin <[EMAIL PROTECTED]> wrote:
[...]
In this solution, there are some issues with who
controls the instances of the allocator that allocates
Data and instances that delete the Data.
Which issues concern you?
The potential problem is this. Let's assume that
I forgot (or didn't know) that shared_ptr creates
a copy of Deleter.
I can technically create a local allocator class
that will keep the heap control block
as a data memeber.
*pseudo-code*
template< typename T >
sturct my_allocator
{
my_heap_control _heap;
T* create()
{
return _heap.create();
}
void operator()( T* d )
{
_heap.destroy(d);
}
};
Now if I do something like the following code (that looks perfecly
fine).
f()
{
my_allocator a;
{
shared_ptr s( a.create(), a );
...
int* n = a.create();
a(n);
}
}
I got a problem, because by the time when 's' deletes my data
the heap state is changed while 's' still has an old copy
of the heap state.
* If we state that boost allocators must be implemented
like statless policies not like real data types,
this problem is not a big deal.
I am not understanding the above at all, maybe because I don't
know what you mean by "heap control block" or "stateless policies"
or "real data types".
But I guess it is not big deal as soon as people understand it.
If are to use template template parameters,
Yes, the template template parameter is a pain. Probably better to
take an arbitrary allocator object and rebind it:
template
shared_ptr(T* p, D d, const A& a)
: px(p), pn(p, d, A::rebind<
(a))
{
detail::sp_enable_shared_from_this(p, p, pn);
}
it would be nice to be able to define one allocator
type for both counter and data.
template
class Allocator>
shared_ptr(
Data*,
Allocator,
const Allocator >& );
This constructor would conflict with the constructors that take a
deleter, which must be a function pointer or function object, and
which might not use an allocator.
Right, a new constructor is not needed.
But for the purpose of creating shared objects using an allocator
you could instead use a factory function something like:
template
shared_ptr
allocate_shared_data(const Data& data, Allocator allocator)
{
struct deleter {
Allocator a;
deleter(const Allocator& a) : a(a) {}
void operator()(Data* p) {
a.deallocate(p,1);
a.destroy(p);
}
};
Data* p = allocator.allocate(1);
allocator.construct(p,data);
return shared_ptr(p,deleter(allocator),allocator);
}
I like it.
Or you can add operator()(Data*) to your allocator
and use it directly in shared_ptr and STL.
Right.
template< typename T >
struct my_allocator : std::alocator
{
void operator()(T* d)
{
try
{
d->~T();
}
catch(...)
{
deallocate(d, 1);
throw std::runtime_error("");
}
deallocate(d, 1);
return;
}
};
shared_ptr s( p, my_allocator(), my_allocator() );
I am not sure how to specify the counter allocator in this case
What is legal syntax for template template parameters in functions?
We just got rid of the template template parameter.
Do you have to put the whole thing in.
shared_ptr s( p, my_allocator(),
my_allocator >() );
or there is another way?
my_allocator a;
shared_ptr s(p,a,a);
___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Gregory Colvin <[EMAIL PROTECTED]> wrote:
[...]
> > In this solution, there are some issues with who
> > controls the instances of the allocator that allocates
> > Data and instances that delete the Data.
>
> Which issues concern you?
The potential problem is this. Let's assume that
I forgot (or didn't know) that shared_ptr creates
a copy of Deleter.
I can technically create a local allocator class
that will keep the heap control block
as a data memeber.
*pseudo-code*
template< typename T >
sturct my_allocator
{
my_heap_control _heap;
T* create()
{
return _heap.create();
}
void operator()( T* d )
{
_heap.destroy(d);
}
};
Now if I do something like the following code (that looks perfecly fine).
f()
{
my_allocator a;
{
shared_ptr s( a.create(), a );
...
int* n = a.create();
a(n);
}
}
I got a problem, because by the time when 's' deletes my data
the heap state is changed while 's' still has an old copy
of the heap state.
* If we state that boost allocators must be implemented
like statless policies not like real data types,
this problem is not a big deal.
>
> > But I guess it is not big deal as soon as people understand it.
> > If are to use template template parameters,
>
> Yes, the template template parameter is a pain. Probably better to
> take an arbitrary allocator object and rebind it:
>
> template
> shared_ptr(T* p, D d, const A& a)
> : px(p), pn(p, d, A::rebind<
> >(a))
> {
> detail::sp_enable_shared_from_this(p, p, pn);
> }
>
> > it would be nice to be able to define one allocator
> > type for both counter and data.
> >
> > template
> > class Allocator>
> > shared_ptr(
> >Data*,
> >Allocator,
> >const Allocator > Deleter> >& );
>
> This constructor would conflict with the constructors that take a
> deleter, which must be a function pointer or function object, and
> which might not use an allocator.
Right, a new constructor is not needed.
> But for the purpose of creating shared objects using an allocator
> you could instead use a factory function something like:
>
> template
> shared_ptr
> allocate_shared_data(const Data& data, Allocator allocator)
> {
>struct deleter {
> Allocator a;
> deleter(const Allocator& a) : a(a) {}
> void operator()(Data* p) {
> a.deallocate(p,1);
> a.destroy(p);
> }
>};
>Data* p = allocator.allocate(1);
>allocator.construct(p,data);
>return shared_ptr(p,deleter(allocator),allocator);
> }
I like it.
Or you can add operator()(Data*) to your allocator
and use it directly in shared_ptr and STL.
template< typename T >
struct my_allocator : std::alocator
{
void operator()(T* d)
{
try
{
d->~T();
}
catch(...)
{
deallocate(d, 1);
throw std::runtime_error("");
}
deallocate(d, 1);
return;
}
};
shared_ptr s( p, my_allocator(), my_allocator() );
I am not sure how to specify the counter allocator in this case
What is legal syntax for template template parameters in functions?
Do you have to put the whole thing in.
shared_ptr s( p, my_allocator(), my_allocator >() );
or there is another way?
Eugene
__
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com
___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
On Thursday, Aug 28, 2003, at 19:40 America/Denver, E. Gladyshev wrote:
--- Gregory Colvin <[EMAIL PROTECTED]> wrote:
shared_ptr doesn't allocate the data, it only deletes it, which is
the job of the
current deleter parameter. And the counter type is by design not
part of the
shared_ptr type, so it doesn't belong as parameter to the shared_ptr
template.
See:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/boost/boost/boost/
shared_ptr.hpp
So you what you might want is to add something more like this to
shared_ptr:
template
class Allocator>
shared_ptr(
Data*,
Deleter,
const Allocator >& );
The idea being that you can use this constructor to get complete
control over how
a particular shared_ptr manages memory.
In this solution, there are some issues with who
controls the instances of the allocator that allocates
Data and instances that delete the Data.
Which issues concern you?
But I guess it is not big deal as soon as people understand it.
If are to use template template parameters,
Yes, the template template parameter is a pain. Probably better to
take an arbitrary allocator object and rebind it:
template
shared_ptr(T* p, D d, const A& a)
: px(p), pn(p, d, A::rebind<
>(a))
{
detail::sp_enable_shared_from_this(p, p, pn);
}
it would be nice to be able to define one allocator
type for both counter and data.
template
class Allocator>
shared_ptr(
Data*,
Allocator,
const Allocator >& );
This constructor would conflict with the constructors that take a
deleter, which must be a function pointer or function object, and
which might not use an allocator.
But for the purpose of creating shared objects using an allocator
you could instead use a factory function something like:
template
shared_ptr
allocate_shared_data(const Data& data, Allocator allocator)
{
struct deleter {
Allocator a;
deleter(const Allocator& a) : a(a) {}
void operator()(Data* p) {
a.deallocate(p,1);
a.destroy(p);
}
};
Data* p = allocator.allocate(1);
allocator.construct(p,data);
return shared_ptr(p,deleter(allocator),allocator);
}
___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Gregory Colvin <[EMAIL PROTECTED]> wrote: > > shared_ptr doesn't allocate the data, it only deletes it, which is the > job of the > current deleter parameter. And the counter type is by design not part > of the > shared_ptr type, so it doesn't belong as parameter to the shared_ptr > template. > See: > > http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/boost/boost/boost/ > shared_ptr.hpp > > So you what you might want is to add something more like this to > shared_ptr: > > template > class Allocator> > shared_ptr( >Data*, >Deleter, >const Allocator Deleter> >& ); > > The idea being that you can use this constructor to get complete > control over how > a particular shared_ptr manages memory. In this solution, there are some issues with who controls the instances of the allocator that allocates Data and instances that delete the Data. But I guess it is not big deal as soon as people understand it. If are to use template template parameters, it would be nice to be able to define one allocator type for both counter and data. template class Allocator> shared_ptr( Data*, Allocator, const Allocator >& ); Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Gregory Colvin <[EMAIL PROTECTED]> wrote:
> On Thursday, Aug 28, 2003, at 16:26 America/Denver, E. Gladyshev wrote:
> > --- Gregory Colvin <[EMAIL PROTECTED]> wrote:
> >>> How will I even know it, the documentation is completely
> >>> ignorant on the memory issues.
> >>
> >> Perhaps because you work with the authors of the documentation to
> >> make it sure it says what needs saying?
> >
> > Are the documentation authors monitoring this mailing list?
> >
> >> And I have no objection myself to adding an allocator parameter
> >> to the shared_ptr constructor, or to making some other change that
> >> serves the purpose. So if you need a change, why not just do it,
> >> try it out, and submit a patch?
> >
> > How about
> >
> > template< typename T, typename Counter = int >
> > shared_ptr
> > {
> >typedef Counter counter; //counter type should be public
> >
> >template , typename
> > CounterAlloc=std::allocator
> >>
> >shared_ptr( const DataAlloc& da = DataAlloc(), const IntAlloc ia& =
> > CountAlloc() );
> > };
>
> shared_ptr doesn't allocate the data, it only deletes it, which is the
> job of the
> current deleter parameter. And the counter type is by design not part
> of the
> shared_ptr type, so it doesn't belong as parameter to the shared_ptr
> template.
> See:
>
> http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/boost/boost/boost/
> shared_ptr.hpp
>
> So you what you might want is to add something more like this to
> shared_ptr:
>
> template
> class Allocator>
> shared_ptr(
>Data*,
>Deleter,
>const Allocator Deleter> >& );
>
> The idea being that you can use this constructor to get complete
> control over how
> a particular shared_ptr manages memory.
There are some issues with ho
__
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com
___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
On Thursday, Aug 28, 2003, at 16:26 America/Denver, E. Gladyshev wrote:
--- Gregory Colvin <[EMAIL PROTECTED]> wrote:
How will I even know it, the documentation is completely
ignorant on the memory issues.
Perhaps because you work with the authors of the documentation to
make it sure it says what needs saying?
Are the documentation authors monitoring this mailing list?
And I have no objection myself to adding an allocator parameter
to the shared_ptr constructor, or to making some other change that
serves the purpose. So if you need a change, why not just do it,
try it out, and submit a patch?
How about
template< typename T, typename Counter = int >
shared_ptr
{
typedef Counter counter; //counter type should be public
template , typename
CounterAlloc=std::allocator
shared_ptr( const DataAlloc& da = DataAlloc(), const IntAlloc ia& =
CountAlloc() );
};
shared_ptr doesn't allocate the data, it only deletes it, which is the
job of the
current deleter parameter. And the counter type is by design not part
of the
shared_ptr type, so it doesn't belong as parameter to the shared_ptr
template.
See:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/boost/boost/boost/
shared_ptr.hpp
So you what you might want is to add something more like this to
shared_ptr:
template
class Allocator>
shared_ptr(
Data*,
Deleter,
const Allocator >& );
The idea being that you can use this constructor to get complete
control over how
a particular shared_ptr manages memory.
___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
On Thursday, Aug 28, 2003, at 16:54 America/Denver, E. Gladyshev wrote: --- Gregory Colvin <[EMAIL PROTECTED]> wrote: And I have no objection myself to adding an allocator parameter to the shared_ptr constructor, or to making some other change that serves the purpose. So if you need a change, why not just do it, try it out, and submit a patch? Just wondering what is the submission procedure exactly? Dunno. My last submission was too long ago. For example I believe that shared_ptr must absolutely have the counter type as a template parameter. And I believe it absolutely must not: http://std.dkuug.dk/jtc1/sc22/wg21/docs/papers/2003/n1450.html ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
On Thursday 28 August 2003 06:26 pm, E. Gladyshev wrote:
> --- Gregory Colvin <[EMAIL PROTECTED]> wrote:
> > Perhaps because you work with the authors of the documentation to
> > make it sure it says what needs saying?
>
> Are the documentation authors monitoring this mailing list?
The developers generally write the documentation, or at the very least have
authority over what it says.
> How about
>
> template< typename T, typename Counter = int >
> shared_ptr
> {
>typedef Counter counter; //counter type should be public
>
>template , typename
> CounterAlloc=std::allocator
>
>shared_ptr( const DataAlloc& da = DataAlloc(), const IntAlloc ia& =
> CountAlloc() ); };
I suggest that you:
(1) flesh out the interface you want,
(2) decide what the specific semantics of the interface are, and
(3) put together at least a toy implementation (at least try to compile the
interface specification; the above isn't valid C++, for instance); then
(4) start a new thread to discuss the semantics you need and the syntax you
want.
> Actually this is probably what the whole thread is about
> to get people to try to find the best practicle solution
> for memory management in the modern C++ age.
> I am not sure that STL style allocators is the best
> possible solution.
Nor is anyone else. But the point is that grand visions cannot go anywhere
without specific examples. Remember: Boost policies are implemented bottom-up
not top-down.
You have suggested a way to fix your problem for shared_ptr, and I suggest you
follow my recommendation above to come up with a proposal that does what you
need (but do note the comments Peter has already made regarding interfaces
similar to the one you suggest above).
Doug
___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Gregory Colvin <[EMAIL PROTECTED]> wrote: > > And I have no objection myself to adding an allocator parameter > to the shared_ptr constructor, or to making some other change that > serves the purpose. So if you need a change, why not just do it, > try it out, and submit a patch? Just wondering what is the submission procedure exactly? For example I believe that shared_ptr must absolutely have the counter type as a template parameter. Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Douglas Gregor <[EMAIL PROTECTED]> wrote: > On Thursday 28 August 2003 04:40 pm, Gregory Colvin wrote: > > I also have no objection, and much sympathy, for having a clear > > memory management policy for Boost libraries. But again, it is a > > matter of people who care about and understand the issue doing the > > necessary work, just like everything else here at Boost. > > Moreover, it's not a matter of convincing certain people that a clear memory > management policy should be adopted by Boost and handed down for developers > to do the work. Boost doesn't work that way. Policies always come from the > bottom up: someone has a problem, so they fix it in the libraries that matter > most to them. With that knowledge of _how_ to fix the problem correctly, they > can approach other developers and say "hey, I think we should fix this > problem in library X; here's how we did it in library Y". Eventually, most of > the libraries will support it, and _then_ we can approve it as a Boost > "policy" so that future libraries will follow it. > > The most productive thing we could do right now would be to end this policy > discussion. Start with smart_ptr and address *specific* documentation and > *specific* implementation problems in this library, with resolutions specific > to that library. Is there a library that does it well? Reference that library > and state why it does it well, so that others may follow. I agree but there is a problem. It is very easy to put an STL allocator in the shared_ptr constructor. However I don't think that you'll be able to say "hey, I think we should fix this problem in library X; here's how we did it in library Y" The question is should we try to find a solution that can work for other libraries so that we could say "hey,..."? Do the library authors have some ideas? Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Gregory Colvin <[EMAIL PROTECTED]> wrote:
> > How will I even know it, the documentation is completely
> > ignorant on the memory issues.
>
> Perhaps because you work with the authors of the documentation to
> make it sure it says what needs saying?
Are the documentation authors monitoring this mailing list?
> And I have no objection myself to adding an allocator parameter
> to the shared_ptr constructor, or to making some other change that
> serves the purpose. So if you need a change, why not just do it,
> try it out, and submit a patch?
How about
template< typename T, typename Counter = int >
shared_ptr
{
typedef Counter counter; //counter type should be public
template , typename
CounterAlloc=std::allocator
>
shared_ptr( const DataAlloc& da = DataAlloc(), const IntAlloc ia& = CountAlloc() );
};
>
> I also have no objection, and much sympathy, for having a clear
> memory management policy for Boost libraries. But again, it is a
> matter of people who care about and understand the issue doing the
> necessary work, just like everything else here at Boost.
I believe that the library authors should care about
memory management most.
If they don't understand something, they should ask people around,
but I am convinced that the boost authors are the best (honest).
I am sure that they can find a practical solution if they
only wanted to.
Actually this is probably what the whole thread is about
to get people to try to find the best practicle solution
for memory management in the modern C++ age.
I am not sure that STL style allocators is the best
possible solution.
Eugene
__
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com
___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
On Thursday 28 August 2003 04:40 pm, Gregory Colvin wrote: > I also have no objection, and much sympathy, for having a clear > memory management policy for Boost libraries. But again, it is a > matter of people who care about and understand the issue doing the > necessary work, just like everything else here at Boost. Moreover, it's not a matter of convincing certain people that a clear memory management policy should be adopted by Boost and handed down for developers to do the work. Boost doesn't work that way. Policies always come from the bottom up: someone has a problem, so they fix it in the libraries that matter most to them. With that knowledge of _how_ to fix the problem correctly, they can approach other developers and say "hey, I think we should fix this problem in library X; here's how we did it in library Y". Eventually, most of the libraries will support it, and _then_ we can approve it as a Boost "policy" so that future libraries will follow it. The most productive thing we could do right now would be to end this policy discussion. Start with smart_ptr and address *specific* documentation and *specific* implementation problems in this library, with resolutions specific to that library. Is there a library that does it well? Reference that library and state why it does it well, so that others may follow. Doug ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
On Thursday, Aug 28, 2003, at 10:46 America/Denver, E. Gladyshev wrote: --- Peter Dimov <[EMAIL PROTECTED]> wrote: You can use all smart pointers except shared_ptr and shared_array as they do not allocate any memory. In particular, intrusive_ptr is a good candidate if memory is a concern as it has smaller memory footprint than shared_ptr. Thanks, I'll consider it next time. However if Boost doesn't have a clear memory management concept, how can I guarantee that the next time around, intrusive_ptr or something else in boost won't start allocating memory under the covers. It is perfectly legal in Boost. Of course future releases of Boost are free to change in completely arbitrary ways that break anything we want. The same is true of the C++ standard, operating system APIs, CPUs, and everything else. So there are no guarantees that you can release a library today and have it keep working with all future releases of Boost, C++, etc. Nonetheless, backwards compatibility matters, and Boost is no more likely to change in ways that break your code than anything else. And to the extent you remain active in Boost, you can help see to it that such changes don't happen. How will I even know it, the documentation is completely ignorant on the memory issues. Perhaps because you work with the authors of the documentation to make it sure it says what needs saying? You can also use Boost.Regex, it is completely allocator-enabled. ;-) I never said that I am a huge allocators fan but at least it seems that regex has one of the most consistent memory management concepts in the library. :) And I have no objection myself to adding an allocator parameter to the shared_ptr constructor, or to making some other change that serves the purpose. So if you need a change, why not just do it, try it out, and submit a patch? I also have no objection, and much sympathy, for having a clear memory management policy for Boost libraries. But again, it is a matter of people who care about and understand the issue doing the necessary work, just like everything else here at Boost. ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
On Thu, 2003-08-28 at 17:46, E. Gladyshev wrote: > --- Peter Dimov <[EMAIL PROTECTED]> wrote: > > > > You can use all smart pointers except shared_ptr and shared_array as they do > > not allocate any memory. In particular, intrusive_ptr is a good candidate if > > memory is a concern as it has smaller memory footprint than shared_ptr. > > Thanks, I'll consider it next time. > However if Boost doesn't have a clear memory management > concept, how can I guarantee that the next time around, > intrusive_ptr or something else in boost > won't start allocating memory under the covers. > It is perfectly legal in Boost. > How will I even know it, the documentation is completely > ignorant on the memory issues. Both dynamic_bitset and multi_array are allocator enabled. array isn't for obvious reasons. What you are looking for is a policy based smart pointer. Boost does not *yet* have one of these, although David Held has one under development in the sandbox but I don't know what state it is in. Policy based design is fairly recent and people are experimenting with them. But then you probably wouldn't want to put experimental code into production ;-). /ikh ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Peter Dimov <[EMAIL PROTECTED]> wrote: > > You can use all smart pointers except shared_ptr and shared_array as they do > not allocate any memory. In particular, intrusive_ptr is a good candidate if > memory is a concern as it has smaller memory footprint than shared_ptr. Thanks, I'll consider it next time. However if Boost doesn't have a clear memory management concept, how can I guarantee that the next time around, intrusive_ptr or something else in boost won't start allocating memory under the covers. It is perfectly legal in Boost. How will I even know it, the documentation is completely ignorant on the memory issues. > You can also use Boost.Regex, it is completely allocator-enabled. ;-) I never said that I am a huge allocators fan but at least it seems that regex has one of the most consistent memory management concepts in the library. :) Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
E. Gladyshev wrote: [...] > I am actually using a separate heap in my general purpose exception > handler. > My exception handler is quite complex, it creates a log, compresses > it, etc. The nice thing about it that I can still use STL containers > while processing exceptions. All I had to do is to implement a > protected allocator. I had to write a simple smart pointer for my > handler to work around Boost limitations. Basically in my handler, > I could use almost nothing from Boost except the Metaprogramming stuff. You can use all smart pointers except shared_ptr and shared_array as they do not allocate any memory. In particular, intrusive_ptr is a good candidate if memory is a concern as it has smaller memory footprint than shared_ptr. You can also use Boost.Regex, it is completely allocator-enabled. ;-) ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
Joe Gottman wrote: > >Has anyone considered adding a new template constructor to shared_ptr > that takes an allocator as an extra template parameter? Something like > > template > shared_ptr(Y *p, D d, Alloc alloc); Yes, I have. ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
Daryle Walker wrote: > > That's an invalid argument, since the STL containers give you a default > value for the allocator argument, which makes containers work out of > the box, and the default is "for free." I'm afraid it is not. The allocator parameter has a cost. It (a) prevents some very useful optimizations, (b) implementations that take it seriously (i.e. respect ::pointer, ::construct etc) make me suffer a measurable abstraction penalty. >> Bugs that have no workarounds are always fixed first. :-) > > You're making a worse assumption here. Who says that the need for a > different allocator class is because the default one > has-bugs/is-slow/plain-old-sucks? I say that the primary motivation to customize memory management details is that the default memory manager is slow. The standard allocators have a somewhat different story. They were originally intended to encapsulate the memory model, not the memory management details; this was in the near/far days. The original STL did its own memory management (free lists) independent of the allocator argument. Someone that has actually been present at the relevant meetings can fill the details why the allocators are now what they are (I'm tempted to use Alexander Terekhov's favorite characterization here :-)). > There can be an orthogonal need for a different allocator class, which no > improvement on the default one can fix. Oh yes, there "can be". And there "can be" a need for something else, too. > In this case, the Java user is 100% screwed since they can't change > the memory manager at all. [...] My point exactly. This is why the Java memory manager beats C++'s memory manager in the long run, because the user can't fix it. ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
E. Gladyshev wrote: > > Sure he could have replaced the system allocator (I actually > suggested it too) but why would he want to do it? Because this saves time in the long run. Once you have a non-broken allocator you can use third party libraries as-is, without need for modification, and you can donate it to the community if you're after the fame (aren't we all.) > The standard system allocator worked just fine for the rest of his > program. Why would he want to implement a full blown memory manager. dlmalloc is not that hard to download. :-) Incidentally, you can #define BOOST_SP_USE_STD_ALLOCATOR or BOOST_SP_USE_QUICK_ALLOCATOR to tell shared_ptr to use std::allocator or boost::detail::quick_allocator for counts when your malloc is slow. But it's better to just replace the global new/delete so that every "new X" in the program benefits. And it's better yet to submit a "your malloc is slow, here's the link to dlmalloc which is five times faster on this real code" bug report. (BTW, "five times faster" is not a figure of speech. I do have real measurements. Had to let you know.) ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: what happened to allocators in boost?
"E. Gladyshev" <[EMAIL PROTECTED]> writes: > --- David Abrahams <[EMAIL PROTECTED]> wrote: >> "E. Gladyshev" <[EMAIL PROTECTED]> writes: >> >> > --- David Abrahams <[EMAIL PROTECTED]> wrote: > >> > Yes, I am sure. I shipped one of my libraries to a guy who >> > works on various DSP chips. The native memory allocator >> > on one of the DSP was very slow (some specific issues with >> > how the DSP addresses the memory, I don't remember the details). >> > >> > This guy had to add his own new/delete operator to some of >> > my internal classes to get about 10 times performance improvement. >> >> None of that indicates that he couldn't have replaced the native >> allocator with a fast general-purpose allocator. >> >> > No, I really don't think that there is a hope for general purpose >> > allocators. >> >> Why not? > > Sure he could have replaced the system allocator (I actually suggested it too) > but why would he want to do it? > The standard system allocator worked just fine for the rest of his program. > Why would he want to implement a full blown memory manager. He would not. He would pick an off-the-shelf memory manager such as dlmalloc that has been shown to perform well in a wide variety of applications. > The problem was that my library had couple classes that get > allocated/deleted very often it was the biggest performance hit. He > just overloaded the new/delete in these classes and built a very > simple fixed-size memory manager in a separate heap just for that. > > In fact a program might require different allocation > strategies depending on the context. Replacing the global > new/delete may not be an option. Hypothetical or real? -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Daryle Walker <[EMAIL PROTECTED]> wrote: > On Monday, August 25, 2003, at 2:57 PM, Peter Dimov wrote: > has-bugs/is-slow/plain-old-sucks? There can be an orthogonal need for > a different allocator class, which no improvement on the default one > can fix. > > In this case, the Java user is 100% screwed since they can't change the > memory manager at all. Or at least, can't change the manager without > changing it everywhere. (Allocator policies allow mix-and-match.) I am glad that boost is not a Java library. :) Don't get me wrong, boost is an amazing library. I am learning from it every minute I look at it, great job guys thank you. However I was a bit shocked when I realized how ignorant boost is about memory policies. For instance boost::signals use std::vector<>, but it doesn't care to do anything about the allocator parameter. Some libraries would mix global new/delete calls with user defined memory policies and all that under the covers and the list goes on and on. I don't like the STL allocators too much but I'd like to see a modern C++ library that has consistent and clear memory policies. Instead of having an allocator template parameter for each class, perhaps we could design a global memory management class that can be scaled horizontally on the data type basis. It will require some way to associate a user defined allocator with data type. Then we would need to define STL-style allocators that will call the boost memory manager. The boost memory policy could be along the lines: 1. All boost classes have to use boost::allocator<> with STL allocators internally. std::vector >; 2. If a boost class needs to create an object it must call boost::mm. int *x = boost::getmm()->create(1); ... boost::getmm()->destroy(x,1); 3. The user can customize memory policies for her class by calling regmm boost::getmm()->regmm( MyAllocator() ); IMHO, in terms of memory management, this simple policy will make boost to look more like a C++ library not Java. It doesn't provide the flexibility of STL allocators but it does give the user some control over the memory. Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: what happened to allocators in boost?
On Monday, August 25, 2003, at 2:57 PM, Peter Dimov wrote:
E. Gladyshev wrote:
--- Peter Dimov <[EMAIL PROTECTED]> wrote:
In this context, an allocator parameter is only an excuse for
implementors to avoid doing the necessary work to make function<>
useful out of the box. "You can always use a custom allocator",
right?
Considering the variety of real life requirements and platforms, it
is not practical to make a library that work out of the box.
The danger in that statement is that it looks obvious, but it's not.
It is true in some sotuations (not as frequent as is generally
believed) and false in others, and when it doesn't really hold but is
axiomatically accepted as truth, it serves as exactly the excuse I was
talking about. "It is not practical to make a library that works out
of the box, so users should be happy that they are getting a library
that can be made to work with a "small" additional investment. Of
course we'll gladly sell you the additional bits and pieces."
That's an invalid argument, since the STL containers give you a default
value for the allocator argument, which makes containers work out of
the box, and the default is "for free."
It is interesting to compare the C++ approach with the Java approach
to memory management in this context. C++ "pragmatically" accepts that
the shipped malloc is often inadequate and provides several types of
hooks that you can use to work around the problem. Java does not. If
the JVM has a broken memory manager you lose and can do nothing about
it. This gives C++ a short term advantage, but in the long term Java
wins, since JVM implementors have no choice but to optimize their
memory managers till they are blue in the face.
Bugs that have no workarounds are always fixed first. :-)
You're making a worse assumption here. Who says that the need for a
different allocator class is because the default one
has-bugs/is-slow/plain-old-sucks? There can be an orthogonal need for
a different allocator class, which no improvement on the default one
can fix.
In this case, the Java user is 100% screwed since they can't change the
memory manager at all. Or at least, can't change the manager without
changing it everywhere. (Allocator policies allow mix-and-match.)
That is why STL has allocators. In one project, if STL did not have
allocators, I would have to implement my own containers. I was so
happy that I didn't have to do it.
So far my experience indicates that people only bother with allocators
when std::allocator is inadequate, i.e. slow. Your case may, of
course, be different. It happens sometimes.
At least a library should be consistent about memory management.
This is an arbitrary requirement.
Taking a quick look at the Standard Library, all of the direct
containers that use dynamic memory take an allocator argument, except
for the maligned std::valarray<>.
The issue is how consistent boost is about it (STL is very
consistent)? It seems that boost doesn't have any idea about how it
manages its memory.
A fairly aggressive way to state it, but I see your point. It is true
that Boost does not impose any requirements on how the particular
libraries obtain and manage their memory. But you need to realize that
"do nothing" can be a conscious design decision. Lack of strict
requirements means more freedom for the implementations to do what
they consider best.
This last sentence is probably why std::valarray<> keeps its memory
policy private.
As an example, consider a hypothetical std::vector that does not
have an allocator argument and "has no idea how it manages its
memory." Do you see the implications of that?
1. vector::resize and vector::reserve can now realloc in place since
they can talk directly to the memory manager.
2. On many implementations, vector can now be made as efficient as
new T[], since new T[] often stores the equivalent of size() and
capacity() anyway to allow delete[] to do the right thing.
For example if a boost class X uses std::list, does it have to expose
the std::list<> allocator or should it use the standard
std::allocator<>.
template
struct X
{
std::list l;
};
OR
template >
struct X
{
std::list l;
};
Which definition is more friendly to boost?
The correct answer depends on X's interface. Your toy example is not a
good illustration as this kind of "interface" is rarely seen.
Is it allowed for a boost libarary to use global new/delete? If so,
should it be documented?
Boost libraries are allowed to use the C++ language without
restrictions.
___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- David Abrahams <[EMAIL PROTECTED]> wrote: > "E. Gladyshev" <[EMAIL PROTECTED]> writes: > > > --- David Abrahams <[EMAIL PROTECTED]> wrote: > > Yes, I am sure. I shipped one of my libraries to a guy who > > works on various DSP chips. The native memory allocator > > on one of the DSP was very slow (some specific issues with > > how the DSP addresses the memory, I don't remember the details). > > > > This guy had to add his own new/delete operator to some of > > my internal classes to get about 10 times performance improvement. > > None of that indicates that he couldn't have replaced the native > allocator with a fast general-purpose allocator. > > > No, I really don't think that there is a hope for general purpose > > allocators. > > Why not? Sure he could have replaced the system allocator (I actually suggested it too) but why would he want to do it? The standard system allocator worked just fine for the rest of his program. Why would he want to implement a full blown memory manager. The problem was that my library had couple classes that get allocated/deleted very often it was the biggest performance hit. He just overloaded the new/delete in these classes and built a very simple fixed-size memory manager in a separate heap just for that. In fact a program might require different allocation strategies depending on the context. Replacing the global new/delete may not be an option. Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
RE: [boost] Re: what happened to allocators in boost?
Title: RE: [boost] Re: what happened to allocators in boost? --- David Abrahams <[EMAIL PROTECTED]> wrote: > "E. Gladyshev" <[EMAIL PROTECTED]> writes: > >> Are you sure that >> there's no good general-purpose allocator which they can use? > >Yes, I am sure. I shipped one of my libraries to a guy who >works on various DSP chips. The native memory allocator >on one of the DSP was very slow (some specific issues with >how the DSP addresses the memory, I don't remember the details). > >This guy had to add his own new/delete operator to some of >my internal classes to get about 10 times performance improvement. > >No, I really don't think that there is a hope for general purpose allocators. If it was a system wide performance problem on that DSP wouldn't he have been better off replacing the system allocator completely rather then only for a few classes? Glen
RE: [boost] Re: what happened to allocators in boost?
> -Original Message- > From: David Abrahams [mailto:[EMAIL PROTECTED] > > "E. Gladyshev" <[EMAIL PROTECTED]> writes: > > I am afraid that some of the boost libraries (as they are now) won't > > work for a big group of real time and embedded developers, where > > customization is the key. > > Is it really (other than culturally, I mean)? Are you sure that > there's no good general-purpose allocator which they can use? What is general purpose? does general purpose include (presumably all at once as iiuyc "there can be only one"): Determinism. Memory usage efficiency. Ability to share allocated objects between threads. Low/no impact on scheduling latency. Speed. Freedom from fragmentation over an arbitrarily long uptime (years). Bonus marks for the last on systems with no mmu. Policy based allocators anyone? But then, I'd probably wan't (need?) policy based smart pointers to use the allocators with. At least some of the above conflicting requirements imply policy decisions about the pointers to this storage. The low latency, but no sharing between threads allocator rather implies that managing the object lifetime using a thread-safe shared_ptr is inefficient at best and dangerous at worst - including the allocator in the type wouldn't be a bad thing here. That suggests to me that policy based smart pointers are the way to go for some (all?) such users/uses. I don't think this in any way detracts from the utility of boost::shared_ptr<> as it stands - a lot of the time, I just "want it to work". Nevertheless there are systems for which which I can't/won't use it (at least not until I've had a chance to digest Peter Dimov's suggestions better). Just one user opinion - ymmv. Regards Darryl Green. ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: what happened to allocators in boost?
"E. Gladyshev" <[EMAIL PROTECTED]> writes: > --- David Abrahams <[EMAIL PROTECTED]> wrote: >> "E. Gladyshev" <[EMAIL PROTECTED]> writes: >> >> > I am afraid that some of the boost libraries (as they are now) won't >> > work for a big group of real time and embedded developers, where >> > customization is the key. >> >> Is it really (other than culturally, I mean)? > > The culture is changing. :) > >> Are you sure that >> there's no good general-purpose allocator which they can use? > > Yes, I am sure. I shipped one of my libraries to a guy who > works on various DSP chips. The native memory allocator > on one of the DSP was very slow (some specific issues with > how the DSP addresses the memory, I don't remember the details). > > This guy had to add his own new/delete operator to some of > my internal classes to get about 10 times performance improvement. None of that indicates that he couldn't have replaced the native allocator with a fast general-purpose allocator. > No, I really don't think that there is a hope for general purpose > allocators. Why not? -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- David Abrahams <[EMAIL PROTECTED]> wrote: > "E. Gladyshev" <[EMAIL PROTECTED]> writes: > > > I am afraid that some of the boost libraries (as they are now) won't > > work for a big group of real time and embedded developers, where > > customization is the key. > > Is it really (other than culturally, I mean)? The culture is changing. :) > Are you sure that > there's no good general-purpose allocator which they can use? Yes, I am sure. I shipped one of my libraries to a guy who works on various DSP chips. The native memory allocator on one of the DSP was very slow (some specific issues with how the DSP addresses the memory, I don't remember the details). This guy had to add his own new/delete operator to some of my internal classes to get about 10 times performance improvement. No, I really don't think that there is a hope for general purpose allocators. Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: what happened to allocators in boost?
"E. Gladyshev" <[EMAIL PROTECTED]> writes: > I am afraid that some of the boost libraries (as they are now) won't > work for a big group of real time and embedded developers, where > customization is the key. Is it really (other than culturally, I mean)? Are you sure that there's no good general-purpose allocator which they can use? -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: what happened to allocators in boost?
"Peter Dimov" <[EMAIL PROTECTED]> writes: > E. Gladyshev wrote: >> --- Gregory Colvin <[EMAIL PROTECTED]> wrote: >>> For shared_ptr the count is allocated by the following line in the >>> shared_count >>> constructor: >>> >>> new sp_counted_base_impl(p, d); >>> >>> So it might be possible to make the allocation customizable by >>> specializing >>> sp_counted_base_impl. >> >> I think it would be great. >> >> However there is another problem. >> You have to new your object. >> shared_ptr s( new MyClass ); > > You can allocate your object however you want: > > shared_ptr s(MyClass::allocate(), MyClass::deallocate); > > and of course there is also the textbook way of defining a suitable > MyClass::operator new. That leaves out the question of count allocation. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: what happened to allocators in boost?
E. Gladyshev wrote: IMO, if boost is to become a C++ standard, we'd have to resolve this conflict. Well it is already somewhat standard, since a part of it has been accepted for the library TR (including the smart pointers). See: http://anubis.dkuug.dk/jtc1/sc22/wg21/docs/library_technical_report.html Regards, Dirk Gerrits ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
> > --- Peter Dimov <[EMAIL PROTECTED]> wrote:
> The danger in that statement is that it looks obvious, but it's not. It is
> true in some sotuations (not as frequent as is generally believed) and false
> in others, and when it doesn't really hold but is axiomatically accepted as
> truth, it serves as exactly the excuse I was talking about. "It is not
> practical to make a library that works out of the box, so users should be
> happy that they are getting a library that can be made to work with a
> "small" additional investment. Of course we'll gladly sell you the
> additional bits and pieces."
>
> It is interesting to compare the C++ approach with the Java approach to
> memory management in this context. C++ "pragmatically" accepts that the
> shipped malloc is often inadequate and provides several types of hooks that
> you can use to work around the problem. Java does not. If the JVM has a
> broken memory manager you lose and can do nothing about it. This gives C++ a
> short term advantage, but in the long term Java wins, since JVM implementors
> have no choice but to optimize their memory managers till they are blue in
> the face.
>
> Bugs that have no workarounds are always fixed first. :-)
It is an arguable position of course. I always thought that one of
the powers of a good C++ library is that it doesn't try to solve all
my problems but rather provide a convinient and efficient way to customize it.
I think that generic programming is all about that. In fact a generic
template library is compiled into a custom library for each application.
The developer provides data types and the library and C++ compiler do the rest.
The user data types are not hooks at all?
If you consider the memory as a data type and the memory management
as a policy then it is no different from any other template parameter in the library.
> So far my experience indicates that people only bother with allocators when
> std::allocator is inadequate, i.e. slow. Your case may, of course, be
> different. It happens sometimes.
It is not even an issue of an adequate malloc/free (they will never be
adequate for every case (another arguable statement :) ).
In one of my project, the standard malloc/free were quite adequate, but I had to use
a separate memory heap for some of my modules. Overloading
the global new/delete was not an option because the the rest of the code
need to be using the standard memory management. In this case,
the allocators proved to be very useful.
> A fairly aggressive way to state it, but I see your point. It is true that
> Boost does not impose any requirements on how the particular libraries
> obtain and manage their memory. But you need to realize that "do nothing"
> can be a conscious design decision. Lack of strict requirements means more
> freedom for the implementations to do what they consider best.
>
> As an example, consider a hypothetical std::vector that does not have an
> allocator argument and "has no idea how it manages its memory." Do you see
> the implications of that?
>
> 1. vector::resize and vector::reserve can now realloc in place since they
> can talk directly to the memory manager.
>
> 2. On many implementations, vector can now be made as efficient as new
> T[], since new T[] often stores the equivalent of size() and capacity()
> anyway to allow delete[] to do the right thing.
It is a problem with the STL allocator<> interface. An adequate allocator<>
interface should have realloc()/getsize() methods.
> > For example if a boost class X uses std::list, does it have to expose
> > the std::list<> allocator or should it use the standard
> > std::allocator<>.
> > template
> > struct X
> > {
> >std::list l;
> > };
> >
> > OR
> >
> > template >
> > struct X
> > {
> >std::list l;
> > };
> >
> > Which definition is more friendly to boost?
>
> The correct answer depends on X's interface. Your toy example is not a good
> illustration as this kind of "interface" is rarely seen.
How does it depend on the interface exactly? Is it possible to give
a good illustration?
> > Is it allowed for a boost libarary to use global new/delete? If so,
> > should it be documented?
>
> Boost libraries are allowed to use the C++ language without restrictions.
It means that I can never assume anything about memory calls in boost?
I understand your point about providing maximum flexibility to boost
developers but I am not sure it'll work well for some users.
Perhaps it'll be nice to the user to at least give her
some tips/notes on when a boost type makes use
of the standard memory manager.
The C++ developers have been spoiled by the STL allocator<> concept.
They may now be expecting too much... :)
Eugene
__
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com
___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
--- Peter Dimov <[EMAIL PROTECTED]> wrote:
> In this context, an allocator parameter is only an excuse for implementors
> to avoid doing the necessary work to make function<> useful out of the box.
> "You can always use a custom allocator", right?
Considering the variety of real life requirements and platforms, it is not practical
to make a
library that work out of the box. That is why STL has allocators. In one project, if
STL did not
have allocators, I would have to implement my own containers. I was so happy that I
didn't have
to do it.
At least a library should be consistent about memory management.
The issue is how consistent boost is about it (STL is very consistent)? It seems that
boost
doesn't have any idea about how it manages its memory.
For example if a boost class X uses std::list, does it have to expose the std::list<>
allocator or
should it use the standard std::allocator<>.
template
struct X
{
std::list l;
};
OR
template >
struct X
{
std::list l;
};
Which definition is more friendly to boost?
Is it allowed for a boost libarary to use global new/delete? If so, should it be
documented?
Eugene
__
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com
___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
Andreas Huber wrote: > > If function no longer has the allocator argument, how should we then > use function in a hard real-time environment? In this context, an allocator parameter is only an excuse for implementors to avoid doing the necessary work to make function<> useful out of the box. "You can always use a custom allocator", right? ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: what happened to allocators in boost?
Douglas Gregor wrote: > The allocator design focused on the benefits one could get from > specialized allocators for containers, e.g., data structures that may > allocate large chunks of memory that are expected to be used > together. They don't really give us much for components like > shared_ptr that allocate one thing (and > note that shared_ptr does not allocate the pointer it stores, > although it does allocate a reference count). boost::function > supports an allocator argument, but the C++ committee is considering > removing this allocator from the library TR version of function<> for > precisely this reason. If function no longer has the allocator argument, how should we then use function in a hard real-time environment? Does the committee assume that such systems have deterministic allocators or that users replace global new/delete anyway? Thanks & Regards, Andreas ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: what happened to allocators in boost?
[EMAIL PROTECTED] wrote: > > Hi there. I believe that components such as shared_ptr could still > benifit from allocators that are shared at the class level; for > example, a singleton pool allocator could be used by all instances of > the shared_ptr template. Loki uses exactly this, and it gains a > performance increase of ~100% > compared to using the standard ::operator new and ::operator delete > under MSVC++.NET 2003, when constructing and destructing lots of > shared_ptr objects (or to be more accurate, shared_count objects), > i.e.: This is a problem with MSVC's ::operator new, not with shared_ptr's interface. Many compilers already have an optimized small object allocator. Use #define BOOST_SP_USE_QUICK_ALLOCATOR if count allocation is a bottleneck, but keep in mind that this won't help the rest of "new X" allocations. > std::vector > myDoubleVector(1); This line doesn't allocate any counts, BTW. ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: what happened to allocators in boost?
- Original Message - From: "Douglas Gregor" <[EMAIL PROTECTED]> Newsgroups: gmane.comp.lib.boost.devel Sent: Tuesday, August 05, 2003 8:32 PM Subject: Re: what happened to allocators in boost? > The allocator design focused on the benefits one could get from specialized > allocators for containers, e.g., data structures that may allocate large > chunks of memory that are expected to be used together. They don't really > give us much for components like shared_ptr that allocate one thing (and > note that shared_ptr does not allocate the pointer it stores, although it > does allocate a reference count). boost::function supports an allocator > argument, but the C++ committee is considering removing this allocator from > the library TR version of function<> for precisely this reason. > > Doug > Hi there. I believe that components such as shared_ptr could still benifit from allocators that are shared at the class level; for example, a singleton pool allocator could be used by all instances of the shared_ptr template. Loki uses exactly this, and it gains a performance increase of ~100% compared to using the standard ::operator new and ::operator delete under MSVC++.NET 2003, when constructing and destructing lots of shared_ptr objects (or to be more accurate, shared_count objects), i.e.: std::vector > myDoubleVector(1); I personally think that the benifits are still there, and with things such as template typedefs and template aliases being introduced by the standards commitee, using templates with a lot of arguments may be less painful. Jesse Towner ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
