[protobuf] Re: gcc4.4 fixes: std::tr1::unordered_map
Oh, funky. Sorry, I don't have a Mac to test. Do you have automated builds going? From my experience gcc 4.0.x versions were somewhat buggy, while the 4.1.x branch is reasonable. I am surprised that std::tr1 was actually present in that STL... Perhaps it is an Apple-brewed combo? Oleg. On Wed, Jan 6, 2010 at 8:44 PM, Kenton Varda ken...@google.com wrote: Worked around with r291. Must test on all platforms all over again... sigh. On Wed, Jan 6, 2010 at 8:29 PM, Kenton Varda ken...@google.com wrote: The implementation of tr1::hashtable on OSX 1.5 (GCC 4.0.1) is broken. find_node() is apparently not declared const, meaning calling find() on a const hash_map does not compile. /cry On Mon, Dec 21, 2009 at 11:24 AM, Kenton Varda ken...@google.com wrote: Arrghh. I didn't mean to add that... I just wrote it so that I could hit F3 and have eclipse show me the file, then forgot to delete it. Fixed. Thanks for pointing that out; I'm not normally so sloppy. On Mon, Dec 21, 2009 at 11:19 AM, Oleg Smolsky oleg.smol...@gmail.com wrote: BTW, you've added this line to stubs/hash.h #include ext/hash_map but it should not be there. Includes are already handled by these: #include HASH_MAP_H #include HASH_SET_H Oleg. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: gcc4.4 fixes: std::tr1::unordered_map
Alright, thanks.
Oleg.
On Dec 21, 10:59 am, Kenton Varda ken...@google.com wrote:
Yes, I see now. But your solution -- constructing string objects every time
the hash function is run -- is very slow. I've submitted r259 with this
implementation instead:
template
struct hashconst char* {
inline size_t operator()(const char* str) const {
size_t result = 0;
for (; *str != '\0'; str++) {
result = 5 * result + *str;
}
return result;
}
};
On Mon, Dec 21, 2009 at 10:44 AM, Oleg Smolsky oleg.smol...@gmail.comwrote:
Hey Kenton, you also need the two updated google::protobuf::hash
specializations that ensure correct hashing of const char * and
std::string keys.
I've attached the complete header, so that you can read it without
having to apply the patch.
Oleg.
On Mon, Dec 21, 2009 at 10:26 AM, Kenton Varda ken...@google.com wrote:
Thanks for the fix! Unfortunately your re-ordering makes it hard for me
to
see what actually changed. Also, your style doesn't match the Google
style
guide.
Is the following sufficient for hash.h, given your changes to
stl_hash.m4?
Index: src/google/protobuf/stubs/hash.h
===
--- src/google/protobuf/stubs/hash.h (revision 258)
+++ src/google/protobuf/stubs/hash.h (working copy)
@@ -152,14 +152,14 @@
template typename Key, typename Data,
typename HashFcn = hashKey,
typename EqualKey = std::equal_toKey
-class hash_map : public HASH_NAMESPACE::hash_map
+class hash_map : public HASH_NAMESPACE::HASH_MAP_CLASS
Key, Data, HashFcn, EqualKey {
};
template typename Key,
typename HashFcn = hashKey,
typename EqualKey = std::equal_toKey
-class hash_set : public HASH_NAMESPACE::hash_set
+class hash_set : public HASH_NAMESPACE::HASH_SET_CLASS
Key, HashFcn, EqualKey {
};
On Wed, Dec 16, 2009 at 9:36 PM, Oleg Smolsky oleg.smol...@gmail.com
wrote:
Hey Kenton, attached is a patch that clears deprecated headers
warnings emitted when building protobuf with g++4.4.
The fix has two parts:
a) discover and use std::tr1::unordered_map when it is available
b) ensure that string hashing is available and working
I've tested the updated code with g++4.4, g++4.1 and g++3.4.
Unfortunately the last two share the same old crusty libstdc++ due to
the way Redhad built it...
P.S. I've taken a liberty to reformat and re-order declarations in
.../stubs/hash.h while debugging the hash issue.
Oleg.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Link errors with protobuf 2.2.0a
On Dec 17, 12:27 pm, Kenton Varda ken...@google.com wrote: On Thu, Dec 17, 2009 at 2:47 AM, a...@sibmail.ru wrote: Hello, You should use same runtime library setting to build your application and protobuffer libraries. Specifically, debug vs. release, single-threaded vs. multi-threaded, etc. Unfortunately, MSVC's STL implementation for each of these library implementations is different. VS2008 does not have a single-threaded CRT any more, but configuring debug/release and static/dynamic linking in a consistent way is vital. Oleg. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: Memory leaks on VS2008/C++
I cannot get stack traces with VS. I can add a macro to every
compilation unit that wraps new so that __FILE__ and __LINE__ are
recorded. It's quite a bit of work to re-arrange your header files,
though. Would that suffice? Alternatively, do you happen to have a 3rd
party tracker like Bounds Checker?
Oleg.
On 2009/5/15 15:58, Kenton Varda wrote:
Can you get stack traces for these?
I ran the whole protobuf test suite with the google-perftools leak
checker and didn't detect any leaks.
On Thu, May 14, 2009 at 4:05 PM, Oleg Smolsky o...@smolsky.net
mailto:o...@smolsky.net wrote:
Hi Kenton, thank you for letting me know. I've just got and built
this version of protoc/libprotobuf and run my unit test - there
are still a few leaks according to VS2008. So, there is a
difference between v2.1.0 and the destructor-based patch that I
had cleaned up for v2.0.3.
The output is below. Do you happen to recognize the 24 byte
structure that starts with a pointer and then contains 0x? :)
Kind regards,
Oleg.
Detected memory leaks!
Dumping objects -
{1438} normal block at 0x00D72620, 24 bytes long.
Data: h ( 68 A6 28 00 FF FF FF FF 00 00 00 00 00 00 00 00
{1437} normal block at 0x00D725C8, 24 bytes long.
Data: 0 ( 30 A6 28 00 FF FF FF FF 00 00 00 00 00 00 00 00
{946} normal block at 0x00246AB0, 24 bytes long.
Data: ( A0 A1 28 00 FF FF FF FF 00 00 00 00 00 00 00 00
{945} normal block at 0x00249CD0, 24 bytes long.
Data: h ( 68 A1 28 00 FF FF FF FF 00 00 00 00 00 00 00 00
{934} normal block at 0x00249840, 24 bytes long.
Data: 0 ( 30 A1 28 00 FF FF FF FF 00 00 00 00 00 00 00 00
{812} normal block at 0x002469B0, 24 bytes long.
Data: x ( 78 9D 28 00 FF FF FF FF 00 00 00 00 00 00 00 00
{685} normal block at 0x00D71FB0, 24 bytes long.
Data: ( D0 9C 28 00 FF FF FF FF 00 00 00 00 00 00 00 00
{667} normal block at 0x00D71608, 24 bytes long.
Data: ` ( 60 9C 28 00 FF FF FF FF 00 00 00 00 00 00 00 00
{666} normal block at 0x00D715B0, 24 bytes long.
Data: ( ( 28 9C 28 00 FF FF FF FF 00 00 00 00 00 00 00 00
{663} normal block at 0x00D713D8, 24 bytes long.
Data: Pu( 50 75 28 00 FF FF FF FF 00 00 00 00 00 00 00 00
{148} normal block at 0x00245820, 24 bytes long.
Data: pt( 70 74 28 00 FF FF FF FF 00 00 00 00 00 00 00 00
{147} normal block at 0x002457C8, 24 bytes long.
Data: 8t( 38 74 28 00 FF FF FF FF 00 00 00 00 00 00 00 00
{136} normal block at 0x00245290, 24 bytes long.
Data: t( 00 74 28 00 FF FF FF FF 00 00 00 00 00 00 00 00
{135} normal block at 0x00245238, 24 bytes long.
Data: s( C8 73 28 00 FF FF FF FF 00 00 00 00 00 00 00 00
Object dump complete.
On 2009/5/14 13:08, Kenton Varda wrote:
FYI: This is fixed in the 2.1.0 release (you can call
google::protobuf::ShutdownProtobufLibrary() to clean up all leaks).
On Fri, Mar 27, 2009 at 7:07 PM, Kenton Varda ken...@google.com
mailto:ken...@google.com wrote:
On Fri, Mar 27, 2009 at 4:30 PM, Oleg Smolsky
o...@smolsky.net mailto:o...@smolsky.net wrote:
So, the next question is - would it ever make its way
into your SVN? If so, do you see a need for a switch of
some kind?
Yeah, reviewing and committing this is on my todo list for
the next release.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Memory leaks on VS2008/C++
Actually, I got lucky - the culprit was not in headers. It's this line
in once.cc:
if (internal_ == NULL) internal_ = new GoogleOnceInternal;
The log is as follows:
Detected memory leaks!
Dumping objects -
..\src\google\protobuf\stubs\once.cc(65) : {1438} client block at
0x00B92620, subtype 0, 24 bytes long.
Data: h - 68 A6 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {1437} client block at
0x00B925C8, subtype 0, 24 bytes long.
Data: 0 - 30 A6 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {946} client block at
0x00246AB0, subtype 0, 24 bytes long.
Data: - A0 A1 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {945} client block at
0x00249CD0, subtype 0, 24 bytes long.
Data: h - 68 A1 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {934} client block at
0x00249840, subtype 0, 24 bytes long.
Data: 0 - 30 A1 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {812} client block at
0x002469B0, subtype 0, 24 bytes long.
Data: x - 78 9D 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(73) : {685} client block at
0x00B91FB0, subtype 0, 24 bytes long.
Data: - D0 9C 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(73) : {667} client block at
0x00B91608, subtype 0, 24 bytes long.
Data: ` - 60 9C 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {666} client block at
0x00B915B0, subtype 0, 24 bytes long.
Data: ( - 28 9C 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {663} client block at
0x00B913D8, subtype 0, 24 bytes long.
Data: Pu- 50 75 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {148} client block at
0x00245820, subtype 0, 24 bytes long.
Data: pt- 70 74 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {147} client block at
0x002457C8, subtype 0, 24 bytes long.
Data: 8t- 38 74 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {136} client block at
0x00245290, subtype 0, 24 bytes long.
Data: t- 00 74 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {135} client block at
0x00245238, subtype 0, 24 bytes long.
Data: s- C8 73 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
Object dump complete.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Memory leaks on VS2008/C++
...sent the message too quickly.
So, the fix is trivial, yet it comes down to the same very discussion as
the original issue. In this case it's cleaning up after the cleaners :)
It's a global pointer that is initialized when global constructors
(a.k.a. static initializers) run. The pointer to the heap block is never
lost, but it is never deallocated either. The fix is trivial - all that
is needed is the destructor. This, of course, assumes that the main
thread would exit only after all additional workers are dead.
Oleg.
On 2009/5/15 17:28, Oleg Smolsky wrote:
Actually, I got lucky - the culprit was not in headers. It's this line
in once.cc:
if (internal_ == NULL) internal_ = new GoogleOnceInternal;
The log is as follows:
Detected memory leaks!
Dumping objects -
..\src\google\protobuf\stubs\once.cc(65) : {1438} client block at
0x00B92620, subtype 0, 24 bytes long.
Data: h - 68 A6 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {1437} client block at
0x00B925C8, subtype 0, 24 bytes long.
Data: 0 - 30 A6 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {946} client block at
0x00246AB0, subtype 0, 24 bytes long.
Data: - A0 A1 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {945} client block at
0x00249CD0, subtype 0, 24 bytes long.
Data: h - 68 A1 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {934} client block at
0x00249840, subtype 0, 24 bytes long.
Data: 0 - 30 A1 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {812} client block at
0x002469B0, subtype 0, 24 bytes long.
Data: x - 78 9D 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(73) : {685} client block at
0x00B91FB0, subtype 0, 24 bytes long.
Data: - D0 9C 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(73) : {667} client block at
0x00B91608, subtype 0, 24 bytes long.
Data: ` - 60 9C 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {666} client block at
0x00B915B0, subtype 0, 24 bytes long.
Data: ( - 28 9C 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {663} client block at
0x00B913D8, subtype 0, 24 bytes long.
Data: Pu- 50 75 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {148} client block at
0x00245820, subtype 0, 24 bytes long.
Data: pt- 70 74 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {147} client block at
0x002457C8, subtype 0, 24 bytes long.
Data: 8t- 38 74 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {136} client block at
0x00245290, subtype 0, 24 bytes long.
Data: t- 00 74 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {135} client block at
0x00245238, subtype 0, 24 bytes long.
Data: s- C8 73 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
Object dump complete.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Memory leaks on VS2008/C++
Here we go.
On 2009/5/15 18:56, Kenton Varda wrote:
Ugh, I missed that one because I was doing my testing on Linux and
that is in Windows-only code.
Ideally I'd prefer if each GoogleOnce object registered a shutdown
function for itself. But I suppose that could create a deadlock
because the shutdown function registry is once-initialized. So, sure,
use destructors. Do you want to send me a patch since you're set up
to test it?
On Fri, May 15, 2009 at 5:41 PM, Oleg Smolsky o...@smolsky.net
mailto:o...@smolsky.net wrote:
...sent the message too quickly.
So, the fix is trivial, yet it comes down to the same very
discussion as the original issue. In this case it's cleaning up
after the cleaners :) It's a global pointer that is initialized
when global constructors (a.k.a. static initializers) run. The
pointer to the heap block is never lost, but it is never
deallocated either. The fix is trivial - all that is needed is the
destructor. This, of course, assumes that the main thread would
exit only after all additional workers are dead.
Oleg.
On 2009/5/15 17:28, Oleg Smolsky wrote:
Actually, I got lucky - the culprit was not in headers. It's
this line in once.cc:
if (internal_ == NULL) internal_ = new GoogleOnceInternal;
The log is as follows:
Detected memory leaks!
Dumping objects -
..\src\google\protobuf\stubs\once.cc(65) : {1438} client block
at 0x00B92620, subtype 0, 24 bytes long.
Data: h - 68 A6 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {1437} client block
at 0x00B925C8, subtype 0, 24 bytes long.
Data: 0 - 30 A6 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {946} client block
at 0x00246AB0, subtype 0, 24 bytes long.
Data: - A0 A1 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {945} client block
at 0x00249CD0, subtype 0, 24 bytes long.
Data: h - 68 A1 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {934} client block
at 0x00249840, subtype 0, 24 bytes long.
Data: 0 - 30 A1 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {812} client block
at 0x002469B0, subtype 0, 24 bytes long.
Data: x - 78 9D 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(73) : {685} client block
at 0x00B91FB0, subtype 0, 24 bytes long.
Data: - D0 9C 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(73) : {667} client block
at 0x00B91608, subtype 0, 24 bytes long.
Data: ` - 60 9C 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {666} client block
at 0x00B915B0, subtype 0, 24 bytes long.
Data: ( - 28 9C 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {663} client block
at 0x00B913D8, subtype 0, 24 bytes long.
Data: Pu- 50 75 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {148} client block
at 0x00245820, subtype 0, 24 bytes long.
Data: pt- 70 74 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {147} client block
at 0x002457C8, subtype 0, 24 bytes long.
Data: 8t- 38 74 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {136} client block
at 0x00245290, subtype 0, 24 bytes long.
Data: t- 00 74 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
..\src\google\protobuf\stubs\once.cc(65) : {135} client block
at 0x00245238, subtype 0, 24 bytes long.
Data: s- C8 73 2D 00 FF FF FF FF 00 00 00 00 00 00 00 00
Object dump complete.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
diff -aru protobuf-2.1.0/src/google/protobuf/stubs/once.cc protobuf-2.1.0 -
Copy/src/google/protobuf/stubs/once.cc
--- protobuf-2.1.0/src/google/protobuf/stubs/once.cc2009-05-13
13:36:36.0 -0700
+++ protobuf-2.1.0 - Copy/src/google/protobuf/stubs/once.cc 2009-05-15
19:18:09.405418300 -0700
@@ -56,6 +56,12 @@
CRITICAL_SECTION critical_section;
};
+GoogleOnceType::~GoogleOnceType
Re: Memory leaks on VS2008/C++
On 3/27/2009 3:19 PM, Oleg Smolsky wrote: Weird... DescriptorPool is destroyed, and so it DescriptorPool::Tables which frees its 52 outstanding allocations... OK, I made a silly mistake - I was dumping allocations before static destructors were executed. Oops. I've just verified the test and the code - the merged/updated patch that I posted at the beginning of the thread does address all leaks. So, the next question is - would it ever make its way into your SVN? If so, do you see a need for a switch of some kind? Kind regards, Oleg. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
