Fixed by holding a vector of GtfsRealTimeReader() pointers instead of the 
actual objects. Still no idea what causes it.

Am Donnerstag, 22. Mai 2014 20:56:22 UTC+2 schrieb 
patrick...@googlemail.com:
>
> Okay, I narrowed it down.
>
> I am creating a pool of GTFS real-time readers. For this, I created a 
> wrapper class GtfsRealTimeReader. GtfsRealTimeReader.h basically looks like 
> this (not including header guards and basic includes):
>
> // Reads GTFS realtime protocol buffer feeds
> class GtfsRealTimeReader {
>  public:
>   GtfsRealTimeReader() {}
>
>   // fetch new updates
>   void getTripUpdates();
> };
>
> GtfsRealTimeReader.cpp looks like this (in the test application):
>
> #include <gtfs-realtime.pb.h>
> #include "GtfsRealTimeReader.h"
> #include "easylogging.h"
>
> using std::string;
> using transit_realtime::FeedMessage;
> using transit_realtime::FeedEntity;
> using transit_realtime::TripUpdate_StopTimeEvent;
> using transit_realtime::TripDescriptor;
> using transit_realtime::TripUpdate_StopTimeUpdate_ScheduleRelationship;
> using transit_realtime::TripUpdate_StopTimeUpdate;
>
> void GtfsRealTimeReader::getTripUpdates() {
>   GOOGLE_PROTOBUF_VERIFY_VERSION;
>   FeedMessage fm;
>   LOG(INFO) << fm.GetDescriptor()->DebugString();
> }
>
> *The following code WORKS:*
>
>   GtfsRealTimeReader g();
>   g.getTripUpdates();
>
> *The following code WORKS:*
>
>   GtfsRealTimeReader g();
>   g.getTripUpdates();
>   GtfsRealTimeReader gg();
>   gg.getTripUpdates();
>   GtfsRealTimeReader ggg();
>   ggg.getTripUpdates();
>
> output is the DebugString() message.
>
> *The following code SEGFAULTS:*
>
>   vector<GtfsRealTimeReader> realTimeReaders;
>   realTimeReaders.push_back(GtfsRealTimeReader());
>   GtfsRealTimeReader g();
>   g.getTripUpdates();
>
> I can create an arbitrary number of GtfsRealTimeReader objects, they all 
> work fine. As soon as I push one of them onto a vector and call 
> getTripUpdates() on any of them, the segfault occurs.
>
>
> Am Donnerstag, 22. Mai 2014 20:07:43 UTC+2 schrieb Feng Xiao:
>>
>> Can you post the code of the simple program you used to reproduce the 
>> error? (including the .pb.h/.pb.cc or the .proto file)
>>
>> On Thu, May 22, 2014 at 10:58 AM, <patrick...@googlemail.com> wrote:
>>
>>> Thank your very much for your answer! The result of
>>>
>>>   GOOGLE_PROTOBUF_VERIFY_VERSION;
>>>   FeedMessage fm;
>>>   LOG(INFO) << fm.GetDescriptor()->DebugString();
>>>
>>> in both applications is
>>>
>>> ==13935== Invalid read of size 8
>>> ==13935==    at 0x510EFF2: 
>>> google::protobuf::DescriptorPool::FindFileByName(std::string const&) const 
>>> (in /usr/lib/x86_64-linux-gnu/libprotobuf.so.8.0.0)
>>> ==13935==    by 0x47F3F4: 
>>> transit_realtime::protobuf_AssignDesc_gtfs_2drealtime_2eproto() (in 
>>> /home/patrick/secure/repo/trajserver-internal2/build/trajhttpserv)
>>> ==13935==    by 0x50E22CF: 
>>> google::protobuf::internal::FunctionClosure0::Run() (in 
>>> /usr/lib/x86_64-linux-gnu/libprotobuf.so.8.0.0)
>>> ==13935==    by 0x50E24F0: google::protobuf::GoogleOnceInitImpl(long*, 
>>> google::protobuf::Closure*) (in 
>>> /usr/lib/x86_64-linux-gnu/libprotobuf.so.8.0.0)
>>> ==13935==    by 0x474464: transit_realtime::FeedMessage::GetMetadata() 
>>> const (in /home/patrick/secure/repo/trajserver-internal2/build/trajhttpserv)
>>> ==13935==    by 0x442D40: GtfsRealTimeReader::getTripUpdates() (in 
>>> /home/patrick/secure/repo/trajserver-internal2/build/trajhttpserv)
>>> ==13935==    by 0x4824AB: 
>>> updateRealtimeReaders(std::vector<GtfsRealTimeReader, 
>>> std::allocator<GtfsRealTimeReader> >*, unsigned int) (in 
>>> /home/patrick/secure/repo/trajserver-internal2/build/trajhttpserv)
>>> ==13935==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
>>> ==13935== 
>>> [22/05/2014 19:55:47,646579] FATAL: CRASH HANDLED; Application has 
>>> crashed due to [SIGSEGV] signal
>>>
>>>
>>> Am Donnerstag, 22. Mai 2014 19:47:21 UTC+2 schrieb Feng Xiao:
>>>>
>>>> Can you try if the following code can produce any meaningful results? I 
>>>> suggest you try this once in your original binary and then create a simple 
>>>> program calling the following code only (and only linking the .proto file 
>>>> without anything else in your project). If both fail (segfault), it might 
>>>> be a bug in protobuf generated code. If only the former fails, the problem 
>>>> should be somewhere else in your project (memory corruption bugs most 
>>>> likely).
>>>>
>>>> LOG(INFO) << fm.GetDescriptor()->DebugString();
>>>>
>>>> Note that you shouldn't change the generated code. It won't work.
>>>>
>>>>
>>>>  On Thu, May 22, 2014 at 9:28 AM, <patrick...@googlemail.com> wrote:
>>>>
>>>>>  I am still experiencing this problem. Any help is highly 
>>>>> appreciated, I tried everything.
>>>>>
>>>>> The problem only occurs if bad data is fed to the parseFromString() 
>>>>> function. The following code crashes with a segmentation fault if 
>>>>> readBuffer contains bad data (currently, the content of 
>>>>> http://www.google.com). If readBuffer contains well-formed 
>>>>> GTFS-realtime data, everything works as expected:
>>>>>
>>>>>   if (!(fm.ParseFromString(readBuffer))) {
>>>>>     LOG(WARNING) << "Failed to parse realtime GTFS.";
>>>>>     return;
>>>>>   }
>>>>>
>>>>> This is the line in the gtfs-realtime.pb.cc (in method 
>>>>> protobuf_AssignDesc_headers_2fprotobuf_2fgtfs_2drealtime_2eproto()) 
>>>>> that causes the crash:
>>>>>
>>>>>   const ::google::protobuf::FileDescriptor* file =
>>>>>     ::google::protobuf::DescriptorPool::generated_
>>>>> pool()->FindFileByName(
>>>>>       "headers/protobuf/gtfs-realtime.proto");
>>>>>
>>>>> For some reason, it tries to read the original .proto file if bad data 
>>>>> is received. I tried changing this line to an absolute path, but the 
>>>>> problem persists. The gtfs-realtime.proto file resides in 
>>>>> headers/protobuf. 
>>>>> protoc is called like this:
>>>>>
>>>>> protoc -I=headers/protobuf --cpp_out=. headers/protobuf/gtfs-
>>>>> realtime.proto 
>>>>>
>>>>> I tried several locations for -cpp_out, but it didnt help.
>>>>>
>>>>> This is the valgrind output after the crash:
>>>>>
>>>>> ==4015== Invalid read of size 8
>>>>> ==4015==    at 0x5108FF2: 
>>>>> google::protobuf::DescriptorPool::FindFileByName(std::string 
>>>>> const&) const (in /usr/lib/x86_64-linux-gnu/libprotobuf.so.8.0.0)
>>>>> ==4015==    by 0x47F3F4: transit_realtime::protobuf_
>>>>> AssignDesc_gtfs_2drealtime_2eproto() (in /home/patrick/repos/
>>>>> trajserver/trajhttpserv)
>>>>> ==4015==    by 0x50DC2CF: 
>>>>> google::protobuf::internal::FunctionClosure0::Run() 
>>>>> (in /usr/lib/x86_64-linux-gnu/libprotobuf.so.8.0.0)
>>>>> ==4015==    by 0x50DC4F0: google::protobuf::GoogleOnceInitImpl(long*, 
>>>>> google::protobuf::Closure*) (in /usr/lib/x86_64-linux-gnu/
>>>>> libprotobuf.so.8.0.0)
>>>>> ==4015==    by 0x474444: transit_realtime::FeedMessage::GetMetadata() 
>>>>> const (in /home/patrick/repos/trajserver/trajhttpserv)
>>>>> ==4015==    by 0x513FF6F: google::protobuf::Message::GetTypeName() 
>>>>> const (in /usr/lib/x86_64-linux-gnu/libprotobuf.so.8.0.0)
>>>>> ==4015==    by 0x50EB090: ??? (in /usr/lib/x86_64-linux-gnu/
>>>>> libprotobuf.so.8.0.0)
>>>>> ==4015==    by 0x50EB891: 
>>>>> google::protobuf::MessageLite::ParseFromString(std::string 
>>>>> const&) (in /usr/lib/x86_64-linux-gnu/libprotobuf.so.8.0.0)
>>>>> ==4015==    by 0x442E83: GtfsRealTimeReader::getTripUpdates() (in 
>>>>> /home/patrick/repos/trajserver/trajhttpserv)
>>>>> ==4015==    by 0x4824AB: 
>>>>> updateRealtimeReaders(std::vector<GtfsRealTimeReader, 
>>>>> std::allocator<GtfsRealTimeReader> >*, unsigned int) (in 
>>>>> /home/patrick/repos/trajserver/trajhttpserv)
>>>>> ==4015==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
>>>>> ==4015== 
>>>>> [22/05/2014 15:21:02,241645] FATAL: CRASH HANDLED; Application has 
>>>>> crashed due to [SIGSEGV] signal
>>>>>
>>>>>
>>>>> Again, this crash happens every time I try to parse some random string 
>>>>> as a GTFS-realtime protobuf feed.
>>>>>
>>>>> I reproduced the segfault on three different machines: a Ubuntu 13.10 
>>>>> machine with libprotoc 2.4.1, a Ubuntu 14.04 machine with libprotoc 2.5.0 
>>>>> and a Debian machine with libprotoc 2.4.1. I tried different c++ version, 
>>>>> Ii tried -O1, -O2, -O3, I tried fixing the gtfs-realtime.pb.cc line 
>>>>> by hand, I played around with the protoc parameters, I compiled and 
>>>>> installed libprotoc 2.5.0 by hand on all 3 machines, I tried parsing the 
>>>>> feed from a stream and from a string, I created an application that only 
>>>>> reads http://www.google.com and feeds it to parseFromString() 
>>>>> function - the behaviour is always exactly the same: no problem on 
>>>>> well-formed data, segfault on bad data.
>>>>>
>>>>> I am using the original gtfs-realtime.proto file from the GTFS 
>>>>> realtime page.
>>>>>
>>>>> Anyone?
>>>>>
>>>>>
>>>>> Am Freitag, 2. Mai 2014 15:29:48 UTC+2 schrieb 
>>>>> patrick...@googlemail.com:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am using a straightforward C++ client to fetch GTFS-realtime feeds. 
>>>>>> Feeds are read into a string with libcurl and than parsed:
>>>>>>
>>>>>> GOOGLE_PROTOBUF_VERIFY_VERSION; 
>>>>>> FeedMessage fm;
>>>>>>
>>>>>> /** fetching realtime feed with libcurl into readBuffer **/
>>>>>>
>>>>>> if (!fm.ParseFromString(readBuffer)) {  // this line crashes
>>>>>>   LOG(ERROR) << "Failed to parse realtime GTFS.";
>>>>>>   return;
>>>>>> }
>>>>>>
>>>>>> I experience random segfault crashes during parsing. All of them 
>>>>>> oocure while parsing a very specific feed. The segfault is caused by
>>>>>>
>>>>>> google::protobuf::DescriptorPool::FindFileByName
>>>>>>
>>>>>> Backtrace:
>>>>>>
>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>> #0  0x00007ffff7901f3e in 
>>>>>> google::protobuf::DescriptorPool::FindFileByName(std::string 
>>>>>> const&) const () from /usr/lib/libprotobuf.so.7
>>>>>> #1  0x000000000047e045 in transit_realtime::protobuf_Ass
>>>>>> ignDesc_headers_2fprotobuf_2fgtfs_2drealtime_2eproto() () at 
>>>>>> headers/protobuf/gtfs-realtime.pb.cc:80
>>>>>> #2  0x00007ffff7683400 in pthread_once ()
>>>>>>     at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S:104
>>>>>> #3  0x0000000000471593 in transit_realtime::FeedMessage::GetMetadata() 
>>>>>> const
>>>>>>     () at /usr/include/google/protobuf/stubs/once.h:115
>>>>>> #4  0x00007ffff7932274 in google::protobuf::Message::GetTypeName() 
>>>>>> const ()
>>>>>>    from /usr/lib/libprotobuf.so.7
>>>>>> #5  0x00007ffff78e2f17 in ?? () from /usr/lib/libprotobuf.so.7
>>>>>> #6  0x00007ffff78e3714 in 
>>>>>> google::protobuf::MessageLite::ParseFromString(std::string 
>>>>>> const&) () from /usr/lib/libprotobuf.so.7
>>>>>>
>>>>>>
>>>>>> Since the segfaults are _always_ caused by the same feed, I strongly 
>>>>>> suspect that it is bad data that causes the crash. The random nature of 
>>>>>> the 
>>>>>> crashes could be explained by the feed returning a bad message every few 
>>>>>> few hours. Nevertheless, for bad data, ParseFromString() should return 
>>>>>> false, and not crash completely.
>>>>>>
>>>>>> I am using g++ 4.8.1 with -O3 -g -std=c++0x
>>>>>> protoc --version returns "libprotoc 2.4.1"
>>>>>>
>>>>>> Thanks for any help :) 
>>>>>>
>>>>>>
>>>>>>
>>>>>>  
>>>>>>
>>>>>>
>>>>>>  -- 
>>>>> You received this message because you are subscribed to the Google 
>>>>> Groups "Protocol Buffers" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>>> an email to protobuf+u...@googlegroups.com.
>>>>> To post to this group, send email to prot...@googlegroups.com.
>>>>>
>>>>> Visit this group at http://groups.google.com/group/protobuf.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>>  -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Protocol Buffers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to protobuf+u...@googlegroups.com.
>>> To post to this group, send email to prot...@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/protobuf.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Reply via email to