[grpc-io] grpc application crashes in protobuf init after dlopen()

2019-01-06 Thread MMMike 
( linux C++ grpc protobuf static library init crash ) (keywords)

My application/service has a grpc API, so it is also a grpc service.
It has plugins, one of which is a grpc client of another service.
By plugin I mean features that can be loaded when needed (maybe hours 
later) by loading the shared library with dlopen().
The plugin and my application both use the same version/build of grpc.

When the plugin is loaded via dlopen(), the application crashes, apparently 
re-initializing the protobuf.
See the stack below.

When I make grpc on linux, the build includes libprotobuf.a which we 
install with the application.
I posit the re-init could be solved if the application and the plugin 
didn't both link with libprotobuf.a
Is this correct?  Is there a better way for me to do this?  
Is there a better explanation for the stack below? 

I think if the grpc build created libprotobuf.so instead of libprotobuf.a, 
and everything used a single instance of libprotobuf.so, the re-init 
wouldn't be a problem.
If necessary, the application startup could dlopen("libprotobuf.so", 
RTLD_NOW|RTLD_NODELETE) libprotobuf.so with flags to never unload the 
shared lib. 
If it is never unloaded I hope the re-init wouldn't happen.

Grpc source includes protobuf source via git submodules
I tried modifying grpc's Makefile to produce libprotobuf.so.
The grpc Makefile builds protobuf using: ./configure --disable-shared 
--enable-static $(PROTOBUF_CONFIG_OPTS)
The make fails if I try --enable-shared
Flags --enable-shared and --disable-static are common but don't seem to be 
fully implemented in protobuf.
I don't know how to convince protobuf to build libprotobuf.so for me.

Question 1: Am I on the right track with the cause of the crash? How do 
people prevent grpc's protobuf from crashing in one-time init code?

Question 2: How do I replace libprotobuf.a with libprotobuf.so in the grpc 
make or in the protobuf automake/configure? (assuming question 1 is on the 
right track.)  

Help I need advice. Please and thank you.


(gdb) bt
#0  0x7fc4af68e875 in raise () from /lib64/libc.so.6
#1  0x7fc4af68fe51 in abort () from /lib64/libc.so.6
#2  0x7fc4b00ec325 in __gnu_cxx::__verbose_terminate_handler() ()
   from /usr/lib64/libstdc++.so.6
#3  0x7fc4b00ea4c6 in ?? () from /usr/lib64/libstdc++.so.6
#4  0x7fc4b00ea4f3 in std::terminate() () from /usr/lib64/libstdc++.so.6
#5  0x7fc4b00ea712 in __cxa_throw () from /usr/lib64/libstdc++.so.6
#6  0x7fc4b44c70d8 in google::protobuf::internal::LogMessage::Finish 
(this=0x7fc49a7f46d0)
at google/protobuf/stubs/common.cc:269
#7  0x7fc4b44d992a in 
google::protobuf::DescriptorPool::InternalAddGeneratedFile (
encoded_file_descriptor=0x7fc49c755de0 
, size=489) at 
google/protobuf/descriptor.cc:1394
#8  0x7fc49c684cfc in protobuf_ghost_2eproto::AddDescriptorsImpl ()
at 
/tcwork/swbuilder/7182982361a3dd28/cmic_core/build/src/Grpc/ghostGrpc/ghost.pb.cc:155
#9  0x7fc4b44c7f90 in google::protobuf::internal::FunctionClosure0::Run 
(this=0x7fc49a7f4760)
at ./google/protobuf/stubs/callback.h:129
#10 0x7fc4b44c9491 in google::protobuf::GoogleOnceInitImpl (
once=0x7fc49c9a90e0 , 
closure=)
at google/protobuf/stubs/once.cc:83
#11 0x7fc4b44b6f6b in google::protobuf::GoogleOnceInit (
once=0x7fc49c9a90e0 ,
init_func=0x7fc49c684ce2 )
at 
/var/opt/teradata/swrepository/local/SM_SWREPOSITORY.SLES_11_SP3-devel/grpc-1.11.0-0.x86_64/usr/include/google/protobuf/stubs/once.h:128
#12 0x7fc49c684d2e in protobuf_ghost_2eproto::AddDescriptors ()
at 
/tcwork/swbuilder/7182982361a3dd28/cmic_core/build/src/Grpc/ghostGrpc/ghost.pb.cc:162
#13 0x7fc49c68ac67 in 
protobuf_ghost_2eproto::StaticDescriptorInitializer::StaticDescriptorInitializer
 
(this=0x7fc49c9a90c0 
)
at 
/tcwork/swbuilder/7182982361a3dd28/cmic_core/build/src/Grpc/ghostGrpc/ghost.pb.cc:167
#14 0x7fc49c6893cf in __static_initialization_and_destruction_0 
(__initialize_p=1,
__priority=65535)
at 
/tcwork/swbuilder/7182982361a3dd28/cmic_core/build/src/Grpc/ghostGrpc/ghost.pb.cc:169
#15 0x7fc49c6893e4 in _GLOBAL__sub_I_ghost.pb.cc(void) ()
at 
/tcwork/swbuilder/7182982361a3dd28/cmic_core/build/src/Grpc/ghostGrpc/ghost.pb.cc:1472
#16 0x7fc49c755756 in __do_global_ctors_aux () from 
/opt/teradata/cmic/lib/libghostGrpc.so
#17 0x7fc49c67c28b in _init () from 
/opt/teradata/cmic/lib/libghostGrpc.so
#18 0x7fc49c9ef298 in ?? () from 
/opt/teradata/cmic/lib/libMEPlugin_Disk_Array_SDHK.so
#19 0x7fc4b6f7f268 in call_init () from /lib64/ld-linux-x86-64.so.2
#20 0x7fc4b6f7f397 in _dl_init_internal () from 
/lib64/ld-linux-x86-64.so.2
#21 0x7fc4b6f836b6 in dl_open_worker () from /lib64/ld-linux-x86-64.so.2
#22 0x7fc4b6f7eef6 in _dl_catch_error () from 
/lib64/ld-linux-x86-64.so.2
#23 0x7fc4b6f82eab in _dl_open () from /lib64/ld-linux-x86-64.so.2
#24 0x7fc4b071ef9b in dlopen_doit () from /lib64/libdl.so.2
#25 0x7fc4b6f7eef6 in

Re: [grpc-io] Proposal: descriptor (.pb) to .proto files

2019-01-06 Thread Derek Perez 
For Java, I wrote a library for generating protobuf source files using a
builder style pattern here:
GitHub.com/perezd/protopoet

On Sun, Jan 6, 2019, 5:39 AM Josh Humphries  wrote:

> FWIW, I have a Go implementation of the same functionality here:
> https://godoc.org/github.com/jhump/protoreflect/desc/protoprint
>
> 
> *Josh Humphries*
> jh...@bluegosling.com
>
>
> On Sun, Jan 6, 2019 at 7:48 AM Alex Van Boxel 
> wrote:
>
>> Hi,
>>
>> I'm currently prototyping a Descriptor to .proto files dumper. I'm
>> wondering if it would be something that would be of interest to include in
>> the java-utils part of *grpc-java*?
>>
>> We're planning to use it for dynamically generated schema's, that we then
>> dump on a filesystem to check into git.
>>
>> (it's work in progress, but already generates pretty complete proto files)
>>
>> https://github.com/anemos-io/metastore/blob/master/server/src/main/java/io/anemos/metastore/ProtoFileWriter.java
>>
>> If this is something that could be included I'll do the effort of getting
>> an environment up that is able to build grpc-java.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "grpc.io" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to grpc-io+unsubscr...@googlegroups.com.
>> To post to this group, send email to grpc-io@googlegroups.com.
>> Visit this group at https://groups.google.com/group/grpc-io.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/grpc-io/2a4fdf8d-b5f6-40cd-bb00-a99533ae4e08%40googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups "
> grpc.io" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to grpc-io+unsubscr...@googlegroups.com.
> To post to this group, send email to grpc-io@googlegroups.com.
> Visit this group at https://groups.google.com/group/grpc-io.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/grpc-io/CAO78j%2BJHpHrjPv%2BRqRhh8wa4VVHqC1AT30NYeJ2F0uhBK023cQ%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/CAD7-yfv5j5ZX%3D5dCJkLdk23-Df5aXQq0U4vvu6hfCFthB3YikA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [grpc-io] Proposal: descriptor (.pb) to .proto files

2019-01-06 Thread Josh Humphries 
FWIW, I have a Go implementation of the same functionality here:
https://godoc.org/github.com/jhump/protoreflect/desc/protoprint


*Josh Humphries*
jh...@bluegosling.com


On Sun, Jan 6, 2019 at 7:48 AM Alex Van Boxel 
wrote:

> Hi,
>
> I'm currently prototyping a Descriptor to .proto files dumper. I'm
> wondering if it would be something that would be of interest to include in
> the java-utils part of *grpc-java*?
>
> We're planning to use it for dynamically generated schema's, that we then
> dump on a filesystem to check into git.
>
> (it's work in progress, but already generates pretty complete proto files)
>
> https://github.com/anemos-io/metastore/blob/master/server/src/main/java/io/anemos/metastore/ProtoFileWriter.java
>
> If this is something that could be included I'll do the effort of getting
> an environment up that is able to build grpc-java.
>
> --
> You received this message because you are subscribed to the Google Groups "
> grpc.io" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to grpc-io+unsubscr...@googlegroups.com.
> To post to this group, send email to grpc-io@googlegroups.com.
> Visit this group at https://groups.google.com/group/grpc-io.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/grpc-io/2a4fdf8d-b5f6-40cd-bb00-a99533ae4e08%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/CAO78j%2BJHpHrjPv%2BRqRhh8wa4VVHqC1AT30NYeJ2F0uhBK023cQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[grpc-io] Proposal: descriptor (.pb) to .proto files

2019-01-06 Thread Alex Van Boxel 
Hi,

I'm currently prototyping a Descriptor to .proto files dumper. I'm 
wondering if it would be something that would be of interest to include in 
the java-utils part of *grpc-java*?

We're planning to use it for dynamically generated schema's, that we then 
dump on a filesystem to check into git.

(it's work in progress, but already generates pretty complete proto files)
https://github.com/anemos-io/metastore/blob/master/server/src/main/java/io/anemos/metastore/ProtoFileWriter.java

If this is something that could be included I'll do the effort of getting 
an environment up that is able to build grpc-java.

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/2a4fdf8d-b5f6-40cd-bb00-a99533ae4e08%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.