[
https://issues.apache.org/jira/browse/KUDU-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Serbin updated KUDU-3334:
--------------------------------
Description:
As it turned out, there are conflicts during static initialization of symbols
related to protobuf extensions in Impala which links in {{libkudu_client}}
while dynamically linking in {{libprotobuf}} on itself. It seems the issue
might be caused by the fact of having two independent pools of descriptors, so
eventually impala complains about that in run-time:
{noformat}
[libprotobuf ERROR google/protobuf/descriptor.cc:3620] Invalid proto descriptor
for file "kudu/rpc/rpc_header.proto":
[libprotobuf ERROR google/protobuf/descriptor.cc:3623]
kudu.rpc.track_rpc_result: ".google.protobuf.MethodOptions" is not defined.
[libprotobuf ERROR google/protobuf/descriptor.cc:3623] kudu.rpc.authz_method:
".google.protobuf.MethodOptions" is not defined.
[libprotobuf ERROR google/protobuf/descriptor.cc:3623]
kudu.rpc.default_authz_method: ".google.protobuf.ServiceOptions" is not defined.
{noformat}
>From examining the libkudu_client, it turns out some related protobuf symbols
>are discoverable:
{noformat}
$ nm -g libkudu_client.so | grep protobuf | grep desc
00000000006469a0 R
_ZN50TableStruct_google_2fprotobuf_2fdescriptor_2eproto7offseftsE
0000000000751a20 D descriptor_table_google_2fprotobuf_2fany_2eproto
0000000000751580 D descriptor_table_google_2fprotobuf_2fdescriptor_2eproto
0000000000751ac0 D
descriptor_table_google_2fprotobuf_2fsource_5fcontext_2eproto0000000000751b60 D
descriptor_table_google_2fprotobuf_2ftype_2eproto
0000000000751cc0 D descriptor_table_google_2fprotobuf_2fwrappers_2eproto
0000000000751900 D
scc_info_DescriptorProto_ExtensionRange_google_2fprotobuf_2fdescriptor_2eproto
00000000007519f0 D
scc_info_DescriptorProto_ReservedRange_google_2fprotobuf_2fdescriptor_2eproto
0000000000751920 D
scc_info_DescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
00000000007519d0 D
scc_info_EnumDescriptorProto_EnumReservedRange_google_2fprotobuf_2fdescriptor_2eproto
00000000007518c0 D
scc_info_EnumDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
00000000007518a0 D scc_info_EnumOptions_google_2fprotobuf_2fdescriptor_2eproto
0000000000751880 D
scc_info_EnumValueDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
0000000000751860 D
scc_info_EnumValueOptions_google_2fprotobuf_2fdescriptor_2eproto
0000000000751840 D
scc_info_ExtensionRangeOptions_google_2fprotobuf_2fdescriptor_2eproto
0000000000751820 D
scc_info_FieldDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
0000000000751800 D scc_info_FieldOptions_google_2fprotobuf_2fdescriptor_2eproto
00000000007517a0 D
scc_info_FileDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
0000000000751780 D
scc_info_FileDescriptorSet_google_2fprotobuf_2fdescriptor_2eproto
0000000000751760 D scc_info_FileOptions_google_2fprotobuf_2fdescriptor_2eproto
00000000007519b0 D
scc_info_GeneratedCodeInfo_Annotation_google_2fprotobuf_2fdescriptor_2eproto
0000000000751740 D
scc_info_GeneratedCodeInfo_google_2fprotobuf_2fdescriptor_2eproto
0000000000751720 D
scc_info_MessageOptions_google_2fprotobuf_2fdescriptor_2eproto
0000000000751700 D
scc_info_MethodDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
00000000007516e0 D scc_info_MethodOptions_google_2fprotobuf_2fdescriptor_2eproto
00000000007516c0 D
scc_info_OneofDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
00000000007516a0 D scc_info_OneofOptions_google_2fprotobuf_2fdescriptor_2eproto
0000000000751660 D
scc_info_ServiceDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
0000000000751640 D
scc_info_ServiceOptions_google_2fprotobuf_2fdescriptor_2eproto
0000000000751990 D
scc_info_SourceCodeInfo_Location_google_2fprotobuf_2fdescriptor_2eproto
0000000000751620 D
scc_info_SourceCodeInfo_google_2fprotobuf_2fdescriptor_2eproto
0000000000751970 D
scc_info_UninterpretedOption_NamePart_google_2fprotobuf_2fdescriptor_2eproto
0000000000751600 D
scc_info_UninterpretedOption_google_2fprotobuf_2fdescriptor_2eproto
{noformat}
So, it seems the issue might be addressed by hiding the symbols from the
libprotobuf library linked in statically by libkudu_client.
I found this article which seems to be relevant:
https://stackoverflow.com/questions/2222162/how-to-apply-fvisibility-option-to-symbols-in-static-libraries
was:
As it turned out, there are conflicts during static initialization of symbols
related to protobuf extensions in Impala which links in {{libkudu_client}}
while dynamically linking in {{libprotobuf}} on itself. It seems the issue
might be caused by the fact of having two separate registries of for protobuf
types descriptors, so eventually impala complains about that in run-time:
{noformat}
[libprotobuf ERROR google/protobuf/descriptor.cc:3620] Invalid proto descriptor
for file "kudu/rpc/rpc_header.proto":
[libprotobuf ERROR google/protobuf/descriptor.cc:3623]
kudu.rpc.track_rpc_result: ".google.protobuf.MethodOptions" is not defined.
[libprotobuf ERROR google/protobuf/descriptor.cc:3623] kudu.rpc.authz_method:
".google.protobuf.MethodOptions" is not defined.
[libprotobuf ERROR google/protobuf/descriptor.cc:3623]
kudu.rpc.default_authz_method: ".google.protobuf.ServiceOptions" is not defined.
{noformat}
>From examining the libkudu_client, it turns out some related protobuf symbols
>are discoverable:
{noformat}
$ nm -g libkudu_client.so | grep protobuf | grep desc
00000000006469a0 R
_ZN50TableStruct_google_2fprotobuf_2fdescriptor_2eproto7offseftsE
0000000000751a20 D descriptor_table_google_2fprotobuf_2fany_2eproto
0000000000751580 D descriptor_table_google_2fprotobuf_2fdescriptor_2eproto
0000000000751ac0 D
descriptor_table_google_2fprotobuf_2fsource_5fcontext_2eproto0000000000751b60 D
descriptor_table_google_2fprotobuf_2ftype_2eproto
0000000000751cc0 D descriptor_table_google_2fprotobuf_2fwrappers_2eproto
0000000000751900 D
scc_info_DescriptorProto_ExtensionRange_google_2fprotobuf_2fdescriptor_2eproto
00000000007519f0 D
scc_info_DescriptorProto_ReservedRange_google_2fprotobuf_2fdescriptor_2eproto
0000000000751920 D
scc_info_DescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
00000000007519d0 D
scc_info_EnumDescriptorProto_EnumReservedRange_google_2fprotobuf_2fdescriptor_2eproto
00000000007518c0 D
scc_info_EnumDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
00000000007518a0 D scc_info_EnumOptions_google_2fprotobuf_2fdescriptor_2eproto
0000000000751880 D
scc_info_EnumValueDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
0000000000751860 D
scc_info_EnumValueOptions_google_2fprotobuf_2fdescriptor_2eproto
0000000000751840 D
scc_info_ExtensionRangeOptions_google_2fprotobuf_2fdescriptor_2eproto
0000000000751820 D
scc_info_FieldDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
0000000000751800 D scc_info_FieldOptions_google_2fprotobuf_2fdescriptor_2eproto
00000000007517a0 D
scc_info_FileDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
0000000000751780 D
scc_info_FileDescriptorSet_google_2fprotobuf_2fdescriptor_2eproto
0000000000751760 D scc_info_FileOptions_google_2fprotobuf_2fdescriptor_2eproto
00000000007519b0 D
scc_info_GeneratedCodeInfo_Annotation_google_2fprotobuf_2fdescriptor_2eproto
0000000000751740 D
scc_info_GeneratedCodeInfo_google_2fprotobuf_2fdescriptor_2eproto
0000000000751720 D
scc_info_MessageOptions_google_2fprotobuf_2fdescriptor_2eproto
0000000000751700 D
scc_info_MethodDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
00000000007516e0 D scc_info_MethodOptions_google_2fprotobuf_2fdescriptor_2eproto
00000000007516c0 D
scc_info_OneofDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
00000000007516a0 D scc_info_OneofOptions_google_2fprotobuf_2fdescriptor_2eproto
0000000000751660 D
scc_info_ServiceDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
0000000000751640 D
scc_info_ServiceOptions_google_2fprotobuf_2fdescriptor_2eproto
0000000000751990 D
scc_info_SourceCodeInfo_Location_google_2fprotobuf_2fdescriptor_2eproto
0000000000751620 D
scc_info_SourceCodeInfo_google_2fprotobuf_2fdescriptor_2eproto
0000000000751970 D
scc_info_UninterpretedOption_NamePart_google_2fprotobuf_2fdescriptor_2eproto
0000000000751600 D
scc_info_UninterpretedOption_google_2fprotobuf_2fdescriptor_2eproto
{noformat}
So, it seems the issue might be addressed by hiding the symbols from the
libprotobuf library linked in statically by libkudu_client.
I found this article which seems to be relevant:
https://stackoverflow.com/questions/2222162/how-to-apply-fvisibility-option-to-symbols-in-static-libraries
> Clarify whether it's possible to hide libprotobuf symbols exported from
> libkudu_client
> --------------------------------------------------------------------------------------
>
> Key: KUDU-3334
> URL: https://issues.apache.org/jira/browse/KUDU-3334
> Project: Kudu
> Issue Type: Task
> Reporter: Alexey Serbin
> Assignee: Alexey Serbin
> Priority: Major
>
> As it turned out, there are conflicts during static initialization of symbols
> related to protobuf extensions in Impala which links in {{libkudu_client}}
> while dynamically linking in {{libprotobuf}} on itself. It seems the issue
> might be caused by the fact of having two independent pools of descriptors,
> so eventually impala complains about that in run-time:
> {noformat}
> [libprotobuf ERROR google/protobuf/descriptor.cc:3620] Invalid proto
> descriptor for file "kudu/rpc/rpc_header.proto":
> [libprotobuf ERROR google/protobuf/descriptor.cc:3623]
> kudu.rpc.track_rpc_result: ".google.protobuf.MethodOptions" is not defined.
> [libprotobuf ERROR google/protobuf/descriptor.cc:3623]
> kudu.rpc.authz_method: ".google.protobuf.MethodOptions" is not defined.
> [libprotobuf ERROR google/protobuf/descriptor.cc:3623]
> kudu.rpc.default_authz_method: ".google.protobuf.ServiceOptions" is not
> defined.
> {noformat}
> From examining the libkudu_client, it turns out some related protobuf symbols
> are discoverable:
> {noformat}
> $ nm -g libkudu_client.so | grep protobuf | grep desc
> 00000000006469a0 R
> _ZN50TableStruct_google_2fprotobuf_2fdescriptor_2eproto7offseftsE
> 0000000000751a20 D descriptor_table_google_2fprotobuf_2fany_2eproto
> 0000000000751580 D descriptor_table_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751ac0 D
> descriptor_table_google_2fprotobuf_2fsource_5fcontext_2eproto0000000000751b60
> D descriptor_table_google_2fprotobuf_2ftype_2eproto
> 0000000000751cc0 D descriptor_table_google_2fprotobuf_2fwrappers_2eproto
> 0000000000751900 D
> scc_info_DescriptorProto_ExtensionRange_google_2fprotobuf_2fdescriptor_2eproto
> 00000000007519f0 D
> scc_info_DescriptorProto_ReservedRange_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751920 D
> scc_info_DescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
> 00000000007519d0 D
> scc_info_EnumDescriptorProto_EnumReservedRange_google_2fprotobuf_2fdescriptor_2eproto
> 00000000007518c0 D
> scc_info_EnumDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
> 00000000007518a0 D scc_info_EnumOptions_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751880 D
> scc_info_EnumValueDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751860 D
> scc_info_EnumValueOptions_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751840 D
> scc_info_ExtensionRangeOptions_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751820 D
> scc_info_FieldDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751800 D
> scc_info_FieldOptions_google_2fprotobuf_2fdescriptor_2eproto
> 00000000007517a0 D
> scc_info_FileDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751780 D
> scc_info_FileDescriptorSet_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751760 D scc_info_FileOptions_google_2fprotobuf_2fdescriptor_2eproto
> 00000000007519b0 D
> scc_info_GeneratedCodeInfo_Annotation_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751740 D
> scc_info_GeneratedCodeInfo_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751720 D
> scc_info_MessageOptions_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751700 D
> scc_info_MethodDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
> 00000000007516e0 D
> scc_info_MethodOptions_google_2fprotobuf_2fdescriptor_2eproto
> 00000000007516c0 D
> scc_info_OneofDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
> 00000000007516a0 D
> scc_info_OneofOptions_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751660 D
> scc_info_ServiceDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751640 D
> scc_info_ServiceOptions_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751990 D
> scc_info_SourceCodeInfo_Location_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751620 D
> scc_info_SourceCodeInfo_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751970 D
> scc_info_UninterpretedOption_NamePart_google_2fprotobuf_2fdescriptor_2eproto
> 0000000000751600 D
> scc_info_UninterpretedOption_google_2fprotobuf_2fdescriptor_2eproto
> {noformat}
> So, it seems the issue might be addressed by hiding the symbols from the
> libprotobuf library linked in statically by libkudu_client.
> I found this article which seems to be relevant:
> https://stackoverflow.com/questions/2222162/how-to-apply-fvisibility-option-to-symbols-in-static-libraries
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
