[protobuf] Re: Windows, VisualStudio, C++, and building libraries

[Martin Scholz]

I assume you included your Debug folder as an additional library folder and 
added all the libs as dependencies in your Linker options.

Am Dienstag, 26. November 2019 00:38:27 UTC+1 schrieb Stéphane Charette:
>
> (Windows10, Visual Studio Community 2019, protobufs v3.10.1, gRPC v1.20.0.)
>
> I need to use protocol buffers and gRPC to get some data, and then combine 
> that with other things to write a plugin for an existing Windows 
> application.  So my plugin will be a .dll that is dynamically loaded by the 
> application which I didn't write and cannot control.
>
> For that reason, I wanted gRPC and protocol buffers to be statically 
> linked into my .dll.  This way users will only have a single .dll file they 
> need to manage.  But I'm having a very difficult time figuring out how to 
> properly build protobufs and grpc for Windows so I can link a .lib file 
> into the rest of my code.
>
> I first tried to use vcpkg, like this:
>
> vcpkg.exe install zlib:x64-windows-static
> vcpkg.exe install c-ares:x64-windows-static
> vcpkg.exe install openssl:x64-windows-static
> vcpkg.exe install protobuf[zlib]:x64-windows-static
> vcpkg.exe install grpc:x64-windows-static
>
> If I remember correctly, this led to some link-time problems because the 
> libs use /MT, while the application uses/expects /MD.  So then I switched 
> to this:
>
> vcpkg.exe install zlib:x64-windows
> vcpkg.exe install c-ares:x64-windows
> vcpkg.exe install openssl:x64-windows
> vcpkg.exe install protobuf[zlib]:x64-windows
> vcpkg.exe install grpc:x64-windows
>
> But I believe this gave me .dll files instead of .lib files.  I even tried 
> this from a post I saw somewhere:
>
> SET ( VCPKG_CRT_LINKAGE dynamic )
> SET ( VCPKG_LIBRARY_LINKAGE static )
>
> So now I'm trying to build protobufs and grpc from scratch.  E.g.,
>
> git clone -b v3.10.1 https://github.com/protocolbuffers/protobuf.git
> cd protobuf
> git submodule update --init --recursive
> cd cmake
> mkdir build64
> cd build64
> cmake -DCMAKE_BUILD_TYPE=Debug -Dprotobuf_BUILD_TESTS=OFF 
> -Dprotobuf_WITH_ZLIB=OFF ..
>
> I then copied the resulting .lib files into my plugin project, but I'm 
> still seeing several unresolved external symbols, such as this:
>
> 1>app.pb.obj : error LNK2019: unresolved external symbol 
> "__declspec(dllimport) public: void __cdecl 
> google::protobuf::internal::ArenaStringPtr::ClearToEmpty(class 
> std::basic_string,class 
> std::allocator > const *,class google::protobuf::Arena *)" 
> (__imp_?ClearToEmpty@ArenaStringPtr@internal@protobuf@google@@QEAAXPEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@PEAVArena@34@@Z)
>  
> referenced in function "public: static void __cdecl 
> google::protobuf::internal::MapTypeHandler<9,class 
> std::basic_string,class 
> std::allocator > >::Clear(struct 
> google::protobuf::internal::ArenaStringPtr *,class google::protobuf::Arena 
> *)" 
> (?Clear@?$MapTypeHandler@$08V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@internal@protobuf@google@@SAXPEAUArenaStringPtr@234@PEAVArena@34@@Z)
>
> So I'm still barking up the wrong tree.
>
> The files I'm attempting to linking against are:
>
> ${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/gpr.lib
> ${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/grpc.lib
> ${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/grpc_cronet.lib
> ${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/grpc_unsecure.lib
> ${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/grpc++_cronet.lib
> ${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/grpc++_unsecure.lib
> ${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/grpc++.lib
> ${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/address_sorting.lib
> ${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/cares.lib
> ${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/zlibd.lib
> ${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/libprotobuf-lited.lib
> ${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/libprotobufd.lib
> ${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/libprotocd.lib
>
> This was my attempt to manually fixing up the .lib against which to link.  
> Originally, I had tried it like this:
>
> FIND_PACKAGE ( Threads REQUIRED)
> FIND_PACKAGE ( ZLIB REQUIRED)
> FIND_PACKAGE ( c-ares CONFIG REQUIRED)
> FIND_PACKAGE ( OpenSSL REQUIRED)
> FIND_PACKAGE ( protobuf CONFIG REQUIRED)
> FIND_PACKAGE ( gRPC CONFIG REQUIRED)
>
> SET ( SPLICE_EXTERNAL_DEPS
> Threads::Threads
> protobuf::libprotoc protobuf::libprotobuf protobuf::libprotobuf-lite
> gRPC::gpr gRPC::grpc gRPC::grpc++ gRPC::grpc_cronet
> # c-ares::cares_static
> c-ares::cares # c-ares::cares_static ? should I be using static?
> OpenSSL::SSL OpenSSL::Crypto
> ZLIB::ZLIB
> )
>
> Another possibly-related question:  gRPC seems to include protocol 
> buffers.  Am I reading that correct?  By building gRPC, am I already 
> building protobufs, and thus shouldn't bother with protobufs directly?
>
> Thanks for any pointers,
>
> Stéphane
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an 

[protobuf] Windows, VisualStudio, C++, and building libraries

[Stéphane Charette]

(Windows10, Visual Studio Community 2019, protobufs v3.10.1, gRPC v1.20.0.)

I need to use protocol buffers and gRPC to get some data, and then combine 
that with other things to write a plugin for an existing Windows 
application.  So my plugin will be a .dll that is dynamically loaded by the 
application which I didn't write and cannot control.

For that reason, I wanted gRPC and protocol buffers to be statically linked 
into my .dll.  This way users will only have a single .dll file they need 
to manage.  But I'm having a very difficult time figuring out how to 
properly build protobufs and grpc for Windows so I can link a .lib file 
into the rest of my code.

I first tried to use vcpkg, like this:

vcpkg.exe install zlib:x64-windows-static
vcpkg.exe install c-ares:x64-windows-static
vcpkg.exe install openssl:x64-windows-static
vcpkg.exe install protobuf[zlib]:x64-windows-static
vcpkg.exe install grpc:x64-windows-static

If I remember correctly, this led to some link-time problems because the 
libs use /MT, while the application uses/expects /MD.  So then I switched 
to this:

vcpkg.exe install zlib:x64-windows
vcpkg.exe install c-ares:x64-windows
vcpkg.exe install openssl:x64-windows
vcpkg.exe install protobuf[zlib]:x64-windows
vcpkg.exe install grpc:x64-windows

But I believe this gave me .dll files instead of .lib files.  I even tried 
this from a post I saw somewhere:

SET ( VCPKG_CRT_LINKAGE dynamic )
SET ( VCPKG_LIBRARY_LINKAGE static )

So now I'm trying to build protobufs and grpc from scratch.  E.g.,

git clone -b v3.10.1 https://github.com/protocolbuffers/protobuf.git
cd protobuf
git submodule update --init --recursive
cd cmake
mkdir build64
cd build64
cmake -DCMAKE_BUILD_TYPE=Debug -Dprotobuf_BUILD_TESTS=OFF 
-Dprotobuf_WITH_ZLIB=OFF ..

I then copied the resulting .lib files into my plugin project, but I'm 
still seeing several unresolved external symbols, such as this:

1>app.pb.obj : error LNK2019: unresolved external symbol 
"__declspec(dllimport) public: void __cdecl 
google::protobuf::internal::ArenaStringPtr::ClearToEmpty(class 
std::basic_string,class 
std::allocator > const *,class google::protobuf::Arena *)" 
(__imp_?ClearToEmpty@ArenaStringPtr@internal@protobuf@google@@QEAAXPEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@PEAVArena@34@@Z)
 
referenced in function "public: static void __cdecl 
google::protobuf::internal::MapTypeHandler<9,class 
std::basic_string,class 
std::allocator > >::Clear(struct 
google::protobuf::internal::ArenaStringPtr *,class google::protobuf::Arena 
*)" 
(?Clear@?$MapTypeHandler@$08V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@internal@protobuf@google@@SAXPEAUArenaStringPtr@234@PEAVArena@34@@Z)

So I'm still barking up the wrong tree.

The files I'm attempting to linking against are:

${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/gpr.lib
${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/grpc.lib
${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/grpc_cronet.lib
${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/grpc_unsecure.lib
${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/grpc++_cronet.lib
${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/grpc++_unsecure.lib
${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/grpc++.lib
${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/address_sorting.lib
${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/cares.lib
${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/zlibd.lib
${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/libprotobuf-lited.lib
${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/libprotobufd.lib
${CMAKE_HOME_DIRECTORY}/3rd-party/Debug/libprotocd.lib

This was my attempt to manually fixing up the .lib against which to link.  
Originally, I had tried it like this:

FIND_PACKAGE ( Threads REQUIRED)
FIND_PACKAGE ( ZLIB REQUIRED)
FIND_PACKAGE ( c-ares CONFIG REQUIRED)
FIND_PACKAGE ( OpenSSL REQUIRED)
FIND_PACKAGE ( protobuf CONFIG REQUIRED)
FIND_PACKAGE ( gRPC CONFIG REQUIRED)

SET ( SPLICE_EXTERNAL_DEPS
Threads::Threads
protobuf::libprotoc protobuf::libprotobuf protobuf::libprotobuf-lite
gRPC::gpr gRPC::grpc gRPC::grpc++ gRPC::grpc_cronet
# c-ares::cares_static
c-ares::cares # c-ares::cares_static ? should I be using static?
OpenSSL::SSL OpenSSL::Crypto
ZLIB::ZLIB
)

Another possibly-related question:  gRPC seems to include protocol 
buffers.  Am I reading that correct?  By building gRPC, am I already 
building protobufs, and thus shouldn't bother with protobufs directly?

Thanks for any pointers,

Stéphane

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/c933f9ed-9971-4543-8069-3a3bad54fa0c%40googlegroups.com.

Re: [protobuf] use of custom field-options during code-generation

['Adam Cozzette' via Protocol Buffers]

Your code looks right as far as I can tell. In C++ there's no need to
manually register extensions because they register themselves before main()
begins. Could you try calling DebugString() on the FieldDescriptor that you
expect to have the extension? Seeing a human-readable summary of that
descriptor might offer some insight.

On Mon, Nov 25, 2019 at 12:28 PM Leo Hilbert  wrote:

> Hi guys,
>
> *Background:*
> I'm currently building/prototyping a modified version of the protoc
> java-compiler. I need to change some things from the
> standard-implementation for my usecase, but want to keep most of it.
> To avoid starting from scratch I copied the java-generation-code to my own
> project and started to modify it. I'm a java-dev, so C++ proves to be quite
> the challenge, but I'm learning a lot and am nearly finished with what I
> wanted to do. However now I'm stuck and need your help.
>
> Here is a quick overview of the relevant folders in my project (
> https://github.com/leohilbert/protoc-gen-java-leo ):
>
>- *include-folder*: "h-files" that I copied over (otherwise no classes
>from the protoc-libraries were found)
>- *java-folder*: for my java-library. Also contains my
>test-proto-files for now (java/src/test/proto).
>- *src/google/protobuf/compiler/java_leo*: copied and modifed
>java-generation code from the official repo
>- *src/javaleo/proto*: the generated c++ code for my custom
>FieldOption (java/src/test/proto/options.proto)
>
> btw: if you have any feedback on my setup, it's greatly appreciated! I'm a
> C++ noob and trying to understand everything as I go. :)
>
> *Now to my actual problem:*
> If you navigate to src/google/protobuf/compiler/java_leo/java_field.cc
> 
>  in
> line 64 you can see that I'm trying to read the value from my custom-field
> option by calling
> string test = field->options().GetExtension(javaleo::proto::javatype);
>
> This is *always *empty. Although I imported and used this exact custom
> FieldOption
> 
> in my addressbook.proto
> 
> .
> The C++-Class for the option.proto is also generated and add in the
> CMakeList (src/javaleo/proto).
> "javaleo::proto::javatype" *does* compile, so at least it seems to be in
> the executable somewhere.
>
> What makes me suspicious is that I did not register this custom-extension
> anywhere in my code. I'm thinking of something like
> "ExtensionRegistry.registerExtension(javaleo::proto::javatype);" to add to
> the beginning of my plugin (main.cpp). Otherwise how should protoc know how
> this field should be interpreted. Am I on the right track here, or did I
> understand this whole system wrong.
>
> I debugged this a lot already and am running out of ideas. Searching for a
> solution online is also really tricky, because not a lot of people have
> compiled there own protoc-compiler.. But maybe I'm looking in the wrong
> direction.
>
> I hope my issue is somewhat clear, it's hard describing it when knowing so
> little about c++ and how protoc works internally.. ^^
>
> TLDR: I need to get the value of my custom-option during code-generation,
> but my field-option is always ignored. Help :c
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/protobuf/8135f496-9ec3-4e0d-bff2-dd37d85792bc%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/CADqAXr4GoT%3DhfU7-GkpiVyy3CNE5hnUK0Fti-1S2byCfje%3D8yA%40mail.gmail.com.

[protobuf] use of custom field-options during code-generation

[Leo Hilbert]

Hi guys,

*Background:*
I'm currently building/prototyping a modified version of the protoc 
java-compiler. I need to change some things from the 
standard-implementation for my usecase, but want to keep most of it.
To avoid starting from scratch I copied the java-generation-code to my own 
project and started to modify it. I'm a java-dev, so C++ proves to be quite 
the challenge, but I'm learning a lot and am nearly finished with what I 
wanted to do. However now I'm stuck and need your help.

Here is a quick overview of the relevant folders in my project ( 
https://github.com/leohilbert/protoc-gen-java-leo ):

   - *include-folder*: "h-files" that I copied over (otherwise no classes 
   from the protoc-libraries were found)
   - *java-folder*: for my java-library. Also contains my test-proto-files 
   for now (java/src/test/proto).
   - *src/google/protobuf/compiler/java_leo*: copied and modifed 
   java-generation code from the official repo
   - *src/javaleo/proto*: the generated c++ code for my custom FieldOption 
   (java/src/test/proto/options.proto)
   
btw: if you have any feedback on my setup, it's greatly appreciated! I'm a 
C++ noob and trying to understand everything as I go. :) 

*Now to my actual problem:*
If you navigate to src/google/protobuf/compiler/java_leo/java_field.cc 

 in 
line 64 you can see that I'm trying to read the value from my custom-field 
option by calling 
string test = field->options().GetExtension(javaleo::proto::javatype);

This is *always *empty. Although I imported and used this exact custom 
FieldOption 

 
in my addressbook.proto 

.
The C++-Class for the option.proto is also generated and add in the 
CMakeList (src/javaleo/proto).
"javaleo::proto::javatype" *does* compile, so at least it seems to be in 
the executable somewhere.

What makes me suspicious is that I did not register this custom-extension 
anywhere in my code. I'm thinking of something like 
"ExtensionRegistry.registerExtension(javaleo::proto::javatype);" to add to 
the beginning of my plugin (main.cpp). Otherwise how should protoc know how 
this field should be interpreted. Am I on the right track here, or did I 
understand this whole system wrong.

I debugged this a lot already and am running out of ideas. Searching for a 
solution online is also really tricky, because not a lot of people have 
compiled there own protoc-compiler.. But maybe I'm looking in the wrong 
direction.

I hope my issue is somewhat clear, it's hard describing it when knowing so 
little about c++ and how protoc works internally.. ^^

TLDR: I need to get the value of my custom-option during code-generation, 
but my field-option is always ignored. Help :c

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/8135f496-9ec3-4e0d-bff2-dd37d85792bc%40googlegroups.com.