[protobuf] Issue 686 in protobuf: Protoc does not provide a dependency manifest output file for build tools like make
Status: New Owner: liuj...@google.com Labels: Type-Defect Priority-Medium New issue 686 by richardg...@gmail.com: Protoc does not provide a dependency manifest output file for build tools like make https://code.google.com/p/protobuf/issues/detail?id=686 Protobuf files can have import statements and include directories, so the full dependency tree is not known by the build system before executing protoc. This makes it hard for the build system to know when to schedule any given protoc command. Make & other build systems can use a dependency file to trigger a rebuild when a dependency changes. GCC and other compilers expose this with the -MF flag, which outputs a file in the format output_filename: input_file \ input_file2 \ input_file3 ... Protobuf knows the transitive set of input files, but doesn't expose this information in this format. Patch to follow. -- You received this message because this project is configured to send all issue notifications to this address. You may adjust your notification preferences at: https://code.google.com/hosting/settings -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
Re: [protobuf] Can protobuf work with C++ templates?
Got a new skill :P
Thanks very much!
--- Original Message ---
From: "Stephen Tu"
Sent: January 22, 2015 1:28 AM
To: "Ji Wan"
Cc: protobuf@googlegroups.com
Subject: Re: [protobuf] Can protobuf work with C++ templates?
This is not really a protobuf question, moreso a C++ question. But anyways,
the typical way to do this is:
template struct MatrixTraits { };
template <> struct MatrixTraits { typedef DoubleMatrix type; };
template <> struct MatrixTraits{ typedef FloatMatrix type; };
template class Matrix {
typename MatrixTraits::type mat_data_;
};
On Mon, Jan 19, 2015 at 5:41 PM, Ji Wan wrote:
> Suppose I have two message types `DoubleMatrix` and `FloatMatrix`, and a
> template class `Matrix`:
>
>
> message DoubleMatrix {
> required uint32 rows = 1;
> required uint32 cols = 2;
> repeated double data = 3 [packed=true];
> }
>
>
> message FloatMatrix {
> required uint32 rows = 1;
> required uint32 cols = 2;
> repeated float data = 3 [packed=true];
> }
>
>
> template
> class Matrix {
> MSTType mat_data_;
> };
>
>
>
> Is it possible to make `MSTType` as `FloatMatrix` if `DType` is `float`, as
> `DoubleMatrix` if `DType` is `double`?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at http://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
[protobuf] Building for RTEMS ARM
Hello,
I am building Protocol Buffers from github for RTEMS
(http://www.rtems.org/) to run on a Xilinx Zynq (ARM CortexA9) and I am
making some progress but I have some questions.
I am using the current development head for RTEMS which is called 4.11 and
it is stable and almost about to be released. The gcc is:
$ arm-rtems4.11-gcc -v
Using built-in specs.
COLLECT_GCC=/opt/work/rtems/4.11/bin/arm-rtems4.11-gcc
COLLECT_LTO_WRAPPER=/opt/work/rtems/4.11/libexec/gcc/arm-rtems4.11/4.9.2/lto
-wrapper
Target: arm-rtems4.11
Configured with: ../gcc-4.9.2/configure --prefix=/opt/work/rtems/4.11 --
bindir=/opt/work/rtems/4.11/bin --exec_prefix=/opt/work/rtems/4.11 --
includedir=/opt/work/rtems/4.11/include --libdir=/opt/work/rtems/4.11/lib --
libexecdir=/opt/work/rtems/4.11/libexec --mandir=/opt/work/rtems/4.11/share/man
--infodir=/opt/work/rtems/4.11/share/info --datadir=/opt/work/rtems/4.11/share
--build=x86_64-freebsd10.0 --host=x86_64-freebsd10.0 --target=arm-rtems4.11
--disable-libstdcxx-pch --with-gnu-as --with-gnu-ld --verbose --with-newlib
--with-system-zlib --disable-nls --without-included-gettext
--disable-win32-registry
--enable-version-specific-runtime-libs --disable-lto
--enable-newlib-io-c99-formats
--enable-newlib-iconv --enable-newlib-iconv-encodings=big5,cp775,cp850,cp852
,cp855,cp866,euc_jp,euc_kr,euc_tw,iso_8859_1,iso_8859_10,iso_8859_11,
iso_8859_13,iso_8859_14,iso_8859_15,iso_8859_2,iso_8859_3,iso_8859_4,
iso_8859_5,iso_8859_6,iso_8859_7,iso_8859_8,iso_8859_9,iso_ir_111,koi8_r,
koi8_ru,koi8_u,koi8_uni,ucs_2,ucs_2_internal,ucs_2be,ucs_2le,ucs_4,
ucs_4_internal,ucs_4be,ucs_4le,us_ascii,utf_16,utf_16be,utf_16le,utf_8,
win_1250,win_1251,win_1252,win_1253,win_1254,win_1255,win_1256,win_1257,win_1258
--enable-threads --disable-plugin --enable-obsolete --enable-languages=c,c++
Thread model: rtems
gcc version 4.9.2 20141030 (RTEMS 4.11, RSB
e7cbf74fe2c94f0a233b8a7efcac3e75c239333c, Newlib
de616601501c4f82968683e80c112604a2d40222) (GCC)
The first are errors related to Atromics. We are adding -fpermissive to
make then warning. Here are a few of the many warnings we are seeing:
from ../../protobuf.git/src/google/protobuf/arenastring.h:
40,
from ../../protobuf.git/src/google/protobuf/descriptor.pb.h
:23,
from ../../protobuf.git/src/google/protobuf/compiler/ruby/
ruby_generator.cc:36:
../../protobuf.git/src/google/protobuf/stubs/once.h: In function 'void
google::protobuf::GoogleOnceInit(google::protobuf::ProtobufOnceType*, void
(*)())':
../../protobuf.git/src/google/protobuf/stubs/once.h:125:34: warning:
invalid conversion from 'google::protobuf::ProtobufOnceType* {aka int*}' to
'const
volatile Atomic32* {aka const volatile long int*}' [-fpermissive]
if (internal::Acquire_Load(once) != ONCE_STATE_DONE) {
^
In file included from ../../protobuf.git/src/google/protobuf/stubs/atomicops
.h:205:0,
from ../../protobuf.git/src/google/protobuf/stubs/
atomic_sequence_num.h:33,
from ../../protobuf.git/src/google/protobuf/arena.h:38,
from ../../protobuf.git/src/google/protobuf/descriptor.pb.h
:22,
from ../../protobuf.git/src/google/protobuf/compiler/ruby/
ruby_generator.cc:36:
../../protobuf.git/src/google/protobuf/stubs/atomicops_internals_arm_gcc.h:
136:17: note: initializing argument 1 of 'google::protobuf::internal::Atomic32
google::protobuf::internal::Acquire_Load(const volatile Atomic32*)'
The other issue is RTEMS is not self hosting and so always cross-compiled.
Is there a way to disable running tests ? It makes no sense to run the
tests when the build and host are not the same.
Thanks
Chris
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
Re: [protobuf] Re: Protobuf Buffers v3.0.0-alpha-1
The proto3 design discussion has lasted for more than half a year and all of them happened as an internal process. We have a lot of design docs, email exchanges and weekly design meetings, but they are not available for public consumption. Currently we are preparing public documentations for proto3 and it's targeted to be publicized late this quarter. On Wed Jan 21 2015 at 6:00:35 AM Sumit Kumar wrote: > Specially makes difficult for adoption in financial applications, the > has_field was one of the key reasons to migrate over to protofuf. > > Financial applications need differentiation in-between 0 value set and not > set. Eg: Limit order with 0 price is valid but with no price set is > invalid. Likewise market order with no price set is valid and with any > other price set is invalid (including the 0 value). And there are many > other cases, but anyway if the decision is made then not much value > discussing it. > > Regards, > Sumit Kumar > > On 17 Jan 2015, at 10:52 am, V.B. wrote: > > I suppose what I'm really wondering is: > a) How does it simplify the language implementations exactly? > b) Why was that not the case for *non*-primitives, which still have > presence logic? > > > On Friday, January 16, 2015 at 6:39:56 PM UTC-5, Feng Xiao wrote: >> >> The reason for dropping field presence is more of the same with dropping >> default values. Basically we want to simplify protobuf and make it easier >> to implement efficiently in more languages. We are preparing the proto3 >> documentation and will share more information about the trade-offs we have >> made. >> On Fri Jan 16 2015 at 12:17:25 PM V.B. wrote: >> >>> Can I ask for more details about why presence logic was removed (e.g. >>> hasFoo() ) for primitives? This has been a very useful feature for us. >>> >>> >>> 1. Removal of field presence logic for primitive value fields, removal of required fields, and removal of default values. This makes proto3 significantly easier to implement with open struct representations, as in languages like Android Java, Objective C, or Go. >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Protocol Buffers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to protobuf+u...@googlegroups.com. >>> To post to this group, send email to prot...@googlegroups.com. >>> Visit this group at http://groups.google.com/group/protobuf. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to protobuf+unsubscr...@googlegroups.com. > To post to this group, send email to protobuf@googlegroups.com. > Visit this group at http://groups.google.com/group/protobuf. > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to protobuf+unsubscr...@googlegroups.com. > To post to this group, send email to protobuf@googlegroups.com. > Visit this group at http://groups.google.com/group/protobuf. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
Re: [protobuf] Can protobuf work with C++ templates?
This is not really a protobuf question, moreso a C++ question. But anyways,
the typical way to do this is:
template struct MatrixTraits { };
template <> struct MatrixTraits { typedef DoubleMatrix type; };
template <> struct MatrixTraits{ typedef FloatMatrix type; };
template class Matrix {
typename MatrixTraits::type mat_data_;
};
On Mon, Jan 19, 2015 at 5:41 PM, Ji Wan wrote:
> Suppose I have two message types `DoubleMatrix` and `FloatMatrix`, and a
> template class `Matrix`:
>
>
> message DoubleMatrix {
> required uint32 rows = 1;
> required uint32 cols = 2;
> repeated double data = 3 [packed=true];
> }
>
>
> message FloatMatrix {
> required uint32 rows = 1;
> required uint32 cols = 2;
> repeated float data = 3 [packed=true];
> }
>
>
> template
> class Matrix {
> MSTType mat_data_;
> };
>
>
>
> Is it possible to make `MSTType` as `FloatMatrix` if `DType` is `float`, as
> `DoubleMatrix` if `DType` is `double`?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at http://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
Re: [protobuf] Re: Protobuf Buffers v3.0.0-alpha-1
Specially makes difficult for adoption in financial applications, the has_field was one of the key reasons to migrate over to protofuf. Financial applications need differentiation in-between 0 value set and not set. Eg: Limit order with 0 price is valid but with no price set is invalid. Likewise market order with no price set is valid and with any other price set is invalid (including the 0 value). And there are many other cases, but anyway if the decision is made then not much value discussing it. Regards, Sumit Kumar > On 17 Jan 2015, at 10:52 am, V.B. wrote: > > I suppose what I'm really wondering is: > a) How does it simplify the language implementations exactly? > b) Why was that not the case for non-primitives, which still have presence > logic? > > >> On Friday, January 16, 2015 at 6:39:56 PM UTC-5, Feng Xiao wrote: >> The reason for dropping field presence is more of the same with dropping >> default values. Basically we want to simplify protobuf and make it easier to >> implement efficiently in more languages. We are preparing the proto3 >> documentation and will share more information about the trade-offs we have >> made. >>> On Fri Jan 16 2015 at 12:17:25 PM V.B. wrote: >>> Can I ask for more details about why presence logic was removed (e.g. >>> hasFoo() ) for primitives? This has been a very useful feature for us. >>> >>> 1. Removal of field presence logic for primitive value fields, removal of required fields, and removal of default values. This makes proto3 significantly easier to implement with open struct representations, as in languages like Android Java, Objective C, or Go. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Protocol Buffers" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to protobuf+u...@googlegroups.com. >>> To post to this group, send email to prot...@googlegroups.com. >>> Visit this group at http://groups.google.com/group/protobuf. >>> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to protobuf+unsubscr...@googlegroups.com. > To post to this group, send email to protobuf@googlegroups.com. > Visit this group at http://groups.google.com/group/protobuf. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
Re: [protobuf] Re: Protobuf Buffers v3.0.0-alpha-1
Which forum was this discussed (dropping of default) ? May be reading it would give some insights into the details. Regards, Sumit Kumar > On 13 Jan 2015, at 2:40 pm, Arjun Satish wrote: > > Would it be possible to re-introduce this feature in a subsequent release? It > seems like you are still using it under-the-hood. And because of the benefits > I mentioned above, I strongly feel that it will only help the community. > > Best, > >> On Wed, Jan 7, 2015 at 8:01 PM, Feng Xiao wrote: >> +liujisi, who should have a better idea of why default value is dropped from >> proto3 and what alternatives users can rely on. >> >> Internally the design of proto3 has been discussed among a group of people >> for quite a long time, but most of them haven't subscribed this forum >> though... >>> On Wed, Jan 7, 2015 at 18:52 Arjun Satish wrote: >>> Did anyone get a chance to look at this request? >>> >>> >>> >>> On Wednesday, December 17, 2014 3:54:12 PM UTC-8, Arjun Satish wrote: Hey guys, Thanks for all the hard work! I have a question regarding the decision to drop support for default values. Fields which are set to their default values are not serialized. I noticed that in the new code (3.0.0-alpha-1 for Java), this condition still holds true. But the default values used are the standard ones (0 for int64/int32 etc) and cannot be specified in the .proto file. In some of my code, I had reasons to use non-zero default values (-1 for some integers, 1024 for some others, 3.14 for some doubles etc). Using the old protocol buffers, this was trivial to implement. This was a great feature as we could save atleast 2 bytes for every "untouched" field (which comes in handy when we persist the data :-)). Is there any way we can retain specification of default values in the .proto files and using them in the generated encoders/decoders? Thanks very much! Looking forward to the 3.0 release! Best, Arjun Satish > On Wednesday, December 10, 2014 8:51:01 PM UTC-8, Feng Xiao wrote: > Hi all, > > I just published protobuf v3.0.0-alpha-1 on our github site: > https://github.com/google/protobuf/releases/tag/v3.0.0-alpha-1 > > This is the first alpha release of protobuf v3.0.0. In protobuf v3.0.0, > we will add a new protobuf language version (aka proto3) and support a > wider range of programming languages (to name a few: ruby, php, node.js, > objective-c). This alpha version contains C++ and Java implementation > with partial proto3 support (see below for details). In future releases > we will add support for more programming languages and implement the full > proto3 feature set. Besides proto3, this alpha version also includes two > other new features: map fields and arena allocation. They are implemented > for both proto3 and the old protobuf language version (aka proto2). > > We are currently working on the documentation of these new features and > when it's ready it will be updated to our protobuf developer guide. For > the time being if you have any questions regarding proto3 or other new > features, please post your question in the discussion group. > > CHANGS > === > Version 3.0.0-alpha-1 (C++/Java): > > General > * Introduced Protocol Buffers language version 3 (aka proto3). > > When protobuf was initially opensourced it implemented Protocol > Buffers > language version 2 (aka proto2), which is why the version number > started from v2.0.0. From v3.0.0, a new language version (proto3) is > introduced while the old version (proto2) will continue to be > supported. > > The main intent of introducing proto3 is to clean up protobuf before > pushing the language as the foundation of Google's new API platform. > In proto3, the language is simplified, both for ease of use and to > make it available in a wider range of programming languages. At the > same time a few features are added to better support common idioms > found in APIs. > > The following are the main new features in language version 3: > > 1. Removal of field presence logic for primitive value fields, > removal > of required fields, and removal of default values. This makes > proto3 > significantly easier to implement with open struct > representations, > as in languages like Android Java, Objective C, or Go. > 2. Removal of unknown fields. > 3. Removal of extensions, which are instead replaced by a new > standard > type called Any. > 4. Fix semantics for unknown enum values. > 5. Addition of maps. > 6. Addition of a small set of st
