[protobuf] Re: Issue 188 in protobuf: protobuf fails to link after compiling with LDFLAGS=-Wl,--as-needed because of missing -lpthread
Comment #31 on issue 188 by xarthisius.kk: protobuf fails to link after compiling with LDFLAGS=-Wl,--as-needed because of missing -lpthread http://code.google.com/p/protobuf/issues/detail?id=188 For comparison, the Free Software Foundation has... I've contributed to several FSF projects in the past. AFAIR the most complicated thing to do was to sign a patch (for octave if I remember correctly). When I do it incognito it wants me to login to my @google.com account. Is there a support for that page/license thing? I won't be having net access till 2/01/2011. I'll revisit the problem after I get back. Merry Xmas! Kacper Kowalik -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Reducing code size of protocol buffers
This is C++. I had not tried optimizing for CODE_SIZE. I understood that CODE_SIZE was the precursor to and not as compact as LITE_RUNTIME. Yes, I believe they are tag numbers, the enumeration-style numbering of each possible response or request. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 188 in protobuf: protobuf fails to link after compiling with LDFLAGS=-Wl,--as-needed because of missing -lpthread
Comment #32 on issue 188 by ken...@google.com: protobuf fails to link after compiling with LDFLAGS=-Wl,--as-needed because of missing -lpthread http://code.google.com/p/protobuf/issues/detail?id=188 It looks like our CLA form may in fact be broken. :( I'll try to get it fixed. I'm as frustrated by all this as you are... -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Sorting Protocol Buffers with Hadoop MapReduce
Hmm, you might want to consider parsing both messages into DynamicMessages (or into the generated message classes if you have them compiled in) and then using a protobuf-reflection-based algorithm (not to be confused with Java reflection; see Message#getField()). Your current code is very much coupled to the wire format, and relies on tags being in a particular order (which is not technically guaranteed). On the other hand, your code is probably faster than a reflection-based algorithm would be. On Thu, Dec 16, 2010 at 3:18 PM, Owen O'Malley omal...@apache.org wrote: All, I'm hooking in ProtoBuf (as well as Avro, and Thrift) into Hadoop MapReduce. In order for that to make sense, I need to be able to sort on the protobuf messages. Hadoop uses compare function over the bytes of two serialized objects. Obviously, I could just use a memcmp, but that will lead to a sort order that is hard to explain to users. This function should lead to the obvious sort order, which will be much easier to understand. The rough idea is that I iterate over the fields sorted to be in id order and compare them based on their type. If a message is missing a field that has a default value, the default value is used. The code is here: http://bit.ly/f8Scfo It compiles and works in simple testing. I need to do more testing, but I thought I'd see if anyone here would be willing to take a look at my code for correctness and soundness. -- Owen -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Why not allow null values in Java for scalar types?
There are problems with this:
1) Boxing and unboxing primitives is relatively expensive, compared to just
passing them as primitives. If performance matters to you at all (and for
many protobuf users, it does), you probably don't want this.
2) If you accept messages from untrusted sources, your resulting code will
be more prone to security problems. For example, say you write some code
where the field foo is declared optional but, in practice, is always set.
It's likely that you're going to end up with some code that *assumes* that
it is set, and so doesn't check for null. This code may pass all your tests
because people unfortunately don't usually test against invalid inputs. If
some malicious user then sends you a message missing that field, your code
is going to crash. The protobuf design avoids that by having the getters
return a default value if the field is not set, so if you forget to check,
it's not a huge deal.
On Fri, Dec 17, 2010 at 8:25 AM, James james.j.re...@gmail.com wrote:
I am new to using Protocol Buffers and was really surprised why a
compiler option is not available to control how scalar types are
mapped in Java when generating the message objects. I understand the
concept of having a null int in C++ is not possible, but isn't is
really backwards to have to call another method in order to check if a
value has been set in Java? Having a null reference in Java does
exactly this!
In reading the reference docs:
http://code.google.com/apis/protocolbuffers/docs/reference/java-generated.html
The compiler will generate the following accessor methods in both the
message class and its builder:
* bool hasFoo(): Returns true if the field is set.
* int getFoo(): Returns the current value of the field. If the
field is not set, returns the default value.
Why not give protoc another control flag that would map all primitive
data types to their corresponding objects? (i.e. This would just map
a int - Integer or double - Double or a long - Long) This surely
would not break the protocol, but would give a much cleaner and Java
like way of programming. I just feel like I am working in Java but
with a C++ like interface. Java now has auto-boxing so really if
you like working with int and doubles instead of the Object
counterparts, you would still be okay...
Wouldn't you rather code like a Java Programmer by writing:
if(getFoo() != null)
{
// do something with getFoo()
}
instead of:
if(hasFoo())
{
// do something with getFoo()
}
Think of how this works when you are not trying to pass along the data
to another method that already takes an Integer object and that method
is ok with the reference being null.
I rather write:
doSomethingWithFoo(getFoo());
instead of
doSomethingWithFoo(hasFoo() ? getFoo() : null);
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] ambiguous Builder errors when compiling a Maven project
I've never seen this error. It doesn't make much sense -- protobuf
generated code should not contain any imports. Are you modifying the
generated code in some way?
On Sun, Dec 19, 2010 at 11:47 PM, Xiao Ling bardm...@gmail.com wrote:
Hi All,
I have a generated protobuf java file which compiles alone under both
ECJ and Oracle jdk. When the file is compiled together with other Java
and Scala files in a Maven project, the compilation keeps outputing
some error messages as follows:
DocumentProtos.java:1092: error: reference to Builder is ambiguous;
it is both defined in object Document and imported subsequently by
import Mention._
public Builder newBuilderForType() { return newBuilder(); }
^
DocumentProtos.java:1096: error: reference to Builder is ambiguous;
it is both defined in object Document and imported subsequently by
import Mention._
public Builder toBuilder() { return newBuilder(this); }
Does anybody have any idea why so?
Thanks,
-XL
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 188 in protobuf: protobuf fails to link after compiling with LDFLAGS=-Wl,--as-needed because of missing -lpthread
Comment #33 on issue 188 by ken...@google.com: protobuf fails to link after compiling with LDFLAGS=-Wl,--as-needed because of missing -lpthread http://code.google.com/p/protobuf/issues/detail?id=188 The issue should be fixed now. Sounds like it's too late, though. :( -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] ambiguous Builder errors when compiling a Maven project
You're right. There is no imports in the code. The message was printed
out by maven. As said, the file compiles ok by its own.
Here is the situation:
I have both scala and java code in my project. As long as there is
some scala code calls the protobuf code (which is in java), the
maven-scala-plugin (that is reponsible for compiling both scala and
java code) gives the error messages.
Thanks,
--Xiao
On Wed, Dec 22, 2010 at 4:13 PM, Kenton Varda ken...@google.com wrote:
I've never seen this error. It doesn't make much sense -- protobuf
generated code should not contain any imports. Are you modifying the
generated code in some way?
On Sun, Dec 19, 2010 at 11:47 PM, Xiao Ling bardm...@gmail.com wrote:
Hi All,
I have a generated protobuf java file which compiles alone under both
ECJ and Oracle jdk. When the file is compiled together with other Java
and Scala files in a Maven project, the compilation keeps outputing
some error messages as follows:
DocumentProtos.java:1092: error: reference to Builder is ambiguous;
it is both defined in object Document and imported subsequently by
import Mention._
public Builder newBuilderForType() { return newBuilder(); }
^
DocumentProtos.java:1096: error: reference to Builder is ambiguous;
it is both defined in object Document and imported subsequently by
import Mention._
public Builder toBuilder() { return newBuilder(this); }
Does anybody have any idea why so?
Thanks,
-XL
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Reading and writing multiple messages to a file
You can only have one CodedInputStream wrapping a ZeroCopyInputStream at a
time, otherwise they will confuse each other with buffering conflicts.
CodedInputStreams are cheap to construct, so you can just make one on the
stack in a block, like:
{
CodedInputStream coded_input(input_stream_p.get());
success = coded_input.ReadLittleEndian64(size);
}
On Tue, Dec 21, 2010 at 6:56 AM, rodrigo benenson
rodrigo.benen...@gmail.com wrote:
Hello all !
I am trying to save multiple Protocol Buffer messages into a file and then
reading them back.
All messages (except the first) have the same type.
My code now works. But I had to use a workaround instead. The code looks as
follows.
boost::uint64_t size;
const bool read_size_success =
input_coded_stream_p-ReadLittleEndian64(size);
bool read_data_success = false;
const bool use_zero_copy_stream = false;
if(use_zero_copy_stream)
{
read_data_success =
data.ParseFromBoundedZeroCopyStream(input_stream_p.get(),
static_castint(size));
}
else
{ // work around
std::string buffer;
input_coded_stream_p-ReadString(buffer, size);
read_data_success = data.ParseFromString(buffer);
}
if (read_size_success == false or read_data_success == false)
{
throw std::runtime_error(Failed to read a data message during
DataSequenceDataType::read);
}
The code above works fine when using use_zero_copy_stream == falsehowever
when I set
use_zero_copy_stream to true then the read fails (read_data_success ==
false).
Obviously input_coded_stream_p was constructed using input_stream_p.
Why is ParseFromBoundedZeroCopyStream not working as expected ? What is
the proper way of using it ?
Thanks for your answers.
Best regards,
rodrigob.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] ambiguous Builder errors when compiling a Maven project
Sorry, I don't know how the scala plugin works. It sounds like it has a
bug. Perhaps it generates its own code based on the Java code, and somehow
that code ends up broken given this particular input...
On Wed, Dec 22, 2010 at 4:39 PM, Xiao Ling bardm...@gmail.com wrote:
You're right. There is no imports in the code. The message was printed
out by maven. As said, the file compiles ok by its own.
Here is the situation:
I have both scala and java code in my project. As long as there is
some scala code calls the protobuf code (which is in java), the
maven-scala-plugin (that is reponsible for compiling both scala and
java code) gives the error messages.
Thanks,
--Xiao
On Wed, Dec 22, 2010 at 4:13 PM, Kenton Varda ken...@google.com wrote:
I've never seen this error. It doesn't make much sense -- protobuf
generated code should not contain any imports. Are you modifying the
generated code in some way?
On Sun, Dec 19, 2010 at 11:47 PM, Xiao Ling bardm...@gmail.com wrote:
Hi All,
I have a generated protobuf java file which compiles alone under both
ECJ and Oracle jdk. When the file is compiled together with other Java
and Scala files in a Maven project, the compilation keeps outputing
some error messages as follows:
DocumentProtos.java:1092: error: reference to Builder is ambiguous;
it is both defined in object Document and imported subsequently by
import Mention._
public Builder newBuilderForType() { return newBuilder(); }
^
DocumentProtos.java:1096: error: reference to Builder is ambiguous;
it is both defined in object Document and imported subsequently by
import Mention._
public Builder toBuilder() { return newBuilder(this); }
Does anybody have any idea why so?
Thanks,
-XL
--
You received this message because you are subscribed to the Google
Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] ambiguous Builder errors when compiling a Maven project
I agree. I'm going to ask the maven-scala-plugin guys then.
Thanks,
--Xiao
On Wed, Dec 22, 2010 at 4:53 PM, Kenton Varda ken...@google.com wrote:
Sorry, I don't know how the scala plugin works. It sounds like it has a
bug. Perhaps it generates its own code based on the Java code, and somehow
that code ends up broken given this particular input...
On Wed, Dec 22, 2010 at 4:39 PM, Xiao Ling bardm...@gmail.com wrote:
You're right. There is no imports in the code. The message was printed
out by maven. As said, the file compiles ok by its own.
Here is the situation:
I have both scala and java code in my project. As long as there is
some scala code calls the protobuf code (which is in java), the
maven-scala-plugin (that is reponsible for compiling both scala and
java code) gives the error messages.
Thanks,
--Xiao
On Wed, Dec 22, 2010 at 4:13 PM, Kenton Varda ken...@google.com wrote:
I've never seen this error. It doesn't make much sense -- protobuf
generated code should not contain any imports. Are you modifying the
generated code in some way?
On Sun, Dec 19, 2010 at 11:47 PM, Xiao Ling bardm...@gmail.com wrote:
Hi All,
I have a generated protobuf java file which compiles alone under both
ECJ and Oracle jdk. When the file is compiled together with other Java
and Scala files in a Maven project, the compilation keeps outputing
some error messages as follows:
DocumentProtos.java:1092: error: reference to Builder is ambiguous;
it is both defined in object Document and imported subsequently by
import Mention._
public Builder newBuilderForType() { return newBuilder(); }
^
DocumentProtos.java:1096: error: reference to Builder is ambiguous;
it is both defined in object Document and imported subsequently by
import Mention._
public Builder toBuilder() { return newBuilder(this); }
Does anybody have any idea why so?
Thanks,
-XL
--
You received this message because you are subscribed to the Google
Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
