Re: [protobuf] Re: Truncated varint decoding a message just built
Strange. It looks like there is a bug in the type checker for booleans -- it will accept any integer. However, this doesn't explain the encoding error. Looking at the code, I can't see how this could happen; any non-zero integer value should result in 1 being written to the wire. The problem you describe sounds like an obscure problem that happens in C++, but I can't see why it would happen in Python. What version of protobufs are you using? On Thu, Dec 23, 2010 at 10:10 PM, BigBaaadBob bigbaaad...@gmail.com wrote: Turns out it is simple: put a big number into a bool. The encoding side doesn't complain, but the decoder does. A bug on my part, but suboptimal behavior for the library. On Dec 22, 6:09 pm, Kenton Varda ken...@google.com wrote: Can you provide a small, self-contained example program demonstrating this problem? On Wed, Dec 22, 2010 at 4:52 PM, BigBaaadBob bigbaaad...@gmail.com wrote: I have Python code doing something like this: m.whatever = somevalue; etc. s = m.SerializeToString() m.Clear() m.ParseFromString(s) And the m.ParseFromString throws google.protobuf.message.DecodeError: Truncated varint. (And this is the only error thrown! If I print m I don't get any errors.) It seems like protobuf should be able to read something it successfully creates, no? Suggestions? -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com protobuf%2bunsubscr...@googlegroups.c om . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Stronger introspection
You can use the type's descriptor to inspect the fields it contains and what
all their types are. MyType.DESCRIPTOR is the descriptor for MyType. Once
you've figured out what field you want to read/write, use getattr() /
setattr().
Note that the C++ and Java interfaces for this are somewhat cleaner and more
mature than the Python interface. But you should be able to do it in Python
too.
On Thu, Dec 23, 2010 at 10:33 PM, BigBaaadBob bigbaaad...@gmail.com wrote:
I have a couple of applications where I need to do some generic coding
with protobufs. In one case I have a gui that needs to display
selected contents out of a message, and in another case I need to
transcode from a different message format into a protobuf. It is
trivial to hand-write hardcoded stuff for both of these, but it
violates the DRY principle, and is error prone, etc.
What I want to do in both cases (the gui) is, by conventionally using
the same names for the gui or other message fields as the fields in
the protobuf messages, use introspection to do the field mapping.
So, for example, given a list of PyQT widgets, I would use the widget
names to go find a like-named field in the protobuf message. But
since the protobuf message is likely to be composite, and in fact it
might be a subfield I need, I have to do some serious introspection.
Also, I want to do this kind of at init time BEFORE I receive a
message.
For example, I might have a PyQT widget named tt_m1, and I want to get
the .temperatures.tt_m1.val field out of the following protobuf
message structure by setting up a mapping between the widget and the
protobuf field at init time. I'd like to do similar things in the
opposite direction: given a selected set of protobuf fields, transcode
from another message into protobuf contents.
Suggestions?
message Analog {
required double val = 1;
enum AnalogStatus {
SENSOR_NOMINAL = 0;
SENSOR_UNDER_LIMIT = 1;
SENSOR_OVER_LIMIT = 2;
SENSOR_FAULT = 3;
SENSOR_NO_DATA = 4;
}
required AnalogStatus status = 2;
optional double limit = 3;
}
message Temperatures {
optional Analog tt_m1 = 1;
optional Analog tt_m2 = 2;
optional Analog tt_m3 = 3;
optional Analog tt_m4 = 4;
}
message StatusUpdate {
...stuff...
optional Temperatures temperatures = 7;
...stuff...
}
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Re: Converting JSON Text Into PB Bytes Type
You should probably contact the authors of protobuf-java-format; I'm not
sure if they pay attention to this list.
Base64 is the best way to encode arbitrary (non-text) data as text.
However, it's really up to the JSON converter code you are using to decide
what format to use. As far as I know, JSON does not specify any particular
encoding for arbitrary bytes.
On Sat, Dec 25, 2010 at 2:40 AM, yalmasri y.alma...@gmail.com wrote:
Thanks for the reply.
I encoded Hello world in Base64 and I got this SGVsbG8gd29ybGQ=.
That doesn't look close to the format I sent in the question. Here's
the code snippet I used:
import org.apache.commons.codec.binary.Base64;
public class Codec {
public static void main(String[] args) {
try {
String clearText = Hello world;
// Base64
System.out.println(Encoded: +
Base64.encodeBase64String(clearText.getBytes()));
} catch (Exception e) {
e.printStackTrace();
}
}
}
I'm using this: http://code.google.com/p/protobuf-java-format/ for
JSON -- PB conversion
On Dec 23, 3:48 am, Kenton Varda ken...@google.com wrote:
You will probably need to base64-encode bytes fields before embedding
them
in JSON, since JSON is a text format.
Which JSON - protobuf converter are you using? Or did you write your
own?
On Tue, Dec 21, 2010 at 11:41 PM, yalmasri y.alma...@gmail.com wrote:
Hello,
One of our customers is still using legacy Java 1.4 and therefore
could not use PB. For that we created an intermediate service that
receives text messages in plain JSON format then convert them into
generated PB objects.
In one object we have a field named desc of type bytes which will
hold a textual description of the arriving message. When the customer
wants to send English text for desc the value in JSON will be plain
readable English, but if non-ASCII is to be used, how the value in
JSON will look like? It look to me something like:
\0069\0062\006J\0062\006J\000 \006\'\006D\0069\006E\006J\006D\006\f
\000 \006D\006B\006/\000 \006\'\006F\006*\006G\006I\000
But I really don't know how to get this out of non-ASCII characters.
Can anyone help?
--
You received this message because you are subscribed to the Google
Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
protobuf%2bunsubscr...@googlegroups.comprotobuf%252bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] how can we deal with multi-lingual string? (c++)
The dirty little secret of std::wstring is that it does not actually deal with non-ASCII characters on all platforms. On some platforms wchar_t is 8-bit just like char! You should avoid using wstring and wchar_t for this reason; define your own types that are exactly what you need. For protocol buffers, we take the convention of using regular 8-bit chars but always using UTF-8 encoding. On Sat, Dec 25, 2010 at 8:10 PM, alcohol alcoho...@gmail.com wrote: without std::wstring support, how can we deal with strings consists of Ascii, Chinese,Japanese, Korean characters? -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Transfer protocol buffer object to POJO java object
Hi,
I have a protocol buffer object like:
message PBPerson
{
optional string lastName = 1;
optional string firstName = 2;
}
I also have another java object
class JavaPerson
{
private String lastName;
private String firstName;
// setter and getter
}
I want to convert from PBPerson to JavaPerson and vice versa and I
don't want to do it manually by writing getter and setter for each
fields because my object can have more than 10 fields. Is there a
utility from protocol buffer that helps me doing that. So the would
expect a utility class that does something like:
JavaPerson javaPerson = PBConverter.convert(PBPerson pbPerson);
PBPerson pbPerson = PBConverter.convert(JavaPerson javaPerson);
Thanks,
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
[protobuf] C# client to Java based server
Hello, I originally was using JAX-WS to communicate between client and server. This works well when my client is C# and server is Java based service. I have been asked to speed up the performance using Protocol Buffers to do binary serialization instead of text-based serialization. Knowing that I can no longer use Soap and had to figure out another transport mechanism. Is RPC the best way? Can I use RPC when my client and server are using C# and Java respectively? I noticed the example of CXF-protobuf link that shows how you can make the connection between Java and Java using RPC, but nothing about if the Client is C#. thanks, Tommy -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: how can we deal with multi-lingual string? (c++)
I'd say use UTF-8 for all strings and you are good to go. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Linker issues: undefined reference to `a_message__get_packed_size'
I am using the protobuf-c extension out there.
Getting the following errors:
gcc tut.c tut.pb-c.c -o tut -lprotobuf-c -lprotobuf -lpthread
/tmp/ccBHwTBl.o: In function `main':
tut.c:(.text+0xab): undefined reference to
`a_message__get_packed_size'
tut.c:(.text+0xcc): undefined reference to `a_message__pack'
collect2: ld returned 1 exit status
tut.c
#include tut.pb-
c.h
#include
stdio.h
#include
stdlib.h
int main(int argc, char **argv)
{
AMessage msg =
AMESSAGE__INIT;
void
*buf;
unsigned
len;
if (argc 2 || argc 3)
{
fprintf(stderr, usage: pack a [b]
\n);
return
1;
}
msg.a =
atoi(argv[1]);
if (argc == 3)
{
msg.has_b =
1;
msg.b =
atoi(argv[2]);
}
len =
a_message__get_packed_size(msg);
buf =
malloc(len);
a_message__pack(msg,
buf);
fwrite(buf, len, 1,
stdout);
return
0;
}
tut-pb-c.h
--
/* Generated by the protocol buffer compiler. DO NOT EDIT! */
#ifndef PROTOBUF_C_tut_2eproto__INCLUDED
#define PROTOBUF_C_tut_2eproto__INCLUDED
#include google/protobuf-c/protobuf-c.h
PROTOBUF_C_BEGIN_DECLS
typedef struct _AMessage AMessage;
/* --- enums --- */
/* --- messages --- */
struct _AMessage
{
ProtobufCMessage base;
int32_t a;
protobuf_c_boolean has_b;
int32_t b;
};
#define AMESSAGE__INIT \
{ PROTOBUF_C_MESSAGE_INIT (amessage__descriptor) \
, 0, 0,0 }
/* AMessage methods */
void amessage__init
(AMessage *message);
size_t amessage__get_packed_size
(const AMessage *message);
size_t amessage__pack
(const AMessage *message,
uint8_t *out);
size_t amessage__pack_to_buffer
(const AMessage *message,
ProtobufCBuffer *buffer);
AMessage *
amessage__unpack
(ProtobufCAllocator *allocator,
size_t len,
const uint8_t *data);
void amessage__free_unpacked
(AMessage *message,
ProtobufCAllocator *allocator);
/* --- per-message closures --- */
typedef void (*AMessage_Closure)
(const AMessage *message,
void *closure_data);
/* --- services --- */
/* --- descriptors --- */
extern const ProtobufCMessageDescriptor amessage__descriptor;
PROTOBUF_C_END_DECLS
#endif /* PROTOBUF_tut_2eproto__INCLUDED */
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] C# client to Java based server
Protocol buffers itself has no built-in RPC implementation. You have to find an RPC implementation that supports whatever languages you are interested in, or write your own. It's not too hard to write a simple RPC implementation given protocol buffers as a base. Sending protobufs over HTTP is a popular solution that lets you leverage existing infrastructure. On Sun, Dec 26, 2010 at 11:55 PM, Tommy tommyhan...@gmail.com wrote: Hello, I originally was using JAX-WS to communicate between client and server. This works well when my client is C# and server is Java based service. I have been asked to speed up the performance using Protocol Buffers to do binary serialization instead of text-based serialization. Knowing that I can no longer use Soap and had to figure out another transport mechanism. Is RPC the best way? Can I use RPC when my client and server are using C# and Java respectively? I noticed the example of CXF-protobuf link that shows how you can make the connection between Java and Java using RPC, but nothing about if the Client is C#. thanks, Tommy -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Transfer protocol buffer object to POJO java object
I don't know of any existing tools for this. You could write code that does
this via reflection (protobuf reflection on the protobuf object, and basic
java reflection on the POJO). Or, you could write a protoc plugin which
generates the code you need, though that will be a lot more complicated.
On Tue, Dec 28, 2010 at 3:55 PM, Sean Nguyen sontran...@gmail.com wrote:
Hi,
I have a protocol buffer object like:
message PBPerson
{
optional string lastName = 1;
optional string firstName = 2;
}
I also have another java object
class JavaPerson
{
private String lastName;
private String firstName;
// setter and getter
}
I want to convert from PBPerson to JavaPerson and vice versa and I
don't want to do it manually by writing getter and setter for each
fields because my object can have more than 10 fields. Is there a
utility from protocol buffer that helps me doing that. So the would
expect a utility class that does something like:
JavaPerson javaPerson = PBConverter.convert(PBPerson pbPerson);
PBPerson pbPerson = PBConverter.convert(JavaPerson javaPerson);
Thanks,
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] C# client to Java based server
On Tue, Dec 28, 2010 at 15:14, Kenton Varda ken...@google.com wrote: Protocol buffers itself has no built-in RPC implementation. You have to find an RPC implementation that supports whatever languages you are interested in, or write your own. It's not too hard to write a simple RPC implementation given protocol buffers as a base. Sending protobufs over HTTP is a popular solution that lets you leverage existing infrastructure. Good place to start looking for implementations is http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns On Sun, Dec 26, 2010 at 11:55 PM, Tommy tommyhan...@gmail.com wrote: Hello, I originally was using JAX-WS to communicate between client and server. This works well when my client is C# and server is Java based service. I have been asked to speed up the performance using Protocol Buffers to do binary serialization instead of text-based serialization. Knowing that I can no longer use Soap and had to figure out another transport mechanism. Is RPC the best way? Can I use RPC when my client and server are using C# and Java respectively? I noticed the example of CXF-protobuf link that shows how you can make the connection between Java and Java using RPC, but nothing about if the Client is C#. thanks, Tommy -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
