Re: Implement ShutdownProtobufLibrary().

2009-05-05 Thread Jason Hsueh
I believe the reasoning against this was for the case where the client doesn't care whether the memory is freed - in particular, if the shutdown is happening right before the process exits, then it would be doing a bunch of work to free memory that's going back to the OS anyway. Apparently this

[protobuf] Re: Seek in a codedoutputstream

2009-11-14 Thread Jason Hsueh
Why can't you write the actual size of data rather than a 0? Just use the ByteSize() method to find out how much data will be written for a protocol buffer. On Sat, Nov 14, 2009 at 11:39 AM, Fishtank saptarshi.g...@gmail.com wrote: Hello, I'm using the following idiom ZeroCopyOutputStream*

[protobuf] Re: Seek in a codedoutputstream

2009-11-16 Thread Jason Hsueh
, Jason Hsueh jas...@google.com wrote: Why can't you write the actual size of data rather than a 0? Just use the ByteSize() method to find out how much data will be written for a protocol buffer. On Sat, Nov 14, 2009 at 11:39 AM, Fishtank saptarshi.g...@gmail.com wrote: Hello, I'm

[protobuf] Re: Regd: Resolving Wire type ambiguities

2009-11-16 Thread Jason Hsueh
You can decode the protocol buffer with just wire type + tag number, but you won't know the original types without a proto definition. Everything would be treated as an unknown field. You could access these by iterating through the UnknownFieldSet, but again, you can't recover the original types.

Re: [protobuf] Memory leak (fragmentation?)

2009-11-24 Thread Jason Hsueh
Are you sure the leak is due to persistCacheToFile()? It looks like your FileManager::loadCacheFromFile() method is leaking the protocol buffer when you are reading the cache back in. If that's not the case, can you send a small reproduction of the problem? On Tue, Nov 24, 2009 at 2:03 AM, Greg

Re: [protobuf] Java -using GPB -which type of stream should I be using over sockets?

2009-12-02 Thread Jason Hsueh
Protocol buffers implement their own serialization routines: the writeTo(OutputStream output) method will write the binary format to the OutputStream. From a socket, just do: MyProtoBuf pb = new MyProtoBuf; ... pb.writeTo(socket.getOutputStream()); On Wed, Dec 2, 2009 at 7:47 AM, Nigel Pickard

Re: [protobuf] Re: Java -using GPB -which type of stream should I be using over sockets?

2009-12-02 Thread Jason Hsueh
On Wed, Dec 2, 2009 at 12:13 PM, Nigel Pickard pickard.ni...@gmail.comwrote: Thanks for the reply Adam -I did a simple sub into my code with your suggestion, but it doesn't work. However, now it's made me think of more questions. If protocol buffers doesn't use object I/O streams,

Re: [protobuf] Re: Java -using GPB -which type of stream should I be using over sockets?

2009-12-02 Thread Jason Hsueh
On Wed, Dec 2, 2009 at 4:15 PM, Nigel Pickard pickard.ni...@gmail.comwrote: When the Java test app works, I'd say the IO stream is transparent. I use the same simple object instance each time (have a hard coded method that simply returns the same object so it's identical), and I simply call

Re: [protobuf] does java api have something like DescriptorPool

2009-12-09 Thread Jason Hsueh
No, there isn't an equivalent to the DescriptorPool in Java. If you know the types that you want you can build a mapping yourself. Or if you'd be interested in porting the C++ DescriptorPool to java that would be great! 2009/12/9 Romain François francoisrom...@free.fr Hello, Given a

Re: [protobuf] Reading Multiple Protobufs from a File

2009-12-09 Thread Jason Hsueh
Indeed, CodedOutputStream and CodedInputStream aren't presently available in Python. Want to add it? You should also be able to implement what you want without the help of these classes. http://code.google.com/apis/protocolbuffers/docs/techniques.html#streamingdescribes what to do - once you have

Re: [protobuf] Re: Build problem with nested enum

2009-12-09 Thread Jason Hsueh
Thanks. Is there an ADD or DELETE macro that gets defined somewhere in your MSVC? It doesn't seem like the compiler has a problem with previous enum typedefs, so I'm guessing that there may be a name conflict with your enum names, and it's expanding to code which no longer compiles. Can you try

Re: [protobuf] Builders containing builders

2009-12-16 Thread Jason Hsueh
You should be able to pass a builder to some method, and have the method modify some value of the builder just fine. Modifying a value in a sub-message is a bit more inconvenient, but still doable. If you have some type Bar in type Foo like: message Bar { optional int32 a = 1; optional int32

Re: [protobuf] Message limit

2010-01-12 Thread Jason Hsueh
The limit applies to the data source from which a message is parsed. So if you want to parse a serialization of Foo, it applies to Foo. But if you parse a bunch of Bar messages one by one, and add them individually to Bar, then the limit only applies to each individual Bar. You can change the

Re: [protobuf] Any j2me implementations that support extensions?

2010-01-27 Thread Jason Hsueh
It appears the ones listed at http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns do not, though I've only taken a cursory glance at their pages. Anyone on the mailing list holding out on us? :-) On Wed, Jan 27, 2010 at 2:11 AM, Juan Amiguet jamig...@gmail.com wrote: Hi All, Does any one

Re: [protobuf] analysing field sizes

2010-01-27 Thread Jason Hsueh
(Since you said getserializedsize I'm assuming Java) There isn't a precanned way to get a breakdown of the serialized sizes, but I don't think you need to reimplement the entire parser: if you take a serialized message, you could set up a CodedInputStream and use a fairly simple loop of

Re: [protobuf] how to get RepeatedField object

2010-02-11 Thread Jason Hsueh
No, there isn't a way to get the RepeatedField from the reflection interface. You can only do so via the generated interface. On Thu, Feb 11, 2010 at 5:57 AM, Romain Francois romain.francois.r.enthusi...@gmail.com wrote: Hello, How can I get hold of a RepeatedField object to manage a

Re: [protobuf] how to get RepeatedField object

2010-02-12 Thread Jason Hsueh
will not ever use a RepeatedField directly; they will use the get-by-index, set-by-index, and add accessors that are generated for all repeated fields What if I do want to use RepeatedField ? Romain On 02/11/2010 06:50 PM, Jason Hsueh wrote: No, there isn't a way to get

Re: [protobuf] Third-Party Add-ons - Update Request - for Protocol Buffer Editor

2010-02-17 Thread Jason Hsueh
Updated the wiki page. Thanks for writing the tool! On Wed, Feb 17, 2010 at 1:40 AM, Bruce bm_...@yahoo.com.au wrote: Third-Party Add-ons - Update Request - for Proto Buffer Editor Could you please add my project http://code.google.com/p/protobufeditor/ to the Third-Party Add-ons (Other

Re: [protobuf] Invalid embedded descriptor Dependencies passed to FileDescriptor.buildFrom() don't match

2010-02-24 Thread Jason Hsueh
I believe you should be running protoc from the same directory, using the same --proto_path, for both commands. The imports should be relative to the common root directory. modify the import in project2\proto\TaskProtos.proto: import project1/proto/ProtoException.proto $ protoc --proto_path=.

Re: [protobuf] Using std::string as a zero-copy buffer

2010-03-04 Thread Jason Hsueh
I don't know about any other implementations, but a word of caution: I think that this would be especially complicated/hacky for gcc 4.3, which as I recall switched to a copy-on-write implementation for std::string. You'd probably want to make sure that the performance difference is really worth

Re: [protobuf] Using std::string as a zero-copy buffer

2010-03-04 Thread Jason Hsueh
(__len); } } See the code for basic_string::append, for example. Given that, it shouldn't be too ugly to do it from outside, but these things are always easy to mess up. Sean On Thu, Mar 4, 2010 at 1:09 PM, Jason Hsueh jas...@google.com wrote: I don't know about any other

Re: [protobuf] Using std::string as a zero-copy buffer

2010-03-04 Thread Jason Hsueh
use -fno-strict-aliasing. On Thu, Mar 4, 2010 at 10:09 AM, Jason Hsueh jas...@google.com wrote: I don't know about any other implementations, but a word of caution: I think that this would be especially complicated/hacky for gcc 4.3, which as I recall switched to a copy-on-write implementation

Re: [protobuf] Variant-length encoding : is it OK to be wasteful?

2010-03-10 Thread Jason Hsueh
On Wed, Mar 10, 2010 at 7:48 AM, Marc Gravell marc.grav...@gmail.comwrote: The variable-length encoding allows multiple representations of the same value - for example, 1 could be written as: 0001 or it could be (I believe): 1000 00011000 Is there anything

Re: [protobuf] Disabling the 'gen_cpp' output of Services?

2010-03-10 Thread Jason Hsueh
In your .proto file, just specify the file-level option cc_generic_services = false. Comments from descriptor.proto: // Should generic services be generated in each language? Generic services // are not specific to any particular RPC system. They are generated by the // main code

Re: [protobuf] How to get the number of the repeated data?

2010-03-17 Thread Jason Hsueh
This is covered in the tutorial - select the one for your language from http://code.google.com/apis/protocolbuffers/docs/overview.html On Tue, Mar 16, 2010 at 8:57 PM, jungle www1...@gmail.com wrote: here is .proto message foo{ required xxx repeated xxx } how to figure out the exact

Re: [protobuf] How to get optional data?

2010-03-17 Thread Jason Hsueh
The getters for embedded messages return a reference - see http://code.google.com/apis/protocolbuffers/docs/reference/cpp-generated.html On Tue, Mar 16, 2010 at 9:02 PM, jungle www1...@gmail.com wrote: here is the .proto message Bar{ required int32 l = 1; } message Foo{ option Bar

Re: [protobuf] modifying lists within a message

2010-03-17 Thread Jason Hsueh
Messages are indeed immutable once built. getXXXList() for both the builder and the message should return unmodifiable lists: if you try to modify them you should get an exception. If you haven't already, take a look at the Java tutorial here:

Re: [protobuf] File size of the serialized records

2010-03-22 Thread Jason Hsueh
If you're measuring using sizeof(), you won't account for memory allocated by subobjects (strings and submessages are stored as pointers). You should use Message::SpaceUsed() instead. The inmemory vs serialized size is going to depend on your proto definition and how you use it. If you have a lot

Re: [protobuf] Why getting errors when exanging large messages

2010-03-24 Thread Jason Hsueh
The first message from message_lite.cc:106 is telling you that some required fields were missing. Have you verified that each of the fields listed was set? If you don't want required fields checking, you need to use one of the ParsePartial* to skip required field checking. The second message is

[protobuf] Re:

2010-03-25 Thread Jason Hsueh
+protobuf mailing list This is a bit outside the scope of protocol buffers - the core library does not provide any such functionality. I am personally not that familiar with their usage, but I don't see any technical limitations to using protocol buffers with shared memory. It would be a bit

Re: [protobuf] c++: Print contents of a serialized file using its descriptor file

2010-04-02 Thread Jason Hsueh
On Fri, Apr 2, 2010 at 9:13 AM, Navin ng06d...@gmail.com wrote: 1. For the person example, I create a descriptor as follows: E:\Desktop\prototype_MAIN\protoc.exe person.proto -- descriptor_set_out=msg.protocomp 2. I create a Person object and serialized it to file. person.pb 3. I want to

Re: [protobuf] Re: c++: Print contents of a serialized file using its descriptor file

2010-04-02 Thread Jason Hsueh
. On Apr 2, 1:42 pm, Jason Hsueh jas...@google.com wrote: On Fri, Apr 2, 2010 at 9:13 AM, Navin ng06d...@gmail.com wrote: 1. For the person example, I create a descriptor as follows: E:\Desktop\prototype_MAIN\protoc.exe person.proto -- descriptor_set_out=msg.protocomp 2. I create a Person

Re: [protobuf] Re: c++: Print contents of a serialized file using its descriptor file

2010-04-02 Thread Jason Hsueh
to their package. Would the use in dynamic_message_unittest.cc have helped? It basically does what's described here, except it gets the descriptor protos from the compiled-in types. On Apr 2, 6:21 pm, Jason Hsueh jas...@google.com wrote: On Fri, Apr 2, 2010 at 2:40 PM, Navin ng06d...@gmail.com

Re: [protobuf] Re: c++: Print contents of a serialized file using its descriptor file

2010-04-05 Thread Jason Hsueh
::ShutdownProtobufLibrary(); return 0; } On Apr 2, 9:39 pm, Jason Hsueh jas...@google.com wrote: On Fri, Apr 2, 2010 at 6:20 PM, Navin ng06d...@gmail.com wrote: I went on a tangent with the use of TextFormat::Parse. I somehow thought that would help me read the data, but you have

Re: [protobuf] 65k string limit in MS Visual C++ 2005

2010-04-06 Thread Jason Hsueh
This sounds like a C++ version of the Java issue fixed with r189http://code.google.com/p/protobuf/source/detail?r=189: the encoded descriptor data in the generated file exceeded the Java string literal limit. This large descriptor was previously built in C++ but was removed from the test suite in

Re: [protobuf] Protocol buffers and CentOS

2010-04-06 Thread Jason Hsueh
How did you install the python protobuf library, and what was the output? On Tue, Apr 6, 2010 at 11:15 AM, Eric yangcoo...@gmail.com wrote: Hello, I have installed python (2.6.5) and GPB on CentOS (it's a VM not sure if that's relevant) but am not able to run. In it's simplest version I do:

Re: [protobuf] Re: Tutorial example make failure

2010-04-06 Thread Jason Hsueh
The test you mentioned verifies that descriptor.pb.{h,cc} and plugin.pb.{h,cc} that's in the source matches what the compiler would generate. The test can fail if you've made changes affecting what the compiler outputs, or if you've made changes to those files directly. On Tue, Apr 6, 2010 at

Re: [protobuf] Protocol buffers and CentOS

2010-04-07 Thread Jason Hsueh
): File string, line 1, in module File google/__init__.py, line 1, in module __import__('pkg_resources').declare_namespace(__name__) ImportError: No module named pkg_resources thanks again, Eric On Tue, Apr 6, 2010 at 4:40 PM, Jason Hsueh jas...@google.com wrote: Eric - Those commands

Re: [protobuf] multi arch build of protobuf on macosx

2010-04-07 Thread Jason Hsueh
Kenton, do you have any experience compiling protobufs on OSX? On Wed, Apr 7, 2010 at 9:34 AM, Romain Francois romain.francois.r.enthusi...@gmail.com wrote: Hello, Please bear with me if this is obvious ... I'm new to OSX. Are there instructions somewhere to build protobuf for both i386

Re: [protobuf] Minimum set of libraries/files to run Python and Google Protocol Buffer (CentOS)

2010-04-07 Thread Jason Hsueh
It looks like you just copied over the source code. You need to install the modules in a place where python can find them: the python/setup.py script does this for you. On Wed, Apr 7, 2010 at 3:29 PM, Sherry Yang yangcoo...@gmail.com wrote: Kenton, Thanks for the followup. I actually copied

Re: [protobuf] i am not able to compile the files which includes import statements

2010-04-09 Thread Jason Hsueh
Can you provide more information? What are the commands you ran? What are the errors? On Fri, Apr 9, 2010 at 4:09 AM, prashant waykar waykar.prash...@gmail.comwrote: Hello, can someone please help me out as i am new to PB. i am getting compile errors in the files which consists import

Re: [protobuf] compile issue with import statements having path info

2010-04-13 Thread Jason Hsueh
I think if you run everything from the root of your imports this all goes away. The protobuf_AddDesc_* functions are file-level methods used to initialize various bits of static data. The names for these methods are generated based on the pathname of the .proto. When b.proto imports a.proto, it

Re: [protobuf] Importing one proto file in another

2010-04-14 Thread Jason Hsueh
When you invoke the proto compiler are you passing a value for -I (the import path) such that abc/first.proto can be found? It defaults to the current working directory. On Wed, Apr 14, 2010 at 9:20 AM, DK deepakkumar.si...@gmail.com wrote: Hi, I have a proto file (first.proto) in folder abc

Re: [protobuf] RepeatedPtrFieldstd::string::iterator ?

2010-04-14 Thread Jason Hsueh
On Wed, Apr 14, 2010 at 1:15 PM, Andrew Kimpton awkimp...@gmail.com wrote: Do I have a typo (or some other misunderstanding) or is there no support for iterators with RepeatedPtrFields ? Iterators are only available in RepeatedFields ? google::protobuf::RepeatedPtrFieldstd::string::iterator

Re: [protobuf] Protobuf 2.3 and compiler warnings

2010-04-15 Thread Jason Hsueh
Yes, these are safe to ignore. This is also addressed in r302: http://code.google.com/p/protobuf/source/detail?r=302 On Thu, Apr 15, 2010 at 12:40 PM, Fishtank saptarshi.g...@gmail.com wrote: Hello, I ran protoc --cpp_out on a proto file using 2.3.0 compiler . I replaced the my previously

Re: [protobuf] parseFrom suggestion

2010-04-16 Thread Jason Hsueh
The builders already have a mergeFrom() method with the second signature: http://code.google.com/apis/protocolbuffers/docs/reference/java/com/google/protobuf/AbstractMessage.Builder.html#mergeFrom(byte[], int, int) On Fri, Apr 16, 2010 at 3:02 PM, Fishtank saptarshi.g...@gmail.com wrote:

Re: [protobuf] Suggesting a new Wire Type which encodes length by nested field count rather than bytes

2010-04-19 Thread Jason Hsueh
It's not clear to me what the number of fields buys you. How do you intend to serialize this large data if it doesn't fit into memory? The usual solution to this is to split the large message into smaller fragments - in your example, say, 10 messages each containing 100 nested messages. You can

Re: [protobuf] Usage of Protocol Buffer with map reduce framework

2010-04-19 Thread Jason Hsueh
As far as I recall, mapreduce just works with raw bytes. If you want to use protocol buffers within map reduce, you just need to use the serialization routines to convert from the raw bytes to your proto. On Mon, Apr 19, 2010 at 4:06 AM, stuti awasthi stutic...@gmail.com wrote: Hi all, I just

Re: [protobuf] How to handle options in compiler plugin?

2010-04-20 Thread Jason Hsueh
Your options extensions can live in a .proto file in the location of your choosing. protoc doesn't need to know about your options: it will simply parse the options into the UninterpretedOptions message. Then in the .proto files where you want to use your options, you need to put the option names

Re: [protobuf] parseFrom with ExtensionRegistryLite when upgrading from 2.0.3 to 2.3

2010-04-21 Thread Jason Hsueh
ExtensionRegistry extends ExtensionRegistryLite, and the generated parseFrom()/mergeFrom() routines don't need the descriptor-based features that are added by ExtensionRegistry. Is this causing problems for you? On Wed, Apr 21, 2010 at 9:35 AM, skay she...@pobox.com wrote: I'm upgrading from

Re: [protobuf] protobuf == SQL

2010-04-21 Thread Jason Hsueh
You could write a bridge between the database and protobufs using reflection. It is probably simplest if each field in the protobuf corresponds to a column in the database table. Then when processing the result of a SQL query, you can just iterate over the columns, find the corresponding field in

Re: [protobuf] parseFrom with ExtensionRegistryLite when upgrading from 2.0.3 to 2.3

2010-04-21 Thread Jason Hsueh
for ExtensionRegistry. m = returnType.getMethod(parseFrom, Array.newInstance(Byte.TYPE, 0).getClass(), ExtensionRegistry.class); Hence I was wondering why things changed. On Wed, Apr 21, 2010 at 12:24 PM, Jason Hsueh jas...@google.com wrote: ExtensionRegistry extends ExtensionRegistryLite

Re: [protobuf] packed repeated fields / extensions

2010-04-26 Thread Jason Hsueh
The packed option is specified in the FieldOptions proto: const FieldDescriptor* f = ...; if (f-options().packed()) { // field is packed } On Mon, Apr 26, 2010 at 2:29 PM, adamdms adam.kwintkiew...@gmail.comwrote: how to check whether the field / expansion (FieldDescriptor *) is repeated

Re: [protobuf] Seg Fault with MutableRepeatedMessage()

2010-05-04 Thread Jason Hsueh
Your msg is completely empty. There aren't any entries in the repeated field - FieldSize() should be returning 0. Your loop needs to terminate when i size, not i = size - accessing the 1st element of an empty repeated field is going to return an invalid pointer. FYI bounds checking is done in

Re: [protobuf] Avoid attributes/ Cyclic support

2010-05-04 Thread Jason Hsueh
On Mon, May 3, 2010 at 11:29 PM, Fabio Maulo fabioma...@gmail.com wrote: - Is there a way to avoid attributes ? I mean something like an internal metadata where we can define the identifier of each member. I'm not sure what you mean by this. By attributes do you mean field options? Can you

Re: [protobuf] Re: Seg Fault with MutableRepeatedMessage()

2010-05-04 Thread Jason Hsueh
a fundamental step here =) Thanks for the help! -Ryan On May 4, 12:23 pm, Jason Hsueh jas...@google.com wrote: Your msg is completely empty. There aren't any entries in the repeated field - FieldSize() should be returning 0. Your loop needs to terminate when i size, not i = size

Re: [protobuf] protobuf CXF RPC impl

2010-05-05 Thread Jason Hsueh
Added to the list, thanks! On Wed, May 5, 2010 at 7:46 AM, orbang gyorgy.or...@gmail.com wrote: Hi all, I've created a pluggable protobuf RPC impl based on Apache CXF: http://code.google.com/p/cxf-protobuf/ Could you please list it on the

Re: [protobuf] Detect type of message

2010-05-14 Thread Jason Hsueh
Check out the docs: http://code.google.com/apis/protocolbuffers/docs/techniques.html#union On Fri, May 14, 2010 at 12:45 PM, Vince vincedup...@gmail.com wrote: Hi, If I want to serialize many message in the same pipe, do I need one master message that includes all the others in optional

Re: [protobuf] Nested messages accessor methods

2010-05-14 Thread Jason Hsueh
Yes, you can. What is your target language? The accessor names are generated differently - in C++ you would get lower-cased names. Have you looked at the generated header? You may also want to see http://code.google.com/apis/protocolbuffers/docs/style.html On Fri, May 14, 2010 at 1:29 AM, Jacques

Re: [protobuf] Can I add a new item into an existing list of enumerations safely?

2010-05-14 Thread Jason Hsueh
Clients with the old binary will not be able to read messages containing the new enum value. This would get treated as an unknown field, meaning that the type field would not be sent in an older client. Since your field is also required, it would fail to parse. So, with this example, you would

Re: [protobuf] How to get rid of metadata only extensions dependencies

2010-05-14 Thread Jason Hsueh
No, there isn't any way to remove the dependency. The compiler doesn't try to detect why you defined an import. On Fri, May 14, 2010 at 2:38 PM, Igor Gatis igorga...@gmail.com wrote: So, let's say I'm developing a plugin which, say, add or removes builder pattern to java generator (this is

Re: [protobuf] Re: Can I add a new item into an existing list of enumerations safely?

2010-05-14 Thread Jason Hsueh
... ). So, later we have: enum Type { POP = 0, PUSH = 1, TOP = 2, TAIL = 3, UNKNOWN = 100 } and so on ... Or it really does not matter? Is this the recommended approach for this pattern? Or should I be using extends? -john On May 14, 3:15 pm, Jason Hsueh jas...@google.com wrote

Re: [protobuf] Re: Can I add a new item into an existing list of enumerations safely?

2010-05-14 Thread Jason Hsueh
extensions; I think extensions become smaller when you have 8 fields or so. On Fri, May 14, 2010 at 5:44 PM, Jason Hsueh jas...@google.com wrote: It doesn't matter that much. I only set UNKNOWN = 2 on the assumption that pop and push were already assigned and you needed to preserve wire

Re: [protobuf] Deadlock problems with protobuf static initialization in Java

2010-05-16 Thread Jason Hsueh
I think this is http://b/issue?id=2267627 On Sat, May 15, 2010 at 10:02 PM, Igor Gatis igorga...@gmail.com wrote: Agreed. So I guess I need to start looking for the deadlock in my code. Do you know any tool to help me out on that? On Sun, May 16, 2010 at 1:38 AM, Christopher Smith

Re: [protobuf] Deadlock problems with protobuf static initialization in Java

2010-05-16 Thread Jason Hsueh
Err. Got my mailing lists mixed up. I think this is due to http://bugs.sun.com/view_bug.do?bug_id=6501158 The suggested workaround has been to force initialization of your classes in main, before threads are spawned. On Sun, May 16, 2010 at 1:18 AM, Jason Hsueh jas...@google.com wrote: I think

Re: [protobuf] systesting custom utf8 validation on remote c++ node using protocol buffers from python

2010-05-17 Thread Jason Hsueh
If you compile with the macro GOOGLE_PROTOBUF_UTF8_VALIDATION_ENABLED defined, the C++ code will do UTF8 validation. However, it doesn't prevent the data from serializing or parsing, it will simply log an error message. How would you like it to fail? On Mon, May 17, 2010 at 3:15 PM, JT Olds

Re: [protobuf] Protocol buffers and large data sets

2010-05-17 Thread Jason Hsueh
There is a default byte size limit of 64MB when parsing protocol buffers - if a message is larger than that, it will fail to parse. This can be configured if you really need to parse larger messages, but it is generally not recommended. Additionally, ByteSize() returns a 32-bit integer, so there's

Re: [protobuf] Implementing protobuf in symbian

2010-06-01 Thread Jason Hsueh
Can you provide more details? What commands are you running, and what are your errors? In your previous thread it sounded like you were using C++, but now you mention Java - which language are you trying to use? Also, are you attempting to use the lite library or the full library? It seems most

Re: [protobuf] Protoc error on Linux Mint (protoc-gen-ccp: program not found)

2010-06-02 Thread Jason Hsueh
On Wed, Jun 2, 2010 at 5:31 AM, mark.t.macdon...@googlemail.com mark.t.macdon...@googlemail.com wrote: Hi, I'm running Linux Mint, and trying to get started with protobuf-2.3.0. My install was as follows (no problems reported that I could see): ./configure --prefix=/usr make clean make

Re: [protobuf] using the new plugin infrastructure

2010-06-02 Thread Jason Hsueh
Does the test plugin help? There is a test code generator that is used in the unittest for command_line_interface.cc: http://code.google.com/p/protobuf/source/browse/trunk/src/google/protobuf/compiler/mock_code_generator.cc On Tue, Jun 1, 2010 at 4:00 AM, marcmo oliver.muel...@gmail.com wrote:

Re: [protobuf] Questions/Ideas about Protobuf

2010-06-02 Thread Jason Hsueh
On Tue, Jun 1, 2010 at 6:21 AM, bnh baoneng...@gmail.com wrote: I'm using a protobuf as the protocol for a distributed system.But now I have some questions about protobuf a.Whether protobuf provides the inteface for user-defined allocator because sometimes I find 'malloc' cost too much?

Re: [protobuf] Re: Questions/Ideas about Protobuf

2010-06-03 Thread Jason Hsueh
,I can only hold 10,000 pageids 4.so the server app knows it,so app just serialize 10,000 pageids into memory instead of 1,000,000 pageids. I hope I clarify it now.. If the protobuf doesn't implement it, do you have any idea about it?. On Jun 3, 12:40 am, Jason Hsueh jas...@google.com wrote

Re: [protobuf] Re: Questions/Ideas about Protobuf

2010-06-03 Thread Jason Hsueh
is based on protobuf. On the top,my app faces a lot of small stream packets in protobuf] Linkedin:http://www.linkedin.com/in/dirlt 于 2010/6/4 0:21, Jason Hsueh 写道: This really needs to be handled in the application since protobuf has no idea which fields are expendable or can be truncated

Re: [protobuf] What is the default value for a message used as a field type within another message?

2010-06-04 Thread Jason Hsueh
No, protobuf accessors never return null. For message fields, you get the default instance of the message type. On Fri, Jun 4, 2010 at 9:32 AM, Jax jax...@gmail.com wrote: This should be pretty straightforward, but I couldn't find it in any of the documentation on default values. A slight

Re: [protobuf] Options extensions for dynamic messages?

2010-06-07 Thread Jason Hsueh
Hm. Can you send the code snippet you are using to try to access your custom option? You should just be accessing the options proto from FieldDescriptor::options(), and calling GetExtension() on that. Also, UnknownFields doesn't have anything to do with options extensions. Did you mean

Re: [protobuf] Options extensions for dynamic messages?

2010-06-08 Thread Jason Hsueh
. Rather convoluted, but we don't have a better way. On Mon, Jun 7, 2010 at 3:25 PM, Jason Hsueh jas...@google.com wrote: Hm. Can you send the code snippet you are using to try to access your custom option? You should just be accessing the options proto from FieldDescriptor::options(), and calling

Re: [protobuf] question about SerializeToString output for cpp

2010-06-11 Thread Jason Hsueh
Yes, '\0' may appear in the binary format. output-size() will return the correct result: C++ string can store null characters without any issue. However, strlen(output-c_str()) and other calls that assume null-terminated C strings will not. On Thu, Jun 10, 2010 at 11:30 PM, mistlike

Re: [protobuf] Python protobuf speed and latest Protobuf version

2010-06-18 Thread Jason Hsueh
The last release, 2.3.0, includes significant improvments to Python: http://protobuf.googlecode.com/svn/trunk/CHANGES.txt There have also been further improvements since then (the C extension-based implementation mentioned in

Re: [protobuf] Way to figure out the fields that are modified

2010-06-21 Thread Jason Hsueh
On Mon, Jun 21, 2010 at 8:23 AM, Ananth Arockia ananth.aroc...@gmail.comwrote: Hello, I am exploring protobuf to use it in my project. I would like to have a message defined in a .proto file. All the fields in this message would be optional due to the fact that the message producer in my

Re: [protobuf] Re: Java: Deleting from lists in protobufs

2010-06-23 Thread Jason Hsueh
On Wed, Jun 23, 2010 at 3:01 PM, Dheeraj Pandey dhee...@alumni.utexas.netwrote: Java doesn't have such a thing. Is this by design? If yes, would love to understand the rationale of why clearing-everything-and-reinserting-(N-1)-values is so much better than: (a) providing an API to do so, and

Re: [protobuf] Re: using the new plugin infrastructure

2010-06-30 Thread Jason Hsueh
Yes, you can write the plug-in in the language of your choice. On Wed, Jun 30, 2010 at 7:08 AM, qrilka qri...@gmail.com wrote: Is it possible to use other languages? E.g. python? Kind regards, Kirill Zaborsky On 2 июн, 20:25, Jason Hsueh jas...@google.com wrote: Does the test plugin

Re: [protobuf] custom protocol

2010-06-30 Thread Jason Hsueh
The library provides a standard text format (TextFormat classes in C++ and Java, text_format.py for python). If that doesn't work for you, you could write your own serializers around reflection, much like TextFormat. On Wed, Jun 30, 2010 at 7:05 AM, Vladimir Danishevsky danishev...@gmail.com

Re: [protobuf] hasXXX methods

2010-07-02 Thread Jason Hsueh
See http://code.google.com/apis/protocolbuffers/docs/cpptutorial.html under the section The Protocol Buffer API On Fri, Jul 2, 2010 at 11:08 AM, jd unicom...@gmail.com wrote: Where are the hasXXX methods that are available in C++? -- You received this message because you are subscribed to

Re: [protobuf] Why do field default values not get serialised?

2010-07-14 Thread Jason Hsueh
On Wed, Jul 14, 2010 at 1:47 PM, mark.t.macdon...@googlemail.com mark.t.macdon...@googlemail.com wrote: I am new and want to understand the purpose/usage of default values in the .proto file. E.g: test { optional string a 1 [default =test]; } When I create a message object and read the

Re: [protobuf] Protobuf as generic marshalling layer

2010-07-14 Thread Jason Hsueh
On Tue, Jul 13, 2010 at 1:38 AM, Galder Zamarreno market...@zamarreno.comwrote: Hi, I want to use protobuf as a generic marshalling mechanism and I'm having some issues with the unmarshalling part, because I need knowledge of what needs to be unmarshalled, i.e. in Java, access to static

Re: [protobuf] Building FileDescriptorSet out of FileDescriptor

2010-07-21 Thread Jason Hsueh
Ordering is important because any dependencies need to be built before they are used - if you give an arbitrary order there may be errors complaining about undefined symbols or imports not being found. A depth first ordering will work, and that's exactly what protoc does when producing

Re: [protobuf] Building FileDescriptorSet out of FileDescriptor

2010-07-22 Thread Jason Hsueh
. On Wed, Jul 21, 2010 at 11:38 PM, Nader Salehi sal...@alumni.usc.eduwrote: If any dependency needs to be build beforehand, then how is topological ordering is supported? On 7/21/2010 17:39 Jason Hsueh writes: Ordering is important because any dependencies need to be built before

Re: [protobuf] Extensions in Java

2010-07-27 Thread Jason Hsueh
The C++ implementation does have a singleton registry, which allows it to magically parse extensions that are linked into the binary. This behavior can be somewhat confusing though since it relies on what the linker decides to do with generated code. The Java implementation uses an object oriented

Re: [protobuf] why ParseFromIstream using std::cin always returns true using a pipe?

2010-07-28 Thread Jason Hsueh
Messages without any required fields are allowed to have an empty serialization, so the library cannot assume that parsing empty input is an error. You just need to test cin.eof() separately. On Wed, Jul 28, 2010 at 7:18 AM, jetcube pmlo...@gmail.com wrote: I have a simple app that received

Re: [protobuf] Gzip

2010-07-29 Thread Jason Hsueh
What exactly do you want to know about them? There are various details about how they adapt zlib to the ZeroCopy*Stream interfaces, but as a client you shouldn't need to worry about that. You can use them just like any other ZeroCopyStream. On Thu, Jul 29, 2010 at 8:58 AM, Julian González

Re: [protobuf] Gzip

2010-07-29 Thread Jason Hsueh
but first I got an error telling me zlib.h could not be found and I was also wondering if I can use GzipOutputStream along with CodedOutputStream? El 29 de julio de 2010 14:59, Jason Hsueh jas...@google.com escribió: What exactly do you want to know about them? There are various details about

Re: [protobuf] libcurl integrtion

2010-07-30 Thread Jason Hsueh
Not at all familiar with libcurl, but from lookng at man pages... On Wed, Jul 28, 2010 at 6:56 PM, Birch moa...@gmail.com wrote: I am not well versed in c++, just a disclamer. I have muddled through creating a small client and I want to post the binary output via HTTP. Here is the source I

Re: [protobuf] ByteString example

2010-07-30 Thread Jason Hsueh
What are you trying to do? Does the Javadoc help? http://code.google.com/apis/protocolbuffers/docs/reference/java/com/google/protobuf/ByteString.html On Fri, Jul 30, 2010 at 8:07 AM, David david.kum...@gmail.com wrote: Hi, I am a bit ashamed to ask this, but does anyone know how to use

Re: [protobuf] Please add j2me implementation in ThirdPartAddOns page

2010-08-03 Thread Jason Hsueh
Added to the wiki. On Thu, Jul 29, 2010 at 7:45 PM, Igor Gatis igorga...@gmail.com wrote: Here it is: http://code.google.com/p/protobuf-j2me/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to

Re: [protobuf] Re: Java implementation questions

2010-08-05 Thread Jason Hsueh
Groups are primarily deprecated because it used to be the case that groups could not be reused in other types. Now groups can be reused, but other messages must use the length-delimited format, rather than the group format. In theory, the fact that groups do not require a length prefix makes them

Re: [protobuf] How to use SetString reflection method

2010-08-06 Thread Jason Hsueh
The setters need a (non-const) pointer to the message object. It appears fractal_handle is a reference since the getter, which takes a const reference to a message, compiles. On Fri, Aug 6, 2010 at 4:42 AM, mark.t.macdon...@googlemail.com mark.t.macdon...@googlemail.com wrote: Can you please

Re: [protobuf] Scala protocol buffers protoc plugin

2010-08-11 Thread Jason Hsueh
Added to the wiki, thanks! On Wed, Aug 11, 2010 at 12:57 AM, Jeff Plaisance jeffplaisa...@gmail.comwrote: Hello, I have written a compiler plugin that generates type safe scala wrappers for the java protoc output. It creates wrappers for each message type that use the scala Option type for

Re: [protobuf] How to retrieve parameters using tag numbers using CodedInputStream

2010-08-11 Thread Jason Hsueh
Suppose those strings are all defined as optional fields in the message. They may not be present in the serialization, so the decoder needs to read a tag to determine the field number and type before handling the value. On Wed, Aug 11, 2010 at 8:37 AM, Prakash Rao prakashrao1...@gmail.comwrote:

Re: [protobuf] Re: flushing the data

2010-08-11 Thread Jason Hsueh
ParseFromFileDescriptor will read until there is no more input. This is consistent with behavior when parsing from other types of input streams, as the message wire format is not self delimiting. For file descriptors, that means you have to reach EOF. As such, ParseFromFileDescriptor is only

Re: [protobuf] Re: beginner question about extensions

2010-08-11 Thread Jason Hsueh
Is this in a loop? It appears there are two occurrences of that log line, and one of them has a completely empty message. On Wed, Aug 11, 2010 at 4:44 PM, Jun8 ctaski...@gmail.com wrote: libprotobuf ERROR google/protobuf/message_lite.cc:123] Can't parse message of type myevents.Event because

  1   2   3   >