Re: [protobuf] Re: ProtocolBuffer repeated tasks in Java..
On Sat, Dec 12, 2009 at 11:21 AM, atkretsch atkret...@gmail.com wrote:
You can reuse the builder object by clearing it after each iteration
like so:
No you can't. This coincidentally appeared to work in earlier versions but
it was never supposed to work according to the docs. In 2.2.0 it throws an
exception. We were actually going to introduce an optimization to
automatically freelist the builders, which would have possibly lead to data
corruption for people who tried to clear and reuse builders illegally. We
ended up giving up on the optimization for other reasons.
Person.Builder builder = Person.newBuilder();
for(...) {
builder.clear();
builder.setFname(...);
builder.setLName(...);
builder.setEmail(...)
myResponseBuilder.addPerson(builder.build());
}
This is a bit more efficient since you're only creating one new object
per iteration instead of two. If you're creating a lot of these
objects or this loop is in a critical path, this could be a decent
speedup.
On Dec 11, 12:35 pm, Programmer 09 macster2...@gmail.com wrote:
I am new to ProtocolBuffer and wanted to verify that I am doing the
right thing here:
I have proto definitions as:
Person {
required string fname =1;
required string lname = 2;
required string email = 3;
}
MyResponse {
repeated Person persons = 1;
}
What is the most effecient way to generate a myResponse with 100
persons? Is this effecieint?
// assume I have a array of hashmap of person data.
// there is the myResponseBuilder which is a builder for MyResponse.
for (int i = 0; i 100; i++)
Person.Builder personBuilder = Person.newBuilder();
personBuilder.setFname(nameFromMap);
personBuilder.setLname(nameFromMap);
personBuilder.setEmail(emailFromMap);
myResponseBuilder.addPerson(personBuilder);
}
Is this the right way to do this? Or is there a better effecient way
instead of creating a builder for each person data? Any pointers
appreciated! Thanks!
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Re: protobuf rpc over http
If you're interested I have added explanatory comments to the socket-level protocol definition for Captain Proto (formerly pbcap): http://code.google.com/p/capnproto/source/browse/proto/capnproto.proto I still have no conventions for HTTP transport. On Thu, Dec 10, 2009 at 3:33 PM, Kenton Varda ken...@google.com wrote: On Thu, Dec 10, 2009 at 3:02 PM, Mikhail Opletayev opleta...@gmail.comwrote: Will there be a way to directly connect to the service or a client must go thru the discovery service each time a connection is open? Why build this into the protocol when the discovery service is just as good? Note that the protocol supports sending a request to the returned service before actually waiting for it to be returned. In other words, a conversation might look like: client: Open service Foo. client: When request 1 is complete, call method Bar() on the result. server: Foo opened successfully. Here is a reference. server: Bar completed. Here is the response. By keeping service naming out of the low-level protocol, we keep it nice and simple, which makes it easier to write a high-quality implementation. I also intend to provide a standard discovery service interface, but it will be a layer that sits on top of the raw protocol implementation, rather than being built into it. Applications will be able to use the raw protocol without using the standard discovery service if desired. What do you mean? The global Stream message exists only to define the protocol. It is not actually used at runtime -- individual messages are read from the stream one at a time. From pbcap.proto it looked to me as all requests and responses must be wrapped inside a Stream message. I guess I'll need to take a look at the implementation before I ask more questions, I only had time to take a quick peak earlier today. Maybe what you don't like is the fact that the app-specific request message is embedded as bytes in pbcap.CallRequest? Currently this requires a reduntant copy of all those bytes. However, I was thinking of adding an optimization to the protobuf implementation to avoid this -- we could add a way to construct a new ByteString that is a substring of some other ByteString in constant time (with no copies). Then, parsing bytes fields -- when the original input is a ByteString -- would not require a copy, making things efficient. P.S. I hope my questions don't derail this thread too much as the original question was about HTTP RPC. On Dec 10, 4:42 pm, Kenton Varda ken...@google.com wrote: On Thu, Dec 10, 2009 at 2:37 PM, Mikhail Opletayev opleta...@gmail.com wrote: Interesting. Essentially a discovery service for protobuf RPC. I am not quite sure what you mean by pointers to other services. Is it going to reference them by name or a more complex structure containing full endpoint information? Currently it references them by an ID number that is tied to the particular connection. So, each time a new service object is returned on the connection, a new ID number is assigned to it. Also, is it going to be an extension to pbcap or something completely new? Not an extension -- this *is* pbcap. It supports this already. (Note that I'm planning to change the name to Captain Proto, aka capnproto, to avoid the confusion with pcap.) The reason I am asking is because some patterns in pbcap (such as wrapping up everything into a global Stream message) are rather questionable and not without consequences. What do you mean? The global Stream message exists only to define the protocol. It is not actually used at runtime -- individual messages are read from the stream one at a time. Regards, On Dec 10, 4:22 pm, Kenton Varda ken...@google.com wrote: On Thu, Dec 10, 2009 at 8:13 AM, Mikhail Opletayev opleta...@gmail.com wrote: It's great news that you working on a standard way to communicate between Protocol Buffers implementations! You don't need to send the service name at all. The server should already know what kind of service it is exporting. I think its handy to export several services from the same end point, especially if you are running RPC over something else than HTTP. If you were to run Protocol Buffers RPC over plain sockets you'd probably want to publish a bunch of services on the same port. This is exactly my point. If you use the service type name to identify the service, then you can only export one service of each type. Instead, some other name -- having nothing to do with the type name -- should be used to identify the service. Actually, the implementation I'm working on doesn't even identify services by names. Instead, when you first connect on a port, you connect to the default service for that port. However, the default service can send back
Re: [protobuf] protocol message was rejected because it was too big (Turn off?)
Thanks much. Regards Saptarshi On Sat, Dec 12, 2009 at 4:40 PM, Kenton Varda ken...@google.com wrote: Actually, this is a better link: http://code.google.com/apis/protocolbuffers/docs/reference/cpp/google.protobuf.common.html#SetLogHandler.details For some reason the auto-generated documentation is failing to hyperlink to the definition of LogHandler, but it is shown among the typedefs at the top of that page: typedef void LogHandler(LogLevel level, const char *filename, int line, const string message) On Sat, Dec 12, 2009 at 1:36 PM, Kenton Varda ken...@google.com wrote: All messages are written to stderr (not stdout), which is usually reserved for human-readable error messages. However, you can redirect the messages using google::protobuf::SetLogHandler() as documented here: http://code.google.com/apis/protocolbuffers/docs/reference/cpp/google.protobuf.common.html#SetLogHandler On Sat, Dec 12, 2009 at 10:50 AM, Saptarshi Guha saptarshi.g...@gmail.com wrote: Hello, I am using Protocol Buffers to serialize some data. To begin with I do realize that I shouldn't be using PB for serializing very large messages, but given that I am, I have to deal with these messages. E.g I have a message of 381MB, so naturally I get this error when parsing: libprotobuf ERROR google/protobuf/io/coded_stream.cc:196] A protocol message was rejected because it was too big (more than 67108864 bytes). To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h. I viewed the header file and see what I have to do next. I'll fix my code soon, till then: My program redirects standard error(and output) and re-encodes functions that write to these streams. However writing to s.out and s.err are through special functions. PB, does not use my functions. Other libraries (and I only use PB) writing to s.err and s.out can adversely affect my program. Q. Is there a flag I can set to not display the warning? i.e silently fail? I'm not using CodedInputstream, instead I use ParseFromArray (i have read in the bytes with m own functions) Regards Saptarshi -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
