Thanks Jason, I have implemented my system following your suggestion.
On Aug 11, 5:36 pm, Jason Hsueh <jas...@google.com> wrote: > ParseFromFileDescriptor will read until there is no more input. This is > consistent with behavior when parsing from other types of input streams, as > the message wire format is not self delimiting. For file descriptors, that > means you have to reach EOF. As such, ParseFromFileDescriptor is only useful > when the whole file contains a single message. > > One approach to writing multiple messages to the same stream is to use a > length-delimited format: write the size of the message, then serialize the > message itself. On the receiver side, you would set up a FileInputStream, > and wrap a CodedInputStream around that. You can read the size of the > messages from the stream and then use PushLimit and PopLimit to control how > much data is read. > > On Wed, Aug 11, 2010 at 2:19 PM, nirajshr <niraj...@gmail.com> wrote: > > Hi Kenton, > > > I really appreciate your quick response. As you mentioned in your > > response, the protobuf library does write all the data to the file > > descriptor before returning from SerializeToFileDescriptor(). But, I > > have to explicitly close the file descriptor (using close(fd) ) before > > the data gets flushed. As a result, ParseFromFileDescriptor keeps > > waiting for the input until the file descriptor has been closed. > > > I tried breaking this process into two parts to see if the problem > > persists: > > In the first program, > > 1) Firstly, I used the SerializeToString() function to write to a > > string, > > 2) Then, I wrote the string to a file descriptor using the > > write(char*, buffer length) function. > > > Then, I tested the following two ways to receive the data at the other > > end of the socket: > > In second program, > > 1) I used ParseFromFileDescriptor() function to receive the data. The > > problem persisted in this case. The first program had to call > > close(fd) in order for the second program to receive the data. > > > 2) Alternatively, I used read(char* str, buff_len) & > > ParseFromString() functions to receive the data. In this case, the > > program immediately received the data, without having to close the > > file descriptor in the first program. > > > I know that this issue might be outside the scope of google protocol > > buffers. But, this contradictory results show that the > > ParseFromFileDescriptor() function expects an explicit close of the > > filedescriptor before it can retrieve all the data. I guess this makes > > sense as ParseFromFileDescriptor() function has no way of knowing the > > length of the data. Having to close the file descriptor as a way to > > mark the end of the transmission of data is confusing. The > > SerializeToFileDescriptor() function has to take care of this so that > > ParseFromFileDescriptor() function has an idea about the size of data > > being received. > > > I am sorry to bother you with this long post, but, your insight will > > be very helpful. > > > On Aug 10, 5:40 pm, Kenton Varda <ken...@google.com> wrote: > > > The protobuf library writes all data to the file descriptor before > > returning > > > from SerializeToFileDescriptor(). So, the problem is not in protobufs. > > > Sorry. > > > > On Tue, Aug 10, 2010 at 10:58 AM, nirajshr <niraj...@gmail.com> wrote: > > > > > Particularly, I am using the SerializeToFileDescriptor() function to > > > > send the serialized data over a socket interface. The problem is that > > > > the data does not get sent before I explicitly close the socket using > > > > close(fileDescriptor). I even tried letting the google protocol buffer > > > > object go out of scope (hoping the destructor would be called, and > > > > hence flushing of the data). I am sending large amounts of data > > > > (around 1MB). I did not face this problem while sending smaller amount > > > > of data. > > > > > Isn't there any way I can force flushing of data without destroying > > > > the google protocol buffer object? > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Protocol Buffers" group. > > > > To post to this group, send email to proto...@googlegroups.com. > > > > To unsubscribe from this group, send email to > > > > protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.com> > > <protobuf%2bunsubscr...@googlegroups.com<protobuf%252bunsubscr...@googlegroups.com> > > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/protobuf?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Protocol Buffers" group. > > To post to this group, send email to proto...@googlegroups.com. > > To unsubscribe from this group, send email to > > protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
