Re: [protobuf] message forward declaration

2011-01-21 Thread Jason Hsueh 
There are many things that need to be read from imported .proto files to
determine if the .proto is valid, or to produce correct code. e.g.:
- need to differentiate between enum and message imports
- when referencing a qualified type like foo.bar.Baz.Qux, you need to know
what components are package specifiers, and which are objects. This changes
the forward declaration scheme: if that was package foo.bar; message Bar {
message Qux { } }, the declaration is namespace foo { namespace bar { class
Bar_Qux; } }, whereas maybe Bar is just a namespace, so you should instead
produce namespace foo { namespace bar { namespace Bar { class Qux; } } }
- options like java_multiple_files affects the generated code
- when defining extensions, you need to know whether the extendee accepts
extensions, and what range of extension numbers the extendee allows

To use the .proto, you need to build the imported proto files anyway, so
even if you could capture all of this in a simple forward-declaration-type
scheme, I don't think you would save very much.

On Wed, Jan 19, 2011 at 9:02 AM, George  wrote:

> Hi,
>
> It looks to me that the generated result of a single proto file
> doesn’t change significantly based on the content of the imported
> proto files, but mostly on the fact that it is imported.
>
> Did you have considered replacing the need of actual import with some
> kind of message forward declaration?
>
> In my opinion having a single proto file self-sufficient even in case
> it actually refers messages from other proto files could significantly
> simplify the build process.
>
> What the protobuf community thinks about this?
>
> Thanks,
> George
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To post to this group, send email to protobuf@googlegroups.com.
> To unsubscribe from this group, send email to
> protobuf+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/protobuf?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.

Re: [protobuf] Problem with accents in python while unpacking a message

2011-01-21 Thread Jason Hsueh 
Can you provide a small, self-contained reproduction of the program?

On Thu, Jan 20, 2011 at 10:39 AM, Louhike  wrote:

> Hi,
> I'm using Google Protobuf with python on a project.
> My problem is I get an error while my program tries to build an
> instance with the function google.protobuf.text_format.merge() if the
> message contains accents (“utf-8 can't decode the byte \xe9” with the
> character 'é' for example).
> I need to keep the accents but I don’t find a solution  to do it. It
> may seem simple but I’m in an early learning phase of programming and
> I'm often stuck on little things like that. Any help would be useful.
>
> Thanks,
> Louhike
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To post to this group, send email to protobuf@googlegroups.com.
> To unsubscribe from this group, send email to
> protobuf+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/protobuf?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.

Re: [protobuf] java_multiple_files option

2011-01-21 Thread Jason Hsueh 
This contains file-level code. In addition to any file-scope extensions you
may have defined, it provides the file's descriptor and initialization code
for the .proto file.

On Thu, Jan 20, 2011 at 2:20 AM, Antoine DESSAIGNE <
antoine.dessai...@gmail.com> wrote:

> Hi everyone,
>
> First I want to say thank you for creating Protobuf, it truely makes my
> life easier. And I'm always thankful for that kind of thing :)
>
> I'm currently going deeper and deeper in Protobuf in order to learn how to
> use it more efficiently.
>
> I found the "java_multiple_files" option that is really great but I was
> wondering why it generates an extra class named after the .proto file. What
> is the use of this class?
>
> Thanks a lot,
>
> Antoine.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To post to this group, send email to protobuf@googlegroups.com.
> To unsubscribe from this group, send email to
> protobuf+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/protobuf?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.

Re: [protobuf] Dealing with Corrupted Protocol Buffers

2011-01-21 Thread Jason Hsueh 
It will be rather difficult to correct for the error. The point at which the
parse fails may not be the point of corruption: e.g., the corruption may be
in a byte that is part of a varint, and the continuation bit may be set when
it shouldn't. Similarly you could have a corruption in the length delimiter
for a string or nested message field. Both could cause you to read more
bytes than you should have for that particular field. The encoding is dense
enough that the parser may merrily consume more bytes before encountering an
error to complain about.

You can try to mess with the bytes; you might be able to deal with errors
using some assumptions about the serialized data based on your protocol. But
in general, and going forward, you should write small messages in a
container format that allows for error recovery. Various threads from this
search
discuss
this issue.

On Thu, Jan 20, 2011 at 7:11 PM, Julius Schorzman  wrote:

> Thanks for the tip on CodedInputStream Evan!   I will explore it and
> if I get anything out of it will report back my findings for anyone
> else dealing with this issue.
>
> On Thu, Jan 20, 2011 at 6:27 PM, Evan Jones  wrote:
> > On Jan 20, 2011, at 2:48 , julius-schorzman wrote:
> >>
> >> My question is -- can anything be done to retrieve part of the file?
> >> It would be nice to know at which point in the file the problematic
> >> message occurred, and then I could crop to that point or do some
> >> manual exception -- but unfortunately this exception is very general.
> >> I find it hard to believe that a single mis-saved bit makes the whole
> >> file worthless.
> >
> > You are correct: your entire data is not worthless, but at the point of
> the
> > error, you will need some manual intervention to figure out what is going
> > on.
> >
> > It is probably possible to figure out the byte offset where this error
> > occurs. The CodedInputStream tracks some sort of bytesRead counter, I
> seem
> > to recall. However, this will require you to modify the source.
> >
> >
> >> I also find it curious that the source provides no way (that I can
> >> tell) to get at any lower level data in the p.b. since whenever I try
> >> to do anything with it it throws an exception.  Best I can tell I will
> >> have to write from scratch my own code to decode the p.b. file.
> >
> > The lowest level tools that are provided is CodedInputStream. But yes,
> you
> > will effectively have to "parse" the message yourself. Look at the code
> that
> > is generated for the mergeFrom method of your message to get an idea for
> how
> > it works, and you can read the encoding documentation:
> >
> > http://code.google.com/apis/protocolbuffers/docs/encoding.html
> >
> > You can definitely figure out what is going on, but it will be a bit of a
> > pain. Good luck,
> >
> > Evan Jones
> >
> > --
> > http://evanjones.ca/
> >
> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To post to this group, send email to protobuf@googlegroups.com.
> To unsubscribe from this group, send email to
> protobuf+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/protobuf?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.

Re: [protobuf] cannot get extensions from serialized proto -- please help

2011-01-21 Thread Jason Hsueh 
You didn't provide a code snippet, so it's hard to say, but I would guess
that you need to provide an ExtensionRegistry and pass that to the parsing
method. (It looks like you are working in Java)

e.g.
ExtensionRegistry extensionRegistry = ExtensionRegistry.newInstance();
YourFileContainerProto.registerAllExtensions(extensionRegistry);
Results r = Results.parseFrom(data, extensionRegistry);

On Fri, Jan 21, 2011 at 8:39 AM, hp  wrote:

> test below is failing..please please help
>
> message Results {
>  extensions 100 to max
>
> }
>
>
> message Item1 {
>  extend Resultes {
>repeated Item1 items = 100;
>}
> }
>
>
> message Item2 {
>  extend Resultes {
>repeated Item2 items = 100;
>}
> }
>
>
> Results.getExtension(items) does not return anything when converted to
> and from byte array.. Works fine otherwise
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To post to this group, send email to protobuf@googlegroups.com.
> To unsubscribe from this group, send email to
> protobuf+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/protobuf?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.

[protobuf] cannot get extensions from serialized proto -- please help

2011-01-21 Thread hp 
test below is failing..please please help

message Results {
  extensions 100 to max

}


message Item1 {
 extend Resultes {
repeated Item1 items = 100;
}
}


message Item2 {
 extend Resultes {
repeated Item2 items = 100;
}
}


Results.getExtension(items) does not return anything when converted to
and from byte array.. Works fine otherwise

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.