[protobuf] How to import .proto file in different package correctly?

[Alexander Luya]

I have two packages like this

com.abc.
 protobuf.
share.proto
 depart.
detect.proto 

and the conent of  share.proto like this:

syntax = "proto3";
package com.adc.protobuf;
message Test{}
   
and the content of detect.proto like this:

syntax = "proto3";
package com.adc.depart;
import "com/abc/protobuf/share.proto"

and compile share.proto in it's dir like this:

protoc -I=. --python_out=. share.proto

then compile detect.proto in it's dir like this:

protoc -I=/pathToSrcDir/ -I=. --python_out=. detect.proto 
and

pathToSrcDir has been added to PYTHONPATH

all compilations work fine,but when run a python script which 

from com.abc.depart import detect_pb2

got this error   

TypeError: Couldn't build proto file into descriptor pool!
Invalid proto descriptor for file "detect.proto":
  detect.proto: Import "com/abc/protobuf/share.proto" has not been 
loaded.
  com.abc.depert.XClass.ymethod: "com.abc.protobuf.Test" seems to be 
defined in "share.proto", which is not imported by "detect.proto".  To use 
it here, please add the necessary import.

How to solve this import problem?

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Python CPP Implementation

[Matthew Heritage]

Apologies - here is the link to the docs, referring to the Python C++ as 
experimental:
https://developers.google.com/protocol-buffers/docs/reference/python-generated#cpp_impl
Scroll down to the bottom:
"There is also an **experimental** C++ implementation for Python messages via a 
Python extension for better performance. Implementation type is controlled by 
an environment variable PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION (valid values: 
"cpp" and "python"). The default value is currently "python" but will be 
changed to "cpp" in future release.
Note that the environment variable needs to be set before installing the 
protobuf library, in order to build and install the python extension. The C++ 
implementation also requires CPython platforms. See python/INSTALL.txt for 
detailed install instructions."
This is a test disclaimer.

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Python CPP Implementation

['Feng Xiao' via Protocol Buffers]

On Mon, Jan 15, 2018 at 4:34 AM, Matthew Heritage <
matthew.c.herit...@gmail.com> wrote:

> According to the docs, the C++ Python implementation is experimental, but
> this has been the case in the docs for 7 years.
> Is it really still experimental or are the docs out of date?
>
A link to the said doc? The doc needs update. Python C++ support is not
experimental any more and has been used in production both inside/outside
Google. +jieluo


> Is there a date when the default Python implementation will be switched to
> C++ from Python? The performance difference is huge.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] decode does not work on client side!. illegal buffer

['Bo Yang' via Protocol Buffers]

Our js protobuf cannnot decode json format. Ours can only decode binary
format AFAIK.

On Tue, Jan 16, 2018 at 9:35 AM  wrote:

> my question is do you have a procedure to decode protobuf on client side.
>
>
> On Tuesday, January 16, 2018 at 11:33:08 AM UTC-6, Bo Yang wrote:
>
>> Sorry, protobuf.js is not maintained by us. Please contact the owner.
>>
>> On Tue, Jan 16, 2018 at 9:21 AM  wrote:
>>
> protobuf.js version: 6.8.4
>>>
>>> I am using Django-channels in the backed and javascript WebSockets on
>>> the front end. I have a set of 10 .proto files which import one or more of
>>> these .proto files. I was able to serialize my JSON on the server side,
>>> send data over the wire and receive base64 encoded message. I am having
>>> difficulty trying to convert this message to JSON.
>>>
>>> I have encountered following problems.
>>>
>>>- if I convert my actual data to the uint8array buffer and pass it
>>>into decode: illegal buffer
>>>- if i use toObject() prints completely wrong object. the structure
>>>of obj is correct but the values that go into this template are erratic.
>>>
>>> --
>>>
>>> ---protobuf.load("/static/proto/web.proto",function(err,root){
>>>
>>> wssm = root.lookupType("web.WebSocketServerMessage");
>>> var obj1 = wssm.toObject(atob(message.data), {
>>> enums: String,  // enums as string names
>>> longs: String,  // longs as strings (requires long.js)
>>> bytes: String,  // bytes as base64 encoded strings
>>> defaults: true, // includes default values
>>> arrays: true,   // populates empty arrays (repeated fields) even if 
>>> defaults=false
>>> objects: true,  // populates empty objects (map fields) even if 
>>> defaults=false
>>> oneofs: true// includes virtual oneof fields set to the present 
>>> field's name
>>> });
>>>
>>> --
>>>
>>> ---
>>> Object { type: "REPLY", reply: null, exception: null, data: null } 
>>> ##COMMENTS: type is wrong, expected to have values in reply and data.
>>>
>>>
>>> it will be helpful if you could point me to a snippet.
>>> Note: I am not using node, need help only in js client.
>>>
>>> my message on client looks like :
>>> [image: image]
>>>
>>>
>>>
>>>
>>> CODE:::
>>>
>>> i have tried out what you have suggested it dosent work . please look at
>>> the program and commented output.
>>>
>>> `var wssm=null
>>> protobuf.load("/static/proto/web.proto",function(err,root){
>>>
>>> console.log(err);
>>> wssm = root.lookupType("web.WebSocketServerMessage");
>>>
>>> socket.onmessage = function (message2){
>>>
>>> console.log("BASE64 encoded : ",message2.data);
>>>
>>> /*00:07:56.460 BASE64 encoded :
>>> CAQilwEIBxAIGpABCo0BChkKFy9DRlMvQ0ZFX0VWUy9DbWRDb3VudGVyEgQIAigAGgQIAigAIJeRr+KPLCivyprrnAkwADgBWhcyMDE4LTAxLTE2VDAwOjA3OjU2LjE3NWIXMTk4MC0wMS0xN1QxNzozNzoyOS44NDe4AfOcr+KPLMIBFzIwMTgtMDEtMTZUMDA6MDc6NTcuNjc1
>>> 1 client2.js:294:17*/
>>>
>>> console.log("applied atob() : ",atob(message2.data));
>>>
>>> /
>>>
>>>
>>> *00:07:56.462 applied atob() :
>>> ��"�/CFS/CFE_EVS/ErrCounter(*/
>>>
>>> console.log("protobuf.util.base64 :
>>> ",protobuf.util.base64.decode(message2.data));
>>>
>>> /*TypeError: buffer is undefined[Learn More] protobuf.js:204:17
>>> decode
>>> http://cdn.rawgit.com/dcodeIO/protobuf.js/6.8.4/dist/protobuf.js:204:17
>>> Session_Maintainance.prototype.subscribeTelemetry/>> http://127.0.0.1:8000/client2.js:296:57*/
>>>
>>> });`
>>>
>>> :::
>>>
>>>
>>>
>>> # web.proto
>>> syntax="proto2";
>>>
>>> package web;
>>> option java_package = "org.yamcs.protobuf";
>>>
>>> import "yamcs.proto";
>>> import "archive.proto";
>>> import "commanding.proto";
>>> import "pvalue.proto";
>>> import "alarms.proto";
>>> import "yamcsManagement.proto";
>>>
>>> message WebSocketClientMessage {
>>> optional uint32 protocolVersion = 1;
>>> optional uint32 sequenceNumber = 2;
>>> optional string resource = 3;
>>> optional string operation = 4;
>>> optional bytes data = 5;
>>> }
>>>
>>> message WebSocketServerMessage {
>>> enum MessageType {
>>> REPLY = 2; // one-time rpc-style ACK reply
>>> EXCEPTION = 3; // one-time rpc-style exception reply
>>> DATA = 4; // various subscribed data
>>> }
>>>
>>> message WebSocketReplyData {
>>> optional uint32 protocolVersion = 1;
>>> optional int32 sequenceNumber = 2; // allow for -1
>>> optional string type = 3;
>>> optional string message = 4;
>>> optional bytes data = 5; // Structured data
>>> }
>>>
>>> message WebSocketExceptionData {
>>> optional uint32 protocolVersion = 1;
>>> optional int32 sequenceNumber = 2; // allow for -1
>>> optional string type = 3;
>>> optional string message = 4;
>>> optional bytes data = 5; // Structured exception data
>>> }
>>>
>>> // Could use protobuf3 Any-type some day.
>>> message WebSocketSubscriptionData {
>>> optional uint32 sequenceNumber = 1;
>>> optional yamcs.ProtoDataType type = 2;
>>> optional pvalue.ParameterData 

Re: [protobuf] decode does not work on client side!. illegal buffer

['Bo Yang' via Protocol Buffers]

Sorry, protobuf.js is not maintained by us. Please contact the owner.

On Tue, Jan 16, 2018 at 9:21 AM  wrote:

> protobuf.js version: 6.8.4
>
> I am using Django-channels in the backed and javascript WebSockets on the
> front end. I have a set of 10 .proto files which import one or more of
> these .proto files. I was able to serialize my JSON on the server side,
> send data over the wire and receive base64 encoded message. I am having
> difficulty trying to convert this message to JSON.
>
> I have encountered following problems.
>
>- if I convert my actual data to the uint8array buffer and pass it
>into decode: illegal buffer
>- if i use toObject() prints completely wrong object. the structure of
>obj is correct but the values that go into this template are erratic.
>
> --
>
> ---protobuf.load("/static/proto/web.proto",function(err,root){
>
> wssm = root.lookupType("web.WebSocketServerMessage");
> var obj1 = wssm.toObject(atob(message.data), {
> enums: String,  // enums as string names
> longs: String,  // longs as strings (requires long.js)
> bytes: String,  // bytes as base64 encoded strings
> defaults: true, // includes default values
> arrays: true,   // populates empty arrays (repeated fields) even if 
> defaults=false
> objects: true,  // populates empty objects (map fields) even if 
> defaults=false
> oneofs: true// includes virtual oneof fields set to the present 
> field's name
> });
>
> --
>
> ---
> Object { type: "REPLY", reply: null, exception: null, data: null } 
> ##COMMENTS: type is wrong, expected to have values in reply and data.
>
>
> it will be helpful if you could point me to a snippet.
> Note: I am not using node, need help only in js client.
>
> my message on client looks like :
> [image: image]
>
>
>
>
> CODE:::
>
> i have tried out what you have suggested it dosent work . please look at
> the program and commented output.
>
> `var wssm=null
> protobuf.load("/static/proto/web.proto",function(err,root){
>
> console.log(err);
> wssm = root.lookupType("web.WebSocketServerMessage");
>
> socket.onmessage = function (message2){
>
> console.log("BASE64 encoded : ",message2.data);
>
> /*00:07:56.460 BASE64 encoded :
> CAQilwEIBxAIGpABCo0BChkKFy9DRlMvQ0ZFX0VWUy9DbWRDb3VudGVyEgQIAigAGgQIAigAIJeRr+KPLCivyprrnAkwADgBWhcyMDE4LTAxLTE2VDAwOjA3OjU2LjE3NWIXMTk4MC0wMS0xN1QxNzozNzoyOS44NDe4AfOcr+KPLMIBFzIwMTgtMDEtMTZUMDA6MDc6NTcuNjc1
> 1 client2.js:294:17*/
>
> console.log("applied atob() : ",atob(message2.data));
>
> /
>
>
> *00:07:56.462 applied atob() :
> ��"�/CFS/CFE_EVS/ErrCounter(*/
>
> console.log("protobuf.util.base64 :
> ",protobuf.util.base64.decode(message2.data));
>
> /*TypeError: buffer is undefined[Learn More] protobuf.js:204:17
> decode
> http://cdn.rawgit.com/dcodeIO/protobuf.js/6.8.4/dist/protobuf.js:204:17
> Session_Maintainance.prototype.subscribeTelemetry/ http://127.0.0.1:8000/client2.js:296:57*/
>
> });`
>
> :::
>
>
>
> # web.proto
> syntax="proto2";
>
> package web;
> option java_package = "org.yamcs.protobuf";
>
> import "yamcs.proto";
> import "archive.proto";
> import "commanding.proto";
> import "pvalue.proto";
> import "alarms.proto";
> import "yamcsManagement.proto";
>
> message WebSocketClientMessage {
> optional uint32 protocolVersion = 1;
> optional uint32 sequenceNumber = 2;
> optional string resource = 3;
> optional string operation = 4;
> optional bytes data = 5;
> }
>
> message WebSocketServerMessage {
> enum MessageType {
> REPLY = 2; // one-time rpc-style ACK reply
> EXCEPTION = 3; // one-time rpc-style exception reply
> DATA = 4; // various subscribed data
> }
>
> message WebSocketReplyData {
> optional uint32 protocolVersion = 1;
> optional int32 sequenceNumber = 2; // allow for -1
> optional string type = 3;
> optional string message = 4;
> optional bytes data = 5; // Structured data
> }
>
> message WebSocketExceptionData {
> optional uint32 protocolVersion = 1;
> optional int32 sequenceNumber = 2; // allow for -1
> optional string type = 3;
> optional string message = 4;
> optional bytes data = 5; // Structured exception data
> }
>
> // Could use protobuf3 Any-type some day.
> message WebSocketSubscriptionData {
> optional uint32 sequenceNumber = 1;
> optional yamcs.ProtoDataType type = 2;
> optional pvalue.ParameterData parameterData = 3;
> optional commanding.CommandHistoryEntry command = 4;
> optional yamcsManagement.ProcessorInfo processorInfo = 5;
> optional yamcsManagement.ClientInfo clientInfo = 6;
> optional yamcsManagement.Statistics statistics = 7;
> optional yamcs.Event event = 8;
> optional archive.StreamData streamData = 9;
> optional alarms.AlarmData alarmData = 10;
> optional yamcs.TimeInfo timeInfo = 11;
> optional yamcsManagement.LinkEvent linkEvent = 12;
> optional commanding.CommandQueueInfo commandQueueInfo = 13;
> optional commanding.CommandQueueEvent commandQueueEvent = 14;
> 

[protobuf] decode does not work on client side!. illegal buffer

[kbibireddy]

protobuf.js version: 6.8.4

I am using Django-channels in the backed and javascript WebSockets on the 
front end. I have a set of 10 .proto files which import one or more of 
these .proto files. I was able to serialize my JSON on the server side, 
send data over the wire and receive base64 encoded message. I am having 
difficulty trying to convert this message to JSON.

I have encountered following problems.

   - if I convert my actual data to the uint8array buffer and pass it into 
   decode: illegal buffer
   - if i use toObject() prints completely wrong object. the structure of 
   obj is correct but the values that go into this template are erratic.

--

---protobuf.load("/static/proto/web.proto",function(err,root){

wssm = root.lookupType("web.WebSocketServerMessage");
var obj1 = wssm.toObject(atob(message.data), {
enums: String,  // enums as string names
longs: String,  // longs as strings (requires long.js)
bytes: String,  // bytes as base64 encoded strings
defaults: true, // includes default values
arrays: true,   // populates empty arrays (repeated fields) even if 
defaults=false
objects: true,  // populates empty objects (map fields) even if 
defaults=false
oneofs: true// includes virtual oneof fields set to the present field's 
name
});

--

---
Object { type: "REPLY", reply: null, exception: null, data: null } ##COMMENTS: 
type is wrong, expected to have values in reply and data.


it will be helpful if you could point me to a snippet.
Note: I am not using node, need help only in js client.

my message on client looks like :
[image: image]




CODE:::

i have tried out what you have suggested it dosent work . please look at 
the program and commented output.

`var wssm=null
protobuf.load("/static/proto/web.proto",function(err,root){

console.log(err);
wssm = root.lookupType("web.WebSocketServerMessage");

socket.onmessage = function (message2){

console.log("BASE64 encoded : ",message2.data);

/*00:07:56.460 BASE64 encoded : 
CAQilwEIBxAIGpABCo0BChkKFy9DRlMvQ0ZFX0VWUy9DbWRDb3VudGVyEgQIAigAGgQIAigAIJeRr+KPLCivyprrnAkwADgBWhcyMDE4LTAxLTE2VDAwOjA3OjU2LjE3NWIXMTk4MC0wMS0xN1QxNzozNzoyOS44NDe4AfOcr+KPLMIBFzIwMTgtMDEtMTZUMDA6MDc6NTcuNjc1
 
1 client2.js:294:17*/

console.log("applied atob() : ",atob(message2.data));

/


*00:07:56.462 applied atob() : ��"�/CFS/CFE_EVS/ErrCounter(*
/

console.log("protobuf.util.base64 : 
",protobuf.util.base64.decode(message2.data));

/*TypeError: buffer is undefined[Learn More] protobuf.js:204:17
decode 
http://cdn.rawgit.com/dcodeIO/protobuf.js/6.8.4/dist/protobuf.js:204:17
Session_Maintainance.prototype.subscribeTelemetry/http://127.0.0.1:8000/client2.js:296:57*/

});`

:::



# web.proto
syntax="proto2";

package web;
option java_package = "org.yamcs.protobuf";

import "yamcs.proto";
import "archive.proto";
import "commanding.proto";
import "pvalue.proto";
import "alarms.proto";
import "yamcsManagement.proto";

message WebSocketClientMessage {
optional uint32 protocolVersion = 1;
optional uint32 sequenceNumber = 2;
optional string resource = 3;
optional string operation = 4;
optional bytes data = 5;
}

message WebSocketServerMessage {
enum MessageType {
REPLY = 2; // one-time rpc-style ACK reply
EXCEPTION = 3; // one-time rpc-style exception reply
DATA = 4; // various subscribed data
}

message WebSocketReplyData {
optional uint32 protocolVersion = 1;
optional int32 sequenceNumber = 2; // allow for -1
optional string type = 3;
optional string message = 4;
optional bytes data = 5; // Structured data
}

message WebSocketExceptionData {
optional uint32 protocolVersion = 1;
optional int32 sequenceNumber = 2; // allow for -1
optional string type = 3;
optional string message = 4;
optional bytes data = 5; // Structured exception data
}

// Could use protobuf3 Any-type some day.
message WebSocketSubscriptionData {
optional uint32 sequenceNumber = 1;
optional yamcs.ProtoDataType type = 2;
optional pvalue.ParameterData parameterData = 3;
optional commanding.CommandHistoryEntry command = 4;
optional yamcsManagement.ProcessorInfo processorInfo = 5;
optional yamcsManagement.ClientInfo clientInfo = 6;
optional yamcsManagement.Statistics statistics = 7;
optional yamcs.Event event = 8;
optional archive.StreamData streamData = 9;
optional alarms.AlarmData alarmData = 10;
optional yamcs.TimeInfo timeInfo = 11;
optional yamcsManagement.LinkEvent linkEvent = 12;
optional commanding.CommandQueueInfo commandQueueInfo = 13;
optional commanding.CommandQueueEvent commandQueueEvent = 14;
optional yamcs.TmPacketData tmPacket = 15;
optional ConnectionInfo connectionInfo = 16;

optional WebSocketExtensionData extensionData = 100;

}

optional MessageType type = 1;
optional WebSocketReplyData reply = 2;
optional WebSocketExceptionData exception = 3;
optional WebSocketSubscriptionData data = 4;
}

// Escape hatch for providing non-core Yamcs data over web socket
message