If only UTF-8 messages are being parsed successfully, then that sounds like a clue to me that you probably need to base64-encode the serialized message. I have only limited experience with JavaScript, but from what I understand, JavaScript strings can only represent Unicode and not raw binary data. So if you need to store raw binary in a JavaScript string then you probably want to base64-encode it (and then decode it on the other side before parsing the message).
On Thu, Feb 17, 2022 at 11:28 AM Csaba Szigeti <[email protected]> wrote: > I am really sorry. Posting again with correct formatting this time : > > I would like to pass a protobuf message from Javascript to a C++ > WebAssembly module. > > I do receive, and can successfully parse the simpler protobuf messages on > the C++ side, however, protobuf is failing to parse the less-simple > protobuf messages. > > Here is the code that I am using : > > ```javascript > goog.module('protobufMessageGenerator'); > goog.require('proto.my_proto.MyMessage'); > > function generateMyMessage() { > var msg = new proto.my_proto.MyMessage() > var internal = new proto.my_proto.MyMessageInternal() > internal.setAccentColor("#CC1D24") > > internal.setLogo("iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAYHSURBVHgB7ZhrbBRVFMf/s7vdR9ul7ba09CXQoqVFqcHwFhMQEh4GDSF+ggQhIT6CCUaN/YCaRsWo6QfAKAkvJVEILzECBSE8CyG0lfAoBQu0ttvndrfbZZ8zu9czd22FD+Lc4cNK3F8yO5mZvXfO/95zzzl3JEbgMcaAx5ykgESTFJBokgISTVJAovk/C2BgoTB8decRbG6GXmJuDzy1xxDp7oYeTNADlU+D586ja/v3QCiEjBdmwTZ+vFAXUBT0/Lib+rmASGsbitdVwTxqFEQRnwEyvmfrDjg3fYOSTz6GOTcXaRPGqxOimZjfj9aqdQjfakHp+mpElQhSS0ugCyZEjLkOHGSNk6azcGsbU1xu1jBpGou0tfNnmlAUdmfte+za3IUs6vMxz+FadnnqLMZCIaYHoRmQO7vRUf0ZshfNh3l0Mbx1ddyFTFmZ9FTS1Ifn2HG49/+MgjVvwJCWhv5DR2DOdgBmM/SgXQC5SNfmLaBRh2PRAm6w9+RpGK1WSCkal1I0it5t38E0KhcZL84GCwZx72I9DKmp0DoA0CuABYLw0GgZcxxIe7qCGxO8fkPE9RG62wpfw2XYp0zmRoc6nVD6+iBJ+oxX0Swg3N5OL+uHJT8fEr2chSOI9Ll4KGUkRguBphswyBHYSsdCHXG5qxdSNIYozYRQFNAjQBnwUPiIwWhP59dMlhFTjSchUYoqDydunNzv5mfTiAzuMUPtot5B+lGgB80CJIsVjKZaNZxfG6gpHYxCoNzv+cd20YEByJ4BLkKyWPi9mByO92Ey8rM84OWDoaJQQnN+/S29R5sgzQIs+XkwWMyQXf18tFRjDPY0soZRFu35618PuoHS04ur8xejq2aDai5sY0bzQYh09/L/mjKz+HXMH0CMZsF7/CSal6+C2eEYFvdvaM7EppEjYZtQzheuarC5oBC2okL4nZ2QnR3wNzRi8Mw5hNvaaVZkmCjB2WdMQ8bk53Cv8TeyN0YJr5yCQA78l69w4dbCAgqfKUBERseXNQjRmirdWANrWZnmoKQ9jBqNyF2xjPu899RZqBnE+uQ4/qizZiPaqtcjeKcV7n0/IUKi7GS4+3AtAtea+GIHtVPjfvb8eQhcuYbQ7bswUfw3ZWdDouwuDwziqc2bqCQpg0hQEkhkDI6FC5A2Yypce/fzUbOOHcOfpE+bgvK9P6Do/bUwkquF2zsQIffJfmkhLDTKjNZAy5p30LxiFXwUiVS6t27ng6LOoopj8SIY0tMhGo0EijkJEmXLkq8+x83lK+H89AsMnDrN7xtsajJLgbm4CBUH98Bz9AS5UhtCLbdhKS6G//cWPFH1LlIKC7lve44eR+sH62BMS4WPZkNFCQSG3yMEEyQWi7F79Y2svrSCddds4OeW19eweC10/xEn4nSyW6vfVBsO9cAP1649rKFsIvuj6kPWUDSOdW7ZxvQgXo2Sg/qvXod95nTkrV5JkSST54e/R2/oiJOSl4cx1R/ddyv+PNjSgpxXlyCLXIeH5+E+xBDeDzAKeb2796CUXAk2Gy/EDOkPqWXIz1Pych+4FSLjXb8cQcWunXxfAIMEoy0VehCegcEzZ2GhEGkrj29g1NxgKSgQ6qN3x0445s2hivYJGGhdMUqI5ryR0IOwgP5DtRjx/HQ+aipKhGqb8jLN7RktVveJU8h+ZbF6RVlZJgGU5ErGQg9iAsK0B25sROqzlXFj6Foml0qvrITW8OdvvknOxpBaXg7V7cJd3TyhWSiC6UFsQ0N7AcXrHY7dQQqTNnpxSkE+tIY/dfFaqb1kjddFvkv1yJwzm0K0BXoQEqD4vFQHMb4w1b2xa98BZM2dA5HUqXh9GBKr7jHcR39FztIl0ItQFDLZ7ZCozumjKKQWc94LF1Hw9lsiXcCUMQLBpmZ4Dh+B9+x5pD8zAbYKwS8a9yOUNWSFNS97jV0qHscaKyfzhCYK1UnUdgqrLyplTS8vZbTPYI+CpP6ICFbre2/dBdgrJyKFrwW1uVj6V0uMwI2byJg1E8bMDDwKwgL+ayQ/7iaapIBEkxSQaJICEk1SQKL5E+3sNu+yTCFZAAAAAElFTkSuQmCC") > msg.setInternal(internal) > return msg > } > ``` > > Using the google-closure-compiler, I compile this protobuf message > generator to pure Javascript. > I use this code to pass the message to the C++ WebAssembly module : > > ```javascript > function sendMessage() { > > Module.self().setMyMessage(module$contents$protobufMessageGenerator_generateMyMessage().serializeBinary()) > } > sendMessage() > ``` > > I receive the message on the C++ side using this code : > > ```cpp > void JavascriptPublicInterface::setMyMessage(const QString &myMessage) > { > if (_myMessage != myMessage) > { > _myMessage = myMessage; > emit myMessageChanged(); > } > } > > void JavascriptPublicInterface::setMyMessageEMS(const std::string > &myMessage) > { > setMyMessage(QString::fromStdString(myMessage)); > } > > //... > > void startListeningToMyMessage(std::function<void(euf_proto::MyMessage > myMessage)> listener) > { > const auto changedListener = [listener]() { > const auto newMyMessageString = > JavascriptPublicInterface::self()->myMessage(); > my_proto::MyMessage myMessagePb; > if > (myMessagePb.ParseFromString(newMyMessageString.toStdString())) > { > qDebug() << "C++ : Successfully parsed new > MyMessage: " << newMyMessageString; > } > else > { > qDebug() << "C++ : Failed to parse new MyMessage: > " << newMyMessageString; > } > > qDebug() << "MyMessage received: " << > QString::fromStdString(myMessagePb.DebugString()); > > listener(myMessagePb); > }; > if (!QObject::connect(JavascriptPublicInterface::self(), > &JavascriptPublicInterface::myMessageChanged, changedListener)) > { > throw std::runtime_error("Failed to connect to > JavascriptPublicInterface myMessageChanged"); > } > } > ``` > > In the Javascript console I see : "C++ : Failed to parse new MyMessage: > ..." > > I guess that it can be related to the fact that Javascript is using UTF-16 > and C++ is using UTF-8 encoding (because the messages that are UTF-8 only > can be successfully parsed), so I tried to fix the sending and receiving > like this: > > ```javascript > //For more info see : > https://emscripten.org/docs/api_reference/emscripten.h.html?highlight=stringtoutf8 > function convertUtf16ToUtf8(input) { > var uint8Arr = input > var jsString = String.fromCharCode.apply(null, uint8Arr) > var lengthBytes = lengthBytesUTF8(jsString)+1 > var stringOnWasmHeap = _malloc(lengthBytes) > stringToUTF8(jsString, stringOnWasmHeap, lengthBytes) > return stringOnWasmHeap > } > > function sendMessage() { > var myMessage = > convertUtf16ToUtf8(module$contents$protobufMessageGenerator_generateMyMessage().serializeBinary()) > Module.self().setMyMessage(myMessage) > _free(myMessage) > } > sendMessage() > ``` > > And I modified the parsing on the C++ side like this : > > ```cpp > void JavascriptPublicInterface::setMyMessageEMS(const int pointer) > { > const char* msg = (char*)pointer; > setMyMessage(QString::fromStdString(msg)); > } > ``` > > But I still get this error : "C++ : Failed to parse new MyMessage: ..." > > How could I fix this? > > (I am using protobuf 3.19.1 (same version on both the Javascript and the > C++ side).) > > On Thursday, February 17, 2022 at 8:25:26 PM UTC+1 Csaba Szigeti wrote: > >> I would like to pass a protobuf message from Javascript to a C++ >> WebAssembly module. >> >> I do receive, and can successfully parse the simpler protobuf messages on >> the C++ side, however, protobuf is failing to parse the less-simple >> protobuf messages. >> >> Here is the code that I am using : >> ``` >> goog.module('protobufMessageGenerator'); >> goog.require('proto.my_proto.MyMessage'); function generateMyMessage() { >> var msg = new proto.my_proto.MyMessage() var internal = new >> proto.my_proto.MyMessageInternal() internal.setAccentColor("#CC1D24") >> internal.setLogo("iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAYHSURBVHgB7ZhrbBRVFMf/s7vdR9ul7ba09CXQoqVFqcHwFhMQEh4GDSF+ggQhIT6CCUaN/YCaRsWo6QfAKAkvJVEILzECBSE8CyG0lfAoBQu0ttvndrfbZZ8zu9czd22FD+Lc4cNK3F8yO5mZvXfO/95zzzl3JEbgMcaAx5ykgESTFJBokgISTVJAovk/C2BgoTB8decRbG6GXmJuDzy1xxDp7oYeTNADlU+D586ja/v3QCiEjBdmwTZ+vFAXUBT0/Lib+rmASGsbitdVwTxqFEQRnwEyvmfrDjg3fYOSTz6GOTcXaRPGqxOimZjfj9aqdQjfakHp+mpElQhSS0ugCyZEjLkOHGSNk6azcGsbU1xu1jBpGou0tfNnmlAUdmfte+za3IUs6vMxz+FadnnqLMZCIaYHoRmQO7vRUf0ZshfNh3l0Mbx1ddyFTFmZ9FTS1Ifn2HG49/+MgjVvwJCWhv5DR2DOdgBmM/SgXQC5SNfmLaBRh2PRAm6w9+RpGK1WSCkal1I0it5t38E0KhcZL84GCwZx72I9DKmp0DoA0CuABYLw0GgZcxxIe7qCGxO8fkPE9RG62wpfw2XYp0zmRoc6nVD6+iBJ+oxX0Swg3N5OL+uHJT8fEr2chSOI9Ll4KGUkRguBphswyBHYSsdCHXG5qxdSNIYozYRQFNAjQBnwUPiIwWhP59dMlhFTjSchUYoqDydunNzv5mfTiAzuMUPtot5B+lGgB80CJIsVjKZaNZxfG6gpHYxCoNzv+cd20YEByJ4BLkKyWPi9mByO92Ey8rM84OWDoaJQQnN+/S29R5sgzQIs+XkwWMyQXf18tFRjDPY0soZRFu35618PuoHS04ur8xejq2aDai5sY0bzQYh09/L/mjKz+HXMH0CMZsF7/CSal6+C2eEYFvdvaM7EppEjYZtQzheuarC5oBC2okL4nZ2QnR3wNzRi8Mw5hNvaaVZkmCjB2WdMQ8bk53Cv8TeyN0YJr5yCQA78l69w4dbCAgqfKUBERseXNQjRmirdWANrWZnmoKQ9jBqNyF2xjPu899RZqBnE+uQ4/qizZiPaqtcjeKcV7n0/IUKi7GS4+3AtAtea+GIHtVPjfvb8eQhcuYbQ7bswUfw3ZWdDouwuDwziqc2bqCQpg0hQEkhkDI6FC5A2Yypce/fzUbOOHcOfpE+bgvK9P6Do/bUwkquF2zsQIffJfmkhLDTKjNZAy5p30LxiFXwUiVS6t27ng6LOoopj8SIY0tMhGo0EijkJEmXLkq8+x83lK+H89AsMnDrN7xtsajJLgbm4CBUH98Bz9AS5UhtCLbdhKS6G//cWPFH1LlIKC7lve44eR+sH62BMS4WPZkNFCQSG3yMEEyQWi7F79Y2svrSCddds4OeW19eweC10/xEn4nSyW6vfVBsO9cAP1649rKFsIvuj6kPWUDSOdW7ZxvQgXo2Sg/qvXod95nTkrV5JkSST54e/R2/oiJOSl4cx1R/ddyv+PNjSgpxXlyCLXIeH5+E+xBDeDzAKeb2796CUXAk2Gy/EDOkPqWXIz1Pych+4FSLjXb8cQcWunXxfAIMEoy0VehCegcEzZ2GhEGkrj29g1NxgKSgQ6qN3x0445s2hivYJGGhdMUqI5ryR0IOwgP5DtRjx/HQ+aipKhGqb8jLN7RktVveJU8h+ZbF6RVlZJgGU5ErGQg9iAsK0B25sROqzlXFj6Foml0qvrITW8OdvvknOxpBaXg7V7cJd3TyhWSiC6UFsQ0N7AcXrHY7dQQqTNnpxSkE+tIY/dfFaqb1kjddFvkv1yJwzm0K0BXoQEqD4vFQHMb4w1b2xa98BZM2dA5HUqXh9GBKr7jHcR39FztIl0ItQFDLZ7ZCozumjKKQWc94LF1Hw9lsiXcCUMQLBpmZ4Dh+B9+x5pD8zAbYKwS8a9yOUNWSFNS97jV0qHscaKyfzhCYK1UnUdgqrLyplTS8vZbTPYI+CpP6ICFbre2/dBdgrJyKFrwW1uVj6V0uMwI2byJg1E8bMDDwKwgL+ayQ/7iaapIBEkxSQaJICEk1SQKL5E+3sNu+yTCFZAAAAAElFTkSuQmCC") >> msg.setInternal(internal) return msg } >> ``` >> Using the google-closure-compiler, I compile this protobuf message >> generator to pure Javascript. I use this code to pass the message to the >> C++ WebAssembly module : >> ``` >> function sendMessage() { >> Module.self().setMyMessage(module$contents$protobufMessageGenerator_generateMyMessage().serializeBinary()) >> } sendMessage() >> ``` >> I receive the message on the C++ side using this code : >> ``` >> void JavascriptPublicInterface::setMyMessage(const QString &myMessage) { >> if (_myMessage != myMessage) { _myMessage = myMessage; emit >> myMessageChanged(); } } void >> JavascriptPublicInterface::setMyMessageEMS(const std::string &myMessage) { >> setMyMessage(QString::fromStdString(myMessage)); } //... void >> startListeningToMyMessage(std::function<void(euf_proto::MyMessage >> myMessage)> listener) { const auto changedListener = [listener]() { const >> auto newMyMessageString = JavascriptPublicInterface::self()->myMessage(); >> my_proto::MyMessage myMessagePb; if >> (myMessagePb.ParseFromString(newMyMessageString.toStdString())) { qDebug() >> << "C++ : Successfully parsed new MyMessage: " << newMyMessageString; } >> else { qDebug() << "C++ : Failed to parse new MyMessage: " << >> newMyMessageString; } qDebug() << "MyMessage received: " << >> QString::fromStdString(myMessagePb.DebugString()); listener(myMessagePb); >> }; if (!QObject::connect(JavascriptPublicInterface::self(), >> &JavascriptPublicInterface::myMessageChanged, changedListener)) { throw >> std::runtime_error("Failed to connect to JavascriptPublicInterface >> myMessageChanged"); } } >> ``` >> >> >> In the Javascript console I see : "C++ : Failed to parse new MyMessage: >> ..." >> >> I guess that it can be related to the fact that Javascript is using >> UTF-16 and C++ is using UTF-8 encoding (because the messages that are UTF-8 >> only can be successfully parsed), so I tried to fix the sending and >> receiving like this: >> >> ``` >> //For more info see : >> https://emscripten.org/docs/api_reference/emscripten.h.html?highlight=stringtoutf8 >> function convertUtf16ToUtf8(input) { var uint8Arr = input var jsString = >> String.fromCharCode.apply(null, uint8Arr) var lengthBytes = >> lengthBytesUTF8(jsString)+1 var stringOnWasmHeap = _malloc(lengthBytes) >> stringToUTF8(jsString, stringOnWasmHeap, lengthBytes) return >> stringOnWasmHeap } function sendMessage() { var myMessage = >> convertUtf16ToUtf8(module$contents$protobufMessageGenerator_generateMyMessage().serializeBinary()) >> Module.self().setMyMessage(myMessage) _free(myMessage) } sendMessage() >> ``` >> And I modified the parsing on the C++ side like this : >> ``` >> void JavascriptPublicInterface::setMyMessageEMS(const int pointer) { >> const char* msg = (char*)pointer; >> setMyMessage(QString::fromStdString(msg)); } >> ``` >> >> But I still get this error : "C++ : Failed to parse new MyMessage: ..." >> >> How could I fix this? >> >> (I am using protobuf 3.19.1 (same version on both the Javascript and the >> C++ side).) >> > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/protobuf/9ce5f69e-b6de-4627-b8e1-71b68da574b3n%40googlegroups.com > <https://groups.google.com/d/msgid/protobuf/9ce5f69e-b6de-4627-b8e1-71b68da574b3n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/CADqAXr7L_Kv%2Bm5Kte5Y%3DHeO9hLzobzr77umbYD64hhXsy4GZ5w%40mail.gmail.com.
