Hi I could do a little more things.
Interestingly, capnp.exe built under x64 can do decode message. I bet it does different processing than MessageReader's do and simply follow message format structure. I made a test code using MapViewOfFile and FlatArrayMessageReader,. It can read messages that I couldnt by using StreamFdMessageReader. But easily fails with a little large data, say 32 entries in first level List and 1 entry in the second level List, getting following exception: Exception: capnp\serialize.c++:51: failed: expected array.size() >= offset + segmentSize; Message ends prematurely in first segment. stack: 7ff6a2af4b35 7ff6a2af3577 7ff6a2af14b6 7ff6a2b02323 7ffc1aa97c23 7ffc1b18d720 So I think I'm stuck. Any advice on how to proceed will be greatly appreciated. Thank you in advance. 2021年2月16日火曜日 16:23:38 UTC+9 [email protected]: > Hi > > So, trying to understand what is happening. > > using capnp.exe (32bit) in capnproto-tools-win32-0.8.0, message can be > read correctly. > capnp.exe decode sparse4d.capnp Dir4D < myoutput.bin > By looking at hex dump of the message file, it looks like a proper message > format. > > So, writing new message is okay but reading it is not working properly. > Using capnproto-0.8.0 built with VS2017/x64 , and create a new message, > save it to a file, and then reading it by my program fails. > Since all happens when I do message.getRoot<Dir4D>(), this "Premature > EOF" situation happens inside capnp library. > > Is there any way to fix this? or to narrow down what is wrong? > > Thank you. > > 2021年2月16日火曜日 4:56:44 UTC+9 [email protected]: > >> Hello >> >> I have been learning this wonderful software. >> So I have been looking for an opportunity to use capnproto for my project, >> and now I have it but I need to use it with MSVC 2017(recently updated to >> latest version 15.9.33), x64. >> I only need serialization and no need for RPC or async. >> This is my very first time posting a message to this community, please >> let me know if I'm not doing right here. >> >> I have a schema whose basic construct is List of List of struct ( x,y >> points in 4d expressed by two level lists). Second level will have a few >> hundreds of entries and first level will have lots of entries. >> And I am now stuck with a issue. If I put many entries in either of lists >> (more than a few hundreds) and write that to a file, then when I read the >> message, I get exception like this: >> Any help would be very appreciated. >> >> printSparseDir4D: std::exception: kj\io.c++:53: failed: expected n >= >> minBytes; Premature EOF >> stack: 7ff672faac46 7ff672fa3fd6 7ff672fa2de5 7ff672fa1308 7ff672fb15d3 >> 7ffc1aa97c23 7ffc1b18d720 >> >> Now I found that writing 4 entries in first level, two entries in second >> level (two x,y points each in 4 frames), I get the same error. >> >> I'm putting my schema and my read/write code below. I'm using these codes >> with capnproto0.8.0. But actually this schema is a little striped version >> than the one I want to use. When I used that a little more complex schema, >> I faced similar issue with capnproto 0.7.0 and 0.8.0. >> >> I built capnproto following the steps in the website, had to skip >> heavy-tests project and kj-async (0.8.0), hoping the build was ok. >> >> Is there anything missing or done wrong? >> It would be very much appreciated for any help. >> Thank you. >> >> >> sparse4d.capnp : >> >> # unique file ID, generated by `capnp id` >> @0xae7dda29a38d355f; >> >> struct Sparse4DHeader { >> tSize @0 :Int32; >> zSize @1 :Int32; >> ySize @2 :Int32; >> xSize @3 :Int32; >> } >> >> struct Dir4D { >> header @0 :Sparse4DHeader; >> planes @1 :List(SparsePlane); >> >> struct SparsePlane { >> t @0 :UInt16; >> z @1 :UInt16; >> points @2 :List(Point); >> } >> } >> >> struct Point { >> x @0 :UInt16; >> y @1 :UInt16; >> intensity @2 :Float32; >> } >> >> >> And read/write code: >> >> /** >> * Copyright (c) 2021 ASTOM R&D >> * Author: Yojiro Nobukuni >> */ >> //#include "pch.h" >> >> #include "../sparse4d.capnp.h" >> #include <capnp/message.h> >> #include <capnp/serialize-packed.h> >> #include <iostream> >> #include <ios> >> >> #include <sys/types.h> >> #include <sys/stat.h> >> #include <fcntl.h> >> //#include <unistd.h> >> >> /** >> * g++ writer.cpp photodeco_kernel.capnp.c++ -o writer -lcapnp -lkj >> * ./writer > test.message >> */ >> >> /** >> * Write PhotodecoKernel message to fd >> * @param int fd fd to read message in >> */ >> void writeSparse4D(int fd, unsigned int nofPlanes, unsigned int >> nofPoints, bool packed) >> { >> ::capnp::MallocMessageBuilder message; >> >> auto dir4d = message.initRoot<Dir4D>(); >> auto header = dir4d.initHeader(); >> header.setTSize(7); >> header.setZSize(7); >> header.setXSize(7); >> header.setYSize(7); >> >> auto planes = dir4d.initPlanes(nofPlanes); >> for (auto plane : planes) { >> plane.setT(9); >> plane.setZ(8); >> auto points = plane.initPoints(nofPoints); >> #if 0 >> // This one or the other one in #else block, same >> behavior. >> for (unsigned int i = 0; i < nofPoints; i++) { >> points[i].setX(1); >> points[i].setY(2); >> points[i].setIntensity(3); >> } >> #else >> for (auto p : points) { >> p.setX(1); >> p.setY(2); >> p.setIntensity(3); >> } >> #endif >> } >> >> try { >> if (packed) >> writePackedMessageToFd(fd, message); >> else >> writeMessageToFd(fd, message); >> } >> catch (std::exception& e) { >> std::cout << "writeSparse4D: std::exception: " << e.what() << std::endl; >> return; >> } >> } >> >> >> /** >> * Print PhotodecoKernerl kernel in text >> * @param PhotodecoKernel::Reader kernel >> */ >> void printDir4D(Dir4D::Reader dir4d, bool printPoints) >> { >> auto header = dir4d.getHeader(); >> std::cout << " sizes[T,Z,Y,X] = " >> << header.getTSize() << ", " << header.getZSize() << ", " << >> header.getYSize() << ", " << header.getXSize() << ", " >> << std::endl; >> //std::cout << " nofPoints = " << header.getNofPoints() << std::endl; >> >> if (!printPoints) return; >> >> size_t nofPoints = 0; >> auto planes = dir4d.getPlanes(); >> for (auto plane : planes) { >> const auto t = plane.getT(); >> const auto z = plane.getZ(); >> std::cout << "plane(t,z): " << t << ", " << z << std::endl; >> for (auto p : plane.getPoints()) { >> std::cout << t << ", " << z << ", " << p.getY() << ", " << p.getX() << ", >> " << p.getIntensity() << std::endl; >> nofPoints++; >> } >> } >> >> std::cout << "Total nof KernelPoints = " << nofPoints << std::endl; >> } >> >> /** >> * Read PhotodecoKernerl message from fd and print it in text >> * @param int fd >> * @param bool packed >> */ >> void printSparseDir4D(int fd, bool printPoints, bool packed) >> { >> std::cout << "printSparseDir4D" << std::endl; >> >> // readerOptions for reading message. >> capnp::ReaderOptions readerOptions; >> readerOptions.traversalLimitInWords = 1000 * 1000 * 1000 * >> readerOptions.traversalLimitInWords; >> //readerOptions.nestingLimit; >> >> ::capnp::MessageReader *message; >> try { >> if (packed) >> message = new ::capnp::PackedFdMessageReader(fd, readerOptions); >> else >> message = new ::capnp::StreamFdMessageReader(fd, readerOptions); >> >> auto dir4d = message->getRoot<Dir4D>(); >> printDir4D(dir4d, printPoints); >> } >> catch (std::exception& e) { >> std::cout << "printSparseDir4D: std::exception: " << e.what() << >> std::endl; >> return; >> } >> >> >> delete message; >> } >> >> >> > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/2d1ae382-e739-4ad0-9e74-cbdcc0ab1a0fn%40googlegroups.com.
