l3303 commented on issue #994: URL: https://github.com/apache/incubator-fury/issues/994#issuecomment-2162826646
This feature is very necessary. Refer to protostuff implemetation, it will alloc a very small byte arry(default to 4096) to buffer binary during serialization, when buffer is full then write all into target output stream. This may solve your concern about perfermance issue. Back to fury, I found that even fury stream deseialization(FuryInputStream) is not real streaming as well. It will read all binary into MemoryBuffer then process deserialization. This may waste memory when object is big. My way to optimize this is to read bytes only within MemoryBuffer's size each time, then process deseialization, once buffer is not enough then refill buffer to its size. I've implemented a demo base on this solution and works in my situation. Looking forward fury can implement real streaming in serialization & deserialization -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
