[GitHub] [incubator-brpc] leaf-potato commented on issue #2052: 传输大量数据必须用butil::IOBuf 吗？
leaf-potato commented on issue #2052: URL: https://github.com/apache/incubator-brpc/issues/2052#issuecomment-1375721168 目前似乎没看到相关的文档或者测试数据，可以问下仓库的管理员~ 简单说下个人的理解： 1. 使用proto设置reponse：用户设置response => 序列化数据（框架） => 发送 2. attament传序列化数据：用户设置reponse => 序列化数据（用户） => 发送 可以看到两种方式都需要存储序列化后的数据，区别在于序列化的地方不同。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@brpc.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@brpc.apache.org For additional commands, e-mail: dev-h...@brpc.apache.org
[GitHub] [incubator-brpc] leaf-potato commented on issue #2052: 传输大量数据必须用butil::IOBuf 吗？
leaf-potato commented on issue #2052: URL: https://github.com/apache/incubator-brpc/issues/2052#issuecomment-1373070266 > 也就是发送数据量 >=2G 的时候在发送端的 copy 无法消除，其他的 copy 都可以消除，这样可以更高效。 序列化数据拷贝到attachment可以看下IOBuf的`append_user_data`方法是否可以解决 Q3：可以参考[高效率排查server卡顿](https://github.com/apache/incubator-brpc/blob/master/docs/cn/server_debugging.md)文档先进行排查 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@brpc.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@brpc.apache.org For additional commands, e-mail: dev-h...@brpc.apache.org
[GitHub] [incubator-brpc] leaf-potato commented on issue #2052: 传输大量数据必须用butil::IOBuf 吗？
leaf-potato commented on issue #2052: URL: https://github.com/apache/incubator-brpc/issues/2052#issuecomment-1369936547 > Q3：通过 perf，我们看到 bthread 中的 sys call 占总体 sys call 的 20% 左右，不知道是我们使用姿势有问题，或者是有优化参数可以控制？ 可以参考 [高效率排查server卡顿](https://github.com/apache/incubator-brpc/blob/master/docs/cn/server_debugging.md)文档先自行排查下。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@brpc.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@brpc.apache.org For additional commands, e-mail: dev-h...@brpc.apache.org
[GitHub] [incubator-brpc] leaf-potato commented on issue #2052: 传输大量数据必须用butil::IOBuf 吗？
leaf-potato commented on issue #2052: URL: https://github.com/apache/incubator-brpc/issues/2052#issuecomment-1369923528 > 也就是发送数据量 >=2G 的时候在发送端的 copy 无法消除，其他的 copy 都可以消除，这样可以更高效。 我理解不管使用PB还是attachment发送数据都会涉及到一次拷贝，只是这个拷贝发生的地方不同： 1. PB发送数据：用户设置PB => PB序列化（框架）=> 发送。拷贝发生在框架拿到PB序列化的结果。 2. attachment：用户PB序列化 => attachment => 发送。拷贝发生在用户序列化PB给attachment赋值。 > 在限制范围内，attachment和proto的区别在于序列化和反序列化，如果attachment是传proto序列化后的数据，理论上两者差距不大。 综上，用户更多考虑的是如何给PB赋值时减少拷贝，比方使用PB的Swap方法进行数据交换等。最终得@wwbmmm来double check下避免得出错误结论~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@brpc.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@brpc.apache.org For additional commands, e-mail: dev-h...@brpc.apache.org
[GitHub] [incubator-brpc] leaf-potato commented on issue #2052: 传输大量数据必须用butil::IOBuf 吗？
leaf-potato commented on issue #2052: URL: https://github.com/apache/incubator-brpc/issues/2052#issuecomment-1364497672 重新看了下文档，`chunk_PB`在使用异步请求时，`CallMethod`结束后可以释放，前提在done中不需要`chunkPB`信息  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@brpc.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@brpc.apache.org For additional commands, e-mail: dev-h...@brpc.apache.org
[GitHub] [incubator-brpc] leaf-potato commented on issue #2052: 传输大量数据必须用butil::IOBuf 吗？
leaf-potato commented on issue #2052: URL: https://github.com/apache/incubator-brpc/issues/2052#issuecomment-1363987901 Q1：在client、server两端都是brpc的前提下，attachment更算是proto message的一种补充。proto官方给出大小是有限制的，默认是64MB，硬制是2GB。在限制范围内，attachment和proto的区别在于序列化和反序列化，如果attachment是传proto序列化后的数据，理论上两者差距不大。 Q2：理论上chunkPB是要大于rpc请求的生命周期 1. 发送端如何去判断请求已经发送完了呢？同步请求rpc server，_stub函数返回时整个rpc交互已经完成收到response信息；异步请求rpc，_stub函数返回只代表请求提交给了rpc框架，结束时调用用户的回调函数；brpc应该没提供接口告诉用户当前请求已经发送了。 2. 在调closure前rpc并未结束，chunkPB都是有效的。 个人看法，如有错误还请指教~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@brpc.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@brpc.apache.org For additional commands, e-mail: dev-h...@brpc.apache.org
