[
https://issues.apache.org/jira/browse/ARROW-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16326742#comment-16326742
]
ASF GitHub Bot commented on ARROW-2000:
---------------------------------------
robertnishihara commented on a change in pull request #1479: ARROW-2000:
[Plasma] Deduplicate file descriptors when replying to GetRequest.
URL: https://github.com/apache/arrow/pull/1479#discussion_r161659428
##########
File path: cpp/src/plasma/format/plasma.fbs
##########
@@ -156,9 +156,17 @@ table PlasmaGetReply {
// objects if not all requested objects are stored and sealed
// in the local Plasma store.
object_ids: [string];
- // Plasma object information, in the same order as their IDs.
+ // Plasma object information, in the same order as their IDs. The number of
+ // elements in both object_ids and plasma_objects arrays must agree.
plasma_objects: [PlasmaObjectSpec];
- // The number of elements in both object_ids and plasma_objects arrays must
agree.
+ // A list of the file descriptors in the store that correspond to the file
+ // descriptors being sent to the client. The length of this list is the
number
+ // of file descriptors that the store will send to the client after this
+ // message.
+ store_file_descriptors: [int];
+ // Size in bytes of the segment for each store file descriptor (needed to
call
+ // mmap). This list must have the same length as store_file_descriptors.
+ mmap_sizes: [long];
Review comment:
Consider getting rid of this info from the `PlasmaObjectSpec`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Deduplicate file descriptors when plasma store replies to get request.
> ----------------------------------------------------------------------
>
> Key: ARROW-2000
> URL: https://issues.apache.org/jira/browse/ARROW-2000
> Project: Apache Arrow
> Issue Type: Bug
> Components: Plasma (C++)
> Reporter: Robert Nishihara
> Assignee: Robert Nishihara
> Priority: Major
> Labels: pull-request-available
>
> Right now when the plasma store replies to a GetRequest from a client, it
> sends many file descriptors over the relevant socket (by calling
> {{send_fd}}). However, many of these file descriptors are redundant and so we
> should deduplicate them before sending.
>
> Note that I often see the error "Failed to send file descriptor, retrying."
> printed when getting around 100 objects from the store. This may alleviate
> that.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
