Yuxin Wu created ARROW-2045:
-------------------------------
Summary: More primitive operations on plasma store
Key: ARROW-2045
URL: https://issues.apache.org/jira/browse/ARROW-2045
Project: Apache Arrow
Issue Type: New Feature
Components: Plasma (C++)
Reporter: Yuxin Wu
Hi Developers,
I found plasma store very useful – it's fast and simple to use. However, I
think there are more operations that can make it a more general IPC/messaging
tool and potentially helpful in more scenarios.
Conceptually, an object store can support the following "put" methods:
# Evict when full
# Wait for space when full, perhaps with a timeout (i.e. blocking)
# Return failure when full (i.e. non-blocking)
And the following "get" methods:
# Wait for the object to appear (i.e. blocking)
# Return failure when object doesn't exist (i.e. non-blocking)
# Remove the object after get
Some of the above features can be implemented with others. But some of them are
primitives (e.g. return failure when full) that needs to be supported.
My use case: I wanted to use plasma to send/recv large buffers between
processes, i.e. build a message passing interface on top of shared memory.
Plasma has made it quite easy (only have to send/recv the id) and efficient
(faster than unix pipe). But "evict when full" is now the only available "put"
method, so that could create many trouble if I want to ensure message delivery.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
