hi Eric -- I don't see a particular issue with this. You might look at how Ray interacts with Plasma for scheduling tasks to its workers.
Are you running to any problems with this yet? - Wes On Thu, Jan 24, 2019 at 5:59 AM Eric Talbot <[email protected]> wrote: > > We're trying to set up a system that adds objects to plasma from > scala/JVM, and then a separate Python process consumes the objects. > > The Python piece looks something like this: > > client = plasma.connect(...)client.subscribe() > > while True: > > object_id, data_size, meta_size = client.get_next_notification() > if data_size >= 0: > data, = client.get_buffers([object_id]) > > func(pyarrow.BufferReader(data).read()) > client.delete([object_id]) > > > > > >
