Lu Qi created ARROW-1792:
-----------------------------
Summary: [Plasma C++] continuous write tensor failed
Key: ARROW-1792
URL: https://issues.apache.org/jira/browse/ARROW-1792
Project: Apache Arrow
Issue Type: Bug
Components: Plasma (C++)
Environment: ubuntu 14.04 gcc 4.8.4
Reporter: Lu Qi
start plasma using "plasma_store -m 8000000000 -s /tmp/plasma"
write tensor in python using
for i in range(10):
client = plasma.connect("/tmp/plasma", "", 0)
x = np.random.rand(1000,1000,5*256).astype("float32") // write 5 GB
object_id = pa.plasma.ObjectID(random_object_id())
tensor = pa.Tensor.from_numpy(x)
data_size = pa.get_tensor_size(tensor)
buf = client.create(object_id, data_size)
stream = pa.FixedSizeBufferWriter(buf)
stream.set_memcopy_threads(6)
pa.write_tensor(tensor, stream)
client.seal(object_id)
// client.release(object_id)
print(i)
The error is like below:
pyarrow.lib.PlasmaStoreFull: object does not fit in the plasma store
If I add "client.release(object_id)" ,the error is:
/arrow/cpp/src/plasma/client.cc296 Check failed: object_entry !=
objects_in_use_.end()
Also,sometimes error is:
buf = client.create(object_id, data_size)
File "pyarrow/plasma.pyx", line 301, in pyarrow.plasma.PlasmaClient.create
(/arrow/python/build/temp.linux-x86_64-2.7/plasma.cxx:4382)
File "pyarrow/error.pxi", line 79, in pyarrow.lib.check_status
(/arrow/python/build/temp.linux-x86_64-2.7/lib.cxx:7888)
pyarrow.lib.ArrowIOError: Broken pipe
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
