[
https://issues.apache.org/jira/browse/ARROW-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16566448#comment-16566448
]
Robert Nishihara commented on ARROW-2956:
-----------------------------------------
I'm not sure we support memory sizes this small. However, if we don't support
it, then we need to fail when starting the plasma store in the first place
instead of letting it fail later.
This issue probably has to do with the default malloc sizes in this file
https://github.com/apache/arrow/blob/d48dce2cfebdbd044a8260d0a77f5fe3d89a4a2d/cpp/src/plasma/malloc.cc#L47
> [Python]Arrow plasma throws ArrowIOError and process crashed
> ------------------------------------------------------------
>
> Key: ARROW-2956
> URL: https://issues.apache.org/jira/browse/ARROW-2956
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Reporter: He Kaisheng
> Priority: Major
>
> hello,
> We start a plasma store with 100k memory. when storage is full, it throws
> ArrowIOError and the *process crashed,* not the expected PlasmaStoreFull
> error.
> code:
> {code:java}
> import pyarrow.plasma as plasma
> import numpy as np
> plasma_client = plasma.connect(plasma_socket, '', 0)
> ref = []
> for _ in range(1000):
> obj_id = plasma_client.put(np.random.randint(100, size=(100, 100),
> dtype=np.int16))
> data = plasma_client.get(obj_id)
> ref.append(data)
> {code}
> error:
> {noformat}
> ---------------------------------------------------------------------------
> ArrowIOError Traceback (most recent call last)
> <ipython-input-2-07ec7f6f4b82> in <module>()
> 2 ref = []
> 3 for _ in range(1000):
> ----> 4 obj_id = plasma_client.put(np.random.randint(100, size=(100,
> 100), dtype=np.int16))
> 5 data = plasma_client.get(obj_id)
> 6 ref.append(data)
> plasma.pyx in pyarrow.plasma.PlasmaClient.put()
> plasma.pyx in pyarrow.plasma.PlasmaClient.create()
> error.pxi in pyarrow.lib.check_status()
> ArrowIOError: Encountered unexpected EOF{noformat}
> this problem doesn't exist when dtype is np.int64 or share memory is
> larger(like more than 100M), it seems so strange, anybody knows the reason?
> Thanks a lot.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
