[
https://issues.apache.org/jira/browse/ARROW-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383324#comment-16383324
]
Antoine Pitrou commented on ARROW-2237:
---------------------------------------
By the way, setting up 20000 huge pages seems excessive. Here, {{echo 20000 >
/proc/sys/vm/nr_hugepages}} swaps wildly as it seems to evict existing normal
pages to make place for huge pages (am I analyzing this right?). This is using
a {{4.13.0}} kernel on a 16GB RAM machine.
Otherwise, yes, configuring {{/proc/sys/vm/hugetlb_shm_group}} and
{{/proc/sys/vm/nr_hugepages}} and fixing the test to use {{/dev/hugepages}}
makes it pass here. I wonder if there's a way to skip it or otherwise give
better directions if hugepages support isn't usable on the system.
> [Python] Huge tables test failure
> ---------------------------------
>
> Key: ARROW-2237
> URL: https://issues.apache.org/jira/browse/ARROW-2237
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Reporter: Antoine Pitrou
> Priority: Major
> Fix For: 0.9.0
>
>
> This is a new failure here (Ubuntu 16.04, x86-64):
> {code}
> _____________________________ test_use_huge_pages
> _____________________________
> Traceback (most recent call last):
> File "/home/antoine/arrow/python/pyarrow/tests/test_plasma.py", line 779,
> in test_use_huge_pages
> create_object(plasma_client, 100000000)
> File "/home/antoine/arrow/python/pyarrow/tests/test_plasma.py", line 80, in
> create_object
> seal=seal)
> File "/home/antoine/arrow/python/pyarrow/tests/test_plasma.py", line 69, in
> create_object_with_id
> memory_buffer = client.create(object_id, data_size, metadata)
> File "plasma.pyx", line 302, in pyarrow.plasma.PlasmaClient.create
> File "error.pxi", line 79, in pyarrow.lib.check_status
> pyarrow.lib.ArrowIOError: /home/antoine/arrow/cpp/src/plasma/client.cc:192
> code: PlasmaReceive(store_conn_, MessageType_PlasmaCreateReply, &buffer)
> /home/antoine/arrow/cpp/src/plasma/protocol.cc:46 code: ReadMessage(sock,
> &type, buffer)
> Encountered unexpected EOF
> ---------------------------- Captured stderr call
> -----------------------------
> Allowing the Plasma store to use up to 0.1GB of memory.
> Starting object store with directory /mnt/hugepages and huge page support
> enabled
> create_buffer failed to open file /mnt/hugepages/plasmapSNc0X
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
