[ 
https://issues.apache.org/jira/browse/ARROW-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383324#comment-16383324
 ] 

Antoine Pitrou commented on ARROW-2237:
---------------------------------------

By the way, setting up 20000 huge pages seems excessive. Here, {{echo 20000 > 
/proc/sys/vm/nr_hugepages}} swaps wildly as it seems to evict existing normal 
pages to make place for huge pages (am I analyzing this right?). This is using 
a {{4.13.0}} kernel on a 16GB RAM machine.

Otherwise, yes, configuring {{/proc/sys/vm/hugetlb_shm_group}} and 
{{/proc/sys/vm/nr_hugepages}} and fixing the test to use {{/dev/hugepages}} 
makes it pass here. I wonder if there's a way to skip it or otherwise give 
better directions if hugepages support isn't usable on the system.

> [Python] Huge tables test failure
> ---------------------------------
>
>                 Key: ARROW-2237
>                 URL: https://issues.apache.org/jira/browse/ARROW-2237
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>            Reporter: Antoine Pitrou
>            Priority: Major
>             Fix For: 0.9.0
>
>
> This is a new failure here (Ubuntu 16.04, x86-64):
> {code}
> _____________________________ test_use_huge_pages 
> _____________________________
> Traceback (most recent call last):
>   File "/home/antoine/arrow/python/pyarrow/tests/test_plasma.py", line 779, 
> in test_use_huge_pages
>     create_object(plasma_client, 100000000)
>   File "/home/antoine/arrow/python/pyarrow/tests/test_plasma.py", line 80, in 
> create_object
>     seal=seal)
>   File "/home/antoine/arrow/python/pyarrow/tests/test_plasma.py", line 69, in 
> create_object_with_id
>     memory_buffer = client.create(object_id, data_size, metadata)
>   File "plasma.pyx", line 302, in pyarrow.plasma.PlasmaClient.create
>   File "error.pxi", line 79, in pyarrow.lib.check_status
> pyarrow.lib.ArrowIOError: /home/antoine/arrow/cpp/src/plasma/client.cc:192 
> code: PlasmaReceive(store_conn_, MessageType_PlasmaCreateReply, &buffer)
> /home/antoine/arrow/cpp/src/plasma/protocol.cc:46 code: ReadMessage(sock, 
> &type, buffer)
> Encountered unexpected EOF
> ---------------------------- Captured stderr call 
> -----------------------------
> Allowing the Plasma store to use up to 0.1GB of memory.
> Starting object store with directory /mnt/hugepages and huge page support 
> enabled
> create_buffer failed to open file /mnt/hugepages/plasmapSNc0X
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to