[
https://issues.apache.org/jira/browse/ARROW-12241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316047#comment-17316047
]
Yibo Cai commented on ARROW-12241:
----------------------------------
Hmm..., looks the keyboard interrupt is generated deliberately to test the
cancellation. But why it stops the pytest main process?
> [Python] Parallel csv reader test failed
> ----------------------------------------
>
> Key: ARROW-12241
> URL: https://issues.apache.org/jira/browse/ARROW-12241
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Reporter: Yibo Cai
> Priority: Major
>
> CI job is okay, but it failed from my side. Tested on x86 skylake server with
> 32 cores, and Apple M1 with 8 cores.
> Maybe I missed something?
> Test steps:
> {code:bash}
> $ cmake -GNinja -DCMAKE_BUILD_TYPE=Release -DARROW_COMPUTE=ON
> -DARROW_PARQUET=ON -DARROW_BUILD_TESTS=ON
> -DCMAKE_INSTALL_PREFIX=$(pwd)/_install -DCMAKE_INSTALL_LIBDIR=lib
> -DARROW_PYTHON=ON -DCMAKE_CXX_COMPILER=/usr/bin/clang++-9
> -DCMAKE_C_COMPILER=/usr/bin/clang-9 ..
> $ ninja install
> $ cd ~/arrow/python
> # set LD_LIBRARY_PATH, ARROW_HOME to newly built binaries
> $ python setup.py build_ext --inplace
> $ pytest pyarrow
> {code}
> Error log:
> {code:bash}
> =======================================================================================
> test session starts
> ========================================================================================
> platform linux -- Python 3.6.9, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
> rootdir: /home/cyb/arrow/python, configfile: setup.cfg
> plugins: lazy-fixture-0.6.3, hypothesis-5.41.2
> collected 3802 items / 3 skipped / 3799 selected
>
>
> pyarrow/tests/test_adhoc_memory_leak.py s
>
> [ 0%]
> pyarrow/tests/test_array.py
> ......................s...........................................................................................s.............................................
> [ 4%]
> ............................ss.........
>
> [ 5%]
> pyarrow/tests/test_builder.py ....
>
> [ 5%]
> pyarrow/tests/test_cffi.py ......
>
> [ 5%]
> pyarrow/tests/test_compute.py
> ....................................................................................................................................................
> [ 9%]
> pyarrow/tests/test_convert_builtin.py
> ......................................................................................................................................................
> [ 13%]
> ................................................................................................................................x..................................................x........
> [ 18%]
> ...............ssssss.....................................................................................................sssssss
> [ 21%]
> pyarrow/tests/test_csv.py
> ....................................F.......................
> =============================================================================================
> FAILURES
> =============================================================================================
> ______________________________________________________________________________
> TestParallelCSVRead.test_cancellation
> _______________________________________________________________________________
> self = <pyarrow.tests.test_csv.TestParallelCSVRead
> testMethod=test_cancellation>
> def test_cancellation(self):
> if (threading.current_thread().ident !=
> threading.main_thread().ident):
> pytest.skip("test only works from main Python thread")
> if sys.version_info >= (3, 8):
> raise_signal = signal.raise_signal
> elif os.name == 'nt':
> # On Windows, os.kill() doesn't actually send a signal,
> # it just terminates the process with the given exit code.
> pytest.skip("test requires Python 3.8+ on Windows")
> else:
> # On Unix, emulate raise_signal() with os.kill().
> def raise_signal(signum):
> os.kill(os.getpid(), signum)
> large_csv = b"a,b,c\n" + b"1,2,3\n" * 30000000
> def signal_from_thread():
> time.sleep(0.2)
> raise_signal(signal.SIGINT)
> t1 = time.time()
> with pytest.raises(KeyboardInterrupt) as exc_info:
> threading.Thread(target=signal_from_thread).start()
> > self.read_bytes(large_csv)
> E Failed: DID NOT RAISE <class 'KeyboardInterrupt'>
> pyarrow/tests/test_csv.py:927: Failed
> =========================================================================================
> warnings summary
> =========================================================================================
> ../../archery/lib/python3.6/distutils/__init__.py:4
> /home/cyb/archery/lib/python3.6/distutils/__init__.py:4:
> DeprecationWarning: the imp module is deprecated in favour of importlib; see
> the module's documentation for alternative uses
> import imp
> -- Docs: https://docs.pytest.org/en/stable/warnings.html
> =====================================================================================
> short test summary info
> ======================================================================================
> FAILED pyarrow/tests/test_csv.py::TestParallelCSVRead::test_cancellation -
> Failed: DID NOT RAISE <class 'KeyboardInterrupt'>
> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> KeyboardInterrupt
> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> pyarrow/error.pxi:221: KeyboardInterrupt
> (to show a full traceback on KeyboardInterrupt use --full-trace)
> ================================================================= 1 failed,
> 864 passed, 21 skipped, 2 xfailed, 1 warning in 4.19s
> ==================================================================
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
