Raúl Cumplido created ARROW-17014:
-------------------------------------
Summary: [Python][C++] Appveyor cython tests are failing
Key: ARROW-17014
URL: https://issues.apache.org/jira/browse/ARROW-17014
Project: Apache Arrow
Issue Type: Bug
Components: Continuous Integration, Python
Reporter: Raúl Cumplido
Fix For: 9.0.0
Two cython tests are failing on Appveyor builds
(https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/44114739/job/rresg43jos1wpbtv):
{code:java}
_______________________________ test_cython_api
_______________________________tmpdir =
local('C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\pytest-of-appveyor\\pytest-0\\test_cython_api0')
@pytest.mark.cython def test_cython_api(tmpdir): """ Basic
test for the Cython API. """ # Fail early if cython is not found
import cython # noqa with tmpdir.as_cwd(): # Set
up temporary workspace pyx_file = 'pyarrow_cython_example.pyx'
shutil.copyfile(os.path.join(here, pyx_file),
os.path.join(str(tmpdir), pyx_file)) # Create setup.py file
setup_code = setup_template.format(pyx_file=pyx_file,
compiler_opts=compiler_opts,
test_ld_path=test_ld_path) with open('setup.py',
'w') as f: f.write(setup_code) # ARROW-2263: Make
environment with this pyarrow/ package first on the # PYTHONPATH,
for local dev environments subprocess_env =
test_util.get_modified_env_with_pythonpath() # Compile extension
module> subprocess.check_call([sys.executable, 'setup.py',
'build_ext', '--inplace'],
env=subprocess_env)pyarrow\tests\test_cython.py:107: _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _popenargs =
(['C:\\Miniconda37-x64\\envs\\arrow\\python.exe', 'setup.py', 'build_ext',
'--inplace'],)kwargs = {'env': {'7ZIP': '"C:\\Program Files\\7-Zip\\7z.exe"',
'ALLUSERSPROFILE': 'C:\\ProgramData', 'APPDATA':
'C:\\Users\\appveyor\\AppData\\Roaming', 'APPVEYOR': 'True', ...}}retcode =
1cmd = ['C:\\Miniconda37-x64\\envs\\arrow\\python.exe', 'setup.py',
'build_ext', '--inplace'] def check_call(*popenargs, **kwargs):
"""Run command with arguments. Wait for command to complete. If the
exit code was zero then return, otherwise raise CalledProcessError. The
CalledProcessError object will have the return code in the returncode
attribute. The arguments are the same as for the call function.
Example: check_call(["ls", "-l"]) """ retcode =
call(*popenargs, **kwargs) if retcode: cmd =
kwargs.get("args") if cmd is None: cmd =
popenargs[0]> raise CalledProcessError(retcode, cmd)E
subprocess.CalledProcessError: Command
'['C:\\Miniconda37-x64\\envs\\arrow\\python.exe', 'setup.py', 'build_ext',
'--inplace']' returned non-zero exit status
1.C:\Miniconda37-x64\envs\arrow\lib\subprocess.py:364:
CalledProcessError---------------------------- Captured stdout call
-----------------------------Compiling pyarrow_cython_example.pyx because it
changed.[1/1] Cythonizing pyarrow_cython_example.pyxExtension module:
<setuptools.extension.Extension('pyarrow_cython_example') at 0x2804221ae10>
['C:\\Miniconda37-x64\\envs\\arrow\\lib\\site-packages\\numpy\\core\\include',
'c:\\projects\\arrow\\python\\pyarrow\\include'] ['arrow', 'arrow_python']
['c:\\projects\\arrow\\python\\pyarrow',
'C:/Miniconda37-x64/envs/arrow/Library/lib',
'C:\\Miniconda37-x64\\envs\\arrow\\Library\\lib']running build_extbuilding
'pyarrow_cython_example' extensioncreating buildcreating
build\temp.win-amd64-cpython-38creating
build\temp.win-amd64-cpython-38\Release"C:\Program Files (x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe" /c
/nologo /O2 /W3 /GL /DNDEBUG /MD
-IC:\Miniconda37-x64\envs\arrow\lib\site-packages\numpy\core\include
-Ic:\projects\arrow\python\pyarrow\include
-IC:\Miniconda37-x64\envs\arrow\include -IC:\Miniconda37-x64\envs\arrow\Include
"-IC:\Program Files (x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\ATLMFC\include" "-IC:\Program
Files (x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files
(x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\cppwinrt" "-IC:\Program Files (x86)\Microsoft
Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\ATLMFC\include"
"-IC:\Program Files (x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files
(x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\cppwinrt" /EHsc /Tppyarrow_cython_example.cpp
/Fobuild\temp.win-amd64-cpython-38\Release\pyarrow_cython_example.objpyarrow_cython_example.cppC:\Program
Files (x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1271):
error C2338: You've instantiated std::aligned_storage<Len, Align> with an
extended alignment (in other words, Align > alignof(max_align_t)). Before VS
2017 15.8, the member type would non-conformingly have an alignment of only
alignof(max_align_t). VS 2017 15.8 was fixed to handle this correctly, but the
fix inherently changes layout and breaks binary compatibility (*only* for uses
of aligned_storage with extended alignments). Please define either (1)
_ENABLE_EXTENDED_ALIGNED_STORAGE to acknowledge that you understand this
message and that you actually want a type with an extended alignment, or (2)
_DISABLE_EXTENDED_ALIGNED_STORAGE to silence this message and get the old
non-conformant behavior.C:\Program Files (x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1291):
note: see reference to class template instantiation
'std::_Aligned<192,64,double,false>' being compiledC:\Program Files
(x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1298):
note: see reference to class template instantiation
'std::_Aligned<192,64,int,false>' being compiledC:\Program Files
(x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1305):
note: see reference to class template instantiation
'std::_Aligned<192,64,short,false>' being compiledC:\Program Files
(x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1312):
note: see reference to class template instantiation
'std::_Aligned<192,64,char,false>' being compiledC:\Program Files
(x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1320):
note: see reference to class template instantiation
'std::aligned_storage<192,64>' being compiledC:\Program Files (x86)\Microsoft
Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1352):
note: see reference to alias template instantiation
'std::aligned_storage_t<192,64>' being
compiledc:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(132):
note: see reference to class template instantiation
'std::aligned_union<0,arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>'
being
compiledc:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(140):
note: see reference to class template instantiation
'arrow::util::detail::VariantStorage<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>'
being
compiledc:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(157):
note: see reference to class template instantiation
'arrow::util::detail::VariantImpl<arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>>'
being
compiledc:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(157):
note: see reference to class template instantiation
'arrow::util::detail::VariantImpl<arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>,std::shared_ptr<arrow::ArrayData>>'
being
compiledc:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(245):
note: see reference to class template instantiation
'arrow::util::detail::VariantImpl<arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>,arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>'
being
compiledc:\projects\arrow\python\pyarrow\include\arrow/compute/exec.h(313):
note: see reference to class template instantiation
'arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>'
being
compiledc:\projects\arrow\python\pyarrow\include\arrow/vendored/datetime/date.h(182):
note: see reference to class template instantiation
'std::chrono::duration<int,std::ratio<86400,1>>' being
compiled---------------------------- Captured stderr call
-----------------------------error: command 'C:\\Program Files (x86)\\Microsoft
Visual
Studio\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX86\\x64\\cl.exe'
failed with exit code 2_____________________________ test_visit_strings
______________________________tmpdir =
local('C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\pytest-of-appveyor\\pytest-0\\test_visit_strings0')
@pytest.mark.cython def test_visit_strings(tmpdir): with
tmpdir.as_cwd(): # Set up temporary workspace pyx_file =
'bound_function_visit_strings.pyx'
shutil.copyfile(os.path.join(here, pyx_file),
os.path.join(str(tmpdir), pyx_file)) # Create setup.py file
setup_code = setup_template.format(pyx_file=pyx_file,
compiler_opts=compiler_opts,
test_ld_path=test_ld_path) with open('setup.py',
'w') as f: f.write(setup_code) subprocess_env =
test_util.get_modified_env_with_pythonpath() # Compile extension
module> subprocess.check_call([sys.executable, 'setup.py',
'build_ext', '--inplace'],
env=subprocess_env)pyarrow\tests\test_cython.py:162: _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _popenargs =
(['C:\\Miniconda37-x64\\envs\\arrow\\python.exe', 'setup.py', 'build_ext',
'--inplace'],)kwargs = {'env': {'7ZIP': '"C:\\Program Files\\7-Zip\\7z.exe"',
'ALLUSERSPROFILE': 'C:\\ProgramData', 'APPDATA':
'C:\\Users\\appveyor\\AppData\\Roaming', 'APPVEYOR': 'True', ...}}retcode =
1cmd = ['C:\\Miniconda37-x64\\envs\\arrow\\python.exe', 'setup.py',
'build_ext', '--inplace'] def check_call(*popenargs, **kwargs):
"""Run command with arguments. Wait for command to complete. If the
exit code was zero then return, otherwise raise CalledProcessError. The
CalledProcessError object will have the return code in the returncode
attribute. The arguments are the same as for the call function.
Example: check_call(["ls", "-l"]) """ retcode =
call(*popenargs, **kwargs) if retcode: cmd =
kwargs.get("args") if cmd is None: cmd =
popenargs[0]> raise CalledProcessError(retcode, cmd)E
subprocess.CalledProcessError: Command
'['C:\\Miniconda37-x64\\envs\\arrow\\python.exe', 'setup.py', 'build_ext',
'--inplace']' returned non-zero exit status
1.C:\Miniconda37-x64\envs\arrow\lib\subprocess.py:364:
CalledProcessError---------------------------- Captured stdout call
-----------------------------Compiling bound_function_visit_strings.pyx because
it changed.[1/1] Cythonizing bound_function_visit_strings.pyxExtension module:
<setuptools.extension.Extension('bound_function_visit_strings') at
0x28ba9d0df00>
['C:\\Miniconda37-x64\\envs\\arrow\\lib\\site-packages\\numpy\\core\\include',
'c:\\projects\\arrow\\python\\pyarrow\\include'] ['arrow', 'arrow_python']
['c:\\projects\\arrow\\python\\pyarrow',
'C:/Miniconda37-x64/envs/arrow/Library/lib',
'C:\\Miniconda37-x64\\envs\\arrow\\Library\\lib']running build_extbuilding
'bound_function_visit_strings' extensioncreating buildcreating
build\temp.win-amd64-cpython-38creating
build\temp.win-amd64-cpython-38\Release"C:\Program Files (x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe" /c
/nologo /O2 /W3 /GL /DNDEBUG /MD
-IC:\Miniconda37-x64\envs\arrow\lib\site-packages\numpy\core\include
-Ic:\projects\arrow\python\pyarrow\include
-IC:\Miniconda37-x64\envs\arrow\include -IC:\Miniconda37-x64\envs\arrow\Include
"-IC:\Program Files (x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\ATLMFC\include" "-IC:\Program
Files (x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files
(x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\cppwinrt" "-IC:\Program Files (x86)\Microsoft
Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\ATLMFC\include"
"-IC:\Program Files (x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files
(x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\cppwinrt" /EHsc
/Tpbound_function_visit_strings.cpp
/Fobuild\temp.win-amd64-cpython-38\Release\bound_function_visit_strings.objbound_function_visit_strings.cppC:\Program
Files (x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1271):
error C2338: You've instantiated std::aligned_storage<Len, Align> with an
extended alignment (in other words, Align > alignof(max_align_t)). Before VS
2017 15.8, the member type would non-conformingly have an alignment of only
alignof(max_align_t). VS 2017 15.8 was fixed to handle this correctly, but the
fix inherently changes layout and breaks binary compatibility (*only* for uses
of aligned_storage with extended alignments). Please define either (1)
_ENABLE_EXTENDED_ALIGNED_STORAGE to acknowledge that you understand this
message and that you actually want a type with an extended alignment, or (2)
_DISABLE_EXTENDED_ALIGNED_STORAGE to silence this message and get the old
non-conformant behavior.C:\Program Files (x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1291):
note: see reference to class template instantiation
'std::_Aligned<192,64,double,false>' being compiledC:\Program Files
(x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1298):
note: see reference to class template instantiation
'std::_Aligned<192,64,int,false>' being compiledC:\Program Files
(x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1305):
note: see reference to class template instantiation
'std::_Aligned<192,64,short,false>' being compiledC:\Program Files
(x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1312):
note: see reference to class template instantiation
'std::_Aligned<192,64,char,false>' being compiledC:\Program Files
(x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1320):
note: see reference to class template instantiation
'std::aligned_storage<192,64>' being compiledC:\Program Files (x86)\Microsoft
Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1352):
note: see reference to alias template instantiation
'std::aligned_storage_t<192,64>' being
compiledc:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(132):
note: see reference to class template instantiation
'std::aligned_union<0,arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>'
being
compiledc:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(140):
note: see reference to class template instantiation
'arrow::util::detail::VariantStorage<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>'
being
compiledc:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(157):
note: see reference to class template instantiation
'arrow::util::detail::VariantImpl<arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>>'
being
compiledc:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(157):
note: see reference to class template instantiation
'arrow::util::detail::VariantImpl<arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>,std::shared_ptr<arrow::ArrayData>>'
being
compiledc:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(245):
note: see reference to class template instantiation
'arrow::util::detail::VariantImpl<arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>,arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>'
being
compiledc:\projects\arrow\python\pyarrow\include\arrow/compute/exec.h(313):
note: see reference to class template instantiation
'arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>'
being
compiledc:\projects\arrow\python\pyarrow\include\arrow/vendored/datetime/date.h(182):
note: see reference to class template instantiation
'std::chrono::duration<int,std::ratio<86400,1>>' being
compiled---------------------------- Captured stderr call
-----------------------------error: command 'C:\\Program Files (x86)\\Microsoft
Visual
Studio\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX86\\x64\\cl.exe'
failed with exit code 2 {code}
It seems to have been failing on master since this was merged
https://github.com/apache/arrow/commit/6cc37cf2d1ba72c46b64fbc7ac499bd0d7296d20
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
