Re: [pypy-dev] Link errors while translating with VS2010 and 64bit
Hi Tasos, On Wed, Feb 2, 2011 at 1:15 AM, Tasos Vogiatzoglou tvog...@gmail.com wrote: You are right. My mistake. It was picking the 2.6 32bit python. Now is working and tests are passing A few extra notes. What is needed to make this work is three-fold: * make sure that in the C code, intptr_t is generated instead of long for lltype.Signed, and similarly uintptr_t instead of unsigned long for lltype.Unsigned. This involves mainly fixing translator/c/primitive.py to declare types using intptr_t instead of long, and probably fixing a number of other uses of long in the .py files in that directory. * make sure that in the hand-written headers translator/c/src/*.h, we use intptr_t instead of long whenever it's needed (mostly everywhere, I suppose, but not absolutely everywhere, e.g. not in calls to external C functions that are declared to take or return a long). The changes above should fix the generation of code. Now the third part is the longest, probably. You need to distinguish the two types used in RPython, which are lltype.Signed and rffi.LONG, and which are equal so far. Of course rffi.LONG should remain equivalent to the C long. This probably needs looking around in all modules and in pypy/rlib/, to make sure that we use the correct one, either lltype.Signed or rffi.LONG. Fortunately, errors resulting from this confusion will probably just give translation-time errors, so we can fix the places one after the other. But you need to carefully check the declaration of external functions that are implemented in translator/c/src/*.h, like pypy/module/signal/interp_signal.py (this one happens to use mostly rffi.INT, so it should work right now, with the exception of LONG_STRUCT that uses a lltype.Signed). A bientôt, Armin. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Link errors while translating with VS2010 and 64bit
Armin, thanks for the directions. I'll start working on them and I'll report back. Regards, Tasos On 3 Φεβ 2011, at 14:30, Armin Rigo ar...@tunes.org wrote: Hi Tasos, On Wed, Feb 2, 2011 at 1:15 AM, Tasos Vogiatzoglou tvog...@gmail.com wrote: You are right. My mistake. It was picking the 2.6 32bit python. Now is working and tests are passing A few extra notes. What is needed to make this work is three-fold: * make sure that in the C code, intptr_t is generated instead of long for lltype.Signed, and similarly uintptr_t instead of unsigned long for lltype.Unsigned. This involves mainly fixing translator/c/primitive.py to declare types using intptr_t instead of long, and probably fixing a number of other uses of long in the .py files in that directory. * make sure that in the hand-written headers translator/c/src/*.h, we use intptr_t instead of long whenever it's needed (mostly everywhere, I suppose, but not absolutely everywhere, e.g. not in calls to external C functions that are declared to take or return a long). The changes above should fix the generation of code. Now the third part is the longest, probably. You need to distinguish the two types used in RPython, which are lltype.Signed and rffi.LONG, and which are equal so far. Of course rffi.LONG should remain equivalent to the C long. This probably needs looking around in all modules and in pypy/rlib/, to make sure that we use the correct one, either lltype.Signed or rffi.LONG. Fortunately, errors resulting from this confusion will probably just give translation-time errors, so we can fix the places one after the other. But you need to carefully check the declaration of external functions that are implemented in translator/c/src/*.h, like pypy/module/signal/interp_signal.py (this one happens to use mostly rffi.INT, so it should work right now, with the exception of LONG_STRUCT that uses a lltype.Signed). A bientôt, Armin. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
[pypy-dev] Link errors while translating with VS2010 and 64bit
Amaury, It seems that there is a general issue with the compiler/link . I did the translations without the _hashlib and ssl and after a while I got the following errors. [platform:ERROR]Creating library c:\users\flatline\appdata\local\temp\usessi on-default-57\testing_1\libpypy-c.lib and object c:\users\flatline\appdata\local\temp\usession-default-57\testing_1\libpypy-c.exp [platform:ERROR] implement_52.obj : error LNK2019: unresolved external symbol _inflateEnd referenced in function _pypy_g_ccall_inflateEnd__z_streamPtr [platform:ERROR] implement_64.obj : error LNK2019: unresolved external symbol _inflateInit2_ referenced in function _pypy_g_ccall_inflateInit2___z_streamPtr_Sig ned_arrayPtr [platform:ERROR] implement_64.obj : error LNK2019: unresolved external symbol _inflate referenced in function _pypy_g_ccall_inflate__z_streamPtr_Signed [platform:ERROR] implement_67.obj : error LNK2019: unresolved external symbol _deflateEnd referenced in function _pypy_g_ccall_deflateEnd__z_streamPtr [platform:ERROR] implement_69.obj : error LNK2019: unresolved external symbol _deflate referenced in function _pypy_g_ccall_deflate__z_streamPtr_Signed [platform:ERROR] implement_70.obj : error LNK2019: unresolved external symbol _adler32 referenced in function _pypy_g_ccall_adler32__Unsigned_arrayPtr_Unsigned [platform:ERROR] implement_71.obj : error LNK2019: unresolved external symbol _deflateInit2_ referenced in function _pypy_g_ccall_deflateInit2___z_streamPtr_Sig ned_Signed_S [platform:ERROR] implement_71.obj : error LNK2019: unresolved external symbol _crc32 referenced in function _pypy_g_ccall_crc32__Unsigned_arrayPtr_Unsigned [platform:ERROR] c:\users\flatline\appdata\local\temp\usession-default-57\testing_1\libpypy-c.dll : fatal error LNK1120: 8 unresolved externals [Timer] Timings: [Timer] annotate --- 1447.8 s [Timer] rtype_lltype --- 768.2 s [Timer] backendopt_lltype --- 415.9 s [Timer] stackcheckinsertion_lltype --- 60.1 s [Timer] database_c --- 498.3 s [Timer] source_c --- 1190.5 s [Timer] compile_c --- 216.4 s [Timer] === [Timer] Total: --- 4597.3 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR]File translate.py, line 290, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR]File c:\Users\flatline\development\pypy\src\pypy\translator\driver.py, line 812, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR]File c:\Users\flatline\development\pypy\src\pypy\translator\tool\taskengine.py, line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR]File c:\Users\flatline\development\pypy\src\pypy\translator\driver.py, line 288, in _do [translation:ERROR] res = func() [translation:ERROR]File c:\Users\flatline\development\pypy\src\pypy\translator\driver.py, line 575, in task_compile_c [translation:ERROR] cbuilder.compile(**kwds) [translation:ERROR]File c:\Users\flatline\development\pypy\src\pypy\translator\c\genc.py, line 527, in compile [translation:ERROR] self.executable_name = compiler.build(shared=shared) [translation:ERROR]File c:\Users\flatline\development\pypy\src\pypy\translator\c\genc.py, line 95, in build [translation:ERROR] return self._build(shared=shared) [translation:ERROR]File c:\Users\flatline\development\pypy\src\pypy\translator\c\genc.py, line 90, in _build [translation:ERROR] standalone=not shared) [translation:ERROR]File c:\Users\flatline\development\pypy\src\pypy\translator\platform\__init__.py, line 52, in compile [translation:ERROR] return self._finish_linking(ofiles, eci, outputfilename, standalone) [translation:ERROR]File c:\Users\flatline\development\pypy\src\pypy\translator\platform\__init__.py, line 194, in _finish_linking [translation:ERROR] return self._link(cc_link, ofiles, largs, standalone, exe_name) [translation:ERROR]File c:\Users\flatline\development\pypy\src\pypy\translator\platform\windows.py, line 176, in _link [translation:ERROR] self._execute_c_compiler(self.link, args, exe_name) [translation:ERROR]File c:\Users\flatline\development\pypy\src\pypy\translator\platform\__init__.py, line 118, in _execute_c_compiler [translation:ERROR] self._handle_error(returncode, stderr, stdout, outname) [translation:ERROR]File c:\Users\flatline\development\pypy\src\pypy\translator\platform\windows.py, line 199, in _handle_error [translation:ERROR] raise CompilationError(stdout, stderr) [translation:ERROR] CompilationError: CompilationError err=' Creating library c:\\users\\flatline\\appdata\\local\\temp\\usession-default-57\\testing_1\\lib pypy-c.lib and
Re: [pypy-dev] Link errors while translating with VS2010 and 64bit
Hi, 2011/2/1 Tasos Vogiatzoglou tvog...@gmail.com: Amaury, It seems that there is a general issue with the compiler/link . I did the translations without the _hashlib and ssl and after a while I got the following errors. [...] [platform:ERROR] implement_52.obj : error LNK2019: unresolved external symbol _inflateEnd referenced in function _pypy_g_ccall_inflateEnd__z_streamPtr We have never tried the win64 platform, and I don't have access to a Windows 64bit machine. But I suspect that even if you removed all external dependencies, the result would not work; pypy's compilation tools implicitly assume that sizeof(long)==sizeof(void*) Before running a translation, could you run the tests in pypy/translator/c ? in the pypy directory, run: python test_all.py translator/c I'd like at least the files test_genc and test_newgc to pass without errors. -- Amaury Forgeot d'Arc ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Link errors while translating with VS2010 and 64bit
On Tue, Feb 1, 2011 at 10:11 AM, Amaury Forgeot d'Arc amaur...@gmail.com wrote: Hi, 2011/2/1 Tasos Vogiatzoglou tvog...@gmail.com: Amaury, It seems that there is a general issue with the compiler/link . I did the translations without the _hashlib and ssl and after a while I got the following errors. [...] [platform:ERROR] implement_52.obj : error LNK2019: unresolved external symbol _inflateEnd referenced in function _pypy_g_ccall_inflateEnd__z_streamPtr We have never tried the win64 platform, and I don't have access to a Windows 64bit machine. But I suspect that even if you removed all external dependencies, the result would not work; pypy's compilation tools implicitly assume that sizeof(long)==sizeof(void*) Before running a translation, could you run the tests in pypy/translator/c ? in the pypy directory, run: python test_all.py translator/c I'd like at least the files test_genc and test_newgc to pass without errors. Surprisingly enough, I have -- Amaury Forgeot d'Arc ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Link errors while translating with VS2010 and 64bit
Amaury, I'll try and I'll let you know. Thanks, Tasos On 1 Φεβ 2011, at 10:11, Amaury Forgeot d'Arc amaur...@gmail.com wrote: Hi, 2011/2/1 Tasos Vogiatzoglou tvog...@gmail.com: Amaury, It seems that there is a general issue with the compiler/link . I did the translations without the _hashlib and ssl and after a while I got the following errors. [...] [platform:ERROR] implement_52.obj : error LNK2019: unresolved external symbol _inflateEnd referenced in function _pypy_g_ccall_inflateEnd__z_streamPtr We have never tried the win64 platform, and I don't have access to a Windows 64bit machine. But I suspect that even if you removed all external dependencies, the result would not work; pypy's compilation tools implicitly assume that sizeof(long)==sizeof(void*) Before running a translation, could you run the tests in pypy/translator/c ? in the pypy directory, run: python test_all.py translator/c I'd like at least the files test_genc and test_newgc to pass without errors. -- Amaury Forgeot d'Arc ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Link errors while translating with VS2010 and 64bit
Amaury, You were right, there are some issues with type sizes. test_genc and test_newgc are failing. I'll try to see what I can do to fix them. Are there any directions I should move towards ? Thanks, Tasos On Tue, Feb 1, 2011 at 10:20 AM, Tasos Vogiatzoglou tvog...@gmail.com wrote: Amaury, I'll try and I'll let you know. Thanks, Tasos On 1 Φεβ 2011, at 10:11, Amaury Forgeot d'Arc amaur...@gmail.com wrote: Hi, 2011/2/1 Tasos Vogiatzoglou tvog...@gmail.com: Amaury, It seems that there is a general issue with the compiler/link . I did the translations without the _hashlib and ssl and after a while I got the following errors. [...] [platform:ERROR] implement_52.obj : error LNK2019: unresolved external symbol _inflateEnd referenced in function _pypy_g_ccall_inflateEnd__z_streamPtr We have never tried the win64 platform, and I don't have access to a Windows 64bit machine. But I suspect that even if you removed all external dependencies, the result would not work; pypy's compilation tools implicitly assume that sizeof(long)==sizeof(void*) Before running a translation, could you run the tests in pypy/translator/c ? in the pypy directory, run: python test_all.py translator/c I'd like at least the files test_genc and test_newgc to pass without errors. -- Amaury Forgeot d'Arc ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Link errors while translating with VS2010 and 64bit
Hi, A naive guess would be the different sizes between 64bit windows, and 64 bit linux? (long int is 32bit on windows and 64bit on linux) From http://en.wikipedia.org/wiki/64-bit 64-bit data models Data model short (integer) int long (integer) long longpointers/size_t Sample operating systems LLP64/ IL32P64 16 32 32 64 64 Microsoft Windows (X64/IA-64) LP64/ I32LP64 16 32 64 64 64 Most Unix and Unix-like systems, e.g. Solaris, Linux, and Mac OS X ILP64 16 64 64 64 64 HAL Computer Systems port of Solaris to SPARC64 SILP64 64 64 64 64 64 Unicos On Tue, Feb 1, 2011 at 5:10 PM, Tasos Vogiatzoglou tvog...@gmail.com wrote: Amaury, You were right, there are some issues with type sizes. test_genc and test_newgc are failing. I'll try to see what I can do to fix them. Are there any directions I should move towards ? Thanks, Tasos On Tue, Feb 1, 2011 at 10:20 AM, Tasos Vogiatzoglou tvog...@gmail.com wrote: Amaury, I'll try and I'll let you know. Thanks, Tasos On 1 Φεβ 2011, at 10:11, Amaury Forgeot d'Arc amaur...@gmail.com wrote: Hi, 2011/2/1 Tasos Vogiatzoglou tvog...@gmail.com: Amaury, It seems that there is a general issue with the compiler/link . I did the translations without the _hashlib and ssl and after a while I got the following errors. [...] [platform:ERROR] implement_52.obj : error LNK2019: unresolved external symbol _inflateEnd referenced in function _pypy_g_ccall_inflateEnd__z_streamPtr We have never tried the win64 platform, and I don't have access to a Windows 64bit machine. But I suspect that even if you removed all external dependencies, the result would not work; pypy's compilation tools implicitly assume that sizeof(long)==sizeof(void*) Before running a translation, could you run the tests in pypy/translator/c ? in the pypy directory, run: python test_all.py translator/c I'd like at least the files test_genc and test_newgc to pass without errors. -- Amaury Forgeot d'Arc ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Link errors while translating with VS2010 and 64bit
2011/2/1 René Dudfield ren...@gmail.com: Hi, A naive guess would be the different sizes between 64bit windows, and 64 bit linux? (long int is 32bit on windows and 64bit on linux) From http://en.wikipedia.org/wiki/64-bit 64-bit data models Data model short (integer) int long (integer) long long pointers/size_t Sample operating systems LLP64/ IL32P64 16 32 32 64 64 Microsoft Windows (X64/IA-64) LP64/ I32LP64 16 32 64 64 64 Most Unix and Unix-like systems, e.g. Solaris, Linux, and Mac OS X ILP64 16 64 64 64 64 HAL Computer Systems port of Solaris to SPARC64 SILP64 64 64 64 64 64 Unicos From what Amaury said, you need to identify places where long is used and it is assumed that it has 64bits. A first guess, in general, is that _every_ use of long assumes it to be 64bit on 64bits systems. What about using intptr_t (or uintptr_t) from stdint.h? IOW, you'd simply replace long by this type everywhere, including in functions which generate C code through some library. I don't know the details, could somebody help here? IIRC this header was introduced in C99, and for sure this type is optional - i.e., it must be declared if it is available at all. Of course, there might be systems where this type is not available, so you need to provide a fallback (in a configure-like fashion). If this header is available on (modern) Windows environments (the only system where long is 32 bits), using long as fallback is safe. Cheers, -- Paolo Giarrusso - Ph.D. Student http://www.informatik.uni-marburg.de/~pgiarrusso/ ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Link errors while translating with VS2010 and 64bit
2011/2/2 Tasos Vogiatzoglou tvog...@gmail.com: After that change I had lot's of linking issues like [platform:ERROR] testing_25.obj : error LNK2019: unresolved external symbol __imp_Py_BuildValue referenced in function malloc_counters [platform:ERROR] testing_25.obj : error LNK2019: unresolved external symbol __imp_PyMethod_New referenced in function gencfunc_descr_get [platform:ERROR] testing_25.obj : error LNK2019: unresolved external symbol __imp__Py_NoneStruct referenced in function gencfunc_descr_get Any idea ? You are certainly building an extension module for the host Python (the one which runs the tests). You should check that the link command finds the correct Python27.lib, corresponding to your installation of Python. btw, are you really running a 64bit Python? -- Amaury Forgeot d'Arc ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
