[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-607061503 @leezu seems like adding both 5.2 and 7.0 and then dropping 7.0 would trigger a normal run...but each run should be independent, right? See here: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-gpu/detail/PR-17808/44/ With @josephevans 's fix + pip install I think the CI is working now. But we are getting some Jenkins issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-607061503 @leezu seems like adding both 5.2 and 7.0 and then dropping 7.0 would trigger a normal run...but each run should be independent, right? See here: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-gpu/detail/PR-17808/44/ With @josephevans 's fix I think the CI is working now. Update: still runs into segfault... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-607061503 @leezu seems like adding both 5.2 and 7.0 and then dropping 7.0 would trigger a normal run... See here: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-gpu/detail/PR-17808/44/ With @josephevans 's fix I think the CI is working now. Update: still runs into segfault... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-607061503 @leezu seems like adding both 5.2 and 7.0 and then drop 7.0 would trigger a normal run... See here: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-gpu/detail/PR-17808/44/ With @josephevans 's fix I think the CI is working now. Update: still run into segfault... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-607061503 @leezu seems like adding both 5.2 and 7.0 and then drop 7.0 would trigger a normal run... See here: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-gpu/detail/PR-17808/44/ With @josephevans 's fix I think the CI is working now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-607061503 @leezu seems like adding both 5.2 and 7.0 and then drop 7.0 would trigger a normal run... See here: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-gpu/detail/PR-17808/44/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-605897612 > Test in WIN_GPU fail at > > ``` > OSError: [WinError 126] The specified module could not be found > ``` > > This one means the libmxnet.dll is not in the DLL Search path [acc to Stackoverflow] > Fix would be to add it into the environment > https://stackoverflow.com/questions/43987081/openslide-python-import-error > > What's surprising is : libmxnet.dll & mxnet_70.dll are being packed into windows_package.7z and unpacked during test phase correctly. > While WIN_CPU tests don't give this error, WIN_GPU fails @larroy says he used to run into the problem and it was solved by packing opencv binary. In this case the opencv is already in the environment path and therefore be found. Although WinError 126 could also occur if a dependent dll is missing, in my local tests when I deliberately created that situation, segmentation fault (access violation at 0x) is thrown instead. @larroy also suggests using dependency walker to look at the dependent dlls. The only dependent dll of libmxnet.dll is cudart64_102.dll (not including the win32 dlls). For mxnet_70.dll, the libraries of opencv and openblas are already in the ami's environment path. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-605897612 > Test in WIN_GPU fail at > > ``` > OSError: [WinError 126] The specified module could not be found > ``` > > This one means the libmxnet.dll is not in the DLL Search path [acc to Stackoverflow] > Fix would be to add it into the environment > https://stackoverflow.com/questions/43987081/openslide-python-import-error > > What's surprising is : libmxnet.dll & mxnet_70.dll are being packed into windows_package.7z and unpacked during test phase correctly. > While WIN_CPU tests don't give this error, WIN_GPU fails @larroy says he used to run into the problem and it was solved by packing opencv binary. In this case the opencv is already in the environment path and therefore be found. Although WinError 126 could also occur if a dependent dll is missing, in my local tests when I deliberately created that situation, segmentation fault (access violation at 0x) is thrown instead. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-605897612 > Test in WIN_GPU fail at > > ``` > OSError: [WinError 126] The specified module could not be found > ``` > > This one means the libmxnet.dll is not in the DLL Search path [acc to Stackoverflow] > Fix would be to add it into the environment > https://stackoverflow.com/questions/43987081/openslide-python-import-error > > What's surprising is : libmxnet.dll & mxnet_70.dll are being packed into windows_package.7z and unpacked during test phase correctly. > While WIN_CPU tests don't give this error, WIN_GPU fails @larroy says he used to run into the problem and it was solved by packing opencv binary. In this case the opencv is already in the environment path and it should be found. Although WinError 126 could also occur if a dependent dll is missing, in my local tests when I deliberately created that situation, segmentation fault (access violation at 0x) is thrown instead. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-605611140 > Sorry but I don't believe that you are testing the development on a VM when this kind of commit is necessary. Was testing on one instance and made commits on another...manually copied the code and missed some parts... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-605572269 > @vexilligera as discussed offline, lets try testing locally for WIN_GPU and WIN_GPU_MKLDNN build 10 times each (since 1 run takes 20-30mins) to come up with some basis... (ideally would have tried 100 times but given the resource & time constraints) On my local test, the WIN_GPU_MKLDNN is much more flaky than WIN_GPU, as all WIN_GPU builds passed while about 1/3 of WIN_GPU_MKLDNN builds failed, based on my historical test data. @ChaiBapchya suggests introducing a maximum retry number to circumvent this flaky issue as pytorch has done here https://github.com/pytorch/pytorch/pull/35375 @haojin2 suggests us removing the WIN_GPU_MKLDNN test entirely since MKLDNN doesn't make much sense as we are running on GPU, and GPU_MKLDNN case is covered on other platforms. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-605572269 > @vexilligera as discussed offline, lets try testing locally for WIN_GPU and WIN_GPU_MKLDNN build 10 times each (since 1 run takes 20-30mins) to come up with some basis... (ideally would have tried 100 times but given the resource & time constraints) On my local test, the WIN_GPU_MKLDNN is much more flaky than WIN_GPU, as all WIN_GPU builds passed while about 1/3 of WIN_GPU_MKLDNN builds failed, based on my historical test data. @haojin2 suggests us removing the WIN_GPU_MKLDNN test entirely since MKLDNN doesn't make much sense as we are running on GPU, and GPU_MKLDNN case is covered on other platforms. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-605447906 WIN_GPU_MKLDNN is having a flaky issue similar to https://github.com/pytorch/pytorch/issues/25393 Also need to update dmlc-core to the latest to support VS2019 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-605447906 WIN_GPU_MKLDNN is having a flaky issue similar to https://github.com/pytorch/pytorch/issues/25393 Also need to update 3rdparty/dmlc-core to the latest to support VS2019 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-605447906 WIN_GPU_MKLDNN is having a flaky issue similar to https://github.com/pytorch/pytorch/issues/25393 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-605447906 WIN_GPU_MKLDNN is having a flaky issue similar to https://github.com/pytorch/pytorch/issues/25393 Also it would be great if someone could help update 3rdparty/dmlc-core, since there was a necessary fix just merged days ago to support VS2019 build and the GPU build just failed because of that, thank you! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-605447906 WIN_GPU_MKLDNN is having a flaky issue similar to https://github.com/pytorch/pytorch/issues/25393 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs
vexilligera edited a comment on issue #17808: [WIP] Windows dev environment configuration, update install instructions from source in the docs URL: https://github.com/apache/incubator-mxnet/pull/17808#issuecomment-600951836 > @vexilligera what is the status of this PR? I'm trying to update the AMI to VS2019 and CUDA 10.2 and there's a bug with dmlc-core, probably caused by different MSVC standards. I'm working on that. If it's an issue with MSVC then I will try to use Clang-cl for building. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
