roywei commented on issue #14981: [CI][NightlyTestsForBinaries] Test Large Tensor: GPU Failing URL: https://github.com/apache/incubator-mxnet/issues/14981#issuecomment-498748546 Currently, both CPU and GPU tests have been disabled due to the same memory issue. Had a discussion with @access2rohit and @apeforest, we can try a few things: 1. change to P3 instances here https://github.com/apache/incubator-mxnet/blob/master/tests/nightly/JenkinsfileForBinaries#L82 2. further increase shared memory to 50G 3. stop running large tensor test parallelly with other tests. We are having problems testing the above solutions on CI machines that have multiple jobs running in parallel.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
