HeartSaVioR commented on issue #27064: [SPARK-30246]OneForOneStreamManager might leak memory in connectionTerminated URL: https://github.com/apache/spark/pull/27064#issuecomment-572804389 Oh wait I'm seeing failure "after" installing "resource", but the test fails elsewhere which is odd. ``` Running PySpark tests. Output is in /Users/jlim/WorkArea/ScalaProjects/spark/python/unit-tests.log Will test against the following Python executables: ['/usr/local/bin/python3', 'python2.7', 'pypy'] Will test the following Python tests: ['pyspark.tests.test_worker'] Starting test(/usr/local/bin/python3): pyspark.tests.test_worker Starting test(pypy): pyspark.tests.test_worker Starting test(python2.7): pyspark.tests.test_worker test_memory_limit (pyspark.tests.test_worker.WorkerMemoryTest) ... Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). /Users/jlim/WorkArea/ScalaProjects/spark/python/pyspark/context.py:219: DeprecationWarning: Support for Python 2 and Python 3 prior to version 3.6 is deprecated as of Spark 3.0. See also the plan for dropping Python 2 support at https://spark.apache.org/news/plan-for-dropping-python-2-support.html. DeprecationWarning) [Stage 0:> (0 + 1) / 1]Current mem limits: 9223372036854775807 of max 9223372036854775807 Setting mem limits to 1048576 of max 1048576 /Users/jlim/WorkArea/ScalaProjects/spark/python/pyspark/tests/test_worker.py:190: RuntimeWarning: Parent module 'pyspark.tests' not found while handling absolute import import resource ok test_reuse_worker_of_parallelize_xrange (pyspark.tests.test_worker.WorkerReuseTest) ... FAIL test_accumulator_when_reuse_worker (pyspark.tests.test_worker.WorkerTests) ... ok test_after_exception (pyspark.tests.test_worker.WorkerTests) ... ok test_after_jvm_exception (pyspark.tests.test_worker.WorkerTests) ... ok test_cancel_task (pyspark.tests.test_worker.WorkerTests) ... /Users/jlim/WorkArea/ScalaProjects/spark/python/pyspark/tests/test_worker.py:45: RuntimeWarning: Parent module 'pyspark.tests' not found while handling absolute import import os /Users/jlim/WorkArea/ScalaProjects/spark/python/pyspark/tests/test_worker.py:46: RuntimeWarning: Parent module 'pyspark.tests' not found while handling absolute import import time ok test_python_exception_non_hanging (pyspark.tests.test_worker.WorkerTests) ... ok test_reuse_worker_after_take (pyspark.tests.test_worker.WorkerTests) ... ok test_with_different_versions_of_python (pyspark.tests.test_worker.WorkerTests) ... ok ====================================================================== Finished test(/usr/local/bin/python3): pyspark.tests.test_worker (15s) FAIL: test_reuse_worker_of_parallelize_xrange (pyspark.tests.test_worker.WorkerReuseTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/jlim/WorkArea/ScalaProjects/spark/python/pyspark/tests/test_worker.py", line 176, in test_reuse_worker_of_parallelize_xrange self.assertTrue(pid in previous_pids) AssertionError: False is not true ---------------------------------------------------------------------- Ran 9 tests in 15.360s FAILED (failures=1) ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
