[ https://issues.apache.org/jira/browse/SPARK-32186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197391#comment-17197391 ]
Apache Spark commented on SPARK-32186: -------------------------------------- User 'itholic' has created a pull request for this issue: https://github.com/apache/spark/pull/29781 > Development - Debugging > ----------------------- > > Key: SPARK-32186 > URL: https://issues.apache.org/jira/browse/SPARK-32186 > Project: Spark > Issue Type: Sub-task > Components: Documentation, PySpark > Affects Versions: 3.1.0 > Reporter: Hyukjin Kwon > Assignee: Hyukjin Kwon > Priority: Major > Fix For: 3.1.0 > > > 1. Python Profiler: > https://spark.apache.org/docs/2.3.0/api/python/_modules/pyspark/profiler.html, > > {code} > >>> sc._conf.set("spark.python.profile", "true") > >>> rdd = sc.parallelize(range(100)).map(str) > >>> rdd.count() > 100 > >>> sc.show_profiles() > {code} > 2. Python Debugger, e.g.) > https://www.jetbrains.com/help/pycharm/remote-debugging-with-product.html#remote-interpreter > \(?\) > 3. Monitoring Python Workers: {{top}}, `{{ls}}`, etc. \(?\) > 4. PyCharm setup \(?\) (at https://issues.apache.org/jira/browse/SPARK-32189) -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org