Jennifer Jones created ZEPPELIN-1448: ----------------------------------------
Summary: Add CPU time to paragraph output for %pyspark, %spark, %python, and %sql interpreters Key: ZEPPELIN-1448 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1448 Project: Zeppelin Issue Type: New Feature Components: Interpreters, pySpark, python-interpreter, zeppelin-interpreter, zeppelin-server, zeppelin-zengine Affects Versions: 0.6.1 Environment: Mac OS X El Capitan Version 10.11.6, Spark 2.0.0, Zeppelin 0.6.1, and the Anaconda distribution of Python 3.5.2. Reporter: Jennifer Jones Fix For: 0.6.2, 0.7.0, 0.6.1 In Zeppelin, when using the PySpark interpreter (%pyspark) in a cell or "paragraph," I want the output to list the CPU time in addition to the evaluated output and the actual (physical) time elapsed. A specific example of a statement I want to time (not necessarily limited to sql queries) is something like this: %pyspark ... sqlctx = SQLContext(sc) ... sqlctx.sql("SELECT feature1, feature2, feature3 FROM tableName " + "WHERE feature3 = 'a' LIMIT 100").show() or a sql count over the total number of rows in the table. If I use Zeppelin with the Hive interpreter in a paragraph (%hive), the output automatically includes actual time, CPU time, and the evaluated output. Similarly, if I use IPython (either in a shell or in a Jupyter notebook), I can preface a statement with %time to have the output returned along with the CPU time. Please add the capability to return CPU time in a (PySpark, etc.) paragraph in a Zeppelin notebook. -- This message was sent by Atlassian JIRA (v6.3.4#6332)