GitHub user sadikovi opened a pull request:
https://github.com/apache/zeppelin/pull/1499
[ZEPPELIN-1528] Add progress information in addition to progress percentage
### What is this PR for?
PR adds progress information in addition to progress percentage, allowing
to provide links to Spark UI, for example. Currently done by adding
`getProgressInfo()` method to the interpreter API that returns list of
`InterpreterProgressInfo`, similar to completion API in this sense.
Main changes are here:
-
`zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java
`
-
`zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/Job.java`
- `zeppelin-interpreter/src/main/thrift/RemoteInterpreterService.thrift`
-
`spark/src/main/scala/org/apache/zeppelin/spark/SparkJobDetailsListener.scala`
- `zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js`
- `zeppelin-web/src/app/notebook/paragraph/paragraph-progressInfo.html`
### What type of PR is it?
Feature
### Todos
* [ ] - Add unittests for `SparkJobDetailsListener`
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1528
### How should this be tested?
Currently tested manually, by building distribution and launching simple
Spark jobs in paragraph.
Also inspecting that progress info is updated as jobs get executed and
final result is returned correctly in stdout in paragraph (initially had
problems with this since there is a difference in displaying between 0.6.x and
0.7.x).
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No.
* Is there breaking changes for older versions? Possibly.
* Does this needs documentation? I am not sure, I added javadoc for the
method.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sadikovi/zeppelin spark-progress-info
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1499.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1499
----
commit 7b09074c5a13393a398b4eae6a7e438f8928952f
Author: Ivan Sadikov <[email protected]>
Date: 2016-10-04T19:59:36Z
update thrift
commit b50441260f59542df47ed719f806f9b763bf65c2
Author: Ivan Sadikov <[email protected]>
Date: 2016-10-05T03:55:48Z
add spark job details
commit 0bb8c4f9fe1fc2622420afffbfe524a7e41932cf
Author: Ivan Sadikov <[email protected]>
Date: 2016-10-05T20:45:20Z
add progress info api
commit dfef21ee022e03ab2c0c08f69445fdb1520aadd2
Author: Ivan Sadikov <[email protected]>
Date: 2016-10-06T04:11:00Z
update job details listener
commit 50b822dfb403c5c3e24db4a5ebbce16434c79bff
Author: Ivan Sadikov <[email protected]>
Date: 2016-10-06T04:44:25Z
fix case match
commit 7edd759c67b4e9dd74043029b8d3409e1cd156f4
Author: sadikovi <[email protected]>
Date: 2016-10-06T18:19:14Z
Merge remote-tracking branch 'origin/master' into spark-progress-info
commit 1157fbfd125c4eedb7b504fe5ce56c26fed6bdf0
Author: Ivan Sadikov <[email protected]>
Date: 2016-10-09T19:39:39Z
add final progress info update
commit 71aa17cb1359ed1e2cac5384d8f7e57f27c6fcb8
Author: Ivan Sadikov <[email protected]>
Date: 2016-10-10T04:36:06Z
improve final progress info update
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
