[GitHub] zeppelin issue #1754: [ZEPPELIN-1800] Update scio doc - fix type $ should be...
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1754 @felixcheung thanks for the review - tests fail, but I doubt it's due to this change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin pull request #1755: [ZEPPELIN-1801] To force end table - append %te...
GitHub user ravwojdyla opened a pull request:
https://github.com/apache/zeppelin/pull/1755
[ZEPPELIN-1801] To force end table - append %text
### What is this PR for?
It seems there is a bug in scio introduced by recent changes. Table
rendered from scio helper methods would include interpreter results as well as
the data.
### What type of PR is it?
Bug Fix
### Todos
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1801
### How should this be tested?
Try:
```
%beam.scio
val (sc, args) = ContextAndArgs(argz)
sc.parallelize(Seq("foo", "foo", "bar")).countByValue.closeAndDisplay()
```
before and after the change - and notice extra meaningless columns before
the fix.
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ravwojdyla/incubator-zeppelin fix_1801
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1755.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1755
commit 49bd51855defcfb289ec4327546a8ca5cd7bd157
Author: Rafal Wojdyla <r...@spotify.com>
Date: 2016-12-13T16:58:56Z
To force end table - append %text
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1754: [ZEPPELIN-1800] Update scio doc - fix type $ sh...
GitHub user ravwojdyla opened a pull request: https://github.com/apache/zeppelin/pull/1754 [ZEPPELIN-1800] Update scio doc - fix type $ should be % ### What is this PR for? Update scio doc - fix type $ should be % ### What type of PR is it? Documentation ### Todos ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1800 ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/ravwojdyla/incubator-zeppelin fix_1800 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1754.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1754 commit d4287a9ef84e2c3b2299d61fb8586862ecb6d91e Author: Rafal Wojdyla <r...@spotify.com> Date: 2016-12-13T15:42:20Z Update scio doc - fix type $ should be % --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1633: Fix formatting/typos/style in scio doc
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1633 Is jira ticket needed for this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin pull request #1633: Fix formatting/typos/style in scio doc
GitHub user ravwojdyla opened a pull request: https://github.com/apache/zeppelin/pull/1633 Fix formatting/typos/style in scio doc ### What is this PR for? Fix formatting/typos/style in scio doc ### What type of PR is it? Documentation You can merge this pull request into a Git repository by running: $ git pull https://github.com/ravwojdyla/incubator-zeppelin fix_scio_doc Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1633.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1633 commit 888ec68b4783f5e397bf7dd527bd5d65479a18dc Author: Rafal Wojdyla <r...@spotify.com> Date: 2016-11-15T00:26:37Z Fix formatting/typos/style in scio doc --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 Those failures on travis seem unrelated - restarting build. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 Rebased to solve conflict in `zeppelin-distribution/src/bin_license/LICENSE`. Are there any other comments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 @felixcheung correct - i missed it - updated doc/examples. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 ping @felixcheung --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 ping @felixcheung --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 @felixcheung after going through the `BeamInterpreter`, I don't see immediate technical advantage of sharing the group, current implementation of `BeamInterpreter`, is actually pure Java interpreter, they would not share any configuration. That said from strictly logical point of view it makes sense for them to be in the same group - given they share the same model. What kind of changes are needed to put scio in the same group as beam? And what is gonna be the impact on users? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 @felixcheung I see, thanks for help, is 95dd9bb enough? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 @felixcheung well - yes and no. scio is currently being contributed to beam, but current implementation still uses dataflow sdk. maybe we can consolidate it once scio migration to beam is done? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 ping @Leemoonsoo @felixcheung --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 @AhyoungRyu @zjffdu code was reviewed by @nevillelyh, build looks green. Any other comments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 @nevillelyh (the creator of scio) - could you please review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 @zjffdu @AhyoungRyu are there any other comments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 The tests are failing for: ``` Tests run: 9, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 119.221 sec <<< FAILURE! - in org.apache.zeppelin.integration.ParagraphActionsIT testWidth(org.apache.zeppelin.integration.ParagraphActionsIT) Time elapsed: 21.445 sec <<< FAILURE! java.lang.AssertionError: New Width is : 4 Expected: but: was at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20) at org.junit.Assert.assertThat(Assert.java:865) at org.junit.rules.ErrorCollector$1.call(ErrorCollector.java:65) at org.junit.rules.ErrorCollector.checkSucceeds(ErrorCollector.java:78) at org.junit.rules.ErrorCollector.checkThat(ErrorCollector.java:63) at org.apache.zeppelin.integration.ParagraphActionsIT.testWidth(ParagraphActionsIT.java:314) testWidth(org.apache.zeppelin.integration.ParagraphActionsIT) Time elapsed: 21.445 sec <<< FAILURE! java.lang.AssertionError: New Width is : 8 Expected: but: was at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20) at org.junit.Assert.assertThat(Assert.java:865) at org.junit.rules.ErrorCollector$1.call(ErrorCollector.java:65) at org.junit.rules.ErrorCollector.checkSucceeds(ErrorCollector.java:78) at org.junit.rules.ErrorCollector.checkThat(ErrorCollector.java:63) at org.apache.zeppelin.integration.ParagraphActionsIT.testWidth(ParagraphActionsIT.java:314) ``` I can't see the connection of those with this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 @AhyoungRyu updated licenses. The travis failures do not seems to be related could you please restart the travis jobs? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 @AhyoungRyu sorry I missed it. Will go ahead and add all the licenses. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 ping @AhyoungRyu @zjffdu --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 @AhyoungRyu @zjffdu are there any more comment, how do you want to proceed? the error on travis does not seems to be related? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: Closes [ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 @AhyoungRyu @zjffdu could you please review when your have time. all the todos are âï¸ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1471: [WIP][ZEPPELIN-1505] Add Scio interpreter
Github user ravwojdyla commented on the issue: https://github.com/apache/zeppelin/pull/1471 @AhyoungRyu @zjffdu thanks for prompt review. I have some questions in the comments, plus fixes coming in. I have also changed the topic of the PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin pull request #1471: Closes ZEPPELIN-1505: Add Scio interpreter
GitHub user ravwojdyla opened a pull request: https://github.com/apache/zeppelin/pull/1471 Closes ZEPPELIN-1505: Add Scio interpreter ### What is this PR for? Closes #ZEPPELIN-1505. Adds Scio interpreter. Scio is a Scala DSL on top of Dataflow/Beam. ### What type of PR is it? Improvement - ZEPPELIN-1505 ### Todos * [ ] - test integration with zeppelin context * [ ] - what to do about code completion? * [ ] - add more tests? ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1505 ### How should this be tested? ``` mvn -pl scio,zeppelin-display,zeppelin-interpreter -Dtest='org.apache.zeppelin.scio.*' -DfailIfNoTests=false test ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/ravwojdyla/incubator-zeppelin scio Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1471.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1471 commit b85c3c028f13064345cb149f1d162c41d5bc4dba Author: Rafal Wojdyla <r...@spotify.com> Date: 2016-09-28T04:36:20Z Add Scio interpreter commit c8463c6540021b782851090ce359c263717905eb Author: Rafal Wojdyla <r...@spotify.com> Date: 2016-09-29T01:16:42Z Add Scio interpreter tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
