[GitHub] zeppelin issue #3024: [ZEPPELIN-3545] save all tables to ResourcePool
Github user mebelousov commented on the issue: https://github.com/apache/zeppelin/pull/3024 @zjffdu thank you about ResourcePool impoving. Share please your vision how it would be. For example, a paragraph has 5 table results. How user will define which of them would be added into ResourcePool? ---
[jira] [Created] (ZEPPELIN-3559) z.show only shows 13 rows from dataframe of 100
Matthew J Meyer created ZEPPELIN-3559:
-
Summary: z.show only shows 13 rows from dataframe of 100
Key: ZEPPELIN-3559
URL: https://issues.apache.org/jira/browse/ZEPPELIN-3559
Project: Zeppelin
Issue Type: Bug
Components: GUI
Affects Versions: 0.8.1
Reporter: Matthew J Meyer
Attachments: Screenshot from 2018-06-20 22-27-24.png
Running:
{{%spark}}
{{val l = (1 to 100).toSeq}}
{{val v = List.tabulate(100)(n => n * n)}}
{{val df = sc.parallelize(l zip v).toDF("label","value")}}
{{df.show(100)}}
{{z.show(df)}}
results in expected interpreter output of spark showing 100 rows but the table
shown in zeppelin via z.show only shows 13 rows.
If I switch to use the any of the chart visualizations (bar, line, area, etc),
zeppelin correctly shows the expected 100 elements on the x axis.
zeppelin.spark.maxResult is set to default of 1000
Zeppelin was built from branch-0.8 using:
mvn clean package -Pspark-2.1 -Phadoop-2.4 -Pscala-2.10 -DskipTests
Running the same on 0.7.3 results in expected 100 rows in zeppelin table.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] zeppelin issue #3013: [ZEPPELIN-3511] remove old button "Download Data as CS...
Github user sanjaydasgupta commented on the issue: https://github.com/apache/zeppelin/pull/3013 Thanks for the ideas @Tagar If we focus on long term advantages and stability, then continuing to maintain the old button will be a disadvantage. If the old button is retained, it will have to be revisited (implement, test, document) every time the [ui-grid](http://ui-grid.info/) menu adds a new feature or changes an existing feature. This is clearly not desirable, and should not be our choice of the way forward. I had offered to provide a transition mechanism based on a visually similar button, but just as a transition mechanism - to be removed in 1 or 2 releases. The ui-grid already has some nice features (like PDF output) that zeppelin users have requested, and going with ui-grid will IMHO be a good decision. ---
[GitHub] zeppelin issue #3024: [ZEPPELIN-3545] save all tables to ResourcePool
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/3024 Thanks @Savalek for this contribution, but I think putting all tables into ResourcePool doesn't make sense. As it would occupy lots of memory. I plan to introduce paragraph level properties (ZEPPELIN-3348), so that user can control whether to put the interpreter result into ResourcePool. ---
[GitHub] zeppelin pull request #3036: [ZEPPELIN-3554] Flaky test - InterpreterContext...
Github user asfgit closed the pull request at: https://github.com/apache/zeppelin/pull/3036 ---
[GitHub] zeppelin issue #3029: ZEPPELIN-621 - Bump Java version to Java 8
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/3029 @alexott Travis fails, could you help check that ? ---
[GitHub] zeppelin issue #3029: ZEPPELIN-621 - Bump Java version to Java 8
Github user jongyoul commented on the issue: https://github.com/apache/zeppelin/pull/3029 I have a simple question. Is `Vargrant` related to JDK8 directly? ---
Re: Disable Flaky Test
Hi, I think it's good but we might need a plan when we enable them again. Basically, I agree with it. On Tue, Jun 19, 2018 at 1:49 AM, Jeff Zhang wrote: > Hi Folks, > > Seems there's no objection on this, so I assume everyone agree on it. I > have merged this PR. From now on, please double check the travis status > before you merge your PR if you are committer and make sure the travis > build status before you ask others to review if you are not committer. As I > mentioned before, you need to add unit test for your PR and rerun the > travis if it still fails. If your PR make the travis failed, every > committer has the privilege to revert your PR. > > > > Jeff Zhang 于2018年6月15日周五 上午9:46写道： > > > > > BTW, please* add unit test for any new PR* unless it is very trivial > > change and other committer agree it is OK without unit test. If you find > > it is difficult to add unit test. There will be several reasons according > > my experience. > > > > 1. It is due to the architecture and evolution of zeppelin, as more and > > more components and features added to zeppelin, many things may be > coupled > > together which make it difficult to add unit test. In this case, please > > comment in your PR why it is difficult to add unit test, and what is your > > plan for that. Code refactoring in some component of zeppelin or unit > test. > > > > 2. It is due to your PR that introduce too much complexity or didn't do > it > > in a elegant approach. In this case, please rethink your PR to improve > it. > > > > > > > > > > Jeff Zhang 于2018年6月15日周五 上午8:59写道： > > > >> Hi Folks, > >> > >> The zeppelin travis become unstable again recently. I know it is really > >> annoying and frustrated, so I create a PR to disable these flaky tests. > >> There still some flaky test, but happens in low frequency. According my > >> experience, if you still have travis build failure after this PR, you > can > >> try to rerun the travis build to get travis build pass. Just rerun the > >> failed one, don't run the whole travis build matrix and also please > clear > >> travis cache first. If there's no objection, I would commit this PR. And > >> after that, please keep travis build passed before you commit any new > >> changes. Thanks. > >> > >> https://github.com/apache/zeppelin/pull/3020 > >> > >> > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
[GitHub] zeppelin issue #3036: [ZEPPELIN-3554] Flaky test - InterpreterContextTest.te...
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/3036 LGTM ---
[GitHub] zeppelin pull request #2997: Miscellaneous documentation fixes
Github user asfgit closed the pull request at: https://github.com/apache/zeppelin/pull/2997 ---
[GitHub] zeppelin issue #3029: ZEPPELIN-621 - Bump Java version to Java 8
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/3029 We also need to update .travis.yml to use only java 8 ---
[jira] [Created] (ZEPPELIN-3558) Refactor Scheduler component
Jeff Zhang created ZEPPELIN-3558: Summary: Refactor Scheduler component Key: ZEPPELIN-3558 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3558 Project: Zeppelin Issue Type: Improvement Reporter: Jeff Zhang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ZEPPELIN-3557) Remove interpreter profiles
Jeff Zhang created ZEPPELIN-3557: Summary: Remove interpreter profiles Key: ZEPPELIN-3557 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3557 Project: Zeppelin Issue Type: Improvement Reporter: Jeff Zhang There's several interpreter specific profiles, such as beams, scio and etc. It is better to remove them otherwise it is very likely to miss to build them in travis so that we may miss the build errors. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zeppelin issue #2997: Miscellaneous documentation fixes
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2997 LGTM, will merge it soon ---
[GitHub] zeppelin issue #2997: Miscellaneous documentation fixes
Github user alexott commented on the issue: https://github.com/apache/zeppelin/pull/2997 Done, thank you for review! ---
[GitHub] zeppelin issue #2997: Miscellaneous documentation fixes
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2997 @alexott Could you do rebase to resolve the conflicts ? ---
Re: PRs for documentation?
Sorry, Alex, 0.8.0 RC5 is out today. I am afraid we could not get it into 0.8 if the voting pass, but I will merge it into next release. And thanks for the contribution. Alex Ott 于2018年6月20日周三 下午3:28写道： > Hello Jeff > > I know that problem with OSS projects - it's always not enough time :-( > > My changes are quite small, mostly formatting, and it would be nice to get > them into 0.8 branch, as some documentation is really looks not accurate > because of incorrect formatting. > Regarding this one - should I open a separate PR against 0.8 to make it > into release? > > > On Wed, Jun 20, 2018 at 1:32 AM, Jianfeng (Jeff) Zhang < > jzh...@hortonworks.com> wrote: > > > > > It is not necessary to create ticket for document change if it is trivial > > change, otherwise you still need to create ticket. > > > > Regarding the review, I am sorry that there¹s no much bandwidth to review > > for committers, if there¹s no response in 3 days, please ping someone or > > send mail in dev mail list. > > > > > > > > Best Regard, > > Jeff Zhang > > > > > > > > > > > > On 6/19/18, 9:50 PM, "Alex Ott" wrote: > > > > >Hi all > > > > > >I have a question - should I file JIRA for documentation > > >fixes/improvements? > > >For example, I opened PR some time ago ( > > >https://github.com/apache/zeppelin/pull/2997) that fixes multiple > > problems > > >and improves documentation. But it still not merged... (conflict occur > > >afterwards - I'll rebase soon) > > > > > >-- > > >With best wishes,Alex Ott > > >http://alexott.net/ > > >Twitter: alexott_en (English), alexott (Russian) > > > > > > > -- > With best wishes,Alex Ott > http://alexott.net/ > Twitter: alexott_en (English), alexott (Russian) >
Re: PRs for documentation?
Hello Jeff I know that problem with OSS projects - it's always not enough time :-( My changes are quite small, mostly formatting, and it would be nice to get them into 0.8 branch, as some documentation is really looks not accurate because of incorrect formatting. Regarding this one - should I open a separate PR against 0.8 to make it into release? On Wed, Jun 20, 2018 at 1:32 AM, Jianfeng (Jeff) Zhang < jzh...@hortonworks.com> wrote: > > It is not necessary to create ticket for document change if it is trivial > change, otherwise you still need to create ticket. > > Regarding the review, I am sorry that there¹s no much bandwidth to review > for committers, if there¹s no response in 3 days, please ping someone or > send mail in dev mail list. > > > > Best Regard, > Jeff Zhang > > > > > > On 6/19/18, 9:50 PM, "Alex Ott" wrote: > > >Hi all > > > >I have a question - should I file JIRA for documentation > >fixes/improvements? > >For example, I opened PR some time ago ( > >https://github.com/apache/zeppelin/pull/2997) that fixes multiple > problems > >and improves documentation. But it still not merged... (conflict occur > >afterwards - I'll rebase soon) > > > >-- > >With best wishes,Alex Ott > >http://alexott.net/ > >Twitter: alexott_en (English), alexott (Russian) > > -- With best wishes,Alex Ott http://alexott.net/ Twitter: alexott_en (English), alexott (Russian)
[GitHub] zeppelin pull request #3038: ZEPPELIN-3556. set spark.yarn.submit.waitAppCom...
GitHub user zjffdu opened a pull request: https://github.com/apache/zeppelin/pull/3038 ZEPPELIN-3556. set spark.yarn.submit.waitAppCompletion to false in yarn-cluster mode ### What is this PR for? Set `spark.yarn.submit.waitAppCompletion` to false in yarn-cluster mode, so that in the interpreter log we won't see many logs of monitor yarn app status which is useless. ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-3556 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-3556 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/3038.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3038 commit 7474987a693c48489bdc453890e0f72baa3e3697 Author: Jeff Zhang Date: 2018-06-20T06:56:11Z ZEPPELIN-3556. set spark.yarn.submit.waitAppCompletion to false in yarn-cluster mode ---
[jira] [Created] (ZEPPELIN-3556) set spark.yarn.submit.waitAppCompletion to false in yarn-cluster mode
Jeff Zhang created ZEPPELIN-3556: Summary: set spark.yarn.submit.waitAppCompletion to false in yarn-cluster mode Key: ZEPPELIN-3556 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3556 Project: Zeppelin Issue Type: Improvement Reporter: Jeff Zhang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zeppelin issue #3019: [ZEPPELIN-3540] Show Spark inline UI in Non-Secure mod...
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/3019 @shahsank3t I didn't try this PR, but I didn't see backend pass `spark.ui.allowFramingFrom` to frontend, so how does frontend know whether it should show spark ui inline or in another windows of the old method ? ---
[GitHub] zeppelin pull request #3037: [ZEPPELIN-3555] Zeppelin auth fails if `activeD...
GitHub user prabhjyotsingh opened a pull request: https://github.com/apache/zeppelin/pull/3037 [ZEPPELIN-3555] Zeppelin auth fails if `activeDirectoryRealm.groupRolesMap` is not specified. ### What is this PR for? Zeppelin auth fails if `activeDirectoryRealm.groupRolesMap` is not specified. ### What type of PR is it? [Bug Fix] ### What is the Jira issue? * [ZEPPELIN-3555](https://issues.apache.org/jira/browse/ZEPPELIN-3555) ### How should this be tested? Zeppelin auth fails if `groupRolesMap` is not specified in `ActiveDirectoryGroupRealm`, with this PR following config should work; ``` activeDirectoryRealm = org.apache.zeppelin.realm.ActiveDirectoryGroupRealm activeDirectoryRealm.systemUsername = userNameA activeDirectoryRealm.systemPassword = passwordA activeDirectoryRealm.searchBase = CN=Users,DC=SOME_GROUP,DC=COMPANY,DC=COM activeDirectoryRealm.url = ldap://ldap.test.com:389 #activeDirectoryRealm.groupRolesMap = "CN=admin,OU=groups,DC=SOME_GROUP,DC=COMPANY,DC=COM":"admin","CN=finance,OU=groups,DC=SOME_GROUP,DC=COMPANY,DC=COM":"finance","CN=hr,OU=groups,DC=SOME_GROUP,DC=COMPANY,DC=COM":"hr" activeDirectoryRealm.authorizationCachingEnabled = false ``` ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? N/A * Is there breaking changes for older versions? N/A * Does this needs documentation? N/A You can merge this pull request into a Git repository by running: $ git pull https://github.com/prabhjyotsingh/zeppelin ZEPPELIN-3555 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/3037.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3037 commit 6cb408a6e5741ee4f52672def084d4eee546948a Author: Prabhjyot Singh Date: 2018-06-20T06:13:45Z ZEPPELIN-3555: fix NPE in groupRolesMap for ActiveDirectoryGroupRealm Change-Id: Ifb172642cb4b8579eeb71255358b456f378c9143 ---
