[GitHub] [zeppelin] florpor commented on issue #3363: [ZEPPELIN-4144] After refresh the revision tools are sometimes hidden

2019-05-18 Thread GitBox
florpor commented on issue #3363: [ZEPPELIN-4144] After refresh the revision 
tools are sometimes hidden
URL: https://github.com/apache/zeppelin/pull/3363#issuecomment-493693390
 
 
   @Leemoonsoo I enabled travis and it took a few force-push-retries for all 
the jobs to succeed in one run.
   Now it seems that the travis_check.py script has had some error in the 
Jenkins job.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] asfgit closed pull request #3353: [ZEPPELIN-4132]. Spark Interpreter has issue of SPARK-22393

2019-05-17 Thread GitBox
asfgit closed pull request #3353: [ZEPPELIN-4132]. Spark Interpreter has issue 
of SPARK-22393
URL: https://github.com/apache/zeppelin/pull/3353
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu opened a new pull request #3367: [ZEPPELIN-4147]. Fail to rename folder

2019-05-16 Thread GitBox
zjffdu opened a new pull request #3367: [ZEPPELIN-4147]. Fail to rename folder
URL: https://github.com/apache/zeppelin/pull/3367
 
 
   ### What is this PR for?
   
   It is a trivial bug fox for rename folder failure. The root cause is that 
the folder path is not including leading `/` which is required by NotebookRepo. 
   
   ### What type of PR is it?
   [Bug Fix]
   
   ### Todos
   * [ ] - Task
   
   ### What is the Jira issue?
   * https://jira.apache.org/jira/browse/ZEPPELIN-4147
   
   ### How should this be tested?
   * CI Pass
   
   ### Screenshots (if appropriate)
   
   ### Questions:
   * Does the licenses files need update? No
   * Is there breaking changes for older versions? No
   * Does this needs documentation? No
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] asfgit closed pull request #3360: [ZEPPELIN-4152]. Use spark 2.2 as the default profile

2019-05-16 Thread GitBox
asfgit closed pull request #3360: [ZEPPELIN-4152]. Use spark 2.2 as the default 
profile
URL: https://github.com/apache/zeppelin/pull/3360
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] asfgit closed pull request #3366: [ZEPPELIN-4148]. Move GitNotebookRepo into zeppelin-zengine as default NotebookRepo

2019-05-16 Thread GitBox
asfgit closed pull request #3366: [ZEPPELIN-4148]. Move GitNotebookRepo into 
zeppelin-zengine as default NotebookRepo
URL: https://github.com/apache/zeppelin/pull/3366
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] felixcheung commented on issue #3360: [ZEPPELIN-4152]. Use spark 2.2 as the default profile

2019-05-16 Thread GitBox
felixcheung commented on issue #3360: [ZEPPELIN-4152]. Use spark 2.2 as the 
default profile
URL: https://github.com/apache/zeppelin/pull/3360#issuecomment-493283789
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] felixcheung commented on issue #3360: [ZEPPELIN-4152]. Use spark 2.2 as the default profile

2019-05-16 Thread GitBox
felixcheung commented on issue #3360: [ZEPPELIN-4152]. Use spark 2.2 as the 
default profile
URL: https://github.com/apache/zeppelin/pull/3360#issuecomment-493283757
 
 
   sure, this could be a big problem...


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] asfgit closed pull request #3203: [ZEPPELIN-3626] Cluster server and client module

2019-05-16 Thread GitBox
asfgit closed pull request #3203: [ZEPPELIN-3626] Cluster server and client 
module
URL: https://github.com/apache/zeppelin/pull/3203
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu opened a new pull request #3366: [ZEPPELIN-4148]. Move GitNotebookRepo into zeppelin-zengine as default t NotebookRepo

2019-05-16 Thread GitBox
zjffdu opened a new pull request #3366: [ZEPPELIN-4148]. Move GitNotebookRepo 
into zeppelin-zengine as default t NotebookRepo
URL: https://github.com/apache/zeppelin/pull/3366
 
 
   ### What is this PR for?
   We should make zeppelin-zengine work without building plugins. So it make 
sense to move `GitNotebookRepo` into zeppelin-zengine as it is also the default 
NotebookRepo if not specified in `zeppelin-site.xml`
   
   ### What type of PR is it?
   [ Improvement ]
   
   ### Todos
   * [ ] - Task
   
   ### What is the Jira issue?
   * https://jira.apache.org/jira/browse/ZEPPELIN-4148
   
   ### How should this be tested?
   * CI pass
   
   ### Screenshots (if appropriate)
   
   ### Questions:
   * Does the licenses files need update? No
   * Is there breaking changes for older versions? No
   * Does this needs documentation? No
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] conker84 commented on issue #3034: ZEPPELIN-3552. Support Scala 2.12 of SparkInterpreter

2019-05-16 Thread GitBox
conker84 commented on issue #3034: ZEPPELIN-3552. Support Scala 2.12 of 
SparkInterpreter
URL: https://github.com/apache/zeppelin/pull/3034#issuecomment-493022286
 
 
   Any news on this?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on issue #3365: [ZEPPELIN-4160] Fixed Move this note to trash icon always show

2019-05-16 Thread GitBox
liuxunorg commented on issue #3365: [ZEPPELIN-4160] Fixed Move this note to 
trash icon always show
URL: https://github.com/apache/zeppelin/pull/3365#issuecomment-492946354
 
 
   @felixcheung , More busy, I suggest to find out the problem and solve it. :-)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on issue #3360: [ZEPPELIN-4152]. Use spark 2.2 as the default profile

2019-05-16 Thread GitBox
zjffdu commented on issue #3360: [ZEPPELIN-4152]. Use spark 2.2 as the default 
profile
URL: https://github.com/apache/zeppelin/pull/3360#issuecomment-492941721
 
 
   I tried to do that, but some other interpreters don't support scala-2.11. So 
it would take much more time to support scala-2.11 fully in zeppelin. I plan to 
move some interpreter out of zeppelin project to subproject, so that each 
interpreter can has its own build environment and dependency. This PR is a 
temporary solution before that  


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] felixcheung commented on issue #3360: [ZEPPELIN-4152]. Use spark 2.2 as the default profile

2019-05-16 Thread GitBox
felixcheung commented on issue #3360: [ZEPPELIN-4152]. Use spark 2.2 as the 
default profile
URL: https://github.com/apache/zeppelin/pull/3360#issuecomment-492940403
 
 
   hm. I guess we need a plan to move to 2.11


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] felixcheung commented on issue #3365: [ZEPPELIN-4160] Fixed Move this note to trash icon always show

2019-05-16 Thread GitBox
felixcheung commented on issue #3365: [ZEPPELIN-4160] Fixed Move this note to 
trash icon always show
URL: https://github.com/apache/zeppelin/pull/3365#issuecomment-492940152
 
 
   ok. do you think we should add more tests for ZEPPELIN-2619?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-16 Thread GitBox
zjffdu commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter
URL: https://github.com/apache/zeppelin/pull/3359#issuecomment-492935457
 
 
   @liuxunorg Based on the above comments, I think we should keep the old shell 
interpreter as the batch mode and the new shell interpreter as interactive 
mode.  Personally I prefer to keep them in one interpreter group so that they 
can be launched in one JVM and collaborate with each other.
   
   Regarding the `interpolations`, it should already support, as it inherits 
`AbstractInterpreter` which support `interpolations`. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on a change in pull request #3364: [ZEPPELIN-4155] Unable get available HostAddress in multi-NIC environment

2019-05-15 Thread GitBox
zjffdu commented on a change in pull request #3364: [ZEPPELIN-4155] Unable get 
available HostAddress in multi-NIC environment
URL: https://github.com/apache/zeppelin/pull/3364#discussion_r284523852
 
 

 ##
 File path: 
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterUtils.java
 ##
 @@ -93,22 +97,53 @@ public static TServerSocket createTServerSocket(String 
portRange)
 throw new IOException("No available port in the portRange: " + portRange);
   }
 
+  // 1) Multiple NetworkCard will be configured on some servers
+  // 2) In the docker container environment, A container will also generate 
multiple virtual NetworkCard
 
 Review comment:
   Could you add java doc to explain the logic ?
   To me, it seems to involves 2 steps:
   1. find all available networkcards
   2. loo all the neetworkcards, and use the first non-loopback ip address.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on a change in pull request #3364: [ZEPPELIN-4155] Unable get available HostAddress in multi-NIC environment

2019-05-15 Thread GitBox
zjffdu commented on a change in pull request #3364: [ZEPPELIN-4155] Unable get 
available HostAddress in multi-NIC environment
URL: https://github.com/apache/zeppelin/pull/3364#discussion_r284523852
 
 

 ##
 File path: 
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterUtils.java
 ##
 @@ -93,22 +97,53 @@ public static TServerSocket createTServerSocket(String 
portRange)
 throw new IOException("No available port in the portRange: " + portRange);
   }
 
+  // 1) Multiple NetworkCard will be configured on some servers
+  // 2) In the docker container environment, A container will also generate 
multiple virtual NetworkCard
 
 Review comment:
   Could you add java doc to explain the logic ?
   To me, it seems to involves 2 steps:
   1. find all available networkcards
   2. loo all the neetworkcards, and use the first non-loop ip address.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on a change in pull request #3364: [ZEPPELIN-4155] Unable get available HostAddress in multi-NIC environment

2019-05-15 Thread GitBox
zjffdu commented on a change in pull request #3364: [ZEPPELIN-4155] Unable get 
available HostAddress in multi-NIC environment
URL: https://github.com/apache/zeppelin/pull/3364#discussion_r284523852
 
 

 ##
 File path: 
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterUtils.java
 ##
 @@ -93,22 +97,53 @@ public static TServerSocket createTServerSocket(String 
portRange)
 throw new IOException("No available port in the portRange: " + portRange);
   }
 
+  // 1) Multiple NetworkCard will be configured on some servers
+  // 2) In the docker container environment, A container will also generate 
multiple virtual NetworkCard
 
 Review comment:
   Could you add java doc to explain the logic ?
   To me, it seems to involves 2 steps:
   1. find all available networkcards
   2. loop all the neetworkcards, and use the first non-loopback ip address.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on a change in pull request #3364: [ZEPPELIN-4155] Unable get available HostAddress in multi-NIC environment

2019-05-15 Thread GitBox
zjffdu commented on a change in pull request #3364: [ZEPPELIN-4155] Unable get 
available HostAddress in multi-NIC environment
URL: https://github.com/apache/zeppelin/pull/3364#discussion_r284523456
 
 

 ##
 File path: 
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterUtils.java
 ##
 @@ -93,22 +97,53 @@ public static TServerSocket createTServerSocket(String 
portRange)
 throw new IOException("No available port in the portRange: " + portRange);
   }
 
+  // 1) Multiple NetworkCard will be configured on some servers
+  // 2) In the docker container environment, A container will also generate 
multiple virtual NetworkCard
   public static String findAvailableHostAddress() throws UnknownHostException, 
SocketException {
-InetAddress address = InetAddress.getLocalHost();
-if (address.isLoopbackAddress()) {
-  for (NetworkInterface networkInterface : Collections
-  .list(NetworkInterface.getNetworkInterfaces())) {
-if (!networkInterface.isLoopback()) {
-  for (InterfaceAddress interfaceAddress : 
networkInterface.getInterfaceAddresses()) {
-InetAddress a = interfaceAddress.getAddress();
-if (a instanceof Inet4Address) {
-  return a.getHostAddress();
-}
+List netlist = new ArrayList();
+
+// Get all the network cards in the current environment
+Enumeration netInterfaces = NetworkInterface.getNetworkInterfaces();
+while (netInterfaces.hasMoreElements()) {
+  NetworkInterface networkInterface = 
(NetworkInterface)netInterfaces.nextElement();
+  LOGGER.info("networkInterface = " + networkInterface.toString());
 
 Review comment:
   -> LOGGER.debug


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on a change in pull request #3364: [ZEPPELIN-4155] Unable get available HostAddress in multi-NIC environment

2019-05-15 Thread GitBox
zjffdu commented on a change in pull request #3364: [ZEPPELIN-4155] Unable get 
available HostAddress in multi-NIC environment
URL: https://github.com/apache/zeppelin/pull/3364#discussion_r284523426
 
 

 ##
 File path: 
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterUtils.java
 ##
 @@ -93,22 +97,53 @@ public static TServerSocket createTServerSocket(String 
portRange)
 throw new IOException("No available port in the portRange: " + portRange);
   }
 
+  // 1) Multiple NetworkCard will be configured on some servers
+  // 2) In the docker container environment, A container will also generate 
multiple virtual NetworkCard
   public static String findAvailableHostAddress() throws UnknownHostException, 
SocketException {
-InetAddress address = InetAddress.getLocalHost();
-if (address.isLoopbackAddress()) {
-  for (NetworkInterface networkInterface : Collections
-  .list(NetworkInterface.getNetworkInterfaces())) {
-if (!networkInterface.isLoopback()) {
-  for (InterfaceAddress interfaceAddress : 
networkInterface.getInterfaceAddresses()) {
-InetAddress a = interfaceAddress.getAddress();
-if (a instanceof Inet4Address) {
-  return a.getHostAddress();
-}
+List netlist = new ArrayList();
+
+// Get all the network cards in the current environment
+Enumeration netInterfaces = NetworkInterface.getNetworkInterfaces();
+while (netInterfaces.hasMoreElements()) {
+  NetworkInterface networkInterface = 
(NetworkInterface)netInterfaces.nextElement();
+  LOGGER.info("networkInterface = " + networkInterface.toString());
+  if (networkInterface.isLoopback()) {
+// Filter lo network card
+continue;
+  }
+  // When all the network cards are obtained by the above method,
+  // The order obtained is the reverse of the order of the NICs
+  // seen in the server with the ifconfig command.
+  // Therefore, when you want to traverse from the first NIC,
+  // Need to reverse the elements in Enumeration
+  netlist.add(0, networkInterface);
+}
+
+for (NetworkInterface list:netlist) {
+  Enumeration enumInetAddress = list.getInetAddresses();
+
+  while (enumInetAddress.hasMoreElements()) {
+InetAddress ip = (InetAddress) enumInetAddress.nextElement();
+LOGGER.info("ip = " + ip.toString());
 
 Review comment:
   -> LOGGER.debug 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on issue #3203: [ZEPPELIN-3626] Cluster server and client module

2019-05-15 Thread GitBox
liuxunorg commented on issue #3203: [ZEPPELIN-3626] Cluster server and client 
module
URL: https://github.com/apache/zeppelin/pull/3203#issuecomment-492885349
 
 
   This feature has been running in our company for 2 years and there is no 
problem.
   Will merge if no more comments.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on issue #3365: [ZEPPELIN-4160] Fixed Move this note to trash icon always show

2019-05-15 Thread GitBox
liuxunorg commented on issue #3365: [ZEPPELIN-4160] Fixed Move this note to 
trash icon always show
URL: https://github.com/apache/zeppelin/pull/3365#issuecomment-492884302
 
 
   > can you explain the change from `[0]` to `[1]`?
   
   @felixcheung , This is because of the problem caused by this 
[ZEPPELIN-2619](https://github.com/apache/zeppelin/commit/085efeb646f3a95b39bf555b5b04551119e3c0b9).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on a change in pull request #3365: [ZEPPELIN-4160] Fixed Move this note to trash icon always show

2019-05-15 Thread GitBox
liuxunorg commented on a change in pull request #3365: [ZEPPELIN-4160] Fixed 
Move this note to trash icon always show
URL: https://github.com/apache/zeppelin/pull/3365#discussion_r284510848
 
 

 ##
 File path: zeppelin-web/src/app/notebook/notebook.controller.js
 ##
 @@ -237,7 +237,8 @@ function NotebookCtrl($scope, $route, $routeParams, 
$location, $rootScope,
   };
 
   $scope.isTrash = function(note) {
-return note ? note.name.split('/')[0] === TRASH_FOLDER_ID : false;
+console.log('note.path = ' + note.path);
 
 Review comment:
   ok.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-15 Thread GitBox
liuxunorg commented on issue #3359: [ZEPPELIN-4131] Refactoring shell 
interpreter
URL: https://github.com/apache/zeppelin/pull/3359#issuecomment-492882297
 
 
   @Tagar , this new shell interpreter support `Kerberos auto refresh`, but not 
support `interpolations`.
   
   I think either use this new interpreter with the `%sh.terminal` keyword? 
What do you think?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] Tagar commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-15 Thread GitBox
Tagar commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter
URL: https://github.com/apache/zeppelin/pull/3359#issuecomment-492824721
 
 
   As part of Zeppelin notes we have static %sh paragraphs (non-interactive) 
with 
   or without variable embeddings (variable interpolations ) like described in 
   
https://zeppelin.apache.org/docs/0.8.1/interpreter/shell.html#object-interpolation
 
   
   Would this new implementation keep that possible? (without typing in a 
command each time)
   
   I totally agree it's a very cool feature, but we may need to preserve old 
good `%sh` interpreter
   if this new approach takes some of its functionality away. 
   
   Thoughts?
   
   Thank you @liuxunorg 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] Leemoonsoo commented on issue #3363: [ZEPPELIN-4144] After refresh the revision tools are sometimes hidden

2019-05-15 Thread GitBox
Leemoonsoo commented on issue #3363: [ZEPPELIN-4144] After refresh the revision 
tools are sometimes hidden
URL: https://github.com/apache/zeppelin/pull/3363#issuecomment-492770214
 
 
   Thanks @florpor for the contribution!
   Could you check message from Jenkins and try setup travis, and make CI test 
pass?
   
   ```
   Looks like travis-ci is not configured for your fork.
   Please setup by swich on 'zeppelin' repository at 
https://travis-ci.org/profile and travis-ci.
   And then make sure 'Build branch updates' option is enabled in the settings 
https://travis-ci.org/florpor/zeppelin/settings.
   
   To trigger CI after setup, you will need ammend your last commit with
   git commit --amend
   git push your-remote HEAD --force
   
   See 
http://zeppelin.apache.org/contribution/contributions.html#continuous-integration.
   Build step 'Execute shell' marked build as failure
   Putting comment on the pull request
   Finished: FAILURE
   ```
   
   Let me know if you need any help setting up CI.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] felixcheung commented on a change in pull request #3364: [ZEPPELIN-4155] Unable get available HostAddress in multi-NIC environment

2019-05-15 Thread GitBox
felixcheung commented on a change in pull request #3364: [ZEPPELIN-4155] Unable 
get available HostAddress in multi-NIC environment
URL: https://github.com/apache/zeppelin/pull/3364#discussion_r284328179
 
 

 ##
 File path: 
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterUtils.java
 ##
 @@ -93,22 +97,53 @@ public static TServerSocket createTServerSocket(String 
portRange)
 throw new IOException("No available port in the portRange: " + portRange);
   }
 
+  // 1) Multiple NetworkCard will be configured on some servers
+  // 2) In the docker container environment, A container will also generate 
multiple virtual NetworkCard
   public static String findAvailableHostAddress() throws UnknownHostException, 
SocketException {
-InetAddress address = InetAddress.getLocalHost();
-if (address.isLoopbackAddress()) {
-  for (NetworkInterface networkInterface : Collections
-  .list(NetworkInterface.getNetworkInterfaces())) {
-if (!networkInterface.isLoopback()) {
-  for (InterfaceAddress interfaceAddress : 
networkInterface.getInterfaceAddresses()) {
-InetAddress a = interfaceAddress.getAddress();
-if (a instanceof Inet4Address) {
-  return a.getHostAddress();
-}
+List netlist = new ArrayList();
+
+// Get all the network cards in the current environment
+Enumeration netInterfaces = NetworkInterface.getNetworkInterfaces();
+while (netInterfaces.hasMoreElements()) {
+  NetworkInterface networkInterface = 
(NetworkInterface)netInterfaces.nextElement();
+  LOGGER.info("networkInterface = " + networkInterface.toString());
+  if (networkInterface.isLoopback()) {
+// Filter lo network card
+continue;
+  }
+  // When all the network cards are obtained by the above method,
+  // The order obtained is the reverse of the order of the NICs
+  // seen in the server with the ifconfig command.
+  // Therefore, when you want to traverse from the first NIC,
+  // Need to reverse the elements in Enumeration
+  netlist.add(0, networkInterface);
+}
+
+for (NetworkInterface list:netlist) {
+  Enumeration enumInetAddress = list.getInetAddresses();
+
+  while (enumInetAddress.hasMoreElements()) {
+InetAddress ip = (InetAddress) enumInetAddress.nextElement();
+LOGGER.info("ip = " + ip.toString());
+if (!ip.isLoopbackAddress()) {
+  if (ip.getHostAddress().equalsIgnoreCase("127.0.0.1")){
 
 Review comment:
   isn't `isLoopbackAddress` enough?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] felixcheung commented on a change in pull request #3365: [ZEPPELIN-4160] Fixed Move this note to trash icon always show

2019-05-15 Thread GitBox
felixcheung commented on a change in pull request #3365: [ZEPPELIN-4160] Fixed 
Move this note to trash icon always show
URL: https://github.com/apache/zeppelin/pull/3365#discussion_r284327576
 
 

 ##
 File path: zeppelin-web/src/app/notebook/notebook.controller.js
 ##
 @@ -237,7 +237,8 @@ function NotebookCtrl($scope, $route, $routeParams, 
$location, $rootScope,
   };
 
   $scope.isTrash = function(note) {
-return note ? note.name.split('/')[0] === TRASH_FOLDER_ID : false;
+console.log('note.path = ' + note.path);
 
 Review comment:
   remove this before merging?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on issue #3365: [ZEPPELIN-4160] Fixed Move this note to trash icon always show

2019-05-15 Thread GitBox
liuxunorg commented on issue #3365: [ZEPPELIN-4160] Fixed Move this note to 
trash icon always show
URL: https://github.com/apache/zeppelin/pull/3365#issuecomment-492710080
 
 
   @zjffdu , Please help me review code. Thanks!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on issue #3364: [ZEPPELIN-4155] Unable get available HostAddress in multi-NIC environment

2019-05-15 Thread GitBox
liuxunorg commented on issue #3364: [ZEPPELIN-4155] Unable get available 
HostAddress in multi-NIC environment
URL: https://github.com/apache/zeppelin/pull/3364#issuecomment-492710247
 
 
   @zjffdu , Please help me review code. Thanks!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg opened a new pull request #3365: [ZEPPELIN-4160] Fixed Move this note to trash icon always show

2019-05-15 Thread GitBox
liuxunorg opened a new pull request #3365: [ZEPPELIN-4160] Fixed Move this note 
to trash icon always show
URL: https://github.com/apache/zeppelin/pull/3365
 
 
   ### What is this PR for?
   when the note moved to trash, the note should show 'Remove this note 
permanently', it does not work correctly after clicked again, more ~Trash 
folder will be created
   
   
   ### What type of PR is it?
   [Bug Fix]
   
   ### Todos
   * [ ] - Task
   
   ### What is the Jira issue?
   * https://issues.apache.org/jira/browse/ZEPPELIN-4160
   
   ### How should this be tested?
   * [CI pass](https://travis-ci.org/liuxunorg/zeppelin/builds/532764123)
   
   ### Screenshots (if appropriate)
   
   ### Questions:
   * Does the licenses files need update? NO
   * Is there breaking changes for older versions? NO
   * Does this needs documentation? NO
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-15 Thread GitBox
liuxunorg commented on issue #3359: [ZEPPELIN-4131] Refactoring shell 
interpreter
URL: https://github.com/apache/zeppelin/pull/3359#issuecomment-492696041
 
 
   # The function is exactly the same as the terminal emulator
   ## curl + tar ScreenRecord
   
![shell-curl-tar](https://user-images.githubusercontent.com/3677382/57786530-34efa180-7766-11e9-9d91-520743ca0d60.gif)
   
   ## python + gcc ScreenRecord
   
![shell-python-gcc](https://user-images.githubusercontent.com/3677382/57786554-420c9080-7766-11e9-86dd-8c3bc82dc858.gif)
   
   ## call script ScreenRecord
   
![shll-script](https://user-images.githubusercontent.com/3677382/57786572-4df85280-7766-11e9-96a0-83e9a70307d7.gif)
   
   
   
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] FireArrow commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-15 Thread GitBox
FireArrow commented on issue #3359: [ZEPPELIN-4131] Refactoring shell 
interpreter
URL: https://github.com/apache/zeppelin/pull/3359#issuecomment-492659189
 
 
   We are using %sh pretty much exactly like @Leemoonsoo described it above. I 
think this is a really cool feature, but I don't think it should replace the 
%sh magic, but rather get its own, new, magic (as @Tagar suggested above). 
`%terminal` or `%shell` perhaps?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on issue #3360: [ZEPPELIN-4152]. Use spark 2.2 as the default profile

2019-05-15 Thread GitBox
zjffdu commented on issue #3360: [ZEPPELIN-4152]. Use spark 2.2 as the default 
profile
URL: https://github.com/apache/zeppelin/pull/3360#issuecomment-492656521
 
 
   Unfortunately, spark 2.3 doesn't support scala 2.10 which is default 
profile. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] Leemoonsoo commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-15 Thread GitBox
Leemoonsoo commented on issue #3359: [ZEPPELIN-4131] Refactoring shell 
interpreter
URL: https://github.com/apache/zeppelin/pull/3359#issuecomment-492652747
 
 
   I can think a use case, such as
   
   ```
   %sh
   # download some data
   curl -O http://
   
   # extract
   tar -xzf 
   ```
   
   ```
   %python
   
   # load downloaded data
   df = pd.read_csv("...")
   ...
   ```
   
   This case, %sh interpreter may better have a script to run, rather than 
expecting interactive user input.
   Is this new %sh interpreter implementation capable of doing this as well?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on a change in pull request #3361: ZEPPELIN-4153. Move zeppelin-interpreter-api jar out of interpreter jar

2019-05-14 Thread GitBox
zjffdu commented on a change in pull request #3361: ZEPPELIN-4153. Move 
zeppelin-interpreter-api jar out of interpreter jar
URL: https://github.com/apache/zeppelin/pull/3361#discussion_r284069568
 
 

 ##
 File path: cassandra/pom.xml
 ##
 @@ -58,7 +58,13 @@
 org.apache.zeppelin
 zeppelin-interpreter
 ${project.version}
-test
+provided
 
 Review comment:
   right, I fixed it 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-14 Thread GitBox
liuxunorg commented on issue #3359: [ZEPPELIN-4131] Refactoring shell 
interpreter
URL: https://github.com/apache/zeppelin/pull/3359#issuecomment-492481141
 
 
   @Tagar This is a good question, @zjffdu  Is there any special use of the 
shell interpreter that needs to be retained?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on a change in pull request #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-14 Thread GitBox
liuxunorg commented on a change in pull request #3359: [ZEPPELIN-4131] 
Refactoring shell interpreter
URL: https://github.com/apache/zeppelin/pull/3359#discussion_r284067703
 
 

 ##
 File path: shell/src/main/resources/ui_templates/terminal-dashboard.jinja
 ##
 @@ -0,0 +1,40 @@
+{#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+-#}
+
+  
+Terminal - {{ HOST_NAME }} ({{ HOST_IP }})
+STATUS: 
+CONNECT
 
 Review comment:
   Ok, I am correcting it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on a change in pull request #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-14 Thread GitBox
liuxunorg commented on a change in pull request #3359: [ZEPPELIN-4131] 
Refactoring shell interpreter
URL: https://github.com/apache/zeppelin/pull/3359#discussion_r284067723
 
 

 ##
 File path: shell/src/main/resources/ui_templates/terminal-dashboard.jinja
 ##
 @@ -0,0 +1,40 @@
+{#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+-#}
+
+  
+Terminal - {{ HOST_NAME }} ({{ HOST_IP }})
+STATUS: 
+CONNECT
+DISCONNECT
 
 Review comment:
   Ok, I am correcting it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] Tagar commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-14 Thread GitBox
Tagar commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter
URL: https://github.com/apache/zeppelin/pull/3359#issuecomment-492314076
 
 
   Very cool.
   Perhaps this should be a new interpreter called Terminal? 
   And leave Shell interpreter as is? 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] Tagar commented on a change in pull request #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-14 Thread GitBox
Tagar commented on a change in pull request #3359: [ZEPPELIN-4131] Refactoring 
shell interpreter
URL: https://github.com/apache/zeppelin/pull/3359#discussion_r283893958
 
 

 ##
 File path: shell/src/main/resources/ui_templates/terminal-dashboard.jinja
 ##
 @@ -0,0 +1,40 @@
+{#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+-#}
+
+  
+Terminal - {{ HOST_NAME }} ({{ HOST_IP }})
+STATUS: 
+CONNECT
+DISCONNECT
 
 Review comment:
   disconnectED


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] Tagar commented on a change in pull request #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-14 Thread GitBox
Tagar commented on a change in pull request #3359: [ZEPPELIN-4131] Refactoring 
shell interpreter
URL: https://github.com/apache/zeppelin/pull/3359#discussion_r283893737
 
 

 ##
 File path: shell/src/main/resources/ui_templates/terminal-dashboard.jinja
 ##
 @@ -0,0 +1,40 @@
+{#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+-#}
+
+  
+Terminal - {{ HOST_NAME }} ({{ HOST_IP }})
+STATUS: 
+CONNECT
 
 Review comment:
   CONNECTED?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg opened a new pull request #3364: [ZEPPELIN-4155] Unable get available HostAddress in multi-NIC environment

2019-05-14 Thread GitBox
liuxunorg opened a new pull request #3364: [ZEPPELIN-4155] Unable get available 
HostAddress in multi-NIC environment
URL: https://github.com/apache/zeppelin/pull/3364
 
 
   ### What is this PR for?
   1. Multiple NetworkCard will be configured on some servers.
   2. In the docker container environment, A container will also generate 
multiple virtual NetworkCard.
   
   `RemoteInterpreterUtils#findAvailableHostAddress()` sometimes can't get the 
correct network card information.
   
   
   ### What type of PR is it?
   [Improvement]
   
   ### Todos
   * [ ] - Task
   
   ### What is the Jira issue?
   * https://issues.apache.org/jira/browse/ZEPPELIN-4155
   
   ### How should this be tested?
   * [CI pass](https://travis-ci.org/liuxunorg/zeppelin/builds/532212106)
   
   ### Screenshots (if appropriate)
   
   ### Questions:
   * Does the licenses files need update? NO
   * Is there breaking changes for older versions? NO
   * Does this needs documentation? NO
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] asfgit closed pull request #3362: [ZEPPELIN-4159] Fixed ZeppelinIT#testAngularDisplay() failure

2019-05-14 Thread GitBox
asfgit closed pull request #3362: [ZEPPELIN-4159] Fixed 
ZeppelinIT#testAngularDisplay() failure
URL: https://github.com/apache/zeppelin/pull/3362
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] felixcheung commented on a change in pull request #3361: ZEPPELIN-4153. Move zeppelin-interpreter-api jar out of interpreter jar

2019-05-14 Thread GitBox
felixcheung commented on a change in pull request #3361: ZEPPELIN-4153. Move 
zeppelin-interpreter-api jar out of interpreter jar
URL: https://github.com/apache/zeppelin/pull/3361#discussion_r283649347
 
 

 ##
 File path: cassandra/pom.xml
 ##
 @@ -58,7 +58,13 @@
 org.apache.zeppelin
 zeppelin-interpreter
 ${project.version}
-test
+provided
 
 Review comment:
   hmm, is this right?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] florpor opened a new pull request #3363: [ZEPPELIN-4144] After refresh the revision tools are sometimes hidden

2019-05-13 Thread GitBox
florpor opened a new pull request #3363: [ZEPPELIN-4144] After refresh the 
revision tools are sometimes hidden
URL: https://github.com/apache/zeppelin/pull/3363
 
 
   ### What is this PR for?
   Fixes issue where after refreshing a note's page the revision controls in 
the navigation bar are hidden since listConfigurations is only requested in the 
navbar controller after a loginSuccess broadcast.
   This currently works sometimes because another controller (note-import) 
sends a listConfigurations request when it is loaded and navbar controller is 
sometimes loaded by the time the response arrives, but I can reproduce it 
easily using Firefox (gif attached to Jira issue).
   
   
   ### What type of PR is it?
   Bug Fix
   
   ### Todos
   
   ### What is the Jira issue?
   
[https://issues.apache.org/jira/browse/ZEPPELIN-4144](https://issues.apache.org/jira/browse/ZEPPELIN-4144)
   
   ### How should this be tested?
   Added unit and e2e tests to make sure listConfigurations is called and 
controls are visible.
   Seems to me like the issue is easier to reproduce using Firefox. Was able to 
do so using the official 0.8.1 docker image.
   
   ### Questions:
   * Does the licenses files need update? No
   * Is there breaking changes for older versions? No
   * Does this needs documentation? No
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on issue #3362: [ZEPPELIN-4159] Fixed ZeppelinIT#testAngularDisplay() failure

2019-05-13 Thread GitBox
zjffdu commented on issue #3362: [ZEPPELIN-4159] Fixed 
ZeppelinIT#testAngularDisplay() failure
URL: https://github.com/apache/zeppelin/pull/3362#issuecomment-492080337
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on issue #3362: [ZEPPELIN-4159] Fixed ZeppelinIT#testAngularDisplay() failure

2019-05-13 Thread GitBox
liuxunorg commented on issue #3362: [ZEPPELIN-4159] Fixed 
ZeppelinIT#testAngularDisplay() failure
URL: https://github.com/apache/zeppelin/pull/3362#issuecomment-492075663
 
 
   @zjffdu, @felixcheung  , [CI 
pass](https://travis-ci.org/liuxunorg/zeppelin/builds/532093683), Please help 
me review code, Thanks!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg opened a new pull request #3362: [ZEPPELIN-4159] Fixed ZeppelinIT#testAngularDisplay() failure

2019-05-13 Thread GitBox
liuxunorg opened a new pull request #3362: [ZEPPELIN-4159] Fixed 
ZeppelinIT#testAngularDisplay() failure
URL: https://github.com/apache/zeppelin/pull/3362
 
 
   ### What is this PR for?
   1. setTextOfParagraph(1, "println(\"%angular BindingTest_{{myVar}}_\")");
   2. driver.findElement(By.xpath(getParagraphXPath(1) + 
"//div[@id=\"angularTestButton\"]")).click();
   
   ```
   org.openqa.selenium.NoSuchElementException: Unable to locate element: 
{"method":"xpath","selector":"(//div[@ng-controller=\"ParagraphCtrl\"])[1]//div[@id=\"angularTestButton\"]"}
   Command duration or timeout: 0 milliseconds
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
   at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
   at 
org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:214)
   at 
org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:166)
   at 
org.openqa.selenium.remote.http.JsonHttpResponseCodec.reconstructValue(JsonHttpResponseCodec.java:40)
   at 
org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:80)
   at 
org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:44)
   at 
org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:164)
   at 
org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
   at 
org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:601)
   at 
org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:371)
   at 
org.openqa.selenium.remote.RemoteWebDriver.findElementByXPath(RemoteWebDriver.java:473)
   at org.openqa.selenium.By$ByXPath.findElement(By.java:361)
   at 
org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:363)
   at 
org.apache.zeppelin.integration.ZeppelinIT.testAngularDisplay(ZeppelinIT.java:117)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
   at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
   at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
   at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
   at org.junit.rules.Verifier$1.evaluate(Verifier.java:35)
   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
   at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
   at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
   at 
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
   at 
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
   at 
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
   at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
   Caused by: org.openqa.selenium.NoSuchElementException: Unable to locate 
element: 
{"method":"xpath","selector":"(//div[@ng-controller=\"ParagraphCtrl\"])[1]//div[@id=\"angularTestButton\"]"}
   ```
   
   
   ### What type of PR is it?
   [Bug Fix]
   
   
   ### What is the Jira issue?
   * https://issues.apache.org/jira/browse/ZEPPELIN-4159
   
   ### How should this be tested?
   * [CI pass](https://travis-ci.org/liuxunorg/zeppelin/builds/532093683)
   
   ### Screenshots (if appropriate)
   
   ### Questions:
   * Does the licenses files need update? NO
   * Is there breaking changes for older versions? NO
   * Does this needs documentation? NO
   


This is an automated message from the Apache Git 

[GitHub] [zeppelin] zjffdu opened a new pull request #3361: ZEPPELIN-4153. Move zeppelin-interpreter-api jar out of interpreter jar

2019-05-13 Thread GitBox
zjffdu opened a new pull request #3361: ZEPPELIN-4153. Move 
zeppelin-interpreter-api jar out of interpreter jar
URL: https://github.com/apache/zeppelin/pull/3361
 
 
   ### What is this PR for?
   This PR is to move zeppelin-interpreter-api jar out of interpreter jar, and 
put it under folder ${ZEPPELIN_HOME}/interpreter which is shared between all 
interpreters.
   
   
   ### What type of PR is it?
   [Improvement]
   
   ### Todos
   * [ ] - Task
   
   ### What is the Jira issue?
   * https://issues.apache.org/jira/browse/ZEPPELIN-4153
   
   ### How should this be tested?
   * CI pass
   
   ### Screenshots (if appropriate)
   
   ### Questions:
   * Does the licenses files need update? No
   * Is there breaking changes for older versions? No
   * Does this needs documentation? No
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] dbanda commented on issue #3315: Update twittr maven to HTTPS [ZEPPELIN-4156]

2019-05-13 Thread GitBox
dbanda commented on issue #3315: Update twittr maven to HTTPS [ZEPPELIN-4156]
URL: https://github.com/apache/zeppelin/pull/3315#issuecomment-491924996
 
 
   > ping @dbanda Could you address the comments ?
   
   Addressed. Added the Jira and updated PR title


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] asfgit closed pull request #3348: changed-readme-for-python-interpreter-for-apache-zeppelin

2019-05-13 Thread GitBox
asfgit closed pull request #3348: 
changed-readme-for-python-interpreter-for-apache-zeppelin
URL: https://github.com/apache/zeppelin/pull/3348
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on issue #3315: update twittr maven to HTTPS

2019-05-13 Thread GitBox
zjffdu commented on issue #3315: update twittr maven to HTTPS
URL: https://github.com/apache/zeppelin/pull/3315#issuecomment-491829926
 
 
   ping @dbanda Could you address the comments ? 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on issue #3348: changed-readme-for-python-interpreter-for-apache-zeppelin

2019-05-13 Thread GitBox
zjffdu commented on issue #3348: 
changed-readme-for-python-interpreter-for-apache-zeppelin
URL: https://github.com/apache/zeppelin/pull/3348#issuecomment-491829180
 
 
   Thanks @d3adp007 , merging


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu opened a new pull request #3360: [ZEPPELIN-4152]. Use spark 2.2 as the default profile

2019-05-13 Thread GitBox
zjffdu opened a new pull request #3360: [ZEPPELIN-4152]. Use spark 2.2 as the 
default profile
URL: https://github.com/apache/zeppelin/pull/3360
 
 
   ### What is this PR for?
   Trivial PR to use spark 2.2 as the default profile so that the default build 
command can pass
   
   ```
   mvn clean package -DskipTests
   ```
   
   
   ### What type of PR is it?
   [ Improvement ]
   
   ### Todos
   * [ ] - Task
   
   ### What is the Jira issue?
   * https://issues.apache.org/jira/browse/ZEPPELIN-4152
   
   ### How should this be tested?
   * CI pass
   
   ### Screenshots (if appropriate)
   
   ### Questions:
   * Does the licenses files need update? No
   * Is there breaking changes for older versions? No
   * Does this needs documentation? No
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-12 Thread GitBox
liuxunorg commented on issue #3359: [ZEPPELIN-4131] Refactoring shell 
interpreter
URL: https://github.com/apache/zeppelin/pull/3359#issuecomment-491684736
 
 
   @zjffdu , `hterm_all.js` is exported via the chromium libapps project. 
`hterm_all.js` relies on several submodules in chromium libapps. `hterm_all.js` 
cannot be downloaded directly.
   Therefore, Need to perform the following operations in the official 
documentation to generate `hterm_all.js`.
   
   ## Get the code
   
   The “libapps” git repository contains hterm and its dependencies. Clone this 
repo into a parent directory of your choice. In this example we'll create 
`~/src/libapps/`:
   
   ```
   $ mkdir ~/src
   $ cd ~/src
   $ git clone https://chromium.googlesource.com/apps/libapps
   ```
   
   ## Build hterm
   
   The build process bundles some resources as JavaScript source and 
concatenates the JavaScript into a single file (`hterm_all.js`).
   
   ```
   $ cd ~/src/libapps
   $ ./hterm/bin/mkdist.sh
   ```
   
   The files are all written to `./hterm/dist/js/`. Copy the `hterm_all.js` 
file into your project.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-12 Thread GitBox
liuxunorg commented on issue #3359: [ZEPPELIN-4131] Refactoring shell 
interpreter
URL: https://github.com/apache/zeppelin/pull/3359#issuecomment-491682950
 
 
   @zjffdu , 《Refactoring zeppelin shell interpreter Design Document》 : 
https://docs.google.com/document/d/1-XUkXY06cMtEUBxVHVyqQPkyCkv2BjYhAQj-W1CBxNI/edit


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-12 Thread GitBox
zjffdu commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter
URL: https://github.com/apache/zeppelin/pull/3359#issuecomment-491682054
 
 
   @liuxunorg Is it possible to download hterm_all.js at building time instead 
of put into repo ? And could you write a simple design doc for it since it is a 
redesign of shell interpreter. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on issue #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-12 Thread GitBox
liuxunorg commented on issue #3359: [ZEPPELIN-4131] Refactoring shell 
interpreter
URL: https://github.com/apache/zeppelin/pull/3359#issuecomment-491677070
 
 
   @zjffdu , @felixcheung , Please help me review code. Thanks! :-)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg opened a new pull request #3359: [ZEPPELIN-4131] Refactoring shell interpreter

2019-05-12 Thread GitBox
liuxunorg opened a new pull request #3359: [ZEPPELIN-4131] Refactoring shell 
interpreter
URL: https://github.com/apache/zeppelin/pull/3359
 
 
   ### What is this PR for?
   Realize the same function as the terminal, The shell-by-command can be 
executed by the shell interpreter.
   
   [Refactoring zeppelin shell interpreter Design 
Document](https://docs.google.com/document/d/1-XUkXY06cMtEUBxVHVyqQPkyCkv2BjYhAQj-W1CBxNI/edit)
   
   ### What type of PR is it?
   [Improvement]
   
   ### What is the Jira issue?
   * https://issues.apache.org/jira/browse/ZEPPELIN-4131
   
   ### How should this be tested?
   * [CI Pass](https://travis-ci.org/liuxunorg/zeppelin/builds/531594343)
   
   ### Screenshots (if appropriate)
   
![shell-pr](https://user-images.githubusercontent.com/3677382/57596651-5e4dd900-757e-11e9-8bfc-ff4875af499c.gif)
   
   
   ### Questions:
   * Does the licenses files need update? NO
   * Is there breaking changes for older versions? NO
   * Does this needs documentation? YES
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on issue #3358: [ZEPPELIN-2444] Better error message on shell timeout being reached.

2019-05-10 Thread GitBox
liuxunorg commented on issue #3358: [ZEPPELIN-2444] Better error message on 
shell timeout being reached.
URL: https://github.com/apache/zeppelin/pull/3358#issuecomment-491252614
 
 
   @epugh , I will pay attention to the problem you reported. Thank you! :-)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] epugh commented on issue #3358: [ZEPPELIN-2444] Better error message on shell timeout being reached.

2019-05-10 Thread GitBox
epugh commented on issue #3358: [ZEPPELIN-2444] Better error message on shell 
timeout being reached.
URL: https://github.com/apache/zeppelin/pull/3358#issuecomment-491230906
 
 
   That definitly looks like a powerful feature!  I saw it briefly when I was 
researching this PR….  I wasn’t sure the timeline to put this in!  If this is 
the future, and the future is soon, please do close my PR in favor of this!
   
   
   > On May 10, 2019, at 5:42 AM, Xun Liu  wrote:
   > 
   > @epugh  , I refactored the shell interpreter and 
are preparing to submit the PR, Has made the shell interpreter function exactly 
the same as the terminal。
   > ZEPPELIN-4131 , 
Refactoring zeppelin shell interpreter Design Document 

   >  

   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub 
, or mute 
the thread 
.
   > 
   
   ___
   Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | 
http://www.opensourceconnections.com  | 
My Free/Busy   
   Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 

 
   This e-mail and all contents, including attachments, is considered to be 
Company Confidential unless explicitly stated otherwise, regardless of whether 
attachments are marked as such.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] liuxunorg commented on issue #3358: [ZEPPELIN-2444] Better error message on shell timeout being reached.

2019-05-10 Thread GitBox
liuxunorg commented on issue #3358: [ZEPPELIN-2444] Better error message on 
shell timeout being reached.
URL: https://github.com/apache/zeppelin/pull/3358#issuecomment-491228442
 
 
   @epugh ,  I refactored the shell interpreter and are preparing to submit the 
PR, Has made the shell interpreter function exactly the same as the terminal。
   [ZEPPELIN-4131](https://jira.apache.org/jira/browse/ZEPPELIN-4131),  
[Refactoring zeppelin shell interpreter Design 
Document](https://docs.google.com/document/d/1-XUkXY06cMtEUBxVHVyqQPkyCkv2BjYhAQj-W1CBxNI/edit#heading=h.hcfsn8dd2dw4)
   
   
![shell-pr](https://user-images.githubusercontent.com/3677382/57518103-e3e94300-734a-11e9-964b-fe95c6989ea9.gif)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] epugh commented on issue #3358: [ZEPPELIN-2444] Better error message on shell timeout being reached.

2019-05-10 Thread GitBox
epugh commented on issue #3358: [ZEPPELIN-2444] Better error message on shell 
timeout being reached.
URL: https://github.com/apache/zeppelin/pull/3358#issuecomment-491224213
 
 
   It may be…  I didn’t realize that it was until months later when I google 
around, and found the issue ZEPPELIN-2444!  
   
   When I got the SIGTERM error, I though it was my shell command (in my case a 
python pip install) had died due to something random, I didn’t think “oh, it 
took too long and hit a time out”….  
   
   > On May 10, 2019, at 2:35 AM, Felix Cheung  wrote:
   > 
   > @felixcheung commented on this pull request.
   > 
   > In shell/src/main/java/org/apache/zeppelin/shell/ShellInterpreter.java 
:
   > 
   > > @@ -132,7 +132,7 @@ public InterpreterResult internalInterpret(String 
cmd,
   >String message = outStream.toString();
   >if (exitValue == 143) {
   >  code = Code.INCOMPLETE;
   > -message += "Paragraph received a SIGTERM\n";
   > +message += "Timeout of " + getProperty(TIMEOUT_PROPERTY) + " ms 
to run reached.\n";
   > hmm, I'm not sure these are the same? SIGTERM is not a timeout?
   > 
   > —
   > You are receiving this because you authored the thread.
   > Reply to this email directly, view it on GitHub 
, or 
mute the thread 
.
   > 
   
   ___
   Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | 
http://www.opensourceconnections.com  | 
My Free/Busy   
   Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 

 
   This e-mail and all contents, including attachments, is considered to be 
Company Confidential unless explicitly stated otherwise, regardless of whether 
attachments are marked as such.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] felixcheung commented on a change in pull request #3358: [ZEPPELIN-2444] Better error message on shell timeout being reached.

2019-05-10 Thread GitBox
felixcheung commented on a change in pull request #3358: [ZEPPELIN-2444] Better 
error message on shell timeout being reached.
URL: https://github.com/apache/zeppelin/pull/3358#discussion_r282761598
 
 

 ##
 File path: shell/src/main/java/org/apache/zeppelin/shell/ShellInterpreter.java
 ##
 @@ -132,7 +132,7 @@ public InterpreterResult internalInterpret(String cmd,
   String message = outStream.toString();
   if (exitValue == 143) {
 code = Code.INCOMPLETE;
-message += "Paragraph received a SIGTERM\n";
+message += "Timeout of " + getProperty(TIMEOUT_PROPERTY) + " ms to run 
reached.\n";
 
 Review comment:
   hmm, I'm not sure these are the same? SIGTERM is not a timeout?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] epugh opened a new pull request #3358: Fix for ZEPPELIN-2444

2019-05-09 Thread GitBox
epugh opened a new pull request #3358: Fix for ZEPPELIN-2444
URL: https://github.com/apache/zeppelin/pull/3358
 
 
   ### What is this PR for?
   Cue the user what the error they are getting is due to.  I would get 
spurious timeouts, and I had no idea it was a timeout, and that I had the 
option of extending the timeout.  Same issue as the reporter of ZEPPELIN-2444.  
   
   
   ### What type of PR is it?
   Bug Fix
   
   ### Todos
   
   
   ### What is the Jira issue?
   https://issues.apache.org/jira/browse/ZEPPELIN-2444
   
   ### How should this be tested?
   Run sleep 61 and verify you get a nice error message.  Unit test was updated.
   
   ### Screenshots (if appropriate)
   
   ### Questions:
   * Does the licenses files need update? No
   * Is there breaking changes for older versions? No
   * Does this needs documentation? No.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] jongyoul commented on a change in pull request #3353: [ZEPPELIN-4132]. Spark Interpreter has issue of SPARK-22393

2019-05-04 Thread GitBox
jongyoul commented on a change in pull request #3353: [ZEPPELIN-4132]. Spark 
Interpreter has issue of SPARK-22393
URL: https://github.com/apache/zeppelin/pull/3353#discussion_r280989658
 
 

 ##
 File path: .travis.yml
 ##
 @@ -109,7 +109,7 @@ matrix:
 - sudo: required
   jdk: "oraclejdk8"
   dist: trusty
-  env: BUILD_PLUGINS="true" PYTHON="3" SCALA_VER="2.11" 
PROFILE="-Pspark-2.2 -Pscala-2.11 -Phadoop2 -Pintegration" SPARKR="true" 
BUILD_FLAG="install -DskipTests -DskipRat -am" TEST_FLAG="test -DskipRat -am" 
MODULES="-pl 
zeppelin-interpreter-integration,zeppelin-web,spark/spark-dependencies" 
TEST_PROJECTS="-Dtest=ZeppelinSparkClusterTest22,SparkIntegrationTest22,org.apache.zeppelin.spark.*
 -DfailIfNoTests=false"
+  env: BUILD_PLUGINS="true" PYTHON="3" SCALA_VER="2.11" 
PROFILE="-Pspark-2.2 -Pscala-2.10 -Phadoop2 -Pintegration" SPARKR="true" 
BUILD_FLAG="install -DskipTests -DskipRat -am" TEST_FLAG="test -DskipRat -am" 
MODULES="-pl 
zeppelin-interpreter-integration,zeppelin-web,spark/spark-dependencies" 
TEST_PROJECTS="-Dtest=ZeppelinSparkClusterTest22,SparkIntegrationTest22,org.apache.zeppelin.spark.*
 -DfailIfNoTests=false"
 
 Review comment:
   May we need to change `SCALA_VER` as well?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] chrissng commented on a change in pull request #3240: [ZEPPELIN-3840] Zeppelin on Kubernetes

2019-05-03 Thread GitBox
chrissng commented on a change in pull request #3240: [ZEPPELIN-3840] Zeppelin 
on Kubernetes
URL: https://github.com/apache/zeppelin/pull/3240#discussion_r280949311
 
 

 ##
 File path: 
zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sStandardInterpreterLauncher.java
 ##
 @@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.interpreter.launcher;
+
+import java.io.File;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.nio.charset.Charset;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+
+import com.google.common.annotations.VisibleForTesting;
+import org.apache.zeppelin.conf.ZeppelinConfiguration;
+import org.apache.zeppelin.interpreter.recovery.RecoveryStorage;
+import org.apache.zeppelin.interpreter.remote.RemoteInterpreterUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Interpreter Launcher which use shell script to launch the interpreter 
process.
+ */
+public class K8sStandardInterpreterLauncher extends InterpreterLauncher {
+
+  private static final Logger LOGGER = 
LoggerFactory.getLogger(K8sStandardInterpreterLauncher.class);
+  private final Kubectl kubectl;
+  private InterpreterLaunchContext context;
+
+
+  public K8sStandardInterpreterLauncher(ZeppelinConfiguration zConf, 
RecoveryStorage recoveryStorage) throws IOException {
+super(zConf, recoveryStorage);
+kubectl = new Kubectl(zConf.getK8sKubectlCmd());
+kubectl.setNamespace(getNamespace());
+  }
+
+  @VisibleForTesting
+  K8sStandardInterpreterLauncher(ZeppelinConfiguration zConf, RecoveryStorage 
recoveryStorage, Kubectl kubectl) {
+super(zConf, recoveryStorage);
+this.kubectl = kubectl;
+  }
+
+
+  /**
+   * Check if i'm running inside of kubernetes or not.
+   * It should return truth regardless of ZeppelinConfiguration.getRunMode().
+   *
+   * Normally, unless Zeppelin is running on Kubernetes, 
K8sStandardInterpreterLauncher shouldn't even have initialized.
+   * However, when ZeppelinConfiguration.getRunMode() is force 'k8s', 
InterpreterSetting.getLauncherPlugin() will try
+   * to use K8sStandardInterpreterLauncher. This is useful for development. It 
allows Zeppelin server running on your
+   * IDE and creates your interpreters in Kubernetes. So any code changes on 
Zeppelin server or kubernetes yaml spec
+   * can be applied without re-building docker image.
+   * @return
+   */
+  boolean isRunningOnKubernetes() {
+if (new File("/var/run/secrets/kubernetes.io").exists()) {
+  return true;
+} else {
+  return false;
+}
+  }
+
+  /**
+   * Get current namespace
+   * @throws IOException
+   */
+  String getNamespace() throws IOException {
+if (isRunningOnKubernetes()) {
+  return 
readFile("/var/run/secrets/kubernetes.io/serviceaccount/namespace", 
Charset.defaultCharset()).trim();
+} else {
+  return "default";
+}
+  }
+
+  /**
+   * Get hostname. It should be the same to Service name (and Pod name) of the 
Kubernetes
+   * @return
+   */
+  String getHostname() {
+try {
+  return InetAddress.getLocalHost().getHostName();
+} catch (UnknownHostException e) {
+  return "localhost";
+}
+  }
+
+  /**
+   * get Zeppelin server host dns.
+   * return ..svc.cluster.local
+   * @throws IOException
+   */
+  private String getZeppelinServiceHost() throws IOException {
+if (isRunningOnKubernetes()) {
+  return String.format("%s.%s.svc.cluster.local",
+  getHostname(), // service name and pod name should be the same
 
 Review comment:
   That should work  


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] Leemoonsoo commented on a change in pull request #3240: [ZEPPELIN-3840] Zeppelin on Kubernetes

2019-05-03 Thread GitBox
Leemoonsoo commented on a change in pull request #3240: [ZEPPELIN-3840] 
Zeppelin on Kubernetes
URL: https://github.com/apache/zeppelin/pull/3240#discussion_r280916925
 
 

 ##
 File path: 
zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/Kubectl.java
 ##
 @@ -0,0 +1,157 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.interpreter.launcher;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.gson.Gson;
+import java.util.ArrayList;
+import java.util.Arrays;
+
+import org.apache.commons.exec.*;
+import org.apache.commons.io.IOUtils;
+
+import java.io.*;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class Kubectl {
+  private final Logger LOGGER = LoggerFactory.getLogger(Kubectl.class);
+  private final String kubectlCmd;
+  private final Gson gson = new Gson();
+  private String namespace;
+
+  public Kubectl(String kubectlCmd) {
+this.kubectlCmd = kubectlCmd;
+  }
+
+  /**
+   * Override namespace. Otherwise use namespace provided in schema
+   * @param namespace
+   */
+  public void setNamespace(String namespace) {
+this.namespace = namespace;
+  }
+
+  public String getNamespace() {
+return namespace;
+  }
+
+  public String apply(String spec) throws IOException {
+return execAndGet(new String[]{"apply", "-f", "-"}, spec);
+  }
+
+  public String delete(String spec) throws IOException {
+return execAndGet(new String[]{"delete", "-f", "-"}, spec);
+  }
+
+  public String wait(String resource, String waitFor, int timeoutSec) throws 
IOException {
+try {
+  return execAndGet(new String[]{
+  "wait",
+  resource,
+  String.format("--for=%s", waitFor),
+  String.format("--timeout=%ds", timeoutSec)});
+} catch (IOException e) {
+  if ("delete".equals(waitFor) && e.getMessage().contains("NotFound")) {
+LOGGER.info("{} Not found. Maybe already deleted.", resource);
+return "";
+  } else {
+throw e;
+  }
+}
+  }
+
+  public ExecuteWatchdog portForward(String resource, String [] ports) throws 
IOException {
+DefaultExecutor executor = new DefaultExecutor();
+CommandLine cmd = new CommandLine(kubectlCmd);
+cmd.addArguments("port-forward");
+cmd.addArguments(resource);
+cmd.addArguments(ports);
+
+ExecuteWatchdog watchdog = new ExecuteWatchdog(-1);
+executor.setWatchdog(watchdog);
+
+executor.execute(cmd, new ExecuteResultHandler() {
+  @Override
+  public void onProcessComplete(int i) {
+LOGGER.info("Port-forward stopped");
+  }
+
+  @Override
+  public void onProcessFailed(ExecuteException e) {
+LOGGER.debug("port-forward process exit", e);
+  }
+});
+
+return watchdog;
+  }
+
+  String execAndGet(String [] args) throws IOException {
+return execAndGet(args, "");
+  }
+
+  @VisibleForTesting
+  String execAndGet(String [] args, String stdin) throws IOException {
+InputStream ins = IOUtils.toInputStream(stdin);
+ByteArrayOutputStream stdout = new ByteArrayOutputStream();
+ByteArrayOutputStream stderr = new ByteArrayOutputStream();
+ArrayList argsToOverride = new ArrayList<>(Arrays.asList(args));
+
+// set namespace
+if (namespace != null) {
+  argsToOverride.add("--namespace=" + namespace);
+}
+
+LOGGER.info("kubectl " + argsToOverride);
+LOGGER.debug(stdin);
+
+try {
+  int exitCode = execute(
+  argsToOverride.toArray(new String[0]),
+  ins,
+  stdout,
+  stderr
+  );
+
+  if (exitCode == 0) {
+String output = new String(stdout.toByteArray());
+return output;
+  } else {
+String output = new String(stderr.toByteArray());
+throw new IOException(String.format("non zero return code (%d). %s", 
exitCode, output));
+  }
+} catch (Exception e) {
+  String output = new String(stderr.toByteArray());
+  throw new IOException(output, e);
+}
+  }
+
+  public int execute(String [] args, InputStream stdin, OutputStream stdout, 
OutputStream stderr) throws IOException {
+

[GitHub] [zeppelin] Leemoonsoo commented on a change in pull request #3240: [ZEPPELIN-3840] Zeppelin on Kubernetes

2019-05-03 Thread GitBox
Leemoonsoo commented on a change in pull request #3240: [ZEPPELIN-3840] 
Zeppelin on Kubernetes
URL: https://github.com/apache/zeppelin/pull/3240#discussion_r280916321
 
 

 ##
 File path: 
zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sStandardInterpreterLauncher.java
 ##
 @@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.interpreter.launcher;
+
+import java.io.File;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.nio.charset.Charset;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+
+import com.google.common.annotations.VisibleForTesting;
+import org.apache.zeppelin.conf.ZeppelinConfiguration;
+import org.apache.zeppelin.interpreter.recovery.RecoveryStorage;
+import org.apache.zeppelin.interpreter.remote.RemoteInterpreterUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Interpreter Launcher which use shell script to launch the interpreter 
process.
+ */
+public class K8sStandardInterpreterLauncher extends InterpreterLauncher {
+
+  private static final Logger LOGGER = 
LoggerFactory.getLogger(K8sStandardInterpreterLauncher.class);
+  private final Kubectl kubectl;
+  private InterpreterLaunchContext context;
+
+
+  public K8sStandardInterpreterLauncher(ZeppelinConfiguration zConf, 
RecoveryStorage recoveryStorage) throws IOException {
+super(zConf, recoveryStorage);
+kubectl = new Kubectl(zConf.getK8sKubectlCmd());
+kubectl.setNamespace(getNamespace());
+  }
+
+  @VisibleForTesting
+  K8sStandardInterpreterLauncher(ZeppelinConfiguration zConf, RecoveryStorage 
recoveryStorage, Kubectl kubectl) {
+super(zConf, recoveryStorage);
+this.kubectl = kubectl;
+  }
+
+
+  /**
+   * Check if i'm running inside of kubernetes or not.
+   * It should return truth regardless of ZeppelinConfiguration.getRunMode().
+   *
+   * Normally, unless Zeppelin is running on Kubernetes, 
K8sStandardInterpreterLauncher shouldn't even have initialized.
+   * However, when ZeppelinConfiguration.getRunMode() is force 'k8s', 
InterpreterSetting.getLauncherPlugin() will try
+   * to use K8sStandardInterpreterLauncher. This is useful for development. It 
allows Zeppelin server running on your
+   * IDE and creates your interpreters in Kubernetes. So any code changes on 
Zeppelin server or kubernetes yaml spec
+   * can be applied without re-building docker image.
+   * @return
+   */
+  boolean isRunningOnKubernetes() {
+if (new File("/var/run/secrets/kubernetes.io").exists()) {
+  return true;
+} else {
+  return false;
+}
+  }
+
+  /**
+   * Get current namespace
+   * @throws IOException
+   */
+  String getNamespace() throws IOException {
+if (isRunningOnKubernetes()) {
+  return 
readFile("/var/run/secrets/kubernetes.io/serviceaccount/namespace", 
Charset.defaultCharset()).trim();
+} else {
+  return "default";
+}
+  }
+
+  /**
+   * Get hostname. It should be the same to Service name (and Pod name) of the 
Kubernetes
+   * @return
+   */
+  String getHostname() {
+try {
+  return InetAddress.getLocalHost().getHostName();
+} catch (UnknownHostException e) {
+  return "localhost";
+}
+  }
+
+  /**
+   * get Zeppelin server host dns.
+   * return ..svc.cluster.local
+   * @throws IOException
+   */
+  private String getZeppelinServiceHost() throws IOException {
+if (isRunningOnKubernetes()) {
+  return String.format("%s.%s.svc.cluster.local",
+  getHostname(), // service name and pod name should be the same
 
 Review comment:
   Good point. Notebook serving (https://github.com/apache/zeppelin/pull/3356) 
has an exact problem while TestTask runs Zeppelin using `Job`, and ServingTask 
runs Zeppelin using `Deployment`.
   
   In serving implementation, K8sStandardInterpreterLauncher has modified to 
looking for `SERVICE_NAME` env variable. Check 
https://github.com/apache/zeppelin/pull/3356/files#diff-dfad7c26921ec97aa7bca7fc3180408fL109.
 Does it make sense?


[GitHub] [zeppelin] Leemoonsoo commented on a change in pull request #3240: [ZEPPELIN-3840] Zeppelin on Kubernetes

2019-05-03 Thread GitBox
Leemoonsoo commented on a change in pull request #3240: [ZEPPELIN-3840] 
Zeppelin on Kubernetes
URL: https://github.com/apache/zeppelin/pull/3240#discussion_r280914489
 
 

 ##
 File path: 
zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
 ##
 @@ -0,0 +1,382 @@
+package org.apache.zeppelin.interpreter.launcher;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.Maps;
+import com.google.gson.Gson;
+import com.google.gson.reflect.TypeToken;
+import java.io.File;
+import java.io.IOException;
+import java.util.*;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.apache.commons.exec.ExecuteWatchdog;
+import org.apache.commons.lang3.ArrayUtils;
+import org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess;
+import org.apache.zeppelin.interpreter.remote.RemoteInterpreterUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class K8sRemoteInterpreterProcess extends RemoteInterpreterProcess {
+  private static final Logger LOGGER = 
LoggerFactory.getLogger(K8sStandardInterpreterLauncher.class);
+  private static final int K8S_INTERPRETER_SERVICE_PORT = 12321;
+  private final Kubectl kubectl;
+  private final String interpreterGroupId;
+  private final String interpreterGroupName;
+  private final String interpreterSettingName;
+  private final File specTempaltes;
+  private final String containerImage;
+  private final Properties properties;
+  private final Map envs;
+  private final String zeppelinServiceHost;
+  private final String zeppelinServiceRpcPort;
+
+  private final Gson gson = new Gson();
+  private final String podName;
+  private final boolean portForward;
+  private final String sparkImage;
+  private ExecuteWatchdog portForwardWatchdog;
+  private int podPort = K8S_INTERPRETER_SERVICE_PORT;
+
+  private AtomicBoolean started = new AtomicBoolean(false);
+
+  public K8sRemoteInterpreterProcess(
+  Kubectl kubectl,
+  File specTemplates,
+  String containerImage,
+  String interpreterGroupId,
+  String interpreterGroupName,
+  String interpreterSettingName,
+  Properties properties,
+  Map envs,
+  String zeppelinServiceHost,
+  String zeppelinServiceRpcPort,
+  boolean portForward,
+  String sparkImage,
+  int connectTimeout
+  ) {
+super(connectTimeout);
+this.kubectl = kubectl;
+this.specTempaltes = specTemplates;
+this.containerImage = containerImage;
+this.interpreterGroupId = interpreterGroupId;
+this.interpreterGroupName = interpreterGroupName;
+this.interpreterSettingName = interpreterSettingName;
+this.properties = properties;
+this.envs = new HashMap(envs);
+this.zeppelinServiceHost = zeppelinServiceHost;
+this.zeppelinServiceRpcPort = zeppelinServiceRpcPort;
+this.portForward = portForward;
+this.sparkImage = sparkImage;
+this.podName = interpreterGroupName.toLowerCase() + "-" + 
getRandomString(6);
+  }
+
+
+  /**
+   * Get interpreter pod name
+   * @return
+   */
+  @VisibleForTesting
+  String getPodName() {
+return podName;
+  }
+
+  @Override
+  public String getInterpreterSettingName() {
+return interpreterSettingName;
+  }
+
+  @Override
+  public void start(String userName) throws IOException {
+// create new pod
+apply(specTempaltes, false);
+kubectl.wait(String.format("pod/%s", getPodName()), "condition=Ready", 
getConnectTimeout()/1000);
+
+if (portForward) {
+  podPort = 
RemoteInterpreterUtils.findRandomAvailablePortOnAllLocalInterfaces();
+  portForwardWatchdog = kubectl.portForward(
+  String.format("pod/%s", getPodName()),
+  new String[] {
+  String.format("%s:%s", podPort, K8S_INTERPRETER_SERVICE_PORT)
+  });
+}
+
+long startTime = System.currentTimeMillis();
+
+// wait until interpreter send started message through thrift rpc
+synchronized (started) {
+  if (!started.get()) {
+try {
+  started.wait(getConnectTimeout());
+} catch (InterruptedException e) {
+  LOGGER.error("Remote interpreter is not accessible");
+}
+  }
+}
+
+if (!started.get()) {
+  LOGGER.info(
+  String.format("Interpreter pod creation is time out in %d seconds",
+  getConnectTimeout()/1000));
+}
+
+// waits for interpreter thrift rpc server ready
+while (System.currentTimeMillis() - startTime < getConnectTimeout()) {
+  if (RemoteInterpreterUtils.checkIfRemoteEndpointAccessible(getHost(), 
getPort())) {
+break;
+  } else {
+try {
+  Thread.sleep(1000);
+} catch (InterruptedException e) {
+}
+  }
+}
+  }
+
+  @Override
+  public void stop() {
+// delete pod
+try {
+  apply(specTempaltes, true);
+} catch (IOException e) {

[GitHub] [zeppelin] chrissng commented on a change in pull request #3240: [ZEPPELIN-3840] Zeppelin on Kubernetes

2019-05-02 Thread GitBox
chrissng commented on a change in pull request #3240: [ZEPPELIN-3840] Zeppelin 
on Kubernetes
URL: https://github.com/apache/zeppelin/pull/3240#discussion_r280657802
 
 

 ##
 File path: 
zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/Kubectl.java
 ##
 @@ -0,0 +1,157 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.interpreter.launcher;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.gson.Gson;
+import java.util.ArrayList;
+import java.util.Arrays;
+
+import org.apache.commons.exec.*;
+import org.apache.commons.io.IOUtils;
+
+import java.io.*;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class Kubectl {
+  private final Logger LOGGER = LoggerFactory.getLogger(Kubectl.class);
+  private final String kubectlCmd;
+  private final Gson gson = new Gson();
+  private String namespace;
+
+  public Kubectl(String kubectlCmd) {
+this.kubectlCmd = kubectlCmd;
+  }
+
+  /**
+   * Override namespace. Otherwise use namespace provided in schema
+   * @param namespace
+   */
+  public void setNamespace(String namespace) {
+this.namespace = namespace;
+  }
+
+  public String getNamespace() {
+return namespace;
+  }
+
+  public String apply(String spec) throws IOException {
+return execAndGet(new String[]{"apply", "-f", "-"}, spec);
+  }
+
+  public String delete(String spec) throws IOException {
+return execAndGet(new String[]{"delete", "-f", "-"}, spec);
+  }
+
+  public String wait(String resource, String waitFor, int timeoutSec) throws 
IOException {
+try {
+  return execAndGet(new String[]{
+  "wait",
+  resource,
+  String.format("--for=%s", waitFor),
+  String.format("--timeout=%ds", timeoutSec)});
+} catch (IOException e) {
+  if ("delete".equals(waitFor) && e.getMessage().contains("NotFound")) {
+LOGGER.info("{} Not found. Maybe already deleted.", resource);
+return "";
+  } else {
+throw e;
+  }
+}
+  }
+
+  public ExecuteWatchdog portForward(String resource, String [] ports) throws 
IOException {
+DefaultExecutor executor = new DefaultExecutor();
+CommandLine cmd = new CommandLine(kubectlCmd);
+cmd.addArguments("port-forward");
+cmd.addArguments(resource);
+cmd.addArguments(ports);
+
+ExecuteWatchdog watchdog = new ExecuteWatchdog(-1);
+executor.setWatchdog(watchdog);
+
+executor.execute(cmd, new ExecuteResultHandler() {
+  @Override
+  public void onProcessComplete(int i) {
+LOGGER.info("Port-forward stopped");
+  }
+
+  @Override
+  public void onProcessFailed(ExecuteException e) {
+LOGGER.debug("port-forward process exit", e);
+  }
+});
+
+return watchdog;
+  }
+
+  String execAndGet(String [] args) throws IOException {
+return execAndGet(args, "");
+  }
+
+  @VisibleForTesting
+  String execAndGet(String [] args, String stdin) throws IOException {
+InputStream ins = IOUtils.toInputStream(stdin);
+ByteArrayOutputStream stdout = new ByteArrayOutputStream();
+ByteArrayOutputStream stderr = new ByteArrayOutputStream();
+ArrayList argsToOverride = new ArrayList<>(Arrays.asList(args));
+
+// set namespace
+if (namespace != null) {
+  argsToOverride.add("--namespace=" + namespace);
+}
+
+LOGGER.info("kubectl " + argsToOverride);
+LOGGER.debug(stdin);
+
+try {
+  int exitCode = execute(
+  argsToOverride.toArray(new String[0]),
+  ins,
+  stdout,
+  stderr
+  );
+
+  if (exitCode == 0) {
+String output = new String(stdout.toByteArray());
+return output;
+  } else {
+String output = new String(stderr.toByteArray());
+throw new IOException(String.format("non zero return code (%d). %s", 
exitCode, output));
+  }
+} catch (Exception e) {
+  String output = new String(stderr.toByteArray());
+  throw new IOException(output, e);
+}
+  }
+
+  public int execute(String [] args, InputStream stdin, OutputStream stdout, 
OutputStream stderr) throws IOException {
+DefaultExecutor 

[GitHub] [zeppelin] chrissng commented on a change in pull request #3240: [ZEPPELIN-3840] Zeppelin on Kubernetes

2019-05-02 Thread GitBox
chrissng commented on a change in pull request #3240: [ZEPPELIN-3840] Zeppelin 
on Kubernetes
URL: https://github.com/apache/zeppelin/pull/3240#discussion_r280651655
 
 

 ##
 File path: 
zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sStandardInterpreterLauncher.java
 ##
 @@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.interpreter.launcher;
+
+import java.io.File;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.nio.charset.Charset;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+
+import com.google.common.annotations.VisibleForTesting;
+import org.apache.zeppelin.conf.ZeppelinConfiguration;
+import org.apache.zeppelin.interpreter.recovery.RecoveryStorage;
+import org.apache.zeppelin.interpreter.remote.RemoteInterpreterUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Interpreter Launcher which use shell script to launch the interpreter 
process.
+ */
+public class K8sStandardInterpreterLauncher extends InterpreterLauncher {
+
+  private static final Logger LOGGER = 
LoggerFactory.getLogger(K8sStandardInterpreterLauncher.class);
+  private final Kubectl kubectl;
+  private InterpreterLaunchContext context;
+
+
+  public K8sStandardInterpreterLauncher(ZeppelinConfiguration zConf, 
RecoveryStorage recoveryStorage) throws IOException {
+super(zConf, recoveryStorage);
+kubectl = new Kubectl(zConf.getK8sKubectlCmd());
+kubectl.setNamespace(getNamespace());
+  }
+
+  @VisibleForTesting
+  K8sStandardInterpreterLauncher(ZeppelinConfiguration zConf, RecoveryStorage 
recoveryStorage, Kubectl kubectl) {
+super(zConf, recoveryStorage);
+this.kubectl = kubectl;
+  }
+
+
+  /**
+   * Check if i'm running inside of kubernetes or not.
+   * It should return truth regardless of ZeppelinConfiguration.getRunMode().
+   *
+   * Normally, unless Zeppelin is running on Kubernetes, 
K8sStandardInterpreterLauncher shouldn't even have initialized.
+   * However, when ZeppelinConfiguration.getRunMode() is force 'k8s', 
InterpreterSetting.getLauncherPlugin() will try
+   * to use K8sStandardInterpreterLauncher. This is useful for development. It 
allows Zeppelin server running on your
+   * IDE and creates your interpreters in Kubernetes. So any code changes on 
Zeppelin server or kubernetes yaml spec
+   * can be applied without re-building docker image.
+   * @return
+   */
+  boolean isRunningOnKubernetes() {
+if (new File("/var/run/secrets/kubernetes.io").exists()) {
+  return true;
+} else {
+  return false;
+}
+  }
+
+  /**
+   * Get current namespace
+   * @throws IOException
+   */
+  String getNamespace() throws IOException {
+if (isRunningOnKubernetes()) {
+  return 
readFile("/var/run/secrets/kubernetes.io/serviceaccount/namespace", 
Charset.defaultCharset()).trim();
+} else {
+  return "default";
+}
+  }
+
+  /**
+   * Get hostname. It should be the same to Service name (and Pod name) of the 
Kubernetes
+   * @return
+   */
+  String getHostname() {
+try {
+  return InetAddress.getLocalHost().getHostName();
+} catch (UnknownHostException e) {
+  return "localhost";
+}
+  }
+
+  /**
+   * get Zeppelin server host dns.
+   * return ..svc.cluster.local
+   * @throws IOException
+   */
+  private String getZeppelinServiceHost() throws IOException {
+if (isRunningOnKubernetes()) {
+  return String.format("%s.%s.svc.cluster.local",
+  getHostname(), // service name and pod name should be the same
 
 Review comment:
   @Leemoonsoo Great work on making Zeppelin work in Kubernetes!
   
   I have a concern with this approach of matching Service and Pod names via 
hostname. This will not work for zeppelin pods that are provisioned via 
Deployments or StatefulSets as pod names are dynamically generated during 
creation.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to 

[GitHub] [zeppelin] chrissng commented on a change in pull request #3240: [ZEPPELIN-3840] Zeppelin on Kubernetes

2019-05-02 Thread GitBox
chrissng commented on a change in pull request #3240: [ZEPPELIN-3840] Zeppelin 
on Kubernetes
URL: https://github.com/apache/zeppelin/pull/3240#discussion_r280651243
 
 

 ##
 File path: 
zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
 ##
 @@ -0,0 +1,382 @@
+package org.apache.zeppelin.interpreter.launcher;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.Maps;
+import com.google.gson.Gson;
+import com.google.gson.reflect.TypeToken;
+import java.io.File;
+import java.io.IOException;
+import java.util.*;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.apache.commons.exec.ExecuteWatchdog;
+import org.apache.commons.lang3.ArrayUtils;
+import org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess;
+import org.apache.zeppelin.interpreter.remote.RemoteInterpreterUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class K8sRemoteInterpreterProcess extends RemoteInterpreterProcess {
+  private static final Logger LOGGER = 
LoggerFactory.getLogger(K8sStandardInterpreterLauncher.class);
+  private static final int K8S_INTERPRETER_SERVICE_PORT = 12321;
+  private final Kubectl kubectl;
+  private final String interpreterGroupId;
+  private final String interpreterGroupName;
+  private final String interpreterSettingName;
+  private final File specTempaltes;
+  private final String containerImage;
+  private final Properties properties;
+  private final Map envs;
+  private final String zeppelinServiceHost;
+  private final String zeppelinServiceRpcPort;
+
+  private final Gson gson = new Gson();
+  private final String podName;
+  private final boolean portForward;
+  private final String sparkImage;
+  private ExecuteWatchdog portForwardWatchdog;
+  private int podPort = K8S_INTERPRETER_SERVICE_PORT;
+
+  private AtomicBoolean started = new AtomicBoolean(false);
+
+  public K8sRemoteInterpreterProcess(
+  Kubectl kubectl,
+  File specTemplates,
+  String containerImage,
+  String interpreterGroupId,
+  String interpreterGroupName,
+  String interpreterSettingName,
+  Properties properties,
+  Map envs,
+  String zeppelinServiceHost,
+  String zeppelinServiceRpcPort,
+  boolean portForward,
+  String sparkImage,
+  int connectTimeout
+  ) {
+super(connectTimeout);
+this.kubectl = kubectl;
+this.specTempaltes = specTemplates;
+this.containerImage = containerImage;
+this.interpreterGroupId = interpreterGroupId;
+this.interpreterGroupName = interpreterGroupName;
+this.interpreterSettingName = interpreterSettingName;
+this.properties = properties;
+this.envs = new HashMap(envs);
+this.zeppelinServiceHost = zeppelinServiceHost;
+this.zeppelinServiceRpcPort = zeppelinServiceRpcPort;
+this.portForward = portForward;
+this.sparkImage = sparkImage;
+this.podName = interpreterGroupName.toLowerCase() + "-" + 
getRandomString(6);
+  }
+
+
+  /**
+   * Get interpreter pod name
+   * @return
+   */
+  @VisibleForTesting
+  String getPodName() {
+return podName;
+  }
+
+  @Override
+  public String getInterpreterSettingName() {
+return interpreterSettingName;
+  }
+
+  @Override
+  public void start(String userName) throws IOException {
+// create new pod
+apply(specTempaltes, false);
+kubectl.wait(String.format("pod/%s", getPodName()), "condition=Ready", 
getConnectTimeout()/1000);
+
+if (portForward) {
+  podPort = 
RemoteInterpreterUtils.findRandomAvailablePortOnAllLocalInterfaces();
+  portForwardWatchdog = kubectl.portForward(
+  String.format("pod/%s", getPodName()),
+  new String[] {
+  String.format("%s:%s", podPort, K8S_INTERPRETER_SERVICE_PORT)
+  });
+}
+
+long startTime = System.currentTimeMillis();
+
+// wait until interpreter send started message through thrift rpc
+synchronized (started) {
+  if (!started.get()) {
+try {
+  started.wait(getConnectTimeout());
+} catch (InterruptedException e) {
+  LOGGER.error("Remote interpreter is not accessible");
+}
+  }
+}
+
+if (!started.get()) {
+  LOGGER.info(
+  String.format("Interpreter pod creation is time out in %d seconds",
+  getConnectTimeout()/1000));
+}
+
+// waits for interpreter thrift rpc server ready
+while (System.currentTimeMillis() - startTime < getConnectTimeout()) {
+  if (RemoteInterpreterUtils.checkIfRemoteEndpointAccessible(getHost(), 
getPort())) {
+break;
+  } else {
+try {
+  Thread.sleep(1000);
+} catch (InterruptedException e) {
+}
+  }
+}
+  }
+
+  @Override
+  public void stop() {
+// delete pod
+try {
+  apply(specTempaltes, true);
+} catch (IOException e) {
+ 

[GitHub] [zeppelin] zjffdu commented on issue #3353: [ZEPPELIN-4132]. Spark Interpreter has issue of SPARK-22393

2019-05-02 Thread GitBox
zjffdu commented on issue #3353: [ZEPPELIN-4132]. Spark Interpreter has issue 
of SPARK-22393
URL: https://github.com/apache/zeppelin/pull/3353#issuecomment-488673631
 
 
   Merge if no more comment


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] pmuntyanu commented on issue #2373: [ZEPPELIN-2598] Securing Zeppelin with OpenID Connect

2019-05-02 Thread GitBox
pmuntyanu commented on issue #2373: [ZEPPELIN-2598] Securing Zeppelin with 
OpenID Connect
URL: https://github.com/apache/zeppelin/pull/2373#issuecomment-488583737
 
 
   Hi @ajaygk95 , it worked! I even changed the `preferred_username` to the 
`email`  - for my use case, it is better. With regards to the roles and 
permissions, I think it is possible to use `keycloak` roles and 
[RequireAnyRoleAuthorizer](https://github.com/pac4j/pac4j/blob/15419a42a572f67c97b2978911fe8ec66dadf0c7/pac4j-core/src/main/java/org/pac4j/core/authorization/authorizer/RequireAnyRoleAuthorizer.java)
 which we can setup via java-bean-like shiro.ini file. In the end, it is just 
an idea and I have not tested it yet, but it looks like it can work. More 
details are [here](https://github.com/bujiio/buji-pac4j/wiki/Apply-security) 
and example is 
[here](https://github.com/bujiio/buji-pac4j/wiki/Security-configuration)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] ajaygk95 edited a comment on issue #2373: [ZEPPELIN-2598] Securing Zeppelin with OpenID Connect

2019-05-01 Thread GitBox
ajaygk95 edited a comment on issue #2373: [ZEPPELIN-2598] Securing Zeppelin 
with OpenID Connect
URL: https://github.com/apache/zeppelin/pull/2373#issuecomment-488297287
 
 
   > 
   > 
   > @ajaygk95 , thanks for sharing - it worked for us! There are two not 
resolved questions still:
   > 
   > * is it possible to define roles and users and have different 
permissions by using OpenID and Keycloak?
   > 
   > * how to get real username in zeppelin in the top right corner instead 
of id from Keycloak?
   
   Hi @pmuntyanu ,
   I am still figuring out the authorization part. No clue on this yet
   For the username issue you can use `pac4jRealm.principalNameAttribute = 
preferred_username` under pac4jRealm section in your shiro.ini. This will 
display the actual username in your zeppelin UI.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] ajaygk95 commented on issue #2373: [ZEPPELIN-2598] Securing Zeppelin with OpenID Connect

2019-05-01 Thread GitBox
ajaygk95 commented on issue #2373: [ZEPPELIN-2598] Securing Zeppelin with 
OpenID Connect
URL: https://github.com/apache/zeppelin/pull/2373#issuecomment-488297287
 
 
   > 
   > 
   > @ajaygk95 , thanks for sharing - it worked for us! There are two not 
resolved questions still:
   > 
   > * is it possible to define roles and users and have different 
permissions by using OpenID and Keycloak?
   > 
   > * how to get real username in zeppelin in the top right corner instead 
of id from Keycloak?
   
   Hi @pmuntyanu ,
   I am also still figuring out the authorization part of user and role 
mapping. 
   For the username issue you can use `pac4jRealm.principalNameAttribute = 
preferred_username` under pac4jRealm section in your shiro.ini. This will 
display the actual username in your zeppelin UI.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] pmuntyanu commented on issue #2373: [ZEPPELIN-2598] Securing Zeppelin with OpenID Connect

2019-05-01 Thread GitBox
pmuntyanu commented on issue #2373: [ZEPPELIN-2598] Securing Zeppelin with 
OpenID Connect
URL: https://github.com/apache/zeppelin/pull/2373#issuecomment-488276472
 
 
   @ajaygk95 , thanks for sharing -  it worked for us! There are two not 
resolved questions still:
   * is it possible to define roles and users and have different permissions by 
using OpenID and Keycloak?
   * how to get real username in zeppelin in the top right corner instead of id 
from Keycloak?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] sirianni closed pull request #3280: [MINOR][ZEPPELIN-1919] Reduce font size in markdown output

2019-04-30 Thread GitBox
sirianni closed pull request #3280: [MINOR][ZEPPELIN-1919] Reduce font size in 
markdown output
URL: https://github.com/apache/zeppelin/pull/3280
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] RussellSpitzer closed pull request #3357: [ZEPPELIN-4136] -Use a match expression to determine SparkR Security

2019-04-29 Thread GitBox
RussellSpitzer closed pull request #3357: [ZEPPELIN-4136] -Use a match 
expression to determine SparkR Security
URL: https://github.com/apache/zeppelin/pull/3357
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] RussellSpitzer commented on issue #3357: [ZEPPELIN-4136] -Use a match expression to determine SparkR Security

2019-04-29 Thread GitBox
RussellSpitzer commented on issue #3357: [ZEPPELIN-4136] -Use a match 
expression to determine SparkR Security
URL: https://github.com/apache/zeppelin/pull/3357#issuecomment-487690798
 
 
   I'm really not liking what i'm doing here. I think i'm going to try to just 
push the version number in our distribution.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] RussellSpitzer edited a comment on issue #3357: [ZEPPELIN-4136] -Use a match expression to determine SparkR Security

2019-04-29 Thread GitBox
RussellSpitzer edited a comment on issue #3357: [ZEPPELIN-4136] -Use a match 
expression to determine SparkR Security
URL: https://github.com/apache/zeppelin/pull/3357#issuecomment-487671905
 
 
   @felixcheung I read the code wrong, to avoid the version check for Python I 
have it check whether py4j supports the auth token method.
   
   I don't really like this solution ... but I can't think of another


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] RussellSpitzer commented on issue #3357: [ZEPPELIN-4136] -Use a match expression to determine SparkR Security

2019-04-29 Thread GitBox
RussellSpitzer commented on issue #3357: [ZEPPELIN-4136] -Use a match 
expression to determine SparkR Security
URL: https://github.com/apache/zeppelin/pull/3357#issuecomment-487671905
 
 
   @felixcheung I read the code wrong, to avoid the version check for Python I 
have it check whether py4j supports the auth token method.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] RussellSpitzer commented on issue #3357: [ZEPPELIN-4136] -Use a match expression to determine SparkR Security

2019-04-29 Thread GitBox
RussellSpitzer commented on issue #3357: [ZEPPELIN-4136] -Use a match 
expression to determine SparkR Security
URL: https://github.com/apache/zeppelin/pull/3357#issuecomment-487648939
 
 
   Not sure why i'm having such pain with this type matching :/


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] RussellSpitzer commented on issue #3357: [ZEPPELIN-4136] -Use a match expression to determine SparkR Security

2019-04-29 Thread GitBox
RussellSpitzer commented on issue #3357: [ZEPPELIN-4136] -Use a match 
expression to determine SparkR Security
URL: https://github.com/apache/zeppelin/pull/3357#issuecomment-487645040
 
 
   @felixcheung From what I can tell, pyspark just assumes that the secret is 
always passed 
https://github.com/apache/zeppelin/blob/8fa534343beccb797f9f1850ec43673ac830b73e/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java#L121


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] felixcheung commented on issue #3357: [ZEPPELIN-4136] -Use a match expression to determine SparkR Security

2019-04-29 Thread GitBox
felixcheung commented on issue #3357: [ZEPPELIN-4136] -Use a match expression 
to determine SparkR Security
URL: https://github.com/apache/zeppelin/pull/3357#issuecomment-487642797
 
 
   this is the same for pyspark, right?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] RussellSpitzer commented on issue #3357: [ZEPPELIN-4136] -Use a match expression to determine SparkR Security

2019-04-29 Thread GitBox
RussellSpitzer commented on issue #3357: [ZEPPELIN-4136] -Use a match 
expression to determine SparkR Security
URL: https://github.com/apache/zeppelin/pull/3357#issuecomment-487625794
 
 
   Of course
   
   On Mon, Apr 29, 2019 at 10:25 AM Jeff Zhang 
   wrote:
   
   > Thanks for the contribution @RussellSpitzer
   >  , could you remove
   > SparkVersion#isSecretSocketSupported ?
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > , or 
mute
   > the thread
   > 

   > .
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on issue #3357: Use a match expression to determine SparkR Security

2019-04-29 Thread GitBox
zjffdu commented on issue #3357: Use a match expression to determine SparkR 
Security
URL: https://github.com/apache/zeppelin/pull/3357#issuecomment-487624893
 
 
   Thanks for the contribution @RussellSpitzer , could you remove 
SparkVersion#isSecretSocketSupported ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] RussellSpitzer opened a new pull request #3357: Use a match expression to determine SparkR Security

2019-04-29 Thread GitBox
RussellSpitzer opened a new pull request #3357: Use a match expression to 
determine SparkR Security
URL: https://github.com/apache/zeppelin/pull/3357
 
 
   
   ### What is this PR for?
   Previously the code dependend on hard coded Spark Versions, this made
   some distributions of Spark incompatible because they included security
   backports not listed in the official distribution versions. This instead
   checks the class itself to determine whether or not the new Secret
   method should be utilized
   
   
   ### What type of PR is it?
   [Bug Fix | Refactoring]
   
   ### Todos
   Change from version check to Match
   
   ### What is the Jira issue?
   https://issues.apache.org/jira/browse/ZEPPELIN-4136
   
   ### How should this be tested?
   Gonna try out CI
   
   ### Screenshots (if appropriate)
   
   ### Questions:
   * Does the licenses files need update? Nope
   * Is there breaking changes for older versions? Nope
   * Does this needs documentation? Nope
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] jongyoul commented on issue #3297: [ZEPPELIN-3671] Add info about running interpreters to API and JMX

2019-04-28 Thread GitBox
jongyoul commented on issue #3297: [ZEPPELIN-3671] Add info about running 
interpreters to API and JMX
URL: https://github.com/apache/zeppelin/pull/3297#issuecomment-487431789
 
 
   Changing name left only.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] asfgit closed pull request #3355: [ZEPPELIN-4133]. Idle sessions are no longer being closed even though TimeoutLifecycleManagement is configured properly

2019-04-28 Thread GitBox
asfgit closed pull request #3355: [ZEPPELIN-4133]. Idle sessions are no longer 
being closed even though TimeoutLifecycleManagement is configured properly
URL: https://github.com/apache/zeppelin/pull/3355
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on issue #3355: [ZEPPELIN-4133]. Idle sessions are no longer being closed even though TimeoutLifecycleManagement is configured properly

2019-04-27 Thread GitBox
zjffdu commented on issue #3355: [ZEPPELIN-4133]. Idle sessions are no longer 
being closed even though TimeoutLifecycleManagement is configured properly
URL: https://github.com/apache/zeppelin/pull/3355#issuecomment-487332034
 
 
   Will merge if no more comments


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu removed a comment on issue #3034: ZEPPELIN-3552. Support Scala 2.12 of SparkInterpreter

2019-04-27 Thread GitBox
zjffdu removed a comment on issue #3034: ZEPPELIN-3552. Support Scala 2.12 of 
SparkInterpreter
URL: https://github.com/apache/zeppelin/pull/3034#issuecomment-484813441
 
 
   @conker84 I make some updates to this PR, and it basically works in my local 
environment. But I still need more time to refine this PR. I suppose this could 
be done in 0.9.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] felixcheung commented on issue #3355: [ZEPPELIN-4133]. Idle sessions are no longer being closed even though TimeoutLifecycleManagement is configured properly

2019-04-26 Thread GitBox
felixcheung commented on issue #3355: [ZEPPELIN-4133]. Idle sessions are no 
longer being closed even though TimeoutLifecycleManagement is configured 
properly
URL: https://github.com/apache/zeppelin/pull/3355#issuecomment-487258005
 
 
   LG


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] mephi42 closed pull request #2816: [ZEPPELIN-3263] Do not use Firefox >=55 for integration testing

2019-04-26 Thread GitBox
mephi42 closed pull request #2816: [ZEPPELIN-3263] Do not use Firefox >=55 for 
integration testing
URL: https://github.com/apache/zeppelin/pull/2816
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] mephi42 closed pull request #2815: [ZEPPELIN-3262] Make install_external_dependencies.sh work on macOS

2019-04-26 Thread GitBox
mephi42 closed pull request #2815: [ZEPPELIN-3262] Make 
install_external_dependencies.sh work on macOS
URL: https://github.com/apache/zeppelin/pull/2815
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] Leemoonsoo opened a new pull request #3356: [ZEPPELIN-3994] Notebook serving

2019-04-25 Thread GitBox
Leemoonsoo opened a new pull request #3356: [ZEPPELIN-3994] Notebook serving
URL: https://github.com/apache/zeppelin/pull/3356
 
 
   ### What is this PR for?
   This PR implements [notebook 
serving](https://issues.apache.org/jira/browse/ZEPPELIN-3994) explained 
[here](https://docs.google.com/document/d/1YA6q8W9yO8a88xzLDYs9zv_fKu2_cnB58rmQbakxi1I/edit?usp=sharing).
   
   While this PR is already quite large, front-end implementation will have 
separate PR.
   This PR is work in progress.
   
   ### How to run 
   
   1. Prepare kubernetes cluster
   2. Prepare persistentVolumeClaim with name 'task-context-volume-claim' in 
your kubernetes cluster
   3. Build Zeppelin docker image (follow instructions 
[here](https://github.com/apache/zeppelin/blob/master/docs/quickstart/kubernetes.md)
   4. Deploy api router by running `kubectl apply -f 
k8s/serving-api-router/serving-api-router.yaml`
   5. Start Zeppelin on Kubernetes by running `kubectl apply -f 
k8s/zeppelin-server.yaml`
   6. Port forward to the api router (e.g. `kubectl port-forward 
 8080:80`) and browse `localhost:8080`
   
   ### Summary of changes
   To be updated
   
   ### What type of PR is it?
   Feature
   
   ### Todos
   
   ### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3994
   
   ### How should this be tested?
   * First time? Setup Travis CI as described on 
https://zeppelin.apache.org/contribution/contributions.html#continuous-integration
   * Strongly recommended: add automated unit tests for any new or changed 
behavior
   * Outline any manual steps to test the PR here.
   
   ### Questions:
   * Does the licenses files need update? yes
   * Is there breaking changes for older versions? no
   * Does this needs documentation? yes
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on a change in pull request #3353: [ZEPPELIN-4132]. Spark Interpreter has issue of SPARK-22393

2019-04-25 Thread GitBox
zjffdu commented on a change in pull request #3353: [ZEPPELIN-4132]. Spark 
Interpreter has issue of SPARK-22393
URL: https://github.com/apache/zeppelin/pull/3353#discussion_r278430016
 
 

 ##
 File path: 
spark/scala-2.11/src/main/scala/org/apache/zeppelin/spark/SparkScala211Interpreter.scala
 ##
 @@ -43,7 +44,7 @@ class SparkScala211Interpreter(override val conf: SparkConf,
 
   lazy override val LOGGER: Logger = LoggerFactory.getLogger(getClass)
 
-  private var sparkILoop: ILoop = _
+  private var sparkILoop: SparkILoop = _
 
 Review comment:
   Scala210Interpreter already did that


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [zeppelin] zjffdu commented on a change in pull request #3355: [ZEPPELIN-4133]. Idle sessions are no longer being closed even though TimeoutLifecycleManagement is configured properly

2019-04-25 Thread GitBox
zjffdu commented on a change in pull request #3355: [ZEPPELIN-4133]. Idle 
sessions are no longer being closed even though TimeoutLifecycleManagement is 
configured properly
URL: https://github.com/apache/zeppelin/pull/3355#discussion_r278429581
 
 

 ##
 File path: 
zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/lifecycle/TimeoutLifecycleManager.java
 ##
 @@ -30,29 +32,30 @@
   private long checkInterval;
   private long timeoutThreshold;
 
-  private Timer checkTimer;
+  private ScheduledExecutorService checkScheduler;
 
   public TimeoutLifecycleManager(ZeppelinConfiguration zConf) {
 this.checkInterval = zConf.getLong(ZeppelinConfiguration.ConfVars
 .ZEPPELIN_INTERPRETER_LIFECYCLE_MANAGER_TIMEOUT_CHECK_INTERVAL);
 this.timeoutThreshold = zConf.getLong(
 
ZeppelinConfiguration.ConfVars.ZEPPELIN_INTERPRETER_LIFECYCLE_MANAGER_TIMEOUT_THRESHOLD);
-this.checkTimer = new Timer(true);
-this.checkTimer.scheduleAtFixedRate(new TimerTask() {
-  @Override
-  public void run() {
+this.checkScheduler = Executors.newScheduledThreadPool(1);
+this.checkScheduler.scheduleAtFixedRate(() -> {
+  try {
 long now = System.currentTimeMillis();
 for (Map.Entry entry : 
interpreterGroups.entrySet()) {
   ManagedInterpreterGroup interpreterGroup = entry.getKey();
   Long lastTimeUsing = entry.getValue();
-  if ((now - lastTimeUsing) > timeoutThreshold )  {
+  if ((now - lastTimeUsing) > timeoutThreshold) {
 LOGGER.info("InterpreterGroup {} is timeout.", 
interpreterGroup.getId());
 interpreterGroup.close();
 interpreterGroups.remove(entry.getKey());
   }
 }
+  } catch (Exception e) {
+LOGGER.warn("Fail to run periodical checking task", e);
 
 Review comment:
   Make sense to me. I updated it. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


  1   2   3   4   5   6   7   8   9   10   >