[GitHub] [incubator-livy] yiheng commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift serv
yiheng commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r314167215
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/operation/SparkCatalogOperation.scala
##
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.thriftserver.operation
+
+import org.apache.commons.lang.StringUtils
+import org.apache.hive.service.cli._
+
+import org.apache.livy.thriftserver.LivyThriftSessionManager
+import org.apache.livy.thriftserver.serde.ThriftResultSet
+import org.apache.livy.thriftserver.session.{CleanupCatalogResultJob,
FetchCatalogResultJob}
+
+/**
+ * SparkCatalogOperation is the base class for operations which need to fetch
catalog information
+ * from spark session.
+ */
+abstract class SparkCatalogOperation(
+sessionHandle: SessionHandle,
+opType: OperationType,
+sessionManager: LivyThriftSessionManager)
+ extends Operation(sessionHandle, opType) {
+
+ // The initialization need to be lazy in order not to block when the
instance is created
+ protected lazy val rscClient = {
+// This call is blocking, we are waiting for the session to be ready.
+sessionManager.getLivySession(sessionHandle).client.get
Review comment:
From this
[code](https://github.com/apache/incubator-livy/blob/80daadef02ae57b2a5487c6f92e0f7df558d4864/thriftserver/server/src/main/scala/org/apache/livy/thriftserver/LivyThriftSessionManager.scala#L94),
it seems if we cannot get a session, an error will be thrown in livy session
manager.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao closed pull request #192: [LIVY-620] Spark batch session always ends with success when configuration is master yarn and deploy-mode client
jerryshao closed pull request #192: [LIVY-620] Spark batch session always ends with success when configuration is master yarn and deploy-mode client URL: https://github.com/apache/incubator-livy/pull/192 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on issue #192: [LIVY-620] Spark batch session always ends with success when configuration is master yarn and deploy-mode client
jerryshao commented on issue #192: [LIVY-620] Spark batch session always ends with success when configuration is master yarn and deploy-mode client URL: https://github.com/apache/incubator-livy/pull/192#issuecomment-521503480 LGTM, merging to master branch, thanks for the contribution. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] codecov-io commented on issue #192: [LIVY-620] Spark batch session always ends with success when configuration is master yarn and deploy-mode client
codecov-io commented on issue #192: [LIVY-620] Spark batch session always ends with success when configuration is master yarn and deploy-mode client URL: https://github.com/apache/incubator-livy/pull/192#issuecomment-521501836 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/192?src=pr&el=h1) Report > Merging [#192](https://codecov.io/gh/apache/incubator-livy/pull/192?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/e7f23e06606ff79aec1c22e79a96f959cb89a8be?src=pr&el=desc) will **increase** coverage by `40.46%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-livy/pull/192?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #192 +/- ## = + Coverage 28.33% 68.79% +40.46% - Complexity 343 914 +571 = Files 100 100 Lines 5679 5682+3 Branches855 856+1 = + Hits 1609 3909 +2300 + Misses 3739 1215 -2524 - Partials331 558 +227 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/192?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...la/org/apache/livy/server/batch/BatchSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/192/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uLnNjYWxh) | `86.59% <100%> (+2.55%)` | `16 <0> (+2)` | :arrow_up: | | [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/192/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `35.96% <0%> (+0.98%)` | `11% <0%> (ø)` | :arrow_down: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/192/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `95.87% <0%> (+1.03%)` | `21% <0%> (+3%)` | :arrow_up: | | [.../main/scala/org/apache/livy/server/WebServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/192/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvV2ViU2VydmVyLnNjYWxh) | `53.33% <0%> (+1.66%)` | `10% <0%> (+1%)` | :arrow_up: | | [...la/org/apache/livy/utils/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-livy/pull/192/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9TcGFya1Byb2Nlc3NCdWlsZGVyLnNjYWxh) | `54.44% <0%> (+2.22%)` | `11% <0%> (+1%)` | :arrow_up: | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/192/diff?src=pr&el=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (+2.77%)` | `2% <0%> (ø)` | :arrow_down: | | [...e/livy/server/interactive/InteractiveSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/192/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvaW50ZXJhY3RpdmUvSW50ZXJhY3RpdmVTZXNzaW9uLnNjYWxh) | `69.11% <0%> (+3.66%)` | `44% <0%> (+2%)` | :arrow_up: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/192/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <0%> (+5%)` | `10% <0%> (ø)` | :arrow_down: | | [...ain/scala/org/apache/livy/server/JsonServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/192/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvSnNvblNlcnZsZXQuc2NhbGE=) | `38.46% <0%> (+5.76%)` | `18% <0%> (+4%)` | :arrow_up: | | ... and [71 more](https://codecov.io/gh/apache/incubator-livy/pull/192/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/192?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/192?src=pr&el=footer). Last update [e7f23e0...98577dd](https://codecov.io/gh/apache/incubator-livy/pull/192?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). --
[GitHub] [incubator-livy] jerryshao commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift s
jerryshao commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r314163812
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/operation/SparkCatalogOperation.scala
##
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.thriftserver.operation
+
+import org.apache.commons.lang.StringUtils
+import org.apache.hive.service.cli._
+
+import org.apache.livy.thriftserver.LivyThriftSessionManager
+import org.apache.livy.thriftserver.serde.ThriftResultSet
+import org.apache.livy.thriftserver.session.{CleanupCatalogResultJob,
FetchCatalogResultJob}
+
+/**
+ * SparkCatalogOperation is the base class for operations which need to fetch
catalog information
+ * from spark session.
+ */
+abstract class SparkCatalogOperation(
+sessionHandle: SessionHandle,
+opType: OperationType,
+sessionManager: LivyThriftSessionManager)
+ extends Operation(sessionHandle, opType) {
+
+ // The initialization need to be lazy in order not to block when the
instance is created
+ protected lazy val rscClient = {
+// This call is blocking, we are waiting for the session to be ready.
+sessionManager.getLivySession(sessionHandle).client.get
Review comment:
Shall we check if client is not null, `require(client != null)`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift server
codecov-io edited a comment on issue #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift server URL: https://github.com/apache/incubator-livy/pull/194#issuecomment-521240478 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=h1) Report > Merging [#194](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/e7f23e06606ff79aec1c22e79a96f959cb89a8be?src=pr&el=desc) will **increase** coverage by `40.28%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #194 +/- ## = + Coverage 28.33% 68.62% +40.28% - Complexity 343 912 +569 = Files 100 100 Lines 5679 5679 Branches855 855 = + Hits 1609 3897 +2288 + Misses 3739 1224 -2515 - Partials331 558 +227 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `35.96% <0%> (+0.98%)` | `11% <0%> (ø)` | :arrow_down: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `95.87% <0%> (+1.03%)` | `21% <0%> (+3%)` | :arrow_up: | | [.../main/scala/org/apache/livy/server/WebServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvV2ViU2VydmVyLnNjYWxh) | `53.33% <0%> (+1.66%)` | `10% <0%> (+1%)` | :arrow_up: | | [...la/org/apache/livy/server/batch/BatchSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uLnNjYWxh) | `86.17% <0%> (+2.12%)` | `14% <0%> (ø)` | :arrow_down: | | [...la/org/apache/livy/utils/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9TcGFya1Byb2Nlc3NCdWlsZGVyLnNjYWxh) | `54.44% <0%> (+2.22%)` | `11% <0%> (+1%)` | :arrow_up: | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (+2.77%)` | `2% <0%> (ø)` | :arrow_down: | | [...e/livy/server/interactive/InteractiveSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvaW50ZXJhY3RpdmUvSW50ZXJhY3RpdmVTZXNzaW9uLnNjYWxh) | `69.11% <0%> (+3.66%)` | `44% <0%> (+2%)` | :arrow_up: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <0%> (+5%)` | `10% <0%> (ø)` | :arrow_down: | | [...ain/scala/org/apache/livy/server/JsonServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvSnNvblNlcnZsZXQuc2NhbGE=) | `38.46% <0%> (+5.76%)` | `18% <0%> (+4%)` | :arrow_up: | | [.../apache/livy/server/batch/CreateBatchRequest.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQ3JlYXRlQmF0Y2hSZXF1ZXN0LnNjYWxh) | `68.75% <0%> (+6.25%)` | `19% <0%> (+1%)` | :arrow_up: | | ... and [70 more](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codeco
[GitHub] [incubator-livy] codecov-io edited a comment on issue #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift server
codecov-io edited a comment on issue #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift server URL: https://github.com/apache/incubator-livy/pull/194#issuecomment-521240478 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=h1) Report > Merging [#194](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/e7f23e06606ff79aec1c22e79a96f959cb89a8be?src=pr&el=desc) will **increase** coverage by `40.28%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #194 +/- ## = + Coverage 28.33% 68.62% +40.28% - Complexity 343 912 +569 = Files 100 100 Lines 5679 5679 Branches855 855 = + Hits 1609 3897 +2288 + Misses 3739 1224 -2515 - Partials331 558 +227 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `35.96% <0%> (+0.98%)` | `11% <0%> (ø)` | :arrow_down: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `95.87% <0%> (+1.03%)` | `21% <0%> (+3%)` | :arrow_up: | | [.../main/scala/org/apache/livy/server/WebServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvV2ViU2VydmVyLnNjYWxh) | `53.33% <0%> (+1.66%)` | `10% <0%> (+1%)` | :arrow_up: | | [...la/org/apache/livy/server/batch/BatchSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uLnNjYWxh) | `86.17% <0%> (+2.12%)` | `14% <0%> (ø)` | :arrow_down: | | [...la/org/apache/livy/utils/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9TcGFya1Byb2Nlc3NCdWlsZGVyLnNjYWxh) | `54.44% <0%> (+2.22%)` | `11% <0%> (+1%)` | :arrow_up: | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (+2.77%)` | `2% <0%> (ø)` | :arrow_down: | | [...e/livy/server/interactive/InteractiveSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvaW50ZXJhY3RpdmUvSW50ZXJhY3RpdmVTZXNzaW9uLnNjYWxh) | `69.11% <0%> (+3.66%)` | `44% <0%> (+2%)` | :arrow_up: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <0%> (+5%)` | `10% <0%> (ø)` | :arrow_down: | | [...ain/scala/org/apache/livy/server/JsonServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvSnNvblNlcnZsZXQuc2NhbGE=) | `38.46% <0%> (+5.76%)` | `18% <0%> (+4%)` | :arrow_up: | | [.../apache/livy/server/batch/CreateBatchRequest.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQ3JlYXRlQmF0Y2hSZXF1ZXN0LnNjYWxh) | `68.75% <0%> (+6.25%)` | `19% <0%> (+1%)` | :arrow_up: | | ... and [70 more](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codeco
[GitHub] [incubator-livy] codecov-io edited a comment on issue #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift server
codecov-io edited a comment on issue #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift server URL: https://github.com/apache/incubator-livy/pull/194#issuecomment-521240478 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=h1) Report > Merging [#194](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/e7f23e06606ff79aec1c22e79a96f959cb89a8be?src=pr&el=desc) will **increase** coverage by `40.28%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #194 +/- ## = + Coverage 28.33% 68.62% +40.28% - Complexity 343 912 +569 = Files 100 100 Lines 5679 5679 Branches855 855 = + Hits 1609 3897 +2288 + Misses 3739 1224 -2515 - Partials331 558 +227 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `35.96% <0%> (+0.98%)` | `11% <0%> (ø)` | :arrow_down: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `95.87% <0%> (+1.03%)` | `21% <0%> (+3%)` | :arrow_up: | | [.../main/scala/org/apache/livy/server/WebServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvV2ViU2VydmVyLnNjYWxh) | `53.33% <0%> (+1.66%)` | `10% <0%> (+1%)` | :arrow_up: | | [...la/org/apache/livy/server/batch/BatchSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uLnNjYWxh) | `86.17% <0%> (+2.12%)` | `14% <0%> (ø)` | :arrow_down: | | [...la/org/apache/livy/utils/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9TcGFya1Byb2Nlc3NCdWlsZGVyLnNjYWxh) | `54.44% <0%> (+2.22%)` | `11% <0%> (+1%)` | :arrow_up: | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (+2.77%)` | `2% <0%> (ø)` | :arrow_down: | | [...e/livy/server/interactive/InteractiveSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvaW50ZXJhY3RpdmUvSW50ZXJhY3RpdmVTZXNzaW9uLnNjYWxh) | `69.11% <0%> (+3.66%)` | `44% <0%> (+2%)` | :arrow_up: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <0%> (+5%)` | `10% <0%> (ø)` | :arrow_down: | | [...ain/scala/org/apache/livy/server/JsonServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvSnNvblNlcnZsZXQuc2NhbGE=) | `38.46% <0%> (+5.76%)` | `18% <0%> (+4%)` | :arrow_up: | | [.../apache/livy/server/batch/CreateBatchRequest.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQ3JlYXRlQmF0Y2hSZXF1ZXN0LnNjYWxh) | `68.75% <0%> (+6.25%)` | `19% <0%> (+1%)` | :arrow_up: | | ... and [70 more](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codeco
[GitHub] [incubator-livy] gumartinm opened a new pull request #192: [LIVY-620] Spark batch session always ends with success when configuration is master yarn and deploy-mode client
gumartinm opened a new pull request #192: [LIVY-620] Spark batch session always ends with success when configuration is master yarn and deploy-mode client URL: https://github.com/apache/incubator-livy/pull/192 ## What changes were proposed in this pull request? Batch session should end with state dead when process exits with no 0 return code. https://issues.apache.org/jira/browse/LIVY-620 ## How was this patch tested? 1. Unit Test (included in this PR) Submit batch session that runs forever, wait 2 seconds, kill that batch session and expect for state dead. 2. Also currently used in production environment. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jerryshao closed pull request #192: [LIVY-620] Spark batch session always ends with success when configuration is master yarn and deploy-mode client
jerryshao closed pull request #192: [LIVY-620] Spark batch session always ends with success when configuration is master yarn and deploy-mode client URL: https://github.com/apache/incubator-livy/pull/192 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] yiheng commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift serv
yiheng commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r314153269
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/LivyOperationManager.scala
##
@@ -187,6 +187,64 @@ class LivyOperationManager(val livyThriftSessionManager:
LivyThriftSessionManage
})
}
+ @throws[HiveSQLException]
+ def getTables(
+ sessionHandle: SessionHandle,
+ catalogName: String,
+ schemaName: String,
+ tableName: String,
+ tableTypes: util.List[String]): OperationHandle = {
+ executeOperation(sessionHandle, {
+val op = new GetTablesOperation(sessionHandle, catalogName, schemaName,
tableName, tableTypes,
Review comment:
fixed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] yiheng commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift serv
yiheng commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r314152190
##
File path:
thriftserver/session/src/main/java/org/apache/livy/thriftserver/session/GetTablesJob.java
##
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.thriftserver.session;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import static scala.collection.JavaConversions.seqAsJavaList;
+
+import org.apache.spark.sql.catalyst.TableIdentifier;
+import org.apache.spark.sql.catalyst.catalog.CatalogTable;
+import org.apache.spark.sql.catalyst.catalog.CatalogTableType;
+import org.apache.spark.sql.catalyst.catalog.SessionCatalog;
+
+public class GetTablesJob extends SparkCatalogJob {
+ private final String databasePattern;
+ private final String tablePattern;
+ private final List tableTypes = new ArrayList();
+
+ public GetTablesJob(
+ String databasePattern,
+ String tablePattern,
+ List tableTypes,
+ String sessionId,
+ String jobId) {
+super(sessionId, jobId);
+this.databasePattern = databasePattern;
+this.tablePattern = tablePattern;
+if (tableTypes != null) {
+ for (String type : tableTypes) {
+this.tableTypes.add(type.toUpperCase());
+ }
+}
+ }
+
+ @Override
+ protected List fetchCatalogObjects(SessionCatalog catalog) {
+List tableList = new ArrayList();
+List databases =
seqAsJavaList(catalog.listDatabases(databasePattern));
+for (String db: databases) {
+ List tableIdentifiers =
+seqAsJavaList(catalog.listTables(db, tablePattern));
+ for(TableIdentifier tableIdentifier: tableIdentifiers) {
+CatalogTable table =
catalog.getTempViewOrPermanentTableMetadata(tableIdentifier);
+String type = convertTableType(table.tableType().name());
+if (tableTypes.isEmpty() || tableTypes.contains(type)) {
+ tableList.add(
+new Object[] {
+ DEFAULT_HIVE_CATALOG,
+ table.database(),
+ table.identifier().table(),
+ type,
+ table.comment().isDefined() ? table.comment().get() : ""
+}
+);
Review comment:
fixed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] yiheng commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift serv
yiheng commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r314152217
##
File path:
thriftserver/session/src/main/java/org/apache/livy/thriftserver/session/GetSchemasJob.java
##
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.thriftserver.session;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import static scala.collection.JavaConversions.seqAsJavaList;
+
+import org.apache.spark.sql.catalyst.catalog.SessionCatalog;
+
+public class GetSchemasJob extends SparkCatalogJob {
+ private final String schemaName;
+
+ public GetSchemasJob(String schemaName, String sessionId, String jobId) {
+super(sessionId, jobId);
+this.schemaName = schemaName;
+ }
+
+ @Override
+ protected List fetchCatalogObjects(SessionCatalog catalog) {
+List databases = seqAsJavaList(catalog.listDatabases(schemaName));
+List schemas = new ArrayList<>();
+for(String db : databases) {
Review comment:
fixed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] yiheng commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift serv
yiheng commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r314152234
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/operation/SparkCatalogOperation.scala
##
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.thriftserver.operation
+
+import org.apache.commons.lang.StringUtils
+import org.apache.hive.service.cli._
+
+import org.apache.livy.thriftserver.LivyThriftSessionManager
+import org.apache.livy.thriftserver.serde.ThriftResultSet
+import org.apache.livy.thriftserver.session.{CleanupCatalogResultJob,
FetchCatalogResultJob}
+
+/**
+ * SparkCatalogOperation is the base class for operations which need to fetch
catalog information
+ * from spark session.
+ */
+abstract class SparkCatalogOperation(
+sessionHandle: SessionHandle,
+opType: OperationType,
+sessionManager: LivyThriftSessionManager)
+ extends Operation(sessionHandle, opType) {
+
+ // The initialization need to be lazy in order not to block when the
instance is created
+ protected lazy val rscClient = {
+// This call is blocking, we are waiting for the session to be ready.
+sessionManager.getLivySession(sessionHandle).client.get
+ }
+
+ protected lazy val jobId = {
+this.opHandle.getHandleIdentifier.getPublicId.toString + "-" +
+ this.opHandle.getHandleIdentifier.getSecretId.toString
+ }
+
+ protected lazy val sessionId = {
+sessionHandle.getSessionId.toString
+ }
+
+ @throws[HiveSQLException]
+ override def close(): Unit = {
+val cleaned = rscClient.submit(new CleanupCatalogResultJob(sessionId,
jobId)).get()
+if (!cleaned) {
+ warn(s"Fail to cleanup fetch catalog job (session = ${sessionId}), " +
+"this message can be ignored if the job failed.")
+}
+setState(OperationState.CLOSED)
+ }
+
+ @throws[HiveSQLException]
+ override def cancel(stateAfterCancel: OperationState): Unit = {
+setState(OperationState.CANCELED)
+// Spark fetch schema is not a really spark job. It only run on driver and
cannot be cancelled
+throw new UnsupportedOperationException("SparkCatalogOperation.cancel()")
+ }
+
+ /**
+ * Convert wildchars and escape sequence from JDBC format to
datanucleous/regex
+ *
+ * This is ported from Spark Hive Thrift MetaOperation.
+ */
+ protected def convertIdentifierPattern(pattern: String, datanucleusFormat:
Boolean): String = {
+if (pattern == null) {
+ convertPattern("%", datanucleusFormat = true)
+} else {
+ convertPattern(pattern, datanucleusFormat)
+}
+ }
+
+ /**
+ * Convert wildchars and escape sequence of schema pattern from JDBC format
to datanucleous/regex
+ * The schema pattern treats empty string also as wildchar.
+ *
+ * This is ported from Spark Hive Thrift MetaOperation.
+ */
+ protected def convertSchemaPattern(pattern: String): String = {
+if (StringUtils.isEmpty(pattern)) {
+ convertPattern("%", datanucleusFormat = true)
+} else {
+ convertPattern(pattern, datanucleusFormat = true)
+}
+ }
+
+ private def convertPattern(pattern: String, datanucleusFormat: Boolean):
String = {
+val wStr = if (datanucleusFormat) "*" else ".*"
+pattern
+ .replaceAll("([^])%", "$1" + wStr)
+ .replaceAll("%", "%")
+ .replaceAll("^%", wStr)
+ .replaceAll("([^])_", "$1.")
+ .replaceAll("_", "_")
+ .replaceAll("^_", ".")
+ }
+
+ override def getNextRowSet(orientation: FetchOrientation, maxRowsL: Long):
ThriftResultSet = {
+validateFetchOrientation(orientation)
+assertState(Seq(OperationState.FINISHED))
+setHasResultSet(true)
+val maxRows = maxRowsL.toInt
+val results = rscClient.submit(new FetchCatalogResultJob(sessionId, jobId,
maxRows)).get()
+
+val rowSet = ThriftResultSet.apply(getResultSetSchema, protocolVersion)
+import scala.collection.JavaConverters._
+results.asScala.foreach {r => rowSet.addRow(r.asInstanceOf[Array[Any]])}
Review comment:
fixed
[GitHub] [incubator-livy] yantzu closed pull request #195: [LIVY-634] To support reuse hive PasswdAuthenticationProvider implementation in ThriftServer
yantzu closed pull request #195: [LIVY-634] To support reuse hive PasswdAuthenticationProvider implementation in ThriftServer URL: https://github.com/apache/incubator-livy/pull/195 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] yantzu opened a new pull request #195: [LIVY-634] To support reuse hive PasswdAuthenticationProvider implementation in ThriftServer
yantzu opened a new pull request #195: [LIVY-634] To support reuse hive PasswdAuthenticationProvider implementation in ThriftServer URL: https://github.com/apache/incubator-livy/pull/195 ## What changes were proposed in this pull request? Currently Livy share same PasswdAuthenticationProvider interface with HiveServer2, but not able to reuse PasswdAuthenticationProvider implementation like LdapAuthenticationProviderImpl and PamAuthenticationProviderImpl. We can extend livy CustomAuthenticationProvider to support hive PasswdAuthenticationProvider implementation. For example, to use LdapAuthenticationProviderImpl we can add such configuration in livy.conf: livy.server.thrift.authentication = CUSTOM livy.server.thrift.custom.authentication.class = org.apache.hive.service.auth.LdapAuthenticationProviderImpl livy.server.thrift.authentication.custom.hive.server2.authentication.ldap.url = livy.server.thrift.authentication.custom.hive.server2.authentication.ldap.Domain = https://issues.apache.org/jira/browse/LIVY-634 ## How was this patch tested? Tested manually. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift s
jerryshao commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r313839254
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/operation/SparkCatalogOperation.scala
##
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.thriftserver.operation
+
+import org.apache.commons.lang.StringUtils
+import org.apache.hive.service.cli._
+
+import org.apache.livy.thriftserver.LivyThriftSessionManager
+import org.apache.livy.thriftserver.serde.ThriftResultSet
+import org.apache.livy.thriftserver.session.{CleanupCatalogResultJob,
FetchCatalogResultJob}
+
+/**
+ * SparkCatalogOperation is the base class for operations which need to fetch
catalog information
+ * from spark session.
+ */
+abstract class SparkCatalogOperation(
+sessionHandle: SessionHandle,
+opType: OperationType,
+sessionManager: LivyThriftSessionManager)
+ extends Operation(sessionHandle, opType) {
+
+ // The initialization need to be lazy in order not to block when the
instance is created
+ protected lazy val rscClient = {
+// This call is blocking, we are waiting for the session to be ready.
+sessionManager.getLivySession(sessionHandle).client.get
+ }
+
+ protected lazy val jobId = {
+this.opHandle.getHandleIdentifier.getPublicId.toString + "-" +
+ this.opHandle.getHandleIdentifier.getSecretId.toString
+ }
+
+ protected lazy val sessionId = {
+sessionHandle.getSessionId.toString
+ }
+
+ @throws[HiveSQLException]
+ override def close(): Unit = {
+val cleaned = rscClient.submit(new CleanupCatalogResultJob(sessionId,
jobId)).get()
+if (!cleaned) {
+ warn(s"Fail to cleanup fetch catalog job (session = ${sessionId}), " +
+"this message can be ignored if the job failed.")
+}
+setState(OperationState.CLOSED)
+ }
+
+ @throws[HiveSQLException]
+ override def cancel(stateAfterCancel: OperationState): Unit = {
+setState(OperationState.CANCELED)
+// Spark fetch schema is not a really spark job. It only run on driver and
cannot be cancelled
+throw new UnsupportedOperationException("SparkCatalogOperation.cancel()")
+ }
+
+ /**
+ * Convert wildchars and escape sequence from JDBC format to
datanucleous/regex
+ *
+ * This is ported from Spark Hive Thrift MetaOperation.
+ */
+ protected def convertIdentifierPattern(pattern: String, datanucleusFormat:
Boolean): String = {
+if (pattern == null) {
+ convertPattern("%", datanucleusFormat = true)
+} else {
+ convertPattern(pattern, datanucleusFormat)
+}
+ }
+
+ /**
+ * Convert wildchars and escape sequence of schema pattern from JDBC format
to datanucleous/regex
+ * The schema pattern treats empty string also as wildchar.
+ *
+ * This is ported from Spark Hive Thrift MetaOperation.
+ */
+ protected def convertSchemaPattern(pattern: String): String = {
+if (StringUtils.isEmpty(pattern)) {
+ convertPattern("%", datanucleusFormat = true)
+} else {
+ convertPattern(pattern, datanucleusFormat = true)
+}
+ }
+
+ private def convertPattern(pattern: String, datanucleusFormat: Boolean):
String = {
+val wStr = if (datanucleusFormat) "*" else ".*"
+pattern
+ .replaceAll("([^])%", "$1" + wStr)
+ .replaceAll("%", "%")
+ .replaceAll("^%", wStr)
+ .replaceAll("([^])_", "$1.")
+ .replaceAll("_", "_")
+ .replaceAll("^_", ".")
+ }
+
+ override def getNextRowSet(orientation: FetchOrientation, maxRowsL: Long):
ThriftResultSet = {
+validateFetchOrientation(orientation)
+assertState(Seq(OperationState.FINISHED))
+setHasResultSet(true)
+val maxRows = maxRowsL.toInt
+val results = rscClient.submit(new FetchCatalogResultJob(sessionId, jobId,
maxRows)).get()
+
+val rowSet = ThriftResultSet.apply(getResultSetSchema, protocolVersion)
+import scala.collection.JavaConverters._
+results.asScala.foreach {r => rowSet.addRow(r.asInstanceOf[Array[Any]])}
Review comment:
nit: `fo
[GitHub] [incubator-livy] jerryshao commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift s
jerryshao commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r313841090
##
File path:
thriftserver/session/src/main/java/org/apache/livy/thriftserver/session/GetSchemasJob.java
##
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.thriftserver.session;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import static scala.collection.JavaConversions.seqAsJavaList;
+
+import org.apache.spark.sql.catalyst.catalog.SessionCatalog;
+
+public class GetSchemasJob extends SparkCatalogJob {
+ private final String schemaName;
+
+ public GetSchemasJob(String schemaName, String sessionId, String jobId) {
+super(sessionId, jobId);
+this.schemaName = schemaName;
+ }
+
+ @Override
+ protected List fetchCatalogObjects(SessionCatalog catalog) {
+List databases = seqAsJavaList(catalog.listDatabases(schemaName));
+List schemas = new ArrayList<>();
+for(String db : databases) {
Review comment:
nit: space after `for`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift s
jerryshao commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r313841612
##
File path:
thriftserver/session/src/main/java/org/apache/livy/thriftserver/session/GetTablesJob.java
##
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.thriftserver.session;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import static scala.collection.JavaConversions.seqAsJavaList;
+
+import org.apache.spark.sql.catalyst.TableIdentifier;
+import org.apache.spark.sql.catalyst.catalog.CatalogTable;
+import org.apache.spark.sql.catalyst.catalog.CatalogTableType;
+import org.apache.spark.sql.catalyst.catalog.SessionCatalog;
+
+public class GetTablesJob extends SparkCatalogJob {
+ private final String databasePattern;
+ private final String tablePattern;
+ private final List tableTypes = new ArrayList();
+
+ public GetTablesJob(
+ String databasePattern,
+ String tablePattern,
+ List tableTypes,
+ String sessionId,
+ String jobId) {
+super(sessionId, jobId);
+this.databasePattern = databasePattern;
+this.tablePattern = tablePattern;
+if (tableTypes != null) {
+ for (String type : tableTypes) {
+this.tableTypes.add(type.toUpperCase());
+ }
+}
+ }
+
+ @Override
+ protected List fetchCatalogObjects(SessionCatalog catalog) {
+List tableList = new ArrayList();
+List databases =
seqAsJavaList(catalog.listDatabases(databasePattern));
+for (String db: databases) {
+ List tableIdentifiers =
+seqAsJavaList(catalog.listTables(db, tablePattern));
+ for(TableIdentifier tableIdentifier: tableIdentifiers) {
+CatalogTable table =
catalog.getTempViewOrPermanentTableMetadata(tableIdentifier);
+String type = convertTableType(table.tableType().name());
+if (tableTypes.isEmpty() || tableTypes.contains(type)) {
+ tableList.add(
+new Object[] {
+ DEFAULT_HIVE_CATALOG,
+ table.database(),
+ table.identifier().table(),
+ type,
+ table.comment().isDefined() ? table.comment().get() : ""
+}
+);
Review comment:
move this line to above.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift s
jerryshao commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r313832654
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/LivyOperationManager.scala
##
@@ -187,6 +187,64 @@ class LivyOperationManager(val livyThriftSessionManager:
LivyThriftSessionManage
})
}
+ @throws[HiveSQLException]
+ def getTables(
+ sessionHandle: SessionHandle,
+ catalogName: String,
+ schemaName: String,
+ tableName: String,
+ tableTypes: util.List[String]): OperationHandle = {
+ executeOperation(sessionHandle, {
+val op = new GetTablesOperation(sessionHandle, catalogName, schemaName,
tableName, tableTypes,
Review comment:
nit: indention here and above seems not correct.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift server
codecov-io edited a comment on issue #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift server URL: https://github.com/apache/incubator-livy/pull/194#issuecomment-521240478 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=h1) Report > Merging [#194](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/e7f23e06606ff79aec1c22e79a96f959cb89a8be?src=pr&el=desc) will **increase** coverage by `40.37%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master#194 +/- ## + Coverage 28.33% 68.7% +40.37% - Complexity 343 911 +568 Files 100 100 Lines 56795679 Branches855 855 + Hits 16093902 +2293 + Misses 37391219 -2520 - Partials331 558 +227 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `35.96% <0%> (+0.98%)` | `11% <0%> (ø)` | :arrow_down: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `95.87% <0%> (+1.03%)` | `21% <0%> (+3%)` | :arrow_up: | | [.../main/scala/org/apache/livy/server/WebServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvV2ViU2VydmVyLnNjYWxh) | `53.33% <0%> (+1.66%)` | `10% <0%> (+1%)` | :arrow_up: | | [...la/org/apache/livy/server/batch/BatchSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uLnNjYWxh) | `86.17% <0%> (+2.12%)` | `14% <0%> (ø)` | :arrow_down: | | [...la/org/apache/livy/utils/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9TcGFya1Byb2Nlc3NCdWlsZGVyLnNjYWxh) | `54.44% <0%> (+2.22%)` | `11% <0%> (+1%)` | :arrow_up: | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (+2.77%)` | `2% <0%> (ø)` | :arrow_down: | | [...e/livy/server/interactive/InteractiveSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvaW50ZXJhY3RpdmUvSW50ZXJhY3RpdmVTZXNzaW9uLnNjYWxh) | `69.11% <0%> (+3.66%)` | `44% <0%> (+2%)` | :arrow_up: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <0%> (+5%)` | `10% <0%> (ø)` | :arrow_down: | | [...ain/scala/org/apache/livy/server/JsonServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvSnNvblNlcnZsZXQuc2NhbGE=) | `38.46% <0%> (+5.76%)` | `18% <0%> (+4%)` | :arrow_up: | | [.../apache/livy/server/batch/CreateBatchRequest.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQ3JlYXRlQmF0Y2hSZXF1ZXN0LnNjYWxh) | `68.75% <0%> (+6.25%)` | `19% <0%> (+1%)` | :arrow_up: | | ... and [70 more](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://co
[GitHub] [incubator-livy] codecov-io commented on issue #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift server
codecov-io commented on issue #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift server URL: https://github.com/apache/incubator-livy/pull/194#issuecomment-521240478 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=h1) Report > Merging [#194](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/e7f23e06606ff79aec1c22e79a96f959cb89a8be?src=pr&el=desc) will **increase** coverage by `40.37%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master#194 +/- ## + Coverage 28.33% 68.7% +40.37% - Complexity 343 911 +568 Files 100 100 Lines 56795679 Branches855 855 + Hits 16093902 +2293 + Misses 37391219 -2520 - Partials331 558 +227 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `35.96% <0%> (+0.98%)` | `11% <0%> (ø)` | :arrow_down: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `95.87% <0%> (+1.03%)` | `21% <0%> (+3%)` | :arrow_up: | | [.../main/scala/org/apache/livy/server/WebServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvV2ViU2VydmVyLnNjYWxh) | `53.33% <0%> (+1.66%)` | `10% <0%> (+1%)` | :arrow_up: | | [...la/org/apache/livy/server/batch/BatchSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uLnNjYWxh) | `86.17% <0%> (+2.12%)` | `14% <0%> (ø)` | :arrow_down: | | [...la/org/apache/livy/utils/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9TcGFya1Byb2Nlc3NCdWlsZGVyLnNjYWxh) | `54.44% <0%> (+2.22%)` | `11% <0%> (+1%)` | :arrow_up: | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (+2.77%)` | `2% <0%> (ø)` | :arrow_down: | | [...e/livy/server/interactive/InteractiveSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvaW50ZXJhY3RpdmUvSW50ZXJhY3RpdmVTZXNzaW9uLnNjYWxh) | `69.11% <0%> (+3.66%)` | `44% <0%> (+2%)` | :arrow_up: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <0%> (+5%)` | `10% <0%> (ø)` | :arrow_down: | | [...ain/scala/org/apache/livy/server/JsonServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvSnNvblNlcnZsZXQuc2NhbGE=) | `38.46% <0%> (+5.76%)` | `18% <0%> (+4%)` | :arrow_up: | | [.../apache/livy/server/batch/CreateBatchRequest.scala](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQ3JlYXRlQmF0Y2hSZXF1ZXN0LnNjYWxh) | `68.75% <0%> (+6.25%)` | `19% <0%> (+1%)` | :arrow_up: | | ... and [70 more](https://codecov.io/gh/apache/incubator-livy/pull/194/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/194?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.i
[GitHub] [incubator-livy] codecov-io commented on issue #196: [LIVY-606] fix the invalid conf of pyspark.archives and sparkr.package
codecov-io commented on issue #196: [LIVY-606] fix the invalid conf of pyspark.archives and sparkr.package URL: https://github.com/apache/incubator-livy/pull/196#issuecomment-521238783 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/196?src=pr&el=h1) Report > Merging [#196](https://codecov.io/gh/apache/incubator-livy/pull/196?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/e7f23e06606ff79aec1c22e79a96f959cb89a8be?src=pr&el=desc) will **increase** coverage by `40.41%`. > The diff coverage is `66.66%`. [](https://codecov.io/gh/apache/incubator-livy/pull/196?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #196 +/- ## = + Coverage 28.33% 68.74% +40.41% - Complexity 343 909 +566 = Files 100 100 Lines 5679 5676-3 Branches855 855 = + Hits 1609 3902 +2293 + Misses 3739 1215 -2524 - Partials331 559 +228 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/196?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...e/livy/server/interactive/InteractiveSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvaW50ZXJhY3RpdmUvSW50ZXJhY3RpdmVTZXNzaW9uLnNjYWxh) | `68.82% <66.66%> (+3.38%)` | `44 <0> (+2)` | :arrow_up: | | [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `35.46% <0%> (+0.49%)` | `11% <0%> (ø)` | :arrow_down: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `95.87% <0%> (+1.03%)` | `21% <0%> (+3%)` | :arrow_up: | | [.../main/scala/org/apache/livy/server/WebServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvV2ViU2VydmVyLnNjYWxh) | `53.33% <0%> (+1.66%)` | `10% <0%> (+1%)` | :arrow_up: | | [...la/org/apache/livy/server/batch/BatchSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uLnNjYWxh) | `86.17% <0%> (+2.12%)` | `14% <0%> (ø)` | :arrow_down: | | [...la/org/apache/livy/utils/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9TcGFya1Byb2Nlc3NCdWlsZGVyLnNjYWxh) | `54.44% <0%> (+2.22%)` | `11% <0%> (+1%)` | :arrow_up: | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (+2.77%)` | `2% <0%> (ø)` | :arrow_down: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <0%> (+5%)` | `10% <0%> (ø)` | :arrow_down: | | [...ain/scala/org/apache/livy/server/JsonServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvSnNvblNlcnZsZXQuc2NhbGE=) | `38.46% <0%> (+5.76%)` | `18% <0%> (+4%)` | :arrow_up: | | ... and [71 more](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/196?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/196?src=pr&el=footer). Last update [e7f23e0...59b4c1e](https://codecov.io/gh/apache/incubator-livy/pull/196?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated messag
[GitHub] [incubator-livy] codecov-io edited a comment on issue #196: [LIVY-606] fix the invalid conf of pyspark.archives and sparkr.package
codecov-io edited a comment on issue #196: [LIVY-606] fix the invalid conf of pyspark.archives and sparkr.package URL: https://github.com/apache/incubator-livy/pull/196#issuecomment-521238783 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/196?src=pr&el=h1) Report > Merging [#196](https://codecov.io/gh/apache/incubator-livy/pull/196?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/e7f23e06606ff79aec1c22e79a96f959cb89a8be?src=pr&el=desc) will **increase** coverage by `40.41%`. > The diff coverage is `66.66%`. [](https://codecov.io/gh/apache/incubator-livy/pull/196?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #196 +/- ## = + Coverage 28.33% 68.74% +40.41% - Complexity 343 909 +566 = Files 100 100 Lines 5679 5676-3 Branches855 855 = + Hits 1609 3902 +2293 + Misses 3739 1215 -2524 - Partials331 559 +228 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/196?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...e/livy/server/interactive/InteractiveSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvaW50ZXJhY3RpdmUvSW50ZXJhY3RpdmVTZXNzaW9uLnNjYWxh) | `68.82% <66.66%> (+3.38%)` | `44 <0> (+2)` | :arrow_up: | | [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `35.46% <0%> (+0.49%)` | `11% <0%> (ø)` | :arrow_down: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `95.87% <0%> (+1.03%)` | `21% <0%> (+3%)` | :arrow_up: | | [.../main/scala/org/apache/livy/server/WebServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvV2ViU2VydmVyLnNjYWxh) | `53.33% <0%> (+1.66%)` | `10% <0%> (+1%)` | :arrow_up: | | [...la/org/apache/livy/server/batch/BatchSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uLnNjYWxh) | `86.17% <0%> (+2.12%)` | `14% <0%> (ø)` | :arrow_down: | | [...la/org/apache/livy/utils/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9TcGFya1Byb2Nlc3NCdWlsZGVyLnNjYWxh) | `54.44% <0%> (+2.22%)` | `11% <0%> (+1%)` | :arrow_up: | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (+2.77%)` | `2% <0%> (ø)` | :arrow_down: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <0%> (+5%)` | `10% <0%> (ø)` | :arrow_down: | | [...ain/scala/org/apache/livy/server/JsonServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvSnNvblNlcnZsZXQuc2NhbGE=) | `38.46% <0%> (+5.76%)` | `18% <0%> (+4%)` | :arrow_up: | | ... and [71 more](https://codecov.io/gh/apache/incubator-livy/pull/196/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/196?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/196?src=pr&el=footer). Last update [e7f23e0...59b4c1e](https://codecov.io/gh/apache/incubator-livy/pull/196?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated
[GitHub] [incubator-livy] runzhiwang closed pull request #196: [LIVY-606] fix the invalid conf of pyspark.archives and sparkr.package
runzhiwang closed pull request #196: [LIVY-606] fix the invalid conf of pyspark.archives and sparkr.package URL: https://github.com/apache/incubator-livy/pull/196 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] runzhiwang opened a new pull request #196: [LIVY-606] fix the invalid conf of pyspark.archives and sparkr.package
runzhiwang opened a new pull request #196: [LIVY-606] fix the invalid conf of pyspark.archives and sparkr.package URL: https://github.com/apache/incubator-livy/pull/196 ## What changes were proposed in this pull request? 1. Remove livy.pyspark.archives and livy.sparkr.package in livy.conf, which can be set in livy-client.conf. Remove getting PYSPARK_ARCHIVES from livy.conf in findPySparkArchives() 2. Remove livy.rsc.pyspark.archives in livy-client.conf, which can't work because spark will upload the pyspark.zip and py4j-0.10.7-src.zip from local to hdfs. 3. Avoid livy.rsc.sparkr.package in builderProperties overwritting by findSparkRArchive(). ## How was this patch tested? Config the livy.rsc.sparkr.package by hdfs path, then livy will not upload zip from local to hdfs every time a session is started. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] yiheng commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift serv
yiheng commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r313847060
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/operation/GetColumnsOperation.scala
##
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.thriftserver.operation
+
+import org.apache.commons.lang.exception.ExceptionUtils
+import org.apache.hive.service.cli.{HiveSQLException, OperationState,
OperationType, SessionHandle}
+
+import org.apache.livy.Logging
+import org.apache.livy.thriftserver.types.{BasicDataType, Field, Schema}
+import org.apache.livy.thriftserver.LivyThriftSessionManager
+import org.apache.livy.thriftserver.session.{GetColumnsJob, GetFunctionsJob}
+
+class GetColumnsOperation(
+sessionHandle: SessionHandle,
+catalogName: String,
+schemaName: String,
+tableName: String,
+columnName: String,
+sessionManager: LivyThriftSessionManager)
+ extends SparkCatalogOperation(
+sessionHandle, OperationType.GET_COLUMNS, sessionManager) with Logging {
+
+ info("Starting GetColumnsOperation")
+
+ @throws(classOf[HiveSQLException])
+ override protected def runInternal(): Unit = {
+setState(OperationState.RUNNING)
+info("Fetching columns list")
+try {
+ rscClient.submit(new GetColumnsJob(
+convertSchemaPattern(schemaName),
+convertIdentifierPattern(tableName, datanucleusFormat = true),
+if (columnName == null) null else convertIdentifierPattern(
+ columnName, datanucleusFormat = false),
+seesionId,
+jobId
+ )).get()
+
+ setState(OperationState.FINISHED)
+ info("Fetching column list has been successfully finished")
+} catch {
+ case e: Throwable =>
+error("Remote job meet an exception: " +
ExceptionUtils.getFullStackTrace(e))
Review comment:
fixed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] yiheng commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift serv
yiheng commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r313701841
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/operation/GetColumnsOperation.scala
##
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.thriftserver.operation
+
+import org.apache.commons.lang.exception.ExceptionUtils
+import org.apache.hive.service.cli.{HiveSQLException, OperationState,
OperationType, SessionHandle}
+
+import org.apache.livy.Logging
+import org.apache.livy.thriftserver.types.{BasicDataType, Field, Schema}
+import org.apache.livy.thriftserver.LivyThriftSessionManager
+import org.apache.livy.thriftserver.session.{GetColumnsJob, GetFunctionsJob}
+
+class GetColumnsOperation(
+sessionHandle: SessionHandle,
+catalogName: String,
+schemaName: String,
+tableName: String,
+columnName: String,
+sessionManager: LivyThriftSessionManager)
+ extends SparkCatalogOperation(
+sessionHandle, OperationType.GET_COLUMNS, sessionManager) with Logging {
+
+ info("Starting GetColumnsOperation")
+
+ @throws(classOf[HiveSQLException])
+ override protected def runInternal(): Unit = {
+setState(OperationState.RUNNING)
+info("Fetching columns list")
+try {
+ rscClient.submit(new GetColumnsJob(
+convertSchemaPattern(schemaName),
+convertIdentifierPattern(tableName, datanucleusFormat = true),
+if (columnName == null) null else convertIdentifierPattern(
+ columnName, datanucleusFormat = false),
+seesionId,
+jobId
+ )).get()
+
+ setState(OperationState.FINISHED)
+ info("Fetching column list has been successfully finished")
+} catch {
+ case e: Throwable =>
+error("Remote job meet an exception: " +
ExceptionUtils.getFullStackTrace(e))
Review comment:
`error("", e)` only print the first line of exception, while
`ExceptionUtils.getFullStackTrace(e)` print all the exception stack, which can
help us locate the error source
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] runzhiwang closed pull request #196: [LIVY-606] fix the invalid conf of pyspark.archives and sparkr.package
runzhiwang closed pull request #196: [LIVY-606] fix the invalid conf of pyspark.archives and sparkr.package URL: https://github.com/apache/incubator-livy/pull/196 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] runzhiwang opened a new pull request #196: [LIVY-606] fix the invalid conf of pyspark.archives and sparkr.package
runzhiwang opened a new pull request #196: [LIVY-606] fix the invalid conf of pyspark.archives and sparkr.package URL: https://github.com/apache/incubator-livy/pull/196 ## What changes were proposed in this pull request? 1. Remove livy.pyspark.archives and livy.sparkr.package in livy.conf, which can be set in livy-client.conf. Remove getting PYSPARK_ARCHIVES from livy.conf in findPySparkArchives() 2. Remove livy.rsc.pyspark.archives in livy-client.conf, which can't work because spark will upload the pyspark.zip and py4j-0.10.7-src.zip from local to hdfs. 3. Avoid livy.rsc.sparkr.package in builderProperties overwritting by findSparkRArchive(). ## How was this patch tested? Config the livy.rsc.sparkr.package by hdfs path, then livy will not upload zip from local to hdfs every time a session is started. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jerryshao edited a comment on issue #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase
jerryshao edited a comment on issue #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase URL: https://github.com/apache/incubator-livy/pull/198#issuecomment-521209657 @runzhiwang can you please also manually verify if thriftserver can be worked without issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on issue #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase
jerryshao commented on issue #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase URL: https://github.com/apache/incubator-livy/pull/198#issuecomment-521209657 @runzhiwang can you please also verify if thriftserver can be worked without issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #199: [LIVY-573] Add tests for operation logs retrieval
codecov-io edited a comment on issue #199: [LIVY-573] Add tests for operation logs retrieval URL: https://github.com/apache/incubator-livy/pull/199#issuecomment-521201698 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/199?src=pr&el=h1) Report > Merging [#199](https://codecov.io/gh/apache/incubator-livy/pull/199?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/e7f23e06606ff79aec1c22e79a96f959cb89a8be?src=pr&el=desc) will **increase** coverage by `40.2%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-livy/pull/199?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #199 +/- ## + Coverage 28.33% 68.53% +40.2% - Complexity 343 909 +566 Files 100 100 Lines 5679 5679 Branches855 855 + Hits 1609 3892+2283 + Misses 3739 1228-2511 - Partials331 559 +228 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/199?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `35.46% <0%> (+0.49%)` | `11% <0%> (ø)` | :arrow_down: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `95.87% <0%> (+1.03%)` | `21% <0%> (+3%)` | :arrow_up: | | [.../main/scala/org/apache/livy/server/WebServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvV2ViU2VydmVyLnNjYWxh) | `53.33% <0%> (+1.66%)` | `10% <0%> (+1%)` | :arrow_up: | | [...la/org/apache/livy/server/batch/BatchSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uLnNjYWxh) | `86.17% <0%> (+2.12%)` | `14% <0%> (ø)` | :arrow_down: | | [...la/org/apache/livy/utils/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9TcGFya1Byb2Nlc3NCdWlsZGVyLnNjYWxh) | `54.44% <0%> (+2.22%)` | `11% <0%> (+1%)` | :arrow_up: | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (+2.77%)` | `2% <0%> (ø)` | :arrow_down: | | [...e/livy/server/interactive/InteractiveSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvaW50ZXJhY3RpdmUvSW50ZXJhY3RpdmVTZXNzaW9uLnNjYWxh) | `69.11% <0%> (+3.66%)` | `44% <0%> (+2%)` | :arrow_up: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <0%> (+5%)` | `10% <0%> (ø)` | :arrow_down: | | [...ain/scala/org/apache/livy/server/JsonServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvSnNvblNlcnZsZXQuc2NhbGE=) | `38.46% <0%> (+5.76%)` | `18% <0%> (+4%)` | :arrow_up: | | [.../apache/livy/server/batch/CreateBatchRequest.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQ3JlYXRlQmF0Y2hSZXF1ZXN0LnNjYWxh) | `68.75% <0%> (+6.25%)` | `19% <0%> (+1%)` | :arrow_up: | | ... and [70 more](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/199?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/199?src=pr&el=footer). Last update [e
[GitHub] [incubator-livy] codecov-io commented on issue #199: [LIVY-573] Add tests for operation logs retrieval
codecov-io commented on issue #199: [LIVY-573] Add tests for operation logs retrieval URL: https://github.com/apache/incubator-livy/pull/199#issuecomment-521201698 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/199?src=pr&el=h1) Report > Merging [#199](https://codecov.io/gh/apache/incubator-livy/pull/199?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/e7f23e06606ff79aec1c22e79a96f959cb89a8be?src=pr&el=desc) will **increase** coverage by `40.2%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-livy/pull/199?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #199 +/- ## + Coverage 28.33% 68.53% +40.2% - Complexity 343 909 +566 Files 100 100 Lines 5679 5679 Branches855 855 + Hits 1609 3892+2283 + Misses 3739 1228-2511 - Partials331 559 +228 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/199?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `35.46% <0%> (+0.49%)` | `11% <0%> (ø)` | :arrow_down: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `95.87% <0%> (+1.03%)` | `21% <0%> (+3%)` | :arrow_up: | | [.../main/scala/org/apache/livy/server/WebServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvV2ViU2VydmVyLnNjYWxh) | `53.33% <0%> (+1.66%)` | `10% <0%> (+1%)` | :arrow_up: | | [...la/org/apache/livy/server/batch/BatchSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uLnNjYWxh) | `86.17% <0%> (+2.12%)` | `14% <0%> (ø)` | :arrow_down: | | [...la/org/apache/livy/utils/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9TcGFya1Byb2Nlc3NCdWlsZGVyLnNjYWxh) | `54.44% <0%> (+2.22%)` | `11% <0%> (+1%)` | :arrow_up: | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (+2.77%)` | `2% <0%> (ø)` | :arrow_down: | | [...e/livy/server/interactive/InteractiveSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvaW50ZXJhY3RpdmUvSW50ZXJhY3RpdmVTZXNzaW9uLnNjYWxh) | `69.11% <0%> (+3.66%)` | `44% <0%> (+2%)` | :arrow_up: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <0%> (+5%)` | `10% <0%> (ø)` | :arrow_down: | | [...ain/scala/org/apache/livy/server/JsonServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvSnNvblNlcnZsZXQuc2NhbGE=) | `38.46% <0%> (+5.76%)` | `18% <0%> (+4%)` | :arrow_up: | | [.../apache/livy/server/batch/CreateBatchRequest.scala](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQ3JlYXRlQmF0Y2hSZXF1ZXN0LnNjYWxh) | `68.75% <0%> (+6.25%)` | `19% <0%> (+1%)` | :arrow_up: | | ... and [70 more](https://codecov.io/gh/apache/incubator-livy/pull/199/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/199?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/199?src=pr&el=footer). Last update [e7f23e0.
[GitHub] [incubator-livy] codecov-io edited a comment on issue #189: [LIVY-616] Livy Server discovery
codecov-io edited a comment on issue #189: [LIVY-616] Livy Server discovery URL: https://github.com/apache/incubator-livy/pull/189#issuecomment-517236826 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/189?src=pr&el=h1) Report > Merging [#189](https://codecov.io/gh/apache/incubator-livy/pull/189?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/1ce266d78fb3475af6154262629ff92f5d342ae6?src=pr&el=desc) will **decrease** coverage by `0.04%`. > The diff coverage is `87.5%`. [](https://codecov.io/gh/apache/incubator-livy/pull/189?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #189 +/- ## - Coverage 68.71% 68.66% -0.05% - Complexity 908 909 +1 Files 100 103 +3 Lines 5674 5710 +36 Branches854 854 + Hits 3899 3921 +22 - Misses 1219 1231 +12 - Partials556 558 +2 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/189?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...he/livy/server/recovery/FileSystemStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvRmlsZVN5c3RlbVN0YXRlU3RvcmUuc2NhbGE=) | `63.04% <ø> (ø)` | `11 <0> (ø)` | :arrow_down: | | [...c/main/java/org/apache/livy/LivyClientBuilder.java](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-YXBpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L0xpdnlDbGllbnRCdWlsZGVyLmphdmE=) | `83.05% <ø> (ø)` | `17 <0> (ø)` | :arrow_down: | | [...a/org/apache/livy/server/recovery/StateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU3RhdGVTdG9yZS5zY2FsYQ==) | `78.94% <ø> (-4.39%)` | `1 <0> (ø)` | | | [...c/src/main/java/org/apache/livy/rsc/RSCClient.java](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDbGllbnQuamF2YQ==) | `75.15% <ø> (+1.24%)` | `20 <0> (ø)` | :arrow_down: | | [...n/java/org/apache/livy/client/http/HttpClient.java](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-Y2xpZW50LWh0dHAvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2xpdnkvY2xpZW50L2h0dHAvSHR0cENsaWVudC5qYXZh) | `86.15% <100%> (+0.66%)` | `16 <1> (+1)` | :arrow_up: | | [...ala/org/apache/livy/scalaapi/LivyScalaClient.scala](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-c2NhbGEtYXBpL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zY2FsYWFwaS9MaXZ5U2NhbGFDbGllbnQuc2NhbGE=) | `83.87% <100%> (+0.53%)` | `8 <1> (+1)` | :arrow_up: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `96.01% <100%> (+0.16%)` | `21 <0> (ø)` | :arrow_down: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <100%> (ø)` | `10 <1> (ø)` | :arrow_down: | | [...che/livy/server/recovery/BlackholeStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvQmxhY2tob2xlU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `5 <1> (ø)` | :arrow_down: | | [.../org/apache/livy/server/discovery/JsonMapper.scala](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvZGlzY292ZXJ5L0pzb25NYXBwZXIuc2NhbGE=) | `100% <100%> (ø)` | `0 <0> (?)` | | | ... and [16 more](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/189?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/189?src=pr&el=footer). Last update [1ce266d...18c10ed](https://codecov.io/gh/apache/incubator-livy/pul
[GitHub] [incubator-livy] codecov-io edited a comment on issue #189: [LIVY-616] Livy Server discovery
codecov-io edited a comment on issue #189: [LIVY-616] Livy Server discovery URL: https://github.com/apache/incubator-livy/pull/189#issuecomment-517236826 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/189?src=pr&el=h1) Report > Merging [#189](https://codecov.io/gh/apache/incubator-livy/pull/189?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/1ce266d78fb3475af6154262629ff92f5d342ae6?src=pr&el=desc) will **decrease** coverage by `0.04%`. > The diff coverage is `87.5%`. [](https://codecov.io/gh/apache/incubator-livy/pull/189?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #189 +/- ## - Coverage 68.71% 68.66% -0.05% - Complexity 908 909 +1 Files 100 103 +3 Lines 5674 5710 +36 Branches854 854 + Hits 3899 3921 +22 - Misses 1219 1231 +12 - Partials556 558 +2 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/189?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...he/livy/server/recovery/FileSystemStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvRmlsZVN5c3RlbVN0YXRlU3RvcmUuc2NhbGE=) | `63.04% <ø> (ø)` | `11 <0> (ø)` | :arrow_down: | | [...c/main/java/org/apache/livy/LivyClientBuilder.java](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-YXBpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L0xpdnlDbGllbnRCdWlsZGVyLmphdmE=) | `83.05% <ø> (ø)` | `17 <0> (ø)` | :arrow_down: | | [...a/org/apache/livy/server/recovery/StateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU3RhdGVTdG9yZS5zY2FsYQ==) | `78.94% <ø> (-4.39%)` | `1 <0> (ø)` | | | [...c/src/main/java/org/apache/livy/rsc/RSCClient.java](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDbGllbnQuamF2YQ==) | `75.15% <ø> (+1.24%)` | `20 <0> (ø)` | :arrow_down: | | [...n/java/org/apache/livy/client/http/HttpClient.java](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-Y2xpZW50LWh0dHAvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2xpdnkvY2xpZW50L2h0dHAvSHR0cENsaWVudC5qYXZh) | `86.15% <100%> (+0.66%)` | `16 <1> (+1)` | :arrow_up: | | [...ala/org/apache/livy/scalaapi/LivyScalaClient.scala](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-c2NhbGEtYXBpL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zY2FsYWFwaS9MaXZ5U2NhbGFDbGllbnQuc2NhbGE=) | `83.87% <100%> (+0.53%)` | `8 <1> (+1)` | :arrow_up: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `96.01% <100%> (+0.16%)` | `21 <0> (ø)` | :arrow_down: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <100%> (ø)` | `10 <1> (ø)` | :arrow_down: | | [...che/livy/server/recovery/BlackholeStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvQmxhY2tob2xlU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `5 <1> (ø)` | :arrow_down: | | [.../org/apache/livy/server/discovery/JsonMapper.scala](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvZGlzY292ZXJ5L0pzb25NYXBwZXIuc2NhbGE=) | `100% <100%> (ø)` | `0 <0> (?)` | | | ... and [16 more](https://codecov.io/gh/apache/incubator-livy/pull/189/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/189?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/189?src=pr&el=footer). Last update [1ce266d...18c10ed](https://codecov.io/gh/apache/incubator-livy/pul
[GitHub] [incubator-livy] yiheng opened a new pull request #199: [LIVY-573] Add tests for operation logs retrieval
yiheng opened a new pull request #199: [LIVY-573] Add tests for operation logs retrieval URL: https://github.com/apache/incubator-livy/pull/199 ## What changes were proposed in this pull request? Add unit test for operation logs retrieval. ## How was this patch tested? New unit tests and existing unit test This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on issue #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase
jerryshao commented on issue #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase URL: https://github.com/apache/incubator-livy/pull/198#issuecomment-521181417 CC @mgaido91 , would you please help to review. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase
codecov-io edited a comment on issue #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase URL: https://github.com/apache/incubator-livy/pull/198#issuecomment-521144705 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=h1) Report > Merging [#198](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/e7f23e06606ff79aec1c22e79a96f959cb89a8be?src=pr&el=desc) will **increase** coverage by `40.44%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #198 +/- ## = + Coverage 28.33% 68.77% +40.44% - Complexity 343 913 +570 = Files 100 100 Lines 5679 5679 Branches855 855 = + Hits 1609 3906 +2297 + Misses 3739 1216 -2523 - Partials331 557 +226 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `35.46% <0%> (+0.49%)` | `11% <0%> (ø)` | :arrow_down: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `95.87% <0%> (+1.03%)` | `21% <0%> (+3%)` | :arrow_up: | | [.../main/scala/org/apache/livy/server/WebServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvV2ViU2VydmVyLnNjYWxh) | `53.33% <0%> (+1.66%)` | `10% <0%> (+1%)` | :arrow_up: | | [...la/org/apache/livy/server/batch/BatchSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uLnNjYWxh) | `86.17% <0%> (+2.12%)` | `14% <0%> (ø)` | :arrow_down: | | [...la/org/apache/livy/utils/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9TcGFya1Byb2Nlc3NCdWlsZGVyLnNjYWxh) | `54.44% <0%> (+2.22%)` | `11% <0%> (+1%)` | :arrow_up: | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (+2.77%)` | `2% <0%> (ø)` | :arrow_down: | | [...e/livy/server/interactive/InteractiveSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvaW50ZXJhY3RpdmUvSW50ZXJhY3RpdmVTZXNzaW9uLnNjYWxh) | `69.11% <0%> (+3.66%)` | `44% <0%> (+2%)` | :arrow_up: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <0%> (+5%)` | `10% <0%> (ø)` | :arrow_down: | | [...ain/scala/org/apache/livy/server/JsonServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvSnNvblNlcnZsZXQuc2NhbGE=) | `38.46% <0%> (+5.76%)` | `18% <0%> (+4%)` | :arrow_up: | | [.../apache/livy/server/batch/CreateBatchRequest.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQ3JlYXRlQmF0Y2hSZXF1ZXN0LnNjYWxh) | `68.75% <0%> (+6.25%)` | `19% <0%> (+1%)` | :arrow_up: | | ... and [70 more](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/19
[GitHub] [incubator-livy] codecov-io commented on issue #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase
codecov-io commented on issue #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase URL: https://github.com/apache/incubator-livy/pull/198#issuecomment-521144705 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=h1) Report > Merging [#198](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/e7f23e06606ff79aec1c22e79a96f959cb89a8be?src=pr&el=desc) will **increase** coverage by `40.44%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #198 +/- ## = + Coverage 28.33% 68.77% +40.44% - Complexity 343 913 +570 = Files 100 100 Lines 5679 5679 Branches855 855 = + Hits 1609 3906 +2297 + Misses 3739 1216 -2523 - Partials331 557 +226 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `35.46% <0%> (+0.49%)` | `11% <0%> (ø)` | :arrow_down: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `95.87% <0%> (+1.03%)` | `21% <0%> (+3%)` | :arrow_up: | | [.../main/scala/org/apache/livy/server/WebServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvV2ViU2VydmVyLnNjYWxh) | `53.33% <0%> (+1.66%)` | `10% <0%> (+1%)` | :arrow_up: | | [...la/org/apache/livy/server/batch/BatchSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uLnNjYWxh) | `86.17% <0%> (+2.12%)` | `14% <0%> (ø)` | :arrow_down: | | [...la/org/apache/livy/utils/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9TcGFya1Byb2Nlc3NCdWlsZGVyLnNjYWxh) | `54.44% <0%> (+2.22%)` | `11% <0%> (+1%)` | :arrow_up: | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (+2.77%)` | `2% <0%> (ø)` | :arrow_down: | | [...e/livy/server/interactive/InteractiveSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvaW50ZXJhY3RpdmUvSW50ZXJhY3RpdmVTZXNzaW9uLnNjYWxh) | `69.11% <0%> (+3.66%)` | `44% <0%> (+2%)` | :arrow_up: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <0%> (+5%)` | `10% <0%> (ø)` | :arrow_down: | | [...ain/scala/org/apache/livy/server/JsonServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvSnNvblNlcnZsZXQuc2NhbGE=) | `38.46% <0%> (+5.76%)` | `18% <0%> (+4%)` | :arrow_up: | | [.../apache/livy/server/batch/CreateBatchRequest.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQ3JlYXRlQmF0Y2hSZXF1ZXN0LnNjYWxh) | `68.75% <0%> (+6.25%)` | `19% <0%> (+1%)` | :arrow_up: | | ... and [70 more](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/198?src=p
[GitHub] [incubator-livy] codecov-io edited a comment on issue #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase
codecov-io edited a comment on issue #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase URL: https://github.com/apache/incubator-livy/pull/198#issuecomment-521144705 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=h1) Report > Merging [#198](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/e7f23e06606ff79aec1c22e79a96f959cb89a8be?src=pr&el=desc) will **increase** coverage by `40.44%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #198 +/- ## = + Coverage 28.33% 68.77% +40.44% - Complexity 343 913 +570 = Files 100 100 Lines 5679 5679 Branches855 855 = + Hits 1609 3906 +2297 + Misses 3739 1216 -2523 - Partials331 557 +226 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `35.46% <0%> (+0.49%)` | `11% <0%> (ø)` | :arrow_down: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `95.87% <0%> (+1.03%)` | `21% <0%> (+3%)` | :arrow_up: | | [.../main/scala/org/apache/livy/server/WebServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvV2ViU2VydmVyLnNjYWxh) | `53.33% <0%> (+1.66%)` | `10% <0%> (+1%)` | :arrow_up: | | [...la/org/apache/livy/server/batch/BatchSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uLnNjYWxh) | `86.17% <0%> (+2.12%)` | `14% <0%> (ø)` | :arrow_down: | | [...la/org/apache/livy/utils/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9TcGFya1Byb2Nlc3NCdWlsZGVyLnNjYWxh) | `54.44% <0%> (+2.22%)` | `11% <0%> (+1%)` | :arrow_up: | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (+2.77%)` | `2% <0%> (ø)` | :arrow_down: | | [...e/livy/server/interactive/InteractiveSession.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvaW50ZXJhY3RpdmUvSW50ZXJhY3RpdmVTZXNzaW9uLnNjYWxh) | `69.11% <0%> (+3.66%)` | `44% <0%> (+2%)` | :arrow_up: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `80% <0%> (+5%)` | `10% <0%> (ø)` | :arrow_down: | | [...ain/scala/org/apache/livy/server/JsonServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvSnNvblNlcnZsZXQuc2NhbGE=) | `38.46% <0%> (+5.76%)` | `18% <0%> (+4%)` | :arrow_up: | | [.../apache/livy/server/batch/CreateBatchRequest.scala](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQ3JlYXRlQmF0Y2hSZXF1ZXN0LnNjYWxh) | `68.75% <0%> (+6.25%)` | `19% <0%> (+1%)` | :arrow_up: | | ... and [70 more](https://codecov.io/gh/apache/incubator-livy/pull/198/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/198?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/19
[GitHub] [incubator-livy] o-shevchenko edited a comment on issue #189: [LIVY-616] Livy Server discovery
o-shevchenko edited a comment on issue #189: [LIVY-616] Livy Server discovery URL: https://github.com/apache/incubator-livy/pull/189#issuecomment-517642960 Refactored configuration to provide a more flexible mechanism for configuration, simplified code ``` # Livy Server discovery # ZooKeeper quorum URLs, e.g. host1:port1,host2:port2 # livy.zookeeper.url = # Name of base Livy znode. Default livy # livy.zookeeper.namespace = livy # Name of Livy Server znode. Uses livy.zookeeper.namespace as parent. # By default will be /livy/server.uri # livy.server.zookeeper.namespace = server.uri # Number of trials to establish the connection to ZooKeeper quorum # livy.server.zookeeper.connection.max.retries = 3 # Sleep time between connection retries to ZooKeeper quorum # livy.server.zookeeper.connection.retry.interval.ms = 500 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] runzhiwang closed pull request #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase
runzhiwang closed pull request #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase URL: https://github.com/apache/incubator-livy/pull/198 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] runzhiwang opened a new pull request #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase
runzhiwang opened a new pull request #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase URL: https://github.com/apache/incubator-livy/pull/198 ## What changes were proposed in this pull request? Fix travis fail to build by excluding the dependency of hbase. ## How was this patch tested? existed UT and IT. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] yiheng commented on a change in pull request #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase
yiheng commented on a change in pull request #198: [LIVY-635] Fix travis fail to build by excluding the dependency of hbase URL: https://github.com/apache/incubator-livy/pull/198#discussion_r313731321 ## File path: thriftserver/server/pom.xml ## @@ -39,6 +39,10 @@ tomcat * + + org.apache.hbase Review comment: This dependency scope is not test. Not sure if this exclusion is safe This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
