subject:"\[GitHub\] spark pull request #18303\: \[SPARK\-19824\]\[Core\] Update JsonProtocol to keep c..."

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/18303


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread jiangxb1987

Github user jiangxb1987 commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122610140
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
--- End diff --

Good point, I'll remove this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596758
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
+   * `webuiaddress` the address used in web UI
+   * `cores` total cores of the worker
+   * `coresused` allocated cores of the worker
+   * `coresfree` free cores of the worker
+   * `memory` total memory of the worker
+   * `memoryused` allocated memory of the worker
+   * `memoryfree` free memory of the worker
+   * `state` state of the worker, see [[WorkerState]]
+   * `lastheartbeat` time in milliseconds that the latest heart 
beat message from the
+   * worker is received.
+   */
+  def writeWorkerInfo(obj: WorkerInfo): JObject = {
+("id" -> obj.id) ~
+("host" -> obj.host) ~
+("port" -> obj.port) ~
+("address" -> obj.hostPort) ~
+("webuiaddress" -> obj.webUiAddress) ~
+("cores" -> obj.cores) ~
+("coresused" -> obj.coresUsed) ~
+("coresfree" -> obj.coresFree) ~
+("memory" -> obj.memory) ~
+("memoryused" -> obj.memoryUsed) ~
+("memoryfree" -> obj.memoryFree) ~
+("state" -> obj.state.toString) ~
+("lastheartbeat" -> obj.lastHeartbeat)
+  }
 
+  /**
+   * Export the [[ApplicationInfo]] to a Json object, an 
[[ApplicationInfo]] consists of the
+   * information of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the application
+   * `starttime` time in milliseconds that the application starts
+   * `name` a name describes the application
+   * `cores` total cores granted to the application
+   * `user` name of the user who submitted the application
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `submitdate` time in Date that the application is submitted
+   * `state` state of the application, see [[ApplicationState]]
+   * `duration` time in milliseconds that the application has been 
running
+   */
   def writeApplicationInfo(obj: ApplicationInfo): JObject = {
 ("starttime" -> obj.startTime) ~
 ("id" -> obj.id) ~
 ("name" -> obj.desc.name) ~
-("cores" -> obj.desc.maxCores) ~
+("cores" -> obj.coresGranted) ~
 ("user" -> obj.desc.user) ~
 ("memoryperslave" -> obj.desc.memoryPerExecutorMB) ~
 ("submitdate" -> obj.submitDate.toString) ~
 ("state" -> obj.state.toString) ~
 ("duration" -> obj.duration)
   }
 
+  /**
+   * Export the [[ApplicationDescription]] to a Json object, an 
[[ApplicationDescription]] consists
+   * of the description of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `name` a name describes the application
+   * `cores` max cores can be allocated to the application, 0 
means unlimited
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `user` name of the user who submitted the application
+   * `command` the command string that submitted the application
+   */
   def writeApplicationDescription(obj: ApplicationDescription): JObject = {

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596741
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
+   * `webuiaddress` the address used in web UI
+   * `cores` total cores of the worker
+   * `coresused` allocated cores of the worker
+   * `coresfree` free cores of the worker
+   * `memory` total memory of the worker
+   * `memoryused` allocated memory of the worker
+   * `memoryfree` free memory of the worker
+   * `state` state of the worker, see [[WorkerState]]
+   * `lastheartbeat` time in milliseconds that the latest heart 
beat message from the
+   * worker is received.
+   */
+  def writeWorkerInfo(obj: WorkerInfo): JObject = {
+("id" -> obj.id) ~
+("host" -> obj.host) ~
+("port" -> obj.port) ~
+("address" -> obj.hostPort) ~
+("webuiaddress" -> obj.webUiAddress) ~
+("cores" -> obj.cores) ~
+("coresused" -> obj.coresUsed) ~
+("coresfree" -> obj.coresFree) ~
+("memory" -> obj.memory) ~
+("memoryused" -> obj.memoryUsed) ~
+("memoryfree" -> obj.memoryFree) ~
+("state" -> obj.state.toString) ~
+("lastheartbeat" -> obj.lastHeartbeat)
+  }
 
+  /**
+   * Export the [[ApplicationInfo]] to a Json object, an 
[[ApplicationInfo]] consists of the
+   * information of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the application
+   * `starttime` time in milliseconds that the application starts
+   * `name` a name describes the application
+   * `cores` total cores granted to the application
+   * `user` name of the user who submitted the application
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `submitdate` time in Date that the application is submitted
+   * `state` state of the application, see [[ApplicationState]]
+   * `duration` time in milliseconds that the application has been 
running
+   */
   def writeApplicationInfo(obj: ApplicationInfo): JObject = {
 ("starttime" -> obj.startTime) ~
 ("id" -> obj.id) ~
 ("name" -> obj.desc.name) ~
-("cores" -> obj.desc.maxCores) ~
+("cores" -> obj.coresGranted) ~
 ("user" -> obj.desc.user) ~
 ("memoryperslave" -> obj.desc.memoryPerExecutorMB) ~
 ("submitdate" -> obj.submitDate.toString) ~
 ("state" -> obj.state.toString) ~
 ("duration" -> obj.duration)
   }
 
+  /**
+   * Export the [[ApplicationDescription]] to a Json object, an 
[[ApplicationDescription]] consists
+   * of the description of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `name` a name describes the application
+   * `cores` max cores can be allocated to the application, 0 
means unlimited
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `user` name of the user who submitted the application
+   * `command` the command string that submitted the application
+   */
   def writeApplicationDescription(obj: ApplicationDescription): JObject = {

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596663
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
+   * `webuiaddress` the address used in web UI
+   * `cores` total cores of the worker
+   * `coresused` allocated cores of the worker
+   * `coresfree` free cores of the worker
+   * `memory` total memory of the worker
+   * `memoryused` allocated memory of the worker
+   * `memoryfree` free memory of the worker
+   * `state` state of the worker, see [[WorkerState]]
+   * `lastheartbeat` time in milliseconds that the latest heart 
beat message from the
+   * worker is received.
+   */
+  def writeWorkerInfo(obj: WorkerInfo): JObject = {
+("id" -> obj.id) ~
+("host" -> obj.host) ~
+("port" -> obj.port) ~
+("address" -> obj.hostPort) ~
+("webuiaddress" -> obj.webUiAddress) ~
+("cores" -> obj.cores) ~
+("coresused" -> obj.coresUsed) ~
+("coresfree" -> obj.coresFree) ~
+("memory" -> obj.memory) ~
+("memoryused" -> obj.memoryUsed) ~
+("memoryfree" -> obj.memoryFree) ~
+("state" -> obj.state.toString) ~
+("lastheartbeat" -> obj.lastHeartbeat)
+  }
 
+  /**
+   * Export the [[ApplicationInfo]] to a Json object, an 
[[ApplicationInfo]] consists of the
+   * information of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the application
+   * `starttime` time in milliseconds that the application starts
+   * `name` a name describes the application
+   * `cores` total cores granted to the application
+   * `user` name of the user who submitted the application
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `submitdate` time in Date that the application is submitted
+   * `state` state of the application, see [[ApplicationState]]
+   * `duration` time in milliseconds that the application has been 
running
+   */
   def writeApplicationInfo(obj: ApplicationInfo): JObject = {
 ("starttime" -> obj.startTime) ~
 ("id" -> obj.id) ~
 ("name" -> obj.desc.name) ~
-("cores" -> obj.desc.maxCores) ~
+("cores" -> obj.coresGranted) ~
 ("user" -> obj.desc.user) ~
 ("memoryperslave" -> obj.desc.memoryPerExecutorMB) ~
 ("submitdate" -> obj.submitDate.toString) ~
 ("state" -> obj.state.toString) ~
 ("duration" -> obj.duration)
   }
 
+  /**
+   * Export the [[ApplicationDescription]] to a Json object, an 
[[ApplicationDescription]] consists
+   * of the description of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `name` a name describes the application
+   * `cores` max cores can be allocated to the application, 0 
means unlimited
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `user` name of the user who submitted the application
+   * `command` the command string that submitted the application
+   */
   def writeApplicationDescription(obj: ApplicationDescription): JObject = {

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596651
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
+   * `webuiaddress` the address used in web UI
+   * `cores` total cores of the worker
+   * `coresused` allocated cores of the worker
+   * `coresfree` free cores of the worker
+   * `memory` total memory of the worker
+   * `memoryused` allocated memory of the worker
+   * `memoryfree` free memory of the worker
+   * `state` state of the worker, see [[WorkerState]]
+   * `lastheartbeat` time in milliseconds that the latest heart 
beat message from the
+   * worker is received.
+   */
+  def writeWorkerInfo(obj: WorkerInfo): JObject = {
+("id" -> obj.id) ~
+("host" -> obj.host) ~
+("port" -> obj.port) ~
+("address" -> obj.hostPort) ~
+("webuiaddress" -> obj.webUiAddress) ~
+("cores" -> obj.cores) ~
+("coresused" -> obj.coresUsed) ~
+("coresfree" -> obj.coresFree) ~
+("memory" -> obj.memory) ~
+("memoryused" -> obj.memoryUsed) ~
+("memoryfree" -> obj.memoryFree) ~
+("state" -> obj.state.toString) ~
+("lastheartbeat" -> obj.lastHeartbeat)
+  }
 
+  /**
+   * Export the [[ApplicationInfo]] to a Json object, an 
[[ApplicationInfo]] consists of the
+   * information of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the application
+   * `starttime` time in milliseconds that the application starts
+   * `name` a name describes the application
+   * `cores` total cores granted to the application
+   * `user` name of the user who submitted the application
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `submitdate` time in Date that the application is submitted
+   * `state` state of the application, see [[ApplicationState]]
+   * `duration` time in milliseconds that the application has been 
running
+   */
   def writeApplicationInfo(obj: ApplicationInfo): JObject = {
 ("starttime" -> obj.startTime) ~
 ("id" -> obj.id) ~
 ("name" -> obj.desc.name) ~
-("cores" -> obj.desc.maxCores) ~
+("cores" -> obj.coresGranted) ~
 ("user" -> obj.desc.user) ~
 ("memoryperslave" -> obj.desc.memoryPerExecutorMB) ~
 ("submitdate" -> obj.submitDate.toString) ~
 ("state" -> obj.state.toString) ~
 ("duration" -> obj.duration)
   }
 
+  /**
+   * Export the [[ApplicationDescription]] to a Json object, an 
[[ApplicationDescription]] consists
+   * of the description of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `name` a name describes the application
+   * `cores` max cores can be allocated to the application, 0 
means unlimited
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `user` name of the user who submitted the application
+   * `command` the command string that submitted the application
+   */
   def writeApplicationDescription(obj: ApplicationDescription): JObject = {

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596647
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
+   * `webuiaddress` the address used in web UI
+   * `cores` total cores of the worker
+   * `coresused` allocated cores of the worker
+   * `coresfree` free cores of the worker
+   * `memory` total memory of the worker
+   * `memoryused` allocated memory of the worker
+   * `memoryfree` free memory of the worker
+   * `state` state of the worker, see [[WorkerState]]
+   * `lastheartbeat` time in milliseconds that the latest heart 
beat message from the
+   * worker is received.
+   */
+  def writeWorkerInfo(obj: WorkerInfo): JObject = {
+("id" -> obj.id) ~
+("host" -> obj.host) ~
+("port" -> obj.port) ~
+("address" -> obj.hostPort) ~
+("webuiaddress" -> obj.webUiAddress) ~
+("cores" -> obj.cores) ~
+("coresused" -> obj.coresUsed) ~
+("coresfree" -> obj.coresFree) ~
+("memory" -> obj.memory) ~
+("memoryused" -> obj.memoryUsed) ~
+("memoryfree" -> obj.memoryFree) ~
+("state" -> obj.state.toString) ~
+("lastheartbeat" -> obj.lastHeartbeat)
+  }
 
+  /**
+   * Export the [[ApplicationInfo]] to a Json object, an 
[[ApplicationInfo]] consists of the
+   * information of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the application
+   * `starttime` time in milliseconds that the application starts
+   * `name` a name describes the application
+   * `cores` total cores granted to the application
+   * `user` name of the user who submitted the application
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `submitdate` time in Date that the application is submitted
+   * `state` state of the application, see [[ApplicationState]]
+   * `duration` time in milliseconds that the application has been 
running
+   */
   def writeApplicationInfo(obj: ApplicationInfo): JObject = {
 ("starttime" -> obj.startTime) ~
 ("id" -> obj.id) ~
 ("name" -> obj.desc.name) ~
-("cores" -> obj.desc.maxCores) ~
+("cores" -> obj.coresGranted) ~
 ("user" -> obj.desc.user) ~
 ("memoryperslave" -> obj.desc.memoryPerExecutorMB) ~
 ("submitdate" -> obj.submitDate.toString) ~
 ("state" -> obj.state.toString) ~
 ("duration" -> obj.duration)
   }
 
+  /**
+   * Export the [[ApplicationDescription]] to a Json object, an 
[[ApplicationDescription]] consists
+   * of the description of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `name` a name describes the application
+   * `cores` max cores can be allocated to the application, 0 
means unlimited
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `user` name of the user who submitted the application
+   * `command` the command string that submitted the application
+   */
   def writeApplicationDescription(obj: ApplicationDescription): JObject = {

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596621
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
+   * `webuiaddress` the address used in web UI
+   * `cores` total cores of the worker
+   * `coresused` allocated cores of the worker
+   * `coresfree` free cores of the worker
+   * `memory` total memory of the worker
+   * `memoryused` allocated memory of the worker
+   * `memoryfree` free memory of the worker
+   * `state` state of the worker, see [[WorkerState]]
+   * `lastheartbeat` time in milliseconds that the latest heart 
beat message from the
+   * worker is received.
+   */
+  def writeWorkerInfo(obj: WorkerInfo): JObject = {
+("id" -> obj.id) ~
+("host" -> obj.host) ~
+("port" -> obj.port) ~
+("address" -> obj.hostPort) ~
+("webuiaddress" -> obj.webUiAddress) ~
+("cores" -> obj.cores) ~
+("coresused" -> obj.coresUsed) ~
+("coresfree" -> obj.coresFree) ~
+("memory" -> obj.memory) ~
+("memoryused" -> obj.memoryUsed) ~
+("memoryfree" -> obj.memoryFree) ~
+("state" -> obj.state.toString) ~
+("lastheartbeat" -> obj.lastHeartbeat)
+  }
 
+  /**
+   * Export the [[ApplicationInfo]] to a Json object, an 
[[ApplicationInfo]] consists of the
+   * information of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the application
+   * `starttime` time in milliseconds that the application starts
+   * `name` a name describes the application
+   * `cores` total cores granted to the application
+   * `user` name of the user who submitted the application
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `submitdate` time in Date that the application is submitted
+   * `state` state of the application, see [[ApplicationState]]
+   * `duration` time in milliseconds that the application has been 
running
+   */
   def writeApplicationInfo(obj: ApplicationInfo): JObject = {
 ("starttime" -> obj.startTime) ~
 ("id" -> obj.id) ~
 ("name" -> obj.desc.name) ~
-("cores" -> obj.desc.maxCores) ~
+("cores" -> obj.coresGranted) ~
 ("user" -> obj.desc.user) ~
 ("memoryperslave" -> obj.desc.memoryPerExecutorMB) ~
 ("submitdate" -> obj.submitDate.toString) ~
 ("state" -> obj.state.toString) ~
 ("duration" -> obj.duration)
   }
 
+  /**
+   * Export the [[ApplicationDescription]] to a Json object, an 
[[ApplicationDescription]] consists
+   * of the description of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `name` a name describes the application
+   * `cores` max cores can be allocated to the application, 0 
means unlimited
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `user` name of the user who submitted the application
+   * `command` the command string that submitted the application
+   */
   def writeApplicationDescription(obj: ApplicationDescription): JObject = {

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596609
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
+   * `webuiaddress` the address used in web UI
+   * `cores` total cores of the worker
+   * `coresused` allocated cores of the worker
+   * `coresfree` free cores of the worker
+   * `memory` total memory of the worker
+   * `memoryused` allocated memory of the worker
+   * `memoryfree` free memory of the worker
+   * `state` state of the worker, see [[WorkerState]]
+   * `lastheartbeat` time in milliseconds that the latest heart 
beat message from the
+   * worker is received.
+   */
+  def writeWorkerInfo(obj: WorkerInfo): JObject = {
+("id" -> obj.id) ~
+("host" -> obj.host) ~
+("port" -> obj.port) ~
+("address" -> obj.hostPort) ~
+("webuiaddress" -> obj.webUiAddress) ~
+("cores" -> obj.cores) ~
+("coresused" -> obj.coresUsed) ~
+("coresfree" -> obj.coresFree) ~
+("memory" -> obj.memory) ~
+("memoryused" -> obj.memoryUsed) ~
+("memoryfree" -> obj.memoryFree) ~
+("state" -> obj.state.toString) ~
+("lastheartbeat" -> obj.lastHeartbeat)
+  }
 
+  /**
+   * Export the [[ApplicationInfo]] to a Json object, an 
[[ApplicationInfo]] consists of the
+   * information of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the application
+   * `starttime` time in milliseconds that the application starts
+   * `name` a name describes the application
+   * `cores` total cores granted to the application
+   * `user` name of the user who submitted the application
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `submitdate` time in Date that the application is submitted
+   * `state` state of the application, see [[ApplicationState]]
+   * `duration` time in milliseconds that the application has been 
running
+   */
   def writeApplicationInfo(obj: ApplicationInfo): JObject = {
 ("starttime" -> obj.startTime) ~
 ("id" -> obj.id) ~
 ("name" -> obj.desc.name) ~
-("cores" -> obj.desc.maxCores) ~
+("cores" -> obj.coresGranted) ~
 ("user" -> obj.desc.user) ~
 ("memoryperslave" -> obj.desc.memoryPerExecutorMB) ~
 ("submitdate" -> obj.submitDate.toString) ~
 ("state" -> obj.state.toString) ~
 ("duration" -> obj.duration)
   }
 
+  /**
+   * Export the [[ApplicationDescription]] to a Json object, an 
[[ApplicationDescription]] consists
+   * of the description of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `name` a name describes the application
+   * `cores` max cores can be allocated to the application, 0 
means unlimited
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `user` name of the user who submitted the application
+   * `command` the command string that submitted the application
+   */
   def writeApplicationDescription(obj: ApplicationDescription): JObject = {

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596599
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
+   * `webuiaddress` the address used in web UI
+   * `cores` total cores of the worker
+   * `coresused` allocated cores of the worker
+   * `coresfree` free cores of the worker
+   * `memory` total memory of the worker
+   * `memoryused` allocated memory of the worker
+   * `memoryfree` free memory of the worker
+   * `state` state of the worker, see [[WorkerState]]
+   * `lastheartbeat` time in milliseconds that the latest heart 
beat message from the
+   * worker is received.
+   */
+  def writeWorkerInfo(obj: WorkerInfo): JObject = {
+("id" -> obj.id) ~
+("host" -> obj.host) ~
+("port" -> obj.port) ~
+("address" -> obj.hostPort) ~
+("webuiaddress" -> obj.webUiAddress) ~
+("cores" -> obj.cores) ~
+("coresused" -> obj.coresUsed) ~
+("coresfree" -> obj.coresFree) ~
+("memory" -> obj.memory) ~
+("memoryused" -> obj.memoryUsed) ~
+("memoryfree" -> obj.memoryFree) ~
+("state" -> obj.state.toString) ~
+("lastheartbeat" -> obj.lastHeartbeat)
+  }
 
+  /**
+   * Export the [[ApplicationInfo]] to a Json object, an 
[[ApplicationInfo]] consists of the
+   * information of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the application
+   * `starttime` time in milliseconds that the application starts
+   * `name` a name describes the application
+   * `cores` total cores granted to the application
+   * `user` name of the user who submitted the application
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `submitdate` time in Date that the application is submitted
+   * `state` state of the application, see [[ApplicationState]]
+   * `duration` time in milliseconds that the application has been 
running
+   */
   def writeApplicationInfo(obj: ApplicationInfo): JObject = {
 ("starttime" -> obj.startTime) ~
 ("id" -> obj.id) ~
 ("name" -> obj.desc.name) ~
-("cores" -> obj.desc.maxCores) ~
+("cores" -> obj.coresGranted) ~
 ("user" -> obj.desc.user) ~
 ("memoryperslave" -> obj.desc.memoryPerExecutorMB) ~
 ("submitdate" -> obj.submitDate.toString) ~
 ("state" -> obj.state.toString) ~
 ("duration" -> obj.duration)
   }
 
+  /**
+   * Export the [[ApplicationDescription]] to a Json object, an 
[[ApplicationDescription]] consists
+   * of the description of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `name` a name describes the application
+   * `cores` max cores can be allocated to the application, 0 
means unlimited
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `user` name of the user who submitted the application
+   * `command` the command string that submitted the application
--- End diff --

`the command string that submitted` -> `the command submitted`

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596579
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
+   * `webuiaddress` the address used in web UI
+   * `cores` total cores of the worker
+   * `coresused` allocated cores of the worker
+   * `coresfree` free cores of the worker
+   * `memory` total memory of the worker
+   * `memoryused` allocated memory of the worker
+   * `memoryfree` free memory of the worker
+   * `state` state of the worker, see [[WorkerState]]
+   * `lastheartbeat` time in milliseconds that the latest heart 
beat message from the
+   * worker is received.
+   */
+  def writeWorkerInfo(obj: WorkerInfo): JObject = {
+("id" -> obj.id) ~
+("host" -> obj.host) ~
+("port" -> obj.port) ~
+("address" -> obj.hostPort) ~
+("webuiaddress" -> obj.webUiAddress) ~
+("cores" -> obj.cores) ~
+("coresused" -> obj.coresUsed) ~
+("coresfree" -> obj.coresFree) ~
+("memory" -> obj.memory) ~
+("memoryused" -> obj.memoryUsed) ~
+("memoryfree" -> obj.memoryFree) ~
+("state" -> obj.state.toString) ~
+("lastheartbeat" -> obj.lastHeartbeat)
+  }
 
+  /**
+   * Export the [[ApplicationInfo]] to a Json object, an 
[[ApplicationInfo]] consists of the
+   * information of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the application
+   * `starttime` time in milliseconds that the application starts
+   * `name` a name describes the application
+   * `cores` total cores granted to the application
+   * `user` name of the user who submitted the application
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `submitdate` time in Date that the application is submitted
+   * `state` state of the application, see [[ApplicationState]]
+   * `duration` time in milliseconds that the application has been 
running
+   */
   def writeApplicationInfo(obj: ApplicationInfo): JObject = {
 ("starttime" -> obj.startTime) ~
 ("id" -> obj.id) ~
 ("name" -> obj.desc.name) ~
-("cores" -> obj.desc.maxCores) ~
+("cores" -> obj.coresGranted) ~
 ("user" -> obj.desc.user) ~
 ("memoryperslave" -> obj.desc.memoryPerExecutorMB) ~
 ("submitdate" -> obj.submitDate.toString) ~
 ("state" -> obj.state.toString) ~
 ("duration" -> obj.duration)
   }
 
+  /**
+   * Export the [[ApplicationDescription]] to a Json object, an 
[[ApplicationDescription]] consists
+   * of the description of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `name` a name describes the application
--- End diff --

-> the description of the application


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596590
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
+   * `webuiaddress` the address used in web UI
+   * `cores` total cores of the worker
+   * `coresused` allocated cores of the worker
+   * `coresfree` free cores of the worker
+   * `memory` total memory of the worker
+   * `memoryused` allocated memory of the worker
+   * `memoryfree` free memory of the worker
+   * `state` state of the worker, see [[WorkerState]]
+   * `lastheartbeat` time in milliseconds that the latest heart 
beat message from the
+   * worker is received.
+   */
+  def writeWorkerInfo(obj: WorkerInfo): JObject = {
+("id" -> obj.id) ~
+("host" -> obj.host) ~
+("port" -> obj.port) ~
+("address" -> obj.hostPort) ~
+("webuiaddress" -> obj.webUiAddress) ~
+("cores" -> obj.cores) ~
+("coresused" -> obj.coresUsed) ~
+("coresfree" -> obj.coresFree) ~
+("memory" -> obj.memory) ~
+("memoryused" -> obj.memoryUsed) ~
+("memoryfree" -> obj.memoryFree) ~
+("state" -> obj.state.toString) ~
+("lastheartbeat" -> obj.lastHeartbeat)
+  }
 
+  /**
+   * Export the [[ApplicationInfo]] to a Json object, an 
[[ApplicationInfo]] consists of the
+   * information of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the application
+   * `starttime` time in milliseconds that the application starts
+   * `name` a name describes the application
+   * `cores` total cores granted to the application
+   * `user` name of the user who submitted the application
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `submitdate` time in Date that the application is submitted
+   * `state` state of the application, see [[ApplicationState]]
+   * `duration` time in milliseconds that the application has been 
running
+   */
   def writeApplicationInfo(obj: ApplicationInfo): JObject = {
 ("starttime" -> obj.startTime) ~
 ("id" -> obj.id) ~
 ("name" -> obj.desc.name) ~
-("cores" -> obj.desc.maxCores) ~
+("cores" -> obj.coresGranted) ~
 ("user" -> obj.desc.user) ~
 ("memoryperslave" -> obj.desc.memoryPerExecutorMB) ~
 ("submitdate" -> obj.submitDate.toString) ~
 ("state" -> obj.state.toString) ~
 ("duration" -> obj.duration)
   }
 
+  /**
+   * Export the [[ApplicationDescription]] to a Json object, an 
[[ApplicationDescription]] consists
+   * of the description of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `name` a name describes the application
+   * `cores` max cores can be allocated to the application, 0 
means unlimited
--- End diff --

`max cores can be ` -> `max cores that can be  `


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596570
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
+   * `webuiaddress` the address used in web UI
+   * `cores` total cores of the worker
+   * `coresused` allocated cores of the worker
+   * `coresfree` free cores of the worker
+   * `memory` total memory of the worker
+   * `memoryused` allocated memory of the worker
+   * `memoryfree` free memory of the worker
+   * `state` state of the worker, see [[WorkerState]]
+   * `lastheartbeat` time in milliseconds that the latest heart 
beat message from the
+   * worker is received.
+   */
+  def writeWorkerInfo(obj: WorkerInfo): JObject = {
+("id" -> obj.id) ~
+("host" -> obj.host) ~
+("port" -> obj.port) ~
+("address" -> obj.hostPort) ~
+("webuiaddress" -> obj.webUiAddress) ~
+("cores" -> obj.cores) ~
+("coresused" -> obj.coresUsed) ~
+("coresfree" -> obj.coresFree) ~
+("memory" -> obj.memory) ~
+("memoryused" -> obj.memoryUsed) ~
+("memoryfree" -> obj.memoryFree) ~
+("state" -> obj.state.toString) ~
+("lastheartbeat" -> obj.lastHeartbeat)
+  }
 
+  /**
+   * Export the [[ApplicationInfo]] to a Json object, an 
[[ApplicationInfo]] consists of the
+   * information of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the application
+   * `starttime` time in milliseconds that the application starts
+   * `name` a name describes the application
+   * `cores` total cores granted to the application
+   * `user` name of the user who submitted the application
+   * `memoryperslave` minimal memory in MB required to each 
executor
+   * `submitdate` time in Date that the application is submitted
+   * `state` state of the application, see [[ApplicationState]]
+   * `duration` time in milliseconds that the application has been 
running
+   */
   def writeApplicationInfo(obj: ApplicationInfo): JObject = {
 ("starttime" -> obj.startTime) ~
 ("id" -> obj.id) ~
 ("name" -> obj.desc.name) ~
-("cores" -> obj.desc.maxCores) ~
+("cores" -> obj.coresGranted) ~
 ("user" -> obj.desc.user) ~
 ("memoryperslave" -> obj.desc.memoryPerExecutorMB) ~
 ("submitdate" -> obj.submitDate.toString) ~
 ("state" -> obj.state.toString) ~
 ("duration" -> obj.duration)
   }
 
+  /**
+   * Export the [[ApplicationDescription]] to a Json object, an 
[[ApplicationDescription]] consists
--- End diff --

`, an` -> `. An`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596562
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
+   * `webuiaddress` the address used in web UI
+   * `cores` total cores of the worker
+   * `coresused` allocated cores of the worker
+   * `coresfree` free cores of the worker
+   * `memory` total memory of the worker
+   * `memoryused` allocated memory of the worker
+   * `memoryfree` free memory of the worker
+   * `state` state of the worker, see [[WorkerState]]
+   * `lastheartbeat` time in milliseconds that the latest heart 
beat message from the
+   * worker is received.
+   */
+  def writeWorkerInfo(obj: WorkerInfo): JObject = {
+("id" -> obj.id) ~
+("host" -> obj.host) ~
+("port" -> obj.port) ~
+("address" -> obj.hostPort) ~
+("webuiaddress" -> obj.webUiAddress) ~
+("cores" -> obj.cores) ~
+("coresused" -> obj.coresUsed) ~
+("coresfree" -> obj.coresFree) ~
+("memory" -> obj.memory) ~
+("memoryused" -> obj.memoryUsed) ~
+("memoryfree" -> obj.memoryFree) ~
+("state" -> obj.state.toString) ~
+("lastheartbeat" -> obj.lastHeartbeat)
+  }
 
+  /**
+   * Export the [[ApplicationInfo]] to a Json object, an 
[[ApplicationInfo]] consists of the
--- End diff --

`, an` -> `. An`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596533
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
+   * `webuiaddress` the address used in web UI
+   * `cores` total cores of the worker
+   * `coresused` allocated cores of the worker
+   * `coresfree` free cores of the worker
+   * `memory` total memory of the worker
+   * `memoryused` allocated memory of the worker
+   * `memoryfree` free memory of the worker
+   * `state` state of the worker, see [[WorkerState]]
+   * `lastheartbeat` time in milliseconds that the latest heart 
beat message from the
+   * worker is received.
+   */
+  def writeWorkerInfo(obj: WorkerInfo): JObject = {
+("id" -> obj.id) ~
+("host" -> obj.host) ~
+("port" -> obj.port) ~
+("address" -> obj.hostPort) ~
+("webuiaddress" -> obj.webUiAddress) ~
+("cores" -> obj.cores) ~
+("coresused" -> obj.coresUsed) ~
+("coresfree" -> obj.coresFree) ~
+("memory" -> obj.memory) ~
+("memoryused" -> obj.memoryUsed) ~
+("memoryfree" -> obj.memoryFree) ~
+("state" -> obj.state.toString) ~
+("lastheartbeat" -> obj.lastHeartbeat)
+  }
 
+  /**
+   * Export the [[ApplicationInfo]] to a Json object, an 
[[ApplicationInfo]] consists of the
+   * information of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the application
+   * `starttime` time in milliseconds that the application starts
+   * `name` a name describes the application
--- End diff --

-> the description of the application


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596518
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
+   * `webuiaddress` the address used in web UI
+   * `cores` total cores of the worker
+   * `coresused` allocated cores of the worker
+   * `coresfree` free cores of the worker
+   * `memory` total memory of the worker
+   * `memoryused` allocated memory of the worker
+   * `memoryfree` free memory of the worker
+   * `state` state of the worker, see [[WorkerState]]
+   * `lastheartbeat` time in milliseconds that the latest heart 
beat message from the
+   * worker is received.
+   */
+  def writeWorkerInfo(obj: WorkerInfo): JObject = {
+("id" -> obj.id) ~
+("host" -> obj.host) ~
+("port" -> obj.port) ~
+("address" -> obj.hostPort) ~
+("webuiaddress" -> obj.webUiAddress) ~
+("cores" -> obj.cores) ~
+("coresused" -> obj.coresUsed) ~
+("coresfree" -> obj.coresFree) ~
+("memory" -> obj.memory) ~
+("memoryused" -> obj.memoryUsed) ~
+("memoryfree" -> obj.memoryFree) ~
+("state" -> obj.state.toString) ~
+("lastheartbeat" -> obj.lastHeartbeat)
+  }
 
+  /**
+   * Export the [[ApplicationInfo]] to a Json object, an 
[[ApplicationInfo]] consists of the
+   * information of an application.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the application
+   * `starttime` time in milliseconds that the application starts
--- End diff --

Switch the order?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596434
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
+   * worker.
+   *
+   * @return a Json object containing the following fields:
+   * `id` a string identifier of the worker
+   * `host` the host that the worker is running on
+   * `port` the port that the worker is bound to
+   * `address` ${host}:${port}
--- End diff --

If we already has host and port, why we still need the `address`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122596343
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
 import org.json4s.JsonDSL._
 
 import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, 
WorkerStateResponse}
-import org.apache.spark.deploy.master.{ApplicationInfo, DriverInfo, 
WorkerInfo}
+import org.apache.spark.deploy.master._
+import org.apache.spark.deploy.master.RecoveryState.MasterState
 import org.apache.spark.deploy.worker.ExecutorRunner
 
 private[deploy] object JsonProtocol {
- def writeWorkerInfo(obj: WorkerInfo): JObject = {
-   ("id" -> obj.id) ~
-   ("host" -> obj.host) ~
-   ("port" -> obj.port) ~
-   ("webuiaddress" -> obj.webUiAddress) ~
-   ("cores" -> obj.cores) ~
-   ("coresused" -> obj.coresUsed) ~
-   ("coresfree" -> obj.coresFree) ~
-   ("memory" -> obj.memory) ~
-   ("memoryused" -> obj.memoryUsed) ~
-   ("memoryfree" -> obj.memoryFree) ~
-   ("state" -> obj.state.toString) ~
-   ("lastheartbeat" -> obj.lastHeartbeat)
- }
+  /**
+   * Export the [[WorkerInfo]] to a Json object, a [[WorkerInfo]] consists 
of the information of a
--- End diff --

`, a` -> `. A`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-16 Thread jiangxb1987

Github user jiangxb1987 commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122384185
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -54,7 +55,7 @@ private[deploy] object JsonProtocol {
 
   def writeApplicationDescription(obj: ApplicationDescription): JObject = {
--- End diff --

Let me do this at the weekend in this PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-16 Thread gatorsmile

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/18303#discussion_r122377229
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -54,7 +55,7 @@ private[deploy] object JsonProtocol {
 
   def writeApplicationDescription(obj: ApplicationDescription): JObject = {
--- End diff --

IMO, we need to add the function descriptions for these public functions 
and provide a clear definition for each field. How about adding a few JIRAs and 
ask the open source community to do it?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-14 Thread jiangxb1987

GitHub user jiangxb1987 opened a pull request:

https://github.com/apache/spark/pull/18303

[SPARK-19824][Core] Update JsonProtocol to keep consistent with the UI

## What changes were proposed in this pull request?

Fix any inconsistent part in JsonProtocol with the UI.
This PR also contains the modifications in #17181 

## How was this patch tested?

Updated JsonProtocolSuite.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jiangxb1987/spark json-protocol

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/18303.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #18303


commit 244bbae71c2aa0b9f173ad7ac16ad0440eaab99c
Author: Xingbo Jiang 
Date:   2017-06-14T14:55:16Z

update JsonProtocol




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

21 matches

Site Navigation

Mail list logo

Footer information