[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2925 ---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2925#discussion_r241802156
--- Diff:
integration/spark2/src/main/scala/org/apache/carbondata/spark/thriftserver/CarbonThriftServer.scala
---
@@ -48,8 +48,13 @@ object CarbonThriftServer {
System.exit(0)
}
+val master = Option(System.getProperty("spark.master"))
--- End diff --
ok, done
---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2925#discussion_r241802106 --- Diff: integration/spark2/pom.xml --- @@ -134,6 +134,11 @@ + --- End diff -- ok, done ---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user jackylk commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2925#discussion_r241774500 --- Diff: integration/spark2/pom.xml --- @@ -134,6 +134,11 @@ + --- End diff -- I see `org.apache.httpcomponents` is already introduced in other modules, please unify the version for this jar in all places ---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user jackylk commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2925#discussion_r241736440 --- Diff: integration/spark2/pom.xml --- @@ -134,6 +134,11 @@ + --- End diff -- I have never encounter this problem before, can you describe the scenario in more detail, in what case NoClassDefFoundError will be thrown? ---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2925#discussion_r241736214
--- Diff:
integration/spark2/src/main/scala/org/apache/carbondata/spark/thriftserver/CarbonThriftServer.scala
---
@@ -48,8 +48,13 @@ object CarbonThriftServer {
System.exit(0)
}
+val master = Option(System.getProperty("spark.master"))
--- End diff --
I think better to let user give --master local, user can control this.
---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2925#discussion_r235027703
--- Diff:
examples/spark2/src/main/scala/org/apache/carbondata/examples/S3Example.scala
---
@@ -157,8 +157,8 @@ object S3Example {
}
def getSparkMaster(args: Array[String]): String = {
- if (args.length == 5) args(4)
- else if (args(3).contains("spark:") || args(3).contains("mesos:"))
args(3)
- else "local"
-}
--- End diff --
the format is incorrect. we should optimize it.
```def getS3EndPoint(args: Array[String]): String = { def
getS3EndPoint(args: Array[String]): String = {
if (args.length >= 4 && args(3).contains(".com")) args(3)
if (args.length >= 4 && args(3).contains(".com")) args(3)
else "" else ""
}```
---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user akashrn5 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2925#discussion_r235015512
--- Diff:
examples/spark2/src/main/scala/org/apache/carbondata/examples/S3Example.scala
---
@@ -157,8 +157,8 @@ object S3Example {
}
def getSparkMaster(args: Array[String]): String = {
- if (args.length == 5) args(4)
- else if (args(3).contains("spark:") || args(3).contains("mesos:"))
args(3)
- else "local"
-}
--- End diff --
revert the unnecessary changes
---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2925#discussion_r234982825 --- Diff: integration/spark2/pom.xml --- @@ -134,6 +134,11 @@ + --- End diff -- I meet more than two times. This issue also happen in spark on k8s before. You can try in original cluster. It's not convenient for user to add httpclient jars to you cluster seperately. What's effect if we add this dependency? ---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2925#discussion_r234962295
--- Diff:
integration/spark2/src/main/scala/org/apache/carbondata/spark/thriftserver/CarbonThriftServer.scala
---
@@ -48,8 +48,13 @@ object CarbonThriftServer {
System.exit(0)
}
+val master = Option(System.getProperty("spark.master"))
--- End diff --
ok
---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user kunal642 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2925#discussion_r234962085 --- Diff: integration/spark2/pom.xml --- @@ -134,6 +134,11 @@ + --- End diff -- But nobody else is facing this issue. And also none of our environments are getting this exception. Maybe u need to add httpclient jars to you cluster seperately? ---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2925#discussion_r234541941
--- Diff:
integration/spark2/src/main/scala/org/apache/carbondata/spark/thriftserver/CarbonThriftServer.scala
---
@@ -48,8 +48,13 @@ object CarbonThriftServer {
System.exit(0)
}
+val master = Option(System.getProperty("spark.master"))
--- End diff --
No, it's support CarbonThriftServer run in local/cluster, which convenient
for users. Before if user want to run CarbonThriftServer in local, user need
add master("local")
---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2925#discussion_r234541398
--- Diff: integration/spark2/pom.xml ---
@@ -134,6 +134,11 @@
+
--- End diff --
Because when I use
org.apache.carbondata.spark.thriftserver.CarbonThriftServe and use beeline to
connect, it throw NoClassDefFoundErrorãNoClassDefFoundError:
org/apache/http/impl/conn/PoolingClientConnectionManagerã
PoolingClientConnectionManager is belong to httpclient, so I add:
org.apache.httpcomponents
httpclient
${httpclient.version}
---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user kunal642 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2925#discussion_r234191527 --- Diff: integration/spark2/pom.xml --- @@ -134,6 +134,11 @@ + --- End diff -- Why is this dependency needed in spark2? ---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2925#discussion_r234191597
--- Diff:
integration/spark2/src/main/scala/org/apache/carbondata/spark/thriftserver/CarbonThriftServer.scala
---
@@ -48,8 +48,13 @@ object CarbonThriftServer {
System.exit(0)
}
+val master = Option(System.getProperty("spark.master"))
--- End diff --
Was this the reason for NoClassDefFoundError?
---
[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...
GitHub user xubo245 opened a pull request: https://github.com/apache/carbondata/pull/2925 [CARBONDATA-3102] Fix NoClassDefFoundError when use thriftServer and beeline Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? No - [ ] Any backward compatibility impacted? No - [ ] Document update required? No - [ ] Testing done No - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. No You can merge this pull request into a Git repository by running: $ git pull https://github.com/xubo245/carbondata CARBONDATA-3102_fixBeelineThriftServerError Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2925.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2925 commit 86201f124c8857d70d50730fa33d93cd250d6d59 Author: xubo245 Date: 2018-11-15T12:36:12Z [CARBONDATA-3102] Fix NoClassDefFoundError when use thriftServer and beeline ---
