[GitHub] incubator-carbondata issue #765: [CARBONDATA-887]lazy rdd iterator for Inser...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/765
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1530/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #751: [CARBONDATA-816] Added Example for Hive Int...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/751
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1529/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (CARBONDATA-891) Fix compilation issue of AlterTableValidationTestCase generate new folder "carbon.store"

[Liang Chen (JIRA)]

Liang Chen created CARBONDATA-891:
-

 Summary: Fix compilation issue of AlterTableValidationTestCase 
generate new folder "carbon.store"
 Key: CARBONDATA-891
 URL: https://issues.apache.org/jira/browse/CARBONDATA-891
 Project: CarbonData
  Issue Type: Bug
  Components: build, core
Reporter: Liang Chen
Assignee: Liang Chen
 Fix For: 1.1.0-incubating


Fix compilation issue of AlterTableValidationTestCase generate new folder 
"carbon.store"



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata pull request #751: [CARBONDATA-816] Added Example for H...

[anubhav100]

Github user anubhav100 commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/751#discussion_r110546718
  
--- Diff: 
integration/hive/src/main/scala/org/apache/carbondata/hiveexample/HiveExample.scala
 ---
@@ -0,0 +1,166 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.hiveexample
+
+import java.io.File
+import java.sql.{DriverManager, ResultSet, SQLException, Statement}
+
+import org.apache.spark.sql.SparkSession
+
+import org.apache.carbondata.common.logging.LogServiceFactory
+import org.apache.carbondata.hive.server.HiveEmbeddedServer2
+
+object HiveExample {
+
+  private val driverName: String = "org.apache.hive.jdbc.HiveDriver"
+
+  /**
+   * @param args
+   * @throws SQLException
+   */
+  @throws[SQLException]
+  def main(args: Array[String]) {
+val rootPath = new File(this.getClass.getResource("/").getPath
++ "../../../..").getCanonicalPath
+val store = s"$rootPath/integration/hive/target/store"
+val warehouse = s"$rootPath/integration/hive/target/warehouse"
+val metaStore_Db = 
s"$rootPath/integration/hive/target/carbon_metaStore_db"
+val logger = 
LogServiceFactory.getLogService(this.getClass.getCanonicalName)
+
+import org.apache.spark.sql.CarbonSession._
+
+System.setProperty("hadoop.home.dir", "/")
+
+val carbon = SparkSession
+  .builder()
+  .master("local")
+  .appName("HiveExample")
+  .config("carbon.sql.warehouse.dir", warehouse).enableHiveSupport()
+  .getOrCreateCarbonSession(
+store, metaStore_Db)
+
+val carbonJarPath = 
s"$rootPath/assembly/target/scala-2.11/carbondata_2.11-1.1" +
+s".0-incubating-SNAPSHOT-shade-hadoop2.7.2.jar"
--- End diff --

@chenliang613  Wildcards are not supported in both hive.aux.jars.path and 
add jar.

i solved this problem by specifying both version of hadoop

val carbonHadoopJarPath = 
s"$rootPath/assembly/target/scala-2.11/carbondata_2.11-1.1" +
  ".0-incubating-SNAPSHOT-shade-hadoop2.7.2.jar"

val carbon_DefaultHadoopVersion_JarPath =
  s"$rootPath/assembly/target/scala-2.11/carbondata_2.11-1.1" +
  ".0-incubating-SNAPSHOT-shade-hadoop2.2.0.jar"

first i tried to add jar with 2.7.2 version if it fails then i tried with 
hadoop 2.2.0 version

 try {
  stmt
.execute(s"ADD JAR $carbonHadoopJarPath")
}
catch {
  case exception: Exception =>
logger.warn(s"Jar Not Found $carbonHadoopJarPath"+"Looking For 
hadoop 2.2.0 version jar")
try {
  stmt
.execute(s"ADD JAR $carbon_DefaultHadoopVersion_JarPath")
}
catch {
  case exception: Exception => logger
.error(s"Exception Occurs:Neither One of Jar is Found 
$carbon_DefaultHadoopVersion_JarPath,$carbonHadoopJarPath"+"Atleast One Should 
Be Build")
hiveEmbeddedServer2.stop()
}
}


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (CARBONDATA-884) [Documentation] information on assembly jar to be provided in Quick Start

[Liang Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liang Chen resolved CARBONDATA-884.
---
   Resolution: Fixed
Fix Version/s: 1.1.0-incubating

> [Documentation] information on assembly jar to be provided in Quick Start
> -
>
> Key: CARBONDATA-884
> URL: https://issues.apache.org/jira/browse/CARBONDATA-884
> Project: CarbonData
>  Issue Type: Improvement
>Reporter: Gururaj Shetty
>Assignee: Sanoj MG
>Priority: Minor
> Fix For: 1.1.0-incubating
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> In Quick start we have mentioned the below command:
> Start Spark shell by running the following command in the Spark directory:
> ./bin/spark-shell --jars 
> It is better to mention for the user from where to take the assembly jar.
> For example: the assembly jar will be present in the target folder when you 
> build the project.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata pull request #770: [CARBONDATA-884] Assembly jar inform...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-carbondata/pull/770


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #770: [CARBONDATA-884] Assembly jar information i...

[chenliang613]

Github user chenliang613 commented on the issue:

https://github.com/apache/incubator-carbondata/pull/770
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (CARBONDATA-890) For Spark 2.1 LRU cache size at driver is getting configured with the executor lru cache size.

[Mohammad Shahid Khan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mohammad Shahid Khan reassigned CARBONDATA-890:
---

Assignee: Mohammad Shahid Khan

> For Spark 2.1 LRU cache size at driver is getting configured with the 
> executor lru cache size.
> --
>
> Key: CARBONDATA-890
> URL: https://issues.apache.org/jira/browse/CARBONDATA-890
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Mohammad Shahid Khan
>Assignee: Mohammad Shahid Khan
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CARBONDATA-890) For Spark 2.1 LRU cache size at driver is getting configured with the executor lru cache size.

[Mohammad Shahid Khan (JIRA)]

Mohammad Shahid Khan created CARBONDATA-890:
---

 Summary: For Spark 2.1 LRU cache size at driver is getting 
configured with the executor lru cache size.
 Key: CARBONDATA-890
 URL: https://issues.apache.org/jira/browse/CARBONDATA-890
 Project: CarbonData
  Issue Type: Bug
Reporter: Mohammad Shahid Khan






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata issue #730: [WIP]NullPointerException is getting thrown...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/730
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1528/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #771: [CARBONDATA-883] Fixed cast exception for i...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/771
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1527/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #771: [CARBONDATA-883] Fixed cast exceptio...

[kunal642]

GitHub user kunal642 opened a pull request:

https://github.com/apache/incubator-carbondata/pull/771

[CARBONDATA-883] Fixed cast exception for integer data types in 
RestructureBasedVectorResultCollector

The following issues are fixed in this PR:-
1. In RestructureUtil, int and short datatypes were being parsed as long 
and then being cast to long which was throwing cast exception.

2. Added an entry for SHORT in carbonKeywords because we were allowing the 
user to create a table with short datatype but alter table was failing due to 
missing SHORT keyword.
Now short and smallInt are both acceptable and both will be converted to 
smallint.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kunal642/incubator-carbondata CARBONDATA-883

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/771.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #771


commit 9db005af3022b2221eb8af1077f14a62075a76c3
Author: kunal642 
Date:   2017-04-07T13:26:16Z

fixed cast exception for integer data types in 
RestructureBasedVectorResultCollector

commit 0be853c83a2ce81256a0c54c5fc9d27db6e8c9da
Author: kunal642 
Date:   2017-04-09T08:45:24Z

added parser for short dataype




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #730: [WIP]NullPointerException is getting thrown...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/730
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1526/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #770: [CARBONDATA-884] Assembly jar information i...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/770
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1525/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #730: [WIP]NullPointerException is getting thrown...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/730
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1524/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #770: [CARBONDATA-884] Assembly jar inform...

[sanoj-mg]

GitHub user sanoj-mg opened a pull request:

https://github.com/apache/incubator-carbondata/pull/770

[CARBONDATA-884] Assembly jar information in Quick Start guide

Documentation : Added details of assembly jar in Quick Start guide  




You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sanoj-mg/incubator-carbondata 
CARBONDATA-884-quickstart-assembly-jar

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/770.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #770


commit c3af07efc2484d07b12d27d2d2b7adfa002c6f47
Author: Sanoj MG 
Date:   2017-04-09T08:30:41Z

Added assembly jar information in Quick Start guide




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (CARBONDATA-884) [Documentation] information on assembly jar to be provided in Quick Start

[Sanoj MG (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sanoj MG reassigned CARBONDATA-884:
---

Assignee: Sanoj MG

> [Documentation] information on assembly jar to be provided in Quick Start
> -
>
> Key: CARBONDATA-884
> URL: https://issues.apache.org/jira/browse/CARBONDATA-884
> Project: CarbonData
>  Issue Type: Improvement
>Reporter: Gururaj Shetty
>Assignee: Sanoj MG
>Priority: Minor
>
> In Quick start we have mentioned the below command:
> Start Spark shell by running the following command in the Spark directory:
> ./bin/spark-shell --jars 
> It is better to mention for the user from where to take the assembly jar.
> For example: the assembly jar will be present in the target folder when you 
> build the project.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata issue #769: [CARBONDATA-888] Added include and exclude ...

[sanoj-mg]

Github user sanoj-mg commented on the issue:

https://github.com/apache/incubator-carbondata/pull/769
  
@jackylk I have changed test cases. Can you please review this pr. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #769: [CARBONDATA-888] Added include and exclude ...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/769
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1523/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #767: fix sort_columns issue(12-dev)

[jackylk]

Github user jackylk commented on the issue:

https://github.com/apache/incubator-carbondata/pull/767
  
please rebase


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (CARBONDATA-854) Carbondata with Datastax / Cassandra

[Sanoj MG (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sanoj MG reassigned CARBONDATA-854:
---

Assignee: Sanoj MG

> Carbondata with Datastax / Cassandra
> 
>
> Key: CARBONDATA-854
> URL: https://issues.apache.org/jira/browse/CARBONDATA-854
> Project: CarbonData
>  Issue Type: Improvement
>  Components: spark-integration
>Affects Versions: 1.1.0-incubating
> Environment: Datastax DSE 5.0 ( DSE analytics )
>Reporter: Sanoj MG
>Assignee: Sanoj MG
>Priority: Minor
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> I am trying to get Carbondata working in a Datastax DSE 5.0 cluster. 
> An exception is thrown while trying to create Carbondata table from spark 
> shell. Below are the steps: 
> scala> import com.datastax.spark.connector._
> scala> import org.apache.spark.sql.SaveMode
> scala> import org.apache.spark.sql.CarbonContext
> scala> import org.apache.spark.sql.types._
> scala> val cc = new CarbonContext(sc, "cfs://127.0.0.1/opt/CarbonStore")
> scala> val df = 
> cc.read.parquet("file:///home/cassandra/testdata-30day/cassandra/zone.parquet")
> scala> df.write.format("carbondata").option("tableName", 
> "zone").option("compress", 
> "true").option("TempCSV","false").mode(SaveMode.Overwrite).save()
> Below exception is thrown and it fails to create carbondata table. 
> java.io.FileNotFoundException: /opt/CarbonStore/default/zone/Metadata/schema 
> (No such file or directory)
> at java.io.FileOutputStream.open0(Native Method)
> at java.io.FileOutputStream.open(FileOutputStream.java:270)
> at java.io.FileOutputStream.(FileOutputStream.java:213)
> at java.io.FileOutputStream.(FileOutputStream.java:133)
> at 
> org.apache.carbondata.core.datastore.impl.FileFactory.getDataOutputStream(FileFactory.java:207)
> at 
> org.apache.carbondata.core.writer.ThriftWriter.open(ThriftWriter.java:84)
> at 
> org.apache.spark.sql.hive.CarbonMetastore.createTableFromThrift(CarbonMetastore.scala:293)
> at 
> org.apache.spark.sql.execution.command.CreateTable.run(carbonTableSchema.scala:163)
> at 
> org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:58)
> at 
> org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:56)
> at 
> org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:70)
> at 
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132)
> at 
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130)
> at 
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
> at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130)
> at 
> org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:55)
> at 
> org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:55)
> at org.apache.spark.sql.DataFrame.(DataFrame.scala:145)
> at org.apache.spark.sql.DataFrame.(DataFrame.scala:130)
> at org.apache.spark.sql.CarbonContext.sql(CarbonContext.scala:139)
> at 
> org.apache.carbondata.spark.CarbonDataFrameWriter.saveAsCarbonFile(CarbonDataFrameWriter.scala:39)
> at 
> org.apache.spark.sql.CarbonSource.createRelation(CarbonDatasourceRelation.scala:109)
> at 
> org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:222)
> at 
> org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:148)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata issue #767: fix sort_columns issue(12-dev)

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/767
  
Build Failed  with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1522/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (CARBONDATA-836) Error in load using dataframe - columns containing comma

[Sanoj MG (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sanoj MG reassigned CARBONDATA-836:
---

Assignee: Sanoj MG

> Error in load using dataframe  - columns containing comma
> -
>
> Key: CARBONDATA-836
> URL: https://issues.apache.org/jira/browse/CARBONDATA-836
> Project: CarbonData
>  Issue Type: Bug
>  Components: spark-integration
>Affects Versions: 1.1.0-incubating
> Environment: HDP sandbox 2.5, Spark 1.6.2
>Reporter: Sanoj MG
>Assignee: Sanoj MG
>Priority: Minor
> Fix For: NONE
>
>
> While trying to load data into Carabondata table using dataframe, the columns 
> containing commas are not properly loaded. 
> Eg: 
> scala> df.show(false)
> +---+--+---++-+--+
> |Country|Branch|Name   |Address |ShortName|Status|
> +---+--+---++-+--+
> |2  |1 |Main Branch|, Dubai, UAE|UHO  |256   |
> +---+--+---++-+--+
> scala>  df.write.format("carbondata").option("tableName", 
> "Branch1").option("compress", "true").mode(SaveMode.Overwrite).save()
> scala> cc.sql("select * from branch1").show(false)
> +---+--+---+---+-+--+
> |country|branch|name   |address|shortname|status|
> +---+--+---+---+-+--+
> |2  |1 |Main Branch|   | Dubai   |null  |
> +---+--+---+---+-+--+



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata pull request #751: [CARBONDATA-816] Added Example for H...

[chenliang613]

Github user chenliang613 commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/751#discussion_r110533556
  
--- Diff: 
integration/hive/src/main/scala/org/apache/carbondata/hiveexample/HiveExample.scala
 ---
@@ -0,0 +1,166 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.hiveexample
+
+import java.io.File
+import java.sql.{DriverManager, ResultSet, SQLException, Statement}
+
+import org.apache.spark.sql.SparkSession
+
+import org.apache.carbondata.common.logging.LogServiceFactory
+import org.apache.carbondata.hive.server.HiveEmbeddedServer2
+
+object HiveExample {
+
+  private val driverName: String = "org.apache.hive.jdbc.HiveDriver"
+
+  /**
+   * @param args
+   * @throws SQLException
+   */
+  @throws[SQLException]
+  def main(args: Array[String]) {
+val rootPath = new File(this.getClass.getResource("/").getPath
++ "../../../..").getCanonicalPath
+val store = s"$rootPath/integration/hive/target/store"
+val warehouse = s"$rootPath/integration/hive/target/warehouse"
+val metaStore_Db = 
s"$rootPath/integration/hive/target/carbon_metaStore_db"
+val logger = 
LogServiceFactory.getLogService(this.getClass.getCanonicalName)
+
+import org.apache.spark.sql.CarbonSession._
+
+System.setProperty("hadoop.home.dir", "/")
+
+val carbon = SparkSession
+  .builder()
+  .master("local")
+  .appName("HiveExample")
+  .config("carbon.sql.warehouse.dir", warehouse).enableHiveSupport()
+  .getOrCreateCarbonSession(
+store, metaStore_Db)
+
+val carbonJarPath = 
s"$rootPath/assembly/target/scala-2.11/carbondata_2.11-1.1" +
+s".0-incubating-SNAPSHOT-shade-hadoop2.7.2.jar"
--- End diff --

For hadoop version of assembly jar , please don't give the fixed version 
number(2.7.2), how about using 2.* ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---