[GitHub] incubator-carbondata issue #793: [CARBONDATA-909] Added option to specify si...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/incubator-carbondata/pull/793
  
LGTM, thanks for working on this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #749: [CARBONDATA-854] Datastax CFS file s...

[ravipesala]

Github user ravipesala commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/749#discussion_r111532440
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/datastore/filesystem/CFSCarbonFile.java
 ---
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.core.datastore.filesystem;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.carbondata.common.logging.LogService;
+import org.apache.carbondata.common.logging.LogServiceFactory;
+import org.apache.carbondata.core.datastore.impl.FileFactory;
+
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+
+
+public class CFSCarbonFile extends AbstractDFSCarbonFile {
--- End diff --

Please add comments to the class


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (CARBONDATA-917) count(*) doesn't work

[anubhav tarar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

anubhav tarar reassigned CARBONDATA-917:


Assignee: anubhav tarar

> count(*) doesn't work
> -
>
> Key: CARBONDATA-917
> URL: https://issues.apache.org/jira/browse/CARBONDATA-917
> Project: CarbonData
>  Issue Type: Bug
>  Components: data-query
> Environment: scala 2.1, Hive 1.2.1
>Reporter: Neha Bhardwaj
>Assignee: anubhav tarar
>Priority: Minor
> Attachments: abc.csv
>
>
> Select query with count(*) fails to render output
> Steps to reproduce:
> 1) In Spark Shell :
> a) Create Table -
> import org.apache.spark.sql.SparkSession
> import org.apache.spark.sql.CarbonSession._
> val carbon = 
> SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession("hdfs://localhost:54310/opt/data")
> scala> carbon.sql(" create table abc(id int, name string) stored by 
> 'carbondata' ").show
> b) Load Data - 
> scala> carbon.sql(""" load data inpath 'hdfs://localhost:54310/Files/abc.csv' 
> into table abc """ ).show
> 2) In Hive :
> a) Add Jars - 
> add jar 
> /home/neha/incubator-carbondata/assembly/target/scala-2.11/carbondata_2.11-1.1.0-incubating-SNAPSHOT-shade-hadoop2.7.2.jar;
> add jar /opt/spark-2.1.0-bin-hadoop2.7/jars/spark-catalyst_2.11-2.1.0.jar;
> add jar 
> /home/neha/incubator-carbondata/integration/hive/carbondata-hive-1.1.0-incubating-SNAPSHOT.jar;
> b) Create Table -
> create table abc(id int,name string);
> c) Alter location - 
> hive> alter table abc set LOCATION 
> 'hdfs://localhost:54310/opt/data/default/abc' ;
> d) Set Properties - 
> set hive.mapred.supports.subdirectories=true;
> set mapreduce.input.fileinputformat.input.dir.recursive=true;
> d) Alter FileFormat -
> alter table abc set FILEFORMAT
> INPUTFORMAT "org.apache.carbondata.hive.MapredCarbonInputFormat"
> OUTPUTFORMAT "org.apache.carbondata.hive.MapredCarbonOutputFormat"
> SERDE "org.apache.carbondata.hive.CarbonHiveSerDe";
> e) Query -
> hive> select count(*) from abc;
> Expected Output : 
> ResultSet should display the count of the number of rows in the table.
> Result:
> Query ID = hduser_20170412181449_85a7db42-42a1-450c-9931-dc7b3b00b412
> Total jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
>   set hive.exec.reducers.bytes.per.reducer=
> In order to limit the maximum number of reducers:
>   set hive.exec.reducers.max=
> In order to set a constant number of reducers:
>   set mapreduce.job.reduces=
> Job running in-process (local Hadoop)
> 2017-04-12 18:14:53,949 Stage-1 map = 0%,  reduce = 0%
> Ended Job = job_local220086106_0001 with errors
> Error during job, obtaining debugging information...
> Job Tracking URL: http://localhost:8080/
> FAILED: Execution Error, return code 2 from 
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> MapReduce Jobs Launched: 
> Stage-Stage-1:  HDFS Read: 0 HDFS Write: 0 FAIL
> Total MapReduce CPU Time Spent: 0 msec



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata issue #777: [CARBONDATA-863] Moved AlterTableAddColumnR...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/777
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1619/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #793: [CARBONDATA-909] Added option to specify si...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/793
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1618/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #793: [CARBONDATA-909] Added option to specify si...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/793
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1617/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #769: [CARBONDATA-888] Added include and e...

[sanoj-mg]

Github user sanoj-mg closed the pull request at:

https://github.com/apache/incubator-carbondata/pull/769


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #781: [CARBONDATA-900] Is null query on a newly a...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/781
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1616/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #788: [CARBONDATA-914] Clear BTree and Dic...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-carbondata/pull/788


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (CARBONDATA-914) Clear BTree and Dictionary instances from LRU cache on table drop

[Venkata Ramana G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Venkata Ramana G resolved CARBONDATA-914.
-
Resolution: Fixed

> Clear BTree and Dictionary instances from LRU cache on table drop
> -
>
> Key: CARBONDATA-914
> URL: https://issues.apache.org/jira/browse/CARBONDATA-914
> Project: CarbonData
>  Issue Type: Improvement
>Reporter: Manish Gupta
>Assignee: Manish Gupta
>Priority: Minor
> Fix For: 1.1.0-incubating
>
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> After drop table dictionary and BTree instances are not getting cleared from 
> driver memory. Due to this memory will keep growing and after some time GC 
> problems will occur. In real case scenarios usually driver memory is on lower 
> side hence it is more prone to GC problems.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata pull request #786: [CARBONDATA-888] Added options to in...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-carbondata/pull/786


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #781: [CARBONDATA-900] Is null query on a newly a...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/781
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1614/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #788: [CARBONDATA-914] Clear BTree and Dictionary...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/788
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1613/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #788: [CARBONDATA-914] Clear BTree and Dictionary...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/788
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1612/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #781: [CARBONDATA-900] Is null query on a newly a...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/781
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1611/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (CARBONDATA-883) Select query after alter is not working fine with integer datatype.

[Venkata Ramana G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Venkata Ramana G resolved CARBONDATA-883.
-
   Resolution: Fixed
Fix Version/s: 0.1.1-incubating

> Select query after alter is not working fine with integer datatype.
> ---
>
> Key: CARBONDATA-883
> URL: https://issues.apache.org/jira/browse/CARBONDATA-883
> Project: CarbonData
>  Issue Type: Bug
> Environment: CREATE TABLE uniqdata (CUST_ID int,CUST_NAME 
> String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, 
> BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), 
> DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 
> double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format' 
> TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into 
> table uniqdata OPTIONS('DELIMITER'=',' , 
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> 0: jdbc:hive2://192.168.2.126:1> alter table default.uniqdata1 add 
> columns(ACTIVE_EMUI_VERSION int) 
> TBLPROPERTIES('DEFAULT.VALUE.ACTIVE_EMUI_VERSION'='67890');
> +-+--+
> | Result  |
> +-+--+
> +-+--+
> No rows selected (0.265 seconds)
> 0: jdbc:hive2://192.168.2.126:1> select distinct(ACTIVE_EMUI_VERSION) 
> from uniqdata1 ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure: 
> Task 0 in stage 22.0 failed 1 times, most recent failure: Lost task 0.0 in 
> stage 22.0 (TID 1211, localhost, executor driver): 
> java.lang.ClassCastException: java.lang.Long cannot be cast to 
> java.lang.Integer
>   at 
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingMeasures(RestructureBasedVectorResultCollector.java:193)
>   at 
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:98)
>   at 
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
>   at 
> org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
>   at 
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
>   at 
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
>   at 
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
>   at 
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
>  Source)
>   at 
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithKeys$(Unknown
>  Source)
>   at 
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
>  Source)
>   at 
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
>   at 
> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
>   at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
>   at 
> org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
>   at 
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
>   at 
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
>   at org.apache.spark.scheduler.Task.run(Task.scala:99)
>   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Driver stacktrace: (state=,code=0)
> 0: jdbc:hive2://192.168.2.126:1> 
>Reporter: SWATI RAO
>Assignee: Kunal Kapoor
> Fix For: 0.1.1-incubating
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION 
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES 

[GitHub] incubator-carbondata issue #771: [CARBONDATA-883] Fixed cast exception for i...

[gvramana]

Github user gvramana commented on the issue:

https://github.com/apache/incubator-carbondata/pull/771
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #771: [CARBONDATA-883] Fixed cast exceptio...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-carbondata/pull/771


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #795: [CARBONDATA-923]fix issue of insertInto rea...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/795
  
Build Failed  with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1610/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #795: [CARBONDATA-923]fix issue of insertI...

[QiangCai]

GitHub user QiangCai opened a pull request:

https://github.com/apache/incubator-carbondata/pull/795

[CARBONDATA-923]fix issue of insertInto read from OneRowRelation

Reproduce:
create table OneRowTable(col1 string, col2 string, col3 int, col4 double) 
stored by 'carbondata'
insert into OneRowTable select '0.1', 'a.b', 1, 1.2

Exception:
org.apache.spark.sql.AnalysisException: cannot resolve '`0.1`' given input 
columns: [0.1, a.b, 1, 1.2];;
'Project ['0.1, 'a.b]
+- Project 0.1 AS 0.1#11, a.b AS a.b#12, 1 AS 1#13, 1.2 AS 1.2#14
+- OneRowRelation$

Solution:
rename output attribute of select sub-query

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/QiangCai/incubator-carbondata fixInsertInto

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/795.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #795


commit 7c2bb5f970155a854f2b807e8aaf32de8bca43d8
Author: QiangCai 
Date:   2017-04-13T13:09:12Z

insertInto read from OneRowRelation




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (CARBONDATA-923) InserInto read from one row not working

[QiangCai (JIRA)]

QiangCai created CARBONDATA-923:
---

 Summary: InserInto read from one row not working
 Key: CARBONDATA-923
 URL: https://issues.apache.org/jira/browse/CARBONDATA-923
 Project: CarbonData
  Issue Type: Bug
Reporter: QiangCai
Assignee: QiangCai


Reproduce:
create table OneRowTable(col1 string, col2 string, col3 int, col4 double) 
stored by 'carbondata'
insert into OneRowTable select '0.1', 'a.b', 1, 1.2

Exception:
org.apache.spark.sql.AnalysisException: cannot resolve '`0.1`' given input 
columns: [0.1, a.b, 1, 1.2];;
'Project ['0.1, 'a.b]
+- Project [0.1 AS 0.1#11, a.b AS a.b#12, 1 AS 1#13, 1.2 AS 1.2#14]
   +- OneRowRelation$



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata issue #701: [CARBONDATA-822] Added unsafe sort for buck...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/701
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1609/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #777: [CARBONDATA-863] Moved AlterTableAddColumnR...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/777
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1608/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #777: [CARBONDATA-863] Moved AlterTableAddColumnR...

[kunal642]

Github user kunal642 commented on the issue:

https://github.com/apache/incubator-carbondata/pull/777
  
retest this please



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Closed] (CARBONDATA-913) dead lock problem in unsafe batch parallel read merge sort

[Mohammad Shahid Khan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mohammad Shahid Khan closed CARBONDATA-913.
---

> dead lock problem in unsafe batch parallel read merge sort
> --
>
> Key: CARBONDATA-913
> URL: https://issues.apache.org/jira/browse/CARBONDATA-913
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Mohammad Shahid Khan
>Assignee: Mohammad Shahid Khan
>Priority: Critical
> Attachments: unsafeBatchMergesort_threading issue.txt
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CARBONDATA-913) dead lock problem in unsafe batch parallel read merge sort

[Mohammad Shahid Khan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mohammad Shahid Khan resolved CARBONDATA-913.
-
Resolution: Fixed

fixed with https://github.com/apache/incubator-carbondata/pull/783

> dead lock problem in unsafe batch parallel read merge sort
> --
>
> Key: CARBONDATA-913
> URL: https://issues.apache.org/jira/browse/CARBONDATA-913
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Mohammad Shahid Khan
>Assignee: Mohammad Shahid Khan
>Priority: Critical
> Attachments: unsafeBatchMergesort_threading issue.txt
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata issue #794: [CARBONDATA-857] Added Alter Table document...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/794
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1607/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #794: [CARBONDATA-857] Added Alter Table d...

[PallaviSingh1992]

GitHub user PallaviSingh1992 opened a pull request:

https://github.com/apache/incubator-carbondata/pull/794

[CARBONDATA-857] Added Alter Table documentation

Added Documentation for following alter commands:

Add Column
Drop Columns
Change Data Type
Also Restructured the DDL Document in an appropriate sequence.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/PallaviSingh1992/incubator-carbondata 
feature/CARBONDATA-857

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/794.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #794


commit 948d4e36366ca1ef171318db4f96654e9d23e925
Author: PallaviSingh1992 
Date:   2017-04-13T11:39:04Z

added alter table documentation




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #777: [CARBONDATA-863] Moved AlterTableAddColumnR...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/777
  
Build Failed  with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1606/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (CARBONDATA-904) ArrayIndexOutOfBoundsException

[Rahul Kumar (JIRA)]

[ 
https://issues.apache.org/jira/browse/CARBONDATA-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967456#comment-15967456
 ] 

Rahul Kumar commented on CARBONDATA-904:


[~swati.rao] there is a blank-space in value *' 2019-07-05 13:07:30'* . this 
behaviour is not supported in Spark-1.6 . 
You can try same query without blank-space.


> ArrayIndexOutOfBoundsException 
> ---
>
> Key: CARBONDATA-904
> URL: https://issues.apache.org/jira/browse/CARBONDATA-904
> Project: CarbonData
>  Issue Type: Bug
> Environment: Spark1.6
>Reporter: SWATI RAO
>Assignee: Rahul Kumar
> Fix For: 1.1.0-incubating
>
> Attachments: Test_Data1_h1.csv, Test_Data1_h1.csv
>
>
> Or operator is not working properly.
> When we execute these query in hive it is working fine but when we execute 
> the same in carbondata it throws an exception:
> java.lang.ArrayIndexOutOfBoundsException
> HIVE:
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary_h1 (c1_int 
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string 
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) ROW FORMAT DELIMITED 
> FIELDS TERMINATED BY ',' ;
> +-+--+
> | result  |
> +-+--+
> +-+--+
> No rows selected (1.177 seconds)
> 0: jdbc:hive2://hadoop-master:1> load data local inpath 
> '/opt/Carbon/CarbonData/TestData/Data/Test_Data1_h1.csv' OVERWRITE INTO TABLE 
> Test_Boundary_h1 ;
> +-+--+
> | Result  |
> +-+--+
> +-+--+
> No rows selected (0.437 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c6_Timestamp,max(c6_Timestamp) 
> from Test_Boundary_h1 where c6_Timestamp ='2017-07-01 12:07:28' or 
> c6_Timestamp ='2019-07-05 13:07:30' or c6_Timestamp = '1999-01-06 10:05:29' 
> group by c6_Timestamp ;
> +++--+
> |  c6_Timestamp  |  _c1   |
> +++--+
> | 2017-07-01 12:07:28.0  | 2017-07-01 12:07:28.0  |
> +++--+
> 1 row selected (1.637 seconds)
> CARBONDATA:
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary (c1_int 
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string 
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY 
> 'org.apache.carbondata.format' ;
> +-+--+
> | Result  |
> +-+--+
> +-+--+
> No rows selected (4.48 seconds)
> 0: jdbc:hive2://hadoop-master:1> LOAD DATA INPATH 
> 'hdfs://192.168.2.145:54310/BabuStore/Data/Test_Data1.csv' INTO table 
> Test_Boundary 
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
>  ;
> +-+--+
> | Result  |
> +-+--+
> +-+--+
> No rows selected (4.445 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c6_Timestamp,max(c6_Timestamp) 
> from Test_Boundary where c6_Timestamp ='2017-07-01 12:07:28' or c6_Timestamp 
> =' 2019-07-05 13:07:30' or c6_Timestamp = '1999-01-06 10:05:29' group by 
> c6_Timestamp ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure: 
> Task 0 in stage 5.0 failed 4 times, most recent failure: Lost task 0.3 in 
> stage 5.0 (TID 8, hadoop-master): java.lang.RuntimeException: 
> java.util.concurrent.ExecutionException: 
> java.lang.ArrayIndexOutOfBoundsException: 0
>   at 
> org.apache.carbondata.core.scan.processor.AbstractDataBlockIterator.updateScanner(AbstractDataBlockIterator.java:136)
>   at 
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.next(DataBlockIteratorImpl.java:50)
>   at 
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.next(DataBlockIteratorImpl.java:32)
>   at 
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.getBatchResult(DetailQueryResultIterator.java:50)
>   at 
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:41)
>   at 
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:31)
>   at 
> org.apache.carbondata.core.scan.result.iterator.ChunkRowIterator.(ChunkRowIterator.java:41)
>   at 
> org.apache.carbondata.hadoop.CarbonRecordReader.initialize(CarbonRecordReader.java:79)
>   at 
> org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.scala:204)
>   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
>   at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
>   at 
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
>   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
>   at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
>   at 
> 

[GitHub] incubator-carbondata pull request #788: [CARBONDATA-914] Clear BTree and Dic...

[manishgupta88]

GitHub user manishgupta88 reopened a pull request:

https://github.com/apache/incubator-carbondata/pull/788

[CARBONDATA-914] Clear BTree and Dictionary instances from LRU cache on 
table drop

Changes includes:
1. Support cleaning of BTree and dictionary instances when a table is being 
dropped.
2. Support clearing the access count for each segment immediately after 
block pruning rather then loading all the segments first and at lats clearing 
access count for all the segments together.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manishgupta88/incubator-carbondata 
clear_driver_cache_after_drop_table

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/788.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #788


commit c9aab5a433ef88413164b2ef02b8d0c35c6c824f
Author: manishgupta88 
Date:   2017-04-11T06:51:35Z

Problem: After drop table dictionary and BTree instances are not getting 
cleared from driver memory. Due to this memory will keep growing and after some 
time GC problems will occur. In real case scenarios usually driver memory is on 
lower side hence it is more prone to GC problems.

Solution:
1. When a table is being clear BTree and dictionary instances from LRU 
cache.
2. Clear the access count for each segment immediately after block pruning 
rather then loading all the segments first and at lats clearing access count 
for all the segments together.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #788: [CARBONDATA-914] Clear BTree and Dic...

[manishgupta88]

Github user manishgupta88 closed the pull request at:

https://github.com/apache/incubator-carbondata/pull/788


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (CARBONDATA-904) ArrayIndexOutOfBoundsException

[Rahul Kumar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rahul Kumar reassigned CARBONDATA-904:
--

Assignee: Rahul Kumar

> ArrayIndexOutOfBoundsException 
> ---
>
> Key: CARBONDATA-904
> URL: https://issues.apache.org/jira/browse/CARBONDATA-904
> Project: CarbonData
>  Issue Type: Bug
> Environment: Spark1.6
>Reporter: SWATI RAO
>Assignee: Rahul Kumar
> Fix For: 1.1.0-incubating
>
> Attachments: Test_Data1_h1.csv, Test_Data1_h1.csv
>
>
> Or operator is not working properly.
> When we execute these query in hive it is working fine but when we execute 
> the same in carbondata it throws an exception:
> java.lang.ArrayIndexOutOfBoundsException
> HIVE:
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary_h1 (c1_int 
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string 
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) ROW FORMAT DELIMITED 
> FIELDS TERMINATED BY ',' ;
> +-+--+
> | result  |
> +-+--+
> +-+--+
> No rows selected (1.177 seconds)
> 0: jdbc:hive2://hadoop-master:1> load data local inpath 
> '/opt/Carbon/CarbonData/TestData/Data/Test_Data1_h1.csv' OVERWRITE INTO TABLE 
> Test_Boundary_h1 ;
> +-+--+
> | Result  |
> +-+--+
> +-+--+
> No rows selected (0.437 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c6_Timestamp,max(c6_Timestamp) 
> from Test_Boundary_h1 where c6_Timestamp ='2017-07-01 12:07:28' or 
> c6_Timestamp ='2019-07-05 13:07:30' or c6_Timestamp = '1999-01-06 10:05:29' 
> group by c6_Timestamp ;
> +++--+
> |  c6_Timestamp  |  _c1   |
> +++--+
> | 2017-07-01 12:07:28.0  | 2017-07-01 12:07:28.0  |
> +++--+
> 1 row selected (1.637 seconds)
> CARBONDATA:
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary (c1_int 
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string 
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY 
> 'org.apache.carbondata.format' ;
> +-+--+
> | Result  |
> +-+--+
> +-+--+
> No rows selected (4.48 seconds)
> 0: jdbc:hive2://hadoop-master:1> LOAD DATA INPATH 
> 'hdfs://192.168.2.145:54310/BabuStore/Data/Test_Data1.csv' INTO table 
> Test_Boundary 
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
>  ;
> +-+--+
> | Result  |
> +-+--+
> +-+--+
> No rows selected (4.445 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c6_Timestamp,max(c6_Timestamp) 
> from Test_Boundary where c6_Timestamp ='2017-07-01 12:07:28' or c6_Timestamp 
> =' 2019-07-05 13:07:30' or c6_Timestamp = '1999-01-06 10:05:29' group by 
> c6_Timestamp ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure: 
> Task 0 in stage 5.0 failed 4 times, most recent failure: Lost task 0.3 in 
> stage 5.0 (TID 8, hadoop-master): java.lang.RuntimeException: 
> java.util.concurrent.ExecutionException: 
> java.lang.ArrayIndexOutOfBoundsException: 0
>   at 
> org.apache.carbondata.core.scan.processor.AbstractDataBlockIterator.updateScanner(AbstractDataBlockIterator.java:136)
>   at 
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.next(DataBlockIteratorImpl.java:50)
>   at 
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.next(DataBlockIteratorImpl.java:32)
>   at 
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.getBatchResult(DetailQueryResultIterator.java:50)
>   at 
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:41)
>   at 
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:31)
>   at 
> org.apache.carbondata.core.scan.result.iterator.ChunkRowIterator.(ChunkRowIterator.java:41)
>   at 
> org.apache.carbondata.hadoop.CarbonRecordReader.initialize(CarbonRecordReader.java:79)
>   at 
> org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.scala:204)
>   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
>   at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
>   at 
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
>   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
>   at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
>   at 
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
>   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
>   at 

[GitHub] incubator-carbondata issue #755: [CARBONDATA-881] Load status is successful ...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/incubator-carbondata/pull/755
  
Please handle at SegmentStatusManager.updateDeletionStatus as well.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (CARBONDATA-898) When select query and alter table rename table is triggered concurrently, NullPointerException is getting thrown

[Ravindra Pesala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravindra Pesala resolved CARBONDATA-898.

   Resolution: Fixed
Fix Version/s: 1.1.0-incubating

> When select query and alter table rename table is triggered concurrently, 
> NullPointerException is getting thrown
> 
>
> Key: CARBONDATA-898
> URL: https://issues.apache.org/jira/browse/CARBONDATA-898
> Project: CarbonData
>  Issue Type: Bug
> Environment: Spark 2.1
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Minor
> Fix For: 1.1.0-incubating
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> When user triggers select query and alter table rename table command 
> concurrently, Select query is throwning NullPointerException if the files 
> does not exist in hdfs.
> When dictionary file or schema file does not exist, File not found exception 
> should be thrown



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata pull request #730: [CARBONDATA-898]NullPointerException...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-carbondata/pull/730


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #730: [CARBONDATA-898]NullPointerException is get...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/incubator-carbondata/pull/730
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (CARBONDATA-919) result_size query stats is not giving proper row count if vector reader is enabled.

[Ravindra Pesala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravindra Pesala resolved CARBONDATA-919.

   Resolution: Fixed
Fix Version/s: 1.1.0-incubating

> result_size query stats is not giving proper row count if vector reader is 
> enabled.
> ---
>
> Key: CARBONDATA-919
> URL: https://issues.apache.org/jira/browse/CARBONDATA-919
> Project: CarbonData
>  Issue Type: Bug
> Environment: Spark 2.1
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Trivial
> Fix For: 1.1.0-incubating
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Incase of vector reader, we return columnarbatch which will have row count as 
> size of the batch, whereas we are incrementing the row count with 1 & the 
> result is printed on the query stats log
> Moved result_Size calculation into respective reader and logging the results 
> after the task completes in executor.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata pull request #790: [CARBONDATA-919]result_size in query...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-carbondata/pull/790


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #790: [CARBONDATA-919]result_size in query...

[ravipesala]

Github user ravipesala commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/790#discussion_r111359840
  
--- Diff: 
integration/spark2/src/main/java/org/apache/carbondata/spark/vectorreader/VectorizedCarbonRecordReader.java
 ---
@@ -147,7 +148,11 @@ public VectorizedCarbonRecordReader(QueryModel 
queryModel) {
   }
 
   @Override public Object getCurrentValue() throws IOException, 
InterruptedException {
-if (returnColumnarBatch) return columnarBatch;
+if (returnColumnarBatch) {
+  rowCount += columnarBatch.numValidRows();
+  return columnarBatch;
+}
+rowCount += 1;
--- End diff --

ok


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #790: [CARBONDATA-919]result_size in query statis...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/incubator-carbondata/pull/790
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (CARBONDATA-903) data load is not failing even though bad records exists in the data in case of unsafe sort or batch sort

[Ravindra Pesala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravindra Pesala resolved CARBONDATA-903.

   Resolution: Fixed
Fix Version/s: 1.1.0-incubating

> data load is not failing even though bad records exists in the data in case 
> of unsafe sort or batch sort
> 
>
> Key: CARBONDATA-903
> URL: https://issues.apache.org/jira/browse/CARBONDATA-903
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Mohammad Shahid Khan
>Assignee: Mohammad Shahid Khan
>Priority: Critical
> Fix For: 1.1.0-incubating
>
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata pull request #783: [CARBONDATA-903] data load is not fa...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-carbondata/pull/783


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #777: [CARBONDATA-863] Moved AlterTableAddColumnR...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/777
  
Build Failed  with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1603/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #790: [CARBONDATA-919]result_size in query...

[nareshpr]

Github user nareshpr commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/790#discussion_r111356217
  
--- Diff: 
integration/spark2/src/main/java/org/apache/carbondata/spark/vectorreader/VectorizedCarbonRecordReader.java
 ---
@@ -147,7 +148,11 @@ public VectorizedCarbonRecordReader(QueryModel 
queryModel) {
   }
 
   @Override public Object getCurrentValue() throws IOException, 
InterruptedException {
-if (returnColumnarBatch) return columnarBatch;
+if (returnColumnarBatch) {
+  rowCount += columnarBatch.numValidRows();
+  return columnarBatch;
+}
+rowCount += 1;
--- End diff --

if incase returnColumnarBatch is false, we are return only 1 row.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #788: [CARBONDATA-914] Clear BTree and Dictionary...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/788
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1602/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #779: [CARBONDATA-899] Added Support for DecimalT...

[bhavya411]

Github user bhavya411 commented on the issue:

https://github.com/apache/incubator-carbondata/pull/779
  
I have fixed the spacing issues as well as indentations , please review.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #793: [CARBONDATA-909] Added option to specify si...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/793
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1600/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (CARBONDATA-921) selecting columns out of order in hive doesn't work

[Neha Bhardwaj (JIRA)]

Neha Bhardwaj created CARBONDATA-921:


 Summary: selecting columns out of order in hive doesn't work
 Key: CARBONDATA-921
 URL: https://issues.apache.org/jira/browse/CARBONDATA-921
 Project: CarbonData
  Issue Type: Bug
  Components: data-query
 Environment: spark 2.1, hive 1.2.1
Reporter: Neha Bhardwaj
Priority: Minor
 Attachments: abc.csv

Selecting columns non sequentially(out of the order) fails to render output

Steps to reproduce:
1) In Spark Shell :
a) Create Table -
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.CarbonSession._
val carbon = 
SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession("hdfs://localhost:54310/opt/data")

scala> carbon.sql(" create table abc(id int, name string) stored by 
'carbondata' ").show

b) Load Data -
scala> carbon.sql(""" load data inpath 'hdfs://localhost:54310/Files/abc.csv' 
into table abc """ ).show

2) In Hive :
a) Add Jars -
add jar 
/home/neha/incubator-carbondata/assembly/target/scala-2.11/carbondata_2.11-1.1.0-incubating-SNAPSHOT-shade-hadoop2.7.2.jar;
add jar /opt/spark-2.1.0-bin-hadoop2.7/jars/spark-catalyst_2.11-2.1.0.jar;
add jar 
/home/neha/incubator-carbondata/integration/hive/carbondata-hive-1.1.0-incubating-SNAPSHOT.jar;

b) Create Table -
create table abc(id int,name string);

c) Alter location -
hive> alter table abc set LOCATION 
'hdfs://localhost:54310/opt/data/default/abc' ;

d) Set Properties -
set hive.mapred.supports.subdirectories=true;
set mapreduce.input.fileinputformat.input.dir.recursive=true;

d) Alter FileFormat -
alter table abc set FILEFORMAT
INPUTFORMAT "org.apache.carbondata.hive.MapredCarbonInputFormat"
OUTPUTFORMAT "org.apache.carbondata.hive.MapredCarbonOutputFormat"
SERDE "org.apache.carbondata.hive.CarbonHiveSerDe";

e) Queries -
hive> select id from abc; //Works Fine(Column in order)
hive> select name from abc;   //Doesn't Work(Column out of order)
hive> select id,name from abc;//Works Fine(Columns in order)
hive> select name,id from abc;//Doesn't Work(Columns out of order)

Expected output : Query - hive> select name,id from abc;
display data of the columns specified.

Actual output : Query - hive> select name,id from abc;
OK
Failed with exception java.io.IOException:java.lang.ClassCastException: 
java.lang.String cannot be cast to java.lang.Long
Time taken: 0.079 seconds



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata pull request #793: [CARBONDATA-909] Added option to spe...

[sanoj-mg]

GitHub user sanoj-mg opened a pull request:

https://github.com/apache/incubator-carbondata/pull/793

[CARBONDATA-909] Added option to specify single pass load in data frame

Added single pass load option  in data frame writer ( spark 1.6 )

Tested with HDP 2.5 / Spark 1.6



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sanoj-mg/incubator-carbondata 
CARBONDATA-909-single-pass-dataframe-dev

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/793.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #793


commit 4f08b54a8bfced79dd14c18ac2c49dfd17f4486c
Author: Sanoj MG 
Date:   2017-04-12T18:32:51Z

Added option to specify single pass load in data frame




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #792: [WIP]Changed CarbonEnv object to class and ...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/792
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1599/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (CARBONDATA-899) Added Support for DecimalType and Timestamp for spark-2.1 for integration/presto

[Bhavya Aggarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhavya Aggarwal updated CARBONDATA-899:
---
Request participants:   (was: )
 Description: Added Support for DecimalType and Timestamp for 
spark-2.1 for integration/presto.   (was: The Decimal Type correct support is 
added as well as issues related to timestamp and date are resolved.)
 Summary: Added Support for DecimalType and Timestamp for 
spark-2.1 for integration/presto  (was: Added Support for Decimal data type and 
Fixed the timestamp and date issues for Spark-2.1)

> Added Support for DecimalType and Timestamp for spark-2.1 for 
> integration/presto
> 
>
> Key: CARBONDATA-899
> URL: https://issues.apache.org/jira/browse/CARBONDATA-899
> Project: CarbonData
>  Issue Type: Sub-task
>  Components: presto-integration
>Reporter: Bhavya Aggarwal
>Assignee: Bhavya Aggarwal
>Priority: Minor
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Added Support for DecimalType and Timestamp for spark-2.1 for 
> integration/presto. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata issue #791: [CARBONDATA-920] Updated useful-tips-on-car...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/791
  
Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #791: [CARBONDATA-920] Updated useful-tips...

[vandana7]

GitHub user vandana7 opened a pull request:

https://github.com/apache/incubator-carbondata/pull/791

[CARBONDATA-920] Updated useful-tips-on-carbondata.md

Updated useful-tips-on-carbondata.md file as there were some examples which 
were  throwing  errors while execution.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vandana7/incubator-carbondata UPDATION-IN-DOCS

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/791.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #791


commit f5a37279b35729f2bf44bc007cd6d9b52f21dd27
Author: vandana 
Date:   2017-04-13T06:03:35Z

updation in useful-tips-on-carbondata.md




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #755: [CARBONDATA-881] Load status is successful ...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/755
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1598/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---