[jira] [Assigned] (CARBONDATA-3161) Pipe "|" dilimiter is not working for streaming table

2018-12-17 Thread Pawan Malwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pawan Malwal reassigned CARBONDATA-3161:


Assignee: (was: Pawan Malwal)

> Pipe "|" dilimiter is not working for streaming table
> -
>
> Key: CARBONDATA-3161
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3161
> Project: CarbonData
>  Issue Type: Bug
>  Components: data-load
>Reporter: Pawan Malwal
>Priority: Minor
>
> csv data with "|" as a dilimiter is not getting loaded into streaming table 
> correctly.
> *DDL:*
> create table table1_st(begintime TIMESTAMP, deviceid STRING, statcycle INT, 
> topologypath STRING, devicetype STRING, rebootnum INT) stored by 'carbondata' 
> TBLPROPERTIES('SORT_SCOPE'='GLOBAL_SORT','sort_columns'='deviceid,begintime','streaming'
>  ='true');
> *Run in spark shell:*
> import org.apache.spark.sql.SparkSession;
> import org.apache.spark.sql.SparkSession.Builder;
> import org.apache.spark.sql.CarbonSession;
> import org.apache.spark.sql.CarbonSession.CarbonBuilder;
> import org.apache.spark.sql.streaming._
> import org.apache.carbondata.streaming.parser._
> val enableHiveSupport = SparkSession.builder().enableHiveSupport();
> val carbon=new 
> CarbonBuilder(enableHiveSupport).getOrCreateCarbonSession("hdfs://hacluster/user/hive/warehouse/")
> val df=carbon.readStream.text("/user/*.csv")
> val qrymm_0001 = 
> df.writeStream.format("carbondata").option(CarbonStreamParser.CARBON_STREAM_PARSER,
>  
> CarbonStreamParser.CARBON_STREAM_PARSER_CSV).{color:#FF}*option("delimiter","|")*{color}.option("header","false").option("dbName","stdb").option("checkpointLocation",
>  
> "/tmp/tb1").option("bad_records_action","FORCE").option("tableName","table1_st").trigger(ProcessingTime(6000)).option("carbon.streaming.auto.handoff.enabled","true").option("TIMESTAMPFORMAT","-dd-MM
>  HH:mm:ss").start
>  
> *Sample records:*
>  begintime| deviceid| statcycle| topologypath| devicetype| rebootnum
>  2018-10-01 00:00:00|Device1|0|dsad|STB|9
>  2018-10-01 00:05:00|Device1|0|Rsad|STB|4
>  2018-10-01 00:10:00|Device1|0|fsf|STB|6
>  2018-10-01 00:15:00|Device1|0|fdgf|STB|8



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CARBONDATA-3161) Pipe "|" dilimiter is not working for streaming table

2018-12-11 Thread Pawan Malwal (JIRA)
Pawan Malwal created CARBONDATA-3161:


 Summary: Pipe "|" dilimiter is not working for streaming table
 Key: CARBONDATA-3161
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3161
 Project: CarbonData
  Issue Type: Bug
  Components: data-load
Reporter: Pawan Malwal
Assignee: Pawan Malwal


csv data with "|" as a dilimiter is not getting loaded into streaming table 
correctly.

*DDL:*

create table table1_st(begintime TIMESTAMP, deviceid STRING, statcycle INT, 
topologypath STRING, devicetype STRING, rebootnum INT) stored by 'carbondata' 
TBLPROPERTIES('SORT_SCOPE'='GLOBAL_SORT','sort_columns'='deviceid,begintime','streaming'
 ='true');

*Run in spark shell:*

import org.apache.spark.sql.SparkSession;
import org.apache.spark.sql.SparkSession.Builder;
import org.apache.spark.sql.CarbonSession;
import org.apache.spark.sql.CarbonSession.CarbonBuilder;
import org.apache.spark.sql.streaming._
import org.apache.carbondata.streaming.parser._


val enableHiveSupport = SparkSession.builder().enableHiveSupport();
val carbon=new 
CarbonBuilder(enableHiveSupport).getOrCreateCarbonSession("hdfs://hacluster/user/hive/warehouse/")
val df=carbon.readStream.text("/user/*.csv")

val qrymm_0001 = 
df.writeStream.format("carbondata").option(CarbonStreamParser.CARBON_STREAM_PARSER,
 
CarbonStreamParser.CARBON_STREAM_PARSER_CSV).{color:#FF}*option("delimiter","|")*{color}.option("header","false").option("dbName","stdb").option("checkpointLocation",
 
"/tmp/tb1").option("bad_records_action","FORCE").option("tableName","table1_st").trigger(ProcessingTime(6000)).option("carbon.streaming.auto.handoff.enabled","true").option("TIMESTAMPFORMAT","-dd-MM
 HH:mm:ss").start

 

*Sample records:*
 begintime| deviceid| statcycle| topologypath| devicetype| rebootnum
 2018-10-01 00:00:00|Device1|0|dsad|STB|9
 2018-10-01 00:05:00|Device1|0|Rsad|STB|4
 2018-10-01 00:10:00|Device1|0|fsf|STB|6
 2018-10-01 00:15:00|Device1|0|fdgf|STB|8



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CARBONDATA-3076) Set streaming=false not working for streaming table.

2018-11-05 Thread Pawan Malwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pawan Malwal updated CARBONDATA-3076:
-
Environment: Spark2.1
Description: 
1. Create streaming table.

CREATE TABLE table_st(CUST_ID String,CUST_NAME String) STORED BY 
'org.apache.carbondata.format' TBLPROPERTIES('streaming'='true');

2. Alter table to set 'streaming'='false'

alter table table_st set tblproperties('streaming'='false');

_Alter table throws below exception_

_Error: 
org.apache.carbondata.common.exceptions.sql.MalformedCarbonCommandException: 
Streaming property can not be changed once it is 'true' (state=,code=0)_

  was:
Create streaming table.

CREATE TABLE table_st(CUST_ID String,CUST_NAME String) STORED BY 
'org.apache.carbondata.format' TBLPROPERTIES('streaming'='true');

Alter table 


> Set streaming=false not working for streaming table.
> 
>
> Key: CARBONDATA-3076
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3076
> Project: CarbonData
>  Issue Type: Bug
>  Components: data-query
>Affects Versions: 1.5.1
> Environment: Spark2.1
>Reporter: Pawan Malwal
>Priority: Minor
>
> 1. Create streaming table.
> CREATE TABLE table_st(CUST_ID String,CUST_NAME String) STORED BY 
> 'org.apache.carbondata.format' TBLPROPERTIES('streaming'='true');
> 2. Alter table to set 'streaming'='false'
> alter table table_st set tblproperties('streaming'='false');
> _Alter table throws below exception_
> _Error: 
> org.apache.carbondata.common.exceptions.sql.MalformedCarbonCommandException: 
> Streaming property can not be changed once it is 'true' (state=,code=0)_



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CARBONDATA-3076) Set streaming=false not working for streaming table.

2018-11-05 Thread Pawan Malwal (JIRA)
Pawan Malwal created CARBONDATA-3076:


 Summary: Set streaming=false not working for streaming table.
 Key: CARBONDATA-3076
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3076
 Project: CarbonData
  Issue Type: Bug
  Components: data-query
Affects Versions: 1.5.1
Reporter: Pawan Malwal


Create streaming table.

CREATE TABLE table_st(CUST_ID String,CUST_NAME String) STORED BY 
'org.apache.carbondata.format' TBLPROPERTIES('streaming'='true');

Alter table 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CARBONDATA-2849) No of executors running simultaneously are less impacting the compaction performance

2018-08-09 Thread Pawan Malwal (JIRA)
Pawan Malwal created CARBONDATA-2849:


 Summary: No of executors running simultaneously are less impacting 
the compaction performance
 Key: CARBONDATA-2849
 URL: https://issues.apache.org/jira/browse/CARBONDATA-2849
 Project: CarbonData
  Issue Type: Bug
  Components: data-load
Affects Versions: 1.4.1
 Environment: Spark 2.1
Reporter: Pawan Malwal


Steps :

Compaction is performed for huge data.

 

Actual Issue : 

Only 10 executors are running same job simultaneously compared to 17 executors 
in previous carbon versions due to which compaction performance is impacted

 

Expected Issue : 

No of executors running in current carbon version should be more similar to 
earlier carbon versions. This should help improve carbon compaction performance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CARBONDATA-2848) Task failed during 1st load and restarted on same executor

2018-08-09 Thread Pawan Malwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pawan Malwal updated CARBONDATA-2848:
-
Description: 
Steps :

Huge data load performed. (table has 3.5 billion records)

 

Actual Issue :

One of the task failed during first load and restarted again in the same 
executor and took double the time to load first segment, rest of the 5 load ran 
properly.

 Due to this failure and task rerun Segment_0 has got double the number of 
.carbondata files and even the number of records loaded twice.

 

Expected :

Task should not fail in 1st load. Even after failed task restarts carbondata 
files and records loaded should not double.

 

  was:
Steps :

Huge data load performed. (table has 3.5 billion records)

 

Actual Issue :

One of the task failed during first load and restarted again in the same 
executor and took double the time to load first segment, rest of the 5 load ran 
properly.

 Due to this failure and task rerun Segment_0 has got double the number of 
.carbondata files and even the number of records loaded twice.

 

Expected :

Task should not fail in 1st load. 

 


> Task failed during 1st load and restarted on same executor
> --
>
> Key: CARBONDATA-2848
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2848
> Project: CarbonData
>  Issue Type: Bug
>  Components: data-load
>Affects Versions: 1.4.1
> Environment: Spark 2.1
>Reporter: Pawan Malwal
>Priority: Minor
>
> Steps :
> Huge data load performed. (table has 3.5 billion records)
>  
> Actual Issue :
> One of the task failed during first load and restarted again in the same 
> executor and took double the time to load first segment, rest of the 5 load 
> ran properly.
>  Due to this failure and task rerun Segment_0 has got double the number of 
> .carbondata files and even the number of records loaded twice.
>  
> Expected :
> Task should not fail in 1st load. Even after failed task restarts carbondata 
> files and records loaded should not double.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CARBONDATA-2848) Task failed during 1st load and restarted on same executor

2018-08-09 Thread Pawan Malwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pawan Malwal updated CARBONDATA-2848:
-
Description: 
Steps :

Huge data load performed. (table has 3.5 billion records)

 

Actual Issue :

One of the task failed during first load and restarted again in the same 
executor and took double the time to load first segment, rest of the 5 load ran 
properly.

 Due to this failure and task rerun Segment_0 has got double the number of 
.carbondata files and even the number of records loaded twice.

 

Expected :

Task should not fail in 1st load. 

 

  was:
Steps :

Huge data load performed.

 

Actual Issue :

One of the task failed during first load and restarted again in the same 
executor and took double the time to load first segment, rest of the 5 load ran 
properly.

 

Expected :

Task should not fail in 1st load. 

 


> Task failed during 1st load and restarted on same executor
> --
>
> Key: CARBONDATA-2848
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2848
> Project: CarbonData
>  Issue Type: Bug
>  Components: data-load
>Affects Versions: 1.4.1
> Environment: Spark 2.1
>Reporter: Pawan Malwal
>Priority: Minor
>
> Steps :
> Huge data load performed. (table has 3.5 billion records)
>  
> Actual Issue :
> One of the task failed during first load and restarted again in the same 
> executor and took double the time to load first segment, rest of the 5 load 
> ran properly.
>  Due to this failure and task rerun Segment_0 has got double the number of 
> .carbondata files and even the number of records loaded twice.
>  
> Expected :
> Task should not fail in 1st load. 
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CARBONDATA-2848) Task failed during 1st load and restarted on same executor

2018-08-09 Thread Pawan Malwal (JIRA)
Pawan Malwal created CARBONDATA-2848:


 Summary: Task failed during 1st load and restarted on same executor
 Key: CARBONDATA-2848
 URL: https://issues.apache.org/jira/browse/CARBONDATA-2848
 Project: CarbonData
  Issue Type: Bug
  Components: data-load
Affects Versions: 1.4.1
 Environment: Spark 2.1
Reporter: Pawan Malwal


Steps :

Huge data load performed.

 

Actual Issue :

One of the task failed during first load and restarted again in the same 
executor and took double the time to load first segment, rest of the 5 load ran 
properly.

 

Expected :

Task should not fail in 1st load. 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CARBONDATA-2847) In S3 Alter Rename is throwing Null Pointer Exception.But table is getting renamed.

2018-08-09 Thread Pawan Malwal (JIRA)
Pawan Malwal created CARBONDATA-2847:


 Summary: In S3 Alter Rename is throwing Null Pointer Exception.But 
table is getting renamed.
 Key: CARBONDATA-2847
 URL: https://issues.apache.org/jira/browse/CARBONDATA-2847
 Project: CarbonData
  Issue Type: Bug
  Components: other
Affects Versions: 1.4.1
 Environment: Spark 2.2
Reporter: Pawan Malwal


Steps :

User tries to alter table rename in S3.

 

Actual Issue :

In S3 Alter Rename is throwing Null Pointer Exception.But table is getting 
renamed.

 

Expected:

The alter rename should not throw exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CARBONDATA-1993) Mismatch in Carbon properties default values and corresponding template and document

2018-01-05 Thread Pawan Malwal (JIRA)

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pawan Malwal updated CARBONDATA-1993:
-
Summary: Mismatch in Carbon properties default values and corresponding 
template and document  (was: Mismatch in Carbon properties default values and 
corresponding and template, document)

> Mismatch in Carbon properties default values and corresponding template and 
> document
> 
>
> Key: CARBONDATA-1993
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1993
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Pawan Malwal
>Assignee: Pawan Malwal
>Priority: Minor
>
> Carbon properties default values has to be fixed with correct values and 
> corresponding template, document needs to be updated.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (CARBONDATA-1993) Mismatch in Carbon properties default values and corresponding and template, document

2018-01-05 Thread Pawan Malwal (JIRA)
Pawan Malwal created CARBONDATA-1993:


 Summary: Mismatch in Carbon properties default values and 
corresponding and template, document
 Key: CARBONDATA-1993
 URL: https://issues.apache.org/jira/browse/CARBONDATA-1993
 Project: CarbonData
  Issue Type: Bug
Reporter: Pawan Malwal
Assignee: Pawan Malwal
Priority: Minor


Carbon properties default values has to be fixed with correct values and 
corresponding template, document needs to be updated.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (CARBONDATA-1247) Block pruning not working for date type data type column

2017-12-13 Thread Pawan Malwal (JIRA)

[ 
https://issues.apache.org/jira/browse/CARBONDATA-1247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289364#comment-16289364
 ] 

Pawan Malwal commented on CARBONDATA-1247:
--

[https://github.com/apache/carbondata/pull/1656]

> Block pruning not working for date type data type column
> 
>
> Key: CARBONDATA-1247
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1247
> Project: CarbonData
>  Issue Type: Bug
>  Components: sql
> Environment: Standalone
>Reporter: krishna reddy
>Assignee: Pawan Malwal
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> 1. create table if not exists test_date (id int,first_name String,last_name 
> string,email string,gender string,dob date) stored by 'carbondata'
> 2. LOAD DATA LOCAL INPATH 'D:/data/MOCK_DATA_24.csv' into table test_date
> 3. LOAD DATA LOCAL INPATH 'D:/data/MOCK_DATA_25.csv' into table test_date
> 4. select dob from test_date where dob = '2016-06-24'
> Actual Result: In the logs it is going to 2 blocks
>  Identified no.of.blocks: 2,
>  no.of.tasks: 2,
>  no.of.nodes: 0,
>  parallelism: 1
> Expected: It should select only one block



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Closed] (CARBONDATA-1889) Block pruning not working for date type column

2017-12-13 Thread Pawan Malwal (JIRA)

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pawan Malwal closed CARBONDATA-1889.

Resolution: Duplicate

> Block pruning not working for date type column
> --
>
> Key: CARBONDATA-1889
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1889
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Pawan Malwal
>Assignee: Pawan Malwal
>
> spark.sql(s"""create table test_dateType(c1 int, c2 string, c3 smallint, 
> c4 bigint, c5 short, c6 date) stored by 'carbondata'""".stripMargin).show()
> spark.sql(s"""insert into test_dateType select 
> 1,'111',111,11,,'2017-12-12'""".stripMargin)
> spark.sql(s"""insert into test_dateType select 
> 2,'222',222,22,,'2018-11-11'""".stripMargin)
> spark.sql(s"""insert into test_dateType select 
> 3,'333',333,33,,'2019-10-10'""".stripMargin)
> spark.sql(s"""select * from test_dateType where c6 = '2018-11-11' 
> """.stripMargin).show(200,false)
> Check for "Identified no.of.blocks" in logs
> It shows :  Identified no.of.blocks: 3
> Only 1 block should be selected but all the 3 blocks are getting selected



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (CARBONDATA-1247) Block pruning not working for date type data type column

2017-12-13 Thread Pawan Malwal (JIRA)

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pawan Malwal reassigned CARBONDATA-1247:


Assignee: Pawan Malwal  (was: Babulal)

> Block pruning not working for date type data type column
> 
>
> Key: CARBONDATA-1247
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1247
> Project: CarbonData
>  Issue Type: Bug
>  Components: sql
> Environment: Standalone
>Reporter: krishna reddy
>Assignee: Pawan Malwal
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> 1. create table if not exists test_date (id int,first_name String,last_name 
> string,email string,gender string,dob date) stored by 'carbondata'
> 2. LOAD DATA LOCAL INPATH 'D:/data/MOCK_DATA_24.csv' into table test_date
> 3. LOAD DATA LOCAL INPATH 'D:/data/MOCK_DATA_25.csv' into table test_date
> 4. select dob from test_date where dob = '2016-06-24'
> Actual Result: In the logs it is going to 2 blocks
>  Identified no.of.blocks: 2,
>  no.of.tasks: 2,
>  no.of.nodes: 0,
>  parallelism: 1
> Expected: It should select only one block



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (CARBONDATA-1889) Block pruning not working for date type column

2017-12-12 Thread Pawan Malwal (JIRA)

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pawan Malwal updated CARBONDATA-1889:
-
Description: 
spark.sql(s"""create table test_dateType(c1 int, c2 string, c3 smallint, c4 
bigint, c5 short, c6 date) stored by 'carbondata'""".stripMargin).show()

spark.sql(s"""insert into test_dateType select 
1,'111',111,11,,'2017-12-12'""".stripMargin)
spark.sql(s"""insert into test_dateType select 
2,'222',222,22,,'2018-11-11'""".stripMargin)
spark.sql(s"""insert into test_dateType select 
3,'333',333,33,,'2019-10-10'""".stripMargin)

spark.sql(s"""select * from test_dateType where c6 = '2018-11-11' 
""".stripMargin).show(200,false)

Check for "Identified no.of.blocks" in logs

It shows :  Identified no.of.blocks: 3

Only 1 block should be selected but all the 3 blocks are getting selected

  was:
spark.sql(s"""create table test_dateType(c1 int, c2 string, c3 smallint, c4 
bigint, c5 short, c6 date) stored by 'carbondata'""".stripMargin).show()

spark.sql(s"""insert into test_dateType select 
1,'111',111,11,,'2017-12-12'""".stripMargin)
spark.sql(s"""insert into test_dateType select 
2,'222',222,22,,'2018-11-11'""".stripMargin)
spark.sql(s"""insert into test_dateType select 
3,'333',333,33,,'2019-10-10'""".stripMargin)

spark.sql(s"""select * from test_dateType where c6 = '2018-11-11' 
""".stripMargin).show(200,false)

Check for "Identified no.of.blocks" in logs

It shows :  Identified no.of.blocks: 3

Only 1 block should be selected


> Block pruning not working for date type column
> --
>
> Key: CARBONDATA-1889
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1889
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Pawan Malwal
>Assignee: Pawan Malwal
>
> spark.sql(s"""create table test_dateType(c1 int, c2 string, c3 smallint, 
> c4 bigint, c5 short, c6 date) stored by 'carbondata'""".stripMargin).show()
> spark.sql(s"""insert into test_dateType select 
> 1,'111',111,11,,'2017-12-12'""".stripMargin)
> spark.sql(s"""insert into test_dateType select 
> 2,'222',222,22,,'2018-11-11'""".stripMargin)
> spark.sql(s"""insert into test_dateType select 
> 3,'333',333,33,,'2019-10-10'""".stripMargin)
> spark.sql(s"""select * from test_dateType where c6 = '2018-11-11' 
> """.stripMargin).show(200,false)
> Check for "Identified no.of.blocks" in logs
> It shows :  Identified no.of.blocks: 3
> Only 1 block should be selected but all the 3 blocks are getting selected



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (CARBONDATA-1889) Block pruning not working for date type column

2017-12-12 Thread Pawan Malwal (JIRA)
Pawan Malwal created CARBONDATA-1889:


 Summary: Block pruning not working for date type column
 Key: CARBONDATA-1889
 URL: https://issues.apache.org/jira/browse/CARBONDATA-1889
 Project: CarbonData
  Issue Type: Bug
Reporter: Pawan Malwal
Assignee: Pawan Malwal


spark.sql(s"""create table test_dateType(c1 int, c2 string, c3 smallint, c4 
bigint, c5 short, c6 date) stored by 'carbondata'""".stripMargin).show()

spark.sql(s"""insert into test_dateType select 
1,'111',111,11,,'2017-12-12'""".stripMargin)
spark.sql(s"""insert into test_dateType select 
2,'222',222,22,,'2018-11-11'""".stripMargin)
spark.sql(s"""insert into test_dateType select 
3,'333',333,33,,'2019-10-10'""".stripMargin)

spark.sql(s"""select * from test_dateType where c6 = '2018-11-11' 
""".stripMargin).show(200,false)

Check for "Identified no.of.blocks" in logs

It shows :  Identified no.of.blocks: 3

Only 1 block should be selected



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (CARBONDATA-1671) Support set/unset table comment for ALTER table

2017-11-15 Thread Pawan Malwal (JIRA)

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pawan Malwal updated CARBONDATA-1671:
-
Description: Table comment set/unset using **ALTER TABLE  SET/UNSET 
TBLPROPERTIES** 

> Support set/unset table comment for ALTER table
> ---
>
> Key: CARBONDATA-1671
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1671
> Project: CarbonData
>  Issue Type: Sub-task
>Reporter: Pawan Malwal
>Assignee: Pawan Malwal
>
> Table comment set/unset using **ALTER TABLE  SET/UNSET TBLPROPERTIES** 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (CARBONDATA-1671) Support set/unset table comment for ALTER table

2017-11-15 Thread Pawan Malwal (JIRA)

[ 
https://issues.apache.org/jira/browse/CARBONDATA-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254813#comment-16254813
 ] 

Pawan Malwal commented on CARBONDATA-1671:
--

Features proposed in this jira::

Table comment set/unset using **ALTER TABLE  SET/UNSET TBLPROPERTIES** query is 
not supported

eg: ALTER TABLE table_with_comment SET TBLPROPERTIES("comment"= "modified 
comment)
If user wants to alter the table properties and adds/updates table comment, 
inorder  to handle this requirement, update the comment field value with the 
newly added/modified comment when user executes **ALTER TABLE  SET 
TBLPROPERTIES** query.

This will also take care of unsetting the table comment when user executes 
query  **ALTER TABLE  UNSET TBLPROPERTIES** inorder to unset or remove table 
comment.
eg: ALTER TABLE table_comment UNSET TBLPROPERTIES IF EXISTS ('comment')

> Support set/unset table comment for ALTER table
> ---
>
> Key: CARBONDATA-1671
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1671
> Project: CarbonData
>  Issue Type: Sub-task
>Reporter: Pawan Malwal
>Assignee: Pawan Malwal
>
> Table comment set/unset using **ALTER TABLE  SET/UNSET TBLPROPERTIES** 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (CARBONDATA-1671) Support set/unset table comment for ALTER table

2017-11-06 Thread Pawan Malwal (JIRA)
Pawan Malwal created CARBONDATA-1671:


 Summary: Support set/unset table comment for ALTER table
 Key: CARBONDATA-1671
 URL: https://issues.apache.org/jira/browse/CARBONDATA-1671
 Project: CarbonData
  Issue Type: Sub-task
Reporter: Pawan Malwal
Assignee: Pawan Malwal






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (CARBONDATA-1502) Compaction Support: Enhance Test Cases for Struct DataType

2017-09-20 Thread Pawan Malwal (JIRA)
Pawan Malwal created CARBONDATA-1502:


 Summary: Compaction Support: Enhance Test Cases for Struct DataType
 Key: CARBONDATA-1502
 URL: https://issues.apache.org/jira/browse/CARBONDATA-1502
 Project: CarbonData
  Issue Type: Sub-task
Reporter: Pawan Malwal
Assignee: Pawan Malwal
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)