[jira] [Assigned] (CARBONDATA-610) Update is working with String Datatype now but Date Datatype still not working
[
https://issues.apache.org/jira/browse/CARBONDATA-610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ravikiran reassigned CARBONDATA-610:
Assignee: sounak chakraborty (was: ravikiran)
> Update is working with String Datatype now but Date Datatype still not working
> --
>
> Key: CARBONDATA-610
> URL: https://issues.apache.org/jira/browse/CARBONDATA-610
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
>Affects Versions: 1.0.0-incubating
> Environment: SPARK=1.6
>Reporter: abhishek giri
>Assignee: sounak chakraborty
> Attachments: 3000_UniqData.csv, Screenshot from 2017-01-10
> 12-32-48.png
>
>
> Update query is not working with spark 1.6 and String and date
> Create Table Query = CREATE TABLE uniqdata (CUST_ID int,CUST_NAME
> String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp,
> BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10),
> DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2
> double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format'
> TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB");
> Load data query = LOAD DATA INPATH
> 'hdfs://localhost:54310/tmp/testData/3000_UniqData.csv' into table uniqdata
> OPTIONS('FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
>
> Query for String = update uniqdata set (CUST_NAME)=('tony stark');
> Query for Date = update uniqdata set (dob)=('2016-12-11') where cust_name =
> 'CUST_NAME_01999';
> After update NULL is being inserted as shown below:
> -+-+--+---+--+--+
> | cust_id | cust_name |active_emui_version | dob
> | doj | bigint_column1 | bigint_column2 |
> decimal_column1 | decimal_column2 |double_column1|
> double_column2 | integer_column1 |
> +--+++++-+-+-+-+--+---+--+--+
> | 10987| NULL | ACTIVE_EMUI_VERSION_01987 | 1975-06-11 01:00:03.0
> | 1975-06-11 02:00:03.0 | 123372038841| -223372034867 |
> 12345.6808882656| 22345.6808882656| 1.12345674897976E10 |
> -1.12345674897976E10 | 1988 |
> | 10988| NULL | ACTIVE_EMUI_VERSION_01988 | 1975-06-12 01:00:03.0
> | 1975-06-12 02:00:03.0 | 123372038842| -223372034866 |
> 12345.6808892656| 22345.6808892656| 1.12345674897976E10 |
> -1.12345674897976E10 | 1989 |
> | 10989| NULL | ACTIVE_EMUI_VERSION_01989 | 1975-06-13 01:00:03.0
> | 1975-06-13 02:00:03.0 | 123372038843| -223372034865 |
> 12345.6808902656| 22345.6808902656| 1.12345674897976E10 |
> -1.12345674897976E10 | 1990 |
> | 10990| NULL | ACTIVE_EMUI_VERSION_01990 | 1975-06-14 01:00:03.0
> | 1975-06-14 02:00:03.0 | 123372038844| -223372034864 |
> 12345.6808912656| 22345.6808912656| 1.12345674897976E10 |
> -1.12345674897976E10 | 1991 |
> | 10991| NULL | ACTIVE_EMUI_VERSION_01991 | 1975-06-15 01:00:03.0
> | 1975-06-15 02:00:03.0 | 123372038845| -223372034863 |
> 12345.6808922656| 22345.6808922656| 1.12345674897976E10 |
> -1.12345674897976E10 | 1992 |
> | 10992| NULL | ACTIVE_EMUI_VERSION_01992 | 1975-06-16 01:00:03.0
> | 1975-06-16 02:00:03.0 | 123372038846| -223372034862 |
> 12345.6808932656| 22345.6808932656| 1.12345674897976E10 |
> -1.12345674897976E10 | 1993 |
> | 10993| NULL | ACTIVE_EMUI_VERSION_01993 | 1975-06-17 01:00:03.0
> | 1975-06-17 02:00:03.0 | 123372038847| -223372034861 |
> 12345.6808942656| 22345.6808942656| 1.12345674897976E10 |
> -1.12345674897976E10 | 1994 |
> | 10994| NULL | ACTIVE_EMUI_VERSION_01994 | 1975-06-18 01:00:03.0
> | 1975-06-18 02:00:03.0 | 123372038848| -223372034860 |
> 12345.6808952656| 22345.6808952656| 1.12345674897976E10 |
> -1.12345674897976E10 | 1995 |
> | 10995| NULL | ACTIVE_EMUI_VERSION_01995 | 1975-06-19 01:00:03.0
> | 1975-06-19 02:00:03.0 | 123372038849| -223372034859 |
> 12345.6808962656| 22345.6808962656| 1.12345674897976E10 |
> -1.12345674897976E10 | 1996 |
> | 10996| NULL |
[jira] [Closed] (CARBONDATA-415) Fixing of the Findbugs and other static issues.
[ https://issues.apache.org/jira/browse/CARBONDATA-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ravikiran closed CARBONDATA-415. Resolution: Fixed > Fixing of the Findbugs and other static issues. > --- > > Key: CARBONDATA-415 > URL: https://issues.apache.org/jira/browse/CARBONDATA-415 > Project: CarbonData > Issue Type: Improvement > Components: core, spark-integration >Affects Versions: 0.1.1-incubating >Reporter: ravikiran >Assignee: ravikiran >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CARBONDATA-663) Major compaction is not working properly as per the configuration
[
https://issues.apache.org/jira/browse/CARBONDATA-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831280#comment-15831280
]
ravikiran commented on CARBONDATA-663:
--
hi ,
Please find below the working of the Major compaction .
1. major compaction is a size based compaction. assume X is the size
configured . for example X = 10 mb
2. Major compaction will be done as long as the segments are inside this 10 mb
size. here the size considered is not the CSV input file size. size will be
calculated from the segments files i.e carbondata files , index files of a
segment.
3. if a size of 1 segment is above 10 mb then that segment wont be considered
for merging.
In the above description please check if the size is calculated properly.
> Major compaction is not working properly as per the configuration
> -
>
> Key: CARBONDATA-663
> URL: https://issues.apache.org/jira/browse/CARBONDATA-663
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
>Affects Versions: 1.0.0-incubating
> Environment: Spark - 2.1
>Reporter: Anurag Srivastava
> Attachments: logs, sample_str_more1.csv, show_segment.png,
> show_segments_after_compaction.png
>
>
> I have set property *carbon.major.compaction.size= 3* and load data which is
> the size of 5 MB and when I perform compaction it compacted, but initially it
> shouldn't be perform. Here is the queries :
> *create table :* create table test_major_compaction(id Int,name string)stored
> by 'carbondata';
> *Load Data :* Load two segments.
> LOAD DATA inpath 'hdfs://localhost:54310/sample_str_more1.csv' INTO table
> test_major_compaction options('DELIMITER'=',', 'FILEHEADER'='id,
> name','QUOTECHAR'='"');
> *Show segments :* show segments for table test_major_compaction;
> !https://issues.apache.org/jira/secure/attachment/12848287/show_segment.png!
> *Alter Table :* ALTER TABLE test_major_compaction COMPACT 'MAJOR';
> *Show segments :* Again see the segments :
> show segments for table test_major_compaction;
> !https://issues.apache.org/jira/secure/attachment/12848286/show_segments_after_compaction.png!
> I have attached all the data with the it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CARBONDATA-615) Update query store wrong value for Date data type
[
https://issues.apache.org/jira/browse/CARBONDATA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831732#comment-15831732
]
ravikiran commented on CARBONDATA-615:
--
FIxed this issue with PR #551. please check this issue in latest release.
> Update query store wrong value for Date data type
> -
>
> Key: CARBONDATA-615
> URL: https://issues.apache.org/jira/browse/CARBONDATA-615
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
>Affects Versions: 1.0.0-incubating
> Environment: Spark 1.6 Spark 2.1
>Reporter: Anurag Srivastava
>Assignee: ravikiran
>Priority: Minor
> Attachments: 2000_UniqData.csv, update_dob.png
>
>
> I am trying to update DOB column with Date Data Type. It is storing a day
> before date which I have mentioned for updating in DOB column.
> *Create Table :* CREATE TABLE uniqdata (CUST_ID int,CUST_NAME
> char(30),ACTIVE_EMUI_VERSION string, DOB Date, DOJ Date, BIGINT_COLUMN1
> bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double, INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format';
> *Load Data :* LOAD DATA INPATH 'hdfs://localhost:54310/2000_UniqData.csv'
> into table uniqdata OPTIONS ('DELIMITER'=','
> ,'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','SINGLE_PASS'='true');
> *Update Query :* update uniqdata set (dob)=(to_date('2016-12-01')) where
> cust_name = 'CUST_NAME_01999';
> *Expected Result :* It should update DOB column with *2016-12-01*.
> *Actual Result :* It is updating DOB column with *2016-11-30*.
> !https://issues.apache.org/jira/secure/attachment/12846515/update_dob.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CARBONDATA-219) compaction with out data load is failing.
ravikiran created CARBONDATA-219: Summary: compaction with out data load is failing. Key: CARBONDATA-219 URL: https://issues.apache.org/jira/browse/CARBONDATA-219 Project: CarbonData Issue Type: Bug Components: spark-integration Reporter: ravikiran -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-281) improve the test cases in LCM module.
ravikiran created CARBONDATA-281: Summary: improve the test cases in LCM module. Key: CARBONDATA-281 URL: https://issues.apache.org/jira/browse/CARBONDATA-281 Project: CarbonData Issue Type: Improvement Components: spark-integration Affects Versions: 0.1.0-incubating Reporter: ravikiran Assignee: ravikiran Priority: Minor improving the test cases in the lcm. adding the test cases for the compaction with boundary test cases. added test cases to verify the minor compaction threshold check. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-261) clean files is updating the stale segments to the table status.
ravikiran created CARBONDATA-261: Summary: clean files is updating the stale segments to the table status. Key: CARBONDATA-261 URL: https://issues.apache.org/jira/browse/CARBONDATA-261 Project: CarbonData Issue Type: Bug Components: data-load, data-query Affects Versions: 0.1.0-incubating Reporter: ravikiran Assignee: ravikiran in clean files it will read the table status first and it will delete the physical locations and update the table metadata and then take the lock for writing and after that it will write the old read list to the file. but this is wrong as in the time between the read and write the table status would have been changed. so before writing it should read the table status again and update the changes and then write. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-849) if alter table ddl is executed on non existing table, then error message is wrong.
ravikiran created CARBONDATA-849: Summary: if alter table ddl is executed on non existing table, then error message is wrong. Key: CARBONDATA-849 URL: https://issues.apache.org/jira/browse/CARBONDATA-849 Project: CarbonData Issue Type: Bug Components: sql Reporter: ravikiran Assignee: ravikiran Priority: Minor The error message getting while running alter on the non existing table is : Exception in thread "main" org.apache.carbondata.spark.exception.MalformedCarbonCommandException: Unsupported alter operation on hive table but this is not correct. The hive table has blocked the alter DDL on its tables. So Carbon should be consistent with HIVE. Correct msg : Operation not allowed: alter table name compact 'minor' -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-880) when explain extended is done on a query then store path is getting printed to the user.
ravikiran created CARBONDATA-880: Summary: when explain extended is done on a query then store path is getting printed to the user. Key: CARBONDATA-880 URL: https://issues.apache.org/jira/browse/CARBONDATA-880 Project: CarbonData Issue Type: Bug Components: data-query Reporter: ravikiran Assignee: ravikiran when explain extended is done on a query then store path is getting printed to the user in carbonhadooprelation. the path should not be shown to the user. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (CARBONDATA-880) when explain extended is done on a query then store path is getting printed to the user.
[ https://issues.apache.org/jira/browse/CARBONDATA-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ravikiran updated CARBONDATA-880: - Request participants: (was: ) Description: when explain extended is done on a query then store path is getting printed to the user in carbonhadooprelation. the path should not be shown to the user. example : CarbonDatasourceHadoopRelation(org.apache.spark.sql.CarbonSession@1ed6b3b9,[Ljava.lang.String;@21b14680,Map(path -> file:/D:/Carbon/incubator-carbondata/examples/spark2/target/warehouse/carbon_table, serialization.format -> 1, dbname -> default, tablepath -> D:Carbonincubator-carbondata/examples/spark2/target/storedefaultcarbon_table, tablename -> carbon_table),Some(StructType(StructField(shortField,ShortType,true), StructField(intField,IntegerType,true) was: when explain extended is done on a query then store path is getting printed to the user in carbonhadooprelation. the path should not be shown to the user. > when explain extended is done on a query then store path is getting printed > to the user. > > > Key: CARBONDATA-880 > URL: https://issues.apache.org/jira/browse/CARBONDATA-880 > Project: CarbonData > Issue Type: Bug > Components: data-query >Reporter: ravikiran >Assignee: ravikiran > > when explain extended is done on a query then store path is getting printed > to the user in carbonhadooprelation. > the path should not be shown to the user. > example : > CarbonDatasourceHadoopRelation(org.apache.spark.sql.CarbonSession@1ed6b3b9,[Ljava.lang.String;@21b14680,Map(path > -> > file:/D:/Carbon/incubator-carbondata/examples/spark2/target/warehouse/carbon_table, > serialization.format -> 1, dbname -> default, tablepath -> > D:Carbonincubator-carbondata/examples/spark2/target/storedefaultcarbon_table, > tablename -> > carbon_table),Some(StructType(StructField(shortField,ShortType,true), > StructField(intField,IntegerType,true) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Issue Comment Deleted] (CARBONDATA-880) when explain extended is done on a query then store path is getting printed to the user.
[ https://issues.apache.org/jira/browse/CARBONDATA-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ravikiran updated CARBONDATA-880: - Comment: was deleted (was: below is the example : CarbonDatasourceHadoopRelation(org.apache.spark.sql.CarbonSession@1ed6b3b9,[Ljava.lang.String;@21b14680,Map(path -> file:/D:/Carbon/incubator-carbondata/examples/spark2/target/warehouse/carbon_table, serialization.format -> 1, dbname -> default, tablepath -> D:Carbonincubator-carbondata/examples/spark2/target/storedefaultcarbon_table, tablename -> carbon_table),Some(StructType(StructField(shortField,ShortType,true), StructField(intField,IntegerType,true)) > when explain extended is done on a query then store path is getting printed > to the user. > > > Key: CARBONDATA-880 > URL: https://issues.apache.org/jira/browse/CARBONDATA-880 > Project: CarbonData > Issue Type: Bug > Components: data-query >Reporter: ravikiran >Assignee: ravikiran > > when explain extended is done on a query then store path is getting printed > to the user in carbonhadooprelation. > the path should not be shown to the user. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CARBONDATA-880) when explain extended is done on a query then store path is getting printed to the user.
[ https://issues.apache.org/jira/browse/CARBONDATA-880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958877#comment-15958877 ] ravikiran commented on CARBONDATA-880: -- below is the example : CarbonDatasourceHadoopRelation(org.apache.spark.sql.CarbonSession@1ed6b3b9,[Ljava.lang.String;@21b14680,Map(path -> file:/D:/Carbon/incubator-carbondata/examples/spark2/target/warehouse/carbon_table, serialization.format -> 1, dbname -> default, tablepath -> D:Carbonincubator-carbondata/examples/spark2/target/storedefaultcarbon_table, tablename -> carbon_table),Some(StructType(StructField(shortField,ShortType,true), StructField(intField,IntegerType,true) > when explain extended is done on a query then store path is getting printed > to the user. > > > Key: CARBONDATA-880 > URL: https://issues.apache.org/jira/browse/CARBONDATA-880 > Project: CarbonData > Issue Type: Bug > Components: data-query >Reporter: ravikiran >Assignee: ravikiran > > when explain extended is done on a query then store path is getting printed > to the user in carbonhadooprelation. > the path should not be shown to the user. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-875) create database ddl is creating the database folder with case sensitive name.
ravikiran created CARBONDATA-875: Summary: create database ddl is creating the database folder with case sensitive name. Key: CARBONDATA-875 URL: https://issues.apache.org/jira/browse/CARBONDATA-875 Project: CarbonData Issue Type: Bug Components: sql Reporter: ravikiran Assignee: ravikiran Priority: Minor Create database DBNAME. here the database name should be case insensitive. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
