[jira] [Created] (CARBONDATA-672) Complex data type is not working while fetching it from Database
Anurag Srivastava created CARBONDATA-672:
Summary: Complex data type is not working while fetching it from
Database
Key: CARBONDATA-672
URL: https://issues.apache.org/jira/browse/CARBONDATA-672
Project: CarbonData
Issue Type: Bug
Components: data-query
Affects Versions: 1.0.0-incubating
Environment: Spark - 2.1
Reporter: Anurag Srivastava
Attachments: complex_data_type.png, ct5.csv, log, pass_fetch.png
I created a table with an complex data type and then load data into it. Data
loaded successfully but when I am trying to fetch data it is giving me error.
*Create Table :*
create table ct10(id Int,data array)stored by 'carbondata';
*Load Data :*
LOAD DATA inpath 'hdfs://localhost:54310/ct4.csv' INTO table ct1
options('DELIMITER'=',', 'FILEHEADER'='id,
data','QUOTECHAR'='"','COMPLEX_DELIMITER_LEVEL_1'='#');
*Data Query :*
Select * from ct10;
*Error :*
While when we use query *select id from ct10;*, its working fine.
I am attaching screen shot, CSV and log with it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CARBONDATA-663) Major compaction is not woking properly
Anurag Srivastava created CARBONDATA-663:
Summary: Major compaction is not woking properly
Key: CARBONDATA-663
URL: https://issues.apache.org/jira/browse/CARBONDATA-663
Project: CarbonData
Issue Type: Bug
Components: data-query
Affects Versions: 1.0.0-incubating
Environment: Spark - 2.1
Reporter: Anurag Srivastava
Attachments: logs, sample_str_more1.csv, show_segment.png,
show_segments_after_compaction.png
I have set property *carbon.major.compaction.size= 3* and load data which is
the size of 5 MB and when I perform compaction it compacted, but initially it
shouldn't be perform. Here is the queries :
*create table :* create table test_major_compaction(id Int,name string)stored
by 'carbondata';
*Load Data :* Load two segments.
LOAD DATA inpath 'hdfs://localhost:54310/sample_str_more1.csv' INTO table
test_major_compaction options('DELIMITER'=',', 'FILEHEADER'='id,
name','QUOTECHAR'='"');
*Show segments :* show segments for table test_major_compaction;
*Alter Table :* ALTER TABLE test_major_compaction COMPACT 'MAJOR';
*Show segments :* Again see the segments :
show segments for table test_major_compaction;
I have attached all the data with the it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CARBONDATA-649) Rand() function is not working while updating data
Anurag Srivastava created CARBONDATA-649:
Summary: Rand() function is not working while updating data
Key: CARBONDATA-649
URL: https://issues.apache.org/jira/browse/CARBONDATA-649
Project: CarbonData
Issue Type: Bug
Components: data-query
Affects Versions: 1.0.0-incubating
Environment: Spark 1.6
Reporter: Anurag Srivastava
Priority: Minor
I am using update functionality with the *rand(1)* and *rand()* which return
deterministic value or random value.
But as I run query it gives error.
*Create Table :* CREATE TABLE uniqdata_date_dictionary (CUST_ID int,CUST_NAME
string,ACTIVE_EMUI_VERSION string, DOB date, DOJ date, BIGINT_COLUMN1
bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"=
"256 MB","DICTIONARY_EXCLUDE"="DOB,DOJ");
*Load Data :* LOAD DATA INPATH 'hdfs://localhost:54310/2000_UniqData.csv' into
table uniqdata OPTIONS ('DELIMITER'=','
,'QUOTECHAR'='""','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','MAXCOLUMNS'='12');
*Query-1 :* Update uniqdata set (decimal_column1) = (rand());
*Query-2 :* Update uniqdata set (decimal_column1) = (rand(1));
Expected Result : Table created.
Actual Result : Error:
org.apache.carbondata.spark.exception.MalformedCarbonCommandException:
DICTIONARY_EXCLUDE is unsupported for date data type column: dob (state=,code=0)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CARBONDATA-641) DICTIONARY_EXCLUDE is not working with 'DATE' column
Anurag Srivastava created CARBONDATA-641:
Summary: DICTIONARY_EXCLUDE is not working with 'DATE' column
Key: CARBONDATA-641
URL: https://issues.apache.org/jira/browse/CARBONDATA-641
Project: CarbonData
Issue Type: Bug
Components: core
Affects Versions: 1.0.0-incubating
Environment: Spark - 1.6 and Spark - 2.1
Reporter: Anurag Srivastava
I am trying to create a table with *"DICTIONARY_EXCLUDE"* and this property is
now working for *"DATE"* Data Type.
*Query :* CREATE TABLE uniqdata_date_dictionary (CUST_ID int,CUST_NAME
string,ACTIVE_EMUI_VERSION string, DOB date, DOJ date, BIGINT_COLUMN1
bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"=
"256 MB","DICTIONARY_EXCLUDE"="DOB,DOJ");
*Expected Result :* Table created.
*Actual Result :* Error:
org.apache.carbondata.spark.exception.MalformedCarbonCommandException:
DICTIONARY_EXCLUDE is unsupported for date data type column: dob (state=,code=0)
But is is working fine, If I use 'TIMESTAMP' in place of 'DATE'.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Unable to Assign Jira to me
Hello Team, I am working on JIRA [CARBONDATA-542] and want to assign this JIRA to me. But I am not able to assign it to me. How can I assign it to me ? -- *Thanks®ards* *Anurag Srivastava**Software Consultant* *Knoldus Software LLP* *India - US - Canada* *Twitter <http://www.twitter.com/anuragknoldus> | FB <http://www.facebook.com/anuragsrivastava.06> | LinkedIn <https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>*
[jira] [Created] (CARBONDATA-630) Unable to use string function on string/char data type column
Anurag Srivastava created CARBONDATA-630:
Summary: Unable to use string function on string/char data type
column
Key: CARBONDATA-630
URL: https://issues.apache.org/jira/browse/CARBONDATA-630
Project: CarbonData
Issue Type: Bug
Components: data-query
Affects Versions: 1.0.0-incubating
Environment: SPARK-2.1.0
Reporter: Anurag Srivastava
Priority: Minor
Attachments: 2000_UniqData.csv, exception.png
I am trying to execute string function like: reverse, concat, lower, upper with
the string/char column but it is giving error and when I am giving direct
string value to it, it is working.
*Create Table :* CREATE TABLE uniqdata_char (CUST_ID int,CUST_NAME
char(30),ACTIVE_EMUI_VERSION char(30), DOB timestamp, DOJ timestamp,
BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10),
DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2
double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format'
TBLPROPERTIES ('TABLE_BLOCKSIZE'= '256 MB');
*Load Data : * LOAD DATA INPATH 'hdfs://localhost:54310/2000_UniqData.csv' into
table uniqdata_char OPTIONS ('DELIMITER'=','
,'QUOTECHAR'='""','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','MAXCOLUMNS'='12');
*Query :* select Lower(cust_name) from uniqdata_char;
After running the query I am getting error.
But when I am running :
select Lower('TESTING') from uniqdata_char;
It is working fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CARBONDATA-615) Update query store wrong value for Date data type
Anurag Srivastava created CARBONDATA-615:
Summary: Update query store wrong value for Date data type
Key: CARBONDATA-615
URL: https://issues.apache.org/jira/browse/CARBONDATA-615
Project: CarbonData
Issue Type: Bug
Components: data-query
Affects Versions: 1.0.0-incubating
Reporter: Anurag Srivastava
Priority: Minor
Attachments: 2000_UniqData.csv, update_dob.png
I am trying to update DOB column with Date Data Type. It is storing a day
before date which I have mentioned for updating in DOB column.
*Create Table :* CREATE TABLE uniqdata (CUST_ID int,CUST_NAME
char(30),ACTIVE_EMUI_VERSION string, DOB Date, DOJ Date, BIGINT_COLUMN1
bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double, INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format';
*Load Data :* LOAD DATA INPATH 'hdfs://localhost:54310/2000_UniqData.csv' into
table uniqdata OPTIONS ('DELIMITER'=','
,'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','SINGLE_PASS'='true');
*Update Query :* update uniqdata set (dob)=(to_date('2016-12-01')) where
cust_name = 'CUST_NAME_01999';
*Expected Result :* It should update DOB column with *2016-12-01*.
*Actual Result :* It is updating DOB column with *2016-11-30*.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CARBONDATA-612) Bucket is not support in Spark-1.6
Anurag Srivastava created CARBONDATA-612: Summary: Bucket is not support in Spark-1.6 Key: CARBONDATA-612 URL: https://issues.apache.org/jira/browse/CARBONDATA-612 Project: CarbonData Issue Type: Bug Components: data-query Affects Versions: 1.0.0-incubating Reporter: Anurag Srivastava Priority: Minor I am trying to use bucket feature on Spark-1.6 and It create table on Spark-1.6 but as I know that Bucket functionality support from Spark-2.x. So when we are trying to create table with bucket in Spark-1.6, it should provide error message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-603) Unable to use filter with Date Data Type
Anurag Srivastava created CARBONDATA-603:
Summary: Unable to use filter with Date Data Type
Key: CARBONDATA-603
URL: https://issues.apache.org/jira/browse/CARBONDATA-603
Project: CarbonData
Issue Type: Bug
Components: data-query
Affects Versions: 1.0.0-incubating
Reporter: Anurag Srivastava
Attachments: Date.png
I am creating table with *DATE* Data Type and loading data with CSV into the
table.
After that as I run the select query with *WHERE* clause, it converted value as
NULL and provide me result with Null Value.
*Create Table :* CREATE TABLE uniqdata (CUST_ID int,CUST_NAME
char(30),ACTIVE_EMUI_VERSION string, DOB date, DOJ date, BIGINT_COLUMN1
bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"=
"256 MB");
*Load Data :* LOAD DATA INPATH 'hdfs://localhost:54310/2000_UniqData.csv' into
table uniqdata OPTIONS('DELIMITER'=','
,'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
*Select Query :* Select cust_id, cust_name, dob from uniqdata where
dob='1975-06-22';
It is working fine on hive. I am attaching CSV
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Select query is not working.
Hello, I have taken latest code at today (5/01/2017) and build code with spark 1.6. After that I put the latest jar in carbonlib in spark and start thrift server. When I have started running query I was able to run the create and load query but for the "*select*" query it is giving me error : *org.apache.carbondata.core.carbon.datastore.exception.IndexBuilderException: Block B-tree loading failed* I have raised JIRA ISSUE for the same. Please look at there for further information and stack trace. Here is the link : https://issues.apache.org/jira/browse/CARBONDATA-597 -- *Thanks®ards* *Anurag Srivastava**Software Consultant* *Knoldus Software LLP* *India - US - Canada* * Twitter <http://www.twitter.com/anuragknoldus> | FB <http://www.facebook.com/anuragsrivastava.06> | LinkedIn <https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>*
[jira] [Created] (CARBONDATA-597) Unable to fetch data with "select" query
Anurag Srivastava created CARBONDATA-597: Summary: Unable to fetch data with "select" query Key: CARBONDATA-597 URL: https://issues.apache.org/jira/browse/CARBONDATA-597 Project: CarbonData Issue Type: Bug Components: data-query Affects Versions: 1.0.0-incubating Reporter: Anurag Srivastava Attachments: ErrorLog.png I am running Carbon Data with beeline and I am able to Create Table and Load Data but as I run *select * from table_name;*, Its giving me error : *Block B-tree loading failed* PFA for stack Trace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-586) Create table with 'Char' data type but it workes as 'String' data type
Anurag Srivastava created CARBONDATA-586:
Summary: Create table with 'Char' data type but it workes as
'String' data type
Key: CARBONDATA-586
URL: https://issues.apache.org/jira/browse/CARBONDATA-586
Project: CarbonData
Issue Type: Bug
Components: data-load
Affects Versions: 1.0.0-incubating
Environment: Cluster
Reporter: Anurag Srivastava
Priority: Minor
I am trying to use Char data type with Carbon Data latest version and it
created successfully. When I started loading data in this that time I found
that it is taking data more then its size.
I have checked it with hive and there it is working fine.
EX :-
1. *Carbon Data :*
1.1 create table test_carbon (name char(10)) stored by
'org.apache.carbondata.format';
1.2 desc test_carbon;
*Output :*
+-+--+--+--+
| col_name | data_type | comment |
+-+--+--+
| name| string | |
+-+--+--+
1.3 LOAD DATA INPATH 'hdfs://localhost:54310/test.csv' into table test_carbon
OPTIONS ('FILEHEADER'='name');
1.4 select * from test_carbon;
*Output :*
++
|name |
++
| Anurag Srivasrata |
| Robert|
| james james |
++
2. *Hive :*
2.1 create table test_hive (name char(10));
2.2 desc test_hive;
*Output :*
+-+--+-+
| col_name | data_type | comment |
+-+--+-+
| name| char(10)| NULL |
+-+--+-+
2.3 LOAD DATA INPATH 'hdfs://localhost:54310/test.csv' into table test_hive;
2.4 select * from test_hive;
*Output :*
++
|name |
++
| james jame |
| Anurag Sri|
| Robert |
++
So as hive truncate remaining string with Char data type in carbon data it
should work like hive.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CARBONDATA-583) Replace Function is not working for string/char
Anurag Srivastava created CARBONDATA-583:
Summary: Replace Function is not working for string/char
Key: CARBONDATA-583
URL: https://issues.apache.org/jira/browse/CARBONDATA-583
Project: CarbonData
Issue Type: Bug
Affects Versions: 1.0.0-incubating
Reporter: Anurag Srivastava
Priority: Minor
I am running "replace" function but it is giving error : "undefined function
replace".
Query : select replace('aaabbccaabb', 'aaa', 't');
Expected Result : "tbbccaabb"
Result : Error: org.apache.spark.sql.AnalysisException: undefined function
replace; line 1 pos 30 (state=,code=0)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Int accepting range of BigInt
Hello Raghu,
We can not validate data on the basis of schema because when we give "Int"
data type it stores in schema as a "BigInt" data type and then it starts
taking range of "BigInt". When we provide "BigInt" data type, it also
stores as a "BigInt". That time it will not be possible to differentiate
between "Int" and "BigInt".
Thanks.
Anurag Srivastava
On Tue, Dec 27, 2016 at 1:02 AM, Raghunandan S <
carbondatacontributi...@gmail.com> wrote:
> Dear anurag,
> Not required to be stored as int.if we validate the data based on the
> schema definition would suffice
> On Mon, 26 Dec 2016 at 1:10 PM, Anurag Srivastava
> wrote:
>
> > Hello everyone,
> >
> > I am working on resolving jira bug
> > https://issues.apache.org/jira/browse/CARBONDATA-370
> > While resolving this I found that when I create a table with Int as one
> of
> > the columns and then if I do Desc table, it shows the Int column being
> > created as BigInt :-
> >
> > So, the Int column is created as BigInt in CarbonData.
> >
> > Query :
> >
> > cc.sql("""
> >CREATE TABLE IF NOT EXISTS t3
> >(ID BigInt, date Timestamp, country String,
> >name String, phonetype String, serialname char(10), salary Int)
> >STORED BY 'carbondata'
> >""")
> >
> >
> >
> > Output :
> >
> > +---+---+--+
> > | col_name | data_type | comment|
> > +---+---+--+
> > |id | bigint | |
> > | date| timestamp | |
> > | country | string | |
> > | name | string | |
> > | phonetype| string | |
> > |serialname| string | |
> > |salary | bigint | |
> > +--+-+-+
> >
> >
> > My query is that to resolve the issue of out of range values being
> accepted
> > for Int, should we be creating the Int column as Int and not BigInt.
> >
> > Thanks and Regards
> > Anurag Srivastava
> >
> > On Fri, Nov 4, 2016 at 11:58 AM, manish gupta >
> > wrote:
> >
> > > Hi Pallavi,
> > >
> > > This is a bug in the system. We store Int as bigInt but value beyond
> the
> > > integer range should not be allowed for storage in integer datatype
> > column.
> > > From my opinion you can raise a Jira issue for this.
> > >
> > > Regards
> > > Manish Gupta
> > >
> > > On Thu, Nov 3, 2016 at 11:06 PM, Kumar Vishal <
> kumarvishal1...@gmail.com
> > >
> > > wrote:
> > >
> > > > Hi Pallavi,
> > > >Currently in carbon int data type is stored as big
> int.
> > > Can
> > > > u please re-upload the image.
> > > >
> > > > -Regards
> > > > Kumar Vishal
> > > >
> > > > On Thu, Nov 3, 2016 at 10:09 AM, Pallavi Singh <
> > pallavi.si...@knoldus.in
> > > >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I would like to know when we execute the query :
> > > > > create table employee(id int, name String) stored by 'carbondata';
> > > > > the table created is :
> > > > >
> > > > > [image: Inline image 1]
> > > > >
> > > > > the Int is converted into BigInt and Int supports range beyond
> > > > > (-2147483648 to 2147483647)
> > > > > --
> > > > > Regards | Pallavi Singh
> > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > *Thanks®ards*
> >
> >
> > *Anurag Srivastava**Software Consultant*
> > *Knoldus Software LLP*
> >
> > *India - US - Canada*
> > * Twitter <http://www.twitter.com/anuragknoldus> | FB
> > <http://www.facebook.com/anuragsrivastava.06> | LinkedIn
> > <https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>*
> >
>
--
*Thanks®ards*
*Anurag Srivastava**Software Consultant*
*Knoldus Software LLP*
*India - US - Canada*
* Twitter <http://www.twitter.com/anuragknoldus> | FB
<http://www.facebook.com/anuragsrivastava.06> | LinkedIn
<https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>*
Re: Int accepting range of BigInt
Hello everyone,
I am working on resolving jira bug
https://issues.apache.org/jira/browse/CARBONDATA-370
While resolving this I found that when I create a table with Int as one of
the columns and then if I do Desc table, it shows the Int column being
created as BigInt :-
So, the Int column is created as BigInt in CarbonData.
Query :
cc.sql("""
CREATE TABLE IF NOT EXISTS t3
(ID BigInt, date Timestamp, country String,
name String, phonetype String, serialname char(10), salary Int)
STORED BY 'carbondata'
""")
Output :
+---+---+--+
| col_name | data_type | comment|
+---+---+--+
|id | bigint | |
| date| timestamp | |
| country | string | |
| name | string | |
| phonetype| string | |
|serialname| string | |
|salary | bigint | |
+--+-+-+
My query is that to resolve the issue of out of range values being accepted
for Int, should we be creating the Int column as Int and not BigInt.
Thanks and Regards
Anurag Srivastava
On Fri, Nov 4, 2016 at 11:58 AM, manish gupta
wrote:
> Hi Pallavi,
>
> This is a bug in the system. We store Int as bigInt but value beyond the
> integer range should not be allowed for storage in integer datatype column.
> From my opinion you can raise a Jira issue for this.
>
> Regards
> Manish Gupta
>
> On Thu, Nov 3, 2016 at 11:06 PM, Kumar Vishal
> wrote:
>
> > Hi Pallavi,
> >Currently in carbon int data type is stored as big int.
> Can
> > u please re-upload the image.
> >
> > -Regards
> > Kumar Vishal
> >
> > On Thu, Nov 3, 2016 at 10:09 AM, Pallavi Singh >
> > wrote:
> >
> > > Hi,
> > >
> > > I would like to know when we execute the query :
> > > create table employee(id int, name String) stored by 'carbondata';
> > > the table created is :
> > >
> > > [image: Inline image 1]
> > >
> > > the Int is converted into BigInt and Int supports range beyond
> > > (-2147483648 to 2147483647)
> > > --
> > > Regards | Pallavi Singh
> > >
> > >
> >
>
--
*Thanks®ards*
*Anurag Srivastava**Software Consultant*
*Knoldus Software LLP*
*India - US - Canada*
* Twitter <http://www.twitter.com/anuragknoldus> | FB
<http://www.facebook.com/anuragsrivastava.06> | LinkedIn
<https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>*
[jira] [Created] (CARBONDATA-551) Implement unit test cases for classes in processing package
Anurag Srivastava created CARBONDATA-551: Summary: Implement unit test cases for classes in processing package Key: CARBONDATA-551 URL: https://issues.apache.org/jira/browse/CARBONDATA-551 Project: CarbonData Issue Type: Test Reporter: Anurag Srivastava Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-543) Implement unit test cases for DataBlockIteratorImpl, IntermediateFileMerger and SortDataRows classes
Anurag Srivastava created CARBONDATA-543: Summary: Implement unit test cases for DataBlockIteratorImpl, IntermediateFileMerger and SortDataRows classes Key: CARBONDATA-543 URL: https://issues.apache.org/jira/browse/CARBONDATA-543 Project: CarbonData Issue Type: Test Reporter: Anurag Srivastava Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-541) Implement unit test cases for processing.newflow.dictionary package
Anurag Srivastava created CARBONDATA-541: Summary: Implement unit test cases for processing.newflow.dictionary package Key: CARBONDATA-541 URL: https://issues.apache.org/jira/browse/CARBONDATA-541 Project: CarbonData Issue Type: Test Reporter: Anurag Srivastava Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Drop Database Unexpected Behaviour
Hi, I have created a database in hive and Carbondata then dropped both databases when I executed *show table* in hive, it gives an exception that database does not exists but in carbon data it does not give any exception. *Output in Hive -* hive> CREATE DATABASE SAMPLE; OK hive (default)> use SAMPLE; OK hive (SAMPLE)> drop database SAMPLE; OK hive (SAMPLE)> use SAMPLE; FAILED: SemanticException [Error 10072]: Database does not exist: SAMPLE hive (SAMPLE)> show tables; FAILED: SemanticException [Error 10072]: Database does not exist: SAMPLE *Output in Carbondata :* 0: jdbc:hive2://localhost:1> CREATE DATABASE SAMPLE; 0: jdbc:hive2://localhost:1> USE SAMPLE; 0: jdbc:hive2://localhost:1> DROP DATABASE SAMPLE; 0: jdbc:hive2://localhost:1> USE SAMPLE; Error: org.apache.spark.sql.execution.QueryExecutionException: FAILED: SemanticException [Error 10072]: Database does not exist: SAMPLE (state=,code=0) 0: jdbc:hive2: 0: jdbc:hive2://localhost:1>show tables; tableName isTemporary No rows selected (0.066 seconds) *Proposed Solution : * When we looked into the problem we found that it delete all the table and information from the database and made it empty so that we could see any record there and we find empty result on console. We can add a flag with database because if it delete all the record, it will quite difficult to know that we drop the database or create the database because in both scenario we don't have any table. So for the handle this situation we can set flag = 1 when we create a database or set flag = 0 when we drop the database and return a message or exception on the console when user run *show table* command. -- *Thanks®ards* *Anurag Srivastava**Software Consultant* *Knoldus Software LLP* *India - US - Canada* * Twitter <http://www.twitter.com/anuragknoldus> | FB <http://www.facebook.com/anuragsrivastava.06> | LinkedIn <https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>*
Support of Float Data Type in Carbon Data
Hi, Carbon Data is not supporting Float Data type. Do we need to fix this Jira issue [CARBONDATA-390] <https://issues.apache.org/jira/browse/CARBONDATA-390>? I think float data type should have its own range. So do we need to support range for Float data type? Proposed Solution : We have to make changes in the following file : We have to provide support during parser in CarbonSqlParser class. We have to add Float Data Type in the DataTypeConverterUtil. -- *Thanks®ards* *Anurag Srivastava**Software Consultant* *Knoldus Software LLP* *India - US - Canada* * Twitter <http://www.twitter.com/anuragknoldus> | FB <http://www.facebook.com/anuragsrivastava.06> | LinkedIn <https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>*
[jira] [Created] (CARBONDATA-515) Implement unit test cases for processing.newflow.converter package
Anurag Srivastava created CARBONDATA-515: Summary: Implement unit test cases for processing.newflow.converter package Key: CARBONDATA-515 URL: https://issues.apache.org/jira/browse/CARBONDATA-515 Project: CarbonData Issue Type: Test Reporter: Anurag Srivastava Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-496) Implement unit test cases for core.carbon.datastore package
Anurag Srivastava created CARBONDATA-496: Summary: Implement unit test cases for core.carbon.datastore package Key: CARBONDATA-496 URL: https://issues.apache.org/jira/browse/CARBONDATA-496 Project: CarbonData Issue Type: Test Reporter: Anurag Srivastava Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-494) Implement unit test cases for filter.executer package
Anurag Srivastava created CARBONDATA-494: Summary: Implement unit test cases for filter.executer package Key: CARBONDATA-494 URL: https://issues.apache.org/jira/browse/CARBONDATA-494 Project: CarbonData Issue Type: Test Reporter: Anurag Srivastava Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-475) Implement unit test cases for core.carbon.querystatics package
Anurag Srivastava created CARBONDATA-475: Summary: Implement unit test cases for core.carbon.querystatics package Key: CARBONDATA-475 URL: https://issues.apache.org/jira/browse/CARBONDATA-475 Project: CarbonData Issue Type: Bug Reporter: Anurag Srivastava Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-474) Implement unit test cases for core.datastorage.store.columnar package
Anurag Srivastava created CARBONDATA-474: Summary: Implement unit test cases for core.datastorage.store.columnar package Key: CARBONDATA-474 URL: https://issues.apache.org/jira/browse/CARBONDATA-474 Project: CarbonData Issue Type: Bug Reporter: Anurag Srivastava Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Please vote and advise on building thrift files
+1 for proposal 1 On Thu, Nov 17, 2016 at 3:56 PM, Kumar Vishal wrote: > +1 for proposal 1 > > -Regards > Kumar Vishal > > On Nov 17, 2016 09:58, "金铸" wrote: > > > +1 for proposal 1 > > > > > > 在 2016/11/17 12:13, 邢冰 写道: > > > >> +1 for proposal 1 > >> > >> thx > >> > >> > >> > >> > >> 发自网易邮箱大师 > >> On 11/17/2016 12:09, Ravindra Pesala wrote: > >> +1 for proposal 1 > >> > >> On 17 November 2016 at 08:23, Xiaoqiao He wrote: > >> > >> +1 for proposal 1. > >>> > >>> On Thu, Nov 17, 2016 at 10:31 AM, ZhuWilliam > >>> wrote: > >>> > >>> +1 for proposal 1 . > >>>> > >>>> Auto generated code should not be added to project. Also most the of > >>>> time > >>>> ,people who dive into carbondata may not touch format code. > >>>> > >>>> > >>>> > >>>> -- > >>>> View this message in context: http://apache-carbondata- > >>>> mailing-list-archive.1130556.n5.nabble.com/Please-vote-and- > >>>> advise-on-building-thrift-files-tp2952p2957.html > >>>> Sent from the Apache CarbonData Mailing List archive mailing list > >>>> archive > >>>> at Nabble.com. > >>>> > >>>> > >> > >> -- > >> Thanks & Regards, > >> Ravi > >> > > > > > > > > > > > > --- > > Confidentiality Notice: The information contained in this e-mail and any > > accompanying attachment(s) > > is intended only for the use of the intended recipient and may be > > confidential and/or privileged of > > Neusoft Corporation, its subsidiaries and/or its affiliates. If any > reader > > of this communication is > > not the intended recipient, unauthorized use, forwarding, printing, > > storing, disclosure or copying > > is strictly prohibited, and may be unlawful.If you have received this > > communication in error,please > > immediately notify the sender by return e-mail, and delete the original > > message and all copies from > > your system. Thank you. > > > > --- > > > -- *Thanks®ards* *Anurag Srivastava**Software Consultant* *Knoldus Software LLP* *India - US - Canada* * Twitter <http://www.twitter.com/anuragknoldus> | FB <http://www.facebook.com/anuragsrivastava.06> | LinkedIn <https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>*
[jira] [Created] (CARBONDATA-413) Implement unit test cases for scan.expression package
Anurag Srivastava created CARBONDATA-413: Summary: Implement unit test cases for scan.expression package Key: CARBONDATA-413 URL: https://issues.apache.org/jira/browse/CARBONDATA-413 Project: CarbonData Issue Type: Improvement Reporter: Anurag Srivastava Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-397) Use of ANTLR instead of CarbonSqlParser for parsing queries
Anurag Srivastava created CARBONDATA-397: Summary: Use of ANTLR instead of CarbonSqlParser for parsing queries Key: CARBONDATA-397 URL: https://issues.apache.org/jira/browse/CARBONDATA-397 Project: CarbonData Issue Type: Improvement Reporter: Anurag Srivastava Priority: Minor We are using CarbonSqlParser for parsing queries but we can use ANTLR for the same. we could better handle query parsing with ANTLR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-396) Implement test cases for datastorage package
Anurag Srivastava created CARBONDATA-396: Summary: Implement test cases for datastorage package Key: CARBONDATA-396 URL: https://issues.apache.org/jira/browse/CARBONDATA-396 Project: CarbonData Issue Type: Test Reporter: Anurag Srivastava Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Use of ANTLR instead of CarbonSqlParser
Hi, We can use ANTLR with Spark1.5/1.6. There is no compatibility issue. We can replace the carbon parser with the ANTLR even when CarbonData is integrated with Spark 1.5/1.6. And when we are integrating with Spark 2.0, not much changes will be required in the ANTLR parser. But, if integration with Spark 2.0 is planned in near future, we can switch to ANTLR parser at that time as well. On Mon, Nov 7, 2016 at 6:59 AM, Jacky Li wrote: > Hi, > > It is because CarbonData currently is integrated with Spark 1.5/1.6 and > CarbonContext is based on HiveContext, so it is based on the Hive parser in > HiveContext. But you are right, there is no design limitation about this, > Carbon can switch to use ANTLR. I see in Spark 2.0, spark is using ANTLR as > well. So maybe it is a good time to switch the parser when doing > integration with Spark 2.0. > What is your idea? > > Regards, > Jacky > > > > 在 2016年11月4日，下午3:47，Anurag Srivastava 写道： > > > > Hi, > > > > We are using CarbonSqlParser for parsing the queries but we can use ANTLR > > for the same. > > > > Is there any specific reason for using CarbonSqlParser because as per my > > concern we could better handle with ANTLR. > > > > > > -- > > *Thanks®ards* > > > > > > *Anurag Srivastava**Software Consultant* > > *Knoldus Software LLP* > > > > *India - US - Canada* > > * Twitter <http://www.twitter.com/anuragknoldus> | FB > > <http://www.facebook.com/anuragsrivastava.06> | LinkedIn > > <https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>* > > -- *Thanks®ards* *Anurag Srivastava**Software Consultant* *Knoldus Software LLP* *India - US - Canada* * Twitter <http://www.twitter.com/anuragknoldus> | FB <http://www.facebook.com/anuragsrivastava.06> | LinkedIn <https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>*
Re: Int accepting range of BigInt
Please find Image. On Fri, Nov 4, 2016 at 1:31 PM, Pallavi Singh wrote: > > > > On Fri, Nov 4, 2016 at 1:28 PM, Sangeeta Gulia > wrote: > >> Please find the image attached. >> >> On Fri, Nov 4, 2016 at 1:24 PM, Pallavi Singh >> wrote: >> >>> Please find the image as an attachment >>> >>> On Thu, Nov 3, 2016 at 11:06 PM, Kumar Vishal >> > wrote: >>> >>>> Hi Pallavi, >>>>Currently in carbon int data type is stored as big int. >>>> Can >>>> u please re-upload the image. >>>> >>>> -Regards >>>> Kumar Vishal >>>> >>>> On Thu, Nov 3, 2016 at 10:09 AM, Pallavi Singh < >>>> pallavi.si...@knoldus.in> >>>> wrote: >>>> >>>> > Hi, >>>> > >>>> > I would like to know when we execute the query : >>>> > create table employee(id int, name String) stored by 'carbondata'; >>>> > the table created is : >>>> > >>>> > [image: Inline image 1] >>>> > >>>> > the Int is converted into BigInt and Int supports range beyond >>>> > (-2147483648 to 2147483647) >>>> > -- >>>> > Regards | Pallavi Singh >>>> > >>>> > >>>> >>> >>> >>> >>> -- >>> Regards | Pallavi Singh >>> Software Consultant >>> Knoldus Software LLP >>> pallavi.si...@knoldus.in >>> +91-9911235949 >>> >> >> >> >> -- >> Warm Regards, >> >> Sangeeta Gulia >> Software Consultant >> Knoldus Software LLP >> > > > > -- > Regards | Pallavi Singh > Software Consultant > Knoldus Software LLP > pallavi.si...@knoldus.in > +91-9911235949 > -- *Thanks®ards* *Anurag Srivastava**Software Consultant* *Knoldus Software LLP* *India - US - Canada* * Twitter <http://www.twitter.com/anuragknoldus> | FB <http://www.facebook.com/anuragsrivastava.06> | LinkedIn <https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>*
Use of ANTLR instead of CarbonSqlParser
Hi, We are using CarbonSqlParser for parsing the queries but we can use ANTLR for the same. Is there any specific reason for using CarbonSqlParser because as per my concern we could better handle with ANTLR. -- *Thanks®ards* *Anurag Srivastava**Software Consultant* *Knoldus Software LLP* *India - US - Canada* * Twitter <http://www.twitter.com/anuragknoldus> | FB <http://www.facebook.com/anuragsrivastava.06> | LinkedIn <https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>*
[jira] [Created] (CARBONDATA-340) Implement test cases for load package in core module
Anurag Srivastava created CARBONDATA-340: Summary: Implement test cases for load package in core module Key: CARBONDATA-340 URL: https://issues.apache.org/jira/browse/CARBONDATA-340 Project: CarbonData Issue Type: Test Reporter: Anurag Srivastava Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Subscribe mailing list
Hello , I want add my mail in your mailing list. -- *Thanks®ards* *Anurag Srivastava**Software Consultant* *Knoldus Software LLP* *India - US - Canada* * Twitter <http://www.twitter.com/anuragknoldus> | FB <http://www.facebook.com/anuragsrivastava.06> | LinkedIn <https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>*
