Hi Anurag, Carbon has value based compression for storing the measure column data, in store it will be stored as int. I think only the bug is in desc table as we are updating the data type in schmea.
My query is that to resolve the issue of out of range values being accepted for Int, should we be creating the Int column as Int and not BigInt. Already one discussion is going on for this [Discussion] Parsing values during data load should adopt a strict check or lenient check mechanism -Regards Kumar Vishal On Tue, Dec 27, 2016 at 2:15 PM, Anurag Srivastava <anu...@knoldus.com> wrote: > Hello Raghu, > > We can not validate data on the basis of schema because when we give "Int" > data type it stores in schema as a "BigInt" data type and then it starts > taking range of "BigInt". When we provide "BigInt" data type, it also > stores as a "BigInt". That time it will not be possible to differentiate > between "Int" and "BigInt". > > Thanks. > > Anurag Srivastava > > On Tue, Dec 27, 2016 at 1:02 AM, Raghunandan S < > carbondatacontributi...@gmail.com> wrote: > > > Dear anurag, > > Not required to be stored as int.if we validate the data based on the > > schema definition would suffice > > On Mon, 26 Dec 2016 at 1:10 PM, Anurag Srivastava <anu...@knoldus.com> > > wrote: > > > > > Hello everyone, > > > > > > I am working on resolving jira bug > > > https://issues.apache.org/jira/browse/CARBONDATA-370 > > > While resolving this I found that when I create a table with Int as one > > of > > > the columns and then if I do Desc table, it shows the Int column being > > > created as BigInt :- > > > > > > So, the Int column is created as BigInt in CarbonData. > > > > > > Query : > > > > > > cc.sql(""" > > > CREATE TABLE IF NOT EXISTS t3 > > > (ID BigInt, date Timestamp, country String, > > > name String, phonetype String, serialname char(10), salary Int) > > > STORED BY 'carbondata' > > > """) > > > > > > > > > > > > Output : > > > > > > +---------------+---------------+--------------+ > > > | col_name | data_type | comment| > > > +---------------+---------------+--------------+ > > > | id | bigint | | > > > | date | timestamp | | > > > | country | string | | > > > | name | string | | > > > | phonetype| string | | > > > |serialname| string | | > > > | salary | bigint | | > > > +--------------+-----------------+-------------+ > > > > > > > > > My query is that to resolve the issue of out of range values being > > accepted > > > for Int, should we be creating the Int column as Int and not BigInt. > > > > > > Thanks and Regards > > > Anurag Srivastava > > > > > > On Fri, Nov 4, 2016 at 11:58 AM, manish gupta < > tomanishgupt...@gmail.com > > > > > > wrote: > > > > > > > Hi Pallavi, > > > > > > > > This is a bug in the system. We store Int as bigInt but value beyond > > the > > > > integer range should not be allowed for storage in integer datatype > > > column. > > > > From my opinion you can raise a Jira issue for this. > > > > > > > > Regards > > > > Manish Gupta > > > > > > > > On Thu, Nov 3, 2016 at 11:06 PM, Kumar Vishal < > > kumarvishal1...@gmail.com > > > > > > > > wrote: > > > > > > > > > Hi Pallavi, > > > > > Currently in carbon int data type is stored as big > > int. > > > > Can > > > > > u please re-upload the image. > > > > > > > > > > -Regards > > > > > Kumar Vishal > > > > > > > > > > On Thu, Nov 3, 2016 at 10:09 AM, Pallavi Singh < > > > pallavi.si...@knoldus.in > > > > > > > > > > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > I would like to know when we execute the query : > > > > > > create table employee(id int, name String) stored by > 'carbondata'; > > > > > > the table created is : > > > > > > > > > > > > [image: Inline image 1] > > > > > > > > > > > > the Int is converted into BigInt and Int supports range beyond > > > > > > (-2147483648 to 2147483647) > > > > > > -- > > > > > > Regards | Pallavi Singh > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > *Thanks®ards* > > > > > > > > > *Anurag Srivastava**Software Consultant* > > > *Knoldus Software LLP* > > > > > > *India - US - Canada* > > > * Twitter <http://www.twitter.com/anuragknoldus> | FB > > > <http://www.facebook.com/anuragsrivastava.06> | LinkedIn > > > <https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>* > > > > > > > > > -- > *Thanks®ards* > > > *Anurag Srivastava**Software Consultant* > *Knoldus Software LLP* > > *India - US - Canada* > * Twitter <http://www.twitter.com/anuragknoldus> | FB > <http://www.facebook.com/anuragsrivastava.06> | LinkedIn > <https://in.linkedin.com/pub/anurag-srivastava/5a/b6/441>* >
