[jira] [Created] (CARBONDATA-4052) Select query on SI table after insert overwrite is giving wrong result.

2020-11-22 Thread Nihal kumar ojha (Jira)
Nihal kumar ojha created CARBONDATA-4052:


 Summary: Select query on SI table after insert overwrite is giving 
wrong result.
 Key: CARBONDATA-4052
 URL: https://issues.apache.org/jira/browse/CARBONDATA-4052
 Project: CarbonData
  Issue Type: Bug
Reporter: Nihal kumar ojha


# Create carbon table.
 # Create SI table on the same carbon table.
 # Do load or insert operation.
 # Run query insert overwrite on maintable.
 # Now select query on SI table is showing old as well as new data which should 
be only new data.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-4046) Select count(*) fails on partition table.

2020-11-02 Thread Nihal kumar ojha (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-4046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nihal kumar ojha updated CARBONDATA-4046:
-
Description: 
Steps to reproduce

1. set property `carbon.read.partition.hive.direct=false`

2. Create table which contain more than one partition column.

3. run query select count (*)

 

It fails with exception as `Key not found`.

 

create table partition_cache(a string) partitioned by(b int, c String) stored 
as carbondata;

insert into partition_cache select 'k',1,'nihal';

select count(*) from partition_cache where b = 1;

  was:
Steps to reproduce

1. set property `carbon.read.partition.hive.direct=false`

2. Create table which contain more than one partition column.

3. run query select count (*)

 

It fails with exception as `Key not found`.


> Select count(*) fails on partition table.
> -
>
> Key: CARBONDATA-4046
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4046
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Nihal kumar ojha
>Priority: Major
>
> Steps to reproduce
> 1. set property `carbon.read.partition.hive.direct=false`
> 2. Create table which contain more than one partition column.
> 3. run query select count (*)
>  
> It fails with exception as `Key not found`.
>  
> create table partition_cache(a string) partitioned by(b int, c String) stored 
> as carbondata;
> insert into partition_cache select 'k',1,'nihal';
> select count(*) from partition_cache where b = 1;



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (CARBONDATA-4046) Select count(*) fails on partition table.

2020-11-02 Thread Nihal kumar ojha (Jira)
Nihal kumar ojha created CARBONDATA-4046:


 Summary: Select count(*) fails on partition table.
 Key: CARBONDATA-4046
 URL: https://issues.apache.org/jira/browse/CARBONDATA-4046
 Project: CarbonData
  Issue Type: Bug
Reporter: Nihal kumar ojha


Steps to reproduce

1. set property `carbon.read.partition.hive.direct=false`

2. Create table which contain more than one partition column.

3. run query select count (*)

 

It fails with exception as `Key not found`.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (CARBONDATA-3880) How to start JDBC service in distributed index

2020-10-14 Thread Nihal kumar ojha (Jira)


[ 
https://issues.apache.org/jira/browse/CARBONDATA-3880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213825#comment-17213825
 ] 

Nihal kumar ojha commented on CARBONDATA-3880:
--

Hi, please follow the below steps to configure the distributed index server 
with JDBC.

1. Add these properties in spark-defaults.conf
 spark.yarn.keytab=
 spark.carbon.indexserver.keytab=
 spark.carbon.indexserver.principal=spark2x/hadoop.hadoop@hadoop.com
 spark.yarn.principal=spark2x/hadoop.hadoop@hadoop.com

2. Add following configuration in carbon.properties(Ensure the 
carbon.properties is configured in spark-defaults.conf in driver extra java 
option)
 carbon.enable.index.server=true
 carbon.indexserver.enable.prepriming=true
 carbon.indexserver.HA.enabled=true
 carbon.max.executor.lru.cache.size=-1
 carbon.disable.index.server.fallback=false
 carbon.indexserver.zookeeper.dir=/indexserver2x
 carbon.index.server.port=

Then run below spark-submit command at $spark_home location
bin/spark-submit  --num-executors 2 --master yarn --class 
org.apache.carbondata.indexserver.indexserver  


then start spark JDBCserver as usual.
Queries should reflect in yarn UI, in the index server and spark JDBC 
application.
 

>  How to start JDBC service in distributed index
> ---
>
> Key: CARBONDATA-3880
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3880
> Project: CarbonData
>  Issue Type: Bug
>  Components: core
>Affects Versions: 2.0.0
>Reporter: li
>Priority: Major
> Fix For: 2.1.0
>
>
> How to start JDBC service in distributed index



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (CARBONDATA-3892) An exception occurred when modifying the table name using SparkSession

2020-10-14 Thread Nihal kumar ojha (Jira)


[ 
https://issues.apache.org/jira/browse/CARBONDATA-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213785#comment-17213785
 ] 

Nihal kumar ojha commented on CARBONDATA-3892:
--

Hi, I was trying to reproduce this issue but not getting reproduced.
I am using the query "ALTER TABLE oldTable RENAME to newTable".
please correct me if I am wrong.
Or if there is some other configuration then please add here.

> An exception occurred when modifying the table name using SparkSession
> --
>
> Key: CARBONDATA-3892
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3892
> Project: CarbonData
>  Issue Type: Bug
>  Components: spark-integration
>Affects Versions: 2.0.0
>Reporter: li
>Priority: Blocker
>
> Exception in thread "main" java.lang.LinkageError: ClassCastException: 
> attempting to 
> castjar:file:/usr/hdp/2.6.5.0-292/spark2/carbonlib/apache-carbondata-1.6.1-bin-spark2.2.1-hadoop2.7.2.jar!/javax/ws/rs/ext/RuntimeDelegate.classtojar:file:/usr/hdp/2.6.5.0-292/spark2/carbonlib/apache-carbondata-1.6.1-bin-spark2.2.1-hadoop2.7.2.jar!/javax/ws/rs/ext/RuntimeDelegate.class



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (CARBONDATA-3806) Create bloom datamap fails with null pointer exception

2020-10-06 Thread Nihal kumar ojha (Jira)


[ 
https://issues.apache.org/jira/browse/CARBONDATA-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208575#comment-17208575
 ] 

Nihal kumar ojha commented on CARBONDATA-3806:
--

This issue was handled in
PR: https://github.com/apache/carbondata/pull/3775

> Create bloom datamap fails with null pointer exception
> --
>
> Key: CARBONDATA-3806
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3806
> Project: CarbonData
>  Issue Type: Bug
>  Components: data-query
>Affects Versions: 1.6.1
> Environment: Spark 2.3.2
>Reporter: Chetan Bhat
>Priority: Major
>
> Create bloom datamap fails with null pointer exception
> create table brinjal_bloom (imei string,AMSize string,channelsId 
> string,ActiveCountry string, Activecity string,gamePointId 
> double,deviceInformationId double,productionDate Timestamp,deliveryDate 
> timestamp,deliverycharge double) STORED BY 'carbondata' 
> TBLPROPERTIES('table_blocksize'='1');
> LOAD DATA INPATH 'hdfs://hacluster/chetan/vardhandaterestruct.csv' INTO TABLE 
> brinjal_bloom OPTIONS('DELIMITER'=',', 'QUOTECHAR'= 
> '"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'= 
> 'imei,deviceInformationId,AMSize,channelsId,ActiveCountry,Activecity,gamePointId,productionDate,deliveryDate,deliverycharge');
> 0: jdbc:hive2://10.20.255.171:23040/default> CREATE DATAMAP dm_brinjal4 ON 
> TABLE brinjal_bloom USING 'bloomfilter' DMPROPERTIES ('INDEX_COLUMNS' = 
> 'AMSize', 'BLOOM_SIZE'='64', 'BLOOM_FPP'='0.1');
> Error: org.apache.spark.SparkException: Job aborted due to stage failure: 
> Task 0 in stage 210.0 failed 4 times, most recent failure: Lost task 0.3 in 
> stage 210.0 (TID 1477, vm2, executor 2): java.lang.NullPointerException
>  at 
> org.apache.carbondata.core.datamap.Segment.getCommittedIndexFile(Segment.java:150)
>  at 
> org.apache.carbondata.core.util.BlockletDataMapUtil.getTableBlockUniqueIdentifiers(BlockletDataMapUtil.java:198)
>  at 
> org.apache.carbondata.core.indexstore.blockletindex.BlockletDataMapFactory.getTableBlockIndexUniqueIdentifiers(BlockletDataMapFactory.java:176)
>  at 
> org.apache.carbondata.core.indexstore.blockletindex.BlockletDataMapFactory.getDataMaps(BlockletDataMapFactory.java:154)
>  at 
> org.apache.carbondata.core.indexstore.blockletindex.BlockletDataMapFactory.getSegmentProperties(BlockletDataMapFactory.java:425)
>  at 
> org.apache.carbondata.datamap.IndexDataMapRebuildRDD.internalCompute(IndexDataMapRebuildRDD.scala:359)
>  at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:84)
>  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
>  at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
>  at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
>  at org.apache.spark.scheduler.Task.run(Task.scala:109)
>  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  at java.lang.Thread.run(Thread.java:745)
> Driver stacktrace: (state=,code=0)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (CARBONDATA-3912) Clean file requests are failing in case of multiple load due to concurrent locking.

2020-10-06 Thread Nihal kumar ojha (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nihal kumar ojha resolved CARBONDATA-3912.
--
Fix Version/s: 2.1.0
   Resolution: Fixed

This issue was handled in 
PR: https://github.com/apache/carbondata/pull/3871

> Clean file requests are failing in case of multiple load due to concurrent 
> locking.
> ---
>
> Key: CARBONDATA-3912
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3912
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Nihal kumar ojha
>Priority: Minor
> Fix For: 2.1.0
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> If multiple loads are fired at the same time then clean file requests are 
> failing due to failing in lock acquiring.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (CARBONDATA-3992) Drop Index is throwing null pointer exception.

2020-10-01 Thread Nihal kumar ojha (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nihal kumar ojha resolved CARBONDATA-3992.
--
Resolution: Fixed

Fixed in PR:
https://github.com/apache/carbondata/pull/3928

> Drop Index is throwing null pointer exception.
> --
>
> Key: CARBONDATA-3992
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3992
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Nihal kumar ojha
>Priority: Minor
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Index server set to true but index server is not running.
> Create an index as 'carbondata' and try to drop the index -> throwing null 
> pointer exception.
> IndexStoreMandaer.Java -> line 98



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (CARBONDATA-3992) Drop Index is throwing null pointer exception.

2020-09-16 Thread Nihal kumar ojha (Jira)
Nihal kumar ojha created CARBONDATA-3992:


 Summary: Drop Index is throwing null pointer exception.
 Key: CARBONDATA-3992
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3992
 Project: CarbonData
  Issue Type: Bug
Reporter: Nihal kumar ojha


Index server set to true but index server is not running.
Create an index as 'carbondata' and try to drop the index -> throwing null 
pointer exception.

IndexStoreMandaer.Java -> line 98



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3964) Select * from table or select count(*) without filter is throwing null pointer exception.

2020-08-27 Thread Nihal kumar ojha (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nihal kumar ojha updated CARBONDATA-3964:
-
Priority: Minor  (was: Major)

> Select * from table or select count(*) without filter is throwing null 
> pointer exception.
> -
>
> Key: CARBONDATA-3964
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3964
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Nihal kumar ojha
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Steps to reproduce.
> 1. Create a table.
> 2. Load around 500 segments and more than 1 million records.
> 3. Running query select(*) or select count(*) without filter is throwing null 
> pointer exception.
> File: TableIndex.java
> Method: pruneWithMultiThread
> line: 447
> Reason: filter.getresolver() is null.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (CARBONDATA-3964) Select * from table or select count(*) without filter is throwing null pointer exception.

2020-08-27 Thread Nihal kumar ojha (Jira)
Nihal kumar ojha created CARBONDATA-3964:


 Summary: Select * from table or select count(*) without filter is 
throwing null pointer exception.
 Key: CARBONDATA-3964
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3964
 Project: CarbonData
  Issue Type: Bug
Reporter: Nihal kumar ojha


Steps to reproduce.
1. Create a table.
2. Load around 500 segments and more than 1 million records.
3. Running query select(*) or select count(*) without filter is throwing null 
pointer exception.

File: TableIndex.java
Method: pruneWithMultiThread
line: 447
Reason: filter.getresolver() is null.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (CARBONDATA-3947) Insert Into Select Operation is throwing exception for hive read/write operation in carbon.

2020-08-09 Thread Nihal kumar ojha (Jira)
Nihal kumar ojha created CARBONDATA-3947:


 Summary: Insert Into Select Operation is throwing exception for 
hive read/write operation in carbon.
 Key: CARBONDATA-3947
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3947
 Project: CarbonData
  Issue Type: Bug
Reporter: Nihal kumar ojha


CREATE TABLE hive_carbon_table1(id INT, name STRING, scale DECIMAL, country 
STRING, salary DOUBLE) stored by 
'org.apache.carbondata.hive.CarbonStorageHandler';

INSERT into hive_carbon_table1 SELECT 1, 'RAM', '2.3', 'INDIA', 3500";
   
CREATE TABLE hive_carbon_table2(id INT, name STRING, scale DECIMAL, country 
STRING, salary DOUBLE) stored by 
'org.apache.carbondata.hive.CarbonStorageHandler';

INSERT into hive_carbon_table2 SELECT * FROM hive_carbon_table1";   -> Throwing 
exception as "CarbonData file is not present in the table location"




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (CARBONDATA-3928) Handle the Strings which length is greater than 32000 as a bad record.

2020-07-27 Thread Nihal kumar ojha (Jira)
Nihal kumar ojha created CARBONDATA-3928:


 Summary: Handle the Strings which length is greater than 32000 as 
a bad record.
 Key: CARBONDATA-3928
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3928
 Project: CarbonData
  Issue Type: Task
Reporter: Nihal kumar ojha


Currently, when the string length exceeds 32000 then the load is failed.
Suggestion:
1. Bad record can handle string length greater than 32000 and load should not 
be failed because only a few records string length is greater than 32000.
2. Include some more information in the log message like which record and 
column have the problem.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (CARBONDATA-3912) Clean file requests are failing in case of multiple load due to concurrent locking.

2020-07-17 Thread Nihal kumar ojha (Jira)
Nihal kumar ojha created CARBONDATA-3912:


 Summary: Clean file requests are failing in case of multiple load 
due to concurrent locking.
 Key: CARBONDATA-3912
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3912
 Project: CarbonData
  Issue Type: Bug
Reporter: Nihal kumar ojha


If multiple loads are fired at the same time then clean file requests are 
failing due to failing in lock acquiring.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3855) Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.

2020-07-08 Thread Nihal kumar ojha (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nihal kumar ojha updated CARBONDATA-3855:
-
Attachment: CarbonData SDK support load from file.pdf

> Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.
> --
>
> Key: CARBONDATA-3855
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3855
> Project: CarbonData
>  Issue Type: New Feature
>Reporter: Nihal kumar ojha
>Priority: Major
> Attachments: CarbonData SDK support load from file.pdf
>
>  Time Spent: 7h 10m
>  Remaining Estimate: 0h
>
> Please find the solution document attached.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3855) Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.

2020-07-08 Thread Nihal kumar ojha (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nihal kumar ojha updated CARBONDATA-3855:
-
Attachment: (was: CarbonData SDK support load from file.pdf)

> Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.
> --
>
> Key: CARBONDATA-3855
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3855
> Project: CarbonData
>  Issue Type: New Feature
>Reporter: Nihal kumar ojha
>Priority: Major
> Attachments: CarbonData SDK support load from file.pdf
>
>  Time Spent: 7h 10m
>  Remaining Estimate: 0h
>
> Please find the solution document attached.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3855) Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.

2020-06-24 Thread Nihal kumar ojha (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nihal kumar ojha updated CARBONDATA-3855:
-
Attachment: CarbonData SDK support load from file.pdf

> Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.
> --
>
> Key: CARBONDATA-3855
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3855
> Project: CarbonData
>  Issue Type: New Feature
>Reporter: Nihal kumar ojha
>Priority: Major
> Attachments: CarbonData SDK support load from file.pdf
>
>
> Please find the solution document attached.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3855) Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.

2020-06-24 Thread Nihal kumar ojha (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nihal kumar ojha updated CARBONDATA-3855:
-
Attachment: (was: CarbonData SDK support load from file .pdf)

> Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.
> --
>
> Key: CARBONDATA-3855
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3855
> Project: CarbonData
>  Issue Type: New Feature
>Reporter: Nihal kumar ojha
>Priority: Major
>
> Please find the solution document attached.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (CARBONDATA-3855) Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.

2020-06-12 Thread Nihal kumar ojha (Jira)
Nihal kumar ojha created CARBONDATA-3855:


 Summary: Support Carbondata SDK to load data from parquet, ORC, 
CSV, Avro and JSON.
 Key: CARBONDATA-3855
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3855
 Project: CarbonData
  Issue Type: New Feature
Reporter: Nihal kumar ojha
 Attachments: CarbonData SDK support load from file .pdf

Please find the solution document attached.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)