[jira] [Updated] (CARBONDATA-647) Dictionary Lock issue with load data through beeline.
[
https://issues.apache.org/jira/browse/CARBONDATA-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DEEPAK MEHRA updated CARBONDATA-647:
Attachment: 2000_UniqData.csv
> Dictionary Lock issue with load data through beeline.
> --
>
> Key: CARBONDATA-647
> URL: https://issues.apache.org/jira/browse/CARBONDATA-647
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
>Affects Versions: 1.0.0-incubating
> Environment: Spark 1.6.2
>Reporter: DEEPAK MEHRA
> Attachments: 2000_UniqData.csv, DriverLog, ExecutorLog
>
>
> I am executing below-mentioned query on beeline and it throws dictionary file
> is locked for updation. Dictionary file is random every time I create a new
> table followed by load.
> Example
> CREATE TABLE uni (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string,
> DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' ;
> LOAD DATA INPATH 'hdfs://hadoop-master:54311/payal/2000_UniqData.csv' into
> table uni OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')
> ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 7.0 failed 4 times, most recent failure: Lost task 0.3 in
> stage 7.0 (TID 47, hadoop-slave-3): java.lang.RuntimeException: Dictionary
> file cust_name is locked for updation. Please try after some time
> at scala.sys.package$.error(package.scala:27)
> at
> org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerateRDD$$anon$1.(CarbonGlobalDictionaryRDD.scala:396)
> at
> org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerateRDD.compute(CarbonGlobalDictionaryRDD.scala:334)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Driver stacktrace: (state=,code=0)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] incubator-carbondata pull request #419: [CARBONDATA-523] Added test cases fo...
Github user anubhav100 closed the pull request at: https://github.com/apache/incubator-carbondata/pull/419 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (CARBONDATA-468) Add pruning in driver side to improve query performance
[ https://issues.apache.org/jira/browse/CARBONDATA-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-468: --- Assignee: kumar vishal (was: Ravindra Pesala) > Add pruning in driver side to improve query performance > --- > > Key: CARBONDATA-468 > URL: https://issues.apache.org/jira/browse/CARBONDATA-468 > Project: CarbonData > Issue Type: Sub-task >Reporter: Ravindra Pesala >Assignee: kumar vishal > > Add pruning using bucketing information in driver side to improve query > performance -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CARBONDATA-646) Bad record handling is not correct for Int data type
[ https://issues.apache.org/jira/browse/CARBONDATA-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825525#comment-15825525 ] Ramakrishna commented on CARBONDATA-646: same way this is not handled in Update query, giving in consistent data: 0: jdbc:hive2://172.168.100.212:23040> select * from t_carbn02; ++---+--++--+ | active_status | item_type_cd | qty_day_avg | qty_total | ++---+--++--+ | TRUE | 0 | 1| 2 | ++---+--++--+ 1 row selected (0.088 seconds) 0: jdbc:hive2://172.168.100.212:23040> update t_carbn02 set (item_type_cd) = (1.3); +-+--+ | Result | +-+--+ +-+--+ No rows selected (5.528 seconds) 0: jdbc:hive2://172.168.100.212:23040> select * from t_carbn02; ++---+--++--+ | active_status | item_type_cd | qty_day_avg | qty_total | ++---+--++--+ | TRUE | 1 | 1| 2 | ++---+--++--+ 1 row selected (0.093 seconds) 0: jdbc:hive2://172.168.100.212:23040> update t_carbn02 set (item_type_cd) = (item_type_cd/2); +-+--+ | Result | +-+--+ +-+--+ No rows selected (5.544 seconds) 0: jdbc:hive2://172.168.100.212:23040> select * from t_carbn02; ++---+--++--+ | active_status | item_type_cd | qty_day_avg | qty_total | ++---+--++--+ | TRUE | 0 | 1| 2 | ++---+--++--+ > Bad record handling is not correct for Int data type > > > Key: CARBONDATA-646 > URL: https://issues.apache.org/jira/browse/CARBONDATA-646 > Project: CarbonData > Issue Type: Bug >Reporter: Ramakrishna >Priority: Minor > > With Bad record handling as default, > If Char value is given for Int data type, that is handled properly(moving > NULL). > If Decimal values is given for Int Data type, it is stripping the decimal, > where it should consider this as bad record and move NULL. > Bad record csv: > TRUE,2.7,423.0,A,2003454300, > 121.5,4.99,2.44,SE3423ee,asfdsffdfg,EtryTRWT,2012-01-12 > 03:14:05.123456729,2012-01-20 > 0: jdbc:hive2://172.168.100.212:23040> select * from t_carbn01 where > qty_total is NULL; > ++---+--++-+--+-+-++-+--++--+--+ > | active_status | item_type_cd | qty_day_avg | qty_total | sell_price > | sell_pricep | discount_price | profit | item_code | item_name | > outlet_name | update_time | create_date | > ++---+--++-+--+-+-++-+--++--+--+ > | TRUE | 2 | 423 | NULL | > 2003454304 | 121.5| 4.99| 2.44| SE3423ee | > asfdsffdfg | EtryTRWT | 2012-01-12 03:14:05.0 | 2012-01-20 | > ++---+--++-+--+-+-++-+--++-- > 0: jdbc:hive2://172.168.100.212:23040> desc t_carbn01; > +-+---+--+--+ > |col_name | data_type | comment | > +-+---+--+--+ > | active_status | string| | > | item_type_cd| bigint| | > | qty_day_avg | bigint| | > | qty_total | bigint| | > | sell_price | bigint| | > | sell_pricep | double| | > | discount_price | double| | > | profit | decimal(3,2) | | > | item_code | string| | > | item_name | string| | > | outlet_name | string| | > | update_time | timestamp | | > | create_date | string| | > +-+---+--+--+ > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-646) Bad record handling is not correct for Int data type
Ramakrishna created CARBONDATA-646: -- Summary: Bad record handling is not correct for Int data type Key: CARBONDATA-646 URL: https://issues.apache.org/jira/browse/CARBONDATA-646 Project: CarbonData Issue Type: Bug Reporter: Ramakrishna Priority: Minor With Bad record handling as default, If Char value is given for Int data type, that is handled properly(moving NULL). If Decimal values is given for Int Data type, it is stripping the decimal, where it should consider this as bad record and move NULL. Bad record csv: TRUE,2.7,423.0,A,2003454300, 121.5,4.99,2.44,SE3423ee,asfdsffdfg,EtryTRWT,2012-01-12 03:14:05.123456729,2012-01-20 0: jdbc:hive2://172.168.100.212:23040> select * from t_carbn01 where qty_total is NULL; ++---+--++-+--+-+-++-+--++--+--+ | active_status | item_type_cd | qty_day_avg | qty_total | sell_price | sell_pricep | discount_price | profit | item_code | item_name | outlet_name | update_time | create_date | ++---+--++-+--+-+-++-+--++--+--+ | TRUE | 2 | 423 | NULL | 2003454304 | 121.5| 4.99| 2.44| SE3423ee | asfdsffdfg | EtryTRWT | 2012-01-12 03:14:05.0 | 2012-01-20 | ++---+--++-+--+-+-++-+--++-- 0: jdbc:hive2://172.168.100.212:23040> desc t_carbn01; +-+---+--+--+ |col_name | data_type | comment | +-+---+--+--+ | active_status | string| | | item_type_cd| bigint| | | qty_day_avg | bigint| | | qty_total | bigint| | | sell_price | bigint| | | sell_pricep | double| | | discount_price | double| | | profit | decimal(3,2) | | | item_code | string| | | item_name | string| | | outlet_name | string| | | update_time | timestamp | | | create_date | string| | +-+---+--+--+ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-533) Write Unit Test Case For carbondataprocessing.newflow.steps
[ https://issues.apache.org/jira/browse/CARBONDATA-533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rahul Kumar closed CARBONDATA-533. -- Resolution: Later > Write Unit Test Case For carbondataprocessing.newflow.steps > --- > > Key: CARBONDATA-533 > URL: https://issues.apache.org/jira/browse/CARBONDATA-533 > Project: CarbonData > Issue Type: Improvement >Reporter: Rahul Kumar > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-534) Write Unit Test Case For DataLoadExecuter and DataLoadProcessBuilder
[ https://issues.apache.org/jira/browse/CARBONDATA-534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rahul Kumar closed CARBONDATA-534. -- Resolution: Later > Write Unit Test Case For DataLoadExecuter and DataLoadProcessBuilder > > > Key: CARBONDATA-534 > URL: https://issues.apache.org/jira/browse/CARBONDATA-534 > Project: CarbonData > Issue Type: Improvement >Reporter: Rahul Kumar > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-553) Create integration test-case for dataframe API
[ https://issues.apache.org/jira/browse/CARBONDATA-553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rahul Kumar closed CARBONDATA-553. -- Resolution: Fixed > Create integration test-case for dataframe API > -- > > Key: CARBONDATA-553 > URL: https://issues.apache.org/jira/browse/CARBONDATA-553 > Project: CarbonData > Issue Type: Test >Reporter: Rahul Kumar >Assignee: Rahul Kumar > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-396) Implement test cases for datastorage package
[ https://issues.apache.org/jira/browse/CARBONDATA-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anurag Srivastava closed CARBONDATA-396. Resolution: Later > Implement test cases for datastorage package > > > Key: CARBONDATA-396 > URL: https://issues.apache.org/jira/browse/CARBONDATA-396 > Project: CarbonData > Issue Type: Test >Reporter: Anurag Srivastava >Priority: Trivial > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-551) Implement unit test cases for classes in processing package
[ https://issues.apache.org/jira/browse/CARBONDATA-551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anurag Srivastava closed CARBONDATA-551. Resolution: Later > Implement unit test cases for classes in processing package > --- > > Key: CARBONDATA-551 > URL: https://issues.apache.org/jira/browse/CARBONDATA-551 > Project: CarbonData > Issue Type: Test >Reporter: Anurag Srivastava >Priority: Trivial > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-541) Implement unit test cases for processing.newflow.dictionary package
[ https://issues.apache.org/jira/browse/CARBONDATA-541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anurag Srivastava closed CARBONDATA-541. Resolution: Later > Implement unit test cases for processing.newflow.dictionary package > --- > > Key: CARBONDATA-541 > URL: https://issues.apache.org/jira/browse/CARBONDATA-541 > Project: CarbonData > Issue Type: Test >Reporter: Anurag Srivastava >Priority: Trivial > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-515) Implement unit test cases for processing.newflow.converter package
[ https://issues.apache.org/jira/browse/CARBONDATA-515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anurag Srivastava closed CARBONDATA-515. Resolution: Later > Implement unit test cases for processing.newflow.converter package > -- > > Key: CARBONDATA-515 > URL: https://issues.apache.org/jira/browse/CARBONDATA-515 > Project: CarbonData > Issue Type: Test >Reporter: Anurag Srivastava >Priority: Trivial > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-496) Implement unit test cases for core.carbon.datastore package
[ https://issues.apache.org/jira/browse/CARBONDATA-496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anurag Srivastava closed CARBONDATA-496. Resolved > Implement unit test cases for core.carbon.datastore package > --- > > Key: CARBONDATA-496 > URL: https://issues.apache.org/jira/browse/CARBONDATA-496 > Project: CarbonData > Issue Type: Test >Reporter: Anurag Srivastava >Priority: Trivial > Fix For: 1.0.0-incubating > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-475) Implement unit test cases for core.carbon.querystatics package
[ https://issues.apache.org/jira/browse/CARBONDATA-475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anurag Srivastava closed CARBONDATA-475. Resolved > Implement unit test cases for core.carbon.querystatics package > -- > > Key: CARBONDATA-475 > URL: https://issues.apache.org/jira/browse/CARBONDATA-475 > Project: CarbonData > Issue Type: Test >Reporter: Anurag Srivastava >Priority: Trivial > Fix For: 1.0.0-incubating > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-413) Implement unit test cases for scan.expression package
[ https://issues.apache.org/jira/browse/CARBONDATA-413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anurag Srivastava closed CARBONDATA-413. Resolved > Implement unit test cases for scan.expression package > - > > Key: CARBONDATA-413 > URL: https://issues.apache.org/jira/browse/CARBONDATA-413 > Project: CarbonData > Issue Type: Improvement >Reporter: Anurag Srivastava >Priority: Trivial > Fix For: 1.0.0-incubating > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-474) Implement unit test cases for core.datastorage package
[ https://issues.apache.org/jira/browse/CARBONDATA-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anurag Srivastava closed CARBONDATA-474. Resolved > Implement unit test cases for core.datastorage package > -- > > Key: CARBONDATA-474 > URL: https://issues.apache.org/jira/browse/CARBONDATA-474 > Project: CarbonData > Issue Type: Bug >Reporter: Anurag Srivastava >Priority: Trivial > Fix For: 1.0.0-incubating > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-494) Implement unit test cases for filter.executer package
[ https://issues.apache.org/jira/browse/CARBONDATA-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anurag Srivastava closed CARBONDATA-494. Resolution: Later > Implement unit test cases for filter.executer package > - > > Key: CARBONDATA-494 > URL: https://issues.apache.org/jira/browse/CARBONDATA-494 > Project: CarbonData > Issue Type: Test >Reporter: Anurag Srivastava >Priority: Trivial > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-555) Configure Integration testcases to be run on hadoop cluster
[ https://issues.apache.org/jira/browse/CARBONDATA-555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rahul Kumar closed CARBONDATA-555. -- Resolution: Won't Fix > Configure Integration testcases to be run on hadoop cluster > --- > > Key: CARBONDATA-555 > URL: https://issues.apache.org/jira/browse/CARBONDATA-555 > Project: CarbonData > Issue Type: Improvement >Reporter: Rahul Kumar >Assignee: Rahul Kumar > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-340) Implement test cases for load package in core module
[ https://issues.apache.org/jira/browse/CARBONDATA-340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anurag Srivastava closed CARBONDATA-340. Resoved > Implement test cases for load package in core module > > > Key: CARBONDATA-340 > URL: https://issues.apache.org/jira/browse/CARBONDATA-340 > Project: CarbonData > Issue Type: Test >Reporter: Anurag Srivastava >Priority: Trivial > Fix For: 1.0.0-incubating > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-526) Write Unit TestCase For org.apache.carbondata.scan.filter.resolver
[ https://issues.apache.org/jira/browse/CARBONDATA-526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rahul Kumar closed CARBONDATA-526. -- Resolution: Later > Write Unit TestCase For org.apache.carbondata.scan.filter.resolver > -- > > Key: CARBONDATA-526 > URL: https://issues.apache.org/jira/browse/CARBONDATA-526 > Project: CarbonData > Issue Type: Improvement >Reporter: Rahul Kumar > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-carbondata pull request #466: [CARBONDATA-555] Configure Integrati...
Github user rahulforallp closed the pull request at: https://github.com/apache/incubator-carbondata/pull/466 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata pull request #399: [CARBONDATA-494] Added unit test cas...
Github user abhisheknoldus closed the pull request at: https://github.com/apache/incubator-carbondata/pull/399 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (CARBONDATA-645) Query Analysis exception while running Delete segment
Ramakrishna created CARBONDATA-645: -- Summary: Query Analysis exception while running Delete segment Key: CARBONDATA-645 URL: https://issues.apache.org/jira/browse/CARBONDATA-645 Project: CarbonData Issue Type: Bug Reporter: Ramakrishna Priority: Minor Delete segment query does not work 0: jdbc:hive2://172.168.100.212:23040> delete from table t_carbn01 where segment.id in (3.1); Error: org.apache.spark.sql.AnalysisException: Table not found: table; line 1 pos 26 (state=,code=0) 0: jdbc:hive2://172.168.100.212:23040> show tables; ++--+--+ | tableName | isTemporary | ++--+--+ | t_carbn01 | false| -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-carbondata issue #541: [CARBONDATA-641]supported date type for dic...
Github user mohammadshahidkhan commented on the issue: https://github.com/apache/incubator-carbondata/pull/541 @anubhav100 1. Please check you have missed my second comment TableCreator.isStringAndTimestampColDictionaryExclude 2. Please give meaning full name to method isStringAndTimestampColDictionaryExclude in TableCreator and CarbonDDLSqlParser --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Closed] (CARBONDATA-531) Eliminate spark dependency in carbon core
[ https://issues.apache.org/jira/browse/CARBONDATA-531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacky Li closed CARBONDATA-531. --- Resolution: Invalid Because the code base has changed a lot, this improvement will be consider later > Eliminate spark dependency in carbon core > - > > Key: CARBONDATA-531 > URL: https://issues.apache.org/jira/browse/CARBONDATA-531 > Project: CarbonData > Issue Type: Improvement >Affects Versions: 0.2.0-incubating >Reporter: Jacky Li >Assignee: Jacky Li > Fix For: 1.0.0-incubating > > Time Spent: 1h > Remaining Estimate: 0h > > Clean up the interface and take out Spark dependency on Carbon-core module. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-carbondata pull request #432: [CARBONDATA-531] remove spark depend...
Github user jackylk closed the pull request at: https://github.com/apache/incubator-carbondata/pull/432 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata issue #432: [CARBONDATA-531] remove spark dependency in...
Github user jackylk commented on the issue: https://github.com/apache/incubator-carbondata/pull/432 This PR has too many conflict, I will close it and raise it again --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Closed] (CARBONDATA-514) Select string type columns will return error.
[
https://issues.apache.org/jira/browse/CARBONDATA-514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cao, Lionel closed CARBONDATA-514.
--
Resolution: Fixed
> Select string type columns will return error.
> -
>
> Key: CARBONDATA-514
> URL: https://issues.apache.org/jira/browse/CARBONDATA-514
> Project: CarbonData
> Issue Type: Bug
> Components: sql
>Affects Versions: 1.0.0-incubating
>Reporter: Cao, Lionel
> Attachments: Screenshot.png
>
>
> The data successfully loaded and count(*) is OK, but when I tried to query
> the detail data, it returns below error:
> scala> cc.sql("desc carbontest_002").show
> +-+-+---+
> | col_name|data_type|comment|
> +-+-+---+
> | vin| string| |
> |data_date| string| |
> +-+-+---+
> scala> cc.sql("load data inpath
> 'hdfs://nameservice2/user/appuser/lucao/mydata4.csv' into table
> default.carbontest_002 OPTIONS('DELIMITER'=',')")
> WARN 07-12 16:30:30,241 - main skip empty input file:
> hdfs://nameservice2/user/appuser/lucao/mydata4.csv/_SUCCESS
> AUDIT 07-12 16:30:34,338 - [*.com][appuser][Thread-1]Data load request has
> been received for table default.carbontest_002
> AUDIT 07-12 16:30:38,410 - [*.com][appuser][Thread-1]Data load is successful
> for default.carbontest_002
> res12: org.apache.spark.sql.DataFrame = []
> scala> cc.sql("select count(*) from carbontest_002")
> res14: org.apache.spark.sql.DataFrame = [_c0: bigint]
> scala> res14.show
> +---+
> |_c0|
> +---+
> |100|
> +---+
> scala> cc.sql("select vin, count(*) as cnt from carbontest_002 group by
> vin").show
> WARN 07-12 16:32:04,250 - Lost task 1.0 in stage 20.0 (TID 40, *.com):
> java.lang.ClassCastException: java.lang.String cannot be cast to
> java.lang.Integer
> at scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:106)
> at
> org.apache.spark.sql.catalyst.expressions.BaseGenericInternalRow$class.getInt(rows.scala:41)
>
> at
> org.apache.spark.sql.catalyst.expressions.GenericMutableRow.getInt(rows.scala:248)
>
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown
> Source)
> at
> org.apache.spark.sql.CarbonScan$$anonfun$1$$anon$1.next(CarbonScan.scala:155)
> at
> org.apache.spark.sql.CarbonScan$$anonfun$1$$anon$1.next(CarbonScan.scala:149)
> at
> org.apache.spark.sql.execution.aggregate.TungstenAggregationIterator.processInputs(TungstenAggregationIterator.scala:512)
>
> at
> org.apache.spark.sql.execution.aggregate.TungstenAggregationIterator.(TungstenAggregationIterator.scala:686)
>
> at
> org.apache.spark.sql.execution.aggregate.TungstenAggregate$$anonfun$doExecute$1$$anonfun$2.apply(TungstenAggregate.scala:95)
>
> at
> org.apache.spark.sql.execution.aggregate.TungstenAggregate$$anonfun$doExecute$1$$anonfun$2.apply(TungstenAggregate.scala:86)
>
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
>
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
>
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
> ERROR 07-12 16:32:04,516 - Task 1 in stage 20.0 failed 4 times; aborting job
> WARN 07-12 16:32:04,600 - Lost task 0.1 in stage 20.0 (TID 45, *):
> TaskKilled (killed intentionally)
> ERROR 07-12 16:32:04,604 - Listener SQLListener threw an exception
> java.lang.NullPointerException
> at
> org.apache.spark.sql.execution.ui.SQLListener.onTaskEnd(SQLListener.scala:167)
>
> at
> org.apache.spark.scheduler.SparkListenerBus$class.onPostEvent(SparkListenerBus.scala:42)
>
> at
> org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
>
> at
> org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
>
> at org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:55)
> at
>
[jira] [Commented] (CARBONDATA-514) Select string type columns will return error.
[
https://issues.apache.org/jira/browse/CARBONDATA-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824886#comment-15824886
]
Cao, Lionel commented on CARBONDATA-514:
Hi Ravi,
Tested the current master branch, all successed.
Thanks,
Lionel
> Select string type columns will return error.
> -
>
> Key: CARBONDATA-514
> URL: https://issues.apache.org/jira/browse/CARBONDATA-514
> Project: CarbonData
> Issue Type: Bug
> Components: sql
>Affects Versions: 1.0.0-incubating
>Reporter: Cao, Lionel
> Attachments: Screenshot.png
>
>
> The data successfully loaded and count(*) is OK, but when I tried to query
> the detail data, it returns below error:
> scala> cc.sql("desc carbontest_002").show
> +-+-+---+
> | col_name|data_type|comment|
> +-+-+---+
> | vin| string| |
> |data_date| string| |
> +-+-+---+
> scala> cc.sql("load data inpath
> 'hdfs://nameservice2/user/appuser/lucao/mydata4.csv' into table
> default.carbontest_002 OPTIONS('DELIMITER'=',')")
> WARN 07-12 16:30:30,241 - main skip empty input file:
> hdfs://nameservice2/user/appuser/lucao/mydata4.csv/_SUCCESS
> AUDIT 07-12 16:30:34,338 - [*.com][appuser][Thread-1]Data load request has
> been received for table default.carbontest_002
> AUDIT 07-12 16:30:38,410 - [*.com][appuser][Thread-1]Data load is successful
> for default.carbontest_002
> res12: org.apache.spark.sql.DataFrame = []
> scala> cc.sql("select count(*) from carbontest_002")
> res14: org.apache.spark.sql.DataFrame = [_c0: bigint]
> scala> res14.show
> +---+
> |_c0|
> +---+
> |100|
> +---+
> scala> cc.sql("select vin, count(*) as cnt from carbontest_002 group by
> vin").show
> WARN 07-12 16:32:04,250 - Lost task 1.0 in stage 20.0 (TID 40, *.com):
> java.lang.ClassCastException: java.lang.String cannot be cast to
> java.lang.Integer
> at scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:106)
> at
> org.apache.spark.sql.catalyst.expressions.BaseGenericInternalRow$class.getInt(rows.scala:41)
>
> at
> org.apache.spark.sql.catalyst.expressions.GenericMutableRow.getInt(rows.scala:248)
>
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown
> Source)
> at
> org.apache.spark.sql.CarbonScan$$anonfun$1$$anon$1.next(CarbonScan.scala:155)
> at
> org.apache.spark.sql.CarbonScan$$anonfun$1$$anon$1.next(CarbonScan.scala:149)
> at
> org.apache.spark.sql.execution.aggregate.TungstenAggregationIterator.processInputs(TungstenAggregationIterator.scala:512)
>
> at
> org.apache.spark.sql.execution.aggregate.TungstenAggregationIterator.(TungstenAggregationIterator.scala:686)
>
> at
> org.apache.spark.sql.execution.aggregate.TungstenAggregate$$anonfun$doExecute$1$$anonfun$2.apply(TungstenAggregate.scala:95)
>
> at
> org.apache.spark.sql.execution.aggregate.TungstenAggregate$$anonfun$doExecute$1$$anonfun$2.apply(TungstenAggregate.scala:86)
>
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
>
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
>
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
> ERROR 07-12 16:32:04,516 - Task 1 in stage 20.0 failed 4 times; aborting job
> WARN 07-12 16:32:04,600 - Lost task 0.1 in stage 20.0 (TID 45, *):
> TaskKilled (killed intentionally)
> ERROR 07-12 16:32:04,604 - Listener SQLListener threw an exception
> java.lang.NullPointerException
> at
> org.apache.spark.sql.execution.ui.SQLListener.onTaskEnd(SQLListener.scala:167)
>
> at
> org.apache.spark.scheduler.SparkListenerBus$class.onPostEvent(SparkListenerBus.scala:42)
>
> at
> org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
>
> at
> org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
[jira] [Closed] (CARBONDATA-290) When part of table name has database name, then query will show segment path not found
[ https://issues.apache.org/jira/browse/CARBONDATA-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay closed CARBONDATA-290. -- Resolution: Fixed > When part of table name has database name, then query will show segment path > not found > --- > > Key: CARBONDATA-290 > URL: https://issues.apache.org/jira/browse/CARBONDATA-290 > Project: CarbonData > Issue Type: Bug >Reporter: Jay >Assignee: Jay >Priority: Minor > > When part of table name has database name, ex: > in default database, CREATE TABLE IF NOT EXISTS t3default > then load and then query, we will get the exception that segment not found -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-556) when loading, it may happen that multiple tasks in one node, while other node has no task.
[ https://issues.apache.org/jira/browse/CARBONDATA-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay closed CARBONDATA-556. -- Resolution: Invalid > when loading, it may happen that multiple tasks in one node, while other node > has no task. > -- > > Key: CARBONDATA-556 > URL: https://issues.apache.org/jira/browse/CARBONDATA-556 > Project: CarbonData > Issue Type: Improvement >Reporter: Jay >Assignee: Jay >Priority: Minor > Time Spent: 1h 20m > Remaining Estimate: 0h > > the algorithm of carbondata loading is aim to ensure one task in one node , > but in cluster, it may happen that mutilple tasks in one node, while other > node has no tasks, which cost more time because of out of balance load. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-402) carbon should support CreateAsSelect
[ https://issues.apache.org/jira/browse/CARBONDATA-402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay closed CARBONDATA-402. -- Resolution: Invalid > carbon should support CreateAsSelect > - > > Key: CARBONDATA-402 > URL: https://issues.apache.org/jira/browse/CARBONDATA-402 > Project: CarbonData > Issue Type: Improvement >Reporter: Jay >Assignee: Jay >Priority: Minor > Time Spent: 1h 40m > Remaining Estimate: 0h > > provide support for CreateAsSelect, > the syntax is hive Syntax, like below: > Create TABLE table4 STORED BY 'carbondata' AS SELECT * FROM table3 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-269) Change delete segment parser because its may confused with hive grammar
[ https://issues.apache.org/jira/browse/CARBONDATA-269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay closed CARBONDATA-269. -- Resolution: Invalid > Change delete segment parser because its may confused with hive grammar > --- > > Key: CARBONDATA-269 > URL: https://issues.apache.org/jira/browse/CARBONDATA-269 > Project: CarbonData > Issue Type: Improvement >Reporter: Jay >Assignee: Jay >Priority: Minor > > the delete segment grammar in carbon is too similar with that delete grammar > in hive, while they have different usage. so change carbon's grammar to make > carbon easy use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-carbondata issue #537: fix unapproved licenses
Github user QiangCai commented on the issue: https://github.com/apache/incubator-carbondata/pull/537 I agree. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata issue #537: fix unapproved licenses
Github user chenliang613 commented on the issue: https://github.com/apache/incubator-carbondata/pull/537 @QiangCai Please keep consistent with scala file header(same with spark) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata issue #537: fix unapproved licenses
Github user QiangCai commented on the issue: https://github.com/apache/incubator-carbondata/pull/537 @chenliang613 @jackylk @ravipesala now the license header of Java file is different with the license header of Scala file. Which one we should choose? java file header: ``` /* * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file * distributed with this work for additional information * regarding copyright ownership. The ASF licenses this file * to you under the Apache License, Version 2.0 (the * "License"); you may not use this file except in compliance * with the License. You may obtain a copy of the License at * *http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, * software distributed under the License is distributed on an * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY * KIND, either express or implied. See the License for the * specific language governing permissions and limitations * under the License. */ ``` scala file header:(same with spark) ``` /* * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with * this work for additional information regarding copyright ownership. * The ASF licenses this file to You under the Apache License, Version 2.0 * (the "License"); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * *http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Closed] (CARBONDATA-104) To support varchar datatype
[ https://issues.apache.org/jira/browse/CARBONDATA-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangshunyu closed CARBONDATA-104. -- Resolution: Invalid > To support varchar datatype > --- > > Key: CARBONDATA-104 > URL: https://issues.apache.org/jira/browse/CARBONDATA-104 > Project: CarbonData > Issue Type: New Feature >Reporter: zhangshunyu >Assignee: zhangshunyu >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-274) Use exist method in CarbonMetastoreCatalog to read/write thrift TableInfo
[ https://issues.apache.org/jira/browse/CARBONDATA-274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangshunyu closed CARBONDATA-274. -- Resolution: Invalid > Use exist method in CarbonMetastoreCatalog to read/write thrift TableInfo > - > > Key: CARBONDATA-274 > URL: https://issues.apache.org/jira/browse/CARBONDATA-274 > Project: CarbonData > Issue Type: Improvement > Components: spark-integration >Affects Versions: 0.1.1-incubating >Reporter: zhangshunyu >Assignee: zhangshunyu >Priority: Minor > > Use exist method in CarbonMetastoreCatalog to read/write thrift TableInfo -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-carbondata issue #541: [CARBONDATA-641]supported date type for dic...
Github user CarbonDataQA commented on the issue: https://github.com/apache/incubator-carbondata/pull/541 Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/635/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (CARBONDATA-604) Use Kettle wrong option in Thrift server give wrong error message
[
https://issues.apache.org/jira/browse/CARBONDATA-604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravindra Pesala updated CARBONDATA-604:
---
Assignee: Manish Gupta (was: Ravindra Pesala)
> Use Kettle wrong option in Thrift server give wrong error message
> -
>
> Key: CARBONDATA-604
> URL: https://issues.apache.org/jira/browse/CARBONDATA-604
> Project: CarbonData
> Issue Type: Bug
> Environment: Spark 1.6
>Reporter: Harmeet Singh
>Assignee: Manish Gupta
>Priority: Trivial
>
> I am using Thrift Server with following configuration:
> -Duse.kettle=default
> Here kettle value is wrong and when I run the thrift server, the thrift
> server is running successfully. After that, I am going to start beeline. In
> beeline, I am executing load command without any kettle option as below:
> LOAD DATA INPATH 'hdfs://localhost:54310/csvs/3000_UniqData.csv' into table
> uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> After executing above command, I am getting Error:
> java.lang.IllegalArgumentException: For input string: "default"
> (state=,code=0)
> This error, seems to be wrong, because I am not using any default value in
> load query, and default is used with thrift.
> So I have following Suggestions:
> 1. Change the error message, while load query is executing.
> 2. While starting thrift server with wrong value, thrift give us an error and
> not start.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CARBONDATA-522) New data loading flowcauses testcase failures like big decimal etc
[ https://issues.apache.org/jira/browse/CARBONDATA-522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala resolved CARBONDATA-522. Resolution: Fixed Fix Version/s: 1.0.0-incubating > New data loading flowcauses testcase failures like big decimal etc > -- > > Key: CARBONDATA-522 > URL: https://issues.apache.org/jira/browse/CARBONDATA-522 > Project: CarbonData > Issue Type: Bug >Reporter: Lionx >Assignee: Ravindra Pesala > Fix For: 1.0.0-incubating > > > Pls check http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/105/. > new data flow cause test regressions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CARBONDATA-640) Insert Query with Hardcoded values is not working
[ https://issues.apache.org/jira/browse/CARBONDATA-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824376#comment-15824376 ] Ravindra Pesala commented on CARBONDATA-640: [~vyom_huawei] Please verify this scenario once with Hive and check the behavior. > Insert Query with Hardcoded values is not working > - > > Key: CARBONDATA-640 > URL: https://issues.apache.org/jira/browse/CARBONDATA-640 > Project: CarbonData > Issue Type: Bug > Components: sql >Affects Versions: 1.0.0-incubating >Reporter: Vyom Rastogi >Priority: Minor > Labels: Spark-1.6 > Attachments: insert_data.png > > > 1)Creating table employees,Managers > create table employees(name string, empid string, mgrid string, mobileno > bigint) stored by 'carbondata'; > 2)create table managers(name string, empid string, mgrid string, mobileno > bigint) stored by 'carbondata'; > Insert into Select Queries > insert into managers select 'harry','h2399','v788232',99823230205; > Error Description: > Error: org.apache.spark.sql.AnalysisException: Failed to recognize predicate > ''. Failed rule: 'regularBody' in statement; line 1 pos 65 > (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-149) Dictionary file has issue when executor task fails
[ https://issues.apache.org/jira/browse/CARBONDATA-149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-149: --- Assignee: Ashok Kumar > Dictionary file has issue when executor task fails > -- > > Key: CARBONDATA-149 > URL: https://issues.apache.org/jira/browse/CARBONDATA-149 > Project: CarbonData > Issue Type: Bug >Reporter: Ashok Kumar >Assignee: Ashok Kumar >Priority: Minor > > 1. Add more logging when dictionary file write is successfull > 2.Throw error when truncation of dictionary file fails > 3.While reading dictionary into cache, do not add empty chunks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CARBONDATA-167) UndeclaredThrowableException thrown instead of data loading fail when fileheader has unsupported characters in file/command
[ https://issues.apache.org/jira/browse/CARBONDATA-167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala resolved CARBONDATA-167. Resolution: Not A Problem Fix Version/s: 0.2.0-incubating > UndeclaredThrowableException thrown instead of data loading fail when > fileheader has unsupported characters in file/command > --- > > Key: CARBONDATA-167 > URL: https://issues.apache.org/jira/browse/CARBONDATA-167 > Project: CarbonData > Issue Type: Bug >Reporter: zhangshunyu >Priority: Minor > Fix For: 0.2.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-225) Sort data based on data type while loading the data for Dictionary exclude columns
[ https://issues.apache.org/jira/browse/CARBONDATA-225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-225: --- Issue Type: Improvement (was: Bug) > Sort data based on data type while loading the data for Dictionary exclude > columns > -- > > Key: CARBONDATA-225 > URL: https://issues.apache.org/jira/browse/CARBONDATA-225 > Project: CarbonData > Issue Type: Improvement >Reporter: kumar vishal > > Currently only String data type is supported for No Dictionary Exclude > columns to support numeric data type we need to sort based on actual data > type by passing the chain comparator based on data type. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CARBONDATA-243) Filters on dictionary column not always work.
[
https://issues.apache.org/jira/browse/CARBONDATA-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824367#comment-15824367
]
Ravindra Pesala commented on CARBONDATA-243:
I think this issue got resolved in latest master. Please verify it once
> Filters on dictionary column not always work.
> -
>
> Key: CARBONDATA-243
> URL: https://issues.apache.org/jira/browse/CARBONDATA-243
> Project: CarbonData
> Issue Type: Bug
> Components: examples
>Affects Versions: 0.2.0-incubating
> Environment: Ubuntu 14.04
>Reporter: Yue Shang
>Priority: Minor
>
> Filter on dictionary column not always work.
> Table loaded contains 100 columns, 100M rows, and one of the column c100's
> cardinality is 10M. Dictionary include c100.
> DDL statement as follows:
> CREATE TABLE IF NOT EXISTS big_table4
> (
> c100 Int,
> c1 Int, c2 Int, c3 Int, c4 Int, c5 Int, c6 Int, c7 Int, c8 Int, c9
> Int, c10 Int, c11 Int, c12 Int, c13 Int, c14 Int, c15 Int, c16 Int, c17 Int,
> c18 Int, c19 Int, c20 Int,
> c21 Int, c22 Int, c23 Int, c24 Int, c25 Int, c26 Int, c27 Int, c28
> Int, c29 Int, c30 Int, c31 Int, c32 Int, c33 Int, c34 Int, c35 Int, c36 Int,
> c37 Int, c38 Int, c39 Int, c40 Int,
> c41 Int, c42 Int, c43 Int, c44 Int, c45 Int, c46 Int, c47 Int, c48
> Int, c49 Int, c50 Int, c51 Int, c52 Int, c53 Int, c54 Int, c55 Int, c56 Int,
> c57 Int, c58 Int, c59 Int, c60 Int,
> c61 Int, c62 Int, c63 Int, c64 Int, c65 Int, c66 Int, c67 Int, c68
> Int, c69 Int, c70 Int, c71 Int, c72 Int, c73 Int, c74 Int, c75 Int, c76 Int,
> c77 Int, c78 Int, c79 Int, c80 Int,
> c81 Int, c82 Int, c83 Int, c84 Int, c85 Int, c86 Int, c87 Int, c88
> Int, c89 Int, c90 Int, c91 Int, c92 Int, c93 Int, c94 Int, c95 Int, c96 Int,
> c97 Int, c98 Int, c99 Int,
> c101 String
> )
> STORED BY 'carbondata'
> TBLPROPERTIES ("DICTIONARY_INCLUDE"="c101,c100, c1,
> c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,c14,
> c15")
> I tried this query to make sure the value i'm querying is exist.
> select c100 from big_table4 where c100 like '1234_'
> ​+-+
> | c100|
> +-+
> |12340|
> |12341|
> |12342|
> |12343|
> |12344|
> |12345|
> |12346|
> |12347|
> |12348|
> |12349|
> +-+
> ​
> But when I tried: " select c100 from big_table4 where c100 like '12345'​ ",
> I got a runtime error(Log is at the bottom of this mail).
> But the most wield part is, some values can be queried while others not.
> cc.sql("select c100 from big_table4 where c100 like '116'")
> This query will get the exact correct answer.
> Any idea about this?
>
>log
>
> ​ERROR 14-09 17:16:52,024 - [Executor task launch
> worker-0][partitionID:table4;queryID:275884499876481_0]
> java.lang.NullPointerException
> at
> org.apache.carbondata.scan.result.iterator.AbstractDetailQueryResultIterator.intialiseInfos(AbstractDetailQueryResultIterator.java:95)
> at
> org.apache.carbondata.scan.result.iterator.AbstractDetailQueryResultIterator.(AbstractDetailQueryResultIterator.java:87)
> at
> org.apache.carbondata.scan.result.iterator.DetailQueryResultIterator.(DetailQueryResultIterator.java:47)
> at
> org.apache.carbondata.scan.executor.impl.DetailQueryExecutor.execute(DetailQueryExecutor.java:39)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.(CarbonScanRDD.scala:193)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.scala:174)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:264)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:264)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:264)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:264)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:264)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:264)
> at
[jira] [Resolved] (CARBONDATA-358) Compaction is not working in latest release
[ https://issues.apache.org/jira/browse/CARBONDATA-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala resolved CARBONDATA-358. Resolution: Fixed Fix Version/s: 0.2.0-incubating > Compaction is not working in latest release > --- > > Key: CARBONDATA-358 > URL: https://issues.apache.org/jira/browse/CARBONDATA-358 > Project: CarbonData > Issue Type: Bug > Components: data-query >Affects Versions: 0.2.0-incubating > Environment: stand alone environment >Reporter: Krishna Reddy > Fix For: 0.2.0-incubating > > > Compaction feature is not working in open source 0.2 release (Major, Minor > and Auto) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CARBONDATA-442) Query result mismatching with Hive
[
https://issues.apache.org/jira/browse/CARBONDATA-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravindra Pesala resolved CARBONDATA-442.
Resolution: Won't Fix
Fix Version/s: 1.0.0-incubating
Not an issue.
> Query result mismatching with Hive
> --
>
> Key: CARBONDATA-442
> URL: https://issues.apache.org/jira/browse/CARBONDATA-442
> Project: CarbonData
> Issue Type: Bug
>Reporter: SWATI RAO
> Fix For: 1.0.0-incubating
>
>
> => I created table using following command :
> create table Carbon_automation_test5 (imei string,deviceInformationId int,MAC
> string,deviceColor string,device_backColor string,modelId string,marketName
> string,AMSize string,ROMSize string,CUPAudit string,CPIClocked string,series
> string,productionDate string,bomCode string,internalModels string,
> deliveryTime string, channelsId string,channelsName string , deliveryAreaId
> string, deliveryCountry string, deliveryProvince string, deliveryCity
> string,deliveryDistrict string, deliveryStreet string,oxSingleNumber string,
> ActiveCheckTime string, ActiveAreaId string, ActiveCountry string,
> ActiveProvince string, Activecity string, ActiveDistrict string, ActiveStreet
> string, ActiveOperatorId string, Active_releaseId string, Active_EMUIVersion
> string,Active_operaSysVersion string, Active_BacVerNumber string,
> Active_BacFlashVer string,Active_webUIVersion string, Active_webUITypeCarrVer
> string,Active_webTypeDataVerNumber string, Active_operatorsVersion string,
> Active_phonePADPartitionedVersions string,Latest_YEAR int, Latest_MONTH int,
> Latest_DAY int, Latest_HOUR string, Latest_areaId string, Latest_country
> string, Latest_province string, Latest_city string,Latest_district string,
> Latest_street string, Latest_releaseId string,Latest_EMUIVersion string,
> Latest_operaSysVersion string, Latest_BacVerNumber string,Latest_BacFlashVer
> string, Latest_webUIVersion string, Latest_webUITypeCarrVer
> string,Latest_webTypeDataVerNumber string, Latest_operatorsVersion
> string,Latest_phonePADPartitionedVersions string, Latest_operatorId
> string,gamePointDescription string, gamePointId int,contractNumber int)
> stored by 'org.apache.carbondata.format'
> => Load csv to table :
> LOAD DATA INPATH 'hdfs://localhost:54310/user/hduser/100_olap.csv' INTO table
> Carbon_automation_test5 OPTIONS('DELIMITER'= ',' ,'QUOTECHAR'= '"',
> 'FILEHEADER'=
> 'imei,deviceInformationId,MAC,deviceColor,device_backColor,modelId,marketName,AMSize,ROMSize,CUPAudit,CPIClocked,series,productionDate,bomCode,internalModels,deliveryTime,channelsId,channelsName,deliveryAreaId,deliveryCountry,deliveryProvince,deliveryCity,deliveryDistrict,deliveryStreet,oxSingleNumber,contractNumber,ActiveCheckTime,ActiveAreaId,ActiveCountry,ActiveProvince,Activecity,ActiveDistrict,ActiveStreet,ActiveOperatorId,Active_releaseId,Active_EMUIVersion,Active_operaSysVersion,Active_BacVerNumber,Active_BacFlashVer,Active_webUIVersion,Active_webUITypeCarrVer,Active_webTypeDataVerNumber,Active_operatorsVersion,Active_phonePADPartitionedVersions,Latest_YEAR,Latest_MONTH,Latest_DAY,Latest_HOUR,Latest_areaId,Latest_country,Latest_province,Latest_city,Latest_district,Latest_street,Latest_releaseId,Latest_EMUIVersion,Latest_operaSysVersion,Latest_BacVerNumber,Latest_BacFlashVer,Latest_webUIVersion,Latest_webUITypeCarrVer,Latest_webTypeDataVerNumber,Latest_operatorsVersion,Latest_phonePADPartitionedVersions,Latest_operatorId,gamePointId,gamePointDescription')
> =>now executed SELECT querry :
> SELECT Carbon_automation_test5.AMSize AS AMSize,
> Carbon_automation_test5.ActiveCountry AS ActiveCountry,
> Carbon_automation_test5.Activecity AS Activecity ,
> SUM(Carbon_automation_test5.gamePointId) AS Sum_gamePointId FROM ( SELECT
> AMSize,ActiveCountry,gamePointId, Activecity FROM (select * from
> Carbon_automation_test5) SUB_QRY ) Carbon_automation_test5 INNER JOIN (
> SELECT ActiveCountry, Activecity, AMSize FROM (select * from
> Carbon_automation_test5) SUB_QRY ) Carbon_automation_vmall_test1 ON
> Carbon_automation_test5.AMSize = Carbon_automation_vmall_test1.AMSize WHERE
> NOT(Carbon_automation_test5.AMSize <= '3RAM size') GROUP BY
> Carbon_automation_test5.AMSize, Carbon_automation_test5.ActiveCountry,
> Carbon_automation_test5.Activecity ORDER BY Carbon_automation_test5.AMSize
> ASC, Carbon_automation_test5.ActiveCountry ASC,
> Carbon_automation_test5.Activecity ASC;
> +++-+--+--+
> | AMSize | ActiveCountry | Activecity | Sum_gamePointId |
> +++-+--+--+
> | 4RAM size | Chinese| changsha| 200860 |
> | 4RAM size | Chinese| guangzhou | 38016|
> | 4RAM size | Chinese| shenzhen| 49610
[jira] [Assigned] (CARBONDATA-476) storeLocation start with file:/// cause table not found exceptioin
[ https://issues.apache.org/jira/browse/CARBONDATA-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala reassigned CARBONDATA-476: -- Assignee: Ravindra Pesala > storeLocation start with file:/// cause table not found exceptioin > -- > > Key: CARBONDATA-476 > URL: https://issues.apache.org/jira/browse/CARBONDATA-476 > Project: CarbonData > Issue Type: Bug > Components: data-load >Affects Versions: 1.0.0-incubating >Reporter: WilliamZhu >Assignee: Ravindra Pesala >Priority: Minor > > When you set carbondata's storelocation starts with file:///, carbondata will > fail to create table meta,and it do not raise any exception > the code if found is in CarbonMetastore.createTableFromThrift > ``` > val thriftWriter = new ThriftWriter(schemaFilePath, false) > thriftWriter.open() > thriftWriter.write(thriftTableInfo)//if writing failed, it will be quiet. > thriftWriter.close() > ``` -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-carbondata issue #541: [CARBONDATA-641]supported date type for dic...
Github user CarbonDataQA commented on the issue: https://github.com/apache/incubator-carbondata/pull/541 Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/634/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (CARBONDATA-562) Carbon Context initialization is failed with spark 1.6.3
[ https://issues.apache.org/jira/browse/CARBONDATA-562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-562: --- Assignee: Naresh P R > Carbon Context initialization is failed with spark 1.6.3 > - > > Key: CARBONDATA-562 > URL: https://issues.apache.org/jira/browse/CARBONDATA-562 > Project: CarbonData > Issue Type: Bug > Components: core >Affects Versions: 0.2.0-incubating >Reporter: Babulal >Assignee: Naresh P R > > Carbon is compiled with below command > mvn clean -P spark-1.6,provided -Dspark.version=1.6.3 -DskipTests package > start carbon context in yarn client mode > bin/spark-shell --master yarn-client --executor-memory 2G --executor-cores > 2 --driver-memory 2G --num-executors 3 > __ > / __/__ ___ _/ /__ > _\ \/ _ \/ _ `/ __/ '_/ >/___/ .__/\_,_/_/ /_/\_\ version 1.6.3 > /_/ > Using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_111) > Type in expressions to have them evaluated. > scala> import org.apache.spark.sql.CarbonContext; > import org.apache.spark.sql.CarbonContext > scala> val cc=new CarbonContext(sc,"hdfs://hacluser/opt/CarbonStore"); > e of dependencies) > scala.tools.reflect.ToolBoxError: reflective compilation has failed: > not enough arguments for constructor Optimizer: (conf: > org.apache.spark.sql.catalyst.CatalystConf)org.apache.spark.sql.catalyst.optimizer.Optimizer. > Unspecified value parameter conf. > at > scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal.throwIfErrors(ToolBoxFactory.scala:314) > at > org.apache.spark.sql.CodeGenerateFactory.(CodeGenerateFactory.scala:32) > at > org.apache.spark.sql.CodeGenerateFactory$.init(CodeGenerateFactory.scala:67) > at org.apache.spark.sql.CarbonContext.(CarbonContext.scala:58) > at org.apache.spark.sql.CarbonContext.(CarbonContext.scala:52) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-477) Can we merge storeLocation and metaLocation into one?
[ https://issues.apache.org/jira/browse/CARBONDATA-477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-477: --- Issue Type: Improvement (was: Bug) > Can we merge storeLocation and metaLocation into one? > - > > Key: CARBONDATA-477 > URL: https://issues.apache.org/jira/browse/CARBONDATA-477 > Project: CarbonData > Issue Type: Improvement > Components: core >Affects Versions: 1.0.0-incubating >Reporter: WilliamZhu >Priority: Minor > > storeLocation is needed ,but meta locations is really needed? we should keep > carbondata configuration simple and should not confuse people who is not > familiar with it. > Also the default value of storeLocation may we use hive's warehouse? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CARBONDATA-479) Guarantee consistency for keyword LOCAL and file path in data loading command
[
https://issues.apache.org/jira/browse/CARBONDATA-479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824348#comment-15824348
]
Ravindra Pesala commented on CARBONDATA-479:
LOCAL is only used to keep the syntax synchronize with Hive. Right now there is
no effect of LOCAL keyword on loading flow.
> Guarantee consistency for keyword LOCAL and file path in data loading command
> -
>
> Key: CARBONDATA-479
> URL: https://issues.apache.org/jira/browse/CARBONDATA-479
> Project: CarbonData
> Issue Type: Bug
>Reporter: Lionx
>Priority: Minor
>
> In CarbonSqlParser.scala,
> protected lazy val loadDataNew: Parser[LogicalPlan] =
> LOAD ~> DATA ~> opt(LOCAL) ~> INPATH ~> stringLit ~ opt(OVERWRITE) ~
> (INTO ~> TABLE ~> (ident <~ ".").? ~ ident) ~
> (OPTIONS ~> "(" ~> repsep(loadOptions, ",") <~ ")").? <~ opt(";") ^^ {
> case filePath ~ isOverwrite ~ table ~ optionsList =>
> val (databaseNameOp, tableName) = table match {
> case databaseName ~ tableName => (databaseName,
> tableName.toLowerCase())
> }
> if (optionsList.isDefined) {
> validateOptions(optionsList)
> }
> val optionsMap = optionsList.getOrElse(List.empty[(String,
> String)]).toMap
> LoadTable(databaseNameOp, tableName, filePath, Seq(), optionsMap,
> isOverwrite.isDefined)
> }
> It seems that using Keyword LOCAL impacts noting. Loading data from hdfs or
> file just depends on the path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CARBONDATA-514) Select string type columns will return error.
[
https://issues.apache.org/jira/browse/CARBONDATA-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824346#comment-15824346
]
Ravindra Pesala commented on CARBONDATA-514:
[~lucao] Please verify it once on latest master and let us know the status of
it.
> Select string type columns will return error.
> -
>
> Key: CARBONDATA-514
> URL: https://issues.apache.org/jira/browse/CARBONDATA-514
> Project: CarbonData
> Issue Type: Bug
> Components: sql
>Affects Versions: 1.0.0-incubating
>Reporter: Cao, Lionel
> Attachments: Screenshot.png
>
>
> The data successfully loaded and count(*) is OK, but when I tried to query
> the detail data, it returns below error:
> scala> cc.sql("desc carbontest_002").show
> +-+-+---+
> | col_name|data_type|comment|
> +-+-+---+
> | vin| string| |
> |data_date| string| |
> +-+-+---+
> scala> cc.sql("load data inpath
> 'hdfs://nameservice2/user/appuser/lucao/mydata4.csv' into table
> default.carbontest_002 OPTIONS('DELIMITER'=',')")
> WARN 07-12 16:30:30,241 - main skip empty input file:
> hdfs://nameservice2/user/appuser/lucao/mydata4.csv/_SUCCESS
> AUDIT 07-12 16:30:34,338 - [*.com][appuser][Thread-1]Data load request has
> been received for table default.carbontest_002
> AUDIT 07-12 16:30:38,410 - [*.com][appuser][Thread-1]Data load is successful
> for default.carbontest_002
> res12: org.apache.spark.sql.DataFrame = []
> scala> cc.sql("select count(*) from carbontest_002")
> res14: org.apache.spark.sql.DataFrame = [_c0: bigint]
> scala> res14.show
> +---+
> |_c0|
> +---+
> |100|
> +---+
> scala> cc.sql("select vin, count(*) as cnt from carbontest_002 group by
> vin").show
> WARN 07-12 16:32:04,250 - Lost task 1.0 in stage 20.0 (TID 40, *.com):
> java.lang.ClassCastException: java.lang.String cannot be cast to
> java.lang.Integer
> at scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:106)
> at
> org.apache.spark.sql.catalyst.expressions.BaseGenericInternalRow$class.getInt(rows.scala:41)
>
> at
> org.apache.spark.sql.catalyst.expressions.GenericMutableRow.getInt(rows.scala:248)
>
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown
> Source)
> at
> org.apache.spark.sql.CarbonScan$$anonfun$1$$anon$1.next(CarbonScan.scala:155)
> at
> org.apache.spark.sql.CarbonScan$$anonfun$1$$anon$1.next(CarbonScan.scala:149)
> at
> org.apache.spark.sql.execution.aggregate.TungstenAggregationIterator.processInputs(TungstenAggregationIterator.scala:512)
>
> at
> org.apache.spark.sql.execution.aggregate.TungstenAggregationIterator.(TungstenAggregationIterator.scala:686)
>
> at
> org.apache.spark.sql.execution.aggregate.TungstenAggregate$$anonfun$doExecute$1$$anonfun$2.apply(TungstenAggregate.scala:95)
>
> at
> org.apache.spark.sql.execution.aggregate.TungstenAggregate$$anonfun$doExecute$1$$anonfun$2.apply(TungstenAggregate.scala:86)
>
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
>
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
>
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
> ERROR 07-12 16:32:04,516 - Task 1 in stage 20.0 failed 4 times; aborting job
> WARN 07-12 16:32:04,600 - Lost task 0.1 in stage 20.0 (TID 45, *):
> TaskKilled (killed intentionally)
> ERROR 07-12 16:32:04,604 - Listener SQLListener threw an exception
> java.lang.NullPointerException
> at
> org.apache.spark.sql.execution.ui.SQLListener.onTaskEnd(SQLListener.scala:167)
>
> at
> org.apache.spark.scheduler.SparkListenerBus$class.onPostEvent(SparkListenerBus.scala:42)
>
> at
> org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
>
> at
>
[jira] [Resolved] (CARBONDATA-589) carbon spark shell is not working with spark 2.0
[ https://issues.apache.org/jira/browse/CARBONDATA-589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala resolved CARBONDATA-589. Resolution: Won't Fix Fix Version/s: 1.0.0-incubating We are not going to support carbon spark shell from this version onwards. User can directly use spark-shell. > carbon spark shell is not working with spark 2.0 > > > Key: CARBONDATA-589 > URL: https://issues.apache.org/jira/browse/CARBONDATA-589 > Project: CarbonData > Issue Type: Bug > Components: build >Affects Versions: 1.0.0-incubating > Environment: Spark 2.0 >Reporter: anubhav tarar >Priority: Minor > Fix For: 1.0.0-incubating > > > carbon shell is not working with spark 2.0 version > here are the logs > ./carknoldus@knoldus:~/Desktop/open source/incubator-carbondata/bin$ > ./carbon-spark-shell > java.lang.ClassNotFoundException: org.apache.spark.repl.carbon.Main > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:348) > at org.apache.spark.util.Utils$.classForName(Utils.scala:225) > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:686) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CARBONDATA-552) Unthrown FilterUnsupportedException in catch block
[
https://issues.apache.org/jira/browse/CARBONDATA-552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravindra Pesala resolved CARBONDATA-552.
Resolution: Fixed
Fix Version/s: 1.0.0-incubating
It is already handled in code
> Unthrown FilterUnsupportedException in catch block
> --
>
> Key: CARBONDATA-552
> URL: https://issues.apache.org/jira/browse/CARBONDATA-552
> Project: CarbonData
> Issue Type: Bug
> Components: core
>Reporter: Jaechang Nam
>Priority: Trivial
> Fix For: 1.0.0-incubating
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> new FilterUnsupportedException(e) is not thrown in
> core/src/main/java/org/apache/carbondata/scan/filter/resolver/RowLevelRangeFilterResolverImpl.java
> (found in the current github snapshot,
> ddeb00425537ff59bdfba76779c5d96287e07d2e)
> {code}
> 230 }
> 231 } catch (FilterIllegalMemberException e) {
> 232 new FilterUnsupportedException(e);
> 233 }
> 234 return filterValuesList;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CARBONDATA-604) Use Kettle wrong option in Thrift server give wrong error message
[
https://issues.apache.org/jira/browse/CARBONDATA-604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravindra Pesala reassigned CARBONDATA-604:
--
Assignee: Ravindra Pesala
> Use Kettle wrong option in Thrift server give wrong error message
> -
>
> Key: CARBONDATA-604
> URL: https://issues.apache.org/jira/browse/CARBONDATA-604
> Project: CarbonData
> Issue Type: Bug
> Environment: Spark 1.6
>Reporter: Harmeet Singh
>Assignee: Ravindra Pesala
>Priority: Trivial
>
> I am using Thrift Server with following configuration:
> -Duse.kettle=default
> Here kettle value is wrong and when I run the thrift server, the thrift
> server is running successfully. After that, I am going to start beeline. In
> beeline, I am executing load command without any kettle option as below:
> LOAD DATA INPATH 'hdfs://localhost:54310/csvs/3000_UniqData.csv' into table
> uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> After executing above command, I am getting Error:
> java.lang.IllegalArgumentException: For input string: "default"
> (state=,code=0)
> This error, seems to be wrong, because I am not using any default value in
> load query, and default is used with thrift.
> So I have following Suggestions:
> 1. Change the error message, while load query is executing.
> 2. While starting thrift server with wrong value, thrift give us an error and
> not start.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CARBONDATA-635) ClassCastException in Spark 2.1 Cluster mode in insert query when name of column is changed and When the orders of columns are changed in the tables
[
https://issues.apache.org/jira/browse/CARBONDATA-635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824333#comment-15824333
]
Ravindra Pesala commented on CARBONDATA-635:
Please verify this issue once on latest master.
> ClassCastException in Spark 2.1 Cluster mode in insert query when name of
> column is changed and When the orders of columns are changed in the tables
>
>
> Key: CARBONDATA-635
> URL: https://issues.apache.org/jira/browse/CARBONDATA-635
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
>Affects Versions: 1.0.0-incubating
> Environment: Spark 2.1 Cluster mode
>Reporter: Harsh Sharma
>Priority: Minor
> Attachments: 2000_UniqData.csv, driverlog
>
>
> : SCENARIO 1 :::
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> CREATE TABLE student (CUST_ID2 int,CUST_ADDR String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA inpath 'hdfs://hadoop-master:54311/data/2000_UniqData.csv' INTO
> table uniqdata options('DELIMITER'=',', 'FILEHEADER'='CUST_ID, CUST_NAME,
> ACTIVE_EMUI_VERSION, DOB, DOJ, BIGINT_COLUMN1, BIGINT_COLUMN2,
> DECIMAL_COLUMN1, DECIMAL_COLUMN2, Double_COLUMN1, Double_COLUMN2,
> INTEGER_COLUMN1');
> insert into student select * from uniqdata;
> : SCENARIO 2 :::
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double) STORED BY
> 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB");
> CREATE TABLE student (ACTIVE_EMUI_VERSION string, DOB timestamp, CUST_ID
> int,CUST_NAME String, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double) STORED BY
> 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA inpath 'hdfs://hadoop-master:54311/data/2000_UniqData.csv' INTO
> table uniqdata options('DELIMITER'=',', 'FILEHEADER'='CUST_ID, CUST_NAME,
> ACTIVE_EMUI_VERSION, DOB, DOJ, BIGINT_COLUMN1, BIGINT_COLUMN2,
> DECIMAL_COLUMN1, DECIMAL_COLUMN2, Double_COLUMN1, Double_COLUMN2,
> INTEGER_COLUMN1');
> Above two scenarios have the same result and exception as below,
> 0: jdbc:hive2://hadoop-master:1> insert into student select * from
> uniqdata;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 26.0 failed 4 times, most recent failure: Lost task 0.3 in
> stage 26.0 (TID 38, 192.168.2.176, executor 0): java.lang.ClassCastException:
> org.apache.spark.unsafe.types.UTF8String cannot be cast to java.lang.Integer
> at scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:101)
> at
> org.apache.spark.sql.CarbonDictionaryDecoder$$anonfun$doExecute$1$$anonfun$7$$anon$1$$anonfun$next$1.apply$mcVI$sp(CarbonDictionaryDecoder.scala:186)
> at
> org.apache.spark.sql.CarbonDictionaryDecoder$$anonfun$doExecute$1$$anonfun$7$$anon$1$$anonfun$next$1.apply(CarbonDictionaryDecoder.scala:183)
> at
> org.apache.spark.sql.CarbonDictionaryDecoder$$anonfun$doExecute$1$$anonfun$7$$anon$1$$anonfun$next$1.apply(CarbonDictionaryDecoder.scala:183)
> at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:74)
> at
> org.apache.spark.sql.CarbonDictionaryDecoder$$anonfun$doExecute$1$$anonfun$7$$anon$1.next(CarbonDictionaryDecoder.scala:183)
> at
> org.apache.spark.sql.CarbonDictionaryDecoder$$anonfun$doExecute$1$$anonfun$7$$anon$1.next(CarbonDictionaryDecoder.scala:174)
> at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
>
[jira] [Updated] (CARBONDATA-642) Delete Subquery is not working while creating and loading 2 tables
[
https://issues.apache.org/jira/browse/CARBONDATA-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravindra Pesala updated CARBONDATA-642:
---
Assignee: sounak chakraborty
> Delete Subquery is not working while creating and loading 2 tables
> ---
>
> Key: CARBONDATA-642
> URL: https://issues.apache.org/jira/browse/CARBONDATA-642
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.0.0-incubating
>Reporter: Vyom Rastogi
>Assignee: sounak chakraborty
>Priority: Minor
> Labels: Spark-1.6
>
> Create table uiqdata
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format'
> TBLPROPERTIES('DICTIONARY_INCLUDE'='BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,INTEGER_COLUMN1,CUST_ID');
> Create table uniqdata1
> CREATE TABLE uniqdata1 (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format'
> TBLPROPERTIES('DICTIONARY_INCLUDE'='BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,INTEGER_COLUMN1,CUST_ID');
> Load the data into uniqdata
> LOAD DATA INPATH 'hdfs://hacluster/vyom/2000_UniqData.csv' into table
> uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
> 'BAD_RECORDS_ACTION'='REDIRECT','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> Load the data into uniqdata1
> LOAD DATA INPATH 'hdfs://hacluster/vyom/2000_UniqData.csv' into table
> uniqdata1 OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
> 'BAD_RECORDS_ACTION'='REDIRECT','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> Delete Subquery with below error
> Error: org.apache.spark.sql.AnalysisException:
> Unsupported language features in query: select tupleId from uniqdata1 a where
> a.CUST_ID in (Select b.CUST_ID from (Select c.CUST_ID from uniqdata c ) b)
> TOK_QUERY 1, 0,45, 20
> TOK_FROM 1, 4,8, 20
> TOK_TABREF 1, 6,8, 20
> TOK_TABNAME 1, 6,6, 20
> uniqdata1 1, 6,6, 20
> a 1, 8,8, 30
> TOK_INSERT 0, -1,45, 0
> TOK_DESTINATION 0, -1,-1, 0
> TOK_DIR 0, -1,-1, 0
> TOK_TMP_FILE 0, -1,-1, 0
> TOK_SELECT 1, 0,2, 7
> TOK_SELEXPR 1, 2,2, 7
> TOK_TABLE_OR_COL 1, 2,2, 7
> tupleId 1, 2,2, 7
> TOK_WHERE 1, 10,45, 49
> TOK_SUBQUERY_EXPR 1, 13,45, 49
> TOK_SUBQUERY_OP 1, 17,17, 49
> in 1, 17,17, 49
> TOK_QUERY 1, 19,45, 99
> TOK_FROM 1, 26,44, 99
> TOK_SUBQUERY 1, 28,44, 99
> TOK_QUERY 1, 29,40, 99
> TOK_FROM 1, 35,40, 99
> TOK_TABREF 1, 38,40, 99
> TOK_TABNAME 1, 38,38, 99
> uniqdata 1, 38,38, 99
> c 1, 40,40, 108
> TOK_INSERT 0, -1,33, 0
> TOK_DESTINATION 0, -1,-1, 0
> TOK_DIR 0, -1,-1, 0
> TOK_TMP_FILE 0, -1,-1, 0
> TOK_SELECT 1, 29,33, 84
> TOK_SELEXPR 1, 31,33, 84
> . 1, 31,33, 84
> TOK_TABLE_OR_COL 1, 31,31, 83
> c 1, 31,31, 83
> CUST_ID 1, 33,33, 85
> b 1, 44,44, 112
> TOK_INSERT 0, -1,24, 0
> TOK_DESTINATION 0, -1,-1, 0
> TOK_DIR 0, -1,-1, 0
> TOK_TMP_FILE 0, -1,-1, 0
> TOK_SELECT 1, 20,24, 61
> TOK_SELEXPR 1, 22,24, 61
> Csv: 2000_UniqData.csv
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CARBONDATA-85) please support insert into carbon table from other format table
[ https://issues.apache.org/jira/browse/CARBONDATA-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala resolved CARBONDATA-85. --- Resolution: Fixed Fix Version/s: 1.0.0-incubating > please support insert into carbon table from other format table > --- > > Key: CARBONDATA-85 > URL: https://issues.apache.org/jira/browse/CARBONDATA-85 > Project: CarbonData > Issue Type: Wish > Components: spark-integration >Affects Versions: 0.2.0-incubating >Reporter: Shoujie Zhuo >Assignee: QiangCai > Fix For: 1.0.0-incubating > > > exmaple: > insert into table2 select * from table1 > table2 is stored by org.apache.carbondata.format, and table 1 is a parquet > table -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-85) please support insert into carbon table from other format table
[ https://issues.apache.org/jira/browse/CARBONDATA-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-85: -- Assignee: QiangCai > please support insert into carbon table from other format table > --- > > Key: CARBONDATA-85 > URL: https://issues.apache.org/jira/browse/CARBONDATA-85 > Project: CarbonData > Issue Type: Wish > Components: spark-integration >Affects Versions: 0.2.0-incubating >Reporter: Shoujie Zhuo >Assignee: QiangCai > > exmaple: > insert into table2 select * from table1 > table2 is stored by org.apache.carbondata.format, and table 1 is a parquet > table -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-carbondata issue #541: [CARBONDATA-641]supported date type for dic...
Github user CarbonDataQA commented on the issue: https://github.com/apache/incubator-carbondata/pull/541 Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/633/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (CARBONDATA-104) To support varchar datatype
[ https://issues.apache.org/jira/browse/CARBONDATA-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-104: --- Assignee: zhangshunyu > To support varchar datatype > --- > > Key: CARBONDATA-104 > URL: https://issues.apache.org/jira/browse/CARBONDATA-104 > Project: CarbonData > Issue Type: New Feature >Reporter: zhangshunyu >Assignee: zhangshunyu >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CARBONDATA-164) Add template for pull requests
[ https://issues.apache.org/jira/browse/CARBONDATA-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala resolved CARBONDATA-164. Resolution: Fixed Fix Version/s: 0.2.0-incubating > Add template for pull requests > -- > > Key: CARBONDATA-164 > URL: https://issues.apache.org/jira/browse/CARBONDATA-164 > Project: CarbonData > Issue Type: New Feature >Reporter: Ajay Yadava > Fix For: 0.2.0-incubating > > > As per [discussion on the mailing > list|https://www.mail-archive.com/dev@carbondata.incubator.apache.org/msg00643.html], > this JIRA is to add a template for pull requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CARBONDATA-172) Pushdown spark known and unknown expression and equijoin to carbon
[ https://issues.apache.org/jira/browse/CARBONDATA-172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824321#comment-15824321 ] Ravindra Pesala commented on CARBONDATA-172: I guess Its not a good idea to pushdown these operations to carbon layer. > Pushdown spark known and unknown expression and equijoin to carbon > -- > > Key: CARBONDATA-172 > URL: https://issues.apache.org/jira/browse/CARBONDATA-172 > Project: CarbonData > Issue Type: Improvement >Reporter: Ashok Kumar >Assignee: Ashok Kumar >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > 1.Pushdown spark unknown expression > 2.Pushdown spark known expression > 3.Pushdown spark equijoin -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-172) Pushdown spark known and unknown expression and equijoin to carbon
[ https://issues.apache.org/jira/browse/CARBONDATA-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-172: --- Assignee: Ashok Kumar > Pushdown spark known and unknown expression and equijoin to carbon > -- > > Key: CARBONDATA-172 > URL: https://issues.apache.org/jira/browse/CARBONDATA-172 > Project: CarbonData > Issue Type: Improvement >Reporter: Ashok Kumar >Assignee: Ashok Kumar >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > 1.Pushdown spark unknown expression > 2.Pushdown spark known expression > 3.Pushdown spark equijoin -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-269) Change delete segment parser because its may confused with hive grammar
[ https://issues.apache.org/jira/browse/CARBONDATA-269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-269: --- Assignee: Jay > Change delete segment parser because its may confused with hive grammar > --- > > Key: CARBONDATA-269 > URL: https://issues.apache.org/jira/browse/CARBONDATA-269 > Project: CarbonData > Issue Type: Improvement >Reporter: Jay >Assignee: Jay >Priority: Minor > > the delete segment grammar in carbon is too similar with that delete grammar > in hive, while they have different usage. so change carbon's grammar to make > carbon easy use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-carbondata issue #537: fix unapproved licenses
Github user CarbonDataQA commented on the issue: https://github.com/apache/incubator-carbondata/pull/537 Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/632/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata issue #537: fix unapproved licenses
Github user QiangCai commented on the issue: https://github.com/apache/incubator-carbondata/pull/537 @ravipesala already added header check to java stype --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (CARBONDATA-402) carbon should support CreateAsSelect
[ https://issues.apache.org/jira/browse/CARBONDATA-402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-402: --- Assignee: Jay > carbon should support CreateAsSelect > - > > Key: CARBONDATA-402 > URL: https://issues.apache.org/jira/browse/CARBONDATA-402 > Project: CarbonData > Issue Type: Improvement >Reporter: Jay >Assignee: Jay >Priority: Minor > Time Spent: 1h 40m > Remaining Estimate: 0h > > provide support for CreateAsSelect, > the syntax is hive Syntax, like below: > Create TABLE table4 STORED BY 'carbondata' AS SELECT * FROM table3 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-424) Data Load will fail for badrecord and "bad_records_action" is fail
[ https://issues.apache.org/jira/browse/CARBONDATA-424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-424: --- Issue Type: Bug (was: New Feature) > Data Load will fail for badrecord and "bad_records_action" is fail > -- > > Key: CARBONDATA-424 > URL: https://issues.apache.org/jira/browse/CARBONDATA-424 > Project: CarbonData > Issue Type: Bug > Components: core, data-load, spark-integration >Reporter: Akash R Nilugal >Assignee: Akash R Nilugal > Time Spent: 2h 10m > Remaining Estimate: 0h > > Whenever there is a bad record found in csv file and the BAD_RECORDS_ACTION' > is FAIL then data load will fail with an error message which gives > information about the badrecord because of which data load is failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-424) Data Load will fail for badrecord and "bad_records_action" is fail
[ https://issues.apache.org/jira/browse/CARBONDATA-424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-424: --- Assignee: Akash R Nilugal > Data Load will fail for badrecord and "bad_records_action" is fail > -- > > Key: CARBONDATA-424 > URL: https://issues.apache.org/jira/browse/CARBONDATA-424 > Project: CarbonData > Issue Type: New Feature > Components: core, data-load, spark-integration >Reporter: Akash R Nilugal >Assignee: Akash R Nilugal > Time Spent: 2h 10m > Remaining Estimate: 0h > > Whenever there is a bad record found in csv file and the BAD_RECORDS_ACTION' > is FAIL then data load will fail with an error message which gives > information about the badrecord because of which data load is failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CARBONDATA-468) Add pruning in driver side to improve query performance
[ https://issues.apache.org/jira/browse/CARBONDATA-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala reassigned CARBONDATA-468: -- Assignee: Ravindra Pesala > Add pruning in driver side to improve query performance > --- > > Key: CARBONDATA-468 > URL: https://issues.apache.org/jira/browse/CARBONDATA-468 > Project: CarbonData > Issue Type: Sub-task >Reporter: Ravindra Pesala >Assignee: Ravindra Pesala > > Add pruning using bucketing information in driver side to improve query > performance -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CARBONDATA-466) Implement bucketing table in carbondata
[
https://issues.apache.org/jira/browse/CARBONDATA-466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravindra Pesala reassigned CARBONDATA-466:
--
Assignee: Ravindra Pesala
> Implement bucketing table in carbondata
> ---
>
> Key: CARBONDATA-466
> URL: https://issues.apache.org/jira/browse/CARBONDATA-466
> Project: CarbonData
> Issue Type: New Feature
>Reporter: Ravindra Pesala
>Assignee: Ravindra Pesala
>
> Bucketing is the useful feature when user wants to join big tables. And also
> it is useful in driver level partition pruning to improve query performance.
> User can add buckets on any dimension column (except complex types) as follows
> {code}
> CREATE TABLE test(user_id BIGINT, firstname STRING, lastname STRING)
> CLUSTERED BY(user_id) INTO 32 BUCKETS
> STORED BY 'carbondata';
> {code}
> In the above example column user_id is hash partitioned and creates 32 bucket
> files in carbondata. So while doing the join with other table on bucketed
> column it can select same buckets and do the join with out shuffling.
> Carbon format changes
> 1. Bucketing information needs to be stored inside schema thrift file
> 2. Bucket id can be stored inside every carbondata index file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CARBONDATA-469) Leveraging Carbondata's bucketing info for optimized Join operation
[ https://issues.apache.org/jira/browse/CARBONDATA-469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala resolved CARBONDATA-469. Resolution: Fixed Fix Version/s: 1.0.0-incubating > Leveraging Carbondata's bucketing info for optimized Join operation > --- > > Key: CARBONDATA-469 > URL: https://issues.apache.org/jira/browse/CARBONDATA-469 > Project: CarbonData > Issue Type: Sub-task >Reporter: Ravindra Pesala >Assignee: Ravindra Pesala > Fix For: 1.0.0-incubating > > > Optimize join in spark using bucketing information to avoid shuffling when > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CARBONDATA-469) Leveraging Carbondata's bucketing info for optimized Join operation
[ https://issues.apache.org/jira/browse/CARBONDATA-469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala reassigned CARBONDATA-469: -- Assignee: Ravindra Pesala > Leveraging Carbondata's bucketing info for optimized Join operation > --- > > Key: CARBONDATA-469 > URL: https://issues.apache.org/jira/browse/CARBONDATA-469 > Project: CarbonData > Issue Type: Sub-task >Reporter: Ravindra Pesala >Assignee: Ravindra Pesala > Fix For: 1.0.0-incubating > > > Optimize join in spark using bucketing information to avoid shuffling when > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-531) Eliminate spark dependency in carbon core
[ https://issues.apache.org/jira/browse/CARBONDATA-531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-531: --- Assignee: Jacky Li > Eliminate spark dependency in carbon core > - > > Key: CARBONDATA-531 > URL: https://issues.apache.org/jira/browse/CARBONDATA-531 > Project: CarbonData > Issue Type: Improvement >Affects Versions: 0.2.0-incubating >Reporter: Jacky Li >Assignee: Jacky Li > Fix For: 1.0.0-incubating > > Time Spent: 40m > Remaining Estimate: 0h > > Clean up the interface and take out Spark dependency on Carbon-core module. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-527) Greater than/less-than/Like filters optimization for dictionary encoded columns
[ https://issues.apache.org/jira/browse/CARBONDATA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-527: --- Assignee: Sujith > Greater than/less-than/Like filters optimization for dictionary encoded > columns > --- > > Key: CARBONDATA-527 > URL: https://issues.apache.org/jira/browse/CARBONDATA-527 > Project: CarbonData > Issue Type: New Feature >Reporter: Sujith >Assignee: Sujith > Time Spent: 50m > Remaining Estimate: 0h > > Current design > In greater than/less-than/Like filters, system first iterates each row > present in the dictionary cache for identifying valid filter actual members > by applying the filter expression , once evaluation done system will hold the > list of identified valid filter actual member values(String), now in next > step again system will look up the dictionary cache in order to identify the > dictionary surrogate values of the identified members. this look up is an > additional cost to our system even though the look up methodology is an > binary search in dictionary cache. > > Proposed design/solution: > Identify the dictionary surrogate values in filter expression evaluation step > itself when actual dictionary values will be scanned for identifying valid > filter members . > Keep a dictionary counter variable which will be increased when system > iterates through the dictionary cache in order to retrieve each actual > member stored in dictionary cache , after this system will evaluate each row > against the filter expression to identify whether its a valid filter member > or not, while doing this process itself counter value can be taken as valid > selected dictionary value since the actual member values and its dictionary > values will be kept in same order in dictionary cache as the iteration order. > thus it will eliminate the further dictionary look up step which is required > to retrieve the dictionary surrogate value against identified actual valid > filter member. this can also increase significantly the filter query > performance of such filter queries which require expression evaluation to > identify it the filter members by looking up dictionary cache, like greater > than/less-than/Like filters . > Note : this optimization is applicable for dictionary columns. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CARBONDATA-538) Add test case to spark2 integration
[ https://issues.apache.org/jira/browse/CARBONDATA-538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala resolved CARBONDATA-538. Resolution: Fixed > Add test case to spark2 integration > --- > > Key: CARBONDATA-538 > URL: https://issues.apache.org/jira/browse/CARBONDATA-538 > Project: CarbonData > Issue Type: Improvement >Reporter: Jacky Li >Assignee: QiangCai > Fix For: 1.0.0-incubating > > > Currently spark2 integration has very few test case, it should be improved -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-538) Add test case to spark2 integration
[ https://issues.apache.org/jira/browse/CARBONDATA-538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-538: --- Assignee: QiangCai > Add test case to spark2 integration > --- > > Key: CARBONDATA-538 > URL: https://issues.apache.org/jira/browse/CARBONDATA-538 > Project: CarbonData > Issue Type: Improvement >Reporter: Jacky Li >Assignee: QiangCai > Fix For: 1.0.0-incubating > > > Currently spark2 integration has very few test case, it should be improved -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-553) Create integration test-case for dataframe API
[ https://issues.apache.org/jira/browse/CARBONDATA-553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-553: --- Assignee: Rahul Kumar > Create integration test-case for dataframe API > -- > > Key: CARBONDATA-553 > URL: https://issues.apache.org/jira/browse/CARBONDATA-553 > Project: CarbonData > Issue Type: Test >Reporter: Rahul Kumar >Assignee: Rahul Kumar > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-565) Clean up code suggested by IDE analyzer
[ https://issues.apache.org/jira/browse/CARBONDATA-565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-565: --- Assignee: Jacky Li > Clean up code suggested by IDE analyzer > --- > > Key: CARBONDATA-565 > URL: https://issues.apache.org/jira/browse/CARBONDATA-565 > Project: CarbonData > Issue Type: Improvement >Reporter: Jacky Li >Assignee: Jacky Li > Fix For: 1.0.0-incubating > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-556) when loading, it may happen that multiple tasks in one node, while other node has no task.
[ https://issues.apache.org/jira/browse/CARBONDATA-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-556: --- Assignee: Jay > when loading, it may happen that multiple tasks in one node, while other node > has no task. > -- > > Key: CARBONDATA-556 > URL: https://issues.apache.org/jira/browse/CARBONDATA-556 > Project: CarbonData > Issue Type: Improvement >Reporter: Jay >Assignee: Jay >Priority: Minor > Time Spent: 1h 20m > Remaining Estimate: 0h > > the algorithm of carbondata loading is aim to ensure one task in one node , > but in cluster, it may happen that mutilple tasks in one node, while other > node has no tasks, which cost more time because of out of balance load. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-carbondata issue #537: fix unapproved licenses
Github user CarbonDataQA commented on the issue: https://github.com/apache/incubator-carbondata/pull/537 Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/631/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (CARBONDATA-644) Select query fails randomly on spark shell
[
https://issues.apache.org/jira/browse/CARBONDATA-644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824254#comment-15824254
]
Ravindra Pesala commented on CARBONDATA-644:
I think you have configured "fs.defaultFS" in your conf/ *.xml files. If you
configure this parameter as hdfs then it always look this path into hadoop file
system. Please verify it once.
> Select query fails randomly on spark shell
> --
>
> Key: CARBONDATA-644
> URL: https://issues.apache.org/jira/browse/CARBONDATA-644
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
>Affects Versions: 1.0.0-incubating
> Environment: spark 1.6.2
>Reporter: Deepti Bhardwaj
>Assignee: Ravindra Pesala
>Priority: Minor
> Attachments: connectdemo1.csv, eror-log-while-select
>
>
> I created a carbon table and loaded it with data(csv attached) via spark
> shell by running the below command from $SPARK_HOME/bin :
> ./spark-shell
> and then to get the carbon context :
> import org.apache.spark.sql.CarbonContext;
> val cc = new CarbonContext(sc);
> CREATE AND LOAD COMMANDS:
> scala>cc.sql("CREATE TABLE connectdemo1 (name String, gender String, province
> String , singler String, age Int) STORED BY 'org.apache.carbondata.format'");
> scala>cc.sql("LOAD DATA inpath
> 'hdfs://localhost:54310/BabuStore/Data/uniqdata/connectdemo1.csv' INTO table
> connectdemo1 options('DELIMITER'=',','FILEHEADER'='name, gender, province,
> singler, age')");
> The above two operations were successful but when I tried to run the select
> query, it failed giving me the java.io.FileNotFoundException(see the log
> attached)
> scala> cc.sql("select * from connectdemo1").show();
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] incubator-carbondata pull request #370: [CARBONDATA-457] Added Unit Test cas...
Github user PallaviSingh1992 closed the pull request at: https://github.com/apache/incubator-carbondata/pull/370 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata pull request #428: [CARBONDATA-529] - Added Unit Test c...
Github user PallaviSingh1992 closed the pull request at: https://github.com/apache/incubator-carbondata/pull/428 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Closed] (CARBONDATA-414) Access array elements using index than Loop
[ https://issues.apache.org/jira/browse/CARBONDATA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pallavi Singh closed CARBONDATA-414. > Access array elements using index than Loop > --- > > Key: CARBONDATA-414 > URL: https://issues.apache.org/jira/browse/CARBONDATA-414 > Project: CarbonData > Issue Type: Improvement > Environment: Spark 1.6 >Reporter: Pallavi Singh >Priority: Trivial > Fix For: 1.0.0-incubating > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-529) Add Unit Tests for processing.newflow.parser package
[ https://issues.apache.org/jira/browse/CARBONDATA-529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pallavi Singh closed CARBONDATA-529. Resolution: Fixed > Add Unit Tests for processing.newflow.parser package > > > Key: CARBONDATA-529 > URL: https://issues.apache.org/jira/browse/CARBONDATA-529 > Project: CarbonData > Issue Type: Test > Environment: Spark 1.6 >Reporter: Pallavi Singh >Priority: Trivial > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-428) Remove Redundant Condition Checks
[ https://issues.apache.org/jira/browse/CARBONDATA-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pallavi Singh closed CARBONDATA-428. Resolution: Fixed > Remove Redundant Condition Checks > - > > Key: CARBONDATA-428 > URL: https://issues.apache.org/jira/browse/CARBONDATA-428 > Project: CarbonData > Issue Type: Improvement > Environment: Spark 1.6 >Reporter: Pallavi Singh >Priority: Trivial > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-426) replace if else with conditional operator
[ https://issues.apache.org/jira/browse/CARBONDATA-426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pallavi Singh closed CARBONDATA-426. Resolution: Fixed > replace if else with conditional operator > - > > Key: CARBONDATA-426 > URL: https://issues.apache.org/jira/browse/CARBONDATA-426 > Project: CarbonData > Issue Type: Improvement > Environment: Spark 1.6 >Reporter: Pallavi Singh >Priority: Trivial > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-carbondata pull request #532: [CARBONDATA-638] Move package in car...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-carbondata/pull/532 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (CARBONDATA-638) Move and refine package in carbon-core module
[ https://issues.apache.org/jira/browse/CARBONDATA-638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Chen resolved CARBONDATA-638. --- Resolution: Fixed > Move and refine package in carbon-core module > - > > Key: CARBONDATA-638 > URL: https://issues.apache.org/jira/browse/CARBONDATA-638 > Project: CarbonData > Issue Type: Improvement >Reporter: Jacky Li >Assignee: Jacky Li > Fix For: 1.0.0-incubating > > Time Spent: 1h 50m > Remaining Estimate: 0h > > As a step to refactor code to make it more modular, this PR move following > package in carbon-core module > move org.apache.carbondata.core.carbon to org.apache.carbondata.core > move org.apache.carbondata.common.ext to org.apache.carbondata.core.service > rename org.apache.carbondata.core.update to org.apache.carbondata.core.mutate > and move org.apache.carbondata.common.iudprocessor.iuddata to > org.apache.carbondata.core.mutate.data > move org.apache.carbondata.core.partition to org.apache.carbondata.processing > move org.apache.carbondata.fileoperation to org.apache.carbondata.core.atomic > move org.apache.carbondata.locks to > org.apache.carbondata.core.updatestatus.locks > move CarbonDataLoadSchema to carbon-processing > move all Ideintifier class to org.apaceh.carbondata.core.metadata > move org.apache.carbondata.core.datastorage to > org.apache.carbondata.core.datastore -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-carbondata issue #532: [CARBONDATA-638] Move package in carbon-cor...
Github user chenliang613 commented on the issue: https://github.com/apache/incubator-carbondata/pull/532 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (CARBONDATA-638) Move and refine package in carbon-core module
[ https://issues.apache.org/jira/browse/CARBONDATA-638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Chen updated CARBONDATA-638: -- Description: As a step to refactor code to make it more modular, this PR move following package in carbon-core module move org.apache.carbondata.core.carbon to org.apache.carbondata.core move org.apache.carbondata.common.ext to org.apache.carbondata.core.service rename org.apache.carbondata.core.update to org.apache.carbondata.core.mutate and move org.apache.carbondata.common.iudprocessor.iuddata to org.apache.carbondata.core.mutate.data move org.apache.carbondata.core.partition to org.apache.carbondata.processing move org.apache.carbondata.fileoperation to org.apache.carbondata.core.atomic move org.apache.carbondata.locks to org.apache.carbondata.core.updatestatus.locks move CarbonDataLoadSchema to carbon-processing move all Ideintifier class to org.apaceh.carbondata.core.metadata move org.apache.carbondata.core.datastorage to org.apache.carbondata.core.datastore was: move org.apache.carbondata.core.carbon to org.apache.carbondata.core move org.apache.carbondata.common.ext to org.apache.carbondata.core.service move org.apache.carbondata.common.iudprocessor.iuddata to org.apache.carbondata.core.update move org.apache.carbondata.core.partition to org.apache.carbondata.processing move org.apache.carbondata.fileoperation to org.apache.carbondata.core.fileoperation move org.apache.carbondata.locks to org.apache.carbondata.core.locks move CarbonDataLoadSchema to carbon-processing move all Ideintifier class to org.apaceh.carbondata.core.metadata move org.apache.carbondata.core.datastorage to org.apache.carbondata.core.datastore > Move and refine package in carbon-core module > - > > Key: CARBONDATA-638 > URL: https://issues.apache.org/jira/browse/CARBONDATA-638 > Project: CarbonData > Issue Type: Improvement >Reporter: Jacky Li >Assignee: Jacky Li > Fix For: 1.0.0-incubating > > Time Spent: 1.5h > Remaining Estimate: 0h > > As a step to refactor code to make it more modular, this PR move following > package in carbon-core module > move org.apache.carbondata.core.carbon to org.apache.carbondata.core > move org.apache.carbondata.common.ext to org.apache.carbondata.core.service > rename org.apache.carbondata.core.update to org.apache.carbondata.core.mutate > and move org.apache.carbondata.common.iudprocessor.iuddata to > org.apache.carbondata.core.mutate.data > move org.apache.carbondata.core.partition to org.apache.carbondata.processing > move org.apache.carbondata.fileoperation to org.apache.carbondata.core.atomic > move org.apache.carbondata.locks to > org.apache.carbondata.core.updatestatus.locks > move CarbonDataLoadSchema to carbon-processing > move all Ideintifier class to org.apaceh.carbondata.core.metadata > move org.apache.carbondata.core.datastorage to > org.apache.carbondata.core.datastore -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-638) Move and refine package in carbon-core module
[ https://issues.apache.org/jira/browse/CARBONDATA-638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Chen updated CARBONDATA-638: -- Summary: Move and refine package in carbon-core module (was: Move package in carbon-core module) > Move and refine package in carbon-core module > - > > Key: CARBONDATA-638 > URL: https://issues.apache.org/jira/browse/CARBONDATA-638 > Project: CarbonData > Issue Type: Improvement >Reporter: Jacky Li >Assignee: Jacky Li > Fix For: 1.0.0-incubating > > Time Spent: 1.5h > Remaining Estimate: 0h > > move org.apache.carbondata.core.carbon to org.apache.carbondata.core > move org.apache.carbondata.common.ext to org.apache.carbondata.core.service > move org.apache.carbondata.common.iudprocessor.iuddata to > org.apache.carbondata.core.update > move org.apache.carbondata.core.partition to org.apache.carbondata.processing > move org.apache.carbondata.fileoperation to > org.apache.carbondata.core.fileoperation > move org.apache.carbondata.locks to org.apache.carbondata.core.locks > move CarbonDataLoadSchema to carbon-processing > move all Ideintifier class to org.apaceh.carbondata.core.metadata > move org.apache.carbondata.core.datastorage to > org.apache.carbondata.core.datastore -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-carbondata pull request #222: [CARBONDATA-221] Fix the bug of inve...
Github user Zhangshunyu closed the pull request at: https://github.com/apache/incubator-carbondata/pull/222 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata pull request #254: [CARBONDATA-326] Create wrong table ...
Github user Zhangshunyu closed the pull request at: https://github.com/apache/incubator-carbondata/pull/254 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata issue #532: [CARBONDATA-638] Move package in carbon-cor...
Github user CarbonDataQA commented on the issue: https://github.com/apache/incubator-carbondata/pull/532 Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/630/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata issue #532: [CARBONDATA-638] Move package in carbon-cor...
Github user CarbonDataQA commented on the issue: https://github.com/apache/incubator-carbondata/pull/532 Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/629/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
