[GitHub] carbondata issue #2347: [CARBONDATA-2554] Added support for logical type
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2347 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5015/ ---
[GitHub] carbondata issue #2347: [CARBONDATA-2554] Added support for logical type
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2347 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6176/ ---
[GitHub] carbondata issue #2347: [CARBONDATA-2554] Added support for logical type
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2347 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5151/ ---
[jira] [Resolved] (CARBONDATA-2555) SDK Reader should have isTransactionalTable = false by default, to be inline with SDK writer
[ https://issues.apache.org/jira/browse/CARBONDATA-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor resolved CARBONDATA-2555. -- Resolution: Fixed Fix Version/s: 1.4.0 > SDK Reader should have isTransactionalTable = false by default, to be inline > with SDK writer > > > Key: CARBONDATA-2555 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2555 > Project: CarbonData > Issue Type: Bug >Reporter: Ajantha Bhat >Assignee: Ajantha Bhat >Priority: Major > Fix For: 1.4.0 > > Time Spent: 3h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2352: [CARBONDATA-2555]Fixed SDK reader set default...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2352 ---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Github user kunal642 commented on the issue: https://github.com/apache/carbondata/pull/2352 LGTM ---
[GitHub] carbondata issue #2354: [WIP] Remove dead code
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2354 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6175/ ---
[GitHub] carbondata issue #2354: [WIP] Remove dead code
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2354 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5150/ ---
[GitHub] carbondata issue #2290: [CARBONDATA-2389] Search mode support lucene datamap
Github user xubo245 commented on the issue: https://github.com/apache/carbondata/pull/2290 @jackylk CI pass ---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2352 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5013/ ---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2352 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5149/ ---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2352 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6174/ ---
[GitHub] carbondata issue #2290: [CARBONDATA-2389] Search mode support lucene datamap
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2290 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5012/ ---
[GitHub] carbondata issue #2290: [CARBONDATA-2389] Search mode support lucene datamap
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2290 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6173/ ---
[GitHub] carbondata issue #2290: [CARBONDATA-2389] Search mode support lucene datamap
Github user xubo245 commented on the issue: https://github.com/apache/carbondata/pull/2290 retest this please ---
[GitHub] carbondata issue #2354: [WIP] Remove dead code
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2354 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5148/ ---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2352 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5147/ ---
[GitHub] carbondata issue #2290: [CARBONDATA-2389] Search mode support lucene datamap
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2290 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5146/ ---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2352 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5010/ ---
[GitHub] carbondata issue #2290: [CARBONDATA-2389] Search mode support lucene datamap
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2290 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5145/ ---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2352 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6171/ ---
[GitHub] carbondata issue #2347: [CARBONDATA-2554] Added support for logical type
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2347 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5144/ ---
[GitHub] carbondata issue #2354: [WIP] Remove dead code
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2354 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6172/ ---
[GitHub] carbondata issue #2290: [CARBONDATA-2389] Search mode support lucene datamap
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2290 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6170/ ---
[GitHub] carbondata issue #2354: [WIP] Remove dead code
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2354 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5011/ ---
[GitHub] carbondata pull request #2354: [WIP] Remove dead code
GitHub user sraghunandan opened a pull request: https://github.com/apache/carbondata/pull/2354 [WIP] Remove dead code Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/sraghunandan/carbondata-1 improve_coverage_new Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2354.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2354 commit a8041c06255eba3587601b9433d29dd50466d57b Author: sraghunandan Date: 2017-09-07T12:05:22Z remove dead code ---
[GitHub] carbondata issue #2347: [CARBONDATA-2554] Added support for logical type
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2347 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5143/ ---
[GitHub] carbondata issue #2348: [CARBONDATA-2546] Fixed the ArrayIndexOutOfBoundsExc...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2348 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6166/ ---
[GitHub] carbondata issue #2347: [CARBONDATA-2554] Added support for logical type
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2347 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6169/ ---
[GitHub] carbondata issue #2348: [CARBONDATA-2546] Fixed the ArrayIndexOutOfBoundsExc...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2348 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5005/ ---
[jira] [Created] (CARBONDATA-2564) [MV] Query Failed when MV created when same name column used in both table
Babulal created CARBONDATA-2564: --- Summary: [MV] Query Failed when MV created when same name column used in both table Key: CARBONDATA-2564 URL: https://issues.apache.org/jira/browse/CARBONDATA-2564 Project: CarbonData Issue Type: Bug Reporter: Babulal create table mvtest8( name string,age int,salary int) stored by 'carbondata'; create table mvtest9( name string,age int,salary int) stored by 'carbondata'; insert into mvtest9 select 'a',12,12; insert into mvtest8 select 'a',12,12; 0: jdbc:hive2://10.18.222.231:23040> select * from mvtest8 join mvtest9 on mvtest8.name=mvtest9.name; +---+--+-+---+--+-+--+ | name | age | salary | name | age | salary | +---+--+-+---+--+-+--+ | a | 12 | 12 | a | 12 | 12 | +---+--+-+---+--+-+--+ 0: jdbc:hive2://10.18.222.231:23040> create datamap mv8andmv9map using 'mv' as select * from mvtest8 join mvtest9 on mvtest8.name=mvtest9.name; +-+--+ | Result | +-+--+ +-+--+ No rows selected (0.52 seconds) 0: jdbc:hive2://10.18.222.231:23040> rebuild datamap mv8andmv9map; +-+--+ | Result | +-+--+ +-+–+ 0: jdbc:hive2://10.18.222.231:23040> explain select * from mvtest8 join mvtest9 on mvtest8.name=mvtest9.name; Error: org.apache.spark.sql.AnalysisException: Reference 'name' is ambiguous, could be: name#1888, name#2412.; line 1 pos 28 (state=,code=0) 0: jdbc:hive2://10.18.222.231:23040> 0: jdbc:hive2://10.18.222.231:23040> desc formatted mv8andmv9map_table; +---+---+---+--+ | col_name | data_type | comment | +---+---+---+--+ | mvtest8_name | string | KEY COLUMN,null | | mvtest8_age | int | MEASURE,null | | mvtest8_salary | int | MEASURE,null | | mvtest9_name | string | KEY COLUMN,null | | mvtest9_age | int | MEASURE,null | | mvtest9_salary | int | MEASURE,null | | | | | | ##Detailed Table Information | | | | Database Name | default | | | Table Name | mv8andmv9map_table | | | CARBON Store Path | hdfs://hacluster/user/hive/warehouse/carbon.store/default/mv8andmv9map_table | | | Comment | | | | Table Block Size | 1024 MB | | | Table Data Size | 1484 | | | Table Index Size | 940 | | | Last Update Time | 1527610493258 | | | SORT_SCOPE | LOCAL_SORT | LOCAL_SORT | | Streaming | false | | | | | | | ##Detailed Column property | | | | ADAPTIVE | | | | SORT_COLUMNS | mvtest8_name,mvtest9_name | | +---+---+---+--+ 22 rows selected (0.19 seconds) 0: jdbc:hive2://10.18.222.231:23040> select mvtest8.name from mvtest8 join mvtest9 on mvtest8.name=mvtest9.name; Error: org.apache.spark.sql.AnalysisException: Reference 'name' is ambiguous, could be: name#1888, name#2412.; line 3 pos 10 (state=,code=0) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata issue #2348: [CARBONDATA-2546] Fixed the ArrayIndexOutOfBoundsExc...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2348 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5142/ ---
[jira] [Updated] (CARBONDATA-2313) Support Non Transactional carbon table
[ https://issues.apache.org/jira/browse/CARBONDATA-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajantha Bhat updated CARBONDATA-2313: - Attachment: carbon NonTranscational Table_v1.0.pdf > Support Non Transactional carbon table > -- > > Key: CARBONDATA-2313 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2313 > Project: CarbonData > Issue Type: New Feature >Reporter: Ajantha Bhat >Assignee: sounak chakraborty >Priority: Major > Attachments: carbon NonTranscational Table_v1.0.pdf > > Time Spent: 21h 50m > Remaining Estimate: 0h > > h5. Support unmanaged carbon table -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata issue #2290: [CARBONDATA-2389] Search mode support lucene datamap
Github user xubo245 commented on the issue: https://github.com/apache/carbondata/pull/2290 retest this please ---
[jira] [Updated] (CARBONDATA-2313) Support Non Transactional carbon table
[ https://issues.apache.org/jira/browse/CARBONDATA-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajantha Bhat updated CARBONDATA-2313: - Attachment: (was: carbon unamanged table_V1.0.pdf) > Support Non Transactional carbon table > -- > > Key: CARBONDATA-2313 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2313 > Project: CarbonData > Issue Type: New Feature >Reporter: Ajantha Bhat >Assignee: sounak chakraborty >Priority: Major > Time Spent: 21h 50m > Remaining Estimate: 0h > > h5. Support unmanaged carbon table -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CARBONDATA-2546) It will throw exception when give same column twice in projection and tries to print it.
[ https://issues.apache.org/jira/browse/CARBONDATA-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacky Li resolved CARBONDATA-2546. -- Resolution: Fixed Fix Version/s: 1.4.1 > It will throw exception when give same column twice in projection and tries > to print it. > > > Key: CARBONDATA-2546 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2546 > Project: CarbonData > Issue Type: Bug >Reporter: xubo245 >Assignee: xubo245 >Priority: Major > Fix For: 1.4.1 > > Time Spent: 6h 50m > Remaining Estimate: 0h > > It will throw exception when give same column twice in projection and tries > to print it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2348: [CARBONDATA-2546] Fixed the ArrayIndexOutOfBo...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2348 ---
[GitHub] carbondata issue #2348: [CARBONDATA-2546] Fixed the ArrayIndexOutOfBoundsExc...
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/2348 LGTM ---
[jira] [Resolved] (CARBONDATA-2558) Optimize carbon schema reader interface of SDK
[ https://issues.apache.org/jira/browse/CARBONDATA-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacky Li resolved CARBONDATA-2558. -- Resolution: Fixed Fix Version/s: 1.4.1 > Optimize carbon schema reader interface of SDK > -- > > Key: CARBONDATA-2558 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2558 > Project: CarbonData > Issue Type: Improvement >Reporter: xubo245 >Assignee: xubo245 >Priority: Major > Fix For: 1.4.1 > > Time Spent: 2h > Remaining Estimate: 0h > > Optimize carbon schema reader interface of SDK -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2353: [CARBONDATA-2558] Optimize carbon schema read...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2353 ---
[GitHub] carbondata issue #2353: [CARBONDATA-2558] Optimize carbon schema reader inte...
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/2353 LGTM ---
[GitHub] carbondata issue #2353: [CARBONDATA-2558] Optimize carbon schema reader inte...
Github user xubo245 commented on the issue: https://github.com/apache/carbondata/pull/2353 @jackylk @sounakr CI pass, please review it ---
[GitHub] carbondata issue #2347: [CARBONDATA-2554] Added support for logical type
Github user kunal642 commented on the issue: https://github.com/apache/carbondata/pull/2347 retest this please ---
[GitHub] carbondata issue #2290: [CARBONDATA-2389] Search mode support lucene datamap
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2290 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6168/ ---
[GitHub] carbondata issue #2347: [CARBONDATA-2554] Added support for logical type
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2347 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6167/ ---
[GitHub] carbondata issue #2353: [CARBONDATA-2558] Optimize carbon schema reader inte...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2353 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5141/ ---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2352 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5003/ ---
[GitHub] carbondata issue #2353: [CARBONDATA-2558] Optimize carbon schema reader inte...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2353 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6164/ ---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2352 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6163/ ---
[GitHub] carbondata issue #2348: [CARBONDATA-2546] Fixed the ArrayIndexOutOfBoundsExc...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2348 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5001/ ---
[GitHub] carbondata pull request #2290: [CARBONDATA-2389] Search mode support lucene ...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2290#discussion_r191451235
--- Diff:
core/src/main/java/org/apache/carbondata/core/readcommitter/LatestFilesReadCommittedScope.java
---
@@ -42,10 +39,27 @@
public class LatestFilesReadCommittedScope implements ReadCommittedScope {
private String carbonFilePath;
+ private String segmentId;
private ReadCommittedIndexFileSnapShot readCommittedIndexFileSnapShot;
private LoadMetadataDetails[] loadMetadataDetails;
- public LatestFilesReadCommittedScope(String path) {
+ /**
+ * a new constructor of this class, which supports obtain lucene index
in search mode
+ *
+ * @param path carbon file path
+ * @param segmentId segment id
+ */
+ public LatestFilesReadCommittedScope(String path, String segmentId) {
+this.carbonFilePath = path;
--- End diff --
ok, done.
segmentId can be null.
---
[GitHub] carbondata issue #2347: [CARBONDATA-2554] Added support for logical type
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2347 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5140/ ---
[GitHub] carbondata pull request #2290: [CARBONDATA-2389] Search mode support lucene ...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2290#discussion_r191450259
--- Diff:
core/src/main/java/org/apache/carbondata/core/readcommitter/LatestFilesReadCommittedScope.java
---
@@ -42,10 +39,27 @@
public class LatestFilesReadCommittedScope implements ReadCommittedScope {
private String carbonFilePath;
+ private String segmentId;
private ReadCommittedIndexFileSnapShot readCommittedIndexFileSnapShot;
private LoadMetadataDetails[] loadMetadataDetails;
- public LatestFilesReadCommittedScope(String path) {
+ /**
+ * a new constructor of this class, which supports obtain lucene index
in search mode
--- End diff --
ok, remove it
---
[GitHub] carbondata pull request #2290: [CARBONDATA-2389] Search mode support lucene ...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2290#discussion_r191450160
--- Diff:
core/src/main/java/org/apache/carbondata/core/readcommitter/LatestFilesReadCommittedScope.java
---
@@ -42,10 +39,27 @@
public class LatestFilesReadCommittedScope implements ReadCommittedScope {
private String carbonFilePath;
+ private String segmentId;
private ReadCommittedIndexFileSnapShot readCommittedIndexFileSnapShot;
private LoadMetadataDetails[] loadMetadataDetails;
- public LatestFilesReadCommittedScope(String path) {
+ /**
+ * a new constructor of this class, which supports obtain lucene index
in search mode
+ *
+ * @param path carbon file path
+ * @param segmentId segment id
+ */
+ public LatestFilesReadCommittedScope(String path, String segmentId) {
+this.carbonFilePath = path;
+this.segmentId = segmentId;
+try {
+ takeCarbonIndexFileSnapShot();
+} catch (IOException ex) {
+ throw new RuntimeException("Error while taking index snapshot", ex);
+}
+ }
+
+ public LatestFilesReadCommittedScope(String path) {
--- End diff --
ok, done
---
[GitHub] carbondata pull request #2290: [CARBONDATA-2389] Search mode support lucene ...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2290#discussion_r191448581
--- Diff:
core/src/main/java/org/apache/carbondata/core/datamap/DataMapStoreManager.java
---
@@ -96,13 +96,19 @@ private DataMapStoreManager() {
String dbName = carbonTable.getDatabaseName();
String tableName = carbonTable.getTableName();
String dmName = dataMap.getDataMapSchema().getDataMapName();
- boolean isDmVisible = sessionInfo.getSessionParams().getProperty(
- String.format("%s%s.%s.%s",
CarbonCommonConstants.CARBON_DATAMAP_VISIBLE,
- dbName, tableName, dmName),
"true").trim().equalsIgnoreCase("true");
- if (!isDmVisible) {
-LOGGER.warn(String.format("Ignore invisible datamap %s on table
%s.%s",
-dmName, dbName, tableName));
-dataMapIterator.remove();
+ if (sessionInfo != null) {
--- End diff --
ok,done
---
[jira] [Created] (CARBONDATA-2563) Explain query with Order by operator is fired Spark Job which is increase explain query time
Babulal created CARBONDATA-2563: --- Summary: Explain query with Order by operator is fired Spark Job which is increase explain query time Key: CARBONDATA-2563 URL: https://issues.apache.org/jira/browse/CARBONDATA-2563 Project: CarbonData Issue Type: Bug Reporter: Babulal Attachments: image-2018-05-29-20-02-58-129.png Create Table (hive or carbon ) create table justtesthive( name string,age int) insert into justtesthive select 'babu',12; WithCarbonSession means startCarbonThriftserver Without CarbonSession (with SparkSession) 0: jdbc:hive2://10.18.222.231:23040> explain select name from justtesthive order by name; ++--+ | plan | ++--+ | == Physical Plan == *Sort [name#15 ASC NULLS FIRST], true, 0 +- Exchange rangepartitioning(name#15 ASC NULLS FIRST, 200) +- HiveTableScan [name#15], HiveTableRelation `default`.`justtesthive`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, [name#15, age#16] | ++--+ 1 row selected (0.089 seconds) 0: jdbc:hive2://10.18.222.231:23040> With CarbonSession. 0: jdbc:hive2://10.18.222.231:23040> explain select name from justtesthive order by name; +--+--+ | plan | +--+--+ | == CarbonData Profiler == | | == Physical Plan == *Sort [name#1867 ASC NULLS FIRST], true, 0 +- Exchange rangepartitioning(name#1867 ASC NULLS FIRST, 200) +- HiveTableScan [name#1867], HiveTableRelation `default`.`justtesthive`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, [name#1867, age#1868] | +--+--+ 2 rows selected (11.609 seconds) Time taken by explain is 10 sec for CarbonSession because SparkJob is fired , but for Explain Job should not fired . !image-2018-05-29-20-02-58-129.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2290: [CARBONDATA-2389] Search mode support lucene ...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2290#discussion_r191447617
--- Diff:
core/src/main/java/org/apache/carbondata/core/datamap/DataMapStoreManager.java
---
@@ -96,13 +96,19 @@ private DataMapStoreManager() {
String dbName = carbonTable.getDatabaseName();
String tableName = carbonTable.getTableName();
String dmName = dataMap.getDataMapSchema().getDataMapName();
- boolean isDmVisible = sessionInfo.getSessionParams().getProperty(
- String.format("%s%s.%s.%s",
CarbonCommonConstants.CARBON_DATAMAP_VISIBLE,
- dbName, tableName, dmName),
"true").trim().equalsIgnoreCase("true");
- if (!isDmVisible) {
-LOGGER.warn(String.format("Ignore invisible datamap %s on table
%s.%s",
-dmName, dbName, tableName));
-dataMapIterator.remove();
+ if (sessionInfo != null) {
+boolean isDmVisible = sessionInfo.getSessionParams().getProperty(
+String.format("%s%s.%s.%s",
CarbonCommonConstants.CARBON_DATAMAP_VISIBLE,
+dbName, tableName, dmName),
"true").trim().equalsIgnoreCase("true");
+if (!isDmVisible) {
+ LOGGER.warn(String.format("Ignore invisible datamap %s on table
%s.%s",
+ dmName, dbName, tableName));
+ dataMapIterator.remove();
+}
+ } else {
+// TODO: need support get the visible status of datamp in the
future
+String message = "Carbon session info is null";
+LOGGER.audit(message);
--- End diff --
ok,done
---
[GitHub] carbondata pull request #2290: [CARBONDATA-2389] Search mode support lucene ...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2290#discussion_r191447507
--- Diff:
datamap/lucene/src/main/java/org/apache/carbondata/datamap/lucene/LuceneDataMapFactoryBase.java
---
@@ -235,7 +236,8 @@ public DataMapBuilder createBuilder(Segment segment,
String shardName) {
}
for (CarbonFile indexDir : indexDirs) {
// Filter out the tasks which are filtered through CG datamap.
- if
(!segment.getFilteredIndexShardNames().contains(indexDir.getName())) {
+ if (getDataMapLevel() != DataMapLevel.FG &&
+
!segment.getFilteredIndexShardNames().contains(indexDir.getName())) {
--- End diff --
ok,done
---
[GitHub] carbondata pull request #2290: [CARBONDATA-2389] Search mode support lucene ...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2290#discussion_r191447282
--- Diff:
hadoop/src/main/java/org/apache/carbondata/hadoop/CarbonRecordReader.java ---
@@ -77,8 +77,10 @@ public void initialize(InputSplit inputSplit,
TaskAttemptContext context)
} else {
throw new RuntimeException("unsupported input split type: " +
inputSplit);
}
-List tableBlockInfoList =
CarbonInputSplit.createBlocks(splitList);
-queryModel.setTableBlockInfos(tableBlockInfoList);
+if (queryModel.getTableBlockInfos().isEmpty()) {
--- End diff --
ok, done
---
[GitHub] carbondata issue #2353: [CARBONDATA-2558] Optimize carbon schema reader inte...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2353 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5002/ ---
[GitHub] carbondata pull request #2290: [CARBONDATA-2389] Search mode support lucene ...
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2290#discussion_r191442174 --- Diff: core/src/main/java/org/apache/carbondata/core/scan/scanner/impl/BlockletFilterScanner.java --- @@ -169,6 +169,10 @@ private BlockletScannedResult executeFilter(RawBlockletColumnChunks rawBlockletC // apply filter on actual data, for each page BitSetGroup bitSetGroup = this.filterExecuter.applyFilter(rawBlockletColumnChunks, useBitSetPipeLine); +// if bitSetGroup is nul, then new BitSetGroup object, which can avoid NPE --- End diff -- before PR2242, it will be null if the bitSetGroup of rawBlockletColumnChunks is null, so we can remove this now ---
[GitHub] carbondata issue #2348: [CARBONDATA-2546] Fixed the ArrayIndexOutOfBoundsExc...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2348 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6162/ ---
[GitHub] carbondata pull request #2290: [CARBONDATA-2389] Search mode support lucene ...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2290#discussion_r191432727
--- Diff:
store/search/src/main/java/org/apache/carbondata/store/worker/SearchRequestHandler.java
---
@@ -158,22 +163,36 @@ private QueryModel prune(int queryId, CarbonTable
table, QueryModel queryModel,
CarbonMultiBlockSplit mbSplit, DataMapExprWrapper datamap) throws
IOException {
Objects.requireNonNull(datamap);
List segments = new LinkedList<>();
+HashMap uniqueSegments = new HashMap<>();
for (CarbonInputSplit split : mbSplit.getAllSplits()) {
- segments.add(
- Segment.toSegment(split.getSegmentId(),
- new LatestFilesReadCommittedScope(table.getTablePath(;
+ String segmentId = split.getSegmentId();
+ if (uniqueSegments.get(segmentId) == null) {
+segments.add(Segment.toSegment(
+segmentId,
+new LatestFilesReadCommittedScope(table.getTablePath(),
segmentId)));
+uniqueSegments.put(segmentId, 1);
+ } else {
+uniqueSegments.put(segmentId, uniqueSegments.get(segmentId) + 1);
+ }
+}
+
+List distributables =
datamap.toDistributable(segments);
+List prunnedBlocklets = new
LinkedList();
+for (int i = 0; i < distributables.size(); i++) {
+ DataMapDistributable dataMapDistributable =
distributables.get(i).getDistributable();
+ prunnedBlocklets.addAll(datamap.prune(dataMapDistributable, null));
}
-List prunnedBlocklets = datamap.prune(segments,
null);
-List pathToRead = new LinkedList<>();
-for (ExtendedBlocklet prunnedBlocklet : prunnedBlocklets) {
- pathToRead.add(prunnedBlocklet.getPath());
+HashMap pathToRead = new HashMap<>();
+for (ExtendedBlocklet prunedBlocklet : prunnedBlocklets) {
+ pathToRead.put(prunedBlocklet.getFilePath(), prunedBlocklet);
}
List blocks = queryModel.getTableBlockInfos();
List blockToRead = new LinkedList<>();
for (TableBlockInfo block : blocks) {
- if (pathToRead.contains(block.getFilePath())) {
+ if (pathToRead.keySet().contains(block.getFilePath())) {
+
block.setDataMapWriterPath(pathToRead.get(block.getFilePath()).getDataMapWriterPath());
--- End diff --
if not set this, the bitSetGroup will be null (reason refer
org.apache.carbondata.core.indexstore.blockletindex.BlockletDataRefNode#getIndexedData).
And it will throw exception in
org.apache.carbondata.core.scan.filter.executer.RowLevelFilterExecuterImpl#applyFilter(org.apache.carbondata.core.scan.processor.RawBlockletColumnChunks,
boolean)
---
[GitHub] carbondata issue #2351: [CARBONDATA-2559] task id set for each carbonReader ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2351 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5139/ ---
[GitHub] carbondata issue #2347: [CARBONDATA-2554] Added support for logical type
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2347 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6161/ ---
[GitHub] carbondata issue #2348: [CARBONDATA-2546] Fixed the ArrayIndexOutOfBoundsExc...
Github user xubo245 commented on the issue: https://github.com/apache/carbondata/pull/2348 retest this please ---
[GitHub] carbondata issue #2348: [CARBONDATA-2546] Fixed the ArrayIndexOutOfBoundsExc...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2348 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4998/ ---
[GitHub] carbondata pull request #2290: [CARBONDATA-2389] Search mode support lucene ...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2290#discussion_r191417277
--- Diff: store/search/src/main/scala/org/apache/spark/rpc/Master.scala ---
@@ -142,7 +142,7 @@ class Master(sparkConf: SparkConf) {
(address,
schedulable.ref.ask[ShutdownResponse](ShutdownRequest("user")))
}
futures.foreach { case (address, future) =>
- ThreadUtils.awaitResult(future, Duration.apply("10s"))
+ ThreadUtils.awaitResult(future, Duration.apply("100s"))
--- End diff --
ok, down, default value is 10s
---
[GitHub] carbondata issue #2347: [CARBONDATA-2554] Added support for logical type
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2347 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4999/ ---
[GitHub] carbondata issue #2353: [CARBONDATA-2558] Optimize carbon schema reader inte...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2353 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5138/ ---
[GitHub] carbondata issue #2351: [CARBONDATA-2559] task id set for each carbonReader ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2351 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6159/ ---
[GitHub] carbondata issue #2353: [CARBONDATA-2558] Optimize carbon schema reader inte...
Github user xubo245 commented on the issue: https://github.com/apache/carbondata/pull/2353 retest this please ---
[jira] [Assigned] (CARBONDATA-2561) Carbondata support external datasource
[ https://issues.apache.org/jira/browse/CARBONDATA-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xuchuanyin reassigned CARBONDATA-2561: -- Assignee: (was: xuchuanyin) > Carbondata support external datasource > -- > > Key: CARBONDATA-2561 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2561 > Project: CarbonData > Issue Type: New Feature >Reporter: xuchuanyin >Priority: Major > > Proposal for Carbondata to support external datasource. > The external datasource means that the data is not stored as CarbonData file > format, it may be a CSV/Parquet/Orc table and it may also be a table in HBase. > By using CarbonData, user can create indexes (known as DataMap in CarbonData) > and Materialize Views on the external datasource to accelerate the query > performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata issue #2347: [CARBONDATA-2554] Added support for logical type
Github user sounakr commented on the issue: https://github.com/apache/carbondata/pull/2347 @kunal642 . Please check all these logicalType can be supported in the current PR. 1. Time-millis 2. Time-micros   2. duration  ---
[jira] [Assigned] (CARBONDATA-2561) Carbondata support external datasource
[ https://issues.apache.org/jira/browse/CARBONDATA-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xuchuanyin reassigned CARBONDATA-2561: -- Assignee: xuchuanyin > Carbondata support external datasource > -- > > Key: CARBONDATA-2561 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2561 > Project: CarbonData > Issue Type: New Feature >Reporter: xuchuanyin >Assignee: xuchuanyin >Priority: Major > > Proposal for Carbondata to support external datasource. > The external datasource means that the data is not stored as CarbonData file > format, it may be a CSV/Parquet/Orc table and it may also be a table in HBase. > By using CarbonData, user can create indexes (known as DataMap in CarbonData) > and Materialize Views on the external datasource to accelerate the query > performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CARBONDATA-2562) Support creating datamaps on external CSV datasource
xuchuanyin created CARBONDATA-2562: -- Summary: Support creating datamaps on external CSV datasource Key: CARBONDATA-2562 URL: https://issues.apache.org/jira/browse/CARBONDATA-2562 Project: CarbonData Issue Type: Sub-task Reporter: xuchuanyin Assignee: xuchuanyin Support creating indexed datamap on external CSV datasource. Support rebuilding the indexed datamap for the external CSV datasource. Query on external datasource make use of datamap if it is available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CARBONDATA-2561) Carbondata support external datasource
xuchuanyin created CARBONDATA-2561: -- Summary: Carbondata support external datasource Key: CARBONDATA-2561 URL: https://issues.apache.org/jira/browse/CARBONDATA-2561 Project: CarbonData Issue Type: New Feature Reporter: xuchuanyin Proposal for Carbondata to support external datasource. The external datasource means that the data is not stored as CarbonData file format, it may be a CSV/Parquet/Orc table and it may also be a table in HBase. By using CarbonData, user can create indexes (known as DataMap in CarbonData) and Materialize Views on the external datasource to accelerate the query performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2347: [CARBONDATA-2554] Added support for logical t...
Github user sounakr commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2347#discussion_r191403062
--- Diff:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/AvroCarbonWriter.java ---
@@ -177,13 +198,26 @@ private static Field prepareFields(Schema.Field
avroField) {
String FieldName = avroField.name();
Schema childSchema = avroField.schema();
Schema.Type type = childSchema.getType();
+LogicalType logicalType = childSchema.getLogicalType();
switch (type) {
case BOOLEAN:
return new Field(FieldName, DataTypes.BOOLEAN);
case INT:
-return new Field(FieldName, DataTypes.INT);
+if (logicalType instanceof LogicalTypes.Date) {
+ return new Field(FieldName, DataTypes.DATE);
+} else {
+ LOGGER.warn("Unsupported logical type. Considering Data Type as
INT for " + childSchema
+ .getName());
+ return new Field(FieldName, DataTypes.INT);
+}
case LONG:
-return new Field(FieldName, DataTypes.LONG);
+if (logicalType instanceof LogicalTypes.TimestampMillis) {
--- End diff --
Don't we have to check the TimeStampMicros and TimeStamp logicaltypes
---
[GitHub] carbondata pull request #2347: [CARBONDATA-2554] Added support for logical t...
Github user sounakr commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2347#discussion_r191404125
--- Diff:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/AvroCarbonWriter.java ---
@@ -221,13 +255,22 @@ private static Field prepareFields(Schema.Field
avroField) {
private static StructField prepareSubFields(String FieldName, Schema
childSchema) {
Schema.Type type = childSchema.getType();
+LogicalType logicalType = childSchema.getLogicalType();
switch (type) {
case BOOLEAN:
return new StructField(FieldName, DataTypes.BOOLEAN);
case INT:
-return new StructField(FieldName, DataTypes.INT);
+if (logicalType == null) {
+ return new StructField(FieldName, DataTypes.INT);
--- End diff --
Make the checks in sync with avroFieldToObject logicaltype checks
---
[GitHub] carbondata issue #2353: [CARBONDATA-2558] Optimize carbon schema reader inte...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2353 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6160/ ---
[GitHub] carbondata pull request #2347: [CARBONDATA-2554] Added support for logical t...
Github user sounakr commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2347#discussion_r191398287
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/datatypes/PrimitiveDataType.java
---
@@ -316,15 +321,32 @@ public int getSurrogateIndex() {
if (!this.carbonDimension.getUseActualData()) {
byte[] value = null;
if (isDirectDictionary) {
- int surrogateKey =
dictionaryGenerator.getOrGenerateKey(parsedValue);
+ int surrogateKey;
+ if (dictionaryGenerator instanceof DirectDictionary
+ && input instanceof Long) {
+surrogateKey = ((DirectDictionary)
dictionaryGenerator).generateKey((long) input);
+ } else {
+surrogateKey =
dictionaryGenerator.getOrGenerateKey(parsedValue);
+ }
if (surrogateKey ==
CarbonCommonConstants.INVALID_SURROGATE_KEY) {
value = new byte[0];
} else {
value = ByteUtil.toBytes(surrogateKey);
}
} else {
- value =
DataTypeUtil.getBytesBasedOnDataTypeForNoDictionaryColumn(parsedValue,
- this.carbonDimension.getDataType(), dateFormat);
+ if (this.carbonDimension.getDataType().equals(DataTypes.DATE)
+ ||
this.carbonDimension.getDataType().equals(DataTypes.TIMESTAMP)
+ && input instanceof Long) {
--- End diff --
Add a comment on which case input will be long
---
[GitHub] carbondata pull request #2347: [CARBONDATA-2554] Added support for logical t...
Github user sounakr commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2347#discussion_r191396859
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/datatypes/PrimitiveDataType.java
---
@@ -288,7 +288,12 @@ public int getSurrogateIndex() {
logHolder.setReason(message);
}
} else {
-surrogateKey = dictionaryGenerator.getOrGenerateKey(parsedValue);
+if (dictionaryGenerator instanceof DirectDictionary
+&& input instanceof Long) {
--- End diff --
Rectify the indentation.
---
[GitHub] carbondata pull request #2347: [CARBONDATA-2554] Added support for logical t...
Github user sounakr commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2347#discussion_r191399826
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/loading/steps/InputProcessorStepWithNoConverterImpl.java
---
@@ -313,7 +315,22 @@ private CarbonRowBatch getBatch() {
throw new CarbonDataLoadingException("Loading Exception", e);
}
} else {
-newData[i] = data[orderOfData[i]];
+DataType dataType = dataFields[i].getColumn().getDataType();
+if (dataType == DataTypes.DATE && data[orderOfData[i]]
instanceof Long) {
+ DirectDictionaryGenerator directDictionaryGenerator =
--- End diff --
why everytime a new directDictionaryGenerator object is needed? It can also
be a member variable of InputProcessorStepWithNoConverterImpl and initialize
only once.
---
[GitHub] carbondata pull request #2347: [CARBONDATA-2554] Added support for logical t...
Github user sounakr commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2347#discussion_r191396937
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/datatypes/PrimitiveDataType.java
---
@@ -316,15 +321,32 @@ public int getSurrogateIndex() {
if (!this.carbonDimension.getUseActualData()) {
byte[] value = null;
if (isDirectDictionary) {
- int surrogateKey =
dictionaryGenerator.getOrGenerateKey(parsedValue);
+ int surrogateKey;
+ if (dictionaryGenerator instanceof DirectDictionary
--- End diff --
Rectify the indentation
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Github user kunal642 commented on the issue: https://github.com/apache/carbondata/pull/2352 retest this please ---
[GitHub] carbondata issue #2350: [CARBONDATA-2553] support ZSTD compression for sort ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2350 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5137/ ---
[GitHub] carbondata issue #2351: [CARBONDATA-2559] task id set for each carbonReader ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2351 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4997/ ---
[GitHub] carbondata issue #2348: [CARBONDATA-2546] Fixed the ArrayIndexOutOfBoundsExc...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2348 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4995/ ---
[jira] [Resolved] (CARBONDATA-2552) Fix Data Mismatch for Complex Data type Array of Timestamp with Dictionary Include
[ https://issues.apache.org/jira/browse/CARBONDATA-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kumar vishal resolved CARBONDATA-2552. -- Resolution: Fixed Fix Version/s: 1.4.0 > Fix Data Mismatch for Complex Data type Array of Timestamp with Dictionary > Include > -- > > Key: CARBONDATA-2552 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2552 > Project: CarbonData > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > Fix For: 1.4.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata issue #2348: [CARBONDATA-2546] Fixed the ArrayIndexOutOfBoundsExc...
Github user brijoobopanna commented on the issue: https://github.com/apache/carbondata/pull/2348 retest sdv please ---
[GitHub] carbondata pull request #2349: [CARBONDATA-2552]Fix Data Mismatch for Comple...
Github user Indhumathi27 closed the pull request at: https://github.com/apache/carbondata/pull/2349 ---
[GitHub] carbondata pull request #2347: [CARBONDATA-2554] Added support for logical t...
Github user sounakr commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2347#discussion_r191386255
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/datatypes/PrimitiveDataType.java
---
@@ -288,7 +288,12 @@ public int getSurrogateIndex() {
logHolder.setReason(message);
}
} else {
-surrogateKey = dictionaryGenerator.getOrGenerateKey(parsedValue);
+if (dictionaryGenerator instanceof DirectDictionary
--- End diff --
@Override public void writeByteArray(Object input, DataOutputStream
dataOutputStream,
BadRecordLogHolder logHolder) throws IOException,
DictionaryGenerationException {
String parsedValue =
input == null ? null : DataTypeUtil.parseValue(input.toString(),
carbonDimension);
If the input is long, is it needed to parse as toString?
---
[GitHub] carbondata issue #2349: [CARBONDATA-2552]Fix Data Mismatch for Complex Data ...
Github user kumarvishal09 commented on the issue: https://github.com/apache/carbondata/pull/2349 LGTM ---
[GitHub] carbondata issue #2348: [CARBONDATA-2546] Fixed the ArrayIndexOutOfBoundsExc...
Github user xubo245 commented on the issue: https://github.com/apache/carbondata/pull/2348 retest this please ---
[GitHub] carbondata issue #2348: [CARBONDATA-2546] Fixed the ArrayIndexOutOfBoundsExc...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2348 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6157/ ---
[GitHub] carbondata issue #2349: [CARBONDATA-2552]Fix Data Mismatch for Complex Data ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2349 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6156/ ---
[GitHub] carbondata issue #2353: [CARBONDATA-2558] Optimize carbon schema reader inte...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2353 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4996/ ---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Github user sounakr commented on the issue: https://github.com/apache/carbondata/pull/2352 LGTM ---
[GitHub] carbondata issue #2350: [CARBONDATA-2553] support ZSTD compression for sort ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2350 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4994/ ---
[GitHub] carbondata issue #2349: [CARBONDATA-2552]Fix Data Mismatch for Complex Data ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2349 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5136/ ---
