[GitHub] carbondata pull request #2134: [CARBONDATA-2310] Refactored code to improve ...

[dhatchayani]

GitHub user dhatchayani opened a pull request:

https://github.com/apache/carbondata/pull/2134

[CARBONDATA-2310] Refactored code to improve Distributable interface

Refactored code to improve Distributable interface

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [x] Testing done
Manual Testing
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dhatchayani/carbondata distributable

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2134.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2134


commit dfad93daa4c9cf549f5a9d0d4362024354964454
Author: dhatchayani 
Date:   2018-04-03T05:49:43Z

[CARBONDATA-2310] Refactored code to improve Distributable interface




---

[jira] [Created] (CARBONDATA-2310) Refactored code to improve Distributable interface

[dhatchayani (JIRA)]

dhatchayani created CARBONDATA-2310:
---

 Summary: Refactored code to improve Distributable interface
 Key: CARBONDATA-2310
 URL: https://issues.apache.org/jira/browse/CARBONDATA-2310
 Project: CarbonData
  Issue Type: Improvement
Reporter: dhatchayani
Assignee: dhatchayani






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata issue #2131: [WIP] Support unmanaged carbon table read and write

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2131
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4272/



---

[GitHub] carbondata issue #2125: [CARBONDATA-2299]Support showing all segment informa...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2125
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4271/



---

[GitHub] carbondata issue #2125: [CARBONDATA-2299]Support showing all segment informa...

[zzcclp]

Github user zzcclp commented on the issue:

https://github.com/apache/carbondata/pull/2125
  
retest this please


---

[GitHub] carbondata pull request #2131: [WIP] Support unmanaged carbon table read and...

[gvramana]

Github user gvramana commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2131#discussion_r178717012
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/readcommitter/LatestFilesReadCommitted.java
 ---
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.core.readcommitter;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.Map;
+
+import org.apache.carbondata.core.datamap.Segment;
+import org.apache.carbondata.core.datastore.filesystem.CarbonFile;
+import 
org.apache.carbondata.core.indexstore.blockletindex.SegmentIndexFileStore;
+import org.apache.carbondata.core.statusmanager.LoadMetadataDetails;
+import org.apache.carbondata.core.statusmanager.SegmentStatus;
+import org.apache.carbondata.core.util.path.CarbonTablePath;
+
+public class LatestFilesReadCommitted implements ReadCommitted {
+
+  private String carbonFilePath;
+  private ReadCommittedIndexFileSnapShot readCommittedIndexFileSnapShot;
+
+  public LatestFilesReadCommitted(String path) {
+this.carbonFilePath = path;
+try {
+  takeCarbonIndexFileSnapShot();
+} catch (IOException ex) {
+  // TODO Put proper Log and throw the exception out.
+  System.out.println("Error while reding index file");
+}
+  }
+
+  @Override public LoadMetadataDetails[] getSegmentList() throws 
IOException {
+// Read the Segment path and form the LoadMetadataDetails array.
+File fs = new File(carbonFilePath);
+
+if (fs.isDirectory()) {
+
+  CarbonFile[] carbonIndexFiles = 
SegmentIndexFileStore.getCarbonIndexFiles(carbonFilePath);
--- End diff --

should read from readCommittedIndexFileSnapShot, not filesystem


---

[GitHub] carbondata pull request #2131: [WIP] Support unmanaged carbon table read and...

[gvramana]

Github user gvramana commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2131#discussion_r178715161
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/readcommitter/LatestFilesReadCommitted.java
 ---
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.core.readcommitter;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.Map;
+
+import org.apache.carbondata.core.datamap.Segment;
+import org.apache.carbondata.core.datastore.filesystem.CarbonFile;
+import 
org.apache.carbondata.core.indexstore.blockletindex.SegmentIndexFileStore;
+import org.apache.carbondata.core.statusmanager.LoadMetadataDetails;
+import org.apache.carbondata.core.statusmanager.SegmentStatus;
+import org.apache.carbondata.core.util.path.CarbonTablePath;
+
+public class LatestFilesReadCommitted implements ReadCommitted {
+
+  private String carbonFilePath;
+  private ReadCommittedIndexFileSnapShot readCommittedIndexFileSnapShot;
+
+  public LatestFilesReadCommitted(String path) {
+this.carbonFilePath = path;
+try {
+  takeCarbonIndexFileSnapShot();
+} catch (IOException ex) {
+  // TODO Put proper Log and throw the exception out.
+  System.out.println("Error while reding index file");
+}
+  }
+
+  @Override public LoadMetadataDetails[] getSegmentList() throws 
IOException {
+// Read the Segment path and form the LoadMetadataDetails array.
+File fs = new File(carbonFilePath);
+
+if (fs.isDirectory()) {
+
+  CarbonFile[] carbonIndexFiles = 
SegmentIndexFileStore.getCarbonIndexFiles(carbonFilePath);
+  LoadMetadataDetails[] loadMetadataDetailsArray =
+  new LoadMetadataDetails[carbonIndexFiles.length];
+  int loadCount = 0;
+  for (int i = 0; i < carbonIndexFiles.length; i++) {
--- End diff --

There can be multiple indexfiles under same logical segment/transactionid


---

[GitHub] carbondata issue #2125: [CARBONDATA-2299]Support showing all segment informa...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2125
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3541/



---

[GitHub] carbondata pull request #2131: [WIP] Support unmanaged carbon table read and...

[gvramana]

Github user gvramana commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2131#discussion_r178714736
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/indexstore/blockletindex/SegmentIndexFileStore.java
 ---
@@ -200,6 +216,90 @@ public void readAllIIndexOfSegment(CarbonFile[] 
carbonFiles) throws IOException
 return indexFiles;
   }
 
+  /**
+   * Read all index file names of the carbon File Path.
+   *
+   * @return
+   * @throws IOException
+   */
+  public Map getReadCommittedIndexFilesFromPath(String 
carbonFilePath)
+  throws IOException {
+// Get only those index files which are mentioned in the
+CarbonFile[] carbonIndexFiles = getCarbonIndexFiles(carbonFilePath, 
carbonIndexMap);
+Map indexFiles = new HashMap<>();
+for (int i = 0; i < carbonIndexFiles.length; i++) {
+  if 
(carbonIndexFiles[i].getName().endsWith(CarbonTablePath.MERGE_INDEX_FILE_EXT)) {
+List indexFilesFromMergeFile =
+
getIndexFilesFromMergeFile(carbonIndexFiles[i].getCanonicalPath());
+for (String file : indexFilesFromMergeFile) {
+  
indexFiles.put(carbonIndexFiles[i].getParentFile().getAbsolutePath()
+  + CarbonCommonConstants.FILE_SEPARATOR + file, 
carbonIndexFiles[i].getName());
+}
+  } else if 
(carbonIndexFiles[i].getName().endsWith(CarbonTablePath.INDEX_FILE_EXT)) {
+indexFiles.put(carbonIndexFiles[i].getAbsolutePath(), null);
+  }
+}
+return indexFiles;
+  }
+
+  /**
+   * Read all index file names of the segment
+   *
+   * @return
+   * @throws IOException
+   */
+  public Map getReadCommittedIndexFilesFromPath(String 
path, List segments)
+  throws IOException {
+// Only returns indexes matching the segment.
+Map indexFiles = new HashMap<>();
+for (Segment seg : segments) {
+  CarbonFile[] carbonIndexFiles = getCarbonIndexFiles(path, 
carbonIndexMap);
+  for (int i = 0; i < carbonIndexFiles.length; i++) {
--- End diff --

Duplicate logic across functions


---

[GitHub] carbondata issue #2125: [CARBONDATA-2299]Support showing all segment informa...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2125
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4769/



---

[GitHub] carbondata issue #2133: [CARBONDATA-2304][Compaction] Prefetch rowbatch duri...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2133
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4270/



---

[GitHub] carbondata issue #2125: [CARBONDATA-2299]Support showing all segment informa...

[zzcclp]

Github user zzcclp commented on the issue:

https://github.com/apache/carbondata/pull/2125
  
retest this please


---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4269/



---

[GitHub] carbondata issue #2131: [WIP] Support unmanaged carbon table read and write

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2131
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3540/



---

[GitHub] carbondata issue #2125: [CARBONDATA-2299]Support showing all segment informa...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2125
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4767/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4268/



---

[GitHub] carbondata issue #2131: [WIP] Support unmanaged carbon table read and write

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2131
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4768/



---

[GitHub] carbondata issue #2125: [CARBONDATA-2299]Support showing all segment informa...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2125
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3539/



---

[GitHub] carbondata pull request #2125: [CARBONDATA-2299]Support showing all segment ...

[zzcclp]

Github user zzcclp commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2125#discussion_r178699449
  
--- Diff: 
integration/spark2/src/test/scala/org/apache/spark/util/CarbonCommandSuite.scala
 ---
@@ -190,4 +190,21 @@ class CarbonCommandSuite extends Spark2QueryTest with 
BeforeAndAfterAll {
 dropTable(tableName)
   }
 
+  test("show history segments") {
+val tableName = "test_tablestatus_history"
+sql(s"drop table if exists ${tableName}")
+sql(s"create table ${tableName} (name String, age int) stored by 
'carbondata' "
+  + 
"TBLPROPERTIES('AUTO_LOAD_MERGE'='true','COMPACTION_LEVEL_THRESHOLD'='2,2')")
+val carbonTable = 
CarbonMetadata.getInstance().getCarbonTable("default", tableName)
+sql(s"insert into ${tableName} select 'abc1',1")
+sql(s"insert into ${tableName} select 'abc2',2")
+sql(s"insert into ${tableName} select 'abc3',3")
+assert(sql(s"show segments for table ${tableName}").collect().length 
== 4)
+assert(sql(s"show history segments for table 
${tableName}").collect().length == 4)
+sql(s"clean files for table ${tableName}")
+assert(sql(s"show segments for table ${tableName}").collect().length 
== 2)
+assert(sql(s"show history segments for table 
${tableName}").collect().length == 4)
--- End diff --

Done


---

[GitHub] carbondata issue #2133: [CARBONDATA-2304][Compaction] Prefetch rowbatch duri...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2133
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3537/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4267/



---

[GitHub] carbondata issue #2133: [CARBONDATA-2304][Compaction] Prefetch rowbatch duri...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2133
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4765/



---

[jira] [Created] (CARBONDATA-2309) Add strategy to generate bigger carbondata files in case of small amount of data

[xuchuanyin (JIRA)]

xuchuanyin created CARBONDATA-2309:
--

 Summary: Add strategy to generate bigger carbondata files in case 
of small amount of data
 Key: CARBONDATA-2309
 URL: https://issues.apache.org/jira/browse/CARBONDATA-2309
 Project: CarbonData
  Issue Type: Improvement
  Components: data-load
Reporter: xuchuanyin
Assignee: xuchuanyin


In some scenario, the input amount of loading data is small, but carbondata 
still distribute them to each executors (nodes) to do local-sort, thus 
resulting to small carbondata files generated by each executor. 

In  some extreme conditions, if the cluster is big enough or if the amount of 
data is small enough, the carbondata file contains only one blocklet or page.

I  think a new strategy should be introduced to solve the above problem.

The new strategy should:
 # be able to control the minimum amount of input data for each node
 # ignore data locality otherwise it may always choose a small portion of 
particular nodes



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4266/



---

[GitHub] carbondata pull request #2133: [CARBONDATA-2304][Compaction] Prefetch rowbat...

[xuchuanyin]

GitHub user xuchuanyin opened a pull request:

https://github.com/apache/carbondata/pull/2133

[CARBONDATA-2304][Compaction] Prefetch rowbatch during compaction

Add a configuration to enable prefetch during compaction.

During compaction, carbondata will query on the segments and retrieve a 
row， then it will sort the rows and produce the final carbondata file.

Currently we find the poor performance in retrieving the rows, so adding 
prefetch for the rows will surely improve the compaction performance.

In my local tests, compacting 4 segments each with 100 thousand rows costs 
30s with prefetch and 50s without prefetch.

In my tests in a larger cluster, compacting 6 segments each with 18GB raw 
data costs 45min with prefetch and 57min without prefetch.

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [x] Any interfaces changed?
 `NO`
 - [x] Any backward compatibility impacted?
 `NO`
 - [x] Document update required?
`Add a configuration, will update it later`
 - [x] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
`Yes`
- How it is tested? Please attach test report.
`Tested in local and a 3-node cluster`
- Is it a performance related change? Please attach the performance 
test report.
`Compaction performance has been enhanced by 25+%`
- Any additional information to help reviewers in testing this 
change.
   
 - [x] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
`Not related`


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xuchuanyin/carbondata 0402_compaction_prefetch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2133.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2133


commit 41869effa326052b46088f68dd1d6ccc5f7525e5
Author: xuchuanyin 
Date:   2018-04-02T12:38:17Z

Prefetch rowbatch during compaction

Add a configuration to enable prefetch during compaction.




---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4265/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4264/



---

[GitHub] carbondata issue #2113: [WIP][LUCENE_DATAMAP]load issue in lucene datamap, m...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2113
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4263/



---

[GitHub] carbondata issue #2131: [WIP] Support unmanaged carbon table read and write

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2131
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4262/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4261/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4764/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3536/



---

[GitHub] carbondata issue #2123: [CARBONDATA-2297] Support SEARCH_MODE for basic filt...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2123
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4260/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4763/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3535/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3534/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4762/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4761/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3533/



---

[GitHub] carbondata issue #2128: [CARBONDATA-2303] [WIP] If dataload is failed for pa...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4258/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3532/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4760/



---

[GitHub] carbondata issue #2083: [CARBONDATA-2269]Support Query On PreAggregate table...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2083
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4257/



---

[GitHub] carbondata issue #2113: [WIP][LUCENE_DATAMAP]load issue in lucene datamap, m...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2113
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4758/



---

[GitHub] carbondata pull request #2131: [WIP] Support unmanaged carbon table read and...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2131#discussion_r178594069
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/datamap/TableDataMap.java ---
@@ -79,26 +80,27 @@
*
* @param segments
* @param filterExp
+   * @param readCommitted
* @return
*/
   public List prune(List segments, 
FilterResolverIntf filterExp,
-  List partitions) throws IOException {
+  List partitions, ReadCommitted readCommitted) throws 
IOException {
--- End diff --

ReadCommited is implemented in order to set the read scope. Currently for 
Unmanaged Table the Read Scope is LatestFileRead which is going to take a 
snapshot of the list of Index files during query phase starting i.e. in 
CarbonInputSplit. In later phase of query life cycle even though new files get 
placed, only files read during initial snapshot is only returned. 
Similarly TableStatusReadCommitter is going to take a snapshot of Table 
Status file and will be used by Managed Table. One time snapshot of 
LoadMetadataDetails will be taken in CarbonTableInputFormat and later the Table 
Status snapshot will be only referred to return the list of files. So in 
parallel in case Table Status gets updated then new updated segment and file 
list wont be reflected. This way we defined the read commit scope. 


---

[GitHub] carbondata issue #2127: [CARBONDATA-2301][SDK] CarbonStore interface and two...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4256/



---

[GitHub] carbondata issue #2113: [WIP][LUCENE_DATAMAP]load issue in lucene datamap, m...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2113
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3530/



---

[GitHub] carbondata issue #2126: [CARBONDATA-2300] Add ENABLE_UNSAFE_IN_QUERY_EXECUTI...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2126
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4255/



---

[GitHub] carbondata issue #2131: [WIP] Support unmanaged carbon table read and write

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2131
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3529/



---

[GitHub] carbondata pull request #2131: [WIP] Support unmanaged carbon table read and...

[jackylk]

Github user jackylk commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2131#discussion_r178571705
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/datamap/TableDataMap.java ---
@@ -79,26 +80,27 @@
*
* @param segments
* @param filterExp
+   * @param readCommitted
* @return
*/
   public List prune(List segments, 
FilterResolverIntf filterExp,
-  List partitions) throws IOException {
+  List partitions, ReadCommitted readCommitted) throws 
IOException {
--- End diff --

Can you explain what is ReadCommitted and why is it needed?


---

[jira] [Resolved] (CARBONDATA-2300) Add ENABLE_UNSAFE_IN_QUERY_EXECUTION as a configuration parameter in presto integration

[Jacky Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacky Li resolved CARBONDATA-2300.
--
   Resolution: Fixed
Fix Version/s: 1.4.0

> Add ENABLE_UNSAFE_IN_QUERY_EXECUTION as a configuration parameter in presto 
> integration
> ---
>
> Key: CARBONDATA-2300
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2300
> Project: CarbonData
>  Issue Type: Task
>  Components: presto-integration
>Reporter: Liang Chen
>Assignee: Liang Chen
>Priority: Minor
> Fix For: 1.4.0
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Add ENABLE_UNSAFE_IN_QUERY_EXECUTION as a configuration parameter in presto 
> integration.
> The default is true, provide this configuration parameter for users to 
> disable the unsafe in query execution.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata pull request #2126: [CARBONDATA-2300] Add ENABLE_UNSAFE_IN_QUERY_...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/2126


---

[GitHub] carbondata issue #2126: [CARBONDATA-2300] Add ENABLE_UNSAFE_IN_QUERY_EXECUTI...

[jackylk]

Github user jackylk commented on the issue:

https://github.com/apache/carbondata/pull/2126
  
LGTM


---

[GitHub] carbondata issue #2123: [CARBONDATA-2297] Support SEARCH_MODE for basic filt...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2123
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4755/



---

[GitHub] carbondata issue #2131: [WIP] Support unmanaged carbon table read and write

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2131
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4254/



---

[GitHub] carbondata pull request #2123: [CARBONDATA-2297] Support SEARCH_MODE for bas...

[kevinjmh]

Github user kevinjmh commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2123#discussion_r178568134
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/scan/executor/impl/SearchModeVectorDetailQueryExecutor.java
 ---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.core.scan.executor.impl;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+
+import org.apache.carbondata.common.CarbonIterator;
+import org.apache.carbondata.common.logging.LogService;
+import org.apache.carbondata.common.logging.LogServiceFactory;
+import org.apache.carbondata.core.constants.CarbonCommonConstants;
+import 
org.apache.carbondata.core.scan.executor.exception.QueryExecutionException;
+import org.apache.carbondata.core.scan.executor.infos.BlockExecutionInfo;
+import org.apache.carbondata.core.scan.model.QueryModel;
+import 
org.apache.carbondata.core.scan.result.iterator.SearchModeResultIterator;
+import org.apache.carbondata.core.util.CarbonProperties;
+
+/**
+ * Below class will be used to execute the detail query and returns 
columnar vectors.
+ */
+public class SearchModeVectorDetailQueryExecutor extends 
AbstractQueryExecutor {
+  private static final LogService LOGGER =
+  
LogServiceFactory.getLogService(SearchModeVectorDetailQueryExecutor.class.getName());
+  private static ExecutorService executorService;
+
+  static {
+int nThread;
+try {
+  nThread = Integer.parseInt(CarbonProperties.getInstance()
+  .getProperty(CarbonCommonConstants.CARBON_SEARCH_MODE_THREAD,
+  
CarbonCommonConstants.CARBON_SEARCH_MODE_THREAD_DEFAULT));
+} catch (NumberFormatException e) {
+  nThread = 
Integer.parseInt(CarbonCommonConstants.CARBON_SEARCH_MODE_THREAD_DEFAULT);
+  LOGGER.warn("The carbon.search.mode.thread is invalid. Using the 
default value " + nThread);
+}
+executorService = Executors.newFixedThreadPool(nThread);
--- End diff --

Setting number of threads is a way to limit the parallelism in case of 
dealing huge number of splits in a task. As for static ExecutorService, it can 
avoid to create a new thread pool for every query and keep the thread pool 
standby for next query. 


---

[GitHub] carbondata issue #2127: [CARBONDATA-2301][SDK] CarbonStore interface and two...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4253/



---

[GitHub] carbondata issue #2131: [WIP] Support unmanaged carbon table read and write

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2131
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4757/



---

[GitHub] carbondata issue #2109: [CARBONDATA-2294] Partition preaggregate support

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2109
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4756/



---

[GitHub] carbondata issue #2123: [CARBONDATA-2297] Support SEARCH_MODE for basic filt...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2123
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3527/



---

[GitHub] carbondata issue #2131: [WIP] Support unmanaged carbon table read and write

[ajantha-bhat]

Github user ajantha-bhat commented on the issue:

https://github.com/apache/carbondata/pull/2131
  
retest this please


---

[GitHub] carbondata pull request #2123: [CARBONDATA-2297] Support SEARCH_MODE for bas...

[xuchuanyin]

Github user xuchuanyin commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2123#discussion_r178560655
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/scan/executor/impl/SearchModeVectorDetailQueryExecutor.java
 ---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.core.scan.executor.impl;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+
+import org.apache.carbondata.common.CarbonIterator;
+import org.apache.carbondata.common.logging.LogService;
+import org.apache.carbondata.common.logging.LogServiceFactory;
+import org.apache.carbondata.core.constants.CarbonCommonConstants;
+import 
org.apache.carbondata.core.scan.executor.exception.QueryExecutionException;
+import org.apache.carbondata.core.scan.executor.infos.BlockExecutionInfo;
+import org.apache.carbondata.core.scan.model.QueryModel;
+import 
org.apache.carbondata.core.scan.result.iterator.SearchModeResultIterator;
+import org.apache.carbondata.core.util.CarbonProperties;
+
+/**
+ * Below class will be used to execute the detail query and returns 
columnar vectors.
+ */
+public class SearchModeVectorDetailQueryExecutor extends 
AbstractQueryExecutor {
+  private static final LogService LOGGER =
+  
LogServiceFactory.getLogService(SearchModeVectorDetailQueryExecutor.class.getName());
+  private static ExecutorService executorService;
+
+  static {
+int nThread;
+try {
+  nThread = Integer.parseInt(CarbonProperties.getInstance()
+  .getProperty(CarbonCommonConstants.CARBON_SEARCH_MODE_THREAD,
+  
CarbonCommonConstants.CARBON_SEARCH_MODE_THREAD_DEFAULT));
+} catch (NumberFormatException e) {
+  nThread = 
Integer.parseInt(CarbonCommonConstants.CARBON_SEARCH_MODE_THREAD_DEFAULT);
+  LOGGER.warn("The carbon.search.mode.thread is invalid. Using the 
default value " + nThread);
+}
+executorService = Executors.newFixedThreadPool(nThread);
--- End diff --

Why should we specify the #threads ?

Besides, since executorService is static, all SearchModeResultIterator will 
use the same pool. Is it intended？


---

[GitHub] carbondata issue #2056: [CARBONDATA-2238][DataLoad] Merge and spill in-memor...

[xuchuanyin]

Github user xuchuanyin commented on the issue:

https://github.com/apache/carbondata/pull/2056
  
@jackylk can you review and merge this PR and #2052 ?


---

[GitHub] carbondata issue #2132: [CARBONDATA-2308] Support concurrent loading and com...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2132
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4252/



---

[GitHub] carbondata issue #2128: [CARBONDATA-2303] [WIP] If dataload is failed for pa...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4752/



---

[GitHub] carbondata issue #2126: [CARBONDATA-2300] Add ENABLE_UNSAFE_IN_QUERY_EXECUTI...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2126
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3526/



---

[GitHub] carbondata issue #2126: [CARBONDATA-2300] Add ENABLE_UNSAFE_IN_QUERY_EXECUTI...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2126
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4753/



---

[GitHub] carbondata issue #2128: [CARBONDATA-2303] [WIP] If dataload is failed for pa...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3525/



---

[GitHub] carbondata issue #2056: [CARBONDATA-2238][DataLoad] Merge and spill in-memor...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2056
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4251/



---

[GitHub] carbondata issue #2126: [CARBONDATA-2300] Add ENABLE_UNSAFE_IN_QUERY_EXECUTI...

[chenliang613]

Github user chenliang613 commented on the issue:

https://github.com/apache/carbondata/pull/2126
  
retest this please


---

[GitHub] carbondata issue #2083: [CARBONDATA-2269]Support Query On PreAggregate table...

[kumarvishal09]

Github user kumarvishal09 commented on the issue:

https://github.com/apache/carbondata/pull/2083
  
retest sdv please


---

[GitHub] carbondata issue #2126: [CARBONDATA-2300] Add ENABLE_UNSAFE_IN_QUERY_EXECUTI...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2126
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4749/



---

[GitHub] carbondata issue #2127: [CARBONDATA-2301][SDK] CarbonStore interface and two...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4751/



---

[GitHub] carbondata issue #2127: [CARBONDATA-2301][SDK] CarbonStore interface and two...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3524/



---

[GitHub] carbondata issue #2131: [WIP] Support unmanaged carbon table read and write

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2131
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4250/



---

[GitHub] carbondata pull request #2129: [CARBONDATA-2298][BACKPORT-1.3]Delete segment...

[zzcclp]

Github user zzcclp closed the pull request at:

https://github.com/apache/carbondata/pull/2129


---

[GitHub] carbondata issue #2014: [CARBONDATA-2198] Fixed bug for streaming data for b...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2014
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4249/



---

[GitHub] carbondata pull request #2124: [CARBONDATA-2298]Delete segment lock files be...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/2124


---

[GitHub] carbondata issue #2124: [CARBONDATA-2298]Delete segment lock files before up...

[jackylk]

Github user jackylk commented on the issue:

https://github.com/apache/carbondata/pull/2124
  
LGTM


---

[jira] [Resolved] (CARBONDATA-2298) Delete segment lock files before update metadata

[Jacky Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacky Li resolved CARBONDATA-2298.
--
Resolution: Fixed

> Delete segment lock files before update metadata
> 
>
> Key: CARBONDATA-2298
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2298
> Project: CarbonData
>  Issue Type: Improvement
>Affects Versions: 1.4.0, 1.3.2
>Reporter: Zhichao  Zhang
>Assignee: Zhichao  Zhang
>Priority: Minor
> Fix For: 1.4.0, 1.3.2
>
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> If there are some COMPACTED segments and their last modified time is within 
> one hour, the segment lock files deletion operation will not be executed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata issue #2129: [CARBONDATA-2298][BACKPORT-1.3]Delete segment lock f...

[jackylk]

Github user jackylk commented on the issue:

https://github.com/apache/carbondata/pull/2129
  
LGTM


---

[GitHub] carbondata issue #2127: [CARBONDATA-2301][SDK] CarbonStore interface and two...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3523/



---

[jira] [Resolved] (CARBONDATA-2302) Fix some bugs when separate visible and invisible segments info into two files

[Jacky Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacky Li resolved CARBONDATA-2302.
--
Resolution: Fixed

> Fix some bugs when separate visible and invisible segments info into two files
> --
>
> Key: CARBONDATA-2302
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2302
> Project: CarbonData
>  Issue Type: Bug
>  Components: core, data-load
>Affects Versions: 1.4.0
>Reporter: Zhichao  Zhang
>Assignee: Zhichao  Zhang
>Priority: Major
> Fix For: 1.4.0
>
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> There are some bugs when separate visible and invisible segments info into 
> two files:
>  # It will not delete physical data of history segments after separating
>  # Generate duplicated segment id.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata issue #2127: [CARBONDATA-2301][SDK] CarbonStore interface and two...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4750/



---

[GitHub] carbondata pull request #2130: [CARBONDATA-2302]Fix some bugs when separate ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/2130


---

[GitHub] carbondata issue #2126: [CARBONDATA-2300] Add ENABLE_UNSAFE_IN_QUERY_EXECUTI...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2126
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3522/



---

[GitHub] carbondata issue #2127: [CARBONDATA-2301][SDK] CarbonStore interface and two...

[jackylk]

Github user jackylk commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
retest this please


---

[GitHub] carbondata issue #2131: [WIP] Support unmanaged carbon table read and write

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2131
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4248/



---

[GitHub] carbondata issue #1942: [CARBONDATA-2136] Fixed bug related to data load for...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1942
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4247/



---

[GitHub] carbondata pull request #2126: [CARBONDATA-2300] Add ENABLE_UNSAFE_IN_QUERY_...

[chenliang613]

Github user chenliang613 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2126#discussion_r178530872
  
--- Diff: 
integration/presto/src/main/java/org/apache/carbondata/presto/impl/CarbonTableReader.java
 ---
@@ -372,6 +372,11 @@ public TBase create() {
   
CarbonProperties.getInstance().addProperty(CarbonCommonConstants.UNSAFE_WORKING_MEMORY_IN_MB,
   config.getUnsafeMemoryInMb());
 }
+if(config.getEnableUnsafeInQueryExecution() != null) {
+  CarbonProperties.getInstance()
--- End diff --

done


---

[GitHub] carbondata issue #1942: [CARBONDATA-2136] Fixed bug related to data load for...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1942
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4246/



---

[GitHub] carbondata issue #2131: [WIP] Support unmanaged carbon table read and write

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2131
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3521/



---

[GitHub] carbondata issue #2132: [CARBONDATA-2308] Support concurrent loading and com...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2132
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4745/



---

[GitHub] carbondata issue #2131: [WIP] Support unmanaged carbon table read and write

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2131
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4748/



---

[GitHub] carbondata issue #1942: [CARBONDATA-2136] Fixed bug related to data load for...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1942
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3519/



---

[GitHub] carbondata issue #2127: [CARBONDATA-2301][SDK] CarbonStore interface and two...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4747/



---

[GitHub] carbondata issue #2127: [CARBONDATA-2301][SDK] CarbonStore interface and two...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3520/



---

[GitHub] carbondata issue #2128: [CARBONDATA-2303] [WIP] If dataload is failed for pa...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4245/



---

[GitHub] carbondata issue #1942: [CARBONDATA-2136] Fixed bug related to data load for...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1942
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4746/



---

[GitHub] carbondata issue #2132: [CARBONDATA-2308] Support concurrent loading and com...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2132
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3518/



---