[GitHub] carbondata issue #2017: [CARBONDATA-2217]fix drop partition for non existing...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2017
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2759/



---

[GitHub] carbondata issue #2017: [CARBONDATA-2217]fix drop partition for non existing...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2017
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4003/



---

[GitHub] carbondata issue #2018: [WIP] Added validation for external partition locati...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2018
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2758/



---

[GitHub] carbondata issue #2018: [WIP] Added validation for external partition locati...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2018
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4002/



---

[GitHub] carbondata pull request #2018: [WIP] Added validation for external partition...

[ravipesala]

GitHub user ravipesala opened a pull request:

https://github.com/apache/carbondata/pull/2018

[WIP] Added validation for external partition location to use same schema.

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ravipesala/incubator-carbondata 
partition-external-location

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2018.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2018


commit fe48c162db006d8e3c67d062d070051a3fd03886
Author: ravipesala 
Date:   2018-03-01T06:34:53Z

Added validation for external partition location to use same schema.




---

[GitHub] carbondata issue #2017: [CARBONDATA-2217]fix drop partition for non existing...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2017
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2757/



---

[GitHub] carbondata issue #2017: [CARBONDATA-2217]fix drop partition for non existing...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2017
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4001/



---

[GitHub] carbondata pull request #2017: [CARBONDATA-2217]fix drop partition for non e...

[akashrn5]

GitHub user akashrn5 opened a pull request:

https://github.com/apache/carbondata/pull/2017

[CARBONDATA-2217]fix drop partition for non existing partition and set 
FactTimeStamp during compaction for partition table

**Problem:**
1)when drop partition is fired for a column which does not exists , it 
throws null pointer exception
2)select * is not working when clean files operation is fired after second 
level of compaction, it throws exception sometimes

**Solution:**
1)have a null check , if column does not exists
2)give different timestamp for fact files during compaction to avoid 
deletion of files during clean files
3) This PR also contains fix for creating a pre aggregate table with same 
name which has already created in other database

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [x] Any interfaces changed?
 NA
 - [x] Any backward compatibility impacted?
 NA
 - [x] Document update required?
NA
 - [x] Testing done
Added UTs
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [x] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
NA


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/akashrn5/incubator-carbondata partition_global

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2017.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2017


commit 3b66b06a3ed733ae938271361ff9f1af81fbf7a7
Author: akashrn5 
Date:   2018-02-28T11:58:43Z

fix drop partition for non existing partition and set FactTimeStamp during 
compaction for partition table




---

[jira] [Resolved] (CARBONDATA-2215) Add the description of Carbon Stream Parser into streaming-guide.md

[QiangCai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

QiangCai resolved CARBONDATA-2215.
--
Resolution: Fixed

> Add the description of Carbon Stream Parser into streaming-guide.md
> ---
>
> Key: CARBONDATA-2215
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2215
> Project: CarbonData
>  Issue Type: Task
>  Components: docs
>Reporter: Zhichao  Zhang
>Assignee: Zhichao  Zhang
>Priority: Trivial
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Add the description of Carbon Stream Parser into streaming-guide.md



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata pull request #2016: [CARBONDATA-2215][Docs]Add the description of...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/2016


---

[GitHub] carbondata issue #2016: [CARBONDATA-2215][Docs]Add the description of Carbon...

[QiangCai]

Github user QiangCai commented on the issue:

https://github.com/apache/carbondata/pull/2016
  
LGTM


---

[GitHub] carbondata pull request #1825: [CARBONDATA-2032][DataLoad] directly write ca...

[xuchuanyin]

Github user xuchuanyin commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1825#discussion_r171470463
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/store/writer/AbstractFactDataWriter.java
 ---
@@ -386,29 +380,42 @@ protected void writeIndexFile() throws IOException, 
CarbonDataWriterException {
 .getIndexHeader(localCardinality, thriftColumnSchemaList, 
model.getBucketId());
 // get the block index info thrift
 List blockIndexThrift = 
CarbonMetadataUtil.getBlockIndexInfo(blockIndexInfoList);
-// randomly choose a temp location for index file
-String[] tempLocations = model.getStoreLocation();
-String chosenTempLocation = tempLocations[new 
Random().nextInt(tempLocations.length)];
-LOGGER.info("Randomly choose index file location: " + 
chosenTempLocation);
+String indexFileName;
+if (enableDirectlyWriteData2Hdfs) {
+  String rawFileName = model.getCarbonDataDirectoryPath() + 
File.separator + CarbonTablePath
+  
.getCarbonIndexFileName(model.getCarbonDataFileAttributes().getTaskId(),
+  model.getBucketId(), model.getTaskExtension(),
+  "" + model.getCarbonDataFileAttributes().getFactTimeStamp());
+  indexFileName = FileFactory.getUpdatedFilePath(rawFileName, 
FileFactory.FileType.HDFS);
+} else {
+  // randomly choose a temp location for index file
+  String[] tempLocations = model.getStoreLocation();
+  String chosenTempLocation = tempLocations[new 
Random().nextInt(tempLocations.length)];
+  LOGGER.info("Randomly choose index file location: " + 
chosenTempLocation);
+  indexFileName = chosenTempLocation + File.separator + CarbonTablePath
+  
.getCarbonIndexFileName(model.getCarbonDataFileAttributes().getTaskId(),
+  model.getBucketId(), model.getTaskExtension(),
+  "" + model.getCarbonDataFileAttributes().getFactTimeStamp());
+}
 
-String fileName = chosenTempLocation + File.separator + CarbonTablePath
-
.getCarbonIndexFileName(model.getCarbonDataFileAttributes().getTaskId(),
-model.getBucketId(), model.getTaskExtension(),
-"" + model.getCarbonDataFileAttributes().getFactTimeStamp());
 CarbonIndexFileWriter writer = new CarbonIndexFileWriter();
 // open file
-writer.openThriftWriter(fileName);
+writer.openThriftWriter(indexFileName);
 // write the header first
 writer.writeThrift(indexHeader);
 // write the indexes
 for (BlockIndex blockIndex : blockIndexThrift) {
   writer.writeThrift(blockIndex);
 }
 writer.close();
-// copy from temp to actual store location
-CarbonUtil.copyCarbonDataFileToCarbonStorePath(fileName,
-model.getCarbonDataDirectoryPath(),
-fileSizeInBytes);
+if (enableDirectlyWriteData2Hdfs) {
+  executorServiceSubmitList.add(executorService.submit(
+  new CopyThread(indexFileName, FileFactory.FileType.HDFS)));
--- End diff --

use ‘completeHdfsBackendThread’ ~


---

[GitHub] carbondata issue #2008: [CARBONDATA-2211] in case of DDL HndOff should not b...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2008
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2756/



---

[GitHub] carbondata issue #2008: [CARBONDATA-2211] in case of DDL HndOff should not b...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2008
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4000/



---

[GitHub] carbondata pull request #2008: [CARBONDATA-2211] in case of DDL HndOff shoul...

[rahulforallp]

Github user rahulforallp commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2008#discussion_r171465697
  
--- Diff: 
streaming/src/main/scala/org/apache/carbondata/streaming/StreamHandoffRDD.scala 
---
@@ -277,15 +277,21 @@ object StreamHandoffRDD {
*/
   def startStreamingHandoffThread(
   carbonLoadModel: CarbonLoadModel,
-  sparkSession: SparkSession
+  sparkSession: SparkSession,
+  isDDL: Boolean
   ): Unit = {
 // start a new thread to execute streaming segment handoff
 val handoffThread = new Thread() {
--- End diff --

Done


---

[jira] [Created] (CARBONDATA-2217) nullpointer issue drop partition where column does not exists and clean files issue after second level of compaction

[Akash R Nilugal (JIRA)]

Akash R Nilugal created CARBONDATA-2217:
---

 Summary: nullpointer issue  drop partition where column does not 
exists and clean files issue after second level of compaction
 Key: CARBONDATA-2217
 URL: https://issues.apache.org/jira/browse/CARBONDATA-2217
 Project: CarbonData
  Issue Type: Bug
  Components: core, spark-integration
Reporter: Akash R Nilugal
Assignee: Akash R Nilugal


1)when drop partition is fired for a column which does not exists , it throws 
null pointer exception

2)select * is not working when clean files operation is fired after second 
level of compaction

create table comp_dt2(id int,name string) partitioned by (dt date,c4 int) 
stored by 'carbondata';
insert into comp_dt2 select 1,'A','2001-01-01',1;
insert into comp_dt2 select 2,'B','2001-01-01',1;
insert into comp_dt2 select 3,'C','2002-01-01',2;
insert into comp_dt2 select 4,'D','2002-01-01',null;

insert into comp_dt2 select 5,'E','2003-01-01',3;
insert into comp_dt2 select 6,'F','2003-01-01',3;
insert into comp_dt2 select 7,'G','2003-01-01',4;
insert into comp_dt2 select 8,'H','2004-01-01','';

insert into comp_dt2 select 9,'H','2001-01-01',1;
insert into comp_dt2 select 10,'I','2002-01-01',null;
insert into comp_dt2 select 11,'J','2003-01-01',4;
insert into comp_dt2 select 12,'K','2003-01-01',5;

 

clean files for table comp_dt2;

select * from comp_dt2

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata issue #2006: [CARBONDATA-2209] Fixed rename table with partitions...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2006
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3999/



---

[GitHub] carbondata pull request #2014: [CARBONDATA-2198] Fixed bug for streaming dat...

[geetikagupta16]

Github user geetikagupta16 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2014#discussion_r171464754
  
--- Diff: 
hadoop/src/main/java/org/apache/carbondata/hadoop/streaming/CarbonStreamRecordWriter.java
 ---
@@ -181,75 +181,81 @@ private void initializeAtFirstRow() throws 
IOException, InterruptedException {
   initializeAtFirstRow();
 }
 
-// parse and convert row
-currentRow.setData(rowParser.parseRow((Object[]) value));
-converter.convert(currentRow);
-
 // null bit set
 nullBitSet.clear();
-for (int i = 0; i < dataFields.length; i++) {
-  if (null == currentRow.getObject(i)) {
-nullBitSet.set(i);
+Object[] rowData = (Object[]) value;
+currentRow.setRawData(rowData);
+// parse and convert row
+currentRow.setData(rowParser.parseRow(rowData));
+CarbonRow updatedCarbonRow = converter.convert(currentRow);
+if (updatedCarbonRow == null) {
+  output.skipRow();
+  currentRow.clearData();
+} else {
--- End diff --

@mohammadshahidkhan  Yes, else part remains the same


---

[GitHub] carbondata issue #1942: [CARBONDATA-2136] Fixed bug related to data load for...

[mohammadshahidkhan]

Github user mohammadshahidkhan commented on the issue:

https://github.com/apache/carbondata/pull/1942
  
+1 The streaming issue is handled in PR #2014 


---

[GitHub] carbondata issue #1825: [CARBONDATA-2032][DataLoad] directly write carbon da...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1825
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2754/



---

[GitHub] carbondata issue #2016: [CARBONDATA-2215][Docs]Add the description of Carbon...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2016
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2753/



---

[GitHub] carbondata issue #2015: [CARBONDATA-2103]Make show datamaps configurable in ...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2015
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3745/



---

[GitHub] carbondata issue #1956: [HOTFIX] Add partition usage code

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1956
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2752/



---

[GitHub] carbondata issue #1956: [HOTFIX] Add partition usage code

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1956
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3996/



---

[GitHub] carbondata issue #1867: [CARBONDATA-2055][Streaming]Support integrating Stre...

[zzcclp]

Github user zzcclp commented on the issue:

https://github.com/apache/carbondata/pull/1867
  
@chenliang613 @jackylk @QiangCai  please help to review, thanks.


---

[GitHub] carbondata issue #2016: [CARBONDATA-2215][Docs]Add the description of Carbon...

[zzcclp]

Github user zzcclp commented on the issue:

https://github.com/apache/carbondata/pull/2016
  
@QiangCai please help to review, thanks.


---

[GitHub] carbondata issue #1825: [CARBONDATA-2032][DataLoad] directly write carbon da...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1825
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2751/



---

[GitHub] carbondata issue #1825: [CARBONDATA-2032][DataLoad] directly write carbon da...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1825
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3995/



---

[GitHub] carbondata pull request #1825: [CARBONDATA-2032][DataLoad] directly write ca...

[xuchuanyin]

Github user xuchuanyin commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1825#discussion_r171456256
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/store/writer/AbstractFactDataWriter.java
 ---
@@ -386,29 +380,42 @@ protected void writeIndexFile() throws IOException, 
CarbonDataWriterException {
 .getIndexHeader(localCardinality, thriftColumnSchemaList, 
model.getBucketId());
 // get the block index info thrift
 List blockIndexThrift = 
CarbonMetadataUtil.getBlockIndexInfo(blockIndexInfoList);
-// randomly choose a temp location for index file
-String[] tempLocations = model.getStoreLocation();
-String chosenTempLocation = tempLocations[new 
Random().nextInt(tempLocations.length)];
-LOGGER.info("Randomly choose index file location: " + 
chosenTempLocation);
+String indexFileName;
+if (enableDirectlyWriteData2Hdfs) {
+  String rawFileName = model.getCarbonDataDirectoryPath() + 
File.separator + CarbonTablePath
+  
.getCarbonIndexFileName(model.getCarbonDataFileAttributes().getTaskId(),
+  model.getBucketId(), model.getTaskExtension(),
+  "" + model.getCarbonDataFileAttributes().getFactTimeStamp());
+  indexFileName = FileFactory.getUpdatedFilePath(rawFileName, 
FileFactory.FileType.HDFS);
+} else {
+  // randomly choose a temp location for index file
+  String[] tempLocations = model.getStoreLocation();
+  String chosenTempLocation = tempLocations[new 
Random().nextInt(tempLocations.length)];
+  LOGGER.info("Randomly choose index file location: " + 
chosenTempLocation);
+  indexFileName = chosenTempLocation + File.separator + CarbonTablePath
+  
.getCarbonIndexFileName(model.getCarbonDataFileAttributes().getTaskId(),
+  model.getBucketId(), model.getTaskExtension(),
+  "" + model.getCarbonDataFileAttributes().getFactTimeStamp());
+}
 
-String fileName = chosenTempLocation + File.separator + CarbonTablePath
-
.getCarbonIndexFileName(model.getCarbonDataFileAttributes().getTaskId(),
-model.getBucketId(), model.getTaskExtension(),
-"" + model.getCarbonDataFileAttributes().getFactTimeStamp());
 CarbonIndexFileWriter writer = new CarbonIndexFileWriter();
 // open file
-writer.openThriftWriter(fileName);
+writer.openThriftWriter(indexFileName);
 // write the header first
 writer.writeThrift(indexHeader);
 // write the indexes
 for (BlockIndex blockIndex : blockIndexThrift) {
   writer.writeThrift(blockIndex);
 }
 writer.close();
-// copy from temp to actual store location
-CarbonUtil.copyCarbonDataFileToCarbonStorePath(fileName,
-model.getCarbonDataDirectoryPath(),
-fileSizeInBytes);
+if (enableDirectlyWriteData2Hdfs) {
+  executorServiceSubmitList.add(executorService.submit(
+  new CopyThread(indexFileName, FileFactory.FileType.HDFS)));
--- End diff --

Sometimes it's copy，sometimes it's setReplication.

How about using the name ‘CompleteHdfsBackupsThread’?
Or we can still use the name ‘CopyThread’ and add some comments to 
explain since we may remove ‘write temp fact to local’ in the further as we 
discussed before.  At that that we can give it a proper name.


---

[GitHub] carbondata pull request #1825: [CARBONDATA-2032][DataLoad] directly write ca...

[jackylk]

Github user jackylk commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1825#discussion_r171454525
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/store/writer/AbstractFactDataWriter.java
 ---
@@ -386,29 +380,42 @@ protected void writeIndexFile() throws IOException, 
CarbonDataWriterException {
 .getIndexHeader(localCardinality, thriftColumnSchemaList, 
model.getBucketId());
 // get the block index info thrift
 List blockIndexThrift = 
CarbonMetadataUtil.getBlockIndexInfo(blockIndexInfoList);
-// randomly choose a temp location for index file
-String[] tempLocations = model.getStoreLocation();
-String chosenTempLocation = tempLocations[new 
Random().nextInt(tempLocations.length)];
-LOGGER.info("Randomly choose index file location: " + 
chosenTempLocation);
+String indexFileName;
+if (enableDirectlyWriteData2Hdfs) {
+  String rawFileName = model.getCarbonDataDirectoryPath() + 
File.separator + CarbonTablePath
+  
.getCarbonIndexFileName(model.getCarbonDataFileAttributes().getTaskId(),
+  model.getBucketId(), model.getTaskExtension(),
+  "" + model.getCarbonDataFileAttributes().getFactTimeStamp());
+  indexFileName = FileFactory.getUpdatedFilePath(rawFileName, 
FileFactory.FileType.HDFS);
+} else {
+  // randomly choose a temp location for index file
+  String[] tempLocations = model.getStoreLocation();
+  String chosenTempLocation = tempLocations[new 
Random().nextInt(tempLocations.length)];
+  LOGGER.info("Randomly choose index file location: " + 
chosenTempLocation);
+  indexFileName = chosenTempLocation + File.separator + CarbonTablePath
+  
.getCarbonIndexFileName(model.getCarbonDataFileAttributes().getTaskId(),
+  model.getBucketId(), model.getTaskExtension(),
+  "" + model.getCarbonDataFileAttributes().getFactTimeStamp());
+}
 
-String fileName = chosenTempLocation + File.separator + CarbonTablePath
-
.getCarbonIndexFileName(model.getCarbonDataFileAttributes().getTaskId(),
-model.getBucketId(), model.getTaskExtension(),
-"" + model.getCarbonDataFileAttributes().getFactTimeStamp());
 CarbonIndexFileWriter writer = new CarbonIndexFileWriter();
 // open file
-writer.openThriftWriter(fileName);
+writer.openThriftWriter(indexFileName);
 // write the header first
 writer.writeThrift(indexHeader);
 // write the indexes
 for (BlockIndex blockIndex : blockIndexThrift) {
   writer.writeThrift(blockIndex);
 }
 writer.close();
-// copy from temp to actual store location
-CarbonUtil.copyCarbonDataFileToCarbonStorePath(fileName,
-model.getCarbonDataDirectoryPath(),
-fileSizeInBytes);
+if (enableDirectlyWriteData2Hdfs) {
+  executorServiceSubmitList.add(executorService.submit(
+  new CopyThread(indexFileName, FileFactory.FileType.HDFS)));
--- End diff --

The name of CopyThread is confusing, is it copy or rename?


---

[GitHub] carbondata pull request #1825: [CARBONDATA-2032][DataLoad] directly write ca...

[jackylk]

Github user jackylk commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1825#discussion_r171454147
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/store/writer/AbstractFactDataWriter.java
 ---
@@ -239,24 +260,30 @@ private void notifyDataMapBlockEnd() {
 blockletId = 0;
   }
 
-  private String constructFactFileFullPath() {
-String factFilePath =
-this.model.getCarbonDataDirectoryPath() + File.separator + 
this.carbonDataFileName;
-return factFilePath;
-  }
   /**
* Finish writing current file. It will flush stream, copy and rename 
temp file to final file
* @param copyInCurrentThread set to false if want to do data copy in a 
new thread
*/
   protected void commitCurrentFile(boolean copyInCurrentThread) {
 notifyDataMapBlockEnd();
 CarbonUtil.closeStreams(this.fileOutputStream, this.fileChannel);
-if (copyInCurrentThread) {
-  CarbonUtil.copyCarbonDataFileToCarbonStorePath(
-  carbonDataFileTempPath, model.getCarbonDataDirectoryPath(),
-  fileSizeInBytes);
+if (enableDirectlyWriteData2Hdfs) {
+  if (copyInCurrentThread) {
+CarbonUtil.completeRemainingHdfsReplicas(carbonDataFileHdfsPath,
+FileFactory.FileType.HDFS);
+  } else {
+executorServiceSubmitList.add(executorService.submit(
+new CopyThread(carbonDataFileHdfsPath, 
FileFactory.FileType.HDFS)));
--- End diff --

Copy again? or just rename?


---

[GitHub] carbondata issue #1825: [CARBONDATA-2032][DataLoad] directly write carbon da...

[jackylk]

Github user jackylk commented on the issue:

https://github.com/apache/carbondata/pull/1825
  
retest this please


---

[GitHub] carbondata issue #1988: [CARBONDATA-2193] Support register analyzer and opti...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1988
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3744/



---

[GitHub] carbondata issue #2009: [CARBONDATA-2212] event fired while updating the sta...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2009
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2750/



---

[GitHub] carbondata issue #2009: [CARBONDATA-2212] event fired while updating the sta...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2009
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3994/



---

[GitHub] carbondata pull request #2008: [CARBONDATA-2211] in case of DDL HndOff shoul...

[QiangCai]

Github user QiangCai commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2008#discussion_r171448904
  
--- Diff: 
streaming/src/main/scala/org/apache/carbondata/streaming/StreamHandoffRDD.scala 
---
@@ -277,15 +277,21 @@ object StreamHandoffRDD {
*/
   def startStreamingHandoffThread(
   carbonLoadModel: CarbonLoadModel,
-  sparkSession: SparkSession
+  sparkSession: SparkSession,
+  isDDL: Boolean
   ): Unit = {
 // start a new thread to execute streaming segment handoff
 val handoffThread = new Thread() {
--- End diff --

move this code to the inside of the if statement


---

[GitHub] carbondata issue #2009: [CARBONDATA-2212] event fired while updating the sta...

[QiangCai]

Github user QiangCai commented on the issue:

https://github.com/apache/carbondata/pull/2009
  
retest this please


---

[GitHub] carbondata issue #2012: [CARBONDATA-2216][Test] Fix bugs in sdv tests

[xuchuanyin]

Github user xuchuanyin commented on the issue:

https://github.com/apache/carbondata/pull/2012
  
@ravipesala The sdvtest complains that 
```
Error Message
2 did not equal 1
Stacktrace
  org.scalatest.exceptions.TestFailedException: 2 did not equal 1
  at 
org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:500)
  at 
org.scalatest.FunSuite.newAssertionFailedException(FunSuite.scala:1555)
  at 
org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:466)
  at 
org.apache.carbondata.cluster.sdv.generated.MergeIndexTestCase$$anonfun$3.apply$mcV$sp(MergeIndexTestCase.scala:97)
```
But actually I cloned the `sdv-test_data` branch in your repo and tested it 
in local, the dataloading only generated 1 *.index file not 2.

Is there any different from the CI env and Local env?


---

[GitHub] carbondata pull request #2012: [CARBONDATA-2216][Test] Fix bugs in sdv tests

[xuchuanyin]

Github user xuchuanyin closed the pull request at:

https://github.com/apache/carbondata/pull/2012


---

[GitHub] carbondata issue #2015: [CARBONDATA-2103]Make show datamaps configurable in ...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2015
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3743/



---

[GitHub] carbondata issue #2015: [CARBONDATA-2103]Make show datamaps configurable in ...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2015
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3742/



---

[GitHub] carbondata issue #1867: [CARBONDATA-2055][Streaming]Support integrating Stre...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1867
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3741/



---

[GitHub] carbondata issue #2015: [CARBONDATA-2103]Make show datamaps configurable in ...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2015
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3740/



---

[GitHub] carbondata issue #2006: [CARBONDATA-2209] Fixed rename table with partitions...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2006
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3739/



---

[GitHub] carbondata issue #1995: [WIP] File Format Reader

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1995
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3738/



---

[GitHub] carbondata issue #2014: [CARBONDATA-2198] Fixed bug for streaming data for b...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2014
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3737/



---

[GitHub] carbondata issue #1995: [WIP] File Format Reader

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1995
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3736/



---

[GitHub] carbondata issue #2013: [CARBONDATA-2214][Docs]Remove config 'spark.sql.hive...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2013
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3735/



---

[GitHub] carbondata issue #2012: [CARBONDATA-2216][Test] Fix bugs in sdv tests

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2012
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3734/



---

[GitHub] carbondata issue #2016: [CARBONDATA-2215][Docs]Add the description of Carbon...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2016
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3993/



---

[GitHub] carbondata issue #2016: [CARBONDATA-2215][Docs]Add the description of Carbon...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2016
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2749/



---

[GitHub] carbondata issue #2005: [CARBONDATA-2207] Fix testcases after using hive met...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2005
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3733/



---

[GitHub] carbondata issue #1981: [Pre-Agg Test] Added SDV TestCase of preaggregate

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1981
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3732/



---

[GitHub] carbondata issue #2016: [CARBONDATA-2215][Docs]Add the description of Carbon...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2016
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2746/



---

[GitHub] carbondata issue #2016: [CARBONDATA-2215][Docs]Add the description of Carbon...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2016
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3990/



---

[GitHub] carbondata issue #1988: [CARBONDATA-2193] Support register analyzer and opti...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1988
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2745/



---

[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1990
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3731/



---

[GitHub] carbondata pull request #2016: [CARBONDATA-2215][Docs]Add the description of...

[zzcclp]

GitHub user zzcclp opened a pull request:

https://github.com/apache/carbondata/pull/2016

[CARBONDATA-2215][Docs]Add the description of Carbon Stream Parser into 
streaming-guide.md

Add the description of Carbon Stream Parser into streaming-guide.md

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed? No
 
 - [ ] Any backward compatibility impacted? No
 
 - [ ] Document update required? 

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zzcclp/carbondata CARBONDATA-2215

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2016.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2016


commit 449cade5954163a3578435fa624c9c76f8fe91b0
Author: Zhang Zhichao <441586683@...>
Date:   2018-02-28T15:07:38Z

[CARBONDATA-2215][Docs]Add the description of Carbon Stream Parser into 
streaming-guide.md

Add the description of Carbon Stream Parser into streaming-guide.md




---

[GitHub] carbondata issue #2015: [CARBONDATA-2103]Make show datamaps configurable in ...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2015
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2744/



---

[GitHub] carbondata issue #2015: [CARBONDATA-2103]Make show datamaps configurable in ...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2015
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3988/



---

[GitHub] carbondata issue #1988: [CARBONDATA-2193] Support register analyzer and opti...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1988
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3989/



---

[GitHub] carbondata issue #2015: [CARBONDATA-2103]Make show datamaps configurable in ...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2015
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3986/



---

[jira] [Resolved] (CARBONDATA-2216) Error in compilation and execution in sdvtest

[Jacky Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacky Li resolved CARBONDATA-2216.
--
   Resolution: Fixed
Fix Version/s: 1.4.0

> Error in compilation and execution in sdvtest
> -
>
> Key: CARBONDATA-2216
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2216
> Project: CarbonData
>  Issue Type: Bug
>  Components: test
>Reporter: xuchuanyin
>Assignee: xuchuanyin
>Priority: Major
> Fix For: 1.4.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> # Missing imports in code files results in compilation error
>  # Error in tests results in test execution error



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata issue #1988: [CARBONDATA-2193] Support register analyzer and opti...

[jackylk]

Github user jackylk commented on the issue:

https://github.com/apache/carbondata/pull/1988
  
retest sdv please


---

[GitHub] carbondata issue #2015: [CARBONDATA-2103]Make show datamaps configurable in ...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2015
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2742/



---

[GitHub] carbondata issue #1988: [CARBONDATA-2193] Support register analyzer and opti...

[jackylk]

Github user jackylk commented on the issue:

https://github.com/apache/carbondata/pull/1988
  
retest this please


---

[GitHub] carbondata issue #2012: [CARBONDATA-2216][Test] Fix bugs in sdv tests

[jackylk]

Github user jackylk commented on the issue:

https://github.com/apache/carbondata/pull/2012
  
merged into datamap branch


---

[GitHub] carbondata issue #2012: [CARBONDATA-2216][Test] Fix bugs in sdv tests

[jackylk]

Github user jackylk commented on the issue:

https://github.com/apache/carbondata/pull/2012
  
LGTM


---

[GitHub] carbondata issue #1867: [CARBONDATA-2055][Streaming]Support integrating Stre...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1867
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3985/



---

[GitHub] carbondata issue #1867: [CARBONDATA-2055][Streaming]Support integrating Stre...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1867
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2740/



---

[GitHub] carbondata issue #1949: [HOTFIX][CARBONDATA2144] Optimize preaggregate table...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1949
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3730/



---

[GitHub] carbondata issue #2015: [CARBONDATA-2103]Make show datamaps configurable in ...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2015
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2741/



---

[GitHub] carbondata pull request #2015: [CARBONDATA-2103]Make show datamaps configura...

[akashrn5]

Github user akashrn5 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2015#discussion_r171246980
  
--- Diff: 
integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggCreateCommand.scala
 ---
@@ -377,11 +379,15 @@ class TestPreAggCreateCommand extends QueryTest with 
BeforeAndAfterAll {
 sql("DROP DATAMAP IF EXISTS agg0 ON TABLE maintable")
   }
 
-  test("test show tables filterted with datamaps"){
+  test("test show tables filterted with datamaps") {
+
CarbonProperties.getInstance().addProperty(CarbonCommonConstants.CARBON_SHOW_DATAMAPS,"false")
 sql("create table showTables(name string, age int) stored by 
'carbondata'")
-sql("create datamap preAgg on table showTables using 'preaggregate' as 
select sum(age) from showTables")
+sql(
+  "create datamap preAgg on table showTables using 'preaggregate' as 
select sum(age) from showTables")
 sql("show tables").show()
-assert(!sql("show tables").collect().contains("showTables_preagg"))
+checkExistence(sql("show tables"), false, "showtables_preagg")
+
CarbonProperties.getInstance().addProperty(CarbonCommonConstants.CARBON_SHOW_DATAMAPS,CarbonCommonConstants.CARBON_SHOW_DATAMAPS_DEFAULT)
--- End diff --

reset the property to default in afterAll() block


---

[GitHub] carbondata issue #2015: [CARBONDATA-2103]Make show datamaps configurable in ...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2015
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2739/



---

[GitHub] carbondata pull request #2015: [CARBONDATA-2103]Make show datamaps configura...

[ravipesala]

Github user ravipesala commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2015#discussion_r171243483
  
--- Diff: 
integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggCreateCommand.scala
 ---
@@ -377,11 +379,15 @@ class TestPreAggCreateCommand extends QueryTest with 
BeforeAndAfterAll {
 sql("DROP DATAMAP IF EXISTS agg0 ON TABLE maintable")
   }
 
-  test("test show tables filterted with datamaps"){
+  test("test show tables filterted with datamaps") {
+
CarbonProperties.getInstance().addProperty(CarbonCommonConstants.CARBON_SHOW_DATAMAPS,"false")
 sql("create table showTables(name string, age int) stored by 
'carbondata'")
-sql("create datamap preAgg on table showTables using 'preaggregate' as 
select sum(age) from showTables")
+sql(
+  "create datamap preAgg on table showTables using 'preaggregate' as 
select sum(age) from showTables")
 sql("show tables").show()
-assert(!sql("show tables").collect().contains("showTables_preagg"))
+checkExistence(sql("show tables"), false, "showtables_preagg")
+
CarbonProperties.getInstance().addProperty(CarbonCommonConstants.CARBON_SHOW_DATAMAPS,CarbonCommonConstants.CARBON_SHOW_DATAMAPS_DEFAULT)
--- End diff --

Keep it finally block


---

[GitHub] carbondata pull request #2015: [CARBONDATA-2103]Make show datamaps configura...

[akashrn5]

GitHub user akashrn5 opened a pull request:

https://github.com/apache/carbondata/pull/2015

[CARBONDATA-2103]Make show datamaps configurable in show tables command

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [x] Any interfaces changed?
 NA
 - [x] Any backward compatibility impacted?
 NA
 - [x] Document update required?
Yes
 - [x] Testing done
added UT
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [X] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
NA


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/akashrn5/incubator-carbondata show

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2015.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2015


commit 91c630c461069f9bf1558350f8ae67029241e712
Author: akashrn5 
Date:   2018-02-28T12:08:50Z

fix show tables in case of any exception while filtering datamaps




---

[GitHub] carbondata issue #2014: [CARBONDATA-2198] Fixed bug for streaming data for b...

[geetikagupta16]

Github user geetikagupta16 commented on the issue:

https://github.com/apache/carbondata/pull/2014
  
@mohammadshahidkhan Please review. This issue is related to PR #1942 


---

[GitHub] carbondata issue #1989: [CARBONDATA-2194] Exception is improper when use inc...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1989
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3729/



---

[GitHub] carbondata issue #1867: [CARBONDATA-2055][Streaming]Support integrating Stre...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1867
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3728/



---

[GitHub] carbondata issue #2006: [CARBONDATA-2209] Fixed rename table with partitions...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2006
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2738/



---

[GitHub] carbondata issue #2006: [CARBONDATA-2209] Fixed rename table with partitions...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2006
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3982/



---

[GitHub] carbondata issue #2014: [CARBONDATA-2198] Fixed bug for streaming data for b...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2014
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3981/



---

[GitHub] carbondata issue #2014: [CARBONDATA-2198] Fixed bug for streaming data for b...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2014
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2736/



---

[GitHub] carbondata issue #1995: [WIP] File Format Reader

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1995
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2737/



---

[GitHub] carbondata issue #1995: [WIP] File Format Reader

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1995
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3980/



---

[jira] [Resolved] (CARBONDATA-2207) TestCase Fails using Hive Metastore

[Ravindra Pesala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravindra Pesala resolved CARBONDATA-2207.
-
Resolution: Fixed

> TestCase Fails using Hive Metastore
> ---
>
> Key: CARBONDATA-2207
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2207
> Project: CarbonData
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Jatin
>Assignee: Jatin
>Priority: Major
> Fix For: 1.4.0
>
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> Run All the Cabon TestCases using hive metastore out of which some test cases 
> were failing because of not getting carbon table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata pull request #2005: [CARBONDATA-2207] Fix testcases after using h...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/2005


---

[GitHub] carbondata pull request #2014: [CARBONDATA-2198] Fixed bug for streaming dat...

[geetikagupta16]

GitHub user geetikagupta16 opened a pull request:

https://github.com/apache/carbondata/pull/2014

[CARBONDATA-2198] Fixed bug for streaming data for bad_records_action as 
REDIRECT or IGNORE

1. Refactored streaming functionality for bad_records_action as IGNORE or 
REDIRECT
2. Added related test cases

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/geetikagupta16/incubator-carbondata 
CARBONDATA-2198

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2014.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2014


commit ff972f9d32a4eb9786cb9e92c48b574789ae368b
Author: Geetika Gupta 
Date:   2018-02-28T10:39:48Z

1. Refactored streaming functionality for bad_records_action as IGNORE or 
REDIRECT
2. Added related test cases




---

[GitHub] carbondata issue #2005: [CARBONDATA-2207] Fix testcases after using hive met...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2005
  
LGTM


---

[GitHub] carbondata issue #1930: [CARBONDATA-2130] Find some spelling error in Carbon...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1930
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3727/



---

[GitHub] carbondata issue #1995: [WIP] File Format Reader

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1995
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2735/



---

[GitHub] carbondata issue #1995: [WIP] File Format Reader

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1995
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3979/



---

[jira] [Resolved] (CARBONDATA-2214) Remove config 'spark.sql.hive.thriftServer.singleSession' from installation-guide.md

[Liang Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liang Chen resolved CARBONDATA-2214.

   Resolution: Fixed
Fix Version/s: 1.3.1

> Remove config 'spark.sql.hive.thriftServer.singleSession' from 
> installation-guide.md
> 
>
> Key: CARBONDATA-2214
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2214
> Project: CarbonData
>  Issue Type: Task
>  Components: docs
>Reporter: Zhichao  Zhang
>Assignee: Zhichao  Zhang
>Priority: Trivial
> Fix For: 1.3.1
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Remove config 'spark.sql.hive.thriftServer.singleSession' from 
> installation-guide.md



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata pull request #2013: [CARBONDATA-2214][Docs]Remove config 'spark.s...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/2013


---

[GitHub] carbondata issue #2013: [CARBONDATA-2214][Docs]Remove config 'spark.sql.hive...

[chenliang613]

Github user chenliang613 commented on the issue:

https://github.com/apache/carbondata/pull/2013
  
LGTM


---

[GitHub] carbondata issue #1995: [WIP] File Format Reader

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1995
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3726/



---

[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1857
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3725/



---

[GitHub] carbondata issue #2013: [CARBONDATA-2214][Docs]Remove config 'spark.sql.hive...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2013
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3978/



---

[GitHub] carbondata issue #2012: [CARBONDATA-2216][Test] Fix bugs in sdv tests

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2012
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2733/



---

[GitHub] carbondata issue #2012: [CARBONDATA-2216][Test] Fix bugs in sdv tests

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2012
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3977/



---