[GitHub] carbondata pull request #2983: [CARBONDATA-3119] Fixed SDK Write for Complex...

2018-12-17 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/2983


---

[GitHub] carbondata pull request #2983: [CARBONDATA-3119] Fixed SDK Write for Complex...

2018-12-17 Thread kunal642 
Github user kunal642 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2983#discussion_r242403727
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/loading/parser/impl/ArrayParserImpl.java
 ---
@@ -56,6 +56,10 @@ public ArrayObject parse(Object data) {
   }
   return new ArrayObject(array);
 }
+  } else if (value.isEmpty()) {
+Object[] array = new Object[1];
+array[0] = child.parse(value);
--- End diff --

Why not use value instead of child.parse??


---

[GitHub] carbondata pull request #2983: [CARBONDATA-3119] Fixed SDK Write for Complex...

2018-12-11 Thread ajantha-bhat 
Github user ajantha-bhat commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2983#discussion_r240889253
  
--- Diff: 
store/sdk/src/test/java/org/apache/carbondata/sdk/file/CarbonReaderTest.java ---
@@ -2072,4 +1797,35 @@ public void testReadingNullValues() {
 }
   }
 
+  @Test public void testSdkWriteWhenArrayOfStringIsEmpty()
+  throws IOException, InvalidLoadOptionException {
+
+CarbonProperties.getInstance()
+.addProperty(CarbonCommonConstants.CARBON_BAD_RECORDS_ACTION, 
"FAIL");
--- End diff --

same as above, take a backup of CARBON_BAD_RECORDS_ACTION and set it at the 
end of the test case


---

[GitHub] carbondata pull request #2983: [CARBONDATA-3119] Fixed SDK Write for Complex...

2018-12-11 Thread ajantha-bhat 
Github user ajantha-bhat commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2983#discussion_r240889185
  
--- Diff: 
store/sdk/src/test/java/org/apache/carbondata/sdk/file/CarbonReaderTest.java ---
@@ -23,8 +23,11 @@
 import java.util.*;
 
 import org.apache.avro.generic.GenericData;
+
--- End diff --

please revert unwanted changes, only your changes should be there in diff


---

[GitHub] carbondata pull request #2983: [CARBONDATA-3119] Fixed SDK Write for Complex...

2018-12-11 Thread ajantha-bhat 
Github user ajantha-bhat commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2983#discussion_r240889074
  
--- Diff: 
integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/complexType/TestComplexDataType.scala
 ---
@@ -39,6 +39,27 @@ class TestComplexDataType extends QueryTest with 
BeforeAndAfterAll {
   .addProperty(CarbonCommonConstants.CARBON_BAD_RECORDS_ACTION, 
badRecordAction)
   }
 
+  test("test Projection PushDown for Array - String type when Array is 
Empty") {
+CarbonProperties.getInstance()
+  .addProperty(CarbonCommonConstants.CARBON_BAD_RECORDS_ACTION, "FAIL")
--- End diff --

Take a previous CARBON_BAD_RECORDS_ACTION property as backup and set it 
back after this test case, else it affects other test suites


---

[GitHub] carbondata pull request #2983: [CARBONDATA-3119] Fixed SDK Write for Complex...

2018-12-11 Thread ajantha-bhat 
Github user ajantha-bhat commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2983#discussion_r240889108
  
--- Diff: 
integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/complexType/TestComplexDataType.scala
 ---
@@ -39,6 +39,27 @@ class TestComplexDataType extends QueryTest with 
BeforeAndAfterAll {
   .addProperty(CarbonCommonConstants.CARBON_BAD_RECORDS_ACTION, 
badRecordAction)
   }
 
+  test("test Projection PushDown for Array - String type when Array is 
Empty") {
+CarbonProperties.getInstance()
+  .addProperty(CarbonCommonConstants.CARBON_BAD_RECORDS_ACTION, "FAIL")
+sql("drop table if exists table1")
+sql("create table table1 (detail array) stored by 
'carbondata'")
+sql("insert into table1 values('')")
+checkAnswer(sql("select detail[0] from table1"), Seq(Row("")))
+sql("drop table if exists table1")
+  }
+
+  test("test Projection PushDown for Struct - Array type when Array is 
Empty") {
+CarbonProperties.getInstance()
+  .addProperty(CarbonCommonConstants.CARBON_BAD_RECORDS_ACTION, "FAIL")
--- End diff --

same as above


---

[GitHub] carbondata pull request #2983: [CARBONDATA-3119] Fixed SDK Write for Complex...

2018-12-10 Thread shivamasn 
GitHub user shivamasn opened a pull request:

https://github.com/apache/carbondata/pull/2983

[CARBONDATA-3119] Fixed SDK Write for Complex Array Type when Array is Empty

### What was the issue?
For SDK Write , it was going into bad record by returning null on passing 
empty array for Complex Type.
### What has been changed?
Added a check for empty array. This will return an empty array.

 - [ ] Any interfaces changed?
 NA
 - [ ] Any backward compatibility impacted?
 NA
 - [ ] Document update required?
NA
 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
Test  case added
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
NA


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shivamasn/carbondata complex_issue

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2983.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2983


commit a81911b32aa7b586a5ae0c8d206d16fb8921c6fa
Author: namanrastogi 
Date:   2018-12-10T13:34:43Z

Complex Type Empty String Issue Fixed




---