[GitHub] carbondata issue #2253: [CARBONDATA-2421] Remove unused code of CarbonGetTab...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2253
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4386/



---

[GitHub] carbondata issue #2253: [CARBONDATA-2421] Remove unused code of CarbonGetTab...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2253
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5549/



---

[GitHub] carbondata issue #2253: [CARBONDATA-2421] Remove unused code of CarbonGetTab...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2253
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4645/



---

[GitHub] carbondata issue #2199: [CARBONDATA-2370] Added document for presto multinod...

[geetikagupta16]

Github user geetikagupta16 commented on the issue:

https://github.com/apache/carbondata/pull/2199
  
@sgururajshetty @chenliang613. Please review


---

[GitHub] carbondata pull request #2253: [CARBONDATA-2421] Remove unused code of Carbo...

[xuchuanyin]

GitHub user xuchuanyin opened a pull request:

https://github.com/apache/carbondata/pull/2253

[CARBONDATA-2421] Remove unused code of CarbonGetTableDetailCommand

CarbonGetTableDetailCommand is unused by any other class or method, better 
to remove it as well as the related tests.

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [x] Any interfaces changed?
 `NO`
 - [x] Any backward compatibility impacted?
 `NO`
 - [x] Document update required?
`NO`
 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
`Removed tests`
- How it is tested? Please attach test report.
`Tested in local`
- Is it a performance related change? Please attach the performance 
test report.
`NA`
- Any additional information to help reviewers in testing this 
change.
   `NA`
 - [x] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
`NA`


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xuchuanyin/carbondata 
0501_remove_CarbonGetTableCmd

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2253.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2253


commit 0ba4180cabcab753c3422701da77f48d42da4e03
Author: xuchuanyin 
Date:   2018-05-01T04:30:41Z

Remove unused code of CarbonGetTableDetailCommand




---

[jira] [Created] (CARBONDATA-2421) Remove unused CarbonGetTableDetailCommand

[xuchuanyin (JIRA)]

xuchuanyin created CARBONDATA-2421:
--

 Summary: Remove unused CarbonGetTableDetailCommand
 Key: CARBONDATA-2421
 URL: https://issues.apache.org/jira/browse/CARBONDATA-2421
 Project: CarbonData
  Issue Type: Bug
Reporter: xuchuanyin
Assignee: xuchuanyin


CarbonGetTableDetailCommand is unused, better to remove it



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata issue #2252: WIP:[CARBONDATA-2420] Support string longer than 320...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2252
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4385/



---

[GitHub] carbondata issue #2252: WIP:[CARBONDATA-2420] Support string longer than 320...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2252
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5548/



---

[GitHub] carbondata issue #2252: WIP:[CARBONDATA-2420] Support string longer than 320...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2252
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4644/



---

[jira] [Assigned] (CARBONDATA-2339) 下标越界

[xuchuanyin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xuchuanyin reassigned CARBONDATA-2339:
--

Assignee: xuchuanyin

> 下标越界
> 
>
> Key: CARBONDATA-2339
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2339
> Project: CarbonData
>  Issue Type: Bug
>  Components: data-load
>Affects Versions: 1.3.0
>Reporter: niaoshu
>Assignee: xuchuanyin
>Priority: Blocker
> Fix For: NONE
>
>   Original Estimate: 96h
>  Remaining Estimate: 96h
>
> java.lang.ArrayIndexOutOfBoundsException



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (CARBONDATA-2420) Support string longer than 32000 characters

[xuchuanyin (JIRA)]

xuchuanyin created CARBONDATA-2420:
--

 Summary: Support string longer than 32000 characters
 Key: CARBONDATA-2420
 URL: https://issues.apache.org/jira/browse/CARBONDATA-2420
 Project: CarbonData
  Issue Type: Improvement
Reporter: xuchuanyin
Assignee: xuchuanyin


Add a property in creating table 'long_string_columns' to support string 
columns that will contains more than 32000 characters.
Inside carbondata, it use an integer instead of short to store the length of 
bytes content.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata issue #2239: [CARBONDATA-2408] Fix search mode master SaslExcepti...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2239
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4643/



---

[GitHub] carbondata issue #2239: [CARBONDATA-2408] Fix search mode master SaslExcepti...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2239
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4384/



---

[GitHub] carbondata issue #2239: [CARBONDATA-2408] Fix search mode master SaslExcepti...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2239
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5547/



---

[GitHub] carbondata issue #2252: WIP: Support string longer than 32000 characters

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2252
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4642/



---

[GitHub] carbondata issue #2249: [CARBONDATA-2419] sortColumns Order we are getting w...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2249
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4381/



---

[GitHub] carbondata issue #2246: [CARBONDATA-2413] After running CarbonWriter, there ...

[xubo245]

Github user xubo245 commented on the issue:

https://github.com/apache/carbondata/pull/2246
  
@ravipesala @jackylk @QiangCai Can you help to review it?


---

[GitHub] carbondata issue #2252: WIP: Support string longer than 32000 characters

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2252
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4641/



---

[GitHub] carbondata issue #2249: [CARBONDATA-2419] sortColumns Order we are getting w...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2249
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4640/



---

[GitHub] carbondata issue #2252: WIP: Support string longer than 32000 characters

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2252
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4383/



---

[GitHub] carbondata issue #2240: [CARBONDATA-2313] fixed multiple issues in SDK write...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2240
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4382/



---

[GitHub] carbondata issue #2195: [CARBONDATA-2366] fixed concurrent datamap creation ...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2195
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4639/



---

[GitHub] carbondata issue #2240: [CARBONDATA-2313] fixed multiple issues in SDK write...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2240
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5545/



---

[GitHub] carbondata issue #2195: [CARBONDATA-2366] fixed concurrent datamap creation ...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2195
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4380/



---

[GitHub] carbondata issue #2252: WIP: Support string longer than 32000 characters

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2252
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5546/



---

[GitHub] carbondata issue #2249: [CARBONDATA-2419] sortColumns Order we are getting w...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2249
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5544/



---

[GitHub] carbondata issue #2224: [CARBONDATA-2393]TaskNo is not working for SDK

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2224
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4376/



---

[GitHub] carbondata issue #2209: [CARBONDATA-2388][SDK]Avro Record Complex Type Imple...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2209
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4638/



---

[GitHub] carbondata issue #2195: [CARBONDATA-2366] fixed concurrent datamap creation ...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2195
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5543/



---

[GitHub] carbondata issue #2209: [CARBONDATA-2388][SDK]Avro Record Complex Type Imple...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2209
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4379/



---

[GitHub] carbondata pull request #2252: WIP: Support string longer than 32000 charact...

[xuchuanyin]

GitHub user xuchuanyin opened a pull request:

https://github.com/apache/carbondata/pull/2252

WIP: Support string longer than 32000 characters

Add a property in creating table 'long_string_columns' to support string 
columns that will contains more than 32000 characters.
Inside carbondata, it use an integer instead of short to store the length 
of bytes content.

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xuchuanyin/carbondata 
0428_string_longer_than_32000

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2252.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2252


commit 26cbea7f1493d204a1abb4275e052481abccd185
Author: xuchuanyin 
Date:   2018-04-30T15:53:22Z

Support string longer than 32000 characters

Add a property in creating table 'long_string_columns' to support string 
columns that will contains more than 32000 characters.
Inside carbondata, it use an integer instead of short to store the length 
of bytes content.




---

[GitHub] carbondata issue #2227: [CARBONDATA-2401] Date and Timestamp options are not...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2227
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5541/



---

[GitHub] carbondata issue #2209: [CARBONDATA-2388][SDK]Avro Record Complex Type Imple...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2209
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4637/



---

[GitHub] carbondata issue #2249: [CARBONDATA-2419] sortColumns Order we are getting w...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2249
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4636/



---

[GitHub] carbondata issue #2209: [CARBONDATA-2388][SDK]Avro Record Complex Type Imple...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2209
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5542/



---

[GitHub] carbondata issue #2227: [CARBONDATA-2401] Date and Timestamp options are not...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2227
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4378/



---

[GitHub] carbondata issue #2240: [CARBONDATA-2313] fixed multiple issues in SDK write...

[ajantha-bhat]

Github user ajantha-bhat commented on the issue:

https://github.com/apache/carbondata/pull/2240
  
retest this please


---

[jira] [Resolved] (CARBONDATA-2393) TaskNo is not working for SDK

[kumar vishal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

kumar vishal resolved CARBONDATA-2393.
--
Resolution: Fixed

> TaskNo is not working for SDK
> -
>
> Key: CARBONDATA-2393
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2393
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Babulal
>Assignee: Babulal
>Priority: Minor
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> set Task No. to SDK interface as below 
> val fields: Array[Field] = new Array[Field](2)
> fields(0) = new Field("stringField", DataTypes.STRING)
> fields(1) = new Field("intField", DataTypes.INT)
> val builder: CarbonWriterBuilder = CarbonWriter.builder.withSchema(new 
> Schema(fields))
>  
> .isTransactionalTable(true).outputPath("D:/data/yyy").taskNo("5").isTransactionalTable(false)
> val writer: CarbonWriter = builder.buildWriterForCSVInput
> writer.write(Array("babu","12"));
> writer.close()
>  
> check the files 
> index file name is = 328748098884003_batchno0-0-0.carbonindex
> data file is =part-0-328748098884003_batchno0-0-0.carbondata
>  
> Expected 
> index file name is = 5_batchno0-0-0.carbonindex
> data file is =part-0-5_batchno0-0-0.carbondata
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata pull request #2224: [CARBONDATA-2393]TaskNo is not working for SD...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/2224


---

[GitHub] carbondata issue #2224: [CARBONDATA-2393]TaskNo is not working for SDK

[kumarvishal09]

Github user kumarvishal09 commented on the issue:

https://github.com/apache/carbondata/pull/2224
  
LGTM


---

[GitHub] carbondata issue #2224: [CARBONDATA-2393]TaskNo is not working for SDK

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2224
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4635/



---

[GitHub] carbondata issue #2224: [CARBONDATA-2393]TaskNo is not working for SDK

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2224
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5539/



---

[GitHub] carbondata issue #2249: [CARBONDATA-2419] sortColumns Order we are getting w...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2249
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5540/



---

[GitHub] carbondata issue #2249: [CARBONDATA-2419] sortColumns Order we are getting w...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2249
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4377/



---

[GitHub] carbondata issue #2240: [CARBONDATA-2313] fixed multiple issues in SDK write...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2240
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4375/



---

[GitHub] carbondata issue #2224: [CARBONDATA-2393]TaskNo is not working for SDK

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2224
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4373/



---

[GitHub] carbondata issue #2240: [CARBONDATA-2313] fixed multiple issues in SDK write...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2240
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5538/



---

[jira] [Resolved] (CARBONDATA-2333) Block insert overwrite on parent table if any of the child tables are not partitioned on the specified partition columns

[kumar vishal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

kumar vishal resolved CARBONDATA-2333.
--
   Resolution: Fixed
Fix Version/s: 1.4.0

> Block insert overwrite on parent table if any of the child tables are not 
> partitioned on the specified partition columns
> 
>
> Key: CARBONDATA-2333
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2333
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Kunal Kapoor
>Assignee: Kunal Kapoor
>Priority: Minor
> Fix For: 1.4.0
>
>  Time Spent: 9h 20m
>  Remaining Estimate: 0h
>
> Block insert overwrite on parent table if any of the child tables are not 
> partitioned on the specified partition columns.
> This is required because if any aggregate table is not partitioned then 
> overriding the data for one partition may cause data inconsistency between 
> the parent and child
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata pull request #2172: [CARBONDATA-2333] Add validation for insert o...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/2172


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184992863
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/store/TablePage.java 
---
@@ -222,8 +222,9 @@ private void addComplexColumn(int index, int rowId, 
byte[] complexColumns) {
   ByteBuffer byteArrayInput = ByteBuffer.wrap(complexColumns);
   ByteArrayOutputStream byteArrayOutput = new ByteArrayOutputStream();
   DataOutputStream dataOutputStream = new 
DataOutputStream(byteArrayOutput);
-  complexDataType.parseAndBitPack(byteArrayInput, dataOutputStream,
-  model.getComplexDimensionKeyGenerator());
+  int startOffset = 0;
--- End diff --

Removed


---

[GitHub] carbondata issue #2243: [CARBONDATA-2410] Error message correction when colu...

[KanakaKumar]

Github user KanakaKumar commented on the issue:

https://github.com/apache/carbondata/pull/2243
  
@mohammadshahidkhan , can you please add details on why we have to change 
the exception parsing now.. Did we change univocity jar recently?


---

[GitHub] carbondata pull request #2243: [CARBONDATA-2410] Error message correction wh...

[KanakaKumar]

Github user KanakaKumar commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2243#discussion_r184991452
  
--- Diff: 
integration/spark-common/src/main/scala/org/apache/carbondata/spark/util/GlobalDictionaryUtil.scala
 ---
@@ -824,9 +824,11 @@ object GlobalDictionaryUtil {
   def trimErrorMessage(input: String): String = {
 var errorMessage: String = null
 if (input != null && input.contains("TextParsingException:")) {
-  if (input.split("Hint").length > 0 &&
+  if (input.split("Hint").length > 1 &&
--- End diff --

The title of the fix is wrong.. It should be 32000 characters


---

[GitHub] carbondata issue #2240: [CARBONDATA-2313] fixed multiple issues in SDK write...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2240
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4634/



---

[jira] [Resolved] (CARBONDATA-2394) Setting segments in thread local space but not getting reflected in the driver

[Kunal Kapoor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Kapoor resolved CARBONDATA-2394.
--
   Resolution: Fixed
Fix Version/s: 1.3.0
   1.4.0

> Setting segments in thread local space but not getting reflected in the driver
> --
>
> Key: CARBONDATA-2394
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2394
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Rahul Kumar
>Assignee: Rahul Kumar
>Priority: Major
> Fix For: 1.4.0, 1.3.0
>
>  Time Spent: 3h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata pull request #2223: [CARBONDATA-2394] Setting segments in thread ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/2223


---

[GitHub] carbondata issue #2172: [CARBONDATA-2333] Add validation for insert overwrit...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2172
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4374/



---

[GitHub] carbondata issue #2172: [CARBONDATA-2333] Add validation for insert overwrit...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2172
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5537/



---

[GitHub] carbondata issue #2227: [CARBONDATA-2401] Date and Timestamp options are not...

[BJangir]

Github user BJangir commented on the issue:

https://github.com/apache/carbondata/pull/2227
  
retest this please


---

[GitHub] carbondata issue #2224: [CARBONDATA-2393]TaskNo is not working for SDK

[BJangir]

Github user BJangir commented on the issue:

https://github.com/apache/carbondata/pull/2224
  
retest this please


---

[GitHub] carbondata issue #2223: [CARBONDATA-2394] Setting segments in thread local s...

[kunal642]

Github user kunal642 commented on the issue:

https://github.com/apache/carbondata/pull/2223
  
LGTM


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184977270
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/util/CarbonDataProcessorUtil.java
 ---
@@ -267,14 +267,24 @@ private static String 
getComplexTypeString(DataField[] dataFields) {
 return dimString.toString();
   }
 
+  private static String isDictionaryType(CarbonDimension dimension) {
--- End diff --

This is used to append in a String field. 


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184976884
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/scan/complextypes/PrimitiveQueryType.java
 ---
@@ -84,6 +101,9 @@ public PrimitiveQueryType(String name, String 
parentname, int blockIndex,
   DimensionRawColumnChunk[] rawColumnChunks, int rowNumber,
   int pageNumber, DataOutputStream dataOutputStream) throws 
IOException {
 byte[] currentVal = copyBlockDataChunk(rawColumnChunks, rowNumber, 
pageNumber);
+if (!this.isDictionary) {
+  dataOutputStream.writeInt(currentVal.length);
--- End diff --

Below line 105 we are writing the data. Data writing is without any 
condition. 


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184976649
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/PrimitiveDataType.java
 ---
@@ -94,6 +99,11 @@
 
   private CarbonDimension carbonDimension;
 
+  private Boolean isDictionary;
--- End diff --

Done


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184976699
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/scan/complextypes/PrimitiveQueryType.java
 ---
@@ -53,6 +55,21 @@ public PrimitiveQueryType(String name, String 
parentname, int blockIndex,
 this.name = name;
 this.parentname = parentname;
 this.isDirectDictionary = isDirectDictionary;
+this.isDictionary = true;
+  }
+
+
+  public PrimitiveQueryType(String name, String parentname, int blockIndex,
+  org.apache.carbondata.core.metadata.datatype.DataType dataType, int 
keySize,
+  Dictionary dictionary, boolean isDirectDictionary, boolean 
isDictionary) {
--- End diff --

Done. 


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184976629
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/GenericDataType.java
 ---
@@ -71,6 +71,8 @@ void writeByteArray(T input, DataOutputStream 
dataOutputStream)
*/
   void setSurrogateIndex(int surrIndex);
 
+  Boolean getIsColumnDictionary();
--- End diff --

Done


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184976615
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/ArrayDataType.java
 ---
@@ -183,7 +187,27 @@ public void parseAndBitPack(ByteBuffer byteArrayInput, 
DataOutputStream dataOutp
 for (int i = 0; i < dataLength; i++) {
   children.parseAndBitPack(byteArrayInput, dataOutputStream, 
generator);
 }
+  }
+
+  @Override
+  public int parseComplexValue(ByteBuffer byteArrayInput, DataOutputStream 
dataOutputStream,
+  KeyGenerator[] generator, Boolean[][] 
complexDictionaryIndentification, int startOffset)
--- End diff --

removed complexDictionaryIndentification


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184976246
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/ArrayDataType.java
 ---
@@ -183,7 +187,27 @@ public void parseAndBitPack(ByteBuffer byteArrayInput, 
DataOutputStream dataOutp
 for (int i = 0; i < dataLength; i++) {
   children.parseAndBitPack(byteArrayInput, dataOutputStream, 
generator);
 }
+  }
+
+  @Override
+  public int parseComplexValue(ByteBuffer byteArrayInput, DataOutputStream 
dataOutputStream,
+  KeyGenerator[] generator, Boolean[][] 
complexDictionaryIndentification, int startOffset)
+  throws IOException, KeyGenException {
+int dataLength = byteArrayInput.getInt();
+startOffset += Integer.SIZE / Byte.SIZE;
--- End diff --

Added CarbonCommonConstant


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184976155
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/PrimitiveDataType.java
 ---
@@ -245,6 +312,28 @@ public void parseAndBitPack(ByteBuffer byteArrayInput, 
DataOutputStream dataOutp
 dataOutputStream.write(v);
   }
 
+  @Override
+  public int parseComplexValue(ByteBuffer byteArrayInput, DataOutputStream 
dataOutputStream,
+  KeyGenerator[] generator, Boolean[][] 
complexDictionaryIndentification, int startOffset)
--- End diff --

Removed


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184976110
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/PrimitiveDataType.java
 ---
@@ -245,6 +312,28 @@ public void parseAndBitPack(ByteBuffer byteArrayInput, 
DataOutputStream dataOutp
 dataOutputStream.write(v);
   }
 
+  @Override
+  public int parseComplexValue(ByteBuffer byteArrayInput, DataOutputStream 
dataOutputStream,
+  KeyGenerator[] generator, Boolean[][] 
complexDictionaryIndentification, int startOffset)
+  throws IOException, KeyGenException {
+if (!this.isDictionary) {
+  int sizeOfData = byteArrayInput.getInt();
+  startOffset += Integer.SIZE / Byte.SIZE;
--- End diff --

Added the constant


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184976193
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/ArrayDataType.java
 ---
@@ -183,7 +187,27 @@ public void parseAndBitPack(ByteBuffer byteArrayInput, 
DataOutputStream dataOutp
 for (int i = 0; i < dataLength; i++) {
   children.parseAndBitPack(byteArrayInput, dataOutputStream, 
generator);
 }
+  }
+
+  @Override
+  public int parseComplexValue(ByteBuffer byteArrayInput, DataOutputStream 
dataOutputStream,
+  KeyGenerator[] generator, Boolean[][] 
complexDictionaryIndentification, int startOffset)
+  throws IOException, KeyGenException {
+int dataLength = byteArrayInput.getInt();
+startOffset += Integer.SIZE / Byte.SIZE;
 
+dataOutputStream.writeInt(dataLength);
+if (children instanceof PrimitiveDataType) {
+  if (children.getIsColumnDictionary()) {
+
dataOutputStream.writeInt(generator[children.getSurrogateIndex()].getKeySizeInBytes());
+startOffset += Integer.SIZE / Byte.SIZE;
--- End diff --

Added a constant


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184975951
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/loading/CarbonDataLoadConfiguration.java
 ---
@@ -348,6 +356,33 @@ public void 
setCardinalityFinder(DictionaryCardinalityFinder cardinalityFinder)
 return complexKeyGenerators;
   }
 
+  public Boolean[][] createComplexDictionaryFieldIdentification() {
--- End diff --

Removed


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184976034
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerModel.java
 ---
@@ -609,5 +616,13 @@ public short getWritingCoresCount() {
   public DataMapWriterListener getDataMapWriterlistener() {
 return dataMapWriterlistener;
   }
+
+  public Boolean[][] getComplexDictionaryFields() {
--- End diff --

Done


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184975993
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/PrimitiveDataType.java
 ---
@@ -245,6 +312,28 @@ public void parseAndBitPack(ByteBuffer byteArrayInput, 
DataOutputStream dataOutp
 dataOutputStream.write(v);
   }
 
+  @Override
+  public int parseComplexValue(ByteBuffer byteArrayInput, DataOutputStream 
dataOutputStream,
+  KeyGenerator[] generator, Boolean[][] 
complexDictionaryIndentification, int startOffset)
--- End diff --

Removed


---

[GitHub] carbondata issue #2245: [CARBONDATA-2411] infinite loop when sdk writer thro...

[kumarvishal09]

Github user kumarvishal09 commented on the issue:

https://github.com/apache/carbondata/pull/2245
  
LGTM


---

[jira] [Resolved] (CARBONDATA-2411) infinite loop when sdk writer throws Exception

[kumar vishal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

kumar vishal resolved CARBONDATA-2411.
--
Resolution: Fixed

> infinite loop when sdk writer throws Exception
> --
>
> Key: CARBONDATA-2411
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2411
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Babulal
>Assignee: Babulal
>Priority: Minor
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> When SDK CSVWriter throws Error (Cast String Exception). application is stuck 
> and data loading Threads are in waiting state. Use below code to reproduce 
> issue.
>  
> val fields: Array[Field] = new Array[Field](2)
>  fields(0) = new Field("stringField", DataTypes.STRING)
>  fields(1) = new Field("intField", DataTypes.INT)
> val builder: CarbonWriterBuilder = CarbonWriter.builder.withSchema(new 
> Schema(fields))
>  
> .isTransactionalTable(false).outputPath("D:/data/yyy").taskNo("5").isTransactionalTable(false)
>  val writer: CarbonWriter = builder.buildWriterForCSVInput
> writer.write(Array("xyz",1))
> writer.close()
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata pull request #2245: [CARBONDATA-2411] infinite loop when sdk writ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/2245


---

[GitHub] carbondata issue #2240: [CARBONDATA-2313] fixed multiple issues in SDK write...

[kumarvishal09]

Github user kumarvishal09 commented on the issue:

https://github.com/apache/carbondata/pull/2240
  
Many places indentation is not correct, please check


---

[GitHub] carbondata issue #2172: [CARBONDATA-2333] Add validation for insert overwrit...

[kunal642]

Github user kunal642 commented on the issue:

https://github.com/apache/carbondata/pull/2172
  
retest this please


---

[GitHub] carbondata issue #2224: [CARBONDATA-2393]TaskNo is not working for SDK

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2224
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5536/



---

[GitHub] carbondata issue #2245: [CARBONDATA-2411] infinite loop when sdk writer thro...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2245
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4633/



---

[GitHub] carbondata issue #2227: [CARBONDATA-2401] Date and Timestamp options are not...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2227
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5535/



---

[GitHub] carbondata issue #2227: [CARBONDATA-2401] Date and Timestamp options are not...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2227
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4372/



---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184969127
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/loading/CarbonDataLoadConfiguration.java
 ---
@@ -348,6 +356,33 @@ public void 
setCardinalityFinder(DictionaryCardinalityFinder cardinalityFinder)
 return complexKeyGenerators;
   }
 
+  public Boolean[][] createComplexDictionaryFieldIdentification() {
--- End diff --

removed


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184968977
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/PrimitiveDataType.java
 ---
@@ -245,6 +312,28 @@ public void parseAndBitPack(ByteBuffer byteArrayInput, 
DataOutputStream dataOutp
 dataOutputStream.write(v);
   }
 
+  @Override
+  public int parseComplexValue(ByteBuffer byteArrayInput, DataOutputStream 
dataOutputStream,
+  KeyGenerator[] generator, Boolean[][] 
complexDictionaryIndentification, int startOffset)
--- End diff --

Currently no required. Removed


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184968935
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/PrimitiveDataType.java
 ---
@@ -245,6 +312,28 @@ public void parseAndBitPack(ByteBuffer byteArrayInput, 
DataOutputStream dataOutp
 dataOutputStream.write(v);
   }
 
+  @Override
+  public int parseComplexValue(ByteBuffer byteArrayInput, DataOutputStream 
dataOutputStream,
+  KeyGenerator[] generator, Boolean[][] 
complexDictionaryIndentification, int startOffset)
--- End diff --

Removed


---

[GitHub] carbondata issue #2243: [CARBONDATA-2410] Error message correction when colu...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2243
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4366/



---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184964754
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/PrimitiveDataType.java
 ---
@@ -94,6 +99,11 @@
 
   private CarbonDimension carbonDimension;
 
+  private Boolean isDictionary;
+
+  private FieldConverter fieldConverterForNoDictionary;
--- End diff --

Removed


---

[GitHub] carbondata issue #1138: [CARBONDATA-1263]Single pass load does not take defa...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1138
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4369/



---

[GitHub] carbondata issue #2245: [CARBONDATA-2411] infinite loop when sdk writer thro...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2245
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4368/



---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184964228
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/PrimitiveDataType.java
 ---
@@ -94,6 +99,11 @@
 
   private CarbonDimension carbonDimension;
 
+  private Boolean isDictionary;
--- End diff --

done


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184963740
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/GenericDataType.java
 ---
@@ -80,7 +82,13 @@ void writeByteArray(T input, DataOutputStream 
dataOutputStream)
* @throws KeyGenException
*/
   void parseAndBitPack(ByteBuffer byteArrayInput, DataOutputStream 
dataOutputStream,
-  KeyGenerator[] generator) throws IOException, KeyGenException;
+  KeyGenerator[] generator)
+  throws IOException, KeyGenException;
+
+
+  int parseComplexValue(ByteBuffer byteArrayInput, DataOutputStream 
dataOutputStream,
+  KeyGenerator[] generator, Boolean[][] 
complexDictionaryIndentification, int startOffset)
+  throws IOException, KeyGenException;
--- End diff --

complexDictionaryIndentification is removed but StartOffset is used. 


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184963673
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/GenericDataType.java
 ---
@@ -71,6 +71,8 @@ void writeByteArray(T input, DataOutputStream 
dataOutputStream)
*/
   void setSurrogateIndex(int surrIndex);
 
+  Boolean getIsColumnDictionary();
--- End diff --

Done


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184963324
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/ArrayDataType.java
 ---
@@ -183,7 +187,27 @@ public void parseAndBitPack(ByteBuffer byteArrayInput, 
DataOutputStream dataOutp
 for (int i = 0; i < dataLength; i++) {
   children.parseAndBitPack(byteArrayInput, dataOutputStream, 
generator);
 }
+  }
+
+  @Override
+  public int parseComplexValue(ByteBuffer byteArrayInput, DataOutputStream 
dataOutputStream,
+  KeyGenerator[] generator, Boolean[][] 
complexDictionaryIndentification, int startOffset)
+  throws IOException, KeyGenException {
+int dataLength = byteArrayInput.getInt();
+startOffset += Integer.SIZE / Byte.SIZE;
 
+dataOutputStream.writeInt(dataLength);
+if (children instanceof PrimitiveDataType) {
+  if (children.getIsColumnDictionary()) {
+
dataOutputStream.writeInt(generator[children.getSurrogateIndex()].getKeySizeInBytes());
+startOffset += Integer.SIZE / Byte.SIZE;
--- End diff --

Done


---

[GitHub] carbondata issue #2224: [CARBONDATA-2393]TaskNo is not working for SDK

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2224
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4632/



---

[GitHub] carbondata issue #2224: [CARBONDATA-2393]TaskNo is not working for SDK

[BJangir]

Github user BJangir commented on the issue:

https://github.com/apache/carbondata/pull/2224
  
retest this please


---

[GitHub] carbondata issue #2227: [CARBONDATA-2401] Date and Timestamp options are not...

[BJangir]

Github user BJangir commented on the issue:

https://github.com/apache/carbondata/pull/2227
  
retest this please


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184962411
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/ArrayDataType.java
 ---
@@ -183,7 +187,27 @@ public void parseAndBitPack(ByteBuffer byteArrayInput, 
DataOutputStream dataOutp
 for (int i = 0; i < dataLength; i++) {
   children.parseAndBitPack(byteArrayInput, dataOutputStream, 
generator);
 }
+  }
+
+  @Override
+  public int parseComplexValue(ByteBuffer byteArrayInput, DataOutputStream 
dataOutputStream,
+  KeyGenerator[] generator, Boolean[][] 
complexDictionaryIndentification, int startOffset)
--- End diff --

Removed


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184962372
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/ArrayDataType.java
 ---
@@ -171,9 +175,9 @@ public void fillCardinality(List 
dimCardWithComplex) {
   /**
* parse byte array and bit pack
*/
-  @Override
-  public void parseAndBitPack(ByteBuffer byteArrayInput, DataOutputStream 
dataOutputStream,
-  KeyGenerator[] generator) throws IOException, KeyGenException {
+  @Override public void parseAndBitPack(ByteBuffer byteArrayInput,
--- End diff --

Removed


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184962348
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/datatypes/ArrayDataType.java
 ---
@@ -171,9 +175,9 @@ public void fillCardinality(List 
dimCardWithComplex) {
   /**
* parse byte array and bit pack
*/
-  @Override
-  public void parseAndBitPack(ByteBuffer byteArrayInput, DataOutputStream 
dataOutputStream,
-  KeyGenerator[] generator) throws IOException, KeyGenException {
+  @Override public void parseAndBitPack(ByteBuffer byteArrayInput,
--- End diff --

Done


---

[GitHub] carbondata pull request #2209: [CARBONDATA-2388][SDK]Avro Record Complex Typ...

[sounakr]

Github user sounakr commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2209#discussion_r184961740
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/scan/complextypes/PrimitiveQueryType.java
 ---
@@ -108,4 +128,14 @@ public PrimitiveQueryType(String name, String 
parentname, int blockIndex,
 }
 return actualData;
   }
+
+  @Override public Object 
getDataBasedOnDataTypeFromNoDictionary(ByteBuffer data) {
--- End diff --

Done


---