[GitHub] CarbonDataQA commented on issue #3085: [CARBONDATA-3261] support float and byte data type reading from presto carbon

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3085: [CARBONDATA-3261] support float and byte 
data type reading from presto carbon
URL: https://github.com/apache/carbondata/pull/3085#issuecomment-455977295
 
 
   Build Success with Spark 2.2.1, Please check CI 
http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2588/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3082: [CARBONDATA-3260] Fix the Hive stats issue in carbon catalog table

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3082: [CARBONDATA-3260] Fix the Hive stats 
issue in carbon catalog table
URL: https://github.com/apache/carbondata/pull/3082#issuecomment-455976352
 
 
   Build Success with Spark 2.2.1, Please check CI 
http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2589/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] ajantha-bhat commented on a change in pull request #3083: [CARBONDATA-3257] Fix for NO_SORT load and describe formatted being in NO_SORT flow even with Sort Columns given

2019-01-20 Thread GitBox
ajantha-bhat commented on a change in pull request #3083: [CARBONDATA-3257] Fix 
for NO_SORT load and describe formatted being in NO_SORT flow even with Sort 
Columns given
URL: https://github.com/apache/carbondata/pull/3083#discussion_r249346883
 
 

 ##
 File path: 
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonLoadDataCommand.scala
 ##
 @@ -209,16 +209,24 @@ case class CarbonLoadDataCommand(
 * 4. Session property CARBON_OPTIONS_SORT_SCOPE
 * 5. Default Sort Scope LOAD_SORT_SCOPE
 */
-optionsFinal.put("sort_scope",
-  options.getOrElse("sort_scope",
-carbonProperty.getProperty(
-  CarbonLoadOptionConstants.CARBON_TABLE_LOAD_SORT_SCOPE + 
table.getDatabaseName + "." +
-  table.getTableName,
-  tableProperties.asScala.getOrElse("sort_scope",
-
carbonProperty.getProperty(CarbonLoadOptionConstants.CARBON_OPTIONS_SORT_SCOPE,
-  carbonProperty.getProperty(CarbonCommonConstants.LOAD_SORT_SCOPE,
-CarbonCommonConstants.LOAD_SORT_SCOPE_DEFAULT))
-
+if (tableProperties.get("sort_columns") != null &&
+tableProperties.get("sort_scope") == null) {
+  // If there are Sort Columns given for the table and Sort Scope is not 
specified,
+  // we will take it as whichever sort scope given or LOCAL_SORT as default
+  optionsFinal.put("sort_scope", 
carbonProperty.getProperty("carbon.load.sort.scope",
+SortScopeOptions.getSortScope("LOCAL_SORT").toString))
+}
+else {
 
 Review comment:
   keep proper indentation of else.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] ajantha-bhat commented on issue #3083: [CARBONDATA-3257] Fix for NO_SORT load and describe formatted being in NO_SORT flow even with Sort Columns given

2019-01-20 Thread GitBox
ajantha-bhat commented on issue #3083: [CARBONDATA-3257] Fix for NO_SORT load 
and describe formatted being in NO_SORT flow even with Sort Columns given
URL: https://github.com/apache/carbondata/pull/3083#issuecomment-455975469
 
 
   @manishnalla1994 : Also please observe the compaction sort_scope behavior in 
upgrade scenario. Fix it if any issue as part of this


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] ajantha-bhat commented on a change in pull request #3083: [CARBONDATA-3257] Fix for NO_SORT load and describe formatted being in NO_SORT flow even with Sort Columns given

2019-01-20 Thread GitBox
ajantha-bhat commented on a change in pull request #3083: [CARBONDATA-3257] Fix 
for NO_SORT load and describe formatted being in NO_SORT flow even with Sort 
Columns given
URL: https://github.com/apache/carbondata/pull/3083#discussion_r249346414
 
 

 ##
 File path: 
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonLoadDataCommand.scala
 ##
 @@ -209,16 +209,24 @@ case class CarbonLoadDataCommand(
 * 4. Session property CARBON_OPTIONS_SORT_SCOPE
 * 5. Default Sort Scope LOAD_SORT_SCOPE
 */
-optionsFinal.put("sort_scope",
-  options.getOrElse("sort_scope",
-carbonProperty.getProperty(
-  CarbonLoadOptionConstants.CARBON_TABLE_LOAD_SORT_SCOPE + 
table.getDatabaseName + "." +
-  table.getTableName,
-  tableProperties.asScala.getOrElse("sort_scope",
-
carbonProperty.getProperty(CarbonLoadOptionConstants.CARBON_OPTIONS_SORT_SCOPE,
-  carbonProperty.getProperty(CarbonCommonConstants.LOAD_SORT_SCOPE,
-CarbonCommonConstants.LOAD_SORT_SCOPE_DEFAULT))
-
+if (tableProperties.get("sort_columns") != null &&
+tableProperties.get("sort_scope") == null) {
+  // If there are Sort Columns given for the table and Sort Scope is not 
specified,
+  // we will take it as whichever sort scope given or LOCAL_SORT as default
+  optionsFinal.put("sort_scope", 
carbonProperty.getProperty("carbon.load.sort.scope",
+SortScopeOptions.getSortScope("LOCAL_SORT").toString))
 
 Review comment:
   In carbonTableSchemaCommon.scala, same logic there.
   
   see TableNewProcessor. process(). look up for log " // If sort_scope is not 
specified, but sort_columns are present, set sort_scope as"
   
   This logic doesn't handle upgrade case as table is not created again. so 
remove that logic and test agin
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3086: [CARBONDATA-3259] Added documentation for new complex delimiters

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3086: [CARBONDATA-3259] Added documentation 
for new complex delimiters
URL: https://github.com/apache/carbondata/pull/3086#issuecomment-455973111
 
 
   Build Success with Spark 2.3.2, Please check CI 
http://136.243.101.176:8080/job/carbondataprbuilder2.3/10621/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3086: [CARBONDATA-3259] Added documentation for new complex delimiters

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3086: [CARBONDATA-3259] Added documentation 
for new complex delimiters
URL: https://github.com/apache/carbondata/pull/3086#issuecomment-455972267
 
 
   Build Success with Spark 2.2.1, Please check CI 
http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2586/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] NamanRastogi commented on issue #3064: [CARBONDATA-3243] Updated DOC for No-Sort Compaction and a few Fixes

2019-01-20 Thread GitBox
NamanRastogi commented on issue #3064: [CARBONDATA-3243] Updated DOC for 
No-Sort Compaction and a few Fixes
URL: https://github.com/apache/carbondata/pull/3064#issuecomment-455970559
 
 
   @xuchuanyin I have removed the Explanation of Compaction Performance. But 
Document update is still required for `SET 
ALTER.TABLE.LOAD.SORT.SCOPE..` and Sort-Scope in `LOAD DATA` 
command.
   
   Problems:
   1. **SortScopeOptions.getSortScope**
   Default Sort Scope after PR is NO_SORT. So default here also should be 
Default Sort Scope, not some hardcoded value.
   
   2. **CarbonTable.getSortScope**
   After [PR 3014](https://github.com/apache/carbondata/pull/3014), session 
property for table level sort scope is added. This overrides the table level 
Sort Scope provided in `CREATE TABLE` command. CarbonTable.getSortScope should 
also reflect this behavior.
   
   3. **CarbonDDLSqlParser.validateOptions**
   The Sort Scope for Load Data command was not getting verified.
   
   4. **SingleThreadFinalSortFilesMerger.getSortScope**
   Object `inMemorySortTempChunkHolder` should be added to the 
`recordHolderHeapLocal` Only if a row is found in 
`inMemorySortTempChunkHolder`, else not.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Created] (CARBONDATA-3261) support float and byte reading from presto

2019-01-20 Thread Ajantha Bhat (JIRA)
Ajantha Bhat created CARBONDATA-3261:


 Summary: support float and byte reading from presto
 Key: CARBONDATA-3261
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3261
 Project: CarbonData
  Issue Type: Improvement
Reporter: Ajantha Bhat
Assignee: Ajantha Bhat


problem: support float and byte reading from presto

cause: currently float and byte cannot be read in presto due to code issue. It 
was going as double data type. Hence array out of bound issue used to come as 
float/byte read from double stream reader.

solution: Implement a new stream reader for float and byte.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ajantha-bhat commented on issue #3085: [CARBONDATA-3261] support float and byte data type reading from presto carbon

2019-01-20 Thread GitBox
ajantha-bhat commented on issue #3085: [CARBONDATA-3261] support float and byte 
data type reading from presto carbon
URL: https://github.com/apache/carbondata/pull/3085#issuecomment-455969389
 
 
   @ravipesala : please check this


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] xubo245 commented on a change in pull request #3087: Delete unused packages

2019-01-20 Thread GitBox
xubo245 commented on a change in pull request #3087: Delete unused packages
URL: https://github.com/apache/carbondata/pull/3087#discussion_r249341018
 
 

 ##
 File path: 
integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/directdictionary/TimestampNoDictionaryColumnTestCase.scala
 ##
 @@ -20,10 +20,8 @@ package 
org.apache.carbondata.spark.testsuite.directdictionary
 import java.sql.Timestamp
 
 import org.apache.spark.sql.Row
-import org.apache.spark.sql.hive.HiveContext
 import org.scalatest.BeforeAndAfterAll
 import org.apache.carbondata.core.constants.CarbonCommonConstants
 
 Review comment:
   It's seem didn't open code style check, maybe we can raise new PR for 
optimizing it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] xubo245 commented on issue #3087: Delete unused packages

2019-01-20 Thread GitBox
xubo245 commented on issue #3087: Delete unused packages
URL: https://github.com/apache/carbondata/pull/3087#issuecomment-455968329
 
 
   @gouyangyang Please optimize the title,for example: [CARBONDATA-3252] Remove 
unused import


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] gouyangyang commented on issue #3087: Delete unused packages

2019-01-20 Thread GitBox
gouyangyang commented on issue #3087: Delete unused packages
URL: https://github.com/apache/carbondata/pull/3087#issuecomment-455967456
 
 
   Delete unused packages 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] gouyangyang opened a new pull request #3087: Delete unused packages

2019-01-20 Thread GitBox
gouyangyang opened a new pull request #3087: Delete unused packages
URL: https://github.com/apache/carbondata/pull/3087
 
 
   Be sure to do all of the following checklist to help us incorporate 
   your contribution quickly and easily:
   
- [ ] Any interfaces changed?
   

org.apache.carbondata.spark.testsuite.directdictionary.TimestampNoDictionaryColumnTestCase
remove Line 23,26
   
   org.apache.carbondata.spark.testsuite.badrecordloger.BadRecordLoggerTest
remove Line 24,26 line CarbonLoadOptionConstants
remove Line 38
   
   
   
- [ ] Any backward compatibility impacted?

- [ ] Document update required?
   
- [ ] Testing done
   Please provide details on 
   - Whether new unit test cases have been added or why no new tests 
are required?
   - How it is tested? Please attach test report.
   - Is it a performance related change? Please attach the performance 
test report.
   - Any additional information to help reviewers in testing this 
change.
  
- [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] ravipesala opened a new pull request #3088: [DOC] Updated Presto guide

2019-01-20 Thread GitBox
ravipesala opened a new pull request #3088: [DOC] Updated Presto guide
URL: https://github.com/apache/carbondata/pull/3088
 
 
   Be sure to do all of the following checklist to help us incorporate 
   your contribution quickly and easily:
   
- [ ] Any interfaces changed?

- [ ] Any backward compatibility impacted?

- [ ] Document update required?
   
- [ ] Testing done
   Please provide details on 
   - Whether new unit test cases have been added or why no new tests 
are required?
   - How it is tested? Please attach test report.
   - Is it a performance related change? Please attach the performance 
test report.
   - Any additional information to help reviewers in testing this 
change.
  
- [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3081: [SDV] Fix SDV testcase failures

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3081: [SDV] Fix SDV testcase failures
URL: https://github.com/apache/carbondata/pull/3081#issuecomment-455966733
 
 
   Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2365/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3082: [CARBONDATA-3260] Fix the Hive stats issue in carbon catalog table

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3082: [CARBONDATA-3260] Fix the Hive stats 
issue in carbon catalog table
URL: https://github.com/apache/carbondata/pull/3082#issuecomment-455965087
 
 
   Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2364/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] shivamasn commented on a change in pull request #3081: [SDV] Fix SDV testcase failures

2019-01-20 Thread GitBox
shivamasn commented on a change in pull request #3081: [SDV] Fix SDV testcase 
failures
URL: https://github.com/apache/carbondata/pull/3081#discussion_r249338098
 
 

 ##
 File path: 
integration/spark-common-cluster-test/src/test/scala/org/apache/spark/sql/common/util/QueryTest.scala
 ##
 @@ -45,7 +45,7 @@ class QueryTest extends PlanTest with Suite {
   // Add Locale setting
   Locale.setDefault(Locale.US)
   CarbonProperties.getInstance()
-.addProperty(CarbonCommonConstants.VALIDATE_DIRECT_QUERY_ON_DATAMAP, 
"false")
+.addProperty(CarbonCommonConstants.SUPPORT_DIRECT_QUERY_ON_DATAMAP, 
"false")
 
 Review comment:
   @ajantha-bhat Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3085: [WIP] Support float data type reading in presto

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3085: [WIP] Support float data type reading in 
presto
URL: https://github.com/apache/carbondata/pull/3085#issuecomment-455964778
 
 
   Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2363/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3086: [CARBONDATA-3259] Added documentation for new complex delimiters

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3086: [CARBONDATA-3259] Added documentation 
for new complex delimiters
URL: https://github.com/apache/carbondata/pull/3086#issuecomment-455962082
 
 
   Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2362/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] manishnalla1994 commented on a change in pull request #3086: [CARBONDATA-3259] Added documentation for new complex delimiters

2019-01-20 Thread GitBox
manishnalla1994 commented on a change in pull request #3086: [CARBONDATA-3259] 
Added documentation for new complex delimiters
URL: https://github.com/apache/carbondata/pull/3086#discussion_r249333887
 
 

 ##
 File path: docs/dml-of-carbondata.md
 ##
 @@ -135,15 +135,24 @@ CarbonData DML statements are documented here,which 
includes:
 Split the complex type data column in a row (eg., a\001b\001c --> Array = 
{a,b,c}).
 
 ```
-OPTIONS('COMPLEX_DELIMITER_LEVEL_1'='\\\001')
+OPTIONS('COMPLEX_DELIMITER_LEVEL_1'='\001')
 ```
 
   - # COMPLEX_DELIMITER_LEVEL_2:
 
 Split the complex type nested data column in a row. Applies level_1 
delimiter & applies level_2 based on complex data type (eg., a\002b\001c\002d 
--> Array> = {{a,b},{c,d}}).
 
 ```
-OPTIONS('COMPLEX_DELIMITER_LEVEL_2'='\\\002')
+OPTIONS('COMPLEX_DELIMITER_LEVEL_2'='\002')
+```
+
+  - # COMPLEX_DELIMITER_LEVEL_3:
+
+Split the complex type nested data column in a row. Applies level_1 
delimiter, applies level_2 and then level_3 delimiter based on complex data 
type.
+ Used in case of nested Complex Map type. (eg., 
'1\0032\0022\0033\001100\003200\002200\003300' --> Array Of Map> = {{1 -> 2, 2 
-> 3},{100 -> 200, 200 -> 300}}).
 
 Review comment:
   Done.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3086: [CARBONDATA-3259] Added documentation for new complex delimiters

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3086: [CARBONDATA-3259] Added documentation 
for new complex delimiters
URL: https://github.com/apache/carbondata/pull/3086#issuecomment-455959620
 
 
   Build Success with Spark 2.2.1, Please check CI 
http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2585/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] ravipesala commented on a change in pull request #3082: [CARBONDATA-3260] Fix the Hive stats issue in carbon catalog table

2019-01-20 Thread GitBox
ravipesala commented on a change in pull request #3082: [CARBONDATA-3260] Fix 
the Hive stats issue in carbon catalog table
URL: https://github.com/apache/carbondata/pull/3082#discussion_r249333552
 
 

 ##
 File path: streaming/pom.xml
 ##
 @@ -107,33 +107,6 @@
   false
 
   
-  
 
 Review comment:
   It is unnecessary plugin added here. There is no scala tests added in this 
module. Because of it there are duplicate test count is added in CI thats why 
it is removed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3086: [CARBONDATA-3259] Added documentation for new complex delimiters

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3086: [CARBONDATA-3259] Added documentation 
for new complex delimiters
URL: https://github.com/apache/carbondata/pull/3086#issuecomment-455959552
 
 
   Build Success with Spark 2.3.2, Please check CI 
http://136.243.101.176:8080/job/carbondataprbuilder2.3/10620/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Created] (CARBONDATA-3260) Broadcast join is not properly in carbon with spark-2.3.2

2019-01-20 Thread Ravindra Pesala (JIRA)
Ravindra Pesala created CARBONDATA-3260:
---

 Summary: Broadcast join is not properly in carbon with spark-2.3.2
 Key: CARBONDATA-3260
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3260
 Project: CarbonData
  Issue Type: Bug
Reporter: Ravindra Pesala


It seems stats which come from catalog table of hive gives wrong data sizes for 
carbon table. Because of that even large tables are also going to broadcast 
join.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] qiuchenjian commented on a change in pull request #3086: [CARBONDATA-3259] Added documentation for new complex delimiters

2019-01-20 Thread GitBox
qiuchenjian commented on a change in pull request #3086: [CARBONDATA-3259] 
Added documentation for new complex delimiters
URL: https://github.com/apache/carbondata/pull/3086#discussion_r24919
 
 

 ##
 File path: docs/dml-of-carbondata.md
 ##
 @@ -135,15 +135,24 @@ CarbonData DML statements are documented here,which 
includes:
 Split the complex type data column in a row (eg., a\001b\001c --> Array = 
{a,b,c}).
 
 ```
-OPTIONS('COMPLEX_DELIMITER_LEVEL_1'='\\\001')
+OPTIONS('COMPLEX_DELIMITER_LEVEL_1'='\001')
 ```
 
   - # COMPLEX_DELIMITER_LEVEL_2:
 
 Split the complex type nested data column in a row. Applies level_1 
delimiter & applies level_2 based on complex data type (eg., a\002b\001c\002d 
--> Array> = {{a,b},{c,d}}).
 
 ```
-OPTIONS('COMPLEX_DELIMITER_LEVEL_2'='\\\002')
+OPTIONS('COMPLEX_DELIMITER_LEVEL_2'='\002')
+```
+
+  - # COMPLEX_DELIMITER_LEVEL_3:
+
+Split the complex type nested data column in a row. Applies level_1 
delimiter, applies level_2 and then level_3 delimiter based on complex data 
type.
+ Used in case of nested Complex Map type. (eg., 
'1\0032\0022\0033\001100\003200\002200\003300' --> Array Of Map> = {{1 -> 2, 2 
-> 3},{100 -> 200, 200 -> 300}}).
 
 Review comment:
   better to use string instead of int in the example, because it's confused 
with level delimiter
   such as '1\0032\0022\0033\001100\003200\002200\003300'  ---> 
'a\003b\002b\003c\001aaa\003bbb\002bbb\003ccc'


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3086: [CARBONDATA-3259] Added documentation for new complex delimiters

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3086: [CARBONDATA-3259] Added documentation 
for new complex delimiters
URL: https://github.com/apache/carbondata/pull/3086#issuecomment-455950746
 
 
   Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2361/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] manishnalla1994 opened a new pull request #3086: [CARBONDATA-3259] Added documentation for new complex delimiters

2019-01-20 Thread GitBox
manishnalla1994 opened a new pull request #3086: [CARBONDATA-3259] Added 
documentation for new complex delimiters
URL: https://github.com/apache/carbondata/pull/3086
 
 
   Be sure to do all of the following checklist to help us incorporate 
   your contribution quickly and easily:
   
- [ ] Any interfaces changed?

- [ ] Any backward compatibility impacted?

- [ ] Document update required?
   
- [ ] Testing done
   Please provide details on 
   - Whether new unit test cases have been added or why no new tests 
are required?
   - How it is tested? Please attach test report.
   - Is it a performance related change? Please attach the performance 
test report.
   - Any additional information to help reviewers in testing this 
change.
  
- [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Created] (CARBONDATA-3259) Documentation Update

2019-01-20 Thread MANISH NALLA (JIRA)
MANISH NALLA created CARBONDATA-3259:


 Summary: Documentation Update
 Key: CARBONDATA-3259
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3259
 Project: CarbonData
  Issue Type: Sub-task
Reporter: MANISH NALLA
Assignee: MANISH NALLA






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] CarbonDataQA commented on issue #3085: [WIP] Support float data type reading in presto

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3085: [WIP] Support float data type reading in 
presto
URL: https://github.com/apache/carbondata/pull/3085#issuecomment-455942906
 
 
   Build Success with Spark 2.2.1, Please check CI 
http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2584/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3085: [WIP] Support float data type reading in presto

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3085: [WIP] Support float data type reading in 
presto
URL: https://github.com/apache/carbondata/pull/3085#issuecomment-455940676
 
 
   Build Success with Spark 2.3.2, Please check CI 
http://136.243.101.176:8080/job/carbondataprbuilder2.3/10619/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3074: [CARBONDATA-3250] Optimize hive doc

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3074: [CARBONDATA-3250] Optimize hive doc
URL: https://github.com/apache/carbondata/pull/3074#issuecomment-455939262
 
 
   Build Success with Spark 2.2.1, Please check CI 
http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2583/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3074: [CARBONDATA-3250] Optimize hive doc

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3074: [CARBONDATA-3250] Optimize hive doc
URL: https://github.com/apache/carbondata/pull/3074#issuecomment-455939093
 
 
   Build Success with Spark 2.3.2, Please check CI 
http://136.243.101.176:8080/job/carbondataprbuilder2.3/10618/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3085: [WIP] Support float data type reading in presto

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3085: [WIP] Support float data type reading in 
presto
URL: https://github.com/apache/carbondata/pull/3085#issuecomment-455932478
 
 
   Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2360/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3074: [WIP][CARBONDATA-3250] Optimize hive doc

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3074: [WIP][CARBONDATA-3250] Optimize hive doc
URL: https://github.com/apache/carbondata/pull/3074#issuecomment-455931301
 
 
   Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2359/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] ajantha-bhat opened a new pull request #3085: [WIP] Support float data type reading in presto

2019-01-20 Thread GitBox
ajantha-bhat opened a new pull request #3085: [WIP] Support float data type 
reading in presto
URL: https://github.com/apache/carbondata/pull/3085
 
 
   [WIP] Support float data type reading in presto
   
   Be sure to do all of the following checklist to help us incorporate 
   your contribution quickly and easily:
   
- [ ] Any interfaces changed?

- [ ] Any backward compatibility impacted?

- [ ] Document update required?
   
- [ ] Testing done
   Please provide details on 
   - Whether new unit test cases have been added or why no new tests 
are required?
   - How it is tested? Please attach test report.
   - Is it a performance related change? Please attach the performance 
test report.
   - Any additional information to help reviewers in testing this 
change.
  
- [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3084: [CARBONDATA-3258] Add more test case for mv datamap

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3084: [CARBONDATA-3258] Add more test case for 
mv datamap
URL: https://github.com/apache/carbondata/pull/3084#issuecomment-455872208
 
 
   Build Success with Spark 2.3.2, Please check CI 
http://136.243.101.176:8080/job/carbondataprbuilder2.3/10617/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3084: [CARBONDATA-3258] Add more test case for mv datamap

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3084: [CARBONDATA-3258] Add more test case for 
mv datamap
URL: https://github.com/apache/carbondata/pull/3084#issuecomment-455872246
 
 
   Build Success with Spark 2.2.1, Please check CI 
http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2582/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] xubo245 edited a comment on issue #3069: [CARBONDATA-3232] Add test framework and test case for alluxio UT

2019-01-20 Thread GitBox
xubo245 edited a comment on issue #3069: [CARBONDATA-3232] Add test framework 
and test case for alluxio UT
URL: https://github.com/apache/carbondata/pull/3069#issuecomment-455869569
 
 
   @ravipesala removed webapp, please check it again。


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] xubo245 commented on issue #3069: [CARBONDATA-3232] Add test framework and test case for alluxio UT

2019-01-20 Thread GitBox
xubo245 commented on issue #3069: [CARBONDATA-3232] Add test framework and test 
case for alluxio UT
URL: https://github.com/apache/carbondata/pull/3069#issuecomment-455869569
 
 
   @ravipesala removed webapp


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] CarbonDataQA commented on issue #3084: [CARBONDATA-3258] Add more test case for mv datamap

2019-01-20 Thread GitBox
CarbonDataQA commented on issue #3084: [CARBONDATA-3258] Add more test case for 
mv datamap
URL: https://github.com/apache/carbondata/pull/3084#issuecomment-455867770
 
 
   Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2358/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] qiuchenjian opened a new pull request #3084: [CARBONDATA-3258] Add more test case for mv datamap

2019-01-20 Thread GitBox
qiuchenjian opened a new pull request #3084: [CARBONDATA-3258] Add more test 
case for mv datamap
URL: https://github.com/apache/carbondata/pull/3084
 
 
   Be sure to do all of the following checklist to help us incorporate 
   your contribution quickly and easily:
   
- [ ] Any interfaces changed?

- [ ] Any backward compatibility impacted?

- [ ] Document update required?
   
- [ ] Testing done
   more test cases
  
- [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Created] (CARBONDATA-3258) Add more test case for mv datamap

2019-01-20 Thread Chenjian Qiu (JIRA)
Chenjian Qiu created CARBONDATA-3258:


 Summary: Add more test case for mv datamap
 Key: CARBONDATA-3258
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3258
 Project: CarbonData
  Issue Type: Test
  Components: data-query
Reporter: Chenjian Qiu


Add more test case for mv datamap



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)