[jira] [Commented] (KYLIN-5215) The metadata information returned by Kylin pushdown query and Cube query is inconsistent

2022-08-02 Thread gaogao (Jira) 


[ 
https://issues.apache.org/jira/browse/KYLIN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574455#comment-17574455
 ] 

gaogao commented on KYLIN-5215:
---

example:   select  count(*) as "COUNT__stime" from tabletest 

在指标查询的时候加上 as 和双引号就会这样。

cube query result:

{isNullable: 0, displaySize: 19, label: "COUNT__stime", name: "COUNT__stime", 
schemaName: null,…}

pushdown query result:

{isNullable: 0, displaySize: 2147483647, label: "COUNT__STIME", name: 
"COUNT__STIME", schemaName: null,…}

> The metadata information returned by Kylin pushdown query and Cube query is 
> inconsistent
> 
>
> Key: KYLIN-5215
> URL: https://issues.apache.org/jira/browse/KYLIN-5215
> Project: Kylin
>  Issue Type: Bug
>  Components: Spark Engine
>Affects Versions: all, v4.0.1
> Environment: kylin 4.0.1
>Reporter: gaogao
>Priority: Major
> Attachments: cube_query.png, pushdown_query.png
>
>
> 问题：
>  在下压查询中返回的结果  和  cube 查询的结果不一致，下压查询中  columnMetas 的  label 和 name 字段会默认转换成大写！
> 对应源码类
> PushDownRunnerSparkImpl  50行
> ：
>  
> {code:java}
> // fill in selected column meta
> for (int i = 0; i < columnCount; ++i) {
> int nullable = fieldList.get(i).isNullable() ? 1 : 0;
> columnMetas.add(new SelectedColumnMeta(false, false, false, false, nullable, 
> true, Integer.MAX_VALUE,
> fieldList.get(i).getName().toUpperCase(Locale.ROOT), 
> fieldList.get(i).getName().toUpperCase(Locale.ROOT), null, null,
> null, fieldList.get(i).getPrecision(), fieldList.get(i).getScale(), 
> fieldList.get(i).getDataType(),
> fieldList.get(i).getDataTypeName(), false, false, false));
> }
>  
> {code}
>  
> pushdown query:
> !c:\Users\Administrator\AppData\Local\YNote\data\weixinobU7VjlOcNgcm0ZbmNGYzYRPZ__4\401fe7a7ea05495799145f2d413b0da4\clipboard.png!
> cube query:
> !c:\Users\Administrator\AppData\Local\YNote\data\weixinobU7VjlOcNgcm0ZbmNGYzYRPZ__4\0ef826f0229f42b59c4ad946a08bf6f3\clipboard.png!
>  
> 我认为这个问题应该修复，因为在尝试对接superset时发现，有些此类框架会强校验元数据列的大小写，如果默认进行转换会导致无法适配，且cube 
> query和pushdown query查询的元数据列结果不一致这也是无法接受的。
>  
> 如果讨论有需要的话，我可以修复他，谢谢~



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (KYLIN-5215) The metadata information returned by Kylin pushdown query and Cube query is inconsistent

2022-08-02 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/KYLIN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574178#comment-17574178
 ] 

ASF GitHub Bot commented on KYLIN-5215:
---

liyang-kylin commented on PR #1900:
URL: https://github.com/apache/kylin/pull/1900#issuecomment-1202333933

   pending on design discussion in 
https://issues.apache.org/jira/projects/KYLIN/issues/KYLIN-5215




> The metadata information returned by Kylin pushdown query and Cube query is 
> inconsistent
> 
>
> Key: KYLIN-5215
> URL: https://issues.apache.org/jira/browse/KYLIN-5215
> Project: Kylin
>  Issue Type: Bug
>  Components: Spark Engine
>Affects Versions: all, v4.0.1
> Environment: kylin 4.0.1
>Reporter: gaogao
>Priority: Major
> Attachments: cube_query.png, pushdown_query.png
>
>
> 问题：
>  在下压查询中返回的结果  和  cube 查询的结果不一致，下压查询中  columnMetas 的  label 和 name 字段会默认转换成大写！
> 对应源码类
> PushDownRunnerSparkImpl  50行
> ：
>  
> {code:java}
> // fill in selected column meta
> for (int i = 0; i < columnCount; ++i) {
> int nullable = fieldList.get(i).isNullable() ? 1 : 0;
> columnMetas.add(new SelectedColumnMeta(false, false, false, false, nullable, 
> true, Integer.MAX_VALUE,
> fieldList.get(i).getName().toUpperCase(Locale.ROOT), 
> fieldList.get(i).getName().toUpperCase(Locale.ROOT), null, null,
> null, fieldList.get(i).getPrecision(), fieldList.get(i).getScale(), 
> fieldList.get(i).getDataType(),
> fieldList.get(i).getDataTypeName(), false, false, false));
> }
>  
> {code}
>  
> pushdown query:
> !c:\Users\Administrator\AppData\Local\YNote\data\weixinobU7VjlOcNgcm0ZbmNGYzYRPZ__4\401fe7a7ea05495799145f2d413b0da4\clipboard.png!
> cube query:
> !c:\Users\Administrator\AppData\Local\YNote\data\weixinobU7VjlOcNgcm0ZbmNGYzYRPZ__4\0ef826f0229f42b59c4ad946a08bf6f3\clipboard.png!
>  
> 我认为这个问题应该修复，因为在尝试对接superset时发现，有些此类框架会强校验元数据列的大小写，如果默认进行转换会导致无法适配，且cube 
> query和pushdown query查询的元数据列结果不一致这也是无法接受的。
>  
> 如果讨论有需要的话，我可以修复他，谢谢~



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (KYLIN-5215) The metadata information returned by Kylin pushdown query and Cube query is inconsistent

2022-08-02 Thread liyang (Jira) 


[ 
https://issues.apache.org/jira/browse/KYLIN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574176#comment-17574176
 ] 

liyang commented on KYLIN-5215:
---

Kylin 的对外元数据默认都是大写，也就是比如 hive 数据源中用的是小写的表名和列名，从 Kylin 暴漏出来后将都是大写。

估计这也就是为什么原来代码里用了 upperCase() 把下推返回 name & label 也变成默认大写。

例子中的情况比较神奇是，cube 查询返回的 name & label 为什么是小写的值得研究一下。

> The metadata information returned by Kylin pushdown query and Cube query is 
> inconsistent
> 
>
> Key: KYLIN-5215
> URL: https://issues.apache.org/jira/browse/KYLIN-5215
> Project: Kylin
>  Issue Type: Bug
>  Components: Spark Engine
>Affects Versions: all, v4.0.1
> Environment: kylin 4.0.1
>Reporter: gaogao
>Priority: Major
> Attachments: cube_query.png, pushdown_query.png
>
>
> 问题：
>  在下压查询中返回的结果  和  cube 查询的结果不一致，下压查询中  columnMetas 的  label 和 name 字段会默认转换成大写！
> 对应源码类
> PushDownRunnerSparkImpl  50行
> ：
>  
> {code:java}
> // fill in selected column meta
> for (int i = 0; i < columnCount; ++i) {
> int nullable = fieldList.get(i).isNullable() ? 1 : 0;
> columnMetas.add(new SelectedColumnMeta(false, false, false, false, nullable, 
> true, Integer.MAX_VALUE,
> fieldList.get(i).getName().toUpperCase(Locale.ROOT), 
> fieldList.get(i).getName().toUpperCase(Locale.ROOT), null, null,
> null, fieldList.get(i).getPrecision(), fieldList.get(i).getScale(), 
> fieldList.get(i).getDataType(),
> fieldList.get(i).getDataTypeName(), false, false, false));
> }
>  
> {code}
>  
> pushdown query:
> !c:\Users\Administrator\AppData\Local\YNote\data\weixinobU7VjlOcNgcm0ZbmNGYzYRPZ__4\401fe7a7ea05495799145f2d413b0da4\clipboard.png!
> cube query:
> !c:\Users\Administrator\AppData\Local\YNote\data\weixinobU7VjlOcNgcm0ZbmNGYzYRPZ__4\0ef826f0229f42b59c4ad946a08bf6f3\clipboard.png!
>  
> 我认为这个问题应该修复，因为在尝试对接superset时发现，有些此类框架会强校验元数据列的大小写，如果默认进行转换会导致无法适配，且cube 
> query和pushdown query查询的元数据列结果不一致这也是无法接受的。
>  
> 如果讨论有需要的话，我可以修复他，谢谢~



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (KYLIN-5215) The metadata information returned by Kylin pushdown query and Cube query is inconsistent

2022-07-31 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/KYLIN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573391#comment-17573391
 ] 

ASF GitHub Bot commented on KYLIN-5215:
---

liyang-kylin commented on PR #1900:
URL: https://github.com/apache/kylin/pull/1900#issuecomment-1200411047

   Thx for the pr, will come back with more comments in a few days.




> The metadata information returned by Kylin pushdown query and Cube query is 
> inconsistent
> 
>
> Key: KYLIN-5215
> URL: https://issues.apache.org/jira/browse/KYLIN-5215
> Project: Kylin
>  Issue Type: Bug
>  Components: Spark Engine
>Affects Versions: all, v4.0.1
> Environment: kylin 4.0.1
>Reporter: gaogao
>Priority: Major
> Attachments: cube_query.png, pushdown_query.png
>
>
> 问题：
>  在下压查询中返回的结果  和  cube 查询的结果不一致，下压查询中  columnMetas 的  label 和 name 字段会默认转换成大写！
> 对应源码类
> PushDownRunnerSparkImpl  50行
> ：
>  
> {code:java}
> // fill in selected column meta
> for (int i = 0; i < columnCount; ++i) {
> int nullable = fieldList.get(i).isNullable() ? 1 : 0;
> columnMetas.add(new SelectedColumnMeta(false, false, false, false, nullable, 
> true, Integer.MAX_VALUE,
> fieldList.get(i).getName().toUpperCase(Locale.ROOT), 
> fieldList.get(i).getName().toUpperCase(Locale.ROOT), null, null,
> null, fieldList.get(i).getPrecision(), fieldList.get(i).getScale(), 
> fieldList.get(i).getDataType(),
> fieldList.get(i).getDataTypeName(), false, false, false));
> }
>  
> {code}
>  
> pushdown query:
> !c:\Users\Administrator\AppData\Local\YNote\data\weixinobU7VjlOcNgcm0ZbmNGYzYRPZ__4\401fe7a7ea05495799145f2d413b0da4\clipboard.png!
> cube query:
> !c:\Users\Administrator\AppData\Local\YNote\data\weixinobU7VjlOcNgcm0ZbmNGYzYRPZ__4\0ef826f0229f42b59c4ad946a08bf6f3\clipboard.png!
>  
> 我认为这个问题应该修复，因为在尝试对接superset时发现，有些此类框架会强校验元数据列的大小写，如果默认进行转换会导致无法适配，且cube 
> query和pushdown query查询的元数据列结果不一致这也是无法接受的。
>  
> 如果讨论有需要的话，我可以修复他，谢谢~



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (KYLIN-5215) The metadata information returned by Kylin pushdown query and Cube query is inconsistent

2022-07-31 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/KYLIN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573390#comment-17573390
 ] 

ASF GitHub Bot commented on KYLIN-5215:
---

liyang-kylin commented on PR #1900:
URL: https://github.com/apache/kylin/pull/1900#issuecomment-1200410917

   Thx for the pr, will come back with more comments in a few days.




> The metadata information returned by Kylin pushdown query and Cube query is 
> inconsistent
> 
>
> Key: KYLIN-5215
> URL: https://issues.apache.org/jira/browse/KYLIN-5215
> Project: Kylin
>  Issue Type: Bug
>  Components: Spark Engine
>Affects Versions: all, v4.0.1
> Environment: kylin 4.0.1
>Reporter: gaogao
>Priority: Major
> Attachments: cube_query.png, pushdown_query.png
>
>
> 问题：
>  在下压查询中返回的结果  和  cube 查询的结果不一致，下压查询中  columnMetas 的  label 和 name 字段会默认转换成大写！
> 对应源码类
> PushDownRunnerSparkImpl  50行
> ：
>  
> {code:java}
> // fill in selected column meta
> for (int i = 0; i < columnCount; ++i) {
> int nullable = fieldList.get(i).isNullable() ? 1 : 0;
> columnMetas.add(new SelectedColumnMeta(false, false, false, false, nullable, 
> true, Integer.MAX_VALUE,
> fieldList.get(i).getName().toUpperCase(Locale.ROOT), 
> fieldList.get(i).getName().toUpperCase(Locale.ROOT), null, null,
> null, fieldList.get(i).getPrecision(), fieldList.get(i).getScale(), 
> fieldList.get(i).getDataType(),
> fieldList.get(i).getDataTypeName(), false, false, false));
> }
>  
> {code}
>  
> pushdown query:
> !c:\Users\Administrator\AppData\Local\YNote\data\weixinobU7VjlOcNgcm0ZbmNGYzYRPZ__4\401fe7a7ea05495799145f2d413b0da4\clipboard.png!
> cube query:
> !c:\Users\Administrator\AppData\Local\YNote\data\weixinobU7VjlOcNgcm0ZbmNGYzYRPZ__4\0ef826f0229f42b59c4ad946a08bf6f3\clipboard.png!
>  
> 我认为这个问题应该修复，因为在尝试对接superset时发现，有些此类框架会强校验元数据列的大小写，如果默认进行转换会导致无法适配，且cube 
> query和pushdown query查询的元数据列结果不一致这也是无法接受的。
>  
> 如果讨论有需要的话，我可以修复他，谢谢~



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (KYLIN-5215) The metadata information returned by Kylin pushdown query and Cube query is inconsistent

2022-07-25 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/KYLIN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571180#comment-17571180
 ] 

ASF GitHub Bot commented on KYLIN-5215:
---

gaogao110 opened a new pull request, #1900:
URL: https://github.com/apache/kylin/pull/1900

   Fix columnMetas information returned by Kylin pushdown query and Cube query 
is inconsistent
   ## Proposed changes
   1. Delete toUpperCase  from  PushDownRunnerSparkImpl.executeQuery 
   
   `columnMetas.add(new SelectedColumnMeta(false, false, false, 
false, nullable, true, Integer.MAX_VALUE,
   fieldList.get(i).getName().toUpperCase(Locale.ROOT), 
fieldList.get(i).getName().toUpperCase(Locale.ROOT), null, null,
   null, fieldList.get(i).getPrecision(), 
fieldList.get(i).getScale(), fieldList.get(i).getDataType(),
   fieldList.get(i).getDataTypeName(), false, false, 
false));`
   ## Checklist
   -  I have create an issue on [Kylin's 
jira](https://issues.apache.org/jira/projects/KYLIN/issues/KYLIN-5215), and 
have described the bug/feature there in detail
   
   
   




> The metadata information returned by Kylin pushdown query and Cube query is 
> inconsistent
> 
>
> Key: KYLIN-5215
> URL: https://issues.apache.org/jira/browse/KYLIN-5215
> Project: Kylin
>  Issue Type: Bug
>  Components: Spark Engine
>Affects Versions: all, v4.0.1
> Environment: kylin 4.0.1
>Reporter: gaogao
>Priority: Major
> Attachments: cube_query.png, pushdown_query.png
>
>
> 问题：
>  在下压查询中返回的结果  和  cube 查询的结果不一致，下压查询中  columnMetas 的  label 和 name 字段会默认转换成大写！
> 对应源码类
> PushDownRunnerSparkImpl  50行
> ：
>  
> {code:java}
> // fill in selected column meta
> for (int i = 0; i < columnCount; ++i) {
> int nullable = fieldList.get(i).isNullable() ? 1 : 0;
> columnMetas.add(new SelectedColumnMeta(false, false, false, false, nullable, 
> true, Integer.MAX_VALUE,
> fieldList.get(i).getName().toUpperCase(Locale.ROOT), 
> fieldList.get(i).getName().toUpperCase(Locale.ROOT), null, null,
> null, fieldList.get(i).getPrecision(), fieldList.get(i).getScale(), 
> fieldList.get(i).getDataType(),
> fieldList.get(i).getDataTypeName(), false, false, false));
> }
>  
> {code}
>  
> pushdown query:
> !c:\Users\Administrator\AppData\Local\YNote\data\weixinobU7VjlOcNgcm0ZbmNGYzYRPZ__4\401fe7a7ea05495799145f2d413b0da4\clipboard.png!
> cube query:
> !c:\Users\Administrator\AppData\Local\YNote\data\weixinobU7VjlOcNgcm0ZbmNGYzYRPZ__4\0ef826f0229f42b59c4ad946a08bf6f3\clipboard.png!
>  
> 我认为这个问题应该修复，因为在尝试对接superset时发现，有些此类框架会强校验元数据列的大小写，如果默认进行转换会导致无法适配，且cube 
> query和pushdown query查询的元数据列结果不一致这也是无法接受的。
>  
> 如果讨论有需要的话，我可以修复他，谢谢~



--
This message was sent by Atlassian Jira
(v8.20.10#820010)