[jira] [Commented] (KYLIN-3635) Percentile calculation on Spark engine is wrong
[ https://issues.apache.org/jira/browse/KYLIN-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16665130#comment-16665130 ] jiatao.tao commented on KYLIN-3635: --- Hi Shaofeng, do you think we should provide a stronger constraint about this? I've tried to make method reset() abstract so that the subclasses have to implement this, but I found most of the implementation is still empty. What's your opinion about this? > Percentile calculation on Spark engine is wrong > --- > > Key: KYLIN-3635 > URL: https://issues.apache.org/jira/browse/KYLIN-3635 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Affects Versions: v2.3.0, v2.3.1, v2.4.0, v2.3.2, v2.4.1, v2.5.0 >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Major > Fix For: v2.4.2, v2.5.1 > > > As titled; Received reporting that percentile result is wrong when using > Spark engine. Checked the code and found the object was reused and not reset, > the problem won't happen on normal MR as it would be serialized before next > call, but would be a problem on Spark, as the object was cached in-memory and > be serialized later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (KYLIN-3630) Remove unused fields in the implementations of MeasureType
[
https://issues.apache.org/jira/browse/KYLIN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao closed KYLIN-3630.
-
thanks shaofeng.
> Remove unused fields in the implementations of MeasureType
> --
>
> Key: KYLIN-3630
> URL: https://issues.apache.org/jira/browse/KYLIN-3630
> Project: Kylin
> Issue Type: Improvement
>Reporter: jiatao.tao
>Assignee: jiatao.tao
>Priority: Minor
> Fix For: v2.6.0
>
>
> In code RawMeasureType#RawMeasureType, we can see that both funcName and
> dataType are use, and this can be remove cuz there's no constraint on its
> constructor.
> {code:java}
> @SuppressWarnings("unused")
> private final DataType dataType;
> public RawMeasureType(String funcName, DataType dataType) {
> this.dataType = dataType;
> }
> {code}
> And after we done that, we can more easily test MeasureType without passing
> in meaningless args(AggregatorMemEstimateTest#112).
> Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType, other
> implementations can be cleaner.
> {code:java}
> @Override
> public MeasureType createMeasureType(String funcName,
> DataType dataType) {
> return new DimCountDistinctMeasureType();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (KYLIN-3630) Remove unused fields in the implementations of MeasureType
[
https://issues.apache.org/jira/browse/KYLIN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao resolved KYLIN-3630.
---
Resolution: Fixed
> Remove unused fields in the implementations of MeasureType
> --
>
> Key: KYLIN-3630
> URL: https://issues.apache.org/jira/browse/KYLIN-3630
> Project: Kylin
> Issue Type: Improvement
>Reporter: jiatao.tao
>Assignee: jiatao.tao
>Priority: Minor
> Fix For: v2.6.0
>
>
> In code RawMeasureType#RawMeasureType, we can see that both funcName and
> dataType are use, and this can be remove cuz there's no constraint on its
> constructor.
> {code:java}
> @SuppressWarnings("unused")
> private final DataType dataType;
> public RawMeasureType(String funcName, DataType dataType) {
> this.dataType = dataType;
> }
> {code}
> And after we done that, we can more easily test MeasureType without passing
> in meaningless args(AggregatorMemEstimateTest#112).
> Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType, other
> implementations can be cleaner.
> {code:java}
> @Override
> public MeasureType createMeasureType(String funcName,
> DataType dataType) {
> return new DimCountDistinctMeasureType();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3630) remove unused fields in the implementations of MeasureType
[
https://issues.apache.org/jira/browse/KYLIN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao updated KYLIN-3630:
--
Description:
In code RawMeasureType#RawMeasureType, we can see that both funcName and
dataType are use, and this can be remove cuz there's no constraint on its
constructor.
{code:java}
@SuppressWarnings("unused")
private final DataType dataType;
public RawMeasureType(String funcName, DataType dataType) {
this.dataType = dataType;
}
{code}
And after we done that, we can more easily test MeasureType without passing in
meaningless args(AggregatorMemEstimateTest#112).
Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType, other
implementations can be cleaner.
{code:java}
@Override
public MeasureType createMeasureType(String funcName, DataType
dataType) {
return new DimCountDistinctMeasureType();
}
{code}
was:
In code RawMeasureType#RawMeasureType, we can see that both funcName and
dataType are use, and this can be remove cuz there's no constraint on its
constructor.
{code:java}
@SuppressWarnings("unused")
private final DataType dataType;
public RawMeasureType(String funcName, DataType dataType) {
this.dataType = dataType;
}
{code}
And after we done that, we can more easily test MeasureType without passing in
meaningless args.
Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType, other
implementations can be cleaner.
{code:java}
@Override
public MeasureType createMeasureType(String funcName, DataType
dataType) {
return new DimCountDistinctMeasureType();
}
{code}
> remove unused fields in the implementations of MeasureType
> --
>
> Key: KYLIN-3630
> URL: https://issues.apache.org/jira/browse/KYLIN-3630
> Project: Kylin
> Issue Type: Improvement
>Reporter: jiatao.tao
>Assignee: jiatao.tao
>Priority: Minor
>
> In code RawMeasureType#RawMeasureType, we can see that both funcName and
> dataType are use, and this can be remove cuz there's no constraint on its
> constructor.
> {code:java}
> @SuppressWarnings("unused")
> private final DataType dataType;
> public RawMeasureType(String funcName, DataType dataType) {
> this.dataType = dataType;
> }
> {code}
> And after we done that, we can more easily test MeasureType without passing
> in meaningless args(AggregatorMemEstimateTest#112).
> Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType, other
> implementations can be cleaner.
> {code:java}
> @Override
> public MeasureType createMeasureType(String funcName,
> DataType dataType) {
> return new DimCountDistinctMeasureType();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3630) remove unused fields in the implementations of MeasureType
[
https://issues.apache.org/jira/browse/KYLIN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao updated KYLIN-3630:
--
Description:
In code RawMeasureType#RawMeasureType, we can see that both funcName and
dataType are use, and this can be remove cuz there's no constraint on its
constructor.
{code:java}
@SuppressWarnings("unused")
private final DataType dataType;
public RawMeasureType(String funcName, DataType dataType) {
this.dataType = dataType;
}
{code}
And after we done that, we can mo easily test MeasureType without passing in
meaningless args.
Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType, other
{code:java}
@Override
public MeasureType createMeasureType(String funcName, DataType
dataType) {
return new DimCountDistinctMeasureType();
}
{code}
was:
Like in code org.apache.kylin.measure.raw.RawMeasureType#RawMeasureType, we can
see that both funcName and dataType are use, and this can be remove cuz
there's no constraint on its constructor.
And after we done that, we can mo easily test MeasureType without passing in
meaningless args.
Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType
> remove unused fields in the implementations of MeasureType
> --
>
> Key: KYLIN-3630
> URL: https://issues.apache.org/jira/browse/KYLIN-3630
> Project: Kylin
> Issue Type: Improvement
>Reporter: jiatao.tao
>Assignee: jiatao.tao
>Priority: Minor
>
> In code RawMeasureType#RawMeasureType, we can see that both funcName and
> dataType are use, and this can be remove cuz there's no constraint on its
> constructor.
> {code:java}
> @SuppressWarnings("unused")
> private final DataType dataType;
> public RawMeasureType(String funcName, DataType dataType) {
> this.dataType = dataType;
> }
> {code}
> And after we done that, we can mo easily test MeasureType without passing in
> meaningless args.
> Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType, other
> {code:java}
> @Override
> public MeasureType createMeasureType(String funcName,
> DataType dataType) {
> return new DimCountDistinctMeasureType();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3630) remove unused fields in the implementations of MeasureType
[
https://issues.apache.org/jira/browse/KYLIN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao updated KYLIN-3630:
--
Description:
In code RawMeasureType#RawMeasureType, we can see that both funcName and
dataType are use, and this can be remove cuz there's no constraint on its
constructor.
{code:java}
@SuppressWarnings("unused")
private final DataType dataType;
public RawMeasureType(String funcName, DataType dataType) {
this.dataType = dataType;
}
{code}
And after we done that, we can more easily test MeasureType without passing in
meaningless args.
Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType, other
implementations can be cleaner.
{code:java}
@Override
public MeasureType createMeasureType(String funcName, DataType
dataType) {
return new DimCountDistinctMeasureType();
}
{code}
was:
In code RawMeasureType#RawMeasureType, we can see that both funcName and
dataType are use, and this can be remove cuz there's no constraint on its
constructor.
{code:java}
@SuppressWarnings("unused")
private final DataType dataType;
public RawMeasureType(String funcName, DataType dataType) {
this.dataType = dataType;
}
{code}
And after we done that, we can mo easily test MeasureType without passing in
meaningless args.
Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType, other
{code:java}
@Override
public MeasureType createMeasureType(String funcName, DataType
dataType) {
return new DimCountDistinctMeasureType();
}
{code}
> remove unused fields in the implementations of MeasureType
> --
>
> Key: KYLIN-3630
> URL: https://issues.apache.org/jira/browse/KYLIN-3630
> Project: Kylin
> Issue Type: Improvement
>Reporter: jiatao.tao
>Assignee: jiatao.tao
>Priority: Minor
>
> In code RawMeasureType#RawMeasureType, we can see that both funcName and
> dataType are use, and this can be remove cuz there's no constraint on its
> constructor.
> {code:java}
> @SuppressWarnings("unused")
> private final DataType dataType;
> public RawMeasureType(String funcName, DataType dataType) {
> this.dataType = dataType;
> }
> {code}
> And after we done that, we can more easily test MeasureType without passing
> in meaningless args.
> Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType, other
> implementations can be cleaner.
> {code:java}
> @Override
> public MeasureType createMeasureType(String funcName,
> DataType dataType) {
> return new DimCountDistinctMeasureType();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3630) remove unused fields in the implementations of MeasureType
[ https://issues.apache.org/jira/browse/KYLIN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3630: -- Attachment: (was: screenshot-1.png) > remove unused fields in the implementations of MeasureType > -- > > Key: KYLIN-3630 > URL: https://issues.apache.org/jira/browse/KYLIN-3630 > Project: Kylin > Issue Type: Improvement >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Minor > > Like in code org.apache.kylin.measure.raw.RawMeasureType#RawMeasureType, we > can see that both funcName and dataType are use, and this can be remove cuz > there's no constraint on its constructor. > And after we done that, we can mo easily test MeasureType without passing in > meaningless args. > Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3630) remove unused fields in the implementations of MeasureType
[ https://issues.apache.org/jira/browse/KYLIN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3630: -- Attachment: (was: image-2018-10-14-18-56-29-010.png) > remove unused fields in the implementations of MeasureType > -- > > Key: KYLIN-3630 > URL: https://issues.apache.org/jira/browse/KYLIN-3630 > Project: Kylin > Issue Type: Improvement >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Minor > > Like in code org.apache.kylin.measure.raw.RawMeasureType#RawMeasureType, we > can see that both funcName and dataType are use, and this can be remove cuz > there's no constraint on its constructor. > And after we done that, we can mo easily test MeasureType without passing in > meaningless args. > Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3630) remove unused fields in the implementations of MeasureType
[ https://issues.apache.org/jira/browse/KYLIN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3630: -- Description: Like in code org.apache.kylin.measure.raw.RawMeasureType#RawMeasureType, we can see that both funcName and dataType are use, and this can be remove cuz there's no constraint on its constructor. And after we done that, we can mo easily test MeasureType without passing in meaningless args. Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType was: Like in code org.apache.kylin.measure.raw.RawMeasureType#RawMeasureType, we can see that both funcName and dataType are use, and this can be remove cuz there's no constraint on its constructor. And after we done that, we can mo easily test MeasureType without passing in meaningless args. !screenshot-1.png! Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType !image-2018-10-14-18-56-29-010.png! > remove unused fields in the implementations of MeasureType > -- > > Key: KYLIN-3630 > URL: https://issues.apache.org/jira/browse/KYLIN-3630 > Project: Kylin > Issue Type: Improvement >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Minor > > Like in code org.apache.kylin.measure.raw.RawMeasureType#RawMeasureType, we > can see that both funcName and dataType are use, and this can be remove cuz > there's no constraint on its constructor. > And after we done that, we can mo easily test MeasureType without passing in > meaningless args. > Like codes in org.apache.kylin.measure.dim.DimCountDistinctMeasureType -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3630) remove unused fields in the implementations of MeasureType
[ https://issues.apache.org/jira/browse/KYLIN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3630: -- Attachment: screenshot-1.png > remove unused fields in the implementations of MeasureType > -- > > Key: KYLIN-3630 > URL: https://issues.apache.org/jira/browse/KYLIN-3630 > Project: Kylin > Issue Type: Improvement >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Minor > Attachments: image-2018-10-14-18-56-29-010.png, screenshot-1.png > > > !image-2018-10-14-18-56-29-010.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3630) remove unused fields in the implementations of MeasureType
jiatao.tao created KYLIN-3630: - Summary: remove unused fields in the implementations of MeasureType Key: KYLIN-3630 URL: https://issues.apache.org/jira/browse/KYLIN-3630 Project: Kylin Issue Type: Improvement Reporter: jiatao.tao Assignee: jiatao.tao Attachments: image-2018-10-14-18-56-29-010.png !image-2018-10-14-18-56-29-010.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3562) TS conflict when kylin update metadata in HBase
[
https://issues.apache.org/jira/browse/KYLIN-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16627171#comment-16627171
]
jiatao.tao edited comment on KYLIN-3562 at 9/25/18 11:27 AM:
-
[~baibaichen] [~chitin] As a workaround, I think [~yichen.zhou]'s solution
really work.
Try both
{code:java}
$KYLIN_HOME/bin/metastore.sh remove /user/ADMIN
{code}
{code:java}
$KYLIN_HOME/bin/metastore.sh remove /user/admin
{code}
or
{code:java}
$KYLIN_HOME/bin/metastore.sh remove /user
{code}
but this will remove all users.
was (Author: aron.tao):
[~baibaichen] [~chitin] As a workaround, I think [~yichen.zhou]'s solution
really work.The premise is you don't have many users in metastore(Under normal
circumstances, you can't. Cuz now we can not add users manually :), the only
way to add users is from LDAP).
> TS conflict when kylin update metadata in HBase
> ---
>
> Key: KYLIN-3562
> URL: https://issues.apache.org/jira/browse/KYLIN-3562
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Lingang Deng
>Priority: Major
> Attachments: image-2018-09-17-16-40-56-212.png,
> image-2018-09-25-15-03-51-009.png, image-2018-09-25-16-43-50-277.png
>
>
> Error log was as follows,
> {code:java}
> org.apache.kylin.common.persistence.WriteConflictException: Overwriting
> conflict /user/admin, expect old TS 1536928877043, but it is 1536928907207
> at
> org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:325)
> at
> org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:318)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:303)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:282)
> at
> org.apache.kylin.metadata.cachesync.CachedCrudAssist.save(CachedCrudAssist.java:192){code}
>
> what disturbs me the most was that the error was happened several hours,
> then my all build job and query job failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3562) TS conflict when kylin update metadata in HBase
[
https://issues.apache.org/jira/browse/KYLIN-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16627171#comment-16627171
]
jiatao.tao edited comment on KYLIN-3562 at 9/25/18 11:16 AM:
-
[~baibaichen] [~chitin] As a workaround, I think [~yichen.zhou]'s solution
really work.The premise is you don't have many users in metastore(Under normal
circumstances, you can't. Cuz now we can not add users manually :), the only
way to add users is from LDAP).
was (Author: aron.tao):
[~baibaichen] [~chitin] As a workaround, I think [~yichen.zhou]'s solution
really work.The premise is you didn't have many users in metastore(you can't
cuz now we can not add users manually :), the only way to add users is from
LDAP).
> TS conflict when kylin update metadata in HBase
> ---
>
> Key: KYLIN-3562
> URL: https://issues.apache.org/jira/browse/KYLIN-3562
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Lingang Deng
>Priority: Major
> Attachments: image-2018-09-17-16-40-56-212.png,
> image-2018-09-25-15-03-51-009.png, image-2018-09-25-16-43-50-277.png
>
>
> Error log was as follows,
> {code:java}
> org.apache.kylin.common.persistence.WriteConflictException: Overwriting
> conflict /user/admin, expect old TS 1536928877043, but it is 1536928907207
> at
> org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:325)
> at
> org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:318)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:303)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:282)
> at
> org.apache.kylin.metadata.cachesync.CachedCrudAssist.save(CachedCrudAssist.java:192){code}
>
> what disturbs me the most was that the error was happened several hours,
> then my all build job and query job failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3562) TS conflict when kylin update metadata in HBase
[
https://issues.apache.org/jira/browse/KYLIN-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16627165#comment-16627165
]
jiatao.tao edited comment on KYLIN-3562 at 9/25/18 11:15 AM:
-
Hi [~baibaichen], I think your thought about
KylinAuthenticationProvider.authenticate is pretty right. The updating
operation is pretty confusing and useless, I think may be remove this is the
right path.
For KylinUserService, it actually just rename UserService(you can see
UserService's git log), we extracted this for preparing better LDAP support.
was (Author: aron.tao):
Hi [~baibaichen], I think your thought about
KylinAuthenticationProvider.authenticate is right. The updating operation is
pretty confusing and useless, I think may be remove this is the right path.
For KylinUserService, it actually just rename UserService(you can see
UserService's git log), we extracted this for better LDAP support preparation.
> TS conflict when kylin update metadata in HBase
> ---
>
> Key: KYLIN-3562
> URL: https://issues.apache.org/jira/browse/KYLIN-3562
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Lingang Deng
>Priority: Major
> Attachments: image-2018-09-17-16-40-56-212.png,
> image-2018-09-25-15-03-51-009.png, image-2018-09-25-16-43-50-277.png
>
>
> Error log was as follows,
> {code:java}
> org.apache.kylin.common.persistence.WriteConflictException: Overwriting
> conflict /user/admin, expect old TS 1536928877043, but it is 1536928907207
> at
> org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:325)
> at
> org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:318)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:303)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:282)
> at
> org.apache.kylin.metadata.cachesync.CachedCrudAssist.save(CachedCrudAssist.java:192){code}
>
> what disturbs me the most was that the error was happened several hours,
> then my all build job and query job failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3562) TS conflict when kylin update metadata in HBase
[
https://issues.apache.org/jira/browse/KYLIN-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16627171#comment-16627171
]
jiatao.tao commented on KYLIN-3562:
---
[~baibaichen] [~chitin] As a workaround, I think [~yichen.zhou]'s solution
really work.The premise is you didn't have many users in metastore(you can't
cuz now we can not add users manually :), the only way to add users is from
LDAP).
> TS conflict when kylin update metadata in HBase
> ---
>
> Key: KYLIN-3562
> URL: https://issues.apache.org/jira/browse/KYLIN-3562
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Lingang Deng
>Priority: Major
> Attachments: image-2018-09-17-16-40-56-212.png,
> image-2018-09-25-15-03-51-009.png, image-2018-09-25-16-43-50-277.png
>
>
> Error log was as follows,
> {code:java}
> org.apache.kylin.common.persistence.WriteConflictException: Overwriting
> conflict /user/admin, expect old TS 1536928877043, but it is 1536928907207
> at
> org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:325)
> at
> org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:318)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:303)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:282)
> at
> org.apache.kylin.metadata.cachesync.CachedCrudAssist.save(CachedCrudAssist.java:192){code}
>
> what disturbs me the most was that the error was happened several hours,
> then my all build job and query job failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3562) TS conflict when kylin update metadata in HBase
[
https://issues.apache.org/jira/browse/KYLIN-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16627165#comment-16627165
]
jiatao.tao commented on KYLIN-3562:
---
Hi [~baibaichen], I think your thought about
KylinAuthenticationProvider.authenticate is right. The updating operation is
pretty confusing and useless, I think may be remove this is the right path.
For KylinUserService, it actually just rename UserService(you can see
UserService's git log), we extracted this for better LDAP support preparation.
> TS conflict when kylin update metadata in HBase
> ---
>
> Key: KYLIN-3562
> URL: https://issues.apache.org/jira/browse/KYLIN-3562
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Lingang Deng
>Priority: Major
> Attachments: image-2018-09-17-16-40-56-212.png,
> image-2018-09-25-15-03-51-009.png, image-2018-09-25-16-43-50-277.png
>
>
> Error log was as follows,
> {code:java}
> org.apache.kylin.common.persistence.WriteConflictException: Overwriting
> conflict /user/admin, expect old TS 1536928877043, but it is 1536928907207
> at
> org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:325)
> at
> org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:318)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:303)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:282)
> at
> org.apache.kylin.metadata.cachesync.CachedCrudAssist.save(CachedCrudAssist.java:192){code}
>
> what disturbs me the most was that the error was happened several hours,
> then my all build job and query job failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3592) Synchronized should be placed after static in declaration
[ https://issues.apache.org/jira/browse/KYLIN-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626898#comment-16626898 ] jiatao.tao edited comment on KYLIN-3592 at 9/25/18 7:14 AM: In kylin, some places occur `synchronized static`, some occur `static synchronized`, should be unified. see in : https://docs.oracle.com/javase/specs/jls/se8/html/jls-8.html The order is: !screenshot-1.png! was (Author: aron.tao): see in : https://docs.oracle.com/javase/specs/jls/se8/html/jls-8.html The order is: !screenshot-1.png! In kylin, some places occur `synchronized static`, some occur `static synchronized`, should be unified. > Synchronized should be placed after static in declaration > - > > Key: KYLIN-3592 > URL: https://issues.apache.org/jira/browse/KYLIN-3592 > Project: Kylin > Issue Type: Improvement >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Trivial > Attachments: screenshot-1.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3592) Synchronized should be placed after static in declaration
[ https://issues.apache.org/jira/browse/KYLIN-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626898#comment-16626898 ] jiatao.tao commented on KYLIN-3592: --- see in : https://docs.oracle.com/javase/specs/jls/se8/html/jls-8.html The order is: !screenshot-1.png! In kylin, some places occur `synchronized static`, some occur `static synchronized`, should be unified. > Synchronized should be placed after static in declaration > - > > Key: KYLIN-3592 > URL: https://issues.apache.org/jira/browse/KYLIN-3592 > Project: Kylin > Issue Type: Improvement >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Trivial > Attachments: screenshot-1.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3592) Synchronized should be placed after static in declaration
[ https://issues.apache.org/jira/browse/KYLIN-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3592: -- Attachment: screenshot-1.png > Synchronized should be placed after static in declaration > - > > Key: KYLIN-3592 > URL: https://issues.apache.org/jira/browse/KYLIN-3592 > Project: Kylin > Issue Type: Improvement >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Trivial > Attachments: screenshot-1.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3588) Potentially duplicate put to RemoveBlackoutRealizationsRule#filters map
[
https://issues.apache.org/jira/browse/KYLIN-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626866#comment-16626866
]
jiatao.tao commented on KYLIN-3588:
---
seems a synchronized can deal with this.
> Potentially duplicate put to RemoveBlackoutRealizationsRule#filters map
> ---
>
> Key: KYLIN-3588
> URL: https://issues.apache.org/jira/browse/KYLIN-3588
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
>
> Here is related code:
> {code}
> IRealizationFilter filter = filters.get(conf);
> if (filter == null) {
> try {
> Class clz =
> ClassUtil.forName(conf.getQueryRealizationFilter(), IRealizationFilter.class);
> filter =
> clz.getConstructor(KylinConfig.class).newInstance(conf);
> } catch (Exception e) {
> throw new RuntimeException(e);
> }
> filters.put(conf, filter);
> {code}
> Two threads may find out there is no filter for conf, they would proceed with
> creation of IRealizationFilter instances and try to put into the map.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3588) Potentially duplicate put to RemoveBlackoutRealizationsRule#filters map
[
https://issues.apache.org/jira/browse/KYLIN-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao reassigned KYLIN-3588:
-
Assignee: jiatao.tao
> Potentially duplicate put to RemoveBlackoutRealizationsRule#filters map
> ---
>
> Key: KYLIN-3588
> URL: https://issues.apache.org/jira/browse/KYLIN-3588
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
>
> Here is related code:
> {code}
> IRealizationFilter filter = filters.get(conf);
> if (filter == null) {
> try {
> Class clz =
> ClassUtil.forName(conf.getQueryRealizationFilter(), IRealizationFilter.class);
> filter =
> clz.getConstructor(KylinConfig.class).newInstance(conf);
> } catch (Exception e) {
> throw new RuntimeException(e);
> }
> filters.put(conf, filter);
> {code}
> Two threads may find out there is no filter for conf, they would proceed with
> creation of IRealizationFilter instances and try to put into the map.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3575) Unclosed Connection in DriverTest
[
https://issues.apache.org/jira/browse/KYLIN-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625032#comment-16625032
]
jiatao.tao commented on KYLIN-3575:
---
Thanks!
> Unclosed Connection in DriverTest
> -
>
> Key: KYLIN-3575
> URL: https://issues.apache.org/jira/browse/KYLIN-3575
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
>
> In testCalciteProps :
> {code}
> KylinConnection conn = (KylinConnection)
> driver.connect("jdbc:kylin:test_url/test_db", props);
> {code}
> The connection should be closed upon return.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3575) Unclosed Connection in DriverTest
[
https://issues.apache.org/jira/browse/KYLIN-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao reassigned KYLIN-3575:
-
Assignee: jiatao.tao
> Unclosed Connection in DriverTest
> -
>
> Key: KYLIN-3575
> URL: https://issues.apache.org/jira/browse/KYLIN-3575
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
>
> In testCalciteProps :
> {code}
> KylinConnection conn = (KylinConnection)
> driver.connect("jdbc:kylin:test_url/test_db", props);
> {code}
> The connection should be closed upon return.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3492) typo in KylinConfigBase.getDefaultVarcharPrecision
jiatao.tao created KYLIN-3492: - Summary: typo in KylinConfigBase.getDefaultVarcharPrecision Key: KYLIN-3492 URL: https://issues.apache.org/jira/browse/KYLIN-3492 Project: Kylin Issue Type: Improvement Reporter: jiatao.tao Assignee: jiatao.tao -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3379) timestampadd bug fix and add test
[ https://issues.apache.org/jira/browse/KYLIN-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao reassigned KYLIN-3379: - Assignee: jiatao.tao (was: hongbin ma) > timestampadd bug fix and add test > - > > Key: KYLIN-3379 > URL: https://issues.apache.org/jira/browse/KYLIN-3379 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.3.1 >Reporter: hongbin ma >Assignee: jiatao.tao >Priority: Major > Fix For: v2.5.0 > > > complex cases like > timestampadd(MONTH,23,test_kylin_fact.cal_dt) or > timestampadd(MONTH,-23,test_kylin_fact.cal_dt) is not covered. > > And my tests shows this kind of queries will fail IT. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3379) timestampadd bug fix and add test
[ https://issues.apache.org/jira/browse/KYLIN-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3379: -- Summary: timestampadd bug fix and add test (was: timestampadd test coverage is not enough) > timestampadd bug fix and add test > - > > Key: KYLIN-3379 > URL: https://issues.apache.org/jira/browse/KYLIN-3379 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.3.1 >Reporter: hongbin ma >Assignee: hongbin ma >Priority: Major > Fix For: v2.5.0 > > > complex cases like > timestampadd(MONTH,23,test_kylin_fact.cal_dt) or > timestampadd(MONTH,-23,test_kylin_fact.cal_dt) is not covered. > > And my tests shows this kind of queries will fail IT. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3417) Consider replacing ReentrantReadWriteLock with StampedLock
[
https://issues.apache.org/jira/browse/KYLIN-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569436#comment-16569436
]
jiatao.tao commented on KYLIN-3417:
---
Hi [~yuzhih...@gmail.com] , nowadays, kylin's minimum java version requirement
is sitll 1.7, when it comes to 1.8, we can replace ReentrantReadWriteLock in
class AutoReadWriteLock with StampedLock.
> Consider replacing ReentrantReadWriteLock with StampedLock
> --
>
> Key: KYLIN-3417
> URL: https://issues.apache.org/jira/browse/KYLIN-3417
> Project: Kylin
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Major
> Fix For: v2.5.0
>
>
> ReentrantReadWriteLock's are only the right solution when there is long hold
> time due to expensive I/O.
> It is expensive for readers.
> We should see if the lighter {{StampedLock}} can be used instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3483) Imprecise comparison between double and integer division
[
https://issues.apache.org/jira/browse/KYLIN-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569435#comment-16569435
]
jiatao.tao edited comment on KYLIN-3483 at 8/5/18 11:01 AM:
Thanks Ted Yu! Here's the commit https://github.com/apache/kylin/pull/187/files.
was (Author: aron.tao):
Thanks Ted Yu! here's the https://github.com/apache/kylin/pull/187/files.
> Imprecise comparison between double and integer division
>
>
> Key: KYLIN-3483
> URL: https://issues.apache.org/jira/browse/KYLIN-3483
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
>
> In HLLCSerializer :
> {code}
> || averageNumOfElementsInCounter < (m - 5) / (1 +
> registerIndexSize)) {
> {code}
> The integer division on the right side of < would lose remainder, or
> fractional part of the quotient.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3483) Imprecise comparison between double and integer division
[
https://issues.apache.org/jira/browse/KYLIN-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569435#comment-16569435
]
jiatao.tao commented on KYLIN-3483:
---
Thanks Ted Yu! here's the https://github.com/apache/kylin/pull/187/files.
> Imprecise comparison between double and integer division
>
>
> Key: KYLIN-3483
> URL: https://issues.apache.org/jira/browse/KYLIN-3483
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
>
> In HLLCSerializer :
> {code}
> || averageNumOfElementsInCounter < (m - 5) / (1 +
> registerIndexSize)) {
> {code}
> The integer division on the right side of < would lose remainder, or
> fractional part of the quotient.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3482) Unclosed SetAndUnsetThreadLocalConfig in SparkCubingByLayer
[
https://issues.apache.org/jira/browse/KYLIN-3482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao reassigned KYLIN-3482:
-
Assignee: jiatao.tao
> Unclosed SetAndUnsetThreadLocalConfig in SparkCubingByLayer
> ---
>
> Key: KYLIN-3482
> URL: https://issues.apache.org/jira/browse/KYLIN-3482
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
>
> Here is related code:
> {code}
> KylinConfig kylinConfig =
> AbstractHadoopJob.loadKylinConfigFromHdfs(sConf, metaUrl);
>
> KylinConfig.setAndUnsetThreadLocalConfig(kylinConfig);
> {code}
> The return value from setAndUnsetThreadLocalConfig should be closed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3483) Imprecise comparison between double and integer division
[
https://issues.apache.org/jira/browse/KYLIN-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao reassigned KYLIN-3483:
-
Assignee: jiatao.tao
> Imprecise comparison between double and integer division
>
>
> Key: KYLIN-3483
> URL: https://issues.apache.org/jira/browse/KYLIN-3483
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
>
> In HLLCSerializer :
> {code}
> || averageNumOfElementsInCounter < (m - 5) / (1 +
> registerIndexSize)) {
> {code}
> The integer division on the right side of < would lose remainder, or
> fractional part of the quotient.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3310) Use lint for maven-compiler-plugin
[
https://issues.apache.org/jira/browse/KYLIN-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16543937#comment-16543937
]
jiatao.tao commented on KYLIN-3310:
---
NIce! I'll give a try! Thanks.
> Use lint for maven-compiler-plugin
> --
>
> Key: KYLIN-3310
> URL: https://issues.apache.org/jira/browse/KYLIN-3310
> Project: Kylin
> Issue Type: Improvement
> Components: Tools, Build and Test
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Major
>
> lint helps identify structural problems.
> We should enable lint for maven-compiler-plugin
> {code}
> maven-compiler-plugin
> ${maven-compiler-plugin.version}
>
> 1.8
> 1.8
>
> -Xlint:all
> ${compiler.error.flag}
>
> -Xlint:-options
>
> -Xlint:-cast
> -Xlint:-deprecation
> -Xlint:-processing
> -Xlint:-rawtypes
> -Xlint:-serial
> -Xlint:-try
> -Xlint:-unchecked
> -Xlint:-varargs
>
>
>
>
> true
>
> false
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3310) Use lint for maven-compiler-plugin
[
https://issues.apache.org/jira/browse/KYLIN-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao reassigned KYLIN-3310:
-
Assignee: jiatao.tao
> Use lint for maven-compiler-plugin
> --
>
> Key: KYLIN-3310
> URL: https://issues.apache.org/jira/browse/KYLIN-3310
> Project: Kylin
> Issue Type: Improvement
> Components: Tools, Build and Test
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Major
>
> lint helps identify structural problems.
> We should enable lint for maven-compiler-plugin
> {code}
> maven-compiler-plugin
> ${maven-compiler-plugin.version}
>
> 1.8
> 1.8
>
> -Xlint:all
> ${compiler.error.flag}
>
> -Xlint:-options
>
> -Xlint:-cast
> -Xlint:-deprecation
> -Xlint:-processing
> -Xlint:-rawtypes
> -Xlint:-serial
> -Xlint:-try
> -Xlint:-unchecked
> -Xlint:-varargs
>
>
>
>
> true
>
> false
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3417) Consider replacing ReentrantReadWriteLock with StampedLock
[
https://issues.apache.org/jira/browse/KYLIN-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16543935#comment-16543935
]
jiatao.tao commented on KYLIN-3417:
---
Thanks Ted, I will take a glance! Nice to see all these interesting usage.
> Consider replacing ReentrantReadWriteLock with StampedLock
> --
>
> Key: KYLIN-3417
> URL: https://issues.apache.org/jira/browse/KYLIN-3417
> Project: Kylin
> Issue Type: Improvement
>Reporter: Ted Yu
>Priority: Major
> Fix For: v2.5.0
>
>
> ReentrantReadWriteLock's are only the right solution when there is long hold
> time due to expensive I/O.
> It is expensive for readers.
> We should see if the lighter {{StampedLock}} can be used instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3417) Consider replacing ReentrantReadWriteLock with StampedLock
[
https://issues.apache.org/jira/browse/KYLIN-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao reassigned KYLIN-3417:
-
Assignee: jiatao.tao
> Consider replacing ReentrantReadWriteLock with StampedLock
> --
>
> Key: KYLIN-3417
> URL: https://issues.apache.org/jira/browse/KYLIN-3417
> Project: Kylin
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Major
> Fix For: v2.5.0
>
>
> ReentrantReadWriteLock's are only the right solution when there is long hold
> time due to expensive I/O.
> It is expensive for readers.
> We should see if the lighter {{StampedLock}} can be used instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3450) Consider using google re2j
[ https://issues.apache.org/jira/browse/KYLIN-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16543933#comment-16543933 ] jiatao.tao commented on KYLIN-3450: --- Nice advice! Thanks Ted! > Consider using google re2j > -- > > Key: KYLIN-3450 > URL: https://issues.apache.org/jira/browse/KYLIN-3450 > Project: Kylin > Issue Type: Improvement >Reporter: Ted Yu >Priority: Minor > > RE2J : https://github.com/google/re2j > For regular expression patterns with a high degree of alternation, using RE2J > would exhibit higher performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3399) Leaked lookup table in DictionaryGeneratorCLI#processSegment
[
https://issues.apache.org/jira/browse/KYLIN-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao resolved KYLIN-3399.
---
Resolution: Fixed
> Leaked lookup table in DictionaryGeneratorCLI#processSegment
>
>
> Key: KYLIN-3399
> URL: https://issues.apache.org/jira/browse/KYLIN-3399
> Project: Kylin
> Issue Type: Bug
> Components: Client - CLI
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Major
>
> {code}
> for (TableRef lookup : toCheckLookup) {
> logger.info("Checking snapshot of " + lookup);
> JoinDesc join =
> cubeSeg.getModel().getJoinsTree().getJoinByPKSide(lookup);
> cubeMgr.getLookupTable(cubeSeg, join);
> {code}
> The lookup table from the last line doesn't seem to be used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3439) configuration "kylin.web.timezone" is Inconsistent between kylin-defaults.properties and KylinConfigBase.java
[
https://issues.apache.org/jira/browse/KYLIN-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533278#comment-16533278
]
jiatao.tao commented on KYLIN-3439:
---
In kylin-defaults.properties : kylin.web.timezone=GMT+8
In KylinConfigBase : return getOptional("kylin.web.timezone", "GMT+8");
> configuration "kylin.web.timezone" is Inconsistent between
> kylin-defaults.properties and KylinConfigBase.java
> --
>
> Key: KYLIN-3439
> URL: https://issues.apache.org/jira/browse/KYLIN-3439
> Project: Kylin
> Issue Type: Bug
>Reporter: jiatao.tao
>Assignee: jiatao.tao
>Priority: Minor
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3439) configuration "kylin.web.timezone" is Inconsistent between kylin-defaults.properties and KylinConfigBase.java
jiatao.tao created KYLIN-3439: - Summary: configuration "kylin.web.timezone" is Inconsistent between kylin-defaults.properties and KylinConfigBase.java Key: KYLIN-3439 URL: https://issues.apache.org/jira/browse/KYLIN-3439 Project: Kylin Issue Type: Bug Reporter: jiatao.tao Assignee: jiatao.tao -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3399) Leaked lookup table in DictionaryGeneratorCLI#processSegment
[
https://issues.apache.org/jira/browse/KYLIN-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517746#comment-16517746
]
jiatao.tao commented on KYLIN-3399:
---
I add a try-catch clause, so the code may more intuitive.
> Leaked lookup table in DictionaryGeneratorCLI#processSegment
>
>
> Key: KYLIN-3399
> URL: https://issues.apache.org/jira/browse/KYLIN-3399
> Project: Kylin
> Issue Type: Bug
> Components: Client - CLI
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Major
>
> {code}
> for (TableRef lookup : toCheckLookup) {
> logger.info("Checking snapshot of " + lookup);
> JoinDesc join =
> cubeSeg.getModel().getJoinsTree().getJoinByPKSide(lookup);
> cubeMgr.getLookupTable(cubeSeg, join);
> {code}
> The lookup table from the last line doesn't seem to be used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (KYLIN-3321) Set MALLOC_ARENA_MAX in script
[ https://issues.apache.org/jira/browse/KYLIN-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao resolved KYLIN-3321. --- Resolution: Fixed > Set MALLOC_ARENA_MAX in script > -- > > Key: KYLIN-3321 > URL: https://issues.apache.org/jira/browse/KYLIN-3321 > Project: Kylin > Issue Type: Improvement > Components: Environment >Reporter: Ted Yu >Assignee: jiatao.tao >Priority: Major > > conf/setenv.sh would be good place to set MALLOC_ARENA_MAX which prevents > native memory OOM. > See https://github.com/prestodb/presto/issues/8993 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-1948) IntegerDimEnc, does not encode -1 correctly
[ https://issues.apache.org/jira/browse/KYLIN-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao resolved KYLIN-1948. --- Resolution: Fixed > IntegerDimEnc, does not encode -1 correctly > --- > > Key: KYLIN-1948 > URL: https://issues.apache.org/jira/browse/KYLIN-1948 > Project: Kylin > Issue Type: Bug >Reporter: liyang >Assignee: jiatao.tao >Priority: Major > Fix For: v2.4.0 > > > The code for -1 is all 0xff, which is the code for NULL. Need a fix, since -1 > is a common value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3399) Leaked lookup table in DictionaryGeneratorCLI#processSegment
[
https://issues.apache.org/jira/browse/KYLIN-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504429#comment-16504429
]
jiatao.tao commented on KYLIN-3399:
---
Thanks a lot! I'll take a look.
> Leaked lookup table in DictionaryGeneratorCLI#processSegment
>
>
> Key: KYLIN-3399
> URL: https://issues.apache.org/jira/browse/KYLIN-3399
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Major
>
> {code}
> for (TableRef lookup : toCheckLookup) {
> logger.info("Checking snapshot of " + lookup);
> JoinDesc join =
> cubeSeg.getModel().getJoinsTree().getJoinByPKSide(lookup);
> cubeMgr.getLookupTable(cubeSeg, join);
> {code}
> The lookup table from the last line doesn't seem to be used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3398) Inaccurate arithmetic operation in LookupTableToHFileJob#calculateShardNum
[
https://issues.apache.org/jira/browse/KYLIN-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504428#comment-16504428
]
jiatao.tao commented on KYLIN-3398:
---
Thanks a lot! I'll take a look.
> Inaccurate arithmetic operation in LookupTableToHFileJob#calculateShardNum
> --
>
> Key: KYLIN-3398
> URL: https://issues.apache.org/jira/browse/KYLIN-3398
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Major
>
> There're two issues with the following code:
> {code}
> private int calculateShardNum(KylinConfig kylinConfig, long dataSize) {
> long shardSize = kylinConfig.getExtTableSnapshotShardingMB() * 1024 *
> 1024;
> return dataSize < shardSize ? 1 : (int) (Math.ceil(dataSize /
> shardSize));
> {code}
> getExtTableSnapshotShardingMB returns an int. The multiplication is done
> using 32-bit arithmetic, and then used in a context that expects an
> expression of type "long".
> Second, Math.ceil expects a double. The integer division would lose some
> precision.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3398) Inaccurate arithmetic operation in LookupTableToHFileJob#calculateShardNum
[
https://issues.apache.org/jira/browse/KYLIN-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao reassigned KYLIN-3398:
-
Assignee: jiatao.tao (was: nichunen)
> Inaccurate arithmetic operation in LookupTableToHFileJob#calculateShardNum
> --
>
> Key: KYLIN-3398
> URL: https://issues.apache.org/jira/browse/KYLIN-3398
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Major
>
> There're two issues with the following code:
> {code}
> private int calculateShardNum(KylinConfig kylinConfig, long dataSize) {
> long shardSize = kylinConfig.getExtTableSnapshotShardingMB() * 1024 *
> 1024;
> return dataSize < shardSize ? 1 : (int) (Math.ceil(dataSize /
> shardSize));
> {code}
> getExtTableSnapshotShardingMB returns an int. The multiplication is done
> using 32-bit arithmetic, and then used in a context that expects an
> expression of type "long".
> Second, Math.ceil expects a double. The integer division would lose some
> precision.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3399) Leaked lookup table in DictionaryGeneratorCLI#processSegment
[
https://issues.apache.org/jira/browse/KYLIN-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao reassigned KYLIN-3399:
-
Assignee: jiatao.tao (was: nichunen)
> Leaked lookup table in DictionaryGeneratorCLI#processSegment
>
>
> Key: KYLIN-3399
> URL: https://issues.apache.org/jira/browse/KYLIN-3399
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Major
>
> {code}
> for (TableRef lookup : toCheckLookup) {
> logger.info("Checking snapshot of " + lookup);
> JoinDesc join =
> cubeSeg.getModel().getJoinsTree().getJoinByPKSide(lookup);
> cubeMgr.getLookupTable(cubeSeg, join);
> {code}
> The lookup table from the last line doesn't seem to be used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (KYLIN-3339) Decoupling SQL keywords set with specify implementation in query pushdown
[ https://issues.apache.org/jira/browse/KYLIN-3339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao resolved KYLIN-3339. --- Resolution: Fixed > Decoupling SQL keywords set with specify implementation in query pushdown > - > > Key: KYLIN-3339 > URL: https://issues.apache.org/jira/browse/KYLIN-3339 > Project: Kylin > Issue Type: Improvement >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Trivial > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3295) Unused method SQLDigestUtil#appendTsFilterToExecute
[
https://issues.apache.org/jira/browse/KYLIN-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493092#comment-16493092
]
jiatao.tao commented on KYLIN-3295:
---
SQLDigestUtil is no usage, so may be delete this class is better?
> Unused method SQLDigestUtil#appendTsFilterToExecute
> ---
>
> Key: KYLIN-3295
> URL: https://issues.apache.org/jira/browse/KYLIN-3295
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
> Fix For: v2.4.0
>
>
> SQLDigestUtil#appendTsFilterToExecute is not called anywhere.
> {code}
>T ret = action.apply(null);
> {code}
> Passing null to {{apply}} seems incorrect.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-2408) GTRecord work on column block rather than individual columns
[ https://issues.apache.org/jira/browse/KYLIN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao reassigned KYLIN-2408: - Assignee: jiatao.tao > GTRecord work on column block rather than individual columns > > > Key: KYLIN-2408 > URL: https://issues.apache.org/jira/browse/KYLIN-2408 > Project: Kylin > Issue Type: Improvement >Reporter: liyang >Assignee: jiatao.tao >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-1948) IntegerDimEnc, does not encode -1 correctly
[ https://issues.apache.org/jira/browse/KYLIN-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao reassigned KYLIN-1948: - Assignee: jiatao.tao > IntegerDimEnc, does not encode -1 correctly > --- > > Key: KYLIN-1948 > URL: https://issues.apache.org/jira/browse/KYLIN-1948 > Project: Kylin > Issue Type: Bug >Reporter: liyang >Assignee: jiatao.tao >Priority: Major > > The code for -1 is all 0xff, which is the code for NULL. Need a fix, since -1 > is a common value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3295) Unused method SQLDigestUtil#appendTsFilterToExecute
[
https://issues.apache.org/jira/browse/KYLIN-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao reassigned KYLIN-3295:
-
Assignee: jiatao.tao
> Unused method SQLDigestUtil#appendTsFilterToExecute
> ---
>
> Key: KYLIN-3295
> URL: https://issues.apache.org/jira/browse/KYLIN-3295
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
>
> SQLDigestUtil#appendTsFilterToExecute is not called anywhere.
> {code}
>T ret = action.apply(null);
> {code}
> Passing null to {{apply}} seems incorrect.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3321) Set MALLOC_ARENA_MAX in script
[ https://issues.apache.org/jira/browse/KYLIN-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao reassigned KYLIN-3321: - Assignee: jiatao.tao > Set MALLOC_ARENA_MAX in script > -- > > Key: KYLIN-3321 > URL: https://issues.apache.org/jira/browse/KYLIN-3321 > Project: Kylin > Issue Type: Task > Components: Environment >Reporter: Ted Yu >Assignee: jiatao.tao >Priority: Major > > conf/setenv.sh would be good place to set MALLOC_ARENA_MAX which prevents > native memory OOM. > See https://github.com/prestodb/presto/issues/8993 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-743) Kylin to be OLAP source of SparkSQL
[ https://issues.apache.org/jira/browse/KYLIN-743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao reassigned KYLIN-743: Assignee: jiatao.tao > Kylin to be OLAP source of SparkSQL > --- > > Key: KYLIN-743 > URL: https://issues.apache.org/jira/browse/KYLIN-743 > Project: Kylin > Issue Type: New Feature > Components: Spark Engine >Reporter: Luke Han >Assignee: jiatao.tao >Priority: Major > Fix For: Backlog > > > Kylin's MOLAP Cube is great to serve query which has pattern and request very > frequently from clients. > To be more efficiency, SparkSQL could leverage Kylin as it's OLAP option so > that SparkSQL could make decision route SQL to Kylin's cube or underlying > data. > This feature require more deep design about integration between Kylin and > SparkSQL. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3321) Set MALLOC_ARENA_MAX in script
[ https://issues.apache.org/jira/browse/KYLIN-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16443643#comment-16443643 ] jiatao.tao commented on KYLIN-3321: --- (y) > Set MALLOC_ARENA_MAX in script > -- > > Key: KYLIN-3321 > URL: https://issues.apache.org/jira/browse/KYLIN-3321 > Project: Kylin > Issue Type: Task > Components: Environment >Reporter: Ted Yu >Assignee: jiatao.tao >Priority: Major > > conf/setenv.sh would be good place to set MALLOC_ARENA_MAX which prevents > native memory OOM. > See https://github.com/prestodb/presto/issues/8993 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3290) Avoid calling Class#newInstance
[ https://issues.apache.org/jira/browse/KYLIN-3290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao reassigned KYLIN-3290: - Assignee: jiatao.tao > Avoid calling Class#newInstance > --- > > Key: KYLIN-3290 > URL: https://issues.apache.org/jira/browse/KYLIN-3290 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: jiatao.tao >Priority: Minor > > Class#newInstance is deprecated starting in Java 9 - > https://bugs.openjdk.java.net/browse/JDK-6850612 - because it may throw > undeclared checked exceptions. > The suggested replacement is getDeclaredConstructor().newInstance(), which > wraps the checked exceptions in InvocationException. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3317) Replace UUID.randomUUID with deterministic PRNG
[ https://issues.apache.org/jira/browse/KYLIN-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao reassigned KYLIN-3317: - Assignee: jiatao.tao > Replace UUID.randomUUID with deterministic PRNG > --- > > Key: KYLIN-3317 > URL: https://issues.apache.org/jira/browse/KYLIN-3317 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: jiatao.tao >Priority: Major > > Currently UUID.randomUUID is called in various places in the code base. > * It is non-deterministic. > * It uses a single secure random for UUID generation. This uses a single JVM > wide lock, and this can lead to lock contention and other performance > problems. > We should move to something that is deterministic by using seeded PRNGs -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3339) Decoupling SQL keywords set with specify implementation.
jiatao.tao created KYLIN-3339: - Summary: Decoupling SQL keywords set with specify implementation. Key: KYLIN-3339 URL: https://issues.apache.org/jira/browse/KYLIN-3339 Project: Kylin Issue Type: Improvement Reporter: jiatao.tao Assignee: jiatao.tao -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3299) When a column in int data type and encoding as date may will lead build wrong value
[ https://issues.apache.org/jira/browse/KYLIN-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3299: -- Description: When the year is less than 1000, build and query will result 1970. > When a column in int data type and encoding as date may will lead build wrong > value > --- > > Key: KYLIN-3299 > URL: https://issues.apache.org/jira/browse/KYLIN-3299 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Major > > When the year is less than 1000, build and query will result 1970. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3299) When a column in int data type and encoding as date may will lead build wrong value
[ https://issues.apache.org/jira/browse/KYLIN-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3299: -- Summary: When a column in int data type and encoding as date may will lead build wrong value (was: When a column in int data type and encoding as date and the year less than 1000 will lead build error) > When a column in int data type and encoding as date may will lead build wrong > value > --- > > Key: KYLIN-3299 > URL: https://issues.apache.org/jira/browse/KYLIN-3299 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3299) When a column in int data type and encoding as date may will lead build wrong value
[ https://issues.apache.org/jira/browse/KYLIN-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao reassigned KYLIN-3299: - Assignee: jiatao.tao > When a column in int data type and encoding as date may will lead build wrong > value > --- > > Key: KYLIN-3299 > URL: https://issues.apache.org/jira/browse/KYLIN-3299 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3299) When a column in int data type and encoding as date and the year less than 1000 will lead build error
jiatao.tao created KYLIN-3299: - Summary: When a column in int data type and encoding as date and the year less than 1000 will lead build error Key: KYLIN-3299 URL: https://issues.apache.org/jira/browse/KYLIN-3299 Project: Kylin Issue Type: Bug Reporter: jiatao.tao -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3293) FixedLenHexDimEnc return a wrong code length leads to cut bytes error.
[ https://issues.apache.org/jira/browse/KYLIN-3293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3293: -- Description: Cuz > FixedLenHexDimEnc return a wrong code length leads to cut bytes error. > -- > > Key: KYLIN-3293 > URL: https://issues.apache.org/jira/browse/KYLIN-3293 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Major > > Cuz -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3293) FixedLenHexDimEnc return a wrong code length leads to cut bytes error.
jiatao.tao created KYLIN-3293: - Summary: FixedLenHexDimEnc return a wrong code length leads to cut bytes error. Key: KYLIN-3293 URL: https://issues.apache.org/jira/browse/KYLIN-3293 Project: Kylin Issue Type: Bug Reporter: jiatao.tao Assignee: jiatao.tao -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3287) When a shard by column is in dict encoding, dict building error.
[ https://issues.apache.org/jira/browse/KYLIN-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3287: -- Description: When a shard by column is in dict encoding, query this column will result null. > When a shard by column is in dict encoding, dict building error. > > > Key: KYLIN-3287 > URL: https://issues.apache.org/jira/browse/KYLIN-3287 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.3.0 >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Major > Fix For: v2.3.1 > > Attachments: image-2018-03-12-14-14-50-112.png, > image-2018-03-12-14-14-59-193.png, image-2018-03-12-14-15-42-046.png, > image-2018-03-12-14-16-38-657.png, image-2018-03-12-14-17-46-780.png > > > When a shard by column is in dict encoding, query this column will result > null. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3287) When a shard by column is in dict encoding, dict building error.
[ https://issues.apache.org/jira/browse/KYLIN-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3287: -- Description: (was: In dict encoding. !image-2018-03-12-14-14-50-112.png|width=544,height=84! !image-2018-03-12-14-14-59-193.png|width=535,height=631! In int encoding !image-2018-03-12-14-15-42-046.png|width=544,height=87! !image-2018-03-12-14-16-38-657.png!) > When a shard by column is in dict encoding, dict building error. > > > Key: KYLIN-3287 > URL: https://issues.apache.org/jira/browse/KYLIN-3287 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.3.0 >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Major > Fix For: v2.3.1 > > Attachments: image-2018-03-12-14-14-50-112.png, > image-2018-03-12-14-14-59-193.png, image-2018-03-12-14-15-42-046.png, > image-2018-03-12-14-16-38-657.png, image-2018-03-12-14-17-46-780.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3287) When a shard by column is in dict encoding, dict building error.
[ https://issues.apache.org/jira/browse/KYLIN-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3287: -- Summary: When a shard by column is in dict encoding, dict building error. (was: When a shard by column is in dict encoding, dict buildin error.) > When a shard by column is in dict encoding, dict building error. > > > Key: KYLIN-3287 > URL: https://issues.apache.org/jira/browse/KYLIN-3287 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Major > Attachments: image-2018-03-12-14-14-50-112.png, > image-2018-03-12-14-14-59-193.png, image-2018-03-12-14-15-42-046.png, > image-2018-03-12-14-16-38-657.png, image-2018-03-12-14-17-46-780.png > > > In dict encoding. > !image-2018-03-12-14-14-50-112.png|width=544,height=84! > !image-2018-03-12-14-14-59-193.png|width=535,height=631! > In int encoding > !image-2018-03-12-14-15-42-046.png|width=544,height=87! > !image-2018-03-12-14-16-38-657.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3287) When a shard by column is in dict encoding, dict buildin error.
[ https://issues.apache.org/jira/browse/KYLIN-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3287: -- Summary: When a shard by column is in dict encoding, dict buildin error. (was: When a shard by column is in dict encoding, dict building is error.) > When a shard by column is in dict encoding, dict buildin error. > --- > > Key: KYLIN-3287 > URL: https://issues.apache.org/jira/browse/KYLIN-3287 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Major > Attachments: image-2018-03-12-14-14-50-112.png, > image-2018-03-12-14-14-59-193.png, image-2018-03-12-14-15-42-046.png, > image-2018-03-12-14-16-38-657.png, image-2018-03-12-14-17-46-780.png > > > In dict encoding. > !image-2018-03-12-14-14-50-112.png|width=544,height=84! > !image-2018-03-12-14-14-59-193.png|width=535,height=631! > In int encoding > !image-2018-03-12-14-15-42-046.png|width=544,height=87! > !image-2018-03-12-14-16-38-657.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3287) When a shard by column is in dict encoding, dict building is error.
[ https://issues.apache.org/jira/browse/KYLIN-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394845#comment-16394845 ] jiatao.tao commented on KYLIN-3287: --- This is the root clause. !image-2018-03-12-14-17-46-780.png|width=579,height=318! > When a shard by column is in dict encoding, dict building is error. > --- > > Key: KYLIN-3287 > URL: https://issues.apache.org/jira/browse/KYLIN-3287 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Priority: Major > Attachments: image-2018-03-12-14-14-50-112.png, > image-2018-03-12-14-14-59-193.png, image-2018-03-12-14-15-42-046.png, > image-2018-03-12-14-16-38-657.png, image-2018-03-12-14-17-46-780.png > > > In dict encoding. > !image-2018-03-12-14-14-50-112.png|width=544,height=84! > !image-2018-03-12-14-14-59-193.png|width=535,height=631! > In int encoding > !image-2018-03-12-14-15-42-046.png|width=544,height=87! > !image-2018-03-12-14-16-38-657.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3287) When a shard by column is in dict encoding, dict building is error.
[ https://issues.apache.org/jira/browse/KYLIN-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao reassigned KYLIN-3287: - Assignee: jiatao.tao > When a shard by column is in dict encoding, dict building is error. > --- > > Key: KYLIN-3287 > URL: https://issues.apache.org/jira/browse/KYLIN-3287 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Major > Attachments: image-2018-03-12-14-14-50-112.png, > image-2018-03-12-14-14-59-193.png, image-2018-03-12-14-15-42-046.png, > image-2018-03-12-14-16-38-657.png, image-2018-03-12-14-17-46-780.png > > > In dict encoding. > !image-2018-03-12-14-14-50-112.png|width=544,height=84! > !image-2018-03-12-14-14-59-193.png|width=535,height=631! > In int encoding > !image-2018-03-12-14-15-42-046.png|width=544,height=87! > !image-2018-03-12-14-16-38-657.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3287) When a shard by column is in dict encoding, dict building is error.
[ https://issues.apache.org/jira/browse/KYLIN-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3287: -- Description: In dict encoding. !image-2018-03-12-14-14-50-112.png|width=544,height=84! !image-2018-03-12-14-14-59-193.png|width=535,height=631! In int encoding !image-2018-03-12-14-15-42-046.png|width=544,height=87! !image-2018-03-12-14-16-38-657.png! > When a shard by column is in dict encoding, dict building is error. > --- > > Key: KYLIN-3287 > URL: https://issues.apache.org/jira/browse/KYLIN-3287 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Priority: Major > Attachments: image-2018-03-12-14-14-50-112.png, > image-2018-03-12-14-14-59-193.png, image-2018-03-12-14-15-42-046.png, > image-2018-03-12-14-16-38-657.png, image-2018-03-12-14-17-46-780.png > > > In dict encoding. > !image-2018-03-12-14-14-50-112.png|width=544,height=84! > !image-2018-03-12-14-14-59-193.png|width=535,height=631! > In int encoding > !image-2018-03-12-14-15-42-046.png|width=544,height=87! > !image-2018-03-12-14-16-38-657.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3287) When a shard by column is in dict encoding, dict building is error.
[ https://issues.apache.org/jira/browse/KYLIN-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3287: -- Attachment: image-2018-03-12-14-17-46-780.png > When a shard by column is in dict encoding, dict building is error. > --- > > Key: KYLIN-3287 > URL: https://issues.apache.org/jira/browse/KYLIN-3287 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Priority: Major > Attachments: image-2018-03-12-14-14-50-112.png, > image-2018-03-12-14-14-59-193.png, image-2018-03-12-14-15-42-046.png, > image-2018-03-12-14-16-38-657.png, image-2018-03-12-14-17-46-780.png > > > In dict encoding. > !image-2018-03-12-14-14-50-112.png|width=544,height=84! > !image-2018-03-12-14-14-59-193.png|width=535,height=631! > In int encoding > !image-2018-03-12-14-15-42-046.png|width=544,height=87! > !image-2018-03-12-14-16-38-657.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3287) When a shard by column is in dict encoding, dict building is error.
[ https://issues.apache.org/jira/browse/KYLIN-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3287: -- Attachment: image-2018-03-12-14-16-38-657.png > When a shard by column is in dict encoding, dict building is error. > --- > > Key: KYLIN-3287 > URL: https://issues.apache.org/jira/browse/KYLIN-3287 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Priority: Major > Attachments: image-2018-03-12-14-14-50-112.png, > image-2018-03-12-14-14-59-193.png, image-2018-03-12-14-15-42-046.png, > image-2018-03-12-14-16-38-657.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3287) When a shard by column is in dict encoding, dict building is error.
[ https://issues.apache.org/jira/browse/KYLIN-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3287: -- Attachment: image-2018-03-12-14-15-42-046.png > When a shard by column is in dict encoding, dict building is error. > --- > > Key: KYLIN-3287 > URL: https://issues.apache.org/jira/browse/KYLIN-3287 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Priority: Major > Attachments: image-2018-03-12-14-14-50-112.png, > image-2018-03-12-14-14-59-193.png, image-2018-03-12-14-15-42-046.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3287) When a shard by column is in dict encoding, dict building is error.
[ https://issues.apache.org/jira/browse/KYLIN-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3287: -- Attachment: image-2018-03-12-14-14-59-193.png > When a shard by column is in dict encoding, dict building is error. > --- > > Key: KYLIN-3287 > URL: https://issues.apache.org/jira/browse/KYLIN-3287 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Priority: Major > Attachments: image-2018-03-12-14-14-50-112.png, > image-2018-03-12-14-14-59-193.png, image-2018-03-12-14-15-42-046.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3287) When a shard by column is in dict encoding, dict building is error.
[ https://issues.apache.org/jira/browse/KYLIN-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3287: -- Attachment: image-2018-03-12-14-14-50-112.png > When a shard by column is in dict encoding, dict building is error. > --- > > Key: KYLIN-3287 > URL: https://issues.apache.org/jira/browse/KYLIN-3287 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Priority: Major > Attachments: image-2018-03-12-14-14-50-112.png, > image-2018-03-12-14-14-59-193.png, image-2018-03-12-14-15-42-046.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3287) When a shard by column is in dict encoding, dict building is error.
jiatao.tao created KYLIN-3287: - Summary: When a shard by column is in dict encoding, dict building is error. Key: KYLIN-3287 URL: https://issues.apache.org/jira/browse/KYLIN-3287 Project: Kylin Issue Type: Bug Reporter: jiatao.tao -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3174) Default scheduler enhancement
[ https://issues.apache.org/jira/browse/KYLIN-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3174: -- Summary: Default scheduler enhancement (was: Scheduler enhancement) > Default scheduler enhancement > - > > Key: KYLIN-3174 > URL: https://issues.apache.org/jira/browse/KYLIN-3174 > Project: Kylin > Issue Type: Improvement >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3174) Scheduler enhancement
[ https://issues.apache.org/jira/browse/KYLIN-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3174: -- Priority: Minor (was: Major) > Scheduler enhancement > - > > Key: KYLIN-3174 > URL: https://issues.apache.org/jira/browse/KYLIN-3174 > Project: Kylin > Issue Type: Improvement >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3275) Add unit test for StorageCleanupJob
jiatao.tao created KYLIN-3275: - Summary: Add unit test for StorageCleanupJob Key: KYLIN-3275 URL: https://issues.apache.org/jira/browse/KYLIN-3275 Project: Kylin Issue Type: Improvement Reporter: jiatao.tao Assignee: jiatao.tao -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3263) AbstractExecutable's retry has problem
jiatao.tao created KYLIN-3263: - Summary: AbstractExecutable's retry has problem Key: KYLIN-3263 URL: https://issues.apache.org/jira/browse/KYLIN-3263 Project: Kylin Issue Type: Bug Reporter: jiatao.tao Assignee: jiatao.tao -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374038#comment-16374038
]
jiatao.tao commented on KYLIN-3197:
---
Hi [~xingpeng1]
Thanks a lot, any update be updated at this jira soon.
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have test the patch, please review, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiatao.tao updated KYLIN-3197:
--
Attachment: (was: image-2018-02-23-15-30-50-401.png)
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have test the patch, please review, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361734#comment-16361734
]
jiatao.tao edited comment on KYLIN-3197 at 2/13/18 2:54 AM:
Hi [~xingpeng1]
About Redhat we may need further discussion, it's need a full discussion and
comparison or we can solve one problem today, but next day, another problem may
occur. And can you also put your user's ldif?
Besides, I know you use getAdditionalRoles() to solve this problem, for sure,
but what I say is that should we use this method like this way? Can you find
some examples like document or other projects use this way? Not asking for how
it works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
I believe we are not the only one met this problem, we should go and find out
how other people solve this.
Looking forward your opinion.
was (Author: aron.tao):
Hi [~xingpeng1]
About Redhat we may need further discussion, it's need a full discussion and
comparison or we can solve one problem today, but next day, another problem may
occur. And can you also put your user's ldif?
Besides, I know you use getAdditionalRoles() to solve this problem, for sure,
but what I say is that should we use this method like this way? Can you find
some examples like document or other projects use this way? Not asking for how
it works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
I believe we are not the only one met the problem, we should go and find out
how other people solve this.
Looking forward your opinion.
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361734#comment-16361734
]
jiatao.tao edited comment on KYLIN-3197 at 2/13/18 2:53 AM:
Hi [~xingpeng1]
About Redhat we may need further discussion, it's need a full discussion and
comparison or we can solve one problem today, but next day, another problem may
occur. And can you also put your user's ldif?
Besides, I know you use getAdditionalRoles() to solve this problem, for sure,
but what I say is that should we use this method like this way? Can you find
some examples like document or other projects use this way? Not asking for how
it works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
I believe we are not the only one met the problem, we should go and find out
how other people solve this.
Looking forward your opinion.
was (Author: aron.tao):
Hi [~xingpeng1]
About Redhat we may need further discussion, it's need a full discussion and
comparison or we can solve one problem today, but next day, another problem may
occur. And can you also put your user's ldif?
Besides, I know you use getAdditionalRoles() to solve this problem, for sure,
but what I say is that should we use this method like this way? Can you find
some examples like document or other projects use this way? Not asking for how
it works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
>
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361734#comment-16361734
]
jiatao.tao edited comment on KYLIN-3197 at 2/13/18 2:49 AM:
Hi [~xingpeng1]
About Redhat we may need further discussion, it's need a full discussion and
comparison or we can solve one problem today, but next day, another problem may
occur. And can you also put your user's ldif?
Besides, I know you use getAdditionalRoles() to solve this problem, for sure,
but what I say is that should we use this method like this way? Can you find
some examples like document or other projects use this way? Not asking for how
it works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
was (Author: aron.tao):
Hi [~xingpeng1]
About Redhat we may need further discussion, it's need a full discussion and
comparison or we can solve one problem today, but next day, another problem may
occur.And can you also put your user's ldif?
Besides, I know you use getAdditionalRoles() to solve this problem, for sure,
but what I say is that should we use this method like this way? Can you find
some examples like document or other projects use this way? Not asking for how
it works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361734#comment-16361734
]
jiatao.tao edited comment on KYLIN-3197 at 2/13/18 2:49 AM:
Hi [~xingpeng1]
About Redhat we may need further discussion, it's need a full discussion and
comparison or we can solve one problem today, but next day, another problem may
occur.And can you also put your user's ldif?
Besides, I know you use getAdditionalRoles() to solve this problem, for sure,
but what I say is that should we use this method like this way? Can you find
some examples like document or other projects use this way? Not asking for how
it works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
was (Author: aron.tao):
Hi [~xingpeng1]
About Redhat we may need further discussion, can you also put your user's ldif?
Besides, I know you use getAdditionalRoles to solve this problem, for sure, but
what I say is that should we use this method like this way? Can you find some
examples like document or other projects use this way? Not asking for how it
works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have test the patch, please review, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361734#comment-16361734
]
jiatao.tao edited comment on KYLIN-3197 at 2/13/18 2:44 AM:
Hi [~xingpeng1]
About Redhat we may need further discussion, can you also put your user's ldif?
Besides, I know you use getAdditionalRoles to solve this problem, for sure, but
what I say is that should we use this method like this way? Can you find some
examples like document or other projects use this way? Not asking for how it
works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
was (Author: aron.tao):
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Besides, I know you use getAdditionalRoles to solve this problem, for sure, but
what I say is that should we use this method like this way? Can you find some
examples like document or other projects use this way? Not asking for how it
works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have test the patch, please review, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361734#comment-16361734
]
jiatao.tao edited comment on KYLIN-3197 at 2/13/18 2:42 AM:
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Besides, I know you use getAdditionalRoles to solve this problem, for sure, but
what I say is that should we use this method like this way? Can you find some
examples like document or other projects use this way? Not asking for how it
works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
was (Author: aron.tao):
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Besides, I know you use getAdditionalRoles to solve this problem, for sure, but
what I say is that should we use this method like this way? Can you find some
examples like document or other projects use this way? not asking for how it
works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have test the patch, please review, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361734#comment-16361734
]
jiatao.tao edited comment on KYLIN-3197 at 2/13/18 2:41 AM:
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Besides, I know you use getAdditionalRoles to solve this problem, for sure, but
what I say is that should we use this method like this way? Can you find some
examples like document or other projects use this way? not asking for how it
works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
was (Author: aron.tao):
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Besides, I know you use getAdditionalRoles to solve this problem, for sure, but
what I say is that should we use this method like this way? Can you find some
examples like document or other projects use this way, not asking for how it
works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have test the patch, please review, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361734#comment-16361734
]
jiatao.tao edited comment on KYLIN-3197 at 2/13/18 2:40 AM:
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Besides, I know you use getAdditionalRoles to solve this problem, for sure, but
what I say is that should we use this method like this way? Can you find some
examples like document or other projects use this way, not asking for how it
works. We all understand how it works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
was (Author: aron.tao):
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Besides, I know you use getAdditionalRoles to solve this problem, for sure, but
what I say is that should we use this method like this way? Can you find some
examples like document or other projects use this way, not asking for how it
works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have test the patch, please review, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361734#comment-16361734
]
jiatao.tao edited comment on KYLIN-3197 at 2/13/18 2:39 AM:
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Besides, I know you use getAdditionalRoles to solve this problem, for sure, but
what I say is that should we use this method like this way? Can you find some
examples like document or other projects use this way, not asking for how it
works.
{code:java}
The signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
was (Author: aron.tao):
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Besides, I know you use xxx to solve this problem, but what I say is that
should we use this method like this way? Can you find some examples like
document or other project use this way.
{code:java}
the signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have test the patch, please review, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361734#comment-16361734
]
jiatao.tao edited comment on KYLIN-3197 at 2/13/18 2:37 AM:
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Besides, I know you use xxx to solve this problem, but what I say is that
should we use this method like this way? Can you find some examples like
document or other project use this way.
{code:java}
the signature of getAdditionalRoles() seems not the way you use.
Because the Redhat linux can not support the case insensitive ldap username,
that is to say 'getGroupMembershipRoles(userDn, username)' will return empty
Set, so I analyze the spring source code, after
'getGroupMembershipRoles(userDn, username)', there will call
'getAdditionalRoles(user, username)' to get the roles again, then I can get the
real username from the DirContextOperations object.
{code}
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
was (Author: aron.tao):
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Besides, I know you use xxx to solve this problem, but what I say is that
should we use this method like this way? Can you find some examples like
document or other project use this way.
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have test the patch, please review, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361734#comment-16361734
]
jiatao.tao edited comment on KYLIN-3197 at 2/13/18 2:36 AM:
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Besides, I know you use xxx to solve this problem, but what I say is that
should we use this method like this way? Can you find some examples like
document or other project use this way.
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
was (Author: aron.tao):
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have test the patch, please review, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361734#comment-16361734
]
jiatao.tao edited comment on KYLIN-3197 at 2/13/18 2:32 AM:
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
Looking forward your opinion.
was (Author: aron.tao):
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have test the patch, please review, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361734#comment-16361734
]
jiatao.tao commented on KYLIN-3197:
---
Hi [~xingpeng1]
We will consider about Redhat, can you also put your user's ldif?
Also, I think it may be a requirements or issue that need discussion, but not
directly get "cn" from DirContextOperations.
{code:java}
the username passed in is not real one, but the 'WKH', so I find a way to fetch
the real one from DirContextOperations object by 'username =
user.getStringAttribute("cn");'
{code}
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have test the patch, please review, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360224#comment-16360224
]
jiatao.tao edited comment on KYLIN-3197 at 2/12/18 4:49 AM:
-1
[~peng.jianhua] The root clause is even still under discussion, why do you
merge the code???
was (Author: aron.tao):
-1
[~peng.jianhua] The root clause is still under discussion, why do you merge
the code???
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have test the patch, please review, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360305#comment-16360305
]
jiatao.tao edited comment on KYLIN-3197 at 2/12/18 4:41 AM:
Besides, as for you patch, there are two things that are little confusing to
me and it would be great if you can confirm these:
# the signature of getAdditionalRoles() seems not the way you use.
# In your patch you directly get username and not use the name that
getAdditionalRoles(DirContextOperations user, String username) passed in.
{code:java}
username = user.getStringAttribute("cn");
{code}
!image-2018-02-12-12-25-15-793.png|width=615,height=212!
was (Author: aron.tao):
Besides, there are two things that are little confusing to me in your patch and
it would be great if you can confirm these:
# the signature of getAdditionalRoles() seems not the way you use.
# In your patch you directly get username and not use the name that
getAdditionalRoles(DirContextOperations user, String username) passed in.
{code:java}
username = user.getStringAttribute("cn");
{code}
!image-2018-02-12-12-25-15-793.png|width=615,height=212!
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
> '{color:#ff}wkh{color}' to login in, which is normal.
> But when I use '{color:#ff}WKH{color}' to login in, the page does not
> respond.
> I analyze the backgroud code, and find the function of
> 'org.apache.kylin.rest.security.LDAPAuthoritiesPopulator.getGroupMembershipRoles(String,
> String)' has problem.
> When userDn is
> "uid={color:#ff}wkh{color},ou=People,ou=defaultCluster,dc=zdh,dc=com" and
> username is "{color:#ff}WKH{color}", then authorities will be empty Set
> by the follow code:
> {code:java}
> Set authorities = super.getGroupMembershipRoles(userDn,
> username);
> {code}
> So I have added 'getAdditionalRoles' function to get the authorities again.
> I have test the patch, please review, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360290#comment-16360290
]
jiatao.tao edited comment on KYLIN-3197 at 2/12/18 4:40 AM:
Hi [~xingpeng1]
I understand your description, and there's no need post again if there's no
update.
And I test your scenes, as you can see in the previous comment, and can not
reproduce your problem.
Let's put aside your patch first, and the root clause may not the one you
think, I recommend that we first find what's the problem, eg. wrong using?
environment problem? LDAP problem? or it's our Kylin's bug. You can referring
to my experiment. The LDIF config and Kyiln config are as follows.
*As for me,*
{color:#ff}kylin.security.ldap.user-search-pattern=(uid=\{0}){color}
*in your config is very suspicious, ldap may use this as username when
searching group members. And please confirm is this within your expectations.
And may you use {color:#ff}cn{color} for a try?*
Hope you can find the truly root clause.
LDIF
{code:java}
# People, example.com
dn: ou=People,dc=example,dc=com
ou: People
cn: People
objectClass: organizationalRole
objectClass: top
# jenny, People, example.com
dn: cn=jenny,ou=People,dc=example,dc=com
mail: je...@example.io
ou: Analyst
cn: jenny
sn: jenny liu
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: top
userPassword:: ZXhhbXBsZTEyMw==
# admin, Groups, example.com
dn: cn=admin,ou=Groups,dc=example,dc=com
cn: admin
member: cn=jenny,ou=People,dc=example,dc=com
objectClass: groupOfNames
objectClass: top
{code}
{code:java}
kylin.security.ldap.user-search-base=ou=People,dc=example,dc=com
kylin.security.ldap.user-group-search-base=ou=Groups,dc=example,dc=com
kylin.security.ldap.user-search-pattern=(&(cn={0}))
kylin.security.ldap.user-group-search-filter=(|(member={0})(memberUid={1}))
kylin.security.acl.admin-role=admin
{code}
was (Author: aron.tao):
Hi [~xingpeng1]
I understand your description, and there's no need post again if there's no
update.
And I test your scenes, as you can see in the previous comment, and can not
reproduce your problem.
Let's put aside your patch first, and the root clause may not the one you
think, I recommend that we first find what's the problem, eg. wrong using?
environment problem? LDAP problem? or it's our Kylin's bug. You can referring
to my experiment. The LDIF config and Kyiln config are as follows.
*As for me,*
{color:#FF}kylin.security.ldap.user-search-pattern=(uid=\{0}){color}
*in your config is very suspicious, ldap may use this as username to search
group members. And please confirm is this within your expectations. And may you
use {color:#FF}cn{color} for a try?*
Hope you can find the truly root clause.
LDIF
{code:java}
# People, example.com
dn: ou=People,dc=example,dc=com
ou: People
cn: People
objectClass: organizationalRole
objectClass: top
# jenny, People, example.com
dn: cn=jenny,ou=People,dc=example,dc=com
mail: je...@example.io
ou: Analyst
cn: jenny
sn: jenny liu
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: top
userPassword:: ZXhhbXBsZTEyMw==
# admin, Groups, example.com
dn: cn=admin,ou=Groups,dc=example,dc=com
cn: admin
member: cn=jenny,ou=People,dc=example,dc=com
objectClass: groupOfNames
objectClass: top
{code}
{code:java}
kylin.security.ldap.user-search-base=ou=People,dc=example,dc=com
kylin.security.ldap.user-group-search-base=ou=Groups,dc=example,dc=com
kylin.security.ldap.user-search-pattern=(&(cn={0}))
kylin.security.ldap.user-group-search-filter=(|(member={0})(memberUid={1}))
kylin.security.acl.admin-role=admin
{code}
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360290#comment-16360290
]
jiatao.tao edited comment on KYLIN-3197 at 2/12/18 4:39 AM:
Hi [~xingpeng1]
I understand your description, and there's no need post again if there's no
update.
And I test your scenes, as you can see in the previous comment, and can not
reproduce your problem.
Let's put aside your patch first, and the root clause may not the one you
think, I recommend that we first find what's the problem, eg. wrong using?
environment problem? LDAP problem? or it's our Kylin's bug. You can referring
to my experiment. The LDIF config and Kyiln config are as follows.
*As for me,*
{color:#FF}kylin.security.ldap.user-search-pattern=(uid=\{0}){color}
*in your config is very suspicious, ldap may use this as username to search
group members. And please confirm is this within your expectations. And may you
use {color:#FF}cn{color} for a try?*
Hope you can find the truly root clause.
LDIF
{code:java}
# People, example.com
dn: ou=People,dc=example,dc=com
ou: People
cn: People
objectClass: organizationalRole
objectClass: top
# jenny, People, example.com
dn: cn=jenny,ou=People,dc=example,dc=com
mail: je...@example.io
ou: Analyst
cn: jenny
sn: jenny liu
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: top
userPassword:: ZXhhbXBsZTEyMw==
# admin, Groups, example.com
dn: cn=admin,ou=Groups,dc=example,dc=com
cn: admin
member: cn=jenny,ou=People,dc=example,dc=com
objectClass: groupOfNames
objectClass: top
{code}
{code:java}
kylin.security.ldap.user-search-base=ou=People,dc=example,dc=com
kylin.security.ldap.user-group-search-base=ou=Groups,dc=example,dc=com
kylin.security.ldap.user-search-pattern=(&(cn={0}))
kylin.security.ldap.user-group-search-filter=(|(member={0})(memberUid={1}))
kylin.security.acl.admin-role=admin
{code}
was (Author: aron.tao):
Hi [~xingpeng1]
I understand your description, and there's no need post again if there's no
update.
And I test your scenes, as you can see in the previous comment, and can not
reproduce your problem.
Let's put aside your patch first, and the root clause may not the one you
think, I recommend that we first find what's the problem, eg. wrong using?
environment problem? LDAP problem? or it's our Kylin's bug. You can referring
to my experiment. The LDIF config and Kyiln config are as follows.
*As for me, kylin.security.ldap.user-search-pattern=(uid=\{0})*
*in your config is very suspicious, ldap may use this as username to search
group members. And please confirm is this within your expectations. And may you
use cn for a try?*
Hope you can find the truly root clause.
LDIF
{code:java}
# People, example.com
dn: ou=People,dc=example,dc=com
ou: People
cn: People
objectClass: organizationalRole
objectClass: top
# jenny, People, example.com
dn: cn=jenny,ou=People,dc=example,dc=com
mail: je...@example.io
ou: Analyst
cn: jenny
sn: jenny liu
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: top
userPassword:: ZXhhbXBsZTEyMw==
# admin, Groups, example.com
dn: cn=admin,ou=Groups,dc=example,dc=com
cn: admin
member: cn=jenny,ou=People,dc=example,dc=com
objectClass: groupOfNames
objectClass: top
{code}
{code:java}
kylin.security.ldap.user-search-base=ou=People,dc=example,dc=com
kylin.security.ldap.user-group-search-base=ou=Groups,dc=example,dc=com
kylin.security.ldap.user-search-pattern=(&(cn={0}))
kylin.security.ldap.user-group-search-filter=(|(member={0})(memberUid={1}))
kylin.security.acl.admin-role=admin
{code}
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose
[jira] [Comment Edited] (KYLIN-3197) When ldap is opened, I use an ignored case user to login, the page does not respond.
[
https://issues.apache.org/jira/browse/KYLIN-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360290#comment-16360290
]
jiatao.tao edited comment on KYLIN-3197 at 2/12/18 4:38 AM:
Hi [~xingpeng1]
I understand your description, and there's no need post again if there's no
update.
And I test your scenes, as you can see in the previous comment, and can not
reproduce your problem.
Let's put aside your patch first, and the root clause may not the one you
think, I recommend that we first find what's the problem, eg. wrong using?
environment problem? LDAP problem? or it's our Kylin's bug. You can referring
to my experiment. The LDIF config and Kyiln config are as follows.
*As for me, kylin.security.ldap.user-search-pattern=(uid=\{0})*
*in your config is very suspicious, ldap may use this as username to search
group members. And please confirm is this within your expectations. And may you
use cn for a try?*
Hope you can find the truly root clause.
LDIF
{code:java}
# People, example.com
dn: ou=People,dc=example,dc=com
ou: People
cn: People
objectClass: organizationalRole
objectClass: top
# jenny, People, example.com
dn: cn=jenny,ou=People,dc=example,dc=com
mail: je...@example.io
ou: Analyst
cn: jenny
sn: jenny liu
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: top
userPassword:: ZXhhbXBsZTEyMw==
# admin, Groups, example.com
dn: cn=admin,ou=Groups,dc=example,dc=com
cn: admin
member: cn=jenny,ou=People,dc=example,dc=com
objectClass: groupOfNames
objectClass: top
{code}
{code:java}
kylin.security.ldap.user-search-base=ou=People,dc=example,dc=com
kylin.security.ldap.user-group-search-base=ou=Groups,dc=example,dc=com
kylin.security.ldap.user-search-pattern=(&(cn={0}))
kylin.security.ldap.user-group-search-filter=(|(member={0})(memberUid={1}))
kylin.security.acl.admin-role=admin
{code}
was (Author: aron.tao):
Hi [~xingpeng1]
I understand your description, and there's no need post again if there's no
update.
And I test your scenes, as you can see in the previous comment, and can not
reproduce your problem.
Let's put aside your patch first, and the root clause may not the one you
think, I recommend that we first find what's the problem, eg. wrong using?
environment problem? LDAP problem? or it's our Kylin's bug. You can referring
to my experiment. The LDIF config and Kyiln config are as follows.
(As for me, kylin.security.ldap.user-search-pattern=(uid=\{0})
in your config is very suspicious, ldap may use this as username to search
group members. And please confirm is this within your expectations. And may you
use cn for a try?)
Hope you can find the truly root clause.
LDIF
{code:java}
# People, example.com
dn: ou=People,dc=example,dc=com
ou: People
cn: People
objectClass: organizationalRole
objectClass: top
# jenny, People, example.com
dn: cn=jenny,ou=People,dc=example,dc=com
mail: je...@example.io
ou: Analyst
cn: jenny
sn: jenny liu
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: top
userPassword:: ZXhhbXBsZTEyMw==
# admin, Groups, example.com
dn: cn=admin,ou=Groups,dc=example,dc=com
cn: admin
member: cn=jenny,ou=People,dc=example,dc=com
objectClass: groupOfNames
objectClass: top
{code}
{code:java}
kylin.security.ldap.user-search-base=ou=People,dc=example,dc=com
kylin.security.ldap.user-group-search-base=ou=Groups,dc=example,dc=com
kylin.security.ldap.user-search-pattern=(&(cn={0}))
kylin.security.ldap.user-group-search-filter=(|(member={0})(memberUid={1}))
kylin.security.acl.admin-role=admin
{code}
> When ldap is opened, I use an ignored case user to login, the page does not
> respond.
>
>
> Key: KYLIN-3197
> URL: https://issues.apache.org/jira/browse/KYLIN-3197
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.3.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
> Labels: patch
> Fix For: Future
>
> Attachments:
> 0001-KYLIN-3197-When-ldap-is-opened-I-use-an-ignored-case.patch,
> image-2018-01-25-17-22-39-970.png, image-2018-02-06-14-09-32-591.png,
> image-2018-02-08-15-32-25-030.png, image-2018-02-08-15-33-07-277.png,
> image-2018-02-08-15-33-54-480.png, image-2018-02-08-15-35-03-902.png,
> image-2018-02-12-12-15-00-574.png, image-2018-02-12-12-15-28-826.png,
> image-2018-02-12-12-15-39-132.png, image-2018-02-12-12-25-15-793.png
>
>
> When ldap is opened, I config the kylin.properties, and give wkhGroup the
> admin permission.
> {code:java}
> ## Admin roles in LDAP, for ldap and saml
> kylin.security.acl.admin-role=wkhGroup
> {code}
> then I create a new user named 'wkh' whose group is 'wkhGroup', then I use
>
