[
https://issues.apache.org/jira/browse/KYLIN-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17375347#comment-17375347
]
Congling Xia commented on KYLIN-5021:
-------------------------------------
You are right, [~tianhui5]! That is a bug for Kylin uses Spark3. I'll change
the scope of the issue.
The implementation of AttributeSet in Spark3 is slightly different with the one
in Spark2.
In Spark2:
{code:java}
class AttributeSet private (val baseSet: Set[AttributeEquals]) extends
Traversable[Attribute] with Serializable
{code}
And in Spark3:
{code:java}
class AttributeSet private (private val baseSet:
mutable.LinkedHashSet[AttributeEquals]) extends Iterable[Attribute] with
Serializable
{code}
Spark 3 uses HashSet (rather than immutable.Set in Spark2) as the inner
container of AttributeSet. Hash code will be calculated when adding element to
it. So NPE will be raised when constructing AttributeSet with null.
> FilePruner throws NPE when there is no timePartitionColumn in cube
> ------------------------------------------------------------------
>
> Key: KYLIN-5021
> URL: https://issues.apache.org/jira/browse/KYLIN-5021
> Project: Kylin
> Issue Type: Bug
> Reporter: tianhui
> Priority: Major
> Attachments: errorlog, image-2021-07-01-11-58-11-533.png,
> image-2021-07-01-12-04-46-905.png
>
>
> I create a cube without partition clumn in it, then I query it and get a NPE.
> !image-2021-07-01-11-58-11-533.png!
> !image-2021-07-01-12-04-46-905.png!
> The SQL is like: select pid from exception_middle_user where
> EXCEPTION_MIDDLE_USER.TVVERSION ='20.10.21.110'
> I have put errorstack in Attachments.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
