[GitHub] [spark] HeartSaVioR edited a comment on pull request #28326: [SPARK-27340][SS] Alias on TimeWindow expression cause watermark metadata lost

2020-04-27 Thread GitBox
HeartSaVioR edited a comment on pull request #28326: URL: https://github.com/apache/spark/pull/28326#issuecomment-620310834 Sorry for maybe out-of-topic, but I'm seeing that column metadata may have more issues due to not properly defined its usage and how it should be handled. I'm a

[GitHub] [spark] HeartSaVioR edited a comment on pull request #28326: [SPARK-27340][SS] Alias on TimeWindow expression cause watermark metadata lost

2020-04-27 Thread GitBox
HeartSaVioR edited a comment on pull request #28326: URL: https://github.com/apache/spark/pull/28326#issuecomment-620310834 Sorry for maybe out-of-topic, but I'm seeing that column metadata may have more issues due to not properly defined its usage and how it should be handled. I'm a

[GitHub] [spark] HeartSaVioR edited a comment on pull request #28326: [SPARK-27340][SS] Alias on TimeWindow expression cause watermark metadata lost

2020-04-27 Thread GitBox
HeartSaVioR edited a comment on pull request #28326: URL: https://github.com/apache/spark/pull/28326#issuecomment-619831846 > Each Attribute/Alias has its own metadata and can easily be hidden by the outer-most Alias. Yeah I see the concern - I'm not sure the column metadata was cons

[GitHub] [spark] HeartSaVioR edited a comment on pull request #28326: [SPARK-27340][SS] Alias on TimeWindow expression cause watermark metadata lost

2020-04-26 Thread GitBox
HeartSaVioR edited a comment on pull request #28326: URL: https://github.com/apache/spark/pull/28326#issuecomment-619751876 > Sometimes we have to, for example, a + b as c, what should be the metadata of c? The metadata of expression for `a + b` if there's no explicit set of metadat

[GitHub] [spark] HeartSaVioR edited a comment on pull request #28326: [SPARK-27340][SS] Alias on TimeWindow expression cause watermark metadata lost

2020-04-26 Thread GitBox
HeartSaVioR edited a comment on pull request #28326: URL: https://github.com/apache/spark/pull/28326#issuecomment-619751876 > Sometimes we have to, for example, a + b as c, what should be the metadata of c? The metadata of expression for `a + b` if there's no explicit set of metadat