Hi Piyush,

Could you verify that the type of the `timestamp` field in the table
my_kafka_stream is of TIMESTAMP(3) *ROWTIME* type? Could you share how
you create this table?

What you are doing should work and what I suspect is that the
`timestamp` field in the `my_kafka_stream` changed the type somehow.

Best,

Dawid

On 11/11/2019 22:43, Piyush Narang wrote:
>
> Hi folks,
>
>  
>
> We have a Flink streaming Table / SQL job that we were looking to
> migrate from an older Flink release (1.6.x) to 1.9. As part of doing
> so, we have been seeing a few errors which I was trying to figure out
> how to work around. Would appreciate any help / pointers.
>
> Job essentially involves a nested query:
>
> SELECT `timestamp`, cost, partnerid, impression_id, …
>
> FROM my_kafka_stream
>
>  
>
> The kafka stream has a ‘timestamp’ field that tracks event time. We
> register this nested query as “base_query”.
>
>  
>
> We now use this in a couple of outer aggregation queries (different
> outer aggregation queries differ in terms of the time window we
> aggregate over – 1M, 1H, 6H etc):
>
> SELECT
>
>   SUM(cost) AS FLOAT AS CostPerPartner,
>
>   COUNT(impression_id) AS ImpsPerPartner,
>
>   …
>
> FROM
>
>   base_query
>
> GROUP BY
>
>   partnerid,
>
>   HOP(`timestamp`, INTERVAL '30' SECOND, INTERVAL '1' MINUTE)
>
>  
>
> While the outer query would get translated and scheduled as a Flink
> streaming job just fine on 1.6, we are running into this error when we
> try to bump our build to 1.9:
>
> “Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Window can only be
> defined over a time attribute column.”
>
>  
>
> Any suggestions on how we could work around this? I saw a thread
> suggesting using HOP_ROWTIME but if I understand correctly, that would
> mean we would need to do the hop window generation / group by in the
> nested query which we’d like to avoid (as we have a couple of time
> window combinations to generate).
>
>  
>
> Thanks,
>
> -- Piyush
>
>  
>

Attachment: signature.asc
Description: OpenPGP digital signature

Reply via email to