Got it, thank you Jia for noticing that!

Youngwoo

On Tue, Feb 2, 2021 at 11:56 AM Jia Yu <jiayu198...@gmail.com> wrote:

> Hi Youngwoo,
>
> We are aware that the current SQL spatial join does not support streaming
> tables. The Sedona support of Spark structured streaming is on our roadmap
> but it won't be available soon.
>
> Thanks,
> Jia
>
> On Sun, Jan 31, 2021 at 9:31 PM Youngwoo Kim (김영우) <yw...@apache.org>
> wrote:
>
>> Hi,
>>
>> I'm looking into spatial range query using Apache Sedona in spark
>> structured streaming. In my test code, I joined two streams using
>> `st_contains` like following:
>>
>> Dataset<Row> fenced = spark.sql(
>> >   "SELECT * FROM poly, pts WHERE
>> > ST_Contains(ST_PolygonFromText(poly.value,','),
>> > ST_PointFromText(pts.value,','))");
>>
>>
>> I created two streams from `sparkSession.readStream()` and created two
>> views 'poly' and 'pts' respectively. As you expect, I wanted to know
>> whether the polygon contains the point.
>>
>> However, It does not work and I get an exception like this:
>>
>> Caused by: org.apache.spark.sql.AnalysisException: Stream-stream join
>> > without equality predicate is not supported;;
>> >   Join Inner,
>> >  **org.apache.spark.sql.geosparksql.expressions.ST_Contains$**
>>
>>
>> Looks like structured streaming has a restriction on that kind of join
>> query.
>>
>> Please let me know if there is a workaround for spatial joins and also,
>> any
>> plans to support joins in  structured streaming from Sedona.
>>
>> Spark 3.0.1 and GeoSpark 1.3.2-SNAPSHOT for my test env.
>>
>> Thanks,
>> Youngwoo
>>
>

Reply via email to