nirnir93 opened a new pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482
## Is this PR related to a proposed Issue?
#435 #233
## What changes were proposed in this PR?
Deprecating JTS+ in favor of using JTS as a dependency with wrappers
Updating JTS
Imbruced commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-686369822
I will look on this, today evening.
This is an automated message from the Apache Git Service.
To respon
jnh5y commented on a change in pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#discussion_r483034898
##
File path: pom.xml
##
@@ -45,7 +45,8 @@
UTF-8
2.12.8
2.12
-17.0
+23.2
+1.16.1
Review comment:
jiayuasu commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-688177724
@Imbruced Hello Pawel, any updates on this?
This is an automated message from the Apache Git Service.
To
gitter-badger opened a new pull request #483:
URL: https://github.com/apache/incubator-sedona/pull/483
### apache/incubator-sedona now has a Chat Room on Gitter
@jiayuasu has just created a chat room. You can visit it here:
[https://gitter.im/apache/sedona](https://gitter.im/apache/s
jiayuasu closed pull request #483:
URL: https://github.com/apache/incubator-sedona/pull/483
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
Imbruced commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-688506156
@jiayuasu I can see an issue. Python geometry data is translated to jvm
using previous serlializer which assume that Geometry has userData attribute.
Now it has not and
slf007 commented on pull request #210:
URL: https://github.com/apache/incubator-sedona/pull/210#issuecomment-688674494
Excuse me sir ,i am a PHD student in Zhejiang University , I have read a
paper about you concerning the tutorial of GeoSpark,
I wonder could please send me the link abou
jiayuasu commented on pull request #210:
URL: https://github.com/apache/incubator-sedona/pull/210#issuecomment-688683825
Our new website and contact info are clearly listed on this readme. The old
GeoSpark domain is no longer in use.
---
jiayuasu closed pull request #313:
URL: https://github.com/apache/incubator-sedona/pull/313
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
jiayuasu closed pull request #303:
URL: https://github.com/apache/incubator-sedona/pull/303
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
jiayuasu closed pull request #144:
URL: https://github.com/apache/incubator-sedona/pull/144
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
jiayuasu closed pull request #210:
URL: https://github.com/apache/incubator-sedona/pull/210
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
jiayuasu closed pull request #411:
URL: https://github.com/apache/incubator-sedona/pull/411
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
Imbruced commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-689119174
@jiayuasu I fixed it. But I have some doubts. Did anyone check how this PR
affects overall performance ? Do we need this transformations ?
@nirnir93 can you grant me
nirnir93 commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-689124460
@Imbruced I've added you
This is an automated message from the Apache Git Service.
To respond to the mes
nirnir93 commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-689125539
@Imbruced @jiayuasu
@netanel246 and I would welcome any input on performance as we tried out
best to keep the impact to the necessary minimum (considering some
optim
Imbruced commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-689131648
@nirnir93 I am still not able to push changes. :(
This is an automated message from the Apache Git Serv
Imbruced commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-689139826
@nirnir93 I added comments, one file had changes so I added comment within.
For 3 python files I can not find appropriate place so I put changes on top of
Pr changed fil
jiayuasu commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-689301376
@Imbruced Hello Pawel, I didn't see your latest update to that
"move_to_jts_1.16.1" branch. Where did you push your change to?
To me, the only part could affect th
nirnir93 commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-689345424
@Imbruced
- I don't know why you can't see your invite, [try this
link](https://github.com/deprecatejtsplus/GeoSpark/invitations)
- About the second issue - could
jiayuasu commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-689441598
@nirnir93 Ok, if so, I think this will significantly decrease our
performance. Our existing serializer outperforms Kryo in terms of speed and
memory footprint. It can se
jiayuasu edited a comment on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-689441598
@nirnir93 Ok, if so, I think this will significantly decrease our
performance. Our existing serializer outperforms Kryo in terms of speed and
memory footprint. It
nirnir93 commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-689455178
I'll try to re-add GeometrySerde tomorrow evening
This is an automated message from the Apache Git Servi
nirnir93 removed a comment on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-689455178
I'll try to re-add GeometrySerde tomorrow evening
This is an automated message from the Apache G
nirnir93 commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-689620479
@jiayuasu Should we revisit the idea of reflections? We can put all the
parts pertaining to reflections in a different library (JTSplus) and have it
depend on JTS, which
jiayuasu commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-689966366
@nirnir93 I think this is a better idea. Do you think this will work? If you
need, I will add you and @netanel246 to JTSplus contributors.
netanel246 commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-690246807
@jiayuasu We think it should work. What do you think of creating it as a
submodule inside Apache Sedona? this should be more convenient to manage.
---
jnh5y commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-690336850
What do you actually need changed in JTS? I'm a JTS and GeoMesa committer,
and I'd be interested in see Sedona work well with both of these projects.
Can you open is
jiayuasu commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-690657563
@jnh5y @netanel246 Hey Jim, we created an issue before:
https://github.com/locationtech/jts/issues/364
So let me try to create a PR on LocationTech JTS first and s
jnh5y commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-690667575
@jiayuasu cool. I'd suggest using an issue or conversation on Gitter first.
If I recall, there are a few things going on. There was something around
Envelope relate
netanel246 commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-698404338
@jiayuasu Hi Jia, any updates on this?
This is an automated message from the Apache Git Service.
To re
netanel246 commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-698404338
@jiayuasu Hi Jia, any updates on this?
This is an automated message from the Apache Git Service.
To re
jiayuasu commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-699064248
@netanel246 Sorry, was just busy these days. Over this weekend, I will
create a new fork and make a PR on JTS to merge JTSplus functions to JTS. If
some JTSplus function
jiayuasu commented on pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482#issuecomment-700532690
@netanel246 I am currently working on this issue.
I have created a new branch called jtsplus and am implementing my code
there. My plan is as follows:
1. Cr
jiayuasu opened a new pull request #484:
URL: https://github.com/apache/incubator-sedona/pull/484
## Is this PR related to a proposed Issue?
#482
#435
#233
## What changes were proposed in this PR?
Please see my proposal:
https://github.com/apache/incuba
jiayuasu commented on pull request #484:
URL: https://github.com/apache/incubator-sedona/pull/484#issuecomment-703430710
@Imbruced Hi Pawel, although I think there are no actual technical changes
to Sedona itself, it would be good if you can take a look at the Python part.
Most changes are
jiayuasu commented on pull request #484:
URL: https://github.com/apache/incubator-sedona/pull/484#issuecomment-703437703
@Imbruced Apparently, the Python test has failed because of the path issue.
But I believe it should be a quick fix. Can you fix it and push to the Sedona
"jtsplus" branc
jiayuasu edited a comment on pull request #484:
URL: https://github.com/apache/incubator-sedona/pull/484#issuecomment-703437703
@Imbruced Apparently, the Python test has failed
(https://travis-ci.org/github/apache/incubator-sedona/builds/732868873) because
of the path issue. But I believe
Imbruced commented on pull request #484:
URL: https://github.com/apache/incubator-sedona/pull/484#issuecomment-703504794
@jiayuasu I will take care of this today evening.
This is an automated message from the Apache Git Servi
netanel246 commented on a change in pull request #484:
URL: https://github.com/apache/incubator-sedona/pull/484#discussion_r499519523
##
File path: core/pom.xml
##
@@ -32,10 +32,16 @@
jar
+
+org.locationtech.jts
+jts-core
+
jiayuasu commented on a change in pull request #484:
URL: https://github.com/apache/incubator-sedona/pull/484#discussion_r500018075
##
File path: core/pom.xml
##
@@ -35,7 +35,7 @@
org.wololo
jts2geojson
-0.10.0
+0.13.0
jiayuasu commented on a change in pull request #484:
URL: https://github.com/apache/incubator-sedona/pull/484#discussion_r500019542
##
File path: core/pom.xml
##
@@ -32,10 +32,16 @@
jar
+
+org.locationtech.jts
+jts-core
+
netanel246 commented on a change in pull request #484:
URL: https://github.com/apache/incubator-sedona/pull/484#discussion_r500749898
##
File path: core/pom.xml
##
@@ -32,10 +32,16 @@
jar
+
+org.locationtech.jts
+jts-core
+
jiayuasu commented on pull request #484:
URL: https://github.com/apache/incubator-sedona/pull/484#issuecomment-706640944
@Imbruced Hi Pawel, could you please take a look at this? I tried to fix it
by myself but didn't succeed
---
Imbruced commented on pull request #484:
URL: https://github.com/apache/incubator-sedona/pull/484#issuecomment-706704336
@jiayuasu I was able to fix the issue on my local machine. I need to check
if CI works fine.
This is a
Imbruced opened a new pull request #485:
URL: https://github.com/apache/incubator-sedona/pull/485
## Is this PR related to a proposed Issue?
#484
## What changes were proposed in this PR?
Fix failing python tests
## How was this patch tested?
-
## Did this PR include n
Imbruced commented on pull request #485:
URL: https://github.com/apache/incubator-sedona/pull/485#issuecomment-706709177
@jiayuasu That's PR to resolve an issue with Python tests. But one thing
worries me. Can we add to POM, definition of fat jar for Python purposes ?
Currently to make it
Imbruced commented on pull request #485:
URL: https://github.com/apache/incubator-sedona/pull/485#issuecomment-706955991
@jiayuasu
1. Files are used by few tests.
2. I can try to create additional jar file which will be published on PyPI
with wheel files (consists of those missing de
jiayuasu commented on pull request #485:
URL: https://github.com/apache/incubator-sedona/pull/485#issuecomment-707294048
@Imbruced That sounds good!
Let's do it in a separate PR. The next PR in Sedona will be the name change
for all paths, then you can finalize the wheel file there.
jiayuasu merged pull request #485:
URL: https://github.com/apache/incubator-sedona/pull/485
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
jiayuasu merged pull request #484:
URL: https://github.com/apache/incubator-sedona/pull/484
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
jiayuasu closed pull request #482:
URL: https://github.com/apache/incubator-sedona/pull/482
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
jiayuasu opened a new pull request #486:
URL: https://github.com/apache/incubator-sedona/pull/486
## Is this PR related to a proposed Issue?
Prepare the first Sedona release: `1.0.0-incubator `
## What changes were proposed in this PR?
Change all paths to org.apache.sedo
jiayuasu commented on pull request #428:
URL: https://github.com/apache/incubator-sedona/pull/428#issuecomment-714829047
@netanel246 In this PR, you didn't touch anything for geometry serialization
in RDD API, right? It looks like you only add the WKB serializer to SQL API?
--
netanel246 commented on pull request #428:
URL: https://github.com/apache/incubator-sedona/pull/428#issuecomment-715105262
@jiayuasu yep. I didn't change the RDD API.
This is an automated message from the Apache Git Service.
Imbruced commented on pull request #486:
URL: https://github.com/apache/incubator-sedona/pull/486#issuecomment-716130892
@jiayuasu I can see that only viz tests are failing, so I started to work on
Python version. I should finish soon.
Imbruced commented on pull request #486:
URL: https://github.com/apache/incubator-sedona/pull/486#issuecomment-716162211
@jiayuasu Python version is almost there, I need to combine dependencies to
one package and fix one test, but I found out that one of geospark core tests
failing due to
jiayuasu commented on pull request #486:
URL: https://github.com/apache/incubator-sedona/pull/486#issuecomment-716285540
@Imbruced This is because when I cleaned up the code, somehow messed up the
test data files. The error in Viz is also caused by this.
I was very busy in the past w
jiayuasu commented on pull request #486:
URL: https://github.com/apache/incubator-sedona/pull/486#issuecomment-718456085
@Imbruced I have fixed the failed Viz test cases. But the core test does not
fail on my machine:
CRSTransformationTest.testPolygonDistanceJoinWithCRSTransformation:375
Imbruced commented on pull request #486:
URL: https://github.com/apache/incubator-sedona/pull/486#issuecomment-720366786
@jiayuasu I am almost there, today I will create PR.
This is an automated message from the Apache Git S
Imbruced edited a comment on pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487#issuecomment-720739657
@jiayuasu Only changes in documentation left to finish. Please look, if I
missed something. What actually has been done:
1 Python package name change geospark -
Imbruced commented on pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487#issuecomment-720739657
This is an automated message from the Apache Git Service.
To respond to the message, please log on to Git
Imbruced opened a new pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487
## Is this PR related to a proposed Issue?
It is related to existing PR [Sedona-3]
## What changes were proposed in this PR?
Package name change
## How was this patch tested?
Imbruced commented on pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487#issuecomment-721373900
@jiayuasu Can you restart my build ? I dont have such privileges, and I
think Travis build get stuck.
jiayuasu closed pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
jiayuasu commented on pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487#issuecomment-721378091
@Imbruced Unfortunately, I don't have the privilege as well. You can still
push new content to your branch.
If you are done and have nothing to commit any more, I
Imbruced commented on pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487#issuecomment-721870578
@jiayuasu It's ready to review.
This is an automated message from the Apache Git Service.
To respond to
Imbruced commented on pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487#issuecomment-723544195
@jiayuasu I dicided to add python-adapter for simplicity reasons, only
sedona-python-adapter is required to get all functionality. As you can see
within travis script.
jiayuasu merged pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
jiayuasu commented on pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487#issuecomment-723653296
@Imbruced I think the reason why travis-ci took a lot of time on "job
received" is that we are now under Apache organization on Travis-ci. This
organization has hundreds
jiayuasu merged pull request #486:
URL: https://github.com/apache/incubator-sedona/pull/486
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
jiayuasu opened a new pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488
## Is this PR related to a proposed Issue?
https://issues.apache.org/jira/browse/SEDONA-1
This PR is relevant to two JTS PRs made by me:
1. Add the check of userData in equa
jiayuasu commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-727784872
Note that, this PR depends on a JTS 1.16.2 fork:
https://github.com/jiayuasu/jts
1. Although we won't be able to use JTS 1.17 official release (due to the
issues
jiayuasu opened a new pull request #489:
URL: https://github.com/apache/incubator-sedona/pull/489
## Is this PR related to a proposed Issue?
## What changes were proposed in this PR?
## How was this patch tested?
## Did this PR include necessary documentation updates?
jiayuasu merged pull request #489:
URL: https://github.com/apache/incubator-sedona/pull/489
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
Imbruced commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-729245981
@jiayuasu I was able to fix that, need to add some additional tests and
tomorrow I create PR.
This is a
jiayuasu commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-731028505
## Status
This PR is relevant to two JTS PRs made by me:
1. Add the check of userData in equals(object o): locationtech/jts#633 :
**Check UserData in Geomet
jnh5y commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-731441483
@jiayuasu related to the partitioning, is it possible configure a
pre-determined grid to use when partitioning?
-
jiayuasu commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-731953167
Hi all @Imbruced @netanel246 @Sarwat
I managed to fix all the issues. In a nutshell:
## Changes on JTS side
This PR is relevant to two JTS PRs made b
Imbruced commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-733271657
@jiayuasu I am working on that, should be ready within 1-2 days.
This is an automated message from the A
jiayuasu commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-734439471
> ## To-Dos
> 1. Wait for JTS to accept the PR and release JTS 1.18 on Maven Central.
Then I will change the JTS dependency in Sedona to its Maven coordinate. I
expec
jiayuasu opened a new pull request #490:
URL: https://github.com/apache/incubator-sedona/pull/490
## Is this PR related to a proposed Issue?
https://issues.apache.org/jira/browse/SEDONA-3
## What changes were proposed in this PR?
1. Add DISCLAIMER-WIP
2. remove useles
jiayuasu merged pull request #490:
URL: https://github.com/apache/incubator-sedona/pull/490
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
Imbruced commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-735367128
**Ad 2. The duplicates preserving strategy in
JoinQuery.SpatialJoinQuery/DistanceJoinQuery is changed.**
Don't you think that will be the most common question from the
jiayuasu commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-735443464
> **Ad 2. The duplicates preserving strategy in
JoinQuery.SpatialJoinQuery/DistanceJoinQuery is changed.**
> Don't you think that will be the most common question from
jiayuasu commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-735503455
@Imbruced I have fixed this issue. So now SpatialJoinQuery/DistanceJoinQuery
will preserve all duplicates that originally present in both input datasets.
This is even be
Imbruced commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-736072819
@jiayuasu do we need **countWithoutDuplicates** method within SpatialRDD
objects ? It is inconsistent with spatial joins geometry equality checking.
Also it is not suita
Imbruced commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-736073943
@jiayuasu Python version is fixed. I think there is no required docs
changes.
This is an automated mes
Imbruced edited a comment on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-736073943
@jiayuasu Python version is fixed. I think there is no required docs
changes. I start to work on Adapter speed up within Python version.
jiayuasu commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-736087633
@Imbruced countWithoutDuplicates code has been completely removed. Because,
in this new PR, there is no way to support both count with / without dup. We
only have the fo
jiayuasu edited a comment on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-736087633
@Imbruced countWithoutDuplicates code has been completely removed. Because,
in this new PR, there is no way to support both count with / without dup. We
only have
jiayuasu commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-737010119
@Imbruced The Python API failed the test cases...
This is an automated message from the Apache Git Servi
Imbruced commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-737052626
@jiayuasu To me it looks like issue with travis itself. On my local machine
all tests passed.
This is a
jiayuasu commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-737148727
@Imbruced The timeout always happens at
python/core/test_core_spatial_relations.py. The join query particularly calls
the the PythoAdapter.translateSpatialPairRDDWithLis
Imbruced commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-737165865
@jiayuasu Within the tests we are using default spark configuration
(Python). Changing hashset to list increased number of instances needed to
serialize from jvm to Pyth
Imbruced edited a comment on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-737165865
@jiayuasu Within the tests we are using default spark configuration
(Python). Changing hashset to list increased number of instances needed to
serialize from jvm
Imbruced commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-737166598
Also I will make how hashset -> list impacted on memory consumption while
SerDE.
This is an automated m
Imbruced commented on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-737506676
My build getting stuck. Only one thing which should be changed
within python/tests/core/test_core_spatial_relations.py
```python
point_rdd.spatialPartition
Imbruced edited a comment on pull request #488:
URL: https://github.com/apache/incubator-sedona/pull/488#issuecomment-737506676
My build getting stuck (it is queued more than 2 hours). Only one thing
which should be changed
within python/tests/core/test_core_spatial_relations.py
1 - 100 of 1498 matches
Mail list logo
