Re: First Sedona release
Thx Jia. Then it falls under optional use https://www.apache.org/legal/resolved.html#optional Which is totally fine. On Mon, Nov 23, 2020 at 6:50 PM Jia Yu wrote: > Thank you, Felix. I will use the WIP disclaimer. > > To answer Jim's question, GeoTools components use different licenses: > https://docs.geotools.org/latest/userguide/welcome/license.html > > GT-main uses BSD, so its binary can be included in Sedona's release. > Other components in GeoTools use LGPL, but Sedona only uses them for CRS > transformation. I already set the dependency scope to "provided" in > Sedona's POM.xml. If a user wants to use CRS transformation in Sedona, they > will have to add some GeoTools library by themselves. > > > On Mon, Nov 23, 2020 at 6:24 PM Felix Cheung > wrote: > > > On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung > > wrote: > > > > > I’d strongly recommend the community to move towards the first release > > > with the WIP disclaimer > > > > > > > > > https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer > > > > > > https://incubator.apache.org/policy/incubation.html#releases > > > > > > > > > As for the LGPL dependency specifically, a replacement will be needed? > > > > > > > > > To clarify, ok to note in the WIP disclaimer- so it can be released with > > this. > > > > > > > > > > > > On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes wrote: > > > > > >> Hi all, > > >> > > >> Has the fact that one of the dependencies is LGPL (GeoTools) been > > >> discussed / addressed? (See > > >> https://www.apache.org/legal/resolved.html#category-x) > > >> > > >> I'm asking since I don't know if the ASF has any recommended work > > >> arounds for shipping code with licenses that it does not approve of. > > >> > > >> Cheers, > > >> > > >> Jim > > >> > > >> On 11/23/20 1:41 PM, Felix Cheung wrote: > > >> > I can help review around Dev 13 to give a first pass. It should give > > >> you an > > >> > easier path to IPMC vote. > > >> > > > >> > > > >> > On Sun, Nov 22, 2020 at 10:50 PM Jia Yu > > wrote: > > >> > > > >> >> Hi Pawel and everyone, > > >> >> > > >> >> Let's do this in the first Sedona release. But can you please first > > >> fix the > > >> >> Python API for our Move-to-JTS PR, and then work on this one? If > this > > >> >> Python RDD-DF Adapter PR might slow down our progress of releasing > > >> Sedona > > >> >> before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. > > >> >> > > >> >> @everyone > > >> >> Our top priority is to draw the first Sedona release ASAP. Users > have > > >> been > > >> >> waiting for almost six months. Let's push hard to publish the first > > >> Sedona > > >> >> release to Maven Central and PyPI before Christmas. In order to > make > > it > > >> >> happen, > > >> >> > > >> >> Finalize coding and documentation before Dec 6: > > >> >> 1. I believe the Move-to-JTS PR will be done in around one week. > > >> >> 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary > > >> >> 3. I will work on Sedona documentation. > > >> >> 4. @Netanel will work on Sedona support of Spark 2.4 and Scala > 2.11. > > I > > >> will > > >> >> first create a branch for it to illustrate some necessary changes > in > > >> Sedona > > >> >> SQL for Spark 2.4. > > >> >> > > >> >> Final walk-through before Dec 13 > > >> >> 1. Netanel can test the release management for Sedona. > > >> >> 2. Other committers can go through the docs, release notes > > >> >> > > >> >> Community voting before Dec 20 > > >> >> 1. Sedona community voting: before Dec 16 > > >> >> 2. Apache Incubator voting: before Dec 20 > > >> >> > > >> >> Push to Maven Central and PyPi before Dec 24 > > >> >> > > >> >> Please feel free to comment if you have any suggestions! > > >> >> > > >> >> Jia > > >> >> > > >> >> On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński < > > >> pawel93kocin...@gmail.com> > > >> >> wrote: > > >> >> > > >> >>> Hi, > > >> >>> I saw some users reported need to improve Python RDD API in two > > >> >> scenarios: > > >> >>> - converting spatial flat join result to df > > >> >>> - saving spatial flat join result directly to external storage > > >> >>> > > >> >>> Currently SerDe between jvm and Python causes additional time > needed > > >> to > > >> >>> compute the result. I have a local branch with tests where this > > >> >>> functionality is available (need 3-4 days to make it 100% ready), > in > > >> two > > >> >>> above scenarios there will be almost no difference between Python > > and > > >> >> Scala > > >> >>> or Java API. Should I create PR to include this feature within the > > >> first > > >> >>> Sedona release ? > > >> >>> Regards, > > >> >>> Paweł > > >> >>> > > >> >>> pon., 16 lis 2020 o 08:29 Jia Yu > > napisał(a): > > >> >>> > > >> Dear all, > > >> > > >> Thanks for all your suggestions. > > >> > > >> 1. To completely solve the long-overdue JTS issue, I made a > Sedona > > PR > > >> >> and > > >> two JTS PRs. @Jim Hughes , @Paweł Kociński > > >>
Re: First Sedona release
Thank you, Felix. I will use the WIP disclaimer. To answer Jim's question, GeoTools components use different licenses: https://docs.geotools.org/latest/userguide/welcome/license.html GT-main uses BSD, so its binary can be included in Sedona's release. Other components in GeoTools use LGPL, but Sedona only uses them for CRS transformation. I already set the dependency scope to "provided" in Sedona's POM.xml. If a user wants to use CRS transformation in Sedona, they will have to add some GeoTools library by themselves. On Mon, Nov 23, 2020 at 6:24 PM Felix Cheung wrote: > On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung > wrote: > > > I’d strongly recommend the community to move towards the first release > > with the WIP disclaimer > > > > > https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer > > > > https://incubator.apache.org/policy/incubation.html#releases > > > > > > As for the LGPL dependency specifically, a replacement will be needed? > > > > > To clarify, ok to note in the WIP disclaimer- so it can be released with > this. > > > > > > > On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes wrote: > > > >> Hi all, > >> > >> Has the fact that one of the dependencies is LGPL (GeoTools) been > >> discussed / addressed? (See > >> https://www.apache.org/legal/resolved.html#category-x) > >> > >> I'm asking since I don't know if the ASF has any recommended work > >> arounds for shipping code with licenses that it does not approve of. > >> > >> Cheers, > >> > >> Jim > >> > >> On 11/23/20 1:41 PM, Felix Cheung wrote: > >> > I can help review around Dev 13 to give a first pass. It should give > >> you an > >> > easier path to IPMC vote. > >> > > >> > > >> > On Sun, Nov 22, 2020 at 10:50 PM Jia Yu > wrote: > >> > > >> >> Hi Pawel and everyone, > >> >> > >> >> Let's do this in the first Sedona release. But can you please first > >> fix the > >> >> Python API for our Move-to-JTS PR, and then work on this one? If this > >> >> Python RDD-DF Adapter PR might slow down our progress of releasing > >> Sedona > >> >> before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. > >> >> > >> >> @everyone > >> >> Our top priority is to draw the first Sedona release ASAP. Users have > >> been > >> >> waiting for almost six months. Let's push hard to publish the first > >> Sedona > >> >> release to Maven Central and PyPI before Christmas. In order to make > it > >> >> happen, > >> >> > >> >> Finalize coding and documentation before Dec 6: > >> >> 1. I believe the Move-to-JTS PR will be done in around one week. > >> >> 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary > >> >> 3. I will work on Sedona documentation. > >> >> 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. > I > >> will > >> >> first create a branch for it to illustrate some necessary changes in > >> Sedona > >> >> SQL for Spark 2.4. > >> >> > >> >> Final walk-through before Dec 13 > >> >> 1. Netanel can test the release management for Sedona. > >> >> 2. Other committers can go through the docs, release notes > >> >> > >> >> Community voting before Dec 20 > >> >> 1. Sedona community voting: before Dec 16 > >> >> 2. Apache Incubator voting: before Dec 20 > >> >> > >> >> Push to Maven Central and PyPi before Dec 24 > >> >> > >> >> Please feel free to comment if you have any suggestions! > >> >> > >> >> Jia > >> >> > >> >> On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński < > >> pawel93kocin...@gmail.com> > >> >> wrote: > >> >> > >> >>> Hi, > >> >>> I saw some users reported need to improve Python RDD API in two > >> >> scenarios: > >> >>> - converting spatial flat join result to df > >> >>> - saving spatial flat join result directly to external storage > >> >>> > >> >>> Currently SerDe between jvm and Python causes additional time needed > >> to > >> >>> compute the result. I have a local branch with tests where this > >> >>> functionality is available (need 3-4 days to make it 100% ready), in > >> two > >> >>> above scenarios there will be almost no difference between Python > and > >> >> Scala > >> >>> or Java API. Should I create PR to include this feature within the > >> first > >> >>> Sedona release ? > >> >>> Regards, > >> >>> Paweł > >> >>> > >> >>> pon., 16 lis 2020 o 08:29 Jia Yu > napisał(a): > >> >>> > >> Dear all, > >> > >> Thanks for all your suggestions. > >> > >> 1. To completely solve the long-overdue JTS issue, I made a Sedona > PR > >> >> and > >> two JTS PRs. @Jim Hughes , @Paweł Kociński > >> , I, and probably Martin from JTS will > >> take > >> care of these PRs in the coming days. > >> (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 > >> (2) JTS PR: https://github.com/locationtech/jts/pull/633 > >> https://github.com/locationtech/jts/pull/634 > >> > >> 2. To move forward with the first release, I have deleted the > >> "SNAPSHOT" > >> in my JTS 1.16 fork. > >> Most likely, we have to
Re: First Sedona release
On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung wrote: > I’d strongly recommend the community to move towards the first release > with the WIP disclaimer > > https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer > > https://incubator.apache.org/policy/incubation.html#releases > > > As for the LGPL dependency specifically, a replacement will be needed? > To clarify, ok to note in the WIP disclaimer- so it can be released with this. > > On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes wrote: > >> Hi all, >> >> Has the fact that one of the dependencies is LGPL (GeoTools) been >> discussed / addressed? (See >> https://www.apache.org/legal/resolved.html#category-x) >> >> I'm asking since I don't know if the ASF has any recommended work >> arounds for shipping code with licenses that it does not approve of. >> >> Cheers, >> >> Jim >> >> On 11/23/20 1:41 PM, Felix Cheung wrote: >> > I can help review around Dev 13 to give a first pass. It should give >> you an >> > easier path to IPMC vote. >> > >> > >> > On Sun, Nov 22, 2020 at 10:50 PM Jia Yu wrote: >> > >> >> Hi Pawel and everyone, >> >> >> >> Let's do this in the first Sedona release. But can you please first >> fix the >> >> Python API for our Move-to-JTS PR, and then work on this one? If this >> >> Python RDD-DF Adapter PR might slow down our progress of releasing >> Sedona >> >> before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. >> >> >> >> @everyone >> >> Our top priority is to draw the first Sedona release ASAP. Users have >> been >> >> waiting for almost six months. Let's push hard to publish the first >> Sedona >> >> release to Maven Central and PyPI before Christmas. In order to make it >> >> happen, >> >> >> >> Finalize coding and documentation before Dec 6: >> >> 1. I believe the Move-to-JTS PR will be done in around one week. >> >> 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary >> >> 3. I will work on Sedona documentation. >> >> 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. I >> will >> >> first create a branch for it to illustrate some necessary changes in >> Sedona >> >> SQL for Spark 2.4. >> >> >> >> Final walk-through before Dec 13 >> >> 1. Netanel can test the release management for Sedona. >> >> 2. Other committers can go through the docs, release notes >> >> >> >> Community voting before Dec 20 >> >> 1. Sedona community voting: before Dec 16 >> >> 2. Apache Incubator voting: before Dec 20 >> >> >> >> Push to Maven Central and PyPi before Dec 24 >> >> >> >> Please feel free to comment if you have any suggestions! >> >> >> >> Jia >> >> >> >> On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński < >> pawel93kocin...@gmail.com> >> >> wrote: >> >> >> >>> Hi, >> >>> I saw some users reported need to improve Python RDD API in two >> >> scenarios: >> >>> - converting spatial flat join result to df >> >>> - saving spatial flat join result directly to external storage >> >>> >> >>> Currently SerDe between jvm and Python causes additional time needed >> to >> >>> compute the result. I have a local branch with tests where this >> >>> functionality is available (need 3-4 days to make it 100% ready), in >> two >> >>> above scenarios there will be almost no difference between Python and >> >> Scala >> >>> or Java API. Should I create PR to include this feature within the >> first >> >>> Sedona release ? >> >>> Regards, >> >>> Paweł >> >>> >> >>> pon., 16 lis 2020 o 08:29 Jia Yu napisał(a): >> >>> >> Dear all, >> >> Thanks for all your suggestions. >> >> 1. To completely solve the long-overdue JTS issue, I made a Sedona PR >> >> and >> two JTS PRs. @Jim Hughes , @Paweł Kociński >> , I, and probably Martin from JTS will >> take >> care of these PRs in the coming days. >> (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 >> (2) JTS PR: https://github.com/locationtech/jts/pull/633 >> https://github.com/locationtech/jts/pull/634 >> >> 2. To move forward with the first release, I have deleted the >> "SNAPSHOT" >> in my JTS 1.16 fork. >> Most likely, we have to move forward with my JTS 1.16 fork in the >> first >> Sedona release because of the conflict among JTStoGeoJSON, GeoTools, >> and >> JTS 1.17. >> So @Netanel Malka could you please do >> another >> dry-run on the Sedona first release on this Sedona branch: >> >> sedona-1.0-doc: >> https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc >> >> Thanks, >> Jia >> >> On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes >> wrote: >> >> > Hi Mo, >> > >> > I can definitely help. The first step will be for Jia to push a PR >> for >> > the JTS changes. (Since they are his changes, I cannot do this on >> his >> > behalf.) >> > >> > From talking to the lead JTS developer, he wanted to see the >> previous >> > PR (from months/a year+ ago) split up. I think
Re: First Sedona release
I’d strongly recommend the community to move towards the first release with the WIP disclaimer https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer https://incubator.apache.org/policy/incubation.html#releases As for the LGPL dependency specifically, a replacement will be needed? On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes wrote: > Hi all, > > Has the fact that one of the dependencies is LGPL (GeoTools) been > discussed / addressed? (See > https://www.apache.org/legal/resolved.html#category-x) > > I'm asking since I don't know if the ASF has any recommended work > arounds for shipping code with licenses that it does not approve of. > > Cheers, > > Jim > > On 11/23/20 1:41 PM, Felix Cheung wrote: > > I can help review around Dev 13 to give a first pass. It should give you > an > > easier path to IPMC vote. > > > > > > On Sun, Nov 22, 2020 at 10:50 PM Jia Yu wrote: > > > >> Hi Pawel and everyone, > >> > >> Let's do this in the first Sedona release. But can you please first fix > the > >> Python API for our Move-to-JTS PR, and then work on this one? If this > >> Python RDD-DF Adapter PR might slow down our progress of releasing > Sedona > >> before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. > >> > >> @everyone > >> Our top priority is to draw the first Sedona release ASAP. Users have > been > >> waiting for almost six months. Let's push hard to publish the first > Sedona > >> release to Maven Central and PyPI before Christmas. In order to make it > >> happen, > >> > >> Finalize coding and documentation before Dec 6: > >> 1. I believe the Move-to-JTS PR will be done in around one week. > >> 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary > >> 3. I will work on Sedona documentation. > >> 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. I > will > >> first create a branch for it to illustrate some necessary changes in > Sedona > >> SQL for Spark 2.4. > >> > >> Final walk-through before Dec 13 > >> 1. Netanel can test the release management for Sedona. > >> 2. Other committers can go through the docs, release notes > >> > >> Community voting before Dec 20 > >> 1. Sedona community voting: before Dec 16 > >> 2. Apache Incubator voting: before Dec 20 > >> > >> Push to Maven Central and PyPi before Dec 24 > >> > >> Please feel free to comment if you have any suggestions! > >> > >> Jia > >> > >> On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński < > pawel93kocin...@gmail.com> > >> wrote: > >> > >>> Hi, > >>> I saw some users reported need to improve Python RDD API in two > >> scenarios: > >>> - converting spatial flat join result to df > >>> - saving spatial flat join result directly to external storage > >>> > >>> Currently SerDe between jvm and Python causes additional time needed to > >>> compute the result. I have a local branch with tests where this > >>> functionality is available (need 3-4 days to make it 100% ready), in > two > >>> above scenarios there will be almost no difference between Python and > >> Scala > >>> or Java API. Should I create PR to include this feature within the > first > >>> Sedona release ? > >>> Regards, > >>> Paweł > >>> > >>> pon., 16 lis 2020 o 08:29 Jia Yu napisał(a): > >>> > Dear all, > > Thanks for all your suggestions. > > 1. To completely solve the long-overdue JTS issue, I made a Sedona PR > >> and > two JTS PRs. @Jim Hughes , @Paweł Kociński > , I, and probably Martin from JTS will > take > care of these PRs in the coming days. > (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 > (2) JTS PR: https://github.com/locationtech/jts/pull/633 > https://github.com/locationtech/jts/pull/634 > > 2. To move forward with the first release, I have deleted the > "SNAPSHOT" > in my JTS 1.16 fork. > Most likely, we have to move forward with my JTS 1.16 fork in the > first > Sedona release because of the conflict among JTStoGeoJSON, GeoTools, > and > JTS 1.17. > So @Netanel Malka could you please do another > dry-run on the Sedona first release on this Sedona branch: > >> sedona-1.0-doc: > https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc > > Thanks, > Jia > > On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes wrote: > > > Hi Mo, > > > > I can definitely help. The first step will be for Jia to push a PR > for > > the JTS changes. (Since they are his changes, I cannot do this on > his > > behalf.) > > > > From talking to the lead JTS developer, he wanted to see the > previous > > PR (from months/a year+ ago) split up. I think the initial PR should > >> be > > used to discuss what changes are sensible for JTS and where we'll > need > > to push some of the changes to Sedona. > > > > Concretely, I noticed that the Sedona JTS fork changes the toString > on > > Geometry to include printing out the
Re: First Sedona release
Hi all, Has the fact that one of the dependencies is LGPL (GeoTools) been discussed / addressed? (See https://www.apache.org/legal/resolved.html#category-x) I'm asking since I don't know if the ASF has any recommended work arounds for shipping code with licenses that it does not approve of. Cheers, Jim On 11/23/20 1:41 PM, Felix Cheung wrote: I can help review around Dev 13 to give a first pass. It should give you an easier path to IPMC vote. On Sun, Nov 22, 2020 at 10:50 PM Jia Yu wrote: Hi Pawel and everyone, Let's do this in the first Sedona release. But can you please first fix the Python API for our Move-to-JTS PR, and then work on this one? If this Python RDD-DF Adapter PR might slow down our progress of releasing Sedona before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. @everyone Our top priority is to draw the first Sedona release ASAP. Users have been waiting for almost six months. Let's push hard to publish the first Sedona release to Maven Central and PyPI before Christmas. In order to make it happen, Finalize coding and documentation before Dec 6: 1. I believe the Move-to-JTS PR will be done in around one week. 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary 3. I will work on Sedona documentation. 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. I will first create a branch for it to illustrate some necessary changes in Sedona SQL for Spark 2.4. Final walk-through before Dec 13 1. Netanel can test the release management for Sedona. 2. Other committers can go through the docs, release notes Community voting before Dec 20 1. Sedona community voting: before Dec 16 2. Apache Incubator voting: before Dec 20 Push to Maven Central and PyPi before Dec 24 Please feel free to comment if you have any suggestions! Jia On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński wrote: Hi, I saw some users reported need to improve Python RDD API in two scenarios: - converting spatial flat join result to df - saving spatial flat join result directly to external storage Currently SerDe between jvm and Python causes additional time needed to compute the result. I have a local branch with tests where this functionality is available (need 3-4 days to make it 100% ready), in two above scenarios there will be almost no difference between Python and Scala or Java API. Should I create PR to include this feature within the first Sedona release ? Regards, Paweł pon., 16 lis 2020 o 08:29 Jia Yu napisał(a): Dear all, Thanks for all your suggestions. 1. To completely solve the long-overdue JTS issue, I made a Sedona PR and two JTS PRs. @Jim Hughes , @Paweł Kociński , I, and probably Martin from JTS will take care of these PRs in the coming days. (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 (2) JTS PR: https://github.com/locationtech/jts/pull/633 https://github.com/locationtech/jts/pull/634 2. To move forward with the first release, I have deleted the "SNAPSHOT" in my JTS 1.16 fork. Most likely, we have to move forward with my JTS 1.16 fork in the first Sedona release because of the conflict among JTStoGeoJSON, GeoTools, and JTS 1.17. So @Netanel Malka could you please do another dry-run on the Sedona first release on this Sedona branch: sedona-1.0-doc: https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc Thanks, Jia On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes wrote: Hi Mo, I can definitely help. The first step will be for Jia to push a PR for the JTS changes. (Since they are his changes, I cannot do this on his behalf.) From talking to the lead JTS developer, he wanted to see the previous PR (from months/a year+ ago) split up. I think the initial PR should be used to discuss what changes are sensible for JTS and where we'll need to push some of the changes to Sedona. Concretely, I noticed that the Sedona JTS fork changes the toString on Geometry to include printing out the userData. I imagine that may cause trouble for downstream JTS users, so it'd be good to find an alternative. One suggestion would to be add a static method in Sedona for printing a Geometry with its userData object. Cheers, Jim On 11/12/20 12:32 PM, Mohamed Sarwat wrote: Folks, I totally agree with Jim on that. Jim, would you like to take the lead on that - I trust that you can bring this task to completion. Jia, would you please let us know how we can incorporate the changes into the JTS master branch? Thanks, On Nov 12, 2020, at 10:10 AM, Jim Hughes wrote: Hi all, As a JTS committer, I have tried to request that the Sedona project discuss the desired changes to JTS previously. I'd still encourage that. JTS is an active project and I feel that maintaining a fork of JTS is unnecessary and inappropriate. Cheers, Jim On 11/11/20 9:04 PM, Felix Cheung wrote: Ah. You will need to publish it in order for the dependency chain to work on Maven Central However, since you are not the project
