I can help review around Dev 13 to give a first pass. It should give you an easier path to IPMC vote.
On Sun, Nov 22, 2020 at 10:50 PM Jia Yu <jiayu198...@gmail.com> wrote: > Hi Pawel and everyone, > > Let's do this in the first Sedona release. But can you please first fix the > Python API for our Move-to-JTS PR, and then work on this one? If this > Python RDD-DF Adapter PR might slow down our progress of releasing Sedona > before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. > > @everyone > Our top priority is to draw the first Sedona release ASAP. Users have been > waiting for almost six months. Let's push hard to publish the first Sedona > release to Maven Central and PyPI before Christmas. In order to make it > happen, > > Finalize coding and documentation before Dec 6: > 1. I believe the Move-to-JTS PR will be done in around one week. > 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary > 3. I will work on Sedona documentation. > 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. I will > first create a branch for it to illustrate some necessary changes in Sedona > SQL for Spark 2.4. > > Final walk-through before Dec 13 > 1. Netanel can test the release management for Sedona. > 2. Other committers can go through the docs, release notes > > Community voting before Dec 20 > 1. Sedona community voting: before Dec 16 > 2. Apache Incubator voting: before Dec 20 > > Push to Maven Central and PyPi before Dec 24 > > Please feel free to comment if you have any suggestions! > > Jia > > On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński <pawel93kocin...@gmail.com> > wrote: > > > Hi, > > I saw some users reported need to improve Python RDD API in two > scenarios: > > - converting spatial flat join result to df > > - saving spatial flat join result directly to external storage > > > > Currently SerDe between jvm and Python causes additional time needed to > > compute the result. I have a local branch with tests where this > > functionality is available (need 3-4 days to make it 100% ready), in two > > above scenarios there will be almost no difference between Python and > Scala > > or Java API. Should I create PR to include this feature within the first > > Sedona release ? > > Regards, > > Paweł > > > > pon., 16 lis 2020 o 08:29 Jia Yu <jiayu198...@gmail.com> napisał(a): > > > >> Dear all, > >> > >> Thanks for all your suggestions. > >> > >> 1. To completely solve the long-overdue JTS issue, I made a Sedona PR > and > >> two JTS PRs. @Jim Hughes <jhug...@ccri.com> , @Paweł Kociński > >> <pawel93kocin...@gmail.com> , I, and probably Martin from JTS will take > >> care of these PRs in the coming days. > >> (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 > >> (2) JTS PR: https://github.com/locationtech/jts/pull/633 > >> https://github.com/locationtech/jts/pull/634 > >> > >> 2. To move forward with the first release, I have deleted the "SNAPSHOT" > >> in my JTS 1.16 fork. > >> Most likely, we have to move forward with my JTS 1.16 fork in the first > >> Sedona release because of the conflict among JTStoGeoJSON, GeoTools, and > >> JTS 1.17. > >> So @Netanel Malka <netanel...@gmail.com> could you please do another > >> dry-run on the Sedona first release on this Sedona branch: > sedona-1.0-doc: > >> https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc > >> > >> Thanks, > >> Jia > >> > >> On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes <jhug...@ccri.com> wrote: > >> > >>> Hi Mo, > >>> > >>> I can definitely help. The first step will be for Jia to push a PR for > >>> the JTS changes. (Since they are his changes, I cannot do this on his > >>> behalf.) > >>> > >>> From talking to the lead JTS developer, he wanted to see the previous > >>> PR (from months/a year+ ago) split up. I think the initial PR should > be > >>> used to discuss what changes are sensible for JTS and where we'll need > >>> to push some of the changes to Sedona. > >>> > >>> Concretely, I noticed that the Sedona JTS fork changes the toString on > >>> Geometry to include printing out the userData. I imagine that may > cause > >>> trouble for downstream JTS users, so it'd be good to find an > >>> alternative. One suggestion would to be add a static method in Sedona > >>> for printing a Geometry with its userData object. > >>> > >>> Cheers, > >>> > >>> Jim > >>> > >>> On 11/12/20 12:32 PM, Mohamed Sarwat wrote: > >>> > Folks, > >>> > > >>> > I totally agree with Jim on that. Jim, would you like to take the > lead > >>> on that - I trust that you can bring this task to completion. Jia, > would > >>> you please let us know how we can incorporate the changes into the JTS > >>> master branch? > >>> > > >>> > Thanks, > >>> > > >>> >> On Nov 12, 2020, at 10:10 AM, Jim Hughes <jhug...@ccri.com> wrote: > >>> >> > >>> >> Hi all, > >>> >> > >>> >> As a JTS committer, I have tried to request that the Sedona project > >>> discuss the desired changes to JTS previously. I'd still encourage > that. > >>> >> > >>> >> JTS is an active project and I feel that maintaining a fork of JTS > is > >>> unnecessary and inappropriate. > >>> >> > >>> >> Cheers, > >>> >> > >>> >> Jim > >>> >> > >>> >>> On 11/11/20 9:04 PM, Felix Cheung wrote: > >>> >>> Ah. You will need to publish it in order for the dependency chain > to > >>> work > >>> >>> on Maven Central > >>> >>> > >>> >>> However, since you are not the project owner there you might need > to > >>> >>> publish that under a different artifact id. > >>> >>> > >>> >>> In general, it would be best to avoid hard forking another project > >>> like > >>> >>> this. > >>> >>> > >>> >>> > >>> >>>> On Wed, Nov 11, 2020 at 1:05 PM Jia Yu <jiayu198...@gmail.com> > >>> wrote: > >>> >>>> > >>> >>>> Hi Netanel, > >>> >>>> > >>> >>>> That links to this git submodule: > >>> >>>> > https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 > >>> >>>> > >>> >>>> I can easily fix this by changing the version number here to > 1.16.2 > >>> >>>> excluding "SNAPSHOT": > >>> >>>> > https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 > >>> >>>> > >>> >>>> Will this solve the problem? > >>> >>>> > >>> >>>> On Wed, Nov 11, 2020 at 7:40 AM Netanel Malka < > netanel...@gmail.com > >>> > > >>> >>>> wrote: > >>> >>>> > >>> >>>>> Hi Folks, > >>> >>>>> > >>> >>>>> I tried to make a release (dry-run) following by > >>> >>>>> publishing-maven-artifacts > >>> >>>>> <https://infra.apache.org/publishing-maven-artifacts.html>, and > I > >>> >>>>> encountered an issue. > >>> >>>>> > >>> >>>>> On sedona-core, we have jts-core as a dependency with the > SNAPSHOT > >>> >>>>> version. > >>> >>>>> (link > >>> >>>>> < > >>> >>>>> > >>> > https://github.com/apache/incubator-sedona/blob/2e60fc07b0eae78ccae3876d970e677fc9319c40/core/pom.xml#L37 > >>> >>>>> ) > >>> >>>>> > >>> >>>>> As a prerequisite to the release process, we cannot have > >>> dependencies in a > >>> >>>>> SNAPSHOT version. > >>> >>>>> > >>> >>>>> > >>> >>>>> Do you have any clue about how to solve this? > >>> >>>>> > >>> >>>>> > >>> >>>>> On Mon, 9 Nov 2020 at 21:22, Netanel Malka <netan...@sela.co.il> > >>> wrote: > >>> >>>>> > >>> >>>>>> OK. Thanks Felix. > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> Updates: > >>> >>>>>> > >>> >>>>>> * > >>> >>>>>> * Opened a ticket for INFRA to Enable Nexus Access For > >>> Sedona< > >>> >>>>>> https://issues.apache.org/jira/browse/INFRA-21085> > >>> >>>>>> * Followed this< > >>> >>>>>> https://infra.apache.org/publishing-maven-artifacts.html> guide > >>> to test > >>> >>>>>> the maven release process > >>> >>>>>> * I hope to create a PR soon for adjusting the build to > >>> deploy to > >>> >>>>> the > >>> >>>>>> ASF Nexus repository > >>> >>>>>> * The key that signs the artifacts were created and tested. > >>> >>>>>> > >>> >>>>>> Do we want to create a candidate release for the current master > >>> branch? > >>> >>>>>> > >>> >>>>>> Netanel Malka, > >>> >>>>>> Big Data Consultant > >>> >>>>>> [Description: Description: Description: Description: > >>> >>>>>> cid:image001.jpg@01C85203.36A2AF30] > >>> >>>>>> ________________________________ > >>> >>>>>> From: Felix Cheung <felixche...@apache.org> > >>> >>>>>> Sent: Wednesday, November 4, 2020 19:57 > >>> >>>>>> To: dev@sedona.apache.org > >>> >>>>>> Cc: Jinxuan Wu; Mohamed Sarwat; Netanel Malka; Paweł Kociński; > >>> Zongsi > >>> >>>>> Zhang > >>> >>>>>> Subject: Re: First Sedona release > >>> >>>>>> > >>> >>>>>> 1) No you don’t need KEYS file in github only on the release > share > >>> >>>>>> https://dist.apache.org/repos/dist/dev/incubator/ > >>> >>>>>> > >>> >>>>>> 2) as podling you add to > >>> >>>>>> https://dist.apache.org/repos/dist/dev/incubator/ > >>> >>>>>> When you commit via svn you will be able to add a “directory” > for > >>> Sedona > >>> >>>>>> > >>> >>>>>> 2a) for release, you basically do a svn rename to move from dev > to > >>> >>>>> release > >>> >>>>>> “path” > >>> >>>>>> > >>> >>>>>> 3) if you have java based artifacts, yes. You will publish to > >>> Nexus, > >>> >>>>>> staging first and when release is signed off, you can click on > the > >>> >>>>>> interface to make it official, which then automatically sync to > >>> Maven > >>> >>>>>> central. > >>> >>>>>> > >>> >>>>>> Here is a script for example that does release signing and > >>> publication > >>> >>>>> to > >>> >>>>>> Nexus (and staging before release) > >>> >>>>>> > >>> >>>>>> > >>> >>>>> > >>> > https://github.com/apache/spark/blob/master/dev/create-release/release-build.sh > >>> >>>>>> On Wed, Nov 4, 2020 at 2:50 AM Netanel Malka < > >>> netanel...@gmail.com > >>> >>>>> <mailto: > >>> >>>>>> netanel...@gmail.com>> wrote: > >>> >>>>>> Hi, > >>> >>>>>> > >>> >>>>>> I followed the release-signing > >>> >>>>>> <https://infra.apache.org/release-signing.html> doc and created > >>> a key > >>> >>>>> for > >>> >>>>>> signing and hashing. > >>> >>>>>> > >>> >>>>>> I have a few questions: > >>> >>>>>> > >>> >>>>>> 1. Should the KEYS file also be added to the project root > >>> directory > >>> >>>>> on > >>> >>>>>> Github? ( I saw it in Apache Ant) > >>> >>>>>> 2. I saw in release-policy_upload-ci > >>> >>>>>> <http://www.apache.org/legal/release-policy.html#upload-ci> > >>> that we > >>> >>>>>> need > >>> >>>>>> to add a release candidate to > >>> >>>>> https://dist.apache.org/repos/dist/*dev*/ > >>> >>>>>> <TLP > >>> >>>>>> name>/. However, there does not seem to be a directory with > >>> Sedona as > >>> >>>>>> the > >>> >>>>>> TLP name. How may we be able to get a directory with that > >>> name? (Also > >>> >>>>>> for > >>> >>>>>> the *release*) > >>> >>>>>> 3. Do we need to push the artifacts also to ASF Nexus > >>> Repository > >>> >>>>> (beside > >>> >>>>>> Maven Central)? > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> Thanks. > >>> >>>>>> > >>> >>>>>> On Mon, 2 Nov 2020 at 19:21, Netanel Malka < > netanel...@gmail.com > >>> >>>>> <mailto: > >>> >>>>>> netanel...@gmail.com>> wrote: > >>> >>>>>> > >>> >>>>>>> Thanks Felix. > >>> >>>>>>> > >>> >>>>>>> I would be delighted to help. > >>> >>>>>>> I can start with the GPG. > >>> >>>>>>> Can I test it on a some artifact, or I need to wait for the > >>> first > >>> >>>>>> release? > >>> >>>>>>> On Mon, 2 Nov 2020 at 03:17, Felix Cheung < > >>> felixche...@apache.org > >>> >>>>>> <mailto:felixche...@apache.org>> wrote: > >>> >>>>>>>> Great progress! > >>> >>>>>>>> > >>> >>>>>>>> To add, > >>> >>>>>>>> A) I’d strongly recommend the WIP disclaimer - it would be > much > >>> >>>>> easier > >>> >>>>>> to > >>> >>>>>>>> pass with in the first release > >>> >>>>>>>> > https://incubator.apache.org/policy/incubation.html#disclaimers > >>> >>>>>>>> > >>> >>>>>>>> B) more info in signing, checksum > >>> >>>>>>>> https://infra.apache.org/release-signing.html > >>> >>>>>>>> > >>> >>>>>>>> C) signing key should be individual’s and (public key ) > >>> published and > >>> >>>>>> also > >>> >>>>>>>> listed in KEYS file - KEYS file should be located next to the > >>> >>>>> staging > >>> >>>>>>>> (and > >>> >>>>>>>> later release) location, see above > >>> >>>>>>>> > >>> >>>>>>>> D) “correct place” - this is in reference to ASF officIal > >>> staging > >>> >>>>> server > >>> >>>>>>>> http://www.apache.org/legal/release-policy.html#stage > >>> >>>>>>>> And can be “uploaded” by committing to svn > >>> >>>>>>>> http://www.apache.org/legal/release-policy.html#upload-ci > >>> >>>>>>>> > >>> >>>>>>>> E) python / PyPI - > >>> >>>>>>>> https://incubator.apache.org/guides/distribution.html#pypi > >>> >>>>>>>> > >>> >>>>>>>> > >>> >>>>>>>> > >>> >>>>>>>> On Sun, Nov 1, 2020 at 2:17 PM Jia Yu <ji...@apache.org > <mailto: > >>> >>>>>> ji...@apache.org>> wrote: > >>> >>>>>>>>> Hi Netanel, Pawel and other committers, > >>> >>>>>>>>> > >>> >>>>>>>>> While Pawel is working on Python code of Sedona 1.0, let's > >>> focus on > >>> >>>>>>>> other > >>> >>>>>>>>> parts required by the release. Netanel, can you help me with > >>> all > >>> >>>>> the > >>> >>>>>> ASF > >>> >>>>>>>>> incubator requirement items that are not DONE? > >>> >>>>>>>>> > >>> >>>>>>>>> *Here is a checklist for our first Sedona release* > >>> >>>>>>>>> > >>> >>>>>>>>> *ASF incubator requirement > >>> >>>>>>>>> (https://incubator.apache.org/guides/releasemanagement.html > >>> >>>>>>>>> <https://incubator.apache.org/guides/releasemanagement.html > >, > >>> we > >>> >>>>>>>> probably > >>> >>>>>>>>> should read ASF release requirement as well):* > >>> >>>>>>>>> > >>> >>>>>>>>> 1 .Include the word incubating in the release file name: > DONE. > >>> >>>>> Please > >>> >>>>>>>> see > >>> >>>>>>>>> the POM.xml in all directories. > >>> >>>>>>>>> > >>> >>>>>>>>> 2. Include an ASF LICENSE and NOTICE file: DONE. Please see > the > >>> >>>>> GitHub > >>> >>>>>>>>> repo. > >>> >>>>>>>>> > >>> >>>>>>>>> 3. Have valid checksums or signatures: I believe signature > >>> should > >>> >>>>> be > >>> >>>>>>>> done > >>> >>>>>>>>> by the GPG key. Not sure about the checksum. I am also not > sure > >>> >>>>> about > >>> >>>>>>>> the > >>> >>>>>>>>> GPG key requirement of ASF. I use GPG key to sign releases of > >>> >>>>> GeoSpark > >>> >>>>>>>> in > >>> >>>>>>>>> the past. > >>> >>>>>>>>> > >>> >>>>>>>>> 4. Be placed in the correct place on the ASF’s > infrastructure: > >>> we > >>> >>>>>> should > >>> >>>>>>>>> place our releases in two places: Maven, and PyPi. Not sure > >>> how to > >>> >>>>>>>> relate > >>> >>>>>>>>> them to ASF. > >>> >>>>>>>>> > >>> >>>>>>>>> 5. Have a KEYS file to validate the release: this should be > the > >>> >>>>> public > >>> >>>>>>>> key > >>> >>>>>>>>> of our GPG key? > >>> >>>>>>>>> > >>> >>>>>>>>> *Sedona requirement* > >>> >>>>>>>>> > >>> >>>>>>>>> 1. Python path name, file headers, and jars > >>> >>>>>>>>> 2. Project website docs: documentation should use the name, > >>> >>>>> Sedona, in > >>> >>>>>>>> all > >>> >>>>>>>>> tutorials. We should also include the situation of GeoTools > >>> >>>>>>>> dependencies. > >>> >>>>>>>>> Thanks, > >>> >>>>>>>>> Jia > >>> >>>>>>>>> > >>> >>>>>>>>> > >>> >>>>>>>>> On Wed, Oct 14, 2020 at 10:08 PM Jia Yu <ji...@apache.org > >>> <mailto: > >>> >>>>>> ji...@apache.org>> wrote: > >>> >>>>>>>>>> Hi folks, > >>> >>>>>>>>>> > >>> >>>>>>>>>> We will be working on the first Sedona. Please see the JIRA > >>> >>>>> ticket > >>> >>>>>>>> here: > >>> >>>>> > >>> > https://issues.apache.org/jira/projects/SEDONA/issues/SEDONA-3?filter=allopenissues > >>> >>>>>>>>>> Do you think there are any outstanding issues to be fixed as > >>> >>>>> well? > >>> >>>>>>>>>> Thanks, > >>> >>>>>>>>>> Jia > >>> >>>>>>>>>> > >>> >>>>>>> -- > >>> >>>>>>> Best regards, > >>> >>>>>>> Netanel Malka. > >>> >>>>>>> > >>> >>>>>> -- > >>> >>>>>> Best regards, > >>> >>>>>> Netanel Malka. > >>> >>>>>> > >>> >>>>> -- > >>> >>>>> Best regards, > >>> >>>>> Netanel Malka. > >>> >>>>> > >>> > >>> >
