Hi Brandon, This is likely because the two field names attributes are put in the wrong order. Can you try Adapter.toDf(geoRdd, tractRDD.fieldNames, tripRDD.fieldNames, spark) instead?
See our example here: https://sedona.apache.org/tutorial/sql/#spatialpairrdd-to-dataframe On Wed, Aug 25, 2021 at 12:24 PM Graybeal, Brandon (he, him) <[email protected]> wrote: > Hello! > > > > My name is Brandon Graybeal and I am a Data Engineer at Arity, which is a > Telematics company owned by Allstate. We have been using the Geospark > library in our production pipelines for a year now, and it has been great. > However, we are migrating to EMR and Spark 3, so I need to upgrade all our > pipielines to utilize Apache Sedona instead. In the process of following > some of the examples and programming guides on your site, I am running into > some unexpected issues. > > > > To give some context, I am trying to POC the new library from a spark > shell. We are just doing a spatial join on one spark dataframe with a point > geometry to another spark dataframe that has a polygon geometry. The join > appears to work, and whenever I call the function Adapter.toDf(geoRdd, > spark) it returns the left and right geometries properly. However I get an > exception I can’t quite decipher whenever I try to pull other columns > through using Adapter.toDf(geoRdd, tripRDD.fieldNames, tractRDD.fieldNames, > spark). As far as I can tell, I am calling the function the same way that > is shown in the examples folder on Github. > > > > I have attached the scala script that I have been running, as well as the > resulting exception messages. Any guidance or help you all could provide > would be greatly appreciated it! > > > > Thanks! > > *Brandon Graybeal* > > Data Analytics Engineer > > [image: signature_204968942] <https://www.arity.com/> > > 222 W Merchandise Mart Plaza | Suite 875 | Chicago, IL 60654 > > [email protected] | 312-999-7744 | arity.com > <http://www.arity.com/> > [image: signature_74792671] <https://www.linkedin.com/company/arity>[image: > signature_1180521648] <https://twitter.com/arity>[image: > signature_1051572743] <https://www.arity.com/move/>[image: > signature_673829010] <https://www.instagram.com/arityofficial/> > > > CONFIDENTIALITY NOTE: This email and attachments may contain material that > is confidential, proprietary, and/or privileged. It is distributed for the > sole use of the intended recipient, and if it does contain material that is > confidential, proprietary, and/or privileged, it should not be distributed > or forwarded without the sender’s permission. If you are not the intended > recipient, review, reliance, distribution, and forwarding without the > sender’s permission are strictly prohibited and you should immediately > contact the sender and delete all copies. > > >
