Andy, Happy to help, but unfortunately, we don’t have a solution in our hip pocket. We have only encountered this kind of issue when transferring larger numbers of records from legacy systems into Arches. Therefore, our solution is a bit one-off. And, I am a bit surprised that geometry (that I assume was) collected with GPS units would be voluminous enough to surpass that threshold.
That said, what I have done in the past is run the data through an appropriate FME transformer or use Postgis (st_snaptogrid function) to process the data in a manner that resolves the issue. Starting to move beyond my domain of expertise, but it seems likely that there is a gdal function that you could bind to your python ETL/integration code that may do the trick. Wish I could point you to something more specific. Maybe someone else can? One other thing - I highly recommend you start with reducing precision rather than generalizing. You preserve much more integrity of the data and still likely achieve your goal. Adam -- Adam Lodge Geospatial Systems Consultant Farallon Geographics On Sunday, May 1, 2016 at 3:16 PM, Andy Graham wrote: > Thanks for the info Adam. I figured it was something like that. In this > case generalizing the gpx probably is acceptable. Do you guys know of or > have a quick and easy program/code to do this? I can manage this with a few > steps but I visualize this being part of the daily work flow so would prefer > more of a single step, "push a button and it's generalized" type setup. Let > me know. Thanks. > Andy > On May 1, 2016 2:17 PM, "Adam Lodge" <[email protected] > (mailto:[email protected])> wrote: > > Andy, > > > > There is a known limitation - Im guessing from your error message that > > 32766 - to the number of characters that Elastic Search can process in a > > single value. This translates into a geometry limitation because (I think) > > ES indexes geometries as geojson. In short, I think it’s less about the > > size of the file and more about the number of characters necessary to > > describe the geometry. > > > > We have done some tricks to push geometries like these through. They > > include: > > - Reduce the precision of the captured geometries (thereby reducing the > > number of characters after the decimal that gets stored for each vertex) > > - Generalize the geometry (if that is acceptable.) That reduces the number > > of vertices in the geometry. > > > > If you are able to send the file, I would be happy to take a look at it. > > > > Adam > > > > -- > > Adam Lodge > > Geospatial Systems Consultant > > Farallon Geographics > > 415.317.6625 (tel:415.317.6625) > > > > > > On Sunday, May 1, 2016 at 2:08 PM, Andy Graham wrote: > > > > > So I have been trying to add geometry to one of our resources by dragging > > > and dropping a .gpx file onto the map in the edit form for the resource. > > > > > > > > > I can add it to the map just fine but when I go to save I eventually get > > > an error on the webpage that states: > > > > > > TransportError at > > > /resources/SURVEY.E1/default/1510042c-6888-4fc9-8658-cd5cee6a8bdb > > > > > > TransportError(500, u'IllegalArgumentException[Document contains at least > > > one immense term in field="geometries.label" (whose UTF8 encoding is > > > longer than the max length 32766), all of which were skipped. Please > > > correct the analyzer to not produce such terms. The prefix of the first > > > immense term is: \'[77, 85, 76, 84, 73, 76, 73, 78, 69, 83, 84, 82, 73, > > > 78, 71, 40, 40, 45, 49, 50, 48, 46, 53, 54, 57, 57, 53, 49, 48, 51]...\', > > > original message: bytes can be at most 32766 in length; got 100421]; > > > nested: MaxBytesLengthExceededException[bytes can be at most 32766 in > > > length; got 100421]; ') > > > > > > From what I can tell it appears as though there is a size limit on the > > > files used to create the geometry? I have done this successfully with > > > smaller gpx files, this current one is about 300KB. I have converted to > > > a kml but get the same message (albeit with a smaller byte size), and the > > > size of that file is 165KB. > > > > > > Does anyone know for sure if size is the issue here and if there is a way > > > (and how) to increase size limits for files to create geometry? Thanks > > > much. > > > > > > Andy > > > > > > -- > > > -- To post, send email to [email protected] > > > (mailto:[email protected]). To unsubscribe, send email to > > > [email protected] > > > (mailto:[email protected]). For more > > > information, visit https://groups.google.com/d/forum/archesproject?hl=en > > > --- > > > You received this message because you are subscribed to the Google Groups > > > "Arches Project" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [email protected] > > > (mailto:[email protected]). > > > For more options, visit https://groups.google.com/d/optout. > > -- -- To post, send email to [email protected]. To unsubscribe, send email to [email protected]. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
