I opened https://github.com/apache/spark/pull/31461 to track the discussion
further. It narrowly proposes making a few types public.
On Mon, Feb 1, 2021 at 8:52 AM Fitch, Simeon wrote:
> 🙇
>
> On Mon, Feb 1, 2021 at 9:38 AM Sean Owen wrote:
>
>> I'm not hearing any objection to making it public
🙇
On Mon, Feb 1, 2021 at 9:38 AM Sean Owen wrote:
> I'm not hearing any objection to making it public as a @DeveloperApi ?
> anyone object to a PR on that?
>
> On Fri, Jan 29, 2021 at 8:46 AM Sean Owen wrote:
>
>> I'm also interested: are there problems with opening up this API beyond
>> needin
I'm not hearing any objection to making it public as a @DeveloperApi ?
anyone object to a PR on that?
On Fri, Jan 29, 2021 at 8:46 AM Sean Owen wrote:
> I'm also interested: are there problems with opening up this API beyond
> needing to freeze it and keep it stable? it's pretty stable.
> As @De
On Fri, Jan 29, 2021 at 9:46 AM Sean Owen wrote:
> Are there implications for storing UDTs in particular engines or formats?
>
I've found UDTs I/O to Parquet without problem.
They work fine with PySpark with implementation of mirror classes. Without
properly constructed mirror classe they show
I'm also interested: are there problems with opening up this API beyond
needing to freeze it and keep it stable? it's pretty stable.
As @DeveloperApi at least?
Are there implications for storing UDTs in particular engines or formats?
Just making it public for developers, even with a 'use at your ow
Hi,
First time posting here, so apologies if I need to be directing this topic
elsewhere.
I'm the author of RasterFrames, and a contributor to GeoMesa's Spark SQL
module. Both make use of decently low level Catalyst constructs, include
custom UDTs; RasterFrames introduces a geospatial raster type
