unsubscribe
unsubscribe
unsubscribe
unsubscribe
unsubscribe
unsubscribe
Re: Requesting a Plan for Avro-typed Datasets
Hi Please also consider the other 2 alternatives for statically typed Datasets of Avro objects https://github.com/apache/spark/pull/24299 and https://github.com/apache/spark/pull/24367 kind regards -Taoufik On Thu, May 16, 2019 at 9:59 PM Aleksander Eskilson wrote: > Hi all, > > There's been longstanding demand for statically typed Datasets of Avro. > Functionality from the now-deprecated Databricks Spark-Avro project was > folded into Spark, but can still only provide DataFrames over Avro data. As > is discussed in the PR below, there are still drawbacks from not having > fully, statically typed Datasets of Avro. > > There's an open PR adding a first-class Encoder for statically typed > Datasets of Avro: > > https://github.com/apache/spark/pull/22878 : > https://issues.apache.org/jira/browse/SPARK-25789 (originally in > Databricks/spark-avro, https://github.com/databricks/spark-avro/pull/217 > : https://github.com/databricks/spark-avro/issues/169) > > We've tested the content of this PR widely over complex, deeply nested, > Avro structures. It seems ready for a last review and nearly ready for > merger. > > Alek Eskilson > github : bdrillard > -- Taoufik Dachraoui
[SPARK-27457][SQL] modify bean encoder to support avro objects
Hi Is someone looking at the PR https://github.com/apache/spark/pull/24367 ? how much does it take to be fixed with this PR ? PS: note that there is 2 other PRs related to avro encoding support ( https://github.com/apache/spark/pull/24299, and https://github.com/apache/spark/pull/21348) Kind regards -- Taoufik Dachraoui
[SPARK-27457][QL]modify bean encoder to support avro objects
Hi, please have a look to the PR https://github.com/apache/spark/pull/24367 The PR adds support to avro objects by modifying the Encoder javaBean (JavaTypeInference) There is 2 other PRs to add encoder for avro objects: https://github.com/apache/spark/pull/24299, and https://github.com/apache/spark/pull/22878 regards -- Taoufik Dachraoui
Support for Avro datasets
Hi The PR #24299 <https://github.com/apache/spark/pull/24299> proposes a simple solution to add support for Avro objects; your opinions matter, we are eager to have a working solution to create Avro datasets regards -- Taoufik Dachraoui
[SPARK-27388][SQL] expression encoder for objects defined by properties #24299
Hi, I created the PR https://github.com/apache/spark/pull/24299 few days ago and the jira is still unassigned, how can I be sure that the jira/PR are taken into account? Kind regards -- Taoufik Dachraoui
