[
https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Taoufik DACHRAOUI updated SPARK-27388:
--------------------------------------
Description:
Add an expression encoder for objects defined by properties (ie. I call them
Beans). A property of an object is defined by a _setter_ and a _getter_
functions where the _getter_ return type is equal to the _setter_ unique
parameter type and the _getter_ and _setter_ functions have the same name or
the _getter_ name is prefixed by "get" and the _setter_ name is prefixed by
"set"
An example of a Bean:
{code:java}
class Bar {
...
def age():Int = {...}
def age(v:Int):Unit {...}
...
}
class Foo extends Bar {
...
def setName(n:String) {...}
def getName():String = {...}
...
}
{code}
```
The class _Foo_ has 2 properties _age_ and _Name_
also added support for _java.util.List_, _java.util.Map_ and java _Enums_
Avro objects are beans and thus we can create an expression encoder with the
current addition.
Currently complex avro unions are not supported; indeed a complex union is
declared as Object and there cannot be an expression encoder for Object type
(need to use a custom serializer like kryo for example)
was:
Add an expression encoder for objects defined by properties (ie. I call them
Beans). A property of an object is defined by a _setter_ and a _getter_
functions where the _getter_ return type is equal to the _setter_ unique
parameter type and the _getter_ and _setter_ functions have the same name or
the _getter_ name is prefixed by "get" and the _setter_ name is prefixed by
"set"
An example of a Bean:
{code:java}
class Bar {
...
def age():Int = {...}
def age(v:Int):Unit {...}
...
}
class Foo extends Bar {
...
def setName(n:String) {...}
def getName():String = {...}
...
}
{code}
```
The class _Foo_ has 2 properties _age_ and _Name_
also added support for _java.util.List_, _java.util.Map_ and java _Enums_
Currently complex avro unions are not supported; indeed a complex union is
declared as Object and there cannot be an expression encoder for Object type
(need to use a custom serializer like kryo for example)
> expression encoder for avro like objects
> ----------------------------------------
>
> Key: SPARK-27388
> URL: https://issues.apache.org/jira/browse/SPARK-27388
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Affects Versions: 2.4.1
> Reporter: Taoufik DACHRAOUI
> Priority: Major
>
> Add an expression encoder for objects defined by properties (ie. I call them
> Beans). A property of an object is defined by a _setter_ and a _getter_
> functions where the _getter_ return type is equal to the _setter_ unique
> parameter type and the _getter_ and _setter_ functions have the same name or
> the _getter_ name is prefixed by "get" and the _setter_ name is prefixed by
> "set"
>
> An example of a Bean:
> {code:java}
> class Bar {
> ...
> def age():Int = {...}
> def age(v:Int):Unit {...}
> ...
> }
> class Foo extends Bar {
> ...
> def setName(n:String) {...}
> def getName():String = {...}
> ...
> }
> {code}
> ```
> The class _Foo_ has 2 properties _age_ and _Name_
> also added support for _java.util.List_, _java.util.Map_ and java _Enums_
>
> Avro objects are beans and thus we can create an expression encoder with the
> current addition.
> Currently complex avro unions are not supported; indeed a complex union is
> declared as Object and there cannot be an expression encoder for Object type
> (need to use a custom serializer like kryo for example)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
