viirya commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-781801206
cc @cloud-fan
This is an automated message from the Apache Git Service.
To respond to the message, please log on
viirya commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-778581932
It sounds really making sense to me. So seems to have the status quo working
in Java 9+ sounds good idea and no (or just little) harm. Because
`UserDefinedType` is somehow well-k
viirya commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-775709570
> @jnh5y @viirya I got a good question from @marmbrus - can you support
user-defined types by just defining an Encoder for it? so that it can work in a
Dataset?
This sound
viirya commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-773084514
> So what about #16478 @viirya ? is that still a sound proposal for
refactoring the interface? That seems more ideal to add first.
Hm, yea, the refactoring is trying to hid
viirya commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-773084514
> So what about #16478 @viirya ? is that still a sound proposal for
refactoring the interface? That seems more ideal to add first.
Hm, yea, the refactoring is trying to hid
viirya commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-772849433
I just checked current UDT classes. Looks like it still uses internal Spark
classes. Actually as mentioned in previous comment, I don't see any obvious
improvement in UDT in rece
viirya commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-772847017
Hmm, I am not aware of some API changes was made. I closed it because seems
to me these users want is to open UDT API. I'm not sure if these API changes
are in need or not. And y
