Github user koertkuipers commented on the pull request:
https://github.com/apache/spark/pull/6361#issuecomment-118637079
thats fair enough. however keep in mind that kryo is a transitive
dependency of spark, and one that does not upgrade well and has not been
shaded, so you generally force people to use that version for other stuff
as well (within the same programs), including serializing data structures
in files.
On Mon, Jun 29, 2015 at 12:02 PM, Mark Hamstra <[email protected]>
wrote:
> @koertkuipers <https://github.com/koertkuipers> While it may be
> convenient for you that Scalding and Spark have a common serialization
> format when using chill/kryo, that can't be considered anything but a
lucky
> accident. We certainly haven't made guarantees that Spark's serialization
> will maintain compatibility with some external format. While that doesn't
> mean that we should gratuitously change Spark's serialization, it does
mean
> that we shouldn't bind ourselves to a particular format if there are
> compelling reasons to do something other.
>
> â
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/6361#issuecomment-116742527>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
