Copy paste from SO question: http://stackoverflow.com/q/36449368/750216
"Is it possible to use Option[_] member in a case class used with Dataset API? eg. Option[Int] I tried to find an example but could not find any yet. This can probably be done with with a custom encoder (mapping?) but I could not find an example for that yet. This might be achievable using Frameless library: https://github.com/adelbertc/frameless but there should be an easy way to get it done with base Spark libraries. *Update* I am using: "org.apache.spark" %% "spark-core" % "1.6.1" And getting the following error when trying to use an Option[Int]: Unable to find encoder for type stored in a Dataset. Primitive types (Int, String, etc) and Product types (case classes) are supported by importing sqlContext.implicits._ Support for serializing other types will be added in future releases " Thank you -- Razvan Panda .NET Developer | WhatClinic.com T: +353 1 6250520| rpa...@whatclinic.com Global Medical Treatment Ltd, Trading as WhatClinic.com, 12 Duke Lane Upper, Dublin 2, Ireland. Company Registration Number: 428122 ***Email Confidentiality & Disclaimer Notice - This email and any attachments is to be treated as confidential. If received in error, any use, dissemination, distribution, publication or copying of the information contained in this email or attachments is strictly prohibited. We cannot guarantee this email or any attachment is virus free and has not been intercepted and/or amended. The recipient should check this email and any attachments for the presence of viruses***
