[
https://issues.apache.org/jira/browse/LIVY-601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gyorgy Gal updated LIVY-601:
----------------------------
Fix Version/s: 0.10.0
(was: 0.9.0)
This issue has been moved to the 0.10.0 release as part of a bulk update. If
you feel this is moved out inappropriately, feel free to provide justification
and reset the Fix Version to 0.9.0.
> Add support for user defined date format and timezone when transform spark
> result rows to json
> ----------------------------------------------------------------------------------------------
>
> Key: LIVY-601
> URL: https://issues.apache.org/jira/browse/LIVY-601
> Project: Livy
> Issue Type: Improvement
> Components: REPL
> Affects Versions: 0.6.0
> Reporter: iteblog
> Priority: Minor
> Fix For: 0.10.0
>
>
> When Livy get the row data from Spark DataFrame and transform those rows to
> json, we use DefaultFormats to transform Spark timestamp to scala timestamp
> which use default timezone.
> {code:java}
> private implicit def formats = DefaultFormats
> val result = spark.sql(code)
> val schema = parse(result.schema.json)
> // Get the row data
> val rows = result.take(maxResult)
> .map {
> _.toSeq.map {
> // Convert java BigDecimal type to Scala BigDecimal, because current
> version of
> // Json4s doesn't support java BigDecimal as a primitive type
> (LIVY-455).
> case i: java.math.BigDecimal => BigDecimal(i)
> case e => e
> }
> }
> val jRows = Extraction.decompose(rows)
> // DefaultFormats defined in org.json4s.DefaultFormats
> /** Default date format is UTC time.
> */
> object DefaultFormats extends DefaultFormats {
> val UTC = TimeZone.getTimeZone("UTC")
> val losslessDate = {
> def createSdf = {
> val f = new java.text.SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'")
> f.setTimeZone(UTC)
> f
> }
> new ThreadLocal(createSdf)
> }
> }{code}
> It would be useful if we can customize the time zone and time format
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
