[jira] [Commented] (SPARK-19022) Fix tests dependent on OS due to different newline characters

2016-12-29 Thread Apache Spark (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-19022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784908#comment-15784908
 ] 

Apache Spark commented on SPARK-19022:
--

User 'HyukjinKwon' has created a pull request for this issue:
https://github.com/apache/spark/pull/16433

> Fix tests dependent on OS due to different newline characters
> -
>
> Key: SPARK-19022
> URL: https://issues.apache.org/jira/browse/SPARK-19022
> Project: Spark
>  Issue Type: Test
>  Components: Structured Streaming, Tests
>Reporter: Hyukjin Kwon
>Priority: Minor
>
> There are two tests failing on Windows due to the different newlines.
> {code}
>  - StreamingQueryProgress - prettyJson *** FAILED *** (0 milliseconds)
>"{
>  "id" : "39788670-6722-48b7-a248-df6ba08722ac",
>  "runId" : "422282f1-3b81-4b47-a15d-82dda7e69390",
>  "name" : "myName",
>  "timestamp" : "2016-12-05T20:54:20.827Z",
>  "numInputRows" : 678,
>  "inputRowsPerSecond" : 10.0,
>  "durationMs" : {
>"total" : 0
>  },
>  "eventTime" : {
>"avg" : "2016-12-05T20:54:20.827Z",
>"max" : "2016-12-05T20:54:20.827Z",
>"min" : "2016-12-05T20:54:20.827Z",
>"watermark" : "2016-12-05T20:54:20.827Z"
>  },
>  "stateOperators" : [ {
>"numRowsTotal" : 0,
>"numRowsUpdated" : 1
>  } ],
>  "sources" : [ {
>"description" : "source",
>"startOffset" : 123,
>"endOffset" : 456,
>"numInputRows" : 678,
>"inputRowsPerSecond" : 10.0
>  } ],
>  "sink" : {
>"description" : "sink"
>  }
>}" did not equal "{
>  "id" : "39788670-6722-48b7-a248-df6ba08722ac",
>  "runId" : "422282f1-3b81-4b47-a15d-82dda7e69390",
>  "name" : "myName",
>  "timestamp" : "2016-12-05T20:54:20.827Z",
>  "numInputRows" : 678,
>  "inputRowsPerSecond" : 10.0,
>  "durationMs" : {
>"total" : 0
>  },
>  "eventTime" : {
>"avg" : "2016-12-05T20:54:20.827Z",
>"max" : "2016-12-05T20:54:20.827Z",
>"min" : "2016-12-05T20:54:20.827Z",
>"watermark" : "2016-12-05T20:54:20.827Z"
>  },
>  "stateOperators" : [ {
>"numRowsTotal" : 0,
>"numRowsUpdated" : 1
>  } ],
>  "sources" : [ {
>"description" : "source",
>"startOffset" : 123,
>"endOffset" : 456,
>"numInputRows" : 678,
>"inputRowsPerSecond" : 10.0
>  } ],
>  "sink" : {
>"description" : "sink"
>  }
>}" (StreamingQueryStatusAndProgressSuite.scala:36)
> {code}
> {code}
>  - StreamingQueryStatus - prettyJson *** FAILED *** (0 milliseconds)
>"{
>  "message" : "active",
>  "isDataAvailable" : true,
>  "isTriggerActive" : false
>}" did not equal "{
>  "message" : "active",
>  "isDataAvailable" : true,
>  "isTriggerActive" : false
>}" (StreamingQueryStatusAndProgressSuite.scala:115)
>org.scalatest.exceptions.TestFailedException:
> {code}
> The reason is, {{pretty}} in {{org.json4s.pretty}} writes OS-dependent 
> newlines but the string defined in the tests are {{\n}}. This ends up with 
> test failures.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-19022) Fix tests dependent on OS due to different newline characters

2016-12-29 Thread Hyukjin Kwon (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-19022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784894#comment-15784894
 ] 

Hyukjin Kwon commented on SPARK-19022:
--

It seems these are (almost) all instances across the tests on Windows. I will 
double check in the PR again.

> Fix tests dependent on OS due to different newline characters
> -
>
> Key: SPARK-19022
> URL: https://issues.apache.org/jira/browse/SPARK-19022
> Project: Spark
>  Issue Type: Test
>  Components: Structured Streaming, Tests
>Reporter: Hyukjin Kwon
>Priority: Minor
>
> There are two tests failing on Windows due to the different newlines.
> {code}
>  - StreamingQueryProgress - prettyJson *** FAILED *** (0 milliseconds)
>"{
>  "id" : "39788670-6722-48b7-a248-df6ba08722ac",
>  "runId" : "422282f1-3b81-4b47-a15d-82dda7e69390",
>  "name" : "myName",
>  "timestamp" : "2016-12-05T20:54:20.827Z",
>  "numInputRows" : 678,
>  "inputRowsPerSecond" : 10.0,
>  "durationMs" : {
>"total" : 0
>  },
>  "eventTime" : {
>"avg" : "2016-12-05T20:54:20.827Z",
>"max" : "2016-12-05T20:54:20.827Z",
>"min" : "2016-12-05T20:54:20.827Z",
>"watermark" : "2016-12-05T20:54:20.827Z"
>  },
>  "stateOperators" : [ {
>"numRowsTotal" : 0,
>"numRowsUpdated" : 1
>  } ],
>  "sources" : [ {
>"description" : "source",
>"startOffset" : 123,
>"endOffset" : 456,
>"numInputRows" : 678,
>"inputRowsPerSecond" : 10.0
>  } ],
>  "sink" : {
>"description" : "sink"
>  }
>}" did not equal "{
>  "id" : "39788670-6722-48b7-a248-df6ba08722ac",
>  "runId" : "422282f1-3b81-4b47-a15d-82dda7e69390",
>  "name" : "myName",
>  "timestamp" : "2016-12-05T20:54:20.827Z",
>  "numInputRows" : 678,
>  "inputRowsPerSecond" : 10.0,
>  "durationMs" : {
>"total" : 0
>  },
>  "eventTime" : {
>"avg" : "2016-12-05T20:54:20.827Z",
>"max" : "2016-12-05T20:54:20.827Z",
>"min" : "2016-12-05T20:54:20.827Z",
>"watermark" : "2016-12-05T20:54:20.827Z"
>  },
>  "stateOperators" : [ {
>"numRowsTotal" : 0,
>"numRowsUpdated" : 1
>  } ],
>  "sources" : [ {
>"description" : "source",
>"startOffset" : 123,
>"endOffset" : 456,
>"numInputRows" : 678,
>"inputRowsPerSecond" : 10.0
>  } ],
>  "sink" : {
>"description" : "sink"
>  }
>}" (StreamingQueryStatusAndProgressSuite.scala:36)
> {code}
> {code}
>  - StreamingQueryStatus - prettyJson *** FAILED *** (0 milliseconds)
>"{
>  "message" : "active",
>  "isDataAvailable" : true,
>  "isTriggerActive" : false
>}" did not equal "{
>  "message" : "active",
>  "isDataAvailable" : true,
>  "isTriggerActive" : false
>}" (StreamingQueryStatusAndProgressSuite.scala:115)
>org.scalatest.exceptions.TestFailedException:
> {code}
> The reason is, {{pretty}} in {{org.json4s.pretty}} writes OS-dependent 
> newlines but the string defined in the tests are {{\n}}. This ends up with 
> test failures.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org