[jira] [Commented] (SPARK-38056) Structured streaming not working in history server when using LevelDB
[ https://issues.apache.org/jira/browse/SPARK-38056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17559223#comment-17559223 ] Apache Spark commented on SPARK-38056: -- User 'kuwii' has created a pull request for this issue: https://github.com/apache/spark/pull/37007 > Structured streaming not working in history server when using LevelDB > - > > Key: SPARK-38056 > URL: https://issues.apache.org/jira/browse/SPARK-38056 > Project: Spark > Issue Type: Bug > Components: Structured Streaming, Web UI >Affects Versions: 3.1.2, 3.2.0 >Reporter: wy >Assignee: wy >Priority: Major > Fix For: 3.3.0, 3.1.4, 3.2.2 > > Attachments: local-1643373518829 > > > In > [SPARK-31953|https://github.com/apache/spark/commit/4f9667035886a67e6c9a4e8fad2efa390e87ca68], > structured streaming support is added to history server. However this does > not work when spark.history.store.path is set to save app info using LevelDB. > This is because one of the keys of StreamingQueryData, runId, is UUID type, > which is not supported by LevelDB. When replaying event log file in history > server, StreamingQueryStatusListener will throw an exception when writing > info to the store, saying "java.lang.IllegalArgumentException: Type > java.util.UUID not allowed as key.". > Example event log is provided in attachments. When opening it in history > server with spark.history.store.path set to somewhere, no structured > streaming info is available. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38056) Structured streaming not working in history server when using LevelDB
[ https://issues.apache.org/jira/browse/SPARK-38056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489591#comment-17489591 ] Apache Spark commented on SPARK-38056: -- User 'kuwii' has created a pull request for this issue: https://github.com/apache/spark/pull/35464 > Structured streaming not working in history server when using LevelDB > - > > Key: SPARK-38056 > URL: https://issues.apache.org/jira/browse/SPARK-38056 > Project: Spark > Issue Type: Bug > Components: Structured Streaming, Web UI >Affects Versions: 3.1.2, 3.2.0 >Reporter: wy >Assignee: wy >Priority: Major > Fix For: 3.3.0 > > Attachments: local-1643373518829 > > > In > [SPARK-31953|https://github.com/apache/spark/commit/4f9667035886a67e6c9a4e8fad2efa390e87ca68], > structured streaming support is added to history server. However this does > not work when spark.history.store.path is set to save app info using LevelDB. > This is because one of the keys of StreamingQueryData, runId, is UUID type, > which is not supported by LevelDB. When replaying event log file in history > server, StreamingQueryStatusListener will throw an exception when writing > info to the store, saying "java.lang.IllegalArgumentException: Type > java.util.UUID not allowed as key.". > Example event log is provided in attachments. When opening it in history > server with spark.history.store.path set to somewhere, no structured > streaming info is available. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38056) Structured streaming not working in history server when using LevelDB
[ https://issues.apache.org/jira/browse/SPARK-38056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489588#comment-17489588 ] Apache Spark commented on SPARK-38056: -- User 'kuwii' has created a pull request for this issue: https://github.com/apache/spark/pull/35463 > Structured streaming not working in history server when using LevelDB > - > > Key: SPARK-38056 > URL: https://issues.apache.org/jira/browse/SPARK-38056 > Project: Spark > Issue Type: Bug > Components: Structured Streaming, Web UI >Affects Versions: 3.1.2, 3.2.0 >Reporter: wy >Assignee: wy >Priority: Major > Fix For: 3.3.0 > > Attachments: local-1643373518829 > > > In > [SPARK-31953|https://github.com/apache/spark/commit/4f9667035886a67e6c9a4e8fad2efa390e87ca68], > structured streaming support is added to history server. However this does > not work when spark.history.store.path is set to save app info using LevelDB. > This is because one of the keys of StreamingQueryData, runId, is UUID type, > which is not supported by LevelDB. When replaying event log file in history > server, StreamingQueryStatusListener will throw an exception when writing > info to the store, saying "java.lang.IllegalArgumentException: Type > java.util.UUID not allowed as key.". > Example event log is provided in attachments. When opening it in history > server with spark.history.store.path set to somewhere, no structured > streaming info is available. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38056) Structured streaming not working in history server when using LevelDB
[ https://issues.apache.org/jira/browse/SPARK-38056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489589#comment-17489589 ] Apache Spark commented on SPARK-38056: -- User 'kuwii' has created a pull request for this issue: https://github.com/apache/spark/pull/35463 > Structured streaming not working in history server when using LevelDB > - > > Key: SPARK-38056 > URL: https://issues.apache.org/jira/browse/SPARK-38056 > Project: Spark > Issue Type: Bug > Components: Structured Streaming, Web UI >Affects Versions: 3.1.2, 3.2.0 >Reporter: wy >Assignee: wy >Priority: Major > Fix For: 3.3.0 > > Attachments: local-1643373518829 > > > In > [SPARK-31953|https://github.com/apache/spark/commit/4f9667035886a67e6c9a4e8fad2efa390e87ca68], > structured streaming support is added to history server. However this does > not work when spark.history.store.path is set to save app info using LevelDB. > This is because one of the keys of StreamingQueryData, runId, is UUID type, > which is not supported by LevelDB. When replaying event log file in history > server, StreamingQueryStatusListener will throw an exception when writing > info to the store, saying "java.lang.IllegalArgumentException: Type > java.util.UUID not allowed as key.". > Example event log is provided in attachments. When opening it in history > server with spark.history.store.path set to somewhere, no structured > streaming info is available. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38056) Structured streaming not working in history server when using LevelDB
[ https://issues.apache.org/jira/browse/SPARK-38056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17483781#comment-17483781 ] Apache Spark commented on SPARK-38056: -- User 'kuwii' has created a pull request for this issue: https://github.com/apache/spark/pull/35356 > Structured streaming not working in history server when using LevelDB > - > > Key: SPARK-38056 > URL: https://issues.apache.org/jira/browse/SPARK-38056 > Project: Spark > Issue Type: Bug > Components: Structured Streaming, Web UI >Affects Versions: 3.1.2, 3.2.0 >Reporter: Weiyi Kong >Priority: Major > Attachments: local-1643373518829 > > > In > [SPARK-31953|https://github.com/apache/spark/commit/4f9667035886a67e6c9a4e8fad2efa390e87ca68], > structured streaming support is added to history server. However this does > not work when spark.history.store.path is set to save app info using LevelDB. > This is because one of the keys of StreamingQueryData, runId, is UUID type, > which is not supported by LevelDB. When replaying event log file in history > server, StreamingQueryStatusListener will throw an exception when writing > info to the store, saying "java.lang.IllegalArgumentException: Type > java.util.UUID not allowed as key.". > Example event log is provided in attachments. When opening it in history > server with spark.history.store.path set to somewhere, no structured > streaming info is available. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38056) Structured streaming not working in history server when using LevelDB
[ https://issues.apache.org/jira/browse/SPARK-38056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17483779#comment-17483779 ] Apache Spark commented on SPARK-38056: -- User 'kuwii' has created a pull request for this issue: https://github.com/apache/spark/pull/35356 > Structured streaming not working in history server when using LevelDB > - > > Key: SPARK-38056 > URL: https://issues.apache.org/jira/browse/SPARK-38056 > Project: Spark > Issue Type: Bug > Components: Structured Streaming, Web UI >Affects Versions: 3.1.2, 3.2.0 >Reporter: Weiyi Kong >Priority: Major > Attachments: local-1643373518829 > > > In > [SPARK-31953|https://github.com/apache/spark/commit/4f9667035886a67e6c9a4e8fad2efa390e87ca68], > structured streaming support is added to history server. However this does > not work when spark.history.store.path is set to save app info using LevelDB. > This is because one of the keys of StreamingQueryData, runId, is UUID type, > which is not supported by LevelDB. When replaying event log file in history > server, StreamingQueryStatusListener will throw an exception when writing > info to the store, saying "java.lang.IllegalArgumentException: Type > java.util.UUID not allowed as key.". > Example event log is provided in attachments. When opening it in history > server with spark.history.store.path set to somewhere, no structured > streaming info is available. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
