[
https://issues.apache.org/jira/browse/SPARK-12897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15114052#comment-15114052
]
astralidea commented on SPARK-12897:
------------------------------------
It's a problem ! I mean if use StorageLevel such as MEMORY_AND_DISK.when comput
BLOCK is finished it will remove this BLOCK and never see again.but in Tachyon
it will always show in spark UI.therefore with passing of the time.more and
more BLOCK will show in spark UI.In fact some BLOCK does not need to show. may
be it is not spark UI problem it is Tachyon BlockManager logical problem.
> spark ui with tachyon show all Stream Blocks
> --------------------------------------------
>
> Key: SPARK-12897
> URL: https://issues.apache.org/jira/browse/SPARK-12897
> Project: Spark
> Issue Type: Bug
> Components: Block Manager, Web UI
> Affects Versions: 1.6.0
> Environment: "spark.externalBlockStore.url", "tachyon://l-xxx:19998"
> "spark.externalBlockStore.blockManager",
> "org.apache.spark.storage.TachyonBlockManager"
> StorageLevel.OFF_HEAP
> Reporter: astralidea
>
> when I use tachyon .I click spark application UI storage.
> the job already running for 24hours.
> but stream Blocks continued to growing and the page is loading and showing
> all blocks but some block is never use.and loading slowly and slowly.
> I think tachyon should use traditional way. if block is never used it means
> Size is 0.0B.it is not need to show in storage page.just as traditional way.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
