[
https://issues.apache.org/jira/browse/PHOENIX-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14202669#comment-14202669
]
Jeffrey Zhong commented on PHOENIX-1395:
----------------------------------------
OnDiskResultIterator will even delete its spill files when it reaches last row.
While there are couple people complaining about this leaked spill files, I
think the possible cases that we could leak files are as following:
1) Connection doesn't close
2) ResultSet isn't iterated to last row
3) ResulteSet isn't closed after it's used
4) Application multiple threads sharing same connection instance so a
connection may not keep all reference to ResultSets it returned.
5) bugs
For the above suggestions may not work(like finalize, connection close) for
cases that a long running connection is needed therefore using connection close
to clean up spill files isn't enough because spill files are pilling up. It
seems to me we need do two things:
1) For shot live connections, we need to make sure all spill files are deleted
when a connection is closed.
2) For long live connections, lease way is a good option. Each
OnDiskResultIterator.next call will extend the lease and the connection
instance can track its opening ResultSets(we are already doing that today) and
close expired spill files.
> ResultSpooler spill files are left behind in /tmp folder
> --------------------------------------------------------
>
> Key: PHOENIX-1395
> URL: https://issues.apache.org/jira/browse/PHOENIX-1395
> Project: Phoenix
> Issue Type: Bug
> Reporter: Jeffrey Zhong
> Assignee: Alicia Ying Shu
> Fix For: 4.2
>
> Attachments: PHOENIX-1395.patch
>
>
> Recently we found that some ResultSpooler*.bin files left in the tmp folder.
> I think those are due to some client code doesn't call close on the returned
> Resultset(which internally will invoke underlying
> OnDiskResultIterator.close()) or client code get killed during result
> iterating.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
