[jira] [Commented] (CASSANDRASC-64) File descriptor is not being closed on MD5 checksum
[ https://issues.apache.org/jira/browse/CASSANDRASC-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17742205#comment-17742205 ] ASF subversion and git services commented on CASSANDRASC-64: Commit ca8dd2e0381a19ede99ce4b70959ce7a584ea0d2 in cassandra-sidecar's branch refs/heads/trunk from Francisco Guerrero [ https://gitbox.apache.org/repos/asf?p=cassandra-sidecar.git;h=ca8dd2e ] CASSANDRASC-64: File descriptor is not being closed on MD5 checksum This commit fixes an issue where the file descriptors are not being closed during the MD5 checksum during SSTable upload. This can potentially cause the JVM to run out of available file descriptors during execution, and it would force an operator to restart the Sidecar process to return to a working state. In this commit, we ensure the file is closed after the checksum is calculated. Additionally, this commit fixes a rare ConcurrentModificationException encountered in `SSTableImporter` whree the `importQueuePerHost` does not use a thread-safe map. patch by Francisco Guerrero; reviewed by Dinesh Joshi, Yifan Cai for CASSANDRASC-64 > File descriptor is not being closed on MD5 checksum > --- > > Key: CASSANDRASC-64 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-64 > Project: Sidecar for Apache Cassandra > Issue Type: Bug > Components: Rest API >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Normal > Labels: pull-request-available > > The file descriptor is not being closed when an MD5 checksum is performed > during SSTable upload. This could prevent the Sidecar JVM to not be able to > open more files when the ulimit has been reached -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-64) File descriptor is not being closed on MD5 checksum
[ https://issues.apache.org/jira/browse/CASSANDRASC-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17742201#comment-17742201 ] Dinesh Joshi commented on CASSANDRASC-64: - +1, thanks for the patch. > File descriptor is not being closed on MD5 checksum > --- > > Key: CASSANDRASC-64 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-64 > Project: Sidecar for Apache Cassandra > Issue Type: Bug > Components: Rest API >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Normal > Labels: pull-request-available > > The file descriptor is not being closed when an MD5 checksum is performed > during SSTable upload. This could prevent the Sidecar JVM to not be able to > open more files when the ulimit has been reached -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-64) File descriptor is not being closed on MD5 checksum
[ https://issues.apache.org/jira/browse/CASSANDRASC-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17742182#comment-17742182 ] Francisco Guerrero commented on CASSANDRASC-64: --- Latest CI is : https://app.circleci.com/pipelines/github/frankgh/cassandra-sidecar/187 > File descriptor is not being closed on MD5 checksum > --- > > Key: CASSANDRASC-64 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-64 > Project: Sidecar for Apache Cassandra > Issue Type: Bug > Components: Rest API >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Normal > Labels: pull-request-available > > The file descriptor is not being closed when an MD5 checksum is performed > during SSTable upload. This could prevent the Sidecar JVM to not be able to > open more files when the ulimit has been reached -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-64) File descriptor is not being closed on MD5 checksum
[ https://issues.apache.org/jira/browse/CASSANDRASC-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17742176#comment-17742176 ] Francisco Guerrero commented on CASSANDRASC-64: --- Thanks for the review [~djoshi]. I've [added tests|https://github.com/apache/cassandra-sidecar/pull/60/files#diff-75f4fef2f1735f70f9810d36a95a1d7ff58960a86da05bd8c2652007c939f9a9R46] that ensure that the file is being closed. > File descriptor is not being closed on MD5 checksum > --- > > Key: CASSANDRASC-64 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-64 > Project: Sidecar for Apache Cassandra > Issue Type: Bug > Components: Rest API >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Normal > Labels: pull-request-available > > The file descriptor is not being closed when an MD5 checksum is performed > during SSTable upload. This could prevent the Sidecar JVM to not be able to > open more files when the ulimit has been reached -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-64) File descriptor is not being closed on MD5 checksum
[ https://issues.apache.org/jira/browse/CASSANDRASC-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17742174#comment-17742174 ] Yifan Cai commented on CASSANDRASC-64: -- +1 on the patch Thanks for adding the test and addressing my comment > File descriptor is not being closed on MD5 checksum > --- > > Key: CASSANDRASC-64 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-64 > Project: Sidecar for Apache Cassandra > Issue Type: Bug > Components: Rest API >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Normal > Labels: pull-request-available > > The file descriptor is not being closed when an MD5 checksum is performed > during SSTable upload. This could prevent the Sidecar JVM to not be able to > open more files when the ulimit has been reached -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-64) File descriptor is not being closed on MD5 checksum
[ https://issues.apache.org/jira/browse/CASSANDRASC-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17742135#comment-17742135 ] Dinesh Joshi commented on CASSANDRASC-64: - hi [~frankgh] thank you for the patch. It would be a good idea to add a test that will check for file descriptor leaks. You could run several iterations forcing failure and checking that the file descriptors are not accumulating in the test run. > File descriptor is not being closed on MD5 checksum > --- > > Key: CASSANDRASC-64 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-64 > Project: Sidecar for Apache Cassandra > Issue Type: Bug > Components: Rest API >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Normal > Labels: pull-request-available > > The file descriptor is not being closed when an MD5 checksum is performed > during SSTable upload. This could prevent the Sidecar JVM to not be able to > open more files when the ulimit has been reached -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-64) File descriptor is not being closed on MD5 checksum
[
https://issues.apache.org/jira/browse/CASSANDRASC-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17741965#comment-17741965
]
Francisco Guerrero commented on CASSANDRASC-64:
---
Using {{lsof -p }} I was able to see many file descriptors still
open after the MD5 checksum was performed
> File descriptor is not being closed on MD5 checksum
> ---
>
> Key: CASSANDRASC-64
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-64
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Rest API
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Labels: pull-request-available
>
> The file descriptor is not being closed when an MD5 checksum is performed
> during SSTable upload. This could prevent the Sidecar JVM to not be able to
> open more files when the ulimit has been reached
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
