[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16734130#comment-16734130
]
Dimitar Dimitrov commented on CASSANDRA-13692:
--
I have reworked the changes as discussed, and am currently testing.
Initial unit test results are relatively good - 3.0 and 3.11 seem OK (no
failures), and trunk seems to have a bunch of unrelated failures (e.g.
{{SingleSSTableLCSTaskTest}} failing with an OOME,
{{org.apache.cassandra.dht.tokenallocator}} tests failing with a NPE in
{{DatabaseDescriptor.diagnosticEventsEnabled}}). 2.2 has some failures in
{{ScrubTest}} and {{SSTableRewriterTest}} that I'd like to take a closer look
at.
I still don't have initial results for dtests, due to these being harder to
land on a CI VM, and longer to run. I'll update when I have something to report
though.
Here are the draft changes, in case anyone is interested:
|
[2.2|https://github.com/apache/cassandra/compare/cassandra-2.2...dimitarndimitrov:c13692-2.2]
|
|
[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...dimitarndimitrov:c13692-3.0]
|
|
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...dimitarndimitrov:c13692-3.11]
|
|
[trunk|https://github.com/apache/cassandra/compare/trunk...dimitarndimitrov:c13692]
|
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
>Priority: Major
> Labels: lhf
> Attachments: c13692-2.2-dtest-results.PNG,
> c13692-2.2-testall-results.PNG, c13692-3.0-dtest-results-updated.PNG,
> c13692-3.0-dtest-results.PNG, c13692-3.0-testall-results.PNG,
> c13692-3.11-dtest-results-updated.PNG, c13692-3.11-dtest-results.PNG,
> c13692-3.11-testall-results.PNG, c13692-dtest-results-updated.PNG,
> c13692-dtest-results.PNG, c13692-testall-results-updated.PNG,
> c13692-testall-results.PNG
>
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The fixed code throws FSWE and triggers the
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722172#comment-16722172
]
Dimitar Dimitrov commented on CASSANDRA-13692:
--
First, apologies for making this look like a 19th century trans-continental
chess correspondence game...
I think you're absolutely right - after
[{{849a438690aa97a361227781108cc90355dcbcd9}}|https://github.com/apache/cassandra/commit/849a438690aa97a361227781108cc90355dcbcd9],
we return solely candidates from some subset of
{{Directories.dataDirectories}}, all of which are initialized in the
{{clinit}}, so currently it doesn't look like there's a case in which the error
handling could be hit.
I agree with your suggestion (although I'm a tiny bit sad that the
oh-so-laborious test would go as well).
I'll dust off the ancient branches that I had for this and update here with the
corresponding patches soon.
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
>Priority: Major
> Labels: lhf
> Attachments: c13692-2.2-dtest-results.PNG,
> c13692-2.2-testall-results.PNG, c13692-3.0-dtest-results-updated.PNG,
> c13692-3.0-dtest-results.PNG, c13692-3.0-testall-results.PNG,
> c13692-3.11-dtest-results-updated.PNG, c13692-3.11-dtest-results.PNG,
> c13692-3.11-testall-results.PNG, c13692-dtest-results-updated.PNG,
> c13692-dtest-results.PNG, c13692-testall-results-updated.PNG,
> c13692-testall-results.PNG
>
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The fixed code throws FSWE and triggers the failure policy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16513964#comment-16513964
]
Joel Knighton commented on CASSANDRA-13692:
---
Thanks again for the patch. I apologize for my genuinely staggering lateness
here; you deserve better. Some thoughts:
* On all versions, I don't think we'll actually hit the error handling here.
In CASSANDRA-9669, we pushed error handling down into {{getWriteableLocation}}
so that callers don't need to check for null and throw an exception themselves.
* On 2.2, in the error-handling, we still throw a {{RuntimeException}} from
within {{getWriteableLocation}}. I think this probably remains the right thing
to do. in CASSANDRA-11828, for 3.0+, we switched to throwing an
{{FSWriteError}} from {{getWriteableLocation}}. This caused the behavior in
CASSANDRA-12385 which we fixed by adding an {{FSDiskFullWriteError}} that we
throw instead and handle in ACT. I think backporting all that to 2.2 is
probably unnecessary given the age of 2.2.
Am I missing a situation in which we'd hit this error handling? It really seems
to me that we longer return null from {{getWriteableLocation}}. If I'm not, I
think the best thing to do is update the javadocs of affected methods and
remove the unused error handling from {{CompactionAwareWriter}}.
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
>Priority: Major
> Labels: lhf
> Attachments: c13692-2.2-dtest-results.PNG,
> c13692-2.2-testall-results.PNG, c13692-3.0-dtest-results-updated.PNG,
> c13692-3.0-dtest-results.PNG, c13692-3.0-testall-results.PNG,
> c13692-3.11-dtest-results-updated.PNG, c13692-3.11-dtest-results.PNG,
> c13692-3.11-testall-results.PNG, c13692-dtest-results-updated.PNG,
> c13692-dtest-results.PNG, c13692-testall-results-updated.PNG,
> c13692-testall-results.PNG
>
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The fixed code
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16166361#comment-16166361
]
Joel Knighton commented on CASSANDRA-13692:
---
Thanks for the patch (and rebase/test rerun)! I'll review this later today.
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
> Labels: lhf
> Attachments: c13692-2.2-dtest-results.PNG,
> c13692-2.2-testall-results.PNG, c13692-3.0-dtest-results.PNG,
> c13692-3.0-dtest-results-updated.PNG, c13692-3.0-testall-results.PNG,
> c13692-3.11-dtest-results.PNG, c13692-3.11-dtest-results-updated.PNG,
> c13692-3.11-testall-results.PNG, c13692-dtest-results.PNG,
> c13692-dtest-results-updated.PNG, c13692-testall-results.PNG,
> c13692-testall-results-updated.PNG
>
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The fixed code throws FSWE and triggers the failure policy.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16166103#comment-16166103
]
Dimitar Dimitrov commented on CASSANDRA-13692:
--
Okay, it looks like the {{trunk}} dtest abort problems have been fixed - and
here are the new test results that confirm that.
Unfortunately the runs are still not green.
Nevertheless the failures seen in the baseline and my run are almost identical,
with 2 failures from cassci not showing up in my run, and 1 failure from my run
not showing up in cassci.
* To better compare the failures, sort the cassci results by test name, as the
results from my run have also been sorted this way.
|
[2.2|https://github.com/apache/cassandra/compare/cassandra-2.2...dimitarndimitrov:c13692-2.2]
| [testall|^c13692-2.2-testall-results.PNG] |
[dtest|^c13692-2.2-dtest-results.PNG]
([dtest-baseline|https://cassci.datastax.com/job/cassandra-2.2_dtest/lastCompletedBuild/testReport/])
|
|
[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...dimitarndimitrov:c13692-3.0]
| [testall|^c13692-3.0-testall-results.PNG] |
[dtest|^c13692-3.0-dtest-results-updated.PNG]
([dtest-baseline|https://cassci.datastax.com/job/cassandra-3.0_dtest/989/testReport/])
|
|
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...dimitarndimitrov:c13692-3.11]
| [testall|^c13692-3.11-testall-results.PNG]
([testall-baseline|https://cassci.datastax.com/job/cassandra-3.11_testall/lastCompletedBuild/testReport/])
| [dtest|^c13692-3.11-dtest-results-updated.PNG]
([dtest-baseline|https://cassci.datastax.com/job/cassandra-3.11_dtest/165/testReport/])
|
|
[trunk|https://github.com/apache/cassandra/compare/trunk...dimitarndimitrov:c13692]
| [testall|^c13692-testall-results.PNG] |
[*dtest*|^c13692-dtest-results-updated.PNG]
([*dtest-baseline*|https://cassci.datastax.com/job/trunk_dtest/1654/testReport/])
|
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
> Labels: lhf
> Attachments: c13692-2.2-dtest-results.PNG,
> c13692-2.2-testall-results.PNG, c13692-3.0-dtest-results.PNG,
> c13692-3.0-dtest-results-updated.PNG, c13692-3.0-testall-results.PNG,
> c13692-3.11-dtest-results.PNG, c13692-3.11-dtest-results-updated.PNG,
> c13692-3.11-testall-results.PNG, c13692-dtest-results.PNG,
> c13692-testall-results.PNG
>
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
>
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155061#comment-16155061
]
Dimitar Dimitrov commented on CASSANDRA-13692:
--
Here's the table with the updated test results (in bold, trunk dtest stability
issues notwithstanding):
|
[2.2|https://github.com/apache/cassandra/compare/cassandra-2.2...dimitarndimitrov:c13692-2.2]
| [testall|^c13692-2.2-testall-results.PNG] |
[dtest|^c13692-2.2-dtest-results.PNG]
([dtest-baseline|https://cassci.datastax.com/job/cassandra-2.2_dtest/lastCompletedBuild/testReport/])
|
|
[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...dimitarndimitrov:c13692-3.0]
| [testall|^c13692-3.0-testall-results.PNG] |
[*dtest*|^c13692-3.0-dtest-results-updated.PNG]
([*dtest-baseline*|https://cassci.datastax.com/job/cassandra-3.0_dtest/989/testReport/])
|
|
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...dimitarndimitrov:c13692-3.11]
| [testall|^c13692-3.11-testall-results.PNG]
([testall-baseline|https://cassci.datastax.com/job/cassandra-3.11_testall/lastCompletedBuild/testReport/])
| [*dtest*|^c13692-3.11-dtest-results-updated.PNG]
([*dtest-baseline*|https://cassci.datastax.com/job/cassandra-3.11_dtest/165/testReport/])
|
|
[trunk|https://github.com/apache/cassandra/compare/trunk...dimitarndimitrov:c13692]
| [testall|^c13692-testall-results.PNG] | [dtest|^c13692-dtest-results.PNG]
([dtest-baseline|https://cassci.datastax.com/job/trunk_dtest/lastCompletedBuild/testReport/])
|
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
> Labels: lhf
> Attachments: c13692-2.2-dtest-results.PNG,
> c13692-2.2-testall-results.PNG, c13692-3.0-dtest-results.PNG,
> c13692-3.0-testall-results.PNG, c13692-3.11-dtest-results.PNG,
> c13692-3.11-testall-results.PNG, c13692-dtest-results.PNG,
> c13692-testall-results.PNG
>
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154343#comment-16154343
]
Dimitar Dimitrov commented on CASSANDRA-13692:
--
I ended up rebasing the 3.0, 3.11, and trunk versions of the change, and the
results are as follows:
* No more consistency_test.TestConsistency.test_13747 failures for the 3.0 and
3.11 changes. Small differences between actual and expected dtest failures,
which seem to be flaking out.
* The trunk changes are still hitting what seems to be an existing problem
plaguing cassci.datastax.com trunk dtest jobs for the last 10 days or so (see
https://cassci.datastax.com/job/trunk_dtest/).
** The problem is currently being investigated.
Now it looks like the 2.2, 3.0, and 3.11 changes can be accepted as passing the
testall and dtest criteria, only the trunk changes need to be verified, after
the problem affecting trunk gets resolved. I'll post an update once this
happens, but in the meantime, I'd assume it's safe to mark this as "Awaiting
Feedback".
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
> Labels: lhf
> Attachments: c13692-2.2-dtest-results.PNG,
> c13692-2.2-testall-results.PNG, c13692-3.0-dtest-results.PNG,
> c13692-3.0-testall-results.PNG, c13692-3.11-dtest-results.PNG,
> c13692-3.11-testall-results.PNG, c13692-dtest-results.PNG,
> c13692-testall-results.PNG
>
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The fixed code throws FSWE and triggers the failure policy.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152476#comment-16152476
]
Aleksey Yeschenko commented on CASSANDRA-13692:
---
That would explain it. It's a regression test - it's supposed to fail on all
3.0/3.11/trunk branches that don't have CASSANDRA-13747 fix in them.
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
> Labels: lhf
> Attachments: c13692-2.2-dtest-results.PNG,
> c13692-2.2-testall-results.PNG, c13692-3.0-dtest-results.PNG,
> c13692-3.0-testall-results.PNG, c13692-3.11-dtest-results.PNG,
> c13692-3.11-testall-results.PNG, c13692-dtest-results.PNG,
> c13692-testall-results.PNG
>
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The fixed code throws FSWE and triggers the failure policy.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152425#comment-16152425
]
Dimitar Dimitrov commented on CASSANDRA-13692:
--
All observed {{dtest}} failures, including the
consistency_test.TestConsistency.test_13747 failures, reproduce exactly the
same on brand new copies of the cassandra-3.11 and trunk branches of my
apache/cassandra fork.
My fork is by now tens of commits behind the origin, so I'll update the fork,
and re-run the CI jobs for the 3.0, 3.11, and trunk branches.
I'm expecting to see a much more consistent picture this time.
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
> Labels: lhf
> Attachments: c13692-2.2-dtest-results.PNG,
> c13692-2.2-testall-results.PNG, c13692-3.0-dtest-results.PNG,
> c13692-3.0-testall-results.PNG, c13692-3.11-dtest-results.PNG,
> c13692-3.11-testall-results.PNG, c13692-dtest-results.PNG,
> c13692-testall-results.PNG
>
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The fixed code throws FSWE and triggers the failure policy.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150387#comment-16150387
]
Aleksey Yeschenko commented on CASSANDRA-13692:
---
[~dimitarndimitrov] I've noticed and I know who you are. Welcome to the
community (:
FWIW I've run the dtest locally a couple dozen times, and it's passing just
fine.
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
> Labels: lhf
> Attachments: c13692-2.2-dtest-results.PNG,
> c13692-2.2-testall-results.PNG, c13692-3.0-dtest-results.PNG,
> c13692-3.0-testall-results.PNG, c13692-3.11-dtest-results.PNG,
> c13692-3.11-testall-results.PNG, c13692-dtest-results.PNG,
> c13692-testall-results.PNG
>
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The fixed code throws FSWE and triggers the failure policy.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150351#comment-16150351
]
Dimitar Dimitrov commented on CASSANDRA-13692:
--
Ah, sorry, I should have at least attached the failure logs - I didn't attach
the build artifacts, as I wasn't sure if they were sanitized with regard to
non-public data.
I'll sync with a more knowledgeable colleague, and get back to you with the
necessary info.
P.S. Like you've probably noticed, I'm still new to one of the more visible
presences here, and many of the steps in the process are a bit hazy to me -
I'll make sure to improve quickly on that though :)
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
> Labels: lhf
> Attachments: c13692-2.2-dtest-results.PNG,
> c13692-2.2-testall-results.PNG, c13692-3.0-dtest-results.PNG,
> c13692-3.0-testall-results.PNG, c13692-3.11-dtest-results.PNG,
> c13692-3.11-testall-results.PNG, c13692-dtest-results.PNG,
> c13692-testall-results.PNG
>
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The fixed code throws FSWE and triggers the failure policy.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150355#comment-16150355
]
Dimitar Dimitrov commented on CASSANDRA-13692:
--
Some additional observations, after taking yet another look at the test results:
* Although very similar, the 3.11 {{testall}} failures are not exactly the same
as the ones in the baseline.
* The trunk {{dtest}} failures seem to diverge from the pattern of
"common-expected-to-be-unrelated failures plus test_13747 failures". I'll try
to see whether this can be attributed to flakiness (looking closer to the
results, re-running the CI run on the same branch, running another CI run on a
clean branch copy of the trunk, etc.)
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
> Labels: lhf
> Attachments: c13692-2.2-dtest-results.PNG,
> c13692-2.2-testall-results.PNG, c13692-3.0-dtest-results.PNG,
> c13692-3.0-testall-results.PNG, c13692-3.11-dtest-results.PNG,
> c13692-3.11-testall-results.PNG, c13692-dtest-results.PNG,
> c13692-testall-results.PNG
>
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The fixed code throws FSWE and triggers the failure policy.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150291#comment-16150291
]
Aleksey Yeschenko commented on CASSANDRA-13692:
---
[~dimitarndimitrov] Maybe, maybe not. The screenshot is useless to me as I
can't click on test details.
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
> Labels: lhf
> Attachments: c13692-2.2-dtest-results.PNG,
> c13692-2.2-testall-results.PNG, c13692-3.0-dtest-results.PNG,
> c13692-3.0-testall-results.PNG, c13692-3.11-dtest-results.PNG,
> c13692-3.11-testall-results.PNG, c13692-dtest-results.PNG,
> c13692-testall-results.PNG
>
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The fixed code throws FSWE and triggers the failure policy.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150212#comment-16150212
]
Dimitar Dimitrov commented on CASSANDRA-13692:
--
Okay, here are the branches with the proposed changes:
|
[2.2|https://github.com/apache/cassandra/compare/cassandra-2.2...dimitarndimitrov:c13692-2.2]
| [testall|^c13692-2.2-testall-results.png] |
[dtest|^c13692-2.2-dtest-results.png]
([dtest-baseline|https://cassci.datastax.com/job/cassandra-2.2_dtest/lastCompletedBuild/testReport/])
|
|
[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...dimitarndimitrov:c13692-3.0]
| [testall|^c13692-3.0-testall-results.png] |
[dtest|^c13692-3.0-dtest-results.png]
([dtest-baseline|https://cassci.datastax.com/job/cassandra-3.0_dtest/lastCompletedBuild/testReport/])
|
|
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...dimitarndimitrov:c13692-3.11]
| [testall|^c13692-3.11-testall-results.png]
([testall-baseline|https://cassci.datastax.com/job/cassandra-3.11_testall/lastCompletedBuild/testReport/])
|
[dtest|^c13692-3.11-dtest-results.png]
([dtest-baseline|https://cassci.datastax.com/job/cassandra-3.11_dtest/lastCompletedBuild/testReport/])
|
|
[trunk|https://github.com/apache/cassandra/compare/trunk...dimitarndimitrov:c13692]
| [testall|^c13692-testall-results.png] |
[dtest|^c13692-dtest-results.png]
([dtest-baseline|https://cassci.datastax.com/job/trunk_dtest/lastCompletedBuild/testReport/])
|
{{testall}} looks good for all branches, but there's a common theme of
consistency_test.TestConsistency.test_13747 {{dtest}}s failing, in addition to
the common-expected-to-be-unrelated {{dtest}} failures.
My assumption is that this is related to CASSANDRA-13747 (the comments there
seem to corroborate that). [~iamaleksey] , do you have an idea if that could be
the case?
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
> Labels: lhf
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"),
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146000#comment-16146000
]
Dimitar Dimitrov commented on CASSANDRA-13692:
--
Sorry for the late promised update.
I'm currently finalizing the test, which is a bit of a mix between
{{OutOfSpaceTest}}, {{CompactionAwareWriterTest}}, and
{{CompactionsBytemanTest}}.
I reckon that (1) the {{OutOfSpaceTest}}s approach translates well enough for
checking whether failure policies are correctly triggered in this case; (2) the
trickiest part (for me) would be to figure out whether the way
{{CompactionAwareWriter#getWriteDirectory(Iterable, long)}} gets
reached and executed, is correct and relevant.
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
> Labels: lhf
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The fixed code throws FSWE and triggers the failure policy.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16143538#comment-16143538
]
Dimitar Dimitrov commented on CASSANDRA-13692:
--
It looks like a good approach here would be to write a test reproducing the
problem (with one test for each branch throwing a {{RuntimeException}}), then
apply what looks like the obvious solution (replacing the {{RuntimeExceptions
}}with {{FSWriteErrors}}) and see if the failure policy is triggered.
{{org.apache.cassandra.cql3.OutOfSpaceTest}} may be a good starting point, but
I'll also need to understand better under what conditions exactly is
{{CompactionAwareWriter#getWriteDirectory(Iterable, long)}}
triggered.
I'll post another update later today.
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
> Labels: lhf
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The fixed code throws FSWE and triggers the failure policy.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
