[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783124#comment-17783124
]
Berenguer Blasi commented on CASSANDRA-14227:
-
That is weird because we always talked about 'nc' and I remember testing 4.1
with nc. I will have to check...
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 5.0, 5.0-alpha1
>
> Attachments: C14227 Perf check 2023.03.21.pdf, C14227 Perf check
> 2023.05.26.pdf, screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781651#comment-17781651
]
Michael Semb Wever commented on CASSANDRA-14227:
Shouldn't CASSANDRA_4 mode be using the `nb` format ?
It's currently using `nc`, but looking through 4.0 and 4.1 latest versions they
are on `nb`.
(See CASSANDRA-18933)
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 5.0, 5.0-alpha1
>
> Attachments: C14227 Perf check 2023.03.21.pdf, C14227 Perf check
> 2023.05.26.pdf, screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729180#comment-17729180
]
Berenguer Blasi commented on CASSANDRA-14227:
-
Merged. Thx everyone for all the help!
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 5.0
>
> Attachments: C14227 Perf check 2023.03.21.pdf, C14227 Perf check
> 2023.05.26.pdf, screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17727560#comment-17727560
]
Berenguer Blasi commented on CASSANDRA-14227:
-
[~benedict] Everything looks good as far as we can see. Do you think you could
find a gap see if we can +1 and merge :-) You're probably most interested in
the yaml feature flag commit
[here|https://github.com/apache/cassandra/pull/1891/commits/159eabbc7a0dec932864447ecffbaae5ffe66c4c]
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 5.x
>
> Attachments: C14227 Perf check 2023.03.21.pdf, C14227 Perf check
> 2023.05.26.pdf, screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17727558#comment-17727558
]
Andres de la Peña commented on CASSANDRA-14227:
---
Looks good to me after rebase, and CI also looks good. +1 from me. The few
remaining, trivial suggestions about comments can be addressed on commit.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 5.x
>
> Attachments: C14227 Perf check 2023.03.21.pdf, C14227 Perf check
> 2023.05.26.pdf, screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726565#comment-17726565
]
Berenguer Blasi commented on CASSANDRA-14227:
-
Latest perf check after rebases etc LGTM [^C14227 Perf check 2023.05.26.pdf]
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 5.x
>
> Attachments: C14227 Perf check 2023.03.21.pdf, C14227 Perf check
> 2023.05.26.pdf, screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724865#comment-17724865
]
Berenguer Blasi commented on CASSANDRA-14227:
-
Thx, it should be pretty non-controversial and we can always tweak the name etc.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 5.x
>
> Attachments: C14227 Perf check 2023.03.21.pdf, screenshot-1.png,
> screenshot-2.png, screenshot-3.png, screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724840#comment-17724840
]
Benedict Elliott Smith commented on CASSANDRA-14227:
Sorry, the downside of lots of Jira traffic (incl from GitHub comments) is that
I don't check the email notifications for a high traffic ticket.
I won't have time to look at the code soon, but I trust you to have addressed
my concerns given what you describe above. Feel free to proceed.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 5.x
>
> Attachments: C14227 Perf check 2023.03.21.pdf, screenshot-1.png,
> screenshot-2.png, screenshot-3.png, screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724760#comment-17724760
]
Berenguer Blasi commented on CASSANDRA-14227:
-
Hi [~benedict] we're a bit blocked here, if you could find a gap it should be a
quick review for the FF having been moved to yaml and it would be highly
appreciated. Thx.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 5.x
>
> Attachments: C14227 Perf check 2023.03.21.pdf, screenshot-1.png,
> screenshot-2.png, screenshot-3.png, screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723785#comment-17723785
]
Berenguer Blasi commented on CASSANDRA-14227:
-
^The diff to the commit a few days ago is that the property has been moved to
yaml as suggested by [~benedict]
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 5.x
>
> Attachments: C14227 Perf check 2023.03.21.pdf, screenshot-1.png,
> screenshot-2.png, screenshot-3.png, screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723540#comment-17723540
]
Andres de la Peña commented on CASSANDRA-14227:
---
This looks good to me. I have left some final minor nits on the ticket.
If [~benedict] agrees with the approach for compatibility we can probably
rebase and solve [those naming
details|https://github.com/apache/cassandra/pull/1891/files#r1188414272] on the
new CircleCI jobs.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 5.x
>
> Attachments: C14227 Perf check 2023.03.21.pdf, screenshot-1.png,
> screenshot-2.png, screenshot-3.png, screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17721186#comment-17721186
]
Berenguer Blasi commented on CASSANDRA-14227:
-
[~benedict] a feature flag with upgrade tests etc has been added. I have
provided links to the key bits of code
[here|https://github.com/apache/cassandra/pull/1891#issuecomment-1541352411].
If we could +1 I would start getting ready for the merge :-)
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 5.x
>
> Attachments: C14227 Perf check 2023.03.21.pdf, screenshot-1.png,
> screenshot-2.png, screenshot-3.png, screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703527#comment-17703527
]
Berenguer Blasi commented on CASSANDRA-14227:
-
Perf check report post review and big rebase [^C14227 Perf check
2023.03.21.pdf] LGTM
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 5.x
>
> Attachments: C14227 Perf check 2023.03.21.pdf, screenshot-1.png,
> screenshot-2.png, screenshot-3.png, screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17692872#comment-17692872
]
Benedict Elliott Smith commented on CASSANDRA-14227:
I decided to have a quick look at adding support for dynamically determining
the output format based on whether the TTL data requires it, and I noticed that
EncodingStats and EncodingStats.Collector likely treat TTL incorrectly, using
simple min/max on int. This might not cause any bugs, but it might, and we
should fix it - we should probably consider what additional testing we might
need to catch this kind of error elsewhere.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691608#comment-17691608
]
Benedict Elliott Smith commented on CASSANDRA-14227:
This is the canonical example that was given in the original thread more than a
year ago, i.e. that a cluster should not write files that are incompatible with
the version we are upgrading from until the operator agrees the upgrade has
been successful, yes. If there are performance or other regressions encountered
during an upgrade, recovery should be as simple as restarting with the prior
version (until the operator decides the new version is performing adequately).
This minimises risks, without fully eliminating them.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691603#comment-17691603
]
Benjamin Lerer commented on CASSANDRA-14227:
I think, that I am misunderstanding what we call downgradability. What you
propose is some kind of feature flag that delay the problem, no? If a problem
occurs once we switch to the new sstable format then we cannot downgrade
anymore.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691224#comment-17691224
]
Benedict Elliott Smith commented on CASSANDRA-14227:
By default we should reject TTLs past 2038, and while this setting is in place
we should continue to write \-nc\- format sstables. Once the operator
configures longer TTLs, we can write \-oa\- format sstables.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691217#comment-17691217
]
Benjamin Lerer commented on CASSANDRA-14227:
{quote} It looks like we have an option to CAP to 2038, and if we default to
this on upgrade I think a downgrade should be smooth.{quote}
[~benedict] Could you elaborate a bit what you mean? I am not sure that I am
understanding what you have in mind.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688923#comment-17688923
]
Benedict Elliott Smith commented on CASSANDRA-14227:
Downgradeability isn't under continuing discussion at present, no, but you are
welcome to raise it for discussion.
Either way, if it makes it simpler: this patch can easily avoid breaking
downgrade, so it *must* not (or it is subject to my binding -1).
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688862#comment-17688862
]
Berenguer Blasi commented on CASSANDRA-14227:
-
Seems like downgradability, at the time of writing, is still being discussed
iiuc.
Now we're also writing an int, with the uint approach, and the sstable version
change is to signal a negative int being 'undefined data' vs 'a valid uint'.
Regardless of the downgradability discussion _I think_ (not tested) an sstable
scrub would suffice as scrub will either REJECT or CAP those entries. That is
pending the downgradibility discussion detailed requirements: forward
compatibility?, flag?, scrub?
Sthg to take into account though, thx for bringing it up.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688462#comment-17688462
]
Benedict Elliott Smith commented on CASSANDRA-14227:
[~jlewandowski] helpfully pointed out we should consider downgrade for this
patch. It looks like we have an option to CAP to 2038, and if we default to
this on upgrade I think a downgrade should be smooth. I'm not certain we even
need to change the sstable version? Though I haven't looked closely.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683716#comment-17683716
]
Berenguer Blasi commented on CASSANDRA-14227:
-
Putting this back into review with the new Uint approach. I have done a final
round of perf testing:
* Perf testing revolved around 10M rows and runs with random/fixed TTL for
period of 2m, 5m and 10m. The exception being the much longer latency test.
* Longer tests tend to give less noisy results. 14227 vs trunk may randomly
appear one slightly better than the other which I attribute to env noise and
lack of dedicated perf testing HW.
* Latency seems to be aligned as per comment above
* Disk size, flushes etc seem to be aligned. Here is some table stats where
sometimes 14227 is slightly better and sometimes the other way around
!unnamed.png!
The PR has several commits to make it easier to review. Some refactoring of an
earlier commit might be found in a later one but nothing too big and quite
straightforward. I have left a bunch of 14227 TODO comments to bring the
attention of the reviewer just for feedback. CI can be found in the PR.
The only thing to note is the addition of the NONE policy where it's later
removed. That is the only bit it can't be easily collapsed, apologies in
advance.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png, unnamed.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680895#comment-17680895
]
Berenguer Blasi commented on CASSANDRA-14227:
-
Uint POC seems to be OK as well on the latency side. There are latencies on 2
completely different tests with even 2 diff test tooling:
10 averaged runs latency
!screenshot-3.png|thumbnail!
Averaged latency on a 1h test
!screenshot-4.png|thumbnail!
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675917#comment-17675917
]
Berenguer Blasi commented on CASSANDRA-14227:
-
Update: The uint approach is giving good results in the POC on disk size,
memtable, etc. I need to rebase, put capping back and revisit all perf.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17636510#comment-17636510
]
Berenguer Blasi commented on CASSANDRA-14227:
-
After some further perf testing we've found out there is a 3% disk size
increase hit we've like to avoid as discussed in the
[ML|https://lists.apache.org/thread/fyf2d9jlrsor3hmz46qn282o5gwd]. We're
going to experiment and POC a bit a unit encoding to get rid of that and any
extra flushes as well.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619357#comment-17619357
]
Berenguer Blasi commented on CASSANDRA-14227:
-
Hi,
I have been asked to run some profiling to see the impact 'longs' would have. I
have tested on a single node doing inserts with a TTL + selects against that
for 4m at around 100Kops/s after a warm up run. The results are virtual
identical to both trunk and 14227 both at jfr and stress tool perf reporting
Trunk:
!screenshot-1.png!
14227
!screenshot-2.png!
If anything 14227's number are slightly better but probably just test env
noise. GC pauses, total times, latencies, etc are all identical as well. The
test CQL was:
{noformat}
CQL|INSERT INTO test.test (id, type, text) VALUES ($RANDOM_2, $RANDOM_10,
'TTL Profiling') USING TTL $RANDOM_50
CQL|SELECT id, type, text FROM test.test WHERE id=$RANDOM_2
{noformat}
where RANDOM_X means a random number up to X. Here we can see inserts, inserts
colliding and selects. I will be happy to repeat the test if anybody has any
suggestions. I am not posting the jfr for security reasons as the env vars
section contains sensitive data (call me paranoid) but I can share them with
known people on request.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17610845#comment-17610845
]
Berenguer Blasi commented on CASSANDRA-14227:
-
Hi,
moving to this to review. Some comments:
- Max TTL is now 68y and there is a new default NONE overflow policy, the other
ones are kept for backwards compatibility.
- All tests pass but for one upgrade rolling failing 10% of times. I am
investigating it but given it is taking a lot of time and I can't repro lcoally
I prefer to start with the review already.
- The code has a few 'TODO 14227' comments which are points I would like to
discuss with the reviewer such as
{{StreamingTombstoneHistogramBuilder#mergeNearestPoints()}}
- Most files only change int to longs, think about sentinel values while
reviewing and possible side-effects
- The PR has a few self-contained commits for an easier review.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520768#comment-17520768
]
Michael Shuler commented on CASSANDRA-14227:
Removed assignee. I only found a few random state changes by the user in Jira,
so this assignment also looked random. If this was incorrect, please reassign!
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Priority: Urgent
> Fix For: 4.x
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450322#comment-17450322
]
Spiros Ioannou commented on CASSANDRA-14227:
This is a critical issue, that severely limits TTL functionality and future
design, the CAP_NOWARN workaround of CASSANDRA-14092 is already limiting our
data lifecycle. I'm not sure why this is high-severity issue was not planned
for 4.0, what's missing?
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Priority: Urgent
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16892420#comment-16892420
]
Laxmikant Upadhyay commented on CASSANDRA-14227:
[~pauloricardomg] I believe there are many use-cases where users maintain the
TTL of a row while updating it. Hence in all such cases original TTL will be
lost. So, is it worth giving TTL recalculation feature in upgradesstable which
won't be applicable to all users ?
I agree that a permanent fix, changing {{localDeletionTime}} field from integer
to long is simplest way to go but this change will be big in terms changes in
number of classes and we need to carefully analyse the impact as well.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta
>Priority: Normal
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890992#comment-16890992
]
Paulo Motta commented on CASSANDRA-14227:
-
{quote}This results 584043668 and not 63072 ! and then updating the row
with fetched ttl loses the original ttl value (20 years) so resuming
localDeletionTime based on ttl the value will not be feasible if someone
chooses CAP or CAP_NOWARN option as in sstable, TTL value is no more 20 years
(63072).
{quote}
The fact that {{SELECT TTL(*)}} does not return the original TTL does not mean
we cannot resume the original TTL once this is fixed. Data will need to be
rewritten via "upgradesstables" to restore the original TTL once this
limitation is fixed. If you are updating the row with fetched TTL you are
overwriting the original value, so it's expected behavior that you lose the
original TTL.
If you are willing to take a stab at fixing this, I think the simplest approach
is to change the {{localDeletionTime}} field from integer to long.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta
>Priority: Normal
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887860#comment-16887860
]
Laxmikant Upadhyay commented on CASSANDRA-14227:
Resuming the {{localDeletionTime}} value after update on a row with TTL does
not look feasible. Please see the below test case:
*Step 1:*
insert into tab3(id,key,value) VALUES('id2', 'key1', 2) USING TTL 63072;
(20 years).
{code:java}
sstabletojson:
{
"partition" : {
"key" : [ "id2" ],
"position" : 33
},
"rows" : [
{
"type" : "row",
"position" : 75,
"clustering" : [ "key1" ],
"liveness_info" : { "tstamp" : "2019-07-18T07:03:22.198Z", "ttl" : 63072,
"expires_at" : "2038-01-19T03:14:06Z", "expired" : false },
"cells" : [
{ "name" : "value", "value" : 2 }
]
}
]
}{code}
*Step 2:*
1. select ttl(value) from tab3 where id='id2';
This results 584043668 ! and then updating the row with fetched ttl loses the
original ttl value (20 years) so resuming {{localDeletionTime}} based on ttl
the value will not be feasible if someone chooses CAP or CAP_NOWARN option.
2. insert into tab3(id,key,value) VALUES('id2', 'key1', 3) USING TTL 584043668;
{code:java}
sstabletojson:
{
"partition" : {
"key" : [ "id2" ],
"position" : 33
},
"rows" : [
{
"type" : "row",
"position" : 75,
"clustering" : [ "key1" ],
"liveness_info" : { "tstamp" : "2019-07-18T08:54:27.921Z", "ttl" : 584043668,
"expires_at" : "2038-01-19T03:14:06Z", "expired" : false },
"cells" : [
{ "name" : "value", "value" : 3 }
]
}
]
}{code}
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta
>Priority: Normal
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855350#comment-16855350
]
Laxmikant Upadhyay commented on CASSANDRA-14227:
[~pauloricardomg] Upgrade Sstable change to resume the original ttl should be
in scope of this bug or should we open another jira ticket for this?
Note: This change is important for those users who are currently using CAP or
CAP_NOWARN to avoid request rejection but want to resume to originally set TTL
when they upgrade to new cassandra version with fix.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta
>Priority: Normal
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427544#comment-16427544
]
Paulo Motta commented on CASSANDRA-14227:
-
It would be great to get this in for 4.0. The simplest approach would be to
change the {{localDeletionTime}} representation to {{long}} as proposed by
[~VincentWhite], but we need to verify the impact of this on heap for TTL and
non-TTL workloads.
Alternatives include using a unsigned int32 to represent {{localDeletionTime}}
and/or using a later start epoch (maybe 2000 or so) - but this would require
some considerate work to maintain backward compatibility and would difficult
interoperability with other systems - even though it's an internal structure.
[~VincentWhite] would you be willing to work on this and maybe perform some
stress tests to check impact on heap and GC of changing the
{{localDeletionTime}} representation from {{int}} to {{long}} ?
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Paulo Motta
>Priority: Major
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359946#comment-16359946
]
Paulo Motta commented on CASSANDRA-14227:
-
[~VincentWhite] feel free to add your PoC patch from CASSANDRA-14092. :)
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Paulo Motta
>Priority: Major
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
