[jira] [Updated] (IGNITE-17322) [ducktests] Add ability to configure more options for ignite nodes
[ https://issues.apache.org/jira/browse/IGNITE-17322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Korotkov updated IGNITE-17322: - Summary: [ducktests] Add ability to configure more options for ignite nodes (was: [ducktests] Add abilities to configure more options for server and client nodes) > [ducktests] Add ability to configure more options for ignite nodes > -- > > Key: IGNITE-17322 > URL: https://issues.apache.org/jira/browse/IGNITE-17322 > Project: Ignite > Issue Type: Test >Reporter: Sergey Korotkov >Assignee: Sergey Korotkov >Priority: Minor > Labels: ducktests > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-16760) Performance degradation of IgniteTables#tables after configuration changes
[
https://issues.apache.org/jira/browse/IGNITE-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563376#comment-17563376
]
Evgeny Stanilovsky edited comment on IGNITE-16760 at 7/7/22 5:03 AM:
-
[~Denis Chudov] tests with annotation
@Disabled("https://issues.apache.org/jira/browse/IGNITE-16760;) still present
in main, can you take a look ? Additionally i still observe degradation on
annotated test, is it another known issue ?
was (Author: zstan):
[~Denis Chudov] tests with annotation
@Disabled("https://issues.apache.org/jira/browse/IGNITE-16760;) still present
in main, can you take a look ?
> Performance degradation of IgniteTables#tables after configuration changes
> --
>
> Key: IGNITE-16760
> URL: https://issues.apache.org/jira/browse/IGNITE-16760
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha4
>Reporter: Taras Ledkov
>Assignee: Denis Chudov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Performance degradation of configuration changes:
> Steps to reproduce:
> 1. Start cluster with 3 nodes
> 2. Run in the loop
> {code}
> CREATE TABLE TEST(ID INTEGER PRIMARY KEY, V INTEGER)
> for (Table t : ign.tables().tables()) {
> ;
> }
> {code}
> On begin {{IgniteTables#tables}} takes ~ 0.7 sec.
> The time of the operation is grown.
> The time after ~100 iteration is about 20 sec.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17322) [ducktests] Add abilities to configure more options for server and client nodes
[ https://issues.apache.org/jira/browse/IGNITE-17322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Korotkov updated IGNITE-17322: - Labels: ducktests (was: ) > [ducktests] Add abilities to configure more options for server and client > nodes > --- > > Key: IGNITE-17322 > URL: https://issues.apache.org/jira/browse/IGNITE-17322 > Project: Ignite > Issue Type: Test >Reporter: Sergey Korotkov >Priority: Minor > Labels: ducktests > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-17322) [ducktests] Add abilities to configure more options for server and client nodes
[ https://issues.apache.org/jira/browse/IGNITE-17322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Korotkov reassigned IGNITE-17322: Assignee: Sergey Korotkov > [ducktests] Add abilities to configure more options for server and client > nodes > --- > > Key: IGNITE-17322 > URL: https://issues.apache.org/jira/browse/IGNITE-17322 > Project: Ignite > Issue Type: Test >Reporter: Sergey Korotkov >Assignee: Sergey Korotkov >Priority: Minor > Labels: ducktests > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17322) [ducktests] Add abilities to configure more options for server and client nodes
Sergey Korotkov created IGNITE-17322: Summary: [ducktests] Add abilities to configure more options for server and client nodes Key: IGNITE-17322 URL: https://issues.apache.org/jira/browse/IGNITE-17322 Project: Ignite Issue Type: Test Reporter: Sergey Korotkov -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-12852) Comma in field is not supported by COPY command
[ https://issues.apache.org/jira/browse/IGNITE-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YuJue Li reassigned IGNITE-12852: - Assignee: Anton Kurbanov (was: YuJue Li) > Comma in field is not supported by COPY command > --- > > Key: IGNITE-12852 > URL: https://issues.apache.org/jira/browse/IGNITE-12852 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 2.8 >Reporter: YuJue Li >Assignee: Anton Kurbanov >Priority: Critical > Fix For: 2.14 > > Time Spent: 50m > Remaining Estimate: 0h > > CREATE TABLE test(a int,b varchar(100),c int,PRIMARY key(a)); > > a.csv: > 1,"a,b",2 > > COPY FROM '/data/a.csv' INTO test (a,b,c) FORMAT CSV; > > The copy command fails because there is a comma in the second field,but this > is a fully legal and compliant CSV format -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-12852) Comma in field is not supported by COPY command
[ https://issues.apache.org/jira/browse/IGNITE-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563516#comment-17563516 ] YuJue Li commented on IGNITE-12852: --- ok. > Comma in field is not supported by COPY command > --- > > Key: IGNITE-12852 > URL: https://issues.apache.org/jira/browse/IGNITE-12852 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 2.8 >Reporter: YuJue Li >Assignee: YuJue Li >Priority: Critical > Fix For: 2.14 > > Time Spent: 50m > Remaining Estimate: 0h > > CREATE TABLE test(a int,b varchar(100),c int,PRIMARY key(a)); > > a.csv: > 1,"a,b",2 > > COPY FROM '/data/a.csv' INTO test (a,b,c) FORMAT CSV; > > The copy command fails because there is a comma in the second field,but this > is a fully legal and compliant CSV format -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17281) Return error groups from REST API
[ https://issues.apache.org/jira/browse/IGNITE-17281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-17281: - Labels: iep-84 iep-87 ignite-3 (was: iep-87 ignite-3) > Return error groups from REST API > - > > Key: IGNITE-17281 > URL: https://issues.apache.org/jira/browse/IGNITE-17281 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksandr >Assignee: Vyacheslav Koptilin >Priority: Major > Labels: iep-84, iep-87, ignite-3 > Time Spent: 10m > Remaining Estimate: 0h > > After IGNITE-14931 and IGNITE-17193 are done, traceId and human-readable > error code should be returned from any rest component. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-14986) Re-work error handling in meta storage component in accordance with error groups
[ https://issues.apache.org/jira/browse/IGNITE-14986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-14986: - Labels: iep-84 ignite-3 (was: ignite-3) > Re-work error handling in meta storage component in accordance with error > groups > > > Key: IGNITE-14986 > URL: https://issues.apache.org/jira/browse/IGNITE-14986 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Assignee: Vyacheslav Koptilin >Priority: Major > Labels: iep-84, ignite-3 > > Need to introduce a new error group related to Meta Storage Service and add > all needed error codes. > Also, the implementation should using _IgniteInternalException_ and > _IgniteInternalCheckedException_ with specific error codes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-14984) Re-work error handling in table component in accordance with error scopes and prefixes
[ https://issues.apache.org/jira/browse/IGNITE-14984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-14984: - Labels: iep-84 ignite-3 (was: ignite-3) > Re-work error handling in table component in accordance with error scopes and > prefixes > -- > > Key: IGNITE-14984 > URL: https://issues.apache.org/jira/browse/IGNITE-14984 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Priority: Major > Labels: iep-84, ignite-3 > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-14985) Re-work error handling in affinity component in accordance with error scopes and prefixes
[ https://issues.apache.org/jira/browse/IGNITE-14985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-14985: - Labels: iep-84 ignite-3 (was: ignite-3) > Re-work error handling in affinity component in accordance with error scopes > and prefixes > -- > > Key: IGNITE-14985 > URL: https://issues.apache.org/jira/browse/IGNITE-14985 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Priority: Major > Labels: iep-84, ignite-3 > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-14930) Exceptions serialization
[ https://issues.apache.org/jira/browse/IGNITE-14930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-14930: - Labels: iep-84 ignite-3 (was: ignite-3) > Exceptions serialization > > > Key: IGNITE-14930 > URL: https://issues.apache.org/jira/browse/IGNITE-14930 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Priority: Major > Labels: iep-84, ignite-3 > > Since there are other languages than Java, we should have a generic way to > convert exceptions between different representations. Current errors > serialization design is described here: [IEP-76 Thin Client Protocol for > Ignite > 3.0|https://cwiki.apache.org/confluence/display/IGNITE/IEP-76+Thin+Client+Protocol+for+Ignite+3.0] > We should expand the collection of required fields. It should be enough to > deserialize exception objects in Java client as close to the original object > as possible. This includes cause exceptions. > Not yet sure about suppressed exceptions. So, we would need a list of errors, > where the first element is the head and every next element is the cause for > the previous one. Element contains: > Error code (int) > Error message (String) > [Optional] Stack trace, for head only (String) > Full class name (String) > Open problem: > What should we do with standard Java exceptions, like _TimeoutException_ or > _IllegalArgumentException_ , or even _NullPointerException_? Right now it's > better to have a reserved error group for them and assign a specific codes to > all "known" types. > https://cwiki.apache.org/confluence/display/IGNITE/IEP-84%3A+Error+handling -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-13510) Getting status of snapshot execution via command line and jmx
[
https://issues.apache.org/jira/browse/IGNITE-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563432#comment-17563432
]
Sergei Ryzhov commented on IGNITE-13510:
[~xtern] Hi, it's good idea! Thanks!
Yes, i want to finish this patch
> Getting status of snapshot execution via command line and jmx
> -
>
> Key: IGNITE-13510
> URL: https://issues.apache.org/jira/browse/IGNITE-13510
> Project: Ignite
> Issue Type: Task
>Reporter: Sergei Ryzhov
>Assignee: Sergei Ryzhov
>Priority: Major
> Labels: iep-43, ise, snapshot
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> the control.sh utility immediately relinquishes control
> and without using metricExporter it is impossible to understand whether the
> snapshot completed or not
> Restoring
> {code:java}
> Control utility [ver. 2.12.0-SNAPSHOT#20211004-sha1:77de60a7]
> 2021 Copyright(C) Apache Software Foundation
> User: sega
> Time: 2021-10-07T14:18:59.523
> Command [SNAPSHOT] started
> Arguments: --snapshot status --yes
>
> Status of SNAPSHOT operations:
> gridCommandHandlerTest0 -> Restoring to snapshot with name: snapshot_02052020
> gridCommandHandlerTest1 -> Restoring to snapshot with name: snapshot_02052020
> Command [SNAPSHOT] finished with code: 0
> Control utility has completed execution at: 2021-10-07T14:18:59.546
> Execution time: 23 ms
> {code}
> Creating
> {code:java}
> Control utility [ver. 2.12.0-SNAPSHOT#20211004-sha1:77de60a7]
> 2021 Copyright(C) Apache Software Foundation
> User: sega
> Time: 2021-10-07T14:18:55.368
> Command [SNAPSHOT] started
> Arguments: --snapshot status --yes
>
> Status of SNAPSHOT operations:
> gridCommandHandlerTest0 -> Creating the snapshot with name: snapshot_02052020
> gridCommandHandlerTest1 -> Creating the snapshot with name: snapshot_02052020
> Command [SNAPSHOT] finished with code: 0
> Control utility has completed execution at: 2021-10-07T14:18:55.391
> Execution time: 23 ms
> {code}
> No snapshot operation
> {code:java}
> Control utility [ver. 2.12.0-SNAPSHOT#20211004-sha1:77de60a7]
> 2021 Copyright(C) Apache Software Foundation
> User: sega
> Time: 2021-10-07T14:18:58.408
> Command [SNAPSHOT] started
> Arguments: --snapshot status --yes
>
> Status of SNAPSHOT operations:
> gridCommandHandlerTest0 -> No snapshot operation.
> gridCommandHandlerTest1 -> No snapshot operation.
> Command [SNAPSHOT] finished with code: 0
> Control utility has completed execution at: 2021-10-07T14:18:58.439
> Execution time: 31 ms
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-15023) ClassNotFoundException when using security and trying to set a remote listener
[ https://issues.apache.org/jira/browse/IGNITE-15023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563429#comment-17563429 ] Sergei Ryzhov commented on IGNITE-15023: [~dpavlov] hi, now i have a bit free time and i don't know when i can do it but, i'm going to do it > ClassNotFoundException when using security and trying to set a remote listener > -- > > Key: IGNITE-15023 > URL: https://issues.apache.org/jira/browse/IGNITE-15023 > Project: Ignite > Issue Type: Bug >Reporter: Sergei Ryzhov >Assignee: Sergei Ryzhov >Priority: Minor > Labels: ise > Time Spent: 40m > Remaining Estimate: 0h > > ClassNotFoundException when using security and trying to set a remote listener > the issue in using SecurityAwarePredicate on a client node > it makes a wrapper over the remote listener so the class is not passed > through the Peer Class Loader > Caused by: java.lang.ClassNotFoundException: > examples.StartClientXml$$Lambda$703/253380088 > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:348) > at > org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:9064) > at > org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:9002) > at > org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:376) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.descriptorFromCache(OptimizedMarshallerUtils.java:329) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.classDescriptor(OptimizedMarshallerUtils.java:274) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readClass(OptimizedObjectInputStream.java:384) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:329) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:205) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:367) > at > org.apache.ignite.internal.SecurityAwarePredicate.readExternal(SecurityAwarePredicate.java:86) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:560) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:980) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:353) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:205) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:367) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:251) > ... 22 more -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-12852) Comma in field is not supported by COPY command
[ https://issues.apache.org/jira/browse/IGNITE-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563409#comment-17563409 ] Anton Kurbanov commented on IGNITE-12852: - [~liyuj] would it be ok for me to get this ticket assigned back to me? > Comma in field is not supported by COPY command > --- > > Key: IGNITE-12852 > URL: https://issues.apache.org/jira/browse/IGNITE-12852 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 2.8 >Reporter: YuJue Li >Assignee: YuJue Li >Priority: Critical > Fix For: 2.14 > > Time Spent: 50m > Remaining Estimate: 0h > > CREATE TABLE test(a int,b varchar(100),c int,PRIMARY key(a)); > > a.csv: > 1,"a,b",2 > > COPY FROM '/data/a.csv' INTO test (a,b,c) FORMAT CSV; > > The copy command fails because there is a comma in the second field,but this > is a fully legal and compliant CSV format -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-16760) Performance degradation of IgniteTables#tables after configuration changes
[
https://issues.apache.org/jira/browse/IGNITE-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563376#comment-17563376
]
Evgeny Stanilovsky commented on IGNITE-16760:
-
[~Denis Chudov] tests with annotation
@Disabled("https://issues.apache.org/jira/browse/IGNITE-16760;) still present
in main, can you take a look ?
> Performance degradation of IgniteTables#tables after configuration changes
> --
>
> Key: IGNITE-16760
> URL: https://issues.apache.org/jira/browse/IGNITE-16760
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha4
>Reporter: Taras Ledkov
>Assignee: Denis Chudov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Performance degradation of configuration changes:
> Steps to reproduce:
> 1. Start cluster with 3 nodes
> 2. Run in the loop
> {code}
> CREATE TABLE TEST(ID INTEGER PRIMARY KEY, V INTEGER)
> for (Table t : ign.tables().tables()) {
> ;
> }
> {code}
> On begin {{IgniteTables#tables}} takes ~ 0.7 sec.
> The time of the operation is grown.
> The time after ~100 iteration is about 20 sec.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17267) Preparing a checkpoint for a DeltaFilePageStore
[ https://issues.apache.org/jira/browse/IGNITE-17267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563352#comment-17563352 ] Roman Puchkovskiy commented on IGNITE-17267: The patch looks good to me, thank you for the contribution > Preparing a checkpoint for a DeltaFilePageStore > --- > > Key: IGNITE-17267 > URL: https://issues.apache.org/jira/browse/IGNITE-17267 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > Time Spent: 4h 50m > Remaining Estimate: 0h > > To implement the *DeltaFilePageStore*, we need to prepare a checkpoint for > their creation. > What should be done: > # Sort dirty pages (*FullPageId*) by *groupId*, *partitionId*, *pageIdx*; > # In several threads, write dirty pages in batches for each partition and > know the number of these pages. > What can we look at: > * > *org.apache.ignite.internal.pagememory.persistence.checkpoint.CheckpointWorkflow#beginCheckpoint* > * > *org.apache.ignite.internal.pagememory.persistence.checkpoint.CheckpointWorkflow#splitAndSortCheckpointPagesIfNeeded* > * > *org.apache.ignite.internal.pagememory.persistence.checkpoint.Checkpointer#writePages* > Some implementation notes: > * We can replace *CheckpointWorkflow#parallelSortThreshold* with a constant > at 40k, according to the > [paper|https://www.researchgate.net/publication/331742843_Threshold_Analysis_and_Comparison_of_Sequential_and_Parallel_Divide_and_Conquer_Sorting_Algorithms]. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17032) Apache Ignite Docker container does not run correctly if image is run in read only file system mode
[
https://issues.apache.org/jira/browse/IGNITE-17032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563331#comment-17563331
]
Petar Tonkovic commented on IGNITE-17032:
-
Hi [~slukyanov],
Thanks for your comment. Indeed, we have already discovered the workaround with
mounting the tmp folder in the yaml definition:
{code:java}
volumeMounts:
- name: temp-vol
mountPath: /tmp
readOnly: false{code}
However, it might be a nice improvement to have everything working out of the
box as we lost quite some time finding the workaround ourselves and it was not
quite obvious.
Regarding the writable disk, sure Ignite needs to write on disk, but again
usually you provide it the mount which is able to be written to do so, which is
hopefully not in the installation folder. :)
In many cases Kubernetes clusters provided by company infrastructure teams
enforce that the images are run with this read only file system flag as part of
their security requirements and it makes total sense:
# You prepare your installation file system pre-deployment with your Docker
file by building it in the required image;
# After said image is ran, there is no room for any kind of malicious software
doing any kind of injections in your pods.
My company has exactly this security policy enforced and I have struggled a lot
with various software that tries to write some temp files in the installation
folder or something similar when trying to deploy it in a container. They
either forget to let you set the log path somehow (usually via environment
variable) and then connect it to a volume mount that is read/write.
I can imagine that these security measures might be a common occurrence in the
industry, so could be a nice improvement. Thank you for looking into it.
P.S. We have noticed the same script being used in the web console agent (we
used the one from GridGain, not sure if it is the case with the vanilla one as
well), just FYI.
> Apache Ignite Docker container does not run correctly if image is run in read
> only file system mode
> ---
>
> Key: IGNITE-17032
> URL: https://issues.apache.org/jira/browse/IGNITE-17032
> Project: Ignite
> Issue Type: Bug
> Components: build
>Affects Versions: 2.13
>Reporter: Petar Tonkovic
>Priority: Major
>
> When following the Kubernetes deployment tutorials (online:
> https://ignite.apache.org/docs/latest/installation/kubernetes/azure-deployment,
> youtube: [https://youtu.be/38YgdAOs038]), trying to run the official docker
> image () with the --read-only flag is causing errors:
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 52: cannot create
> temp file for here-document: Read-only file system
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 85: [: -lt: unary
> operator expected2022-05-25T14:27:34.504369604+02:00
>
> Since most managed company Kubernetes clusters enforce this read-only flag as
> a security requirement, it would be good to look into these errors.
>
> Later on, we get the following error on starting up:
> class org.apache.ignite.IgniteException: Failed to instantiate Spring XML
> application context (make sure all classes used in Spring configuration are
> present at CLASSPATH) [springUrl=file:/ignite/config/node-configuration.xml]
> at
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1135)
> at org.apache.ignite.Ignition.start(Ignition.java:356)
> at
> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:365)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> instantiate Spring XML application context (make sure all classes used in
> Spring configuration are present at CLASSPATH)
> [springUrl=file:/ignite/config/node-configuration.xml]
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:364)
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:102)
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:96)
> at
> org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:729)2022-05-25T14:27:34.916588365+02:00
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:930)
> at
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:839)2022-05-25T14:27:34.916609431+02:00
> at
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:709)2022-05-25T14:27:34.916622089+02:00
> at
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:678)2022-05-25T14:27:34.916636146+02:00
> at
>
[jira] [Created] (IGNITE-17321) Document which api can work with partition awareness
Luchnikov Alexander created IGNITE-17321: Summary: Document which api can work with partition awareness Key: IGNITE-17321 URL: https://issues.apache.org/jira/browse/IGNITE-17321 Project: Ignite Issue Type: Improvement Components: thin client Reporter: Luchnikov Alexander In javadoc org.apache.ignite.configuration.ClientConfiguration#partitionAwarenessEnabled and in the description of functionality https://ignite.apache.org/docs/latest/thin-clients/java-thin-client#partition-awareness, it is not described with which api this functionality will work and in what cases. For example, will it work with getAll, in a transaction? Describe in the documentation and in the javadoc in which cases it works and with which api. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads
[
https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17261:
-
Description:
Because of lock-free nature, RO reads might interact with writeIntents, meaning
that such intents should be either evaluated as committed, aborted or pending.
In order to perform writeIntent resolution it's required to check txnState
along with adjusting HLC timestamp on txnStates group. If transaction occurred
to be
* pending (there is no explicit txnState for given txnId specified in
writeIntent) or txnState is ABORTED - writeIntent ignored.
* COMMITTED - writeInent returned as RO read response.
Please pay attention that, as was mentioned above, besides writeIntent
resolution itself, it's required to adjust HLC timestamp on txnState partition
in order to guarantee RO reads and tx commit linearizability, meaning that it's
illegal to commit transaction with commitTimestamp less or equal to RO
timestamp that already resolved corresponding writeInent as pending. It's not
only required to perform timestamp adjustment but it's also important to do it
atomically to txnState switch.
There's an open question about following tx design feature:
{code:java}
If the state is NULL, this means txn is not yet started to finish and its
outcome is not known. Send TxStateReq(txId,txnReadTs) to the txCoord to trigger
the coordinator path.{code}
Seems that it's not required, let's discuss it explicitly.
was:
Because of lock-free nature, RO reads might interact with writeIntents, meaning
that such intents should be either evaluated as committed, aborted or pending.
In order to perform writeIntent resolution it's required to check
[txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with
[adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on
txnStates group. If transaction occurred to be
* pending (there is no explicit txnState for given txnId specified in
writeIntent) or txnState is ABORTED - writeIntent ignored.
* COMMITTED - writeInent returned as RO read response.
Please pay attention that, as was mentioned above, besides writeIntent
resolution itself, it's required to adjust HLC timestamp on txnState partition
in order to guarantee RO reads and tx commit linearizability, meaning that it's
illegal to commit transaction with commitTimestamp less or equal to RO
timestamp that already resolved corresponding writeInent as pending. It's not
only required to perform timestamp adjustment but it's also important to do it
atomically to txnState switch.
> Implement write intent resolution logic for RO reads
>
>
> Key: IGNITE-17261
> URL: https://issues.apache.org/jira/browse/IGNITE-17261
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: ignite-3, transaction3_ro
> Attachments: Screenshot from 2022-07-06 17-00-26.png
>
>
> Because of lock-free nature, RO reads might interact with writeIntents,
> meaning that such intents should be either evaluated as committed, aborted or
> pending. In order to perform writeIntent resolution it's required to check
> txnState along with adjusting HLC timestamp on txnStates group. If
> transaction occurred to be
> * pending (there is no explicit txnState for given txnId specified in
> writeIntent) or txnState is ABORTED - writeIntent ignored.
> * COMMITTED - writeInent returned as RO read response.
> Please pay attention that, as was mentioned above, besides writeIntent
> resolution itself, it's required to adjust HLC timestamp on txnState
> partition in order to guarantee RO reads and tx commit linearizability,
> meaning that it's illegal to commit transaction with commitTimestamp less or
> equal to RO timestamp that already resolved corresponding writeInent as
> pending. It's not only required to perform timestamp adjustment but it's also
> important to do it atomically to txnState switch.
> There's an open question about following tx design feature:
> {code:java}
> If the state is NULL, this means txn is not yet started to finish and its
> outcome is not known. Send TxStateReq(txId,txnReadTs) to the txCoord to
> trigger the coordinator path.{code}
> Seems that it's not required, let's discuss it explicitly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads
[ https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17261: - Attachment: Screenshot from 2022-07-06 17-00-26.png > Implement write intent resolution logic for RO reads > > > Key: IGNITE-17261 > URL: https://issues.apache.org/jira/browse/IGNITE-17261 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > Attachments: Screenshot from 2022-07-06 17-00-26.png > > > Because of lock-free nature, RO reads might interact with writeIntents, > meaning that such intents should be either evaluated as committed, aborted or > pending. In order to perform writeIntent resolution it's required to check > [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with > [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] > on txnStates group. If transaction occurred to be > * pending (there is no explicit txnState for given txnId specified in > writeIntent) or txnState is ABORTED - writeIntent ignored. > * COMMITTED - writeInent returned as RO read response. > Please pay attention that, as was mentioned above, besides writeIntent > resolution itself, it's required to adjust HLC timestamp on txnState > partition in order to guarantee RO reads and tx commit linearizability, > meaning that it's illegal to commit transaction with commitTimestamp less or > equal to RO timestamp that already resolved corresponding writeInent as > pending. It's not only required to perform timestamp adjustment but it's also > important to do it atomically to txnState switch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17263) Implement leader to replica safe time propagation
[ https://issues.apache.org/jira/browse/IGNITE-17263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17263: - Description: In order to perform replica reads, it's required either to use read index or check the safe time. Let's recall corresponding section from tx design document. RO transactions can be executed on non-primary replicas. write intent resolution doesn’t help because a write intent for a committed transaction may not be yet replicated to the replica. To mitigate this issue, it’s enough to run readIndex on each mapped partition leader, fetch the commit index and wait on a replica until it’s applied. This will guarantee that all required write intents are replicated and present locally. After that the normal write intern resolution should do the job. There is a second option, which doesn’t require the network RTT. We can use a special low watermark timestamp (safeTs) per replication group, which corresponds to the apply index of a replicated entry, so then an apply index is advanced during the replication, then the safeTs is monotonically incremented too. The HLC used for safeTs advancing is assigned to a replicated entry in an ordered way. Special measures are needed to periodically advance the safeTs if no updates are happening. It’s enough to use a special replication command for this purpose. All we need during RO txn is to wait until a safeTs advances past the RO txn readTs. !Screenshot from 2022-07-06 16-48-30.png! In the picture we have two concurrent transactions mapped to the same partition: T1 and T2. OpReq(w1(x)) and OpReq(w2(x)) are received concurrently. Each write intent is assigned a timestamp in a monotonic order consistent with the replication order. This can be for example done when replication entries are dequeued for processing by replication protocol (we assume entries are replicated successively. It’s not enough only to wait for safeTs - it may never happen due to absence of activity in the partition. Consider the next diagram: !Screenshot from 2022-07-06 16-48-41.png! We need an additional safeTsSync command to propagate a safeTs event in case there are no updates in the partition. Actually, it seems that it's possible to reuse common raft messages such as heartbeatRequests, vote/prevoteRequests together with appendEntriesRequests in order to propagate safeTime from leader to replicas. As was mentioned in [IGNITE-17261|https://issues.apache.org/jira/browse/IGNITE-17261] txnState switch should be linearized with all safe-time propagation requests. was: In order to perform replica reads, it's required either to use read index or check the safe time. Let's recall corresponding section from tx design document. RO transactions can be executed on non-primary replicas. write intent resolution doesn’t help because a write intent for a committed transaction may not be yet replicated to the replica. To mitigate this issue, it’s enough to run readIndex on each mapped partition leader, fetch the commit index and wait on a replica until it’s applied. This will guarantee that all required write intents are replicated and present locally. After that the normal write intern resolution should do the job. There is a second option, which doesn’t require the network RTT. We can use a special low watermark timestamp (safeTs) per replication group, which corresponds to the apply index of a replicated entry, so then an apply index is advanced during the replication, then the safeTs is monotonically incremented too. The HLC used for safeTs advancing is assigned to a replicated entry in an ordered way. Special measures are needed to periodically advance the safeTs if no updates are happening. It’s enough to use a special replication command for this purpose. All we need during RO txn is to wait until a safeTs advances past the RO txn readTs. !Screenshot from 2022-07-06 16-48-30.png! > Implement leader to replica safe time propagation > - > > Key: IGNITE-17263 > URL: https://issues.apache.org/jira/browse/IGNITE-17263 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > Attachments: Screenshot from 2022-07-06 16-48-30.png, Screenshot from > 2022-07-06 16-48-41.png > > > In order to perform replica reads, it's required either to use read index or > check the safe time. Let's recall corresponding section from tx design > document. > RO transactions can be executed on non-primary replicas. write intent > resolution doesn’t help because a write intent for a committed transaction > may not be yet replicated to the replica. To mitigate this issue, it’s enough > to run readIndex on each mapped partition leader, fetch the
[jira] [Updated] (IGNITE-17263) Implement leader to replica safe time propagation
[ https://issues.apache.org/jira/browse/IGNITE-17263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17263: - Attachment: Screenshot from 2022-07-06 16-48-41.png > Implement leader to replica safe time propagation > - > > Key: IGNITE-17263 > URL: https://issues.apache.org/jira/browse/IGNITE-17263 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > Attachments: Screenshot from 2022-07-06 16-48-30.png, Screenshot from > 2022-07-06 16-48-41.png > > > In order to perform replica reads, it's required either to use read index or > check the safe time. Let's recall corresponding section from tx design > document. > RO transactions can be executed on non-primary replicas. write intent > resolution doesn’t help because a write intent for a committed transaction > may not be yet replicated to the replica. To mitigate this issue, it’s enough > to run readIndex on each mapped partition leader, fetch the commit index and > wait on a replica until it’s applied. This will guarantee that all required > write intents are replicated and present locally. After that the normal write > intern resolution should do the job. > There is a second option, which doesn’t require the network RTT. We can use a > special low watermark timestamp (safeTs) per replication group, which > corresponds to the apply index of a replicated entry, so then an apply index > is advanced during the replication, then the safeTs is monotonically > incremented too. The HLC used for safeTs advancing is assigned to a > replicated entry in an ordered way. > Special measures are needed to periodically advance the safeTs if no updates > are happening. It’s enough to use a special replication command for this > purpose. > All we need during RO txn is to wait until a safeTs advances past the RO txn > readTs. > !Screenshot from 2022-07-06 16-48-30.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17263) Implement leader to replica safe time propagation
[ https://issues.apache.org/jira/browse/IGNITE-17263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17263: - Description: In order to perform replica reads, it's required either to use read index or check the safe time. Let's recall corresponding section from tx design document. RO transactions can be executed on non-primary replicas. write intent resolution doesn’t help because a write intent for a committed transaction may not be yet replicated to the replica. To mitigate this issue, it’s enough to run readIndex on each mapped partition leader, fetch the commit index and wait on a replica until it’s applied. This will guarantee that all required write intents are replicated and present locally. After that the normal write intern resolution should do the job. There is a second option, which doesn’t require the network RTT. We can use a special low watermark timestamp (safeTs) per replication group, which corresponds to the apply index of a replicated entry, so then an apply index is advanced during the replication, then the safeTs is monotonically incremented too. The HLC used for safeTs advancing is assigned to a replicated entry in an ordered way. Special measures are needed to periodically advance the safeTs if no updates are happening. It’s enough to use a special replication command for this purpose. All we need during RO txn is to wait until a safeTs advances past the RO txn readTs. !Screenshot from 2022-07-06 16-48-30.png! > Implement leader to replica safe time propagation > - > > Key: IGNITE-17263 > URL: https://issues.apache.org/jira/browse/IGNITE-17263 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > Attachments: Screenshot from 2022-07-06 16-48-30.png, Screenshot from > 2022-07-06 16-48-41.png > > > In order to perform replica reads, it's required either to use read index or > check the safe time. Let's recall corresponding section from tx design > document. > RO transactions can be executed on non-primary replicas. write intent > resolution doesn’t help because a write intent for a committed transaction > may not be yet replicated to the replica. To mitigate this issue, it’s enough > to run readIndex on each mapped partition leader, fetch the commit index and > wait on a replica until it’s applied. This will guarantee that all required > write intents are replicated and present locally. After that the normal write > intern resolution should do the job. > There is a second option, which doesn’t require the network RTT. We can use a > special low watermark timestamp (safeTs) per replication group, which > corresponds to the apply index of a replicated entry, so then an apply index > is advanced during the replication, then the safeTs is monotonically > incremented too. The HLC used for safeTs advancing is assigned to a > replicated entry in an ordered way. > Special measures are needed to periodically advance the safeTs if no updates > are happening. It’s enough to use a special replication command for this > purpose. > All we need during RO txn is to wait until a safeTs advances past the RO txn > readTs. > !Screenshot from 2022-07-06 16-48-30.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17263) Implement leader to replica safe time propagation
[ https://issues.apache.org/jira/browse/IGNITE-17263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17263: - Attachment: Screenshot from 2022-07-06 16-48-30.png > Implement leader to replica safe time propagation > - > > Key: IGNITE-17263 > URL: https://issues.apache.org/jira/browse/IGNITE-17263 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > Attachments: Screenshot from 2022-07-06 16-48-30.png, Screenshot from > 2022-07-06 16-48-41.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads
[ https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17261: - Description: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITTED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself, it's required to adjust HLC timestamp on txnState partition in order to guarantee RO reads and tx commit linearizability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timestamp that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. was: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITTED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself, it's required to adjust HLC timestamp on txnState partition in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timestamp that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. > Implement write intent resolution logic for RO reads > > > Key: IGNITE-17261 > URL: https://issues.apache.org/jira/browse/IGNITE-17261 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > > Because of lock-free nature, RO reads might interact with writeIntents, > meaning that such intents should be either evaluated as committed, aborted or > pending. In order to perform writeIntent resolution it's required to check > [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with > [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] > on txnStates group. If transaction occurred to be > * pending (there is no explicit txnState for given txnId specified in > writeIntent) or txnState is ABORTED - writeIntent ignored. > * COMMITTED - writeInent returned as RO read response. > Please pay attention that, as was mentioned above, besides writeIntent > resolution itself, it's required to adjust HLC timestamp on txnState > partition in order to guarantee RO reads and tx commit linearizability, > meaning that it's illegal to commit transaction with commitTimestamp less or > equal to RO timestamp that already resolved corresponding writeInent as > pending. It's not only required to perform timestamp adjustment but it's also > important to do it atomically to txnState switch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads
[ https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17261: - Description: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself it's required to adjust HLC timestamp on txnState partitiion in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timesmat that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. was: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself it's required to adjust HLC timestamp on txnState partitiion in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timesmat that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. > Implement write intent resolution logic for RO reads > > > Key: IGNITE-17261 > URL: https://issues.apache.org/jira/browse/IGNITE-17261 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > > Because of lock-free nature, RO reads might interact with writeIntents, > meaning that such intents should be either evaluated as committed, aborted or > pending. In order to perform writeIntent resolution it's required to check > [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with > [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] > on txnStates group. If transaction occurred to be > * pending (there is no explicit txnState for given txnId specified in > writeIntent) or txnState is ABORTED - writeIntent ignored. > * COMMITED - writeInent returned as RO read response. > Please pay attention that, as was mentioned above, besides writeIntent > resolution itself it's required to adjust HLC timestamp on txnState > partitiion in order to guarantee RO reads and tx commit inevitability, > meaning that it's illegal to commit transaction with commitTimestamp less or > equal to RO timesmat that already resolved corresponding writeInent as > pending. It's not only required to perform timestamp adjustment but it's also > important to do it atomically to txnState switch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads
[ https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17261: - Description: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITTED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself, it's required to adjust HLC timestamp on txnState partition in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timestamp that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. was: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself, it's required to adjust HLC timestamp on txnState partitiion in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timesmat that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. > Implement write intent resolution logic for RO reads > > > Key: IGNITE-17261 > URL: https://issues.apache.org/jira/browse/IGNITE-17261 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > > Because of lock-free nature, RO reads might interact with writeIntents, > meaning that such intents should be either evaluated as committed, aborted or > pending. In order to perform writeIntent resolution it's required to check > [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with > [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] > on txnStates group. If transaction occurred to be > * pending (there is no explicit txnState for given txnId specified in > writeIntent) or txnState is ABORTED - writeIntent ignored. > * COMMITTED - writeInent returned as RO read response. > Please pay attention that, as was mentioned above, besides writeIntent > resolution itself, it's required to adjust HLC timestamp on txnState > partition in order to guarantee RO reads and tx commit inevitability, meaning > that it's illegal to commit transaction with commitTimestamp less or equal to > RO timestamp that already resolved corresponding writeInent as pending. It's > not only required to perform timestamp adjustment but it's also important to > do it atomically to txnState switch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads
[ https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17261: - Description: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself, it's required to adjust HLC timestamp on txnState partitiion in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timesmat that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. was: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself it's required to adjust HLC timestamp on txnState partitiion in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timesmat that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. > Implement write intent resolution logic for RO reads > > > Key: IGNITE-17261 > URL: https://issues.apache.org/jira/browse/IGNITE-17261 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > > Because of lock-free nature, RO reads might interact with writeIntents, > meaning that such intents should be either evaluated as committed, aborted or > pending. In order to perform writeIntent resolution it's required to check > [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with > [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] > on txnStates group. If transaction occurred to be > * pending (there is no explicit txnState for given txnId specified in > writeIntent) or txnState is ABORTED - writeIntent ignored. > * COMMITED - writeInent returned as RO read response. > Please pay attention that, as was mentioned above, besides writeIntent > resolution itself, it's required to adjust HLC timestamp on txnState > partitiion in order to guarantee RO reads and tx commit inevitability, > meaning that it's illegal to commit transaction with commitTimestamp less or > equal to RO timesmat that already resolved corresponding writeInent as > pending. It's not only required to perform timestamp adjustment but it's also > important to do it atomically to txnState switch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads
[ https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17261: - Description: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself it's required to adjust HLC timestamp on txnState partitiion in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timesmat that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. was: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check txnState along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself it's required to adjust HLC timestamp on txnState partitiion in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timesmat that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. > Implement write intent resolution logic for RO reads > > > Key: IGNITE-17261 > URL: https://issues.apache.org/jira/browse/IGNITE-17261 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > > Because of lock-free nature, RO reads might interact with writeIntents, > meaning that such intents should be either evaluated as committed, aborted or > pending. In order to perform writeIntent resolution it's required to check > [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with > [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] > on txnStates group. If transaction occurred to be > * pending (there is no explicit txnState for given txnId specified in > writeIntent) or txnState is ABORTED - writeIntent ignored. > * COMMITED - writeInent returned as RO read response. > Please pay attention that, as was mentioned above, besides writeIntent > resolution itself it's required to adjust HLC timestamp on txnState > partitiion in order to guarantee RO reads and tx commit inevitability, > meaning that it's illegal to commit transaction with commitTimestamp less or > equal to RO timesmat that already resolved corresponding writeInent as > pending. It's not only required to perform timestamp adjustment but it's also > important to do it atomically to txnState switch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads
[ https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17261: - Description: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check txnState along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself it's required to adjust HLC timestamp on txnState partitiion in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timesmat that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. was: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check [txnState|http://example.com] along with adjusting HLC timestamp on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself it's required to adjust HLC timestamp on txnState partitiion in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timesmat that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. > Implement write intent resolution logic for RO reads > > > Key: IGNITE-17261 > URL: https://issues.apache.org/jira/browse/IGNITE-17261 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > > Because of lock-free nature, RO reads might interact with writeIntents, > meaning that such intents should be either evaluated as committed, aborted or > pending. In order to perform writeIntent resolution it's required to check > txnState along with [adjusting HLC > timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates > group. If transaction occurred to be > * pending (there is no explicit txnState for given txnId specified in > writeIntent) or txnState is ABORTED - writeIntent ignored. > * COMMITED - writeInent returned as RO read response. > Please pay attention that, as was mentioned above, besides writeIntent > resolution itself it's required to adjust HLC timestamp on txnState > partitiion in order to guarantee RO reads and tx commit inevitability, > meaning that it's illegal to commit transaction with commitTimestamp less or > equal to RO timesmat that already resolved corresponding writeInent as > pending. It's not only required to perform timestamp adjustment but it's also > important to do it atomically to txnState switch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads
[ https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17261: - Description: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check [txnState|http://example.com] along with adjusting HLC timestamp on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself it's required to adjust HLC timestamp on txnState partitiion in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timesmat that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. was: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check txnState along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself it's required to adjust HLC timestamp on txnState partitiion in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timesmat that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. > Implement write intent resolution logic for RO reads > > > Key: IGNITE-17261 > URL: https://issues.apache.org/jira/browse/IGNITE-17261 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > > Because of lock-free nature, RO reads might interact with writeIntents, > meaning that such intents should be either evaluated as committed, aborted or > pending. In order to perform writeIntent resolution it's required to check > [txnState|http://example.com] along with adjusting HLC timestamp on txnStates > group. If transaction occurred to be > * pending (there is no explicit txnState for given txnId specified in > writeIntent) or txnState is ABORTED - writeIntent ignored. > * COMMITED - writeInent returned as RO read response. > Please pay attention that, as was mentioned above, besides writeIntent > resolution itself it's required to adjust HLC timestamp on txnState > partitiion in order to guarantee RO reads and tx commit inevitability, > meaning that it's illegal to commit transaction with commitTimestamp less or > equal to RO timesmat that already resolved corresponding writeInent as > pending. It's not only required to perform timestamp adjustment but it's also > important to do it atomically to txnState switch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads
[ https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17261: - Description: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check txnState along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself it's required to adjust HLC timestamp on txnState partitiion in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timesmat that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. was: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check txnState along with adjusting HLC timestamp on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself it's required to adjust HLC timestamp on txnState partitiion in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timesmat that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. > Implement write intent resolution logic for RO reads > > > Key: IGNITE-17261 > URL: https://issues.apache.org/jira/browse/IGNITE-17261 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > > Because of lock-free nature, RO reads might interact with writeIntents, > meaning that such intents should be either evaluated as committed, aborted or > pending. In order to perform writeIntent resolution it's required to check > txnState along with [adjusting HLC > timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates > group. If transaction occurred to be > * pending (there is no explicit txnState for given txnId specified in > writeIntent) or txnState is ABORTED - writeIntent ignored. > * COMMITED - writeInent returned as RO read response. > Please pay attention that, as was mentioned above, besides writeIntent > resolution itself it's required to adjust HLC timestamp on txnState > partitiion in order to guarantee RO reads and tx commit inevitability, > meaning that it's illegal to commit transaction with commitTimestamp less or > equal to RO timesmat that already resolved corresponding writeInent as > pending. It's not only required to perform timestamp adjustment but it's also > important to do it atomically to txnState switch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads
[ https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17261: - Description: Because of lock-free nature, RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check txnState along with adjusting HLC timestamp on txnStates group. If transaction occurred to be * pending (there is no explicit txnState for given txnId specified in writeIntent) or txnState is ABORTED - writeIntent ignored. * COMMITED - writeInent returned as RO read response. Please pay attention that, as was mentioned above, besides writeIntent resolution itself it's required to adjust HLC timestamp on txnState partitiion in order to guarantee RO reads and tx commit inevitability, meaning that it's illegal to commit transaction with commitTimestamp less or equal to RO timesmat that already resolved corresponding writeInent as pending. It's not only required to perform timestamp adjustment but it's also important to do it atomically to txnState switch. was:Because of lock-free nature RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check txnState along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. > Implement write intent resolution logic for RO reads > > > Key: IGNITE-17261 > URL: https://issues.apache.org/jira/browse/IGNITE-17261 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > > Because of lock-free nature, RO reads might interact with writeIntents, > meaning that such intents should be either evaluated as committed, aborted or > pending. In order to perform writeIntent resolution it's required to check > txnState along with adjusting HLC timestamp on txnStates group. If > transaction occurred to be > * pending (there is no explicit txnState for given txnId specified in > writeIntent) or txnState is ABORTED - writeIntent ignored. > * COMMITED - writeInent returned as RO read response. > Please pay attention that, as was mentioned above, besides writeIntent > resolution itself it's required to adjust HLC timestamp on txnState > partitiion in order to guarantee RO reads and tx commit inevitability, > meaning that it's illegal to commit transaction with commitTimestamp less or > equal to RO timesmat that already resolved corresponding writeInent as > pending. It's not only required to perform timestamp adjustment but it's also > important to do it atomically to txnState switch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads
[ https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17261: - Description: Because of lock-free nature RO reads might interact with writeIntents, meaning that such intents should be either evaluated as committed, aborted or pending. In order to perform writeIntent resolution it's required to check txnState along with [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates group. > Implement write intent resolution logic for RO reads > > > Key: IGNITE-17261 > URL: https://issues.apache.org/jira/browse/IGNITE-17261 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_ro > > Because of lock-free nature RO reads might interact with writeIntents, > meaning that such intents should be either evaluated as committed, aborted or > pending. In order to perform writeIntent resolution it's required to check > txnState along with [adjusting HLC > timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates > group. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17311) KafkaToIgniteLoader instantiates several Spring contexts
[ https://issues.apache.org/jira/browse/IGNITE-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-17311: - Labels: IEP-59 (was: ) > KafkaToIgniteLoader instantiates several Spring contexts > > > Key: IGNITE-17311 > URL: https://issues.apache.org/jira/browse/IGNITE-17311 > Project: Ignite > Issue Type: Improvement > Components: extensions >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Major > Labels: IEP-59 > Fix For: 2.14 > > Time Spent: 10m > Remaining Estimate: 0h > > KafkaToIgniteLoader instantiates two SpringApplicationContext during app > start. > This can lead to error in case other beans inside context not designed for > this case. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16650) Exclude ignite-log4j, log4j 1.2.17
[
https://issues.apache.org/jira/browse/IGNITE-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Petrov updated IGNITE-16650:
Description:
log4j 1.2.17 is not supported and contains critical vulnerabilities
https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
I suggest excluding the ignite-log4j module from ignite
Direct vulnerabilities:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571
As a result of the mentioned migration, the following changes will be applied:
1. ignite-log4j.xml will be migrated to log4j2 format. Unfortunately after the
refactoring we will get two configuration ignite-log4j.xml and
ignite-log4j2.xml both in log4j2 format because ignite-log4j2.xml is in use now
and but provide log formatitng different from ignite-log4j.xml.
2. core/src/test/config/log4j-test.xml will not be migrated to log4j2 because
it is used with compatibility tests.
3. core/src/test/config/log4j2-test.xml is refactored to suite current log4j
format. The current version of core/src/test/config/log4j2-test.xml is moved
to the log4j2/src/test/config folder.
4. osgi-paxlogging will be removed because it's only meant to provide some
log4j dependencies. We have no need in them now.
5. Exception logging format will change slightly:
Before:
{code:java}
class org.apache.ignite.IgniteException: Platform error:System.Exception:
EXCEPTION_TEST_Warn
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
at
org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
{code}
After:
{code:java}
org.apache.ignite.IgniteException: Platform error:System.Exception:
EXCEPTION_TEST_Warn
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
at
org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
{code}
As you can see, only the first word "class" is omitted.
6. All other files containing log4j configuration will be refactored to suite
log4j2 and will be renamed if previously their name allowed log4j to
automatically find them in the class path (e.g. log4j.xml -> log4j2.xml and so
on)
was:
log4j 1.2.17 is not supported and contains critical vulnerabilities
https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
I suggest excluding the ignite-log4j module from ignite
Direct vulnerabilities:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571
As a result of the mentioned migration, the following changes will be applied:
1. ignite-log4j.xml will be migrated to log4j2 format. Unfortunately after the
refactoring we will get two configuration ignite-log4j.xml and
ignite-log4j2.xml both in log4j2 format because the main goal of this
refactoring is to keep current log formatting that is used with log4j intact.
Currently ignite-log4j.xml and ignite-log4j2.xml provides different log formats
for log4j and log4j2 respectively.
2. core/src/test/config/log4j-test.xml will not be migrated to log4j2 because
it is used with compatibility tests.
3. core/src/test/config/log4j2-test.xml is refactored to suite current log4j
format. The current version of core/src/test/config/log4j2-test.xml is moved
to the log4j2/src/test/config folder.
4. osgi-paxlogging will be removed because it's only meant to provide some
log4j dependencies. We have no need in them now.
5. Exception logging format will change slightly:
Before:
{code:java}
class org.apache.ignite.IgniteException: Platform error:System.Exception:
EXCEPTION_TEST_Warn
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
at
[jira] [Created] (IGNITE-17320) Implement B+Tree based sorted index storage
Ivan Bessonov created IGNITE-17320: -- Summary: Implement B+Tree based sorted index storage Key: IGNITE-17320 URL: https://issues.apache.org/jira/browse/IGNITE-17320 Project: Ignite Issue Type: Improvement Reporter: Ivan Bessonov Like in IGNITE-17318, binary tuple format is a main goal here, because that's what we're gonna pass into API. Implementing a tree for indexes isn't hard by itself. I expect it to be stored in same partition files as raw storage, everything should be colocated in a single file. What's complicated is the inlining of data into trees pages. I see it as the following: * If tuple has no offset table, we can always store the entire payload in the tree. This is the best case scenario, because the size is fixed and known a priori, we don't even need to store it before the payload. * If tuple has an offset table, the inline size will immediately get bigger. In my view, we will have to: ** store the size of inlined payload, that's 4 bytes ** store null table if it's there, that's a known amount of bytes ** store header and offset table: *** if there are non-fixed-length columns, then a single entry in offset table can be up to 4 bytes *** if there are only fixed-length columns, like ints, floats or even bitsets, the amount of bytes per single entry can be accurately estimated with the upper bound ** then store a good amount of the actual columns data. How much? I'd be generous, but then we would probably have too much space, so all of this is debatable: *** for columns with fixed size, allocate room for entire value *** for strings and numbers (is there something else?) we have to pre-allocate a reasonable amount of bytes. Like, 8, for example. I don't know, there are defaults somewhere in the code of Ignite 2.x, we can use them. So, my point is, there's no new data format for inlined section of the tuple, we should reuse it and thus avoid many possible errors. And, if record fits into a tree page, there's no need in inserting it into a free list. Good! And yes, of course, there has to be a room for row id in inlined section as well. Last, meta tree is still a thing. But, we shouldn't identify indexes by their names, cause there's UUID id or even integer id (see IGNITE-17318). h3. Other ideas I don't like how durable background tasks work in Ignite 2.x, there are always some issues. I would prefer having a general-purposed "recycle bin" in partition and a background cleaner process that would clean it. Maybe this queue should contain other entities in the future. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17319) Remove entries replicated to all nodes from RAFT log in in-memory scenario
Roman Puchkovskiy created IGNITE-17319: -- Summary: Remove entries replicated to all nodes from RAFT log in in-memory scenario Key: IGNITE-17319 URL: https://issues.apache.org/jira/browse/IGNITE-17319 Project: Ignite Issue Type: Improvement Components: persistence Reporter: Roman Puchkovskiy Assignee: Roman Puchkovskiy Fix For: 3.0.0-alpha6 TBD -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16650) Exclude ignite-log4j, log4j 1.2.17
[
https://issues.apache.org/jira/browse/IGNITE-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Petrov updated IGNITE-16650:
Description:
log4j 1.2.17 is not supported and contains critical vulnerabilities
https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
I suggest excluding the ignite-log4j module from ignite
Direct vulnerabilities:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571
As a result of the mentioned migration, the following changes will be applied:
1. ignite-log4j.xml will be migrated to log4j2 format. Unfortunately after the
refactoring we will get two configuration ignite-log4j.xml and
ignite-log4j2.xml both in log4j2 format because the main goal of this
refactoring is to keep current log formatting that is used with log4j intact.
Currently ignite-log4j.xml and ignite-log4j2.xml provides different log formats
for log4j and log4j2 respectively.
2. core/src/test/config/log4j-test.xml will not be migrated to log4j2 because
it is used with compatibility tests.
3. core/src/test/config/log4j2-test.xml is refactored to suite current log4j
format. The current version of core/src/test/config/log4j2-test.xml is moved
to the log4j2/src/test/config folder.
4. osgi-paxlogging will be removed because it's only meant to provide some
log4j dependencies. We have no need in them now.
5. Exception logging format will change slightly:
Before:
{code:java}
class org.apache.ignite.IgniteException: Platform error:System.Exception:
EXCEPTION_TEST_Warn
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
at
org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
{code}
After:
{code:java}
org.apache.ignite.IgniteException: Platform error:System.Exception:
EXCEPTION_TEST_Warn
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
at
org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
{code}
As you can see, only the first word "class" is omitted.
6. All other files containing log4j configuration will be refactored to suite
log4j2 and will be renamed if previously their name allowed log4j to
automatically find them in the class path (e.g. log4j.xml -> log4j2.xml and so
on)
was:
log4j 1.2.17 is not supported and contains critical vulnerabilities
https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
I suggest excluding the ignite-log4j module from ignite
Direct vulnerabilities:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571
As a result of the mentioned migration, the following changes will be applied:
1. ignite-log4j.xml will be migrated to log4j2 format. Unfortunately after the
refactoring we will get two configuration ignite-log4j.xml and
ignite-log4j2.xml both in log4j2 format because the main goal of this
refactoring is to keep current log formatting that is used with log4j intact.
Currently ignite-log4j.xml and ignite-log4j2.xml provides different log formats
for log4j and log4j2 respectively.
2. core/src/test/config/log4j-test.xml will not be migrated to log4j2 because
it is used in compatibility tests.
3. core/src/test/config/log4j2-test.xml is refactored to suite current log4j
format. The current version of core/src/test/config/log4j2-test.xml is moved
to the log4j2/src/test/config folder/
4. osgi-paxlogging will be removed because it's only meant to provide some
log4j dependencies.
5. Exception logging format will change slightly:
Before:
{code:java}
class org.apache.ignite.IgniteException: Platform error:System.Exception:
EXCEPTION_TEST_Warn
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at
[jira] [Updated] (IGNITE-17318) Implement RocksDB based sorted index storage
[ https://issues.apache.org/jira/browse/IGNITE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Bessonov updated IGNITE-17318: --- Labels: ignite-3 (was: ) > Implement RocksDB based sorted index storage > > > Key: IGNITE-17318 > URL: https://issues.apache.org/jira/browse/IGNITE-17318 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Priority: Major > Labels: ignite-3 > > Pretty straightforward. Complicated places are: > * Binary tuples comparator: should be as fast as possible. Some > optimizations might be moved to other issues. > * Thorough testing is required. We have both Java and native comparators > planned. They should behave identically. This means a specific way of writing > tests, to account for this in advance. > * Bounds checking on range scan: > by default, comparator should include the lower bound and exclude the upper > bound. This is how prefix search works. This means that exclusion of the > lower bound (if needed) and inclusion of the upper bound (if needed) +must be > performed manually+ inside of the scan method. > The question is, do we use separate column families for indexes? At one hand, > this increases the number of files and potentially even increases time of the > flush, but on the other, it looks easy (or is it). > Currently, every index has only a UUID id. Just like for tables, we could > create an integer identifier, because why not. This way we could store all > indexes in a single column family without too much overhead. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16650) Exclude ignite-log4j, log4j 1.2.17
[
https://issues.apache.org/jira/browse/IGNITE-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Petrov updated IGNITE-16650:
Description:
log4j 1.2.17 is not supported and contains critical vulnerabilities
https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
I suggest excluding the ignite-log4j module from ignite
Direct vulnerabilities:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571
As a result of the mentioned migration, the following changes will be applied:
1. ignite-log4j.xml will be migrated to log4j2 format. Unfortunately after the
refactoring we will get two configuration ignite-log4j.xml and
ignite-log4j2.xml both in log4j2 format because the main goal of this
refactoring is to keep current log formatting that is used with log4j intact.
Currently ignite-log4j.xml and ignite-log4j2.xml provides different log formats
for log4j and log4j2 respectively.
2. core/src/test/config/log4j-test.xml will not be migrated to log4j2 because
it is used in compatibility tests.
3. core/src/test/config/log4j2-test.xml is refactored to suite current log4j
format. The current version of core/src/test/config/log4j2-test.xml is moved
to the log4j2/src/test/config folder/
4. osgi-paxlogging will be removed because it's only meant to provide some
log4j dependencies.
5. Exception logging format will change slightly:
Before:
{code:java}
class org.apache.ignite.IgniteException: Platform error:System.Exception:
EXCEPTION_TEST_Warn
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
at
org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
{code}
After:
{code:java}
org.apache.ignite.IgniteException: Platform error:System.Exception:
EXCEPTION_TEST_Warn
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
at
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
at
org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
{code}
As we see - only the first "class" word is omitted.
6. All other files containing log4j configuration will be refactored to suite
log4j2 and will be renamed if previously their name allowed log4j to
automatically find them in the class path (e.g. log4j.xml -> log4j2.xml and so
on)
was:
log4j 1.2.17 is not supported and contains critical vulnerabilities
https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
I suggest excluding the ignite-log4j module from ignite
Direct vulnerabilities:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571
> Exclude ignite-log4j, log4j 1.2.17
> --
>
> Key: IGNITE-16650
> URL: https://issues.apache.org/jira/browse/IGNITE-16650
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergei Ryzhov
>Assignee: Mikhail Petrov
>Priority: Major
> Labels: ise
> Time Spent: 20m
> Remaining Estimate: 0h
>
> log4j 1.2.17 is not supported and contains critical vulnerabilities
> https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
> I suggest excluding the ignite-log4j module from ignite
> Direct vulnerabilities:
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571
> As a result of the mentioned migration, the following changes will be applied:
> 1. ignite-log4j.xml will be migrated to log4j2 format. Unfortunately after
> the refactoring we will get two configuration ignite-log4j.xml and
> ignite-log4j2.xml both in log4j2 format because the main goal of this
> refactoring is to keep current log formatting
[jira] [Assigned] (IGNITE-16650) Exclude ignite-log4j, log4j 1.2.17
[ https://issues.apache.org/jira/browse/IGNITE-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Petrov reassigned IGNITE-16650: --- Assignee: Mikhail Petrov (was: Sergei Ryzhov) > Exclude ignite-log4j, log4j 1.2.17 > -- > > Key: IGNITE-16650 > URL: https://issues.apache.org/jira/browse/IGNITE-16650 > Project: Ignite > Issue Type: Bug >Reporter: Sergei Ryzhov >Assignee: Mikhail Petrov >Priority: Major > Labels: ise > Time Spent: 20m > Remaining Estimate: 0h > > log4j 1.2.17 is not supported and contains critical vulnerabilities > https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces > I suggest excluding the ignite-log4j module from ignite > Direct vulnerabilities: > https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305 > https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302 > https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104 > https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17311) KafkaToIgniteLoader instantiates several Spring contexts
[ https://issues.apache.org/jira/browse/IGNITE-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amelchev Nikita updated IGNITE-17311: - Fix Version/s: 2.14 > KafkaToIgniteLoader instantiates several Spring contexts > > > Key: IGNITE-17311 > URL: https://issues.apache.org/jira/browse/IGNITE-17311 > Project: Ignite > Issue Type: Improvement > Components: extensions >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Major > Fix For: 2.14 > > > KafkaToIgniteLoader instantiates two SpringApplicationContext during app > start. > This can lead to error in case other beans inside context not designed for > this case. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17318) Implement RocksDB based sorted index storage
Ivan Bessonov created IGNITE-17318: -- Summary: Implement RocksDB based sorted index storage Key: IGNITE-17318 URL: https://issues.apache.org/jira/browse/IGNITE-17318 Project: Ignite Issue Type: Improvement Reporter: Ivan Bessonov Pretty straightforward. Complicated places are: * Binary tuples comparator: should be as fast as possible. Some optimizations might be moved to other issues. * Thorough testing is required. We have both Java and native comparators planned. They should behave identically. This means a specific way of writing tests, to account for this in advance. * Bounds checking on range scan: by default, comparator should include the lower bound and exclude the upper bound. This is how prefix search works. This means that exclusion of the lower bound (if needed) and inclusion of the upper bound (if needed) +must be performed manually+ inside of the scan method. The question is, do we use separate column families for indexes? At one hand, this increases the number of files and potentially even increases time of the flush, but on the other, it looks easy (or is it). Currently, every index has only a UUID id. Just like for tables, we could create an integer identifier, because why not. This way we could store all indexes in a single column family without too much overhead. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17317) Test IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata is flaky on TC
Sergey Chugunov created IGNITE-17317:
Summary: Test
IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata
is flaky on TC
Key: IGNITE-17317
URL: https://issues.apache.org/jira/browse/IGNITE-17317
Project: Ignite
Issue Type: Bug
Reporter: Sergey Chugunov
Fix For: 2.14
Test history is available here: [TC
link|https://ci.ignite.apache.org/test/-1717009344468901983?currentProjectId=IgniteTests24Java8=%3Cdefault%3E]
Test fails on a cache put operation before a condition being verified is
reached:
{code:java}
org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys
(retry update if possible).: [2]
at
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
at
org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
{code}
There is also a stack trace in logs indicating there is some problem when
writing to WAL:
{code:java}
org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys
(retry update if possible).: [2]
at
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
at
org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2428)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: class
org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException:
Failed to update keys (retry update if possible).: [2]
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:401)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:248)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateResponse(GridDhtAtomicCache.java:3213)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$500(GridDhtAtomicCache.java:145)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:305)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:300)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1151)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:592)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:393)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:319)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:110)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:309)
at
[jira] [Assigned] (IGNITE-17317) Test IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata is flaky on TC
[
https://issues.apache.org/jira/browse/IGNITE-17317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov reassigned IGNITE-17317:
Assignee: Sergey Chugunov
> Test
> IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata
> is flaky on TC
> -
>
> Key: IGNITE-17317
> URL: https://issues.apache.org/jira/browse/IGNITE-17317
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Chugunov
>Assignee: Sergey Chugunov
>Priority: Major
> Fix For: 2.14
>
>
> Test history is available here: [TC
> link|https://ci.ignite.apache.org/test/-1717009344468901983?currentProjectId=IgniteTests24Java8=%3Cdefault%3E]
> Test fails on a cache put operation before a condition being verified is
> reached:
> {code:java}
> org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys
> (retry update if possible).: [2]
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
> at
> org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
> {code}
> There is also a stack trace in logs indicating there is some problem when
> writing to WAL:
> {code:java}
> org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys
> (retry update if possible).: [2]
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
> at
> org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2428)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: class
> org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException:
> Failed to update keys (retry update if possible).: [2]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:401)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:248)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateResponse(GridDhtAtomicCache.java:3213)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$500(GridDhtAtomicCache.java:145)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:305)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:300)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1151)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:592)
[jira] [Updated] (IGNITE-17309) Transactional support for partition scans
[
https://issues.apache.org/jira/browse/IGNITE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17309:
-
Description:
It's required to add transactional support to
{code:java}
org.apache.ignite.internal.table.InternalTable#scan {code}
binding it to
{code:java}
org.apache.ignite.internal.storage.MvPartitionStorage#scan(java.util.function.Predicate,
java.util.UUID) {code}
along with acquiring corresponding locks, namely S_commit(table) - if a
predicate can produce phantom reads, IS_commit(table) - otherwise.
Besides transactional support itself, it worth to introduce direct storage
reads from within PrimaryReplica instead of going through raft.
was:
It's required to add transactional support to
{code:java}
org.apache.ignite.internal.table.InternalTable#scan {code}
binding it to
{code:java}
org.apache.ignite.internal.storage.MvPartitionStorage#scan(java.util.function.Predicate,
java.util.UUID) {code}
along with acquiring corresponding locks, namely S_commit(table) - if a
predicate can produce phantom reads, IS_commit(table) - otherwise.
> Transactional support for partition scans
> -
>
> Key: IGNITE-17309
> URL: https://issues.apache.org/jira/browse/IGNITE-17309
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: ignite-3, transaction3_rw
>
> It's required to add transactional support to
> {code:java}
> org.apache.ignite.internal.table.InternalTable#scan {code}
> binding it to
> {code:java}
> org.apache.ignite.internal.storage.MvPartitionStorage#scan(java.util.function.Predicate,
> java.util.UUID) {code}
> along with acquiring corresponding locks, namely S_commit(table) - if a
> predicate can produce phantom reads, IS_commit(table) - otherwise.
> Besides transactional support itself, it worth to introduce direct storage
> reads from within PrimaryReplica instead of going through raft.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17309) Transactional support for partition scans
[
https://issues.apache.org/jira/browse/IGNITE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17309:
-
Description:
It's required to add transactional support to
{code:java}
org.apache.ignite.internal.table.InternalTable#scan {code}
binding it to
{code:java}
org.apache.ignite.internal.storage.MvPartitionStorage#scan(java.util.function.Predicate,
java.util.UUID) {code}
along with acquiring corresponding locks, namely S_commit(table) - if a
predicate can produce phantom reads, IS_commit(table) - otherwise.
was:
It's required to add transactional support to
{code:java}
org.apache.ignite.internal.table.InternalTable#scan {code}
binding it to
{code:java}
org.apache.ignite.internal.storage.MvPartitionStorage#scan(java.util.function.Predicate,
java.util.UUID) {code}
along with acquiring corresponding locks, namely S_commit(table) - if a
predicate can produce phantom reads, IS_commit(table) - otherwise.
Besides transactional support itself, it worth to ensure that scans together
with other read commands don't linearized wihtin raft, meaning that raft in
lease-based mode shouldn't perform read-index and thus properly perform
multiple scans concurrently. If given assumption occurred to be incorrect it'll
be required to re-implement scans in order to use storage directly without any
raft interaction. It's possible because any RW-reads goes through
PrimaryReplica that is guaranteed to be up-to-date. The only non-trivial moment
here, is that according to raft paper storage itself isn't guaranteed to apply
all commands up to commit index in a synchronous way, meaning that it's
required to await for applyIndex to be greater or equal than commitIndex.
IGNITE-16907 will introduce storage specific applied index, so that we should
only extract commit index to replica somehow.
> Transactional support for partition scans
> -
>
> Key: IGNITE-17309
> URL: https://issues.apache.org/jira/browse/IGNITE-17309
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: ignite-3, transaction3_rw
>
> It's required to add transactional support to
> {code:java}
> org.apache.ignite.internal.table.InternalTable#scan {code}
> binding it to
> {code:java}
> org.apache.ignite.internal.storage.MvPartitionStorage#scan(java.util.function.Predicate,
> java.util.UUID) {code}
> along with acquiring corresponding locks, namely S_commit(table) - if a
> predicate can produce phantom reads, IS_commit(table) - otherwise.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-17220) YCSB benchmark run for ignite2 vs ignite3
[ https://issues.apache.org/jira/browse/IGNITE-17220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Belyak reassigned IGNITE-17220: - Assignee: Alexander Belyak > YCSB benchmark run for ignite2 vs ignite3 > - > > Key: IGNITE-17220 > URL: https://issues.apache.org/jira/browse/IGNITE-17220 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Assignee: Alexander Belyak >Priority: Major > Labels: ignite-3 > > For further investigation of ignite3 performance issues, we need to run the > following benchmarks to compare ignite2 vs ignite3 performance: > * Usual ycsb benchmark with mixed load patterns > * Insert-only ycsb benchmark > For ignite2 and ignite3 in the following configurations: > * 3 ignite nodes setup (so, table must have 1 partition and 3 replicas) > * 1 ignite node setup (so, table must have 1 partitoin and 1 replica) > Also, please provide: > * Hardware configuration of the environment, where benchmark was executed > * JFRs for every node in every run. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-17315) [Extensions] Fix topology-validator-ext compilation
[
https://issues.apache.org/jira/browse/IGNITE-17315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amelchev Nikita resolved IGNITE-17315.
--
Resolution: Fixed
> [Extensions] Fix topology-validator-ext compilation
> ---
>
> Key: IGNITE-17315
> URL: https://issues.apache.org/jira/browse/IGNITE-17315
> Project: Ignite
> Issue Type: Bug
> Components: extensions
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>
> {noformat}
> 05:05:05 [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile
> (default-testCompile) on project ignite-topology-validator-ext: Compilation
> failure: Compilation failure:
> 05:05:05 [ERROR]
> /opt/buildagent/work/9319dd66c384518/modules/topology-validator-ext/src/test/java/org/apache/ignite/plugin/cache/IgniteCacheTopologyValidatorTest.java:[43,53]
> cannot find symbol
> 05:05:05 [ERROR] symbol: class RunnableX
> 05:05:05 [ERROR] location: class
> org.apache.ignite.testframework.GridTestUtils
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17315) [Extensions] Fix topology-validator-ext compilation
[
https://issues.apache.org/jira/browse/IGNITE-17315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563158#comment-17563158
]
Amelchev Nikita commented on IGNITE-17315:
--
Merged into the master.
> [Extensions] Fix topology-validator-ext compilation
> ---
>
> Key: IGNITE-17315
> URL: https://issues.apache.org/jira/browse/IGNITE-17315
> Project: Ignite
> Issue Type: Bug
> Components: extensions
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>
> {noformat}
> 05:05:05 [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile
> (default-testCompile) on project ignite-topology-validator-ext: Compilation
> failure: Compilation failure:
> 05:05:05 [ERROR]
> /opt/buildagent/work/9319dd66c384518/modules/topology-validator-ext/src/test/java/org/apache/ignite/plugin/cache/IgniteCacheTopologyValidatorTest.java:[43,53]
> cannot find symbol
> 05:05:05 [ERROR] symbol: class RunnableX
> 05:05:05 [ERROR] location: class
> org.apache.ignite.testframework.GridTestUtils
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17315) [Extensions] Fix topology-validator-ext compilation
[
https://issues.apache.org/jira/browse/IGNITE-17315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563157#comment-17563157
]
Amelchev Nikita commented on IGNITE-17315:
--
https://ci.ignite.apache.org/buildConfiguration/IgniteExtensions_Tests_TopologyValidator/6669220
> [Extensions] Fix topology-validator-ext compilation
> ---
>
> Key: IGNITE-17315
> URL: https://issues.apache.org/jira/browse/IGNITE-17315
> Project: Ignite
> Issue Type: Bug
> Components: extensions
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>
> {noformat}
> 05:05:05 [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile
> (default-testCompile) on project ignite-topology-validator-ext: Compilation
> failure: Compilation failure:
> 05:05:05 [ERROR]
> /opt/buildagent/work/9319dd66c384518/modules/topology-validator-ext/src/test/java/org/apache/ignite/plugin/cache/IgniteCacheTopologyValidatorTest.java:[43,53]
> cannot find symbol
> 05:05:05 [ERROR] symbol: class RunnableX
> 05:05:05 [ERROR] location: class
> org.apache.ignite.testframework.GridTestUtils
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17316) Thin client pluggable affnity function to use partition awareness
[
https://issues.apache.org/jira/browse/IGNITE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-17316:
-
Description:
We should provie a pluggable affinity for the thin client, thus using the
partition awareness features become possible on only for
{{RendezvousAffinityFunction}}.
Dev-list discussion:
https://lists.apache.org/thread/7n7zbcgw59voxyr08ct3zx2ss5g8q9wh
was:We should provie a pluggable affinity for the thin client, thus using the
partition awareness features become possible on only for
{{RendezvousAffinityFunction}}.
> Thin client pluggable affnity function to use partition awareness
> -
>
> Key: IGNITE-17316
> URL: https://issues.apache.org/jira/browse/IGNITE-17316
> Project: Ignite
> Issue Type: Improvement
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Fix For: 2.14
>
>
> We should provie a pluggable affinity for the thin client, thus using the
> partition awareness features become possible on only for
> {{RendezvousAffinityFunction}}.
> Dev-list discussion:
> https://lists.apache.org/thread/7n7zbcgw59voxyr08ct3zx2ss5g8q9wh
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17316) Thin client pluggable affnity function to use partition awareness
[
https://issues.apache.org/jira/browse/IGNITE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-17316:
-
Component/s: thin client
> Thin client pluggable affnity function to use partition awareness
> -
>
> Key: IGNITE-17316
> URL: https://issues.apache.org/jira/browse/IGNITE-17316
> Project: Ignite
> Issue Type: Improvement
> Components: thin client
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Fix For: 2.14
>
>
> We should provie a pluggable affinity for the thin client, thus using the
> partition awareness features become possible on only for
> {{RendezvousAffinityFunction}}.
> Dev-list discussion:
> https://lists.apache.org/thread/7n7zbcgw59voxyr08ct3zx2ss5g8q9wh
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17316) Thin client pluggable affnity function to use partition awareness
[
https://issues.apache.org/jira/browse/IGNITE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-17316:
-
Affects Version/s: 2.13
> Thin client pluggable affnity function to use partition awareness
> -
>
> Key: IGNITE-17316
> URL: https://issues.apache.org/jira/browse/IGNITE-17316
> Project: Ignite
> Issue Type: Improvement
> Components: thin client
>Affects Versions: 2.13
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Fix For: 2.14
>
>
> We should provie a pluggable affinity for the thin client, thus using the
> partition awareness features become possible on only for
> {{RendezvousAffinityFunction}}.
> Dev-list discussion:
> https://lists.apache.org/thread/7n7zbcgw59voxyr08ct3zx2ss5g8q9wh
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17316) Thin client pluggable affnity function to use partition awareness
Maxim Muzafarov created IGNITE-17316:
Summary: Thin client pluggable affnity function to use partition
awareness
Key: IGNITE-17316
URL: https://issues.apache.org/jira/browse/IGNITE-17316
Project: Ignite
Issue Type: Improvement
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov
Fix For: 2.14
We should provie a pluggable affinity for the thin client, thus using the
partition awareness features become possible on only for
{{RendezvousAffinityFunction}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17081) Implement checkpointIndex for RocksDB
[
https://issues.apache.org/jira/browse/IGNITE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Bessonov updated IGNITE-17081:
---
Description:
Please refer to https://issues.apache.org/jira/browse/IGNITE-16907 for
prerequisites.
Please also familiarize yourself with
https://issues.apache.org/jira/browse/IGNITE-17077 for better understanding,
the description is continued from there.
For RocksDB based storage the recovery process is trivial, because RocksDB has
its own WAL. So, for testing purposes, it would be enough to just store update
index in meta column family.
Immediately we have a write amplification issue, on top of possible performance
degradation. Obvious solution is inherently bad and needs to be improved.
h2. General idea & implementation
Obviously, WAL needs to be disabled (WriteOptions#setDisableWAL). This kinda
breaks RocksDB recovery procedure, we need to take measures to avoid it.
The only feasible way to do so is to use DBOptions#setAtomicFlush in
conjunction with org.rocksdb.WriteBatchWithIndex. This allows RocksDB to save
all column families consistently, if you have batches that cover several CFs.
Basically, {{acquireConsistencyLock()}} would create a thread-local write
batch, that's applied on locks release. Most of RocksDbMvPartitionStorage will
be affected by this change.
NOTE: I believe that scans with unapplied batches should be prohibited for now
(gladly, there's a WriteBatchInterface#count() to check). I don't see any
practical value and a proper way of implementing it, considering how spread-out
in time the scan process is.
h2. Callbacks and RAFT snapshots
Simply storing and reading update index is easy. Reading committed index is
more challenging, I propose caching it and update only from the closure, that
can also be used by RAFT to truncate the log.
For a closure, there are several things to account for during the
implementation:
* DBOptions#setListeners. We need two events - ON_FLUSH_BEGIN and
ON_FLUSH_COMPLETED. All "completed" events go after all "begin" events in
atomic flush mode. And, once you have your first "completed" event ,you have a
guarantee that *all* memtables are already persisted.
This allows easy tracking of RocksDB flushes, monitoring events alteration is
all that's needed.
* Unlike PDS implementation, here we will be writing updateIndex value into a
memtable every time. This makes it harder to find persistedIndex values for
partitions. Gladly, considering the events that we have, during the time
between first "completed" and the very next "begin", the state on disk is fully
consistent. And there's a way to read data from storage avoiding memtable
completely - ReadOptions#setReadTier(PERSISTED_TIER).
Summarizing everything from the above, we should implement following protocol:
{code:java}
During table start: read latest values of update indexes. Store them in an
in-memory structure.
Set "lastEventType = ON_FLUSH_COMPLETED;".
onFlushBegin:
if (lastEventType == ON_FLUSH_BEGIN)
return;
waitForLastAsyncUpdateIndexesRead();
lastEventType = ON_FLUSH_BEGIN;
onFlushCompleted:
if (lastEventType == ON_FLUSH_COMPLETED)
return;
asyncReadUpdateIndexesFromDisk();
lastEventType = ON_FLUSH_COMPLETED;{code}
Reading values from disk must be performed asynchronously to not stall flushing
process. We don't control locks that RocksDb holds while calling listener's
methods.
That asynchronous process would invoke closures that provide presisted
updateIndex values to other components.
NODE: One might say that we should call "waitForLastAsyncUpdateIndexesRead();"
as late as possible just in case. But my implementation says calling it during
the first event. This is fine. I noticed that column families are flushed in
order of their internal ids. These ids correspond to a sequence number of CFs,
and the "default" CF is always created first. This is the exact CF that we use
to store meta. Maybe we're going to change this and create a separate meta CF.
Only then we could start optimizing this part, and only if we'll have an actual
proof that there's a stall in this exact place.
h3. Types of storages
RocksDB is used for:
* tables
* cluster management
* meta-storage
All these types should use the same recovery procedure, but code is located in
different places. I hope that it won't be a big problem and we can do
everything at once.
was:
Please refer to https://issues.apache.org/jira/browse/IGNITE-16907 for
prerequisites.
Please also familiarize yourself with
https://issues.apache.org/jira/browse/IGNITE-17077 for better understanding,
the description is continued from there.
For RocksDB based storage the recovery process is trivial, because RocksDB has
its own WAL. So, for testing purposes, it would be enough to just store update
index in meta column family.
Immediately we have a write amplification issue, on top
[jira] [Created] (IGNITE-17315) [Extensions] Fix topology-validator-ext compilation
Amelchev Nikita created IGNITE-17315:
Summary: [Extensions] Fix topology-validator-ext compilation
Key: IGNITE-17315
URL: https://issues.apache.org/jira/browse/IGNITE-17315
Project: Ignite
Issue Type: Bug
Reporter: Amelchev Nikita
Assignee: Amelchev Nikita
{noformat}
05:05:05 [ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile
(default-testCompile) on project ignite-topology-validator-ext: Compilation
failure: Compilation failure:
05:05:05 [ERROR]
/opt/buildagent/work/9319dd66c384518/modules/topology-validator-ext/src/test/java/org/apache/ignite/plugin/cache/IgniteCacheTopologyValidatorTest.java:[43,53]
cannot find symbol
05:05:05 [ERROR] symbol: class RunnableX
05:05:05 [ERROR] location: class
org.apache.ignite.testframework.GridTestUtils
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17315) [Extensions] Fix topology-validator-ext compilation
[
https://issues.apache.org/jira/browse/IGNITE-17315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amelchev Nikita updated IGNITE-17315:
-
Component/s: extensions
> [Extensions] Fix topology-validator-ext compilation
> ---
>
> Key: IGNITE-17315
> URL: https://issues.apache.org/jira/browse/IGNITE-17315
> Project: Ignite
> Issue Type: Bug
> Components: extensions
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>
> {noformat}
> 05:05:05 [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile
> (default-testCompile) on project ignite-topology-validator-ext: Compilation
> failure: Compilation failure:
> 05:05:05 [ERROR]
> /opt/buildagent/work/9319dd66c384518/modules/topology-validator-ext/src/test/java/org/apache/ignite/plugin/cache/IgniteCacheTopologyValidatorTest.java:[43,53]
> cannot find symbol
> 05:05:05 [ERROR] symbol: class RunnableX
> 05:05:05 [ERROR] location: class
> org.apache.ignite.testframework.GridTestUtils
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17314) .NET: Thin 3.0: Unified exception handling
[ https://issues.apache.org/jira/browse/IGNITE-17314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-17314: Description: Implement a set of public exception types according to IGNITE-17312. > .NET: Thin 3.0: Unified exception handling > -- > > Key: IGNITE-17314 > URL: https://issues.apache.org/jira/browse/IGNITE-17314 > Project: Ignite > Issue Type: Improvement >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: .NET, ignite-3 > Fix For: 3.0.0-alpha6 > > > Implement a set of public exception types according to IGNITE-17312. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17314) .NET: Thin 3.0: Unified exception handling
Pavel Tupitsyn created IGNITE-17314: --- Summary: .NET: Thin 3.0: Unified exception handling Key: IGNITE-17314 URL: https://issues.apache.org/jira/browse/IGNITE-17314 Project: Ignite Issue Type: Improvement Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 3.0.0-alpha6 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17312) Thin 3.0: Unified exception handling
[ https://issues.apache.org/jira/browse/IGNITE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-17312: Description: Java thin client uses the same public API interfaces as the server side. Those user-facing APIs should behave in the same way on the server and client sides, including exceptions that may be thrown. # Remove *IgniteClientException*. # Keep *IgniteClientConnectionException* and other client-specific types. Inherit them from *IgniteException*. # When sending exception details from server to client, include the following: ## Code (group + error, see IgniteException.code) ## TraceId ## Message ## Fully-qualified class name ## OPTIONALLY: stack trace (Sensitive info. Send only when enabled in server-side ClientConnectorConfig. Disabled by default.) was: Java thin client uses the same public API interfaces as the server side. Those user-facing APIs should behave in the same way on the server and client sides, including exceptions that may be thrown. # Remove *IgniteClientException*. # Keep *IgniteClientConnectionException* and other client-specific types. Inherit them from *IgniteException*. # When sending exception details from server to client, include the following: ## Code (group + error, see IgniteException.code) ## TraceId ## Message ## Fully-qualified class name ## OPTIONALLY: stack trace (when enabled in server-side ClientConnectorConfig. Disabled by default.) > Thin 3.0: Unified exception handling > > > Key: IGNITE-17312 > URL: https://issues.apache.org/jira/browse/IGNITE-17312 > Project: Ignite > Issue Type: Improvement > Components: thin client >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > > Java thin client uses the same public API interfaces as the server side. > Those user-facing APIs should behave in the same way on the server and client > sides, including exceptions that may be thrown. > # Remove *IgniteClientException*. > # Keep *IgniteClientConnectionException* and other client-specific types. > Inherit them from *IgniteException*. > # When sending exception details from server to client, include the following: > ## Code (group + error, see IgniteException.code) > ## TraceId > ## Message > ## Fully-qualified class name > ## OPTIONALLY: stack trace (Sensitive info. Send only when enabled in > server-side ClientConnectorConfig. Disabled by default.) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17312) Thin 3.0: Unified exception handling
[ https://issues.apache.org/jira/browse/IGNITE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-17312: Description: Java thin client uses the same public API interfaces as the server side. Those user-facing APIs should behave in the same way on the server and client sides, including exceptions that may be thrown. # Remove *IgniteClientException*. # Keep *IgniteClientConnectionException* and other client-specific types. Inherit them from *IgniteException*. # When sending exception details from server to client, include the following: ## Code (group + error, see IgniteException.code) ## TraceId ## Message ## Fully-qualified class name ## OPTIONALLY: stack trace (when enabled in server-side ClientConnectorConfig. Disabled by default.) was: Java thin client uses the same public API interfaces as the server side. Those user-facing APIs should behave in the same way on the server and client sides, including exceptions that may be thrown. > Thin 3.0: Unified exception handling > > > Key: IGNITE-17312 > URL: https://issues.apache.org/jira/browse/IGNITE-17312 > Project: Ignite > Issue Type: Improvement > Components: thin client >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > > Java thin client uses the same public API interfaces as the server side. > Those user-facing APIs should behave in the same way on the server and client > sides, including exceptions that may be thrown. > # Remove *IgniteClientException*. > # Keep *IgniteClientConnectionException* and other client-specific types. > Inherit them from *IgniteException*. > # When sending exception details from server to client, include the following: > ## Code (group + error, see IgniteException.code) > ## TraceId > ## Message > ## Fully-qualified class name > ## OPTIONALLY: stack trace (when enabled in server-side > ClientConnectorConfig. Disabled by default.) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17312) Thin 3.0: Unified exception handling
[ https://issues.apache.org/jira/browse/IGNITE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-17312: Description: Java thin client uses the same public API interfaces as the server side. Those user-facing APIs should behave in the same way on the server and client sides, including exceptions that may be thrown. was:TODO > Thin 3.0: Unified exception handling > > > Key: IGNITE-17312 > URL: https://issues.apache.org/jira/browse/IGNITE-17312 > Project: Ignite > Issue Type: Improvement > Components: thin client >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > > Java thin client uses the same public API interfaces as the server side. > Those user-facing APIs should behave in the same way on the server and client > sides, including exceptions that may be thrown. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Deleted] (IGNITE-17313) Thin 3.0: Unified exception handling
[ https://issues.apache.org/jira/browse/IGNITE-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn deleted IGNITE-17313: > Thin 3.0: Unified exception handling > > > Key: IGNITE-17313 > URL: https://issues.apache.org/jira/browse/IGNITE-17313 > Project: Ignite > Issue Type: Improvement >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: ignite-3 > > TODO -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17313) Thin 3.0: Unified exception handling
Pavel Tupitsyn created IGNITE-17313: --- Summary: Thin 3.0: Unified exception handling Key: IGNITE-17313 URL: https://issues.apache.org/jira/browse/IGNITE-17313 Project: Ignite Issue Type: Improvement Components: thin client Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 3.0.0-alpha6 TODO -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17312) Thin 3.0: Unified exception handling
Pavel Tupitsyn created IGNITE-17312: --- Summary: Thin 3.0: Unified exception handling Key: IGNITE-17312 URL: https://issues.apache.org/jira/browse/IGNITE-17312 Project: Ignite Issue Type: Improvement Components: thin client Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 3.0.0-alpha6 TODO -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17311) KafkaToIgniteLoader instantiates several Spring contexts
[ https://issues.apache.org/jira/browse/IGNITE-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amelchev Nikita updated IGNITE-17311: - Component/s: extensions > KafkaToIgniteLoader instantiates several Spring contexts > > > Key: IGNITE-17311 > URL: https://issues.apache.org/jira/browse/IGNITE-17311 > Project: Ignite > Issue Type: Improvement > Components: extensions >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Major > > KafkaToIgniteLoader instantiates two SpringApplicationContext during app > start. > This can lead to error in case other beans inside context not designed for > this case. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17311) KafkaToIgniteLoader instantiates several Spring contexts
Amelchev Nikita created IGNITE-17311: Summary: KafkaToIgniteLoader instantiates several Spring contexts Key: IGNITE-17311 URL: https://issues.apache.org/jira/browse/IGNITE-17311 Project: Ignite Issue Type: Improvement Reporter: Amelchev Nikita Assignee: Amelchev Nikita KafkaToIgniteLoader instantiates two SpringApplicationContext during app start. This can lead to error in case other beans inside context not designed for this case. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-17032) Apache Ignite Docker container does not run correctly if image is run in read only file system mode
[
https://issues.apache.org/jira/browse/IGNITE-17032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563063#comment-17563063
]
Stanislav Lukyanov edited comment on IGNITE-17032 at 7/6/22 8:57 AM:
-
Thanks for reporting [~tonkovic].
TLDR: this is fixable, and I think there is a workaround - see it at the end of
the comment.
My first read of this was - well, Ignite should normally be used with a
writable disk, so "Not an issue".
BUT reading the second time I realized that the problem is that we're trying to
write into the *root* volume. We shouldn't do that.
Apparently, the thing that doesn't work is the following line
{code}
version=$(awk -F[\"\.] '{print $2}' <<< ${version})
{code}
In particular, the `<<<` is at fault.
Turns out, in Bash the `<<<` ("here-string") is implemented via `<<`
("here-document") which is in turn implemented via temp files. The root file
system is read-only, hence `/tmp` is also read-only, hence `<<` and `<<<` can't
work. You learn something new everyday, don't you?
[This](https://stackoverflow.com/questions/2500436/how-does-cat-eof-work-in-bash)
explains it.
I'm quite surprised that basic Bash syntax can be unfriendly to cloud but here
we are.
Two things we should do (each of them will suffice, but I'd do both):
1. Get rid of the `<<<` and `<<` in the scripts, just in case.
2. Mount `/tmp` as tmpfs. It should always be tmpfs in all environments, for
many good reasons. I'm a bit surprised it's not tmpfs now, and I'd certainly
change that. The question is whether we can and should do this right in the
Dockerfile, or ask everyone to mount `/tmp` as tmpfs when running. The latter
requires at least docs changes.
As a workaround, I think running the container with an [explicit tmpfs
mount](https://stackoverflow.com/questions/34698620/docker-and-volatile-volumes-ala-tmp)
will do the trick:
{code}
docker run --tmpfs /tmp ...
{code}
Disclaimer: I haven't got my hands dirty yet, so all of the above needs to be
confirmed with tests.
was (Author: slukyanov):
Thanks for reporting [~tonkovic].
TLDR: this is fixable, and I think there is a workaround - see it at the end of
the comment.
My first read of this was - well, Ignite should normally be used with a
writable disk, so "Not an issue".
BUT reading the second time I realized that the problem is that we're trying to
write into the *root* volume. We shouldn't do that.
Apparently, the thing that doesn't work is the following line
```
version=$(awk -F[\"\.] '{print $2}' <<< ${version})
```
In particular, the `<<<` is at fault.
Turns out, in Bash the `<<<` ("here-string") is implemented via `<<`
("here-document") which is in turn implemented via temp files. The root file
system is read-only, hence `/tmp` is also read-only, hence `<<` and `<<<` can't
work. You learn something new everyday, don't you?
[This](https://stackoverflow.com/questions/2500436/how-does-cat-eof-work-in-bash)
explains it.
I'm quite surprised that basic Bash syntax can be unfriendly to cloud but here
we are.
Two things we should do (each of them will suffice, but I'd do both):
1. Get rid of the `<<<` and `<<` in the scripts, just in case.
2. Mount `/tmp` as tmpfs. It should always be tmpfs in all environments, for
many good reasons. I'm a bit surprised it's not tmpfs now, and I'd certainly
change that. The question is whether we can and should do this right in the
Dockerfile, or ask everyone to mount `/tmp` as tmpfs when running. The latter
requires at least docs changes.
As a workaround, I think running the container with an [explicit tmpfs
mount](https://stackoverflow.com/questions/34698620/docker-and-volatile-volumes-ala-tmp)
will do the trick:
```
docker run --tmpfs /tmp ...
```
Disclaimer: I haven't got my hands dirty yet, so all of the above needs to be
confirmed with tests.
> Apache Ignite Docker container does not run correctly if image is run in read
> only file system mode
> ---
>
> Key: IGNITE-17032
> URL: https://issues.apache.org/jira/browse/IGNITE-17032
> Project: Ignite
> Issue Type: Bug
> Components: build
>Affects Versions: 2.13
>Reporter: Petar Tonkovic
>Priority: Major
>
> When following the Kubernetes deployment tutorials (online:
> https://ignite.apache.org/docs/latest/installation/kubernetes/azure-deployment,
> youtube: [https://youtu.be/38YgdAOs038]), trying to run the official docker
> image () with the --read-only flag is causing errors:
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 52: cannot create
> temp file for here-document: Read-only file system
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 85: [: -lt: unary
> operator expected2022-05-25T14:27:34.504369604+02:00
>
[jira] [Comment Edited] (IGNITE-17032) Apache Ignite Docker container does not run correctly if image is run in read only file system mode
[
https://issues.apache.org/jira/browse/IGNITE-17032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563063#comment-17563063
]
Stanislav Lukyanov edited comment on IGNITE-17032 at 7/6/22 8:56 AM:
-
Thanks for reporting [~tonkovic].
TLDR: this is fixable, and I think there is a workaround - see it at the end of
the comment.
My first read of this was - well, Ignite should normally be used with a
writable disk, so "Not an issue".
BUT reading the second time I realized that the problem is that we're trying to
write into the *root* volume. We shouldn't do that.
Apparently, the thing that doesn't work is the following line
```
version=$(awk -F[\"\.] '{print $2}' <<< ${version})
```
In particular, the `<<<` is at fault.
Turns out, in Bash the `<<<` ("here-string") is implemented via `<<`
("here-document") which is in turn implemented via temp files. The root file
system is read-only, hence `/tmp` is also read-only, hence `<<` and `<<<` can't
work. You learn something new everyday, don't you?
[This](https://stackoverflow.com/questions/2500436/how-does-cat-eof-work-in-bash)
explains it.
I'm quite surprised that basic Bash syntax can be unfriendly to cloud but here
we are.
Two things we should do (each of them will suffice, but I'd do both):
1. Get rid of the `<<<` and `<<` in the scripts, just in case.
2. Mount `/tmp` as tmpfs. It should always be tmpfs in all environments, for
many good reasons. I'm a bit surprised it's not tmpfs now, and I'd certainly
change that. The question is whether we can and should do this right in the
Dockerfile, or ask everyone to mount `/tmp` as tmpfs when running. The latter
requires at least docs changes.
As a workaround, I think running the container with an [explicit tmpfs
mount](https://stackoverflow.com/questions/34698620/docker-and-volatile-volumes-ala-tmp)
will do the trick:
```
docker run --tmpfs /tmp ...
```
Disclaimer: I haven't got my hands dirty yet, so all of the above needs to be
confirmed with tests.
was (Author: slukyanov):
Thanks for reporting [~tonkovic].
TLDR: this is fixable, and I think there is a workaround - see it at the end of
the comment.
My first read of this was - well, Ignite should normally be used with a
writable disk, so "Not an issue".
BUT reading the second time I realized that the problem is that we're trying to
write into the *root* volume. We shouldn't do that.
Apparently, the thing that doesn't work is the following line
```
version=$(awk -F[\"\.] '{print $2}' <<< ${version})
```
In particular, the `<<<` is at fault.
Turns out, in Bash the `<<<` ("here-string") is implemented via `<<`
("here-document") which is in turn implemented via temp files. The root file
system is read-only, hence `/tmp` is also read-only, hence `<<` and `<<<` can't
work. You learn something new everyday, don't you?
[This](https://stackoverflow.com/questions/2500436/how-does-cat-eof-work-in-bash)
explains it.
I'm quite surprised that basic Bash syntax can be unfriendly to cloud but here
we are.
Two things we should do (each of them will suffice, but I'd do both):
1. Get rid of the `<<<` and `<<` in the scripts, just in case.
2. Mount `/tmp` as tmpfs. It should always be tmpfs in all environments, for
many good reasons. I'm a bit surprised it's not tmpfs now, and I'd certainly
change that. The question is whether we can and should do this right in the
Dockerfile, or ask everyone to mount `/tmp` as tmpfs when running. The latter
requires at least docs changes.
As a workaround, I think running the container with an [explicit tmpfs
mount](https://stackoverflow.com/questions/34698620/docker-and-volatile-volumes-ala-tmp)
will do the trick:
```
docker run --tmpfs /tmp ...
```
Disclaimer: I haven't got my hands dirty yet, so all of the above needs to be
confirmed with tests.
> Apache Ignite Docker container does not run correctly if image is run in read
> only file system mode
> ---
>
> Key: IGNITE-17032
> URL: https://issues.apache.org/jira/browse/IGNITE-17032
> Project: Ignite
> Issue Type: Bug
> Components: build
>Affects Versions: 2.13
>Reporter: Petar Tonkovic
>Priority: Major
>
> When following the Kubernetes deployment tutorials (online:
> https://ignite.apache.org/docs/latest/installation/kubernetes/azure-deployment,
> youtube: [https://youtu.be/38YgdAOs038]), trying to run the official docker
> image () with the --read-only flag is causing errors:
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 52: cannot create
> temp file for here-document: Read-only file system
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 85: [: -lt: unary
> operator expected2022-05-25T14:27:34.504369604+02:00
>
> Since most
[jira] [Commented] (IGNITE-17032) Apache Ignite Docker container does not run correctly if image is run in read only file system mode
[
https://issues.apache.org/jira/browse/IGNITE-17032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563063#comment-17563063
]
Stanislav Lukyanov commented on IGNITE-17032:
-
Thanks for reporting [~tonkovic].
TLDR: this is fixable, and I think there is a workaround - see it at the end of
the comment.
My first read of this was - well, Ignite should normally be used with a
writable disk, so "Not an issue".
BUT reading the second time I realized that the problem is that we're trying to
write into the *root* volume. We shouldn't do that.
Apparently, the thing that doesn't work is the following line
```
version=$(awk -F[\"\.] '{print $2}' <<< ${version})
```
In particular, the `<<<` is at fault.
Turns out, in Bash the `<<<` ("here-string") is implemented via `<<`
("here-document") which is in turn implemented via temp files. The root file
system is read-only, hence `/tmp` is also read-only, hence `<<` and `<<<` can't
work. You learn something new everyday, don't you?
[This](https://stackoverflow.com/questions/2500436/how-does-cat-eof-work-in-bash)
explains it.
I'm quite surprised that basic Bash syntax can be unfriendly to cloud but here
we are.
Two things we should do (each of them will suffice, but I'd do both):
1. Get rid of the `<<<` and `<<` in the scripts, just in case.
2. Mount `/tmp` as tmpfs. It should always be tmpfs in all environments, for
many good reasons. I'm a bit surprised it's not tmpfs now, and I'd certainly
change that. The question is whether we can and should do this right in the
Dockerfile, or ask everyone to mount `/tmp` as tmpfs when running. The latter
requires at least docs changes.
As a workaround, I think running the container with an [explicit tmpfs
mount](https://stackoverflow.com/questions/34698620/docker-and-volatile-volumes-ala-tmp)
will do the trick:
```
docker run --tmpfs /tmp ...
```
Disclaimer: I haven't got my hands dirty yet, so all of the above needs to be
confirmed with tests.
> Apache Ignite Docker container does not run correctly if image is run in read
> only file system mode
> ---
>
> Key: IGNITE-17032
> URL: https://issues.apache.org/jira/browse/IGNITE-17032
> Project: Ignite
> Issue Type: Bug
> Components: build
>Affects Versions: 2.13
>Reporter: Petar Tonkovic
>Priority: Major
>
> When following the Kubernetes deployment tutorials (online:
> https://ignite.apache.org/docs/latest/installation/kubernetes/azure-deployment,
> youtube: [https://youtu.be/38YgdAOs038]), trying to run the official docker
> image () with the --read-only flag is causing errors:
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 52: cannot create
> temp file for here-document: Read-only file system
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 85: [: -lt: unary
> operator expected2022-05-25T14:27:34.504369604+02:00
>
> Since most managed company Kubernetes clusters enforce this read-only flag as
> a security requirement, it would be good to look into these errors.
>
> Later on, we get the following error on starting up:
> class org.apache.ignite.IgniteException: Failed to instantiate Spring XML
> application context (make sure all classes used in Spring configuration are
> present at CLASSPATH) [springUrl=file:/ignite/config/node-configuration.xml]
> at
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1135)
> at org.apache.ignite.Ignition.start(Ignition.java:356)
> at
> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:365)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> instantiate Spring XML application context (make sure all classes used in
> Spring configuration are present at CLASSPATH)
> [springUrl=file:/ignite/config/node-configuration.xml]
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:364)
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:102)
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:96)
> at
> org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:729)2022-05-25T14:27:34.916588365+02:00
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:930)
> at
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:839)2022-05-25T14:27:34.916609431+02:00
> at
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:709)2022-05-25T14:27:34.916622089+02:00
> at
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:678)2022-05-25T14:27:34.916636146+02:00
> at
>
[jira] [Commented] (IGNITE-17281) Return error groups from REST API
[ https://issues.apache.org/jira/browse/IGNITE-17281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563050#comment-17563050 ] Aleksandr commented on IGNITE-17281: Hi [~slava.koptilin], looks good to me, thanks. > Return error groups from REST API > - > > Key: IGNITE-17281 > URL: https://issues.apache.org/jira/browse/IGNITE-17281 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksandr >Assignee: Vyacheslav Koptilin >Priority: Major > Labels: iep-87, ignite-3 > Time Spent: 10m > Remaining Estimate: 0h > > After IGNITE-14931 and IGNITE-17193 are done, traceId and human-readable > error code should be returned from any rest component. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17215) Write ClusterSnapshotRecord to WAL
[ https://issues.apache.org/jira/browse/IGNITE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Timonin updated IGNITE-17215: Description: For PITR [1] process of recovering based on ClusterSnapshot + archived WALs. It's required to have a point in WAL which splits whole WAL on 2 areas: # Before this point all data changes are contained within ClusterSnapshot, and no need to recover them from WAL archived files. # After this point all data need to be recovered from WAL archived files. It's proposed to write ClusterSnapshotRecord while the checkpoint is running (cp#writeLock has acquired). ClusterSnapshot process guarantees: # there is no active transactions (or any data changes) in moment of running checkpoint. # ClusterSnapshot contains all data pages that will be persisted within this checkpoint process. Then every logical record after begin CheckointRecord doesn't belong to ClusterSnapshot. Then it's safe to write ClusterSnapshotRecord within the checkpoint process. [1] [https://cwiki.apache.org/confluence/pages/editpage.action?pageId=211884314] was: For PITR [1] process of recovering based on ClusterSnapshot + archived WALs. It's required to have a point in WAL which splits whole WAL on 2 areas: # Before this point all data changes are contained within ClusterSnapshot, and no need to recover them from WAL archived files. # After this point all data need to be recovered from WAL archived files. It's proposed to write ClusterSnapshotRecord at the moment checkpoint process starts (cp#writeLock has acquired). ClusterSnapshot process guarantees: # there is no active transactions (or any data changes) in moment of writing begin CheckpointRecord. # ClusterSnapshot consist of data pages that are materialized within this checkpoint process. Then every logical record after begin CheckointRecord doesn't belong to ClusterSnapshot. Then it's safe to write ClusterSnapshotRecord align with CheckpointRecord. [1] [https://cwiki.apache.org/confluence/pages/editpage.action?pageId=211884314] > Write ClusterSnapshotRecord to WAL > -- > > Key: IGNITE-17215 > URL: https://issues.apache.org/jira/browse/IGNITE-17215 > Project: Ignite > Issue Type: New Feature >Reporter: Maksim Timonin >Assignee: Maksim Timonin >Priority: Major > Labels: IEP-89 > Time Spent: 10m > Remaining Estimate: 0h > > For PITR [1] process of recovering based on ClusterSnapshot + archived WALs. > It's required to have a point in WAL which splits whole WAL on 2 areas: > # Before this point all data changes are contained within ClusterSnapshot, > and no need to recover them from WAL archived files. > # After this point all data need to be recovered from WAL archived files. > It's proposed to write ClusterSnapshotRecord while the checkpoint is running > (cp#writeLock has acquired). ClusterSnapshot process guarantees: > # there is no active transactions (or any data changes) in moment of running > checkpoint. > # ClusterSnapshot contains all data pages that will be persisted within this > checkpoint process. > Then every logical record after begin CheckointRecord doesn't belong to > ClusterSnapshot. Then it's safe to write ClusterSnapshotRecord within the > checkpoint process. > [1] > [https://cwiki.apache.org/confluence/pages/editpage.action?pageId=211884314] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17203) SQL API: Implement scale and precision metadata
[ https://issues.apache.org/jira/browse/IGNITE-17203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Orlov updated IGNITE-17203: -- Ignite Flags: (was: Docs Required,Release Notes Required) > SQL API: Implement scale and precision metadata > --- > > Key: IGNITE-17203 > URL: https://issues.apache.org/jira/browse/IGNITE-17203 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Pavel Tupitsyn >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > Time Spent: 40m > Remaining Estimate: 0h > > *scale()* and *precision()* in *ColumnMetadata* interface return incorrect > values. Implement them properly. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17203) SQL API: Implement scale and precision metadata
[ https://issues.apache.org/jira/browse/IGNITE-17203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563035#comment-17563035 ] Konstantin Orlov commented on IGNITE-17203: --- [~amashenkov], I've left a few comments. Please see PR > SQL API: Implement scale and precision metadata > --- > > Key: IGNITE-17203 > URL: https://issues.apache.org/jira/browse/IGNITE-17203 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Pavel Tupitsyn >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > Time Spent: 40m > Remaining Estimate: 0h > > *scale()* and *precision()* in *ColumnMetadata* interface return incorrect > values. Implement them properly. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17279) Mapping of partition states to nodes can erroneously skip lost partitions on the coordinator node
[
https://issues.apache.org/jira/browse/IGNITE-17279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563003#comment-17563003
]
Vladislav Pyatkov commented on IGNITE-17279:
LGTM
> Mapping of partition states to nodes can erroneously skip lost partitions on
> the coordinator node
> -
>
> Key: IGNITE-17279
> URL: https://issues.apache.org/jira/browse/IGNITE-17279
> Project: Ignite
> Issue Type: Bug
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Minor
> Fix For: 2.14
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems that a coordinator node does not correctly update node2part mapping
> for lost partitions.
> {noformat}
> [test-runner-#1%distributed.CachePartitionLostAfterSupplierHasLeftTest%][root]
> dump partitions state for :
> preload sync futures
> nodeId=b57ca812-416d-40d7-bb4f-27199490
> consistentId=distributed.CachePartitionLostAfterSupplierHasLeftTest0
> isDone=true
> nodeId=20fdfa4a-ddf6-4229-b25e-38cd8d31
> consistentId=distributed.CachePartitionLostAfterSupplierHasLeftTest1
> isDone=true
> rebalance futures
> nodeId=b57ca812-416d-40d7-bb4f-27199490 isDone=true res=true topVer=null
> remaining: {}
> nodeId=20fdfa4a-ddf6-4229-b25e-38cd8d31 isDone=true res=false
> topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0]
> remaining: {}
> partition state
> localNodeId=b57ca812-416d-40d7-bb4f-27199490
> grid=distributed.CachePartitionLostAfterSupplierHasLeftTest0
> local part=0 counters=Counter [lwm=200, missed=[], maxApplied=200, hwm=200]
> fullSize=200 *state=LOST* reservations=0 isAffNode=true
> nodeId=20fdfa4a-ddf6-4229-b25e-38cd8d31 part=0 *state=LOST*
> isAffNode=true
> ...
> localNodeId=20fdfa4a-ddf6-4229-b25e-38cd8d31
> grid=distributed.CachePartitionLostAfterSupplierHasLeftTest1
> local part=0 counters=Counter [lwm=0, missed=[], maxApplied=0, hwm=0]
> fullSize=100 *state=LOST* reservations=0 isAffNode=true
> nodeId=b57ca812-416d-40d7-bb4f-27199490 part=0 *state=OWNING*
> isAffNode=true
> ...
> {noformat}
> *Update*:
> The root cause of the issue is that the coordinator node incorrectly
> update mapping nodes to partition states on PME (see
> GridDhtPartitionTopologyImpl.node2part). It seems to me, that the coordinator
> node should set partition state to LOST on all affinity nodes (if this
> partition is assumed as LOST on the coordinator) before creating and sending
> a “full map” message.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
