malte-f19 opened a new issue, #12639:
URL: https://github.com/apache/ignite/issues/12639
We have an issue using Ignite 3 from multiple services. Often data is not
available although it should be.
# TL;DR
- A fresh cluster with three nodes running
- A table with a combined primary key and some data column
- Two applications: One (the sender) is writing to Ignite, the second one
(the receiver) is reading from Ignite.
- After the sender stored multiple rows with the same `id` but with
different sequence numbers, it sends a REST request with the id to the receiver.
- The receiver then tries to select data by the id and doesn't get any
records back.
# Setup
This is the SQL we've run in Ignite:
```sql
CREATE ZONE my_zone (PARTITIONS 25, REPLICAS 1, AUTO SCALE UP 10, AUTO SCALE
DOWN 600) STORAGE PROFILES ['memStorageProfile'];
CREATE TABLE IF NOT EXISTS data_table(
id VARCHAR(50) NOT NULL,
sequence INT NOT NULL,
data VARCHAR(99999) NOT NULL,
PRIMARY KEY (id, sequence)
) ZONE my_zone STORAGE PROFILE 'memStorageProfile';
```
The corresponding entity class is defined like this:
```java
@Table("data_table")
public class DataTable {
@Id
@Column(value = "id", length = 50, nullable = false)
private String id;
@Id
@Column(value = "sequence", nullable = false)
private int sequence;
@Column(value = "data", nullable = false)
private String data;
// Constructors, getters and setters omitted.
}
```
The sender uses the following code to store data into Ignite:
```java
try (IgniteClient igniteClient = IgniteClient
.builder()
.addresses("127.0.0.1:10800", "127.0.0.1:10801", "127.0.0.1:10802")
.build()) {
Table dataTable = igniteClient.tables().table("data_table");
RecordView<DataTable> recordView = dataTable.recordView(DataTable.class);
String id = UUID.randomUUID().toString();
List<DataTable> data = new ArrayList<>();
for (int i = 0; i < 5; i++) {
data.add(new DataTable(id, i, "data-" + i));
}
recordView.upsertAll(null, data);
// Send rest call to receiver
}
```
The receiver is using the following code to get the data:
```java
IgniteClient igniteClient = IgniteClient
.builder()
.addresses("127.0.0.1:10800", "127.0.0.1:10801", "127.0.0.1:10802")
.build();
Table dataTable = igniteClient.tables().table("data_table");
RecordView<DataTable> recordView = dataTable.recordView(DataTable.class);
// Wait to receive the REST call with the id...
Cursor<DataTable> cursor = recordView.query(null, columnValue("id",
equalTo(id)))) {
while (cursor.hasNext()) {
// Do something
}
```
# Actual results
When receiving the REST call (which is sent *after* the `upsertAll` call in
our sender returned) we don't receive any data when running `query`, ie the
cursor is empty.
I added some retry logic to figure out if data will be available after some
time. Here is an example output of that:
```
Data not yet available, waiting 100 ms...
Data not yet available, waiting 200 ms...
Data retrieved successfully after 2 cycles and 300 ms
```
So, it took between 100 and 300ms in this example after the REST request was
received on the receiver side until the data was actually readable. Note it
sometimes takes longer.
# Expected results
After the `upsertAll` returned, other services can read the data.
# Things we tried
We tried different things to get around this, with no success:
- Wrap the `upsertAll` in a transaction.
- Use a read-only transaction when reading the data.
- Immediately after writing the data, we read them in the sender using the
exact same code as the receiver and successfully get the newly inserted data
back.
- Instead of a cluster with three nodes, we created a cluster with a single
node.
- Replace aimem with aipersist.
Why is that happening and how can we get around this? Any hint is much
appreciated. I can also provide the minimal example in case that is helpful.
Thanks in advance!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
