[jira] [Created] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
swy created IGNITE-12374: Summary: Too low performance ~200TPS for single ODBC client Key: IGNITE-12374 URL: https://issues.apache.org/jira/browse/IGNITE-12374 Project: Ignite Issue Type: Improvement Components: clients, odbc Affects Versions: 2.7.5 Environment: Ignite server run on top of Kubernetes, with 2 server nodes, persistence enabled. Both CPU and RAM at server/client server is sufficient according to system reports. Reporter: swy Attachments: odbcsample.cc Hi, in our test ignite performance with ODBC connection is too bad to proceed with product integration. It is about ~200 TPS, each transaction with select+update operation. Please refer to attach sample program. It is just a simple test case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12373) Handshake timeout while opening a channel connection from different nodes
Maxim Muzafarov created IGNITE-12373:
Summary: Handshake timeout while opening a channel connection from
different nodes
Key: IGNITE-12373
URL: https://issues.apache.org/jira/browse/IGNITE-12373
Project: Ignite
Issue Type: Bug
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov
Steps to reproduce,
1. Open transmission sender on the node1
2. Open transmission sender on the node2
3. Simultaneously send a file from node1 to node2 and from node2 to node1.
{code:java}
[2019-11-14
22:37:41,685][ERROR][async-runnable-runner-1][GridIoManagerFileTransmissionSelfTest$BlockingOpenChannelCommunicationSpi]
Failed to read from channel:
java.nio.channels.AsynchronousCloseException
at
java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:205)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:407)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeTcpHandshake(TcpCommunicationSpi.java:3999)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioSession(TcpCommunicationSpi.java:3536)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.openChannel(TcpCommunicationSpi.java:4384)
at
org.apache.ignite.internal.managers.communication.GridIoManagerFileTransmissionSelfTest$BlockingOpenChannelCommunicationSpi.openChannel(GridIoManagerFileTransmissionSelfTest.java:1068)
at
org.apache.ignite.internal.managers.communication.GridIoManager.openChannel(GridIoManager.java:1959)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$6500(GridIoManager.java:229)
at
org.apache.ignite.internal.managers.communication.GridIoManager$TransmissionSender.connect(GridIoManager.java:3145)
at
org.apache.ignite.internal.managers.communication.GridIoManager$TransmissionSender.send(GridIoManager.java:3244)
at
org.apache.ignite.internal.managers.communication.GridIoManager$TransmissionSender.send(GridIoManager.java:3194)
at
org.apache.ignite.internal.managers.communication.GridIoManagerFileTransmissionSelfTest.lambda$testFileHandlerCrossConnections$1(GridIoManagerFileTransmissionSelfTest.java:314)
at
org.apache.ignite.testframework.GridTestUtils.lambda$runAsync$1(GridTestUtils.java:1039)
at
org.apache.ignite.testframework.GridTestUtils.lambda$runAsync$2(GridTestUtils.java:1095)
at
org.apache.ignite.testframework.GridTestUtils$7.call(GridTestUtils.java:1416)
at
org.apache.ignite.testframework.GridTestThread.run(GridTestThread.java:84)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[MTCGA]: new failures in builds [4765969] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. *New stable failure of a flaky test in master IgniteConfigurationTest.TestAllConfigurationProperties https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-7181944984333835627=%3Cdefault%3E=testDetails No changes in the build - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 21:08:56 14-11-2019
Collocated/replicatedOnly flags for Thin JDBC driver
Hi, Igniters During testing Thin JDBC Driver I found some interesting things that would be good to discuss: collocated flag == The flag helps to optimize query against collocated data in advance. We've following: 1. Simple queries against single tables return weird results [3]. It means that either a connection has reset every switch collocated/non-collocated requests or make two connections for collocated and collocated requests respectively. 2. *distrubitedJoins *flag mostly covers the cases: - if it is true we don't concern how the data distributed over cluster - if not we always operate local data no matter it's collocated or not 3. There was an activity to remove it but it was cancelled [2]. replicatedOnly flag === The flag helps to optimize query against replicated tables (caches) and query always operates local data. 1. But why can't we find at preparing the request for execution? 2. For PRIMARY_SYNC cache mode using local node data may lead to inconsistent results. Thus it may be implemented as an explicit hint for a query if user want to do that and accept the possible risks or just re-use *distrubitedJoins=false*. 3. Same concern that it used per JDBC connection and required reconnect for change the flag. I guess both flags should be deprecated and removed (2.8?). Ideally *distrubitedJoins *should be removed as well and replaced by query hints (default *distrubitedJoins=true*) 1. https://apacheignite-sql.readme.io/docs/jdbc-driver 2. https://issues.apache.org/jira/browse/IGNITE-6296 3. https://issues.apache.org/jira/browse/IGNITE-12372 -- Sergey Kozlov GridGain Systems www.gridgain.com
[jira] [Created] (IGNITE-12372) Thin Jdbc driver with collocated=true return per node results
Sergey Kozlov created IGNITE-12372:
--
Summary: Thin Jdbc driver with collocated=true return per node
results
Key: IGNITE-12372
URL: https://issues.apache.org/jira/browse/IGNITE-12372
Project: Ignite
Issue Type: Bug
Components: sql
Affects Versions: 2.7.6
Reporter: Sergey Kozlov
# Start two nodes (in-memory)
# Run sqlline \{{>bin\sqlline.bat -d org.apache.ignite.IgniteJdbcThinDriver -u
jdbc:ignite:thin://127.0.0.1/?collocated=true}}
# Create a table and select aggregation functions:
{noformat}
: jdbc:ignite:thin://127.0.0.1/> create table t1 (a int not null primary key, b
varchar) with "template=partitioned";
No rows affected (0,15 seconds)
0: jdbc:ignite:thin://127.0.0.1/> insert into t1 values (1, 'a'), (2, 'b'), (3,
'c'), (4, 'd');
4 rows affected (0,091 seconds)
0: jdbc:ignite:thin://127.0.0.1/> select * from t1;
+++
| A | B |
+++
| 1 | a |
| 2 | b |
| 4 | d |
| 3 | c |
+++
4 rows selected (0,074 seconds)
0: jdbc:ignite:thin://127.0.0.1/> select count(*) from t1;
++
| COUNT(*) |
++
| 3 |
| 1 |
++
2 rows selected (0,011 seconds)
0: jdbc:ignite:thin://127.0.0.1/> select min(a), max(b) from t1;
+++
| MIN(A) | MAX(B) |
+++
| 1 | d |
| 3 | c |
+++
2 rows selected (0,012 seconds)
0: jdbc:ignite:thin://127.0.0.1/> select min(a), max(a) from t1;
+++
| MIN(A) | MAX(A) |
+++
| 1 | 4 |
| 3 | 3 |
+++
2 rows selected (0,01 seconds)
0: jdbc:ignite:thin://127.0.0.1/>
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12371) Explicit method for starting client nodes
Nikolay Izhikov created IGNITE-12371:
Summary: Explicit method for starting client nodes
Key: IGNITE-12371
URL: https://issues.apache.org/jira/browse/IGNITE-12371
Project: Ignite
Issue Type: Bug
Affects Versions: 2.7.6
Reporter: Nikolay Izhikov
Right now there is almost 500 explicit usage of {{setClientMode}} in tests.
Seems we should support the starting of client nodes in test framework.
We should introduce method {{startClientNode(String name)}} and similar.
This will simplify tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12370) WAL history reservation may fail due to an incorrect determination of the availability of the WAL segment index (under race condition).
Pavel Pereslegin created IGNITE-12370:
-
Summary: WAL history reservation may fail due to an incorrect
determination of the availability of the WAL segment index (under race
condition).
Key: IGNITE-12370
URL: https://issues.apache.org/jira/browse/IGNITE-12370
Project: Ignite
Issue Type: Bug
Reporter: Pavel Pereslegin
For now, {{FileWriteAheadLogManager#hasIndex}} firstly determines that the WAL
segment exists in an archive ({{File.exists}}) and then determines that index
was in the archive (using {{Files.list}}). If the archive file was created
between these operations {{hasIndex}} will return the false-negative result and
the partition map exchange will fail on this node.
Reproducer:
{code:java}
public class IgniteWalHistoryReservationsWithLoadTest extends
GridCommonAbstractTest {
/** {@inheritDoc} */
@Override protected IgniteConfiguration getConfiguration(String gridName)
throws Exception {
IgniteConfiguration cfg = super.getConfiguration(gridName);
cfg.setConsistentId("NODE$" + gridName.charAt(gridName.length() - 1));
DataStorageConfiguration memCfg = new DataStorageConfiguration()
.setDefaultDataRegionConfiguration(
new DataRegionConfiguration()
.setMaxSize(200L * 1024 * 1024)
.setPersistenceEnabled(true))
.setWalMode(WALMode.LOG_ONLY)
.setWalSegmentSize(512 * 1024)
.setCheckpointFrequency(500);
cfg.setDataStorageConfiguration(memCfg);
CacheConfiguration ccfg1 = new CacheConfiguration();
ccfg1.setName("cache1");
ccfg1.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
ccfg1.setAffinity(new RendezvousAffinityFunction(false, 32));
cfg.setCacheConfiguration(ccfg1);
return cfg;
}
@Test
public void testReservationWithConstantLoad() throws Exception {
final IgniteEx node = startGrid(0);
node.cluster().active(true);
AtomicLong cntr = new AtomicLong(100_000);
ConstantLoader ldr = new ConstantLoader(node.cache("cache1"), cntr);
IgniteInternalFuture fut =
GridTestUtils.runMultiThreadedAsync(ldr, 1, "loader");
U.sleep(500);
forceCheckpoint(node);
// Reserve history from the beginning.
node.context().cache().context().database().reserveHistoryForExchange();
long endTime = U.currentTimeMillis() + 60_000;
GridCacheContext ctx = node.cachex("cache1").context();
int grpId = ctx.groupId();
int parts = ctx.topology().partitions();
try {
while (U.currentTimeMillis() < endTime &&
!Thread.currentThread().isInterrupted()) {
try {
for (int p = 0; p < parts; p++) {
boolean reserved =
node.context().cache().context().database().reserveHistoryForPreloading(grpId,
p, cntr.get());
assertTrue("Unable to reserve history [p=" + p + ",
cntr=" + cntr.get() + "]", reserved);
}
} finally {
node.context().cache().context().database().releaseHistoryForPreloading();
}
}
}
finally {
node.context().cache().context().database().releaseHistoryForExchange();
ldr.stop();
}
fut.get(10_000);
}
static class ConstantLoader implements Callable {
private final IgniteCache cache;
private final AtomicLong cntr;
private volatile boolean stop;
ConstantLoader(IgniteCache cache, AtomicLong cntr) {
this.cache = cache;
this.cntr = cntr;
}
@Override public Void call() throws Exception {
while (!stop && !Thread.currentThread().isInterrupted()) {
long n = cntr.getAndIncrement();
cache.put(n, n);
if (n % 100_000 == 0)
log.info("Loaded " + n);
}
return null;
}
public void stop() {
stop = true;
}
}
/** {@inheritDoc} */
@Override protected void beforeTestsStarted() throws Exception {
stopAllGrids();
cleanPersistenceDir();
}
/** {@inheritDoc} */
@Override protected void afterTest() throws Exception {
stopAllGrids();
cleanPersistenceDir();
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12369) JdbcThinClient and Server compatibility broken
Konstantin Orlov created IGNITE-12369:
-
Summary: JdbcThinClient and Server compatibility broken
Key: IGNITE-12369
URL: https://issues.apache.org/jira/browse/IGNITE-12369
Project: Ignite
Issue Type: Bug
Components: sql
Affects Versions: mas
Reporter: Konstantin Orlov
Fix For: mas
Query cancellation support was introduced in IGNITE-5439. From this time server
expects that ALL request have id. It breaks compatibility with clients whose
version prior to 2.8.0.
Furthermore backport of this feature to 8.7 breaks compatibility newer build of
server with older build of client.
See {{ClientListenerProcessor#makeFilters}}. New filter with overrided
_onMessageReceived_ is created here. It tries to read request id and if there
is not enough data to read long value, it crashes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
