[jira] [Commented] (IGNITE-11531) Merge concurrent registrations of the same binary type
[
https://issues.apache.org/jira/browse/IGNITE-11531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16798146#comment-16798146
]
Mikhail Cherkasov commented on IGNITE-11531:
[~akalashnikov] BinaryTypeRegistrationTest#shouldSendOnlyOneMetadataMessage
test fails periodicly, could you please take a look on it:
java.lang.AssertionError:
Expected :1
Actual :2
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.ignite.testframework.junits.JUnitAssertAware.assertEquals(JUnitAssertAware.java:94)
at
org.apache.ignite.internal.processors.cache.BinaryTypeRegistrationTest.shouldSendOnlyOneMetadataMessage(BinaryTypeRegistrationTest.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2102)
at java.lang.Thread.run(Thread.java:748)
> Merge concurrent registrations of the same binary type
> --
>
> Key: IGNITE-11531
> URL: https://issues.apache.org/jira/browse/IGNITE-11531
> Project: Ignite
> Issue Type: Improvement
> Components: binary
>Reporter: Denis Mekhanikov
>Assignee: Anton Kalashnikov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When a binary type is registered multiple times simultaneously, then a lot of
> type versions are generated with the same schema. It leads to long binary
> type registration especially on big topologies.
> The following code sample demonstrates the problem:
> {code:java}
> public class LongRegistration {
> public static void main(String[] args) throws InterruptedException {
> Ignite ignite = Ignition.start(igniteConfig());
> int threadsNum = 50;
> ExecutorService exec = Executors.newFixedThreadPool(threadsNum);
> CyclicBarrier barrier = new CyclicBarrier(threadsNum);
> long startTime = System.currentTimeMillis();
> // register(ignite);
> for (int i = 0; i < threadsNum; i++)
> exec.submit(new TypeRegistrator(ignite, barrier));
> exec.shutdown();
> exec.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS);
> System.out.println("Total registration time: " +
> (System.currentTimeMillis() - startTime));
> }
> private static IgniteConfiguration igniteConfig() {
> IgniteConfiguration igniteCfg = new IgniteConfiguration();
> TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
>
> ipFinder.setAddresses(Collections.singletonList("127.0.0.1:47500..47509"));
> TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi();
> discoverySpi.setLocalAddress("127.0.0.1");
> discoverySpi.setLocalPort(47500);
> discoverySpi.setIpFinder(ipFinder);
> igniteCfg.setDiscoverySpi(discoverySpi);
> return igniteCfg;
> }
> private static void register(Ignite ignite) {
> long startTime = System.currentTimeMillis();
> IgniteBinary binary = ignite.binary();
> BinaryObjectBuilder builder = binary.builder("TestType");
> builder.setField("intField", 1);
> builder.build();
> System.out.println("Registration time: " +
> (System.currentTimeMillis() - startTime));
> }
> private static class TypeRegistrator implements Runnable {
> private Ignite ignite;
> private CyclicBarrier cyclicBarrier;
> TypeRegistrator(Ignite ignite, CyclicBarrier cyclicBarrier) {
> this.ignite = ignite;
> this.cyclicBarrier = cyclicBarrier;
> }
> @Override public void run() {
> try {
> cyclicBarrier.await();
> register(ignite);
> } catch (InterruptedException | BrokenBarrierException e) {
> e.printStackTrace();
> }
> }
>
[jira] [Commented] (IGNITE-11531) Merge concurrent registrations of the same binary type
[
https://issues.apache.org/jira/browse/IGNITE-11531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793527#comment-16793527
]
Anton Kalashnikov commented on IGNITE-11531:
[~agoncharuk] thanks for review. I have added test such you describing.
> Merge concurrent registrations of the same binary type
> --
>
> Key: IGNITE-11531
> URL: https://issues.apache.org/jira/browse/IGNITE-11531
> Project: Ignite
> Issue Type: Improvement
> Components: binary
>Reporter: Denis Mekhanikov
>Assignee: Anton Kalashnikov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When a binary type is registered multiple times simultaneously, then a lot of
> type versions are generated with the same schema. It leads to long binary
> type registration especially on big topologies.
> The following code sample demonstrates the problem:
> {code:java}
> public class LongRegistration {
> public static void main(String[] args) throws InterruptedException {
> Ignite ignite = Ignition.start(igniteConfig());
> int threadsNum = 50;
> ExecutorService exec = Executors.newFixedThreadPool(threadsNum);
> CyclicBarrier barrier = new CyclicBarrier(threadsNum);
> long startTime = System.currentTimeMillis();
> // register(ignite);
> for (int i = 0; i < threadsNum; i++)
> exec.submit(new TypeRegistrator(ignite, barrier));
> exec.shutdown();
> exec.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS);
> System.out.println("Total registration time: " +
> (System.currentTimeMillis() - startTime));
> }
> private static IgniteConfiguration igniteConfig() {
> IgniteConfiguration igniteCfg = new IgniteConfiguration();
> TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
>
> ipFinder.setAddresses(Collections.singletonList("127.0.0.1:47500..47509"));
> TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi();
> discoverySpi.setLocalAddress("127.0.0.1");
> discoverySpi.setLocalPort(47500);
> discoverySpi.setIpFinder(ipFinder);
> igniteCfg.setDiscoverySpi(discoverySpi);
> return igniteCfg;
> }
> private static void register(Ignite ignite) {
> long startTime = System.currentTimeMillis();
> IgniteBinary binary = ignite.binary();
> BinaryObjectBuilder builder = binary.builder("TestType");
> builder.setField("intField", 1);
> builder.build();
> System.out.println("Registration time: " +
> (System.currentTimeMillis() - startTime));
> }
> private static class TypeRegistrator implements Runnable {
> private Ignite ignite;
> private CyclicBarrier cyclicBarrier;
> TypeRegistrator(Ignite ignite, CyclicBarrier cyclicBarrier) {
> this.ignite = ignite;
> this.cyclicBarrier = cyclicBarrier;
> }
> @Override public void run() {
> try {
> cyclicBarrier.await();
> register(ignite);
> } catch (InterruptedException | BrokenBarrierException e) {
> e.printStackTrace();
> }
> }
> }
> }
> {code}
> This code sample leads to registration of 50 versions of the same type. The
> effect is more noticeable if a cluster contains a lot of nodes.
> If you uncomment the call to {{register()}} method, then overall registration
> becomes 10 times faster on topology of 5 nodes.
> Registration of matching types should be merged to avoid long processing of
> such cases.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11531) Merge concurrent registrations of the same binary type
[
https://issues.apache.org/jira/browse/IGNITE-11531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793467#comment-16793467
]
Alexey Goncharuk commented on IGNITE-11531:
---
[~akalashnikov] the code looks good, please add another test which will
register multiple schemas in multiple threads for the same type ID and then
check that the number of propose messages is equal to the number of schemas.
> Merge concurrent registrations of the same binary type
> --
>
> Key: IGNITE-11531
> URL: https://issues.apache.org/jira/browse/IGNITE-11531
> Project: Ignite
> Issue Type: Improvement
> Components: binary
>Reporter: Denis Mekhanikov
>Assignee: Anton Kalashnikov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When a binary type is registered multiple times simultaneously, then a lot of
> type versions are generated with the same schema. It leads to long binary
> type registration especially on big topologies.
> The following code sample demonstrates the problem:
> {code:java}
> public class LongRegistration {
> public static void main(String[] args) throws InterruptedException {
> Ignite ignite = Ignition.start(igniteConfig());
> int threadsNum = 50;
> ExecutorService exec = Executors.newFixedThreadPool(threadsNum);
> CyclicBarrier barrier = new CyclicBarrier(threadsNum);
> long startTime = System.currentTimeMillis();
> // register(ignite);
> for (int i = 0; i < threadsNum; i++)
> exec.submit(new TypeRegistrator(ignite, barrier));
> exec.shutdown();
> exec.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS);
> System.out.println("Total registration time: " +
> (System.currentTimeMillis() - startTime));
> }
> private static IgniteConfiguration igniteConfig() {
> IgniteConfiguration igniteCfg = new IgniteConfiguration();
> TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
>
> ipFinder.setAddresses(Collections.singletonList("127.0.0.1:47500..47509"));
> TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi();
> discoverySpi.setLocalAddress("127.0.0.1");
> discoverySpi.setLocalPort(47500);
> discoverySpi.setIpFinder(ipFinder);
> igniteCfg.setDiscoverySpi(discoverySpi);
> return igniteCfg;
> }
> private static void register(Ignite ignite) {
> long startTime = System.currentTimeMillis();
> IgniteBinary binary = ignite.binary();
> BinaryObjectBuilder builder = binary.builder("TestType");
> builder.setField("intField", 1);
> builder.build();
> System.out.println("Registration time: " +
> (System.currentTimeMillis() - startTime));
> }
> private static class TypeRegistrator implements Runnable {
> private Ignite ignite;
> private CyclicBarrier cyclicBarrier;
> TypeRegistrator(Ignite ignite, CyclicBarrier cyclicBarrier) {
> this.ignite = ignite;
> this.cyclicBarrier = cyclicBarrier;
> }
> @Override public void run() {
> try {
> cyclicBarrier.await();
> register(ignite);
> } catch (InterruptedException | BrokenBarrierException e) {
> e.printStackTrace();
> }
> }
> }
> }
> {code}
> This code sample leads to registration of 50 versions of the same type. The
> effect is more noticeable if a cluster contains a lot of nodes.
> If you uncomment the call to {{register()}} method, then overall registration
> becomes 10 times faster on topology of 5 nodes.
> Registration of matching types should be merged to avoid long processing of
> such cases.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11531) Merge concurrent registrations of the same binary type
[
https://issues.apache.org/jira/browse/IGNITE-11531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793460#comment-16793460
]
Ignite TC Bot commented on IGNITE-11531:
{panel:title=-- Run :: All: No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=3313914buildTypeId=IgniteTests24Java8_RunAll]
> Merge concurrent registrations of the same binary type
> --
>
> Key: IGNITE-11531
> URL: https://issues.apache.org/jira/browse/IGNITE-11531
> Project: Ignite
> Issue Type: Improvement
> Components: binary
>Reporter: Denis Mekhanikov
>Assignee: Anton Kalashnikov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When a binary type is registered multiple times simultaneously, then a lot of
> type versions are generated with the same schema. It leads to long binary
> type registration especially on big topologies.
> The following code sample demonstrates the problem:
> {code:java}
> public class LongRegistration {
> public static void main(String[] args) throws InterruptedException {
> Ignite ignite = Ignition.start(igniteConfig());
> int threadsNum = 50;
> ExecutorService exec = Executors.newFixedThreadPool(threadsNum);
> CyclicBarrier barrier = new CyclicBarrier(threadsNum);
> long startTime = System.currentTimeMillis();
> // register(ignite);
> for (int i = 0; i < threadsNum; i++)
> exec.submit(new TypeRegistrator(ignite, barrier));
> exec.shutdown();
> exec.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS);
> System.out.println("Total registration time: " +
> (System.currentTimeMillis() - startTime));
> }
> private static IgniteConfiguration igniteConfig() {
> IgniteConfiguration igniteCfg = new IgniteConfiguration();
> TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
>
> ipFinder.setAddresses(Collections.singletonList("127.0.0.1:47500..47509"));
> TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi();
> discoverySpi.setLocalAddress("127.0.0.1");
> discoverySpi.setLocalPort(47500);
> discoverySpi.setIpFinder(ipFinder);
> igniteCfg.setDiscoverySpi(discoverySpi);
> return igniteCfg;
> }
> private static void register(Ignite ignite) {
> long startTime = System.currentTimeMillis();
> IgniteBinary binary = ignite.binary();
> BinaryObjectBuilder builder = binary.builder("TestType");
> builder.setField("intField", 1);
> builder.build();
> System.out.println("Registration time: " +
> (System.currentTimeMillis() - startTime));
> }
> private static class TypeRegistrator implements Runnable {
> private Ignite ignite;
> private CyclicBarrier cyclicBarrier;
> TypeRegistrator(Ignite ignite, CyclicBarrier cyclicBarrier) {
> this.ignite = ignite;
> this.cyclicBarrier = cyclicBarrier;
> }
> @Override public void run() {
> try {
> cyclicBarrier.await();
> register(ignite);
> } catch (InterruptedException | BrokenBarrierException e) {
> e.printStackTrace();
> }
> }
> }
> }
> {code}
> This code sample leads to registration of 50 versions of the same type. The
> effect is more noticeable if a cluster contains a lot of nodes.
> If you uncomment the call to {{register()}} method, then overall registration
> becomes 10 times faster on topology of 5 nodes.
> Registration of matching types should be merged to avoid long processing of
> such cases.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
