Re: Real-world use cases: Ignite and High-Performance Computing
To compare the speed, is there a perftest tool for benchmarking the performance? Like this one for Rabbitmq: https://rabbitmq.github.io/rabbitmq-perf-test/stable/htmlsingle/#installation Thank you. Piper On Thu, Dec 2, 2021 at 11:52 PM Denis Magda wrote: > Folks, > > I've just published an article that lists some real-world use cases of > Ignite for high-performance computing. Some of you might be interested in > how the compute APIs are leveraged in practice: > > https://www.gridgain.com/resources/blog/how-apache-ignite-empowers-high-performance-computing-real-use-cases > > That's a short summary of recordings from past conferences and meetup > talks. Thanks to everyone who shared their stories! > > - > Denis >
Re: Real-world use cases: Ignite and High-Performance Computing
Thanks for this valuable sharing. De : "Denis Magda" A : "user" ,"dev" Envoyé: jeudi 2 Décembre 2021 23:52 Objet : Real-world use cases: Ignite and High-Performance Computing Folks, I've just published an article that lists some real-world use cases of Ignite for high-performance computing. Some of you might be interested in how the compute APIs are leveraged in practice: https://www.gridgain.com/resources/blog/how-apache-ignite-empowers-high-performance-computing-real-use-cases That's a short summary of recordings from past conferences and meetup talks. Thanks to everyone who shared their stories! - Denis
Real-world use cases: Ignite and High-Performance Computing
Folks, I've just published an article that lists some real-world use cases of Ignite for high-performance computing. Some of you might be interested in how the compute APIs are leveraged in practice: https://www.gridgain.com/resources/blog/how-apache-ignite-empowers-high-performance-computing-real-use-cases That's a short summary of recordings from past conferences and meetup talks. Thanks to everyone who shared their stories! - Denis
Re: Code Deployment configuration error, start ignite.sh
Oh! it's great. thank you Stephen. I do the cp ./libs/optional/ignite-urideploy/ignite-urideploy-2.11.0.jar ./libs/ then, start ignite.sh successfully. I spent about 1 week for this. Thank you. On 2021/12/02 18:46, Stephen Darlington wrote: Did you copy the ignite-urideploy folder from libs/optional to libs? On 1 Dec 2021, at 19:18, Hajime Kobashi wrote: Hi community. Where is the UriDeploymentSpi.class and source file ? I want something for start ignite, what is wrong. I have some issue, that the ClassNotFoundException occured spi.deployment.uri.UriDeploymentSpi class configuration to XML file with ignite.sh starting. I can not find UriDeploymentSpi class next 3 directory or jars. 1: git clone my local directory. ./ignite/modules/core/src/main/java/org/apache/ignite/spi/deployment 2: download source jar: apache-ignite-2.11.0-src.zip ./apache-ignite-2.11.0-src/modules/core/src/main/java/org/apache/ignite/spi/deployment 3: ignite.sh execution environment jar in libs: ignite-core-2.11.0.jar ./org/apache/ignite/spi/deployment this configuration refer to Code Deployment-Deploying User Code section. https://ignite.apache.org/docs/latest/code-deployment/deploying-user-code. start ignite.sh java exception. configuration XML file name is ./config/testNode1.xml. issue part. === file://freq=5000@localhost/usr/local/bin/ignite/libs === next ignite.sh starting result messages. = [root@sidious apache-ignite]# ./bin/ignite.sh ./config/testNode1.xml class org.apache.ignite.IgniteException: Failed to instantiate Spring XML application context (make sure all classes used in Spring configuration are present at CLASSPATH) [springUrl=file:/opt/apache/apache-ignite/./config/testNode1.xml] at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1098) at org.apache.ignite.Ignition.start(Ignition.java:356) at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:367) Caused by: class org.apache.ignite.IgniteCheckedException: Failed to instantiate Spring XML application context (make sure all classes used in Spring configuration are present at CLASSPATH) [springUrl=file:/opt/apache/apache-ignite/./config/testNode1.xml] at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:387) at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104) at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98) at org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:741) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:942) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:851) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:721) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:690) at org.apache.ignite.Ignition.start(Ignition.java:353) ... 1 more Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.ignite.configuration.IgniteConfiguration#0' defined in URL [file:/opt/apache/apache-ignite/./config/testNode1.xml]: Cannot create inner bean 'org.apache.ignite.spi.deployment.uri.UriDeploymentSpi#3327bd23' of type [org.apache.ignite.spi.deployment.uri.UriDeploymentSpi] while setting bean property 'deploymentSpi'; nested exception is org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [org.apache.ignite.spi.deployment.uri.UriDeploymentSpi] for bean with name 'org.apache.ignite.spi.deployment.uri.UriDeploymentSpi#3327bd23' defined in URL [file:/opt/apache/apache-ignite/./config/testNode1.xml]; nested exception is java.lang.ClassNotFoundException: org.apache.ignite.spi.deployment.uri.UriDeploymentSpi at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:313) at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:122) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1522) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1269) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:551) at
Re: Code Deployment configuration error, start ignite.sh
OS is Rocky linux 8.x / CentOS 7.4, latest updated. JDK is OpenJDK 1.8.0 Thank you yonghua. On 2021/12/02 4:36, yong...@laposte.net wrote: what’s the app env? The OS, JDK version etc. De : "Hajime Kobashi" A : user@ignite.apache.org Envoyé: jeudi 2 Décembre 2021 03:18 Objet : Code Deployment configuration error, start ignite.sh Hi community. Where is the UriDeploymentSpi.class and source file ? I want something for start ignite, what is wrong. I have some issue, that the ClassNotFoundException occured spi.deployment.uri.UriDeploymentSpi class configuration to XML file with ignite.sh starting. I can not find UriDeploymentSpi class next 3 directory or jars. 1: git clone my local directory. ./ignite/modules/core/src/main/java/org/apache/ignite/spi/deployment 2: download source jar: apache-ignite-2.11.0-src.zip ./apache-ignite-2.11.0-src/modules/core/src/main/java/org/apache/ignite/spi/deployment 3: ignite.sh execution environment jar in libs: ignite-core-2.11.0.jar ./org/apache/ignite/spi/deployment this configuration refer to Code Deployment-Deploying User Code section. https://ignite.apache.org/docs/latest/code-deployment/deploying-user-code. start ignite.sh java exception. configuration XML file name is ./config/testNode1.xml. issue part. === file://freq=5000@localhost/usr/local/bin/ignite/libs === next ignite.sh starting result messages. = [root@sidious apache-ignite]# ./bin/ignite.sh ./config/testNode1.xml class org.apache.ignite.IgniteException: Failed to instantiate Spring XML application context (make sure all classes used in Spring configuration are present at CLASSPATH) [springUrl=file:/opt/apache/apache-ignite/./config/testNode1.xml] at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1098) at org.apache.ignite.Ignition.start(Ignition.java:356) at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:367) Caused by: class org.apache.ignite.IgniteCheckedException: Failed to instantiate Spring XML application context (make sure all classes used in Spring configuration are present at CLASSPATH) [springUrl=file:/opt/apache/apache-ignite/./config/testNode1.xml] at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:387) at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104) at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98) at org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:741) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:942) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:851) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:721) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:690) at org.apache.ignite.Ignition.start(Ignition.java:353) ... 1 more Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.ignite.configuration.IgniteConfiguration#0' defined in URL [file:/opt/apache/apache-ignite/./config/testNode1.xml]: Cannot create inner bean 'org.apache.ignite.spi.deployment.uri.UriDeploymentSpi#3327bd23' of type [org.apache.ignite.spi.deployment.uri.UriDeploymentSpi] while setting bean property 'deploymentSpi'; nested exception is org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [org.apache.ignite.spi.deployment.uri.UriDeploymentSpi] for bean with name 'org.apache.ignite.spi.deployment.uri.UriDeploymentSpi#3327bd23' defined in URL [file:/opt/apache/apache-ignite/./config/testNode1.xml]; nested exception is java.lang.ClassNotFoundException: org.apache.ignite.spi.deployment.uri.UriDeploymentSpi at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:313) at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:122) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1522) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1269) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:551) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312) at
Re: Golang thin client
yes I suggest this one because I have been using it. De : "Stephen Darlington" A : user@ignite.apache.org Envoyé: jeudi 2 Décembre 2021 17:48 Objet : Re: Golang thin client There is no “official” Go thin client but I found one that someone has written: https://github.com/amsokol/ignite-go-client > On 2 Dec 2021, at 09:37, Marco Watkin wrote: > > Do you have the official library for go thin client? I saw the website > doesn’t mention that. > > Thanks
Re: Golang thin client
There is no “official” Go thin client but I found one that someone has written: https://github.com/amsokol/ignite-go-client > On 2 Dec 2021, at 09:37, Marco Watkin wrote: > > Do you have the official library for go thin client? I saw the website > doesn’t mention that. > > Thanks
Re: Code Deployment configuration error, start ignite.sh
Did you copy the ignite-urideploy folder from libs/optional to libs? > On 1 Dec 2021, at 19:18, Hajime Kobashi wrote: > > Hi community. > > Where is the UriDeploymentSpi.class and source file ? > I want something for start ignite, what is wrong. > > I have some issue, that the ClassNotFoundException occured > spi.deployment.uri.UriDeploymentSpi class configuration to XML file with > ignite.sh starting. > > I can not find UriDeploymentSpi class next 3 directory or jars. > > 1: git clone my local directory. > ./ignite/modules/core/src/main/java/org/apache/ignite/spi/deployment > 2: download source jar: apache-ignite-2.11.0-src.zip > ./apache-ignite-2.11.0-src/modules/core/src/main/java/org/apache/ignite/spi/deployment > 3: ignite.sh execution environment jar in libs: ignite-core-2.11.0.jar > ./org/apache/ignite/spi/deployment > > this configuration refer to Code Deployment-Deploying User Code section. > https://ignite.apache.org/docs/latest/code-deployment/deploying-user-code. > > start ignite.sh java exception. configuration XML file name is > ./config/testNode1.xml. issue part. > === > > > > > >file://freq=5000@localhost/usr/local/bin/ignite/libs > > > > > === > next ignite.sh starting result messages. > = > [root@sidious apache-ignite]# ./bin/ignite.sh ./config/testNode1.xml > class org.apache.ignite.IgniteException: Failed to instantiate Spring XML > application context (make sure all classes used in Spring configuration are > present at CLASSPATH) > [springUrl=file:/opt/apache/apache-ignite/./config/testNode1.xml] >at > org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1098) >at org.apache.ignite.Ignition.start(Ignition.java:356) >at > org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:367) > Caused by: class org.apache.ignite.IgniteCheckedException: Failed to > instantiate Spring XML application context (make sure all classes used in > Spring configuration are present at CLASSPATH) > [springUrl=file:/opt/apache/apache-ignite/./config/testNode1.xml] >at > org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:387) >at > org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104) >at > org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98) >at > org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:741) >at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:942) >at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:851) >at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:721) >at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:690) >at org.apache.ignite.Ignition.start(Ignition.java:353) >... 1 more > Caused by: org.springframework.beans.factory.BeanCreationException: Error > creating bean with name > 'org.apache.ignite.configuration.IgniteConfiguration#0' defined in URL > [file:/opt/apache/apache-ignite/./config/testNode1.xml]: Cannot create inner > bean 'org.apache.ignite.spi.deployment.uri.UriDeploymentSpi#3327bd23' of type > [org.apache.ignite.spi.deployment.uri.UriDeploymentSpi] while setting bean > property 'deploymentSpi'; nested exception is > org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find > class [org.apache.ignite.spi.deployment.uri.UriDeploymentSpi] for bean with > name 'org.apache.ignite.spi.deployment.uri.UriDeploymentSpi#3327bd23' defined > in URL [file:/opt/apache/apache-ignite/./config/testNode1.xml]; nested > exception is java.lang.ClassNotFoundException: > org.apache.ignite.spi.deployment.uri.UriDeploymentSpi >at > org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:313) >at > org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:122) >at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1522) >at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1269) >at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:551) >at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481) >at
Golang thin client
Do you have the official library for go thin client? I saw the website doesn’t mention that. Thanks
Re: Binary types configuration and cluster configuration consistency between server and client nodes
> we are forced to register all our business objects > changing the cluster configuration to add the new binary types I'm trying to say that you don't need to do that. Instead of listing all binary types in configuration, you can register them at runtime, after the cluster has started. On Thu, Dec 2, 2021 at 11:57 AM Rotondi, Antonio < antonio.roto...@credit-suisse.com> wrote: > Hi Tupitsyn, > > Thanks for your reply. > > > > Can you please clarify your suggestion about removing types form the > static binaryconfiguration? > > > > To clarify, all our caches store objects in binary format. The reason why > we need custom serializers is to allow conversion of basic java types like > Instant and LocalDate (which I suggest you include in the Ignite core) to > the supported Ignite binary ones.. > > I tried with a general serializer setup but that was trying to deserialize > even lambdas sent to the server nodes. On the other side, type specific > serialization configurations at nested level (e.g. an Instant inside a > custom class) does not seem to work as the process manager is only checking > for the top type when assigning serializers and when a not blanket > serializer is configured (in other words it does not seem to recurse inside > not configured types for configured types). > > In this scenario we are forced to register all our business objects > containing types we want to convert to supported binary types with our > seriailzer and every time we add a new type to the cache, on top of > dropping the new classes needed for the new cash configurations into the > server nodes and changing the cluster configuration to add the new binary > types, we also need to modify the cluster configuration for all clients, > including the ones that are not concerned with the new business types. > > Personally I would have made binary types a cache specific configuration > instead of a cluster global one. > > > > Thanks again for all the suggestions you may provide and, please, give me > a pointer to that static type clean-up advise you sent me before. > > > > Regards, > > Antonio > > > > *From:* Pavel Tupitsyn > *Sent:* 30 November 2021 13:49 > *To:* user > *Subject:* Re: Binary types configuration and cluster configuration > consistency between server and client nodes > > > > Hello Antonio, > > > > You can solve this by removing all types from the static > BinaryConfiguration, > > and rely on dynamic type registration instead. > > > > In most cases, you don't need to do anything extra and types will be > registered automatically on first use (e.g. cache.put call). > > If you still encounter an edge case where a type can't be resolved, you > can force the type registration like this: > > ignite.binary().type(MyClass.class) > > > > https://ignite.apache.org/docs/latest/data-modeling/data-modeling > > > > On Tue, Nov 30, 2021 at 4:09 PM Rotondi, Antonio < > antonio.roto...@credit-suisse.com> wrote: > > Hello, > > I have a question regarding an application we have already in production > and for which we are trying to clean as much as possible the dependencies > between server node and client node ones resources. > > In particular we are seeing that it is not possible to set the binary > types configuration in the server cluster configuration without adding that > to all client nodes cluster configuration as well. This is a rather > annoying dependency to manage as it will force the redeployment or patching > of all the client nodes when a new type is added to the cache (think > microservices) . > > I could not find a way to circumvent that as the client needs to connect > tot the gird before it can call a remote task to pull the binary types > configuration and the client will fail to start ignite with an error > stating a difference between the server and client binary type > configuration. > > This server resources dependencies in the client seems to be a general > issue across the whole Ignite architecture, especially because > configuration related ones are not included in class peer loading. > > > > Many thanks for any advise anyone can give me. > > > > Best regards, > > Antonio > > > > > > > === > Please access the attached hyperlink for an important data protection > disclaimer: > https://www.credit-suisse.com/uk/en/legal/privacy-statement.html > If you have received any marketing or product promotion communications in > this > email, you can unsubscribe from receiving such communications by > contacting the > sender of this message or your regular Credit Suisse contact. > > === > > > > > == > Where required by law, and except for FX instruments, pre-trade mid-market > mark > is the arithmetic mean of bid and offer prices unless otherwise stated; > other > regulatory disclosures are at
RE: Binary types configuration and cluster configuration consistency between server and client nodes
Hi Tupitsyn, Thanks for your reply. Can you please clarify your suggestion about removing types form the static binaryconfiguration? To clarify, all our caches store objects in binary format. The reason why we need custom serializers is to allow conversion of basic java types like Instant and LocalDate (which I suggest you include in the Ignite core) to the supported Ignite binary ones.. I tried with a general serializer setup but that was trying to deserialize even lambdas sent to the server nodes. On the other side, type specific serialization configurations at nested level (e.g. an Instant inside a custom class) does not seem to work as the process manager is only checking for the top type when assigning serializers and when a not blanket serializer is configured (in other words it does not seem to recurse inside not configured types for configured types). In this scenario we are forced to register all our business objects containing types we want to convert to supported binary types with our seriailzer and every time we add a new type to the cache, on top of dropping the new classes needed for the new cash configurations into the server nodes and changing the cluster configuration to add the new binary types, we also need to modify the cluster configuration for all clients, including the ones that are not concerned with the new business types. Personally I would have made binary types a cache specific configuration instead of a cluster global one. Thanks again for all the suggestions you may provide and, please, give me a pointer to that static type clean-up advise you sent me before. Regards, Antonio From: Pavel Tupitsyn Sent: 30 November 2021 13:49 To: user Subject: Re: Binary types configuration and cluster configuration consistency between server and client nodes Hello Antonio, You can solve this by removing all types from the static BinaryConfiguration, and rely on dynamic type registration instead. In most cases, you don't need to do anything extra and types will be registered automatically on first use (e.g. cache.put call). If you still encounter an edge case where a type can't be resolved, you can force the type registration like this: ignite.binary().type(MyClass.class) https://ignite.apache.org/docs/latest/data-modeling/data-modeling On Tue, Nov 30, 2021 at 4:09 PM Rotondi, Antonio mailto:antonio.roto...@credit-suisse.com>> wrote: Hello, I have a question regarding an application we have already in production and for which we are trying to clean as much as possible the dependencies between server node and client node ones resources. In particular we are seeing that it is not possible to set the binary types configuration in the server cluster configuration without adding that to all client nodes cluster configuration as well. This is a rather annoying dependency to manage as it will force the redeployment or patching of all the client nodes when a new type is added to the cache (think microservices) . I could not find a way to circumvent that as the client needs to connect tot the gird before it can call a remote task to pull the binary types configuration and the client will fail to start ignite with an error stating a difference between the server and client binary type configuration. This server resources dependencies in the client seems to be a general issue across the whole Ignite architecture, especially because configuration related ones are not included in class peer loading. Many thanks for any advise anyone can give me. Best regards, Antonio === Please access the attached hyperlink for an important data protection disclaimer: https://www.credit-suisse.com/uk/en/legal/privacy-statement.html If you have received any marketing or product promotion communications in this email, you can unsubscribe from receiving such communications by contacting the sender of this message or your regular Credit Suisse contact. === == Where required by law, and except for FX instruments, pre-trade mid-market mark is the arithmetic mean of bid and offer prices unless otherwise stated; other regulatory disclosures are at https://plus.credit-suisse.com == == Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html == === Please access the attached hyperlink for an important data protection disclaimer:
