Re: Getting UnknownHostException: kubernetes.default.svc.cluster.local when using hostNetwork

[camer314]

I found a solution to this which you might want to update your documentation
with.

The answer is to add a dnsPolicy for the pod along with hostNetwork, as per
below:

spec:
  hostNetwork: true
  dnsPolicy: ClusterFirstWithHostNet
  serviceAccountName: ignite
  containers:
# Custom Ignite pod name.
  - name: ignite-node



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Getting UnknownHostException: kubernetes.default.svc.cluster.local when using hostNetwork

[camer314]

Hello,

I have setup a Kubernetes environment with auto discovery and the nodes are
able to find one another no problem. I did have several issues along the way
which this forum has helped resolve, including some modifications to YAML
that differ from those from the Ignite docs.

However, now when I set /hostNetwork: true/, I am getting

*UnknownHostException: kubernetes.default.svc.cluster.local*

If i remove the hostNetwork condition then it all starts to work again.

What is so different about this setting? I thought this was something
standard in Kubernetes/Docker?

This is the fragment of my deployment that is relevant:

spec:
  #hostNetwork: true
  serviceAccountName: ignite
  containers:
# Custom Ignite pod name.
  - name: ignite-node
image: apacheignite/ignite:2.4.0



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Getting Spring XML exception when using Ignite .NET with Kubernetes

[camer314]

I fixed this by following
http://apache-ignite-users.70518.x6.nabble.com/ignite-kubernetes-seems-to-be-missing-the-jackson-annotations-dependency-td25670.html

(Copying a missing JAR)

This gets past that error and now I am left with the following but thats
probably more environmental...

class org.apache.ignite.spi.IgniteSpiException: Failed to retrieve Ignite
pods IP addresses.
at
org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:172)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(TcpDiscoverySpi.java:1900)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.resolvedAddresses(TcpDiscoverySpi.java:1848)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1049)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:910)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:391)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2020)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
at
org.apache.ignite.internal.processors.platform.PlatformAbstractBootstrap.start(PlatformAbstractBootstrap.java:43)
at
org.apache.ignite.internal.processors.platform.PlatformIgnition.start(PlatformIgnition.java:75)
Caused by: java.net.UnknownHostException:
kubernetes.default.svc.cluster.local



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Getting Spring XML exception when using Ignite .NET with Kubernetes

[camer314]

Pruning the list down to the bin distribution gets past that error but now I
am getting:

Unhandled Exception: Apache.Ignite.Core.Common.IgniteException: Java class
is not found (did you set IGNITE_HOME environment variable?):
com/fasterxml/jackson/annotation/JsonView --->
Apache.Ignite.Core.Common.JavaException: java.lang.NoClassDefFoundError:
com/fasterxml/jackson/annotation/JsonView
at
com.fasterxml.jackson.databind.introspect.JacksonAnnotationIntrospector.(JacksonAnnotationIntrospector.java:37)
at
com.fasterxml.jackson.databind.ObjectMapper.(ObjectMapper.java:291)
at
org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:151)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(TcpDiscoverySpi.java:1900)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.resolvedAddresses(TcpDiscoverySpi.java:1848)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1049)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:910)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:391)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2020)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
at
org.apache.ignite.internal.processors.platform.PlatformAbstractBootstrap.start(PlatformAbstractBootstrap.java:43)
at
org.apache.ignite.internal.processors.platform.PlatformIgnition.start(PlatformIgnition.java:75)
Caused by: java.lang.ClassNotFoundException:
com.fasterxml.jackson.annotation.JsonView
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 19 more

   at Apache.Ignite.Core.Impl.Unmanaged.Jni.Env.ExceptionCheck()
   at Apache.Ignite.Core.Impl.Unmanaged.UnmanagedUtils.IgnitionStart(Env
env, String cfgPath, String gridName, Boolean clientMode, Boolean
userLogger, Int64 igniteId, Boolean redirectConsole)
   at Apache.Ignite.Core.Ignition.Start(IgniteConfiguration cfg)
   --- End of inner exception stack trace ---
   at Apache.Ignite.Core.Ignition.Start(IgniteConfiguration cfg)
   at Apache.Ignite.Docker.Program.Main() in /app/Program.cs:line 41




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Getting Spring XML exception when using Ignite .NET with Kubernetes

[camer314]

Ah ok, let me try that.

I was using your Docker project 
https://github.com/ptupitsyn/ignite-net-docker
  . Seems like the nuget
package has different versions bundled together.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Getting Spring XML exception when using Ignite .NET with Kubernetes

[camer314]

 No problem,

This is everything that is logged:

at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:869)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
at
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:381)
... 6 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error
creating bean with name
'org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi#eb21112' defined in URL
[https://wtwdeeplearning.blob.core.windows.net/ignite/spring_config.xml]:
Cannot create inner bean
'org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder#1786f9d5'
of type
[org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder]
while setting bean property 'ipFinder'; nested exception is
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name
'org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder#1786f9d5'
defined in URL
[https://wtwdeeplearning.blob.core.windows.net/ignite/spring_config.xml]:
Initialization of bean failed; nested exception is
java.lang.NullPointerException
at
org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:313)
at
org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:122)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1537)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1284)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
at
org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:299)
... 19 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error
creating bean with name
'org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder#1786f9d5'
defined in URL
[https://wtwdeeplearning.blob.core.windows.net/ignite/spring_config.xml]:
Initialization of bean failed; nested exception is
java.lang.NullPointerException
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:564)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
at
org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:299)
... 25 more
Caused by: java.lang.NullPointerException
at
org.springframework.core.BridgeMethodResolver.findBridgedMethod(BridgeMethodResolver.java:60)
at
org.springframework.beans.GenericTypeAwarePropertyDescriptor.(GenericTypeAwarePropertyDescriptor.java:70)
at
org.springframework.beans.CachedIntrospectionResults.buildGenericTypeAwarePropertyDescriptor(CachedIntrospectionResults.java:366)
at
org.springframework.beans.CachedIntrospectionResults.(CachedIntrospectionResults.java:302)
at
org.springframework.beans.CachedIntrospectionResults.forClass(CachedIntrospectionResults.java:189)
at
org.springframework.beans.BeanWrapperImpl.getCachedIntrospectionResults(BeanWrapperImpl.java:173)
at
org.springframework.beans.BeanWrapperImpl.getLocalPropertyHandler(BeanWrapperImpl.java:226)
at
org.springframework.beans.BeanWrapperImpl.getLocalPropertyHandler(BeanWrapperImpl.java:63)
at
org.springframework.beans.AbstractNestablePropertyAccessor.getPropertyHandler(AbstractNestablePropertyAccessor.java:737)
at
org.springframework.beans.AbstractNestablePropertyAccessor.isWritableProperty(AbstractNestablePropertyAccessor.java:569)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1539)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1284)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
... 27 more
   at 

Re: Getting Spring XML exception when using Ignite .NET with Kubernetes

[camer314]

I figured that I needed the optional kubernetes JARS which are part of the
binary distribution but not he Nuget package, so I added the
ignite-kubernetes libs to my docker image as well, now I get a different
error of null pointer:

Caused by: org.springframework.beans.factory.BeanCreationException: Error
creating bean with name
'org.apache.ignite.configuration.IgniteConfiguration#0' defined in URL
[https://wtwdeeplearning.blob.core.windows.net/ignite/spring_config.xml]:
Cannot create inner bean
'org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi#6eceb130' of type
[org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi] while setting bean
property 'discoverySpi'; nested exception is
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi#6eceb130'
defined in URL
[https://wtwdeeplearning.blob.core.windows.net/ignite/spring_config.xml]:
Cannot create inner bean
'org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder#2eda0940'
of type
[org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder]
while setting bean property 'ipFinder'; nested exception is
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name
'org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder#2eda0940'
defined in URL
[https://wtwdeeplearning.blob.core.windows.net/ignite/spring_config.xml]:
Initialization of bean failed; nested exception is
java.lang.NullPointerException
at
org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:313)
at
org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:122)

My libs folder looks like this:

/app/libs/spring-data-commons-2.0.9.RELEASE.jar
/app/libs/commons-io-2.6.jar
/app/libs/spring-core-5.0.8.RELEASE.jar
/app/libs/spring-core-4.3.18.RELEASE.jar
/app/libs/ignite-spring-data_2.0-2.7.6.jar
/app/libs/commons-rng-core-1.0.jar
/app/libs/ignite-shmem-1.0.0.jar
/app/libs/cache-api-1.0.0.jar
/app/libs/commons-logging-1.1.1.jar
/app/libs/commons-collections-3.2.2.jar
/app/libs/spring-expression-4.3.18.RELEASE.jar
/app/libs/commons-math3-3.6.1.jar
/app/libs/spring-aop-4.3.18.RELEASE.jar
/app/libs/spring-tx-4.3.18.RELEASE.jar
/app/libs/lucene-core-7.4.0.jar
/app/libs/spring-tx-5.0.8.RELEASE.jar
/app/libs/ignite-core-2.7.6.jar
/app/libs/ignite-indexing-2.7.6.jar
/app/libs/spring-beans-4.3.18.RELEASE.jar
/app/libs/commons-beanutils-1.9.3.jar
/app/libs/ignite-spring-data-2.7.6.jar
/app/libs/commons-logging-1.2.jar
/app/libs/h2-1.4.197.jar
/app/libs/commons-codec-1.11.jar
/app/libs/spring-beans-5.0.8.RELEASE.jar
/app/libs/spring-data-commons-1.13.14.RELEASE.jar
/app/libs/spring-context-4.3.18.RELEASE.jar
/app/libs/spring-jdbc-4.3.18.RELEASE.jar
/app/libs/lucene-queryparser-7.4.0.jar
/app/libs/spring-context-5.0.8.RELEASE.jar
/app/libs/commons-lang-2.6.jar
/app/libs/commons-beanutils-1.9.2.jar
/app/libs/commons-rng-simple-1.0.jar
/app/libs/ignite-spring-2.7.6.jar
/app/libs/lucene-analyzers-common-7.4.0.jar
/app/libs/ignite-kubernetes-2.7.6.jar
/app/libs/README.txt
/app/libs/jackson-databind-2.9.6.jar
/app/libs/jackson-core-2.9.6.jar



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Getting Spring XML exception when using Ignite .NET with Kubernetes

[camer314]

Hello,

I have built a docker image of my Ignite .NET application and am
instantiating this as a pod in my Kubernetes deployment. This all works as
expected and Ignite initializes fine.

However, I want my nodes to be discoverable so I am attempting to use the
Spring XML file located  here

  

I have also constructed my .NET code to include this in the configuration:

/Console.WriteLine($"Starting Ignite.NET...");
var config_url =
Environment.GetEnvironmentVariable("CONFIG_URI");

if(String.IsNullOrEmpty(config_url))
{
config_url =
"https://raw.githubusercontent.com/apache/ignite/master/modules/kubernetes/config/example-kube-persistence.xml;;
}

Console.WriteLine($"Using config from [{config_url}]");

var config = new IgniteConfiguration()
{
SpringConfigUrl = config_url,
JvmOptions = new List()
{
"-DIGNITE_QUIET=false"
}
};

Ignition.Start(config);
/

Also, I have the binary distribution of Ignite 2.7.6 in the /libs
subdirectory of my app folder.

When this runs I get the following error but there appears to be no
descriptive message as to whats causing it:

Caused by: org.springframework.beans.factory.BeanCreationException: Error
creating bean with name
'org.apache.ignite.configuration.DataStorageConfiguration#4524411f' defined
in URL
[https://raw.githubusercontent.com/apache/ignite/master/modules/kubernetes/config/example-kube-persistence.xml]:
Cannot create inner bean
'org.apache.ignite.configuration.DataRegionConfiguration#544a2ea6' of type
[org.apache.ignite.configuration.DataRegionConfiguration] while setting bean
property 'defaultDataRegionConfiguration'; nested exception is
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'org.apache.ignite.configuration.DataRegionConfiguration#544a2ea6'
defined in URL
[https://raw.githubusercontent.com/apache/ignite/master/modules/kubernetes/config/example-kube-persistence.xml]:
Initialization of bean failed; nested exception is
java.lang.NullPointerException
at
org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:313)
at
org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:122)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1537)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1284)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
at
org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:299)



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Expected serialization performance of Ignite .NET

[camer314]

Ok thanks.

I have managed to get the times down to 13 seconds write and 7.5 seconds
read, basically double your times, on my 12 processor Intel i7-8850 laptop
which is running 100% so you must have some good kit!

Does Java version have any bearing on performance?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Expected serialization performance of Ignite .NET

[camer314]

It seems a combination of a better spec machine and a parallel for loop has
improved performance, although it still takes 8 seconds to run through all
the cache items.

Here is some basic test code...would appreciate any tips on how to improve
access in this type of usage pattern:

https://wtwdeeplearning.blob.core.windows.net/temp/ignitetest.zip?st=2019-12-03T05%3A47%3A38Z=2019-12-12T05%3A47%3A00Z=rl=2018-03-28=b=t%2FXw4bpAFRo7aKdpIbwLfTFOB4Sv%2FeetSi%2FvVSRjg8w%3D

On my VM it takes 25 seconds to populate and 8 seconds to retrieve.

What is the most efficient way to iterate all items on a local cache? In a
real situation I would not know the keys i have in the cache and the only
way I could get decent throughput on the read was the parallel loop which
implies i know the keys already.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Expected serialization performance of Ignite .NET

[camer314]

I have a 21 property C# class (mix of int and string) and am using
IBinarizable interface as suggested in the documentation.

My cache is configured such that each cache entry is a collection of these
objects, lets say each cache item is a List.

I have 10 million instance of this class. For simplicity lets say each cache
entry holds 10 of these objects, amounting to 1 million cache entries.

Each object is roughly about 120 bytes long, so 10 million = ~1.2 gigabytes
of data stored.

I am using a LOCAL cache and a simple foreach loop over the cache takes in
the region of 25 seconds. This seems like an eternity. I understand there is
a lot of serialization happening, probably a lot of garbage collecting as
well, but it still seems like a large amount of time to effectively move
memory from one location to another.

Does that time seem exorbitant to you given the above specs or is it
expected?

What is the optimal way to lay out cache items locally for cache read
iteration (that is, compute needs to iterate the entire cache)?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Does affinity work with cache mode of LOCAL?

[camer314]

Thank you, that makes sense.

My use case is to match Spark dataframe functionality using only C# if
possible, without using Spark

Specifically we have CSV files we wish to load into the cache and then we
have compute functions that act on those rows, adding columns as they do, so
the cache will be heavy on read/write

To try and improve the initial cache population from file(which can be
millions of rows) I distribute a job to the cluster that each reads a piece
of the file to get some sort of upload parallelization.

I am using affinity keys so that the calculations only have to process the
data on the node they run on, which works fine. But then I thought,
performance would probably improve on the cache population step if i just
used LOCAL caches. Its the same end result, calculations working off only
the data they have on the node. I can maybe live with the downsides of local
cache, which i assume include no fault tolerance or load balancing, if the
speed improvements make it worthwhile.

Anyway, basically to get my desired functionality I have 2 options - either
use affinity keys and affinity compute OR use local caches and broadcast
compute.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Does affinity work with cache mode of LOCAL?

[camer314]

Actually the code is adding data to each nodes individual LOCAL cache, its
just the affinity is not working as expected, all affinity jobs are run on
the node which is invoking them (in my case the client node) rather than
where the actual data exists.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Does affinity work with cache mode of LOCAL?

[camer314]

Actually maybe I do not understand what a LOCAL cache is.

I create a LOCAL cache in my client and I have 2 server nodes. I send a
compute function to each node to load some data into the cache. I assumed as
the cache is LOCAL that the nodes would load the data into their own local
cache?

However, when I send the affinity compute I notice those compute functions
are actually sent to my client application and not the server nodes as
happens in PARTITIONED mode.

But my client app is ClientMode=TRUE so it has no data. If i change it to
ClientMode=FALSE so that it participates in the data load then it is able to
retrieve the records it added to the cache but not the ones the servers
added.

So can I assume there is no concept of a distributed LOCAL cache?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Does affinity work with cache mode of LOCAL?

[camer314]

I have been using affinity keys with a PARTITIONED cache and then use those
keys to send computations to the nodes that have the data which all works as
expected.

I wanted to test LOCAL mode for performance but I found no calculations are
now sent to the nodes.

Is that expected behavior?

How can I send calculations to nodes with a LOCAL cache so that those
calculations only work off the data in the local cache (which is the reason
i was using affinity to begin with, I only ever want my calculations to work
off data that is on that node).

If there is a way for a compute function to access only the keys on the node
without using affinity that would work for me too.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite .NET and TcpDiscoveryStaticIpFinder failure

[camer314]

I created a new VM and it now seems to work ok, I have this message which
would indicate everything is connected (which would be correct, 3 nodes, 2
hosts and 6 CPU total)

H/N/C [hosts=2, nodes=3, CPUs=6]

Also btw, when creating the new VM I had to install Java, I chose
jdk-8u221-windows-x64 to match my working VM. When i tried to start
Apache.Ignite.exe I was faced with the error "Failed to load jvm.dll" even
though everything seemed correct. I them came across the below issue which
recommended installing VC++ 2010 redist however ptupitsyn suggested that was
not needed from 2.4+ and i am using the latest 2.7.6. It was only after
installing the VC++ 2010 libraries that Apache.Ignite.exe was able to run.

http://apache-ignite-users.70518.x6.nabble.com/Ignite-NET-2-2-crashes-on-startup-without-any-information-td17173i20.html



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite .NET and TcpDiscoveryStaticIpFinder failure

[camer314]

I dont know about that, is that maybe because I had both addresses down as
endpoints?

I have modified app.config on both machines to only point to one another:

/   


10.0.2.5:47500..47502


/

   


10.0.2.11:47500..47502


   


Now on the #11 machine the log is a lot cleaner, it just has these messages
repeated:

[11:17:08,959][INFO][tcp-disco-srvr-#3][TcpDiscoverySpi] TCP discovery
accepted incoming connection [rmtAddr=/10.0.2.5, rmtPort=53509]
[11:17:08,959][INFO][tcp-disco-srvr-#3][TcpDiscoverySpi] TCP discovery
spawning a new thread for connection [rmtAddr=/10.0.2.5, rmtPort=53509]
[11:17:08,959][INFO][tcp-disco-sock-reader-#24][TcpDiscoverySpi] Started
serving remote node connection [rmtAddr=/10.0.2.5:53509, rmtPort=53509]
[11:17:08,965][INFO][tcp-disco-sock-reader-#24][TcpDiscoverySpi] Finished
serving remote node connection [rmtAddr=/10.0.2.5:53509, rmtPort=53509

But on the #5 machine the log is entirely void of any detail at all:

[11:14:56,534][INFO][main][IgniteKernal] Non-loopback local IPs: *10.0.2.5*,
10.0.75.1, fe80:0:0:0:0:5efe:a00:205%net2, fe80:0:0:0:0:5efe:a00:4b01%net7,
fe80:0:0:0:35b8:b554:51fb:2fc1%eth5, fe80:0:0:0:812a:7edf:9502:2449%eth4
[11:14:56,534][INFO][main][IgniteKernal] Enabled local MACs:
00E0, 000D3A18192C, 00155D02051F
[11:14:56,571][INFO][main][TcpDiscoverySpi] Connection check threshold is
calculated: 2000
[11:14:56,588][INFO][main][TcpDiscoverySpi] Successfully bound to TCP port
[port=47500, localHost=0.0.0.0/0.0.0.0,
locNodeId=c77edb8f-56b8-405d-b2ba-66f585a30952]






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite .NET and TcpDiscoveryStaticIpFinder failure

[camer314]

I just realised this is the same situation that is logged here  here

  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Ignite .NET and TcpDiscoveryStaticIpFinder failure

[camer314]

I am using 2 Azure virtual machines, both part of the same VNET. The network
security policy is applied at the network level and allows all traffic
in/out from the VNET.

I am trying to run a cluster across both machines. Seeing as Azure does not
support broadcast I am using a static IP list, which are the private IPs of
my two machines like such:

   


10.0.2.5:47500..47509
10.0.2.11:47500..47509




There is not much going on in the way of error messages but the connection
from 11 -> 5 must keep dropping or some other issue is preventing that node
joining.

On the 11 machine I have these log entries, indicating its found the other
node, but then seems to immediately lose the connection?

/[06:50:17,775][INFO][tcp-disco-msg-worker-#2][GridEncryptionManager]
Joining node doesn't have encryption data
[node=41518d17-a16a-48a0-9656-cd3d2b6e0042]
[06:50:17,800][INFO][tcp-disco-msg-worker-#2][TcpDiscoverySpi] New next node
[newNext=TcpDiscoveryNode [id=41518d17-a16a-48a0-9656-cd3d2b6e0042,
addrs=[0:0:0:0:0:0:0:1, 10.0.2.5, 10.0.75.1, 127.0.0.1],
sockAddrs=[camyakoubCPU/10.0.2.5:47500, /10.0.75.1:47500,
/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500,
host.docker.internal/10.0.2.11:47500], discPort=47500, order=0, intOrder=2,
lastExchangeTime=1574751017748, loc=false, ver=2.7.6#20190911-sha1:21f7ca41,
isClient=false]]
[06:50:22,779][INFO][tcp-disco-srvr-#3][TcpDiscoverySpi] TCP discovery
accepted incoming connection [rmtAddr=/10.0.2.5, rmtPort=52234]
[06:50:22,779][INFO][tcp-disco-srvr-#3][TcpDiscoverySpi] TCP discovery
spawning a new thread for connection [rmtAddr=/10.0.2.5, rmtPort=52234]
[06:50:22,780][INFO][tcp-disco-sock-reader-#7][TcpDiscoverySpi] Started
serving remote node connection [rmtAddr=/10.0.2.5:52234, rmtPort=52234]
[06:50:22,788][INFO][tcp-disco-sock-reader-#7][TcpDiscoverySpi] Finished
serving remote node connection [rmtAddr=/10.0.2.5:52234, rmtPort=52234
[06:50:22,805][WARNING][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed to
send message to next node [msg=TcpDiscoveryNodeAddedMessage
[node=TcpDiscoveryNode [id=41518d17-a16a-48a0-9656-cd3d2b6e0042,
addrs=[0:0:0:0:0:0:0:1, 10.0.2.5, 10.0.75.1, 127.0.0.1],
sockAddrs=[camyakoubCPU/10.0.2.5:47500, /10.0.75.1:47500,
/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500,
host.docker.internal/10.0.2.11:47500], discPort=47500, order=0, intOrder=2,
lastExchangeTime=1574751017748, loc=false, ver=2.7.6#20190911-sha1:21f7ca41,
isClient=false],
dataPacket=o.a.i.spi.discovery.tcp.internal.DiscoveryDataPacket@8b3b66a,
discardMsgId=null, discardCustomMsgId=null, top=null, clientTop=null,
gridStartTime=1574750964272, super=TcpDiscoveryAbstractMessage
[sndNodeId=null, id=aa79676ae61-e3b1d49a-a023-435a-961c-13394c08ad0b,
verifierNodeId=e3b1d49a-a023-435a-961c-13394c08ad0b, topVer=0, pendingIdx=0,
failedNodes=null, isClient=false]], next=TcpDiscoveryNode
[id=41518d17-a16a-48a0-9656-cd3d2b6e0042, addrs=[0:0:0:0:0:0:0:1, 10.0.2.5,
10.0.75.1, 127.0.0.1], sockAddrs=[camyakoubCPU/10.0.2.5:47500,
/10.0.75.1:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500,
host.docker.internal/10.0.2.11:47500], discPort=47500, order=0, intOrder=2,
lastExchangeTime=1574751017748, loc=false, ver=2.7.6#20190911-sha1:21f7ca41,
isClient=false], errMsg=Failed to send message to next node
[msg=TcpDiscoveryNodeAddedMessage [node=TcpDiscoveryNode
[id=41518d17-a16a-48a0-9656-cd3d2b6e0042, addrs=[0:0:0:0:0:0:0:1, 10.0.2.5,
10.0.75.1, 127.0.0.1], sockAddrs=[camyakoubCPU/10.0.2.5:47500,
/10.0.75.1:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500,
host.docker.internal/10.0.2.11:47500], discPort=47500, order=0, intOrder=2,
lastExchangeTime=1574751017748, loc=false, ver=2.7.6#20190911-sha1:21f7ca41,
isClient=false],
dataPacket=o.a.i.spi.discovery.tcp.internal.DiscoveryDataPacket@8b3b66a,
discardMsgId=null, discardCustomMsgId=null, top=null, clientTop=null,
gridStartTime=1574750964272, super=TcpDiscoveryAbstractMessage
[sndNodeId=null, id=aa79676ae61-e3b1d49a-a023-435a-961c-13394c08ad0b,
verifierNodeId=e3b1d49a-a023-435a-961c-13394c08ad0b, topVer=0, pendingIdx=0,
failedNodes=null, isClient=false]], next=ClusterNode
[id=41518d17-a16a-48a0-9656-cd3d2b6e0042, order=0, addr=[0:0:0:0:0:0:0:1,
10.0.2.5, 10.0.75.1, 127.0.0.1], daemon=false]]]
[06:50:22,806][WARNING][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Local node
has detected failed nodes and started cluster-wide procedure. To speed up
failure detection please see 'Failure Detection' section under javadoc for
'TcpDiscoverySpi'
[06:50:22,812][INFO][disco-event-worker-#42][GridDiscoveryManager] Added new
node to topology: TcpDiscoveryNode [id=41518d17-a16a-48a0-9656-cd3d2b6e0042,
addrs=[0:0:0:0:0:0:0:1, 10.0.2.5, 10.0.75.1, 127.0.0.1],
sockAddrs=[camyakoubCPU/10.0.2.5:47500, /10.0.75.1:47500,
/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500,
host.docker.internal/10.0.2.11:47500], discPort=47500, order=2, intOrder=2,
lastExchangeTime=1574751017748, loc=false, ver=2.7.6#20190911-sha1:21f7ca41,

Re: Is there a binary of Apache.Ignite.Exe for .NET Core

[camer314]

Perfect thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Is there a binary of Apache.Ignite.Exe for .NET Core

[camer314]

I am using VS Code which complicates things slightly however I managed to get
this working minus service support seeing as ServiceBase is not available in
.NET Core. This is fine for my purposes at the moment.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Is there a binary of Apache.Ignite.Exe for .NET Core

[camer314]

The source code provides solution and proj files for .NET Core however there
is no equivalent for the Apache.Ignite.Exe executable.

I am trying to coerce the existing proj file to build a netcoreapp2.0 but am
running into compiler complaints so was wondering is there is a csproj file
that exists that i can use?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How to perform distributed compute in similar way to Spark vector UDF

[camer314]

Reading a little more in the Java docs about AffinityKey, I am thinking that,
much like vector UDF batch sizing, one way I could easily achieve my result
is to batch my rows into affinity keys. That is, for every 100,000 rows the
affinity key changes for example.

So cache keys [0...9] have affinity key 0, keys [10...19] have
affinity key 1 etc?

If that is the case, may I suggest you update the .NET documentation for
Data Grid regarding Affinity Colocation as it does not mention the use of
AffinityKey or go into anywhere near as much detail as the Java docs.






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

How to perform distributed compute in similar way to Spark vector UDF

[camer314]

I asked this question on  StackOverflow

  

However I probably put too much weight on Spark.

My question really is, how can I load in a large CSV file to the cache and
send compute actions to the nodes which work in a similar way to Pandas UDF.
That is, they work on a subset of the data (rows).

In Ignite I imagine I could load the CSV to a cache using PARTITION mode and
then using affinity compute send functions to the nodes where the data is,
so each node is processing only the data that exists on it. This seems like
a nice way to go, each node is always only processing locally, and the
results of those actions would be adding back to the cache, so presumably
would only add locally as well.

However, I am not entirely sure how the partitioning works. The examples for
affinity show using a single key value.

Is there a way to load a CSV into a cache in PARTITION mode, so Ignite
evenly distributes across the grid but then run a compute job on every node
that works ONLY with the data in its own cache, that way i wont need to care
about keys?

For example, imagine a CSV file that is a matrix of numbers. My distributed
cache would really be a dataframe representation of that file. For arguments
sake lets say my cache is keyed by an increment ID with the data being an
array of doubles and the column names are A,B,C

That ID key is really pretty irrelevant. Its is meaningless to my
application.

Now lets say I wanted to perform the same maths on every row in that
dataframe, with the results being a new column in the cache.

If that formula was D = A * B * C then D becomes a new column.

Ignoring Spark SQL, in Spark I could write a UDF easily that creates column
D by passing columns [A,B,C]. Spark doesnt care about keys or ID columns in
this instance, it just gives you a vector of data and you return a vector of
results.

So in Ignite, how can i replicate that behaviour the most elegantly in code
(.NET), send compute to the grid that collectively processes all rows
without caring about the keys?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Question about memory when uploading CSV using .NET DataStreamer

[camer314]

Ok yes i see. Seems like with my code changes I made to provide the example
that the memory consumption is way more inline with expectations, so I guess
it was a code error on my part.

However, it seems strange that my client node, which has no cache, still
wants to hang onto over 1Gb of heap space even though its using less than
100Mb. Is there no way to release that back?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Question about memory when uploading CSV using .NET DataStreamer

[camer314]

In my sample code i had a bit of a bug, this should be the line to add:

var _ = ldr.AddData(id++,data);

However it doesnt appear to make any difference, this is the state of memory
(with ignite.exe being my client executable). This is paused after insertion
of 1 million rows, why is my client memory usage still so high?

 

If i comment out the AddData then i get:

 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Question about memory when uploading CSV using .NET DataStreamer

[camer314]

Here is my source file and a 1 million row CSV file.

I am not sure whats different between my code and yours but my version
quickly consumes memory on the client side for some reason.

Caveat, I am normally a Python programmer so i might have missed something
obvious...

https://wtwdeeplearning.blob.core.windows.net/ignite/Program.zip?st=2019-11-15T00%3A58%3A20Z=2019-11-25T00%3A58%3A00Z=rl=2018-03-28=b=IkMuGbNJ4YAp5Ko%2BmcqC5PkbSLeuUfQLegMXpj3WNQ0%3D





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Question about memory when uploading CSV using .NET DataStreamer

[camer314]

I have a large CSV file (50 million rows) that i wish to upload to a cache. I
am using .NET and a DataStreamer from my application which is designated as
a client only node.

What i dont understand is i quickly run out of memory on my C# streaming
(client) application while my data node (an instance of Apache.Ignite.exe)
slowly increases RAM usage but not at the rate as my client app does.

So it would seem that either (A) my client IS actually being used to cache
data or (B) there is a memory leak where data that has been sent to the
cache is not released.

As for figures, Apache.Ignite.exe when first started uses 165Mb. After
loading in 1 million records and letting it all settle down,
Apache.Ignite.exe now sits at 450Mb while my client app (the one streaming)
sits at 1.5Gb.

The total size of the input file is 5Gb so 1 million records should really
only be about 100Mb so i dont know how my client even gets to 1.5Gb to begin
with. If i comment out the AddData() then my client never gets past 200Mb so
its certainly something happening in the cache.

Is this expected behaviour? If so then i dont know how to import huge CSV
files without memory issues on the streaming machine.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Confusion about how to connect to Kubernetes cluster remotely from client app

[camer314]

Thanks Ilya,

Am i able to use the .NET API in this way? Connecting to a remote cluster?

I am finding the documentation of the configuration a little hard to
understand in regards to the options for connection. In the Kubernetes YAML
file I see ports for REST, SQL and thin clients, would you use the REST one
for C#?

You also mention I can use thin client, does that support compute?

REST is perhaps the cleanest solution for us.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Confusion about how to connect to Kubernetes cluster remotely from client app

[camer314]

I am new to Ignite and am struggling to figure out how to connect my client
application to my Ignite cluster hosted in Azure Kubernetes.

Effectively I want to remotely connect, that is, I dont want my client
machine to be in the topology, its probably not even in the same country, I
would like to connect (C# or REST) to an Ignite cluster and use both the
cache and compute features. It would seem the only way to do this is via the
'thin client' which does not support compute?

I have heard of 'thick client' nodes which offer full API support but I am
not sure what the requirements are for those.

So my question is, if i have a C# desktop application (or any other piece of
code) which i want to connect remotely to a Ignite cluster, like Azure
Kubernetes, to perform cache/compute and without participating in the
cluster, how can i do that? Is it possible?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/