date:20160609

Re: Slow Transaction Performance

2016-06-09 Thread pragmaticbigdata

I forgot to mention that I have set backups as 0 in order to benchmark the
best possible performance.  So setting primary_sync does not have any
effect. 

Do you have any of your test cases that are resulting in better numbers?



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Slow-Transaction-Performance-tp5548p5566.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Persistence using Cassandra

2016-06-09 Thread ChickyDutt

Hi Alexey

I reinstalled Ignite on C;\Ignite and I got through.


Regards.

On Thu, Jun 9, 2016 at 10:39 PM, Ashish Dutt Sharma <
ash.dutt.sha...@gmail.com> wrote:

> Hi Alexey
>
> I run into the attached error after I run the ignite.bat with reference to
> the server config file as mentioned in Load test examples.
>
> Could you please advise on how to get around it?
>
> On Thu, Jun 9, 2016 at 6:10 PM, Ashish Dutt Sharma <
> ash.dutt.sha...@gmail.com> wrote:
>
>> I decreased the JVM size and it worked. Thank you.
>>
>> Regards.
>>
>> On Thu, Jun 9, 2016 at 5:57 PM, Ashish Dutt Sharma <
>> ash.dutt.sha...@gmail.com> wrote:
>>
>>> Thank you Alexey.
>>>
>>> I am now trying to run the load tests from the source package, After
>>> building from the source and editing relevant properties, I run into the
>>> following error after I run the 'cassandra-load-tests.bat' file.
>>>
>>> Could you please take a look at the attachment and advise on how to
>>> resolve? Please let me know if you need any further information.
>>>
>>> Regards,
>>> Ashish Sharma,
>>>
>>> On Wed, Jun 8, 2016 at 5:43 PM, Alexey Kuznetsov [via Apache Ignite
>>> Users]  wrote:
>>>
 Hi, Ashish Sharma.

 Take a look at docs: https://apacheignite.readme.io/docs/overview
 As far as I see in docs
 https://apacheignite.readme.io/v1.6/docs/base-concepts
 there I found such note;
 
 Current implementation of
 org.apache.ignite.cache.store.cassandra.utils.datasource.DataSource doesn't
 implement Serializable,
  thus for distributed Ignite clusters, CassandraCacheStoreFactory could
 only be setup through Spring XML file, but not from code
 ---

 I think this could be fixed in next releases.



 On Wed, Jun 8, 2016 at 10:35 PM, ChickyDutt <[hidden email]
 > wrote:

> Thank you Alexey.
>
> Do you have an example where the CacheConfiguration store is
> configured using Java ?
>
> Regards,
> Ashish Sharma.
>
> On Mon, May 23, 2016 at 3:28 PM, Alexey Kuznetsov [via Apache Ignite
> Users] <[hidden email]
> > wrote:
>
>> If you need tests - you need to download sources (
>> https://ignite.apache.org/download.cgi#sources).
>> Also, today we released Ignite 1.6 it should be available on Maven
>> now.
>>
>> Hope this help.
>>
>> On Mon, May 23, 2016 at 6:35 PM, ChickyDutt <[hidden email]
>> > wrote:
>>
>>> Mentioned below are the complete logs
>>>
>>> class org.apache.ignite.IgniteException: Failed to instantiate
>>> Spring XML application context
>>> [springUrl=file:/C:/Users/D-UA96OU/Desktop/Ignite
>>> /apache-ignite-fabric-1.6.0-SNAPSHOT-bin/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/config/default-config.xml,
>>> err=Configuration problem: Failed t
>>> o import bean definitions from URL location
>>> [classpath:org/apache/ignite/tests/cassandra/connection-settings.xml]
>>> Offending resource: URL
>>> [file:/C:/Users/D-UA96OU/Desktop/Ignite/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/apache-ignite-fabric-1.6.0-SNAPSHOT-bin
>>> /config/default-config.xml]; nested exception is
>>> org.springframework.beans.factory.BeanDefinitionStoreException: 
>>> IOException
>>> parsing XML docume
>>> nt from class path resource
>>> [org/apache/ignite/tests/cassandra/connection-settings.xml]; nested
>>> exception is java.io.FileNotFoundException: cla
>>> ss path resource
>>> [org/apache/ignite/tests/cassandra/connection-settings.xml] cannot be
>>> opened because it does not exist]
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:906)
>>> at org.apache.ignite.Ignition.start(Ignition.java:350)
>>> at
>>> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:302)
>>> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
>>> instantiate Spring XML application context 
>>> [springUrl=file:/C:/Users/D-UA9
>>> 6OU/Desktop/Ignite/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/config/default-config.xml,
>>> err=Configuration
>>>  problem: Failed to import bean definitions from URL location
>>> [classpath:org/apache/ignite/tests/cassandra/connection-settings.xml]
>>> Offending resource: URL
>>> [file:/C:/Users/D-UA96OU/Desktop/Ignite/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/apache-ignite-fabric-1.6.0-SNAPSHOT-bin
>>> /config/default-config.xml]; nested exception is
>>> org.springframework.beans.factory.BeanDefinitionStoreException: 
>>> IOException
>>> parsing XML docume
>>> nt from class path resource
>>>

Re: Slow Transaction Performance

2016-06-09 Thread amitpa

Setting PRIMARY_SYNC helps for partitioned caches. 
It should be faster.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Slow-Transaction-Performance-tp5548p5563.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Persistence using Cassandra

2016-06-09 Thread ChickyDutt

I decreased the JVM size and it worked. Thank you.

Regards.

On Thu, Jun 9, 2016 at 5:57 PM, Ashish Dutt Sharma <
ash.dutt.sha...@gmail.com> wrote:

> Thank you Alexey.
>
> I am now trying to run the load tests from the source package, After
> building from the source and editing relevant properties, I run into the
> following error after I run the 'cassandra-load-tests.bat' file.
>
> Could you please take a look at the attachment and advise on how to
> resolve? Please let me know if you need any further information.
>
> Regards,
> Ashish Sharma,
>
> On Wed, Jun 8, 2016 at 5:43 PM, Alexey Kuznetsov [via Apache Ignite Users]
>  wrote:
>
>> Hi, Ashish Sharma.
>>
>> Take a look at docs: https://apacheignite.readme.io/docs/overview
>> As far as I see in docs
>> https://apacheignite.readme.io/v1.6/docs/base-concepts
>> there I found such note;
>> 
>> Current implementation of
>> org.apache.ignite.cache.store.cassandra.utils.datasource.DataSource doesn't
>> implement Serializable,
>>  thus for distributed Ignite clusters, CassandraCacheStoreFactory could
>> only be setup through Spring XML file, but not from code
>> ---
>>
>> I think this could be fixed in next releases.
>>
>>
>>
>> On Wed, Jun 8, 2016 at 10:35 PM, ChickyDutt <[hidden email]
>> > wrote:
>>
>>> Thank you Alexey.
>>>
>>> Do you have an example where the CacheConfiguration store is configured
>>> using Java ?
>>>
>>> Regards,
>>> Ashish Sharma.
>>>
>>> On Mon, May 23, 2016 at 3:28 PM, Alexey Kuznetsov [via Apache Ignite
>>> Users] <[hidden email]
>>> > wrote:
>>>
 If you need tests - you need to download sources (
 https://ignite.apache.org/download.cgi#sources).
 Also, today we released Ignite 1.6 it should be available on Maven now.

 Hope this help.

 On Mon, May 23, 2016 at 6:35 PM, ChickyDutt <[hidden email]
 > wrote:

> Mentioned below are the complete logs
>
> class org.apache.ignite.IgniteException: Failed to instantiate Spring
> XML application context [springUrl=file:/C:/Users/D-UA96OU/Desktop/Ignite
> /apache-ignite-fabric-1.6.0-SNAPSHOT-bin/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/config/default-config.xml,
> err=Configuration problem: Failed t
> o import bean definitions from URL location
> [classpath:org/apache/ignite/tests/cassandra/connection-settings.xml]
> Offending resource: URL
> [file:/C:/Users/D-UA96OU/Desktop/Ignite/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/apache-ignite-fabric-1.6.0-SNAPSHOT-bin
> /config/default-config.xml]; nested exception is
> org.springframework.beans.factory.BeanDefinitionStoreException: 
> IOException
> parsing XML docume
> nt from class path resource
> [org/apache/ignite/tests/cassandra/connection-settings.xml]; nested
> exception is java.io.FileNotFoundException: cla
> ss path resource
> [org/apache/ignite/tests/cassandra/connection-settings.xml] cannot be
> opened because it does not exist]
> at
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:906)
> at org.apache.ignite.Ignition.start(Ignition.java:350)
> at
> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:302)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> instantiate Spring XML application context [springUrl=file:/C:/Users/D-UA9
> 6OU/Desktop/Ignite/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/config/default-config.xml,
> err=Configuration
>  problem: Failed to import bean definitions from URL location
> [classpath:org/apache/ignite/tests/cassandra/connection-settings.xml]
> Offending resource: URL
> [file:/C:/Users/D-UA96OU/Desktop/Ignite/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/apache-ignite-fabric-1.6.0-SNAPSHOT-bin
> /config/default-config.xml]; nested exception is
> org.springframework.beans.factory.BeanDefinitionStoreException: 
> IOException
> parsing XML docume
> nt from class path resource
> [org/apache/ignite/tests/cassandra/connection-settings.xml]; nested
> exception is java.io.FileNotFoundException: cla
> ss path resource
> [org/apache/ignite/tests/cassandra/connection-settings.xml] cannot be
> opened because it does not exist]
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391)
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104)
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98)
> at
>

Re: Persistence using Cassandra

2016-06-09 Thread ChickyDutt

Thank you Alexey.

I am now trying to run the load tests from the source package, After
building from the source and editing relevant properties, I run into the
following error after I run the 'cassandra-load-tests.bat' file.

Could you please take a look at the attachment and advise on how to
resolve? Please let me know if you need any further information.

Regards,
Ashish Sharma,

On Wed, Jun 8, 2016 at 5:43 PM, Alexey Kuznetsov [via Apache Ignite Users] <
ml-node+s70518n5535...@n6.nabble.com> wrote:

> Hi, Ashish Sharma.
>
> Take a look at docs: https://apacheignite.readme.io/docs/overview
> As far as I see in docs
> https://apacheignite.readme.io/v1.6/docs/base-concepts
> there I found such note;
> 
> Current implementation of
> org.apache.ignite.cache.store.cassandra.utils.datasource.DataSource doesn't
> implement Serializable,
>  thus for distributed Ignite clusters, CassandraCacheStoreFactory could
> only be setup through Spring XML file, but not from code
> ---
>
> I think this could be fixed in next releases.
>
>
>
> On Wed, Jun 8, 2016 at 10:35 PM, ChickyDutt <[hidden email]
> > wrote:
>
>> Thank you Alexey.
>>
>> Do you have an example where the CacheConfiguration store is configured
>> using Java ?
>>
>> Regards,
>> Ashish Sharma.
>>
>> On Mon, May 23, 2016 at 3:28 PM, Alexey Kuznetsov [via Apache Ignite
>> Users] <[hidden email]
>> > wrote:
>>
>>> If you need tests - you need to download sources (
>>> https://ignite.apache.org/download.cgi#sources).
>>> Also, today we released Ignite 1.6 it should be available on Maven now.
>>>
>>> Hope this help.
>>>
>>> On Mon, May 23, 2016 at 6:35 PM, ChickyDutt <[hidden email]
>>> > wrote:
>>>
 Mentioned below are the complete logs

 class org.apache.ignite.IgniteException: Failed to instantiate Spring
 XML application context [springUrl=file:/C:/Users/D-UA96OU/Desktop/Ignite
 /apache-ignite-fabric-1.6.0-SNAPSHOT-bin/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/config/default-config.xml,
 err=Configuration problem: Failed t
 o import bean definitions from URL location
 [classpath:org/apache/ignite/tests/cassandra/connection-settings.xml]
 Offending resource: URL
 [file:/C:/Users/D-UA96OU/Desktop/Ignite/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/apache-ignite-fabric-1.6.0-SNAPSHOT-bin
 /config/default-config.xml]; nested exception is
 org.springframework.beans.factory.BeanDefinitionStoreException: IOException
 parsing XML docume
 nt from class path resource
 [org/apache/ignite/tests/cassandra/connection-settings.xml]; nested
 exception is java.io.FileNotFoundException: cla
 ss path resource
 [org/apache/ignite/tests/cassandra/connection-settings.xml] cannot be
 opened because it does not exist]
 at
 org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:906)
 at org.apache.ignite.Ignition.start(Ignition.java:350)
 at
 org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:302)
 Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
 instantiate Spring XML application context [springUrl=file:/C:/Users/D-UA9
 6OU/Desktop/Ignite/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/config/default-config.xml,
 err=Configuration
  problem: Failed to import bean definitions from URL location
 [classpath:org/apache/ignite/tests/cassandra/connection-settings.xml]
 Offending resource: URL
 [file:/C:/Users/D-UA96OU/Desktop/Ignite/apache-ignite-fabric-1.6.0-SNAPSHOT-bin/apache-ignite-fabric-1.6.0-SNAPSHOT-bin
 /config/default-config.xml]; nested exception is
 org.springframework.beans.factory.BeanDefinitionStoreException: IOException
 parsing XML docume
 nt from class path resource
 [org/apache/ignite/tests/cassandra/connection-settings.xml]; nested
 exception is java.io.FileNotFoundException: cla
 ss path resource
 [org/apache/ignite/tests/cassandra/connection-settings.xml] cannot be
 opened because it does not exist]
 at
 org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391)
 at
 org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104)
 at
 org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98)
 at
 org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:639)
 at
 org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:840)
 at
 org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:749)
 at

Re: Ignite for Spark on YARN Deployment

2016-06-09 Thread Hongmei Zong

Hi Nikolay,

After I changed the value of IGNITE_XML_CONFIG=/user/hongmei/ignite/config/
 (a HDFS path). Ignite RARN is running now. I use the Hadoop UI console to
check the log of the application, the attached is the *stderr *log information
about containers:

It looks like that the containers are allocated and then completed! The*
stderr *log is very long and the container ID from X01_001 to
XX01_013582.  Finally all these containers are completed.

I have no idea, is there anything not right?

There is no information in *stdout* log.

Thank you!

Hongmei
Logs for container_e24_1464374946035_29722_01_01

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/hdp/2.3.4.0-3485/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/disk2/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_29722/filecache/10/ignite-yarn.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/06/09 10:48:16 INFO impl.ContainerManagementProtocolProxy:
yarn.client.max-cached-nodemanagers-proxies : 0
16/06/09 10:48:16 INFO client.ConfiguredRMFailoverProxyProvider:
Failing over to rm2
Jun 09, 2016 10:48:16 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Application master registered.
Jun 09, 2016 10:48:16 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Jun 09, 2016 10:48:16 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
16/06/09 10:48:17 INFO impl.AMRMClientImpl: Received new token for :
c5hdp108.c5.runwaynine.com:45454
16/06/09 10:48:17 INFO impl.AMRMClientImpl: Received new token for :
c5hdp111.c5.runwaynine.com:45454
Jun 09, 2016 10:48:17 AM org.apache.ignite.yarn.ApplicationMaster
onContainersAllocated
INFO: Launching container: container_e24_1464374946035_29722_01_02.
16/06/09 10:48:17 INFO impl.ContainerManagementProtocolProxy: Opening
proxy : c5hdp108.c5.runwaynine.com:45454
Jun 09, 2016 10:48:17 AM org.apache.ignite.yarn.ApplicationMaster
onContainersAllocated
INFO: Launching container: container_e24_1464374946035_29722_01_03.
16/06/09 10:48:17 INFO impl.ContainerManagementProtocolProxy: Opening
proxy : c5hdp111.c5.runwaynine.com:45454
Jun 09, 2016 10:48:24 AM org.apache.ignite.yarn.ApplicationMaster
onContainersCompleted
INFO: Container completed. Container id:
container_e24_1464374946035_29722_01_02. State: COMPLETE.
Jun 09, 2016 10:48:24 AM org.apache.ignite.yarn.ApplicationMaster
onContainersCompleted
INFO: Container completed. Container id:
container_e24_1464374946035_29722_01_03. State: COMPLETE.
Jun 09, 2016 10:48:24 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Jun 09, 2016 10:48:24 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
16/06/09 10:48:25 INFO impl.AMRMClientImpl: Received new token for :
c5hdp109.c5.runwaynine.com:45454
16/06/09 10:48:25 INFO impl.AMRMClientImpl: Received new token for :
c5hdp105.c5.runwaynine.com:45454
Jun 09, 2016 10:48:25 AM org.apache.ignite.yarn.ApplicationMaster
onContainersAllocated
INFO: Launching container: container_e24_1464374946035_29722_01_04.
16/06/09 10:48:25 INFO impl.ContainerManagementProtocolProxy: Opening
proxy : c5hdp108.c5.runwaynine.com:45454
Jun 09, 2016 10:48:25 AM org.apache.ignite.yarn.ApplicationMaster
onContainersAllocated
INFO: Launching container: container_e24_1464374946035_29722_01_05.
16/06/09 10:48:25 INFO impl.ContainerManagementProtocolProxy: Opening
proxy : c5hdp111.c5.runwaynine.com:45454
Jun 09, 2016 10:48:25 AM org.apache.ignite.yarn.ApplicationMaster
onContainersCompleted
INFO: Container completed. Container id:
container_e24_1464374946035_29722_01_06. State: COMPLETE.
Jun 09, 2016 10:48:25 AM org.apache.ignite.yarn.ApplicationMaster
onContainersCompleted
INFO: Container completed. Container id:
container_e24_1464374946035_29722_01_07. State: COMPLETE.
Jun 09, 2016 10:48:27 AM org.apache.ignite.yarn.ApplicationMaster
onContainersCompleted
INFO: Container completed. Container id:
container_e24_1464374946035_29722_01_04. State: COMPLETE.
Jun 09, 2016 10:48:27 AM org.apache.ignite.yarn.ApplicationMaster
onContainersCompleted
INFO: Container completed. Container id:
container_e24_1464374946035_29722_01_05. State: COMPLETE.
Jun 09, 2016 10:48:27 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Jun 09, 2016 10:48:27 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Jun 09, 2016 10:48:28 AM org.apache.ignite.yarn.ApplicationMaster
onContainersAllocated
INFO: Launching container: container_e24_1464374946035_29722_01_08.
16/06/09 10:48:28 INFO

Re: Ignite for Spark on YARN Deployment

2016-06-09 Thread Hongmei Zong

Hi Nikolay,

I tried to comment the line IGNITE_XML_CONFIG= ,but the Ignite YARN failed.

[hongmei@c5hdpe001 config]$ hadoop jar
/u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
/u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
/u/hongmei/apache-ignite/config/cluster.properties

WARNING: Use "yarn jar" to launch YARN applications.

16/06/09 10:31:43 INFO impl.TimelineClientImpl: Timeline service address:
http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/

16/06/09 10:31:47 INFO client.ConfiguredRMFailoverProxyProvider: Failing
over to rm2

16/06/09 10:31:49 INFO impl.YarnClientImpl: Submitted application
application_1464374946035_29710

Jun 09, 2016 10:31:49 AM org.apache.ignite.yarn.IgniteYarnClient main

INFO: Submitted application. Application id: application_1464374946035_29710

Jun 09, 2016 10:31:59 AM org.apache.ignite.yarn.IgniteYarnClient main

INFO: Application application_1464374946035_29710 is FAILED.

On Thu, Jun 9, 2016 at 10:21 AM, Nikolay Tikhonov 
wrote:

> You set wrong value to IGNITE_XML_CONFIG property. The property should
> contains path to ignite configuration file. For example
>
> IGNITE_XML_CONFIG=/u/hongmei/apache-ignite/config/default-config.xml
>
> I think you can comment this line in property file and ignite will start
> with default configuration.
>
> On Thu, Jun 9, 2016 at 5:08 PM, Hongmei Zong  wrote:
>
>> Hi nikolai,
>>
>> Thank you very much for prompt reply!
>>
>> I did not find the ignite-config.xml file under my ignite home directory(
>>  /u/hongmei/apache-ignite/  ).
>>
>> I find a "default-config.xml" at the path:
>> /u/hongmei/apache-ignite/config/default-config.xml
>>
>> 
>>
>>
>> 
>>
>>
>> http://www.springframework.org/schema/beans;
>>
>>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>>
>>xsi:schemaLocation="
>>
>>http://www.springframework.org/schema/beans
>>
>>http://www.springframework.org/schema/beans/spring-beans.xsd;>
>>
>> 
>>
>> > "org.apache.ignite.configuration.IgniteConfiguration">
>>
>> 
>>
>> > "org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>>
>> 
>>
>> > "org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>>
>> > "addresses">
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>> c5hdpe001.c5.runwaynine.com:47500..47509
>>
>> 
>> c5hdpe002.c5.runwaynine.com:47500..47509
>>
>> 
>> c5hdpe003.c5.runwaynine.com:47500..47509
>>
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>>
>>
>> The "cluster.properties" at the path:
>> /u/hongmei/apache-ignite/config/cluster.properties
>>
>> # The number of nodes in the cluster.
>>
>> IGNITE_NODE_COUNT=2
>>
>>
>> # The number of CPU Cores for each Apache Ignite node.
>>
>> IGNITE_RUN_CPU_PER_NODE=1
>>
>>
>> # The number of Megabytes of RAM for each Apache Ignite node.
>>
>> IGNITE_MEMORY_PER_NODE=2048
>>
>>
>> # The version ignite which will be run on nodes.
>>
>> IGNITE_VERSION=1.6.0
>>
>>
>> # The hdfs directory which will be used for saving Apache Ignite
>> disbributives.
>>
>> IGNITE_RELEASES_DIR=/user/hongmei/ignite-yarn
>>
>>
>> #The directory which will be used for saving Apache Ignite
>> distributives(copy .jar file to it).
>>
>> IGNITE_WORKING_DIR=/user/hongmei/ignite/workdir
>>
>>
>> #The hdfs path to Apache Ignite config file.
>>
>> #IGNITE_XML_CONFIG=/user/yarn/ignite/
>>
>> IGNITE_XML_CONFIG=/user/hongmei/ignite/
>>
>>
>> #The hdfs path to libs which will be added to classpath.
>>
>> IGNITE_USERS_LIBS=/user/hongmei/ignite/libs/
>>
>>
>> #The constraint on slave hosts.
>>
>> #IGNITE_HOSTENAME_CONSTRAINT=
>>
>>
>> IGNITE_LOCAL_WORK_DIR=/u/hongmei/apache-ignite
>>
>>
>>
>> Thank you very much
>>
>> Hongmei
>>
>>
>>
>>
>>
>> On Thu, Jun 9, 2016 at 9:43 AM, Nikolai Tikhonov 
>> wrote:
>>
>>> It seems that your ignite configuration invalid. Could you share 
>>> ignite-config.xml
>>> and /u/hongmei/apache-ignite/config/cluster.properties?
>>>
>>> On Thu, Jun 9, 2016 at 4:35 PM, Hongmei Zong  wrote:
>>>
 Thank you very much Nikolai !

 I found another issue regarding my Ignite YARN

Re: Ignite for Spark on YARN Deployment

2016-06-09 Thread Hongmei Zong

Hi Nikolay,

I tried to comment the line IGNITE_XML_CONFIG= ,but the Ignite YARN failed.

[hongmei@c5hdpe001 config]$ hadoop jar
/u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
/u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
/u/hongmei/apache-ignite/config/cluster.properties

WARNING: Use "yarn jar" to launch YARN applications.

16/06/09 10:31:43 INFO impl.TimelineClientImpl: Timeline service address:
http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/

16/06/09 10:31:47 INFO client.ConfiguredRMFailoverProxyProvider: Failing
over to rm2

16/06/09 10:31:49 INFO impl.YarnClientImpl: Submitted application
application_1464374946035_29710

Jun 09, 2016 10:31:49 AM org.apache.ignite.yarn.IgniteYarnClient main

INFO: Submitted application. Application id: application_1464374946035_29710

Jun 09, 2016 10:31:59 AM org.apache.ignite.yarn.IgniteYarnClient main

INFO: Application application_1464374946035_29710 is FAILED.

On Thu, Jun 9, 2016 at 10:23 AM, Hongmei Zong  wrote:

> Thank you very much Nikolay, I will give it a try! :)
>
> Hongmei
>
> On Thu, Jun 9, 2016 at 10:21 AM, Nikolay Tikhonov 
> wrote:
>
>> You set wrong value to IGNITE_XML_CONFIG property. The property should
>> contains path to ignite configuration file. For example
>>
>> IGNITE_XML_CONFIG=/u/hongmei/apache-ignite/config/default-config.xml
>>
>> I think you can comment this line in property file and ignite will start
>> with default configuration.
>>
>> On Thu, Jun 9, 2016 at 5:08 PM, Hongmei Zong  wrote:
>>
>>> Hi nikolai,
>>>
>>> Thank you very much for prompt reply!
>>>
>>> I did not find the ignite-config.xml file under my ignite home directory(
>>>  /u/hongmei/apache-ignite/  ).
>>>
>>> I find a "default-config.xml" at the path:
>>> /u/hongmei/apache-ignite/config/default-config.xml
>>>
>>> 
>>>
>>>
>>> 
>>>
>>>
>>> http://www.springframework.org/schema/beans;
>>>
>>>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>>>
>>>xsi:schemaLocation="
>>>
>>>http://www.springframework.org/schema/beans
>>>
>>>http://www.springframework.org/schema/beans/spring-beans.xsd;>
>>>
>>> 
>>>
>>> >> "org.apache.ignite.configuration.IgniteConfiguration">
>>>
>>> 
>>>
>>> >> "org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>>>
>>> 
>>>
>>> >> "org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder"
>>> >
>>>
>>> >> "addresses">
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>> c5hdpe001.c5.runwaynine.com:47500..47509
>>>
>>> 
>>> c5hdpe002.c5.runwaynine.com:47500..47509
>>>
>>> 
>>> c5hdpe003.c5.runwaynine.com:47500..47509
>>>
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>>
>>>
>>> The "cluster.properties" at the path:
>>> /u/hongmei/apache-ignite/config/cluster.properties
>>>
>>> # The number of nodes in the cluster.
>>>
>>> IGNITE_NODE_COUNT=2
>>>
>>>
>>> # The number of CPU Cores for each Apache Ignite node.
>>>
>>> IGNITE_RUN_CPU_PER_NODE=1
>>>
>>>
>>> # The number of Megabytes of RAM for each Apache Ignite node.
>>>
>>> IGNITE_MEMORY_PER_NODE=2048
>>>
>>>
>>> # The version ignite which will be run on nodes.
>>>
>>> IGNITE_VERSION=1.6.0
>>>
>>>
>>> # The hdfs directory which will be used for saving Apache Ignite
>>> disbributives.
>>>
>>> IGNITE_RELEASES_DIR=/user/hongmei/ignite-yarn
>>>
>>>
>>> #The directory which will be used for saving Apache Ignite
>>> distributives(copy .jar file to it).
>>>
>>> IGNITE_WORKING_DIR=/user/hongmei/ignite/workdir
>>>
>>>
>>> #The hdfs path to Apache Ignite config file.
>>>
>>> #IGNITE_XML_CONFIG=/user/yarn/ignite/
>>>
>>> IGNITE_XML_CONFIG=/user/hongmei/ignite/
>>>
>>>
>>> #The hdfs path to libs which will be added to classpath.
>>>
>>> IGNITE_USERS_LIBS=/user/hongmei/ignite/libs/
>>>
>>>
>>> #The constraint on slave hosts.
>>>
>>> #IGNITE_HOSTENAME_CONSTRAINT=
>>>
>>>
>>> IGNITE_LOCAL_WORK_DIR=/u/hongmei/apache-ignite
>>>
>>>
>>>
>>> Thank you very much
>>>
>>> Hongmei
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Jun 9, 2016 at 9:43 AM, Nikolai Tikhonov 
>>> wrote:
>>>
 It seems

Re: Ignite for Spark on YARN Deployment

2016-06-09 Thread Hongmei Zong

Thank you very much Nikolay, I will give it a try! :)

Hongmei

On Thu, Jun 9, 2016 at 10:21 AM, Nikolay Tikhonov 
wrote:

> You set wrong value to IGNITE_XML_CONFIG property. The property should
> contains path to ignite configuration file. For example
>
> IGNITE_XML_CONFIG=/u/hongmei/apache-ignite/config/default-config.xml
>
> I think you can comment this line in property file and ignite will start
> with default configuration.
>
> On Thu, Jun 9, 2016 at 5:08 PM, Hongmei Zong  wrote:
>
>> Hi nikolai,
>>
>> Thank you very much for prompt reply!
>>
>> I did not find the ignite-config.xml file under my ignite home directory(
>>  /u/hongmei/apache-ignite/  ).
>>
>> I find a "default-config.xml" at the path:
>> /u/hongmei/apache-ignite/config/default-config.xml
>>
>> 
>>
>>
>> 
>>
>>
>> http://www.springframework.org/schema/beans;
>>
>>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>>
>>xsi:schemaLocation="
>>
>>http://www.springframework.org/schema/beans
>>
>>http://www.springframework.org/schema/beans/spring-beans.xsd;>
>>
>> 
>>
>> > "org.apache.ignite.configuration.IgniteConfiguration">
>>
>> 
>>
>> > "org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>>
>> 
>>
>> > "org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>>
>> > "addresses">
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>> c5hdpe001.c5.runwaynine.com:47500..47509
>>
>> 
>> c5hdpe002.c5.runwaynine.com:47500..47509
>>
>> 
>> c5hdpe003.c5.runwaynine.com:47500..47509
>>
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>>
>>
>> The "cluster.properties" at the path:
>> /u/hongmei/apache-ignite/config/cluster.properties
>>
>> # The number of nodes in the cluster.
>>
>> IGNITE_NODE_COUNT=2
>>
>>
>> # The number of CPU Cores for each Apache Ignite node.
>>
>> IGNITE_RUN_CPU_PER_NODE=1
>>
>>
>> # The number of Megabytes of RAM for each Apache Ignite node.
>>
>> IGNITE_MEMORY_PER_NODE=2048
>>
>>
>> # The version ignite which will be run on nodes.
>>
>> IGNITE_VERSION=1.6.0
>>
>>
>> # The hdfs directory which will be used for saving Apache Ignite
>> disbributives.
>>
>> IGNITE_RELEASES_DIR=/user/hongmei/ignite-yarn
>>
>>
>> #The directory which will be used for saving Apache Ignite
>> distributives(copy .jar file to it).
>>
>> IGNITE_WORKING_DIR=/user/hongmei/ignite/workdir
>>
>>
>> #The hdfs path to Apache Ignite config file.
>>
>> #IGNITE_XML_CONFIG=/user/yarn/ignite/
>>
>> IGNITE_XML_CONFIG=/user/hongmei/ignite/
>>
>>
>> #The hdfs path to libs which will be added to classpath.
>>
>> IGNITE_USERS_LIBS=/user/hongmei/ignite/libs/
>>
>>
>> #The constraint on slave hosts.
>>
>> #IGNITE_HOSTENAME_CONSTRAINT=
>>
>>
>> IGNITE_LOCAL_WORK_DIR=/u/hongmei/apache-ignite
>>
>>
>>
>> Thank you very much
>>
>> Hongmei
>>
>>
>>
>>
>>
>> On Thu, Jun 9, 2016 at 9:43 AM, Nikolai Tikhonov 
>> wrote:
>>
>>> It seems that your ignite configuration invalid. Could you share 
>>> ignite-config.xml
>>> and /u/hongmei/apache-ignite/config/cluster.properties?
>>>
>>> On Thu, Jun 9, 2016 at 4:35 PM, Hongmei Zong  wrote:
>>>
 Thank you very much Nikolai !

 I found another issue regarding my Ignite YARN Integration:

 I  run Ignite YARN application on one client server machine
 'c5hdpe001', screenshot as the following:
 Then I log onto one of the container node and the log is as the
 following:

 [hongmei@c5hdpe001 apache-ignite]$ hadoop jar
 /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
 /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
 /u/hongmei/apache-ignite/config/cluster.properties

 WARNING: Use "yarn jar" to launch YARN applications.

 16/06/09 08:55:20 INFO impl.TimelineClientImpl: Timeline service
 address: http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/

 16/06/09 08:55:22 INFO client.ConfiguredRMFailoverProxyProvider:
 Failing over to rm2

 16/06/09 08:55:24 INFO impl.YarnClientImpl: Submitted application
 application_1464374946035_29511

Re: Ignite for Spark on YARN Deployment

2016-06-09 Thread Nikolay Tikhonov

You set wrong value to IGNITE_XML_CONFIG property. The property should
contains path to ignite configuration file. For example

IGNITE_XML_CONFIG=/u/hongmei/apache-ignite/config/default-config.xml

I think you can comment this line in property file and ignite will start
with default configuration.

On Thu, Jun 9, 2016 at 5:08 PM, Hongmei Zong  wrote:

> Hi nikolai,
>
> Thank you very much for prompt reply!
>
> I did not find the ignite-config.xml file under my ignite home directory(
>  /u/hongmei/apache-ignite/  ).
>
> I find a "default-config.xml" at the path:
> /u/hongmei/apache-ignite/config/default-config.xml
>
> 
>
>
> 
>
>
> http://www.springframework.org/schema/beans;
>
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>
>xsi:schemaLocation="
>
>http://www.springframework.org/schema/beans
>
>http://www.springframework.org/schema/beans/spring-beans.xsd;>
>
> 
>
>  "org.apache.ignite.configuration.IgniteConfiguration">
>
> 
>
>  "org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>
> 
>
>  "org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
>  >
>
> 
>
> 
>
> 
>
> 
>
> 
> c5hdpe001.c5.runwaynine.com:47500..47509
>
> 
> c5hdpe002.c5.runwaynine.com:47500..47509
>
> 
> c5hdpe003.c5.runwaynine.com:47500..47509
>
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
>
>
> The "cluster.properties" at the path:
> /u/hongmei/apache-ignite/config/cluster.properties
>
> # The number of nodes in the cluster.
>
> IGNITE_NODE_COUNT=2
>
>
> # The number of CPU Cores for each Apache Ignite node.
>
> IGNITE_RUN_CPU_PER_NODE=1
>
>
> # The number of Megabytes of RAM for each Apache Ignite node.
>
> IGNITE_MEMORY_PER_NODE=2048
>
>
> # The version ignite which will be run on nodes.
>
> IGNITE_VERSION=1.6.0
>
>
> # The hdfs directory which will be used for saving Apache Ignite
> disbributives.
>
> IGNITE_RELEASES_DIR=/user/hongmei/ignite-yarn
>
>
> #The directory which will be used for saving Apache Ignite
> distributives(copy .jar file to it).
>
> IGNITE_WORKING_DIR=/user/hongmei/ignite/workdir
>
>
> #The hdfs path to Apache Ignite config file.
>
> #IGNITE_XML_CONFIG=/user/yarn/ignite/
>
> IGNITE_XML_CONFIG=/user/hongmei/ignite/
>
>
> #The hdfs path to libs which will be added to classpath.
>
> IGNITE_USERS_LIBS=/user/hongmei/ignite/libs/
>
>
> #The constraint on slave hosts.
>
> #IGNITE_HOSTENAME_CONSTRAINT=
>
>
> IGNITE_LOCAL_WORK_DIR=/u/hongmei/apache-ignite
>
>
>
> Thank you very much
>
> Hongmei
>
>
>
>
>
> On Thu, Jun 9, 2016 at 9:43 AM, Nikolai Tikhonov 
> wrote:
>
>> It seems that your ignite configuration invalid. Could you share 
>> ignite-config.xml
>> and /u/hongmei/apache-ignite/config/cluster.properties?
>>
>> On Thu, Jun 9, 2016 at 4:35 PM, Hongmei Zong  wrote:
>>
>>> Thank you very much Nikolai !
>>>
>>> I found another issue regarding my Ignite YARN Integration:
>>>
>>> I  run Ignite YARN application on one client server machine 'c5hdpe001',
>>> screenshot as the following:
>>> Then I log onto one of the container node and the log is as the
>>> following:
>>>
>>> [hongmei@c5hdpe001 apache-ignite]$ hadoop jar
>>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
>>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
>>> /u/hongmei/apache-ignite/config/cluster.properties
>>>
>>> WARNING: Use "yarn jar" to launch YARN applications.
>>>
>>> 16/06/09 08:55:20 INFO impl.TimelineClientImpl: Timeline service
>>> address: http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/
>>>
>>> 16/06/09 08:55:22 INFO client.ConfiguredRMFailoverProxyProvider: Failing
>>> over to rm2
>>>
>>> 16/06/09 08:55:24 INFO impl.YarnClientImpl: Submitted application
>>> application_1464374946035_29511
>>>
>>> Jun 09, 2016 8:55:24 AM org.apache.ignite.yarn.IgniteYarnClient main
>>>
>>> INFO: Submitted application. Application id:
>>> application_1464374946035_29511
>>>
>>> Jun 09, 2016 8:55:27 AM org.apache.ignite.yarn.IgniteYarnClient main
>>> *INFO: Application application_1464374946035_29511 is RUNNING.*
>>>
>>>
>>> Then I log onto one of the container

Re: Ignite for Spark on YARN Deployment

2016-06-09 Thread Hongmei Zong

Hi nikolai,

Thank you very much for prompt reply!

I did not find the ignite-config.xml file under my ignite home directory(
 /u/hongmei/apache-ignite/  ).

I find a "default-config.xml" at the path:
/u/hongmei/apache-ignite/config/default-config.xml







http://www.springframework.org/schema/beans;

   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;

   xsi:schemaLocation="

   http://www.springframework.org/schema/beans

   http://www.springframework.org/schema/beans/spring-beans.xsd;>
























c5hdpe001.c5.runwaynine.com:47500..47509


c5hdpe002.c5.runwaynine.com:47500..47509


c5hdpe003.c5.runwaynine.com:47500..47509




















The "cluster.properties" at the path:
/u/hongmei/apache-ignite/config/cluster.properties

# The number of nodes in the cluster.

IGNITE_NODE_COUNT=2


# The number of CPU Cores for each Apache Ignite node.

IGNITE_RUN_CPU_PER_NODE=1


# The number of Megabytes of RAM for each Apache Ignite node.

IGNITE_MEMORY_PER_NODE=2048


# The version ignite which will be run on nodes.

IGNITE_VERSION=1.6.0


# The hdfs directory which will be used for saving Apache Ignite
disbributives.

IGNITE_RELEASES_DIR=/user/hongmei/ignite-yarn


#The directory which will be used for saving Apache Ignite
distributives(copy .jar file to it).

IGNITE_WORKING_DIR=/user/hongmei/ignite/workdir


#The hdfs path to Apache Ignite config file.

#IGNITE_XML_CONFIG=/user/yarn/ignite/

IGNITE_XML_CONFIG=/user/hongmei/ignite/


#The hdfs path to libs which will be added to classpath.

IGNITE_USERS_LIBS=/user/hongmei/ignite/libs/


#The constraint on slave hosts.

#IGNITE_HOSTENAME_CONSTRAINT=


IGNITE_LOCAL_WORK_DIR=/u/hongmei/apache-ignite



Thank you very much

Hongmei





On Thu, Jun 9, 2016 at 9:43 AM, Nikolai Tikhonov 
wrote:

> It seems that your ignite configuration invalid. Could you share 
> ignite-config.xml
> and /u/hongmei/apache-ignite/config/cluster.properties?
>
> On Thu, Jun 9, 2016 at 4:35 PM, Hongmei Zong  wrote:
>
>> Thank you very much Nikolai !
>>
>> I found another issue regarding my Ignite YARN Integration:
>>
>> I  run Ignite YARN application on one client server machine 'c5hdpe001',
>> screenshot as the following:
>> Then I log onto one of the container node and the log is as the following:
>>
>> [hongmei@c5hdpe001 apache-ignite]$ hadoop jar
>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
>> /u/hongmei/apache-ignite/config/cluster.properties
>>
>> WARNING: Use "yarn jar" to launch YARN applications.
>>
>> 16/06/09 08:55:20 INFO impl.TimelineClientImpl: Timeline service address:
>> http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/
>>
>> 16/06/09 08:55:22 INFO client.ConfiguredRMFailoverProxyProvider: Failing
>> over to rm2
>>
>> 16/06/09 08:55:24 INFO impl.YarnClientImpl: Submitted application
>> application_1464374946035_29511
>>
>> Jun 09, 2016 8:55:24 AM org.apache.ignite.yarn.IgniteYarnClient main
>>
>> INFO: Submitted application. Application id:
>> application_1464374946035_29511
>>
>> Jun 09, 2016 8:55:27 AM org.apache.ignite.yarn.IgniteYarnClient main
>> *INFO: Application application_1464374946035_29511 is RUNNING.*
>>
>>
>> Then I log onto one of the container node and the log is as the
>> following: "It failed to start grid".
>> Any good suggestion? Thank you very much!!!
>>
>>
>> class org.apache.ignite.IgniteException: Failed to instantiate Spring XML
>> application context
>> [springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/,
>> err=Line 1 in XML document from URL
>> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
>> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
>> 1; columnNumber: 1; Content is not allowed in prolog.]
>> at
>>

Re: Ignite for Spark on YARN Deployment

2016-06-09 Thread Nikolai Tikhonov

It seems that your ignite configuration invalid. Could you share
ignite-config.xml
and /u/hongmei/apache-ignite/config/cluster.properties?

On Thu, Jun 9, 2016 at 4:35 PM, Hongmei Zong  wrote:

> Thank you very much Nikolai !
>
> I found another issue regarding my Ignite YARN Integration:
>
> I  run Ignite YARN application on one client server machine 'c5hdpe001',
> screenshot as the following:
> Then I log onto one of the container node and the log is as the following:
>
> [hongmei@c5hdpe001 apache-ignite]$ hadoop jar
> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
> /u/hongmei/apache-ignite/config/cluster.properties
>
> WARNING: Use "yarn jar" to launch YARN applications.
>
> 16/06/09 08:55:20 INFO impl.TimelineClientImpl: Timeline service address:
> http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/
>
> 16/06/09 08:55:22 INFO client.ConfiguredRMFailoverProxyProvider: Failing
> over to rm2
>
> 16/06/09 08:55:24 INFO impl.YarnClientImpl: Submitted application
> application_1464374946035_29511
>
> Jun 09, 2016 8:55:24 AM org.apache.ignite.yarn.IgniteYarnClient main
>
> INFO: Submitted application. Application id:
> application_1464374946035_29511
>
> Jun 09, 2016 8:55:27 AM org.apache.ignite.yarn.IgniteYarnClient main
> *INFO: Application application_1464374946035_29511 is RUNNING.*
>
>
> Then I log onto one of the container node and the log is as the following:
> "It failed to start grid".
> Any good suggestion? Thank you very much!!!
>
>
> class org.apache.ignite.IgniteException: Failed to instantiate Spring XML
> application context
> [springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/,
> err=Line 1 in XML document from URL
> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
> 1; columnNumber: 1; Content is not allowed in prolog.]
> at
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:904)
> at org.apache.ignite.Ignition.start(Ignition.java:350)
> at
> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:302)
> *Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> instantiate Spring XML application context
> [springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/,
> err=Line 1 in XML document from URL
> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
> 1; columnNumber: 1; Content is not allowed in prolog.]*
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391)
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104)
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98)
> at
> org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:606)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:807)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:716)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:586)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:556)
> at org.apache.ignite.Ignition.start(Ignition.java:347)
> ... 1 more
> *Caused by:
> org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line
> 1 in XML document from URL
> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
> 1; columnNumber: 1; Content is not allowed in prolog.*
> at
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:398)
> at
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:335)
> at
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:303)
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:379)
> ... 9 more
> *Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1;
> Content

Re: Ignite for Spark on YARN Deployment

2016-06-09 Thread Hongmei Zong

Thank you very much Nikolai !

I found another issue regarding my Ignite YARN Integration:

I  run Ignite YARN application on one client server machine 'c5hdpe001',
screenshot as the following:
Then I log onto one of the container node and the log is as the following:

[hongmei@c5hdpe001 apache-ignite]$ hadoop jar
/u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
/u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
/u/hongmei/apache-ignite/config/cluster.properties

WARNING: Use "yarn jar" to launch YARN applications.

16/06/09 08:55:20 INFO impl.TimelineClientImpl: Timeline service address:
http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/

16/06/09 08:55:22 INFO client.ConfiguredRMFailoverProxyProvider: Failing
over to rm2

16/06/09 08:55:24 INFO impl.YarnClientImpl: Submitted application
application_1464374946035_29511

Jun 09, 2016 8:55:24 AM org.apache.ignite.yarn.IgniteYarnClient main

INFO: Submitted application. Application id: application_1464374946035_29511

Jun 09, 2016 8:55:27 AM org.apache.ignite.yarn.IgniteYarnClient main
*INFO: Application application_1464374946035_29511 is RUNNING.*


Then I log onto one of the container node and the log is as the following:
"It failed to start grid".
Any good suggestion? Thank you very much!!!


class org.apache.ignite.IgniteException: Failed to instantiate Spring XML
application context
[springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/,
err=Line 1 in XML document from URL
[file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
1; columnNumber: 1; Content is not allowed in prolog.]
at
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:904)
at org.apache.ignite.Ignition.start(Ignition.java:350)
at
org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:302)
*Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
instantiate Spring XML application context
[springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/,
err=Line 1 in XML document from URL
[file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
1; columnNumber: 1; Content is not allowed in prolog.]*
at
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391)
at
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104)
at
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98)
at
org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:606)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:807)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:716)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:586)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:556)
at org.apache.ignite.Ignition.start(Ignition.java:347)
... 1 more
*Caused by:
org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line
1 in XML document from URL
[file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
1; columnNumber: 1; Content is not allowed in prolog.*
at
org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:398)
at
org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:335)
at
org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:303)
at
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:379)
... 9 more
*Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1;
Content is not allowed in prolog.*
at
com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:198)
at
com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
at
com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:441)

State of initially started cache with CacheRebalanceMode.SYNC ?

2016-06-09 Thread Kristian Rosenvold

The javadoc on CacheRebalanceMode.SYNC seems to indicate that the
cache should block until rebalancing is complete.
When I run the code below, the assert statement fails unless I add the
explicit call to cache.rebalance().get().

Am I doing something wrong ?

Kristian


CacheConfiguration config = new CacheConfiguration();

config.setCacheMode(CacheMode.REPLICATED);
config.setRebalanceMode(CacheRebalanceMode.SYNC);
final IgniteCache cache = ignite.getOrCreateCache(config);
// cache.rebalance().get();
assertThat(cache.localSize(CachePeekMode.ALL)).isEqualTo(knownRemoteSize);

Exception on Ignite shutdown

2016-06-09 Thread colinc

I'm running some tests on an Ignite cluster using an Ignite client. I'm
receiving a stacktrace quite regularly in the client - especially when the
cluster is freshly started. The exception occurs at the point that the
client is closed down (i.e. the try block completes) - see below log trace.

The exception doesn't seem to be related to what the test is actually doing
with Ignite.

public void helloWorldCompute() {
Ignition.setClientMode(true);   
try (Ignite ignite = Ignition.start(CONFIG)) {
ignite.compute().broadcast(() -> 
System.out.println("Hello World"));
}
}

INFO  2016-06-09 11:30:49,385   [Log4J Redirect INFO-1]
AladdinLog4JAppender$SystemOutProxy - [11:30:49] Topology snapshot [ver=2,
servers=1, clients=1, CPUs=8, heap=4.5GB]

[11:30:49,666][SEVERE][tcp-client-disco-sock-writer-#2%null%][TcpDiscoverySpi]
Failed to send message: TcpDiscoveryPingResponse [clientExists=false,
super=TcpDiscoveryAbstractMessage [sndNodeId=null,
id=1e3d6b43551-cdbda7fe-4fd6-4339-855f-25d02ebb03bf, verifierNodeId=null,
topVer=0, pendingIdx=0, failedNodes=null, isClient=true]]
class org.apache.ignite.IgniteCheckedException: Failed to serialize object:
TcpDiscoveryPingResponse [clientExists=false,
super=TcpDiscoveryAbstractMessage [sndNodeId=null,
id=1e3d6b43551-cdbda7fe-4fd6-4339-855f-25d02ebb03bf, verifierNodeId=null,
topVer=0, pendingIdx=0, failedNodes=null, isClient=true]]
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.marshal(JdkMarshaller.java:82)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.writeToSocket(TcpDiscoverySpi.java:1381)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.writeToSocket(TcpDiscoverySpi.java:1352)
at
org.apache.ignite.spi.discovery.tcp.ClientImpl$SocketWriter.body(ClientImpl.java:1071)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
Caused by: java.net.SocketException: Socket closed
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at 
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at
org.apache.ignite.marshaller.jdk.JdkMarshallerOutputStreamWrapper.flush(JdkMarshallerOutputStreamWrapper.java:58)
at
java.io.ObjectOutputStream$BlockDataOutputStream.flush(ObjectOutputStream.java:1823)
at java.io.ObjectOutputStream.flush(ObjectOutputStream.java:719)
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.marshal(JdkMarshaller.java:79)
... 4 more
INFO  2016-06-09 11:30:49,677   [Log4J Redirect INFO-1]
AladdinLog4JAppender$SystemOutProxy - [11:30:49] Ignite node stopped OK
[uptime=00:00:00:292]


Is this a cause for concern, or just something I should ignore? Is there any
way to supress the error?

Thanks,
Colin.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Exception-on-Ignite-shutdown-tp5551.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Maven conflicts within ignite-core from duplicated classes

2016-06-09 Thread Pete Campton

Hello,

I've come across an issue where there are a couple of classes duplicated
from org.jetbrains.annotations within the ignite-core module which conflict
with other part of our code base.

As these files are duplicated rather than being referenced as a maven
dependency I am unable to exclude them or reference a specific version of
the org.jetbrains.annotations artifact.

Specifically, the included version cannot be used to annotate types
ignite-core-1.6.0.jar:org.jetbrains.annotations.Nullable ...
@Target({ElementType.METHOD, ElementType.FIELD, ElementType.PARAMETER,
ElementType.LOCAL_VARIABLE})
annotations-15.0.jar:org.jetbrains.annotations.Nullable ...
@Target({ElementType.METHOD, ElementType.FIELD, ElementType.PARAMETER,
ElementType.LOCAL_VARIABLE, ElementType.TYPE_USE})

As far as I know there isn't an easy way of excluding these files from
within the ignite-core module. Is there are reason why these "external"
files have been duplicated within the ignite code base rather than being
referenced as a maven dependency? Or alternatively, has anyone come across
a way of avoiding these conflicts?

Thanks
Pete

Re: Ignite Write Behind performance

2016-06-09 Thread amitpa

We use it to conservative 2 now in our current setting



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Write-Behind-performance-tp5385p5549.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Slow Transaction Performance

2016-06-09 Thread pragmaticbigdata

I am executing my tests with apache ignite 1.6. With a 5 node cluster
(servers=5, clients=0, CPUs=16, heap=17.0GB), I create a partitioned cache
that is preloaded with 1 million entries (IgniteCache). 

Updating 11k records in this partitioned cache is taking between 1.6 secs
while it takes 17 secs to perform the same update when the cache atomicity
mode is changed to TRANSACTIONAL and the updates (putAll operation) are
executed in a transaction (OPTIMISTIC, READ_COMMITTED).

What are some of the options to improve the transaction performance from 17
secs?

Thanks,
Amit.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Slow-Transaction-Performance-tp5548.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

"Select count(*) " returns incorrect records for distributed cache.

2016-06-09 Thread Rupinder Virk

Hi,

We are using Ignite-1.5.0-final version. On a 3 node cluster we created a
cache containing 10K records with *PARTITIONED * and *REPLICATED * mode.

Why the following query returns *1/3rd records* in PARTITIONED  mode
and *3x records* in REPLICATED  mode.

SqlFieldsQuery qry11 =  new SqlFieldsQuery("select count(*) from  \"" +
TABLE_CACHE_NAME + "\".Table t ");


Thanks,
Rupinder Virk



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Select-count-returns-incorrect-records-for-distributed-cache-tp5546.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Meaning of multiple index types

2016-06-09 Thread colinc

When configuring a cache for use with SqlFieldsQuery, it is necessary to call
setIndexedTypes(). The below example from the documentation defines 3 types
to index:

ccfg.setIndexedTypes(
  MyKey.class, MyValue.class,
  Long.class, MyOtherValue.class,
  UUID.class, String.class
);

If an individual cache can only be used to store objects of one type e.g.
MyValue, then what is the meaning of specifying 3 pairs of types in this
case? It seems possible to write a query for any of the indexed types but it
is only possible to put one type of object in the cache - so querying the
other types returns nothing.

I'm interested in this in case it allows me to index and query across a
graph of objects. For example if the MyValue class has a collection of
MyOtherValue objects, then would it be possible to index and query both
objects in the same cache? Or is it only ever possible to query main cache
type?

Thanks,
Colin.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Meaning-of-multiple-index-types-tp5545.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Slow Transaction Performance

Re: Persistence using Cassandra

Re: Slow Transaction Performance

Re: Persistence using Cassandra

Re: Persistence using Cassandra

Re: Ignite for Spark on YARN Deployment

Re: Ignite for Spark on YARN Deployment

Re: Ignite for Spark on YARN Deployment

Re: Ignite for Spark on YARN Deployment

Re: Ignite for Spark on YARN Deployment

Re: Ignite for Spark on YARN Deployment

Re: Ignite for Spark on YARN Deployment

Re: Ignite for Spark on YARN Deployment

State of initially started cache with CacheRebalanceMode.SYNC ?

Exception on Ignite shutdown

Maven conflicts within ignite-core from duplicated classes

Re: Ignite Write Behind performance

Slow Transaction Performance

"Select count(*) " returns incorrect records for distributed cache.

Meaning of multiple index types

20 matches

Site Navigation

Mail list logo

Footer information