Ignite inside OData WebAPI

[Murthy Kakarlamudi]

Hi,
   I am trying to expose Cache contents as an Odata API. I created the
Odata API using ASP.NET WebAPI. Inside the API implementation, I am
initializing Ignite in cache mode. Below is the method I am using to
retrieve a listing. I have 2 questions.

1. I was not able to use Select option in AsCacheQueryable. How to get a
list of all the entries in the cache from LINQ?
2. Embedded IIS server quits the process after it hits var ignite =
Ignition.Start(cfg); statement. Processing stops beyond this statement. I
confirmed this in debug mode. It does not throw any errors in the console.
Not sure if it is possible to start Ignite in client mode in Web API that
is deployed on IIS.

Any help will be much appreciated. Thanks.

public List GetMovies()
{
var cfg = new IgniteConfiguration
{
BinaryConfiguration = new
Apache.Ignite.Core.Binary.BinaryConfiguration(typeof(Movie),
typeof(StarRating), typeof(Person)),
JvmOptions = new List { "-Xms512m", "-Xmx1024m" }
};
Ignition.ClientMode = true;
var ignite = Ignition.Start(cfg);
var cache = ignite.GetOrCreateCache(new
CacheConfiguration
{
Name = "myMusicCache",
QueryEntities = new[]
{
new QueryEntity(typeof(int), typeof(Movie))

}
});
List movies = new List();
IQueryable> qry =
cache.AsCacheQueryable().Where(m => m.Key == 1);

foreach (ICacheEntry entry in qry)
{
Console.WriteLine("Movie: " + entry.Value);
movies.Add(entry.Value);
}


return movies;
}

Re: Re: Increase Ignite instances can't increase the speed of compute

[胡永亮/Bob]

Hi, Val

Thank you very much, I think this is very helpful for me.

I will change my compute code, and have a try.



Bob
 
From: vkulichenko
Date: 2016-09-23 07:00
To: user@ignite.apache.org
Subject: Re: Re: Increase Ignite instances can't increase the speed of compute
Hi Bob,
 
Most likely these are distributed cache operations and since they are all
synchronous, you wait for each of them one by one. You can try to batch your
updates to improve performance or use data streamer [1]. It would be even
better if you used affinity collocation [2] and made sure that all the
updates are local and network is not used during the computation.
 
[1] https://apacheignite.readme.io/docs/data-streamers
[2] https://apacheignite.readme.io/docs/affinity-collocation
 
-Val
 
 
 
--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Increase-Ignite-instances-can-t-increase-the-speed-of-compute-tp7670p7891.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.


---
Confidentiality Notice: The information contained in this e-mail and any 
accompanying attachment(s)
is intended only for the use of the intended recipient and may be confidential 
and/or privileged of
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of 
this communication is
not the intended recipient, unauthorized use, forwarding, printing,  storing, 
disclosure or copying
is strictly prohibited, and may be unlawful.If you have received this 
communication in error,please
immediately notify the sender by return e-mail, and delete the original message 
and all copies from
your system. Thank you.
---

Re: Logging failing, tried the troubleshoot steps already

[Himetic]

I thought the purpose of using log4j was to avoid the steps in that link? 
The link I included says "you can either add ignite-log4j module to the list
of the used jars so that Ignite would use Log4j as a logging subsystem, or
alter default Spark classpath as described here."  Does "adding it to the
list of used jars" mean something other than adding it to the pom?  I've
added it to the pom and that doesn't seem to be doing anything.

The error is the same before and after adding the log4j jar, doesn't seem to
have changed anything.  This is the runtime error, not a build error. I'm
assuming I missed some important step in activating log4j, I just don't know
what "adding it to the list of used jars" means, if not adding it to the
pom.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Logging-failing-tried-the-troubleshoot-steps-already-tp7882p7895.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Logging failing, tried the troubleshoot steps already

[vkulichenko]

Hi,

First of all, you need not only the ignite-log4j jar, but the whole
ignite-log4j module with dependencies. Refer to [1] for details.

Also as far as I understand, the log4j did replaced the Java logger and you
got a different error. Is this right? If so, you should probably just
configure log4j properly. Refer to log4j docs for this.

[1] http://apacheignite.gridgain.org/v1.2/docs/installation--deployment

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Logging-failing-tried-the-troubleshoot-steps-already-tp7882p7894.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cannot query on a cache using Cassandra as a persistent store

[vkulichenko]

Hi Guang,

Please show the source of the Person class. Does the 'age' field has
@QuerySqlField on it?

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cannot-query-on-a-cache-using-Cassandra-as-a-persistent-store-tp7870p7893.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Re: Increase Ignite instances can't increase the speed of compute

[vkulichenko]

Hi Bob,

Most likely these are distributed cache operations and since they are all
synchronous, you wait for each of them one by one. You can try to batch your
updates to improve performance or use data streamer [1]. It would be even
better if you used affinity collocation [2] and made sure that all the
updates are local and network is not used during the computation.

[1] https://apacheignite.readme.io/docs/data-streamers
[2] https://apacheignite.readme.io/docs/affinity-collocation

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Increase-Ignite-instances-can-t-increase-the-speed-of-compute-tp7670p7891.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite data nodes without participating in transaction

[vkulichenko]

Hi,

DR is not supported out of the box in Ignite. However, GridGain provides
this functionality [1] as a part of there paid solution.

[1] https://gridgain.readme.io/docs/data-center-replication

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-data-nodes-without-participating-in-transaction-tp7856p7892.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: retrieving values from ignite cache but Rdd don´t show values and long time Java

[vkulichenko]

Hi,

Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscr...@ignite.apache.org and follow simple instructions in
the reply.


rodrigo thread wrote
> I have an application that saves Rdd's spark with some values (key, value)
> to ignite cache, then I have another application that gets this cache
> ignite and extract these values (key, value), the problem is when you draw
> can not get the values and show them I'm using Java?
> 
> IgniteRDDInteger, String javaIRDDRet = ic.fromCache(cacheName);   
> JavaRDDTuple2Integer, String> javardd =
> javaIRDDRet.toJavaRDD();
> int j = (int) javardd.count();
> System.out.println("values count =" + j);
> for (int i=0;ij;i++) {
> try {
> Tuple2Integer, String retorno1 =(Tuple2Integer,
> String) javardd.collect().get(i);
> 
> System.out.println(retorno1._1+" "+retorno1._2);
> i++;
> 
> } catch (Exception ex) {
> ex.printStackTrace();
> }
> }

If possible, please provide the full test that will reproduce the issue.
Someone in the community will take a look.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/retrieving-values-from-ignite-cache-but-Rdd-don-t-show-values-and-long-time-Java-tp7812p7888.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: One failing node stalling the whole cluster

[vkulichenko]

Hi Sparkle,

Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscr...@ignite.apache.org and follow simple instructions in
the reply.


sparkle_j wrote
> In one of our tests, we noticed that Ignite's TcpCommunicationSpi object
> is growing and retaining over 70% of heap memory. Not sure why. Is this a
> known issue..? Please let us know if this is already addressed.
> 
> We are testing with Ignite1.5, 10GB heap, with moderate amount of data on
> start up and just connect and disconnect dummy clients to grid. This is
> causing the heap to grow over time and eventually Full GC and node is
> being killed by Ignite. This is also causing cluster wide instability.
> Please see HeapDump screenshots attached.
> 
> TcpCommSpi.png
>   
> 
> dom-tree-1.png
>   
> 
> dom-tree.png
>   
Can you take the full heap dump and upload it somewhere?

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/One-failing-node-stalling-the-whole-cluster-tp5372p7887.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster configuration across data centers

[vkulichenko]

Hi,

Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscr...@ignite.apache.org and follow simple instructions in
the reply.


newbie wrote
> We are evaluating Ignite for caches and have a REPLICATED, TRANSACTIONAL,
> write through and read through cache with zookeeper based discovery.
> 
> Our server setup include some nodes in one data center and some in another
> data center for disaster recovery.
> 
> Including all nodes equally in the cluster seems to increase the Ignite
> transaction time as the ones in other data center have higher network
> latency.
> 
> We would like to configure the cache/cluster so that all cache
> requests/transactions are served from data center one and the nodes in
> data center two are updated 
/
> asynchronously 
/
>  for DR purposes. 
> 
> Can you point towards what we should be trying in terms of
> clusterGroups/clientModes/etc.?
> 
> Also, is it possible to switch the nodes in DR data center to start
> serving cache requests?

There is no out of the box support for this. You can span a single cluster
across two data centers, but this will come with a performance penalty due
to large network latency. If you have two separate clusters, you will need
to make sure that the data is transferred from one data center to another.

If you're interested, GridGain provides such functionality as a part of
their payed solution:
https://gridgain.readme.io/docs/data-center-replication

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cluster-configuration-across-data-centers-tp7795p7884.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Logging failing, tried the troubleshoot steps already

[Himetic]

I'm trying to implement caching using ignite, and when I start it up
(Ignition.start) it throws an error:

Can't load log handler "org.apache.ignite.logger.java.JavaLoggerFileHandler"
java.lang.ClassNotFoundException:
org.apache.ignite.logger.java.JavaLoggerFileHandler
java.lang.ClassNotFoundException:
org.apache.ignite.logger.java.JavaLoggerFileHandler

The help docs say to simply add the ignite-log4j jar
(http://apacheignite.gridgain.org/v1.2/docs/troubleshooting), which I added
to my pom, but it's still throwing the same error.  I noticed an exception
is the build that it didn't have a log4j configuration, so I added that file
which got rid of that exception, but we're still getting the "can't load log
handler" exception, and still not seeing ignite in the logs.

I looked at the alternate troubleshoot steps, but I have no idea what my
SPARK_HOME is supposed to be.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Logging-failing-tried-the-troubleshoot-steps-already-tp7882.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cannot recreate cache after cache.destroy()

[vkulichenko]

Hi Alex,

You're right. I created a ticket for this issue:
https://issues.apache.org/jira/browse/IGNITE-3957

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cannot-recreate-cache-after-cache-destroy-tp7291p7881.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Affinity and Topology versions

[edwardkblk]

Affinity in the cluster is coupled to the topology.  Any affinity based
processing requires a stable topology.  Does Ignite guarantee that all nodes
are executing a transaction on the same affinity version?  Please correct me
if I'm wrong but based on my observation in  Race condition with partition
affinity mapping

  
there seems to be a gap here.  

Also, we would certainly benefit from 
1. exposing access to affintyTopologyVersion from Affinity interface for
tracking purposes and 
2. having functionality to await for specific affinity version before
proceeding with execution

Appreciate your thoughts



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Affinity-and-Topology-versions-tp7880.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Out of order updates for CacheEntryListeners and multi-cache transactions

[ross.anderson]

Interestingly, setting cache key affinity appears to resolve the issue,
however is there any way to avoid this for cases where there isn't a common
cache key on every item, such as foreign keys?

e.g.




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Out-of-order-updates-for-CacheEntryListeners-and-multi-cache-transactions-tp7864p7878.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite LINQ Help

[Murthy Kakarlamudi]

That worked. Thanks Pavel.

On Thu, Sep 22, 2016 at 2:57 AM, Pavel Tupitsyn 
wrote:

> Hi,
>
> Your class is serialized with .NET BinaryFormatter, that's why SQL and
> LINQ does not work [1].
>
> To fix this, remove [Serializable] attribute and register the class in
> BinaryConfiguration:
>
>
>   var cfg = new IgniteConfiguration { BinaryConfiguration = new
> BinaryConfiguration(typeof(MyModel) };
>   using (var ignite = Ignition.Start(cfg))
>   ...
>
>
> [1] https://apacheignite-net.readme.io/docs/serialization
>
> On Thu, Sep 22, 2016 at 6:15 AM, Murthy Kakarlamudi 
> wrote:
>
>> Hi all,
>>  I created a sample example based on the LINQExample provided in
>> Ignite Examples. However I am not getting expected results. Any help is
>> much appreciated. In my for loop at  the end I am expecting the last 10
>> entries. But nothing is getting displayed. Any help is much appreciated.
>>
>> using System;
>> using System.Collections.Generic;
>> using System.Linq;
>> using System.Text;
>> using System.Threading.Tasks;
>> using Apache.Ignite.Core;
>> using Apache.Ignite.Linq;
>> using Apache.Ignite.Core.Cache;
>> using Apache.Ignite.Core.Cache.Configuration;
>> using Apache.Ignite.Core.Cache.Query;
>>
>> using MyIgnite;
>>
>> namespace MyIgniteConsole
>> {
>> class Program
>> {
>> /// 
>> ///
>> /// 
>> /// 
>> static void Main(string[] args)
>> {
>> using (var ignite = Ignition.Start())
>> {
>> Console.WriteLine();
>> var cache = ignite.GetOrCreateCache(new
>> CacheConfiguration
>> {
>> Name = "myCache",
>> QueryEntities = new[]
>> {
>> new QueryEntity(typeof(int), typeof(MyModel))
>>
>> }
>> });
>>
>> for (int i = 0; i < 100; i++)
>> {
>> cache.Put(i, new MyModel(i, "Test"));
>> }
>>
>> Console.WriteLine("Cache size: "+cache.Get(10).MyKey);
>>
>> IQueryable> qry =
>> cache.AsCacheQueryable().Where(obj => obj.Value.MyKey >
>> 90);
>>
>> Console.WriteLine();
>> Console.WriteLine(">>> Objects with Key greater than 90 "
>> );
>>
>> foreach (ICacheEntry entry in qry)
>> Console.WriteLine(">>>" + entry.Value);
>>
>> Console.WriteLine();
>> }
>>
>>
>>
>> Console.ReadLine();
>> }
>> }
>> }
>>
>>
>> *MyModel Class:*
>> using System;
>> using System.Collections.Generic;
>> using System.Linq;
>> using System.Text;
>> using System.Threading.Tasks;
>> using Apache.Ignite.Core.Cache.Configuration;
>>
>> namespace MyIgniteConsole
>> {
>> [Serializable]
>> class MyModel
>> {
>>
>> public MyModel(int MyKeyIn, String MyStringIn)
>> {
>> MyKey = MyKeyIn;
>> MyString = MyStringIn;
>> }
>>
>> [QuerySqlField]
>> public int MyKey { get; set; }
>>
>> public String MyString { get; set; }
>> }
>> }
>>
>>
>

Re: Out of order updates for CacheEntryListeners and multi-cache transactions

[ross.anderson]

I've added a code example to github

https://github.com/rossdanderson/IgniteOutOfOrderUpdate

I appreciate any help here as it basically means there's no way to guarantee
read-after-write consistency for events triggered off of a transaction
involving multiple caches, even if they access the data in exactly the same
order as the transaction changed it.

Cheers,
Ross



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Out-of-order-updates-for-CacheEntryListeners-and-multi-cache-transactions-tp7864p7874.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite LINQ Help

[Pavel Tupitsyn]

Hi,

Your class is serialized with .NET BinaryFormatter, that's why SQL and LINQ
does not work [1].

To fix this, remove [Serializable] attribute and register the class in
BinaryConfiguration:


  var cfg = new IgniteConfiguration { BinaryConfiguration = new
BinaryConfiguration(typeof(MyModel) };
  using (var ignite = Ignition.Start(cfg))
  ...


[1] https://apacheignite-net.readme.io/docs/serialization

On Thu, Sep 22, 2016 at 6:15 AM, Murthy Kakarlamudi 
wrote:

> Hi all,
>  I created a sample example based on the LINQExample provided in
> Ignite Examples. However I am not getting expected results. Any help is
> much appreciated. In my for loop at  the end I am expecting the last 10
> entries. But nothing is getting displayed. Any help is much appreciated.
>
> using System;
> using System.Collections.Generic;
> using System.Linq;
> using System.Text;
> using System.Threading.Tasks;
> using Apache.Ignite.Core;
> using Apache.Ignite.Linq;
> using Apache.Ignite.Core.Cache;
> using Apache.Ignite.Core.Cache.Configuration;
> using Apache.Ignite.Core.Cache.Query;
>
> using MyIgnite;
>
> namespace MyIgniteConsole
> {
> class Program
> {
> /// 
> ///
> /// 
> /// 
> static void Main(string[] args)
> {
> using (var ignite = Ignition.Start())
> {
> Console.WriteLine();
> var cache = ignite.GetOrCreateCache(new
> CacheConfiguration
> {
> Name = "myCache",
> QueryEntities = new[]
> {
> new QueryEntity(typeof(int), typeof(MyModel))
>
> }
> });
>
> for (int i = 0; i < 100; i++)
> {
> cache.Put(i, new MyModel(i, "Test"));
> }
>
> Console.WriteLine("Cache size: "+cache.Get(10).MyKey);
>
> IQueryable> qry =
> cache.AsCacheQueryable().Where(obj => obj.Value.MyKey >
> 90);
>
> Console.WriteLine();
> Console.WriteLine(">>> Objects with Key greater than 90 "
> );
>
> foreach (ICacheEntry entry in qry)
> Console.WriteLine(">>>" + entry.Value);
>
> Console.WriteLine();
> }
>
>
>
> Console.ReadLine();
> }
> }
> }
>
>
> *MyModel Class:*
> using System;
> using System.Collections.Generic;
> using System.Linq;
> using System.Text;
> using System.Threading.Tasks;
> using Apache.Ignite.Core.Cache.Configuration;
>
> namespace MyIgniteConsole
> {
> [Serializable]
> class MyModel
> {
>
> public MyModel(int MyKeyIn, String MyStringIn)
> {
> MyKey = MyKeyIn;
> MyString = MyStringIn;
> }
>
> [QuerySqlField]
> public int MyKey { get; set; }
>
> public String MyString { get; set; }
> }
> }
>
>