James,
That would be enough - everything with the same affinity key will be stored
on the same node. That's assuming, of course, that both caches have equal
affinity configuration (same affinity function, same number of partitions,
etc.).
-Val
--
Sent from:
This is a known issue: https://issues.apache.org/jira/browse/IGNITE-9229.
Looks like it didn't make it to 2.7, but hopefully someone in the community
will pick it up soon and fix.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Prasad,
prasadbhalerao1983 wrote
> Are you saying that when a primary node dies the former backup node
> becomes
> new primary for ALL backup partitions present on it and only primary
> partitions are moved in rebalancing process?
Not for all partitions, but only for those for which primary
Prasad,
When a primary node for a partition dies, former backup node for this
partition becomes new primary. Therefore there is no need to wait for
rebalancing in this case, data is already there. By default job will be
automatically remapped to that node, but with 'withNoFailover()' you'll have
Prasad,
Since you're using withNoFailover(), failover will never happen and the task
will just fail with an exception on client side if primary nodes dies. It's
up to your code to retry in this case.
When you retry, the task will be mapped to the new primary, which is former
backup and therefore
Probably that's the issue :) In any case, Java serialization successfully
deserializes such objects, so I think it's a bug.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Ray,
This sounds suspicious. Please show your configuration and the execution
plan for the query.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Ilya,
Do you know what is the reason for such limitation? It doesn't sounds right
to me, I believe any other marshaller would work just fine with final
fields.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Does the configuration file exist on worker nodes? I looks like it actually
fails to start there for some reason, and then you eventually get this
exception. Are there any other exception in worker/executor logs?
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
If join is indexed and collocated, it still can be pretty fast. Do you have a
particular query that is slower with optimization than without?
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Can you show the full trace? Most likely it's failing on server side because
you don't actually have the class there.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Ray,
Per my understanding, pushdown filters are propagated to Ignite either way,
it's not related to the "optimization". Optimization affects joins,
gropings, aggregations, etc. So, unless I'm missing something, the behavior
you're looking for is achieved by setting
Monil,
You can also use IgniteSpringBean to initialize Ignite separately from
SpringCacheManager, but within Spring context.
So you need to configure following beans:
1. PercentageDataRespository
2. IgniteConfiguration
3. IgniteSpringBean (IgniteConfiguration is injected via 'configuration'
You need to make sure Ignite is started *before* SpringCacheManager.
Basically, you have two options:
1. Start Ignite manually using Ignition.start, and then start Spring app
with SpringCacheManager that references already started Ignite via
'igniteInstanceName' property.
2. Incorporate
Ignition.start is supposed to start an Ignite instance, so passing
spring-cache.xml file that doesn't contain any Ignite configuration doesn't
make sense. The SpringCacheManager bean should be part of the Spring
Application Context, it then will be used as an entry point to Ignite
cluster. It
Gordon,
Generally, having CQ on thin client would definitely be awesome. My only
point is that thin client has several technical limitations that would
introduce multiple "ifs" into the functionality. What exactly those ifs are,
and weather there is still value with all those ifs, is a big
Gaurav,
Web Console receives updates from web agent which periodically polls the
cluster.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Jeff,
Ignite configuration is an XML file which can be quite large. What is the
reason for the requirement to specify it inline?
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Gordon,
Yes, generally we do recommend using thin client for such applications.
However, it doesn't mean that you can't us client node in case it's really
needed, although it might require some additional tuning.
Would you mind telling if you have any other technology in mind? I highly
doubt
Does it work without specifying sessionAffinity?
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Gordon,
Ignite thin client uses request-response model, which is not really suitable
for functionality like this. I would never say never, but I think it's very
unlikely that thin client would get any feature that imply pushing updates
from server to client (this includes near caches, any type of
Hi Alper,
You can create two separate data regions [1], one for the data within the
threshold, another for outside. The latter can have very little memory
allocated and persistence enabled, which would mean that data in this region
is stored only on disc for the most part.
-Val
--
Sent from:
You're confusing object schema and SQL schema. By adding those field via
binary object builder, you amended the former, but not the latter. SQL
schema stayed the same, with (a,b) being key fields not presented in value.
I would not recommend to do this, as you can end up having weird issues (for
Why not use SQL API and execute a 'select *', since that's what you're
actually looking for?
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Prasad,
To achieve the best performance, join criteria should be the same as
collocation criteria. E.g., if you join by userId, then userId should be
used as affinity key. Please learn more about distributed joins here:
https://apacheignite-sql.readme.io/docs/distributed-joins
-Val
--
Sent
Ignite community is helpful even to those who leaves :)
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Prasad,
1. Yes, you will always see it in the execution plan because there are
always two stages. Of course, if only one server node participates in the
execution, the reduce stage is effectively no-op.
2. Yes, you need to put the annotation to make sure you can access it from
queries. I would
Hi Prasad,
1. First plan is for map phase (executed on server side) and the second one
is for reduce phase (executed on client side). Merge scan means that it just
merges result sets from all participating server nodes. Sometime it can
contain additional reduce steps like final groupings,
Serialization of the filter must happen, because it's invoked on server nodes
and therefore needs to be instantiated there. However, peer class loading
can be used for production. If you see this callout in the documentation,
you're probably looking at some old version of it.
-Val
--
Sent
Prasad,
In this case using subscriptionId in query would be a syntax error, because
the name of the field is affinityId. If you use affinityId, however, Ignite
will route the query to a single node. It knows that it's affinity key based
on @AffinityKeyMapped annotation.
-Val
--
Sent from:
There are several ways to do this.
First one is shown in your first message - DefaultDataAffinityKey instance
plays the role of primary key, while enclosed affinityId field plays the of
affinity key, because it's annotated with @AffinityKeyMapped.
In case you don't have classes at all and
Prasad,
Affinity key is always a part of primary key. For example, in your case
primary key consists of two fields - id and subscriptionId. Therefore you
can query by primary key without providing the affinity key. On the other
hand, however, query can be routed to a single node even if only
You're using the Multicast IP finder, therefore nodes discover each other via
multicast even if port ranges do not intersect. Try switching to static IP
finder:
https://apacheignite.readme.io/docs/tcpip-discovery#section-static-ip-finder
-Val
--
Sent from:
Roger,
To be able to change the schema in runtime, you need to make sure there are
no model classes deployed on server nodes and therefore no deserialization
happens. Since you run in embedded mode and have only server nodes, then you
actually can't use POJOs in your data modes at all. You should
You're iterating through some collection of key-value pairs and then use
value from that pair within the entry processor, which is incorrect. You
should always use the one acquired via entry.getValue(); there.
I would actually recommend you to create a separate class instead of using
lambda for
The point here is that Ignite stores data in binary format, and
deserialization happens only on client side. Therefore, you can start server
nodes without deploying Person and Organization classes there, and the
example will still work. On top of that, you can change class definition on
client
Hi Dave,
Currently there is no ability to change any cache configuration parameters
except the ones you mentioned - you turn on/off statistics and modify schema
using DDL.
You can't change persistence settings in runtime as well, because that would
require changing the data region cache is
If you're using a database that doesn't have a driver in Maven Central repo
(SQL Server is one of the examples), you should copy the driver manually
into 'jdbc-drivers' folder. Please refer to README.txt file included in the
project for details.
-Val
--
Sent from:
The error means that there is no such class on classpath. Please check if
that's the case or not.
You can also provide your project and steps to reproduce the issue, I will
take a look then.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Sounds like you're looking for the MERGE command:
https://apacheignite-sql.readme.io/docs/merge
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Roger,
When exactly do you get this exception and what are the steps to reproduce
it? Do you change the set of values in the enum? Do you restart the cluster
when doing this?
Ideally, it would be great if you provide a reproducer that we can just run
to recreate the problem. That would help to
This test stops working though if you replace line 30 with this:
builder.setField("set", Collections.unmodifiableSet(Sets.newHashSet("a",
"b", "c")));
If unmodifiable set is written, it's then read as unmodifiable set as well,
and therefore can't be modified. I believe this is the reason for the
Anand,
I don't think it's a version issue. As I mentioned earlier, you server was
bound to 47501 according to you logs. I believe after restart it bound to
47500 and your client was able to connect.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
You server node bound to port 47501 rather than 47500 for some reason (most
likely the latter was occupied by some other process). Try to add port range
to the IP finder on client side. Replace this line:
ipFinder.setAddresses(Arrays.asList("127.0.0.1", "172.20.98.77"));
with this:
You cannot include JARs into another JAR, Java is not going to add them to
classpath. You should list all required JARs in the '-cp' parameter, or
create an uber-JAR with all the dependencies unpacked there.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
You should use continuous queries for this:
https://apacheignite.readme.io/docs/continuous-queries. They provide exactly
once notification semantics with ordering guarantees.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Hi Jon,
First of all, you don't have to use multicast for discovery. Using static IP
configuration or one other shared IP finder might simplify the setup:
https://apacheignite.readme.io/docs/tcpip-discovery
Second of all, I'm not sure I fully understand what you're trying to
achieve. Are both
Bhaskar,
It's still the same version - 2.2.0.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Bhaskar,
Which version are you on? This configuration was added in 2.5.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Yes, every time topology is changed, assignments will be recalculated and
data will be rebalanced.
By partitioning mechanisms I meant basically the same that Dmitry was
describing before. Ignite automatically distributes cache entries across
nodes based on built-in affinity function. This
Hi the_palakkaran,
Where this requirement is coming from? Why won't you just use partitioning
mechanisms Ignite provides out of the box?
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Prasad,
Parallel updates in a transaction are not possible. As a matter of fact,
they happen during commit, so doing actual 'put' invocations won't provide
any improvement. Also, you transaction seems to be local, so I don't think
you are going to be faster than you are now. However, if
By default Ignite binds to all interfaces (0.0.0.0) and therefore publish all
available IPs. If you want to bind to a specific address, you can set
IgniteConfiguration#localHost property. In this case only this address would
be published.
-Val
--
Sent from:
JP,
Do you have a solution for this? Do you need any more help?
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Prasad,
What is your business use case and what do you mean by processing? Please
describe in more detail. Are you sure you need to use the transaction here
in the first place?
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Prasad,
Transaction is always attached to a single thread. If you do txStart in one
thread, and then a cache operation in any other thread, this operation will
not be enlisted.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
This project demonstrates this functionality:
https://github.com/vkulichenko/IgniteSparkIoT/tree/dataframes
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
There was a discussion on the dev list regarding this:
http://apache-ignite-developers.2346864.n4.nabble.com/Using-GraalVM-instead-of-standard-JVM-td30284.html
I'm sure if anyone tried to run though, feel free to follow up there and
ask.
-Val
--
Sent from:
Ignite stores data in binary form and has its own algorithms for
hashCode/equals that are based on binary representation to avoid
deserialization. Therefore overriding equals would not have any affect.
Actually, I'm not sure I understand why you want to override it in the first
place. The
Nick,
If B is unhealthy, C will not be able to send a heartbeat message to it.
After a timeout, C will consider B as failed and will connect to A. Along
with this connection it will send a NODE_FAILED message that will go through
all the nodes.
Once B is back again, it will try to send a message
JP,
Can you please describe the business case behind this? What are you trying
to achieve on application level? What guarantees are needed and why?
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Raymond,
If you go with approach I described above, I would actually recommend to use
interceptors:
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/CacheInterceptor.html
Continuous query seems to be a bit cumbersome for this.
-Val
--
Sent from:
Raymond,
It sounds like you want to run certain computation on every data update in
the cache, is that right?
To achieve that you can use local continuous queries, but:
- Remote filter would be executed on both primary and backup, so computation
will be executed more than once.
- You can filter
Nick,
Here are some comments on your questions:
1. Heartbeat is always sent to the next node in the ring. That's the whole
point of the ring architecture vs. peer-to-peer.
2. That's not possible, because each discovery message is sent across the
ring, and the ordering of those messages is
Raymond,
If you want to have a single continuous query on one of the server nodes,
then I think singleton service is the best option to achieve that. It will
not only guarantee that there is one query at a time, but will also make
sure to redeploy it in case of failure.
Also I would be accurate
Hi,
The configuration is fine and it does eliminate ranges so that node always
binds to 47500. The only drawback is that if 47500 is not available for
whatever reason, node would not start.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
The only place where Ignite uses UDP is multicast IP finder [1]. Default
number is 47400.
[1]
https://apacheignite.readme.io/docs/cluster-config#multicast-based-discovery
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
This is being developed now, here is the ticket you can track:
https://issues.apache.org/jira/browse/IGNITE-4191
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Prasad,
Probably "inherit" term is not a correct one here, and in general I'm not
the biggest Spring expert, sorry :)
What I meant is that if Ignite is declared as a bean, there can be another
bean that depends on Ignite (IgniteAtomicSequence in this case), right? If
Ignite is declared as a
Statistically it's very unlikely that all big subscriptions (or many of them)
will end up together. So I would check if there is an issue first and go
from there.
Technically you can implement your own AffinityFunction to customize
distribution. But that's not a trivial task as you will have to
Deepesh,
Memory is allocated by a data region, not by a particular cache. So yes, you
can destroy and create caches in a region to reuse its memory.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Most of these seem to ephemeral ports assigned to discovery and communication
clients when they connect to well known configured ports on server side.
That would always happen for any TCP connection.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Sam,
Entry processor is indeed executed within a lock, this is required to
achieve atomicity. So if there is high contention on a single key, requests
will wait for each other. And this is yet another reason to make entry
processor implementation as lightweight as possible so that it does not
1. Can you show full thread dump, not only the thread that creates atomic
sequence? Also are there any exception on server nodes?
2. You can use IgniteSpringBean to initialize Ignite as a bean, and then
inherit other stuff from it.
-Val
--
Sent from:
Are there any exceptions on startup?
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Hi Mauricio,
1. This is correct.
2. Queries return the actual data, therefore you see every entry only once.
The fact that there are backups is internal and is not related to query
results. And actually, I don't think the check you're doing is valid because
different queries are not in sync with
Ray,
You can use IgniteDataFrameSettings#OPTION_CREATE_TABLE_PARAMETERS parameter
to specify custom parameters that Spark should use when creating new table
in Ignite. Provided string will be attached to WITH clause of Ignite's
CREATE TABLE command [1].
For expiry policy in particular, you can
Prasad,
Are you collocating data by subscription? Do you have any actual issue with
data distribution? With 1 subscriptions, I don't see why would you have
one with default settings.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Entries that are evicted according to eviction policy will not be returned by
a SQL query. If your dataset does not fit in memory, consider using native
Ignite persistence storage instead of Cassandra:
https://apacheignite.readme.io/docs/distributed-persistent-store
-Val
--
Sent from:
.withAsync() is actually a deprecated way for asynchronous execution. In the
latest versions, there are separate methods that should be used for this
(e.g. runAsync, callAsync, etc.). Same goes for .NET - there are RunAsync,
CallAsync and others.
As for listening, .NET async methods return
Hi Sam,
What does this byte array represent? You have to array copies in the entry
processor, so I'm not surprised it doesn't perform very well. That's also
the reason why it gets worse when size of the array is increased. The fact
that you're using off-heap also make it worse in this case as it
Ignite doesn't have authentication/authorization capabilities at the moment,
however there are certain plans for that as far as I know.
In the meantime, you can take a look 3rd party vendors like GridGain that
have paid offerings for this:
https://docs.gridgain.com/docs/security-and-audit
-Val
Hi,
Unfortunately, I doubt this description would be efficient to understand
what's going on. Do you have a reproducer that you can share with us?
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Hi Christoph,
Can you try 2.4.0? I know there were some fixes for memory metrics, so there
is a big chance you will not reproduce the issue with the latest version.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
To unsubscribe from this list you should send an email to
user-unsubscr...@ignite.apache.org
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Dave,
Log files location depend on logger configuration. For example, in log4j
it's controlled by "File" parameter of the file appender.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Michael,
There is no much detail to add. Although it's technically possible to
serialize lambdas and anonymous classes, it's usually not recommended, as
they can reference some objects from outside. If these objects are not
serializable (or not intended to be serialized), you're likely to get
Prasad,
1. JTA is an abstraction that is not related to Ignite, so exact details
would depend on how your application is organized. But basically you need to
do the following:
- Acquire the transaction manager (from a container, from Spring app context
or whatever you use for management of such
I actually don't think it relates to initial query, although it definitely
guarantees exactly-once for listener updates. Andrew, am I wrong?
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Prasad,
First, I'm not sure why you think write-though would be slower than using
JTA. I'm probably missing something, but I believe you should at least test
this before making decision.
In any case, if you use JTA, Ignite would automatically enlist any cache
operations into ongoing JTA
There is no native graph support in Ignite. However, for certain use cases it
might be possible to store graph data in a set of caches, and then use
compute APIs [1] to do the processing.
[1] https://apacheignite.readme.io/docs/compute-grid
-Val
--
Sent from:
Never heard of anyone doing this, but I don't see why it wouldn't work. Did
you have any issues while working with this combination?
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Hm.. Not sore what happened exactly in your case, but cache store is never
deployed via peer class loading. It's required that you have a class
explicitly deployed on every node prior to start up.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Can you clarify what you mean by "real-time query" in this case? Why not just
start node C as a client and run a query from it?
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Naveen,
Ignite does not provide such integration out of the box, however there a
commercial offering from GridGain for that:
https://docs.gridgain.com/docs/goldengate-replication
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
This makes sense, and actually that's exactly how initialQuery works. It's
executed after continuous query listener is deployed, so nothing is missed,
but duplicates are possible.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Wilhelm,
First of all, your code doesn't make much sense, it won't compile.
But in any case, Ignite stores data in binary format [1] that allows to not
deserialize values when running SQL queries. Internally, it works only with
fields and would never call a getter. Therefore, you can put
Wilhelm,
When you're working with SQL in Ignite, you should think about data types in
the same way as about tables in relational databases. SQL works with a
particular schema, and therefore if there is a 'Vehicle' table, it can only
store attributes that are generic for all vehicles. There is no
Subash,
Yes, I reproduced it now. Looks like a bug to me, I created a ticket:
https://issues.apache.org/jira/browse/IGNITE-8027
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Subash,
When you run the test, is this the only node, or there are some other nodes
running? I actually can't reproduce the issue even with your code. Here is
what I do:
- Clean up the work directory.
- Run the test, wait for output, stop.
- Comment out 'put' invocations.
- Run again.
The output
1 - 100 of 1797 matches
Mail list logo
