Re: LoadCache Performance decreases with the size of the cache

[steve.hostettler]

Hello,

while investigating, I understood why I do have a lot of locks on Lucene
Documents in java mission control. That is because as soon as there is
String in the index, this is handled by Lucene even if you do not want full
text search (The string being a identifier).


--- From IgniteH2Indexing.java
if (type().valueClass() == String.class) {
try {
luceneIdx = new GridLuceneIndex(ctx, schema.offheap,
schema.spaceName, type);
}
catch (IgniteCheckedException e1) {
throw new IgniteException(e1);
}
}


Altought I do understand why this has been done that way, I wonder whether
it would not be better to let the user choose it. Lucene brings a lot of
features but also has an impact on the performances. In my case, I know that
the index on the string will never ever been searched as a substring.

The Lucene Index management seems to heavily rely on locks and therefore the
more threads the more contention on the index.

Is this a known behavior? Am I missing something.
Furthermore, there seems to be a "rebuildIndexes" method.  I am not sure
what this method do but if it efefctively rebuilds the index then I could do
a measure without indexes (t1) and then a measure with indexes (t2). After
that I rebuild the indexes (t2). Hence, if t1 + t3 < t2 then it means that
disabling the indexes during load make sens from a performance perspective.
Am I correct?






--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/LoadCache-Performance-decreases-with-the-size-of-the-cache-tp9645p9738.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Get a Key from cache vs SQL Indexed parameter search performance

[dkarachentsev]

Hi Oru,

If you need just key/value operations then simple get() will be much faster
than SQL queries. Because indexing, parsing and processing query consumes
resources.

But if you need quite often search for cache entities by their fields, using
of SQL + indexes is preferable, because no need to scan all cache entries.

Thanks!



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Get-a-Key-from-cache-vs-SQL-Indexed-parameter-search-performance-tp9734p9737.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: byte[] as a key in Ignite cache

[Oru]

Thanks for the reply.
I decided to use Strings with ASCII encoding.
Regards,
Oru



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/byte-as-a-key-in-Ignite-cache-tp9728p9736.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Error while writethrough operation in Ignite

[dkarachentsev]

That happens because key and value fields mapped to the same DB columns. You
should use at least one unique column for value mapping. 

Ignite throws away all fields from value that intersect with key fields.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Error-while-writethrough-operation-in-Ignite-tp9696p9735.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Get a Key from cache vs SQL Indexed parameter search performance

[Oru]

Hi There!

I am currently writing my java program to access ignite cache primarily by
using IgniteCache's .get method.

Is it advisable to use indexed SQL based queries (for performance reason
only) ? or the search speed of both is the same?

Thanks
Oru



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Get-a-Key-from-cache-vs-SQL-Indexed-parameter-search-performance-tp9734.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Complex queries

[Anil]

Hi Val,

I have created a complex query and looks good in terms of query plan. But
query execution time is very high.

query -

SELECT
P.serialnumber,
iP.count,
cnt.itemnumber,
cnt.status,
cnt.enddate
FROM  Product P
left JOIN  (  SELECT  serialnumber,  COUNT(*) AS count  FROM Product
GROUP BY serialnumber ) iP
  ON P.serialnumber = iP.serialnumber
JOIN contracts cnt
on P.equipmentid = cnt.equipmentid
 where P.serialnumber = 's3'
 and status = 'Active'

explain plan -

SELECT
IB.SERIALNUMBER,
IIB.COUNT,
CNT.ITEMNUMBER,
CNT.STATUS,
CNT.ENDDATE
FROM PUBLIC.PRODUCT P
 *   /* PUBLIC.SERNO_INDEX: SERIALNUMBER = 's3' */*
/* WHERE P.SERIALNUMBER = 's3'
*/
LEFT OUTER JOIN (
SELECT
SERIALNUMBER,
COUNT(*) AS COUNT
FROM PUBLIC.PRODUCT
GROUP BY SERIALNUMBER
) IP
/* SELECT
SERIALNUMBER,
COUNT(*) AS COUNT
FROM PUBLIC.PRODUCT
  *  /++ PUBLIC.SERNO_INDEX: SERIALNUMBER IS ?1 ++/*
WHERE SERIALNUMBER IS ?1
GROUP BY SERIALNUMBER
  *  /++ group sorted ++/: SERIALNUMBER = P.SERIALNUMBER*
 */
ON P.SERIALNUMBER = IP.SERIALNUMBER
INNER JOIN PUBLIC.CONTRACTS CNT
   * /* PUBLIC.EQ_INDEX: EQUIPMENTID = P.EQUIPMENTID */*
ON P.EQUIPMENTID = CNT.EQUIPMENTID
WHERE (P.SERIALNUMBER = 's3')
AND (STATUS = 'Active')


Contracts are related to Product. So i created product Id reference in
Contract and annotated with @AffinityKeyMapped.

I feel that affinity is not happening. Do you see any issues with above
query or configuration ?

Thanks.


On 20 December 2016 at 02:11, vkulichenko 
wrote:

> I would also create indexes on 'id' fields at least. And do not forget to
> check the execution plan [1] to make sure everything works as expected.
>
> [1] http://apacheignite.gridgain.org/docs/performance-and-debugging
>
> -Val
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Complex-queries-tp9626p9630.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Partitions within single node in Apache Ignite

[Andrey Gura]

Amount of partitions depends on configured affinity function.

RendezvousAffinityFunction with 1024 partitions is default for
replicated or partitioned cache (for single node too). So for example
you can perform scan query by partitions in concurrent threads.
Another example of parallel processing is IgniteCompute.affinityRun()
and IgniteCompute.affinityCall() that can use partition ID as
parameter.

Local cache always has only one partition.

On Sun, Dec 25, 2016 at 11:04 AM, rishi007bansod
 wrote:
> Is there any concept of partitions in Ignite for parallel processing of data
> within single node. That is, can there be more than one partitions within
> one node for multi-threading queries within single node. If there is any
> option, how can we set it?
>
>
>
> --
> View this message in context: 
> http://apache-ignite-users.70518.x6.nabble.com/Partitions-within-single-node-in-Apache-Ignite-tp9726.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite cluster

[Anil]

Hi Val,

Having two independently running clusters is fine with running ignite cache
in each cluster.

But I see cache is stopped because of network segmentation.



*Following the response to one of the query [1] -*

*Call chain:*


*org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.DiscoveryWorker#onSegmentation*
*org.apache.ignite.internal.managers.discovery.GridDiscoveryManager#stopNode*

*org.apache.ignite.Ignition#stop(java.lang.String, boolean)*

*You could change IgniteConfiguration.setSegmentationPolicy()
to SegmentationPolicy.NOOP, so the node will not be stopped upon
segmentation (the default value is STOP).*


I would feel NOOP as default network segmentation policy is better option
than STOP. Network segmentation is just temporary problem and independent
clusters can join as one cluster once is network is good. So STOP not makes
sense in this case.

What do you say ? thanks

1. http://apache-ignite-users.70518.x6.nabble.com/Cache-stopped-td9503.html


Thanks.

On 23 December 2016 at 00:29, vkulichenko 
wrote:

> Anil,
>
> Nothing is stopped. Once again, in case of segmentation you will have two
> independently running clusters. And in of the previous messages you told
> that this is desirable behavior for you as you have a read-only use case.
> Am
> I missing something? Did you run any test that gave different results?
>
> -Val
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Ignite-cluster-tp9527p9707.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: byte[] as a key in Ignite cache

[Vladislav Pyatkov]

Hi,

Your question does not almost clear for me.

>> I realize that strings do take double the space as compared to bytes in
java.

It is not always correct. How about none latin characters?
If you are able to pack string, of course it is good idea for reduce
consume memory.

About byte[] as key, I think you will have hash code issue. You should to
implement key which will by wrapper over byte array.

On Sun, Dec 25, 2016 at 4:39 PM, Oru  wrote:

> Hi There!
> I have a situation where I need to have a Java String as a key in a
> distributed Ignite Cache across multiple server nodes.
>
> I realize that strings do take double the space as compared to bytes in
> java.
>
> So it is possible or recommended to use a byte[] as a key in Ignite Cache?
>
> What's the expert advise on this?
>
> Thanks in advance
> Debasish
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/byte-as-a-key-in-Ignite-cache-tp9728.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Vladislav Pyatkov