RE: Deadlock during cache loading

[Dave Harvey]

Your original stack trace shows a call to your custom stream receiver which
appears to itself call invoke().   I can only guess that your code does, but
it appears to be making an call off node to something that is not returning.

org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.invoke(GatewayProtectedCacheProxy.java:1362)

at

*com.mycompany.myapp.myPackage.dao.ignite.cache.streamer.VersionCheckingStreamReceiver.receive(VersionCheckingStreamReceiver.java:33)*

at

org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:137)

at



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: SqlFieldsQuery Cannot create inner bean 'org.apache.ignite.cache.query.SqlFieldsQuery

[ApacheUser]

Evgenii,
what happens if the user doesn't set that limit or forget to set on client
tool?, 

we set that but some one testing without the lazy=true to prove that Apache
Ignite is not stable. 

Thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: SqlFieldsQuery Cannot create inner bean 'org.apache.ignite.cache.query.SqlFieldsQuery

[Evgenii Zhuravlev]

There is a lazy flag for jdbc string: jdbc:ignite:thin://
192.168.0.15/lazy=true

Evgenii

2018-06-28 22:38 GMT+03:00 ApacheUser :

> Evgenii,
>
> We use Ignite-as Im Memory Database for Tableau and SQL, we dont use Java.
> We use spark to load data into Ingite by Spark streaming realtime data.
> So if any user runs select * from table, the server nodes going OOME. We
> need to control that behaviour i sthere any way?
>
> Thanks
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: SqlFieldsQuery Cannot create inner bean 'org.apache.ignite.cache.query.SqlFieldsQuery

[ApacheUser]

Evgenii,

We use Ignite-as Im Memory Database for Tableau and SQL, we dont use Java.
We use spark to load data into Ingite by Spark streaming realtime data.
So if any user runs select * from table, the server nodes going OOME. We
need to control that behaviour i sthere any way?

Thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: SqlFieldsQuery Cannot create inner bean 'org.apache.ignite.cache.query.SqlFieldsQuery

[Evgenii Zhuravlev]

There is no such field in IgniteConfiguration:
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html

Why do you think that it should work?

You can set lazy flag when you creating SqlFieldsQuery object from java

Evgenii

2018-06-28 20:32 GMT+03:00 ApacheUser :

> Hi Ignite Team,
>
> I am trying set SqlFieldsQuery to seTLazy to avoid OOME on Server nodes. By
> Config file has below setting
>
>
> 
> 
> 
>
>
> 
>
> but getting below error:
>
> []# bin/ignite.sh
> class org.apache.ignite.IgniteException: Failed to instantiate Spring XML
> application context
> [springUrl=file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/config/
> default-config.xml,
> err=Error creating bean with name 'ignite.cfg' defined in URL
> [file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/
> config/default-config.xml]:
> Cannot create inner bean
> 'org.apache.ignite.cache.query.SqlFieldsQuery#5f9d02cb' of type
> [org.apache.ignite.cache.query.SqlFieldsQuery] while setting bean property
> 'SqlFieldsQuery'; nested exception is
> org.springframework.beans.factory.BeanCreationException: Error creating
> bean
> with name 'org.apache.ignite.cache.query.SqlFieldsQuery#5f9d02cb' defined
> in
> URL
> [file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/
> config/default-config.xml]:
> Instantiation of bean failed; nested exception is
> org.springframework.beans.BeanInstantiationException: Failed to
> instantiate
> [org.apache.ignite.cache.query.SqlFieldsQuery]: No default constructor
> found; nested exception is java.lang.NoSuchMethodException:
> org.apache.ignite.cache.query.SqlFieldsQuery.()]
> at
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.
> java:990)
> at org.apache.ignite.Ignition.start(Ignition.java:355)
> at
> org.apache.ignite.startup.cmdline.CommandLineStartup.
> main(CommandLineStartup.java:301)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> instantiate Spring XML application context
> [springUrl=file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/config/
> default-config.xml,
> err=Error creating bean with name 'ignite.cfg' defined in URL
> [file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/
> config/default-config.xml]:
> Cannot create inner bean
> 'org.apache.ignite.cache.query.SqlFieldsQuery#5f9d02cb' of type
> [org.apache.ignite.cache.query.SqlFieldsQuery] while setting bean property
> 'SqlFieldsQuery'; nested exception is
> org.springframework.beans.factory.BeanCreationException: Error creating
> bean
> with name 'org.apache.ignite.cache.query.SqlFieldsQuery#5f9d02cb' defined
> in
> URL
> [file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/
> config/default-config.xml]:
> Instantiation of bean failed; nested exception is
> org.springframework.beans.BeanInstantiationException: Failed to
> instantiate
> [org.apache.ignite.cache.query.SqlFieldsQuery]: No default constructor
> found; nested exception is java.lang.NoSuchMethodException:
> org.apache.ignite.cache.query.SqlFieldsQuery.()]
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.
> applicationContext(IgniteSpringHelperImpl.java:392)
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.
> loadConfigurations(IgniteSpringHelperImpl.java:104)
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.
> loadConfigurations(IgniteSpringHelperImpl.java:98)
> at
> org.apache.ignite.internal.IgnitionEx.loadConfigurations(
> IgnitionEx.java:744)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.
> java:945)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.
> java:854)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.
> java:724)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.
> java:693)
> at org.apache.ignite.Ignition.start(Ignition.java:352)
> ... 1 more
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'ignite.cfg' defined in URL
> [file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/
> config/default-config.xml]:
> Cannot create inner bean
> 'org.apache.ignite.cache.query.SqlFieldsQuery#5f9d02cb' of type
> [org.apache.ignite.cache.query.SqlFieldsQuery] while setting bean property
> 'SqlFieldsQuery'; nested exception is
> org.springframework.beans.factory.BeanCreationException: Error creating
> bean
> with name 'org.apache.ignite.cache.query.SqlFieldsQuery#5f9d02cb' defined
> in
> URL
> [file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/
> config/default-config.xml]:
> Instantiation of bean failed; nested exception is
> org.springframework.beans.BeanInstantiationException: Failed to
> instantiate
> [org.apache.ignite.cache.query.SqlFieldsQuery]: No default constructor
> found; nested exception is java.lang.NoSuchMethodException:
> 

Re: Deadlock during cache loading

[breischl]

Just found a bunch of these in my logs as well. Note this is showing
starvation in the system threadpool, not the datastreamer threadpool, but
perhaps they're related?

[2018-06-28T17:39:55,728Z](grid-timeout-worker-#23)([]) WARN - G - >>>
Possible starvation in striped pool.
Thread name: sys-stripe-4-#5
Queue:
[o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@833da92,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@33e4268f,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@69c904f6,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@5b9aa1b6,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@4deb071d,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@5fa99071,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@7e66c1c6,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@707f48ad,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@65396a50,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@7600549e,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@3e20c369,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@3410de20,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@1ad55918,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@6e054a78,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@5606e75a,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@6455c264,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@54784a6f,
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout@10ea9c12,
Message closure [msg=GridIoMessage [plc=2, topic=TOPIC_CACHE, topicOrd=8,
ordered=false, timeout=0, skipOnTimeout=false,
msg=GridDhtAtomicSingleUpdateRequest [key=KeyCacheObjectImpl [part=468,
val=null, hasValBytes=true], val=CacheObjectImpl [val=null,
hasValBytes=true], prevVal=null, super=GridDhtAtomicAbstractUpdateRequest
[onRes=false, nearNodeId=null, nearFutId=0, flags=, Message closure
[msg=GridIoMessage [plc=2, topic=TOPIC_CACHE, topicOrd=8, ordered=false,
timeout=0, skipOnTimeout=false, msg=GridDhtAtomicSingleUpdateRequest
[key=KeyCacheObjectImpl [part=468, val=null, hasValBytes=true],
val=CacheObjectImpl [val=null, hasValBytes=true], prevVal=null,
super=GridDhtAtomicAbstractUpdateRequest [onRes=false, nearNodeId=null,
nearFutId=0, flags=, Message closure [msg=GridIoMessage [plc=2,
topic=TOPIC_CACHE, topicOrd=8, ordered=false, timeout=0,
skipOnTimeout=false, msg=GridDhtAtomicSingleUpdateRequest
[key=KeyCacheObjectImpl [part=468, val=null, hasValBytes=true],
val=CacheObjectImpl [val=null, hasValBytes=true], prevVal=null,
super=GridDhtAtomicAbstractUpdateRequest [onRes=false, nearNodeId=null,
nearFutId=0, flags=, Message closure [msg=GridIoMessage [plc=2,
topic=TOPIC_CACHE, topicOrd=8, ordered=false, timeout=0,
skipOnTimeout=false, msg=GridDhtAtomicSingleUpdateRequest
[key=KeyCacheObjectImpl [part=468, val=null, hasValBytes=true],
val=CacheObjectImpl [val=null, hasValBytes=true], prevVal=null,
super=GridDhtAtomicAbstractUpdateRequest [onRes=false, nearNodeId=null,
nearFutId=0, flags=, Message closure [msg=GridIoMessage [plc=2,
topic=TOPIC_CACHE, topicOrd=8, ordered=false, timeout=0,
skipOnTimeout=false, msg=GridDhtAtomicSingleUpdateRequest
[key=KeyCacheObjectImpl [part=492, val=null, hasValBytes=true],
val=CacheObjectImpl [val=null, hasValBytes=true], prevVal=null,
super=GridDhtAtomicAbstractUpdateRequest [onRes=false, nearNodeId=null,
nearFutId=0, flags=, Message closure [msg=GridIoMessage [plc=2,
topic=TOPIC_CACHE, topicOrd=8, ordered=false, timeout=0,
skipOnTimeout=false, msg=GridDhtAtomicSingleUpdateRequest
[key=KeyCacheObjectImpl [part=492, val=null, hasValBytes=true],
val=CacheObjectImpl [val=null, hasValBytes=true], prevVal=null,
super=GridDhtAtomicAbstractUpdateRequest [onRes=false, nearNodeId=null,
nearFutId=0, flags=, Message closure [msg=GridIoMessage [plc=2,
topic=TOPIC_CACHE, topicOrd=8, ordered=false, timeout=0,
skipOnTimeout=false, msg=GridDhtAtomicSingleUpdateRequest
[key=KeyCacheObjectImpl [part=492, val=null, hasValBytes=true],
val=CacheObjectImpl [val=null, hasValBytes=true], prevVal=null,
super=GridDhtAtomicAbstractUpdateRequest [onRes=false, nearNodeId=null,
nearFutId=0, flags=, Message closure [msg=GridIoMessage [plc=2,
topic=TOPIC_CACHE, topicOrd=8, ordered=false, timeout=0,
skipOnTimeout=false, msg=GridDhtAtomicSingleUpdateRequest
[key=KeyCacheObjectImpl [part=492, val=null, 

Re: And again... Failed to get page IO instance (page content is corrupted)

[Andrey Mashenkov]

Hi Oleg,

The issue you mentioned IGNITE-8659 [1] is caused by IGNITE-5874 [2] that
will not a part of ignite-2.6 release.
For now, 'ExpiryPolicy with persistence' is totally broken and all it's
fixes are planned to the next 2.7 release.


[1] https://issues.apache.org/jira/browse/IGNITE-8659
[2] https://issues.apache.org/jira/browse/IGNITE-5874

On Tue, Jun 26, 2018 at 11:26 PM Olexandr K 
wrote:

> Hi Andrey,
>
> I see Fix version 2.7 in Jira:
> https://issues.apache.org/jira/browse/IGNITE-8659
> This is a critical bug.. bouncing of server node in not-a-right-time
> causes a catastrophe.
> This mean no availability in fact - I had to clean data folders to start
> my cluster after that
>
> BR, Oleksandr
>
>
> On Fri, Jun 22, 2018 at 4:06 PM, Andrey Mashenkov <
> andrey.mashen...@gmail.com> wrote:
>
>> Hi,
>>
>> We've found and fixed few issues related to ExpiryPolicy usage.
>> Most likely, your issue is [1] and it is planned to ignite 2.6 release.
>>
>> [1] https://issues.apache.org/jira/browse/IGNITE-8659
>>
>>
>> On Fri, Jun 22, 2018 at 8:43 AM Olexandr K 
>> wrote:
>>
>>> Hi Team,
>>>
>>> Issue is still there in 2.5.0
>>>
>>> Steps to reproduce:
>>> 1) start 2 servers + 2 clients topology
>>> 2) start load testing on client nodes
>>> 3) stop server 1
>>> 4) start server 1
>>> 5) stop server 1 again when rebalancing is in progress
>>> => and we got data corrupted here, see error below
>>> => we were not able to restart Ignite cluster after that and need to
>>> perform data folders cleanup...
>>>
>>> 2018-06-21 11:28:01.684 [ttl-cleanup-worker-#43] ERROR  - Critical
>>> system error detected. Will be handled accordingly to configured handler
>>> [hnd=class o.a.i.failure.StopNodeOrHaltFailureHandler,
>>> failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=class
>>> o.a.i.IgniteException: Runtime failure on bounds: [lower=null,
>>> upper=PendingRow [
>>> org.apache.ignite.IgniteException: Runtime failure on bounds:
>>> [lower=null, upper=PendingRow []]
>>> at
>>> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:971)
>>> ~[ignite-core-2.5.0.jar:2.5.0]
>>> at
>>> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:950)
>>> ~[ignite-core-2.5.0.jar:2.5.0]
>>> at
>>> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.expire(IgniteCacheOffheapManagerImpl.java:1024)
>>> ~[ignite-core-2.5.0.jar:2.5.0]
>>> at
>>> org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:197)
>>> ~[ignite-core-2.5.0.jar:2.5.0]
>>> at
>>> org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:137)
>>> [ignite-core-2.5.0.jar:2.5.0]
>>> at
>>> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
>>> [ignite-core-2.5.0.jar:2.5.0]
>>> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
>>> Caused by: java.lang.IllegalStateException: Item not found: 2
>>> at
>>> org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO.findIndirectItemIndex(AbstractDataPageIO.java:341)
>>> ~[ignite-core-2.5.0.jar:2.5.0]
>>> at
>>> org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO.getDataOffset(AbstractDataPageIO.java:450)
>>> ~[ignite-core-2.5.0.jar:2.5.0]
>>> at
>>> org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO.readPayload(AbstractDataPageIO.java:492)
>>> ~[ignite-core-2.5.0.jar:2.5.0]
>>> at
>>> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:150)
>>> ~[ignite-core-2.5.0.jar:2.5.0]
>>> at
>>> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:102)
>>> ~[ignite-core-2.5.0.j
>>>
>>> BR, Oleksandr
>>>
>>> On Thu, Jun 14, 2018 at 2:51 PM, Olexandr K <
>>> olexandr.kundire...@gmail.com> wrote:
>>>
 Upgraded to 2.5.0 and didn't get such error so far..
 Thanks!

 On Wed, Jun 13, 2018 at 4:58 PM, dkarachentsev <
 dkarachent...@gridgain.com> wrote:

> It would be better to upgrade to 2.5, where it is fixed.
> But if you want to overcome this issue in your's version, you need to
> add
> ignite-indexing dependency to your classpath and configure SQL
> indexes. For
> example [1], just modify it to work with Spring in XML:
> 
> 
> org.your.KeyObject
> org.your.ValueObject
> 
> 
>
> [1]
>
> https://apacheignite-sql.readme.io/docs/schema-and-indexes#section-registering-indexed-types
>
> Thanks!
> -Dmitry
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>


>>>
>>
>> --

SqlFieldsQuery Cannot create inner bean 'org.apache.ignite.cache.query.SqlFieldsQuery

[ApacheUser]

Hi Ignite Team,

I am trying set SqlFieldsQuery to seTLazy to avoid OOME on Server nodes. By
Config file has below setting
 








but getting below error:

[]# bin/ignite.sh
class org.apache.ignite.IgniteException: Failed to instantiate Spring XML
application context
[springUrl=file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/config/default-config.xml,
err=Error creating bean with name 'ignite.cfg' defined in URL
[file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/config/default-config.xml]:
Cannot create inner bean
'org.apache.ignite.cache.query.SqlFieldsQuery#5f9d02cb' of type
[org.apache.ignite.cache.query.SqlFieldsQuery] while setting bean property
'SqlFieldsQuery'; nested exception is
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'org.apache.ignite.cache.query.SqlFieldsQuery#5f9d02cb' defined in
URL
[file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/config/default-config.xml]:
Instantiation of bean failed; nested exception is
org.springframework.beans.BeanInstantiationException: Failed to instantiate
[org.apache.ignite.cache.query.SqlFieldsQuery]: No default constructor
found; nested exception is java.lang.NoSuchMethodException:
org.apache.ignite.cache.query.SqlFieldsQuery.()]
at
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:990)
at org.apache.ignite.Ignition.start(Ignition.java:355)
at
org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
instantiate Spring XML application context
[springUrl=file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/config/default-config.xml,
err=Error creating bean with name 'ignite.cfg' defined in URL
[file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/config/default-config.xml]:
Cannot create inner bean
'org.apache.ignite.cache.query.SqlFieldsQuery#5f9d02cb' of type
[org.apache.ignite.cache.query.SqlFieldsQuery] while setting bean property
'SqlFieldsQuery'; nested exception is
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'org.apache.ignite.cache.query.SqlFieldsQuery#5f9d02cb' defined in
URL
[file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/config/default-config.xml]:
Instantiation of bean failed; nested exception is
org.springframework.beans.BeanInstantiationException: Failed to instantiate
[org.apache.ignite.cache.query.SqlFieldsQuery]: No default constructor
found; nested exception is java.lang.NoSuchMethodException:
org.apache.ignite.cache.query.SqlFieldsQuery.()]
at
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:392)
at
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104)
at
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98)
at
org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:744)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:945)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:854)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:724)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:693)
at org.apache.ignite.Ignition.start(Ignition.java:352)
... 1 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'ignite.cfg' defined in URL
[file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/config/default-config.xml]:
Cannot create inner bean
'org.apache.ignite.cache.query.SqlFieldsQuery#5f9d02cb' of type
[org.apache.ignite.cache.query.SqlFieldsQuery] while setting bean property
'SqlFieldsQuery'; nested exception is
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'org.apache.ignite.cache.query.SqlFieldsQuery#5f9d02cb' defined in
URL
[file:/data/ignitedata/apache-ignite-fabric-2.5.0-bin/config/default-config.xml]:
Instantiation of bean failed; nested exception is
org.springframework.beans.BeanInstantiationException: Failed to instantiate
[org.apache.ignite.cache.query.SqlFieldsQuery]: No default constructor
found; nested exception is java.lang.NoSuchMethodException:
org.apache.ignite.cache.query.SqlFieldsQuery.()
at
org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:313)
at
org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:122)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1531)
at

RE: Deadlock during cache loading

[breischl]

Also...

>What you showed that the stream receiver called invoke() and did not get an
answer, not a deadlock. 

It's not that I'm getting back a null, it's that all the threads are blocked
waiting on the invoke() call, and no progress is being made. That sounds a
lot like a deadlock. I guess you could say the problem is that it's just
never returning, but that seems like a distinction without a difference. 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Advice for updating large cache

[matt]

Hi,

We have a service, which is essentially a file system crawler, and we're
using Ignite to store the overall state of the job. The state is represented
by simple objects with fields like ID, Name, Path, and State. The State
field is either "Candidate" or "Document". A Candidate is metadata only, and
is inserted before the content of the file is actually fetched. Once a
Candidate is stored in Ignite, we then send it back to the crawler for it to
read the file and send back the content. Once we receive the content, we
update the State field for the item to Document.

We'd like to be able to support stopping the crawl before it finishes, and
then on the next start, pickup where we left off. This essentially means
crawling all Candidates, but skipping the Documents. This is all straight
forward.

The case that gets tricky, is when the second crawl finishes, we'd like to
then have the option of re-evaluating everything on the next crawl. We could
do this by sending everything to the crawler. The problem is that if _this_
crawl then is stopped before finishing, the state of the items becomes
ambiguous — items that were not crawled have their previous state stuck at
Document, and the items that were crawled also have their new state set from
Document, to Document. This means that re-starting the job causes everything
to be re-crawled.

Obviously this approach is flawed. So we tried the simplest thing we could
think of as a solution: at the end of a job that has finished (and not
manually stopped), update the state of every item in the cache back to
Candidate. And this does the trick. Unfortunately, it is slow - we have a
custom cache store, which may or may not be the bottleneck. While it is
simple, this is indeed a brute-force solution.

So I'm wondering if there's something in Ignite that could help? Or if
anyone has dealt with this kind of problem before and can offer ideas for a
better way?

Thanks!
- Matt



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Best practice for class versioning: marshaller error

[slava.koptilin]

Hello,

In that case, only 'createdTime' fall back to OptimizedMarshaller. 

Thanks,
Slava.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Deadlock during cache loading

[breischl]

>our a stream receiver called invoke() and that in turn did another invoke,
which was the actual bug. 
So Ignite's invoke() implementation called itself?


>It was helpful when we did the invoke using a custom thread pool,
I'm not sure I understand the concept here. Is the idea to have an
ExecutorService in of the StreamReceiver, and use that to call invoke()?


It seems odd that it could get hung up on the get() call since presumably
this is being invoked on the primary, and therefore should not need to go
out anywhere. But I notice that it's still trying to map onto the grid
topology, so maybe if the topology changes before the StreamReceiver is
invoked? Total guess, I have only a vague idea of Ignite internals. 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Deadlock during cache loading

[Dave Harvey]

2.4 should be OK.
What you showed that the stream receiver called invoke() and did not get an
answer, not a deadlock.  Nothing looks particularly wrong there.  When we
created this bug, it was our a stream receiver called invoke() and that in
turn did another invoke, which was the actual bug.

It was helpful when we did the invoke using a custom thread pool, because
the logging reports thread in the custom pool, we could see which node had
active custom threads easily, and then look at what that thread was waiting
for.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Deadlock during cache loading

[breischl]

Thanks Dave. I am using Ignite v2.4.0. Would a newer version potentially
help?

This problem seems to come and go. I didn't hit it for a few days, and now
we've hit it on two deployments in a row. It may be some sort of timing or
external factor that provokes it. The most recent case we hit the same
deadlock in the DataStreamers, but do not have the moderate-CPU behavior or
threads stuck in the StringBuilder code. So it seems like that was another
side effect. 

Any other ideas of things to investigate or try would be fantastic.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Leftover Ignite threads after closing cache

[matt]

Good to know, thanks!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Problems with unlocking multiply held cache locks.

[Jon Tricker]

They are separate issues. One is that the isLocalLocked() method returns 
incorrect results if a remote lock is held.

The other is more serious because, as Denis has commented, it means that two 
nodes can end up holding a lock at the same time.

Although, I appreciate, that if lock code is using the same mechanism under the 
covers maybe one causes the other ... i.e. when a remote lock unlocks it may 
trigger a previously blocked lock() call to check if the lock is held 
elsewhere. If that check gets an incorrect result because of the 
isLocalLocked() it could make the wrong decision ... leading to the second 
problem.

I guess we need to re-tests the second problem after the first is fixed.

-Original Message-
From: aealexsandrov [mailto:aealexsand...@gmail.com]
Sent: 28 June 2018 08:02
To: user@ignite.apache.org
Subject: Re: Problems with unlocking multiply held cache locks.

Hi,

As I remember we already discussed here with Jon:

http://apache-ignite-users.70518.x6.nabble.com/If-a-lock-is-held-by-another-node-IgniteCache-isLocalLocked-appears-to-return-incorrect-results-td22110.html#a22149

that isLocalLocked method works incorrectly in case if several nodes started
in the same JVM. I even file the issue:

https://issues.apache.org/jira/browse/IGNITE-8833

In the current example, several nodes started in the same JVM too. So this
method will not work properly.

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

The information in this e-mail and any attachments is confidential and may be 
legally privileged. It is intended solely for the addressee or addressees. Any 
use or disclosure of the contents of this e-mail/attachments by a not intended 
recipient is unauthorized and may be unlawful. If you have received this e-mail 
in error please notify the sender. Please note that any views or opinions 
presented in this e-mail are solely those of the author and do not necessarily 
represent those of TEMENOS. We recommend that you check this e-mail and any 
attachments against viruses. TEMENOS accepts no liability for any damage caused 
by any malicious code or virus transmitted by this e-mail.

RE: Best practice for class versioning: marshaller error

[Calvin KL Wong, CLSA]

Thanks, Dmitry.

>>2-3. In your case, you have and java.time.Ser in one of the fields of your 
>>POJO (or maybe inside of depended object), and it is Externalizable. In such 
>>case BinaryMarshalelr falls back to OptimizedMarshaller with all the issues. 
>>Try to remove it from your POJOs or make transient.

Given that I have this class:

class NewOrderRequest {
int securityId;
Instant createdTime;
}

Instant has a writeReplace method that returns java.time.Ser.

Is the whole 'NewOrderRequest' fall back to OptimizedMarshaller?  Or only 
'createdTime' fall back to OptimizedMarshaller.

-Calvin

-Original Message-
From: dkarachentsev [mailto:dkarachent...@gridgain.com] 
Sent: Thursday, June 28, 2018 2:59 PM
To: user@ignite.apache.org
Subject: RE: Best practice for class versioning: marshaller error

Hi Calvin,

1. Enlist I mean that if you want, for example, to get to see what fields
present in BinaryObject. In other words, if you want to work with
BinaryObject directly. For POJO serialization/deserialization this should
not be and issue at all.

2-3. In your case, you have and java.time.Ser in one of the fields of your
POJO (or maybe inside of depended object), and it is Externalizable. In such
case BinaryMarshalelr falls back to OptimizedMarshaller with all the issues.
Try to remove it from your POJOs or make transient.

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
The content of this communication is intended for the recipient and is subject 
to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon 
request.
Please consider before printing. CLSA is ISO14001 certified and committed to 
reducing its impact on the environment.

Re: Strange cache size

[Denis Mekhanikov]

Michael,

I still don't get, how you measured the occupied space and how many backups
you have.
Could you clarify?

Denis

вт, 26 июн. 2018 г. в 12:50, Michaelikus :

> This is example of data stored in cache taked from visor.
>
> java.lang.Long | 2147604480 <(214)%20760-4480> |
> o.a.i.i.binary.BinaryObjectImpl |
> Cache.UserObjectCacheItem [hash=684724513, UserName=omguser,
> LastUpdated=System.DateTime [idHash=658840403, hash=1247822310,
> ticks=636656011684408212, dateData=5248342030111796116]]
>
>
> Total  amount of records in cache 86KK+ recs.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Cache size in offheap mem in bytes

[Alex Plehanov]

It's incorrect to use cache object to calculate cache data size. What you
got now is a footprint of the Ignite infrastructure used to manage your
data, but not a footprint of your data itself, since data are stored in
off-heap and this tool only calculate on-heap size of objects referenced by
cache object. As you can see in your sample 1 000 000 objects was created
with 5 longs each, but you got only  2 872 685 longs in footprint.

To roughly estimate off-heap memory needed for cache data you can enable
persistent, put large amount of entries to the cache and then measure size
of cache folder (you should also wait until checkpoint complete to get
correct results). Also page in off-heap memory have some extra header, so
final off-heap memory needed will be a little bit larger then cache folder
size.


2018-06-27 17:55 GMT+03:00 Prasad Bhalerao :

> Hi,
>
> I have written a test program to find the cache size in bytes. I am using
> this  tool
> (memory-measurer) to find the object sizes and its foot prints. I tried to
> use this tool to find the cache size in bytes. I am using on heap cache.
>
> Am I using the correct object (cache in this case) to check the cache size
> in memory? Can some please clarify?
>
> I tried to use Jprofiler in instrumentation mode to check on-heap cache
> size, but could not locate the exact class/objects.
>
> private void pushData(CacheName cacheName, List DataKey>> datas) {
>
>   final IgniteCache cache = ignite.cache(cacheName.name());
>
>   LOGGER.info("cache size in bytes : {}", 
> IpV4RangeTest.bytesToHuman(MemoryMeasurer.measureBytes(cache)));
>
>   TestDataObj data8 = null;
>   for(int ctr=0;ctr<1_000_000;ctr++){
> data8 = new TestDataObj();
> data8.setId(ctr);
> data8.setSubscriptionId(SUBSCRIPTION_ID);
> cache.put(data8.getKey(), data8);
>   }
>
>   LOGGER.info("TestDataObj size in bytes : {}",
>   IpV4RangeTest.bytesToHuman(MemoryMeasurer.measureBytes(data8)));
>
>   LOGGER.info("cache size in bytes : {}",
>   IpV4RangeTest.bytesToHuman(MemoryMeasurer.measureBytes(cache)));
>
>   Footprint footprint = ObjectGraphMeasurer.measure(cache);
>
>   LOGGER.info("{} Footprint={}", cacheName.name(), footprint.toString());
>
>   LOGGER.info("{} size={}", cacheName.name(), cache.size());
>   try {
> Thread.sleep(10);
>   } catch (InterruptedException e) {
> e.printStackTrace();
>   }
> }
>
>
>
> *  Output:*
>
>   *cache size in bytes* : 36.4 MB  *[Empty Cache]*
>
> * TestDataObj size in bytes : 64 byte  cache size in bytes* : 493.23 MB
>   *[After adding 1 million objects]*
>   *Footprint*=Footprint
>   [objects=10962325,
>   references=30951850,
>   primitives={byte=130277671, long=2872685, double=58, float=52015,
> int=11651118, boolean=2156105, char=1905788, short=10457}]
>
>
> *Inside TestDataObj  class:*
>
> public class TestDataObj implements Data {
>
>   @QuerySqlField
>   private long id;
>
>   @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = 
> "ag_domain_nb_override_data", order = 1)})
>   private long assetGroupDomainId;
>
>   @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = 
> "ag_domain_nb_override_data", order = 2)})
>   private long startIp;
>
>   @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = 
> "ag_domain_nb_override_data", order = 3)})
>   private long endIp;
>
>   private long subscriptionId;
>
>   @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = 
> "ag_domain_nb_override_data", order = 4)})
>   private int partitionId;
>
>   private boolean isUpdatedData;
>
> }
>
>
>
> Thanks,
> PRasad
>
> On Wed, Jun 27, 2018 at 3:00 PM dkarachentsev 
> wrote:
>
>> 1) This applicable to Ignite. As it grown from GridGain sometimes it may
>> appear in docs, because missed fro removal.
>> 2) Yes, and I would say overhead could be even bigger. But anyway I cannot
>> say definitely how much, because Ignite doesn't store data sequentially,
>> there a lot of nuances.
>> 3) Ignite transaction caches entries on-heap and this works only for
>> TRANSACTIONAL caches.
>>
>> Thanks!
>> -Dmitry
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>

Re: Problems with unlocking multiply held cache locks.

[Denis Mekhanikov]

Andrew,

The issue, that you filed, shows a different problem. It doesn't address
lock's reentrancy.
Lock's implementation contains a counter, that tracks, how many times it
was acquired.
But it's ignored in the *unlock()* method.
So, I think, the lock shouldn't be released, until the *unlock()* method is
called an appropriate number of times.

Denis

чт, 28 июн. 2018 г. в 10:01, aealexsandrov :

> Hi,
>
> As I remember we already discussed here with Jon:
>
>
> http://apache-ignite-users.70518.x6.nabble.com/If-a-lock-is-held-by-another-node-IgniteCache-isLocalLocked-appears-to-return-incorrect-results-td22110.html#a22149
>
> that isLocalLocked method works incorrectly in case if several nodes
> started
> in the same JVM. I even file the issue:
>
> https://issues.apache.org/jira/browse/IGNITE-8833
>
> In the current example, several nodes started in the same JVM too. So this
> method will not work properly.
>
> BR,
> Andrei
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Problems with unlocking multiply held cache locks.

[aealexsandrov]

Hi,

As I remember we already discussed here with Jon:

http://apache-ignite-users.70518.x6.nabble.com/If-a-lock-is-held-by-another-node-IgniteCache-isLocalLocked-appears-to-return-incorrect-results-td22110.html#a22149

that isLocalLocked method works incorrectly in case if several nodes started
in the same JVM. I even file the issue:

https://issues.apache.org/jira/browse/IGNITE-8833

In the current example, several nodes started in the same JVM too. So this
method will not work properly.

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Best practice for class versioning: marshaller error

[dkarachentsev]

Hi Calvin,

1. Enlist I mean that if you want, for example, to get to see what fields
present in BinaryObject. In other words, if you want to work with
BinaryObject directly. For POJO serialization/deserialization this should
not be and issue at all.

2-3. In your case, you have and java.time.Ser in one of the fields of your
POJO (or maybe inside of depended object), and it is Externalizable. In such
case BinaryMarshalelr falls back to OptimizedMarshaller with all the issues.
Try to remove it from your POJOs or make transient.

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Scaling with SQL query

[dkarachentsev]

Hi,

Reduce will be done on node to which JDBC or thin client connected, it could
be either client or server node.

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/