[DISCUSSION] Request for thread unsafe Compute functionality deprecation.
Hello Igniters !
In the process of Ignite usage i found that some part of Compute functionality
are thread unsafe and seems was designed with such limitations initially.
Example : one (client, but it doesn`t matter at all) instance is shared between
numerous of fabric, all of them calls something like :
IgniteCompute#execute(ComputeTask, T)
or
IgniteCompute#execute(java.lang.Class>, T)
and appropriate «async» methods — what kind of instance will be called is
nondeterministic for now and as a confirmation of my words — i found no tests
covered multi thread usage of Computing i also found nothing on documentation
page [1].
We have all necessary info for correct processing of such cases:
from initiator (ignite.compute(...) starter) side we have Class or it instance
and appropriate class loader which will be wired by class loader id from
execution side.
I create a fix and seems all work perfectly well besides one place, this
functionality :
/**
* Executes given task within the cluster group. For step-by-step explanation
of task execution process
* refer to {@link ComputeTask} documentation.
*
* If task for given name has not been deployed yet, then {@code taskName} will
be
* used as task class name to auto-deploy the task (see {@link
#localDeployTask(Class, ClassLoader)} method).
*/
public R execute(String taskName, T arg) throws IgniteException;
and attendant
/**
* Finds class loader for the given class.
*
* @param rsrcName Class name or class alias to find class loader for.
* @return Deployed class loader, or {@code null} if not deployed.
*/
public DeploymentResource findResource(String rsrcName);
is thread unsafe by default, no guarantee that concurrent call of
localDeployTask and execute will bring expected result.
My proposal is to deprecate (or probably annotate [2], as a minimal —
additionally document it) this methods and to append additional :
public DeploymentResource findResource(String rsrcName, ClassLoader clsLdr);
Only one problem i can observe here, if someone creates new class loaders and
appropriate class instances in loop (i don`t know the purpose) and doesn`t
undeploy them then he will get possibly OOM here.
Such approach will give a possibility to use compute in concurrent scenario. If
there is no objections here i will mark this methods and publish my PR, of
course with additional tests.
What do you think ?
[1] https://ignite.apache.org/docs/latest/code-deployment/peer-class-loading
[2] https://jcip.net/annotations/doc/net/jcip/annotations/NotThreadSafe.html
Re: [Discussion] Kanban board for quick start issues to join the community
Hi Ivan, Pavel Thank you for updating the wiki page. Yes, label do help to find newbie issues. My thoughts related to the Board is to provide a different perspective to find and filter issues. The Kanban board or the Backlog board help in finding issues at Epic / Version level. I observed that there is a new board being created for Ignite project with few issues in 2.11 version. https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=443=IGNITE=planning.nodetail=visible=visible=100 I also observed a new Kanban board being created https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=443=IGNITE I think Kanban board can be very helpful for Release Manager to look into current activity for a Version release and plan the release accordingly. Also in Kanban board you can drag and drop issues from ToDo column to In Progress and finally to Done column. Regards, Saikat On Wed, Jan 27, 2021 at 4:44 AM Ivan Pavlukhin wrote: > Hi, > > Saikat, will "newbie" label work fine for you? I am curious what > benefits of having a board do you see? For example I can think it can > be useful if someone would like to see if there is much activity with > such tickets. Is it interesting for someone? > > 2021-01-24 12:20 GMT+03:00, Pavel Tupitsyn : > > There are quite a lot of issues with the `newbie` label [1], > > we usually recommend them to the new contributors. > > > > This label is also mentioned on wiki [2] > > > > [1] > > > https://issues.apache.org/jira/issues/?jql=project%3Dignite%20and%20labels%3Dnewbie%20and%20status%3Dopen > > [2] https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute > > > > On Sat, Jan 23, 2021 at 8:44 AM Konstantin Boudnik > wrote: > > > >> I guess the same result could be achieved with simple labels, no? > >> > >> With regards, > >>Cos > >> > >> On 22.01.2021 16:25, Saikat Maitra wrote: > >> > Hi, > >> > > >> > I was looking into our community page to find new open issues that I > >> > can > >> > contribute to and the easy and moderate tickets, tickets for beginners > >> and > >> > SQL tasks help me to find open issues to contribute. > >> > > >> > I am also thinking if we should use a project Kanban board in jira and > >> move > >> > these issues in the ToDo column so that it is easy to find and > >> > contribute > >> > to these open issues. > >> > > >> > I have seen few Apache Project like Beam, Flink already uses Kanban > >> > board > >> > > >> > > >> > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=BEAM=325 > >> > > >> > > >> > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=356=FLINK > >> > > >> > Please review and let me know your thoughts. > >> > > >> > Regards > >> > Saikat > >> > > >> > > > > > -- > > Best regards, > Ivan Pavlukhin >
Re: [DISCUSSION] Apache Ignite Release 2.10 (time, scope, manager)
Hi Maxim et al, thank you! The rest of the documentation tasks for important features are also completed. Nevertheless, there are still some items (the ones on my list that do not have any doc tasks) that can be done for the 2.10 release. If you want them to be included, I will try to provide patches ASAP. If not, I am still ready to create the documentation for them and provide it later. So, I will fully rely on your decision. Regards, Nikita вт, 26 янв. 2021 г. в 23:08, Maxim Muzafarov : > Nikita, Folks, > > All documentation tasks from the *[list #2]* [1] that you mentioned > above have been processed and merged to the release branch. > > [1] > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Apache-Ignite-Release-2-10-time-scope-manager-tp50037p51061.html > > On Mon, 25 Jan 2021 at 18:59, Maxim Muzafarov wrote: > > > > Shiva, > > > > I've cherry-picked the issue to 2.10 branch. > > https://issues.apache.org/jira/browse/IGNITE-13912 > > > > Ilya, > > > > Thank you for preparing the patch. I've cherry-picked to 2.10 it too. > > https://issues.apache.org/jira/browse/IGNITE-14039 > > > > On Mon, 25 Jan 2021 at 17:50, Ilya Kasnacheev > wrote: > > > > > > Hello! > > > > > > I have pushed the developer warning, javadoc warning and documentation > > > update to https://issues.apache.org/jira/browse/IGNITE-14039 about the > > > dangers of WAL state change. > > > > > > Please cherry-pick it to 2.10 branch or mention me if it's OK for me > to do > > > this. > > > > > > Regards, > > > -- > > > Ilya Kasnacheev > > > > > > > > > пн, 25 янв. 2021 г. в 14:44, Никита Сафонов >: > > > > > > > Hi Denis, > > > > > > > > Thanks a lot! > > > > I'll update my PR's accordingly. > > > > > > > > Regards, > > > > Nikita > > > > > > > > сб, 23 янв. 2021 г. в 00:59, Denis Magda : > > > > > > > > > Nikita, thanks. I reviewed those and shared some suggestions. > Please take > > > > > them into consideration. > > > > > > > > > > - > > > > > Denis > > > > > > > > > > > > > > > On Fri, Jan 22, 2021 at 1:45 PM Никита Сафонов < > > > > vlasovpavel2...@gmail.com> > > > > > wrote: > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > The following documentation tasks for the important features now > have > > > > > PATCH > > > > > > AVAILABLE status: > > > > > > > > > > > >- Caches warming up strategy - > > > > > > > https://issues.apache.org/jira/browse/IGNITE-13385?src=confmacro > > > > > >- Update on JMX default configuration removal - > > > > > > > https://issues.apache.org/jira/browse/IGNITE-13606?src=confmacro > > > > > >- Control.sh indexes manipulation commands - > > > > > > > https://issues.apache.org/jira/browse/IGNITE-13285?src=confmacro > > > > > > > > > > > > Please see the PR's attached to the tickets. > > > > > > > > > > > > Thank you! > > > > > > > > > > > > Regards, > > > > > > Nikita > > > > > > > > > > > > пт, 22 янв. 2021 г. в 18:03, shm : > > > > > > > > > > > > > Hi All, > > > > > > > Can you please also include a critical ticket > > > > > > > https://issues.apache.org/jira/browse/IGNITE-13912 > > > > > > > to 2.10 In terms of functionality it is a blocker. Still some > > > > > debugging > > > > > > is > > > > > > > going on related to this issue. > > > > > > > > > > > > > > Thanks, > > > > > > > Shiva > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Sent from: > http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > > > > > > > > > > > > > > > > > > >
[jira] [Created] (IGNITE-14075) Python client incorrect hash code calculation for classes as composite keys
Igor Sapego created IGNITE-14075: Summary: Python client incorrect hash code calculation for classes as composite keys Key: IGNITE-14075 URL: https://issues.apache.org/jira/browse/IGNITE-14075 Project: Ignite Issue Type: Bug Components: python, thin client Affects Versions: python-0.3.4 Reporter: Igor Sapego Assignee: Igor Sapego Fix For: python-0.4.0 Python code calculates hash codes for simple binary objects incorrectly, such as ones containing just int and string. Leading to possibility of putting same key twice with same key column values, but different hash code, visible as two rows, and also impossibility of getting entries populated via SQL. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14074) Add ability to skip affinity tests for testing with older ignite versions.
Ivan Daschinskiy created IGNITE-14074: - Summary: Add ability to skip affinity tests for testing with older ignite versions. Key: IGNITE-14074 URL: https://issues.apache.org/jira/browse/IGNITE-14074 Project: Ignite Issue Type: Task Components: python, thin client Reporter: Ivan Daschinskiy -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14073) False alarm to lose all transaction nodes
Vladislav Pyatkov created IGNITE-14073:
--
Summary: False alarm to lose all transaction nodes
Key: IGNITE-14073
URL: https://issues.apache.org/jira/browse/IGNITE-14073
Project: Ignite
Issue Type: Bug
Reporter: Vladislav Pyatkov
Assignee: Vladislav Pyatkov
This exception will happen when losing a primary and other one node during the
transaction.
But it may not be truth, because the transaction will be able to continue on
backups (if they are still alive).
{noformat}
[2021-01-23
22:32:50,584][ERROR][test-runner-#1%near.IgniteTxExceptionNodeFailTest%][root]
Transaction was not committed.
class org.apache.ignite.IgniteException: Failed to commit a transaction (all
partition owners have left the grid, partition data has been lost)
[cacheName=default, partition=3, key=386050343]
at
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1096)
at
org.apache.ignite.internal.processors.cache.transactions.TransactionProxyImpl.commit(TransactionProxyImpl.java:323)
at
org.apache.ignite.internal.processors.cache.distributed.near.IgniteTxExceptionNodeFailTest.cacheWithBackups(IgniteTxExceptionNodeFailTest.java:280)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.apache.ignite.testframework.junits.GridAbstractTest$7.run(GridAbstractTest.java:2367)
at java.lang.Thread.run(Thread.java:748)
Caused by: class
org.apache.ignite.internal.processors.cache.CacheInvalidStateException: Failed
to commit a transaction (all partition owners have left the grid, partition
data has been lost) [cacheName=default, partition=3, key=386050343]
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture$FinishMiniFuture.onNodeLeft(GridNearTxFinishFuture.java:993)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.onNodeLeft(GridNearTxFinishFuture.java:167)
at
org.apache.ignite.internal.processors.cache.GridCacheMvccManager$4.onEvent(GridCacheMvccManager.java:265)
at
org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager$LocalListenerWrapper.onEvent(GridEventStorageManager.java:1393)
at
org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager.notifyListeners(GridEventStorageManager.java:888)
at
org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager.notifyListeners(GridEventStorageManager.java:873)
at
org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager.record0(GridEventStorageManager.java:349)
at
org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager.record(GridEventStorageManager.java:312)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.recordEvent(GridDiscoveryManager.java:2948)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body0(GridDiscoveryManager.java:3164)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body(GridDiscoveryManager.java:2968)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
... 1 more
{noformat}
It will frighten a client, because it looks like a data lose.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [Discussion] Kanban board for quick start issues to join the community
Hi, Saikat, will "newbie" label work fine for you? I am curious what benefits of having a board do you see? For example I can think it can be useful if someone would like to see if there is much activity with such tickets. Is it interesting for someone? 2021-01-24 12:20 GMT+03:00, Pavel Tupitsyn : > There are quite a lot of issues with the `newbie` label [1], > we usually recommend them to the new contributors. > > This label is also mentioned on wiki [2] > > [1] > https://issues.apache.org/jira/issues/?jql=project%3Dignite%20and%20labels%3Dnewbie%20and%20status%3Dopen > [2] https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute > > On Sat, Jan 23, 2021 at 8:44 AM Konstantin Boudnik wrote: > >> I guess the same result could be achieved with simple labels, no? >> >> With regards, >>Cos >> >> On 22.01.2021 16:25, Saikat Maitra wrote: >> > Hi, >> > >> > I was looking into our community page to find new open issues that I >> > can >> > contribute to and the easy and moderate tickets, tickets for beginners >> and >> > SQL tasks help me to find open issues to contribute. >> > >> > I am also thinking if we should use a project Kanban board in jira and >> move >> > these issues in the ToDo column so that it is easy to find and >> > contribute >> > to these open issues. >> > >> > I have seen few Apache Project like Beam, Flink already uses Kanban >> > board >> > >> > >> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=BEAM=325 >> > >> > >> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=356=FLINK >> > >> > Please review and let me know your thoughts. >> > >> > Regards >> > Saikat >> > >> > -- Best regards, Ivan Pavlukhin
[jira] [Created] (IGNITE-14072) Remove copy-paste of response for different versions
Ivan Daschinskiy created IGNITE-14072: - Summary: Remove copy-paste of response for different versions Key: IGNITE-14072 URL: https://issues.apache.org/jira/browse/IGNITE-14072 Project: Ignite Issue Type: Task Components: python, thin client Reporter: Ivan Daschinskiy Currently there are many common code in classed Response140 SqlResponse140 Response130 and SqlResponse130. This should be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14071) .NET: Ignite messaging interoperability
Yaroslav Molochkov created IGNITE-14071: --- Summary: .NET: Ignite messaging interoperability Key: IGNITE-14071 URL: https://issues.apache.org/jira/browse/IGNITE-14071 Project: Ignite Issue Type: Improvement Affects Versions: 2.9.1 Reporter: Yaroslav Molochkov Attachments: MessageHandler.java, Test.cs Currently IGNITE-10075 doesn't provide all needed type interoperability thus ignite messaging works somewhat quirky. Thanks to [~kukushal] for pointing that out. Reproducer is attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: IGNITE-13364: Improve index inline defaults
Stanislav, > I think the most compatibility impact will be on the in-memory caches with > SQL and without explicitly specified inline sizes. I don't think that this is `true` compatibility issue. But I think we should at least mentioned it on our documentation pages and in the release notes. So, I'm +1 to proceed with the merge. Should we include the issue into 2.10 release? On Fri, 8 Jan 2021 at 16:50, Stanislav Lukyanov wrote: > > Hi Igniters, > > I'd like to discuss the change implemented by Evgeniy Rudenko in the ticket > https://issues.apache.org/jira/browse/IGNITE-13364. > I see that the fix is ready for review and merging, and I'm interested in it > so I'd like to pick it up on the last mile. > I also wanted to bring community's attention to it before the merge as it > changes the default behavior. > > The patch changes how SQL index inline size is calculated. > > Specifically: > > 1. When inline size is calculated for a variable-sized field, a constant 10 > (configurable via IGNITE_VARIABLE_TYPE_DEFAULT_INDEX_SIZE) is added to the > calculated size instead of setting the entire calculation result to 10. > For example, consider the following cases > > Index (int, int, string) > Before the change: inline size = 10 > After the change: inline size = 5 + 5 + 10 = 20 > > Index (long, long, long, long, string) > Before the change: inline size = 10 > After the change: inline size = 9 + 9 + 9 + 9 + 10 = 46 > > 2. If there is a VARCHAR_FIXED, e.g. VARCHAR(5), then instead of the default > IGNITE_VARIABLE_TYPE_DEFAULT_INDEX_SIZE Ignite will use the value provided in > the calculation > > 3. If the calculated size exceeds IGNITE_MAX_INDEX_PAYLOAD_SIZE_DEFAULT=64, > it will be truncated to 64. > > All of this only affects calculated inline sizes of new indexes. > Existing indexes should not be affected. > Indexes with explicitly specified inline size should not be affected. > > I think the most compatibility impact will be on the in-memory caches with > SQL and without explicitly specified inline sizes. > After the upgrade these caches may slightly increase in size (because the > inline is likely to become bigger) while SQL is also likely to become faster. > > Please comment. > If there are no concerns, I'll proceed with finding a committer to review and > merge the fix at the end of the next week. > > Thanks, > Stan
[jira] [Created] (IGNITE-14070) Protecting a snapshot from from unauthorized changes
Denis Garus created IGNITE-14070: Summary: Protecting a snapshot from from unauthorized changes Key: IGNITE-14070 URL: https://issues.apache.org/jira/browse/IGNITE-14070 Project: Ignite Issue Type: New Feature Reporter: Denis Garus Assignee: Denis Garus We have to allow Ignite users to check the integrity of snapshot files before restoring them. This opportunity can be done through the Ignite plugin mechanism. -- This message was sent by Atlassian Jira (v8.3.4#803005)
