[MTCGA]: new failures in builds [5131154] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *New test failure in master GridCommandHandlerClusterByClassWithSSLTest.testHelp https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=3954593076795391567=%3Cdefault%3E=testDetails Changes may lead to failure were done by - ilya kasnacheev https://ci.ignite.apache.org/viewModification.html?modId=898732 - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 01:18:55 17-03-2020
Re: подписаться
A note for curious ones. Dev-list requires moderation for messages from unsubscribed addresses. As the initial message was delivered without moderation I suppose Kirill had been already subscribed at that (roughly) moment. Best regards, Ivan Pavlukhin пн, 16 мар. 2020 г. в 15:45, Кирилл Пушенко : > > Thank you! > I have already subscribed. > > пн, 16 мар. 2020 г. в 15:00, Ilya Kasnacheev : > > > Hello! > > > > The following mailto should subscribe you to this list: > > mailto:dev-subscr...@ignite.apache.org > > ?subject=Subscribebody=Please%20subscribe%20me. > > > > As you can notice, both email and subject are different. > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > пн, 16 мар. 2020 г. в 14:05, Maksim Stepachev > >: > > > >> Please type it in English. > >> > >> пн, 16 мар. 2020 г. в 13:28, Кирилл Пушенко : > >> > >> > -- > >> > С уважением, > >> > *Кирилл Пушенко* > >> > > >> > > > > -- > С уважением, > *Кирилл Пушенко*
Re: IGNITE-8152
Hi Aleksei, Thanks for driving the resolution of the ticket. As I see @Taras Ledkov and @Andrey Mashenkov has already stepped in as reviewers and I believe they will look at your changes soon. - Denis On Mon, Mar 16, 2020 at 3:48 AM Aleksei Litsov wrote: > Hello Igniters! > > I send PR for ticket: https://issues.apache.org/jira/browse/IGNITE-8152 > > I fixed all the comments on the ticket. Could you see the solution. If you > need anything else, I am open for suggestions. If everything is ok, please > accept the PR and I will take the next task >
deadlock in system pool on meta update
Classification: Public
Hi,
I've recently tried to apply Ilya's idea
(https://issues.apache.org/jira/browse/IGNITE-12663) of minimizing thread pools
and tried to set system pool to 3 in my own tests.
It caused deadlock on a client node and I think it can happen not only on such
small pool values.
Details are following:
I'm not using persistence currently (if it matters).
On the client note I use ignite compute to call a job on every server node
(there are 3 server nodes in the tests).
Then I've found in logs:
[10:55:21] : [Step 1/1] [2020-03-13 10:55:21,773] { grid-timeout-worker-#8}
[WARN] [o.a.i.i.IgniteKernal] - Possible thread pool starvation detected (no
task completed in last 3ms, is system thread pool size large enough?)
[10:55:21] : [Step 1/1] ^-- System thread pool [active=3, idle=0,
qSize=9]
I see in threaddumps that all 3 system pool workers do the same - processing of
job responses:
"sys-#26" #605 daemon prio=5 os_prio=0 tid=0x64a0a800 nid=0x1f34
waiting on condition [0x7b91d000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
at
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.metadata(CacheObjectBinaryProcessorImpl.java:749)
at
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$1.metadata(CacheObjectBinaryProcessorImpl.java:250)
at
org.apache.ignite.internal.binary.BinaryContext.metadata(BinaryContext.java:1169)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.getOrCreateSchema(BinaryReaderExImpl.java:2005)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.(BinaryReaderExImpl.java:285)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.(BinaryReaderExImpl.java:184)
at
org.apache.ignite.internal.binary.BinaryUtils.doReadObject(BinaryUtils.java:1797)
at
org.apache.ignite.internal.binary.BinaryUtils.deserializeOrUnmarshal(BinaryUtils.java:2160)
at
org.apache.ignite.internal.binary.BinaryUtils.doReadCollection(BinaryUtils.java:2091)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1914)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1714)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readField(BinaryReaderExImpl.java:1982)
at
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:702)
at
org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:187)
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:887)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1762)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1714)
at
org.apache.ignite.internal.binary.BinaryUtils.doReadObject(BinaryUtils.java:1797)
at
org.apache.ignite.internal.binary.BinaryUtils.deserializeOrUnmarshal(BinaryUtils.java:2160)
at
org.apache.ignite.internal.binary.BinaryUtils.doReadCollection(BinaryUtils.java:2091)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1914)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1714)
at
org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:306)
at
org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:100)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:80)
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10493)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:828)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1134)
As I found analyzing this stack trace, unmarshalling a user object the first
time(per type) causes Binary metadata request (despite I've registered this
type in BinaryConfiguration.setTypeConfiguration)
And all this futures will be completed after consequent MetadataResponseMessage
Re: Reference of local service.
Vladimir, >> We won’t affect existing services > How exactly will we affect services without special interface? Please see > the benchmarks in previous email. I talk about backward compatibility, not about performance. But it doesn't matter because... see below. My fault. From discussion I realized that services doesn't require interface. But indeed it does require. If I understand correctly, IgniteServices.service() method could return actual interface implementation instead of interface itself. Am I right? If yes then we can add new method IgniteService.serviceLocalProxy(). It will not break backward compatibility and will always return proxy. On Thu, Mar 12, 2020 at 2:25 PM Vladimir Steshin wrote: > > Andrey, hi. > > >> We won’t affect existing services > > How exactly will we affect services without special interface? Please see > the benchmarks in previous email. > > > >>> what if we generate will generate proxy that collects service’s metrics > only if service will implement some special interface? > > > I don’t like idea enabling/disabling metrics involves code change, > compilation. I believe it should be an external option, probably available > at runtime through JMX. > > > >> we can impose additional requirement for services that want use metrics > out of box. … service must have own interface and only invocations of > methods of this interface will be taken into account for metrics collection. > > Why one more interface? To work via proxy, with remote services user > already has to use an interface additionally to Service. If we introduce > proxy for local services too (as suggested earlier), an interface will be > required. Current IgniteService#serviceProxy() already requires interface > even for local service. I don’t think we need one more special interface. > > > >> user always can use own metrics framework. > > > Since we do not significantly affect services user can use both or disable > our by an option. > > > With the discussion before and the benchmark I propose: > > > - Let IgniteService#serviceProxy() give GridServiceProxy for local services > too. It already requires to work via interface. So it’s safe for user code. > > > - Deprecate IgniteService#service() > > > - Make service metrics enabled by default for all services. > > > - Bring system param which disables metrics by default for all services. > > > - Bring parameter/method in MetricsMxBean which allows disabling/enabling > metrics for all services at run time. > > Makes sense? > > чт, 5 мар. 2020 г., 16:48 Andrey Gura : > > > Hi there, > > > > what if we will generate proxy that collects service's metrics only if > > service will implement some special interface? In such case: > > > > - we won't affect existing services at all. > > - we can impose additional requirement for services that want use > > metrics out of box (i.e. service that implements our special interface > > *must* also have own interface and only invocations of methods of this > > interface will be taken into account for metrics collection). > > - user always can use own metrics framework instead of our (just do > > not implement this new special interface). > > > > About metrics enabling/disabling. At present IGNITE-11927 doesn't > > solve this problem. Just because there is no metrics implementation > > for services :) > > Anyway we should provide a way for configuring service metrics (in > > sense of enabled/disabled) during service deploy. It's easy for cases > > where deploy() methods have ServiceConfiguration as parameter. But > > there are "short cut" methods like deployXxxSingleton(). I have ideas > > how to solve this problem. For example we can introduce "short cut" > > factory methods like nodeSingletonConfiguration(String name, Service > > service) and clusterSingletonConfiguration(String name, Service > > service). This methods will return configuration which has parameters > > for given type of deployment and could be modified, e.g. metrics could > > be enabled. > > > > WDYT? > > > > On Wed, Mar 4, 2020 at 8:42 PM Vladimir Steshin > > wrote: > > > > > > Vyacheslav, Denis, hi again. > > > > > > > > > > > > >>> I agree with the proposal to introduce a new method which returns > > proxy > > > include the case of locally deployed services. > > > > > > > > > > > > I see one is restricted to use an interface for service with > > > IgniteServiceProcessor#serviceProxy(…): > > > > > > > > > > > > A.ensure(svcItf.isInterface(), "Service class must be an interface: " + > > > svcItf); > > > > > > > > > > > > What if we change IgniteService#serviceProxy(...) so that it will return > > > proxy everytime? That looks safe for user code. Doing so we might only > > > deprecate IgniteService#service(...). > > > > > > > > > > > > вт, 3 мар. 2020 г., 11:03 Vyacheslav Daradur : > > > > > > > Denis, finally I understood your arguments about interfaces check, > > thank > > > > you for the explanation. > > > > > > > > I agree with the proposal to introduce a
Re: Ignite 2.8 documentation
Artem, I'll create a ticket for the IGFS docs replacement once we publish the new website that we'll have all the references needed to create new pages that will replace the IGFS content. @Yuriy Gerzhedovich , could you please help with the SQL questions? - Denis On Mon, Mar 16, 2020 at 12:22 AM Artem Budnikov wrote: > Can anyone give some details about those missing features so I can > document them? > > -Artem > > On 15.03.2020 05:31, 18624049226 wrote: > > Hi, > > > > I don't know the missing part. Is there any developer willing to add it? > > > > In addition, as far as I know, IGFS and Hadoop accelerator related > > components of ignite have been discarded, should related documents > > also be deleted? > > > > 在 2020/3/13 上午6:08, Ivan Pavlukhin 写道: > >> Hi, > >> > >>> There are the following functions. I haven't found the related > >>> documents. Please confirm again? > >> Thank you for checking this thoroughly! I looked into SQL and JDBC > >> items. > >> > >>> 1.SQL:Added KILL QUERY command > >> Here it is https://apacheignite-sql.readme.io/docs/kill-query but it > >> seems no link from a parent page [1] > >> > >>> 4.JDBC:Added cache expiry policies > >> Sounds strange because I an not sure that this feature was implemented. > >> > >> All other pointed SQL and JDBS items seems to be missing =( > >> > >> Best regards, > >> Ivan Pavlukhin > >> > >> чт, 12 мар. 2020 г. в 15:04, 18624049226 <18624049...@163.com>: > >>> Hi igniters, > >>> > >>> There are the following functions. I haven't found the related > >>> documents. Please confirm again? > >>> > >>> 1.SQL:Added KILL QUERY command > >>> > >>> 2.SQL:Added ability to specify query parallelism in CREATE TABLE's WITH > >>> "" clause > >>> > >>> 3.SQL:Added default query timeout > >>> > >>> 4.JDBC:Added cache expiry policies > >>> > >>> 5.JDBC:Added support JDBC thin driver: connection timeout > >>> > >>> 6.JDBC:Added support query cancel > >>> > >>> 7.JDBC:Added support query timeout > >>> > >>> 8.REST:Added "caches" param for "top" command > >>> > >>> 9.REST:Added baseline topology commands to REST API > >>> > >>> 10.REST:Added memory policy metrics via REST > >>> > >>> These are great improvements, and without documentation, developers may > >>> not know how to use them. > >>> > >>> > >>> 在 2020/3/12 上午1:40, Artem Budnikov 写道: > Denis, > > I made version 2.8 the main version on readme.io. Everybody can see > it now. > > -Artem > > On Wed, Mar 11, 2020 at 7:35 PM Denis Magda > wrote: > > > Artem, > > > > Understood, let's see what Alexey says. As of now, I would suggest we > > publishing the existing ML pages and improve the content with no > > rush. > > Could you also make a 2.8 version the default one? 2.7.6 is still > > selected > > by default when I navigate to the documentation website. > > > > > > - > > Denis > > > > > > On Wed, Mar 11, 2020 at 9:33 AM Artem Budnikov < > > a.budnikov.ign...@gmail.com> > > wrote: > > > >> Denis, > >> > >> I'm waiting for answers from Alexey. I can't really tell how lont > >> it will > >> take. Of couse, we can publish everything right now and improve some > > pages > >> later. > >> > >> -Artem > >> > >> On Wed, Mar 11, 2020 at 7:13 PM Denis Magda > >> wrote: > >> > >>> Artem, thanks! How much time should it take to roll out the ML > >>> pages? > > Can > >>> we release what's available right now and continue improving the > >>> pages > > in > >>> parallel? > >>> > >>> Maxim, please let me publish the blog post [1] on the apache > >>> website > >> before > >>> sending you'll send an announcement email. The article will > >>> refer to > > many > >>> documentation pages including ML. You'll include a reference to the > > blog > >>> post in the email. > >>> > >>> [1] > >>> > >>> > > > http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2-8-announcement-plan-td46238.html > > > >>> - > >>> Denis > >>> > >>> > >>> On Wed, Mar 11, 2020 at 8:43 AM Maxim Muzafarov > > >> wrote: > Artem, Folks > > > Thank you all. > I'll do announce the release tomorrow in the middle of the day > > (~16.00 > >>> MSK > TZ). > > On Wed, 11 Mar 2020 at 16:06, Artem Budnikov > wrote: > > I created the Apache Ignite Documentation 2.8 with all the new > > pages > except for ML, which I and Alexey are still working on. The > docs are > >> not > published yet, but you can see them under version 2.8.0 if you log > > into > readme.io. The ML pages could take a while, but other than that > the > initial plan on creating the docs is accomplished, so it's no > longer > > an >
Re: [DISCUSSION] Changes in Ignite release process related to documentation
Denis, I agree with you. Also I think that we should move to process which will require documentation updates during work on issue/feature and will part of code review process. Such approach has some useful benefits: - Documentation readiness at the same time when fix/implementation is ready (remember, documentation is part of a product). - Work on documentation and review could discover incompleteness of a fix or a feature on earlier stage (It is usual situation when some aspects were just forgotten, but documentation writing could spotlight such things). On Thu, Mar 12, 2020 at 7:49 PM Denis Magda wrote: > > Igniters, > > With the final 2.8 release steps checked out today by announcing the > version globally (congrats!), it's a proper time to consider and tweak our > release process, making completion of some phases more predictable and > aligned. I would like to dedicate this thread solely to changes related to > the documentation. > > If to do a recap, Ignite 2.8 announcement went out of sync with the > publication of binaries, Maven and other artifacts because our technical > documentation was completed long after the vote had been closed. > > We can easily eliminate such glitches for future releases if agree to start > a vote only if Ignite docs are ready and can be published the same day with > other release artifacts. If the docs are completed and available > internally while the vote goes then we can work on a release blog post > (referring to docs details) and announce the release the same day when the > binaries/docs availability. > > Thoughts? Let's change the process ensuring that the vote can be started > only if technical documentation is ready to be released? > > - > Denis
[jira] [Created] (IGNITE-12791) [IEP-39] Management API to cancel user provided tasks and queries - documentation
Nikolay Izhikov created IGNITE-12791: Summary: [IEP-39] Management API to cancel user provided tasks and queries - documentation Key: IGNITE-12791 URL: https://issues.apache.org/jira/browse/IGNITE-12791 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Ignite provides many API to deploy and execute user-provided code on the server nodes inside the same JVM as the Ignite process runs. Ignite has many APIs that allocate many resources on the server nodes, also. In case of some buggy code that consumes many system resources(CPU, RAM, flood network) or heavy query the whole cluster can become unstable. We should provide to the cluster administrator the ability to stop any user deployed task. JMX beans to cancel listed tasks should be introduced: * Compute task * Service * Continuous query * Transactions * Queries(SQL, Scan, Text) In the scope of IEP-35 System view API introduced. A new API should use the same identifier that is used in corresponding System View. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSSION][IEP-39] Management API to cancel user provided tasks and queries.
Hello, Igniters. PR [1] for the IEP-39 [2] are ready to be reviewed. Ticket [3]. Can someone, please, review my changes I propose to introduce the following features to Ignite management APIs. 1. JMX beans ``` ComputeMXBean#cancel(String sessionId); QueryMXBean#cacncelSQL(String id) QueryMXBean#cacncelScan(String originNodeId, String cacheName, Long id) QueryMXBean#cacncelContinuous(String routineId) TransactionMXBean#cancel(String xid) ServiceMXBean#cancel(String name) ``` 2. control.sh ``` > ./control.sh --kill scan b2d221ca-ab08-4544-b8dc-8475538ed42f default 63 > ./control.sh --kill compute 58b48c3e071-b2d221ca-ab08-4544-b8dc-8475538ed42f > ./control.sh --kill tx 30058c3e071--0bac-6d33--0003 > ./control.sh --kill sql b2d221ca-ab08-4544-b8dc-8475538ed42f_7 > ./control.sh --kill service my-svc > ./control.sh --kill continuous bfca668d-5df8-4879-b97f-bd1712ad01c9 ``` 3. SQL(KILL QUERY for SQL queries already implemented): ``` KILL SCAN_QUERY 'b2d221ca-ab08-4544-b8dc-8475538ed42f' 'default' 63 KILL TX '30058c3e071--0bac-6d33--0003' KILL CONTINUOUS_QUERY 'bfca668d-5df8-4879-b97f-bd1712ad01c9' KILL COMPUTE_TASK '58b48c3e071-b2d221ca-ab08-4544-b8dc-8475538ed42f' KILL SERVICE 'my-svc' ``` [1] https://github.com/apache/ignite/pull/7520 [2] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=145724615 [3] https://issues.apache.org/jira/browse/IGNITE-12632 > 7 февр. 2020 г., в 14:53, Nikolay Izhikov написал(а): > >> IMHO, the control utility is a more natural way of administration/control > of the cluster instead of JMX, for example. > > It’s questionable. > > I don’t mind to improve control.sh in this IEP. > > But, we should discuss to topic - what management utilities we are providing > *AND* supporting and how the should work. > As far as I know we have some REST API, but it seems abandoned and not > supported. > > > >> 7 февр. 2020 г., в 14:45, Вячеслав Коптилин >> написал(а): >> >> Hi Nikolay, >> >> Yes, I think we should add this API to the control utility as well. >> IMHO, the control utility is a more natural way of administration/control >> of the cluster instead of JMX, for example. >> >> Thanks, >> S. >> >> пт, 7 февр. 2020 г. в 11:38, Nikolay Izhikov : >> >>> Hello, Vyacheslav. >>> It seems to me we missed API that should be introduced into control >>> utility. >>> >>> Do you think we should support control.sh for cancel tasks? >>> >>> 7 февр. 2020 г., в 11:04, Alexey Goncharuk >>> написал(а): Alexei, I agree that there should be no principal difficulty with the unique ID >>> for each cancellable object, but I also see Nikolay's point about the wrong copy-paste. I like minimalistic APIs, but in this case the price of a mistake may be high. Let's let a wider circle of community members to speak out. >>> >>> >
Re: [DISCUSSION] Changes in Ignite release process related to documentation
Hi Denis, My suggestion is that in the current version release process, when planning to release a version, first determine the Time, Scope and ReleaseManager. On this basis, we can add another work of documentation, which is to determine which documents need to be written and who should write them,instead of adding documents after all the work is done. 在 2020/3/13 上午12:49, Denis Magda 写道: Igniters, With the final 2.8 release steps checked out today by announcing the version globally (congrats!), it's a proper time to consider and tweak our release process, making completion of some phases more predictable and aligned. I would like to dedicate this thread solely to changes related to the documentation. If to do a recap, Ignite 2.8 announcement went out of sync with the publication of binaries, Maven and other artifacts because our technical documentation was completed long after the vote had been closed. We can easily eliminate such glitches for future releases if agree to start a vote only if Ignite docs are ready and can be published the same day with other release artifacts. If the docs are completed and available internally while the vote goes then we can work on a release blog post (referring to docs details) and announce the release the same day when the binaries/docs availability. Thoughts? Let's change the process ensuring that the vote can be started only if technical documentation is ready to be released? - Denis
Re: Apache Ignite 2.8.1 RELEASE [Time, Scope, Manager]
Folks, I've walked through all the commits to master since 2.8 branch was cut and filtered some tickets that in my opinion are worth including to 2.8.1 release below (note that they are ready end the effort of including them to the release should be low as long as there are no implicit dependencies between tickets). Please share your opinion on whether we should include them to the 2.8.1. IGNITE-12717 SQL: index creation refactoring IGNITE-12590 MERGE INTO query is failing on Ignite client node IGNITE-12671 Update of partition's states can stuck when rebalance completed during exchange IGNITE-11798 Memory leak on unstable topology caused by partition reservation IGNITE-12665 SQL: Potential race on MapResult close. IGNITE-12605 Historical (WAL) rebalance can start on a cleared partition if some baseline node leaves the cluster and then joins back. IGNITE-12654 Some of rentingFutures in GridDhtPartitionTopologyImpl may accumulate a huge number of eviction callbacks IGNITE-12631 Incorrect rewriting wal record type in marshalled mode during iteration IGNITE-12621 Node leave may cause NullPointerException during IO message processing if security is enabled IGNITE-12636 Full rebalance instead of a historical one IGNITE-12618 Affinity cache for version of last server event can be wiped from history IGNITE-12013 NullPointerException is thrown by ExchangeLatchManager during cache creation IGNITE-11797 Fix consistency issues for atomic and mixed tx-atomic cache groups. IGNITE-12557 Destroy of big cache which is not only cache in cache group causes IgniteOOME IGNITE-12567 H2Tree goes into illegal state when non-indexed columns are dropped IGNITE-12569 Can't set serialized enum to a BinaryObject's field IGNITE-12460 Cluster fails to find the node by consistent ID IGNITE-12459 Searching checkpoint record in WAL doesn't work with segment compaction IGNITE-12548 Possible tx desync during recovery on near node left. IGNITE-12546 Prevent partitions owned by other nodes switch their state to MOVING due to counter difference on node join. IGNITE-12551 Partition desync if a partition is evicted then owned again and historically rebalanced IGNITE-12536 Inconsistency between cache data and indexes when cache operation is interrupted IGNITE-12403 Throttle page difference output in PageMemoryTracker IGNITE-12523 Continuously generated thread dumps in failure processor slow down the whole system IGNITE-12489 Error during purges by expiration: Unknown page type
Re: Security Subject of thin client on remote nodes
Hello, Alexei!
I agree with you if we may not care about compatibility at all,
then we can solve the problem much more straightforward way.
In your case, the method GridSecurityProcessor#authenticate will have an
implicit contract:
[ if actx.subject() != null then
returns SecurityContext
else
do authenticate ]
It looks fragile.
When we extend the GridSecurityProcessor, there isn't this problem:
we have the explicit contract and can make default implementation
that throws an unsupported operation exception to enforcing compatibility
check.
In any case, we need to change GridSecurityProcessor implementation.
But I think your proposal to try to find a security context in the node's
attributes first is right
for backward compatibility when Ignite users don't use thin clients.
Summary:
I suggest adding a new method to GridSecurityProcessor because
it has a clear contract and enforces compatibility check natural way.
вс, 15 мар. 2020 г. в 17:13, Alexei Scherbakov :
> Denis Garus,
>
> I've looked at the IEP proposed by you and currently I'm thinking it's not
> immediately required.
>
> The problem of missing SecurityContexts of thin clients can be solved much
> easily.
>
> Below is the stub of a fix, it requires correct implementation of
> method
> org.apache.ignite.internal.processors.security.IgniteSecurityProcessor#authenticatedSubject
> by GridSecurityProcessor:
>
> /** {@inheritDoc} */
> @Override public OperationSecurityContext withContext(UUID nodeId) {
> try {
> SecurityContext ctx0 = secCtxs.get(nodeId);
>
> if (ctx0 == null) {
> ClusterNode node =
> Optional.ofNullable(ctx.discovery().node(nodeId))
> .orElseGet(() ->
> ctx.discovery().historicalNode(nodeId));
>
> // This is a cluster node.
> if (node != null)
> ctx0 = nodeSecurityContext(marsh,
> U.resolveClassLoader(ctx.config()), findNode(nodeId));
> else {
> // This is already authenticated thin client.
> SecuritySubject subj = authenticatedSubject(nodeId);
>
> assert subj != null : "Subject is null " + nodeId;
>
> AuthenticationContext actx = new
> AuthenticationContext();
> actx.subject(subj);
>
> ctx0 = secPrc.authenticate(actx);
> }
> }
>
> secCtxs.putIfAbsent(nodeId, ctx0);
>
> return withContext(ctx0);
> } catch (IgniteCheckedException e) {
> throw new IgniteException(e);
> }
>
> The idea is to create a thin client SecurityContext on a node not having a
> local context using existing SecuritySubject data.
>
> Method
> org.apache.ignite.internal.processors.security.GridSecurityProcessor#authenticate
> should check for not null SecuritySubject field and just recreate
> SecurityContext using passed info (because it's already authenticated).
>
> We have all necessary information in SecuritySubject returned by
>
> org.apache.ignite.internal.processors.security.IgniteSecurityProcessor#authenticatedSubject
> by GridSecurityProcessor method.
>
> Because it is internal API, we may not care about compatibility at all,
> but nevertheless it is possible to add compatibility check in the method
> above. If a feature is not supported the operations from thin clients
> should be forbidden.
>
> You proposal has the similar problem: if GridSecurityProcessor does not
> support retriving context for thin clients, such clients will not be able
> to proceed with operation.
>
> Still, the cleanup of security API is necessary and should be done in 3.0
>
>
>
> чт, 12 мар. 2020 г. в 16:48, VeenaMithare :
>
> > HI ,
> >
> > Created this jira :
> > https://issues.apache.org/jira/browse/IGNITE-12781
> >
> > regards,
> > Veena.
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>
Re: подписаться
Thank you! I have already subscribed. пн, 16 мар. 2020 г. в 15:00, Ilya Kasnacheev : > Hello! > > The following mailto should subscribe you to this list: > mailto:dev-subscr...@ignite.apache.org > ?subject=Subscribebody=Please%20subscribe%20me. > > As you can notice, both email and subject are different. > > Regards, > -- > Ilya Kasnacheev > > > пн, 16 мар. 2020 г. в 14:05, Maksim Stepachev >: > >> Please type it in English. >> >> пн, 16 мар. 2020 г. в 13:28, Кирилл Пушенко : >> >> > -- >> > С уважением, >> > *Кирилл Пушенко* >> > >> > -- С уважением, *Кирилл Пушенко*
Re: Change vendor name in Apache Ignite
Hello! Can you please show an example of that XML (with sensitive information masked) so that we can understand what is going on? Thanks, -- Ilya Kasnacheev пн, 16 мар. 2020 г. в 00:39, Mahmoud Rashwan : > Hello, > > First of all, please let me thank you for the great tool, we're using it > to as a datasource in an application we are developing. > I am wondering as Apache Ignite is an open source, So do I have the > permission to change the vendor name from "Apache Ignite" to "Custom_Name", > because I am in a deep need for a white labeling. > > If yes, How can i achieve that? because when i changed it from xml file > which used to connect as datasource, it fails to connect. > > Thanks in advance, > > Best Regards, > Mahmoud R. > DevOps Engineer >
Re: Apache Ignite 2.8.1 RELEASE [Time, Scope, Manager]
Hello, guys. I propose myself as a release manager for 2.8.1. > 16 марта 2020 г., в 14:19, kpushenko написал(а): > > Hi, igniters! > > I would like to propose a discussion of version 2.9 and scope releases for > 2020 to another discussion thread. > > I understand that there is no objection to release 2.8.1. > I support the date from Maxim and dmagda ~ 1 month to May 1. > > I offer my help with the release of Ignite 2.8.1. I can take on > administrative functions that do not require PMC: a scope release, a > reminder about ticket solutions, jobs without PMC, writing readme and > documentation. I am a new person in the community. But I have been testing > and supporting ignition-based products for the past three years. I want to > benefit the community. > > > > -- > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: подписаться
Hello! The following mailto should subscribe you to this list: mailto:dev-subscr...@ignite.apache.org ?subject=Subscribebody=Please%20subscribe%20me. As you can notice, both email and subject are different. Regards, -- Ilya Kasnacheev пн, 16 мар. 2020 г. в 14:05, Maksim Stepachev : > Please type it in English. > > пн, 16 мар. 2020 г. в 13:28, Кирилл Пушенко : > > > -- > > С уважением, > > *Кирилл Пушенко* > > >
Re: Apache Ignite 2.8.1 RELEASE [Time, Scope, Manager]
Hi, igniters! I would like to propose a discussion of version 2.9 and scope releases for 2020 to another discussion thread. I understand that there is no objection to release 2.8.1. I support the date from Maxim and dmagda ~ 1 month to May 1. I offer my help with the release of Ignite 2.8.1. I can take on administrative functions that do not require PMC: a scope release, a reminder about ticket solutions, jobs without PMC, writing readme and documentation. I am a new person in the community. But I have been testing and supporting ignition-based products for the past three years. I want to benefit the community. -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: подписаться
Please type it in English. пн, 16 мар. 2020 г. в 13:28, Кирилл Пушенко : > -- > С уважением, > *Кирилл Пушенко* >
IGNITE-8152
Hello Igniters! I send PR for ticket: https://issues.apache.org/jira/browse/IGNITE-8152 I fixed all the comments on the ticket. Could you see the solution. If you need anything else, I am open for suggestions. If everything is ok, please accept the PR and I will take the next task
подписаться
-- С уважением, *Кирилл Пушенко*
Re: [DISCUSSION] Improvement in control.sh configuration.
Nikolay, specified ticket only describes the mentioned feature. It's not a ticket of the current proposal. On 16.03.2020 12:14, Dmitrii Ryabov wrote: Hello! Nikolay, yes, but currently, we can't declare user attributes through control.sh. пн, 16 мар. 2020 г., 12:06 Nikolay Izhikov : Hello, Mikhail. Can you, please, double-check link to the ticket? For now it’s a IGNITE-12049 Add user attributes to thin clients which is resolved. 16 марта 2020 г., в 12:04, Mikhail Petrov написал(а): Hello, Igniters. I'd like to propose a small improvement in control.sh configuration, which is to add the ability to pass parameters needed for cluster connection via a special properties file located in the same directory as control.sh script. The connection parameters in the proposed properties file can be considered as defaults and overwritten by those that are specified via the command line (current approach). The proposal can help to avoid unnecessary repeating of cluster connection configuration. Also, some connection parameters (e.g. user attributes [1]) cannot be passed through the command line due to its limited length. WDYT? [1] https://issues.apache.org/jira/browse/IGNITE-12049
Re: [DISCUSSION] Improvement in control.sh configuration.
Hello! Nikolay, yes, but currently, we can't declare user attributes through control.sh. пн, 16 мар. 2020 г., 12:06 Nikolay Izhikov : > Hello, Mikhail. > > Can you, please, double-check link to the ticket? > > For now it’s a IGNITE-12049 Add user attributes to thin clients which is > resolved. > > > 16 марта 2020 г., в 12:04, Mikhail Petrov > написал(а): > > > > Hello, Igniters. > > > > I'd like to propose a small improvement in control.sh configuration, > which is to add the ability to pass parameters needed for cluster > connection via a special properties file located in the same directory as > control.sh script. > > > > The connection parameters in the proposed properties file can be > considered as defaults and overwritten by those that are specified via the > command line (current approach). > > > > The proposal can help to avoid unnecessary repeating of cluster > connection configuration. Also, some connection parameters (e.g. user > attributes [1]) cannot be passed through the command line due to its > limited length. > > > > WDYT? > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12049 > > > >
Re: [DISCUSSION] Improvement in control.sh configuration.
Hello, Mikhail. Can you, please, double-check link to the ticket? For now it’s a IGNITE-12049 Add user attributes to thin clients which is resolved. > 16 марта 2020 г., в 12:04, Mikhail Petrov написал(а): > > Hello, Igniters. > > I'd like to propose a small improvement in control.sh configuration, which is > to add the ability to pass parameters needed for cluster connection via a > special properties file located in the same directory as control.sh script. > > The connection parameters in the proposed properties file can be considered > as defaults and overwritten by those that are specified via the command line > (current approach). > > The proposal can help to avoid unnecessary repeating of cluster connection > configuration. Also, some connection parameters (e.g. user attributes [1]) > cannot be passed through the command line due to its limited length. > > WDYT? > > [1] https://issues.apache.org/jira/browse/IGNITE-12049 >
[DISCUSSION] Improvement in control.sh configuration.
Hello, Igniters. I'd like to propose a small improvement in control.sh configuration, which is to add the ability to pass parameters needed for cluster connection via a special properties file located in the same directory as control.sh script. The connection parameters in the proposed properties file can be considered as defaults and overwritten by those that are specified via the command line (current approach). The proposal can help to avoid unnecessary repeating of cluster connection configuration. Also, some connection parameters (e.g. user attributes [1]) cannot be passed through the command line due to its limited length. WDYT? [1] https://issues.apache.org/jira/browse/IGNITE-12049
[jira] [Created] (IGNITE-12789) Tracking page repairing has no WAL record associated with it
Ivan Bessonov created IGNITE-12789: -- Summary: Tracking page repairing has no WAL record associated with it Key: IGNITE-12789 URL: https://issues.apache.org/jira/browse/IGNITE-12789 Project: Ignite Issue Type: Bug Reporter: Ivan Bessonov Assignee: Ivan Bessonov org.apache.ignite.internal.processors.cache.persistence.tree.io.TrackingPageIO#resetCorruptFlag(long) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12788) Cluster achieved fully rebalanced (PME-free ready) state metric
Anton Vinogradov created IGNITE-12788:
-
Summary: Cluster achieved fully rebalanced (PME-free ready) state
metric
Key: IGNITE-12788
URL: https://issues.apache.org/jira/browse/IGNITE-12788
Project: Ignite
Issue Type: Task
Reporter: Anton Vinogradov
Fix For: 2.9, 2.8.1
Currently, there is no metric responsible for "PME-free ready state achieved"
delivery.
{{GridDhtPartitionsExchangeFuture#rebalanced}} can be used to provide such
metric.
Seems, we should update metric on each
{{GridDhtPartitionsExchangeFuture#onDone}}.
P.s. Late Affinity Assignment should should always set {{true}} to metric value.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Ignite 2.8 documentation
Can anyone give some details about those missing features so I can document them? -Artem On 15.03.2020 05:31, 18624049226 wrote: Hi, I don't know the missing part. Is there any developer willing to add it? In addition, as far as I know, IGFS and Hadoop accelerator related components of ignite have been discarded, should related documents also be deleted? 在 2020/3/13 上午6:08, Ivan Pavlukhin 写道: Hi, There are the following functions. I haven't found the related documents. Please confirm again? Thank you for checking this thoroughly! I looked into SQL and JDBC items. 1.SQL:Added KILL QUERY command Here it is https://apacheignite-sql.readme.io/docs/kill-query but it seems no link from a parent page [1] 4.JDBC:Added cache expiry policies Sounds strange because I an not sure that this feature was implemented. All other pointed SQL and JDBS items seems to be missing =( Best regards, Ivan Pavlukhin чт, 12 мар. 2020 г. в 15:04, 18624049226 <18624049...@163.com>: Hi igniters, There are the following functions. I haven't found the related documents. Please confirm again? 1.SQL:Added KILL QUERY command 2.SQL:Added ability to specify query parallelism in CREATE TABLE's WITH "" clause 3.SQL:Added default query timeout 4.JDBC:Added cache expiry policies 5.JDBC:Added support JDBC thin driver: connection timeout 6.JDBC:Added support query cancel 7.JDBC:Added support query timeout 8.REST:Added "caches" param for "top" command 9.REST:Added baseline topology commands to REST API 10.REST:Added memory policy metrics via REST These are great improvements, and without documentation, developers may not know how to use them. 在 2020/3/12 上午1:40, Artem Budnikov 写道: Denis, I made version 2.8 the main version on readme.io. Everybody can see it now. -Artem On Wed, Mar 11, 2020 at 7:35 PM Denis Magda wrote: Artem, Understood, let's see what Alexey says. As of now, I would suggest we publishing the existing ML pages and improve the content with no rush. Could you also make a 2.8 version the default one? 2.7.6 is still selected by default when I navigate to the documentation website. - Denis On Wed, Mar 11, 2020 at 9:33 AM Artem Budnikov < a.budnikov.ign...@gmail.com> wrote: Denis, I'm waiting for answers from Alexey. I can't really tell how lont it will take. Of couse, we can publish everything right now and improve some pages later. -Artem On Wed, Mar 11, 2020 at 7:13 PM Denis Magda wrote: Artem, thanks! How much time should it take to roll out the ML pages? Can we release what's available right now and continue improving the pages in parallel? Maxim, please let me publish the blog post [1] on the apache website before sending you'll send an announcement email. The article will refer to many documentation pages including ML. You'll include a reference to the blog post in the email. [1] http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2-8-announcement-plan-td46238.html - Denis On Wed, Mar 11, 2020 at 8:43 AM Maxim Muzafarov wrote: Artem, Folks Thank you all. I'll do announce the release tomorrow in the middle of the day (~16.00 MSK TZ). On Wed, 11 Mar 2020 at 16:06, Artem Budnikov wrote: I created the Apache Ignite Documentation 2.8 with all the new pages except for ML, which I and Alexey are still working on. The docs are not published yet, but you can see them under version 2.8.0 if you log into readme.io. The ML pages could take a while, but other than that the initial plan on creating the docs is accomplished, so it's no longer an obstacle to announcing the release. -Artem On Wed, Mar 11, 2020 at 11:50 AM Artem Budnikov < a.budnikov.ign...@gmail.com> wrote: OK, I'm going to create the 2.8 version on readme.io for all documentation pages. If anyone is still working on the docs version 2.7.6, please let me know. I'll post an update in this thread when I finish. Further changes should be made in version 2.8. -Artem On Wed, Mar 11, 2020 at 10:16 AM Anton Vinogradov wrote: Artem, I've updated the Read Repair page On Thu, Mar 5, 2020 at 3:51 PM Artem Budnikov < a.budnikov.ign...@gmail.com> wrote: Anton, Could you please review the page about Read Rapair? https://apacheignite.readme.io/docs/read-repair -Artem On 05.03.2020 12:20, Artem Budnikov wrote: OK, I'll recreate it. Nikolay, please let me know when you are finished with the Metrics and system views documentation. I'm done with the list of docs we identified in this thread and want to publish v. 2.8. -Artem On 05.03.2020 11:55, Maxim Muzafarov wrote: Artem, I've created that. It is not public and can be safely removed since it's a full copy of 2.7.6 (at that moment) On Thu, 5 Mar 2020 at 11:53, Artem Budnikov wrote: Guys, I see that there is already version 2.8 for Ignite docs on readme.io. Who created it and when? I've changed some pages under 2.7.6 version without knowing that there is a newer version. -Artem On
