Re: SQL query timeout: in progress or abandoned
Hello Ivan, I have updated the PR as per our discussion. Please review and share your feedback. Regards, Saikat On Sun, Sep 1, 2019 at 3:20 PM Saikat Maitra wrote: > Hi Ivan, > > I have taken care of review comments and also have shared a question for > the application of default Query timeout value. > > Can you please review and share feedback? > > Regards, > Saikat > > On Sat, Aug 24, 2019 at 7:22 PM Saikat Maitra > wrote: > >> Hi Ivan, >> >> Thank you, I have shared my comments and have few questions related to >> the issue. >> >> Please take a look and share your thoughts. >> >> Regards, >> Saikat >> >> On Tue, Aug 20, 2019 at 4:03 PM Павлухин Иван >> wrote: >> >>> Hi Saikat, >>> >>> I left a comment in JIRA ticket [1]. Also, I invited Andrey to help >>> with a further review. >>> >>> Andrey, could you please step in and continue the review? >>> Unfortunately, for a couple of weeks I have limited access to my >>> computer and cannot do a review in a timely manner. >>> >>> [1] https://issues.apache.org/jira/browse/IGNITE-7285 >>> >>> 2019-08-19 7:24 GMT+11:00, Saikat Maitra : >>> > Hi Ivan, >>> > >>> > I have updated the PR and made changes in IgniteH2Indexing for query >>> > timeout so that default query timeout get used during query execution. >>> > >>> > Please take a look and let me know if this change looks good. >>> > >>> > I will update tests if the approach looks good. >>> > >>> > PR https://github.com/apache/ignite/pull/6490 >>> > >>> > Regards, >>> > >>> > Saikat >>> > >>> > On Sat, Aug 17, 2019 at 8:30 PM Saikat Maitra >> > >>> > wrote: >>> > >>> >> Hi Ivan, Denis >>> >> >>> >> Thank you for your feedback, I am looking into the changes needed for >>> >> this >>> >> issue. >>> >> >>> >> I am also looking into these configurations parameters >>> >> https://apacheignite.readme.io/v2.2/docs/configuration-parameters to >>> see >>> >> if there are similar attributes being used in SqlFieldsQuery and >>> >> SqlQuery. >>> >> >>> >> >>> >> Regards, >>> >> >>> >> Saikat >>> >> >>> >> >>> >> >>> >> On Thu, Aug 15, 2019 at 6:13 AM Павлухин Иван >>> >> wrote: >>> >> >>> >>> Saikat, Denis, >>> >>> >>> >>> I left comments in the ticket [1]. >>> >>> >>> >>> [1] https://issues.apache.org/jira/browse/IGNITE-7285 >>> >>> >>> >>> вт, 13 авг. 2019 г. в 21:53, Denis Magda : >>> >>> > >>> >>> > Hi Saikat, >>> >>> > >>> >>> > Thanks for a quick turnaround! Ivan, could you please step in and >>> do a >>> >>> > review? >>> >>> > >>> >>> > - >>> >>> > Denis >>> >>> > >>> >>> > >>> >>> > On Sun, Aug 11, 2019 at 6:26 AM Saikat Maitra >>> >>> > >>> >>> > wrote: >>> >>> > >>> >>> > > Hi Denis, Ivan >>> >>> > > >>> >>> > > As discussed I have updated the PR and incorporated review >>> comments. >>> >>> > > >>> >>> > > https://github.com/apache/ignite/pull/6490/files >>> >>> > > >>> >>> > > Please take a look and share your feedback. >>> >>> > > >>> >>> > > Regard, >>> >>> > > Saikat >>> >>> > > >>> >>> > > >>> >>> > > >>> >>> > > On Sat, Aug 10, 2019 at 5:51 PM Saikat Maitra < >>> >>> saikat.mai...@gmail.com> >>> >>> > > wrote: >>> >>> > > >>> >>> > > > Hello Denis, Ivan >>> >>> > > > >>> >>> > > > Yes, I can take up the changes for IGNITE-7825. >>> >>> > > > >>> >>> > > > I had a doubt on the usage of the Default Query Timeout. >>> >>> > > > >>> >>> > > > I had raised the PR in an assumption that Default Query Timeout >>> >>> will only >>> >>> > > > be used if user had not provided Cache Query Timeout >>> >>> > > > >>> >>> > > > https://github.com/apache/ignite/pull/6490/files >>> >>> > > > >>> >>> > > > I wanted to discuss if it is correct intended usage of Default >>> >>> > > > Query >>> >>> > > > Timeout or should we reconsider? >>> >>> > > > >>> >>> > > > Regards, >>> >>> > > > Saikat >>> >>> > > > >>> >>> > > > >>> >>> > > > >>> >>> > > > On Fri, Aug 9, 2019 at 12:11 PM Denis Magda >> > >>> >>> wrote: >>> >>> > > > >>> >>> > > >> Ivan, thanks for sharing this discussion. Let's use it for our >>> >>> > > >> conversation. >>> >>> > > >> >>> >>> > > >> - >>> >>> > > >> Denis >>> >>> > > >> >>> >>> > > >> >>> >>> > > >> On Thu, Aug 8, 2019 at 11:15 PM Павлухин Иван >>> >>> > > >> >> >>> > >>> >>> > > >> wrote: >>> >>> > > >> >>> >>> > > >> > Just for the protocol. There was an original dev-list >>> >>> > > >> > discussion >>> >>> [1]. >>> >>> > > >> > Added a link to the ticket as well. >>> >>> > > >> > >>> >>> > > >> > [1] >>> >>> > > >> > >>> >>> > > >> >>> >>> > > >>> >>> >>> http://apache-ignite-developers.2346864.n4.nabble.com/IGNITE-7285-Add-default-query-timeout-td41828.html >>> >>> > > >> > >>> >>> > > >> > пт, 9 авг. 2019 г. в 01:22, Denis Magda >> >: >>> >>> > > >> > > >>> >>> > > >> > > Hey Saikat, >>> >>> > > >> > > >>> >>> > > >> > > Are you still working on this ticket? >>> >>> > > >> > > https://issues.apache.org/jira/browse/IGNITE-7285 >>> >>> > > >> > > >>> >>> > > >> > > Seems that's the last API that doesn't support timeouts - >>> >>> > > >> > > JDBC >>> >>> and >>> >>> > > >> ODBC >>> >>> >
Re: [DISCUSS] Pub/Sub Streamer Implementation
Hello Emmanouil, Can you please share more info about the implementation? We have similar implementation for data streamers with Kafka and RocketMQ https://apacheignite-mix.readme.io/docs/kafka-streamer https://apacheignite-mix.readme.io/docs/rocketmq-streamer Please review and share if you have any questions. Regards, Saikat On Fri, Oct 11, 2019 at 4:56 PM Emmanouil Gkatziouras wrote: > Hi all, > > By mentioning Pub/Sub I mean Google Cloud's Pub/Sub [1] > Yes it is indeed cloud specific. It is the messaging service provided by > GCP. > I have made the integration with it, but I haven't published it yet (I am > waiting for the discussion/ticket and proceed with the contribution). > > Kind regards, > Emmanouil > > [1] https://cloud.google.com/pubsub/docs/overview > > *Emmanouil Gkatziouras* > https://egkatzioura.com/ | > https://www.linkedin.com/in/gkatziourasemmanouil/ > https://github.com/gkatzioura > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda wrote: > > > Hello Emmanouil, > > > > Are you referring to any specific implementation tailored for a cloud? If > > I'm not mistaken you were mentioning AWS/GCE. Is there any Github source > > code? > > > > Here is a list of existing integrations with streaming and pub/sub > > frameworks: > > https://apacheignite-mix.readme.io/docs/overview > > > > with the source code in Ignite master: > > https://github.com/apache/ignite/tree/master/modules > > > > Check out those resources to see what are the most suitable Ignite APIs > for > > such integrations. > > > > @Saikat Maitra , @Roman , > as > > streaming technologies contributors/maintainers, could you please join > this > > thread? > > > > - > > Denis > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil Gkatziouras < > > gkatzio...@gmail.com> > > wrote: > > > > > Hello everyone, > > > > > > I started using Ignite lately. Part of my work involves Pub/Sub, thus I > > > created a pub/sub streamer. > > > Pub/Sub is pretty close to Kafka. > > > Here's how the streamer works. Every node creates an instance of a > > Pub/Sub > > > client. > > > The clients subscribe to the same topic using the same subscription > name. > > > A message is being sent and one of the nodes will receive the message, > > > process it and then forward it to Ignite using the extractor provided. > > > > > > Any thoughts? > > > > > > *Emmanouil Gkatziouras* > > > https://egkatzioura.com/ | > > > https://www.linkedin.com/in/gkatziourasemmanouil/ > > > https://github.com/gkatzioura > > > > > >
Re: [DISCUSS] Pub/Sub Streamer Implementation
Hi all, By mentioning Pub/Sub I mean Google Cloud's Pub/Sub [1] Yes it is indeed cloud specific. It is the messaging service provided by GCP. I have made the integration with it, but I haven't published it yet (I am waiting for the discussion/ticket and proceed with the contribution). Kind regards, Emmanouil [1] https://cloud.google.com/pubsub/docs/overview *Emmanouil Gkatziouras* https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/ https://github.com/gkatzioura On Fri, 11 Oct 2019 at 21:45, Denis Magda wrote: > Hello Emmanouil, > > Are you referring to any specific implementation tailored for a cloud? If > I'm not mistaken you were mentioning AWS/GCE. Is there any Github source > code? > > Here is a list of existing integrations with streaming and pub/sub > frameworks: > https://apacheignite-mix.readme.io/docs/overview > > with the source code in Ignite master: > https://github.com/apache/ignite/tree/master/modules > > Check out those resources to see what are the most suitable Ignite APIs for > such integrations. > > @Saikat Maitra , @Roman , as > streaming technologies contributors/maintainers, could you please join this > thread? > > - > Denis > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil Gkatziouras < > gkatzio...@gmail.com> > wrote: > > > Hello everyone, > > > > I started using Ignite lately. Part of my work involves Pub/Sub, thus I > > created a pub/sub streamer. > > Pub/Sub is pretty close to Kafka. > > Here's how the streamer works. Every node creates an instance of a > Pub/Sub > > client. > > The clients subscribe to the same topic using the same subscription name. > > A message is being sent and one of the nodes will receive the message, > > process it and then forward it to Ignite using the extractor provided. > > > > Any thoughts? > > > > *Emmanouil Gkatziouras* > > https://egkatzioura.com/ | > > https://www.linkedin.com/in/gkatziourasemmanouil/ > > https://github.com/gkatzioura > > >
Re: [DISCUSS] Pub/Sub Streamer Implementation
Hello Emmanouil, Are you referring to any specific implementation tailored for a cloud? If I'm not mistaken you were mentioning AWS/GCE. Is there any Github source code? Here is a list of existing integrations with streaming and pub/sub frameworks: https://apacheignite-mix.readme.io/docs/overview with the source code in Ignite master: https://github.com/apache/ignite/tree/master/modules Check out those resources to see what are the most suitable Ignite APIs for such integrations. @Saikat Maitra , @Roman , as streaming technologies contributors/maintainers, could you please join this thread? - Denis On Thu, Oct 10, 2019 at 7:58 AM Emmanouil Gkatziouras wrote: > Hello everyone, > > I started using Ignite lately. Part of my work involves Pub/Sub, thus I > created a pub/sub streamer. > Pub/Sub is pretty close to Kafka. > Here's how the streamer works. Every node creates an instance of a Pub/Sub > client. > The clients subscribe to the same topic using the same subscription name. > A message is being sent and one of the nodes will receive the message, > process it and then forward it to Ignite using the extractor provided. > > Any thoughts? > > *Emmanouil Gkatziouras* > https://egkatzioura.com/ | > https://www.linkedin.com/in/gkatziourasemmanouil/ > https://github.com/gkatzioura >
Re: Ignite Community Growth Activities and Twitter
Denis, thank you, > > Would you also keep our events page up-to-date? Sure, I will do this gladly. Sally, nice to meet you! пт, 11 окт. 2019 г. в 00:12, Denis Magda : > Kseniya, welcome to the community and thanks for offering the help with > these activities. Most of us are focused on the code over community > + awareness, and that's great to have you among those who can dedicate > more time for community growth and Ignite/ASF promotion. > > Let me also introduce you to Sally Khudairi (ASF VP of Marketing & > Publicity & Sponsor Relations) who can support you from the ASF end. > > I'll send you over the twitter credentials. Would you also keep our events > page up-to-date? [1] Mauricio tended to help us with updates before. > https://ignite.apache.org/events.html > > - > Denis > > > On Thu, Oct 10, 2019 at 12:42 PM Kseniya Romanova < > romanova.ks@gmail.com> wrote: > >> Hi igniters! >> >> For last two years I've been organizing Ignite online and offline events >> for Russia. Now I would like to offer my help with global community growth >> activities: events, working with blog authors, cross-promotion with other >> communities etc. >> >> Could you share access to AI twitter, so I can post sometimes info about >> these activities? >> >> Regards, >> Kseniya >> >
[jira] [Created] (IGNITE-12284) When service class not found on any server nodes, service not deployed without any error
Ilya Kasnacheev created IGNITE-12284:
Summary: When service class not found on any server nodes, service
not deployed without any error
Key: IGNITE-12284
URL: https://issues.apache.org/jira/browse/IGNITE-12284
Project: Ignite
Issue Type: Bug
Components: managed services
Reporter: Ilya Kasnacheev
When service class presents on client node but not on server node, the
following is printed on server:
{code}
[17:57:43,398][SEVERE][srvc-deploy-#44][GridServiceProcessor] Failed to
initialize service (service will not be deployed): FService
class org.apache.ignite.IgniteCheckedException: com.gridgain.datetest.MyService
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10174)
at
org.apache.ignite.internal.processors.service.GridServiceProcessor.copyAndInject(GridServiceProcessor.java:1440)
at
org.apache.ignite.internal.processors.service.GridServiceProcessor.redeploy(GridServiceProcessor.java:1361)
at
org.apache.ignite.internal.processors.service.GridServiceProcessor.processAssignment(GridServiceProcessor.java:1988)
at
org.apache.ignite.internal.processors.service.GridServiceProcessor.onSystemCacheUpdated(GridServiceProcessor.java:1615)
at
org.apache.ignite.internal.processors.service.GridServiceProcessor.access$300(GridServiceProcessor.java:126)
at
org.apache.ignite.internal.processors.service.GridServiceProcessor$ServiceEntriesListener$1.run0(GridServiceProcessor.java:1597)
at
org.apache.ignite.internal.processors.service.GridServiceProcessor$DepRunnable.run(GridServiceProcessor.java:2064)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
com.gridgain.datetest.MyService
at
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:707)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1758)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1717)
at
org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
at
org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10168)
... 10 more
Caused by: java.lang.ClassNotFoundException: com.gridgain.datetest.MyService
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at
org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8775)
at
org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:349)
at
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:698)
... 16 more
{code}
but on client, ServiceDeploymentException is not thrown. Instead, deploy
returns normally.
Code to reproduce:
{code}
public class ServiceStarterMain {
public static void main(String[] args) {
Ignition.setClientMode(true);
Ignite ignite = Ignition.start();
ServiceConfiguration serviceConfiguration = new ServiceConfiguration();
serviceConfiguration.setName("FService");
serviceConfiguration.setMaxPerNodeCount(4);
serviceConfiguration.setTotalCount(10);
serviceConfiguration.setService(new MyService());
ignite.services().deploy(serviceConfiguration); // Exception expected,
but does not happen
}
}
{code}
against a blank Ignite node started from bin distro.
This affects Java and .Net.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Adding experimental support for Intel Optane DC Persistent Memory
Hello Mulugeta, Igniters. Thanks for your interest and efforts in integrating the persistent memory to Ignite. Here are my thoughts on the PR: First of all I was questioning whether we want to use the integration with the pmem library via JNI. Java 14 will have native support for NVME via native mapped byte buffers [1] since the corresponding tickets are already resolved [2],[3]. The suggested integration uses JNI calls quite often (if I read the PR right, there will be tens of JNI calls per a single operation), which may and most likely will undermine the benefits of using PM. On the other hand, there is a PoC/project [4] which successfully demonstrates how the new integration can be used to natively access persistent memory from Java. Additionally, new Unsafe have several performance issues - each put* methods has a map lookup, and CAS method uses a global lock. Removing these issues will change the PR and architecture dramatically. Other than performance and an extra library dependency, there are additional reasons to use custom implementation of PM persistence primitives: - Ignite significantly benefits from having WAL as a single point of data for both failure recovery and historical rebalance. As far as I can guess, the suggested library also uses some sort of journaling in order to support crash recovery. Since we do not want give up on the historical rebalance, we would need to write an additional journal, which then raises a question of how to transact between this journal. - Ignite is supposed to work with large indexes, a I not aware how LLPL handles them. Several researches [5],[6] show that since PM memory has larger access latency than regular RAM, it is beneficial to buffer index pages in regular memory and store leaf pages in PM. There is a generic approach to convert in-memory indexes to PM ones [7], I think it makes sense to investigate whether our existing indexes can be converted to PM. - There is an active IEP which is targeted for a file-based rebalancing. In the suggested implementation this optimization would not work. A better integration should allow to take a snapshot of a partition and transfer it to another node for fast rebalancing. Overall, I think we may take the following path: - Target JEP352 APIs; for now keep using the whole region msync for flush - Introduce PM-based isolated components to support WAL iterator and transaction management. We can start with WAL, then implement a PM-based index based on current research, then a tuple heap with free space management. Each of the components may be developed and tested independently - Do Ignite internals refactoring to properly abstract storage layer from transactions/communication. Currently, there is an abstraction leak in the storage layer, which makes it aware of particular Ignite implementation details (Binary Objects, partition eviction, etc) - Once the storage is separated, we should be able to plug the new components instead of the "classical" ones WDYT? I believe we should create a detailed IEP with concrete proposals on both PM structures design and Ignite internals changes before making any further code changes. [1] https://openjdk.java.net/jeps/352 [2] https://bugs.openjdk.java.net/browse/JDK-8221397 [3] https://bugs.openjdk.java.net/browse/JDK-8221696 [4] https://github.com/jhalliday/mashona [5] https://www.usenix.org/system/files/conference/fast17/fast17-lee.pdf [6] http://www.vldb.org/pvldb/vol8/p786-chen.pdf [7] https://www.cs.utexas.edu/~vijay/papers/sosp19-recipe.pdf [8] https://cwiki.apache.org/confluence/display/IGNITE/IEP-16%3A+Optimization+of+rebalancing вт, 8 окт. 2019 г. в 08:15, Alexey Goncharuk : > Igniters, > > I would like to resurrect this discussion and will review the change again > shortly. If anyone want to join the review - you are welcome! > > ср, 22 авг. 2018 г. в 18:49, Denis Magda : > >> Hi Dmitry, >> >> That's a BSD-3-Clause license if to believe this statement >> "SPDX-License-Identifier: BSD-3-Clause": >> https://github.com/pmem/llpl/blob/master/LICENSE >> >> This license can be used with ASF software: >> https://www.apache.org/legal/resolved.html#category-a >> >> -- >> Denis >> >> On Wed, Aug 22, 2018 at 9:28 AM Dmitriy Pavlov >> wrote: >> >> > Hi Denis, >> > >> > Could you please double check if we can refer to any library licensed to >> > Intel. Can we develop code only version of this support (without >> shipping >> > it in release)? >> > >> > https://github.com/apache/ignite/pull/4381 is quite huge change, >> > including 128 files changed, patch review will require resources from >> > community members to review. I would like to be sure we can include this >> > patch from the legal point of view. >> > >> > Sincerely, >> > Dmitriy Pavlov >> > >> > пт, 3 авг. 2018 г. в 19:23, Dmitriy Pavlov : >> > >> >> Hi Mulugeta, >> >> >> >> I appologise, I've missed that license is already there. So I
Re: Review needed for IGNITE-11410 Sandbox for user-defined code
Hi Denis, In my humble opinion, the security (the sandbox feature is about security, right?) either covers all APIs/subsystems or not. Security should work always and everywhere otherwise it is not security :) > From my point, we should divide the sandbox and features that use it. > Also, I added in the main features of Ignite (cache and compute) the sandbox calls. And at this point, you mixed both in the same pull-request. > I don't see any problem to have the sandbox in the master branch and implement covering for existing and new features if needed. On the other hand, this approach leads to ... ignite-123 [IEP-X] introduces new cool API ignite-124 [IEP-X] improved cool API ignite-125 [IEP-X] fixed a bug ignite-126 [IEP-X] fixed performance drop ignite-127 [IEP-X] Cache API uses new API ignite-127 [IEP-X] Compute grid uses new API ... Why should it be a part of the master branch history? All these things can be done on the feature branch, I think. Anyway, it is up to you. Thanks, S. пт, 11 окт. 2019 г. в 16:31, Denis Garus : > From my point, we should divide the sandbox and features that use it. > The sandbox is fully implemented and has needed tests. > > Also, I added in the main features of Ignite (cache and compute) the > sandbox calls. > > I don't see any problem to have the sandbox in the master branch > and implement covering for existing and new features if needed. > > пт, 11 окт. 2019 г. в 15:21, Вячеслав Коптилин : > > > Hi Denis, > > > > Yep, I understand the scope of the ticket, but... I think it is not a > good > > idea to merge partly implemented feature(s) into the master branch. > > Especially, at this moment. We are at the stage of preparing a new > release > > and I doubt that all improvements, tests (unit tests, integration tests, > > and performance tests) can be implemented before the release branch is > cut > > off. > > Personally, I would prefer to create an epic/feature branch for these > > activities. In that case, we can implement a feature step by step and > merge > > it into the master branch once all components are covered. > > > > > But, sure, we should execute any user-defined code in the sandbox on a > > remote node. Feel free to create issues. > > will do. > > > > Thanks, > > S. > > > > пт, 11 окт. 2019 г. в 14:52, Denis Garus : > > > > > Hello, Slava! > > > > > > The scope of the issue is limited by the following features: > > > > > >- StreamReceiver for DataStreamer; > > >- EntryProcessor; > > >- ComputeJob; > > >- filter and transformer for ScanQuery. > > > > > > But, sure, we should execute any user-defined code in the sandbox on a > > > remote node. > > > Feel free to create issues. > > > > > > Thanks for the feedback! > > > > > > пт, 11 окт. 2019 г. в 13:26, Вячеслав Коптилин < > slava.kopti...@gmail.com > > >: > > > > > > > Hello Denis, Anton, > > > > > > > > Could you please clarify the following aspect? Do we need the same > > > > changes/capabilities related to Continuous Queries, Disco listeners, > > > > CacheStore Factories etc? > > > > > > > > Thanks, > > > > S. > > > > > > > > пт, 11 окт. 2019 г. в 12:24, Anton Vinogradov : > > > > > > > > > Folks, > > > > > > > > > > As a prereviewer, I'd like to say that the solution looks good to > me, > > > but > > > > > fresh eyes would be good. > > > > > > > > > > On Fri, Oct 11, 2019 at 9:40 AM Denis Garus > > > wrote: > > > > > > > > > > > Hello, Igniters! > > > > > > > > > > > > I've raised the PR [1] with the sandbox for AI [2]. > > > > > > Could somebody review it? > > > > > > > > > > > > If you have questions and prefer the Slack, I've created the > > channel > > > > [3]. > > > > > > > > > > > > 1. https://github.com/apache/ignite/pull/6707 > > > > > > 2. https://issues.apache.org/jira/browse/IGNITE-11410 > > > > > > 3. https://app.slack.com/client/T4S1WH2J3/CP8JER880 > > > > > > > > > > > > > > > > > > > > >
Re: Review needed for IGNITE-11410 Sandbox for user-defined code
>From my point, we should divide the sandbox and features that use it. The sandbox is fully implemented and has needed tests. Also, I added in the main features of Ignite (cache and compute) the sandbox calls. I don't see any problem to have the sandbox in the master branch and implement covering for existing and new features if needed. пт, 11 окт. 2019 г. в 15:21, Вячеслав Коптилин : > Hi Denis, > > Yep, I understand the scope of the ticket, but... I think it is not a good > idea to merge partly implemented feature(s) into the master branch. > Especially, at this moment. We are at the stage of preparing a new release > and I doubt that all improvements, tests (unit tests, integration tests, > and performance tests) can be implemented before the release branch is cut > off. > Personally, I would prefer to create an epic/feature branch for these > activities. In that case, we can implement a feature step by step and merge > it into the master branch once all components are covered. > > > But, sure, we should execute any user-defined code in the sandbox on a > remote node. Feel free to create issues. > will do. > > Thanks, > S. > > пт, 11 окт. 2019 г. в 14:52, Denis Garus : > > > Hello, Slava! > > > > The scope of the issue is limited by the following features: > > > >- StreamReceiver for DataStreamer; > >- EntryProcessor; > >- ComputeJob; > >- filter and transformer for ScanQuery. > > > > But, sure, we should execute any user-defined code in the sandbox on a > > remote node. > > Feel free to create issues. > > > > Thanks for the feedback! > > > > пт, 11 окт. 2019 г. в 13:26, Вячеслав Коптилин >: > > > > > Hello Denis, Anton, > > > > > > Could you please clarify the following aspect? Do we need the same > > > changes/capabilities related to Continuous Queries, Disco listeners, > > > CacheStore Factories etc? > > > > > > Thanks, > > > S. > > > > > > пт, 11 окт. 2019 г. в 12:24, Anton Vinogradov : > > > > > > > Folks, > > > > > > > > As a prereviewer, I'd like to say that the solution looks good to me, > > but > > > > fresh eyes would be good. > > > > > > > > On Fri, Oct 11, 2019 at 9:40 AM Denis Garus > > wrote: > > > > > > > > > Hello, Igniters! > > > > > > > > > > I've raised the PR [1] with the sandbox for AI [2]. > > > > > Could somebody review it? > > > > > > > > > > If you have questions and prefer the Slack, I've created the > channel > > > [3]. > > > > > > > > > > 1. https://github.com/apache/ignite/pull/6707 > > > > > 2. https://issues.apache.org/jira/browse/IGNITE-11410 > > > > > 3. https://app.slack.com/client/T4S1WH2J3/CP8JER880 > > > > > > > > > > > > > > >
Re: Review needed for IGNITE-11410 Sandbox for user-defined code
Hi Denis, Yep, I understand the scope of the ticket, but... I think it is not a good idea to merge partly implemented feature(s) into the master branch. Especially, at this moment. We are at the stage of preparing a new release and I doubt that all improvements, tests (unit tests, integration tests, and performance tests) can be implemented before the release branch is cut off. Personally, I would prefer to create an epic/feature branch for these activities. In that case, we can implement a feature step by step and merge it into the master branch once all components are covered. > But, sure, we should execute any user-defined code in the sandbox on a remote node. Feel free to create issues. will do. Thanks, S. пт, 11 окт. 2019 г. в 14:52, Denis Garus : > Hello, Slava! > > The scope of the issue is limited by the following features: > >- StreamReceiver for DataStreamer; >- EntryProcessor; >- ComputeJob; >- filter and transformer for ScanQuery. > > But, sure, we should execute any user-defined code in the sandbox on a > remote node. > Feel free to create issues. > > Thanks for the feedback! > > пт, 11 окт. 2019 г. в 13:26, Вячеслав Коптилин : > > > Hello Denis, Anton, > > > > Could you please clarify the following aspect? Do we need the same > > changes/capabilities related to Continuous Queries, Disco listeners, > > CacheStore Factories etc? > > > > Thanks, > > S. > > > > пт, 11 окт. 2019 г. в 12:24, Anton Vinogradov : > > > > > Folks, > > > > > > As a prereviewer, I'd like to say that the solution looks good to me, > but > > > fresh eyes would be good. > > > > > > On Fri, Oct 11, 2019 at 9:40 AM Denis Garus > wrote: > > > > > > > Hello, Igniters! > > > > > > > > I've raised the PR [1] with the sandbox for AI [2]. > > > > Could somebody review it? > > > > > > > > If you have questions and prefer the Slack, I've created the channel > > [3]. > > > > > > > > 1. https://github.com/apache/ignite/pull/6707 > > > > 2. https://issues.apache.org/jira/browse/IGNITE-11410 > > > > 3. https://app.slack.com/client/T4S1WH2J3/CP8JER880 > > > > > > > > > >
Re: Review needed for IGNITE-11410 Sandbox for user-defined code
Hello, Slava! The scope of the issue is limited by the following features: - StreamReceiver for DataStreamer; - EntryProcessor; - ComputeJob; - filter and transformer for ScanQuery. But, sure, we should execute any user-defined code in the sandbox on a remote node. Feel free to create issues. Thanks for the feedback! пт, 11 окт. 2019 г. в 13:26, Вячеслав Коптилин : > Hello Denis, Anton, > > Could you please clarify the following aspect? Do we need the same > changes/capabilities related to Continuous Queries, Disco listeners, > CacheStore Factories etc? > > Thanks, > S. > > пт, 11 окт. 2019 г. в 12:24, Anton Vinogradov : > > > Folks, > > > > As a prereviewer, I'd like to say that the solution looks good to me, but > > fresh eyes would be good. > > > > On Fri, Oct 11, 2019 at 9:40 AM Denis Garus wrote: > > > > > Hello, Igniters! > > > > > > I've raised the PR [1] with the sandbox for AI [2]. > > > Could somebody review it? > > > > > > If you have questions and prefer the Slack, I've created the channel > [3]. > > > > > > 1. https://github.com/apache/ignite/pull/6707 > > > 2. https://issues.apache.org/jira/browse/IGNITE-11410 > > > 3. https://app.slack.com/client/T4S1WH2J3/CP8JER880 > > > > > >
Re: Review needed for IGNITE-11410 Sandbox for user-defined code
Hello Denis, Anton, Could you please clarify the following aspect? Do we need the same changes/capabilities related to Continuous Queries, Disco listeners, CacheStore Factories etc? Thanks, S. пт, 11 окт. 2019 г. в 12:24, Anton Vinogradov : > Folks, > > As a prereviewer, I'd like to say that the solution looks good to me, but > fresh eyes would be good. > > On Fri, Oct 11, 2019 at 9:40 AM Denis Garus wrote: > > > Hello, Igniters! > > > > I've raised the PR [1] with the sandbox for AI [2]. > > Could somebody review it? > > > > If you have questions and prefer the Slack, I've created the channel [3]. > > > > 1. https://github.com/apache/ignite/pull/6707 > > 2. https://issues.apache.org/jira/browse/IGNITE-11410 > > 3. https://app.slack.com/client/T4S1WH2J3/CP8JER880 > > >
Re: Review needed for IGNITE-11410 Sandbox for user-defined code
Folks, As a prereviewer, I'd like to say that the solution looks good to me, but fresh eyes would be good. On Fri, Oct 11, 2019 at 9:40 AM Denis Garus wrote: > Hello, Igniters! > > I've raised the PR [1] with the sandbox for AI [2]. > Could somebody review it? > > If you have questions and prefer the Slack, I've created the channel [3]. > > 1. https://github.com/apache/ignite/pull/6707 > 2. https://issues.apache.org/jira/browse/IGNITE-11410 > 3. https://app.slack.com/client/T4S1WH2J3/CP8JER880 >
[jira] [Created] (IGNITE-12283) Access restriction to IgniteKernal
Denis Garus created IGNITE-12283: Summary: Access restriction to IgniteKernal Key: IGNITE-12283 URL: https://issues.apache.org/jira/browse/IGNITE-12283 Project: Ignite Issue Type: Task Reporter: Denis Garus IgniteKernal allows a user-defined code to get access to the internal features of Ignite. That behavior leads to security lack. We must encapsulate _IgniteKernal_ to restrict access to it from user-defined code. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12282) Access restriction to the internal package of Ignite
Denis Garus created IGNITE-12282:
Summary: Access restriction to the internal package of Ignite
Key: IGNITE-12282
URL: https://issues.apache.org/jira/browse/IGNITE-12282
Project: Ignite
Issue Type: Task
Reporter: Denis Garus
A user-defined code shouldn't have access to _org.apache.ignite.internal_.*
that is the internal package of Ignite.
To restrict a user-defined code, we need to set the package name to the
_package.access_ security property.
To grand access to a package, we have to use _accessClassInPackage.\{package
name}_ runtime permission.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Review needed for IGNITE-11410 Sandbox for user-defined code
Hello, Igniters! I've raised the PR [1] with the sandbox for AI [2]. Could somebody review it? If you have questions and prefer the Slack, I've created the channel [3]. 1. https://github.com/apache/ignite/pull/6707 2. https://issues.apache.org/jira/browse/IGNITE-11410 3. https://app.slack.com/client/T4S1WH2J3/CP8JER880
