IGNITE-13680

[Nikolay Shubin]

Hello,

My name is Nikolay Shubin.
I want to work on the

   1. IGNITE-13680 


   1. ticket.

I have been logged in to issues.apache.org (username shubin), but I seem to
lack permissions to change the ticket's status (make myself the Assignee,
etc.).
Could you please upgrade my permission so that I can work on this ticket in
accordance with the guidelines
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-Workingonaticket

Sincerely yours,
Nikolay Shubin

Join to dev community

[Aleksey Rostov]

Hello everyone!
My name is Aleksey Rostov, I want to contribute to Ignite.NET. I am
currently a senior .net developer in Luxoft company in Saint Petersburg.
Please, help me to start contributing.
I have created a jira account - arostov.

Re: [DISCUSS] Ignite 3.0 development approach

[Valentin Kulichenko]

Petr,

You have some great points! My comments are below.

-Val

On Fri, Nov 27, 2020 at 4:28 AM Petr Ivanov  wrote:

> More or less, unless we specifically forbid that, I guess
>
> However there is bigger concern: JDK 15 is STS, so after half of a year it
> will be out of support and no major production team will use that JDK in
> their environment.
> I would stick to JDK 11 as it is LTS at least until JDK17, plus — a lot of
> efforts should be made to enforce Apache Ignite be built on JDK 11 alone,
> not to mention 15th version...
>

[Val] I think we will have to stick with Java 11, simply because it's the
current LTS. If we go with 15, almost no one will be able to use Ignite in
production :) We can switch to 17 in the future in case there is any value.


> Also, If we are going to introduce such major changes, I'd like to purpose
> maven project refactoring as well:
> 1. Full revision of all ant-calling tasks with javascript functions and
> alike — the complexity of those are overwhelming, something new should be
> at least researched.
>

[Val] What do we use Ant tasks for? I'm sure we do use them a lot for
release packaging, but it will apparently be significantly simplified. Are
there any other cases?


> 2. Full revisions of profiles (for both root and modules) as there are
> some obsolete ones, and some that do ambiguous or, even worse, totally
> unknown tasks.
>

[Val] Agree - the number of profiles should be at least minimized. In the
best case, we should not have any profiles at all. They are non-intuitive
for developers, and also often confuse IDEs.


> 3. Introduce plugin and dependency management sections to control over and
> concrete versions of software we are relying in our project. Additionally —
> add BOM with all Ignite modules and their dependencies, which should help
> our users to better embed Ignite to their projects.
> 4. Up all versions of plugins and dependencies where possible to there
> current production versions (for plugins — it should be a must if we are
> ever going to build project under latest JDK versions).
>

[Val] Agree with both.


> 5. Prepare project for parallel building, testing and assembling where
> possible for faster development process, with possible additional
> enhancement like incremental rebuild.
>

[Val] Could you elaborate on this? What should be parallelized in your
view, and how exactly it will speed up the process?


> 6. Possibly — research alternate builders, like Gradle (thought there are
> a lot of questions to its race condition issues during multimodule builds).
>

[Val] I'm not sure we need this. Gradle does seem to be nicer than Maven in
many aspects, but this will be a big transition for the community. The
value of such a transition is not clear. We also seem to have much more
experience in Maven than in Gradle.


> And last, but not least — think of migrating to 'CI/CD as a Code' approach
> for our main instrument TeamCity.
> Whole project (both test and release build configurations) can be
> described using DSL (Kotlin in case of TC) and stored in VCS, forcing
> infrastructure changes to go through the same development processes
> including discussions, review, change history and etc.
>

[Val] Huge +1.


>
> Only I am not sure for now about where should the code be stored — in
> separate repository (secure, but disables testing of code with TC settings
> both in single PR), or alongside project's code (can be possible security
> hole).
> That would require additional dev thread I think.
>
>
>
> WDYT?
>
> > On 24 Nov 2020, at 20:04, Pavel Tupitsyn  wrote:
> >
> > If we use Java15 for development, can the resulting package be used from
> a
> > Java11 app (the latest LTS)?
> >
> > On Tue, Nov 24, 2020 at 7:51 PM Andrey Mashenkov <
> andrey.mashen...@gmail.com>
> > wrote:
> >
> >> Jave15 looks awesome.
> >>
> >> * Hidden classes [1] can be used by codegenerators.
> >> * Records [2] can replace boilerplate code like IgniteBiTuple,
> GridTupleX.
> >>
> >> [1] https://openjdk.java.net/jeps/371
> >> [2] https://openjdk.java.net/jeps/384
> >>
> >> On Tue, Nov 24, 2020 at 3:38 PM Alexey Zinoviev  >
> >> wrote:
> >>
> >>> Java 15 is better, VarHandles, ForeignMemory access and so on.
> >>>
> >>> In both cases, I support the Java version 11 and higher for the
> >>> development!
> >>>
> >>> вт, 24 нояб. 2020 г. в 15:21, Andrey Mashenkov <
> >> andrey.mashen...@gmail.com
>  :
> >>>
>  Let's add maven plugins  for sanity checks at the early stage.
>  I've created a ticket for this [1].
> 
>  Also, I've found initial pom.xml has a target version Java 8.
>  Do we intend to move to Java 11 (or may be next LTS) and drop Java 8
> in
>  Ignite 3.0?
> 
>  [1] https://issues.apache.org/jira/browse/IGNITE-13751
> 
>  On Tue, Nov 24, 2020 at 5:40 AM Valentin Kulichenko <
>  valentin.kuliche...@gmail.com> wrote:
> 
> > Folks,
> >
> > I went ahead and created the repository [1]. I also configured a
> 

Re: Issue with custom security plugin and thin clients

[Vishwas Bm]

Hi Denis,


Thanks for the reply.
Yes I was looking for a way to spread the security context to all cluster
nodes when a thin client(sqlline) gets authenticated.
I tried to see if I can use node attributes or user attributes to pass the
information to other nodes. When a cluster of ignite server is already
formed, this will not help as attributes will not be available on remote
nodes.

The node attributes cannot be changed at run time and the attributes will
be available to remote nodes only when they join the cluster.

So I wanted to know, if there is any other way to do this ?
I checked your poc PR for reference,
https://github.com/apache/ignite/pull/7375

In thin client case authenticate node will not be called but authenticate
method is getting called.


Regards,
Vishwas


On Fri, 27 Nov, 2020, 14:29 Denis Garus,  wrote:

> Hello!
>
>
> If I understood your problem correctly, you need to make a thin client's
> security context allowed on a remote node.
>
> When a security plugin does authenticate a thin client, it should spread
> the thin client's security context on the cluster.
>
> How a security context will be transmitted to a remote node is up to the
> plugin's developers.
>
> Also, you have to implement the GridSecurityProcessor.securityContext(UUID
> subjId) method,
>
> the way this method is used in Ignite can see in the task description [1].
>
>
>
>
>1. https://issues.apache.org/jira/browse/IGNITE-12759
>
>
> чт, 26 нояб. 2020 г. в 10:01, Vishwas Bm :
>
> > Hi,
> >
> > I was facing an issue with a custom security plugin and thin remote
> client.
> > I am using Ignite 2.9.0 version and I am hitting below issue
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-41%3A+Security+Context+of+thin+client+on+remote+nodes
> >
> >
> > I had asked the question in the user listing but unfortunately I did not
> > get any reply.
> > So I am posting this question here:
> >
> >
> >
> http://apache-ignite-users.70518.x6.nabble.com/Query-on-implementing-GridSecurityProcessor-td34672.html
> >
> >
> > *Thanks & Regards,*
> >
> > *Vishwas *
> >
>

Re: IEP-61 Technical discussion

[Alexey Goncharuk]

Folks, thanks to everyone who joined the call. Summary:

   - We agree that it may be beneficial to separate metastorage and group
   membership services, however, the abstractions should be clean enough so
   that we could implement group membership via metastorage
   - Production cluster setup will involve an administrator 'init' command
   that will initialize the metastorage raft group. Once the metastorage is
   initialized, all nodes may be restarted arbitrarily
   - HA cluster must contain at least 3 nodes. 2-node cluster will stop
   progress when one of the nodes fails (due to metastorage requirements)
   - We will provide a 'developer' cluster mode which will allow a 1-node
   setup and auto-initialization without the 'init' command
   - We are targeting centralized affinity calculation that will be stored
   to the metastorage. Metastorage downtime does not necessarily mean cluster
   availability (subject to the partition replication protocol choice). It
   would be good to maximally hide the partition object so that we could
   support range partitioning in the future

To discuss at the next meeting (do not hesitate to send questions here
before the meeting):

   - Raft implementation details (API model, porting, etc)
   - Transactions interaction with replication protocol
   - Weaker consistency options

Please add more if I forgot something and let's choose a time for the next
meeting.

--AG

чт, 26 нояб. 2020 г. в 16:12, Kseniya Romanova :

> Done
>
> чт, 26 нояб. 2020 г. в 13:18, Ivan Daschinsky :
>
> > Alexey, is it possible to manage call at 16:00 MSK?
> >
> > чт, 26 нояб. 2020 г. в 12:30, Alexey Goncharuk <
> alexey.goncha...@gmail.com
> > >:
> >
> > > Hi Ivan,
> > >
> > > Unfortunately, the earliest window available for us is 12:00 MSK (1
> hour
> > > slot), or after 14:30 MSK. Let me know what time works best for you.
> > >
> > > ср, 25 нояб. 2020 г. в 21:38, Ivan Daschinsky :
> > >
> > > > Alexey, I kindly ask you to move the meeting a little bit earlier,
> > ideal
> > > > variant -- in the morning.
> > > >
> > > > ср, 25 нояб. 2020 г. в 20:10, Alexey Goncharuk <
> > > alexey.goncha...@gmail.com
> > > > >:
> > > >
> > > > > Folks, let's have the call on Friday, Nov 27th at 18:00 MSK? We can
> > use
> > > > the
> > > > > following waiting room link:
> > > > >
> https://zoom.us/j/99450012496?pwd=RWZmOGhCNWlRK0ZpamdOOTZsYTJ0dz09
> > > > >
> > > > > Let me know if this time works for everybody.
> > > > >
> > > > > ср, 25 нояб. 2020 г. в 16:42, Alexey Goncharuk <
> > > > alexey.goncha...@gmail.com
> > > > > >:
> > > > >
> > > > > > Folks,
> > > > > >
> > > > > > I've made some edits in IEP-61 [1] regarding the group membership
> > > > service
> > > > > > and transaction protocol interaction with the replication
> > > > infrastructure,
> > > > > > please take a look before our Friday call.
> > > > > >
> > > > > > [1]
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-61%3A+Common+Replication+Infrastructure
> > > > > >
> > > > > > пн, 23 нояб. 2020 г. в 13:28, Alexey Goncharuk <
> > > > > alexey.goncha...@gmail.com
> > > > > > >:
> > > > > >
> > > > > >> Thanks, Ivan,
> > > > > >>
> > > > > >> Another protocol for group membership worth checking out is
> RAPID
> > > [1]
> > > > (a
> > > > > >> recent one). Not sure though if there are any available
> > > > implementations
> > > > > for
> > > > > >> it already.
> > > > > >>
> > > > > >> [1]
> > > > >
> > https://www.usenix.org/system/files/conference/atc18/atc18-suresh.pdf
> > > > > >>
> > > > > >> пн, 23 нояб. 2020 г. в 10:46, Ivan Daschinsky <
> > ivanda...@gmail.com
> > > >:
> > > > > >>
> > > > > >>> Also, here is some interesting reading about gossip, SWIM etc.
> > > > > >>>
> > > > > >>> 1 --
> > > > > >>>
> > > >
> http://www.cs.cornell.edu/Info/Projects/Spinglass/public_pdfs/SWIM.pdf
> > > > > >>> 2 --
> > > > > >>>
> > > > > >>>
> > > > >
> > > >
> > >
> >
> http://www.antonkharenko.com/2015/09/swim-distributed-group-membership.html
> > > > > >>> 3 -- https://github.com/hashicorp/memberlist (Foundation
> library
> > > of
> > > > > >>> hashicorp serf)
> > > > > >>> 4 -- https://github.com/scalecube/scalecube-cluster -- (Java
> > > > > >>> implementation
> > > > > >>> of SWIM)
> > > > > >>>
> > > > > >>> чт, 19 нояб. 2020 г. в 16:35, Ivan Daschinsky <
> > ivanda...@gmail.com
> > > >:
> > > > > >>>
> > > > > >>> > >> Friday, Nov 27th work for you? If ok, let's have an open
> > call
> > > > > then.
> > > > > >>> > Yes, great
> > > > > >>> > >> As for the protocol port - we will not be dealing with the
> > > > > >>> > concurrency...
> > > > > >>> > >>Judging by the Rust port, it seems fairly straightforward.
> > > > > >>> > Yes, they chose split transport and logic. But original Go
> > > package
> > > > > from
> > > > > >>> > etcd (see raft/node.go) contains some  heartbeats mechanism
> > etc.
> > > > > >>> > I agree with you, this seems not to be a huge deal to port.
> > > > > >>> >
> > > > 

[jira] [Created] (IGNITE-13771) ODBC crashes on Linux if SQLConnect is called

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13771:


 Summary: ODBC crashes on Linux if SQLConnect is called
 Key: IGNITE-13771
 URL: https://issues.apache.org/jira/browse/IGNITE-13771
 Project: Ignite
  Issue Type: Bug
  Components: odbc
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.10


According to user report the following line 

{code:cpp}

et = SQLConnect(dbc, (SQLCHAR*)DSN, SQL_NTS, (SQLCHAR*)"", SQL_NTS,
(SQLCHAR*)"", SQL_NTS);

{code:cpp}

Gives the following result:
{noformat}
symbol lookup error: /usr/local/lib/libignite-odbc.so: undefined
symbol: SQLGetPrivateProfileString {noformat}
 

Details can be found on userlist: 
https://lists.apache.org/thread.html/r023842d3fa08d9e7c6e8dd6686efaa85b46ea2b14dd22883eecfbe17%40



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Ignite 3.0 development approach

[Ilya Kasnacheev]

Hello!

One can still submit a PR which mines bitcoin and run tests on it. CI/CD
configuration alongside code does not solve the issue that any user may run
code on TC.

Regards,
-- 
Ilya Kasnacheev


пт, 27 нояб. 2020 г. в 16:48, Petr Ivanov :

> > Storing CI/CD code (yaml definitions for Travis/Azure/GH Actions, Jenkins
> > pipelines, etc) in the same repo is very common.
> > Secrets are stored separately (e.g. GitHub secrets), TeamCity probably
> has
> > a similar feature.
>
>
> My main security concern — anyone, including third-party person without
> even committer permissions can modify build configuration in such a way
> that it will be doing something not intend to do (bitcoin mining for
> instance) or even something harmful (like trying to attack underlying TC
> infrastructure). If we are to store CI/CD configuration alongside code,
> some restrictions are required.
>
>
>
> > On 27 Nov 2020, at 15:59, Pavel Tupitsyn  wrote:
> >
> >> migrating to 'CI/CD as a Code'
> >
> > Huge +1 for this.
> >
> >
> >> where should the code be stored ..
> >> alongside project's code (can be possible security hole)
> >
> > Storing CI/CD code (yaml definitions for Travis/Azure/GH Actions, Jenkins
> > pipelines, etc) in the same repo is very common.
> > Secrets are stored separately (e.g. GitHub secrets), TeamCity probably
> has
> > a similar feature.
> >
> >
> > On Fri, Nov 27, 2020 at 3:28 PM Petr Ivanov  wrote:
> >
> >> More or less, unless we specifically forbid that, I guess
> >>
> >> However there is bigger concern: JDK 15 is STS, so after half of a year
> it
> >> will be out of support and no major production team will use that JDK in
> >> their environment.
> >> I would stick to JDK 11 as it is LTS at least until JDK17, plus — a lot
> of
> >> efforts should be made to enforce Apache Ignite be built on JDK 11
> alone,
> >> not to mention 15th version...
> >>
> >>
> >>
> >> Also, If we are going to introduce such major changes, I'd like to
> purpose
> >> maven project refactoring as well:
> >> 1. Full revision of all ant-calling tasks with javascript functions and
> >> alike — the complexity of those are overwhelming, something new should
> be
> >> at least researched.
> >> 2. Full revisions of profiles (for both root and modules) as there are
> >> some obsolete ones, and some that do ambiguous or, even worse, totally
> >> unknown tasks.
> >> 3. Introduce plugin and dependency management sections to control over
> and
> >> concrete versions of software we are relying in our project.
> Additionally —
> >> add BOM with all Ignite modules and their dependencies, which should
> help
> >> our users to better embed Ignite to their projects.
> >> 4. Up all versions of plugins and dependencies where possible to there
> >> current production versions (for plugins — it should be a must if we are
> >> ever going to build project under latest JDK versions).
> >> 5. Prepare project for parallel building, testing and assembling where
> >> possible for faster development process, with possible additional
> >> enhancement like incremental rebuild.
> >> 6. Possibly — research alternate builders, like Gradle (thought there
> are
> >> a lot of questions to its race condition issues during multimodule
> builds).
> >>
> >>
> >>
> >> And last, but not least — think of migrating to 'CI/CD as a Code'
> approach
> >> for our main instrument TeamCity.
> >> Whole project (both test and release build configurations) can be
> >> described using DSL (Kotlin in case of TC) and stored in VCS, forcing
> >> infrastructure changes to go through the same development processes
> >> including discussions, review, change history and etc.
> >>
> >> Only I am not sure for now about where should the code be stored — in
> >> separate repository (secure, but disables testing of code with TC
> settings
> >> both in single PR), or alongside project's code (can be possible
> security
> >> hole).
> >> That would require additional dev thread I think.
> >>
> >>
> >>
> >> WDYT?
> >>
> >>> On 24 Nov 2020, at 20:04, Pavel Tupitsyn  wrote:
> >>>
> >>> If we use Java15 for development, can the resulting package be used
> from
> >> a
> >>> Java11 app (the latest LTS)?
> >>>
> >>> On Tue, Nov 24, 2020 at 7:51 PM Andrey Mashenkov <
> >> andrey.mashen...@gmail.com>
> >>> wrote:
> >>>
>  Jave15 looks awesome.
> 
>  * Hidden classes [1] can be used by codegenerators.
>  * Records [2] can replace boilerplate code like IgniteBiTuple,
> >> GridTupleX.
> 
>  [1] https://openjdk.java.net/jeps/371
>  [2] https://openjdk.java.net/jeps/384
> 
>  On Tue, Nov 24, 2020 at 3:38 PM Alexey Zinoviev <
> zaleslaw@gmail.com
> >>>
>  wrote:
> 
> > Java 15 is better, VarHandles, ForeignMemory access and so on.
> >
> > In both cases, I support the Java version 11 and higher for the
> > development!
> >
> > вт, 24 нояб. 2020 г. в 15:21, Andrey Mashenkov <
>  andrey.mashen...@gmail.com
> >> :
> >
> >> 

Re: [DISCUSS] Ignite 3.0 development approach

[Petr Ivanov]

> Storing CI/CD code (yaml definitions for Travis/Azure/GH Actions, Jenkins
> pipelines, etc) in the same repo is very common.
> Secrets are stored separately (e.g. GitHub secrets), TeamCity probably has
> a similar feature.


My main security concern — anyone, including third-party person without even 
committer permissions can modify build configuration in such a way that it will 
be doing something not intend to do (bitcoin mining for instance) or even 
something harmful (like trying to attack underlying TC infrastructure). If we 
are to store CI/CD configuration alongside code, some restrictions are required.



> On 27 Nov 2020, at 15:59, Pavel Tupitsyn  wrote:
> 
>> migrating to 'CI/CD as a Code'
> 
> Huge +1 for this.
> 
> 
>> where should the code be stored ..
>> alongside project's code (can be possible security hole)
> 
> Storing CI/CD code (yaml definitions for Travis/Azure/GH Actions, Jenkins
> pipelines, etc) in the same repo is very common.
> Secrets are stored separately (e.g. GitHub secrets), TeamCity probably has
> a similar feature.
> 
> 
> On Fri, Nov 27, 2020 at 3:28 PM Petr Ivanov  wrote:
> 
>> More or less, unless we specifically forbid that, I guess
>> 
>> However there is bigger concern: JDK 15 is STS, so after half of a year it
>> will be out of support and no major production team will use that JDK in
>> their environment.
>> I would stick to JDK 11 as it is LTS at least until JDK17, plus — a lot of
>> efforts should be made to enforce Apache Ignite be built on JDK 11 alone,
>> not to mention 15th version...
>> 
>> 
>> 
>> Also, If we are going to introduce such major changes, I'd like to purpose
>> maven project refactoring as well:
>> 1. Full revision of all ant-calling tasks with javascript functions and
>> alike — the complexity of those are overwhelming, something new should be
>> at least researched.
>> 2. Full revisions of profiles (for both root and modules) as there are
>> some obsolete ones, and some that do ambiguous or, even worse, totally
>> unknown tasks.
>> 3. Introduce plugin and dependency management sections to control over and
>> concrete versions of software we are relying in our project. Additionally —
>> add BOM with all Ignite modules and their dependencies, which should help
>> our users to better embed Ignite to their projects.
>> 4. Up all versions of plugins and dependencies where possible to there
>> current production versions (for plugins — it should be a must if we are
>> ever going to build project under latest JDK versions).
>> 5. Prepare project for parallel building, testing and assembling where
>> possible for faster development process, with possible additional
>> enhancement like incremental rebuild.
>> 6. Possibly — research alternate builders, like Gradle (thought there are
>> a lot of questions to its race condition issues during multimodule builds).
>> 
>> 
>> 
>> And last, but not least — think of migrating to 'CI/CD as a Code' approach
>> for our main instrument TeamCity.
>> Whole project (both test and release build configurations) can be
>> described using DSL (Kotlin in case of TC) and stored in VCS, forcing
>> infrastructure changes to go through the same development processes
>> including discussions, review, change history and etc.
>> 
>> Only I am not sure for now about where should the code be stored — in
>> separate repository (secure, but disables testing of code with TC settings
>> both in single PR), or alongside project's code (can be possible security
>> hole).
>> That would require additional dev thread I think.
>> 
>> 
>> 
>> WDYT?
>> 
>>> On 24 Nov 2020, at 20:04, Pavel Tupitsyn  wrote:
>>> 
>>> If we use Java15 for development, can the resulting package be used from
>> a
>>> Java11 app (the latest LTS)?
>>> 
>>> On Tue, Nov 24, 2020 at 7:51 PM Andrey Mashenkov <
>> andrey.mashen...@gmail.com>
>>> wrote:
>>> 
 Jave15 looks awesome.
 
 * Hidden classes [1] can be used by codegenerators.
 * Records [2] can replace boilerplate code like IgniteBiTuple,
>> GridTupleX.
 
 [1] https://openjdk.java.net/jeps/371
 [2] https://openjdk.java.net/jeps/384
 
 On Tue, Nov 24, 2020 at 3:38 PM Alexey Zinoviev >> 
 wrote:
 
> Java 15 is better, VarHandles, ForeignMemory access and so on.
> 
> In both cases, I support the Java version 11 and higher for the
> development!
> 
> вт, 24 нояб. 2020 г. в 15:21, Andrey Mashenkov <
 andrey.mashen...@gmail.com
>> :
> 
>> Let's add maven plugins  for sanity checks at the early stage.
>> I've created a ticket for this [1].
>> 
>> Also, I've found initial pom.xml has a target version Java 8.
>> Do we intend to move to Java 11 (or may be next LTS) and drop Java 8
>> in
>> Ignite 3.0?
>> 
>> [1] https://issues.apache.org/jira/browse/IGNITE-13751
>> 
>> On Tue, Nov 24, 2020 at 5:40 AM Valentin Kulichenko <
>> valentin.kuliche...@gmail.com> wrote:
>> 
>>> Folks,
>>> 

[jira] [Created] (IGNITE-13770) Ignite.dataRegionMetrics throws NPE with empty persisted region

[Ilya Kasnacheev (Jira)]

Ilya Kasnacheev created IGNITE-13770:


 Summary: Ignite.dataRegionMetrics throws NPE with empty persisted 
region
 Key: IGNITE-13770
 URL: https://issues.apache.org/jira/browse/IGNITE-13770
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.9
Reporter: Ilya Kasnacheev
Assignee: Ilya Kasnacheev
 Fix For: 2.9.1


{code}
java.lang.NullPointerException
at 
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointBufferPagesCount(PageMemoryImpl.java:1823)
at 
org.apache.ignite.internal.processors.cache.persistence.DataRegionMetricsImpl.getUsedCheckpointBufferPages(DataRegionMetricsImpl.java:287)
at 
org.apache.ignite.internal.processors.cache.persistence.DataRegionMetricsSnapshot.(DataRegionMetricsSnapshot.java:108)
at 
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.memoryMetrics(IgniteCacheDatabaseSharedManager.java:815)
at 
org.apache.ignite.internal.IgniteKernal.dataRegionMetrics(IgniteKernal.java:3773)
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Ignite 3.0 development approach

[Pavel Tupitsyn]

> migrating to 'CI/CD as a Code'

Huge +1 for this.


> where should the code be stored ..
> alongside project's code (can be possible security hole)

Storing CI/CD code (yaml definitions for Travis/Azure/GH Actions, Jenkins
pipelines, etc) in the same repo is very common.
Secrets are stored separately (e.g. GitHub secrets), TeamCity probably has
a similar feature.


On Fri, Nov 27, 2020 at 3:28 PM Petr Ivanov  wrote:

> More or less, unless we specifically forbid that, I guess
>
> However there is bigger concern: JDK 15 is STS, so after half of a year it
> will be out of support and no major production team will use that JDK in
> their environment.
> I would stick to JDK 11 as it is LTS at least until JDK17, plus — a lot of
> efforts should be made to enforce Apache Ignite be built on JDK 11 alone,
> not to mention 15th version...
>
>
>
> Also, If we are going to introduce such major changes, I'd like to purpose
> maven project refactoring as well:
> 1. Full revision of all ant-calling tasks with javascript functions and
> alike — the complexity of those are overwhelming, something new should be
> at least researched.
> 2. Full revisions of profiles (for both root and modules) as there are
> some obsolete ones, and some that do ambiguous or, even worse, totally
> unknown tasks.
> 3. Introduce plugin and dependency management sections to control over and
> concrete versions of software we are relying in our project. Additionally —
> add BOM with all Ignite modules and their dependencies, which should help
> our users to better embed Ignite to their projects.
> 4. Up all versions of plugins and dependencies where possible to there
> current production versions (for plugins — it should be a must if we are
> ever going to build project under latest JDK versions).
> 5. Prepare project for parallel building, testing and assembling where
> possible for faster development process, with possible additional
> enhancement like incremental rebuild.
> 6. Possibly — research alternate builders, like Gradle (thought there are
> a lot of questions to its race condition issues during multimodule builds).
>
>
>
> And last, but not least — think of migrating to 'CI/CD as a Code' approach
> for our main instrument TeamCity.
> Whole project (both test and release build configurations) can be
> described using DSL (Kotlin in case of TC) and stored in VCS, forcing
> infrastructure changes to go through the same development processes
> including discussions, review, change history and etc.
>
> Only I am not sure for now about where should the code be stored — in
> separate repository (secure, but disables testing of code with TC settings
> both in single PR), or alongside project's code (can be possible security
> hole).
> That would require additional dev thread I think.
>
>
>
> WDYT?
>
> > On 24 Nov 2020, at 20:04, Pavel Tupitsyn  wrote:
> >
> > If we use Java15 for development, can the resulting package be used from
> a
> > Java11 app (the latest LTS)?
> >
> > On Tue, Nov 24, 2020 at 7:51 PM Andrey Mashenkov <
> andrey.mashen...@gmail.com>
> > wrote:
> >
> >> Jave15 looks awesome.
> >>
> >> * Hidden classes [1] can be used by codegenerators.
> >> * Records [2] can replace boilerplate code like IgniteBiTuple,
> GridTupleX.
> >>
> >> [1] https://openjdk.java.net/jeps/371
> >> [2] https://openjdk.java.net/jeps/384
> >>
> >> On Tue, Nov 24, 2020 at 3:38 PM Alexey Zinoviev  >
> >> wrote:
> >>
> >>> Java 15 is better, VarHandles, ForeignMemory access and so on.
> >>>
> >>> In both cases, I support the Java version 11 and higher for the
> >>> development!
> >>>
> >>> вт, 24 нояб. 2020 г. в 15:21, Andrey Mashenkov <
> >> andrey.mashen...@gmail.com
>  :
> >>>
>  Let's add maven plugins  for sanity checks at the early stage.
>  I've created a ticket for this [1].
> 
>  Also, I've found initial pom.xml has a target version Java 8.
>  Do we intend to move to Java 11 (or may be next LTS) and drop Java 8
> in
>  Ignite 3.0?
> 
>  [1] https://issues.apache.org/jira/browse/IGNITE-13751
> 
>  On Tue, Nov 24, 2020 at 5:40 AM Valentin Kulichenko <
>  valentin.kuliche...@gmail.com> wrote:
> 
> > Folks,
> >
> > I went ahead and created the repository [1]. I also configured a
> >>> TeamCity
> > project [2] that runs all available JUnit tests on every PR creation
> >> or
> > update. It also sends the status update to GitHub so that it's
> >>> reflected
>  in
> > the PR itself so that we can do merges directly from GitHub. Basic
> >>> steps
>  to
> > make a change are described on the Wiki page [3].
> >
> > Let me know if you have any questions.
> >
> > [1] https://github.com/apache/ignite-3
> > [2] https://ci.ignite.apache.org/project/ignite3
> > [3]
> >
> >
> 
> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0#ApacheIgnite3.0-DevelopmentProcess
> >
> > -Val
> >
> > On Wed, Nov 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Sure, I'll do that.

On Fri, Nov 27, 2020 at 2:00 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> You can override these values (module, suites) values when running a suite
> on TC. Can you please run these ones which need to be changed individually
> on TC, make sure they run without errors and contain all the needed tests,
> and link to these runs in the ticket? Then I can modify the suites to fit
> those.
>
> I'm not sure that class shadowing will work as we want it to work, e.g., we
> now have two IgniteCacheQuerySelfTestSuite6 with the same FQDN, I'm not
> sure if maven/TC is going to pick both or just one.
> Maybe they should go to a different package, e.g., testsuites/core for
> every suite already present in indexing/spring/etc. Maybe you can rename
> them just now? This will mean a lot less of work reconfiguring suites.
> In TC configurations, suite names are simple class names, not FQ, so no
> changes may be needed at all.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 27 нояб. 2020 г. в 13:03, Max Timonin :
>
> > Hi, sorry for the misleading. I mean "adding ignite-core module *suites*
> to
> > the TeamCity Queries* suite"
> >
> > On Fri, Nov 27, 2020 at 12:44 PM Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com>
> > wrote:
> >
> > > Hello!
> > >
> > > What do you mean by "adding ignite-core to suite"? ignite-core is a top
> > > dependency and its tests are also included in all other modules' tests
> > > classpath since it provides GridAbstractTest.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > пт, 27 нояб. 2020 г. в 01:24, Max Timonin :
> > >
> > > > Hi, Ilya!
> > > >
> > > > So, I've updated PR, fixed comments and removed Core* prefixes. MTCGA
> > > shows
> > > > no blockers, but it was 2 weeks ago, so I've started it again.
> > > >
> > > > If PR is OK then there are some suites that should be updated on TC.
> > > Could
> > > > you please tell me how we can do it?
> > > >
> > > > 1. Add ignite-cassandra-serializers suite:
> > > >
> > > >1. org.apache.ignite.tests.SerializerSuite
> > > >
> > > > 2. Add ignite-core to Queries* TC suite:
> > > >
> > > >1. org.apache.ignite.client.IgniteClientTestSuite
> > > >2. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite
> > > >3. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite2
> > > >4. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite3
> > > >5. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite4
> > > >6. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite5
> > > >7. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite6
> > > >8. org.apache.ignite.suites.IgnitePdsWithIndexingCoreTestSuite
> > > >9. org.apache.ignite.suites.IgniteCacheMvccSqlTestSuite
> > > >
> > > > 3. Remove ignite-indexing from TC suites:
> > > >
> > > >1. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite3
> > > >2. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite4
> > > >3. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite5
> > > >
> > > > 4. Add ignite-core to Spring* TC suite:
> > > >
> > > >1. org.apache.ignite.testsuites.IgniteSpringTestSuite
> > > >
> > > > 5. Add ignite-core suite (depends on uri-deployment module):
> > > >
> > > >1. org.apache.ignite.testsuites.IgniteUriDeploymentTestSuite
> > > >
> > > > 6. Add ignite-core suite to Zookeeper TC suite:
> > > >
> > > >1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
> > > >
> > > > 7. Remove ignite-zookeeper test suite:
> > > >
> > > >1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
> > > >
> > > > 8. Add ignite-ml test suites:
> > > >
> > > >1. org.apache.ignite.ml.math.distances.DistancesTestSuite
> > > >2. org.apache.ignite.ml.naivebayes.NaiveBayesTestSuite
> > > >
> > > >
> > > > On Wed, Nov 25, 2020 at 4:26 PM Ilya Kasnacheev <
> > > ilya.kasnach...@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hello!
> > > > >
> > > > > Yes, we have such tests which depend on ignite-indexing or
> > > ignite-spring.
> > > > > They just need to be included in Spring* or Queries* test suite.
> Then
> > > > they
> > > > > will be executed on TC in the correct context. You can also run
> these
> > > > tests
> > > > > from IDEA by specifying other module as classpath. No need to move
> > the
> > > > > classes around.
> > > > >
> > > > > I will check the PR.
> > > > >
> > > > > Regards,
> > > > > --
> > > > > Ilya Kasnacheev
> > > > >
> > > > >
> > > > > ср, 25 нояб. 2020 г. в 00:22, Max Timonin  >:
> > > > >
> > > > > > Ilya, Anton, Ivan, hi!
> > > > > >
> > > > > > I fix some comments you leave in the PR. Also I checked some test
> > > > suites
> > > > > > and found that some tests are written in the core module but
> depend
> > > on
> > > > > the
> > > > > > indexing module (or other modules). Some of such test classes
> > contain
> > > > > tests
> > > > > > that are related to the core functionality, but some to indexing.
> > I'm
> > > > not
> > > > > > sure 

Re: [DISCUSS] Ignite 3.0 development approach

[Petr Ivanov]

More or less, unless we specifically forbid that, I guess

However there is bigger concern: JDK 15 is STS, so after half of a year it will 
be out of support and no major production team will use that JDK in their 
environment.
I would stick to JDK 11 as it is LTS at least until JDK17, plus — a lot of 
efforts should be made to enforce Apache Ignite be built on JDK 11 alone, not 
to mention 15th version...



Also, If we are going to introduce such major changes, I'd like to purpose 
maven project refactoring as well:
1. Full revision of all ant-calling tasks with javascript functions and alike — 
the complexity of those are overwhelming, something new should be at least 
researched.
2. Full revisions of profiles (for both root and modules) as there are some 
obsolete ones, and some that do ambiguous or, even worse, totally unknown tasks.
3. Introduce plugin and dependency management sections to control over and 
concrete versions of software we are relying in our project. Additionally — add 
BOM with all Ignite modules and their dependencies, which should help our users 
to better embed Ignite to their projects.
4. Up all versions of plugins and dependencies where possible to there current 
production versions (for plugins — it should be a must if we are ever going to 
build project under latest JDK versions).
5. Prepare project for parallel building, testing and assembling where possible 
for faster development process, with possible additional enhancement like 
incremental rebuild.
6. Possibly — research alternate builders, like Gradle (thought there are a lot 
of questions to its race condition issues during multimodule builds).



And last, but not least — think of migrating to 'CI/CD as a Code' approach for 
our main instrument TeamCity.
Whole project (both test and release build configurations) can be described 
using DSL (Kotlin in case of TC) and stored in VCS, forcing infrastructure 
changes to go through the same development processes including discussions, 
review, change history and etc.

Only I am not sure for now about where should the code be stored — in separate 
repository (secure, but disables testing of code with TC settings both in 
single PR), or alongside project's code (can be possible security hole).
That would require additional dev thread I think.



WDYT?

> On 24 Nov 2020, at 20:04, Pavel Tupitsyn  wrote:
> 
> If we use Java15 for development, can the resulting package be used from a
> Java11 app (the latest LTS)?
> 
> On Tue, Nov 24, 2020 at 7:51 PM Andrey Mashenkov 
> wrote:
> 
>> Jave15 looks awesome.
>> 
>> * Hidden classes [1] can be used by codegenerators.
>> * Records [2] can replace boilerplate code like IgniteBiTuple, GridTupleX.
>> 
>> [1] https://openjdk.java.net/jeps/371
>> [2] https://openjdk.java.net/jeps/384
>> 
>> On Tue, Nov 24, 2020 at 3:38 PM Alexey Zinoviev 
>> wrote:
>> 
>>> Java 15 is better, VarHandles, ForeignMemory access and so on.
>>> 
>>> In both cases, I support the Java version 11 and higher for the
>>> development!
>>> 
>>> вт, 24 нояб. 2020 г. в 15:21, Andrey Mashenkov <
>> andrey.mashen...@gmail.com
 :
>>> 
 Let's add maven plugins  for sanity checks at the early stage.
 I've created a ticket for this [1].
 
 Also, I've found initial pom.xml has a target version Java 8.
 Do we intend to move to Java 11 (or may be next LTS) and drop Java 8 in
 Ignite 3.0?
 
 [1] https://issues.apache.org/jira/browse/IGNITE-13751
 
 On Tue, Nov 24, 2020 at 5:40 AM Valentin Kulichenko <
 valentin.kuliche...@gmail.com> wrote:
 
> Folks,
> 
> I went ahead and created the repository [1]. I also configured a
>>> TeamCity
> project [2] that runs all available JUnit tests on every PR creation
>> or
> update. It also sends the status update to GitHub so that it's
>>> reflected
 in
> the PR itself so that we can do merges directly from GitHub. Basic
>>> steps
 to
> make a change are described on the Wiki page [3].
> 
> Let me know if you have any questions.
> 
> [1] https://github.com/apache/ignite-3
> [2] https://ci.ignite.apache.org/project/ignite3
> [3]
> 
> 
 
>>> 
>> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0#ApacheIgnite3.0-DevelopmentProcess
> 
> -Val
> 
> On Wed, Nov 18, 2020 at 4:24 PM Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
> 
>> Thanks, guys. It looks like we are much closer to the consensus
>> now.
>>> I
>> totally on board with the plan, but I would also like to address
>> the
>> short-term needs. As I've already mentioned earlier, there are
>>> several
>> active IEPs, but we still don't have even a preliminary technical
 process
>> for working on these IEPs. I believe this might be frustrating for
>>> the
>> folks who would like to commit code.
>> 
>> The scope we agreed on is quite big, and it will surely take
 significant

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello!

You can override these values (module, suites) values when running a suite
on TC. Can you please run these ones which need to be changed individually
on TC, make sure they run without errors and contain all the needed tests,
and link to these runs in the ticket? Then I can modify the suites to fit
those.

I'm not sure that class shadowing will work as we want it to work, e.g., we
now have two IgniteCacheQuerySelfTestSuite6 with the same FQDN, I'm not
sure if maven/TC is going to pick both or just one.
Maybe they should go to a different package, e.g., testsuites/core for
every suite already present in indexing/spring/etc. Maybe you can rename
them just now? This will mean a lot less of work reconfiguring suites.
In TC configurations, suite names are simple class names, not FQ, so no
changes may be needed at all.

Regards,
-- 
Ilya Kasnacheev


пт, 27 нояб. 2020 г. в 13:03, Max Timonin :

> Hi, sorry for the misleading. I mean "adding ignite-core module *suites* to
> the TeamCity Queries* suite"
>
> On Fri, Nov 27, 2020 at 12:44 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com>
> wrote:
>
> > Hello!
> >
> > What do you mean by "adding ignite-core to suite"? ignite-core is a top
> > dependency and its tests are also included in all other modules' tests
> > classpath since it provides GridAbstractTest.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > пт, 27 нояб. 2020 г. в 01:24, Max Timonin :
> >
> > > Hi, Ilya!
> > >
> > > So, I've updated PR, fixed comments and removed Core* prefixes. MTCGA
> > shows
> > > no blockers, but it was 2 weeks ago, so I've started it again.
> > >
> > > If PR is OK then there are some suites that should be updated on TC.
> > Could
> > > you please tell me how we can do it?
> > >
> > > 1. Add ignite-cassandra-serializers suite:
> > >
> > >1. org.apache.ignite.tests.SerializerSuite
> > >
> > > 2. Add ignite-core to Queries* TC suite:
> > >
> > >1. org.apache.ignite.client.IgniteClientTestSuite
> > >2. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite
> > >3. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite2
> > >4. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite3
> > >5. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite4
> > >6. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite5
> > >7. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite6
> > >8. org.apache.ignite.suites.IgnitePdsWithIndexingCoreTestSuite
> > >9. org.apache.ignite.suites.IgniteCacheMvccSqlTestSuite
> > >
> > > 3. Remove ignite-indexing from TC suites:
> > >
> > >1. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite3
> > >2. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite4
> > >3. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite5
> > >
> > > 4. Add ignite-core to Spring* TC suite:
> > >
> > >1. org.apache.ignite.testsuites.IgniteSpringTestSuite
> > >
> > > 5. Add ignite-core suite (depends on uri-deployment module):
> > >
> > >1. org.apache.ignite.testsuites.IgniteUriDeploymentTestSuite
> > >
> > > 6. Add ignite-core suite to Zookeeper TC suite:
> > >
> > >1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
> > >
> > > 7. Remove ignite-zookeeper test suite:
> > >
> > >1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
> > >
> > > 8. Add ignite-ml test suites:
> > >
> > >1. org.apache.ignite.ml.math.distances.DistancesTestSuite
> > >2. org.apache.ignite.ml.naivebayes.NaiveBayesTestSuite
> > >
> > >
> > > On Wed, Nov 25, 2020 at 4:26 PM Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hello!
> > > >
> > > > Yes, we have such tests which depend on ignite-indexing or
> > ignite-spring.
> > > > They just need to be included in Spring* or Queries* test suite. Then
> > > they
> > > > will be executed on TC in the correct context. You can also run these
> > > tests
> > > > from IDEA by specifying other module as classpath. No need to move
> the
> > > > classes around.
> > > >
> > > > I will check the PR.
> > > >
> > > > Regards,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > ср, 25 нояб. 2020 г. в 00:22, Max Timonin :
> > > >
> > > > > Ilya, Anton, Ivan, hi!
> > > > >
> > > > > I fix some comments you leave in the PR. Also I checked some test
> > > suites
> > > > > and found that some tests are written in the core module but depend
> > on
> > > > the
> > > > > indexing module (or other modules). Some of such test classes
> contain
> > > > tests
> > > > > that are related to the core functionality, but some to indexing.
> I'm
> > > not
> > > > > sure if it is correct to move a whole suite with all tests from the
> > > > > indexing module to the core, as it will hide some core tests from
> the
> > > > core
> > > > > module.
> > > > >
> > > > > I believe that the correct solution is to investigate every such
> test
> > > and
> > > > > move it to the right module. But I think this work will take a lot
> 

[jira] [Created] (IGNITE-13769) Add examples of using the Ignite Spring Data thin client integration.

[Mikhail Petrov (Jira)]

Mikhail Petrov created IGNITE-13769:
---

 Summary:  Add examples of using the Ignite Spring Data thin client 
integration.
 Key: IGNITE-13769
 URL: https://issues.apache.org/jira/browse/IGNITE-13769
 Project: Ignite
  Issue Type: Improvement
Reporter: Mikhail Petrov
Assignee: Mikhail Petrov


It's needed to add examples of using the Ignite Spring Data thin client 
integration.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13768) Provide documentation on using the Spring Data integration with thin client.

[Mikhail Petrov (Jira)]

Mikhail Petrov created IGNITE-13768:
---

 Summary:  Provide documentation on using the Spring Data 
integration with thin client.
 Key: IGNITE-13768
 URL: https://issues.apache.org/jira/browse/IGNITE-13768
 Project: Ignite
  Issue Type: Improvement
Reporter: Mikhail Petrov
Assignee: Mikhail Petrov


It's needed to provide documentation on using the Spring Data integration with 
thin client.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi, sorry for the misleading. I mean "adding ignite-core module *suites* to
the TeamCity Queries* suite"

On Fri, Nov 27, 2020 at 12:44 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> What do you mean by "adding ignite-core to suite"? ignite-core is a top
> dependency and its tests are also included in all other modules' tests
> classpath since it provides GridAbstractTest.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 27 нояб. 2020 г. в 01:24, Max Timonin :
>
> > Hi, Ilya!
> >
> > So, I've updated PR, fixed comments and removed Core* prefixes. MTCGA
> shows
> > no blockers, but it was 2 weeks ago, so I've started it again.
> >
> > If PR is OK then there are some suites that should be updated on TC.
> Could
> > you please tell me how we can do it?
> >
> > 1. Add ignite-cassandra-serializers suite:
> >
> >1. org.apache.ignite.tests.SerializerSuite
> >
> > 2. Add ignite-core to Queries* TC suite:
> >
> >1. org.apache.ignite.client.IgniteClientTestSuite
> >2. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite
> >3. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite2
> >4. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite3
> >5. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite4
> >6. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite5
> >7. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite6
> >8. org.apache.ignite.suites.IgnitePdsWithIndexingCoreTestSuite
> >9. org.apache.ignite.suites.IgniteCacheMvccSqlTestSuite
> >
> > 3. Remove ignite-indexing from TC suites:
> >
> >1. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite3
> >2. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite4
> >3. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite5
> >
> > 4. Add ignite-core to Spring* TC suite:
> >
> >1. org.apache.ignite.testsuites.IgniteSpringTestSuite
> >
> > 5. Add ignite-core suite (depends on uri-deployment module):
> >
> >1. org.apache.ignite.testsuites.IgniteUriDeploymentTestSuite
> >
> > 6. Add ignite-core suite to Zookeeper TC suite:
> >
> >1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
> >
> > 7. Remove ignite-zookeeper test suite:
> >
> >1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
> >
> > 8. Add ignite-ml test suites:
> >
> >1. org.apache.ignite.ml.math.distances.DistancesTestSuite
> >2. org.apache.ignite.ml.naivebayes.NaiveBayesTestSuite
> >
> >
> > On Wed, Nov 25, 2020 at 4:26 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com
> > >
> > wrote:
> >
> > > Hello!
> > >
> > > Yes, we have such tests which depend on ignite-indexing or
> ignite-spring.
> > > They just need to be included in Spring* or Queries* test suite. Then
> > they
> > > will be executed on TC in the correct context. You can also run these
> > tests
> > > from IDEA by specifying other module as classpath. No need to move the
> > > classes around.
> > >
> > > I will check the PR.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > ср, 25 нояб. 2020 г. в 00:22, Max Timonin :
> > >
> > > > Ilya, Anton, Ivan, hi!
> > > >
> > > > I fix some comments you leave in the PR. Also I checked some test
> > suites
> > > > and found that some tests are written in the core module but depend
> on
> > > the
> > > > indexing module (or other modules). Some of such test classes contain
> > > tests
> > > > that are related to the core functionality, but some to indexing. I'm
> > not
> > > > sure if it is correct to move a whole suite with all tests from the
> > > > indexing module to the core, as it will hide some core tests from the
> > > core
> > > > module.
> > > >
> > > > I believe that the correct solution is to investigate every such test
> > and
> > > > move it to the right module. But I think this work will take a lot of
> > > time
> > > > and should be performed in a separate ticket, I will do it in the
> > > > background.
> > > >
> > > > I think currently we should proceed with a way I introduced in PR:
> > > > 1. Create fake suites for all such tests (written in core, suited in
> > > other
> > > > modules: indexing/spring/zookeeper/etc) in the core module. I named
> > such
> > > > suites with prefix Core*.
> > > > 2. Replace tests in modules with links to fake suites.
> > > > 3. Create an umbrella Jira ticket to discover every fake suite and
> > > replace
> > > > it with a new one in the right module.
> > > > 4. Merge this PR for introducing a new travis check to avoid losing
> > > > new tests.
> > > >
> > > > WDYT?
> > > >
> > > > List of such mixed suites:
> > > >
> > > > 1. suite IgniteBinaryCacheQueryTestSuite
> > > >
> > > > test GridCacheQueryIndexingDisabledSelfTest
> > > > test IgniteCacheBinaryObjectsScanSelfTest
> > > > test IgniteCacheBinaryObjectsScanWithEventsSelfTest)
> > > >
> > > >
> > > > 2. suite IgniteCacheQuerySelfTestSuite3
> > > >
> > > > test GridCacheContinuousQueryNodesFilteringTest
> > > >
> > > >
> > > > 3. suite IgniteCacheQuerySelfTestSuite5
> > > 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello!

What do you mean by "adding ignite-core to suite"? ignite-core is a top
dependency and its tests are also included in all other modules' tests
classpath since it provides GridAbstractTest.

Regards,
-- 
Ilya Kasnacheev


пт, 27 нояб. 2020 г. в 01:24, Max Timonin :

> Hi, Ilya!
>
> So, I've updated PR, fixed comments and removed Core* prefixes. MTCGA shows
> no blockers, but it was 2 weeks ago, so I've started it again.
>
> If PR is OK then there are some suites that should be updated on TC. Could
> you please tell me how we can do it?
>
> 1. Add ignite-cassandra-serializers suite:
>
>1. org.apache.ignite.tests.SerializerSuite
>
> 2. Add ignite-core to Queries* TC suite:
>
>1. org.apache.ignite.client.IgniteClientTestSuite
>2. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite
>3. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite2
>4. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite3
>5. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite4
>6. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite5
>7. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite6
>8. org.apache.ignite.suites.IgnitePdsWithIndexingCoreTestSuite
>9. org.apache.ignite.suites.IgniteCacheMvccSqlTestSuite
>
> 3. Remove ignite-indexing from TC suites:
>
>1. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite3
>2. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite4
>3. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite5
>
> 4. Add ignite-core to Spring* TC suite:
>
>1. org.apache.ignite.testsuites.IgniteSpringTestSuite
>
> 5. Add ignite-core suite (depends on uri-deployment module):
>
>1. org.apache.ignite.testsuites.IgniteUriDeploymentTestSuite
>
> 6. Add ignite-core suite to Zookeeper TC suite:
>
>1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
>
> 7. Remove ignite-zookeeper test suite:
>
>1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
>
> 8. Add ignite-ml test suites:
>
>1. org.apache.ignite.ml.math.distances.DistancesTestSuite
>2. org.apache.ignite.ml.naivebayes.NaiveBayesTestSuite
>
>
> On Wed, Nov 25, 2020 at 4:26 PM Ilya Kasnacheev  >
> wrote:
>
> > Hello!
> >
> > Yes, we have such tests which depend on ignite-indexing or ignite-spring.
> > They just need to be included in Spring* or Queries* test suite. Then
> they
> > will be executed on TC in the correct context. You can also run these
> tests
> > from IDEA by specifying other module as classpath. No need to move the
> > classes around.
> >
> > I will check the PR.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 25 нояб. 2020 г. в 00:22, Max Timonin :
> >
> > > Ilya, Anton, Ivan, hi!
> > >
> > > I fix some comments you leave in the PR. Also I checked some test
> suites
> > > and found that some tests are written in the core module but depend on
> > the
> > > indexing module (or other modules). Some of such test classes contain
> > tests
> > > that are related to the core functionality, but some to indexing. I'm
> not
> > > sure if it is correct to move a whole suite with all tests from the
> > > indexing module to the core, as it will hide some core tests from the
> > core
> > > module.
> > >
> > > I believe that the correct solution is to investigate every such test
> and
> > > move it to the right module. But I think this work will take a lot of
> > time
> > > and should be performed in a separate ticket, I will do it in the
> > > background.
> > >
> > > I think currently we should proceed with a way I introduced in PR:
> > > 1. Create fake suites for all such tests (written in core, suited in
> > other
> > > modules: indexing/spring/zookeeper/etc) in the core module. I named
> such
> > > suites with prefix Core*.
> > > 2. Replace tests in modules with links to fake suites.
> > > 3. Create an umbrella Jira ticket to discover every fake suite and
> > replace
> > > it with a new one in the right module.
> > > 4. Merge this PR for introducing a new travis check to avoid losing
> > > new tests.
> > >
> > > WDYT?
> > >
> > > List of such mixed suites:
> > >
> > > 1. suite IgniteBinaryCacheQueryTestSuite
> > >
> > > test GridCacheQueryIndexingDisabledSelfTest
> > > test IgniteCacheBinaryObjectsScanSelfTest
> > > test IgniteCacheBinaryObjectsScanWithEventsSelfTest)
> > >
> > >
> > > 2. suite IgniteCacheQuerySelfTestSuite3
> > >
> > > test GridCacheContinuousQueryNodesFilteringTest
> > >
> > >
> > > 3. suite IgniteCacheQuerySelfTestSuite5
> > >
> > > test ContinuousQueryRemoteFilterMissingInClassPathSelfTest
> > >
> > > 4. suite IgniteCacheQuerySelfTestSuite6
> > >
> > > test CacheContinuousQueryOperationP2PTest
> > >
> > > test CacheContinuousQueryFilterDeploymentFailedTest
> > >
> > >
> > > 5. all tests in suite IgnitePdsWithIndexingCoreTestSuite
> > >
> > >
> > > 6. and some others.
> > >
> > > On Wed, Nov 18, 2020 at 12:38 PM Max Timonin 
> > > wrote:
> > >
> > > > Hi Ilya! Thank you for up the topic. I will 

[jira] [Created] (IGNITE-13767) Remove Python PHP and Node.js thin clients from main repository

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13767:


 Summary: Remove Python PHP and Node.js thin clients from main 
repository
 Key: IGNITE-13767
 URL: https://issues.apache.org/jira/browse/IGNITE-13767
 Project: Ignite
  Issue Type: Improvement
  Components: thin client
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.10


Need to remove the following directories, as we now have separate repos for 
Python, Node.js and PHP thin clients:

modules/platforms/python

modules/platforms/nodejs

modules/platforms/php

 

Also, need to check all the places in code where those directories are 
referenced.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Issue with custom security plugin and thin clients

[Denis Garus]

Hello!


If I understood your problem correctly, you need to make a thin client's
security context allowed on a remote node.

When a security plugin does authenticate a thin client, it should spread
the thin client's security context on the cluster.

How a security context will be transmitted to a remote node is up to the
plugin's developers.

Also, you have to implement the GridSecurityProcessor.securityContext(UUID
subjId) method,

the way this method is used in Ignite can see in the task description [1].




   1. https://issues.apache.org/jira/browse/IGNITE-12759


чт, 26 нояб. 2020 г. в 10:01, Vishwas Bm :

> Hi,
>
> I was facing an issue with a custom security plugin and thin remote client.
> I am using Ignite 2.9.0 version and I am hitting below issue
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-41%3A+Security+Context+of+thin+client+on+remote+nodes
>
>
> I had asked the question in the user listing but unfortunately I did not
> get any reply.
> So I am posting this question here:
>
>
> http://apache-ignite-users.70518.x6.nabble.com/Query-on-implementing-GridSecurityProcessor-td34672.html
>
>
> *Thanks & Regards,*
>
> *Vishwas *
>