[DISCUSSION] Documentation of thin clients (python, php, nodejs)

2021-04-22 Thread Ivan Daschinsky
Igniters, there are some questions regarding the documentation state
of thin clients.

Recently, we have released pyignite 0.4.0. Traditionally,
documentation for python thin client is autogenerated from source and
contains in the same git repository, as the client itself.
Documentation is autogenerated and hosted in familiar for python
developers manner -- on readthedocs.io (Namely,
https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/)

I suppose the same statement is true for other separately developed
clients (nodejs, php).

So I'd like to discuss a current documentation approach for thin clients.
1. I strongly believe that the main documentation site for at least
python thin client should be readthedocs.io.
2. Documentation should be maintained in the same repository as the
thin client itself.
3. As the main documentation's version is tightly coupled with ignite
release cycle, it is by default outdated and doesn't resemble the
latest version of thin client.

I suggest just remove all documentation from the main docs except
simple installation instruction (i.e. pip install pyignite) and link
to readthedocs.io

Regards, Ivan Daschinsky


Re: Review for IGNITE-13112 The current security context should be obtained using the IgniteSecurity interface only.

2021-04-22 Thread Denis Garus
Maksim, ok.

Let me know if you have any questions.

ср, 21 апр. 2021 г. в 17:51, Maksim Stepachev :

> Please wait. I'm watching your review.
>
> вт, 6 апр. 2021 г. в 20:14, Denis Garus :
>
> > Hello, Igniters!
> >
> > I've raised the PR [1] for the issue [2].
> > Could somebody review it?
> >
> > Suggested implementation
> >
> > If Ignite Security (IS) is enabled, then executors, accessed through the
> > PoolProcessor,
> > are wrapped to a security-aware implementation. Security-aware
> > implementation sets proper
> > security context for tasks that the executor performs.
> >
> > The field subject id was deleted from communication requests for cache
> and
> > compute operations;
> > a remote node gets the subject id that initiates the ignite operation
> from
> > GridIoSecurityAwareMessage.
> > IgniteSecurity uses this id to set a proper security context during the
> > execution of the request.
> >
> > Remove GridTaskThreadContextKey#TC_SUBJ_ID,
> > GridCacheContext#subjectIdPerCall;
> > a consumer has to obtain a current security subject id through
> > IgniteSecurity
> > or the set of SecurityUtils methods.
> >
> > For all events that include the subject id field, are set the following
> > rule.
> > If IS is enabled, this field must contain a subject id that initiates
> > an ignite operation, otherwise null.
> >
> > Implement SecurityAwareCustomMessageWrapper for discovery requests that
> act
> > as
> > GridIoSecurityAwareMessage for communication requests. It allows setting
> > proper
> > context during the discovery message execution.
> >
> > Implement SecurityAwareGridRestCommandHandler to allow GridRestProcessor
> > to execute all client requests with the proper security context.
> >
> > 1. https://github.com/apache/ignite/pull/8038
> > 2. https://issues.apache.org/jira/browse/IGNITE-13112
> >
>


Re: Stop sending IGNITE Created e-mails to dev@

2021-04-22 Thread Dmitriy Pavlov
+1 for issues
-1 for TC bot. For the latter I siggest improving analysis to avoid odd
emails and leave only relevant

чт, 22 апр. 2021 г., 18:16 Ivan Pavlukhin :

> > All issues notifications are also sent to iss...@ignite.apache.org so
> one can subscribe to this list in order to track the created tickets.
>
> Does not sound as useful advice. Issues list [1] looks like real
> scrapyard, I doubt that it can be usable for anyone in current flavor.
> Can we send only "Created" notifications there?
>
> [1] https://lists.apache.org/list.html?iss...@ignite.apache.org
>
> 2021-04-21 18:30 GMT+03:00, Ilya Kasnacheev :
> > Hello!
> >
> > INFRA ticket created: https://issues.apache.org/jira/browse/INFRA-21762
> >
> > I have asked to keep sending the created issue notifications for
> > "Contributors 1" role, which is empty at present. So if you wish to keep
> > getting those e-mails, please add yourself to this role or tell me to do
> so
> > for you.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 21 апр. 2021 г. в 17:59, Alexey Goncharuk <
> alexey.goncha...@gmail.com>:
> >
> >> I support the idea. All issues notifications are also sent to
> >> iss...@ignite.apache.org so one can subscribe to this list in order to
> >> track the created tickets. The notifications trash the devlist archive
> UI
> >> and make it extremely difficult to navigate.
> >>
> >> вт, 20 апр. 2021 г. в 18:35, Ilya Kasnacheev  >:
> >>
> >> > Hello, Maxim!
> >> >
> >> > You are free to revert any commit which has led to any new stable test
> >> > failure, or new flaky test that was non-flaky before.
> >> >
> >> > Just revert the change and reopen the ticket.
> >> >
> >> > The problem here is that it's very hard to detect on the spot, most of
> >> > MTCGA e-mails are false positives and even if they are not, it is not
> >> > relevant for most of developers.
> >> >
> >> > WDYT? I'm also still waiting for more input.
> >> >
> >> > Regards,
> >> > --
> >> > Ilya Kasnacheev
> >> >
> >> >
> >> > ср, 14 апр. 2021 г. в 21:26, Maxim Muzafarov :
> >> >
> >> > > +1 for new JIRA issues
> >> > > -1 for MTCGA notifications
> >> > >
> >> > > Why we should hide errors from the dev-list? Who should take care of
> >> > > issues reported by MTCGA.Bot in this case?
> >> > > We must apply stricter rules for such issues: a commit leading to an
> >> > > error must be reverted.
> >> > >
> >> > > On Wed, 14 Apr 2021 at 20:00, Denis Mekhanikov
> >> > > 
> >> > > wrote:
> >> > > >
> >> > > > Huge +1 to this.
> >> > > >
> >> > > > I've already brought up this topic in the past:
> >> > > >
> >> > >
> >> >
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/Bots-on-dev-list-td34406.html
> >> > > > I hope some day newcomers won't need to set up their email filters
> >> when
> >> > > > they come to the developers list.
> >> > > >
> >> > > > Denis
> >> > > >
> >> > > > ср, 14 апр. 2021 г. в 18:07, Atri Sharma :
> >> > > >
> >> > > > > +1 to move issues to the issues list.
> >> > > > >
> >> > > > > For MTCGA, maybe build@?
> >> > > > >
> >> > > > > On Wed, Apr 14, 2021 at 8:35 PM Ilya Kasnacheev
> >> > > > > 
> >> > > wrote:
> >> > > > > >
> >> > > > > > Hello!
> >> > > > > >
> >> > > > > > We have a discussion on how to ensure best engagement in dev@
> >> > list,
> >> > > and
> >> > > > > it
> >> > > > > > seems that Issue Created emails from IGNITE project consume a
> >> > > > > > lot
> >> > of
> >> > > > > screen
> >> > > > > > space, it's hard to spot genuine discussions in
> >> > > > > > https://lists.apache.org/list.html?dev@ignite.apache.org for
> >> > > example.
> >> > > > > >
> >> > > > > > We already have issues@ mailing list. I propose that we stop
> >> > > sending any
> >> > > > > > JIRA emails to dev@. If anyone wishes to get just Created
> >> emails,
> >> > > they
> >> > > > > can
> >> > > > > > subscribe to these messages in their JIRA account settings. I
> >> > imagine
> >> > > > > most
> >> > > > > > of you already filter these messages out, so you may need to
> >> adjust
> >> > > your
> >> > > > > > filters slightly.
> >> > > > > >
> >> > > > > > A distant second is MTCGA messages, which are also
> >> > > > > > autogenerated
> >> > and
> >> > > not
> >> > > > > > informative for most readers of the channel, since they are at
> >> best
> >> > > > > > targeted at a single committer and at worst flaky.
> >> > > > > >
> >> > > > > > Where could we move those? What is your opinion here, on both
> >> > issues?
> >> > > > > >
> >> > > > > > Regards,
> >> > > > >
> >> > > > > --
> >> > > > > Regards,
> >> > > > >
> >> > > > > Atri
> >> > > > > Apache Concerted
> >> > > > >
> >> > >
> >> >
> >>
> >
>
>
> --
>
> Best regards,
> Ivan Pavlukhin
>


Re: Stop sending IGNITE Created e-mails to dev@

2021-04-22 Thread Ivan Pavlukhin
> All issues notifications are also sent to iss...@ignite.apache.org so one can 
> subscribe to this list in order to track the created tickets.

Does not sound as useful advice. Issues list [1] looks like real
scrapyard, I doubt that it can be usable for anyone in current flavor.
Can we send only "Created" notifications there?

[1] https://lists.apache.org/list.html?iss...@ignite.apache.org

2021-04-21 18:30 GMT+03:00, Ilya Kasnacheev :
> Hello!
>
> INFRA ticket created: https://issues.apache.org/jira/browse/INFRA-21762
>
> I have asked to keep sending the created issue notifications for
> "Contributors 1" role, which is empty at present. So if you wish to keep
> getting those e-mails, please add yourself to this role or tell me to do so
> for you.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 21 апр. 2021 г. в 17:59, Alexey Goncharuk :
>
>> I support the idea. All issues notifications are also sent to
>> iss...@ignite.apache.org so one can subscribe to this list in order to
>> track the created tickets. The notifications trash the devlist archive UI
>> and make it extremely difficult to navigate.
>>
>> вт, 20 апр. 2021 г. в 18:35, Ilya Kasnacheev :
>>
>> > Hello, Maxim!
>> >
>> > You are free to revert any commit which has led to any new stable test
>> > failure, or new flaky test that was non-flaky before.
>> >
>> > Just revert the change and reopen the ticket.
>> >
>> > The problem here is that it's very hard to detect on the spot, most of
>> > MTCGA e-mails are false positives and even if they are not, it is not
>> > relevant for most of developers.
>> >
>> > WDYT? I'm also still waiting for more input.
>> >
>> > Regards,
>> > --
>> > Ilya Kasnacheev
>> >
>> >
>> > ср, 14 апр. 2021 г. в 21:26, Maxim Muzafarov :
>> >
>> > > +1 for new JIRA issues
>> > > -1 for MTCGA notifications
>> > >
>> > > Why we should hide errors from the dev-list? Who should take care of
>> > > issues reported by MTCGA.Bot in this case?
>> > > We must apply stricter rules for such issues: a commit leading to an
>> > > error must be reverted.
>> > >
>> > > On Wed, 14 Apr 2021 at 20:00, Denis Mekhanikov
>> > > 
>> > > wrote:
>> > > >
>> > > > Huge +1 to this.
>> > > >
>> > > > I've already brought up this topic in the past:
>> > > >
>> > >
>> >
>> http://apache-ignite-developers.2346864.n4.nabble.com/Bots-on-dev-list-td34406.html
>> > > > I hope some day newcomers won't need to set up their email filters
>> when
>> > > > they come to the developers list.
>> > > >
>> > > > Denis
>> > > >
>> > > > ср, 14 апр. 2021 г. в 18:07, Atri Sharma :
>> > > >
>> > > > > +1 to move issues to the issues list.
>> > > > >
>> > > > > For MTCGA, maybe build@?
>> > > > >
>> > > > > On Wed, Apr 14, 2021 at 8:35 PM Ilya Kasnacheev
>> > > > > 
>> > > wrote:
>> > > > > >
>> > > > > > Hello!
>> > > > > >
>> > > > > > We have a discussion on how to ensure best engagement in dev@
>> > list,
>> > > and
>> > > > > it
>> > > > > > seems that Issue Created emails from IGNITE project consume a
>> > > > > > lot
>> > of
>> > > > > screen
>> > > > > > space, it's hard to spot genuine discussions in
>> > > > > > https://lists.apache.org/list.html?dev@ignite.apache.org for
>> > > example.
>> > > > > >
>> > > > > > We already have issues@ mailing list. I propose that we stop
>> > > sending any
>> > > > > > JIRA emails to dev@. If anyone wishes to get just Created
>> emails,
>> > > they
>> > > > > can
>> > > > > > subscribe to these messages in their JIRA account settings. I
>> > imagine
>> > > > > most
>> > > > > > of you already filter these messages out, so you may need to
>> adjust
>> > > your
>> > > > > > filters slightly.
>> > > > > >
>> > > > > > A distant second is MTCGA messages, which are also
>> > > > > > autogenerated
>> > and
>> > > not
>> > > > > > informative for most readers of the channel, since they are at
>> best
>> > > > > > targeted at a single committer and at worst flaky.
>> > > > > >
>> > > > > > Where could we move those? What is your opinion here, on both
>> > issues?
>> > > > > >
>> > > > > > Regards,
>> > > > >
>> > > > > --
>> > > > > Regards,
>> > > > >
>> > > > > Atri
>> > > > > Apache Concerted
>> > > > >
>> > >
>> >
>>
>


-- 

Best regards,
Ivan Pavlukhin


Re: Empty test results for Security Test Suite ran against master branch on Team City

2021-04-22 Thread Dmitry Pavlov
Hi,

I guess it 's not correct since master tests count is 0. No reason to run a 
suite without tests.

Folks, why it could be the case? I've checked SUITE_NAME and suite in the code 
and it seems to be mached.

Sincerely,
Dmitriy Pavlov

On 2021/04/22 14:29:37, Shishkov Ilya  wrote: 
> Hello, Igniters!
> 
> As I see, there are no test results for Security Test Suite run against
> 'master' branch [1], for 'ignite-2.10' Security tests results are present
> as expected [2].
> Is this situation correct?
> 
> 1.
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5977426?buildTab=log=1878=1579
> 2.
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5976111?buildTab=log=67783=1599.68439.68448
> 


Empty test results for Security Test Suite ran against master branch on Team City

2021-04-22 Thread Shishkov Ilya
Hello, Igniters!

As I see, there are no test results for Security Test Suite run against
'master' branch [1], for 'ignite-2.10' Security tests results are present
as expected [2].
Is this situation correct?

1.
https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5977426?buildTab=log=1878=1579
2.
https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5976111?buildTab=log=67783=1599.68439.68448


[jira] [Created] (IGNITE-14623) Calcite. Sort out test scripts at: sql/aggregate/aggregates/*

2021-04-22 Thread Taras Ledkov (Jira)
Taras Ledkov created IGNITE-14623:
-

 Summary: Calcite. Sort out test scripts at: 
sql/aggregate/aggregates/*
 Key: IGNITE-14623
 URL: https://issues.apache.org/jira/browse/IGNITE-14623
 Project: Ignite
  Issue Type: Task
Reporter: Taras Ledkov
Assignee: Taras Ledkov


Calcite. Sort out test scripts at: {{sql/aggregate/aggregates/*}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: IP Filtering in IPFinders

2021-04-22 Thread Ilya Kasnacheev
Hello!

I'm still not fully convinced, but Val's approach sounds rational to me.

Regards,
-- 
Ilya Kasnacheev


чт, 22 апр. 2021 г. в 12:45, Atri Sharma :

> Hello!
>
> I actually saw the shared container scenario being tried by somebody
> who wanted an external script to monitor all IPs being used by his
> clusters and hence thought of this idea. Another thing that came in
> was the Firewall blocking a few IP addresses, hence the idea.
>
> I feel that the footprint of this change is small, and can be useful
> for esoteric use cases too without really interfering in any existing
> code path. Val's suggestion seems the right way to go since it gives
> the functionality without much change.
>
> Thoughts?
>
> On Thu, Apr 22, 2021 at 2:47 PM Ilya Kasnacheev
>  wrote:
> >
> > Hello!
> >
> > AFAIK, a S3 container, Azure blob container, etc, is a relatively
> > lightweight entity, similar to a table in an SQL database. Why would
> > different clusters need to share the same discovery storage container?
> > When I tested Azure IP finder, it created several blob containers for me
> on
> > demand, based on the parameter passed to IP finder. If I wanted to have
> > more than one cluster it should have been seamless already.
> >
> > I can theoretically see how address filtering may be useful to remove
> > public / private addresses or Docker gateway address, but it is usually
> > handled by setting localHost setting, although requiring tuning it for
> each
> > instance individually. Overall benefit seems to small.
> >
> > This is why I am asking, do you have any specific scenario in mind where
> > this feature is an enabler? How did you arrive at the conclusion to go
> > forward with it?
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > чт, 22 апр. 2021 г. в 07:51, Atri Sharma :
> >
> > > Hi Val,
> > >
> > > Consider a scenario where multiple Ignite clusters are running and for
> > > operational ease (and also compliance, in some cases, e.g. to make
> > > auditing easier), people can configure cloud based IP finders to share
> > > the same container (blob container in Azure, S3 container in AWS etc).
> > >
> > > In such a case, IPs for all clusters will be in the same container.
> > > IPFinders of both the clusters will read the entire list. In this
> > > case, address filtering will help ignore the irrelevant IP addresses.
> > >
> > > Thank you for pointing me to the alternate direction. Let me research
> > > that and revert.
> > >
> > > Atri
> > >
> > > On Wed, Apr 21, 2021 at 10:46 PM Valentin Kulichenko
> > >  wrote:
> > > >
> > > > Hi Atri,
> > > >
> > > > Can you describe the scenario in a little more detail? What exactly
> do
> > > you
> > > > mean by a container shared by multiple clusters? What are the
> > > consequences
> > > > of this? How does the proposed solution solve the problem?
> > > >
> > > > Also, I would suggest revisiting the design - I'm not sure such
> filtering
> > > > should be done on the IP finder level. Why not do this on the SPI
> level
> > > > instead? I would simply add something like "addressFilter" to the
> > > > TcpDiscoverySpi. The filter can be a generic IgnitePredicate, so you
> will
> > > > be able to provide any implementations, including regex or anything
> else.
> > > >
> > > > -Val
> > > >
> > > > On Wed, Apr 21, 2021 at 9:04 AM Atri Sharma  wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > When a container is shared by multiple clusters, then this can be
> > > useful
> > > > > for filtering IPs.
> > > > >
> > > > > Also, things like VPC based barriers can be circumvented using this
> > > > > technique.
> > > > >
> > > > > On Wed, 21 Apr 2021, 15:49 Ilya Kasnacheev, <
> ilya.kasnach...@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hello!
> > > > > >
> > > > > > What are the expected use cases for this feature? Can you please
> > > > > elaborate?
> > > > > >
> > > > > > Thanks,
> > > > > > --
> > > > > > Ilya Kasnacheev
> > > > > >
> > > > > >
> > > > > > ср, 21 апр. 2021 г. в 08:23, Atri Sharma :
> > > > > >
> > > > > > > Hi All,
> > > > > > >
> > > > > > > I have opened the following JIRA for the said topic:
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/IGNITE-14606
> > > > > > >
> > > > > > > The concept is to filter IPs based on a pattern or a blocklist
> in
> > > > > > > IPFinders while consuming IPs. This is more pertinent for cloud
> > > based
> > > > > > > IPFinders since they can have shared containers.
> > > > > > >
> > > > > > > For the moment, I have implemented regex based filtering:
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/IGNITE-14607
> > > > > > >
> > > > > > > for Azure Blob Storage IP Finder. Over time, we can extend the
> > > same to
> > > > > > > other IP finders.
> > > > > > >
> > > > > > > Please see the PR:
> > > > > > >
> > > > > > > https://github.com/apache/ignite/pull/9024
> > > > > > >
> > > > > > > Regards,
> > > > > > >
> > > > > > > Atri
> > > > > > >
> > > > > > > --
> > > > > > > 

Re: IP Filtering in IPFinders

2021-04-22 Thread Atri Sharma
Hello!

I actually saw the shared container scenario being tried by somebody
who wanted an external script to monitor all IPs being used by his
clusters and hence thought of this idea. Another thing that came in
was the Firewall blocking a few IP addresses, hence the idea.

I feel that the footprint of this change is small, and can be useful
for esoteric use cases too without really interfering in any existing
code path. Val's suggestion seems the right way to go since it gives
the functionality without much change.

Thoughts?

On Thu, Apr 22, 2021 at 2:47 PM Ilya Kasnacheev
 wrote:
>
> Hello!
>
> AFAIK, a S3 container, Azure blob container, etc, is a relatively
> lightweight entity, similar to a table in an SQL database. Why would
> different clusters need to share the same discovery storage container?
> When I tested Azure IP finder, it created several blob containers for me on
> demand, based on the parameter passed to IP finder. If I wanted to have
> more than one cluster it should have been seamless already.
>
> I can theoretically see how address filtering may be useful to remove
> public / private addresses or Docker gateway address, but it is usually
> handled by setting localHost setting, although requiring tuning it for each
> instance individually. Overall benefit seems to small.
>
> This is why I am asking, do you have any specific scenario in mind where
> this feature is an enabler? How did you arrive at the conclusion to go
> forward with it?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 22 апр. 2021 г. в 07:51, Atri Sharma :
>
> > Hi Val,
> >
> > Consider a scenario where multiple Ignite clusters are running and for
> > operational ease (and also compliance, in some cases, e.g. to make
> > auditing easier), people can configure cloud based IP finders to share
> > the same container (blob container in Azure, S3 container in AWS etc).
> >
> > In such a case, IPs for all clusters will be in the same container.
> > IPFinders of both the clusters will read the entire list. In this
> > case, address filtering will help ignore the irrelevant IP addresses.
> >
> > Thank you for pointing me to the alternate direction. Let me research
> > that and revert.
> >
> > Atri
> >
> > On Wed, Apr 21, 2021 at 10:46 PM Valentin Kulichenko
> >  wrote:
> > >
> > > Hi Atri,
> > >
> > > Can you describe the scenario in a little more detail? What exactly do
> > you
> > > mean by a container shared by multiple clusters? What are the
> > consequences
> > > of this? How does the proposed solution solve the problem?
> > >
> > > Also, I would suggest revisiting the design - I'm not sure such filtering
> > > should be done on the IP finder level. Why not do this on the SPI level
> > > instead? I would simply add something like "addressFilter" to the
> > > TcpDiscoverySpi. The filter can be a generic IgnitePredicate, so you will
> > > be able to provide any implementations, including regex or anything else.
> > >
> > > -Val
> > >
> > > On Wed, Apr 21, 2021 at 9:04 AM Atri Sharma  wrote:
> > >
> > > > Hi,
> > > >
> > > > When a container is shared by multiple clusters, then this can be
> > useful
> > > > for filtering IPs.
> > > >
> > > > Also, things like VPC based barriers can be circumvented using this
> > > > technique.
> > > >
> > > > On Wed, 21 Apr 2021, 15:49 Ilya Kasnacheev,  > >
> > > > wrote:
> > > >
> > > > > Hello!
> > > > >
> > > > > What are the expected use cases for this feature? Can you please
> > > > elaborate?
> > > > >
> > > > > Thanks,
> > > > > --
> > > > > Ilya Kasnacheev
> > > > >
> > > > >
> > > > > ср, 21 апр. 2021 г. в 08:23, Atri Sharma :
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > I have opened the following JIRA for the said topic:
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/IGNITE-14606
> > > > > >
> > > > > > The concept is to filter IPs based on a pattern or a blocklist in
> > > > > > IPFinders while consuming IPs. This is more pertinent for cloud
> > based
> > > > > > IPFinders since they can have shared containers.
> > > > > >
> > > > > > For the moment, I have implemented regex based filtering:
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/IGNITE-14607
> > > > > >
> > > > > > for Azure Blob Storage IP Finder. Over time, we can extend the
> > same to
> > > > > > other IP finders.
> > > > > >
> > > > > > Please see the PR:
> > > > > >
> > > > > > https://github.com/apache/ignite/pull/9024
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Atri
> > > > > >
> > > > > > --
> > > > > > Regards,
> > > > > >
> > > > > > Atri
> > > > > > Apache Concerted
> > > > > >
> > > > >
> > > >
> >
> > --
> > Regards,
> >
> > Atri
> > Apache Concerted
> >

-- 
Regards,

Atri
Apache Concerted


Re: IP Filtering in IPFinders

2021-04-22 Thread Ilya Kasnacheev
Hello!

AFAIK, a S3 container, Azure blob container, etc, is a relatively
lightweight entity, similar to a table in an SQL database. Why would
different clusters need to share the same discovery storage container?
When I tested Azure IP finder, it created several blob containers for me on
demand, based on the parameter passed to IP finder. If I wanted to have
more than one cluster it should have been seamless already.

I can theoretically see how address filtering may be useful to remove
public / private addresses or Docker gateway address, but it is usually
handled by setting localHost setting, although requiring tuning it for each
instance individually. Overall benefit seems to small.

This is why I am asking, do you have any specific scenario in mind where
this feature is an enabler? How did you arrive at the conclusion to go
forward with it?

Regards,
-- 
Ilya Kasnacheev


чт, 22 апр. 2021 г. в 07:51, Atri Sharma :

> Hi Val,
>
> Consider a scenario where multiple Ignite clusters are running and for
> operational ease (and also compliance, in some cases, e.g. to make
> auditing easier), people can configure cloud based IP finders to share
> the same container (blob container in Azure, S3 container in AWS etc).
>
> In such a case, IPs for all clusters will be in the same container.
> IPFinders of both the clusters will read the entire list. In this
> case, address filtering will help ignore the irrelevant IP addresses.
>
> Thank you for pointing me to the alternate direction. Let me research
> that and revert.
>
> Atri
>
> On Wed, Apr 21, 2021 at 10:46 PM Valentin Kulichenko
>  wrote:
> >
> > Hi Atri,
> >
> > Can you describe the scenario in a little more detail? What exactly do
> you
> > mean by a container shared by multiple clusters? What are the
> consequences
> > of this? How does the proposed solution solve the problem?
> >
> > Also, I would suggest revisiting the design - I'm not sure such filtering
> > should be done on the IP finder level. Why not do this on the SPI level
> > instead? I would simply add something like "addressFilter" to the
> > TcpDiscoverySpi. The filter can be a generic IgnitePredicate, so you will
> > be able to provide any implementations, including regex or anything else.
> >
> > -Val
> >
> > On Wed, Apr 21, 2021 at 9:04 AM Atri Sharma  wrote:
> >
> > > Hi,
> > >
> > > When a container is shared by multiple clusters, then this can be
> useful
> > > for filtering IPs.
> > >
> > > Also, things like VPC based barriers can be circumvented using this
> > > technique.
> > >
> > > On Wed, 21 Apr 2021, 15:49 Ilya Kasnacheev,  >
> > > wrote:
> > >
> > > > Hello!
> > > >
> > > > What are the expected use cases for this feature? Can you please
> > > elaborate?
> > > >
> > > > Thanks,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > ср, 21 апр. 2021 г. в 08:23, Atri Sharma :
> > > >
> > > > > Hi All,
> > > > >
> > > > > I have opened the following JIRA for the said topic:
> > > > >
> > > > > https://issues.apache.org/jira/browse/IGNITE-14606
> > > > >
> > > > > The concept is to filter IPs based on a pattern or a blocklist in
> > > > > IPFinders while consuming IPs. This is more pertinent for cloud
> based
> > > > > IPFinders since they can have shared containers.
> > > > >
> > > > > For the moment, I have implemented regex based filtering:
> > > > >
> > > > > https://issues.apache.org/jira/browse/IGNITE-14607
> > > > >
> > > > > for Azure Blob Storage IP Finder. Over time, we can extend the
> same to
> > > > > other IP finders.
> > > > >
> > > > > Please see the PR:
> > > > >
> > > > > https://github.com/apache/ignite/pull/9024
> > > > >
> > > > > Regards,
> > > > >
> > > > > Atri
> > > > >
> > > > > --
> > > > > Regards,
> > > > >
> > > > > Atri
> > > > > Apache Concerted
> > > > >
> > > >
> > >
>
> --
> Regards,
>
> Atri
> Apache Concerted
>


[jira] [Created] (IGNITE-14622) Snapshot test simplification

2021-04-22 Thread Anton Vinogradov (Jira)
Anton Vinogradov created IGNITE-14622:
-

 Summary: Snapshot test simplification
 Key: IGNITE-14622
 URL: https://issues.apache.org/jira/browse/IGNITE-14622
 Project: Ignite
  Issue Type: Sub-task
Reporter: Anton Vinogradov
Assignee: Anton Vinogradov


Just get rid of useless stuff %) 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: Apache Ignite 3 / RPM

2021-04-22 Thread Petr Ivanov
Unfortunately, no.


However, I think I'll update the package to better resemble with current binary 
archive prepared for alpha2 and update release task.
Due to Bintray closure, I think we can deliver packages as standalone RPM with 
install instructions.

I will update release build anyway, but community will decide whether to 
release package or not.
Nightly builds will be available for enthusiasts with all deliveries possible.


> On 21 Apr 2021, at 17:22, Ilya Kasnacheev  wrote:
> 
> Hello!
> 
> Were you able to get any traction here?
> 
> Apache Ignite 3.0 seems to be in too early stage for packaging, IMHO.
> We see very few interest in improving Apache Ignite 2.x packages, anyway.
> 
> Regards,
> -- 
> Ilya Kasnacheev
> 
> 
> ср, 24 февр. 2021 г. в 19:01, Petr Ivanov :
> 
>> Hi, Igniters.
>> 
>> 
>> Sincerely asking you to help me with testing new RPM package for apache
>> ignite 3.x [1]
>> Currently what is needed to be tested — successful installation and binary
>> run under most of RHEL based distributions: Red Hat, CentOS, Oracle Linux,
>> Fedora, etc.
>> 
>> Any other thoughts and suggestions are more than welcome too.
>> RPM assembly code can be found here [2]
>> 
>> 
>> Thanks in advance!
>> 
>> 
>> 
>> [1]
>> https://issues.apache.org/jira/secure/attachment/13021121/apache-ignite-3.0.0-1.noarch.rpm
>> [2] https://github.com/apache/ignite-3/pull/29/files
>> 
>> 



Re: [VOTE][EXTENSION] Release Apache Ignite performance-statistics-ext, spring-data-all-ext and spring-tx-ext extensions 1.0.0 RC1

2021-04-22 Thread Petr Ivanov
Looks good.


What suites are in question, these ones [1]?



[1] https://ci.ignite.apache.org/project/IgniteExtensions_Tests
> On 22 Apr 2021, at 10:42, Nikita Amelchev  wrote:
> 
> Hi, guys.
> 
> I have prepared PR to fix module names [1, 2]. Could you take a look
> and recheck TC integration, please?
> 
> Note that module names in TC suites should be changed as well.
> 
> [1] https://github.com/apache/ignite-extensions/pull/58
> [2] https://issues.apache.org/jira/browse/IGNITE-14621
> 
> чт, 22 апр. 2021 г. в 00:08, Nikita Amelchev :
> 
>> 
>> I have created the issue to fix modules names:
>> 
>> https://issues.apache.org/jira/browse/IGNITE-14621
>> 
>> ср, 21 апр. 2021 г. в 11:46, Nikita Amelchev :
>>> 
>>> +1 to formalize extension modules names:
>>> ignite-{directory-name}
>>> 
>>> The release script has this issue too. It will work fine with that name.
>>> 
>>> ср, 21 апр. 2021 г. в 10:37, Petr Ivanov :
 
 I checked the modules and there is misnaming issue which I think is 
 critical to test integration automation on TC.
 Can we change maven module names sping-data-2.x-ext to align with 
 directory name? Currently there is underscore in maven module name, which 
 is hyphen in directory name.
 
 
> On 21 Apr 2021, at 10:22, Nikita Amelchev  wrote:
> 
> +1 to postpone the spring-tx-ext extension release.
> 
> So, the following extensions will be released now:
> 
> spring-data-ext
> spring-data-2.0-ext
> spring-data-2.2-ext
> spring-data-commons
> performance-statistics-ext
> 
> вт, 20 апр. 2021 г. в 14:49, Mikhail Petrov :
>> 
>> Igniters,
>> 
>> Changing the scope of Spring dependencies to "provided" in Ignite Spring
>> extensions does not currently work as expected:
>> some versions of Spring that a user can specify via maven configuration
>> for Spring extensions may conflict with the hard-coded version of Spring
>> dependencies that the ignite-spring module relies on.
>> 
>> The issue mentioned above affects the Ignite Spring Transactions
>> integration. And since this integration is included in the Ignite 2.10
>> release, I suggest postponing the release of the Ignite Spring
>> Transactions integration until the above issue is properly fixed.
>> 
>> Any objections?
>> 
>> On 16.04.2021 09:15, Ivan Daschinsky wrote:
>>> -1 From me. There is an absence of NOTICE and LICENSE files in binary
>>> package. Also, there is no source package. These is a violation of 
>>> apache
>>> release policy [1]
>>> [1] https://www.apache.org/legal/release-policy.html
>>> 
>>> чт, 15 апр. 2021 г. в 23:23, Nikita Amelchev :
>>> 
 According to ASF release policy [1] non-PMC committers can sign 
 artifacts:
 
> all artifacts placed in their directory must be signed by a committer,
 preferably by a PMC member.
 
 [1] https://www.apache.org/legal/release-policy.html
 
 чт, 15 апр. 2021 г. в 23:05, Dmitriy Pavlov :
> My best guess that since PMCs have a binding vote in voting in 
> release, a
> PMC member should sign binaries as well. And I suppose that in the 
> past
> only PMC members were signing the release.
> 
> Meanwhile, https://infra.apache.org/release-signing.html does not
 contain
> any mention of PMC role and only committers are mentioned there. It
> might be not an answer, since a lot of projects have PMC=Committer and
 just
> one election.
> 
> Sincerely,
> Dmitriy Pavlov
> 
> чт, 15 апр. 2021 г. в 22:05, Ivan Daschinsky :
> 
>> I'm sorry, but is it OK, that artifacts are signed with signature of
>> non-PMC?
>> 
>> чт, 15 апр. 2021 г. в 19:26, Nikita Amelchev :
>> 
>>> Dear Ignite Community,
>>> 
>>> I have uploaded a release candidate of the following extension
 modules:
>>> performance-statistics-ext
>>> spring-data-ext
>>> spring-data-2.0-ext
>>> spring-data-2.2-ext
>>> spring-data-commons
>>> spring-tx-ext
>>> 
>>> The release candidate of the performance-statistics-ext extension:
>>> 
>>> 
 https://dist.apache.org/repos/dist/dev/ignite/ignite-performance-statistics-ext/1.0.0-rc1/
>>> The following staging can be used for testing:
>>> 
 https://repository.apache.org/content/repositories/orgapacheignite-1509
>>> Tags were created:
>>> 
>>> ignite-performance-statistics-ext-1.0.0-rc1
>>> 
>>> 
 https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=b992d9758278c38e93b375b08e4052954744a436
>>> ignite-spring-data-ext-1.0.0-rc1
>>> 

Re: [VOTE][EXTENSION] Release Apache Ignite performance-statistics-ext, spring-data-all-ext and spring-tx-ext extensions 1.0.0 RC1

2021-04-22 Thread Nikita Amelchev
Hi, guys.

I have prepared PR to fix module names [1, 2]. Could you take a look
and recheck TC integration, please?

Note that module names in TC suites should be changed as well.

[1] https://github.com/apache/ignite-extensions/pull/58
[2] https://issues.apache.org/jira/browse/IGNITE-14621

чт, 22 апр. 2021 г. в 00:08, Nikita Amelchev :

>
> I have created the issue to fix modules names:
>
> https://issues.apache.org/jira/browse/IGNITE-14621
>
> ср, 21 апр. 2021 г. в 11:46, Nikita Amelchev :
> >
> > +1 to formalize extension modules names:
> > ignite-{directory-name}
> >
> > The release script has this issue too. It will work fine with that name.
> >
> > ср, 21 апр. 2021 г. в 10:37, Petr Ivanov :
> > >
> > > I checked the modules and there is misnaming issue which I think is 
> > > critical to test integration automation on TC.
> > > Can we change maven module names sping-data-2.x-ext to align with 
> > > directory name? Currently there is underscore in maven module name, which 
> > > is hyphen in directory name.
> > >
> > >
> > > > On 21 Apr 2021, at 10:22, Nikita Amelchev  wrote:
> > > >
> > > > +1 to postpone the spring-tx-ext extension release.
> > > >
> > > > So, the following extensions will be released now:
> > > >
> > > > spring-data-ext
> > > > spring-data-2.0-ext
> > > > spring-data-2.2-ext
> > > > spring-data-commons
> > > > performance-statistics-ext
> > > >
> > > > вт, 20 апр. 2021 г. в 14:49, Mikhail Petrov :
> > > >>
> > > >> Igniters,
> > > >>
> > > >> Changing the scope of Spring dependencies to "provided" in Ignite 
> > > >> Spring
> > > >> extensions does not currently work as expected:
> > > >> some versions of Spring that a user can specify via maven configuration
> > > >> for Spring extensions may conflict with the hard-coded version of 
> > > >> Spring
> > > >> dependencies that the ignite-spring module relies on.
> > > >>
> > > >> The issue mentioned above affects the Ignite Spring Transactions
> > > >> integration. And since this integration is included in the Ignite 2.10
> > > >> release, I suggest postponing the release of the Ignite Spring
> > > >> Transactions integration until the above issue is properly fixed.
> > > >>
> > > >> Any objections?
> > > >>
> > > >> On 16.04.2021 09:15, Ivan Daschinsky wrote:
> > > >>> -1 From me. There is an absence of NOTICE and LICENSE files in binary
> > > >>> package. Also, there is no source package. These is a violation of 
> > > >>> apache
> > > >>> release policy [1]
> > > >>> [1] https://www.apache.org/legal/release-policy.html
> > > >>>
> > > >>> чт, 15 апр. 2021 г. в 23:23, Nikita Amelchev :
> > > >>>
> > >  According to ASF release policy [1] non-PMC committers can sign 
> > >  artifacts:
> > > 
> > > > all artifacts placed in their directory must be signed by a 
> > > > committer,
> > >  preferably by a PMC member.
> > > 
> > >  [1] https://www.apache.org/legal/release-policy.html
> > > 
> > >  чт, 15 апр. 2021 г. в 23:05, Dmitriy Pavlov :
> > > > My best guess that since PMCs have a binding vote in voting in 
> > > > release, a
> > > > PMC member should sign binaries as well. And I suppose that in the 
> > > > past
> > > > only PMC members were signing the release.
> > > >
> > > > Meanwhile, https://infra.apache.org/release-signing.html does not
> > >  contain
> > > > any mention of PMC role and only committers are mentioned there. It
> > > > might be not an answer, since a lot of projects have PMC=Committer 
> > > > and
> > >  just
> > > > one election.
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > > чт, 15 апр. 2021 г. в 22:05, Ivan Daschinsky :
> > > >
> > > >> I'm sorry, but is it OK, that artifacts are signed with signature 
> > > >> of
> > > >> non-PMC?
> > > >>
> > > >> чт, 15 апр. 2021 г. в 19:26, Nikita Amelchev 
> > > >> :
> > > >>
> > > >>> Dear Ignite Community,
> > > >>>
> > > >>> I have uploaded a release candidate of the following extension
> > >  modules:
> > > >>> performance-statistics-ext
> > > >>> spring-data-ext
> > > >>> spring-data-2.0-ext
> > > >>> spring-data-2.2-ext
> > > >>> spring-data-commons
> > > >>> spring-tx-ext
> > > >>>
> > > >>> The release candidate of the performance-statistics-ext extension:
> > > >>>
> > > >>>
> > >  https://dist.apache.org/repos/dist/dev/ignite/ignite-performance-statistics-ext/1.0.0-rc1/
> > > >>> The following staging can be used for testing:
> > > >>>
> > >  https://repository.apache.org/content/repositories/orgapacheignite-1509
> > > >>> Tags were created:
> > > >>>
> > > >>> ignite-performance-statistics-ext-1.0.0-rc1
> > > >>>
> > > >>>
> > >  https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=b992d9758278c38e93b375b08e4052954744a436
> > > >>> ignite-spring-data-ext-1.0.0-rc1
> > > >>> 

[ANNOUNCE] Apache IGNITE python thin client (pyignite) 0.4.0 released

2021-04-22 Thread Ivan Daschinsky
The Apache Ignite Community is pleased to announce the release of
Apache IGNITE python thin client (pyignite) 0.4.0.

This new release of python thin client contains a lot of changes.
Namely:
1. Partition awareness support.
2. Asyncio support.
3. Numerous performance fixes.
4. Numerous bug fixes.

For the full list of changes, you can refer to the RELEASE_NOTES.
https://ignite.apache.org/releases/pyignite/0.4.0/release_notes.html

You can install this version using pip
>> pip install pyignite==0.4.0

Alternatively, you can download sources and binary packages (wheels) from
here:
https://archive.apache.org/dist/ignite/pyignite/0.4.0/

Please let us know if you have any problems
https://ignite.apache.org/community/resources.html#ask

Regards,
Ivan Daschinsky on behalf of the Apache Ignite community.