Re: Resurrect FairAffinityFunction

[dsetrakyan]

Agree with Val, we should bring it back.

⁣D.​

On Jul 24, 2017, 8:14 PM, at 8:14 PM, Valentin Kulichenko 
 wrote:
>Guys,
>
>Some time ago we removed FairAffinityFunction from the project.
>However, my
>communication with users clearly shows that is was a rush decision.
>Distribution showed by Fair AF is much better than default and for some
>users it's extremely important. Basically, there are cases when
>rendezvous
>function is no-go.
>
>The reason for removal was that it was possible to get inconsistent
>results
>in case multiple caches were created on different topologies. However,
>I
>think this is fixable. As far as I understand, the only thing we need
>to do
>is to maintain a single AffinityFunctionContext for all the caches with
>same affinity function. Currently for each cache we have separate
>context
>which holds the state used by Fair AF. If the state is different, we
>have
>an issue.
>
>The only question is how to check whether two functions are the same or
>not. In case both cache node filter and backup filter are not
>configured,
>this is easy - if number of partitions and excludeNeighbors flag are
>equal
>for two functions, these functions are also equal.
>
>With filters it's a bit more complicated as these are custom
>implementations and in general case we don't know how to compare them.
>Although, to solve this problem, we can enforce user to implement
>equals()
>method for these implementation if Fair AF is used.
>
>I propose to make changes described above and bring Fair AF back.
>
>Thoughts?
>
>-Val

Resurrect FairAffinityFunction

[Valentin Kulichenko]

Guys,

Some time ago we removed FairAffinityFunction from the project. However, my
communication with users clearly shows that is was a rush decision.
Distribution showed by Fair AF is much better than default and for some
users it's extremely important. Basically, there are cases when rendezvous
function is no-go.

The reason for removal was that it was possible to get inconsistent results
in case multiple caches were created on different topologies. However, I
think this is fixable. As far as I understand, the only thing we need to do
is to maintain a single AffinityFunctionContext for all the caches with
same affinity function. Currently for each cache we have separate context
which holds the state used by Fair AF. If the state is different, we have
an issue.

The only question is how to check whether two functions are the same or
not. In case both cache node filter and backup filter are not configured,
this is easy - if number of partitions and excludeNeighbors flag are equal
for two functions, these functions are also equal.

With filters it's a bit more complicated as these are custom
implementations and in general case we don't know how to compare them.
Although, to solve this problem, we can enforce user to implement equals()
method for these implementation if Fair AF is used.

I propose to make changes described above and bring Fair AF back.

Thoughts?

-Val

Re: Timeouts in atomic cache

[Valentin Kulichenko]

Yakov,

Thanks for response. I definitely like the idea of detecting Java level
deadlocks.

As for hangs caused by Ignite internal problems, do we have a ticket for
this as well? Do you have any idea about how this should be implemented?

-Val

On Mon, Jul 24, 2017 at 3:55 AM, Yakov Zhdanov  wrote:

> Val, it seems you spotted and issue. Please file a ticket - I would suggest
> to remove the exceptions entirely as in my understanding timeout logic for
> atomic operation will bring additional overhead, but most of the time
> atomic operations are instant. From timeout perspective, what differs
> atomic operation from a transaction is that you cannot predict when user
> releases lock he acquired inside a transaction, but atomic operation should
> have predictable timeout.
>
> As far as your example. Currently, this will lead to java-level deadlock on
> synchronized sections for the cache entries (but when we move to pure
> thread-per-partition for atomic caches this will not be an issue any more
> https://issues.apache.org/jira/browse/IGNITE-4506). I would suggest we
> file
> a ticket to implement detection of java-level deadlock and allow user to
> configure policy to take appropriate action on deadlock wherever it happens
> - https://issues.apache.org/jira/browse/IGNITE-5811
>
> Any other hang of the atomic operation seem to be caused by issues in
> Ignite's internal machinery - either hanged exchange or problems in message
> processing on some node (e.g. all threads are busy and/or in deadlock)
> which again should result in notifying user and stopping node (by default).
>
> --Yakov
>

Re: Ignite internal events tracing

[Valentin Kulichenko]

Yakov,

How IgniteDiagnosticAware can be used? Is there any information?

-Val

On Mon, Jul 24, 2017 at 3:24 AM, Yakov Zhdanov  wrote:

> Alex, I like the idea very much, but I think we need to rethink the
> implementation approach to make it more generic. Passing parameter to each
> invocation seems dirty to me.
>
> Val,  we already have this. Please
> see org.apache.ignite.internal.IgniteDiagnosticAware
>
> Dmitry, what you suggest will be pretty hard to implement. I would better
> improve self-diagnostic system to extend list of metrics Ignite monitors.
>
> --Yakov
>

Re: [VOTE] Apache Ignite 2.1.0 RC4

[Valentin Kulichenko]

+1 (binding)

On Mon, Jul 24, 2017 at 6:39 AM, Dmitriy Setrakyan 
wrote:

> Anton,
>
> You should treat this vote as a brand new vote. According to Apache rules,
> you need 3 +1 votes and it has to go for 72 hours.
>
> D.
>
> On Mon, Jul 24, 2017 at 8:32 AM, Anton Vinogradov  wrote:
>
> > Igniters,
> >
> > This vote based on same files as RC3.
> > Only one change is that I signed zips with my signature.
> > KEYS files (https://dist.apache.org/repos/dist/release/ignite/KEYS) was
> > updated as well.
> >
> > We already got 5 "+1" at RC3, so, is there any reason to wait 72 hours?
> > This vote will go for 72 hours but may be closed earlier if Konstantin
> > Boudnik confirmed security issue solved.
> >
> > We have uploaded a 2.1.0 release candidate to
> > https://dist.apache.org/repos/dist/dev/ignite/2.1.0-rc4/
> >
> > Git tag name is
> > 2.1.0-rc4
> >
> > This release includes the following changes:
> >
> > Ignite:
> > * Persistent cache store
> > * Added IgniteFuture.listenAsync() and IgniteFuture.chainAsync() mehtods
> > * Deprecated IgniteConfiguration.marshaller
> > * Updated Lucene dependency to version 5.5.2
> > * Machine learning: implemented K-means clusterization algorithm
> optimized
> > for distributed storages
> > * SQL: CREATE TABLE and DROP TABLE commands support
> > * SQL: New thin JDBC driver
> > * SQL: Improved performance of certain queries, when affinity node can be
> > calculated in advance
> > * SQL: Fixed return type of AVG() function
> > * SQL: BLOB type support added to thick JDBC driver
> > * SQL: Improved LocalDate, LocalTime and LocalDateTime support for Java 8
> > * SQL: Added FieldsQueryCursor interface to get fields metadata for
> > SqlFieldsQuery
> > * ODBC: Implemented DML statement batching
> > * Massive performance and stability improvements
> >
> > Ignite.NET:
> > * Automatic remote assembly loading
> > * NuGet-based standalone node deployment
> > * Added conditional data removeal via LINQ DeleteAll
> > * Added TimestampAttribute to control DateTime serialization mode
> > * Added local collections joins support to LINQ.
> >
> > Ignite CPP:
> > * Added Compute::Call and Compute::Broadcast methods
> >
> > Web Console:
> > * Implemented support for UNIQUE indexes for key fields on import model
> > from RDBMS
> > * Added option to show full stack trace on Queries screen
> > * Added PK alias generation on Models screen.
> >
> > Complete list of closed issues:
> > https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%
> > 20fixVersion%20%3D%202.1%20AND%20(status%20%3D%
> > 20closed%20or%20status%20%3D%
> > 20resolved)
> >
> > DEVNOTES
> > https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_
> > plain;f=DEVNOTES.txt;hb=refs/tags/2.1.0-rc4
> >
> > RELEASE NOTES
> > https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_
> > plain;f=RELEASE_NOTES.txt;hb=refs/tags/2.1.0-rc4
> >
> > Please start voting.
> >
> > +1 - to accept Apache Ignite 2.1.0-rc4
> > 0 - don't care either way
> > -1 - DO NOT accept Apache Ignite 2.1.0-rc4 (explain why)
> >
> > This vote will go for 72 hours.
> >
>

Re: Changing public IgniteCompute API to improve changes in 5037 ticket

[Valentin Kulichenko]

Anton,

You can call affinityCallAsync multiple times and then reduce locally.

-Val

On Mon, Jul 24, 2017 at 3:05 AM, Anton Vinogradov 
wrote:

> Val,
>
> > What is the use case for which current affinityRun/Call API doesn't work?
> It does not work for map/reduce.
>
> On Fri, Jul 21, 2017 at 11:42 PM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Maxim,
> >
> > The issue is that it's currently assumed to support job mapping, but it
> > actually doesn't. However, I agree that AffinityKeyMapped annotation
> > doesn't fit the use case well. Let's fix documentation and JavaDoc then.
> >
> > As for the proposed API, it's overcomplicated, took me 15 minutes to
> > understand what it does :)
> >
> > What is the use case for which current affinityRun/Call API doesn't work?
> >
> > -Val
> >
> > On Fri, Jul 21, 2017 at 5:57 AM, Kozlov Maxim 
> > wrote:
> >
> > > Valentin,
> > >
> > > The author of tiket wants to see to provide some API allows to map
> > > ComputeJobs to partitions or keys. If we use @AffinityKeyMapped then
> you
> > > need to enter the cache name parameter, I think this is not convenient
> > for
> > > the user. Therefore, I propose to extend the existing API.
> > > Having consulted with Anton V. decided to make a separate interface
> > > ReducibleTask, which will allow us to have different map logic at each
> > > inheritor.
> > >
> > > Old method, allows to map to node
> > > public interface ComputeTask extends ReducibleTask {
> > > @Nullable public Map
> > > map(List subgrid, @Nullable T arg) throws IgniteException;
> > > }
> > >
> > > Brand new method with mapping to partitions, which solves topology
> change
> > > issues.
> > > public interface AffinityComputeTask extends ReducibleTask {
> > > @Nullable public Map
> > map(@NotnullString
> > > cacheName, List partIds, @Nullable T arg) throws
> > IgniteException;
> > > }
> > >
> > > public interface ReducibleTask extends Serializable {
> > > public ComputeJobResultPolicy result(ComputeJobResult res,
> > > List rcvd) throws IgniteException;
> > >
> > > @Nullable public R reduce(List results) throws
> > > IgniteException;
> > > }
> > >
> > > We also need to implement AffinityComputeTaskAdapter and
> > > AffinityComputeTaskSplitAdapter, for implementation by default. It is
> > > right?
> > >
> > > In the IgniteCompute add:
> > >
> > > @IgniteAsyncSupported
> > > public  R affinityExecute(Class > R>>
> > > taskCls, List partIds, @Nullable T arg) throws
> IgniteException;
> > > @IgniteAsyncSupported
> > > public  R affinityExecute(AffinityComputeTask task,
> > > List partIds, @Nullable T arg) throws IgniteException;
> > >
> > > public  ComputeTaskFuture affinityExecuteAsync(Class > > AffinityComputeTask> taskCls, List partIds, @Nullable T
> > arg)
> > > throws IgniteException;
> > > public  ComputeTaskFuture affinityExecuteAsync(
> > AffinityComputeTask > > R> task, List partIds, @Nullable T arg) throws
> IgniteException;
> > >
> > >
> > > How do you like this idea or do you insist that you need to use
> > > @AffinityKeyMapped to solve the problem?
> > >
> > >
> > > > 13 июля 2017 г., в 6:36, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> написал(а):
> > > >
> > > > Hi Max,
> > > >
> > > > This ticket doesn't assume any API changes, it's about broken
> > > > functionality. I would start with checking what tests we have
> > > > for @AffinityKeyMapped and creating missing one. From what I
> understand
> > > > functionality is broken completely or almost completely, so I guess
> > > testing
> > > > coverage is very weak there.
> > > >
> > > > -Val
> > > >
> > > > On Wed, Jul 12, 2017 at 4:27 PM, Kozlov Maxim 
> > > wrote:
> > > >
> > > >> Igniters,
> > > >>
> > > >> jira: https://issues.apache.org/jira/browse/IGNITE-5037 <
> > > >> https://issues.apache.org/jira/browse/IGNITE-5037>
> > > >> How do you look to solve this ticket by adding two methods to the
> > public
> > > >> IgniteCompute API?
> > > >>
> > > >> @IgniteAsyncSupported
> > > >> public void affinityRun(@NotNull Collection cacheNames,
> > > >> Collection keys, IgniteRunnable job)
> > > >>throws IgniteException;
> > > >>
> > > >> @IgniteAsyncSupported
> > > >> public  R affinityCall(@NotNull Collection cacheNames,
> > > >> Collection keys, IgniteCallable job)
> > > >>throws IgniteException;
> > > >>
> > > >> There is also a question of how to act when changing the topology
> > during
> > > >> the execution of the job.
> > > >> 1) complete with an exception;
> > > >> 2) stop execution and wait until the topology is rebuilt and
> continue
> > > >> execution;
> > > >>
> > > >> I think the second way, do you think?
> > > >>
> > > >> --
> > > >> Best Regards,
> > > >> Max K.
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > >
> > > --
> > > Best Regards,
> > > Max K.
> > >
> > >
> > >
> > >
> > >
> >
>

Re: [VOTE] Apache Ignite 2.1.0 RC3

[Denis Magda]

Cos,

I’ll highly appreciate if you double-check that RC4 is clean and no longer have 
any issues revealed by you:
http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Apache-Ignite-2-1-0-RC4-td19969.html
 


—
Denis

> On Jul 24, 2017, at 11:04 AM, Konstantin Boudnik  wrote:
> 
> Got it. Thank you for the understanding and readiness to deal with the
> finding - that might not look like a big issues for us, but could
> alert some of the users. I will be happy to jump on another
> verification cycle as soon as it is available. Please let me know if I
> can help with anything.
> 
> With best regards,
>  Cos
> --
>  With regards,
> Konstantin (Cos) Boudnik
> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
> 
> Disclaimer: Opinions expressed in this email are those of the author,
> and do not necessarily represent the views of any company the author
> might be affiliated with at the moment of writing.
> 
> 
> On Mon, Jul 24, 2017 at 10:46 AM, Denis Magda  wrote:
>> Hi Cos,
>> 
>>> Which tells me that the private key is simply shared by a number of the
>>> committers. And there's no guarantee that it hasn't been leaked outside of
>>> the group. And that's pretty serious security flaw, actually.
>> 
>> That’s not the case. Sam signed and did final technical steps preparing the 
>> RC3. I took care of other formalities.
>> 
>> Personally, did expect this to be an issue. Agree, let’s fix the process 
>> making sure the release manager signs bundles all the times.
>> 
>>> - why every other RC Vote is started by a different person?
>> 
>> 
>> Summer time, vacations, day offs…
>> 
>> —
>> Denis
>> 
>>> On Jul 22, 2017, at 1:26 PM, Konstantin Boudnik  wrote:
>>> 
>>> Retracting this, found the KEYS (douh...). Still
>>> 
>>> -1 (binding). The release isn't signed by the release manager. Someone else
>>> key is used.
>>> 
>>> - Checked the sha1
>>> - Successfully ran the build
>>> - Checked the signature
>>> - The archive is signed by the key 593A743B belonging to sboi...@apache.org.
>>> However, none of the 2.1.0 RC [VOTE] attempts were started by this person.
>>> Which tells me that the private key is simply shared by a number of the
>>> committers. And there's no guarantee that it hasn't been leaked outside of
>>> the group. And that's pretty serious security flaw, actually.
>>> 
>>> Why the release managers aren't using their own keys? It is easy to generate
>>> and sign the keys following guidelines [1]. Committers' keys are easy to
>>> validate against the Apache repository [2]
>>> 
>>> Things that need to be improved in the next release:
>>> - neither sha1 nor md5 are trustful checksum'ing methods and aren't
>>> guaranteeing the authenticity of the source archive. We should be switching
>>> to at least sha265 or higher. This has been brought up since the incubation.
>>> And warrants for -1 in the next release.
>>> - why every other RC Vote is started by a different person?
>>> 
>>> With regards,
>>> Cos
>>> 
>>> [1] https://people.apache.org/keys/committer/
>>> [2] 
>>> https://www.apache.org/dev/new-committers-guide.html#set-up-security-and-pgp-keys
>>> 
>>> On Sat, Jul 22, 2017 at 01:00PM, Konstantin Boudnik wrote:
 Am I missing the location of the signing keys? I cannot verivy the 
 signature
 of the archive.
 
 -1 (binding) until then.
 
 Thanks
 Cos
 
 On Thu, Jul 20, 2017 at 03:34PM, Denis Magda wrote:
> Igniters,
> 
> Setting off the vote one more time. Hope I’ll be successful this time, 
> keeping fingers crossed :)
> 
> We have uploaded a 2.1.0 release candidate to
> https://dist.apache.org/repos/dist/dev/ignite/2.1.0-rc3/
> 
> Git tag name is
> 2.1.0-rc3
> 
> This release includes the following changes:
> 
> Ignite:
> * Persistent cache store
> * Added IgniteFuture.listenAsync() and IgniteFuture.chainAsync() mehtods
> * Deprecated IgniteConfiguration.marshaller
> * Updated Lucene dependency to version 5.5.2
> * Machine learning: implemented K-means clusterization algorithm optimized
> for distributed storages
> * SQL: CREATE TABLE and DROP TABLE commands support
> * SQL: New thin JDBC driver
> * SQL: Improved performance of certain queries, when affinity node can be
> calculated in advance
> * SQL: Fixed return type of AVG() function
> * SQL: BLOB type support added to thick JDBC driver
> * SQL: Improved LocalDate, LocalTime and LocalDateTime support for Java 8
> * SQL: Added FieldsQueryCursor interface to get fields metadata for
> SqlFieldsQuery
> * ODBC: Implemented DML statement batching
> * Massive performance and stability improvements
> 
> Ignite.NET:
> * Automatic remote assembly loading
> * NuGet-based standalone node deployment
> * Added conditional data 

Re: [VOTE] Apache Ignite 2.1.0 RC3

[Konstantin Boudnik]

Got it. Thank you for the understanding and readiness to deal with the
finding - that might not look like a big issues for us, but could
alert some of the users. I will be happy to jump on another
verification cycle as soon as it is available. Please let me know if I
can help with anything.

With best regards,
  Cos
--
  With regards,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622

Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.


On Mon, Jul 24, 2017 at 10:46 AM, Denis Magda  wrote:
> Hi Cos,
>
>>  Which tells me that the private key is simply shared by a number of the
>>  committers. And there's no guarantee that it hasn't been leaked outside of
>>  the group. And that's pretty serious security flaw, actually.
>
> That’s not the case. Sam signed and did final technical steps preparing the 
> RC3. I took care of other formalities.
>
> Personally, did expect this to be an issue. Agree, let’s fix the process 
> making sure the release manager signs bundles all the times.
>
>> - why every other RC Vote is started by a different person?
>
>
> Summer time, vacations, day offs…
>
> —
> Denis
>
>> On Jul 22, 2017, at 1:26 PM, Konstantin Boudnik  wrote:
>>
>> Retracting this, found the KEYS (douh...). Still
>>
>> -1 (binding). The release isn't signed by the release manager. Someone else
>> key is used.
>>
>> - Checked the sha1
>> - Successfully ran the build
>> - Checked the signature
>> - The archive is signed by the key 593A743B belonging to sboi...@apache.org.
>>  However, none of the 2.1.0 RC [VOTE] attempts were started by this person.
>>  Which tells me that the private key is simply shared by a number of the
>>  committers. And there's no guarantee that it hasn't been leaked outside of
>>  the group. And that's pretty serious security flaw, actually.
>>
>>  Why the release managers aren't using their own keys? It is easy to generate
>>  and sign the keys following guidelines [1]. Committers' keys are easy to
>>  validate against the Apache repository [2]
>>
>> Things that need to be improved in the next release:
>> - neither sha1 nor md5 are trustful checksum'ing methods and aren't
>>  guaranteeing the authenticity of the source archive. We should be switching
>>  to at least sha265 or higher. This has been brought up since the incubation.
>>  And warrants for -1 in the next release.
>> - why every other RC Vote is started by a different person?
>>
>> With regards,
>>  Cos
>>
>> [1] https://people.apache.org/keys/committer/
>> [2] 
>> https://www.apache.org/dev/new-committers-guide.html#set-up-security-and-pgp-keys
>>
>> On Sat, Jul 22, 2017 at 01:00PM, Konstantin Boudnik wrote:
>>> Am I missing the location of the signing keys? I cannot verivy the signature
>>> of the archive.
>>>
>>> -1 (binding) until then.
>>>
>>> Thanks
>>>  Cos
>>>
>>> On Thu, Jul 20, 2017 at 03:34PM, Denis Magda wrote:
 Igniters,

 Setting off the vote one more time. Hope I’ll be successful this time, 
 keeping fingers crossed :)

 We have uploaded a 2.1.0 release candidate to
 https://dist.apache.org/repos/dist/dev/ignite/2.1.0-rc3/

 Git tag name is
 2.1.0-rc3

 This release includes the following changes:

 Ignite:
 * Persistent cache store
 * Added IgniteFuture.listenAsync() and IgniteFuture.chainAsync() mehtods
 * Deprecated IgniteConfiguration.marshaller
 * Updated Lucene dependency to version 5.5.2
 * Machine learning: implemented K-means clusterization algorithm optimized
 for distributed storages
 * SQL: CREATE TABLE and DROP TABLE commands support
 * SQL: New thin JDBC driver
 * SQL: Improved performance of certain queries, when affinity node can be
 calculated in advance
 * SQL: Fixed return type of AVG() function
 * SQL: BLOB type support added to thick JDBC driver
 * SQL: Improved LocalDate, LocalTime and LocalDateTime support for Java 8
 * SQL: Added FieldsQueryCursor interface to get fields metadata for
 SqlFieldsQuery
 * ODBC: Implemented DML statement batching
 * Massive performance and stability improvements

 Ignite.NET:
 * Automatic remote assembly loading
 * NuGet-based standalone node deployment
 * Added conditional data removeal via LINQ DeleteAll
 * Added TimestampAttribute to control DateTime serialization mode
 * Added local collections joins support to LINQ.

 Ignite CPP:
 * Added Compute::Call and Compute::Broadcast methods

 Web Console:
 * Implemented support for UNIQUE indexes for key fields on import model
 from RDBMS
 * Added option to show full stack trace on Queries screen
 * Added PK alias generation on Models screen.

 Complete list of closed issues:
 

Re: Cache Metrics

[Denis Magda]

Guys,

What if we calculate it on both sides? The client will keep the total time 
needed to complete an operation including network hoops while a server (primary 
or backup) will count only local time.

—
Denis

> On Jul 17, 2017, at 7:07 AM, Andrey Gura  wrote:
> 
> Hi,
> 
> I believe that the first solution is better than second because it
> takes into account network communication time. Average time of
> communication between nodes doesn't make sense from my point of view.
> 
> So I vote for #1.
> 
> On Thu, Jul 13, 2017 at 11:52 PM, Вячеслав Коптилин
>  wrote:
>> Hi Experts,
>> 
>> I am working on https://issues.apache.org/jira/browse/IGNITE-3495
>> 
>> A few words about this issue:
>> It is about that the process of gathering/updating of cache metrics is
>> inconsistent in some cases.
>> Let's consider the following simple topology which contains only two nodes:
>> first node is a client node and the second is a server.
>> And client node starts requests to the server node, for instance
>> cache.put(), cache.putAll(), cache.get() etc.
>> In that case, metrics which are related to counters (cache hits, cache
>> misses, removals and puts) are calculated on the server side,
>> while time metrics are updated on the client node.
>> 
>> I think that both metrics (counters and time) should be calculated on the
>> same node. So, there are two obvious solution:
>> 
>> #1 Node that starts some operation is responsible for updating the cache
>> metrics.
>> Pro:
>> - it will allow to get more accurate results of metrics.
>> Contra:
>> - this approach does not work in particular cases. for example, partitioned
>> cache with FULL_ASYNC write synchronization mode.
>> - needs to extend response messages (GridNearAtomicUpdateResponse,
>> GridNearGetResponse etc)
>>  in order to provide additional information from remote node: cache hits,
>> number of removal etc.
>>  So, it will lead to additional pressure on communication channel.
>> Perhaps, this impact will be small - 4 bytes per message or something like
>> that.
>> - backward incompatibility (this is a consequence of the previous point)
>> 
>> #2 Primary node (node that actually executes a request)
>> Pro:
>> - easy to implement
>> - backward compatible
>> Contra:
>> - time metrics will not include the time of communication between nodes, so
>> the results will be less accurate.
>> - perhaps we need to provide additional metric which will allow to get avg
>> time of communication between nodes.
>> 
>> Please let me know about your thoughts.
>> Perhaps, both alternatives are not so good...
>> 
>> Regards,
>> Slava.

Re: [VOTE] Apache Ignite 2.1.0 RC3

[Denis Magda]

Hi Cos,

>  Which tells me that the private key is simply shared by a number of the
>  committers. And there's no guarantee that it hasn't been leaked outside of
>  the group. And that's pretty serious security flaw, actually.

That’s not the case. Sam signed and did final technical steps preparing the 
RC3. I took care of other formalities.

Personally, did expect this to be an issue. Agree, let’s fix the process making 
sure the release manager signs bundles all the times.

> - why every other RC Vote is started by a different person?


Summer time, vacations, day offs…

—
Denis

> On Jul 22, 2017, at 1:26 PM, Konstantin Boudnik  wrote:
> 
> Retracting this, found the KEYS (douh...). Still
> 
> -1 (binding). The release isn't signed by the release manager. Someone else
> key is used.
> 
> - Checked the sha1
> - Successfully ran the build 
> - Checked the signature
> - The archive is signed by the key 593A743B belonging to sboi...@apache.org.
>  However, none of the 2.1.0 RC [VOTE] attempts were started by this person.
>  Which tells me that the private key is simply shared by a number of the
>  committers. And there's no guarantee that it hasn't been leaked outside of
>  the group. And that's pretty serious security flaw, actually.
> 
>  Why the release managers aren't using their own keys? It is easy to generate
>  and sign the keys following guidelines [1]. Committers' keys are easy to
>  validate against the Apache repository [2]
> 
> Things that need to be improved in the next release:
> - neither sha1 nor md5 are trustful checksum'ing methods and aren't
>  guaranteeing the authenticity of the source archive. We should be switching
>  to at least sha265 or higher. This has been brought up since the incubation.
>  And warrants for -1 in the next release.
> - why every other RC Vote is started by a different person?
> 
> With regards,
>  Cos
> 
> [1] https://people.apache.org/keys/committer/
> [2] 
> https://www.apache.org/dev/new-committers-guide.html#set-up-security-and-pgp-keys
> 
> On Sat, Jul 22, 2017 at 01:00PM, Konstantin Boudnik wrote:
>> Am I missing the location of the signing keys? I cannot verivy the signature
>> of the archive.
>> 
>> -1 (binding) until then.
>> 
>> Thanks
>>  Cos
>> 
>> On Thu, Jul 20, 2017 at 03:34PM, Denis Magda wrote:
>>> Igniters,
>>> 
>>> Setting off the vote one more time. Hope I’ll be successful this time, 
>>> keeping fingers crossed :)
>>> 
>>> We have uploaded a 2.1.0 release candidate to
>>> https://dist.apache.org/repos/dist/dev/ignite/2.1.0-rc3/
>>> 
>>> Git tag name is
>>> 2.1.0-rc3
>>> 
>>> This release includes the following changes:
>>> 
>>> Ignite:
>>> * Persistent cache store
>>> * Added IgniteFuture.listenAsync() and IgniteFuture.chainAsync() mehtods
>>> * Deprecated IgniteConfiguration.marshaller
>>> * Updated Lucene dependency to version 5.5.2
>>> * Machine learning: implemented K-means clusterization algorithm optimized
>>> for distributed storages
>>> * SQL: CREATE TABLE and DROP TABLE commands support
>>> * SQL: New thin JDBC driver
>>> * SQL: Improved performance of certain queries, when affinity node can be
>>> calculated in advance
>>> * SQL: Fixed return type of AVG() function
>>> * SQL: BLOB type support added to thick JDBC driver
>>> * SQL: Improved LocalDate, LocalTime and LocalDateTime support for Java 8
>>> * SQL: Added FieldsQueryCursor interface to get fields metadata for
>>> SqlFieldsQuery
>>> * ODBC: Implemented DML statement batching
>>> * Massive performance and stability improvements
>>> 
>>> Ignite.NET:
>>> * Automatic remote assembly loading
>>> * NuGet-based standalone node deployment
>>> * Added conditional data removeal via LINQ DeleteAll
>>> * Added TimestampAttribute to control DateTime serialization mode
>>> * Added local collections joins support to LINQ.
>>> 
>>> Ignite CPP:
>>> * Added Compute::Call and Compute::Broadcast methods
>>> 
>>> Web Console:
>>> * Implemented support for UNIQUE indexes for key fields on import model
>>> from RDBMS
>>> * Added option to show full stack trace on Queries screen
>>> * Added PK alias generation on Models screen.
>>> 
>>> Complete list of closed issues:
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%
>>> 20fixVersion%20%3D%202.1%20AND%20(status%20%3D%20closed%20or%20status%20%3D%
>>> 20resolved)
>>> 
>>> DEVNOTES
>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/tags/2.1.0-rc3
>>> 
>>> RELEASE NOTES
>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_plain;f=RELEASE_NOTES.txt;hb=refs/tags/2.1.0-rc3
>>> 
>>> Please start voting.
>>> 
>>> +1 - to accept Apache Ignite 2.1.0-rc3
>>> 0 - don't care either way
>>> -1 - DO NOT accept Apache Ignite 2.1.0-rc3 (explain why)
>>> 
>>> This vote will go for 72 hours.
>>> 
>>> —
>>> Denis
>>> 
> 
> 

Re: It seems WebSession's removeAttribute does not support HttpSessionBindingListener

[yucigou]

Hi Val, I've registered two listeners with the WebSession cache and also the
binary cache. When a session expires, the corresponding listeners will be
invoked.

https://github.com/apache/ignite/pull/2243/files

Any comments? Thanks.

Yuci



--
View this message in context: 
http://apache-ignite-developers.2346864.n4.nabble.com/It-seems-WebSession-s-removeAttribute-does-not-support-HttpSessionBindingListener-tp19184p19977.html
Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

[jira] [Created] (IGNITE-5819) SQL: add support for TRUNCATE TABLE command.

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-5819:


 Summary: SQL: add support for TRUNCATE TABLE command.
 Key: IGNITE-5819
 URL: https://issues.apache.org/jira/browse/IGNITE-5819
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Andrew Mashenkov
 Fix For: 2.3


Add support for  "TRUNCATE TABLE" command syntax.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5818) SQL: query with condition on affinity column works incorrect in some cases.

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-5818:


 Summary: SQL: query with condition on affinity column works 
incorrect in some cases.
 Key: IGNITE-5818
 URL: https://issues.apache.org/jira/browse/IGNITE-5818
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Andrew Mashenkov


Looks like query optimization [1] makes some kind of keys to be compared 
incorrectly.
E.g. Decimal key. PFA repro.

[1] https://issues.apache.org/jira/browse/IGNITE-4509



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5816) Race in WAL segment leading to ClosedChannelException

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-5816:


 Summary: Race in WAL segment leading to ClosedChannelException
 Key: IGNITE-5816
 URL: https://issues.apache.org/jira/browse/IGNITE-5816
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.1
Reporter: Alexey Goncharuk
Assignee: Alexey Goncharuk
 Fix For: 2.2






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5815) Web Console: Improve Zero Configuration Politics

[Vica Abramova (JIRA)]

Vica Abramova created IGNITE-5815:
-

 Summary: Web Console: Improve Zero Configuration Politics
 Key: IGNITE-5815
 URL: https://issues.apache.org/jira/browse/IGNITE-5815
 Project: Ignite
  Issue Type: Improvement
  Components: UI, wizards
Reporter: Vica Abramova


If the user enters a value in the field and it matches the default value, then 
we generate a string.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5814) Service deploy fails with NPE if it implements ExecutorService interface

[Evgenii Zhuravlev (JIRA)]

Evgenii Zhuravlev created IGNITE-5814:
-

 Summary: Service deploy fails with NPE if it implements 
ExecutorService interface
 Key: IGNITE-5814
 URL: https://issues.apache.org/jira/browse/IGNITE-5814
 Project: Ignite
  Issue Type: Bug
Affects Versions: 1.9
Reporter: Evgenii Zhuravlev






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [VOTE] Apache Ignite 2.1.0 RC4

[Dmitriy Setrakyan]

Anton,

You should treat this vote as a brand new vote. According to Apache rules,
you need 3 +1 votes and it has to go for 72 hours.

D.

On Mon, Jul 24, 2017 at 8:32 AM, Anton Vinogradov  wrote:

> Igniters,
>
> This vote based on same files as RC3.
> Only one change is that I signed zips with my signature.
> KEYS files (https://dist.apache.org/repos/dist/release/ignite/KEYS) was
> updated as well.
>
> We already got 5 "+1" at RC3, so, is there any reason to wait 72 hours?
> This vote will go for 72 hours but may be closed earlier if Konstantin
> Boudnik confirmed security issue solved.
>
> We have uploaded a 2.1.0 release candidate to
> https://dist.apache.org/repos/dist/dev/ignite/2.1.0-rc4/
>
> Git tag name is
> 2.1.0-rc4
>
> This release includes the following changes:
>
> Ignite:
> * Persistent cache store
> * Added IgniteFuture.listenAsync() and IgniteFuture.chainAsync() mehtods
> * Deprecated IgniteConfiguration.marshaller
> * Updated Lucene dependency to version 5.5.2
> * Machine learning: implemented K-means clusterization algorithm optimized
> for distributed storages
> * SQL: CREATE TABLE and DROP TABLE commands support
> * SQL: New thin JDBC driver
> * SQL: Improved performance of certain queries, when affinity node can be
> calculated in advance
> * SQL: Fixed return type of AVG() function
> * SQL: BLOB type support added to thick JDBC driver
> * SQL: Improved LocalDate, LocalTime and LocalDateTime support for Java 8
> * SQL: Added FieldsQueryCursor interface to get fields metadata for
> SqlFieldsQuery
> * ODBC: Implemented DML statement batching
> * Massive performance and stability improvements
>
> Ignite.NET:
> * Automatic remote assembly loading
> * NuGet-based standalone node deployment
> * Added conditional data removeal via LINQ DeleteAll
> * Added TimestampAttribute to control DateTime serialization mode
> * Added local collections joins support to LINQ.
>
> Ignite CPP:
> * Added Compute::Call and Compute::Broadcast methods
>
> Web Console:
> * Implemented support for UNIQUE indexes for key fields on import model
> from RDBMS
> * Added option to show full stack trace on Queries screen
> * Added PK alias generation on Models screen.
>
> Complete list of closed issues:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%
> 20fixVersion%20%3D%202.1%20AND%20(status%20%3D%
> 20closed%20or%20status%20%3D%
> 20resolved)
>
> DEVNOTES
> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_
> plain;f=DEVNOTES.txt;hb=refs/tags/2.1.0-rc4
>
> RELEASE NOTES
> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_
> plain;f=RELEASE_NOTES.txt;hb=refs/tags/2.1.0-rc4
>
> Please start voting.
>
> +1 - to accept Apache Ignite 2.1.0-rc4
> 0 - don't care either way
> -1 - DO NOT accept Apache Ignite 2.1.0-rc4 (explain why)
>
> This vote will go for 72 hours.
>

[VOTE] Apache Ignite 2.1.0 RC4

[Anton Vinogradov]

Igniters,

This vote based on same files as RC3.
Only one change is that I signed zips with my signature.
KEYS files (https://dist.apache.org/repos/dist/release/ignite/KEYS) was
updated as well.

We already got 5 "+1" at RC3, so, is there any reason to wait 72 hours?
This vote will go for 72 hours but may be closed earlier if Konstantin
Boudnik confirmed security issue solved.

We have uploaded a 2.1.0 release candidate to
https://dist.apache.org/repos/dist/dev/ignite/2.1.0-rc4/

Git tag name is
2.1.0-rc4

This release includes the following changes:

Ignite:
* Persistent cache store
* Added IgniteFuture.listenAsync() and IgniteFuture.chainAsync() mehtods
* Deprecated IgniteConfiguration.marshaller
* Updated Lucene dependency to version 5.5.2
* Machine learning: implemented K-means clusterization algorithm optimized
for distributed storages
* SQL: CREATE TABLE and DROP TABLE commands support
* SQL: New thin JDBC driver
* SQL: Improved performance of certain queries, when affinity node can be
calculated in advance
* SQL: Fixed return type of AVG() function
* SQL: BLOB type support added to thick JDBC driver
* SQL: Improved LocalDate, LocalTime and LocalDateTime support for Java 8
* SQL: Added FieldsQueryCursor interface to get fields metadata for
SqlFieldsQuery
* ODBC: Implemented DML statement batching
* Massive performance and stability improvements

Ignite.NET:
* Automatic remote assembly loading
* NuGet-based standalone node deployment
* Added conditional data removeal via LINQ DeleteAll
* Added TimestampAttribute to control DateTime serialization mode
* Added local collections joins support to LINQ.

Ignite CPP:
* Added Compute::Call and Compute::Broadcast methods

Web Console:
* Implemented support for UNIQUE indexes for key fields on import model
from RDBMS
* Added option to show full stack trace on Queries screen
* Added PK alias generation on Models screen.

Complete list of closed issues:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%
20fixVersion%20%3D%202.1%20AND%20(status%20%3D%20closed%20or%20status%20%3D%
20resolved)

DEVNOTES
https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/tags/2.1.0-rc4

RELEASE NOTES
https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_plain;f=RELEASE_NOTES.txt;hb=refs/tags/2.1.0-rc4

Please start voting.

+1 - to accept Apache Ignite 2.1.0-rc4
0 - don't care either way
-1 - DO NOT accept Apache Ignite 2.1.0-rc4 (explain why)

This vote will go for 72 hours.

[GitHub] ignite pull request #2339: Ignite 4181 public api

[andrey-kuznetsov]

GitHub user andrey-kuznetsov opened a pull request:

https://github.com/apache/ignite/pull/2339

Ignite 4181 public api



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrey-kuznetsov/ignite ignite-4181-public-api

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2339.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2339


commit 2e5d1206abb6ab897b1f173bd673f3d2fea7f5c7
Author: andrey-kuznetsov 
Date:   2017-07-24T11:40:15Z

IGNITE-4181: Reproducing with public API

commit 4dc88d742e851f50b3cd276c280dd417843d0dfd
Author: andrey-kuznetsov 
Date:   2017-07-24T13:04:42Z

IGNITE-4181: Reproducer logic improvement




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2285: IGNITE-5123

[YevIgn]

Github user YevIgn closed the pull request at:

https://github.com/apache/ignite/pull/2285


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: IGNITE-5123 Review

[Evgeniy Ignatiev]

Thank you! Many thanks to Dmitry for his attitude and dedication!


On 24.07.2017 16:47, Semyon Boikov wrote:

Evgeniy and Dmitry, thanks for the fix! Merged into master.

Thanks!

On Sun, Jul 23, 2017 at 9:21 PM, Dmitry Pavlov 
wrote:


Hi Evgeniy,

 From my point of view there are no problems with this fix. My testing
didn't show any issues with fix.

Igniters,

Are there any additional comments on this issue? Can we proceed?

Sincerely,
Dmitriy Pavlov

чт, 20 июл. 2017 г. в 20:04, Dmitry Pavlov :


Hi Evgeniy,

Thank you for such a careful research of the issue.

If don’t mind, I would like to do additional tests with this PR changes.

I will come back with result in couple of days

Sincerely,
Dmitriy Pavlov

чт, 20 июл. 2017 г. в 19:18, Evgeniy Ignatiev <

yevgeniy.ignat...@gmail.com

:
onIgniteStart was called in Ignite 1.X in
GridPluginComponent#onKernalStart as one of the calls to the component
callbacks, probably the order in which components were called, ensured
that contract of PluginProvider#onIgniteStart was not violated. But in
2.0 the GridPluginComponent instances are explicitly skipped from this
cycle (lines 1019-1020 in Ignite 2.0 release source) and PluginProviders
are notified before the internal component callbacks.

As far as I can see the change, that moved PluginProvider#onIgniteStart
notification before component callbacks, was introduced by this commit -

https://github.com/apache/ignite/commit/6b7bf97158c097b80bcf5c2150e67a

5210269e6d

- but I have no clue what was the reason.


On 7/20/2017 7:51 PM, Dmitry Pavlov wrote:

Hi Nick,

Thank you for your comment. Was onIgniteStart called after

onKernalStart in

1.9? Or caches were available, but other of initialization was the

same?

Sincerely.
Dmitriy Pavlov

ср, 19 июл. 2017 г. в 17:06, Nick Pordash :


Hi Dmitriy,

The ticket was a regression from 1.9 to 2.0. I don't think anyone

would be

expecting the behavior in 2.0 as it doesn't align with the javadoc

and

has

only been broken since the 2.0 release.

-Nick

On Wed, Jul 19, 2017, 6:55 AM Dmitry Pavlov 

wrote:

Hi Evgeniy,

Thank you. Ignite Basic is one suite from approximately 80 suites

that

covers Ignite by automated tests. Which is why I suggested to use

RunAll

chain in ignite 2.0 group. Yes, several tests may fail, especially

if

it

is

flaky tests or failure is related to the specific JIRA issue.

About change itself: This change seems to be very impact. There is
possiblity that many of existing plugins relies on existing order of
initialization. This change may break plugin initialization in

unexpected

manner.

Could we
- fix javadoc according to existing order in code
- find out new solution?

Sincerely,
Dmitriy Pavlov


ср, 19 июл. 2017 г. в 16:40, Evgeniy Ignatiev <

yevgeniy.ignat...@gmail.com

:


http://ci.ignite.apache.org/viewLog.html?buildId=720722;

tab=buildResultsDiv=IgniteTests_IgniteBasic

- this one - there seem to be no new failed platform tests, other

failed

tests seem to fail in several other reviews too and are unrelated

to

my

changes.


On 19.07.2017 17:35, Dmitry Pavlov wrote:

Hi Evgeniy,

I was not able to find Teamcity run for this change.
Could you please run http://ci.ignite.apache.org test for example

on

branch

pull/2285/head using 'Ignite 2.0 Tests' target 'Run All'.
Or could you please share link to previous run on this changes?

Sincerely,
Dmitriy Pavlov

ср, 19 июл. 2017 г. в 15:26, Anton Vinogradov :


Igniters,

Could somebody review the fix today?

On Wed, Jul 19, 2017 at 1:30 PM, Evgeniy Ignatiev <
yevgeniy.ignat...@gmail.com> wrote:


Hello, Igniters.

Could anyone review my request - https://issues.apache.org/jira
/browse/IGNITE-5123? - My previous pings seems to got lost.

Best regards,

Yevgeniy

[jira] [Created] (IGNITE-5813) Inconsistent cache store state when originating node fails on commit.

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-5813:


 Summary: Inconsistent cache store state when originating node 
fails on commit.
 Key: IGNITE-5813
 URL: https://issues.apache.org/jira/browse/IGNITE-5813
 Project: Ignite
  Issue Type: Bug
  Components: cache
Reporter: Andrew Mashenkov
 Fix For: 2.2


Same tx can be rolled back by cache and commited by CacheStore.
It is possible when originating tx node commit tx successfully, but fails to 
notify other nodes. 

PFA reproducer.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: IGNITE-5123 Review

[Semyon Boikov]

Evgeniy and Dmitry, thanks for the fix! Merged into master.

Thanks!

On Sun, Jul 23, 2017 at 9:21 PM, Dmitry Pavlov 
wrote:

> Hi Evgeniy,
>
> From my point of view there are no problems with this fix. My testing
> didn't show any issues with fix.
>
> Igniters,
>
> Are there any additional comments on this issue? Can we proceed?
>
> Sincerely,
> Dmitriy Pavlov
>
> чт, 20 июл. 2017 г. в 20:04, Dmitry Pavlov :
>
> > Hi Evgeniy,
> >
> > Thank you for such a careful research of the issue.
> >
> > If don’t mind, I would like to do additional tests with this PR changes.
> >
> > I will come back with result in couple of days
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > чт, 20 июл. 2017 г. в 19:18, Evgeniy Ignatiev <
> yevgeniy.ignat...@gmail.com
> > >:
> >
> >> onIgniteStart was called in Ignite 1.X in
> >> GridPluginComponent#onKernalStart as one of the calls to the component
> >> callbacks, probably the order in which components were called, ensured
> >> that contract of PluginProvider#onIgniteStart was not violated. But in
> >> 2.0 the GridPluginComponent instances are explicitly skipped from this
> >> cycle (lines 1019-1020 in Ignite 2.0 release source) and PluginProviders
> >> are notified before the internal component callbacks.
> >>
> >> As far as I can see the change, that moved PluginProvider#onIgniteStart
> >> notification before component callbacks, was introduced by this commit -
> >>
> >> https://github.com/apache/ignite/commit/6b7bf97158c097b80bcf5c2150e67a
> 5210269e6d
> >> - but I have no clue what was the reason.
> >>
> >>
> >> On 7/20/2017 7:51 PM, Dmitry Pavlov wrote:
> >> > Hi Nick,
> >> >
> >> > Thank you for your comment. Was onIgniteStart called after
> >> onKernalStart in
> >> > 1.9? Or caches were available, but other of initialization was the
> same?
> >> >
> >> > Sincerely.
> >> > Dmitriy Pavlov
> >> >
> >> > ср, 19 июл. 2017 г. в 17:06, Nick Pordash :
> >> >
> >> >> Hi Dmitriy,
> >> >>
> >> >> The ticket was a regression from 1.9 to 2.0. I don't think anyone
> >> would be
> >> >> expecting the behavior in 2.0 as it doesn't align with the javadoc
> and
> >> has
> >> >> only been broken since the 2.0 release.
> >> >>
> >> >> -Nick
> >> >>
> >> >> On Wed, Jul 19, 2017, 6:55 AM Dmitry Pavlov 
> >> wrote:
> >> >>
> >> >>> Hi Evgeniy,
> >> >>>
> >> >>> Thank you. Ignite Basic is one suite from approximately 80 suites
> that
> >> >>> covers Ignite by automated tests. Which is why I suggested to use
> >> RunAll
> >> >>> chain in ignite 2.0 group. Yes, several tests may fail, especially
> if
> >> it
> >> >> is
> >> >>> flaky tests or failure is related to the specific JIRA issue.
> >> >>>
> >> >>> About change itself: This change seems to be very impact. There is
> >> >>> possiblity that many of existing plugins relies on existing order of
> >> >>> initialization. This change may break plugin initialization in
> >> unexpected
> >> >>> manner.
> >> >>>
> >> >>> Could we
> >> >>> - fix javadoc according to existing order in code
> >> >>> - find out new solution?
> >> >>>
> >> >>> Sincerely,
> >> >>> Dmitriy Pavlov
> >> >>>
> >> >>>
> >> >>> ср, 19 июл. 2017 г. в 16:40, Evgeniy Ignatiev <
> >> >> yevgeniy.ignat...@gmail.com
> >>  :
> >> 
> >> >>
> >> http://ci.ignite.apache.org/viewLog.html?buildId=720722;
> tab=buildResultsDiv=IgniteTests_IgniteBasic
> >>  - this one - there seem to be no new failed platform tests, other
> >> >> failed
> >>  tests seem to fail in several other reviews too and are unrelated
> to
> >> my
> >>  changes.
> >> 
> >> 
> >>  On 19.07.2017 17:35, Dmitry Pavlov wrote:
> >> > Hi Evgeniy,
> >> >
> >> > I was not able to find Teamcity run for this change.
> >> > Could you please run http://ci.ignite.apache.org test for example
> >> on
> >>  branch
> >> > pull/2285/head using 'Ignite 2.0 Tests' target 'Run All'.
> >> > Or could you please share link to previous run on this changes?
> >> >
> >> > Sincerely,
> >> > Dmitriy Pavlov
> >> >
> >> > ср, 19 июл. 2017 г. в 15:26, Anton Vinogradov :
> >> >
> >> >> Igniters,
> >> >>
> >> >> Could somebody review the fix today?
> >> >>
> >> >> On Wed, Jul 19, 2017 at 1:30 PM, Evgeniy Ignatiev <
> >> >> yevgeniy.ignat...@gmail.com> wrote:
> >> >>
> >> >>> Hello, Igniters.
> >> >>>
> >> >>> Could anyone review my request - https://issues.apache.org/jira
> >> >>> /browse/IGNITE-5123? - My previous pings seems to got lost.
> >> >>>
> >> >>> Best regards,
> >> >>>
> >> >>> Yevgeniy
> >> >>>
> >> >>>
> >> 
> >>
> >>
>

[CANCEL] [VOTE] Apache Ignite 2.1.0 RC3

[Anton Vinogradov]

Officially cancelling the vote according to found sign issues.

*Same *files will be signed again with correct signature and proposed at
new vote.

Re: [VOTE] Apache Ignite 2.1.0 RC3

[Anton Vinogradov]

This vote is closed.

On Sat, Jul 22, 2017 at 11:26 PM, Konstantin Boudnik  wrote:

> Retracting this, found the KEYS (douh...). Still
>
> -1 (binding). The release isn't signed by the release manager. Someone else
> key is used.
>
> - Checked the sha1
> - Successfully ran the build
> - Checked the signature
> - The archive is signed by the key 593A743B belonging to
> sboi...@apache.org.
>   However, none of the 2.1.0 RC [VOTE] attempts were started by this
> person.
>   Which tells me that the private key is simply shared by a number of the
>   committers. And there's no guarantee that it hasn't been leaked outside
> of
>   the group. And that's pretty serious security flaw, actually.
>
>   Why the release managers aren't using their own keys? It is easy to
> generate
>   and sign the keys following guidelines [1]. Committers' keys are easy to
>   validate against the Apache repository [2]
>
> Things that need to be improved in the next release:
> - neither sha1 nor md5 are trustful checksum'ing methods and aren't
>   guaranteeing the authenticity of the source archive. We should be
> switching
>   to at least sha265 or higher. This has been brought up since the
> incubation.
>   And warrants for -1 in the next release.
> - why every other RC Vote is started by a different person?
>
> With regards,
>   Cos
>
> [1] https://people.apache.org/keys/committer/
> [2] https://www.apache.org/dev/new-committers-guide.html#set-
> up-security-and-pgp-keys
>
> On Sat, Jul 22, 2017 at 01:00PM, Konstantin Boudnik wrote:
> > Am I missing the location of the signing keys? I cannot verivy the
> signature
> > of the archive.
> >
> > -1 (binding) until then.
> >
> > Thanks
> >   Cos
> >
> > On Thu, Jul 20, 2017 at 03:34PM, Denis Magda wrote:
> > > Igniters,
> > >
> > > Setting off the vote one more time. Hope I’ll be successful this time,
> keeping fingers crossed :)
> > >
> > > We have uploaded a 2.1.0 release candidate to
> > > https://dist.apache.org/repos/dist/dev/ignite/2.1.0-rc3/
> > >
> > > Git tag name is
> > > 2.1.0-rc3
> > >
> > > This release includes the following changes:
> > >
> > > Ignite:
> > > * Persistent cache store
> > > * Added IgniteFuture.listenAsync() and IgniteFuture.chainAsync()
> mehtods
> > > * Deprecated IgniteConfiguration.marshaller
> > > * Updated Lucene dependency to version 5.5.2
> > > * Machine learning: implemented K-means clusterization algorithm
> optimized
> > > for distributed storages
> > > * SQL: CREATE TABLE and DROP TABLE commands support
> > > * SQL: New thin JDBC driver
> > > * SQL: Improved performance of certain queries, when affinity node can
> be
> > > calculated in advance
> > > * SQL: Fixed return type of AVG() function
> > > * SQL: BLOB type support added to thick JDBC driver
> > > * SQL: Improved LocalDate, LocalTime and LocalDateTime support for
> Java 8
> > > * SQL: Added FieldsQueryCursor interface to get fields metadata for
> > > SqlFieldsQuery
> > > * ODBC: Implemented DML statement batching
> > > * Massive performance and stability improvements
> > >
> > > Ignite.NET:
> > > * Automatic remote assembly loading
> > > * NuGet-based standalone node deployment
> > > * Added conditional data removeal via LINQ DeleteAll
> > > * Added TimestampAttribute to control DateTime serialization mode
> > > * Added local collections joins support to LINQ.
> > >
> > > Ignite CPP:
> > > * Added Compute::Call and Compute::Broadcast methods
> > >
> > > Web Console:
> > > * Implemented support for UNIQUE indexes for key fields on import model
> > > from RDBMS
> > > * Added option to show full stack trace on Queries screen
> > > * Added PK alias generation on Models screen.
> > >
> > > Complete list of closed issues:
> > > https://issues.apache.org/jira/issues/?jql=project%20%
> 3D%20IGNITE%20AND%
> > > 20fixVersion%20%3D%202.1%20AND%20(status%20%3D%
> 20closed%20or%20status%20%3D%
> > > 20resolved)
> > >
> > > DEVNOTES
> > > https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_
> plain;f=DEVNOTES.txt;hb=refs/tags/2.1.0-rc3
> > >
> > > RELEASE NOTES
> > > https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_
> plain;f=RELEASE_NOTES.txt;hb=refs/tags/2.1.0-rc3
> > >
> > > Please start voting.
> > >
> > > +1 - to accept Apache Ignite 2.1.0-rc3
> > > 0 - don't care either way
> > > -1 - DO NOT accept Apache Ignite 2.1.0-rc3 (explain why)
> > >
> > > This vote will go for 72 hours.
> > >
> > > —
> > > Denis
> > >
>
>
>

[jira] [Created] (IGNITE-5812) Implement decorator to auto adjust dopdown width to parent

[Dmitriy Shabalin (JIRA)]

Dmitriy Shabalin created IGNITE-5812:


 Summary: Implement decorator to auto adjust dopdown width to parent
 Key: IGNITE-5812
 URL: https://issues.apache.org/jira/browse/IGNITE-5812
 Project: Ignite
  Issue Type: Improvement
  Components: UI, wizards
Reporter: Dmitriy Shabalin
Assignee: Dmitriy Shabalin
 Fix For: 2.2






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Timeouts in atomic cache

[Yakov Zhdanov]

Val, it seems you spotted and issue. Please file a ticket - I would suggest
to remove the exceptions entirely as in my understanding timeout logic for
atomic operation will bring additional overhead, but most of the time
atomic operations are instant. From timeout perspective, what differs
atomic operation from a transaction is that you cannot predict when user
releases lock he acquired inside a transaction, but atomic operation should
have predictable timeout.

As far as your example. Currently, this will lead to java-level deadlock on
synchronized sections for the cache entries (but when we move to pure
thread-per-partition for atomic caches this will not be an issue any more
https://issues.apache.org/jira/browse/IGNITE-4506). I would suggest we file
a ticket to implement detection of java-level deadlock and allow user to
configure policy to take appropriate action on deadlock wherever it happens
- https://issues.apache.org/jira/browse/IGNITE-5811

Any other hang of the atomic operation seem to be caused by issues in
Ignite's internal machinery - either hanged exchange or problems in message
processing on some node (e.g. all threads are busy and/or in deadlock)
which again should result in notifying user and stopping node (by default).

--Yakov

[jira] [Created] (IGNITE-5811) Detect java-level deadlock and act according to a policy configured.

[Yakov Zhdanov (JIRA)]

Yakov Zhdanov created IGNITE-5811:
-

 Summary: Detect java-level deadlock and act according to a policy 
configured.
 Key: IGNITE-5811
 URL: https://issues.apache.org/jira/browse/IGNITE-5811
 Project: Ignite
  Issue Type: New Feature
Reporter: Yakov Zhdanov


This has something in common with segmentation policy we currently have. User 
should get notified on a deadlock problem and node should take an action (stop 
by default)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Ignite internal events tracing

[Yakov Zhdanov]

Alex, I like the idea very much, but I think we need to rethink the
implementation approach to make it more generic. Passing parameter to each
invocation seems dirty to me.

Val,  we already have this. Please
see org.apache.ignite.internal.IgniteDiagnosticAware

Dmitry, what you suggest will be pretty hard to implement. I would better
improve self-diagnostic system to extend list of metrics Ignite monitors.

--Yakov

[jira] [Created] (IGNITE-5810) Implement configuration import to xml

[Yakov Zhdanov (JIRA)]

Yakov Zhdanov created IGNITE-5810:
-

 Summary: Implement configuration import to xml
 Key: IGNITE-5810
 URL: https://issues.apache.org/jira/browse/IGNITE-5810
 Project: Ignite
  Issue Type: New Feature
Reporter: Yakov Zhdanov


When investigating failures having the entire grid configuration is a must. Now 
Ignite lacks this feature which would be very cool for deployments configured 
from code.

I suggest to implement {{public String Ignite.configurationAsXml();}} and add 
ability to output files and gather them to MBeans and web console.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Changing public IgniteCompute API to improve changes in 5037 ticket

[Anton Vinogradov]

Val,

> What is the use case for which current affinityRun/Call API doesn't work?
It does not work for map/reduce.

On Fri, Jul 21, 2017 at 11:42 PM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Maxim,
>
> The issue is that it's currently assumed to support job mapping, but it
> actually doesn't. However, I agree that AffinityKeyMapped annotation
> doesn't fit the use case well. Let's fix documentation and JavaDoc then.
>
> As for the proposed API, it's overcomplicated, took me 15 minutes to
> understand what it does :)
>
> What is the use case for which current affinityRun/Call API doesn't work?
>
> -Val
>
> On Fri, Jul 21, 2017 at 5:57 AM, Kozlov Maxim 
> wrote:
>
> > Valentin,
> >
> > The author of tiket wants to see to provide some API allows to map
> > ComputeJobs to partitions or keys. If we use @AffinityKeyMapped then you
> > need to enter the cache name parameter, I think this is not convenient
> for
> > the user. Therefore, I propose to extend the existing API.
> > Having consulted with Anton V. decided to make a separate interface
> > ReducibleTask, which will allow us to have different map logic at each
> > inheritor.
> >
> > Old method, allows to map to node
> > public interface ComputeTask extends ReducibleTask {
> > @Nullable public Map
> > map(List subgrid, @Nullable T arg) throws IgniteException;
> > }
> >
> > Brand new method with mapping to partitions, which solves topology change
> > issues.
> > public interface AffinityComputeTask extends ReducibleTask {
> > @Nullable public Map
> map(@NotnullString
> > cacheName, List partIds, @Nullable T arg) throws
> IgniteException;
> > }
> >
> > public interface ReducibleTask extends Serializable {
> > public ComputeJobResultPolicy result(ComputeJobResult res,
> > List rcvd) throws IgniteException;
> >
> > @Nullable public R reduce(List results) throws
> > IgniteException;
> > }
> >
> > We also need to implement AffinityComputeTaskAdapter and
> > AffinityComputeTaskSplitAdapter, for implementation by default. It is
> > right?
> >
> > In the IgniteCompute add:
> >
> > @IgniteAsyncSupported
> > public  R affinityExecute(Class R>>
> > taskCls, List partIds, @Nullable T arg) throws IgniteException;
> > @IgniteAsyncSupported
> > public  R affinityExecute(AffinityComputeTask task,
> > List partIds, @Nullable T arg) throws IgniteException;
> >
> > public  ComputeTaskFuture affinityExecuteAsync(Class > AffinityComputeTask> taskCls, List partIds, @Nullable T
> arg)
> > throws IgniteException;
> > public  ComputeTaskFuture affinityExecuteAsync(
> AffinityComputeTask > R> task, List partIds, @Nullable T arg) throws IgniteException;
> >
> >
> > How do you like this idea or do you insist that you need to use
> > @AffinityKeyMapped to solve the problem?
> >
> >
> > > 13 июля 2017 г., в 6:36, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> написал(а):
> > >
> > > Hi Max,
> > >
> > > This ticket doesn't assume any API changes, it's about broken
> > > functionality. I would start with checking what tests we have
> > > for @AffinityKeyMapped and creating missing one. From what I understand
> > > functionality is broken completely or almost completely, so I guess
> > testing
> > > coverage is very weak there.
> > >
> > > -Val
> > >
> > > On Wed, Jul 12, 2017 at 4:27 PM, Kozlov Maxim 
> > wrote:
> > >
> > >> Igniters,
> > >>
> > >> jira: https://issues.apache.org/jira/browse/IGNITE-5037 <
> > >> https://issues.apache.org/jira/browse/IGNITE-5037>
> > >> How do you look to solve this ticket by adding two methods to the
> public
> > >> IgniteCompute API?
> > >>
> > >> @IgniteAsyncSupported
> > >> public void affinityRun(@NotNull Collection cacheNames,
> > >> Collection keys, IgniteRunnable job)
> > >>throws IgniteException;
> > >>
> > >> @IgniteAsyncSupported
> > >> public  R affinityCall(@NotNull Collection cacheNames,
> > >> Collection keys, IgniteCallable job)
> > >>throws IgniteException;
> > >>
> > >> There is also a question of how to act when changing the topology
> during
> > >> the execution of the job.
> > >> 1) complete with an exception;
> > >> 2) stop execution and wait until the topology is rebuilt and continue
> > >> execution;
> > >>
> > >> I think the second way, do you think?
> > >>
> > >> --
> > >> Best Regards,
> > >> Max K.
> > >>
> > >>
> > >>
> > >>
> > >>
> >
> > --
> > Best Regards,
> > Max K.
> >
> >
> >
> >
> >
>

[GitHub] ignite pull request #2338: IGNITE-5807 added new profile to fix artifacts si...

[oleg-ostanin]

GitHub user oleg-ostanin opened a pull request:

https://github.com/apache/ignite/pull/2338

IGNITE-5807 added new profile to fix artifacts signing, edited DEVNOT…

…ES.txt

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5807

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2338.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2338


commit 6ea8a8262b847d4180e16567d5a8a0973bed6562
Author: oleg-ostanin 
Date:   2017-07-22T10:08:41Z

IGNITE-5807 added new profile to fix artifacts signing, edited DEVNOTES.txt




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-5809) SQL: Add precision and scale support for table fields.

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-5809:


 Summary: SQL: Add precision and scale support for table fields.
 Key: IGNITE-5809
 URL: https://issues.apache.org/jira/browse/IGNITE-5809
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Andrew Mashenkov
 Fix For: 2.2


PFA test.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: "not null" constraint support

[Pavel Tupitsyn]

> QueryEntity is already too complex
> let's make it even more complex

We already see that String-based approach is bad:
* LinkedHashMap fields
* Set keyFields
* Map aliases

Which should have been just QueryField { name, isKey, alias }.
Let's learn from our mistakes and do constraints right.


> we will deprecate it soon

This is not a valid argument.
What is "soon"? 3.0 is not soon, and I think we don't deprecate APIs in
minor releases.
Temporary things tend to become permanent very often.

Thanks,
Pavel


On Fri, Jul 21, 2017 at 9:57 PM, Vladimir Ozerov 
wrote:

> Guys,
>
> QueryEntity is already too complex. We will deprecate it soon in favor of
> brand new SQL API. No need to design FieldConstraint or something similar
> at the moment, let's just stick to "Set notNullFields".
>
> On Fri, Jul 21, 2017 at 7:08 PM, Sergey Chugunov <
> sergey.chugu...@gmail.com>
> wrote:
>
> > Sergey,
> >
> > It may be a good idea to distinguish between field constraints (like "not
> > null" one) which can be applied to only one field; and more complex
> > constraints that involves more than one field.
> >
> > In case of field constraints it is better to simplify our model and allow
> > only one field to appear inside constraint entity class.
> >
> > Something like this:
> > class QueryEntity {
> > ...
> >
> > /** All constraints to be enforced on this QueryEntity. */
> > Set fieldConstraints;
> > }
> >
> > /** Describes all constraints that affect the field. */
> > class FieldConstraint {
> > /** Field name to be examined against all its constraints. */
> > String fieldName;
> >
> > /** Indicates whether check for null is required. */
> > boolean notNull;
> >
> > ... here we would add more flags corresponding to other constraints
> ...
> > }
> >
> > This model has a flaw that if we want to enforce the same constraint on
> > many fields we must define FieldConstraint entity for each of these
> fields.
> > But it has its advantages too: first of all, its simplicity and therefore
> > obvious usage patterns; and easy to implement validation.
> >
> > Thanks,
> > Sergey Ch.
> >
> >
> > On Fri, Jul 21, 2017 at 6:43 PM, Pavel Tupitsyn 
> > wrote:
> >
> > > Sergey, looks good to me!
> > >
> > > I thought of something a bit different, where there is a base class
> > > and each constraint type inherits it, but your design is actually
> better.
> > >
> > > Pavel
> > >
> > > On Fri, Jul 21, 2017 at 6:38 PM, Sergey Kalashnikov <
> > zkilling...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hi Pavel,
> > > >
> > > > Good point! What if we make it the following way?
> > > >
> > > > class QueryEntity {
> > > > ...
> > > >
> > > > /** All constraints to be enforced on this QueryEntity. */
> > > > Set constraint;
> > > > }
> > > >
> > > > /** Describes constraints that affect one or more fields. */
> > > > class QueryConstraint {
> > > > /** Names of fields to be checked. */
> > > > Set fields;
> > > >
> > > > /** Indicates whether check for null is required. */
> > > > boolean notNull;
> > > >
> > > > ... here we would add more flags corresponding to other
> constraints
> > > ...
> > > > }
> > > >
> > > > --
> > > > Best Regards,
> > > > Sergey
> > > >
> > > >
> > > > 2017-07-21 10:54 GMT+03:00 Pavel Tupitsyn :
> > > > > Hi Sergey,
> > > > >
> > > > > This one looks not very good to me:
> > > > >
> > > > >> class QueryEntity {
> > > > >> ...
> > > > >> Set notNullFields;
> > > > >> }
> > > > >
> > > > > What if there are more constraints in future? UNIQUE, CHECK, etc
> etc?
> > > > >
> > > > > Instead we could do something like
> > > > >
> > > > > Set constraints;
> > > > >
> > > > > Which is easily extendable in future.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > Pavel
> > > > >
> > > > > On Thu, Jul 20, 2017 at 11:17 PM, Denis Magda 
> > > wrote:
> > > > >
> > > > >> Hi Sergey,
> > > > >>
> > > > >> That will be an excellent addition to DDL features set.
> > > > >>
> > > > >> The proposed changes look good to from the public API standpoint.
> > > > >>
> > > > >> Alexander P., Sergi, Vovan, please share your thoughts.
> > > > >>
> > > > >> —
> > > > >> Denis
> > > > >>
> > > > >> > On Jul 20, 2017, at 12:27 PM, Sergey Kalashnikov <
> > > > zkilling...@gmail.com>
> > > > >> wrote:
> > > > >> >
> > > > >> > Hi Igniters,
> > > > >> >
> > > > >> > I am working on the ticket
> > > > >> > https://issues.apache.org/jira/browse/IGNITE-5648, which
> purpose
> > is
> > > > to
> > > > >> > add support for NOT NULL constraint for any fields of key or
> value
> > > > >> > stored in Ignite cache.
> > > > >> >
> > > > >> > I would appreciate your comments on the design approach I have
> > > > described
> > > > >> below.
> > > > >> >
> > > > >> > In my opinion, such checks should be enforced both by SQL DML
> and
> > > > >> > cache API to be consistent.
> > 

[GitHub] ignite pull request #2242: Ignite 1.8.8 p1

[ezhuravl]

Github user ezhuravl closed the pull request at:

https://github.com/apache/ignite/pull/2242


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2323: Ignite 1.7.4 p2

[ezhuravl]

Github user ezhuravl closed the pull request at:

https://github.com/apache/ignite/pull/2323


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2263: IGNITE-5467 Exception in communication SPI can st...

[ezhuravl]

Github user ezhuravl closed the pull request at:

https://github.com/apache/ignite/pull/2263


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---