[jira] [Created] (IGNITE-6833) Web Console: runtime errors after each SCSS save

2017-11-06 Thread Ilya Borisov (JIRA) 
Ilya Borisov created IGNITE-6833:


 Summary: Web Console: runtime errors after each SCSS save
 Key: IGNITE-6833
 URL: https://issues.apache.org/jira/browse/IGNITE-6833
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
  Components: wizards
 Environment: Windows 10, MacOS, master branch
Reporter: Ilya Borisov
Assignee: Alexander Kalinin


*What happens:*
Web console frontend throws runtime errors after every SCSS file save. You have 
to save any non-stylesheet file to fix the error.

*How to reproduce:*
1. Checkout master.
2. Start frontend development environment.
3. Save any SCSS file, no changes necessary.
4. Open local web console, open browser console, see the error.

*Expected behavior:*
No errors should happen after SCSS file saves.

*Note:*
I had a hand at this earlier: the error always mentions missing global 
dependency of some sort (i.e. _.head). If you replace all implicit lodash uses 
with explicit imports in every file, error eventually starts to point at 
polyfills (i.e. RegExp.esacpe). Maybe this will be of help, I did not progress 
any further.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Deserialization of BinaryObject's byte arrays

2017-11-06 Thread Vladimir Ozerov 
ByteBuffer is not fundamental data type. It is a thin wrapper over array.
Protocol should contain only fundamental types.

вт, 7 нояб. 2017 г. в 10:05, Andrey Kornev :

> Vladimir,
>
> Could you please elaborate as to why adding ByteBuffer to the Binary
> protocol is “not a good idea”? What is inherently wrong about it?
>
> As for your suggestion, while I certainly see your point, it comes with
> the inconvenience of implementing Binarylizable for every entity type in an
> application, which makes me wonder if the self imposed rules such as “type
> system should be kept as small as possible” could be relaxed in the name of
> greater good and user friendliness. Ignite API is not easy to grok as it is
> already.
>
> Thanks
> Andrey
> _
> From: Vladimir Ozerov >
> Sent: Monday, November 6, 2017 10:22 PM
> Subject: Re: Deserialization of BinaryObject's byte arrays
> To: >
> Cc: Valentin Kulichenko >
>
>
> Hi Andrey,
>
> While we deifnitely need to support byte buffers, adding new type to
> protocol is not very good idea. Binary protocol is the heart of our ssytem,
> which is used not oly for plain Java, but for other platforms and SQL. And
> ByteBuffer is essentially the same byte array, but with different API. What
> you describe is a special case of byte array read/write. For this reason
> our type system should be kept as small as possible and it doesn't require
> new type to be introduced. Instead, we'd better to handle everything on API
> level only. For example, we can add new methods to reader/writer/builder,
> which will handle this. E.g.:
>
> BinaryWriter.writeByteArray();
> BinaryReader.readByteArray();
>
> Vladimir.
>
> On Mon, Nov 6, 2017 at 9:25 PM, Andrey Kornev  >
> wrote:
>
> > Will do! Thanks!
> > 
> > From: Valentin Kulichenko >
> > Sent: Monday, November 6, 2017 9:17 AM
> > To: dev@ignite.apache.org
> > Subject: Re: Deserialization of BinaryObject's byte arrays
> >
> > This use case was discussed couple of times already, so it seems to be
> > pretty important for users. And I like the approach suggested by Andrey.
> >
> > Andrey, can you create a Jira ticket for this change? Would you like to
> > contribute it?
> >
> > -Val
> >
> > On Fri, Nov 3, 2017 at 4:18 PM, Dmitriy Setrakyan  >
> > wrote:
> >
> > > This makes sense to me. Sounds like a useful feature.
> > >
> > > Would be nice to hear what others in the community think?
> > >
> > > D.
> > >
> > > On Fri, Nov 3, 2017 at 1:07 PM, Andrey Kornev <
> andrewkor...@hotmail.com>
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > We store lots of byte arrays (serialized graph-like data structures)
> as
> > > > fields of BinaryObject. Later on, while reading such field,
> > > > BinaryInputStream implementation creates an on-heap array and copies
> > the
> > > > bytes from the BinaryObject's internal backing array to the new
> array.
> > > >
> > > > While in general case it works just fine, in our case, a lot of CPU
> is
> > > > spent on copying of the bytes and significant amount garbage is
> > generated
> > > > as well.
> > > >
> > > > I'd like Ignite Community to consider the following proposal:
> introduce
> > > > support for serialization/deserialization of the ByteBuffer class. A
> > > > BinaryInputStream implementation would special case ByteBuffer fields
> > > same
> > > > way it currently special cases various Java types (collections,
> dates,
> > > > timestamps, enums, etc.)
> > > >
> > > > The application developer would simply call
> > > BinaryObjectBuilder.setField(...)
> > > > passing in an instance of ByteBuffer. During serialization, the
> > > > ByteBuffer's bytes would simply be copied to the backing array
> (similar
> > > to
> > > > how regular byte arrays are serialized today) and in the binary
> > object's
> > > > metadata the field marked as, say, GridBinaryMarshaller.BYTE_BUFFER.
> > > >
> > > > During deserialization of the field, instead of allocating a new byte
> > > > array and transferring the bytes from the BinaryObject's backing
> array,
> > > > BinaryInputStream would simply wrap the required sub-array into a
> > > read-only
> > > > instance of ByteBuffer using "public static ByteBuffer wrap(byte[]
> > array,
> > > > int offset, int length)" and return the instance to the application.
> > > >
> > > > This approach doesn't require any array allocation/data copying and
> is
> > > > faster and significantly reduces pressure on the garbage collector.
> The
> > > > benefits are especially great when application stores significant
> > amount
> >

Re: Deserialization of BinaryObject's byte arrays

2017-11-06 Thread Andrey Kornev 
Vladimir,

Could you please elaborate as to why adding ByteBuffer to the Binary protocol 
is “not a good idea”? What is inherently wrong about it?

As for your suggestion, while I certainly see your point, it comes with the 
inconvenience of implementing Binarylizable for every entity type in an 
application, which makes me wonder if the self imposed rules such as “type 
system should be kept as small as possible” could be relaxed in the name of 
greater good and user friendliness. Ignite API is not easy to grok as it is 
already.

Thanks
Andrey
_
From: Vladimir Ozerov >
Sent: Monday, November 6, 2017 10:22 PM
Subject: Re: Deserialization of BinaryObject's byte arrays
To: >
Cc: Valentin Kulichenko 
>


Hi Andrey,

While we deifnitely need to support byte buffers, adding new type to
protocol is not very good idea. Binary protocol is the heart of our ssytem,
which is used not oly for plain Java, but for other platforms and SQL. And
ByteBuffer is essentially the same byte array, but with different API. What
you describe is a special case of byte array read/write. For this reason
our type system should be kept as small as possible and it doesn't require
new type to be introduced. Instead, we'd better to handle everything on API
level only. For example, we can add new methods to reader/writer/builder,
which will handle this. E.g.:

BinaryWriter.writeByteArray();
BinaryReader.readByteArray();

Vladimir.

On Mon, Nov 6, 2017 at 9:25 PM, Andrey Kornev 
>
wrote:

> Will do! Thanks!
> 
> From: Valentin Kulichenko 
> >
> Sent: Monday, November 6, 2017 9:17 AM
> To: dev@ignite.apache.org
> Subject: Re: Deserialization of BinaryObject's byte arrays
>
> This use case was discussed couple of times already, so it seems to be
> pretty important for users. And I like the approach suggested by Andrey.
>
> Andrey, can you create a Jira ticket for this change? Would you like to
> contribute it?
>
> -Val
>
> On Fri, Nov 3, 2017 at 4:18 PM, Dmitriy Setrakyan 
> >
> wrote:
>
> > This makes sense to me. Sounds like a useful feature.
> >
> > Would be nice to hear what others in the community think?
> >
> > D.
> >
> > On Fri, Nov 3, 2017 at 1:07 PM, Andrey Kornev 
> > >
> > wrote:
> >
> > > Hello,
> > >
> > > We store lots of byte arrays (serialized graph-like data structures) as
> > > fields of BinaryObject. Later on, while reading such field,
> > > BinaryInputStream implementation creates an on-heap array and copies
> the
> > > bytes from the BinaryObject's internal backing array to the new array.
> > >
> > > While in general case it works just fine, in our case, a lot of CPU is
> > > spent on copying of the bytes and significant amount garbage is
> generated
> > > as well.
> > >
> > > I'd like Ignite Community to consider the following proposal: introduce
> > > support for serialization/deserialization of the ByteBuffer class. A
> > > BinaryInputStream implementation would special case ByteBuffer fields
> > same
> > > way it currently special cases various Java types (collections, dates,
> > > timestamps, enums, etc.)
> > >
> > > The application developer would simply call
> > BinaryObjectBuilder.setField(...)
> > > passing in an instance of ByteBuffer. During serialization, the
> > > ByteBuffer's bytes would simply be copied to the backing array (similar
> > to
> > > how regular byte arrays are serialized today) and in the binary
> object's
> > > metadata the field marked as, say, GridBinaryMarshaller.BYTE_BUFFER.
> > >
> > > During deserialization of the field, instead of allocating a new byte
> > > array and transferring the bytes from the BinaryObject's backing array,
> > > BinaryInputStream would simply wrap the required sub-array into a
> > read-only
> > > instance of ByteBuffer using "public static ByteBuffer wrap(byte[]
> array,
> > > int offset, int length)" and return the instance to the application.
> > >
> > > This approach doesn't require any array allocation/data copying and is
> > > faster and significantly reduces pressure on the garbage collector. The
> > > benefits are especially great when application stores significant
> amount
> > of
> > > its data as byte arrays.
> > >
> > > Clearly, this approach equally applies to arrays of other primitive
> > types.
> > >
> > > A minor complication does arise when dealing with off-heap
> BinaryObjects,
> > > but nothing that can't solved with a little bit of Java reflection
> > wizardry
> > > (or by other means).
> > >
> > > Thanks,
> > > Andrey
> > >
> >
>

Re: Deserialization of BinaryObject's byte arrays

2017-11-06 Thread Vladimir Ozerov 
Hi Andrey,

While we deifnitely need to support byte buffers, adding new type to
protocol is not very good idea. Binary protocol is the heart of our ssytem,
which is used not oly for plain Java, but for other platforms and SQL. And
ByteBuffer is essentially the same byte array, but with different API. What
you describe is a special case of byte array read/write. For this reason
our type system should be kept as small as possible and it doesn't require
new type to be introduced. Instead, we'd better to handle everything on API
level only. For example, we can add new methods to reader/writer/builder,
which will handle this. E.g.:

BinaryWriter.writeByteArray();
BinaryReader.readByteArray();

Vladimir.

On Mon, Nov 6, 2017 at 9:25 PM, Andrey Kornev 
wrote:

> Will do! Thanks!
> 
> From: Valentin Kulichenko 
> Sent: Monday, November 6, 2017 9:17 AM
> To: dev@ignite.apache.org
> Subject: Re: Deserialization of BinaryObject's byte arrays
>
> This use case was discussed couple of times already, so it seems to be
> pretty important for users. And I like the approach suggested by Andrey.
>
> Andrey, can you create a Jira ticket for this change? Would you like to
> contribute it?
>
> -Val
>
> On Fri, Nov 3, 2017 at 4:18 PM, Dmitriy Setrakyan 
> wrote:
>
> > This makes sense to me. Sounds like a useful feature.
> >
> > Would be nice to hear what others in the community think?
> >
> > D.
> >
> > On Fri, Nov 3, 2017 at 1:07 PM, Andrey Kornev 
> > wrote:
> >
> > > Hello,
> > >
> > > We store lots of byte arrays (serialized graph-like data structures) as
> > > fields of BinaryObject. Later on, while reading such field,
> > > BinaryInputStream implementation creates an on-heap array and copies
> the
> > > bytes from the BinaryObject's internal backing array to the new array.
> > >
> > > While in general case it works just fine, in our case, a lot of CPU is
> > > spent on copying of the bytes and significant amount garbage is
> generated
> > > as well.
> > >
> > > I'd like Ignite Community to consider the following proposal: introduce
> > > support for serialization/deserialization of the ByteBuffer class.  A
> > > BinaryInputStream implementation would special case ByteBuffer fields
> > same
> > > way it currently special cases various Java types (collections, dates,
> > > timestamps, enums, etc.)
> > >
> > > The application developer would simply call
> > BinaryObjectBuilder.setField(...)
> > > passing in an instance of ByteBuffer. During serialization, the
> > > ByteBuffer's bytes would simply be copied to the backing array (similar
> > to
> > > how regular byte arrays are serialized today) and in the binary
> object's
> > > metadata the field marked as, say, GridBinaryMarshaller.BYTE_BUFFER.
> > >
> > > During deserialization of the field, instead of allocating a new byte
> > > array and transferring the bytes from the BinaryObject's backing array,
> > > BinaryInputStream would simply wrap the required sub-array into a
> > read-only
> > > instance of ByteBuffer using "public static ByteBuffer wrap(byte[]
> array,
> > > int offset, int length)" and return the instance to the application.
> > >
> > > This approach doesn't require any array allocation/data copying and is
> > > faster and significantly reduces pressure on the garbage collector. The
> > > benefits are especially great when application stores significant
> amount
> > of
> > > its data as byte arrays.
> > >
> > > Clearly, this approach equally applies to arrays of other primitive
> > types.
> > >
> > > A minor complication does arise when dealing with off-heap
> BinaryObjects,
> > > but nothing that can't solved with a little bit of Java reflection
> > wizardry
> > > (or by other means).
> > >
> > > Thanks,
> > > Andrey
> > >
> >
>

[GitHub] ignite pull request #2984: IGNITE-6700 Node considered as failed can cause f...

2017-11-06 Thread akuramshingg 
GitHub user akuramshingg opened a pull request:

https://github.com/apache/ignite/pull/2984

IGNITE-6700 Node considered as failed can cause failure of others nodes

Independent asynchronous connection checkers for the previous and the next 
nodes.
TcpDiscoveryHandshakeResponse carries failed node and generates 
TcpDiscoveryNodeFailedMessage on a sender.
Remember the list of recently failed server nodes.
Synchronized access to sendMessageAcrossRing().
Local node freeze detection.

New TcpDiscoverySplitTest based on IgniteCacheTopologySplitAbstractTest.
CacheLateAffinityAssignmentTest and TcpDiscoverySelfTest update.

GridDhtPartitionTopologyImpl update: IGNITE-6433 workaround.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6700-new

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2984.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2984


commit f7875d21ed3364da87cab02b249a14af227a941b
Author: Alexandr Kuramshin 
Date:   2017-11-07T05:55:24Z

IGNITE-6700 Node considered as failed can cause failure of others nodes

Independent asynchronous connection checkers for the previous and the next 
nodes.
TcpDiscoveryHandshakeResponse carries failed node and generates 
TcpDiscoveryNodeFailedMessage on a sender.
Remember the list of recently failed server nodes.
Synchronized access to sendMessageAcrossRing().
Local node freeze detection.

New TcpDiscoverySplitTest based on IgniteCacheTopologySplitAbstractTest.
CacheLateAffinityAssignmentTest and TcpDiscoverySelfTest update.

GridDhtPartitionTopologyImpl update: IGNITE-6433 workaround.




---

[GitHub] ignite pull request #2942: disco update

2017-11-06 Thread akuramshingg 
Github user akuramshingg closed the pull request at:

https://github.com/apache/ignite/pull/2942


---

Re: Make Teamcity Green Again

2017-11-06 Thread Dmitry Pavlov 
"Ignored" means that the test is disabled from the code, using annotation
@Ignore. Not sure it works for our JUnit3 style Java tests. It is used for
.NET.
"Muted" test is disabled in the interface TeamCity, but test code is
actually executed (which is why hanging tests needs to be failed in
addition to mute).

The fact that the run all now became green means all the tests that fall at
this point are flaky. If it is about 200 non-muted flaky tests having the
probability of passing 98%, then the whole chain will pass only with the
probability 0.98 ^ 200 = 1.7%

I suggest continue to identify, fix, or mute such flaky tests to get stable
green RunAll for correct code.

пн, 6 нояб. 2017 г. в 22:57, Denis Magda :

> Dmitriy,
>
> Now it’s time to fix a pretty decent number of muted and ignored tests.
> BTW, what’s the difference between “ignored” and “muted”? I’ve seen the
> former before.
>
> —
> Denis
>
> > On Nov 5, 2017, at 9:58 AM, Dmitry Pavlov  wrote:
> >
> > Hi Igniters,
> >
> > I am happy to report that the activity of monitoring and fixing the tests
> > brought the first significant victory!
> >
> > Today tests launch running all Ignite tests (Ignite 2.0 Tests->RunAll) is
> > completely green:
> https://ci.ignite.apache.org/viewLog.html?buildId=930953
> >
> > This is an reason for being pride, but not a reason to stop this
> activity.
> > Unfortunately, there are still a number of tests with unpredictable
> > failures (flaky, 234 tests), and a number of tests have been muted. Full
> > project scope is available in the article
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Make+Teamcity+Green+Again
> >
> > In any case, thank to all of you who helped and continues to help the
> > project to correct the tests and bugs.
> >
> > Sincerely,
> > Pavlov Dmitry
> >
> >
> > пт, 21 июл. 2017 г. в 14:29, Pavel Tupitsyn :
> >
> >>> Green Again
> >>> Again
> >> As if it ever was green :)
> >>
> >> Of course +1 on this, let me know if you see any .NET-related failures.
> >>
> >> On Fri, Jul 21, 2017 at 1:06 PM, Николай Ижиков  >
> >> wrote:
> >>
> >>> Hello, Igniters.
> >>>
> >>> Also ready to help to #MakeTeamcityGreenAgain !
> >>>
> >>> 21 июля 2017 г. 12:56 PM пользователь "Vyacheslav Daradur" <
> >>> daradu...@gmail.com> написал:
> >>>
>  Hi guys.
> 
>  I vote for #MakeTeamcityGreenAgain. :-)
> 
>  FYI: it had been described and supported previously[1]
> 
>  After the completion of my current task I will try to help with this
>  activity.
> 
>  [1]
>  http://apache-ignite-developers.2346864.n4.nabble.
>  com/Test-failures-td14353.html
> 
>  2017-07-21 12:39 GMT+03:00 Anton Vinogradov :
> 
> > Nikolay,
> >
> > That's also a big problem for me, as reviewer, to accept changes and
>  merge
> > them to master.
> >
> > Dmitriy,
> >
> > Currently we have some contributions *from community* blocked by red
> > Teamcity.
> >
> > On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov <
>  nizhikov@gmail.com>
> > wrote:
> >
> >> +1 to Dmitry Pavlov from me.
> >>
> >> Green team city would be very helpful.
> >>
> >>
> >>> If you are a newcomer, please share your feeling: are you
> >> confused
> >>> by
> >> test
> >>> failures on Teamcity?
> >>
> >> I'm working on my very first ignite issue
> >> and it's not easy to understand TC build results.
> >>
> >>
> >> 21.07.2017 00:30, Dmitriy Setrakyan пишет:
> >>
> >> I think the right question to ask is why do we have RED tests to
> >>> begin
> >>> with. In my view, TC should be green. If there are tests that do
> >> not
> > pass,
> >>> we should disable them and file blocker tickets for the next
> >>> releases.
> >>>
> >>> D.
> >>>
> >>> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov <
> >>> dpavlov@gmail.com
> >
> >>> wrote:
> >>>
> >>> Hi Igniters,
> 
>  We all know that there are a lot of open tasks in Ignite project.
> >>> To
>  be
>  able to handle all these tasks, we need to grow our community.
> >> And
> >>> to
>  achieve that, we should pay special attention to new community
>  members.
> 
>  As recent newcomer I’ve faced the following problem: There is no
> >>> way
>  to
>  find out by yourself whether your pull request changes are
> >> correct
> >>> or
>  not.
>  Contributor can not distinguish pull request test failures and
> >>> master
>  test
>  failures. Experienced Ignite contributor can estimate the damage,
> >>> but
> > how
>  it can be done by new community member?
> 
>  Once failing test is introduced to master it is multiplexed to
> >> all
> > later
>  PRs. As a result, we have tons

Re: Make Teamcity Green Again

2017-11-06 Thread Denis Magda 
Dmitriy, 

Now it’s time to fix a pretty decent number of muted and ignored tests. BTW, 
what’s the difference between “ignored” and “muted”? I’ve seen the former 
before.

—
Denis

> On Nov 5, 2017, at 9:58 AM, Dmitry Pavlov  wrote:
> 
> Hi Igniters,
> 
> I am happy to report that the activity of monitoring and fixing the tests
> brought the first significant victory!
> 
> Today tests launch running all Ignite tests (Ignite 2.0 Tests->RunAll) is
> completely green: https://ci.ignite.apache.org/viewLog.html?buildId=930953
> 
> This is an reason for being pride, but not a reason to stop this activity.
> Unfortunately, there are still a number of tests with unpredictable
> failures (flaky, 234 tests), and a number of tests have been muted. Full
> project scope is available in the article
> https://cwiki.apache.org/confluence/display/IGNITE/Make+Teamcity+Green+Again
> 
> In any case, thank to all of you who helped and continues to help the
> project to correct the tests and bugs.
> 
> Sincerely,
> Pavlov Dmitry
> 
> 
> пт, 21 июл. 2017 г. в 14:29, Pavel Tupitsyn :
> 
>>> Green Again
>>> Again
>> As if it ever was green :)
>> 
>> Of course +1 on this, let me know if you see any .NET-related failures.
>> 
>> On Fri, Jul 21, 2017 at 1:06 PM, Николай Ижиков 
>> wrote:
>> 
>>> Hello, Igniters.
>>> 
>>> Also ready to help to #MakeTeamcityGreenAgain !
>>> 
>>> 21 июля 2017 г. 12:56 PM пользователь "Vyacheslav Daradur" <
>>> daradu...@gmail.com> написал:
>>> 
 Hi guys.
 
 I vote for #MakeTeamcityGreenAgain. :-)
 
 FYI: it had been described and supported previously[1]
 
 After the completion of my current task I will try to help with this
 activity.
 
 [1]
 http://apache-ignite-developers.2346864.n4.nabble.
 com/Test-failures-td14353.html
 
 2017-07-21 12:39 GMT+03:00 Anton Vinogradov :
 
> Nikolay,
> 
> That's also a big problem for me, as reviewer, to accept changes and
 merge
> them to master.
> 
> Dmitriy,
> 
> Currently we have some contributions *from community* blocked by red
> Teamcity.
> 
> On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov <
 nizhikov@gmail.com>
> wrote:
> 
>> +1 to Dmitry Pavlov from me.
>> 
>> Green team city would be very helpful.
>> 
>> 
>>> If you are a newcomer, please share your feeling: are you
>> confused
>>> by
>> test
>>> failures on Teamcity?
>> 
>> I'm working on my very first ignite issue
>> and it's not easy to understand TC build results.
>> 
>> 
>> 21.07.2017 00:30, Dmitriy Setrakyan пишет:
>> 
>> I think the right question to ask is why do we have RED tests to
>>> begin
>>> with. In my view, TC should be green. If there are tests that do
>> not
> pass,
>>> we should disable them and file blocker tickets for the next
>>> releases.
>>> 
>>> D.
>>> 
>>> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov <
>>> dpavlov@gmail.com
> 
>>> wrote:
>>> 
>>> Hi Igniters,
 
 We all know that there are a lot of open tasks in Ignite project.
>>> To
 be
 able to handle all these tasks, we need to grow our community.
>> And
>>> to
 achieve that, we should pay special attention to new community
 members.
 
 As recent newcomer I’ve faced the following problem: There is no
>>> way
 to
 find out by yourself whether your pull request changes are
>> correct
>>> or
 not.
 Contributor can not distinguish pull request test failures and
>>> master
 test
 failures. Experienced Ignite contributor can estimate the damage,
>>> but
> how
 it can be done by new community member?
 
 Once failing test is introduced to master it is multiplexed to
>> all
> later
 PRs. As a result, we have tons of failures and suites timeouts,
>>> which
 waste
 Teamcity agents time.
 
 I suggest to start new activity - MakeTeamcityGreenAgain. I’m
>> sure
 that
 there are a lot of experienced developers in the community who
> understand
 issues with tests very quickly. Please contact me if you are
>> ready
>>> to
 help
 the community to fix current test issues.
 
 I would appreciate any ideas and sharing your vision how we can
 achieve
 green teamcity.
 
 If you are a newcomer, please share your feeling: are you
>> confused
>>> by
 test
 failures on Teamcity?
 
 Sincerely,
 
 Dmitriy Pavlov
 
 
>>> 
> 
 
 
 
 --
 Best Regards, Vyacheslav D.
 
>>> 
>>

Re: Cluster auto activation design proposal

2017-11-06 Thread Dmitriy Setrakyan 
Got it. In that case, I do not think that BaselineNode is a correct
abstraction. How about ClusterEndpoint?

D.

On Sun, Nov 5, 2017 at 2:12 AM, Alexey Goncharuk  wrote:

> Dmitriy,
>
> The main point of having a BaselineNode interface is that baseline topology
> will contain nodes even when they are offline. When a node is offline, most
> of the methods on ClusterNode are meaningless, thus the new interface (for
> example, node ID). I left on the interface only methods which will reliably
> return data.
>
> Ilya,
>
> For now, we must keep the old AffinityFunction interface, but we can change
> it in the AI 3.0.
>
> --AG
>
> 2017-11-04 17:56 GMT+03:00 Dmitriy Setrakyan :
>
> > Alexey, what is the point of BaselineNode interface? Why not just have
> > ClusterNode?
> >
> > On Fri, Nov 3, 2017 at 11:26 PM, Alexey Goncharuk <
> > alexey.goncha...@gmail.com> wrote:
> >
> > > Guys,
> > >
> > > We are getting closer to the baseline topology finalization. As a first
> > > step, I would like to request a review of the baseline topology
> > management
> > > API. The changes are summarized in [1]. In my opinion, changes are
> quite
> > > simple and concise. Also, as a side note, I suggest moving cluster
> > > activation methods to the IgniteCluter facade as well because the
> facade
> > > itself looks like a good place for management API. Looks like the
> > original
> > > decision to place it on Ignite was wrong.
> > >
> > > Thanks!
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-5850
> > >
> > > 2017-09-04 17:46 GMT+03:00 Dmitriy Setrakyan :
> > >
> > > > On Mon, Sep 4, 2017 at 6:13 AM, Sergey Chugunov <
> > > sergey.chugu...@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Dmitriy,
> > > > >
> > > > > I like the idea of ClusterActivator interface.
> > > > >
> > > > > From user perspective it provides the same functionality as the
> > setter
> > > > but
> > > > > in more clear and intuitive way.
> > > > >
> > > >
> > > > BTW, I made a naming mistake in the original email. The setter name
> > > should
> > > > be "setClusterActivator(...).
> > > >
> > > >
> > > > >
> > > > > Also it gives us a good place to put all the documentation about
> the
> > > > > feature.
> > > > >
> > > >
> > > > Agree. Another advantage is that users can now provide custom logic
> for
> > > the
> > > > initial cluster activation.
> > > >
> > > >
> > > > >
> > > > > Any other opinions?
> > > > >
> > > >
> > > > Alexey Goncharuk, given that you provided the initial implementation
> of
> > > the
> > > > cluster activation, can you please take a look at this design and
> > provide
> > > > comments?
> > > >
> > > >
> > > > >
> > > > > On Fri, Sep 1, 2017 at 2:35 PM, Dmitriy Setrakyan <
> > > dsetrak...@apache.org
> > > > >
> > > > > wrote:
> > > > >
> > > > > > How about this:
> > > > > >
> > > > > >
> > > > > > > *interface ClusterActivator {*
> > > > > > > *boolean activate(Collection nodes);**}*
> > > > > >
> > > > > >
> > > > > > Out of the box, we can provide this implementation of the
> > activation
> > > > > > filter:
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > *ClusterInitialActiveSet implements ClusterActivator { *
> > > > > > > *InigeInitialActiveSet(String... addresses);**}*
> > > > > >
> > > > > >
> > > > > > Then user configuration can look as follows:
> > > > > >
> > > > > > *IgniteConfiguration.setActivationFilter(new
> > > > > > > ClusterInitialActiveSet("1.2.3.4", "4.3.2.1", etc));*
> > > > > >
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > D.
> > > > > >
> > > > > > On Fri, Sep 1, 2017 at 1:47 PM, Sergey Chugunov <
> > > > > sergey.chugu...@gmail.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Dmitriy,
> > > > > > >
> > > > > > > The idea is interesting however I cannot come up with a clear
> use
> > > > case
> > > > > > > which can be widely adopted.
> > > > > > > I would give users a simple API at first to cover 80% of their
> > > needs
> > > > > and
> > > > > > > then collect some feedback and start thinking about adding new
> > > > > > > functionality.
> > > > > > >
> > > > > > > Makes sense?
> > > > > > >
> > > > > > > Sergey.
> > > > > > >
> > > > > > > On Thu, Aug 31, 2017 at 3:55 AM, Dmitriy Setrakyan <
> > > > > > dsetrak...@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hm... Can we also ask user to optionally provide a predicate
> > > which
> > > > > will
> > > > > > > > receive a collection of nodes started so far and return true
> if
> > > the
> > > > > > > > activation should happen? Will it be useful?
> > > > > > > >
> > > > > > > > On Wed, Aug 30, 2017 at 6:28 PM, Sergey Chugunov <
> > > > > > > > sergey.chugu...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Nick,
> > > > > > > > >
> > > > > > > > > As I summed up in this thread above, calling setter for
> > initial
> > > > > > > > activation
> > > > > > > > > nodes is not the only option:
> >

Re: Deserialization of BinaryObject's byte arrays

2017-11-06 Thread Andrey Kornev 
Will do! Thanks!

From: Valentin Kulichenko 
Sent: Monday, November 6, 2017 9:17 AM
To: dev@ignite.apache.org
Subject: Re: Deserialization of BinaryObject's byte arrays

This use case was discussed couple of times already, so it seems to be
pretty important for users. And I like the approach suggested by Andrey.

Andrey, can you create a Jira ticket for this change? Would you like to
contribute it?

-Val

On Fri, Nov 3, 2017 at 4:18 PM, Dmitriy Setrakyan 
wrote:

> This makes sense to me. Sounds like a useful feature.
>
> Would be nice to hear what others in the community think?
>
> D.
>
> On Fri, Nov 3, 2017 at 1:07 PM, Andrey Kornev 
> wrote:
>
> > Hello,
> >
> > We store lots of byte arrays (serialized graph-like data structures) as
> > fields of BinaryObject. Later on, while reading such field,
> > BinaryInputStream implementation creates an on-heap array and copies the
> > bytes from the BinaryObject's internal backing array to the new array.
> >
> > While in general case it works just fine, in our case, a lot of CPU is
> > spent on copying of the bytes and significant amount garbage is generated
> > as well.
> >
> > I'd like Ignite Community to consider the following proposal: introduce
> > support for serialization/deserialization of the ByteBuffer class.  A
> > BinaryInputStream implementation would special case ByteBuffer fields
> same
> > way it currently special cases various Java types (collections, dates,
> > timestamps, enums, etc.)
> >
> > The application developer would simply call
> BinaryObjectBuilder.setField(...)
> > passing in an instance of ByteBuffer. During serialization, the
> > ByteBuffer's bytes would simply be copied to the backing array (similar
> to
> > how regular byte arrays are serialized today) and in the binary object's
> > metadata the field marked as, say, GridBinaryMarshaller.BYTE_BUFFER.
> >
> > During deserialization of the field, instead of allocating a new byte
> > array and transferring the bytes from the BinaryObject's backing array,
> > BinaryInputStream would simply wrap the required sub-array into a
> read-only
> > instance of ByteBuffer using "public static ByteBuffer wrap(byte[] array,
> > int offset, int length)" and return the instance to the application.
> >
> > This approach doesn't require any array allocation/data copying and is
> > faster and significantly reduces pressure on the garbage collector. The
> > benefits are especially great when application stores significant amount
> of
> > its data as byte arrays.
> >
> > Clearly, this approach equally applies to arrays of other primitive
> types.
> >
> > A minor complication does arise when dealing with off-heap BinaryObjects,
> > but nothing that can't solved with a little bit of Java reflection
> wizardry
> > (or by other means).
> >
> > Thanks,
> > Andrey
> >
>

Re: Deserialization of BinaryObject's byte arrays

2017-11-06 Thread Valentin Kulichenko 
This use case was discussed couple of times already, so it seems to be
pretty important for users. And I like the approach suggested by Andrey.

Andrey, can you create a Jira ticket for this change? Would you like to
contribute it?

-Val

On Fri, Nov 3, 2017 at 4:18 PM, Dmitriy Setrakyan 
wrote:

> This makes sense to me. Sounds like a useful feature.
>
> Would be nice to hear what others in the community think?
>
> D.
>
> On Fri, Nov 3, 2017 at 1:07 PM, Andrey Kornev 
> wrote:
>
> > Hello,
> >
> > We store lots of byte arrays (serialized graph-like data structures) as
> > fields of BinaryObject. Later on, while reading such field,
> > BinaryInputStream implementation creates an on-heap array and copies the
> > bytes from the BinaryObject's internal backing array to the new array.
> >
> > While in general case it works just fine, in our case, a lot of CPU is
> > spent on copying of the bytes and significant amount garbage is generated
> > as well.
> >
> > I'd like Ignite Community to consider the following proposal: introduce
> > support for serialization/deserialization of the ByteBuffer class.  A
> > BinaryInputStream implementation would special case ByteBuffer fields
> same
> > way it currently special cases various Java types (collections, dates,
> > timestamps, enums, etc.)
> >
> > The application developer would simply call
> BinaryObjectBuilder.setField(...)
> > passing in an instance of ByteBuffer. During serialization, the
> > ByteBuffer's bytes would simply be copied to the backing array (similar
> to
> > how regular byte arrays are serialized today) and in the binary object's
> > metadata the field marked as, say, GridBinaryMarshaller.BYTE_BUFFER.
> >
> > During deserialization of the field, instead of allocating a new byte
> > array and transferring the bytes from the BinaryObject's backing array,
> > BinaryInputStream would simply wrap the required sub-array into a
> read-only
> > instance of ByteBuffer using "public static ByteBuffer wrap(byte[] array,
> > int offset, int length)" and return the instance to the application.
> >
> > This approach doesn't require any array allocation/data copying and is
> > faster and significantly reduces pressure on the garbage collector. The
> > benefits are especially great when application stores significant amount
> of
> > its data as byte arrays.
> >
> > Clearly, this approach equally applies to arrays of other primitive
> types.
> >
> > A minor complication does arise when dealing with off-heap BinaryObjects,
> > but nothing that can't solved with a little bit of Java reflection
> wizardry
> > (or by other means).
> >
> > Thanks,
> > Andrey
> >
>

[GitHub] ignite pull request #2983: Ignite 12992

2017-11-06 Thread ntikhonov 
GitHub user ntikhonov opened a pull request:

https://github.com/apache/ignite/pull/2983

Ignite 12992



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-12992

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2983.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2983


commit 02b194268071b179d291b28472cef5d587e7558a
Author: Alexander Fedotov 
Date:   2017-04-11T09:00:59Z

Fix missing test resource directory for
org.apache.ignite.spi.discovery.tcp
.TcpDiscoveryNodeAttributesUpdateOnReconnectTest.testReconnect

commit 20016a20f780eb3c21f249d3cb74d08018c4eea5
Author: Alexander Fedotov 
Date:   2017-04-11T11:54:06Z

Fix org.apache.ignite.internal.processors.cache.expiry
.IgniteCacheExpiryPolicyAbstractTest#testNearExpiresWithCacheStore

commit 465084da5b00dcfc056d338f5d0a24875ca2af08
Author: Andrey V. Mashenkov 
Date:   2017-04-12T10:01:25Z

IGNITE-4907: Fixed excessive service instances can be started with dynamic 
deployment. This closes #1766.

(cherry picked from commit 0f7ef74)

commit a20c307df1dd000309a273ef93231fdc41a2a81c
Author: dkarachentsev 
Date:   2017-04-13T06:31:17Z

IGNITE-4891 - Fix. Key is deserialized during transactional get() even if 
withKeepBinary is set

(Backport from master)

commit 630558dfeb373f237057e394e8f2f63230d59dab
Author: vladisav 
Date:   2017-04-13T10:24:42Z

ignite-4173 IgniteSemaphore with failoverSafe enabled doesn't release 
permits in case permits owner node left topology

Backport from master.

(cherry picked from commit 76485fc)

commit 870b752c095ed3776e91a65b99763142b9f2ebc0
Author: Vladisav Jelisavcic 
Date:   2017-04-11T11:09:12Z

ignite-1977 - fixed IgniteSemaphore fault tolerance.

Backport from master.

(cherry picked from commit 902bf42)

commit cd0b92950c6691c6fc1a26cb4f7e55f5ee459298
Author: Yakov Zhdanov 
Date:   2017-04-13T12:52:20Z

ignite-4946 GridCacheP2PUndeploySelfTest became failed

(cherry picked from commit d298e75)

commit 405ce563fb7c35627c6e1bb0b68f423ba089c6f2
Author: Dmitriy Shabalin 
Date:   2017-04-14T10:55:38Z

IGNITE-4068 Added common primitive for buttons group. Refactored existing 
button groups.
(cherry picked from commit e5200c2)

commit 60cf48dc175fa288cd74d1189f0e992c9a16dc99
Author: Vasiliy Sisko 
Date:   2017-04-14T11:00:47Z

IGNITE-4886 Catch all errors.
(cherry picked from commit 7e8d9e8)

commit 81c3ed4c0511841f7056677db6063b4eb8d2def0
Author: Alexey Kuznetsov 
Date:   2017-04-14T11:18:23Z

IGNITE-4896 Rewored GridClientNodeBean serialization.
(cherry picked from commit a025268)

commit 4a1415ad01ff9fde30d5c7c02e6d938f1515178d
Author: Andrey V. Mashenkov 
Date:   2017-04-12T10:01:25Z

IGNITE-4907: Fixed excessive service instances can be started with dynamic 
deployment. This closes #1766.

(cherry picked from commit 0f7ef74)

commit e206b9f1fd3dbf927f940d558144a4796479ed5d
Author: vsisko 
Date:   2017-04-14T11:32:30Z

IGNITE-4871 Added Kubernetes IP finder to Cluster configuration screen.
(cherry picked from commit f978ff2)

commit b22738080101536a8af1ed60e70d693897e9bc7c
Author: dkarachentsev 
Date:   2017-04-14T14:54:02Z

ignite-4173 Fix test. Permits must be released on node fail.

(cherry picked from commit 1f867c6)

commit 41c5288606710b9c42983780ac7464a746d09eb0
Author: dkarachentsev 
Date:   2017-04-14T14:56:25Z

Merge remote-tracking branch 'origin/ignite-1.8.6' into ignite-1.8.6

commit 1985496ea98f4d7112a0b99727297427f343e9b2
Author: dkarachentsev 
Date:   2017-04-17T07:20:41Z

Add missed license headers.

commit ae794ab45b22abe1e570d5d29661396ea7c0f951
Author: dkarachentsev 
Date:   2017-04-17T10:40:28Z

IGNITE-4159 Fix version.

commit bf1049741f7a64728bd433f78262ba273f969848
Author: Andrey V. Mashenkov 
Date:   2017-04-17T16:00:30Z

IGNITE-4954 - Configurable expiration timeout for Cassandra session. This 
closes #1785.

commit aca67066ceaf762fcd44ef0715a4ba0f941226a8
Author: dkarachentsev 
Date:   2017-04-18T09:23:28Z

Merge branch 'ignite-1.8.5-p1' into ignite-1.8.6

commit a4c70f12305c15cbf15a6e5eb7c500f883a8b317
Author: agura 
Date:   2017-04-20T17:45:58Z

ignite-5041 NPE in deadlock

[jira] [Created] (IGNITE-6832) handle IO errors while checkpointing

2017-11-06 Thread Alexander Belyak (JIRA) 
Alexander Belyak created IGNITE-6832:


 Summary: handle IO errors while checkpointing
 Key: IGNITE-6832
 URL: https://issues.apache.org/jira/browse/IGNITE-6832
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
Affects Versions: 2.1
Reporter: Alexander Belyak


If we get some IO error (like "No spece left on device") during checkpointing 
(GridCacheDatabaseSharedManager$WriteCheckpointPages:2509) node didn't stop as 
when get same error while writting WAL log and clients will get some "Long 
running cache futures". We must stop node in this case! Better - add some 
internal healthcheck and stop node anyway if  it won't pass for few times (do 
it with different issue).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2899: ignite 12992

2017-11-06 Thread ntikhonov 
Github user ntikhonov closed the pull request at:

https://github.com/apache/ignite/pull/2899


---