Empty cache memory overhead

[Alexandr Kuramshin]

Hi community,

I'd like to share my investigations about the subject.

Even if the caches is off-heap and contains no data, the JVM heap memory
consumed. I'm calling this feature "empty cache memory overhead"
("overhead" later for shot).

The size of the memory consumed depends on many factors, and varying from 1
to 50 Mb per cache on every node in the cluster.

There is real systems uses >1000 caches within the cluster. So the heap
memory consumed on each node will be 50 Gb or more.

I've found that overhead mainly depends on this factors:

1) local partitions count assigned to the node by the affinity function;

1.a) total number of partitions of the affinity function;

1.b) number of backups;

2) IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE

3) IGNITE_AFFINITY_HISTORY_SIZE

After analyzing heapdumps and the sources I've found this countable objects
upon overhead depends:

1) First group.

GridDhtPartitionTopologyImpl = cache count

GridDhtLocalPartition = cache count * local partitions count

GridCircularBuffer$Item = cache count * local partitions count * item
factor (default 32).

Local partitions count = affinity function total partitions / node count *
(1 + number of backups)

Item factor = map capacity for storing ->
IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE / affinity function partitions
count, but minimum 20.

Real values:

GridDhtPartitionTopologyImpl = 1000
Affinity function total partitions = 1024
Node count = 16
Number of backups = 3
Local partitions count = 256
GridDhtLocalPartition = 256_000
GridCircularBuffer$Item = 8_192_000

2) Second group.

GridAffinityAssignmentCache = cache count * node count

GridAffinityAssignment = cache count * node count * assignment factor

Assignment factor depends on topology version and
IGNITE_AFFINITY_HISTORY_SIZE, default 6-7.

Real values:

GridAffinityAssignmentCache = 16_000
GridAffinityAssignment  = 112_000

I think the implementation should be changed in the way the object counts
should depends on cache data size. And the small (or empty) caches should
be more lightweight as possible.

-- 
Thanks,
Alexandr Kuramshin

[jira] [Created] (IGNITE-4539) RocketMQ data streamer

[Roman Shtykh (JIRA)]

Roman Shtykh created IGNITE-4539:


 Summary: RocketMQ data streamer
 Key: IGNITE-4539
 URL: https://issues.apache.org/jira/browse/IGNITE-4539
 Project: Ignite
  Issue Type: New Feature
  Components: streaming
Reporter: Roman Shtykh
Assignee: Roman Shtykh


Streamer for RocketMQ (https://github.com/rocketmq)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4538) BinaryObjectImpl: lack of context information upon deserialization

[Alexandr Kuramshin (JIRA)]

Alexandr Kuramshin created IGNITE-4538:
--

 Summary: BinaryObjectImpl: lack of context information upon 
deserialization
 Key: IGNITE-4538
 URL: https://issues.apache.org/jira/browse/IGNITE-4538
 Project: Ignite
  Issue Type: Improvement
  Components: binary
Affects Versions: 1.8, 1.7
Reporter: Alexandr Kuramshin


Taking an error we don't know the cache name was accessed, the type of 
BinaryClassDescriptor was used, and the entry was accessed (the key of an entry 
should be logged with respect to the *include sensitive* system property).

Such context information should be appended by wrapping inner exception on the 
every key stack frame.

{noformat}
org.apache.ignite.binary.BinaryObjectException: Unexpected flag value [pos=24, 
expected=4, actual=9]
at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.checkFlagNoHandles(BinaryReaderExImpl.java:1423)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.readLongNullable(BinaryReaderExImpl.java:723)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:677)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read(BinaryFieldAccessor.java:639)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:818)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1481)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:717)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:143)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinary(CacheObjectContext.java:272)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinaryIfNeeded(CacheObjectContext.java:160)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinaryIfNeeded(CacheObjectContext.java:147)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinaryIfNeeded(GridCacheContext.java:1706)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$PeekValueExpiryAwareIterator.advance(GridCacheQueryManager.java:2875)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$PeekValueExpiryAwareIterator.(GridCacheQueryManager.java:2814)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$PeekValueExpiryAwareIterator.(GridCacheQueryManager.java:2752)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$5.(GridCacheQueryManager.java:863)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.scanIterator(GridCacheQueryManager.java:863)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.scanQueryLocal(GridCacheQueryManager.java:1436)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter.executeScanQuery(GridCacheQueryAdapter.java:552)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.igniteIterator(GridCacheAdapter.java:4115)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.igniteIterator(GridCacheAdapter.java:4092)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.iterator(IgniteCacheProxy.java:1979)
 ~[ignite-core-1.10.1.ea7.jar:1.10.1.ea7]
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4537) Update Notifier must not transfer system properties

[Denis Magda (JIRA)]

Denis Magda created IGNITE-4537:
---

 Summary: Update Notifier must not transfer system properties
 Key: IGNITE-4537
 URL: https://issues.apache.org/jira/browse/IGNITE-4537
 Project: Ignite
  Issue Type: Bug
Reporter: Denis Magda
Priority: Critical
 Fix For: 1.9


Apache Ignite Update Notifier that is used for sending updates about new Apache 
Ignite versions gathers and transfers all system properties.

The script must not do this. Instead, it has to get only those system 
properties like Java version, OS versions that are needed. Otherwise, the 
script might send sensitive information like passwords stored in the system 
properties.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Timeout for ordered message

[Valentin Kulichenko]

Yakov,

I was actually asking about the public API for messaging
(IgniteMessaging#sendOrdered
method). Here is the JavaDoc:

 * The {@code timeout} parameter specifies how long an out-of-order message
will stay in a queue,
 * waiting for messages that are ordered ahead of it to arrive. If timeout
expires, then all ordered
 * messages that have not arrived before this message will be skipped. When
(and if) expired messages
 * actually do arrive, they will be ignored.

I remember that we indeed had this scenario possible a while ago, but then
implementation of ordered messaging was reworked and 'out-of-order message'
can't happen anymore. Is this correct? If so, I think the timeout should be
removed from the API.

-Val

On Tue, Jan 10, 2017 at 2:34 AM, Yakov Zhdanov 
wrote:

> Val, timeout still makes sense for task sessions. It is possible that all
> jobs mapped to node have been completed, but node still receives
> attributes. Attributes are sent in ordered messages. It may happen so that
> node may start processing jobs for that task again and this way all the
> attributes will be available. If task timeout elapses then messages get
> cleared. Please let me know if you still have questions.
>
> Thanks!
> --
> Yakov Zhdanov, Director R
> *GridGain Systems*
> www.gridgain.com
>
> 2017-01-06 1:02 GMT+03:00 Valentin Kulichenko <
> valentin.kuliche...@gmail.com
> >:
>
> > Folks,
> >
> > Anyone knows what is the purpose of the timeout for
> > IgniteMessaging#sendOrdered method?
> >
> > void sendOrdered(@Nullable Object topic, Object msg, long timeout);
> >
> > It seems to me that in current implementation messages are always ordered
> > on communication level. I.e. if you send messages 1, 2 and 3, there is no
> > way to receive 3 before 2. The only difference between ordered and
> > unordered messages is that ordered are processed one after another in a
> > single thread, while unordered can be processed in parallel.
> >
> > Having said that, timeout for ordered messages doesn't make much sense to
> > me. Is there a scenario that I'm missing?
> >
> > -Val
> >
>

[GitHub] ignite pull request #1419: IGNITE-4531 Use correct property name in BinaryPr...

[alexpaschenko]

GitHub user alexpaschenko opened a pull request:

https://github.com/apache/ignite/pull/1419

IGNITE-4531 Use correct property name in BinaryProperty.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4531

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1419.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1419


commit e930dfa54112a5374ac4acd6c6ac88510fa9d3e9
Author: Alexander Paschenko 
Date:   2017-01-10T20:12:58Z

IGNITE-4531 Use correct property name in BinaryProperty.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: SQL query collocated flag

[Denis Magda]

Sergi,

What does ‘more effective query plan’ exactly means? Will the aggregation 
happen on the data nodes and why this flag set to ‘true’ leads to strange 
behavior described here [1] which sounds like a bug?

As for the documentation how would you rewrite existed setCollocated 
explanation [2] so that it becomes clearer what the flag is for and how it 
works? Please provide your doc version.

[1] http://apache-ignite-users.70518.x6.nabble.com/NOT-IN-in-ignite-td9861.html
[2] 
https://apacheignite.readme.io/docs/configuration-parameters#section--sqlfields-and-sqlfieldsquery-configuration-properties
 


—
Denis

> On Jan 10, 2017, at 9:13 AM, Sergi Vladykin  wrote:
> 
> Yes, this flag indeed is very obscure. Basically it means that if you have
> some query with aggregates (at the top level right now) and you know that
> you will do GROUP BY by affinity key, then Ignite can generate a more
> effective query plan with this flag set.
> 
> Sergi
> 
> 2017-01-10 2:22 GMT+03:00 Valentin Kulichenko > :
> 
>> Folks,
>> 
>> One of the users faced an interesting case [1]. Basically, there is a
>> query [2] which returns correct result when collocated flag is not set, but
>> if it's true, several counts for each node is received by the client
>> without being aggregated. Is this expected? I'm also attaching the test
>> that reproduces behavior.
>> 
>> Actually, the flag seems to be very confusing and documentation
>> description [3] doesn't help a lot. I'm reading it and it's not clear what
>> has to be collocated, what would happen if I incorrectly set the flag to
>> true, etc. Probably, some examples could help. Can someone elaborate?
>> 
>> [1] http://apache-ignite-users.70518.x6.nabble.com/NOT-
>> IN-in-ignite-td9861.html
>> [2] SELECT count(*) as COUNT FROM Person p join table(joinId VARCHAR(50) =
>> ?) i on p.id = i.joinId
>> [3] https://apacheignite.readme.io/docs/configuration-
>> parameters#section--sqlfields-and-sqlfieldsquery-configuration-properties
>> 
>> -Val
>> 

[GitHub] ignite pull request #1418: ignite-4523

[ascherbakoff]

GitHub user ascherbakoff opened a pull request:

https://github.com/apache/ignite/pull/1418

ignite-4523

Allow distributed SQL query execution over explicit set of partitions

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ascherbakoff/ignite ignite-4523

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1418.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1418


commit 153d4a374fe6cf6c80eb895920ec279f483ab1d7
Author: ascherbakoff 
Date:   2017-01-06T15:56:07Z

IGNITE-4523 WIP.

commit 31c5e42e67fc958e77f3014c364a8e14b236bd8d
Author: ascherbakoff 
Date:   2017-01-07T13:11:58Z

IGNITE-4523 WIP.

commit dea221c15b8e2136f35ab6ab0abacb3b658aa626
Author: Aleksei Scherbakov 
Date:   2017-01-10T17:30:10Z

IGNITE-4523 WIP.

commit 3238365c3c04fcd7db309650012f43aa26bd6a8d
Author: Aleksei Scherbakov 
Date:   2017-01-10T17:30:34Z

IGNITE-4523 WIP.

commit 570cfd7457feb6d1bb95ef4dbae77d0c8d0c5e6c
Author: Aleksei Scherbakov 
Date:   2017-01-10T17:32:31Z

Merge branch 'master' of https://github.com/apache/ignite into ignite-4523




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: SQL query collocated flag

[Sergi Vladykin]

Yes, this flag indeed is very obscure. Basically it means that if you have
some query with aggregates (at the top level right now) and you know that
you will do GROUP BY by affinity key, then Ignite can generate a more
effective query plan with this flag set.

Sergi

2017-01-10 2:22 GMT+03:00 Valentin Kulichenko :

> Folks,
>
> One of the users faced an interesting case [1]. Basically, there is a
> query [2] which returns correct result when collocated flag is not set, but
> if it's true, several counts for each node is received by the client
> without being aggregated. Is this expected? I'm also attaching the test
> that reproduces behavior.
>
> Actually, the flag seems to be very confusing and documentation
> description [3] doesn't help a lot. I'm reading it and it's not clear what
> has to be collocated, what would happen if I incorrectly set the flag to
> true, etc. Probably, some examples could help. Can someone elaborate?
>
> [1] http://apache-ignite-users.70518.x6.nabble.com/NOT-
> IN-in-ignite-td9861.html
> [2] SELECT count(*) as COUNT FROM Person p join table(joinId VARCHAR(50) =
> ?) i on p.id = i.joinId
> [3] https://apacheignite.readme.io/docs/configuration-
> parameters#section--sqlfields-and-sqlfieldsquery-configuration-properties
>
> -Val
>

Re: Contributing to Ignite.NET

[Pavel Tupitsyn]

By the way, I'm working on wiki page about Ignite.NET development:
https://cwiki.apache.org/confluence/display/IGNITE/Ignite.NET+Development

On Tue, Jan 10, 2017 at 11:34 AM, Pavel Tupitsyn 
wrote:

> "Pick a Ticket" updated with Ignite.NET section.
>
> On Tue, Jan 10, 2017 at 12:11 AM, Denis Magda  wrote:
>
>> Great, Sergey thanks! I’ve added you to the contributors list on Ignite’s
>> main site.
>> https://ignite.apache.org/community/resources.html#people <
>> https://ignite.apache.org/community/resources.html#people>
>>
>> Pavel, could you add a number of .NET related tickets to this “Pick a
>> Ticket” section?
>> https://ignite.apache.org/community/contribute.html#pick-ticket <
>> https://ignite.apache.org/community/contribute.html#pick-ticket>
>>
>> —
>> Denis
>>
>> > On Jan 9, 2017, at 4:40 AM, Pavel Tupitsyn 
>> wrote:
>> >
>> > Sergey, I've merged your fix for IGNITE-4412.
>> >
>> > Looking forward to more contributions from you!
>> >
>> > Pavel
>> >
>> > On Thu, Jan 5, 2017 at 8:15 PM, Pavel Tupitsyn 
>> wrote:
>> >
>> >> Hi Sergey,
>> >>
>> >> I've added you to project contributors in Jira.
>> >>
>> >> I can suggest the following .NET tickets to get familiar with the
>> project:
>> >>
>> >> IGNITE-4412 .NET: NLogLoggerTest is not thread safe
>> >> 
>> >> IGNITE-3170 .NET: Add user-friendly ToString overrides for public types
>> >> 
>> >> IGNITE-1508 .NET: Create common ToString() method
>> >> 
>> >>
>> >> Let me know if you have any questions.
>> >>
>> >> Thanks,
>> >>
>> >> Pavel.
>> >>
>> >>
>> >> On Thu, Jan 5, 2017 at 8:10 PM, Guru Stron 
>> >> wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> My name is Sergey, i would like to contribute to Ignite.NET.
>> >>> Can you please suggest a ticket for me to start. My Jira username is
>> >>> GuruStron.
>> >>>
>> >>
>> >>
>>
>>
>

[GitHub] ignite pull request #1417: IGNITE-3400: add available size check before data...

[tledkov-gridgain]

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/1417

IGNITE-3400: add available size check before data blocks writing



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-3400

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1417.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1417


commit 1d0da67942b2c0938c8bc1b720ab0fe4225d6dfd
Author: tledkov-gridgain 
Date:   2016-09-07T05:51:47Z

IGNITE-3400 IGFS: Does not correctly deallocate free space in courner case 
- add test

commit c893da70a9757b16b0799adc8eaa29fa1b03d06e
Author: tledkov-gridgain 
Date:   2016-12-21T11:54:33Z

IGNITE-4399: IGFS: Merged IgfsSecondaryFileSystem and 
IgfsSecondaryFileSystemV2 interfaces. This closes #1346.

commit c5882a85f4e3a1f61723ac54fd92f087684df6da
Author: devozerov 
Date:   2016-12-26T11:15:42Z

Merge branch 'master' into ignite-2.0

commit 7e73d0223a3f09cbe0b7094a2c04bdf9d63ca9be
Author: devozerov 
Date:   2016-12-28T09:54:47Z

Merge branch 'master' into ignite-2.0

commit 7d82d6a06b5e9f1f8cd2909b865e37d46b8da03f
Author: devozerov 
Date:   2016-12-28T09:58:11Z

IGNITE-3875: Introduced separate thread pool for data streamer. This closes 
#1173. This closes #1383.

commit a61b0eaff1817d84c0659e8a7e095f29e22800e1
Author: tledkov-gridgain 
Date:   2016-12-28T11:09:38Z

IGNITE-4405: Hadoop: implemented "readLine" method for HadoopDataInStream 
and HadoopDirectDataInput classes. This closes #1358.

commit 2df39a80d80e2575be61a902ccd48615796fcde9
Author: tledkov-gridgain 
Date:   2016-12-28T13:47:24Z

IGNITE-3961: IGFS: Added IgfsSecondaryFileSystem.affintiy() method. This 
closes #1114. This closes #1252.

commit 2e691d80ea4870c3e7b5b127792b66c920f72c39
Author: tledkov-gridgain 
Date:   2016-12-29T08:00:01Z

IGNITE-4462: IGFS: removed grid name from HadoopIgfsEndpoint. This closes 
#1368.

commit a9b1fc2b3840d47d7c978d9296e8ae6bdeb10be5
Author: tledkov-gridgain 
Date:   2016-12-29T08:07:22Z

IGNITE-4459: Hadoop: weighted planned is default one from now on. This 
closes #1391.

commit 1f743465d6875ef48b1835d03a78a0dbaf339bf6
Author: tledkov-gridgain 
Date:   2016-12-29T08:14:10Z

IGNITE-4458: Hadoop: "striped" shuffle mode is default from now on. This 
closes #1390.

commit 6090ebdfcd0ea3840b0d32cb10197b43615e1e89
Author: devozerov 
Date:   2017-01-05T09:23:06Z

Merge branch 'master' into ignite-2.0

commit 0eb6c90bb4752f0a2a54cb332d69054c7587d24c
Author: tledkov-gridgain 
Date:   2017-01-10T11:28:42Z

Merge branch 'ignite-2.0' into ignite-3400

# Conflicts:
#   
modules/core/src/test/java/org/apache/ignite/internal/processors/igfs/IgfsAbstractBaseSelfTest.java

commit e5d00443fbb7f384d93515c5f82c0a000b563ede
Author: tledkov-gridgain 
Date:   2017-01-10T13:26:24Z

IGNITE-3400: add check before block writing

commit 3b261ac85dc17fdea6cba0c2d3bfcc1c83262891
Author: tledkov-gridgain 
Date:   2017-01-10T14:05:04Z

IGNITE-3400: add check available size before writing blocks




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-4536) Update metrics for new offheap storage

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-4536:


 Summary: Update metrics for new offheap storage
 Key: IGNITE-4536
 URL: https://issues.apache.org/jira/browse/IGNITE-4536
 Project: Ignite
  Issue Type: Sub-task
Reporter: Alexey Goncharuk


Since the new implementation of offheap storage and memory pools, we need to 
actualize existing cache metrics



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4535) Add option to store deserialized values on-heap

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-4535:


 Summary: Add option to store deserialized values on-heap
 Key: IGNITE-4535
 URL: https://issues.apache.org/jira/browse/IGNITE-4535
 Project: Ignite
  Issue Type: Sub-task
Reporter: Alexey Goncharuk


Now cache data is always stored in offheap (pagememory). Need implement option 
to also store deserialised values in heap memory, this should be replacement 
for existing CacheConfiguration.memoryMode configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4534) Implement offheap eviction policies based on page memory

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-4534:


 Summary: Implement offheap eviction policies based on page memory
 Key: IGNITE-4534
 URL: https://issues.apache.org/jira/browse/IGNITE-4534
 Project: Ignite
  Issue Type: Sub-task
Reporter: Alexey Goncharuk


Since the internal structure of offheap storage has changed, we need to 
re-implement Fifo, Lru and Sorted eviction policies



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request #1392: IGNITE-4367 .NET: Fix flaky tests

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1392


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1413: IGNITE-4532 .NET: Fix build warnings

[ptupitsyn]

Github user ptupitsyn closed the pull request at:

https://github.com/apache/ignite/pull/1413


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1384: IGNITE-4470 Added support for log file configurat...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1384


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1416: Ignite 4503

[iveselovskiy]

GitHub user iveselovskiy opened a pull request:

https://github.com/apache/ignite/pull/1416

Ignite 4503



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4503

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1416.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1416


commit c893da70a9757b16b0799adc8eaa29fa1b03d06e
Author: tledkov-gridgain 
Date:   2016-12-21T11:54:33Z

IGNITE-4399: IGFS: Merged IgfsSecondaryFileSystem and 
IgfsSecondaryFileSystemV2 interfaces. This closes #1346.

commit c5882a85f4e3a1f61723ac54fd92f087684df6da
Author: devozerov 
Date:   2016-12-26T11:15:42Z

Merge branch 'master' into ignite-2.0

commit 7e73d0223a3f09cbe0b7094a2c04bdf9d63ca9be
Author: devozerov 
Date:   2016-12-28T09:54:47Z

Merge branch 'master' into ignite-2.0

commit 7d82d6a06b5e9f1f8cd2909b865e37d46b8da03f
Author: devozerov 
Date:   2016-12-28T09:58:11Z

IGNITE-3875: Introduced separate thread pool for data streamer. This closes 
#1173. This closes #1383.

commit a61b0eaff1817d84c0659e8a7e095f29e22800e1
Author: tledkov-gridgain 
Date:   2016-12-28T11:09:38Z

IGNITE-4405: Hadoop: implemented "readLine" method for HadoopDataInStream 
and HadoopDirectDataInput classes. This closes #1358.

commit 2df39a80d80e2575be61a902ccd48615796fcde9
Author: tledkov-gridgain 
Date:   2016-12-28T13:47:24Z

IGNITE-3961: IGFS: Added IgfsSecondaryFileSystem.affintiy() method. This 
closes #1114. This closes #1252.

commit 2e691d80ea4870c3e7b5b127792b66c920f72c39
Author: tledkov-gridgain 
Date:   2016-12-29T08:00:01Z

IGNITE-4462: IGFS: removed grid name from HadoopIgfsEndpoint. This closes 
#1368.

commit a9b1fc2b3840d47d7c978d9296e8ae6bdeb10be5
Author: tledkov-gridgain 
Date:   2016-12-29T08:07:22Z

IGNITE-4459: Hadoop: weighted planned is default one from now on. This 
closes #1391.

commit 1f743465d6875ef48b1835d03a78a0dbaf339bf6
Author: tledkov-gridgain 
Date:   2016-12-29T08:14:10Z

IGNITE-4458: Hadoop: "striped" shuffle mode is default from now on. This 
closes #1390.

commit 6090ebdfcd0ea3840b0d32cb10197b43615e1e89
Author: devozerov 
Date:   2017-01-05T09:23:06Z

Merge branch 'master' into ignite-2.0

commit f7609c7f4503b6f74775ee27426c6cb4e4b2fc64
Author: iveselovskiy 
Date:   2017-01-10T12:36:26Z

IGNITE-4503




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1415: Ignite gg 11414

[DmitriyGovorukhin]

GitHub user DmitriyGovorukhin opened a pull request:

https://github.com/apache/ignite/pull/1415

Ignite gg 11414



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-11414

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1415.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1415


commit 7f3f1d67cf23b15dc51e55d01c4969614d4f0668
Author: Dmitriy Govorukhin 
Date:   2016-11-08T18:43:47Z

ignite-gg-11650 update assert  in test

commit e4acc6f59fbdcfb5892c40c0ede52ae027b087c8
Author: EdShangGG 
Date:   2016-11-08T19:21:25Z

GG-11688 Extend Backup SPI to get information about backup from whole 
cluster

commit 0f469651f41112e060ad1e5492396953cdd29313
Author: Dmitriy Govorukhin 
Date:   2016-11-09T13:22:20Z

ignite-gg-11650 activation future check fail nodes

commit 678b4d26ae0812d8d61cdc5f4544b9a9cfce9957
Author: Ilya Lantukh 
Date:   2016-11-09T14:06:13Z

Fixed iteration over indexes.

commit 39defc8af26a985e0f92199c2abad0ec9dbea351
Author: Dmitriy Govorukhin 
Date:   2016-11-09T17:09:50Z

ignite-gg-11650 deploy service test, atomic long test, initLatch for write 
ahead log, get file lock on create dynamic request

commit dde1db01afdc7d8815fbd0a4bffef731ea9251fb
Author: Dmitriy Govorukhin 
Date:   2016-11-11T11:57:25Z

ignite-gg-11650 Stretched cluster DR replication. Small improvements 
activate future

commit 42175527a4f9af70b2a9d2555dc8fa29279bc6ea
Author: Alexey Goncharuk 
Date:   2016-11-12T23:32:52Z

Multithreaded checkpoint

commit b96dbec769e033fb32828b219d0603465266
Author: Dmitriy Govorukhin 
Date:   2016-11-14T17:57:24Z

ignite-gg-11650 concurrent activate cluster

commit c0cd451aef28edc7e750fa4de287990b2fbb1726
Author: Dmitriy Govorukhin 
Date:   2016-11-15T10:23:07Z

ignite-gg-11650 refactoring concurrent activate cluster

commit e7a65ee6286617bfd897595be384576431294cca
Author: Dmitriy Govorukhin 
Date:   2016-11-15T12:48:16Z

ignite-gg-11650 test updates

commit 1a337dbd2e3fb9aa2026782b867adbbfe358584f
Author: Alexey Goncharuk 
Date:   2016-11-16T10:50:11Z

EA3 - Debug info for WAL

commit 8298372048deab4f429d464faf3c9c5fc402166d
Author: Alexey Goncharuk 
Date:   2016-11-16T10:58:50Z

GG-11627 - Fixing concurrent replace in B+Tree leading to an assertion

commit b2999f2e4d490fdb140483e4283f0fe2d8004a67
Author: Sergey Sidorov 
Date:   2016-11-15T15:50:24Z

GG-11414: Remove snapshotableIndex flag from CacheConfiguration, Get rid of 
all usages of GridH2TreeIndex

commit df7b4f65fbea86310c953c7bd41d2fd7ab85831e
Author: Sergey Sidorov 
Date:   2016-11-16T13:50:55Z

tests have been fixed

commit 38a5d0a787531b132fc0109e79b76e10e06423be
Author: Dmitriy Govorukhin 
Date:   2016-11-17T09:44:42Z

ignite-gg-11650 check state cluster before perform operation.

commit e25dcdd645e665feed1b2e262daba5b5f06b22b3
Author: Ilya Lantukh 
Date:   2016-11-17T12:15:02Z

Merge branches 'ignite-db-x-backup-idx' and 'ignite-gg-8.0.1.ea3' of 
https://github.com/gridgain/apache-ignite into ignite-db-x-backup-idx

commit 55282fd29b4b240324797dd8a60683d0ea23b5e6
Author: Dmitriy Govorukhin 
Date:   2016-11-17T12:20:47Z

ignite-gg-11650 check service after activate test

commit 12959744c1b55f2f452379fd6cbfd318bad2673c
Author: EdShangGG 
Date:   2016-11-17T13:37:35Z

GG-11676 Failed to reinitialize local partitions

commit 6bfc6902443b8dc989a28694992e310b3b8aebb4
Author: Ilya Lantukh 
Date:   2016-11-17T14:24:52Z

Merge branch 'GG-11688' of https://github.com/ggprivate/ggprivate into 
ignite-db-x-backup-idx

# Conflicts:
#   
modules/pds/src/main/java/org/gridgain/grid/database/backup/file/FileDatabaseBackupSpi.java

commit 797656e8e3d7632e541f9244518df692698c8f3b
Author: EdShangGG 
Date:   2016-11-17T15:15:24Z

GG-11676 Failed to reinitialize local partitions
-refactoring
-test changes

commit 472b200068a0abfb9ffd179a95f26c6124b08bf2
Author: Ilya Lantukh 
Date:   2016-11-17T17:04:38Z

Partitioned index - WIP.

commit 0abe3047909a10d8efa306984c74f36eb1b41cc3
Author: Dmitriy Govorukhin 
Date:   2016-11-17T18:23:41Z

ignite-gg-11650  store id resolve, system properties for enable

commit 

[GitHub] ignite pull request #1414: Ignite 4525

[dkarachentsev]

GitHub user dkarachentsev opened a pull request:

https://github.com/apache/ignite/pull/1414

Ignite 4525



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4525

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1414.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1414


commit 626f1d5614930bca0ab744a6049e5f83f321d46a
Author: Andrey V. Mashenkov 
Date:   2016-12-23T09:18:07Z

IGNITE-4142: Fixed assertion while update metrics. This closes #1372.

commit 709b1a992121d559c2d8f01a12cce498965c1d88
Author: dkarachentsev 
Date:   2017-01-10T12:38:55Z

ignite-4525 - Near reader is not created when value is loaded from store




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Extend TcpDiscoveryVmIpFinder functionality

[Sergey Kozlov]

I like the idea to use the full range notation like 10.0.0.1..10.0.0.100.
One issue we may face is the using of the addresses from different networks
like 10.0.0.201..10.0.1.100.

On Tue, Jan 10, 2017 at 1:35 PM, Yakov Zhdanov  wrote:

> Guys, before I file a ticket - should we support range notation for each
> byte in IP address? I think we should.
>
> --Yakov
>



-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com

Re: Integration with RocketMQ

[Yakov Zhdanov]

I would place it under Ignite for consistency with other streamers.

--Yakov

[GitHub] ignite pull request #1413: IGNITE-4532 .NET: Fix build warnings

[ptupitsyn]

GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/1413

IGNITE-4532 .NET: Fix build warnings



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ptupitsyn/ignite ignite-4532

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1413.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1413


commit f110b62ea0d3eb18821d41ea9db9848db54f924e
Author: Pavel Tupitsyn 
Date:   2017-01-10T09:36:48Z

IGNITE-4532 .NET: Fix build warnings

commit 58acbbb347e9a3da9fa72f40579238955aac6fe3
Author: Pavel Tupitsyn 
Date:   2017-01-10T09:49:51Z

Suppress processor architecture mismatch warning in x64/x86 modes

commit d52cd0a98e4bc106c2d421fa8085da3cfa53404e
Author: Pavel Tupitsyn 
Date:   2017-01-10T09:59:50Z

Fix log4net project warnings

commit 7df8d34889c89a6d1dd17dfa4dc194f699eacf6c
Author: Pavel Tupitsyn 
Date:   2017-01-10T10:10:50Z

IGNITE-4532 .NET: Suppress CA1822 in test projects

commit f0348d88c91202406280e7ef6905efd4390fac1f
Author: Pavel Tupitsyn 
Date:   2017-01-10T10:12:29Z

wip

commit 6a925b04622997fcb8ac0fcc437a9ec25dec9171
Author: Pavel Tupitsyn 
Date:   2017-01-10T10:15:33Z

Fixing warnings

commit 82677a2d39acb8cb98e7195e014d83b931029a9d
Author: Pavel Tupitsyn 
Date:   2017-01-10T10:17:11Z

wip

commit c3bfcd6f2189258a43f52ea6e6bb7c0611d1b91a
Author: Pavel Tupitsyn 
Date:   2017-01-10T10:18:57Z

wip

commit 24d16e53912d93d62f37f4289234cb6067021828
Author: Pavel Tupitsyn 
Date:   2017-01-10T10:23:15Z

Treat warnings as errors in all projects

commit d111b1e0a6c46672ba6bcc3449c9f152a2ecf432
Author: Pavel Tupitsyn 
Date:   2017-01-10T10:29:21Z

Fix jni project reference in Core




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: IGNITE-3904 - Add an option to serialize specific type(s) with OptimizedMarshaller when binary format is used

[Nikita Amelchev]

We dont have any specific case. You can close this ticket.

2017-01-06 21:26 GMT+03:00 Denis Magda :

> + dev list
>
> Nikita, if this is not the feature that is needed then we can close the
> ticket and I would suggest you picking up another one.
>
> —
> Denis
>
> > On Jan 6, 2017, at 10:18 AM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
> >
> > Denis,
> >
> > Yes, this was the original motivation and with BinaryIdentityResolver
> there is no issue anymore. I don't think there is much sense to make this
> change now, it will only pollute the configuration with another not very
> intuitive property.
> >
> > Nikita, do you have any specific case where you need this change?
> >
> > -Val
> >
> > On Fri, Jan 6, 2017 at 9:42 AM, Denis Magda > wrote:
> > Hi Nikita,
> >
> > Yes, looks like the ticket is waiting to be picked up. I’ve added you to
> the contributors list. Go ahead an assign it on yourself.
> >
> > In the meanwhile, Val, could you remind me what was the main reason this
> ticket was created? There was some issue related to hashCode calculation
> for some Hibernate objects, right? Now we should get over this issue using
> BinaryIdentityResolver [1].
> >
> > [1] https://apacheignite.readme.io/docs/binary-marshaller#
> handling-hash-code-generation-and-equals-execution <
> https://apacheignite.readme.io/docs/binary-marshaller#
> handling-hash-code-generation-and-equals-execution>
> >
> > —
> > Denis
> >
> >> On Jan 6, 2017, at 8:01 AM, Nikita Amelchev  > wrote:
> >>
> >> hello everyone.
> >> Is IGNITE-3904 still relevant?
> >>
> >> I'd like to take it.
> >> Can you add me to contributor list?
> >> Username: NSAmelchev
> >> Full Name: Amelchev Nikita
> >>
> >> --
> >> Best wishes,
> >> Amelchev Nikita
> >
> >
>
>


-- 
Best wishes,
Amelchev Nikita

Re: Extend TcpDiscoveryVmIpFinder functionality

[Yakov Zhdanov]

Guys, before I file a ticket - should we support range notation for each
byte in IP address? I think we should.

--Yakov

Re: Timeout for ordered message

[Yakov Zhdanov]

Val, timeout still makes sense for task sessions. It is possible that all
jobs mapped to node have been completed, but node still receives
attributes. Attributes are sent in ordered messages. It may happen so that
node may start processing jobs for that task again and this way all the
attributes will be available. If task timeout elapses then messages get
cleared. Please let me know if you still have questions.

Thanks!
--
Yakov Zhdanov, Director R
*GridGain Systems*
www.gridgain.com

2017-01-06 1:02 GMT+03:00 Valentin Kulichenko :

> Folks,
>
> Anyone knows what is the purpose of the timeout for
> IgniteMessaging#sendOrdered method?
>
> void sendOrdered(@Nullable Object topic, Object msg, long timeout);
>
> It seems to me that in current implementation messages are always ordered
> on communication level. I.e. if you send messages 1, 2 and 3, there is no
> way to receive 3 before 2. The only difference between ordered and
> unordered messages is that ordered are processed one after another in a
> single thread, while unordered can be processed in parallel.
>
> Having said that, timeout for ordered messages doesn't make much sense to
> me. Is there a scenario that I'm missing?
>
> -Val
>

[GitHub] ignite pull request #1412: Ignite 4489

[alexpaschenko]

GitHub user alexpaschenko opened a pull request:

https://github.com/apache/ignite/pull/1412

Ignite 4489



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4489

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1412.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1412


commit 76d2e43c063660179bf6223b493ae0e9b1aa552e
Author: Alexander Paschenko 
Date:   2016-12-12T10:35:47Z

IGNITE-4363 Non binary UPDATE/inner properties general UPDATE fix

commit 55d0710baec0705625dc77be71441138fe155f41
Author: Alexander Paschenko 
Date:   2016-12-12T10:37:21Z

Merge branch 'master' into ignite-4363

commit ed577b65532c83d1e2863c99d06945af9263af49
Author: Alexander Paschenko 
Date:   2016-12-12T12:48:58Z

IGNITE-4363 Nested fields handling fix for INSERT/MERGE

commit e5f15ddcd39e1b4d5578f12ea78eb3225d1ae184
Author: Alexander Paschenko 
Date:   2016-12-13T16:52:49Z

IGNITE-4363 Removed redundant code

commit 1ef44b5d3a28e6929bd8959aedfc97dbee4abe09
Author: devozerov 
Date:   2016-12-26T15:15:24Z

Merge branch 'master' into ignite-4363

commit d148d576b10e3e5b20983d8f93760068088d7e40
Author: devozerov 
Date:   2016-12-26T15:18:47Z

Cosmetics.

commit d362da5fdb0671d17672b0096d064ce30254638f
Author: Alexander Paschenko 
Date:   2016-12-26T15:39:10Z

IGNITE-4490 Query-less INSERT and MERGE

commit b8dccdc22baf878c74363fb52a86ff30130826bd
Author: Alexander Paschenko 
Date:   2016-12-27T15:10:46Z

Merge branch 'ignite-4490' into ignite-4489

# Conflicts:
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/DmlStatementsProcessor.java
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/dml/UpdatePlanBuilder.java
#   
modules/indexing/src/test/java/org/apache/ignite/internal/processors/cache/IgniteCacheUpdateSqlQuerySelfTest.java

commit 0146046f9e80bf11e215764a38f8427dc48996f1
Author: Alexander Paschenko 
Date:   2016-12-28T08:07:58Z

IGNITE-4489 UPDATE reworked to better distribute entries processing

commit 8c785ccc6c3b4fde3c6580fab7f5f5c0f8d3a154
Author: Alexander Paschenko 
Date:   2016-12-28T08:54:51Z

Handle Dates correctly when parsing Date string without query

commit b0ca3b6133e9d7e8735162f740aec819cf36cbe9
Author: Alexander Paschenko 
Date:   2016-12-28T08:55:55Z

Merge branch 'ignite-4490' into ignite-4489

commit cebfe59ddca606942f1d9843d92f99c76c804fab
Author: Alexander Paschenko 
Date:   2016-12-28T09:01:47Z

Correct missing args handling

commit 600be2be7a8c86ca1873ee75bd273447957a8a0d
Author: Alexander Paschenko 
Date:   2016-12-29T12:29:46Z

IGNITE-4489 Better distributed INSERT and MERGE

commit ebe9c324c589f46455c2c57f4fc9c14de9caef27
Author: Alexander Paschenko 
Date:   2017-01-09T19:44:52Z

IGNITE-4489 DmlStatementsProcessor refactoring for better maintainability 
(moved code to separate classes) + test

commit 947582787aaf9a453eb50b8c28c25a8ecda59089
Author: Alexander Paschenko 
Date:   2017-01-10T10:09:00Z

Merge branch 'master' into ignite-4489




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: IgniteConfiguration.gridName is very confusing

[Pavel Tupitsyn]

I think we should fix log output as well and replace all "grid" occurences
with "instance".

On Tue, Jan 10, 2017 at 12:55 PM, Alexander Fedotov <
alexander.fedot...@gmail.com> wrote:

> Hi,
>
> I think we should leave null as a default value for unnamed Ignite
> instances. At least that change should be considered out of the current
> scope.
>
> What about naming, I'm also renaming log occurrences of "grid" and "grid
> name" where it stands reasonable.
> Are there places in the logging logic where we should prefer name "grid" or
> "grid name" instead of "Ignite instance name" or "Ignite instance name" can
> be used without any semantic impact?
>
> On Sat, Dec 31, 2016 at 11:23 AM, Alexander Fedotov <
> alexander.fedot...@gmail.com> wrote:
>
> > Okay. From the all said above I suppose "instanceName" should work for
> > IgniteConfiguration and "igniteInstanceName" in all other places.
> >
> > Regards,
> > Alexander
> >
> > 31 дек. 2016 г. 3:43 AM пользователь "Dmitriy Setrakyan" <
> > dsetrak...@apache.org> написал:
> >
> > It sounds like it must be unique then. I would propose the following:
> >
> >1. If user defines the instanceName, then we assign it to the node.
> >2. If user does not define the instance name, then we have to give it
> >some unique value, like node ID or PID.
> >
> > Will this change be backward compatible, or should we leave it as null if
> > user does not define it?
> >
> > D.
> >
> > On Fri, Dec 30, 2016 at 4:19 PM, Denis Magda 
> wrote:
> >
> > > Sounds reasonable. Agree that 'instanceName' suits better considering
> > your
> > > explanation.
> > >
> > > --
> > > Denis
> > >
> > > On Friday, December 30, 2016, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > > > This name identifies instance of Ignite, in case there are more than
> > one
> > > > within an application. Here are our API methods around this:
> > > >
> > > > // We provide a name and get newly started *Ignite* instance.
> > > > Ignite ignite = Ignition.start(new
> > > IgniteConfiguration().setGridName(name));
> > > >
> > > > // We provide a name and get existing *Ignite* instance.
> > > > Ignite ignite = Ignition.ignite(name);
> > > >
> > > > This has nothing to do with nodes. For node representation we have
> > > > ClusterNode API, which already has nodeId() method for
> identification.
> > > >
> > > > In other words, if we choose nodeName, we will have both nodeName and
> > > > nodeId in the product, but with absolutely different meaning and used
> > in
> > > > different parts of API. How user is going to understand the
> difference
> > > > between them? In my view, this is even more confusing than current
> > > gridName.
> > > >
> > > > -Val
> > > >
> > > > On Fri, Dec 30, 2016 at 2:42 PM, Denis Magda 
> > > wrote:
> > > >
> > > >> Alexander, frankly speaking I'm still for your original proposal -
> > > >> nodeName. The uniqueness specificities can be set in the doc.
> > > >>
> > > >> --
> > > >> Denis
> > > >>
> > > >> On Friday, December 30, 2016, Alexander Fedotov <
> > > >> alexander.fedot...@gmail.com> wrote:
> > > >> > Well, then may be we should go with one of the below names:
> > > >> >
> > > >> > processNodeName
> > > >> > jvmNodeName
> > > >> > runtimeNodeName
> > > >> > processScopedNodeName
> > > >> > jvmScopedNodeName
> > > >> > runtimeScopedNodeName
> > > >> > processWideNodeName
> > > >> > jvmWideNodeName
> > > >> > runtimeWideNodeName
> > > >> >
> > > >> > Regards,
> > > >> > Alexander
> > > >> >
> > > >> > 31 дек. 2016 г. 12:37 AM пользователь "Denis Magda" <
> > > dma...@apache.org>
> > > >> > написал:
> > > >> >
> > > >> > The parameter specifies a node name which has to be unique per JVM
> > > >> process
> > > >> > (if you start multiple nodes in a single process). In my
> > understanding
> > > it
> > > >> > was mainly introduced to handle these multiple-nodes-per-JVM
> > > scenarios.
> > > >> >
> > > >> > However, several nodes can have the same name cluster wide.
> > > >> >
> > > >> > —
> > > >> > Denis
> > > >> >
> > > >> >
> > > >> >> On Dec 30, 2016, at 1:30 PM, Dmitriy Setrakyan <
> > > dsetrak...@apache.org>
> > > >> > wrote:
> > > >> >>
> > > >> >> Now I am confused. What is the purpose of this configuration
> > > parameter?
> > > >> >>
> > > >> >> On Fri, Dec 30, 2016 at 1:15 PM, Denis Magda 
> > > wrote:
> > > >> >>
> > > >> >>> See Val’s concern in the discussion. I’m absolutely fine with
> > > >> ‘nodeName’.
> > > >> >>>
> > > >> >>> —
> > > >> >>> Denis
> > > >> >>>
> > > >>  On Dec 30, 2016, at 1:13 PM, Dmitriy Setrakyan <
> > > dsetrak...@apache.org
> > > >> >
> > > >> >>> wrote:
> > > >> 
> > > >>  On Fri, Dec 30, 2016 at 1:12 PM, Denis Magda <
> dma...@apache.org>
> > > >> wrote:
> > > >> 
> > > >> > What’s about ‘localNodeName’?
> > > >> >
> > > >> 
> > > >>  Why is it better than "nodeName"? Isn't it obvious that the
> name
> > is
> > > >> for
> > > >> 

Re: IgniteConfiguration.gridName is very confusing

[Alexander Fedotov]

Hi,

I think we should leave null as a default value for unnamed Ignite
instances. At least that change should be considered out of the current
scope.

What about naming, I'm also renaming log occurrences of "grid" and "grid
name" where it stands reasonable.
Are there places in the logging logic where we should prefer name "grid" or
"grid name" instead of "Ignite instance name" or "Ignite instance name" can
be used without any semantic impact?

On Sat, Dec 31, 2016 at 11:23 AM, Alexander Fedotov <
alexander.fedot...@gmail.com> wrote:

> Okay. From the all said above I suppose "instanceName" should work for
> IgniteConfiguration and "igniteInstanceName" in all other places.
>
> Regards,
> Alexander
>
> 31 дек. 2016 г. 3:43 AM пользователь "Dmitriy Setrakyan" <
> dsetrak...@apache.org> написал:
>
> It sounds like it must be unique then. I would propose the following:
>
>1. If user defines the instanceName, then we assign it to the node.
>2. If user does not define the instance name, then we have to give it
>some unique value, like node ID or PID.
>
> Will this change be backward compatible, or should we leave it as null if
> user does not define it?
>
> D.
>
> On Fri, Dec 30, 2016 at 4:19 PM, Denis Magda  wrote:
>
> > Sounds reasonable. Agree that 'instanceName' suits better considering
> your
> > explanation.
> >
> > --
> > Denis
> >
> > On Friday, December 30, 2016, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> > > This name identifies instance of Ignite, in case there are more than
> one
> > > within an application. Here are our API methods around this:
> > >
> > > // We provide a name and get newly started *Ignite* instance.
> > > Ignite ignite = Ignition.start(new
> > IgniteConfiguration().setGridName(name));
> > >
> > > // We provide a name and get existing *Ignite* instance.
> > > Ignite ignite = Ignition.ignite(name);
> > >
> > > This has nothing to do with nodes. For node representation we have
> > > ClusterNode API, which already has nodeId() method for identification.
> > >
> > > In other words, if we choose nodeName, we will have both nodeName and
> > > nodeId in the product, but with absolutely different meaning and used
> in
> > > different parts of API. How user is going to understand the difference
> > > between them? In my view, this is even more confusing than current
> > gridName.
> > >
> > > -Val
> > >
> > > On Fri, Dec 30, 2016 at 2:42 PM, Denis Magda 
> > wrote:
> > >
> > >> Alexander, frankly speaking I'm still for your original proposal -
> > >> nodeName. The uniqueness specificities can be set in the doc.
> > >>
> > >> --
> > >> Denis
> > >>
> > >> On Friday, December 30, 2016, Alexander Fedotov <
> > >> alexander.fedot...@gmail.com> wrote:
> > >> > Well, then may be we should go with one of the below names:
> > >> >
> > >> > processNodeName
> > >> > jvmNodeName
> > >> > runtimeNodeName
> > >> > processScopedNodeName
> > >> > jvmScopedNodeName
> > >> > runtimeScopedNodeName
> > >> > processWideNodeName
> > >> > jvmWideNodeName
> > >> > runtimeWideNodeName
> > >> >
> > >> > Regards,
> > >> > Alexander
> > >> >
> > >> > 31 дек. 2016 г. 12:37 AM пользователь "Denis Magda" <
> > dma...@apache.org>
> > >> > написал:
> > >> >
> > >> > The parameter specifies a node name which has to be unique per JVM
> > >> process
> > >> > (if you start multiple nodes in a single process). In my
> understanding
> > it
> > >> > was mainly introduced to handle these multiple-nodes-per-JVM
> > scenarios.
> > >> >
> > >> > However, several nodes can have the same name cluster wide.
> > >> >
> > >> > —
> > >> > Denis
> > >> >
> > >> >
> > >> >> On Dec 30, 2016, at 1:30 PM, Dmitriy Setrakyan <
> > dsetrak...@apache.org>
> > >> > wrote:
> > >> >>
> > >> >> Now I am confused. What is the purpose of this configuration
> > parameter?
> > >> >>
> > >> >> On Fri, Dec 30, 2016 at 1:15 PM, Denis Magda 
> > wrote:
> > >> >>
> > >> >>> See Val’s concern in the discussion. I’m absolutely fine with
> > >> ‘nodeName’.
> > >> >>>
> > >> >>> —
> > >> >>> Denis
> > >> >>>
> > >>  On Dec 30, 2016, at 1:13 PM, Dmitriy Setrakyan <
> > dsetrak...@apache.org
> > >> >
> > >> >>> wrote:
> > >> 
> > >>  On Fri, Dec 30, 2016 at 1:12 PM, Denis Magda 
> > >> wrote:
> > >> 
> > >> > What’s about ‘localNodeName’?
> > >> >
> > >> 
> > >>  Why is it better than "nodeName"? Isn't it obvious that the name
> is
> > >> for
> > >> >>> the
> > >>  local node?
> > >> >>>
> > >> >>>
> > >> >
> > >>
> > >
> >
>
>
>


-- 
Kind regards,
Alexander.

[jira] [Created] (IGNITE-4533) GridDhtPartitionsExchangeFuture stores unnecessary messages after processing done

[Alexandr Kuramshin (JIRA)]

Alexandr Kuramshin created IGNITE-4533:
--

 Summary: GridDhtPartitionsExchangeFuture stores unnecessary 
messages after processing done
 Key: IGNITE-4533
 URL: https://issues.apache.org/jira/browse/IGNITE-4533
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 1.8, 1.7
Reporter: Alexandr Kuramshin


After GridDhtPartitionsExchangeFuture has been completed, 
GridCachePartitionExchangeManager still stores it in field ExchangeFutureSet 
exchFuts (for race condition handling).

But many GridDhtPartitionsSingleMessage objects stored in field 
ConcurrentMap msgs is not needed after 
the future has been processed.

This map should be cleared in the end of the method onAllReceived().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Contributing to Ignite.NET

[Pavel Tupitsyn]

"Pick a Ticket" updated with Ignite.NET section.

On Tue, Jan 10, 2017 at 12:11 AM, Denis Magda  wrote:

> Great, Sergey thanks! I’ve added you to the contributors list on Ignite’s
> main site.
> https://ignite.apache.org/community/resources.html#people <
> https://ignite.apache.org/community/resources.html#people>
>
> Pavel, could you add a number of .NET related tickets to this “Pick a
> Ticket” section?
> https://ignite.apache.org/community/contribute.html#pick-ticket <
> https://ignite.apache.org/community/contribute.html#pick-ticket>
>
> —
> Denis
>
> > On Jan 9, 2017, at 4:40 AM, Pavel Tupitsyn  wrote:
> >
> > Sergey, I've merged your fix for IGNITE-4412.
> >
> > Looking forward to more contributions from you!
> >
> > Pavel
> >
> > On Thu, Jan 5, 2017 at 8:15 PM, Pavel Tupitsyn 
> wrote:
> >
> >> Hi Sergey,
> >>
> >> I've added you to project contributors in Jira.
> >>
> >> I can suggest the following .NET tickets to get familiar with the
> project:
> >>
> >> IGNITE-4412 .NET: NLogLoggerTest is not thread safe
> >> 
> >> IGNITE-3170 .NET: Add user-friendly ToString overrides for public types
> >> 
> >> IGNITE-1508 .NET: Create common ToString() method
> >> 
> >>
> >> Let me know if you have any questions.
> >>
> >> Thanks,
> >>
> >> Pavel.
> >>
> >>
> >> On Thu, Jan 5, 2017 at 8:10 PM, Guru Stron 
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> My name is Sergey, i would like to contribute to Ignite.NET.
> >>> Can you please suggest a ticket for me to start. My Jira username is
> >>> GuruStron.
> >>>
> >>
> >>
>
>

[GitHub] ignite pull request #1396: IGNITE-4493: ODBC: Added missing diagnostic recor...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1396


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---