[jira] [Created] (IGNITE-7510) IgnitePdsClientNearCachePutGetTest fails flaky on TC

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-7510:


 Summary: IgnitePdsClientNearCachePutGetTest fails flaky on TC
 Key: IGNITE-7510
 URL: https://issues.apache.org/jira/browse/IGNITE-7510
 Project: Ignite
  Issue Type: Test
  Components: persistence
 Environment: Muting this test until this ticket is fixed
Reporter: Alexey Goncharuk






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

RE: Obtaining metadata about items in the cache

[Raymond Wilson]

I’d like to carry this conversation further, cross-posting to dev list:



I now have possible production use cases for accessing cache key
[metadata].



As an example, suppose I want to scan all keys from a cache that may
contain large amounts of data and perform some operation on a few of them,
based on the value of the key itself.



In this use-case the IO bandwidth required for keys & data might be as much
as a 1000 times the bandwidth required for keys alone, even when
considering request parallelization and co-location.



I imagine that Ignite can internally scan cache keys as a part of its
internal query operations. Is that correct? If so, would it be difficult to
expose this kind of functionality in the Ignite API?



Thanks,

Raymond.



*From:* Raymond Wilson [mailto:raymond_wil...@trimble.com]
*Sent:* Monday, December 4, 2017 11:26 PM
*To:* 'u...@ignite.apache.org' 
*Subject:* RE: Obtaining metadata about items in the cache



Thanks Alexey.



This would certainly reduce the IO, but does still require all the data to
be read.



My use case is not really a production one: I want to iterate all items in
the cache to determine if the page size for persistency was suitable.
Reading all the data is not too painful, but a meta data scan would be much
faster, especially if spread across the cluster in your example below.



Raymond.



*From:* Alexey Kukushkin [mailto:kukushkinale...@gmail.com
]
*Sent:* Monday, December 4, 2017 11:10 PM
*To:* u...@ignite.apache.org
*Subject:* Re: Obtaining metadata about items in the cache



Hi Raymond,



I do not think Ignite supports iterating other metadata but you could
minimise IO by:

   - collocated processing (analyse entries locally without sending them
   over the network)
   - working with binary object representation directly (without
   serialisation/deserialisation)

You could send you analysis job to each partition and then execute a local
scan query that would work with binary objects. In the below code I
highlighted the affinityCall, withKeepBinary and setLocal methods you need
to use to achieve the above optimizations:



IgniteCompute compute = ignite.compute(ignite.cluster().forServers());

for (int i = 0; i < ignite.affinity("CacheName").partitions(); ++i) {



compute.*affinityRun*(Collections.singletonList("CacheName"), i, () -> {



IgniteCache cache =
ignite.cache("CacheName").*withKeepBinary*();



IgniteQuery<...> qry = new ScanQuery<>( (k, v) -> { ... };

qry.*setLocal*(true);



QueryCursor cur = cache.query( );

   ...



});



}











On Mon, Dec 4, 2017 at 1:33 AM, Raymond Wilson 
wrote:

Hi,



I’d like to be able to scan all the items in a cache where all I am
interested in is the cache key and other metadata about the cached item
(such as its size).



I can do this now by running a cache query that simple reads out all the
cache items, but this is a lot of IO when I don’t care about the content of
the items themselves.



Does anyone here do this?



Thanks,

Raymond.







-- 

Best regards,

Alexey

Re: Partition loss policy to disable cache completely

[Yakov Zhdanov]

I'm still not sure on what Val has suggested. Dmitry, Val, Do you have any
concrete API/algorithm in mind?

--Yakov

Re: Partition loss policy to disable cache completely

[Dmitriy Setrakyan]

Why not just add a new policy as Val suggested?

⁣D.​

On Jan 23, 2018, 4:44 PM, at 4:44 PM, Yakov Zhdanov  wrote:
>Val, your computation fails once it reaches the absent partition. Agree
>with the point that any new computation should not start. Guys, any
>ideas
>on how to achieve that? I would think of scan/sql query checking that
>there
>is no data loss on current topology version prior to start. Val, please
>note that along with queries that require full data set there can be
>some
>operations that require only limited partitions (most probably only 1).
>So,
>no point in such strict limitations. Agree?
>
>--Yakov

Re: Partition loss policy to disable cache completely

[Yakov Zhdanov]

Val, your computation fails once it reaches the absent partition. Agree
with the point that any new computation should not start. Guys, any ideas
on how to achieve that? I would think of scan/sql query checking that there
is no data loss on current topology version prior to start. Val, please
note that along with queries that require full data set there can be some
operations that require only limited partitions (most probably only 1). So,
no point in such strict limitations. Agree?

--Yakov

Re: Partition loss policy to disable cache completely

[Valentin Kulichenko]

Yakov,

I still think there are valid use cases. From the top of my head - what if
one wants to iterate through multiple partitions and do some calculations?
Locking and transactional semantics are not needed, but if some of the data
is LOST, computation should fail, and new computations should not even
start. Basically, you assume that if two entries are stored in different
partitions and not accessed in same transaction, then these entries are
completely unrelated to each other. From my expirience, this assumption is
incorrect.

-Val

On Tue, Jan 23, 2018 at 11:03 AM, Yakov Zhdanov  wrote:

> Alex, I am against reducing cluster operation. I tried to explain in the
> prev email that it is impossible to have consistent approach here. You can
> prohibit operations only after exchange completes. However, in this case
> plenty of transactions are committed on previous cache topology having
> nodes they do not touch crashed/left the grid.
>
> --Yakov
>
> 2018-01-23 9:28 GMT-08:00 Alexey Goncharuk :
>
> > Valentin,
> >
> > I am ok with having a policy which prohibits all cache operations, and
> this
> > is not very hard to implement. Although, I agree with Yakov - I do not
> see
> > any point in reducing cluster availability when operations can be safely
> > completed.
> >
> > 2018-01-23 2:22 GMT+03:00 Yakov Zhdanov :
> >
> > > Val,
> > >
> > > Your suggestion to prohibit any cache operation on partition loss does
> > not
> > > make sense to me. Why should I care about some partition during
> > particular
> > > operation if I don't access it? Imagine I use data on nodes A and B
> > > performing reads and writes and node C crashes in the middle of tx.
> > Should
> > > my tx be rolled back? I think no.
> > >
> > > As far as difference it seems that IGNORE resets lost status for
> affected
> > > partitions and READ_WRITE_ALL does not.
> > >
> > > * @see Ignite#resetLostPartitions(Collection)
> > > * @see IgniteCache#lostPartitions()
> > >
> > > --Yakov
> > >
> > > 2018-01-17 14:36 GMT-08:00 Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com>:
> > >
> > > > Folks,
> > > >
> > > > Our PartitionLossPolicy allows to disable operations on lost
> > partitions,
> > > > however all available policies allow any operations on partitions
> that
> > > were
> > > > not lost. It seems to me it can be very useful to also have a policy
> > that
> > > > completely blocks the cache in case of data loss. Is it possible to
> add
> > > > one?
> > > >
> > > > And as a side question: what is the difference between READ_WRITE_ALL
> > and
> > > > IGNORE policies? Looks like both allow both read and write on all
> > > > partitions.
> > > >
> > > > -Val
> > > >
> > >
> >
>

[jira] [Created] (IGNITE-7509) Adjust "build" and "getting started" sections of documentation for ML module

[Yury Babak (JIRA)]

Yury Babak created IGNITE-7509:
--

 Summary: Adjust "build" and "getting started" sections of 
documentation for ML module
 Key: IGNITE-7509
 URL: https://issues.apache.org/jira/browse/IGNITE-7509
 Project: Ignite
  Issue Type: Task
  Components: documentation, ml
Reporter: Yury Babak
Assignee: Yury Babak
 Fix For: 2.4






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [SparkDataFrame] Query Optimization. Prototype

[Denis Magda]

Agree. The unsupported functions should be mentioned on the page that will 
cover Ignite ANSI-99 compliance. We have first results available for CORE 
features of the specification:
https://ggsystems.atlassian.net/wiki/spaces/GG/pages/45093646/ANSI+SQL+99 


That’s on my radar. I’ll take care of this.

—
Denis

> On Jan 23, 2018, at 10:31 AM, Dmitriy Setrakyan  wrote:
> 
> I think we need a page listing the unsupported functions with explanation
> why, which is either it does not make sense in Ignite or is planned in
> future release.
> 
> Sergey, do you think you will be able to do it?
> 
> D.
> 
> On Tue, Jan 23, 2018 at 12:05 AM, Serge Puchnin 
> wrote:
> 
>> yes, the Cust function is supporting both Ignite and H2.
>> 
>> I've updated the documentation for next system functions:
>> CASEWHEN Function, CAST, CONVERT, TABLE
>> 
>> https://apacheignite-sql.readme.io/docs/system-functions
>> 
>> And for my mind, next functions aren't applicable for Ignite:
>> ARRAY_GET, ARRAY_LENGTH, ARRAY_CONTAINS, CSVREAD, CSVWRITE, DATABASE,
>> DATABASE_PATH, DISK_SPACE_USED, FILE_READ, FILE_WRITE, LINK_SCHEMA,
>> MEMORY_FREE, MEMORY_USED, LOCK_MODE, LOCK_TIMEOUT, READONLY, CURRVAL,
>> AUTOCOMMIT, CANCEL_SESSION, IDENTITY, NEXTVAL, ROWNUM, SCHEMA,
>> SCOPE_IDENTITY, SESSION_ID, SET, TRANSACTION_ID, TRUNCATE_VALUE, USER,
>> H2VERSION
>> 
>> Also an issue was created for review current documentation:
>> https://issues.apache.org/jira/browse/IGNITE-7496
>> 
>> --
>> BR,
>> Serge
>> 
>> 
>> 
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>> 

Re: Partition loss policy to disable cache completely

[Yakov Zhdanov]

Alex, I am against reducing cluster operation. I tried to explain in the
prev email that it is impossible to have consistent approach here. You can
prohibit operations only after exchange completes. However, in this case
plenty of transactions are committed on previous cache topology having
nodes they do not touch crashed/left the grid.

--Yakov

2018-01-23 9:28 GMT-08:00 Alexey Goncharuk :

> Valentin,
>
> I am ok with having a policy which prohibits all cache operations, and this
> is not very hard to implement. Although, I agree with Yakov - I do not see
> any point in reducing cluster availability when operations can be safely
> completed.
>
> 2018-01-23 2:22 GMT+03:00 Yakov Zhdanov :
>
> > Val,
> >
> > Your suggestion to prohibit any cache operation on partition loss does
> not
> > make sense to me. Why should I care about some partition during
> particular
> > operation if I don't access it? Imagine I use data on nodes A and B
> > performing reads and writes and node C crashes in the middle of tx.
> Should
> > my tx be rolled back? I think no.
> >
> > As far as difference it seems that IGNORE resets lost status for affected
> > partitions and READ_WRITE_ALL does not.
> >
> > * @see Ignite#resetLostPartitions(Collection)
> > * @see IgniteCache#lostPartitions()
> >
> > --Yakov
> >
> > 2018-01-17 14:36 GMT-08:00 Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> >
> > > Folks,
> > >
> > > Our PartitionLossPolicy allows to disable operations on lost
> partitions,
> > > however all available policies allow any operations on partitions that
> > were
> > > not lost. It seems to me it can be very useful to also have a policy
> that
> > > completely blocks the cache in case of data loss. Is it possible to add
> > > one?
> > >
> > > And as a side question: what is the difference between READ_WRITE_ALL
> and
> > > IGNORE policies? Looks like both allow both read and write on all
> > > partitions.
> > >
> > > -Val
> > >
> >
>

Re: [SparkDataFrame] Query Optimization. Prototype

[Dmitriy Setrakyan]

I think we need a page listing the unsupported functions with explanation
why, which is either it does not make sense in Ignite or is planned in
future release.

Sergey, do you think you will be able to do it?

D.

On Tue, Jan 23, 2018 at 12:05 AM, Serge Puchnin 
wrote:

> yes, the Cust function is supporting both Ignite and H2.
>
> I've updated the documentation for next system functions:
> CASEWHEN Function, CAST, CONVERT, TABLE
>
> https://apacheignite-sql.readme.io/docs/system-functions
>
> And for my mind, next functions aren't applicable for Ignite:
> ARRAY_GET, ARRAY_LENGTH, ARRAY_CONTAINS, CSVREAD, CSVWRITE, DATABASE,
> DATABASE_PATH, DISK_SPACE_USED, FILE_READ, FILE_WRITE, LINK_SCHEMA,
> MEMORY_FREE, MEMORY_USED, LOCK_MODE, LOCK_TIMEOUT, READONLY, CURRVAL,
> AUTOCOMMIT, CANCEL_SESSION, IDENTITY, NEXTVAL, ROWNUM, SCHEMA,
> SCOPE_IDENTITY, SESSION_ID, SET, TRANSACTION_ID, TRUNCATE_VALUE, USER,
> H2VERSION
>
> Also an issue was created for review current documentation:
> https://issues.apache.org/jira/browse/IGNITE-7496
>
> --
> BR,
> Serge
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

[GitHub] ignite pull request #3425: IGNITE-7506 setting BaselineTopology in compatibi...

[sergey-chugunov-1985]

GitHub user sergey-chugunov-1985 opened a pull request:

https://github.com/apache/ignite/pull/3425

IGNITE-7506 setting BaselineTopology in compatibilityMode

…n old nodes left the cluster

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7506

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3425.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3425


commit 471a47788fa0fb08c19bbb8b4acb8af0a5ba49f1
Author: Sergey Chugunov 
Date:   2018-01-23T16:43:23Z

IGNITE-7506 establishing of BaselineTopology in compatibilityMode when old 
nodes left the cluster




---

Ignite diagnostic (SQL system views)

[Alex Plehanov]

Hello, Igniters!

For Ignite diagnostic usually it’s helpful to get some Ignite internals
information. But currently, in my opinion, there are no convenient tools
for this purpose:

·Some issues can be solved by analyzing log files. Log files are
useful for dumps, but sometimes they are difficult to read. Also
interesting metrics can’t be received runtime by request, we need to wait
until Ignite will write these metrics by timeout or other events.

·JMX is useful for scalar metrics. Complex and table data can also
be received, but it’s difficult to read, filter and sort them without
processing by specialized external tools. For most frequently used cases
almost duplicating metrics are created to show data in an easy-to-read form.

·Web-console is able to show table and complex data. Perhaps,
someday  web-console will contain all necessary dashboards for most problem
investigation, but some non-trivial queries will not be covered anyway.
Also web-console needs additional infrastructure to work.

·External “home-made” tools can be used for non-trivial cases. They
cover highly specialized cases and usually can’t be used as general purpose
tools.

Sometimes we are forced to use more than one tool and join data by hands
(for example, current thread dump and data from logs).

Often RDBMS for diagnostic purposes provides system views (for example,
DBA_% and V$% in Oracle), which can be queried by SQL. This solution makes
all internal diagnostic information available in a readable form (with all
possible filters and projections) without using any other internal or
external tools. My proposal is to create similar system views in Ignite.

I implement working prototype (PR: [1]). It contains views:

IGNITE_SYSTEM_VIEWS

Registered system views

IGNITE_INSTANCE

Ignite instance

IGNITE_JVM_THREADS

JVM threads

IGNITE_JVM_RUNTIME

JVM runtime

IGNITE_JVM_OS

JVM operating system

IGNITE_CACHES

Ignite caches

IGNITE_CACHE_CLUSTER_METRICS

Ignite cache cluster metrics

IGNITE_CACHE_NODE_METRICS

Ignite cache node metrics

IGNITE_CACHE_GROUPS

Cache groups

IGNITE_NODES

Nodes in topology

IGNITE_NODE_HOSTS

Node hosts

IGNITE_NODE_ADDRESSES

Node addresses

IGNITE_NODE_ATTRIBUTES

Node attributes

IGNITE_NODE_METRICS

Node metrics

IGNITE_TRANSACTIONS

Active transactions

IGNITE_TRANSACTION_ENTRIES

Cache entries used by transaction

IGNITE_TASKS

Active tasks

IGNITE_PART_ASSIGNMENT

Partition assignment map

IGNITE_PART_ALLOCATION

Partition allocation map



There are much more useful views can be implemented (executors diagnostic,
SPIs diagnostic, etc).

Some usage examples:

Cache groups and their partitions, which used by transaction more than 5
minutes long:

SELECT cg.CACHE_OR_GROUP_NAME, te.KEY_PARTITION, count(*) AS ENTITIES_CNT
FROM INFORMATION_SCHEMA.IGNITE_TRANSACTIONS t
JOIN INFORMATION_SCHEMA.IGNITE_TRANSACTION_ENTRIES te ON t.XID = te.XID
JOIN INFORMATION_SCHEMA.IGNITE_CACHES c ON te.CACHE_NAME = c.NAME
JOIN INFORMATION_SCHEMA.IGNITE_CACHE_GROUPS cg ON c.GROUP_ID = cg.ID
WHERE t.START_TIME < TIMESTAMPADD('MINUTE', -5, NOW())
GROUP BY cg.CACHE_OR_GROUP_NAME, te.KEY_PARTITION



Average CPU load on server nodes grouped by operating system:

SELECT na.VALUE, COUNT(n.ID), AVG(nm.AVG_CPU_LOAD) AVG_CPU_LOAD
FROM INFORMATION_SCHEMA.IGNITE_NODES n
JOIN INFORMATION_SCHEMA.IGNITE_NODE_ATTRIBUTES na ON na.NODE_ID = n.ID AND
na.NAME = 'os.name'
JOIN INFORMATION_SCHEMA.IGNITE_NODE_METRICS nm ON nm.NODE_ID = n.ID
WHERE n.IS_CLIENT = false
GROUP BY na.VALUE



Top 5 nodes by puts to cache ‘cache’:

SELECT cm.NODE_ID, cm.CACHE_PUTS FROM
INFORMATION_SCHEMA.IGNITE_CACHE_NODE_METRICS cm
WHERE cm.CACHE_NAME = 'cache'
ORDER BY cm.CACHE_PUTS DESC
LIMIT 5



Does this implementation interesting to someone else? Maybe any views are
redundant? Which additional first-priority views must be implemented? Any
other thoughts or proposal?

[1] https://github.com/apache/ignite/pull/3413

Re: Partition loss policy to disable cache completely

[Alexey Goncharuk]

Valentin,

I am ok with having a policy which prohibits all cache operations, and this
is not very hard to implement. Although, I agree with Yakov - I do not see
any point in reducing cluster availability when operations can be safely
completed.

2018-01-23 2:22 GMT+03:00 Yakov Zhdanov :

> Val,
>
> Your suggestion to prohibit any cache operation on partition loss does not
> make sense to me. Why should I care about some partition during particular
> operation if I don't access it? Imagine I use data on nodes A and B
> performing reads and writes and node C crashes in the middle of tx. Should
> my tx be rolled back? I think no.
>
> As far as difference it seems that IGNORE resets lost status for affected
> partitions and READ_WRITE_ALL does not.
>
> * @see Ignite#resetLostPartitions(Collection)
> * @see IgniteCache#lostPartitions()
>
> --Yakov
>
> 2018-01-17 14:36 GMT-08:00 Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
>
> > Folks,
> >
> > Our PartitionLossPolicy allows to disable operations on lost partitions,
> > however all available policies allow any operations on partitions that
> were
> > not lost. It seems to me it can be very useful to also have a policy that
> > completely blocks the cache in case of data loss. Is it possible to add
> > one?
> >
> > And as a side question: what is the difference between READ_WRITE_ALL and
> > IGNORE policies? Looks like both allow both read and write on all
> > partitions.
> >
> > -Val
> >
>

Re: cache operation failed after transaction rolled back due to deadlock

[Yakov Zhdanov]

Alex Goncharuk, can you please take a look and comment? Test seems to be
valid from my standpoint.

Yakov Zhdanov,
www.gridgain.com

2018-01-22 23:14 GMT-08:00 ALEKSEY KUZNETSOV :

>
> created ticket with reproducer [1]
>
> I think the fix should be to clear context in tx.close()
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7486
>
> > 23 янв. 2018 г., в 1:56, Yakov Zhdanov  написал(а):
> >
> > Guys, can you check if you call tx.close(); or properly use
> > try-with-resources construct?
> >
> > Tx context should not be cleared automatically otherwise user would not
> get
> > any notification that original transaction failed. I believe context
> should
> > be cleared on tx.close().
> >
> > Anyway, let's take a look at reproducer from Alexey first.
> >
> > --Yakov
> >
> > 2018-01-22 2:02 GMT-08:00 ALEKSEY KUZNETSOV :
> >
> >> Sure
> >>
> >> пн, 22 янв. 2018 г. в 12:25, Andrey Gura :
> >>
> >>> It seems that problem isn't related with deadlock detection and should
> be
> >>> reproducible when deadlock detection disabled.
> >>>
> >>> Anyway it sounds like a bug. Could you please file a ticket and provide
> >>> minimal reproducer?
> >>>
> >>> 19 янв. 2018 г. 3:55 PM пользователь "ALEKSEY KUZNETSOV" <
> >>> alkuznetsov...@gmail.com> написал:
> >>>
>  Hi, Igntrs!
> 
> 
> 
>  When you have your transaction rolled back due to detected deadlock,
> >> you
>  are unabled to perform cache operations (in thread where tx was
> started
> >>> and
>  rolled back), because it leads to TransactionTimeoutException.
> 
> 
> 
>  The reason of such behavior is that tx thread map
> (txManager#threadMap)
> >>> was
>  not cleared from tx when roll back occured.
> 
>  In GridNearTxLocal#onTimeout you can find comment on that :
> 
>  *// Note: if rollback asynchronously on timeout should not clear
> thread
>  map*
> 
>  *// since thread started tx still should be able to see this tx.*
> 
>  Cache operation picks up tx from that map and throws exception.
> 
> 
> 
>  So, one must create new thread in order to perform cache operations?
> 
> 
>  --
> 
>  *Best Regards,*
> 
>  *Kuznetsov Aleksey*
> 
> >>>
> >>
> >>
> >> --
> >>
> >> *Best Regards,*
> >>
> >> *Kuznetsov Aleksey*
> >>
>

[jira] [Created] (IGNITE-7508) GridKernalContextImpl::isDaemon creates contention on system properties access

[Stanislav Lukyanov (JIRA)]

Stanislav Lukyanov created IGNITE-7508:
--

 Summary: GridKernalContextImpl::isDaemon creates contention on 
system properties access
 Key: IGNITE-7508
 URL: https://issues.apache.org/jira/browse/IGNITE-7508
 Project: Ignite
  Issue Type: Bug
  Components: general
Reporter: Stanislav Lukyanov


GridKernalContextImpl::isDaemon reads system property IGNITE_DAEMON on every 
call, leading to contention on the system properties lock. The lock is shown as 
contended in the Java Mission Control analysis of a JFR recording of the 
IgnitePutGetBenchmark.

The fix would be to cache IGNITE_DAEMON value (e.g. in IgniteUtils) since it 
isn't supposed to be changed during the JVM's lifetime anyway.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3423: IGNITE-7507: Ignite node performance drop during ...

[dspavlov]

GitHub user dspavlov opened a pull request:

https://github.com/apache/ignite/pull/3423

IGNITE-7507: Ignite node performance drop during checkpoint start

Store metapage eviction causes long checkpoint lock hold time

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7507

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3423.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3423


commit 0d557e567d7b770f2c91306071af9b849f0e66b3
Author: dspavlov 
Date:   2018-01-23T16:20:43Z

IGNITE-7507: Ignite node performance drop during checkpoint start: store 
metapage eviction causes long checkpoint lock hold time




---

[GitHub] ignite pull request #3424: IGNITE-5571 with 4191

[alexpaschenko]

GitHub user alexpaschenko opened a pull request:

https://github.com/apache/ignite/pull/3424

IGNITE-5571 with 4191



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5571-with-4191

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3424.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3424


commit ac475bc18dee0c6f19aa5fb9f1cfeae164deea94
Author: sboikov 
Date:   2017-09-28T10:56:41Z

Merge remote-tracking branch 'remotes/origin/master' into ignite-3478

commit 7f4defd0926325dfb9d6840156dfa64309ae7956
Author: sboikov 
Date:   2017-09-29T11:20:00Z

Merge remote-tracking branch 'remotes/origin/master' into ignite-3478

commit 761e43d3039cf8c58c9c7b0ec2dde68238d71647
Author: sboikov 
Date:   2017-09-29T11:29:03Z

ignite-3479 Coordinators reassign on failure

commit fb3ee2478eccad3a9ab12c42ef99d0364af5194f
Author: sboikov 
Date:   2017-09-29T11:51:28Z

ignite-3478

commit 8cf2aad733aeda8ae4575e093315c052b682b455
Author: sboikov 
Date:   2017-09-29T12:38:37Z

ignite-3478 Support for streamer

commit e8a5a082026c47d40c1b6e482f5a9d83115b927a
Author: sboikov 
Date:   2017-10-02T07:48:05Z

Merge remote-tracking branch 'remotes/origin/master' into ignite-3478

commit 685c688f7a1ec2ea1f5904c5c7df7baff41c15af
Author: sboikov 
Date:   2017-10-02T09:39:29Z

ignite-3478

commit 27b2be4b98e12a32037b1fe6146b85d939c95952
Author: sboikov 
Date:   2017-10-03T07:41:18Z

ignite-3478

commit 07a56028a2150208c79e64c280dc989624c47c47
Author: sboikov 
Date:   2017-10-03T14:34:20Z

ignite-3478

commit fdfe779f063d015dd0fcb7f2880a1b227a3e190f
Author: sboikov 
Date:   2017-10-03T14:34:30Z

ignite-3478

commit fc7b7e21c87689866eee83c6e73771333ab0492e
Author: sboikov 
Date:   2017-10-04T08:35:00Z

Merge remote-tracking branch 'remotes/origin/master' into ignite-3478

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/GridKernalContext.java
#   
modules/core/src/main/java/org/apache/ignite/internal/GridKernalContextImpl.java
#   modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/reader/StandaloneGridKernalContext.java

commit a1d9ddd10dfa935b81703c4ba1e2670fba282389
Author: sboikov 
Date:   2017-10-04T08:35:53Z

Merge remote-tracking branch 'remotes/origin/master' into ignite-3478

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/GridKernalContext.java
#   
modules/core/src/main/java/org/apache/ignite/internal/GridKernalContextImpl.java
#   modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/reader/StandaloneGridKernalContext.java

commit 410c84fcf0d6f7bbe8373f996b6633c4d5b15b61
Author: sboikov 
Date:   2017-10-05T08:40:04Z

Merge remote-tracking branch 'remotes/origin/master' into ignite-3478

commit 921404a6f5b256cf6404822bc7e439f2cad44c5d
Author: sboikov 
Date:   2017-10-09T10:41:58Z

Merge remote-tracking branch 'remotes/origin/master' into ignite-3478

commit fd53c1a8f4e905a7aba469eb5decf38c50b7708e
Author: sboikov 
Date:   2017-10-09T11:42:43Z

ignite-5937 Added BPlusTree.iterate for more optimal mvcc search

commit 61b46c47f01c702a17b683c69a80a7c85c389b84
Author: sboikov 
Date:   2017-10-10T11:39:21Z

Merge remote-tracking branch 'remotes/origin/master' into ignite-3478

commit 69fd367dc2eb9e76d89f344063d9788171cb359f
Author: sboikov 
Date:   2017-10-11T08:18:10Z

Merge remote-tracking branch 'remotes/origin/master' into ignite-3478

commit 970cf47a51dc6e754677b00e85e60effc48083ba
Author: sboikov 
Date:   2017-10-11T08:24:32Z

ignite-3478 Support for removes

commit a8a46084c711ce029d391915e3bfa493220664a0
Author: Alexander Paschenko 
Date:   2017-10-11T16:19:47Z

Merge branch 'master' into ignite-5571

commit 35cb06848097a6db56322358d1d48cacf9f1f35a
Author: Alexander Paschenko 
Date:   2017-10-11T16:21:14Z

Post-merge fx 1

commit 23742962f8d539aac33a7ac953f09a1407b330e9
Author: sboikov 
Date:   2017-10-12T10:36:52Z

Merge remote-tracking branch 'remotes/origin/master' into ignite-3478

commit f29d4bc50801c530ef856d168fb637b0fad1c27b
Author: sboikov 
Date:   2017-10-12T10:43:07Z

ignite-3478 Fixed query ack

commit f23d87afc39118702d44ef1fda707956c59e9724
Author: 

[jira] [Created] (IGNITE-7507) Ignite node performance drop during checkpoint start: store metapage eviction causes long checkpoint lock hold time

[Dmitriy Pavlov (JIRA)]

Dmitriy Pavlov created IGNITE-7507:
--

 Summary: Ignite node performance drop during checkpoint start: 
store metapage eviction causes long checkpoint lock hold time
 Key: IGNITE-7507
 URL: https://issues.apache.org/jira/browse/IGNITE-7507
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov
 Fix For: 2.5


Store metadata Page eviction becomes very expensive operation during checkpoint 
start.

These pages reads hands ignite node until metadata will be loaded from disk.

Following store (paritition) metapages:
- Partition Metadata Page
- Freelist Meta Page
- Partition Counters IO
required during execution of saveStoreMetadata() & markCheckpointBegin()

If this page is not available in memory, it is loaded from disk.
But such loads are done while holding checkpointLock (in write mode).
Example of timing:
- checkpointLockWait=75ms, checkpointLockHoldTime=2653ms, pages=696120

All this time worker threads are not able to put any data to any cache.

It is required to avoid eviction of such pages (evict it with lowest priority 
than dirty page).

(Full stacktrace)   
{noformat} db-checkpoint-thread-#40%checkpoint.IgniteMassLoadSandboxTest1% 
Id=63 WAITING

at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.getUninterruptibly(GridFutureAdapter.java:145)
at 
org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIO.read(AsyncFileIO.java:95)
at 
org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.read(FilePageStore.java:324)
at 
org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.read(FilePageStoreManager.java:306)
at 
org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.read(FilePageStoreManager.java:291)
at 
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:656)
at 
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:576)
at 
org.apache.ignite.internal.processors.cache.persistence.DataStructure.acquirePage(DataStructure.java:130)
at 
org.apache.ignite.internal.processors.cache.persistence.freelist.PagesList.saveMetadata(PagesList.java:301)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.saveStoreMetadata(GridCacheOffheapManager.java:196)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.onCheckpointBegin(GridCacheOffheapManager.java:168)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.markCheckpointBegin(GridCacheDatabaseSharedManager.java:3022)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.doCheckpoint(GridCacheDatabaseSharedManager.java:2719)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseSharedManager.java:2644)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:748)
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7506) GridClusterStateProcessor#compatibilityMode flag never gets reset to false

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-7506:
---

 Summary: GridClusterStateProcessor#compatibilityMode flag never 
gets reset to false
 Key: IGNITE-7506
 URL: https://issues.apache.org/jira/browse/IGNITE-7506
 Project: Ignite
  Issue Type: Bug
Reporter: Sergey Chugunov
Assignee: Sergey Chugunov


When Ignite node of version 2.4+ joins cluster with older versions of Ignite 
running it sets *compatibilityMode* flag to handle **cluster state in a special 
way (excluding BaselineTopology from discovery exchange process and so on).

But when all old nodes are turned off there is no way except full cluster 
restart to enable setting very first **BaselineTopology.

This should be addressed and some way to exit compatibilityMode without full 
cluster restart should be provided.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3421: IGNITE-7500 Partition update counters hot fix

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3421


---

[GitHub] ignite pull request #3422: IGNITE-7443: ODBC now use native batching capabil...

[isapego]

GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/3422

IGNITE-7443: ODBC now use native batching capabilities on the server side



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7443

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3422.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3422


commit b83db1bae0cce492669f3a8bd98908fcdaf101e5
Author: Igor Sapego 
Date:   2018-01-22T16:19:24Z

IGNITE-7443: Implemented for ODBC. Adjusted tests.

commit 1cd6a3af5b663702df2f47b6461fc9e170d27a64
Author: Igor Sapego 
Date:   2018-01-23T10:43:51Z

IGNITE-7443: Added support for SQL_ATTR_PARAM_STATUS_PTR

commit 65e480ffe234ac37d82b8cf9c0758c9baddcb7a3
Author: Igor Sapego 
Date:   2018-01-23T13:05:13Z

IGNITE-7443: Adjusted tests. Fixed bug

commit b4c18d2624e07898a779ea50d45333b4f94752a5
Author: Igor Sapego 
Date:   2018-01-23T14:04:19Z

IGNITE-7443: Adjusted driver info




---

Re: Ignite Semaphore Bug 7090

[Vladisav Jelisavcic]

Hi Tim,

thanks for the update! I left you a comment on Jira.

Best regards,
Vladisav

On Mon, Jan 22, 2018 at 6:17 PM, Tim Onyschak  wrote:

> Hey Vladisav,
>
> I implemented your requests. Take a look, specifically, i created an
> interface to encapsulate the NodeUpdates and let
> the DataStructuresProcessor handle the execution by checking for one type
> as opposed to multiple if checks. In this case it checks for 
> GridCacheNodeUpdate
> instance and executes onNodeRemoved. Let me know what you think.
>
> Thanks,
> Tim
>
>
>
> On Sat, Jan 20, 2018 at 8:10 AM, Vladisav Jelisavcic 
> wrote:
>
>> Hi Tim,
>>
>> I reviewed your contribution and left you some comments on the pr.
>> Thanks!
>>
>> Vladisav
>>
>> On Wed, Jan 17, 2018 at 10:14 PM, Vladisav Jelisavcic <
>> vladis...@gmail.com> wrote:
>>
>>> Hi Tim,
>>>
>>> thank you for the contribution!
>>> I'll do the review soon and let you know.
>>>
>>>
>>>
>>> On Wed, Jan 17, 2018 at 8:56 AM, Yakov Zhdanov 
>>> wrote:
>>>
 Thanks Tim! I hope Vlad can review your patch. If this does not happen
 in
 2-3 days I will take a look. Can you please let me know on weekend if I
 need to?

 --Yakov

 2018-01-16 23:36 GMT+03:00 Tim Onyschak :

 > Hey all,
 >
 > I created a patch and posted to user group, was told feed back would
 be
 > left on the jira. I wanted to see if we could get a fix in with 2.4,
 could
 > somebody please review.
 >
 > http://apache-ignite-users.70518.x6.nabble.com/Semaphore-
 > Stuck-when-no-acquirers-to-assign-permit-td18639.html
 >
 > https://issues.apache.org/jira/browse/IGNITE-7090
 >
 > https://github.com/apache/ignite/pull/3138
 >
 > Thanks,
 > Tim
 >

>>>
>>>
>>
>

[jira] [Created] (IGNITE-7505) Node not in baseline topology may initialize local partition

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-7505:


 Summary: Node not in baseline topology may initialize local 
partition
 Key: IGNITE-7505
 URL: https://issues.apache.org/jira/browse/IGNITE-7505
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Affects Versions: 2.4
Reporter: Alexey Goncharuk
Assignee: Alexey Goncharuk
 Fix For: 2.4


GridDhtPartitionTopologyImpl does not check baseline topology when initializing 
local partition on cache start



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7504) Decision tree documentation

[Yury Babak (JIRA)]

Yury Babak created IGNITE-7504:
--

 Summary: Decision tree documentation
 Key: IGNITE-7504
 URL: https://issues.apache.org/jira/browse/IGNITE-7504
 Project: Ignite
  Issue Type: Task
  Components: documentation, ml
Reporter: Yury Babak
Assignee: Artem Malykh
 Fix For: 2.4


We want to add Decision tree documentation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7503) MLP documentation

[Yury Babak (JIRA)]

Yury Babak created IGNITE-7503:
--

 Summary: MLP documentation
 Key: IGNITE-7503
 URL: https://issues.apache.org/jira/browse/IGNITE-7503
 Project: Ignite
  Issue Type: Sub-task
  Components: documentation, ml
Reporter: Yury Babak
Assignee: Artem Malykh
 Fix For: 2.4


A need to add documentation about MLP



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7502) Baseline topology should affect only persistent caches

[Ilya Lantukh (JIRA)]

Ilya Lantukh created IGNITE-7502:


 Summary: Baseline topology should affect only persistent caches
 Key: IGNITE-7502
 URL: https://issues.apache.org/jira/browse/IGNITE-7502
 Project: Ignite
  Issue Type: Improvement
Reporter: Ilya Lantukh


Non-persistent caches shouldn't restrict affinity assignment to baseline nodes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7501) Improve underlying iterators closing process for cache iterators.

[Roman Kondakov (JIRA)]

Roman Kondakov created IGNITE-7501:
--

 Summary: Improve underlying iterators closing process for cache 
iterators.
 Key: IGNITE-7501
 URL: https://issues.apache.org/jira/browse/IGNITE-7501
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Affects Versions: 2.3
Reporter: Roman Kondakov
 Fix For: 2.5


When we call {{javax.cache.Cache#iterator()}}  we get {{java.util.Iterator}} 
which doesn't have a {{close()}} method. But underlying 
{{GridCloseableIterator}} does have this method and it should be called when 
all the work with the current iterator is done. Currently calling {{close()}} 
for the underlying closeable iterator is delegated to 
{{WeakQueryCloseableIterator}}. So, {{close()}} method is usually called on a 
garbage collection phase, which is not acceptable in some situations. For 
example if MVCC is enabled this *late* iterator closing could dramatically 
increase the active queries tracking list size which  could lead to the 
performance and garbage collection ("vacuum") issues.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3421: IGNITE-7500 Partition update counters hot fix

[Jokser]

GitHub user Jokser opened a pull request:

https://github.com/apache/ignite/pull/3421

IGNITE-7500 Partition update counters hot fix



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7500

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3421.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3421


commit e7ca9b65a68de7752195c8f4d2b5180f3c77d19f
Author: Dmitriy Govorukhin 
Date:   2017-11-13T18:52:47Z

ignite-blt-merge -> ignite-2.4.1

commit cc8168fc184bb7f5e3cc3bbb0743397097f78bfb
Author: Dmitriy Govorukhin 
Date:   2017-11-13T19:13:01Z

merge ignite-pitr-rc1 -> ignite-2.4.1

commit 87e6d74cf6a251c7984f9e68c391f790feccc281
Author: Dmitriy Govorukhin 
Date:   2017-11-14T12:49:33Z

ignite-gg-12877 Compact consistent ID in WAL

commit 9f5a22711baea05bd37ab07c8f928a4837dd83a4
Author: Ilya Lantukh 
Date:   2017-11-14T14:12:28Z

Fixed javadoc.

commit d5af2d78dd8eef8eca8ac5391d31d8c779649bb0
Author: Alexey Kuznetsov 
Date:   2017-11-15T08:09:00Z

IGNITE-6913 Baseline: Added new options to controls.sh for baseline 
manipulations.

commit 713924ce865752b6e99b03bd624136541cea5f9f
Author: Sergey Chugunov 
Date:   2017-11-15T09:03:12Z

IGNITE-5850 failover tests for cache operations during BaselineTopology 
changes

commit b65fd134e748d496f732ec2aa0953a0531f544b8
Author: Ilya Lantukh 
Date:   2017-11-15T12:54:35Z

TX read logging if PITR is enabled.

commit 9b2a567c0e04dc33116b51f88bee75f76e9107d1
Author: Ilya Lantukh 
Date:   2017-11-15T13:45:16Z

TX read logging if PITR is enabled.

commit 993058ccf0b2b8d9e80750c3e45a9ffa31d85dfa
Author: Dmitriy Govorukhin 
Date:   2017-11-15T13:51:54Z

ignite-2.4.1 optimization for store full set node more compacted

commit 1eba521f608d39967aec376b397b7fc800234e54
Author: Dmitriy Govorukhin 
Date:   2017-11-15T13:52:22Z

Merge remote-tracking branch 'professional/ignite-2.4.1' into ignite-2.4.1

commit 564b3fd51f8a7d1d81cb6874df66d0270623049c
Author: Sergey Chugunov 
Date:   2017-11-15T14:00:51Z

IGNITE-5850 fixed issue with initialization of data regions on node 
activation, fixed issue with auto-activation when random node joins inactive 
cluster with existing BLT

commit c6d1fa4da7adfadc80abdc7eaf6452b86a4f6aa4
Author: Sergey Chugunov 
Date:   2017-11-15T16:23:08Z

IGNITE-5850 transitionResult is set earlier when request for changing 
BaselineTopology is sent

commit d65674363163e38a4c5fdd73d1c8d8e1c7610797
Author: Sergey Chugunov 
Date:   2017-11-16T11:59:07Z

IGNITE-5850 new failover tests for changing BaselineTopology up (new node 
added to topology)

commit 20552f3851fe8825191b144179be032965e0b5c6
Author: Sergey Chugunov 
Date:   2017-11-16T12:53:43Z

IGNITE-5850 improved error message when online node is removed from baseline

commit 108bbcae4505ac904a6db774643ad600bfb42c21
Author: Sergey Chugunov 
Date:   2017-11-16T13:45:52Z

IGNITE-5850 BaselineTopology should not change on cluster deactivation

commit deb641ad3bdbf260fa60ad6bf607629652e324bd
Author: Dmitriy Govorukhin 
Date:   2017-11-17T09:45:44Z

ignite-2.4.1 truncate wal and checkpoint history on move/delete snapshot

commit 3c8b06f3659af30d1fd148ccc0f40e216a56c998
Author: Alexey Goncharuk 
Date:   2017-11-17T12:48:12Z

IGNITE-6947 Abandon remap after single map if future is done (fixes NPE)

commit ba2047e5ae7d271a677e0c418375d82d78c4023e
Author: devozerov 
Date:   2017-11-14T12:26:31Z

IGNITE-6901: Fixed assertion during 
IgniteH2Indexing.rebuildIndexesFromHash. This closes #3027.

commit abfc0466d6d61d87255d0fe38cbdf11ad46d4f89
Author: Sergey Chugunov 
Date:   2017-11-17T13:40:57Z

IGNITE-5850 tests for queries in presence of BaselineTopology

commit f4eabaf2a905abacc4c60c01d3ca04f6ca9ec188
Author: Sergey Chugunov 
Date:   2017-11-17T17:23:02Z

IGNITE-5850 implementation for setBaselineTopology(long topVer) migrated 
from wc-251

commit 4edeccd3e0b671aa277f58995df9ff9935baa95a
Author: EdShangGG 
Date:   2017-11-17T18:21:17Z

GG-13074 Multiple snapshot test failures after baseline topology is 
introduced
-adding baseline test to suite
-fixing issues with baseline

commit edae228c8f55990c15ef3044be987dcb00d6c81a
Author: EdShangGG 
Date:   2017-11-18T10:36:41Z

hack with sleep

commit b5bffc7580a4a8ffbcc06f60c282e73979179578

[jira] [Created] (IGNITE-7500) Partition update counters may be inconsistent after rebalancing

[Pavel Kovalenko (JIRA)]

Pavel Kovalenko created IGNITE-7500:
---

 Summary: Partition update counters may be inconsistent after 
rebalancing
 Key: IGNITE-7500
 URL: https://issues.apache.org/jira/browse/IGNITE-7500
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.3
Reporter: Pavel Kovalenko
Assignee: Pavel Kovalenko


Problem:
If partition rebalance requires more than one batch we are not sending `Clear` 
flag for it in the last supply message and as result don't set updateCounter to 
right value.

Temporary solution:
Send `Clear` flags for partitions that were fully rebalanced in the last supply 
message. But we still have a problem with race conditions with setting 
updateCounter during concurrent rebalance and cache load.

General solution:
https://issues.apache.org/jira/browse/IGNITE-6113



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7499) DataRegionMetricsImpl#getPageSize returns ZERO for system data regions

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-7499:


 Summary: DataRegionMetricsImpl#getPageSize returns ZERO for system 
data regions
 Key: IGNITE-7499
 URL: https://issues.apache.org/jira/browse/IGNITE-7499
 Project: Ignite
  Issue Type: Bug
  Components: cache
Reporter: Alexey Kuznetsov
 Fix For: 2.5


Working on IGNITE-7492 I found that DataRegionMetricsImpl#getPageSize returns 
ZERO for system data regions.

Mmeanwhile there org.apache.ignite.internal.pagemem.PageMemory#systemPageSize 
method.

That looks a bit strange, why we need PageSize and SystemPageSize ?

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3398: IGNITE-7456: Fix wrong batch logic in distributed...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3398


---

[jira] [Created] (IGNITE-7498) Crash on startup when DEBUG log level enabled under Java 9

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-7498:
--

 Summary: Crash on startup when DEBUG log level enabled under Java 9
 Key: IGNITE-7498
 URL: https://issues.apache.org/jira/browse/IGNITE-7498
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Pavel Tupitsyn
 Fix For: 2.4


* Enable DEBUG log level in log4j config
* Start Ignite

Result:
{code}
Boot class path mechanism is not supported
at 
java.management/sun.management.RuntimeImpl.getBootClassPath(RuntimeImpl.java:99)
at 
org.apache.ignite.internal.IgniteKernal.ackClassPaths(IgniteKernal.java:2464)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:784)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1940)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7497) Web console should check connection to cluster before sending any task

[Pavel Konstantinov (JIRA)]

Pavel Konstantinov created IGNITE-7497:
--

 Summary: Web console should check connection to cluster before 
sending any task
 Key: IGNITE-7497
 URL: https://issues.apache.org/jira/browse/IGNITE-7497
 Project: Ignite
  Issue Type: Bug
Reporter: Pavel Konstantinov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3420: Fix test license header

[sunnychanwork]

GitHub user sunnychanwork opened a pull request:

https://github.com/apache/ignite/pull/3420

Fix test license header

Fix License header for CassandraSessionImplTest

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sunnychanwork/ignite 
fix-cassandrasessionimpltest-license

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3420.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3420


commit 43d2f18ac24ddfc6f94e2ff0e674242033ebdade
Author: Sunny Chan 
Date:   2018-01-23T09:44:51Z

Fix test license header




---

[GitHub] ignite pull request #3124: IGNITE-6425: Races during transaction finalizatio...

[AMashenkov]

Github user AMashenkov closed the pull request at:

https://github.com/apache/ignite/pull/3124


---

Re: [SparkDataFrame] Query Optimization. Prototype

[Nikolay Izhikov]

Serge, thank you!
Vladimir, guys, can you take a look at another questions:


* Can I know scale and precision of DECIMAL column?
  Example - [3]

* Ignite have some limitation for a *distributed* join. For example, we can 
execute join only for indexed columns. Example - [4].
* Do we have documentation for all limitations of distributed join 
implementation?
* How can I know whether concrete syntactically correct SQL query with 
join can be executed by Ignite or not?




[3] https://gist.github.com/nizhikov/2d67ed68afd473d7251dab7d79135513
[4] https://gist.github.com/nizhikov/a4389fd78636869dd38c13920b5baf2b

В Вт, 23/01/2018 в 01:05 -0700, Serge Puchnin пишет:
> yes, the Cust function is supporting both Ignite and H2.
> 
> I've updated the documentation for next system functions:
> CASEWHEN Function, CAST, CONVERT, TABLE
> 
> https://apacheignite-sql.readme.io/docs/system-functions
> 
> And for my mind, next functions aren't applicable for Ignite:
> ARRAY_GET, ARRAY_LENGTH, ARRAY_CONTAINS, CSVREAD, CSVWRITE, DATABASE,
> DATABASE_PATH, DISK_SPACE_USED, FILE_READ, FILE_WRITE, LINK_SCHEMA,
> MEMORY_FREE, MEMORY_USED, LOCK_MODE, LOCK_TIMEOUT, READONLY, CURRVAL,
> AUTOCOMMIT, CANCEL_SESSION, IDENTITY, NEXTVAL, ROWNUM, SCHEMA,
> SCOPE_IDENTITY, SESSION_ID, SET, TRANSACTION_ID, TRUNCATE_VALUE, USER,
> H2VERSION
> 
> Also an issue was created for review current documentation:
> https://issues.apache.org/jira/browse/IGNITE-7496
> 
> --
> BR,
> Serge
> 
> 
> 
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [SparkDataFrame] Query Optimization. Prototype

[Serge Puchnin]

yes, the Cust function is supporting both Ignite and H2.

I've updated the documentation for next system functions:
CASEWHEN Function, CAST, CONVERT, TABLE

https://apacheignite-sql.readme.io/docs/system-functions

And for my mind, next functions aren't applicable for Ignite:
ARRAY_GET, ARRAY_LENGTH, ARRAY_CONTAINS, CSVREAD, CSVWRITE, DATABASE,
DATABASE_PATH, DISK_SPACE_USED, FILE_READ, FILE_WRITE, LINK_SCHEMA,
MEMORY_FREE, MEMORY_USED, LOCK_MODE, LOCK_TIMEOUT, READONLY, CURRVAL,
AUTOCOMMIT, CANCEL_SESSION, IDENTITY, NEXTVAL, ROWNUM, SCHEMA,
SCOPE_IDENTITY, SESSION_ID, SET, TRANSACTION_ID, TRUNCATE_VALUE, USER,
H2VERSION

Also an issue was created for review current documentation:
https://issues.apache.org/jira/browse/IGNITE-7496

--
BR,
Serge



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[jira] [Created] (IGNITE-7496) Review a SQL documentation

[Sergey Puchnin (JIRA)]

Sergey Puchnin created IGNITE-7496:
--

 Summary: Review a SQL documentation 
 Key: IGNITE-7496
 URL: https://issues.apache.org/jira/browse/IGNITE-7496
 Project: Ignite
  Issue Type: Bug
Reporter: Sergey Puchnin
Assignee: Denis Magda


In a discussion on dev list
http://apache-ignite-developers.2346864.n4.nabble.com/SparkDataFrame-Query-Optimization-Prototype-tt26249.html

was noticed there are some gaps in SQL documentation. 
It's necessary to review it.

I've added next system functions:
CASEWHEN Function, CAST, CONVERT, TABLE

And for my mind, next functions aren't applicable for Ignite:
ARRAY_GET, ARRAY_LENGTH, ARRAY_CONTAINS, CSVREAD, CSVWRITE, DATABASE, 
DATABASE_PATH, DISK_SPACE_USED, FILE_READ, FILE_WRITE, LINK_SCHEMA, 
MEMORY_FREE, MEMORY_USED, LOCK_MODE, LOCK_TIMEOUT, READONLY, CURRVAL, 
AUTOCOMMIT, CANCEL_SESSION, IDENTITY, NEXTVAL, ROWNUM, SCHEMA, SCOPE_IDENTITY, 
SESSION_ID, SET, TRANSACTION_ID, TRUNCATE_VALUE, USER, H2VERSION




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)