[jira] [Created] (IGNITE-8081) Document Kubernetes RBAC configuration to avoid 403 exception

[Denis Magda (JIRA)]

Denis Magda created IGNITE-8081:
---

 Summary: Document Kubernetes RBAC configuration to avoid 403 
exception
 Key: IGNITE-8081
 URL: https://issues.apache.org/jira/browse/IGNITE-8081
 Project: Ignite
  Issue Type: New Feature
  Components: documentation
Reporter: Denis Magda
 Fix For: 2.5


It's reported by the users that sometimes Ignite Kubernetes IP finder fails to 
join the cluster due to security issues. To prevent the exception happening we 
need to document how to set up a Service Account for Ignite pods:
https://stackoverflow.com/questions/49395481/how-to-setmasterurl-in-ignite-xml-config-for-kubernetes-ipfinder/49405879#49405879



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Memory usage per cache

[Denis Magda]

Alexey,

If to rephrase it differently, if a cache/table belongs to a single cache
group then its entries will never be mixed with the entries of another
caches/tables in both data and index pages. Please confirm that my
statement is correct. Also, does that counter accumulates size of
BinaryObjects or the whole data/index pages?

Finally, do you plan to rework CacheMetrics.getOffHeapAllocatedSize method
for this purpose? (btw, to my knowledge the method is broken at the moment
and returns 0).

--
Denis


On Thu, Mar 29, 2018 at 1:20 AM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> Denis,
>
> Currently there is an easy way to add per-cache-group metrics for data and
> index pages. There is an internal counter, but it is not published as an
> MBean metric, we will do this as a part of IEP-6.
>
> As for the per-cache metrics, this can be implemented, but it will take a
> significantly greater effort and most likely will affect performance. As
> Andrey noted, when two caches share the same cache group, multiple entries
> may be written to the same page, this needs special handling. I suggest we
> first start with per-cache-group metrics and then, if there is high demand,
> start thinking about per-cache metrics.
>
> --AG
>
> 2018-03-27 15:18 GMT+03:00 Andrey Kuznetsov :
>
> > I apologize for the previous message sent in hurry. It's imposible to
> > measure the difference between 'precise' and 'estimated' page memory
> usage
> > per cache unless we fully implement approach 2. Current model allows to
> > store objects from several caches in a single page.
> >
> > 2018-03-23 22:21 GMT+03:00 Andrey Kuznetsov :
> >
> > > Denis,
> > >
> > > I'll need to conduct some experiments to estimate the difference. And
> the
> > > answer will depend on numerous parameters: object sizes, number of
> > caches,
> > > that share the same data region and so on.
> > >
> > > пт, 23 марта 2018, 21:53 Denis Magda :
> > >
> > >> Andrey,
> > >>
> > >> How good will be the estimate if we go for 1. and utilize
> > pagesFillFactor
> > >> somehow? In other words, how big can be a difference between 100%
> > precise
> > >> calculation you the approach you're suggesting?
> > >>
> > >> --
> > >> Denis
> > >>
> > >>
> >
> >
> > --
> > Best regards,
> >   Andrey Kuznetsov.
> >
>

[GitHub] ignite pull request #3720: IGNITE-8076 Java Thin Client Authentication

[kukushal]

GitHub user kukushal opened a pull request:

https://github.com/apache/ignite/pull/3720

IGNITE-8076 Java Thin Client Authentication



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8076

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3720.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3720


commit 4925ffc10ce8e8287980eaec38b587512568a302
Author: Alexey Goncharuk 
Date:   2018-01-17T12:26:03Z

IGNITE-7453 Use GridUnsafe.cleanDirectBuffer in PageSnapshot

commit bcd68a058683b2f17b7ac60471b6e7aab3e4f6de
Author: Pavel Tupitsyn 
Date:   2018-01-17T12:38:20Z

IGNITE-7301 .NET: Baseline topology

This closes #3352

commit 66b96316a7775ce8a6e2ff4475185d5929e4998b
Author: devozerov 
Date:   2018-01-17T12:54:17Z

Merge branch 'master' into ignite-2.4

commit 268481c1cf7fe57df24be130eb67c3e3a13afe01
Author: Alexey Goncharuk 
Date:   2018-01-17T13:50:34Z

IGNITE-7453 Use GridUnsafe.cleanDirectBuffer in WalStat

commit db0cd105719c8ae713b13b34d9dca0a8cd45d377
Author: Pavel Tupitsyn 
Date:   2018-01-17T14:05:25Z

IGNITE-6776 .NET: Thin client: Add SQL & LINQ example

This closes #3390

commit c214db879101aa5660e2a50b11cd20964c0bc114
Author: Andrey Gura 
Date:   2018-01-17T12:42:41Z

ignite-7450 FileWriteAheadLogManager always uses RandomAccessFileIOFactory 
now

commit 75c27d5e49d7458e46eb46e6f87a445c3f1320ea
Author: Alexey Kuznetsov 
Date:   2018-01-18T02:25:19Z

IGNITE-7274 Web Console: Support multiple statements on Queries screen.
(cherry picked from commit 1926783)

commit 36cc822935387b2150e690c29bc6992dca0563f7
Author: Dmitriy Shabalin 
Date:   2018-01-18T04:49:08Z

IGNITE-7306 Web Console: Fixed export data from tables.
(cherry picked from commit 1bb60ec)

commit d753298b4012894b56f5c9218325211cd84a21d5
Author: Peter Ivanov 
Date:   2018-01-18T06:18:53Z

IGNITE-7107 Apache Ignite RPM packages

* added changelog to package specification

This closes #3396

commit 63445893f1bc75dc9777184499f7eabc1d4e51b1
Author: Denis Mekhanikov 
Date:   2018-01-18T08:36:18Z

IGNITE-3935 Use PeerDeployAware for streamer transformer - Fixes #3378.

Signed-off-by: Alexey Goncharuk 

commit f3f9f2a24b23027cf0c835140322e41a788932ae
Author: Pavel Tupitsyn 
Date:   2018-01-18T09:05:12Z

IGNITE-7413 Fix SqlDmlExample

This closes #3389

commit 1daa7c41bf1460a4d9a2b0c26a7a317f2fca3fb7
Author: Alexey Kuznetsov 
Date:   2018-01-18T10:14:53Z

IGNITE-7461 UI tools: Actualized data storage configuration.
(cherry picked from commit 577e632)

commit cf0080210d24d9dd8b057f959446fac5f8a4ca01
Author: dpavlov 
Date:   2018-01-18T10:53:29Z

IGNITE-7380 Implemented pluggable Direct IO - Fixes #3226.

Signed-off-by: Alexey Goncharuk 

commit dd06d0bd7ef266bfbe156e858b312d1ac86e8982
Author: Pavel Tupitsyn 
Date:   2018-01-18T12:55:49Z

IGNITE-7465 .NET: Fix SqlDdlExample failure with standalone node

commit 57479ec564e1761716da3d5f9feb7a64c396a9f9
Author: Pavel Tupitsyn 
Date:   2018-01-18T13:45:54Z

.NET: Fix CacheLocalTest.TestTxDeadlockDetection

commit bd6be8a4653322905a3b63850c7e033ce3801ce5
Author: Pavel Tupitsyn 
Date:   2018-01-18T18:25:05Z

.NET: Thin client: Fix OP_BINARY_TYPE_GET schema passing format

commit 97564d160586d6d57d300937e6b8877994e35fc7
Author: rkondakov 
Date:   2018-01-19T08:24:51Z

IGNITE-6456: Ability to separately enable or disable JDBC, ODBC and thin 
client endpoints. This closes #3309.

commit d50274ca8875c9680c12e8786ac355a787ba95e0
Author: Yakov Zhdanov 
Date:   2018-01-18T17:57:17Z

Javadoc enhancements - added @see

commit cb2d3cf22388ab19fb2d34ae5bdfc8f1b608db75
Author: Dmitriy Govorukhin 
Date:   2018-01-18T14:14:26Z

IGNITE-7471 Use soft reference for checkpoint entry contents to avoid 
excessive memory usage

commit 3965923369870bb4e8e57e3332c1a1eb1e5f5ed3
Author: rkondakov 
Date:   2018-01-19T09:00:55Z

IGNITE-6772: SQL exception messages became more informative. This closes 
#3342.

commit ba68cb0fa87f776fcd0499d030c333f182611f41
Author: devozerov 
Date:   2018-01-19T09:03:52Z

Merge remote-tracking branch 'origin/ignite-2.4' into ignite-2.4

commit b54c0c8786bd74aa0abb208f537c29f0c4be4b1e
Author: tledkov-gridgain 
Date:   2018-01-19T09:09:34Z

IGNITE-7248: JDBC: fixed schema resolution for 

[GitHub] ignite pull request #3719: IGNITE-8048 merge query entities for dynamic cach...

[akalash]

GitHub user akalash opened a pull request:

https://github.com/apache/ignite/pull/3719

IGNITE-8048 merge query entities for dynamic cache during joining node



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8048

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3719.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3719


commit f80023baa15e67e0ab42fafafa759667176cb32c
Author: Anton Kalashnikov 
Date:   2018-03-29T14:56:51Z

IGNITE-8048 merge query entities for dynamic cache during joining node




---

[GitHub] ignite pull request #3717: IgniteConfigurationParityTest fix.

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3717


---

[jira] [Created] (IGNITE-8079) Service config variations test has flaky failures in Basic 2 suite: Not serializable

[Dmitriy Pavlov (JIRA)]

Dmitriy Pavlov created IGNITE-8079:
--

 Summary: Service config variations test has flaky failures in 
Basic 2 suite: Not serializable
 Key: IGNITE-8079
 URL: https://issues.apache.org/jira/browse/IGNITE-8079
 Project: Ignite
  Issue Type: Test
Reporter: Dmitriy Pavlov
 Fix For: 2.5


 
IgniteServiceConfigVariationsFullApiTest.testClusterSingletonDeploy 5 
failures in one build
 IgniteServiceConfigVariationsFullApiTest.testDeploy5 failures in one build
 IgniteServiceConfigVariationsFullApiTest.testKeyAffinityDeploy 5 
failures in one build
 IgniteServiceConfigVariationsFullApiTest.testMultipleDeploy5 failures in 
one build
 IgniteServiceConfigVariationsFullApiTest.testNodeSingletonDeploy 

{noformat}
[2018-03-25 
09:11:24,764][ERROR][test-runner-#15278%service.IgniteServiceConfigVariationsFullApiTest%][GridServiceProcessor]
 Failed to marshal service with configured marshaller [name=testService, 
srvc=o.a.i.i.processors.service.IgniteServiceConfigVariationsFullApiTest$TestServiceImpl@587a67fa,
 marsh=JdkMarshaller [clsFilter=null]]
class org.apache.ignite.IgniteCheckedException: Failed to serialize object: 
org.apache.ignite.internal.processors.service.IgniteServiceConfigVariationsFullApiTest$TestServiceImpl@587a67fa
at 
org.apache.ignite.marshaller.jdk.JdkMarshaller.marshal0(JdkMarshaller.java:103)
at 
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:70)
at 
org.apache.ignite.marshaller.jdk.JdkMarshaller.marshal0(JdkMarshaller.java:117)
at 
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
at 
org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10051)
at 
org.apache.ignite.internal.processors.service.GridServiceProcessor.prepareServiceConfigurations(GridServiceProcessor.java:560)
at 
org.apache.ignite.internal.processors.service.GridServiceProcessor.deployAll(GridServiceProcessor.java:612)
at 
org.apache.ignite.internal.processors.service.GridServiceProcessor.deployAll(GridServiceProcessor.java:600)
at 
org.apache.ignite.internal.processors.service.GridServiceProcessor.deployMultiple(GridServiceProcessor.java:488)
at 
org.apache.ignite.internal.processors.service.GridServiceProcessor.deployClusterSingleton(GridServiceProcessor.java:469)
at 
org.apache.ignite.internal.IgniteServicesImpl.deployClusterSingleton(IgniteServicesImpl.java:120)
at 
org.apache.ignite.internal.processors.service.IgniteServiceConfigVariationsFullApiTest$2.run(IgniteServiceConfigVariationsFullApiTest.java:99)
at 
org.apache.ignite.internal.processors.service.IgniteServiceConfigVariationsFullApiTest.testService(IgniteServiceConfigVariationsFullApiTest.java:225)
at 
org.apache.ignite.internal.processors.service.IgniteServiceConfigVariationsFullApiTest$ServiceTestRunnable.run(IgniteServiceConfigVariationsFullApiTest.java:189)
at 
org.apache.ignite.testframework.junits.IgniteConfigVariationsAbstractTest.runInAllDataModes(IgniteConfigVariationsAbstractTest.java:248)
at 
org.apache.ignite.internal.processors.service.IgniteServiceConfigVariationsFullApiTest.testClusterSingletonDeploy(IgniteServiceConfigVariationsFullApiTest.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at junit.framework.TestCase.runTest(TestCase.java:176)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2002)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:133)
at 
org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1917)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.NotSerializableException: 
org.apache.ignite.testframework.junits.IgniteConfigVariationsAbstractTest$PlaneObject
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
at 
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at 
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at 
org.apache.ignite.marshaller.jdk.JdkMarshaller.marshal0(JdkMarshaller.java:98)
... 24 more
{noformat}

Links to test history:
  Basic [2] 

 

[GitHub] ignite pull request #3718: Ignite 2.4.2 p7

[EdShangGG]

GitHub user EdShangGG opened a pull request:

https://github.com/apache/ignite/pull/3718

Ignite 2.4.2 p7



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.4.2-p7

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3718.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3718


commit e7ca9b65a68de7752195c8f4d2b5180f3c77d19f
Author: Dmitriy Govorukhin 
Date:   2017-11-13T18:52:47Z

ignite-blt-merge -> ignite-2.4.1

commit cc8168fc184bb7f5e3cc3bbb0743397097f78bfb
Author: Dmitriy Govorukhin 
Date:   2017-11-13T19:13:01Z

merge ignite-pitr-rc1 -> ignite-2.4.1

commit 87e6d74cf6a251c7984f9e68c391f790feccc281
Author: Dmitriy Govorukhin 
Date:   2017-11-14T12:49:33Z

ignite-gg-12877 Compact consistent ID in WAL

commit 9f5a22711baea05bd37ab07c8f928a4837dd83a4
Author: Ilya Lantukh 
Date:   2017-11-14T14:12:28Z

Fixed javadoc.

commit d5af2d78dd8eef8eca8ac5391d31d8c779649bb0
Author: Alexey Kuznetsov 
Date:   2017-11-15T08:09:00Z

IGNITE-6913 Baseline: Added new options to controls.sh for baseline 
manipulations.

commit 713924ce865752b6e99b03bd624136541cea5f9f
Author: Sergey Chugunov 
Date:   2017-11-15T09:03:12Z

IGNITE-5850 failover tests for cache operations during BaselineTopology 
changes

commit b65fd134e748d496f732ec2aa0953a0531f544b8
Author: Ilya Lantukh 
Date:   2017-11-15T12:54:35Z

TX read logging if PITR is enabled.

commit 9b2a567c0e04dc33116b51f88bee75f76e9107d1
Author: Ilya Lantukh 
Date:   2017-11-15T13:45:16Z

TX read logging if PITR is enabled.

commit 993058ccf0b2b8d9e80750c3e45a9ffa31d85dfa
Author: Dmitriy Govorukhin 
Date:   2017-11-15T13:51:54Z

ignite-2.4.1 optimization for store full set node more compacted

commit 1eba521f608d39967aec376b397b7fc800234e54
Author: Dmitriy Govorukhin 
Date:   2017-11-15T13:52:22Z

Merge remote-tracking branch 'professional/ignite-2.4.1' into ignite-2.4.1

commit 564b3fd51f8a7d1d81cb6874df66d0270623049c
Author: Sergey Chugunov 
Date:   2017-11-15T14:00:51Z

IGNITE-5850 fixed issue with initialization of data regions on node 
activation, fixed issue with auto-activation when random node joins inactive 
cluster with existing BLT

commit c6d1fa4da7adfadc80abdc7eaf6452b86a4f6aa4
Author: Sergey Chugunov 
Date:   2017-11-15T16:23:08Z

IGNITE-5850 transitionResult is set earlier when request for changing 
BaselineTopology is sent

commit d65674363163e38a4c5fdd73d1c8d8e1c7610797
Author: Sergey Chugunov 
Date:   2017-11-16T11:59:07Z

IGNITE-5850 new failover tests for changing BaselineTopology up (new node 
added to topology)

commit 20552f3851fe8825191b144179be032965e0b5c6
Author: Sergey Chugunov 
Date:   2017-11-16T12:53:43Z

IGNITE-5850 improved error message when online node is removed from baseline

commit 108bbcae4505ac904a6db774643ad600bfb42c21
Author: Sergey Chugunov 
Date:   2017-11-16T13:45:52Z

IGNITE-5850 BaselineTopology should not change on cluster deactivation

commit deb641ad3bdbf260fa60ad6bf607629652e324bd
Author: Dmitriy Govorukhin 
Date:   2017-11-17T09:45:44Z

ignite-2.4.1 truncate wal and checkpoint history on move/delete snapshot

commit 3c8b06f3659af30d1fd148ccc0f40e216a56c998
Author: Alexey Goncharuk 
Date:   2017-11-17T12:48:12Z

IGNITE-6947 Abandon remap after single map if future is done (fixes NPE)

commit ba2047e5ae7d271a677e0c418375d82d78c4023e
Author: devozerov 
Date:   2017-11-14T12:26:31Z

IGNITE-6901: Fixed assertion during 
IgniteH2Indexing.rebuildIndexesFromHash. This closes #3027.

commit abfc0466d6d61d87255d0fe38cbdf11ad46d4f89
Author: Sergey Chugunov 
Date:   2017-11-17T13:40:57Z

IGNITE-5850 tests for queries in presence of BaselineTopology

commit f4eabaf2a905abacc4c60c01d3ca04f6ca9ec188
Author: Sergey Chugunov 
Date:   2017-11-17T17:23:02Z

IGNITE-5850 implementation for setBaselineTopology(long topVer) migrated 
from wc-251

commit 4edeccd3e0b671aa277f58995df9ff9935baa95a
Author: EdShangGG 
Date:   2017-11-17T18:21:17Z

GG-13074 Multiple snapshot test failures after baseline topology is 
introduced
-adding baseline test to suite
-fixing issues with baseline

commit edae228c8f55990c15ef3044be987dcb00d6c81a
Author: EdShangGG 
Date:   2017-11-18T10:36:41Z

hack with sleep

commit b5bffc7580a4a8ffbcc06f60c282e73979179578
Author: Ilya Lantukh 

[GitHub] ignite pull request #3717: IgniteConfigurationParityTest fix.

[alexpaschenko]

GitHub user alexpaschenko opened a pull request:

https://github.com/apache/ignite/pull/3717

IgniteConfigurationParityTest fix.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite dotnet-parity-test-fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3717.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3717


commit 0d7d5670534d905b12d70eca02d2ccb9eb5312e4
Author: Alexander Paschenko 
Date:   2018-03-29T13:42:12Z

IgniteConfigurationParityTestFix.




---

[jira] [Created] (IGNITE-8078) Add new metrics for data storage

[Dmitriy Govorukhin (JIRA)]

Dmitriy Govorukhin created IGNITE-8078:
--

 Summary: Add new metrics for data storage
 Key: IGNITE-8078
 URL: https://issues.apache.org/jira/browse/IGNITE-8078
 Project: Ignite
  Issue Type: New Feature
Reporter: Dmitriy Govorukhin


1. Create new MXbean for each index, IndexMxBean
{code}
class IndexMxBean{
/** The number of PUT operations on the index. */
long processedPuts();
/** The number of GET operations on the index. */
long processedGets();
/** The total index size in bytes. */
long getIndexSize();
}
{code}

2. Add new metrics for data storage and cache group.
{code}
class CacheGroupMetricsMXBean{
/** The total index size in bytes */
long getIndexTotalSize();
long getPKIndexTotalSize();
long getTotalsize();
long getDataSize();
String getType();
}
{code}

{code}
class DataRegionMXBean{
long getIndexTotalSize();
long getPKIndexTotalSize();
long getTotalsize();
long getdataSize();
long offheapUsedSize();
long pagesRead();
long pagesWriten();
long pagesReplaced();
long dirtyPagesForNextCheckpoint();
}
{code}

{code}
class DataStorageMXbean{
long getIndexTotalSize();
long getPKIndexTotalSize();
long getTotalsize();
long offHeapSize();
long offheapUsedSize();
long getDataSize();
long pagesRead();
long pagesWriten();
long pagesReplaced();
long checkpointTotalTime();
long dirtyPagesForNextCheckpoint();
}
{code}




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8077) Implement new JMX metrics for transactions

[Dmitriy Govorukhin (JIRA)]

Dmitriy Govorukhin created IGNITE-8077:
--

 Summary: Implement new JMX metrics for transactions
 Key: IGNITE-8077
 URL: https://issues.apache.org/jira/browse/IGNITE-8077
 Project: Ignite
  Issue Type: New Feature
Reporter: Dmitriy Govorukhin


These additional metrics should be implemented to monitor transactions.

{code}
class TransactionsMXbean{
/** The number of transactions which was commited. */
long txCommited();
/** The number of transactions which was rollbacked. */
long txRollBacked();
/** The number of transactions in prepared state. */
long txPrepared();
/** The number of keys locked on the node. */
long lockedKeys();
/** The number of transactions for which node is the initiator. */
int ownerTxs();
}
{code}

For more details see in IgniteTxAdapter and IgniteTxManager.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3687: IGNITE-8025 runMultiThreadedAsync cancellation fi...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3687


---

[GitHub] ignite pull request #3649: IGNITE-7962 Avoid swallowing unexpected exception...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3649


---

[jira] [Created] (IGNITE-8076) Java Thin Client Authentication

[Alexey Kukushkin (JIRA)]

Alexey Kukushkin created IGNITE-8076:


 Summary: Java Thin Client Authentication
 Key: IGNITE-8076
 URL: https://issues.apache.org/jira/browse/IGNITE-8076
 Project: Ignite
  Issue Type: Task
Reporter: Alexey Kukushkin
Assignee: Alexey Kukushkin


Java thin client was merged into branch 2.4.4. Java thin client supports 
authentication by sending user credentials to server. Implement handling 
authentication on the server side.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3673: IGNITE-7990 Integrate MLP with partition based da...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3673


---

[GitHub] ignite pull request #3716: IGNITE-8068: + GridAbstractTest.getFailureHandler...

[x-kreator]

GitHub user x-kreator opened a pull request:

https://github.com/apache/ignite/pull/3716

IGNITE-8068: + GridAbstractTest.getFailureHandler methods.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/x-kreator/ignite ignite-8068

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3716.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3716


commit ac9fe9245cd081f1d13156c1af69564650a49658
Author: Dmitriy Sorokin 
Date:   2018-03-29T10:31:06Z

IGNITE-8068: + GridAbstractTest.getFailureHandler methods.




---

[jira] [Created] (IGNITE-8075) Add support for two new public methods in .NET API

[Alexei Scherbakov (JIRA)]

Alexei Scherbakov created IGNITE-8075:
-

 Summary: Add support for two new public methods in .NET API
 Key: IGNITE-8075
 URL: https://issues.apache.org/jira/browse/IGNITE-8075
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.4
Reporter: Alexei Scherbakov
Assignee: Pavel Tupitsyn
 Fix For: 2.5


Neet to add two described method as part of .NET API.

withLabel
localActiveTransactions

Java implementation is currently available in branch [1]

[1] https://github.com/gridgain/apache-ignite/tree/ignite-6827-2



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8074) Allow changing of tx rollback timeout on exchange in runtime.

[Alexei Scherbakov (JIRA)]

Alexei Scherbakov created IGNITE-8074:
-

 Summary: Allow changing of tx rollback timeout on exchange in 
runtime.
 Key: IGNITE-8074
 URL: https://issues.apache.org/jira/browse/IGNITE-8074
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.4
Reporter: Alexei Scherbakov
 Fix For: 2.5


It's desirable to have the possibility changing in runtime tx rollback timeout, 
introduced in IGNITE-6827.

Simplest implementation: use JMX method call.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Apache Ignite RPM packaging (Stage II)

[Max Shonichev]

1. About packaging/package.sh

1.1. I personally dislike the use of bashism, e.g. like reserved keywords,
double '[[', double '((', but that's ok
for targeted distro (RHEL/Centos). The only requirement is please add more
comments in tough places
to improve code readability and maintanbility.


1.2. few code style comments:

>> name=$(cat /etc/*release | grep ^NAME | sed -r 's|.*"(.*)".*|\1|')

add

   | head -n 1

or else it will fail in case more than one file with *release mask in /etc


1.3. rework prepEnv

>> if [ ! -z installCmd ]; then

this will fail miserably, replace with

   if [ ! -z $installCmd ]; then

1.3.1. however running package.sh under Ubuntu still would fail due to
$installCmd is unpacked to 'apt --no-install-recommends'
and -z 'apt --no-install-recommends' would always return false.

1.3.2. and still, when apt/yum is installed and rpm/rpm-build is not, the
whole prepEnv would fail to ensure all
build-time requirements are installed, because it only checks for apt/yum.

1.3.3. installing requires root privileges. However, no checks for $EUID nor
running apt/yum with 'sudo' present.
Please, add this check, otherwise running package.sh under unprivileged user
with no installed build-time dependencies
will fail with very undescriptive errors.


1.4. getBin

1.4.1. I don't like detecting target apache ignite package version by
parsing embedded changelog. This might fail
when any package text description would contain '*' in the first char of the
string. Either move changelog to separate
Changelog file and %include it in .spec or parse for, e.g. '^Version:
[0-9\.]+'

>> igniteVersion=$(cat rpm/SPECS/apache-ignite.spec | grep '^*' | head -1 |
>> sed -r 's|.*\ -\ (.*)-.*|\1|')

1.4.2.

>> ls ${binName} && binPreparedFlag=true || true

why not use

  if [ ! -f $binName ];

1.4.3. 'curl' is used, but not checked as build-time dependency.

1.4.4. we try to find and download 'apache-ignite-fabric-$igniteVersion',
that's ok for <2.4 releases, but in future
we would need to find and download 'apache-ignite-$igniteVersion' too.
Better add this check as a fallback option, e.g.
try to use curl with both urls in correct order.

  if curl --output /dev/null --silent --head --fail "$url"; then
echo "URL exists: $url"
  else
echo "URL does not exist: $url"
  fi

1.4.5. also strange, but may be ok that we download
apache-ignite-fabric-2.4.0


2. about apache-ignite.spec

2.1. fix typo

>> Apache Ignite's optinal libs and integrations

2.2. fix description

>> C++ files necessary for using Apache Ignite

platform files are not necessary for Ignite, only for integration with .NET,
afaik?

2.3. Tough place with 'Obsoletes:apache-ignite < 2.5.0'

I understand the current logic behind splitting apache-ignite to several
.rpm's and setting obsolete for the old
apache-ignite.

However, for future releases, e.g. if/when we handover this task to another
engineer, it is not quite obvious that one
must (or must not?) update version in 5 different places, including
changelog. Please add a few lines about
'How to package next (new) version of Apache Ignite RPMs' to Confluence.

2.4. firewalld / firewall-cmd is optionally used if installed, but not set
in the dependency. It is OK, but may be we
add this as a post-install note, like we do recommendation messages in node
logs.


3. service scripts : we'll should add a test suite for it, overall looks
pretty solid.







--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[jira] [Created] (IGNITE-8073) Cache read metric is calculated incorrectly in atomic cache.

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-8073:


 Summary: Cache read metric is calculated incorrectly in atomic 
cache.
 Key: IGNITE-8073
 URL: https://issues.apache.org/jira/browse/IGNITE-8073
 Project: Ignite
  Issue Type: Bug
  Components: cache
Reporter: Alexey Kuznetsov


In atomic cache with near enabled we perform put and remove operations.
After it, get operation is called.
Now, cache 'read' metric is calculated incorrectly, because it takes into 
account near cache entry.

Reproducer is attached.

Note that remove operation untracks 'reader' node from dht cache entry, but 
near cache entry still exists. The following test checks it :
GridCacheAtomicNearCacheSelfTest#checkNearCache, see checkReaderRemove().



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Code inspection

[Alexey Goncharuk]

>From what I see, it should be rather easy to get a meaningful number of
inspection failures to get something we can start working with.

Namely, we have:
Overly strong type cast (206) - mechanical work, easy to fix
Assignment replaceable with operator assignment (23) - either mechanical
work, or disable inspection
'expression.equals("literal")' rather than '"literal".equals(expression)'
(49) - mechanical work
'size() == 0' replaceable with 'isEmpty()' (67) - mechanical work
Missorted modifiers (121) - mechanical work
Redundant field initialization (76) - mechanical work or disable inspection
Unnecessary 'this' qualifier (543) - mechanical work or disable inspection
'if' statement could be replaced with conditional expression (244) -
mechanical work or disable inspection
Redundant throws declaration (100) - mechanical work or disable inspection
Redundant suppression (848) - mechanical work
Missing @Override annotation (289) - mechanical work
Property key/value delimiter doesn't match code style settings (2183) -
disable inspection
Unused Property (2180) - disable inspection

For some of the inspections we have to agree whether we enforce a
particular code style (for example, unnecessary 'this' qualifier).
After this is done, the number of failed inspections will drop dramatically
and we can start tracking changes and pay more attention to other
inspection categories.

--AG

2018-03-28 21:19 GMT+03:00 Peter Ivanov :

> Anton, Dmitry is right.
>
> We have to manually add condition when to consider build faulty based on
> how many failed inspection are there.
>
> For now I see this initiative as follows:
> - find more or less correct set of inspections (there are lots of typos and
> other irrelevant to code execution inspections) looking on the results of
> core module build, as it has ~85% of target code;
> - add all modules to composite project and setup schedule at least once a
> week.
>
>
> On Wed, 28 Mar 2018 at 19:09, Dmitry Pavlov  wrote:
>
> > Inspection suites should be failed manually by some fail condition.
> >
> > This question will become actual in future. How to fail such suite on TC?
> >
> > ср, 28 мар. 2018 г. в 18:54, Anton Vinogradov :
> >
> > > Peter,
> > >
> > > Why 44 errors are green?
> > >
> > >
> > >
> > https://ci.ignite.apache.org/viewLog.html?buildId=1145974;
> tab=buildResultsDiv=IgniteTests24Java8_InspectionsAop
> > >
> > > 2018-03-28 16:27 GMT+03:00 Petr Ivanov :
> > >
> > > > After several problems, example run on Aleksey’s configuration is
> > > > complete: https://ci.ignite.apache.org/viewLog.html?buildId=1164652
> <
> > > > https://ci.ignite.apache.org/viewLog.html?buildId=1164652>
> > > >
> > > >
> > > > > On 28 Mar 2018, at 10:28, Petr Ivanov  wrote:
> > > > >
> > > > > Started https://ci.ignite.apache.org/viewLog.html?buildId=1164002
> <
> > > > https://ci.ignite.apache.org/viewQueued.html?itemId=1163998> with
> > > > Aleksey’s inspections profile.
> > > > > Core (long) and AOP (short) modules will be tested as example.
> > > > >
> > > > >
> > > > >
> > > > >> On 27 Mar 2018, at 19:38, Dmitry Pavlov  > >  > > > dpavlov@gmail.com>> wrote:
> > > > >>
> > > > >> Hi Petr,
> > > > >>
> > > > >> Could you please take inspections and run it on AI code base in
> > > > >> https://ci.ignite.apache.org/viewType.html?buildTypeId=
> > > >
> > IgniteTests24Java8_InspectionsCore_IgniteTests24Java8=%3Cdefault%
> > > > 3E=buildTypeStatusDiv  > > > viewType.html?buildTypeId=IgniteTests24Java8_InspectionsCore_
> > > > IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv>
> > > > >> ?
> > > > >>
> > > > >> Sincerely,
> > > > >> Dmitriy Pavlov
> > > > >>
> > > > >> вт, 27 мар. 2018 г. в 19:27, Dmitry Pavlov  >:
> > > > >>
> > > > >>> Alexey, thank you for bring this topic to top.
> > > > >>>
> > > > >>> What do you think about committing this inspections into Ignite
> > code
> > > > base?
> > > > >>>
> > > > >>> What can be our next steps after demonstrating CI check is
> possible
> > > > >>> https://ci.ignite.apache.org/viewType.html?buildTypeId=
> > > >
> > IgniteTests24Java8_InspectionsCore_IgniteTests24Java8=%3Cdefault%
> > > > 3E=buildTypeStatusDiv
> > > > >>> ?
> > > > >>>
> > > > >>> Sincerely,
> > > > >>> Dmitriy Pavlov
> > > > >>>
> > > > >>> вт, 27 мар. 2018 г. в 15:28, Alexey Goncharuk <
> > > > alexey.goncha...@gmail.com
> > > >  :
> > > > >>>
> > > >  Bumping up.
> > > > 
> > > >  Attached is my local inspections profile exported from Idea.
> Let's
> > > run
> > > >  the first iteration and check if it differs significantly from
> > other
> > > >  community members.
> > > > 
> > > >  --AG
> > > > 
> > > >  2018-03-19 16:39 GMT+03:00 Petr Ivanov :
> > > > 
> > > > > Filed https://issues.apache.org/jira/browse/IGNITE-7985 <
> > > > 

[GitHub] ignite pull request #3715: IGNITE-7771 Names of Ignite JMX beans should not ...

[aealeksandrov]

GitHub user aealeksandrov opened a pull request:

https://github.com/apache/ignite/pull/3715

IGNITE-7771 Names of Ignite JMX beans should not be quoted unless required

Names of Ignite JMX beans and bean groups are currently quoted if they 
contain non-alphanumeric characters. This leads to names with spaces, e.g. 
Thread Pools, appearing as "Thread Pools". Moreover, Thread Pools and "Thread 
Pools" are recognized by JMX as distinct names, so code accessing that MBean 
needs to take that into account.

It would be better not to quote the bean and group names unless they 
contain specific control characters. That way users won't need to quote names 
in their JMX clients.

Logic of generation of the ObjectName instances was updates. Also new tests 
were added.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7771

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3715.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3715


commit 262e4d56c855ec1e6a44b32a6e46d82a176dc4dc
Author: Andrei Aleksandrov 
Date:   2018-03-29T08:35:05Z

IGNITE-7771 Names of Ignite JMX beans should not be quoted unless required




---

Re: Memory usage per cache

[Dmitriy Setrakyan]

On Thu, Mar 29, 2018 at 1:20 AM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> Denis,
>
> Currently there is an easy way to add per-cache-group metrics for data and
> index pages. There is an internal counter, but it is not published as an
> MBean metric, we will do this as a part of IEP-6.
>
> As for the per-cache metrics, this can be implemented, but it will take a
> significantly greater effort and most likely will affect performance. As
> Andrey noted, when two caches share the same cache group, multiple entries
> may be written to the same page, this needs special handling. I suggest we
> first start with per-cache-group metrics and then, if there is high demand,
> start thinking about per-cache metrics.
>

I agree with AG. If user really needs  per-cache metrics, then user can use
one cache per group configuration, which is the default, btw.

D.

Re: Memory usage per cache

[Alexey Goncharuk]

Denis,

Currently there is an easy way to add per-cache-group metrics for data and
index pages. There is an internal counter, but it is not published as an
MBean metric, we will do this as a part of IEP-6.

As for the per-cache metrics, this can be implemented, but it will take a
significantly greater effort and most likely will affect performance. As
Andrey noted, when two caches share the same cache group, multiple entries
may be written to the same page, this needs special handling. I suggest we
first start with per-cache-group metrics and then, if there is high demand,
start thinking about per-cache metrics.

--AG

2018-03-27 15:18 GMT+03:00 Andrey Kuznetsov :

> I apologize for the previous message sent in hurry. It's imposible to
> measure the difference between 'precise' and 'estimated' page memory usage
> per cache unless we fully implement approach 2. Current model allows to
> store objects from several caches in a single page.
>
> 2018-03-23 22:21 GMT+03:00 Andrey Kuznetsov :
>
> > Denis,
> >
> > I'll need to conduct some experiments to estimate the difference. And the
> > answer will depend on numerous parameters: object sizes, number of
> caches,
> > that share the same data region and so on.
> >
> > пт, 23 марта 2018, 21:53 Denis Magda :
> >
> >> Andrey,
> >>
> >> How good will be the estimate if we go for 1. and utilize
> pagesFillFactor
> >> somehow? In other words, how big can be a difference between 100%
> precise
> >> calculation you the approach you're suggesting?
> >>
> >> --
> >> Denis
> >>
> >>
>
>
> --
> Best regards,
>   Andrey Kuznetsov.
>

Re: Apache Ignite nightly nuget upload

[Pavel Tupitsyn]

Hi Petr,

Sure, please go ahead.
That was a temporary solution which has become permanent :)

Thank you,
Pavel

On Wed, Mar 28, 2018 at 3:02 PM, Petr Ivanov  wrote:

> Hi, Pavel!
>
>
> While working on moving TC tests to new scheme, I stumbled across this
> build chain ([1] -> [2]).
> I wonder, after preparing nightly build project [3] - can I move task of
> nuget uploading to one of build step of that chain?
>
> IMO, nightly build in tests is a little bit out of place.
>
>
> [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=
> IgniteTests24Java8_IgnitePlatformNetNuGet  viewType.html?buildTypeId=IgniteTests24Java8_IgnitePlatformNetNuGet>
> [2] https://ci.ignite.apache.org/viewType.html?buildTypeId=
> IgniteTests24Java8_IgnitePlatformNetNuGetNightly <
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_
> IgnitePlatformNetNuGetNightly>
> [3] https://ci.ignite.apache.org/project.html?projectId=
> Releases_NightlyRelease_Releases_NightlyRelease=__all_branches__ <
> https://ci.ignite.apache.org/project.html?projectId=
> Releases_NightlyRelease_Releases_NightlyRelease=__all_branches__>