[GitHub] ignite pull request #872: ignite-3455
GitHub user dmagda opened a pull request: https://github.com/apache/ignite/pull/872 ignite-3455 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-3455 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/872.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #872 commit 0e753c38a986cbd46eb24845808ec1d0fc7d1dea Author: dkarachentsev Date: 2016-02-10T09:38:43Z IGNITE-2575: Added validation of IGFS endpoint port value. This closes #469. commit 99028b509c736b79faac7fb8104b1bc3cfe30720 Author: vozerov-gridgain Date: 2016-03-03T09:56:55Z IGFS: Minor refactoring. commit 219238f2793a3f3f9f5705a065c67510c286df1c Author: vozerov-gridgain Date: 2016-03-03T10:25:43Z IGNITE-2754: IGFS: Created separate processor for listing remove operation. commit ff5b68ca69050817794ef4b142c955a186e03de9 Author: vozerov-gridgain Date: 2016-03-14T07:19:23Z IGNITE-2781: IGFS: Force "copyOnRead=false" for meta and data caches. commit 37c4d508c9e40dcba274ae1625d7bf59bfeef144 Author: vozerov-gridgain Date: 2016-03-14T08:49:03Z IGNITE-2810: IGFS: Striped trash directory to reduce contention during removals. commit 54e6991cb1d0b68c4490dede603c9e3ba7cc3b9e Author: vozerov-gridgain Date: 2016-03-14T09:05:39Z IGNITE-2810: IGFS: Minor correction to IgfsUtils.isRootOrTrashId() method. commit 8083391be726c2bbc27e018983ca713e4b25e2a2 Author: vozerov-gridgain Date: 2016-03-14T10:17:58Z IGNITE-2814: IGFS: File lock/unlock/reserve operations are no longer require put/replace on cache. Thin entry processors are used instead. commit a7c1f44420ae96f183abc2e17125f0c9aa0775d5 Author: vozerov-gridgain Date: 2016-03-14T12:57:28Z IGNITE-2828: IGFS: Introduced processor for "updateTimes" operation. commit ba30ddbc599d67f398ffba1263d174f5b58b4b7d Author: thatcoach Date: 2016-03-15T17:46:13Z IGNITE-2838: IGFS: Opimized format of IgfsListingEntry. Now it contains only file ID and boolean flag endicating whether this a directory or file. commit 4f7e3c1c2e82596a26cec3b3587991ae18078b64 Author: vozerov-gridgain Date: 2016-03-16T06:14:49Z IGNITE-2817: IGFS: Optimized "updateProperties" and several other cache operations. Reafactored IgfsMetaManager a bit to simplify work with cache. commit 3e59321ef0ae1d936d94f8f804db45ceeff55844 Author: vozerov-gridgain Date: 2016-03-16T09:31:37Z IGNITE-2842: IGFS: Optimized create/mkdirs operations with help of entry processors. commit f57f3657b1da33abf28f885cd405780dabfd57e3 Author: vozerov-gridgain Date: 2016-03-16T10:22:07Z IGNITE-2846: IGFS: Reworked IgfsMetaManager.updateInfo() operation to use "invoke" instead of "put". commit 8a93c3bf1687e6f2de1a4391d95366d733a44a7d Author: vozerov-gridgain Date: 2016-03-18T13:38:45Z IGNITE-2860: IGFS: Reworked base meta operations. commit 8e9e790e482b8911142bf8b21fa3ad7267a62db6 Author: vozerov-gridgain Date: 2016-03-18T14:07:58Z IGNITE-2834: IGFS: Implemented optional metadata co-location. commit bfa7bf6c3d693406f1dd5121488796687aebbe7d Author: vozerov-gridgain Date: 2016-03-18T14:45:48Z IGNITE-2813: IGFS: Optimized metadata values splitting file and directory into separate classes. commit 8de40f2f8649c9ffecf86202f9fd4efbc3827e83 Author: thatcoach Date: 2016-03-18T18:15:04Z IGNITE-2860: IGFS: Fixed minor bug in append() operation. commit 9a4b5bd720c5ed1f96b82a457fa3eaed1bdbb132 Author: thatcoach Date: 2016-03-19T18:13:35Z IGNITE-2861: IGFS: Moved metadata processors into separate top-level classes to simplify code. Also cleaned up IgfsMetaManager from unused code. commit 2cd0dcb37ce43a4cb07885ddfb2e72392fc814a7 Author: vozerov-gridgain Date: 2016-03-21T07:29:20Z IGNITE-2836: IGFS: Ensured that metadata can be serialized through BinaryMarshaller in the most efficient way. commit 76191ff39456a30246df3aca7c026773d00a8446 Author: vozerov-gridgain Date: 2016-03-21T07:36:26Z IGNITE-2861: Added missing Apache headers. commit ee5ea53bf9c4ad897459466e0b9b5447fc93ec2a Author: vozerov-gridgain Date: 2016-03-22T06:20:32Z IGNITE-2869: IGFS: Slightly improved serialization of IgfsListingEntry. commit 218132dc0c3764966294a5f29ad480af4af7b0ff Author: vozerov-gridgain Date: 2016-03-22T06:23:29Z IGNITE-2868: IGFS: Increased trash concurrency from 16 to 64. commit e886ad0aa800cddb3308fa5f8400902e5879ee3c Author: vozerov-gridgain Date: 2016-03-22T07:28:13Z IGNITE-2811: IGFS: Optimized properties handling. commit 8d95ebacaa01f3f9271a1ce0d1b991dfead1d0c1 Author: vozerov-gridgain Date: 2016-03-22T09:06:51Z IGNITE-2871: IGFS: Removed "path" from IgfsEntryInfo. Purge event is never fired now, it will be fixed as a part of IGNITE-1679. commit
Re: Ignite monitoring
Igor, I think that built-in monitoring facility will add great value to the product. We have to deal with user performance issues pretty often, and it is always a kind of pain to get to the bottom of the problem. We have to ask users for configuration, logs, system config, etc, etc.. Instead, it would be great if we had a single big "switch". If user has performance issue, he turns it on, then perform problematic operations, and then dumps all collected data. We can collect dozens of things: 1) OS/JVM information 2) Ignite configs, logs, etc.. 3) Performance data (CPU, RAM, IO) 4) Metrics 5) JMX data (both Ignite and JVM) 6) Some internal tracing (SQL query plans, how long it takes messages to pass between nodes, etc.) I think the most important part here is good infrastructure (interfaces) and API. So that we can start with something very simple, like collecting configs from all nodes, or starting/stopping shell commands, and then gradually add more and more tracing facilities. Thoughts? Vladimir. On Thu, Jul 14, 2016 at 11:36 PM, Igor Rudyak wrote: > Yakov, as for now I just have well structured scripts to setup Ganglia > agent on Ignite hosts to monitor system metrics like CPU, RAM, IO and etc > (this scripts already included in Ignite 1.6). > > Also experimented with displaying JVM metrics by providing java agent and > specifying MBeans to collect metrics from. But it's rather draft version. > The second problem is, there are plenty of MBeans in Ignite - I just don't > know which to select from. > > Anyway, the original idea was to check with the community if it makes sense > to have such monitoring functionality out of the box. > > Igor Rudyak > > > > On Thu, Jul 14, 2016 at 1:05 AM, Yakov Zhdanov > wrote: > > > Igor, can you please share the changes to scripts you did to support > > monitoring? Can it be done by defining and exporting JAVA_OPTS env > variable > > and then launching ignite.sh? > > > > Thanks! > > > > --Yakov > > > > 2016-07-13 22:45 GMT+03:00 Igor Rudyak : > > > > > Hi guys, > > > > > > While experimenting with large Ignite clusters I found that lack of > > > monitoring is rather critical problem. I know that Ignite provides > number > > > of JMX MBeans to monitor custom metrics in addition to host system > > metrics > > > (CPU, IO, RAM, ). The problem is, there are no out of the box > > solution > > > to monitor all this. > > > > > > Thus you have to manually setup some kind of monitoring tool like > > Graphite, > > > Grafana, Ganglia and etc. Which involves setting up monitoring agents > on > > > all the nodes, uploading JMX agent on all the nodes, selecting > > appropriate > > > metrics from the plenty of JMX MBeans and preparing config files, > tuning > > > Ignite shell scripts to include "java agent" in java launch command. > Lots > > > of work and pain, each time you want to create new Ignite cluster. > > > > > > Probably it makes sense to have all these out of the box, by slightly > > > modifying existing and providing additional shell scripts, to bootstrap > > all > > > monitoring infrastructure? > > > > > > Igor Rudyak > > > > > >
[jira] [Created] (IGNITE-3490) Web console: Refactoring server side code to services, append tests
Andrey Novikov created IGNITE-3490: -- Summary: Web console: Refactoring server side code to services, append tests Key: IGNITE-3490 URL: https://issues.apache.org/jira/browse/IGNITE-3490 Project: Ignite Issue Type: Bug Reporter: Andrey Novikov Move code from routes to services. serve/routes/clusters.js -> serve/services/cluster.js serve/routes/igfs.js -> serve/services/igfs.js ... For example use serve/routes/cache.js -> serve/services/cache.js -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3491) .NET: Allow type name without assembly for type properties in app.config
Pavel Tupitsyn created IGNITE-3491: -- Summary: .NET: Allow type name without assembly for type properties in app.config Key: IGNITE-3491 URL: https://issues.apache.org/jira/browse/IGNITE-3491 Project: Ignite Issue Type: Improvement Components: platforms Affects Versions: 1.6 Reporter: Pavel Tupitsyn Fix For: 1.7 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3492) IGFS: Support file-based evictions
Vladimir Ozerov created IGNITE-3492: --- Summary: IGFS: Support file-based evictions Key: IGNITE-3492 URL: https://issues.apache.org/jira/browse/IGNITE-3492 Project: Ignite Issue Type: Bug Components: IGFS Affects Versions: 1.6 Reporter: Vladimir Ozerov Currently we support only per-block evictions. It means that block of any file can be removed. It might worth considering to drop the whole files instead of file blocks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3493) Web console: New item is not stored as selected.
Vasiliy Sisko created IGNITE-3493: - Summary: Web console: New item is not stored as selected. Key: IGNITE-3493 URL: https://issues.apache.org/jira/browse/IGNITE-3493 Project: Ignite Issue Type: Sub-task Reporter: Vasiliy Sisko # Open cluster page and create one if no clusters exist # Create new cluster with name that will not first in cluster list and save it. Visually new cluster is selected. # Refresh page Selected first cluster, but should be selected created. Check on other pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3494) CPP: Add tests for cross-platform put/get cache operations.
Igor Sapego created IGNITE-3494: --- Summary: CPP: Add tests for cross-platform put/get cache operations. Key: IGNITE-3494 URL: https://issues.apache.org/jira/browse/IGNITE-3494 Project: Ignite Issue Type: Sub-task Components: platforms Affects Versions: 1.6 Reporter: Igor Sapego Assignee: Igor Sapego Fix For: 1.7 We need to have tests that involve cache put from Java and cache get from C++ as well as cache put from C++ and cache get from Java. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3495) CacheMetrics.getAverage###Time is not calculated properly
Denis Magda created IGNITE-3495:
---
Summary: CacheMetrics.getAverage###Time is not calculated properly
Key: IGNITE-3495
URL: https://issues.apache.org/jira/browse/IGNITE-3495
Project: Ignite
Issue Type: Bug
Reporter: Denis Magda
Assignee: Andrey Velichko
{{CacheMetrics.getAverageGetTime}}, {{CacheMetrics.getAveragePutTime}} and
{{CacheMetrics.getAverageRemoveTime}} are not calculated properly in the
following scenario:
- start a server node;
- start a client node that will perform gets and puts;
- {{CacheMetrics.getAverage###Time}} will always be 0 for the server node's
cluster group.
The issue happens because {{CacheMetricsImpl.add###TimeNanos}} method is not
called on the server side when a metric related event happens.
In the attache you can find source that showcases the bug.
- start basic {{ExampleNodeStartup}} using attached configuration
{{example-default.xml}} conjuration;
- start {{ExampleNodeStartupClient}}. You will see that average metrics are not
incremented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3496) .NET: Identify 2.0 API changes
Pavel Tupitsyn created IGNITE-3496: -- Summary: .NET: Identify 2.0 API changes Key: IGNITE-3496 URL: https://issues.apache.org/jira/browse/IGNITE-3496 Project: Ignite Issue Type: Task Components: platforms Affects Versions: 1.7 Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 2.0 2.0 is a chance to introduce breaking changes in the API and fix whatever needs to be fixed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3497) .NET: Improve IgniteConfigurationSection.XSD
Pavel Tupitsyn created IGNITE-3497: -- Summary: .NET: Improve IgniteConfigurationSection.XSD Key: IGNITE-3497 URL: https://issues.apache.org/jira/browse/IGNITE-3497 Project: Ignite Issue Type: Improvement Components: platforms Affects Versions: 1.6 Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 1.7 * make sure that all properties are present (update unit tests) * include all enum values * provide documentation Can we autogenerate XSD from the configuration classes? This is a job for serializer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3498) Possible race leaving an untracked near entry
Alexey Goncharuk created IGNITE-3498: Summary: Possible race leaving an untracked near entry Key: IGNITE-3498 URL: https://issues.apache.org/jira/browse/IGNITE-3498 Project: Ignite Issue Type: Bug Components: cache Affects Versions: ignite-1.4 Reporter: Alexey Goncharuk In GridDhtGetFuture we wait for all ongoing transactions to finish before sending get response if we add reader. If a transaction removes this entry, we may end up with a near entry which does not have a corresponding DHT entry. See TODO in GridDhtGetFuture. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3500) Generate expiry event in GridCacheMapEntry#checkExpired()
Alexey Goncharuk created IGNITE-3500: Summary: Generate expiry event in GridCacheMapEntry#checkExpired() Key: IGNITE-3500 URL: https://issues.apache.org/jira/browse/IGNITE-3500 Project: Ignite Issue Type: Test Components: cache Affects Versions: ignite-1.4 Reporter: Alexey Goncharuk See TODO in GridCacheMapEntry#checkExpired(). Need to add corresponding tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3499) GridCacheReplicatedFullApiMultithreadedSelfTest1 is commented
Alexey Goncharuk created IGNITE-3499: Summary: GridCacheReplicatedFullApiMultithreadedSelfTest1 is commented Key: IGNITE-3499 URL: https://issues.apache.org/jira/browse/IGNITE-3499 Project: Ignite Issue Type: Test Components: cache Affects Versions: ignite-1.4 Reporter: Alexey Goncharuk The whole body of GridCacheReplicatedFullApiMultithreadedSelfTest1 is commented out. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] ignite pull request #873: Ignite 2968
GitHub user agura opened a pull request: https://github.com/apache/ignite/pull/873 Ignite 2968 You can merge this pull request into a Git repository by running: $ git pull https://github.com/agura/incubator-ignite ignite-2968 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/873.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #873 commit 9a4b5bd720c5ed1f96b82a457fa3eaed1bdbb132 Author: thatcoach Date: 2016-03-19T18:13:35Z IGNITE-2861: IGFS: Moved metadata processors into separate top-level classes to simplify code. Also cleaned up IgfsMetaManager from unused code. commit 2cd0dcb37ce43a4cb07885ddfb2e72392fc814a7 Author: vozerov-gridgain Date: 2016-03-21T07:29:20Z IGNITE-2836: IGFS: Ensured that metadata can be serialized through BinaryMarshaller in the most efficient way. commit 76191ff39456a30246df3aca7c026773d00a8446 Author: vozerov-gridgain Date: 2016-03-21T07:36:26Z IGNITE-2861: Added missing Apache headers. commit ee5ea53bf9c4ad897459466e0b9b5447fc93ec2a Author: vozerov-gridgain Date: 2016-03-22T06:20:32Z IGNITE-2869: IGFS: Slightly improved serialization of IgfsListingEntry. commit 218132dc0c3764966294a5f29ad480af4af7b0ff Author: vozerov-gridgain Date: 2016-03-22T06:23:29Z IGNITE-2868: IGFS: Increased trash concurrency from 16 to 64. commit e886ad0aa800cddb3308fa5f8400902e5879ee3c Author: vozerov-gridgain Date: 2016-03-22T07:28:13Z IGNITE-2811: IGFS: Optimized properties handling. commit 8d95ebacaa01f3f9271a1ce0d1b991dfead1d0c1 Author: vozerov-gridgain Date: 2016-03-22T09:06:51Z IGNITE-2871: IGFS: Removed "path" from IgfsEntryInfo. Purge event is never fired now, it will be fixed as a part of IGNITE-1679. commit b286facc4b8c44ab1628039dded6c7527760df73 Author: vozerov-gridgain Date: 2016-03-22T09:34:35Z IGNITE-2806: IGFS: Implemented relaxed consistency model. commit 26f115734e7262d4b4b60f1c6016783f67c66986 Author: vozerov-gridgain Date: 2016-03-22T09:46:23Z IGFS: Added misssing "final" modifiers to FileSystemConfiguration defaults. commit 00a0e4b51c299871ff690bbe6d462cf80dae045e Author: vozerov-gridgain Date: 2016-03-24T07:35:43Z IGNITE-2878: IGFS: Optimzied serialization of IgfsListingEntry and properties map. commit 01a6e86ec4e19d372b8efbc5c497c84f4238a46c Author: vozerov-gridgain Date: 2016-03-24T10:28:30Z IGNITE-2876: Fixed possible starvation in system pool caused by IgfsBlockMessage. This closes #575. commit 59705e008267b1d5926410f95c68bb8ffb8cd93c Author: vozerov-gridgain Date: 2016-03-24T11:29:35Z IGNITE-2883: IGFS: Now IPC messages are handled in a dedicated thread pool. commit 0412c9bfd89b8d2a377588e908f1012c845ac5bc Author: Alexey Kuznetsov Date: 2016-03-30T04:43:19Z Added detail info about keys count in partitions for offheap and swap. commit 28d2a7bf7f35ec4b51fba872ace47cdbc255ded8 Author: Alexey Kuznetsov Date: 2016-03-30T07:42:12Z Minor change to Visor classes: added setters for name and queryMetrics properties. commit a70d2dc48c42f23b1ea64c2a219560efa44fb99f Author: Alexey Kuznetsov Date: 2016-04-04T07:14:42Z Minor fix for Visor query metrics. commit ec2ec9965ae03ef3666b2035a13f444cf2765aec Author: dkarachentsev Date: 2016-03-30T10:29:36Z Now cache plugins are able to unwrap entries passed to EntryProcessor. commit e85a7170534cb66f40386cba689cfe632f4e66db Author: ashutak Date: 2016-04-05T11:56:01Z ignite-2835: Fixed BinaryObjectOffHeapImpl leakage to public code commit d33b4340a68553e59e4adecf78fea79af55bf2ae Author: sboikov Date: 2016-04-05T13:42:50Z ignite-2835 Minor test changes. commit 2397552daf6d8cff9b59515c1c8983abdc60f5f4 Author: vdpyatkov Date: 2016-04-05T15:03:55Z IGNITE-2822 Continuous query local listener can be notified with empty list of events commit 87f7522c56b5735442f8d52dbc63756de53f2464 Author: sboikov Date: 2016-04-06T06:46:47Z gridgain-7.5.11 Increased default affinity history size. commit 86347272e712a0e61a5a763dda3458fde79f29c4 Author: Anton Vinogradov Date: 2016-04-06T08:35:36Z IGNITE-2947 BinaryContext doesn't honor custom loader set through IgniteConfiguration.classLoader commit b2c934d7abe6dfe86e1d09ec592875df987f2120 Author: Alexey Kuznetsov Date: 2016-04-06T10:31:21Z IGNITE-2963 Fixed special case with Date for Oracle JDBC driver. commit e7ab8eef504cdcf8987941e8b7a552ada451aec8 Author: sboikov Date: 2016-04-06T14:38:04Z gridgain-7.5.11 Affinity history cleanup ignoring client discovery event. commit d92e617a71c31ccaa3fd6e4954aa4ccaf494733c Author: Valentin Kulichenko Date: 2016-04-07T01:10:45Z IGNITE-2951 - Stability fixes for cluster with many clients commit bbe1a7e52f177283969e81127216838f37b4205f Author: Anton Vinogradov Date: 2016
Re: Ignite monitoring
I think we can add this functionality to Ignite web console, no? On Mon, Jul 18, 2016 at 11:08 AM, Vladimir Ozerov wrote: > Igor, > > I think that built-in monitoring facility will add great value to the > product. We have to deal with user performance issues pretty often, and it > is always a kind of pain to get to the bottom of the problem. We have to > ask users for configuration, logs, system config, etc, etc.. Instead, it > would be great if we had a single big "switch". If user has performance > issue, he turns it on, then perform problematic operations, and then dumps > all collected data. We can collect dozens of things: > 1) OS/JVM information > 2) Ignite configs, logs, etc.. > 3) Performance data (CPU, RAM, IO) > 4) Metrics > 5) JMX data (both Ignite and JVM) > 6) Some internal tracing (SQL query plans, how long it takes messages to > pass between nodes, etc.) > > I think the most important part here is good infrastructure (interfaces) > and API. So that we can start with something very simple, like collecting > configs from all nodes, or starting/stopping shell commands, and then > gradually add more and more tracing facilities. > > Thoughts? > > Vladimir. > > > On Thu, Jul 14, 2016 at 11:36 PM, Igor Rudyak wrote: > > > Yakov, as for now I just have well structured scripts to setup Ganglia > > agent on Ignite hosts to monitor system metrics like CPU, RAM, IO and etc > > (this scripts already included in Ignite 1.6). > > > > Also experimented with displaying JVM metrics by providing java agent and > > specifying MBeans to collect metrics from. But it's rather draft version. > > The second problem is, there are plenty of MBeans in Ignite - I just > don't > > know which to select from. > > > > Anyway, the original idea was to check with the community if it makes > sense > > to have such monitoring functionality out of the box. > > > > Igor Rudyak > > > > > > > > On Thu, Jul 14, 2016 at 1:05 AM, Yakov Zhdanov > > wrote: > > > > > Igor, can you please share the changes to scripts you did to support > > > monitoring? Can it be done by defining and exporting JAVA_OPTS env > > variable > > > and then launching ignite.sh? > > > > > > Thanks! > > > > > > --Yakov > > > > > > 2016-07-13 22:45 GMT+03:00 Igor Rudyak : > > > > > > > Hi guys, > > > > > > > > While experimenting with large Ignite clusters I found that lack of > > > > monitoring is rather critical problem. I know that Ignite provides > > number > > > > of JMX MBeans to monitor custom metrics in addition to host system > > > metrics > > > > (CPU, IO, RAM, ). The problem is, there are no out of the box > > > solution > > > > to monitor all this. > > > > > > > > Thus you have to manually setup some kind of monitoring tool like > > > Graphite, > > > > Grafana, Ganglia and etc. Which involves setting up monitoring agents > > on > > > > all the nodes, uploading JMX agent on all the nodes, selecting > > > appropriate > > > > metrics from the plenty of JMX MBeans and preparing config files, > > tuning > > > > Ignite shell scripts to include "java agent" in java launch command. > > Lots > > > > of work and pain, each time you want to create new Ignite cluster. > > > > > > > > Probably it makes sense to have all these out of the box, by slightly > > > > modifying existing and providing additional shell scripts, to > bootstrap > > > all > > > > monitoring infrastructure? > > > > > > > > Igor Rudyak > > > > > > > > > >
Re: Ignite monitoring
I think we should have some general API and we could call it from web console and/or from other places. 18 Июл 2016 г. 20:18 пользователь "Dmitriy Setrakyan" написал: > I think we can add this functionality to Ignite web console, no? > > On Mon, Jul 18, 2016 at 11:08 AM, Vladimir Ozerov > wrote: > > > Igor, > > > > I think that built-in monitoring facility will add great value to the > > product. We have to deal with user performance issues pretty often, and > it > > is always a kind of pain to get to the bottom of the problem. We have to > > ask users for configuration, logs, system config, etc, etc.. Instead, it > > would be great if we had a single big "switch". If user has performance > > issue, he turns it on, then perform problematic operations, and then > dumps > > all collected data. We can collect dozens of things: > > 1) OS/JVM information > > 2) Ignite configs, logs, etc.. > > 3) Performance data (CPU, RAM, IO) > > 4) Metrics > > 5) JMX data (both Ignite and JVM) > > 6) Some internal tracing (SQL query plans, how long it takes messages to > > pass between nodes, etc.) > > > > I think the most important part here is good infrastructure (interfaces) > > and API. So that we can start with something very simple, like collecting > > configs from all nodes, or starting/stopping shell commands, and then > > gradually add more and more tracing facilities. > > > > Thoughts? > > > > Vladimir. > > > > > > On Thu, Jul 14, 2016 at 11:36 PM, Igor Rudyak wrote: > > > > > Yakov, as for now I just have well structured scripts to setup Ganglia > > > agent on Ignite hosts to monitor system metrics like CPU, RAM, IO and > etc > > > (this scripts already included in Ignite 1.6). > > > > > > Also experimented with displaying JVM metrics by providing java agent > and > > > specifying MBeans to collect metrics from. But it's rather draft > version. > > > The second problem is, there are plenty of MBeans in Ignite - I just > > don't > > > know which to select from. > > > > > > Anyway, the original idea was to check with the community if it makes > > sense > > > to have such monitoring functionality out of the box. > > > > > > Igor Rudyak > > > > > > > > > > > > On Thu, Jul 14, 2016 at 1:05 AM, Yakov Zhdanov > > > wrote: > > > > > > > Igor, can you please share the changes to scripts you did to support > > > > monitoring? Can it be done by defining and exporting JAVA_OPTS env > > > variable > > > > and then launching ignite.sh? > > > > > > > > Thanks! > > > > > > > > --Yakov > > > > > > > > 2016-07-13 22:45 GMT+03:00 Igor Rudyak : > > > > > > > > > Hi guys, > > > > > > > > > > While experimenting with large Ignite clusters I found that lack of > > > > > monitoring is rather critical problem. I know that Ignite provides > > > number > > > > > of JMX MBeans to monitor custom metrics in addition to host system > > > > metrics > > > > > (CPU, IO, RAM, ). The problem is, there are no out of the box > > > > solution > > > > > to monitor all this. > > > > > > > > > > Thus you have to manually setup some kind of monitoring tool like > > > > Graphite, > > > > > Grafana, Ganglia and etc. Which involves setting up monitoring > agents > > > on > > > > > all the nodes, uploading JMX agent on all the nodes, selecting > > > > appropriate > > > > > metrics from the plenty of JMX MBeans and preparing config files, > > > tuning > > > > > Ignite shell scripts to include "java agent" in java launch > command. > > > Lots > > > > > of work and pain, each time you want to create new Ignite cluster. > > > > > > > > > > Probably it makes sense to have all these out of the box, by > slightly > > > > > modifying existing and providing additional shell scripts, to > > bootstrap > > > > all > > > > > monitoring infrastructure? > > > > > > > > > > Igor Rudyak > > > > > > > > > > > > > > >
Re: Ignite monitoring
On Mon, Jul 18, 2016 at 8:45 PM, Alexey Kuznetsov wrote: > I think we should have some general API and we could call it from web > console and/or from other places. > Agree. The server side support should be sufficient to enable different monitoring connections, including command-line visor, web console, or JMX beans. > 18 Июл 2016 г. 20:18 пользователь "Dmitriy Setrakyan" < > dsetrak...@apache.org> > написал: > > > I think we can add this functionality to Ignite web console, no? > > > > On Mon, Jul 18, 2016 at 11:08 AM, Vladimir Ozerov > > wrote: > > > > > Igor, > > > > > > I think that built-in monitoring facility will add great value to the > > > product. We have to deal with user performance issues pretty often, and > > it > > > is always a kind of pain to get to the bottom of the problem. We have > to > > > ask users for configuration, logs, system config, etc, etc.. Instead, > it > > > would be great if we had a single big "switch". If user has performance > > > issue, he turns it on, then perform problematic operations, and then > > dumps > > > all collected data. We can collect dozens of things: > > > 1) OS/JVM information > > > 2) Ignite configs, logs, etc.. > > > 3) Performance data (CPU, RAM, IO) > > > 4) Metrics > > > 5) JMX data (both Ignite and JVM) > > > 6) Some internal tracing (SQL query plans, how long it takes messages > to > > > pass between nodes, etc.) > > > > > > I think the most important part here is good infrastructure > (interfaces) > > > and API. So that we can start with something very simple, like > collecting > > > configs from all nodes, or starting/stopping shell commands, and then > > > gradually add more and more tracing facilities. > > > > > > Thoughts? > > > > > > Vladimir. > > > > > > > > > On Thu, Jul 14, 2016 at 11:36 PM, Igor Rudyak > wrote: > > > > > > > Yakov, as for now I just have well structured scripts to setup > Ganglia > > > > agent on Ignite hosts to monitor system metrics like CPU, RAM, IO and > > etc > > > > (this scripts already included in Ignite 1.6). > > > > > > > > Also experimented with displaying JVM metrics by providing java agent > > and > > > > specifying MBeans to collect metrics from. But it's rather draft > > version. > > > > The second problem is, there are plenty of MBeans in Ignite - I just > > > don't > > > > know which to select from. > > > > > > > > Anyway, the original idea was to check with the community if it makes > > > sense > > > > to have such monitoring functionality out of the box. > > > > > > > > Igor Rudyak > > > > > > > > > > > > > > > > On Thu, Jul 14, 2016 at 1:05 AM, Yakov Zhdanov > > > > wrote: > > > > > > > > > Igor, can you please share the changes to scripts you did to > support > > > > > monitoring? Can it be done by defining and exporting JAVA_OPTS env > > > > variable > > > > > and then launching ignite.sh? > > > > > > > > > > Thanks! > > > > > > > > > > --Yakov > > > > > > > > > > 2016-07-13 22:45 GMT+03:00 Igor Rudyak : > > > > > > > > > > > Hi guys, > > > > > > > > > > > > While experimenting with large Ignite clusters I found that lack > of > > > > > > monitoring is rather critical problem. I know that Ignite > provides > > > > number > > > > > > of JMX MBeans to monitor custom metrics in addition to host > system > > > > > metrics > > > > > > (CPU, IO, RAM, ). The problem is, there are no out of the box > > > > > solution > > > > > > to monitor all this. > > > > > > > > > > > > Thus you have to manually setup some kind of monitoring tool like > > > > > Graphite, > > > > > > Grafana, Ganglia and etc. Which involves setting up monitoring > > agents > > > > on > > > > > > all the nodes, uploading JMX agent on all the nodes, selecting > > > > > appropriate > > > > > > metrics from the plenty of JMX MBeans and preparing config files, > > > > tuning > > > > > > Ignite shell scripts to include "java agent" in java launch > > command. > > > > Lots > > > > > > of work and pain, each time you want to create new Ignite > cluster. > > > > > > > > > > > > Probably it makes sense to have all these out of the box, by > > slightly > > > > > > modifying existing and providing additional shell scripts, to > > > bootstrap > > > > > all > > > > > > monitoring infrastructure? > > > > > > > > > > > > Igor Rudyak > > > > > > > > > > > > > > > > > > > > >
ReadWriteUpdateLock
Hi everyone, cross-posting from JIRA: I recently came across this post: http://codereview.stackexchange.com/a/31231 Do you think ReadWriteUpdateLock is something we can put to good use here in Ignite? This kind of lock should be more efficient for read-before-write patterns. Best regards, Vladisav
[jira] [Created] (IGNITE-3501) Step-by-step guidance on how to configure and use binary type across Java and .Net
Denis Magda created IGNITE-3501: --- Summary: Step-by-step guidance on how to configure and use binary type across Java and .Net Key: IGNITE-3501 URL: https://issues.apache.org/jira/browse/IGNITE-3501 Project: Ignite Issue Type: Improvement Components: platforms Affects Versions: 1.6 Reporter: Denis Magda Assignee: Denis Magda Priority: Critical Fix For: 1.7 It's quite a common task and endeavor to work with objects, stored in Ignite caches, in their deserialized form on Java and .Net sides. This requires special setup at the configuration level. Presently we don't have a page that list this particular steps with examples. The following can be used as a draft 1) Explicitly configure all your custom types on Java side using BinaryConfiguration.typeConfigurations property [1] or BinaryConfiguration.classNames property [2]. 2) Explicitly configure the same corresponding .Net types on .Net side using PlatformDotNetBinaryConfiguration.types property as it shown under XML tab on this page [3]. Remove BinaryConfiguration bean from .Net XML configuration if you use it before. Set additional required properties to PlatformDotNetBinaryConfiguration bean if BinaryConfiguration used any. 3) If you use custom keys then make sure that both "hashCode" and "equals" methods are implemented properly on both Java and .Net ends. 4) On .Net side you should implement IBinarizable interface [4] or go with Ignite Reflective Serialization [5] for your custom objects. 5) On Java side you can implement Binarizable interface as well or rely on reflective serialization making sure that your objects don't implement Externalizable interface and don't override readObject/writeObject methods [6]. Please let me know if everything works fine on your side after applying the listed steps. Regards, Denis [1] https://apacheignite.readme.io/docs/binary-marshaller#configuring-binary-objects [2] https://ignite.apache.org/releases/1.6.0/javadoc/org/apache/ignite/configuration/BinaryConfiguration.html#setClassNames(java.util.Collection) [3] https://apacheignite-net.readme.io/docs/serialization#ibinarizable [4] https://apacheignite-net.readme.io/docs/serialization#ibinarizable [5] https://apacheignite-net.readme.io/docs/serialization#ignite-reflective-serialization [6] https://apacheignite.readme.io/docs/binary-marshaller#basic-concepts On .Net side the configuration must be provided for App.config, C# config and Spring XML. On Java side the configuration must be provided for Java and Spring XML configs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
