[jira] [Created] (IGNITE-6872) Linear regression should implement Model API
Oleg Ignatenko created IGNITE-6872: -- Summary: Linear regression should implement Model API Key: IGNITE-6872 URL: https://issues.apache.org/jira/browse/IGNITE-6872 Project: Ignite Issue Type: Task Security Level: Public (Viewable by anyone) Components: ml Reporter: Oleg Ignatenko Assignee: Oleg Ignatenko Fix For: 2.4 When linear regression was originally implemented per IGNITE-5012 we had no Model API. Now that this API is available (merged into master with IGNITE-5218) lin regression needs to adapt to implement it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] ignite pull request #3018: Ignite gg 12915
GitHub user mcherkasov opened a pull request: https://github.com/apache/ignite/pull/3018 Ignite gg 12915 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-gg-12915 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3018.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3018 commit d268b32cb252a5f06887d2b803d27ddc20ded95f Author: Igor SapegoDate: 2017-06-16T09:27:35Z IGNITE-4370: Implemented writing of batch of parameters for ODBC. (cherry picked from commit c10be5780589cc84e7929e234e4411d515166e0b) commit 7fbaecc67f1b204162bda4595d6c118ddd45f963 Author: Andrey V. Mashenkov Date: 2017-06-16T17:01:49Z IGNITE-5527: Prevent starvation in stripe pool on unstable topology. commit f81964f59b0ea5b8dfdc8eb2acc34d2a5b8fee07 Author: sboikov Date: 2017-01-10T13:59:17Z Do not evict removed entries, otherwise removes can be lost. (cherry picked from commit 55ac6e7) commit 38f0ea80f3d95be16b38b621b3bcc2910c463997 Author: sboikov Date: 2017-01-10T13:59:17Z Do not evict removed entries, otherwise removes can be lost. (cherry picked from commit 55ac6e7) commit 5dd74ff635de50ff9561ccdb51bdeb620f60c3db Author: sboikov Date: 2017-01-10T13:59:17Z Do not evict removed entries, otherwise removes can be lost. (cherry picked from commit 55ac6e7) commit 799ef99b512fffb90b97d926532bc6b9404efbff Author: Evgenii Zhuravlev Date: 2017-06-21T08:56:53Z Merge remote-tracking branch 'remotes/origin/ignite-1.9.3' into ignite-1.9.4 commit c802b478ef47271f5b8864e0b0ae29925107e75f Author: agura Date: 2017-06-21T15:52:17Z Compilation is fixed commit 5fb5c7e3b54ae4efb7a6a1832ba647677d93e0cd Author: Evgenii Zhuravlev Date: 2017-06-22T06:43:03Z IGNITE-5399 Manual cache rebalancing feature is broken commit 01d41b72ecc3e81dfc8966cc0e395c247037241c Author: Evgenii Zhuravlev Date: 2017-06-21T10:48:15Z GG-12256 H2Indexes are not deleted if key class implements Externalizable commit 5ac9afc719138e37a7d97d9d9db05243eee9a942 Author: Evgenii Zhuravlev Date: 2017-06-22T09:36:14Z IGNITE-5399 add test to testsuite commit a935d40a80e2f928a84a145aba540a45b156687f Author: Evgenii Zhuravlev Date: 2017-06-22T12:10:32Z GG-12256 Minor fixes commit 7e2468770a4eb47a4f61204d8c2000b6ab67c967 Author: nikolay_tikhonov Date: 2017-06-22T13:13:01Z IGNITE-GG-12197 Fixed "Ignore events for discarded update in CLOCK mode". Signed-off-by: nikolay_tikhonov commit 5858efd406bb54352de14a0a7e7f21c2ac7bf899 Author: sboikov Date: 2016-12-16T16:23:29Z IGNITE-2412 - Do not acquire asyncSemaphore for synchronous operations (cherry-picked from master) (cherry picked from commit 82b4073) commit 113a1380da34ea804d68757d39926da97dee09b6 Author: Alexey Goncharuk Date: 2017-06-13T05:20:22Z GG-12355: Backported IO latency test. commit 540ca449f1bd2386d3ba0722afb21dd3a504d044 Author: Alexey Goncharuk Date: 2017-06-13T17:55:38Z GG-12355: Added discovery ring latency test + made it available from MBean (cherry-picked from master). commit 0fc6271d8e39125bf5ee341e50a2665a04fc8b1e Author: Andrey V. Mashenkov Date: 2017-06-21T10:42:12Z GG-12350: GridDhtAtomicSingleUpdateRequest misses topologyVersion() method override. commit f8224d13cf9a6432ba65e0016370ba51bbb544e9 Author: Alexey Goncharuk Date: 2017-06-15T19:49:45Z GG-12299: Make sure concurrent type registrations do not trigger multiple cache updates. commit 4ffc3acfa1bc43bea8c79bfd1864787c15cfc4de Author: Alexey Goncharuk Date: 2017-06-20T04:59:09Z IGNITE-5528 - IS_EVICT_DISABLED flag is not cleared on cache store exception. commit 8cd9e829380f4c91cc9bb126169863286d1cb323 Author: Andrey V. Mashenkov Date: 2017-06-21T12:40:14Z GG-12353: Added local binary context flag. Backport of IGNITE-5223 with fixes. commit 9036ad239d68eff663bc73a81baab2826b054d9a Author: Andrey V. Mashenkov Date: 2017-06-21T15:25:31Z Added MBean for system cache executors. commit ed34a5dc681ea8f284f4d25c5575ad46569cc600 Author: Andrey V. Mashenkov Date: 2017-06-21T15:33:55Z Partial fix of IGNITE-5562. commit
[GitHub] ignite pull request #3017: Ignite-5218: Decision trees fixes.
GitHub user artemmalykh opened a pull request: https://github.com/apache/ignite/pull/3017 Ignite-5218: Decision trees fixes. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-5218 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3017.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3017 commit dbd2b9e14a5a932e2cb1290e8cf3bc6011ae1962 Author: Artem MalykhDate: 2017-10-26T16:01:22Z IGNITE-5218: First version of decision trees. commit 329ccd983aaf9fa05e4ae78347f6acdb07c70a21 Author: Artem Malykh Date: 2017-11-02T13:22:58Z IGNITE-5218: Merge apache master into IGNITE-5218. commit aa12b9ba326271029693dac365806983aac7f0e7 Author: Artem Malykh Date: 2017-11-02T13:26:24Z IGNITE-5218: Deleted isomorphism class. commit 0fb19280b86a215b9a6cac4af6e578b16791f116 Author: Artem Malykh Date: 2017-11-02T13:41:17Z IGNITE-5218: Added license. commit 49e1db802361a3eafa52e461aec53d3e52b63aed Author: Artem Malykh Date: 2017-11-02T13:56:39Z IGNITE-5218: Removed unused class 'EnsembleModel'. commit e4a4382ea179510f2470364255fa97046b5c4f2f Author: Artem Malykh Date: 2017-11-02T14:54:55Z IGNITE-5218: Some editings according to code review comments. commit 71090e3f01b6fe7a3ac9095913fee2862e551c2f Author: Artem Malykh Date: 2017-11-02T15:50:26Z IGNITE-5218: Some editings according to code review comments [1]. commit 8720bc3a31ca4d89593dab28d8af8519fc3994d6 Author: Artem Malykh Date: 2017-11-02T18:43:12Z IGNITE-5218: Some editings according to code review comments [2]. commit 586099d356c2dfc1605ba82b8983cea14c1be59d Author: Artem Malykh Date: 2017-11-03T12:13:55Z IGNITE-5218: Moved path to MNIST into a separate properties file. commit 00730f472d3dece4c0e42799044e1dfa9fe1be7a Author: Artem Malykh Date: 2017-11-03T12:17:55Z IGNITE-5218: Moved columntrees manualrun files to other dir. commit f7c9b99b63d4163fa452d7e3269ca46abdd3ea09 Author: Artem Malykh Date: 2017-11-03T12:18:55Z IGNITE-5218: Changed MNIST paths for arbitrary-looking. commit 70dff5e8539943654dd8c1f32d72b7d6d267956c Author: Artem Malykh Date: 2017-11-03T12:21:53Z IGNITE-5218: Added mnist manualrun properties into .gitignore. commit 29c994e87a20ddb48c3f3f63dc6330dc3a1986a1 Author: Artem Malykh Date: 2017-11-03T13:01:08Z IGNITE-5218: Added some javadoc, fixed example. commit 56e79acef6d291466d97364d897f914c76be3c0f Author: Artem Malykh Date: 2017-11-03T13:44:37Z IGNITE-5218: Elaborated MNIST example description. commit 0da5eb615823702badbd98d56f93708f1674907f Author: Artem Malykh Date: 2017-11-03T13:50:00Z IGNITE-5218: Elaborated MNIST example description[1]. commit a96b8e0776b89d90f519cd4879217e058b031972 Author: Artem Malykh Date: 2017-11-03T13:52:24Z IGNITE-5218: Defaulted config in MNIST example. commit c26fd5a07ac1564cfa75471186ae4edf72d6a20d Author: Artem Malykh Date: 2017-11-03T16:24:55Z IGNITE-5218: Documented MNIST example more carefully. commit 20ca360108ef5b42d6cf3b06aa4dc7068baa75f2 Author: Artem Malykh Date: 2017-11-03T16:26:13Z IGNITE-5218: Reverted default examples config. commit 97fe50f7e94b54040c3f4a74ef0d6f728c8886c5 Author: Artem Malykh Date: 2017-11-03T16:39:50Z IGNITE-5218: Reformated code according to guidelines. commit 182b8dd908e8eb573ccf3f3a40bcc09951c8 Author: Artem Malykh Date: 2017-11-03T16:43:36Z IGNITE-5218: Added package-info into structures package. commit b1bbef960e77fdd1c6f61b75d005296b5608d999 Author: Artem Malykh Date: 2017-11-03T16:58:58Z IGNITE-5218: Changed souts to X. commit f6ae8d5c1d0372ace3eac240e7ddd7823e74323a Author: Artem Malykh Date: 2017-11-03T17:28:53Z IGNITE-5218: Added missing javadocs. commit 234b3d5d90ca8a0cee7dcd9900398dde2ec5a355 Author: Artem Malykh Date: 2017-11-03T17:37:16Z IGNITE-5218: Changed properties. commit bb8d9394d445acac134a4dcba6ee409f2db51399 Author: Artem Malykh Date: 2017-11-03T18:14:42Z IGNITE-5218: Some minor fixes. commit 343d57713e354034553a3ea95d73192c819f5b02 Author: Artem Malykh Date: 2017-11-03T18:25:55Z IGNITE-5218: Some minor fixes[1 ] commit
Re: Data compression in Ignite
Dmitry, Ignite is used by a variety of applications. Some models I saw were made completely of stirngs. Others - of longs and decimals, etc.. It is impossible to either prove or disprove what is the dominant data type. My position is based on experience with Ignite users and approaches used in other databases. Strings are more complex because you approach assumes that there is a common dictionary with strings, and reference to these strings from data pages. As soon as you have cross-page references, you are in trobule, because you need to maintain that dictionary. WIth page based approach we agreed previously, the dictionary is generic (i.e. it can compress not only strings, but any byte sequence), and is located inside the page, meaning that all you need to maintain this dictionary is page lock. On Fri, Nov 10, 2017 at 7:02 PM, Dmitry Pavlovwrote: > Hi Vladimir, > > To my experience string is often used data type in business applications > and moreover, indexed. > > String type doesn't dominate in user models > what is the basis of this assumption? > > Could you explain why String is more complex than byte[] compression. It > seems they both requires dictionaries. > > Sincerely, > Dmitriy Pavlov > > пт, 10 нояб. 2017 г. в 18:57, Vladimir Ozerov : > > > This would require shared dictionary, which is complex to maintain. We > > evaluated this option, but rejected due to complexity. Another important > > thing is that String type doesn't dominate in user models, so I do not > see > > why it should be a kind of special case. > > > > пт, 10 нояб. 2017 г. в 18:45, Dmitry Pavlov : > > > > > Vladimir, > > > > > > orientation on string will also allow us to deduplicate strings in > > objects > > > during unmarshalling from page into heap. > > > > > > Moreover, this can be first simple step of implementating more complex > > > algorithm. > > > > > > Sincerely, > > > Dmitriy Pavlov > > > > > > пт, 10 нояб. 2017 г. в 18:19, Vladimir Ozerov : > > > > > > > Dmitry, > > > > > > > > What we've discussed so far in this topic is essentially the same > > > concept. > > > > We will deduplicate same byte sequences on page level. > > > > > > > > On Fri, Nov 10, 2017 at 6:10 PM, Dmitry Pavlov < > dpavlov@gmail.com> > > > > wrote: > > > > > > > > > Hi Igniters, > > > > > > > > > > What do you think about implementing analogue of Java G1 collector > > > featue > > > > > 'String deduplication': -XX:+UseG1GC -XX:+UseStringDeduplication > > > > > > > > > > Most of business application has almost all objects of type String. > > As > > > > > result char[] array is often on top of heap usage. To reduce > > > consumption > > > > by > > > > > duplicates G1 collector in background identifies and deduplicates > > > strings > > > > > having equal array into one instance (as String is immutable). > > > > > Unfortunately we can’t reuse collector’s feature as Ignite stores > > data > > > > > off-heap. > > > > > > > > > > What if we consider implementing same deduplication feature for > > Ignite > > > > > Durable Memory? > > > > > > > > > > Sincerely, > > > > > Dmitry Pavlov > > > > > > > > > > > > > > > ср, 18 окт. 2017 г. в 18:52, daradurvs : > > > > > > > > > > > Hi, Igniters! > > > > > > > > > > > > Are there any results of researching or a prototype of > compression > > > > > feature? > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble. > com/ > > > > > > > > > > > > > > > > > > > > >
Re: Add new JMX metrics.
Hello, Igniters. I've prepared PR [1] for this task [2], upsource review [3] and tests results [4]. Could someone review it? [1] https://github.com/apache/ignite/pull/3007 [2] https://issues.apache.org/jira/browse/IGNITE-6844 [3] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-394 [4] https://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=projectOverview_Ignite20Tests=pull/3007/head 2017-11-08 15:16 GMT+03:00 Pavel Pereslegin: > Yakov, thanks for your suggestion. > > I will create such MBean to include these metrics. > > 2017-11-07 20:11 GMT+03:00 Yakov Zhdanov : >> This seems to be incorrect place for implementing it. I would think of >> something more generic like IgniteClusterMBean. It should also work for any >> implementation of discovery SPI if one ever appears =) >> >> --Yakov
Re: Data compression in Ignite
Hi Vladimir, To my experience string is often used data type in business applications and moreover, indexed. > String type doesn't dominate in user models what is the basis of this assumption? Could you explain why String is more complex than byte[] compression. It seems they both requires dictionaries. Sincerely, Dmitriy Pavlov пт, 10 нояб. 2017 г. в 18:57, Vladimir Ozerov: > This would require shared dictionary, which is complex to maintain. We > evaluated this option, but rejected due to complexity. Another important > thing is that String type doesn't dominate in user models, so I do not see > why it should be a kind of special case. > > пт, 10 нояб. 2017 г. в 18:45, Dmitry Pavlov : > > > Vladimir, > > > > orientation on string will also allow us to deduplicate strings in > objects > > during unmarshalling from page into heap. > > > > Moreover, this can be first simple step of implementating more complex > > algorithm. > > > > Sincerely, > > Dmitriy Pavlov > > > > пт, 10 нояб. 2017 г. в 18:19, Vladimir Ozerov : > > > > > Dmitry, > > > > > > What we've discussed so far in this topic is essentially the same > > concept. > > > We will deduplicate same byte sequences on page level. > > > > > > On Fri, Nov 10, 2017 at 6:10 PM, Dmitry Pavlov > > > wrote: > > > > > > > Hi Igniters, > > > > > > > > What do you think about implementing analogue of Java G1 collector > > featue > > > > 'String deduplication': -XX:+UseG1GC -XX:+UseStringDeduplication > > > > > > > > Most of business application has almost all objects of type String. > As > > > > result char[] array is often on top of heap usage. To reduce > > consumption > > > by > > > > duplicates G1 collector in background identifies and deduplicates > > strings > > > > having equal array into one instance (as String is immutable). > > > > Unfortunately we can’t reuse collector’s feature as Ignite stores > data > > > > off-heap. > > > > > > > > What if we consider implementing same deduplication feature for > Ignite > > > > Durable Memory? > > > > > > > > Sincerely, > > > > Dmitry Pavlov > > > > > > > > > > > > ср, 18 окт. 2017 г. в 18:52, daradurvs : > > > > > > > > > Hi, Igniters! > > > > > > > > > > Are there any results of researching or a prototype of compression > > > > feature? > > > > > > > > > > > > > > > > > > > > -- > > > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > > > > > > > > > > >
Re: Data compression in Ignite
This would require shared dictionary, which is complex to maintain. We evaluated this option, but rejected due to complexity. Another important thing is that String type doesn't dominate in user models, so I do not see why it should be a kind of special case. пт, 10 нояб. 2017 г. в 18:45, Dmitry Pavlov: > Vladimir, > > orientation on string will also allow us to deduplicate strings in objects > during unmarshalling from page into heap. > > Moreover, this can be first simple step of implementating more complex > algorithm. > > Sincerely, > Dmitriy Pavlov > > пт, 10 нояб. 2017 г. в 18:19, Vladimir Ozerov : > > > Dmitry, > > > > What we've discussed so far in this topic is essentially the same > concept. > > We will deduplicate same byte sequences on page level. > > > > On Fri, Nov 10, 2017 at 6:10 PM, Dmitry Pavlov > > wrote: > > > > > Hi Igniters, > > > > > > What do you think about implementing analogue of Java G1 collector > featue > > > 'String deduplication': -XX:+UseG1GC -XX:+UseStringDeduplication > > > > > > Most of business application has almost all objects of type String. As > > > result char[] array is often on top of heap usage. To reduce > consumption > > by > > > duplicates G1 collector in background identifies and deduplicates > strings > > > having equal array into one instance (as String is immutable). > > > Unfortunately we can’t reuse collector’s feature as Ignite stores data > > > off-heap. > > > > > > What if we consider implementing same deduplication feature for Ignite > > > Durable Memory? > > > > > > Sincerely, > > > Dmitry Pavlov > > > > > > > > > ср, 18 окт. 2017 г. в 18:52, daradurvs : > > > > > > > Hi, Igniters! > > > > > > > > Are there any results of researching or a prototype of compression > > > feature? > > > > > > > > > > > > > > > > -- > > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > > > > > >
[jira] [Created] (IGNITE-6871) Implement new JMX metrics for partitions map monitoring
Aleksey Plekhanov created IGNITE-6871: - Summary: Implement new JMX metrics for partitions map monitoring Key: IGNITE-6871 URL: https://issues.apache.org/jira/browse/IGNITE-6871 Project: Ignite Issue Type: New Feature Security Level: Public (Viewable by anyone) Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov These additional metrics should be implemented to monitor partitions map per each cache group: * Total primary partitions count located on the current node * Total backup partitions count located on the current node * Min/max partition backups left in the cluster for cache group * Maybe some methods to show partitions map/partition distribution statistics in the cluster -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6870) Implement new JMX metric for cache topology validation monitoring
Aleksey Plekhanov created IGNITE-6870: - Summary: Implement new JMX metric for cache topology validation monitoring Key: IGNITE-6870 URL: https://issues.apache.org/jira/browse/IGNITE-6870 Project: Ignite Issue Type: New Feature Security Level: Public (Viewable by anyone) Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov There is no way now to determine from outside the grid that cache is in read-only state after TopologyValidator's "validate" method returns "false" for topology. Implement new JMX metric in CacheMetricsMXBean to show cache's topology validation status. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6868) Implement new JMX metrics for TcpCommunicationSpi monitoring
Aleksey Plekhanov created IGNITE-6868: - Summary: Implement new JMX metrics for TcpCommunicationSpi monitoring Key: IGNITE-6868 URL: https://issues.apache.org/jira/browse/IGNITE-6868 Project: Ignite Issue Type: New Feature Security Level: Public (Viewable by anyone) Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov These additional metrics should be implemented in TcpCommunicationSpi MBean: * Received messages count grouped by message type * Received messages count grouped by sender node * Sent messages count grouped by message type * Sent messages count grouped by receiver node -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6869) Implement new JMX metric for jobs monitoring
Aleksey Plekhanov created IGNITE-6869: - Summary: Implement new JMX metric for jobs monitoring Key: IGNITE-6869 URL: https://issues.apache.org/jira/browse/IGNITE-6869 Project: Ignite Issue Type: New Feature Security Level: Public (Viewable by anyone) Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov There are now some metrics in ClusterLocalNodeMetricsMXBean for monitoring job's execution statistics (min/max/avg execution time). But these metrics gathered since node started and can't be used to calculate average execution time between probes. To resolve this problem new metric "Total jobs execution time" should be implemented. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Data compression in Ignite
Vladimir, orientation on string will also allow us to deduplicate strings in objects during unmarshalling from page into heap. Moreover, this can be first simple step of implementating more complex algorithm. Sincerely, Dmitriy Pavlov пт, 10 нояб. 2017 г. в 18:19, Vladimir Ozerov: > Dmitry, > > What we've discussed so far in this topic is essentially the same concept. > We will deduplicate same byte sequences on page level. > > On Fri, Nov 10, 2017 at 6:10 PM, Dmitry Pavlov > wrote: > > > Hi Igniters, > > > > What do you think about implementing analogue of Java G1 collector featue > > 'String deduplication': -XX:+UseG1GC -XX:+UseStringDeduplication > > > > Most of business application has almost all objects of type String. As > > result char[] array is often on top of heap usage. To reduce consumption > by > > duplicates G1 collector in background identifies and deduplicates strings > > having equal array into one instance (as String is immutable). > > Unfortunately we can’t reuse collector’s feature as Ignite stores data > > off-heap. > > > > What if we consider implementing same deduplication feature for Ignite > > Durable Memory? > > > > Sincerely, > > Dmitry Pavlov > > > > > > ср, 18 окт. 2017 г. в 18:52, daradurvs : > > > > > Hi, Igniters! > > > > > > Are there any results of researching or a prototype of compression > > feature? > > > > > > > > > > > > -- > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > >
[jira] [Created] (IGNITE-6867) Implement new JMX metrics for topology monitoring
Aleksey Plekhanov created IGNITE-6867: - Summary: Implement new JMX metrics for topology monitoring Key: IGNITE-6867 URL: https://issues.apache.org/jira/browse/IGNITE-6867 Project: Ignite Issue Type: New Feature Security Level: Public (Viewable by anyone) Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov These additional metrics and methods should be implemented: * Current topology version * Total server nodes count * Total client nodes count * Method to count nodes filtered by some node attribute * Method to count nodes grouped by some node attribute There is already a ticket to implement first 2 metrics from this list (IGNITE-6844) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] ignite pull request #3016: Ignite 602
GitHub user agura opened a pull request: https://github.com/apache/ignite/pull/3016 Ignite 602 You can merge this pull request into a Git repository by running: $ git pull https://github.com/agura/incubator-ignite ignite-602 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3016.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3016 commit 21879d4cada68df2fe594eb6ccbfb09c98c238f2 Author: Dmitrii RyabovDate: 2017-06-24T11:24:32Z GridToStringBuilder now remember printed objects. commit b7102b467bd2ae4c108aeb988fdff711d06a7e66 Author: Dmitrii Ryabov Date: 2017-06-24T11:39:22Z IdentityHashMap
Re: Data compression in Ignite
Dmitry, What we've discussed so far in this topic is essentially the same concept. We will deduplicate same byte sequences on page level. On Fri, Nov 10, 2017 at 6:10 PM, Dmitry Pavlovwrote: > Hi Igniters, > > What do you think about implementing analogue of Java G1 collector featue > 'String deduplication': -XX:+UseG1GC -XX:+UseStringDeduplication > > Most of business application has almost all objects of type String. As > result char[] array is often on top of heap usage. To reduce consumption by > duplicates G1 collector in background identifies and deduplicates strings > having equal array into one instance (as String is immutable). > Unfortunately we can’t reuse collector’s feature as Ignite stores data > off-heap. > > What if we consider implementing same deduplication feature for Ignite > Durable Memory? > > Sincerely, > Dmitry Pavlov > > > ср, 18 окт. 2017 г. в 18:52, daradurvs : > > > Hi, Igniters! > > > > Are there any results of researching or a prototype of compression > feature? > > > > > > > > -- > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > >
Re: Data compression in Ignite
Hi Igniters, What do you think about implementing analogue of Java G1 collector featue 'String deduplication': -XX:+UseG1GC -XX:+UseStringDeduplication Most of business application has almost all objects of type String. As result char[] array is often on top of heap usage. To reduce consumption by duplicates G1 collector in background identifies and deduplicates strings having equal array into one instance (as String is immutable). Unfortunately we can’t reuse collector’s feature as Ignite stores data off-heap. What if we consider implementing same deduplication feature for Ignite Durable Memory? Sincerely, Dmitry Pavlov ср, 18 окт. 2017 г. в 18:52, daradurvs: > Hi, Igniters! > > Are there any results of researching or a prototype of compression feature? > > > > -- > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ >
[GitHub] ignite pull request #2936: IGNITE-5218: First version of decision trees.
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/2936 ---
[jira] [Created] (IGNITE-6866) Allocate offheap on client
Alexander Belyak created IGNITE-6866: Summary: Allocate offheap on client Key: IGNITE-6866 URL: https://issues.apache.org/jira/browse/IGNITE-6866 Project: Ignite Issue Type: Bug Security Level: Public (Viewable by anyone) Components: general Affects Versions: 2.1 Reporter: Alexander Belyak Often client use the same config file as a server and ignite start offheap memory for client too... but never use it. How it happens: 1) Default memory configuration for server is creating in IgnitionEx.initializeConfiguration() method: if (!myCfg.isClientMode() && myCfg.getMemoryConfiguration() == null) so if ignite configuration already contains memoryConfiguration - it stay there 2) In IgniteCacheDatabaseSharedManager.anActivate method do nothing only: if (cctx.kernalContext().clientNode() && cctx.kernalContext().config().getMemoryConfiguration() == null) return; So if ignite configuration contains memory configuration - it will be allocated. Why its not good: 1) Memory allocation spend virtual memory (OS didn't really allocate memory before first access to it) and if overcommit_memory strategy is set to OVERCOMMIT_NEVER - it can block start client node (maybe first or second one) in same host (see: /proc/sys/vm/overcommit_memory and /proc/sys/vm/overcommit_ratio) 2) In IgniteKernal.checkPhysicalRam() we use maxSize of offheap memory and log warning about memory overusage Good news only one - often in memory configuration really big only maxSize, but initialSize is just about 256Mb so each client really allocate not so many RAM. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] ignite pull request #3015: IGNITE-6836: Implemented query timeout.
GitHub user isapego opened a pull request: https://github.com/apache/ignite/pull/3015 IGNITE-6836: Implemented query timeout. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-6836 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3015.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3015 ---
[jira] [Created] (IGNITE-6865) Wrong map query build using
Alin Andrei Corodescu created IGNITE-6865: - Summary: Wrong map query build using Key: IGNITE-6865 URL: https://issues.apache.org/jira/browse/IGNITE-6865 Project: Ignite Issue Type: Bug Security Level: Public (Viewable by anyone) Components: sql Affects Versions: 2.3 Reporter: Alin Andrei Corodescu -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6864) Apply refactorings to decision trees code.
Artem Malykh created IGNITE-6864: Summary: Apply refactorings to decision trees code. Key: IGNITE-6864 URL: https://issues.apache.org/jira/browse/IGNITE-6864 Project: Ignite Issue Type: Task Security Level: Public (Viewable by anyone) Reporter: Artem Malykh * Think about changes signatures in MNISTUtils, maybe we should use collections or arrays instead of streams. * Add tests for CacheUtils new methods used in ColumnDecisionTreeTrainer * Think about OpenBitSet insead of SparseBitSet and BitSet * Consider making decision tree internal structure classes -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6863) visorcmd: 'cache -a' shows nodes that are not matched by nodeFilter
Denis Mekhanikov created IGNITE-6863: Summary: visorcmd: 'cache -a' shows nodes that are not matched by nodeFilter Key: IGNITE-6863 URL: https://issues.apache.org/jira/browse/IGNITE-6863 Project: Ignite Issue Type: Bug Security Level: Public (Viewable by anyone) Components: visor Affects Versions: 2.3 Reporter: Denis Mekhanikov 'cache -a' command shows all nodes in the detailed summary, even those, which are not matched by a configured node filter. Please find attached archive with the reproducing example project. LOCAL cache also has this issue. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] ignite pull request #3014: IGNITE-6406: SQL: CREATE INDEX should fill index ...
GitHub user dolphin1414 opened a pull request: https://github.com/apache/ignite/pull/3014 IGNITE-6406: SQL: CREATE INDEX should fill index from multiple threads. A parallel index creation and an SQL keyword PARALLEL parsing implemented. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-6406 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3014.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3014 commit 97069d5af3b54a008bd32db01064bfdd90c3f5bd Author: rkondakovDate: 2017-11-10T12:13:29Z IGNITE-6406: Parallel index creation implementated (cherry picked from commit d34f16f0ecd35ea55efb0b2e5f94734c56c9cca3) ---
[GitHub] ignite pull request #3013: IGNITE-6850 SQL: integrate index inline size to C...
GitHub user gg-shq opened a pull request: https://github.com/apache/ignite/pull/3013 IGNITE-6850 SQL: integrate index inline size to CREATE INDEX syntax You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite IGNITE-6850 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3013.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3013 commit be1d297b145322a3d9a90d49e1817a27fc56be24 Author: gg-shqDate: 2017-11-10T11:16:25Z IGNITE-6850: Added INLINE_SIZE option to internal SQL CREATE INDEX statement commit 8f4b677a69036aad486249c507166e5189eca979 Author: gg-shq Date: 2017-11-10T11:21:46Z Merge branch 'master' into IGNITE-6850 commit b0e91486d98d4ae56498f11098a4b57f83e9a69a Author: gg-shq Date: 2017-11-10T11:23:01Z IGNITE-8650: Resolved merge conflicts in SqlCreateIndexCommand.java ---
[jira] [Created] (IGNITE-6862) SparseDistributedMatrixStorage cache config possibly allows read of old state of matrix
Artem Malykh created IGNITE-6862: Summary: SparseDistributedMatrixStorage cache config possibly allows read of old state of matrix Key: IGNITE-6862 URL: https://issues.apache.org/jira/browse/IGNITE-6862 Project: Ignite Issue Type: Bug Security Level: Public (Viewable by anyone) Reporter: Artem Malykh Fix For: 2.4 Because synchronization mode is PRIMARY_SYNC, but by default we have readFromBackup=true, we can read old state of matrix from backups. It seem to be an issue. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] ignite pull request #3012: IGNITE-6861
GitHub user devozerov opened a pull request: https://github.com/apache/ignite/pull/3012 IGNITE-6861 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-6861 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3012.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3012 commit 6ab3e44ff82e47c8904d34e1ddf048598f7e80d1 Author: devozerovDate: 2017-10-19T20:48:34Z Simple lexer implementation. commit adbb1106cadd94bb0b1ca6faed4271739f9ac72c Author: devozerov Date: 2017-10-27T06:19:39Z Merge branch 'master' into vozerov-sql-parse commit 16e932244a756f4057ec274e94b29bdbf0132521 Author: devozerov Date: 2017-10-27T06:24:50Z Merge branch 'master' into vozerov-sql-parse commit 4f36f62cf05c30b23b14db9482b33a6ebc5d3675 Author: devozerov Date: 2017-10-27T06:42:24Z Skeleton. commit f3fa23fd695604015697ee1777212a37efab Author: devozerov Date: 2017-10-27T06:53:25Z Token types. commit 0832c0262d45282e3c451c7dcc95ae465ef51b2a Author: devozerov Date: 2017-10-27T07:07:06Z WIP. commit 27586e19083298501bb35b6c1be4255911d0d2ec Author: devozerov Date: 2017-10-27T07:09:18Z Minors. commit 03abe1e8104c06ca18fb218385aba20bbaf27961 Author: devozerov Date: 2017-10-27T07:24:44Z WIP. commit a7378a8c91391e13c42c56833ae562d54512 Author: devozerov Date: 2017-10-27T07:46:06Z WIP. commit 6b31cdc46e362c401152f97a118c1ac801e86743 Author: devozerov Date: 2017-10-27T07:58:40Z WIP. commit c8b9063d3a99be0a70099da73a8e1b935275373d Author: devozerov Date: 2017-10-27T08:10:57Z WIP on DROP INDEX. commit 8784139e5a375a78286ff560bc65c73c71ef5c1c Author: devozerov Date: 2017-10-27T08:22:16Z DROP INDEX done. commit e02c09d6faebf314bfb3b9a7133838e34524c095 Author: devozerov Date: 2017-10-27T09:10:49Z DROP INDEX. commit e9a904489052e71cc76bc4a5d9c07c866f560724 Author: devozerov Date: 2017-10-27T09:17:58Z Fixes. commit 8f67b2def35ce8df727406e3e97a03a404e8d3af Author: devozerov Date: 2017-10-27T09:49:53Z Finished CREATE INDEX. commit 4de8b28201a7eb028c78990b593784d05a3190ba Author: devozerov Date: 2017-10-27T09:53:37Z Finished CREATE INDEX. commit 4522d8c2395f4410e0df16a5e72b84eb32426e61 Author: devozerov Date: 2017-10-27T09:56:13Z Minors. commit 1732a3f52d4ba0b1c65598cf7601c08d069ad49b Author: devozerov Date: 2017-10-27T13:15:32Z Merge branch 'master' into ignite-6769 commit c987773daec2d03c42e5476036fdf60ef0b1090d Author: devozerov Date: 2017-10-27T13:18:23Z Minors. commit 2d40e5fcbf6077362eaf2cf67952011ed514efee Author: devozerov Date: 2017-10-27T13:31:19Z Look ahead support. commit 52f3dd62c0c02618a7eece916a9b6c36ea0e60bd Author: devozerov Date: 2017-10-27T14:07:02Z Simplifying. commit 2a118f74ea75c4ef917e2412e3e1e4a4cdffdac4 Author: devozerov Date: 2017-10-27T14:08:24Z Minors. commit 58208e4aa7b6fe049fda649e12f08d1da7e93b6a Author: devozerov Date: 2017-10-27T14:23:24Z Refactoring. commit c0281f6fa294523a14254dc3bbd67f054ad57dba Author: devozerov Date: 2017-10-27T14:29:06Z Refactoring. commit a04c1e86cdee61aacf3fa385b68b3ccdc88a20b5 Author: devozerov Date: 2017-10-27T14:46:17Z INDEX TODOs. commit ec9df63abc3acab51bc6352b3d6c359d8b8cc67f Author: devozerov Date: 2017-10-30T09:17:57Z Merge branch 'master' into ignite-6769 commit 3ed74247504858154b396394f5f522cc5e8e610d Author: devozerov Date: 2017-10-30T09:26:22Z Spatial index processing. commit 96772985a0a0137ee403a4ea17e58f3ac833c526 Author: devozerov Date: 2017-10-30T09:38:42Z Started work on CREATE TABLE. commit 547dad042019ae6674a448adde8f689d5f5add1a Author: devozerov Date: 2017-10-30T12:51:47Z WIP on CREATE TABLE. commit a44c0a6ab44218f95dac1b3654da9c03baea013f Author: devozerov Date: 2017-10-30T20:10:20Z Prohibit keywords. ---
[GitHub] ignite pull request #2940: IGNITE-6769
Github user devozerov closed the pull request at: https://github.com/apache/ignite/pull/2940 ---
[jira] [Created] (IGNITE-6861) SQL parser: support CREATE TABLE and DROP TABLE commands
Vladimir Ozerov created IGNITE-6861: --- Summary: SQL parser: support CREATE TABLE and DROP TABLE commands Key: IGNITE-6861 URL: https://issues.apache.org/jira/browse/IGNITE-6861 Project: Ignite Issue Type: Task Security Level: Public (Viewable by anyone) Components: sql Reporter: Vladimir Ozerov Fix For: 2.4 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: GA Grid: Request to contribute GA library to Apache Ignite
Turik, 1) Yes, it`s correct. 2.a) Model API is available, Trainer API in PR(https://github.com/apache/ignite/pull/2936) which should be merged today or tomorrow. 2.b) Yes, Trainer generates Model. Here is the Trainer interface: public interface Trainer { public M train(T data); } Regards, Yury -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
[jira] [Created] (IGNITE-6860) Lack of context information upon serializing and marshalling (writeObject and writeFields)
Alexandr Kuramshin created IGNITE-6860: -- Summary: Lack of context information upon serializing and marshalling (writeObject and writeFields) Key: IGNITE-6860 URL: https://issues.apache.org/jira/browse/IGNITE-6860 Project: Ignite Issue Type: Bug Security Level: Public (Viewable by anyone) Components: general Affects Versions: 2.3 Reporter: Alexandr Kuramshin Fix For: 2.4 Having the stack trace {noformat} Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to marshal object with optimized marshaller: [org.apache.logging.log4j.core.config.AppenderControl@302e61a8] at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:186) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134) at org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496) at org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160) at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663) at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134) at org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496) at org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160) at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663) at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134) at org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496) at org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160) at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663) at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134) at org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496) at org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160) at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663) at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134) at org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496) at org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160) at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663) at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134) at org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496) at
[GitHub] ignite pull request #2967: IGNITE-6812: Readme.md: Update link to TC build a...
Github user dspavlov closed the pull request at: https://github.com/apache/ignite/pull/2967 ---
[GitHub] ignite pull request #2995: IGNITE-6841: Increased ODBC protocol version for ...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/2995 ---
IGNITE-1025 is ready for review (Need to print out warning if IP finder has a lot of addresses on Windows)
Hi, Igniters! I've prepared PR [1] for the issue IGNITE-1025 "Need to print out warning if IP finder has a lot of addresses on Windows" [2] . TeamCity tests look good [3]. Could someone review it? Thanks in advance! [1]https://github.com/apache/ignite/pull/2966 [2]https://issues.apache.org/jira/browse/IGNITE-1025 [3]https://ci.ignite.apache.org/project.html?projectId= Ignite20Tests=8753246349372607337=testDetails_ Ignite20Tests=pull%2F2966%2Fhead -- Ivan Fedotov. ivanan...@gmail.com
[jira] [Created] (IGNITE-6859) Web console: does not work in IE11
Ilya Borisov created IGNITE-6859: Summary: Web console: does not work in IE11 Key: IGNITE-6859 URL: https://issues.apache.org/jira/browse/IGNITE-6859 Project: Ignite Issue Type: Bug Security Level: Public (Viewable by anyone) Components: wizards Environment: WIndows 10, IE11 Reporter: Ilya Borisov Assignee: Ilya Borisov *What happens:* Web console does not work in IE11 at all, it gets stuck at loading screen indefinitely and logs an error into console. *Expected behavior:* Web console should work in IE11. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] ignite pull request #3011: Ignite : Javadocs for cache change request operat...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3011 ---
[GitHub] ignite pull request #2990: IGNITE-6839: delete binary meta before tests, PDS...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/2990 ---