Re: Spark Data Frame support in Ignite

I think the JDBC one is more inefficient, slower requires too much development effort. You can also check the integration of Alluxio with Spark. Then, in general I think JDBC has never designed for large data volumes. It is for executing queries and getting a small or aggregated result set

[jira] [Created] (IGNITE-5907) Add validation to Basic screen for Off-heap size

Pavel Konstantinov created IGNITE-5907: -- Summary: Add validation to Basic screen for Off-heap size Key: IGNITE-5907 URL: https://issues.apache.org/jira/browse/IGNITE-5907 Project: Ignite

Re: Spark Data Frame support in Ignite

Jorn, thanks for your feedback! Can you explain how the direct support would be different from the JDBC support? Thanks, D. On Thu, Aug 3, 2017 at 7:40 AM, Jörn Franke wrote: > These are two different things. Spark applications themselves do not use > JDBC - it is more

Re: Create own SQL parser

Dmitry, Ignite mode in H2 is about parsing only, it does not address other issues pointed out by Vlad. Alas, implementing _parser_ surely won't take no years, whilst things like smart query optimization, etc, probably don't have and can't have proper "finished" state - it's something that you can

[jira] [Created] (IGNITE-5910) Method stopGrid(name) doesn't work in multiJvm mode

Vyacheslav Daradur created IGNITE-5910: -- Summary: Method stopGrid(name) doesn't work in multiJvm mode Key: IGNITE-5910 URL: https://issues.apache.org/jira/browse/IGNITE-5910 Project: Ignite

[GitHub] ignite pull request #2382: IGNITE-5910 Method stopGrid(name) doesn't work in...

GitHub user daradurvs opened a pull request: https://github.com/apache/ignite/pull/2382 IGNITE-5910 Method stopGrid(name) doesn't work in multiJvm mode You can merge this pull request into a Git repository by running: $ git pull https://github.com/daradurvs/ignite ignite-5910

[jira] [Created] (IGNITE-5913) Web console: code (spring\java) is not generated for field 'Long query timeout:' on cluster level in Miscellaneous group

Pavel Konstantinov created IGNITE-5913: -- Summary: Web console: code (spring\java) is not generated for field 'Long query timeout:' on cluster level in Miscellaneous group Key: IGNITE-5913 URL:

[GitHub] ignite pull request #2383: IGNITE-5912: Redis EXPIRE/PEXPIRE commands.

GitHub user shroman opened a pull request: https://github.com/apache/ignite/pull/2383 IGNITE-5912: Redis EXPIRE/PEXPIRE commands. You can merge this pull request into a Git repository by running: $ git pull https://github.com/shroman/ignite IGNITE-5912 Alternatively you can

Re: Thin client protocol message format

Dima, Our goal is to have a format, which will work for both synchronous, asynchronous, single-threaded and multi-threaded clients. All we need to achieve this is "request ID" propagated from request to response. This way 3-rd party developers will be free to decide how to implement the client.

Re: Spark Data Frame support in Ignite

I think the development effort would still be higher. Everything would have to be put via JDBC into Ignite, then checkpointing would have to be done via JDBC (again additional development effort), a lot of conversion from spark internal format to JDBC and back to ignite internal format.

[jira] [Created] (IGNITE-5908) Web console: may failed to open non-root if user is not authorized

Pavel Konstantinov created IGNITE-5908: -- Summary: Web console: may failed to open non-root if user is not authorized Key: IGNITE-5908 URL: https://issues.apache.org/jira/browse/IGNITE-5908

[jira] [Created] (IGNITE-5912) [Redis] EXPIRE/PEXPIRE on keys

Roman Shtykh created IGNITE-5912: Summary: [Redis] EXPIRE/PEXPIRE on keys Key: IGNITE-5912 URL: https://issues.apache.org/jira/browse/IGNITE-5912 Project: Ignite Issue Type: New Feature

Re: Spark Data Frame support in Ignite

On Thu, Aug 3, 2017 at 8:45 AM, Jörn Franke wrote: > I think the JDBC one is more inefficient, slower requires too much > development effort. You can also check the integration of Alluxio with > Spark. > As far as I know, Alluxio is a file system, so it cannot use JDBC.

Re: Spark Data Frame support in Ignite

On Thu, Aug 3, 2017 at 9:04 AM, Jörn Franke wrote: > I think the development effort would still be higher. Everything would > have to be put via JDBC into Ignite, then checkpointing would have to be > done via JDBC (again additional development effort), a lot of conversion

[jira] [Created] (IGNITE-5909) Web console: Implement editable list

Dmitriy Shabalin created IGNITE-5909: Summary: Web console: Implement editable list Key: IGNITE-5909 URL: https://issues.apache.org/jira/browse/IGNITE-5909 Project: Ignite Issue Type:

[jira] [Created] (IGNITE-5911) .NET: EntityFramework cache eager update

Pavel Tupitsyn created IGNITE-5911: -- Summary: .NET: EntityFramework cache eager update Key: IGNITE-5911 URL: https://issues.apache.org/jira/browse/IGNITE-5911 Project: Ignite Issue Type:

[GitHub] ignite pull request #2384: IGNITE-5897: Fix session init/end logic. This fix...

GitHub user nizhikov opened a pull request: https://github.com/apache/ignite/pull/2384 IGNITE-5897: Fix session init/end logic. This fixes tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/nizhikov/ignite IGNITE-5897

Re: Thin client protocol message format

Got it, thanks! On Aug 3, 2017, 11:04 AM, at 11:04 AM, Vladimir Ozerov wrote: >Dima, > >Our goal is to have a format, which will work for both synchronous, >asynchronous, single-threaded and multi-threaded clients. All we need >to >achieve this is "request ID" propagated

Re: Cluster auto activation design proposal

> * Based on some sort of policies when the actual cluster topology differs too much from the baseline or when some critical condition happens (e.g., when there are no more backups for a partition) Good point, Alex! I would even go further. If cluster is active and under load and nodes continue

[jira] [Created] (IGNITE-5915) Add more clear WAL mode documentation and print a warning when NONE mode is used

Alexey Goncharuk created IGNITE-5915: Summary: Add more clear WAL mode documentation and print a warning when NONE mode is used Key: IGNITE-5915 URL: https://issues.apache.org/jira/browse/IGNITE-5915

[jira] [Created] (IGNITE-5917) Fields have no error-triangle in case of incorrect value

Pavel Konstantinov created IGNITE-5917: -- Summary: Fields have no error-triangle in case of incorrect value Key: IGNITE-5917 URL: https://issues.apache.org/jira/browse/IGNITE-5917 Project: Ignite

[jira] [Created] (IGNITE-5918) Adding and searching objects in index tree produce a lot of garbage

Mikhail Cherkasov created IGNITE-5918: - Summary: Adding and searching objects in index tree produce a lot of garbage Key: IGNITE-5918 URL: https://issues.apache.org/jira/browse/IGNITE-5918

Re: Cluster auto activation design proposal

How about naming it "minimal node set" or "required node set"? ⁣D.​ On Aug 3, 2017, 11:15 AM, at 11:15 AM, Yakov Zhdanov wrote: >> * Based on some sort of policies when the actual cluster topology >differs >too much from the baseline or when some critical condition happens

Re: Cluster auto activation design proposal

>From my standpoint name for the concept should emphasize that nodes from the set constitute a target topology - the place where user wants to be. If we go in a "node set" way, what about FixedNodeSet or BaseNodeSet? "restart node set" also is a bit confusing because this concept works not only

[jira] [Created] (IGNITE-5916) Web console: incorrect default value for Cache-Queries & Indexing-SQL index max inline size = -1

Pavel Konstantinov created IGNITE-5916: -- Summary: Web console: incorrect default value for Cache-Queries & Indexing-SQL index max inline size = -1 Key: IGNITE-5916 URL:

Re: Cluster auto activation design proposal

Ю> How about naming it "minimal node set" or "required node set"? Required for what? I would add restart if there are no confusion. --Yakov

Re: Cluster auto activation design proposal

Yakov, I think it is not just restarts, this set of nodes is minimally required for the cluster to function, no? ⁣D.​ On Aug 3, 2017, 11:23 AM, at 11:23 AM, Yakov Zhdanov wrote: >Ю> How about naming it "minimal node set" or "required node set"? > >Required for what? I

Re: Cluster auto activation design proposal

Dmitriy, Obvious connotation of "minimal set" is a set that cannot be decreased. But lets consider the following case: user has a cluster of 50 nodes and decides to switch off 3 nodes for maintenance for a while. Ok, user just does it and then recreates this "minimal node set" to only 47 nodes.

[GitHub] ignite pull request #2390: ignored_tests suite fixes

GitHub user sergey-chugunov-1985 opened a pull request: https://github.com/apache/ignite/pull/2390 ignored_tests suite fixes You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite z_Ignores_check Alternatively

[jira] [Created] (IGNITE-5922) Improve collisionSpi doc - ParallelJobsNumber should be less than PublicThreadPoolSize

Evgenii Zhuravlev created IGNITE-5922: - Summary: Improve collisionSpi doc - ParallelJobsNumber should be less than PublicThreadPoolSize Key: IGNITE-5922 URL: https://issues.apache.org/jira/browse/IGNITE-5922

[GitHub] ignite pull request #2389: IGNITE-5920: Fix the example. Set CacheKeyConfigu...

GitHub user tledkov-gridgain opened a pull request: https://github.com/apache/ignite/pull/2389 IGNITE-5920: Fix the example. Set CacheKeyConfiguration explicitly to… … enable affinity co-location. You can merge this pull request into a Git repository by running: $ git pull

[GitHub] ignite pull request #2371: IGNITE-5211 Classes based constructor for QueryEn...

Github user tledkov-gridgain closed the pull request at: https://github.com/apache/ignite/pull/2371 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] ignite pull request #2386: IGNITE-5890 Estimated time to rebalance completio...

GitHub user DmitriyGovorukhin opened a pull request: https://github.com/apache/ignite/pull/2386 IGNITE-5890 Estimated time to rebalance completion You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite

[GitHub] ignite pull request #2387: IGNITE-5736 Add test of backward-compatibility

GitHub user daradurvs opened a pull request: https://github.com/apache/ignite/pull/2387 IGNITE-5736 Add test of backward-compatibility You can merge this pull request into a Git repository by running: $ git pull https://github.com/daradurvs/ignite ignite-5736 Alternatively

[GitHub] ignite pull request #2388: IGNITE-5211 Classes based constructor for QueryEn...

GitHub user tledkov-gridgain opened a pull request: https://github.com/apache/ignite/pull/2388 IGNITE-5211 Classes based constructor for QueryEntities You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite

[jira] [Created] (IGNITE-5923) ODBC: SQLGetTypeInfo does not work with SQL_ALL_TYPES

Igor Sapego created IGNITE-5923: --- Summary: ODBC: SQLGetTypeInfo does not work with SQL_ALL_TYPES Key: IGNITE-5923 URL: https://issues.apache.org/jira/browse/IGNITE-5923 Project: Ignite Issue

[jira] [Created] (IGNITE-5924) .NET: Decouple Marshaller from Ignite

Pavel Tupitsyn created IGNITE-5924: -- Summary: .NET: Decouple Marshaller from Ignite Key: IGNITE-5924 URL: https://issues.apache.org/jira/browse/IGNITE-5924 Project: Ignite Issue Type:

Re: Zeppelin, MyBatis and Vert.x update to AI 2.1

Hi, I'll do it soon. But there is problem with Zeppelin because CI should be configured on local machine. So I need more time than usually for update Ignite version in Zeppelin (e.g. Ignite 2.0 still not relased in Zeppelin). On Tue, Aug 1, 2017 at 9:13 AM, Roman Shtykh

[GitHub] ignite pull request #2391: Ignite 5655

GitHub user andrey-kuznetsov opened a pull request: https://github.com/apache/ignite/pull/2391 Ignite 5655 First implementation, uses global level configuration option (in BinaryConfiguration) You can merge this pull request into a Git repository by running: $ git pull

[GitHub] ignite pull request #2392: ODBC: SQLGetTypeInfo now works with SQL_ALL_TYPES

GitHub user isapego opened a pull request: https://github.com/apache/ignite/pull/2392 ODBC: SQLGetTypeInfo now works with SQL_ALL_TYPES You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-5923

[GitHub] ignite pull request #2393: IGNITE-5738: SQL: Added support for batching for ...

GitHub user skalashnikov opened a pull request: https://github.com/apache/ignite/pull/2393 IGNITE-5738: SQL: Added support for batching for jdbc2 driver You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite

[jira] [Created] (IGNITE-5925) Get row/col for matrices

Yury Babak created IGNITE-5925: -- Summary: Get row/col for matrices Key: IGNITE-5925 URL: https://issues.apache.org/jira/browse/IGNITE-5925 Project: Ignite Issue Type: Improvement

[jira] [Created] (IGNITE-5927) .NET: DataTable can't be serialized

Pavel Tupitsyn created IGNITE-5927: -- Summary: .NET: DataTable can't be serialized Key: IGNITE-5927 URL: https://issues.apache.org/jira/browse/IGNITE-5927 Project: Ignite Issue Type: Bug

[GitHub] ignite pull request #2394: IGNITE-5880: BLAS integration phase 2

GitHub user ybabak opened a pull request: https://github.com/apache/ignite/pull/2394 IGNITE-5880: BLAS integration phase 2 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-5880 Alternatively you can

[jira] [Created] (IGNITE-5926) Puts on near cache hangs

vinay created IGNITE-5926: - Summary: Puts on near cache hangs Key: IGNITE-5926 URL: https://issues.apache.org/jira/browse/IGNITE-5926 Project: Ignite Issue Type: Bug Affects Versions: 2.1, 2.0

Re: Release notes tools for Ignite releases

If the solution is positive, can anyone see my pull request? 2017-08-03 4:28 GMT+03:00 dsetrakyan [via Apache Ignite Developers] < ml+s2346864n20403...@n4.nabble.com>: > Agree with Denis. I think that we should treat it as a starting point for > the release notes, and further update it before

Re: Spark Data Frame support in Ignite

This JDBC integration is just a Spark data source, which means that Spark will fetch data in its local memory first, and only then apply filters, aggregations, etc. This is obviously slow and doesn't use all advantages Ignite provides. To create useful and valuable integration, we should create a

[GitHub] ignite pull request #2395: IGNITE-5927 .NET: Fix DataTable serialization

GitHub user ptupitsyn opened a pull request: https://github.com/apache/ignite/pull/2395 IGNITE-5927 .NET: Fix DataTable serialization You can merge this pull request into a Git repository by running: $ git pull https://github.com/ptupitsyn/ignite IGNITE-5927 Alternatively

Hang when near cache is used

Folks, One of the users reported an issue with near cache in 2.0: https://issues.apache.org/jira/browse/IGNITE-5926 There is a reproducer attached, I don't see anything obviously wrong and can reproduce the issue. Can someone take a deeper look? -Val

Ignite.close(), G.stop(name, true). Change flag cancel to false

Hi Igniters, I’ve created the simplest example using Ignite 2.1 and persistence (see the code below). I've included Ignite instance into try-with-resources (I think it is default approach for AutoCloseable inheritors). But next time when I started this server I got message: “Ignite node crashed

[jira] [Created] (IGNITE-5921) Reduce contention for free list access

Mikhail Cherkasov created IGNITE-5921: - Summary: Reduce contention for free list access Key: IGNITE-5921 URL: https://issues.apache.org/jira/browse/IGNITE-5921 Project: Ignite Issue

Re: Cluster auto activation design proposal

I also would like to provide more use cases of how BLT is supposed to work (let me call it this way until we come up with a better one): 1. User creates new BLT using WebConsole or other tool and "applies" it to brand-new cluster. 2. User starts up brand-new cluster with desired amount

Re: Cache Metrics

Den, I see at least two problems here: 1. Metrics meaning for end user. How user should interpret metrics in this case. Moreover, average is bad gauge for monitoring because it hides actual latencies. User should have possibility to get accurate metrics in order to build some monitoring that can

Re: Cluster auto activation design proposal

My understanding of Baseline Topology is the set of nodes which are *expected* to be in the cluster. Let me go a little bit further because BT (or whatever name we choose) may and will solve more issues than just auto-activation: 1) More graceful control over rebalancing than just rebalance

[GitHub] ignite pull request #2385: 8.0.3.ea14

GitHub user agoncharuk opened a pull request: https://github.com/apache/ignite/pull/2385 8.0.3.ea14 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-gg-8.0.3.ea14 Alternatively you can review and

Re: Add isPrimary() and isBackup() methods on CacheQueryEntryEvent

Folks, As far as I see, Issue still in PatchAvailable state, what did you mean by "solved"? On Wed, Aug 2, 2017 at 8:01 PM, Kozlov Maxim wrote: > Sure. > > CacheQueryEntryEvent: > > public abstract boolean isBackup(); > public abstract boolean isPrimary(); > > > > 2 авг.

Re: Cluster auto activation design proposal

Igniters, what about Target Node Set? Complete Node Set? As when we reach this topology, we can activate cluster. чт, 3 авг. 2017 г. в 12:58, Sergey Chugunov : > Dmitriy, > > Obvious connotation of "minimal set" is a set that cannot be decreased. > > But lets consider

Re: Cluster auto activation design proposal

>I think it is not just restarts, this set of nodes is minimally required for the cluster to function, no? I don't think so. Cluster can function if there is no data loss. --Yakov

[jira] [Created] (IGNITE-5920) CacheClientBinaryQueryExample return different results if we add non local node

Aleksey Chetaev created IGNITE-5920: --- Summary: CacheClientBinaryQueryExample return different results if we add non local node Key: IGNITE-5920 URL: https://issues.apache.org/jira/browse/IGNITE-5920

Re: Add isPrimary() and isBackup() methods on CacheQueryEntryEvent

Guys, I'm sorry for the misunderstanding, I was tired at the end of the day :-) In the process of working on the task, I had to add 2 methods to the public interface. The first method is #isBackup(), it returns 'true' if cache is being updated on the backup node. The second method is

Re: Cluster auto activation design proposal

>Obvious connotation of "minimal set" is a set that cannot be decreased. >But lets consider the following case: user has a cluster of 50 nodes and >decides to switch off 3 nodes for maintenance for a while. Ok, user just >does it and then recreates this "minimal node set" to only 47 nodes. >So

[jira] [Created] (IGNITE-5919) .NET: EntryProcessorExample closes immediately after execution

Irina Zaporozhtseva created IGNITE-5919: --- Summary: .NET: EntryProcessorExample closes immediately after execution Key: IGNITE-5919 URL: https://issues.apache.org/jira/browse/IGNITE-5919