Re: WAL Archive Issue

2018-02-15 Thread Dmitry Pavlov
Hi Ivan, Excellent idea, I've added this to https://issues.apache.org/jira/browse/IGNITE-7730 ticket. Sinerely, Dmitriy Pavlov ср, 14 февр. 2018 г. в 0:28, Ivan Rakov : > > - applying compressor to segments older than 1 completed checkpoint > ago - > > saves space. > By the way: WAL compressi

Re: Spark data frames

2018-02-15 Thread Dmitry Pavlov
+1 for starting new topic from Nikolay when ' Community Meeting to Introduce Ignite Spark Data Frames' is ready to be announced. пт, 16 февр. 2018 г. в 9:27, Denis Magda : > I'm in :) > > Nikolay, we can use my GoToMeeting account to host the webinar. To draw > more attention I would suggest sta

[jira] [Created] (IGNITE-7730) Improve WAL history size documentation

2018-02-15 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-7730: -- Summary: Improve WAL history size documentation Key: IGNITE-7730 URL: https://issues.apache.org/jira/browse/IGNITE-7730 Project: Ignite Issue Type: Task

[GitHub] ignite pull request #3182: ignite-gg-13116

2018-02-15 Thread agoncharuk
Github user agoncharuk closed the pull request at: https://github.com/apache/ignite/pull/3182 ---

Re: Batch size parameter at DataStreamerCacheUpdaters.batched()

2018-02-15 Thread Roman Guseinov
Hi Val, Thanks for your response. It seems that is public: https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastreamer/DataStreamerCacheUpdaters.java Best Regards, Roman -- Sent from: http://apache-ignite-developers.2346864.n4.nabb

Re: Spark data frames

2018-02-15 Thread Denis Magda
I'm in :) Nikolay, we can use my GoToMeeting account to host the webinar. To draw more attention I would suggest starting a more specific thread titled like "[RSVP] Community Meeting to Introduce Ignite Spark Data Frames". This discussion sounds too generic, folks could simply pass by. Negotiated

[jira] [Created] (IGNITE-7729) Add usage of Roles for Web Console E2E tests

2018-02-15 Thread Alexander Kalinin (JIRA)
Alexander Kalinin created IGNITE-7729: - Summary: Add usage of Roles for Web Console E2E tests Key: IGNITE-7729 URL: https://issues.apache.org/jira/browse/IGNITE-7729 Project: Ignite Issue

[jira] [Created] (IGNITE-7728) Put together a doc that shows how to blend SQL with k/v APIs

2018-02-15 Thread Denis Magda (JIRA)
Denis Magda created IGNITE-7728: --- Summary: Put together a doc that shows how to blend SQL with k/v APIs Key: IGNITE-7728 URL: https://issues.apache.org/jira/browse/IGNITE-7728 Project: Ignite

TeamCity. Ignite RDD tests

2018-02-15 Thread Nikolay Izhikov
Hello, Igniters. I'm working on issue [1]. Team City doesn't collect info about scalatest execution because of wrong pom.xml I'm fixed it in PR [3] It happens there is a 2 broken tests written in Scala - [4] 1. IgniteRDDSpec.IgniteRDD should successfully store data to ignite using saveValues

[jira] [Created] (IGNITE-7727) IgniteRDDSpec. Failing tests

2018-02-15 Thread Nikolay Izhikov (JIRA)
Nikolay Izhikov created IGNITE-7727: --- Summary: IgniteRDDSpec. Failing tests Key: IGNITE-7727 URL: https://issues.apache.org/jira/browse/IGNITE-7727 Project: Ignite Issue Type: Bug

[jira] [Created] (IGNITE-7726) Error in queries screen in Demo mode

2018-02-15 Thread Alexander Kalinin (JIRA)
Alexander Kalinin created IGNITE-7726: - Summary: Error in queries screen in Demo mode Key: IGNITE-7726 URL: https://issues.apache.org/jira/browse/IGNITE-7726 Project: Ignite Issue Type: B

[jira] [Created] (IGNITE-7725) REST: expand parameters list of GetOrCreateCache command

2018-02-15 Thread Alexey Kuznetsov (JIRA)
Alexey Kuznetsov created IGNITE-7725: Summary: REST: expand parameters list of GetOrCreateCache command Key: IGNITE-7725 URL: https://issues.apache.org/jira/browse/IGNITE-7725 Project: Ignite

Re: Next Steps: GA Grid: Request to contribute GA library to Apache Ignite

2018-02-15 Thread techbysample
Denis, Thank you for providing status update. I look forward to hearing from you. Best, Turik Campbell -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Next Steps: GA Grid: Request to contribute GA library to Apache Ignite

2018-02-15 Thread Denis Magda
Guys, Having troubles connecting to ASF subversion to fill in the IP clearance form today. Something messy with a networking environment in my current workplace. Anyway, checked up that we have everything in place to complete the form and finish the process. Please don't merge the pull-request un

[jira] [Created] (IGNITE-7724) SQL COPY: network performance improvements

2018-02-15 Thread Kirill Shirokov (JIRA)
Kirill Shirokov created IGNITE-7724: --- Summary: SQL COPY: network performance improvements Key: IGNITE-7724 URL: https://issues.apache.org/jira/browse/IGNITE-7724 Project: Ignite Issue Type:

Re: SQL Compliance documentation

2018-02-15 Thread Denis Magda
Guess, it's all about taste. If to google for both "SQL compliance" and "SQL conformance," then will see that DB vendors use both terms for the same thing. -- Denis On Wed, Feb 14, 2018 at 5:09 PM, Dmitriy Setrakyan wrote: > Looks great, Prachi, thanks a lot! > > What is the difference between

Re: Apache Ignite 2.4 release

2018-02-15 Thread Denis Magda
Vladimir, I would suggest not to do this because we still need to spend time on testing, documentation, etc. If someone shows interest in this features they can assemble binaries from the master. -- Denis On Thu, Feb 15, 2018 at 6:43 AM, Nikolay Izhikov wrote: > +1 > > В Чт, 15/02/2018 в 17:27

[jira] [Created] (IGNITE-7723) Data loss after node restart with PDS

2018-02-15 Thread Alexandr Kuramshin (JIRA)
Alexandr Kuramshin created IGNITE-7723: -- Summary: Data loss after node restart with PDS Key: IGNITE-7723 URL: https://issues.apache.org/jira/browse/IGNITE-7723 Project: Ignite Issue Type

Re: Transport compression (not store compression)

2018-02-15 Thread Dmitriy Setrakyan
Vova, I think your solution is fine, but I think we will always have some messages compressed and others not. For example, in many cases, especially when messages are relatively small, compressing them will introduce an unnecessary overhead, and most likely slow down the cluster. Why not have comp

Re: Batch size parameter at DataStreamerCacheUpdaters.batched()

2018-02-15 Thread Valentin Kulichenko
Roman, DataStreamerCacheUpdaters class is actually not a part of public API, so I don't see a reason to change it unless there is a need for this internally in Ignite. -Val On Thu, Feb 15, 2018 at 5:52 AM, Roman Guseinov wrote: > Hello Igniters, > > In some cases, batched stream receiver can h

Re: IgniteSet implementation: changes required

2018-02-15 Thread Dmitriy Setrakyan
On Thu, Feb 15, 2018 at 6:08 AM, Vladimir Ozerov wrote: > I do not think indexes is the right approach - set do not have indexes, and > you will have to maintain additional counter for it in order to know when > to stop. > > From what I see there are two distinct problems: > 1) Broken recovery -

[GitHub] ignite pull request #3511: IGNITE-7686: PDS Direct IO failure: IgnitePdsEvic...

2018-02-15 Thread dspavlov
Github user dspavlov closed the pull request at: https://github.com/apache/ignite/pull/3511 ---

[GitHub] ignite pull request #3532: IGNITE-7686: Fix of PDS Direct IO failure: Ignite...

2018-02-15 Thread dspavlov
GitHub user dspavlov opened a pull request: https://github.com/apache/ignite/pull/3532 IGNITE-7686: Fix of PDS Direct IO failure: IgnitePdsEvictionTest.test… …PageEviction You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridga

[GitHub] ignite pull request #3531: IGNITE-7685: Fixed allocation rate.

2018-02-15 Thread andrey-kuznetsov
GitHub user andrey-kuznetsov opened a pull request: https://github.com/apache/ignite/pull/3531 IGNITE-7685: Fixed allocation rate. You can merge this pull request into a Git repository by running: $ git pull https://github.com/andrey-kuznetsov/ignite ignite-7685 Alternatively

IGNITE-7409. Ready to be merged

2018-02-15 Thread Nikolay Izhikov
Hello, Igniters. There is an issue [1] with very small patch - 4 LOC. State is Patch Available. This part of code was introduced be me in [2]. I reviewed patch and want to merge it. Any objections? [1] https://issues.apache.org/jira/browse/IGNITE-7409 [2] https://issues.apache.org/jira/browse/

[jira] [Created] (IGNITE-7722) IgnitePdsCheckpointSimulationWithRealCpDisabledTest generates too many strings

2018-02-15 Thread Alexey Goncharuk (JIRA)
Alexey Goncharuk created IGNITE-7722: Summary: IgnitePdsCheckpointSimulationWithRealCpDisabledTest generates too many strings Key: IGNITE-7722 URL: https://issues.apache.org/jira/browse/IGNITE-7722

Re: [SparkDataFrame] Query Optimization. Prototype

2018-02-15 Thread Nikolay Izhikov
Hello, Valentin. > In general, I don't see a reason to exclude anything (especially joins) from > PR. Please finalize the change and pass it to me for review. I made some minor improvements. Please, review my PR [1] [1] https://github.com/apache/ignite/pull/3397 В Вт, 13/02/2018 в 11:47 -0800,

[GitHub] ignite pull request #2220: Ignite 3935

2018-02-15 Thread 1vanan
Github user 1vanan closed the pull request at: https://github.com/apache/ignite/pull/2220 ---

Re: Apache Ignite 2.4 release

2018-02-15 Thread Nikolay Izhikov
+1 В Чт, 15/02/2018 в 17:27 +0300, Vladimir Ozerov пишет: > Igniters, > > AI 2.4 release was shifted a bit and over this time we implemented two > important SQL features: > 1) COPY command for fast file upload to the cluster [1] > 2) Streaming mode for thin driver [2] > > Both commands are very

Re: Apache Ignite 2.4 release

2018-02-15 Thread Vladimir Ozerov
Igniters, AI 2.4 release was shifted a bit and over this time we implemented two important SQL features: 1) COPY command for fast file upload to the cluster [1] 2) Streaming mode for thin driver [2] Both commands are very important for fast data ingestion into Ignite through SQL. I would like to

Re: IgniteSet implementation: changes required

2018-02-15 Thread Vladimir Ozerov
I do not think indexes is the right approach - set do not have indexes, and you will have to maintain additional counter for it in order to know when to stop. >From what I see there are two distinct problems: 1) Broken recovery - this is just a bug which needs to be fixed. As soon as data is store

Batch size parameter at DataStreamerCacheUpdaters.batched()

2018-02-15 Thread Roman Guseinov
Hello Igniters, In some cases, batched stream receiver can help us to improve performance: try (IgniteDataStreamer streamer = ignite.dataStreamer(cacheName)) { streamer.receiver(DataStreamerCacheUpdaters.batched()); streamer.addData(getData()); } Unfortunately, the bad

Re: Wrapping [Ignite]Interrupted[Checked]Exception in benign exceptions

2018-02-15 Thread Александр Меньшиков
Ilya, I agree with you about reducing the number of unchecked exceptions in public API. Because when you work with grid it can throw about 4 types of different runtime exception. And there is no way except experiments to know about this types. 2018-02-15 15:57 GMT+03:00 Dmitriy Setrakyan : > Ilya

[jira] [Created] (IGNITE-7721) Apache Ignite web session clustering stack after login success

2018-02-15 Thread Sanjeet Jha (JIRA)
Sanjeet Jha created IGNITE-7721: --- Summary: Apache Ignite web session clustering stack after login success Key: IGNITE-7721 URL: https://issues.apache.org/jira/browse/IGNITE-7721 Project: Ignite

[jira] [Created] (IGNITE-7720) Update ODBC cluster configuration: replace OdbcConfiguration with ClientConnectorConfiguration

2018-02-15 Thread Alexey Popov (JIRA)
Alexey Popov created IGNITE-7720: Summary: Update ODBC cluster configuration: replace OdbcConfiguration with ClientConnectorConfiguration Key: IGNITE-7720 URL: https://issues.apache.org/jira/browse/IGNITE-7720

Re: Wrapping [Ignite]Interrupted[Checked]Exception in benign exceptions

2018-02-15 Thread Dmitriy Setrakyan
Ilya, i have looked at the ticket. I am not sure I understand what you are suggesting. Can you provide a "before" and "after" example? D. On Thu, Feb 15, 2018 at 4:40 AM, Ilya Kasnacheev wrote: > Hello Igniters. > > I have stumbled on the problem for which I have created a ticket > https://issu

Wrapping [Ignite]Interrupted[Checked]Exception in benign exceptions

2018-02-15 Thread Ilya Kasnacheev
Hello Igniters. I have stumbled on the problem for which I have created a ticket https://issues.apache.org/jira/browse/IGNITE-7719 Basically it is an awful code smell. On thread interrupt, we wrap InterruptedException with some unrelated exception type, which prevents it from being handled proper

[jira] [Created] (IGNITE-7719) Avoid wrapping InterruptedException in CacheException or IgniteException

2018-02-15 Thread Ilya Kasnacheev (JIRA)
Ilya Kasnacheev created IGNITE-7719: --- Summary: Avoid wrapping InterruptedException in CacheException or IgniteException Key: IGNITE-7719 URL: https://issues.apache.org/jira/browse/IGNITE-7719 Projec

[GitHub] ignite pull request #3530: IGNITE-7042: Trying to configure scala-test plugi...

2018-02-15 Thread nizhikov
GitHub user nizhikov opened a pull request: https://github.com/apache/ignite/pull/3530 IGNITE-7042: Trying to configure scala-test plugin for a TeamCity You can merge this pull request into a Git repository by running: $ git pull https://github.com/nizhikov/ignite IGNITE-7042

[GitHub] ignite pull request #3377: IGNITE-7386: Got rid of ThreadLocalRandom8.

2018-02-15 Thread andrey-kuznetsov
Github user andrey-kuznetsov closed the pull request at: https://github.com/apache/ignite/pull/3377 ---

[GitHub] ignite pull request #2339: Ignite 4181 public api

2018-02-15 Thread andrey-kuznetsov
Github user andrey-kuznetsov closed the pull request at: https://github.com/apache/ignite/pull/2339 ---

[GitHub] ignite pull request #3165: IGNITE-6711: TotalAllocatedPages metric fix.

2018-02-15 Thread andrey-kuznetsov
Github user andrey-kuznetsov closed the pull request at: https://github.com/apache/ignite/pull/3165 ---

[GitHub] ignite pull request #3197: Ignite 6734

2018-02-15 Thread andrey-kuznetsov
Github user andrey-kuznetsov closed the pull request at: https://github.com/apache/ignite/pull/3197 ---

[jira] [Created] (IGNITE-7718) Collections.singleton() and Collections.singletonMap() are not properly serialized by binary marshaller

2018-02-15 Thread Pavel Vinokurov (JIRA)
Pavel Vinokurov created IGNITE-7718: --- Summary: Collections.singleton() and Collections.singletonMap() are not properly serialized by binary marshaller Key: IGNITE-7718 URL: https://issues.apache.org/jira/browse/

[GitHub] ignite pull request #3470: ignite-5804: ScanQuery transformer should be appl...

2018-02-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3470 ---

[GitHub] ignite pull request #3248: IGNITE-6736: Switched GridCacheMapEntry synchroni...

2018-02-15 Thread andrey-kuznetsov
Github user andrey-kuznetsov closed the pull request at: https://github.com/apache/ignite/pull/3248 ---

[GitHub] ignite pull request #3302: IGNITE-7312: Made use of java.util.Base64 for bas...

2018-02-15 Thread andrey-kuznetsov
Github user andrey-kuznetsov closed the pull request at: https://github.com/apache/ignite/pull/3302 ---

[GitHub] ignite pull request #3428: Ignite 7513

2018-02-15 Thread andrey-kuznetsov
Github user andrey-kuznetsov closed the pull request at: https://github.com/apache/ignite/pull/3428 ---

[jira] [Created] (IGNITE-7717) testAssignmentAfterRestarts is flaky on TC

2018-02-15 Thread Pavel Kovalenko (JIRA)
Pavel Kovalenko created IGNITE-7717: --- Summary: testAssignmentAfterRestarts is flaky on TC Key: IGNITE-7717 URL: https://issues.apache.org/jira/browse/IGNITE-7717 Project: Ignite Issue Type:

[jira] [Created] (IGNITE-7716) Red selftest in ML examples

2018-02-15 Thread Yury Babak (JIRA)
Yury Babak created IGNITE-7716: -- Summary: Red selftest in ML examples Key: IGNITE-7716 URL: https://issues.apache.org/jira/browse/IGNITE-7716 Project: Ignite Issue Type: Bug Components

[GitHub] ignite pull request #3529: IGNITE-7698: Page read during replacement should ...

2018-02-15 Thread dspavlov
GitHub user dspavlov opened a pull request: https://github.com/apache/ignite/pull/3529 IGNITE-7698: Page read during replacement should be outside of segmen… …t write lock You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridga

[jira] [Created] (IGNITE-7715) If client cannot find dml entity in local storage, it should ask server for updates

2018-02-15 Thread Pavel Kuznetsov (JIRA)
Pavel Kuznetsov created IGNITE-7715: --- Summary: If client cannot find dml entity in local storage, it should ask server for updates Key: IGNITE-7715 URL: https://issues.apache.org/jira/browse/IGNITE-7715

[jira] [Created] (IGNITE-7714) SQL: COPY command should try to create cache in case table is not found

2018-02-15 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-7714: --- Summary: SQL: COPY command should try to create cache in case table is not found Key: IGNITE-7714 URL: https://issues.apache.org/jira/browse/IGNITE-7714 Project

[GitHub] ignite pull request #3526: IGNITE-7709: SQL COPY file name handling fix

2018-02-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3526 ---

[jira] [Created] (IGNITE-7713) Include cache name to rebalancing finish message

2018-02-15 Thread Alexey Goncharuk (JIRA)
Alexey Goncharuk created IGNITE-7713: Summary: Include cache name to rebalancing finish message Key: IGNITE-7713 URL: https://issues.apache.org/jira/browse/IGNITE-7713 Project: Ignite Iss

Re: Transport compression (not store compression)

2018-02-15 Thread Vladimir Ozerov
I think that we should not guess on how the clients are used. They could be used in any way - in the same network, in another network, in Docker, in hypervisor, etc.. This holds for both thin and thick clients. It is essential that we design configuration API in a way that compression could be enab

Re: EvictableEntry.isCached() state meaning

2018-02-15 Thread Alexey Goncharuk
Andrey, isCached() may return false if the entry was concurrently removed from the heap (note that EvictableEntry covers only on-heap evictions since it is passed to the instance of an EvictionPolicy). Any cache access of the entry key (get or update) will bring the entry back on-heap. --AG 2018