date:20180306

http://apache.org/dist/ignite/2.3.0/ 



> On 6 Mar 2018, at 23:21, Dmitriy Setrakyan  wrote:
> 
> I don't see MD5 files anywhere on the website:
> https://ignite.apache.org/download.cgi
> 
> Where do we have them?
> 
> D.
> 
> 
> On Tue, Mar 6, 2018 at 4:50 AM, Vladimir Ozerov 
> wrote:
> 
>> I am not sure. We do have SHA512 files, which is ok. But we also have MD5
>> files, which are now "SHOULD NOT" be provided. Is it OK that we still have
>> them?
>> 
>> On Tue, Mar 6, 2018 at 10:23 AM, Dmitriy Setrakyan 
>> wrote:
>> 
>>> On Mon, Mar 5, 2018 at 12:57 PM, Denis Magda  wrote:
>>> 
 Igniters,
 
 Do we comply with the next release requirements? Vladimir as a 2.4
>>> release
 manager, could you double check that we are in a good state?
 
>>> 
>>> I think we do. We only provide SHA files for our releases.
>>> 
>>

Re: [VOTE] Apache Ignite 2.4.0 RC1

Nikolai, please file a ticket for this for 2.5 version. I doubt its a
blocker for this release.

D.

On Tue, Mar 6, 2018 at 8:41 PM, Nikolay Izhikov  wrote:

> DEVNOTEX.txt also contains mentioning of jdk7:
>
> ```
> Ignite Hadoop Accelerator Maven Build Instructions
> ...
> 2) Assembly Hadoop Accelerator:
> ...
>NOTE: JDK version should be 1.7.0-* or >= 1.8.0-u40.
> ```
>
> В Ср, 07/03/2018 в 07:25 +0300, Nikolay Izhikov пишет:
> > Hello, Vladimir.
> >
> > I'm looked into
> >
> > apache-ignite-hadoop-2.4.0-bin.zip/HADOOP_README.txt
> >
> > It contains:
> >
> > ```
> > Installation requirements:
> >
> > 1. Windows, Linux, or MacOS environment.
> > 2. Java 7 or 8 (latest update is advisable).
> > ```
> >
> > Seems, it has to be updated because Java 7 is not supported in 2.4
> >
> >
> > В Пн, 05/03/2018 в 22:42 +0300, Vladimir Ozerov пишет:
> > >  Igniters,
> > >
> > > We have uploaded a 2.4.0 release candidate to
> > > https://dist.apache.org/repos/dist/dev/ignite/2.4.0-rc1/
> > >
> > > Git tag name is
> > > 2.4.0-rc1
> > >
> > > This release includes the following changes:
> > >
> > > Ignite:
> > > * Introduced Baseline Affinity Topology
> > > * Ability to disable WAL for cache in runtime through IgniteCluster
> API or
> > > ALTER TABLE command
> > > * Added ability to convert WAL to human-readable form
> > > * Ability to enable/disable JDBC, ODBC and thin client endpoints
> > > * Project moved to Java 8
> > > * Java 7 is no longer supported
> > > * Apache Ignite can be run on Java 9
> > > * Introduced Apache Ignite packaging with RPM
> > > * Spark Data Frames support
> > > * Added integration with Spring 3.x
> > > * Added handler to stop node when IO error is encountered
> > > * EvictionPolicyFactory support added to CacheConfiguration
> > > * Added JMX metrics for memory regions
> > > * Added detailed memory consumption on start and OOM reporting
> > > * Added custom thread pools monitoring
> > > * Support Ignite MR in Kerberized environment without IGFS
> > > * S3 IP finder: support server side encryption and bucket endpoint
> > > configuration
> > > * Multiple fixes and performance optimizations
> > >
> > > Ignite.NET:
> > > * Thin Client initial release
> > > * .NET Core / Linux / macOS support
> > > * Mono platform support
> > > * Removed C++ JNI layer
> > >
> > > SQL:
> > > * Added DEFAULT support to CREATE TABLE command
> > > * Added ALTER TABLE DROP COLUMN support
> > > * Added INLINE_SIZE option to CREATE INDEX command
> > > * Added PARALLEL option to CREATE INDEX command
> > > * Added optional on-heap row cache
> > > * INSERT command now can be executed without explicit column names
> (INSERT
> > > INTO table VALUES ...)
> > > * Allowed multiple caches to share the same schema
> > > * Added support for Java 8 Date and Time API
> > > * Added "notNull" property to QuerySqlField annotation
> > > * Improved COUNT(*) performance
> > > * Fixed a problem causing an exception in certain nodes do not have
> primary
> > > partitions for REPLICATED cache
> > > * Fixed per-thread H2 connection leak.
> > > * Fixed partition exchange hang due to SQL query running inside a
> > > transaction.
> > >
> > > JDBC Driver:
> > > * Optimized batched operations processing
> > >
> > > ODBC Driver:
> > > * Added support of multiple statements execution with one call
> > > * Added support of SQL_ATTR_CONNECTION_TIMEOUT
> > > * Added support of SQL_ATTR_QUERY_TIMEOUT
> > > * Optimized batched operations processing
> > > * Proper handling of ungraceful TCP disconnects (keep-alive connection)
> > > * Fixed error preventing close of executed DML statement
> > > * Fixed table names returned by SQLTables for DDL-created tables
> > >
> > > Machine Learning:
> > > * Implemented Fuzzy c-means algorithm
> > > * Implemented gradient descent for OLS Linear Regression
> > > * Implemented K nearest neighbor algorithm
> > > * Introduced API for datasets
> > > * Introduced API for ML model and trainers
> > > * Introduced common mechanism for group training of models
> > > * Added distributed version of multilayer perceptron
> > > * Added local version of multilayer perceptron
> > > * Added basic import/export functionality for ml models
> > > * Added decision tree algorithm
> > > * Performance optimizations for distributed operations
> > >
> > > Web Console:
> > > * Implemented component for cluster selection and activation
> > > * Implemented support for multiple statements on Queries screen
> > > * Implemented progress indication for 'execute', 'scan' and 'export'
> > > actions on Queries screen
> > > * Added support for ClientConnectorConfiguration
> > > * Added several SQL examples on Queries screen
> > > * Added "Connected clusters" component in header
> > > * Added support for custom SMTP server configuration
> > > * Added detection of CSV separator based on browser locale
> > > * Added "Copy to clipboard" action on Queries screen
> > > * Added duration and node ID in results header and 'Show query' modal

Re: [VOTE] Apache Ignite 2.4.0 RC1

Hello, Vladimir.

I'm looked into 

apache-ignite-hadoop-2.4.0-bin.zip/HADOOP_README.txt

It contains:

```
Installation requirements:

1. Windows, Linux, or MacOS environment.
2. Java 7 or 8 (latest update is advisable).
```

Seems, it has to be updated because Java 7 is not supported in 2.4


В Пн, 05/03/2018 в 22:42 +0300, Vladimir Ozerov пишет:
>  Igniters,
> 
> We have uploaded a 2.4.0 release candidate to
> https://dist.apache.org/repos/dist/dev/ignite/2.4.0-rc1/
> 
> Git tag name is
> 2.4.0-rc1
> 
> This release includes the following changes:
> 
> Ignite:
> * Introduced Baseline Affinity Topology
> * Ability to disable WAL for cache in runtime through IgniteCluster API or
> ALTER TABLE command
> * Added ability to convert WAL to human-readable form
> * Ability to enable/disable JDBC, ODBC and thin client endpoints
> * Project moved to Java 8
> * Java 7 is no longer supported
> * Apache Ignite can be run on Java 9
> * Introduced Apache Ignite packaging with RPM
> * Spark Data Frames support
> * Added integration with Spring 3.x
> * Added handler to stop node when IO error is encountered
> * EvictionPolicyFactory support added to CacheConfiguration
> * Added JMX metrics for memory regions
> * Added detailed memory consumption on start and OOM reporting
> * Added custom thread pools monitoring
> * Support Ignite MR in Kerberized environment without IGFS
> * S3 IP finder: support server side encryption and bucket endpoint
> configuration
> * Multiple fixes and performance optimizations
> 
> Ignite.NET:
> * Thin Client initial release
> * .NET Core / Linux / macOS support
> * Mono platform support
> * Removed C++ JNI layer
> 
> SQL:
> * Added DEFAULT support to CREATE TABLE command
> * Added ALTER TABLE DROP COLUMN support
> * Added INLINE_SIZE option to CREATE INDEX command
> * Added PARALLEL option to CREATE INDEX command
> * Added optional on-heap row cache
> * INSERT command now can be executed without explicit column names (INSERT
> INTO table VALUES ...)
> * Allowed multiple caches to share the same schema
> * Added support for Java 8 Date and Time API
> * Added "notNull" property to QuerySqlField annotation
> * Improved COUNT(*) performance
> * Fixed a problem causing an exception in certain nodes do not have primary
> partitions for REPLICATED cache
> * Fixed per-thread H2 connection leak.
> * Fixed partition exchange hang due to SQL query running inside a
> transaction.
> 
> JDBC Driver:
> * Optimized batched operations processing
> 
> ODBC Driver:
> * Added support of multiple statements execution with one call
> * Added support of SQL_ATTR_CONNECTION_TIMEOUT
> * Added support of SQL_ATTR_QUERY_TIMEOUT
> * Optimized batched operations processing
> * Proper handling of ungraceful TCP disconnects (keep-alive connection)
> * Fixed error preventing close of executed DML statement
> * Fixed table names returned by SQLTables for DDL-created tables
> 
> Machine Learning:
> * Implemented Fuzzy c-means algorithm
> * Implemented gradient descent for OLS Linear Regression
> * Implemented K nearest neighbor algorithm
> * Introduced API for datasets
> * Introduced API for ML model and trainers
> * Introduced common mechanism for group training of models
> * Added distributed version of multilayer perceptron
> * Added local version of multilayer perceptron
> * Added basic import/export functionality for ml models
> * Added decision tree algorithm
> * Performance optimizations for distributed operations
> 
> Web Console:
> * Implemented component for cluster selection and activation
> * Implemented support for multiple statements on Queries screen
> * Implemented progress indication for 'execute', 'scan' and 'export'
> actions on Queries screen
> * Added support for ClientConnectorConfiguration
> * Added several SQL examples on Queries screen
> * Added "Connected clusters" component in header
> * Added support for custom SMTP server configuration
> * Added detection of CSV separator based on browser locale
> * Added "Copy to clipboard" action on Queries screen
> * Added duration and node ID in results header and 'Show query' modal
> * Improved support for binary JDBC types on import from RDBMS
> * Fixed Web Console under IE11
> * Fixed configuration generation for imported model in case of Oracle
> NUMBER(5) data type
> * Fixed output of big numbers in SQL query results
> 
> Visor:
> * Added "-quiet" flag for batch mode
> * Added ability to start cache rebalance
> * Added output of data region metrics to "node" command
> * Added dialog to put/get/remove values to/from cache for simple key types
> * Show valid message for caches when cluster is inactive
> * Fixed 'cache -a' in case of node filter
> * Fixed reading last command line in batch mode
> * Updated eviction policy factory in configs
> 
> Complete list of closed issues:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%202.4%20AND%20(status%20%3D%20closed%20or%20status%20%3D%20resolved)
> 
> DEVNOTES
>

[jira] [Created] (IGNITE-7895) Revise unit tests for Web Console.

2018-03-06 Thread Alexander Kalinin (JIRA)

Alexander Kalinin created IGNITE-7895:
-

 Summary: Revise unit tests for Web Console.
 Key: IGNITE-7895
 URL: https://issues.apache.org/jira/browse/IGNITE-7895
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexander Kalinin
Assignee: Alexander Kalinin
 Fix For: 2.5






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7894) Web console: extract new design collapsible panels into component

2018-03-06 Thread Ilya Borisov (JIRA)

Ilya Borisov created IGNITE-7894:


 Summary: Web console: extract new design collapsible panels into 
component
 Key: IGNITE-7894
 URL: https://issues.apache.org/jira/browse/IGNITE-7894
 Project: Ignite
  Issue Type: Improvement
  Components: wizards
Reporter: Ilya Borisov
Assignee: Ilya Borisov
 Attachments: image-2018-03-07-10-39-32-857.png, 
image-2018-03-07-10-42-01-013.png, image-2018-03-07-10-42-44-368.png

Collapsible panels in new design still don't have a reusable component that 
encapsulates it's behavior and styles.

Where such panels are/will be used:
1. Redesigned cluster configuration forms.
 !image-2018-03-07-10-42-44-368.png! 
2. Current user profile.
 !image-2018-03-07-10-42-01-013.png! 
3. Upcoming queries screen redesign.
!image-2018-03-07-10-39-32-857.png! 

New component should:
1. Have bindings for opened state and opened/closed events.
2. Have a way to insert buttons to the right of header. 

Make sure that there are no leftover unused styles/code left.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

RE: Is ML a part of the Ignite.C# API? eom

2018-03-06 Thread Raymond Wilson

… the reason I ask being that ML is included in the Ignite diagram in the
Ignite.C# documentation page, but there is no documentation section for it.



*From:* Raymond Wilson [mailto:raymond_wil...@trimble.com]
*Sent:* Wednesday, March 7, 2018 3:55 PM
*To:* u...@ignite.apache.org
*Cc:* 'dev@ignite.apache.org' 
*Subject:* Is ML a part of the Ignite.C# API? eom

Re: Ignite nightly build job on Apache Jenkins server

I've started to accumulate information about our tests eco-system here
https://cwiki.apache.org/confluence/display/IGNITE/Continuous+Integration (It
draft now, I'll also send an announcing mail when it will be ready).

I will add this link to the page also.

ср, 7 мар. 2018 г. в 0:03, Dmitriy Setrakyan :

> OK, fine, in that case please send out a link to the TC nightly builds
> whenever it is ready, so the community would know where to get it.
>
> On Tue, Mar 6, 2018 at 12:16 AM, Petr Ivanov  wrote:
>
> > Amount of effort will exceed creating new build on TeamCity.
> > Currently on TeamCity we have prepared and working Release Build project,
> > which can be copied and modified to run at night.
> >
> > A think we have to discuss the delivery of those nightly release
> artifacts
> > to users: some king of page ’Nightly build' at ignite.apache.org <
> > http://ignite.apache.org/> or alike.
> >
> >
> >
> > > On 6 Mar 2018, at 11:05, Dmitriy Setrakyan 
> > wrote:
> > >
> > > On Tue, Mar 6, 2018 at 12:02 AM, Petr Ivanov 
> > wrote:
> > >
> > >> I guess there is no such thing as Nightly RELEASE Builds right now in
> > >> TeamCity.
> > >>
> > >> We have only nightly test builds on master.
> > >>
> > >> I guess there is not much effort in adding nightly release build, so,
> if
> > >> this task has it’s priority — please fill the ticket and I’ll prepare
> > >> corresponding build.
> > >>
> > >
> > > In the mean time, can we fix the Jenkin's build?
> >
> >
>

Re: Ignite nightly build job on Apache Jenkins server

OK, fine, in that case please send out a link to the TC nightly builds
whenever it is ready, so the community would know where to get it.

On Tue, Mar 6, 2018 at 12:16 AM, Petr Ivanov  wrote:

> Amount of effort will exceed creating new build on TeamCity.
> Currently on TeamCity we have prepared and working Release Build project,
> which can be copied and modified to run at night.
>
> A think we have to discuss the delivery of those nightly release artifacts
> to users: some king of page ’Nightly build' at ignite.apache.org <
> http://ignite.apache.org/> or alike.
>
>
>
> > On 6 Mar 2018, at 11:05, Dmitriy Setrakyan 
> wrote:
> >
> > On Tue, Mar 6, 2018 at 12:02 AM, Petr Ivanov 
> wrote:
> >
> >> I guess there is no such thing as Nightly RELEASE Builds right now in
> >> TeamCity.
> >>
> >> We have only nightly test builds on master.
> >>
> >> I guess there is not much effort in adding nightly release build, so, if
> >> this task has it’s priority — please fill the ticket and I’ll prepare
> >> corresponding build.
> >>
> >
> > In the mean time, can we fix the Jenkin's build?
>
>

Re: Let's keep Apache Ignite docs up-to-date

2018-03-06 Thread Denis Magda

Guys,

The migration process is in the progress. Here is a discussion where we
exchanged alternate doc engines (let's keep talking there):
http://apache-ignite-developers.2346864.n4.nabble.com/Move-documentation-from-readme-io-to-GitHub-pages-td16409.html

Add to the JIRA ticket as a watcher if you wish to track the progress:
https://issues.apache.org/jira/browse/IGNITE-7595

The goal is to migrate to the new docs by the next Ignite 2.5 release.

--
Denis

On Tue, Mar 6, 2018 at 2:43 AM, Guru Stron 
wrote:

> Hi, Igniters
>
> Missed this discussion
>
> +1 for git docs, as far as i can see this approach is used by many projects
> and it seems to be quite good.
>
>
> On 6 March 2018 at 11:44, Alexey Goncharuk 
> wrote:
>
> > Igniters,
> >
> > Bumping up this discussion.
> >
> > I have recently found out that we have this process for documenting new
> > releases [1] which looks quite ridiculous to me.
> >
> > First, creating a copy of page with next-version suffix is inconvenient
> and
> > error-prone: the next-version page is not visible to anyone, moreover,
> all
> > suggested edits to current documentation will be lost after the page copy
> > is created. Second, the documentation changes should be transparent to
> > users, but now a regular user cannot even review upcoming changes until
> > they are granted a permission to see/edit hidden pages.
> >
> > Unless we have very strong reasons to keep documentation on readme.io
> (by
> > strong I mean a feature that cannot be implemented using a VCS + doc
> > generator), I would at least spend some time piloting the 'keep docs in
> the
> > VCS' approach.
> >
> > Thoughts?
> >
> > [1] https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
> >
> > 2017-11-02 10:07 GMT+03:00 Dmitry Pavlov :
> >
> > > I don't like git docs idea, it will require to follow whole
> > > PR-review-process that requires long time. IMO it is odd work.
> > > If readme.io provides review process, I suggest to keep it as-is.
> > >
> > > чт, 2 нояб. 2017 г. в 9:57, Dmitriy Setrakyan :
> > >
> > > > On Wed, Nov 1, 2017 at 11:27 PM, Vladimir Ozerov <
> voze...@gridgain.com
> > >
> > > > wrote:
> > > >
> > > > > +1 for moving docs under Git provided that we find a way to update
> > docs
> > > > > outside of AI release as it is possible now with readme.io.
> > > > >
> > > >
> > > > I am HUGE +1 for that. The whole problem is that we haven't found a
> way
> > > > yet. All I want is to update a page and have it commit to GIT and
> > become
> > > > available to public right away. Does anyone know any tool that
> supports
> > > it?
> > > >
> > >
> >
>

Re: Transparent Data Encryption (TDE) in Apache Ignite

Hi Nikolay,

Please note there is cluster-auto activation when it reaches baseline
topology.

As far as I remember to be PCI-DSS compliant it is sufficient to use
encryption at file system level. But it needs to be double-checked. It
requires encrypt transmission of cardholder data across open, public
networks. Could you point me where does it require DB data to be encrypted?

Would this solution have pros in the point of view of security compared
with encrypted FS usage?

Please keep me posted, I would like to join this review.

Sincerely,
Dmitriy Pavlov

вт, 6 мар. 2018 г. в 13:29, Nikolay Izhikov :

> Alexey,
>
> Yes, administrator has to enter password before cluster *activation*(not
> start).
>
> В Вт, 06/03/2018 в 13:27 +0300, Alexey Goncharuk пишет:
> > Nikolay,
> >
> > Does it mean that administrator must enter the MEK password upon Ignite
> > start?
> >
> > 2018-03-06 13:24 GMT+03:00 Nikolay Izhikov :
> >
> > > Hello, Alexey.
> > >
> > > Thank you for very helpfull feedback.
> > > We certainly consider all the issues you written.
> > >
> > > > How encryption keys will be stored and accessed?
> > >
> > > *MEK(Master encryption key)* will be stored in regular java key store
> JKS
> > > [1].
> > > To access it admin must enter key store password.
> > >
> > > *CEK(Cache encryption key)* will be stored inside regular Ignite cache.
> > > It will be encrypted by MEK.
> > > So access to CEK may be done only after one got the MEK.
> > >
> > > Please, see IEP draft for futher information [2].
> > >
> > > > Will pages be encrypted on disk or also in memory?
> > >
> > > Current understanding that only on disk pages will be encrypted.
> > > PCI DSS standart requires only on disk encryption.
> > >
> > > > How do you make sure that encrypted page fits the page size
> > >
> > > AFAIK encryption doesn't affect data size.
> > >
> > > > This significantly increases success of known-plain-text attacks.
> > > > How will you write WAL delta records for encrypted pages?
> > >
> > > I don't have an answer to this questions for now.
> > > So, prior to starting an implementation we returns with answers.
> > >
> > > [1]
> https://docs.oracle.com/javase/7/docs/api/java/security/KeyStore.html
> > > [2] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
> > >
> > >
> > > В Вт, 06/03/2018 в 13:04 +0300, Alexey Goncharuk пишет:
> > > > Guys,
> > > >
> > > > I think this TDE proposal is not thought through enough yet. Please
> > > > consider the following points when writing the IEP:
> > > >
> > > >  * How encryption keys will be stored and accessed? If the
> encryption key
> > > > is stored with the same permissions as the main data storage, the
> whole
> > > > exercise with encryption is self-deception
> > > >  * Will pages be encrypted on disk or also in memory?
> > > >  * How do you make sure that encrypted page fits the page size (I am
> not
> > >
> > > an
> > > > expert in encryption, so I am not sure if it adds an overhead)
> > > >  * As Dmitriy Pavlov mentioned, currently data and index pages are
> highly
> > > > redundant and some of the fields in certain pages are known in
> advance.
> > > > This significantly increases success of known-plain-text attacks.
> How are
> > > > you planning to fix it?
> > > >  * How will you write WAL delta records for encrypted pages? If a
> change
> > >
> > > in
> > > > a single byte will potentially change the whole page, this will
> induce a
> > > > huge write amplification on WAL. How do you encrypt WAL data
> records? How
> > > > will this work with this optimization: [1]
> > > >
> > > > [1] https://ggsystems.atlassian.net/browse/IGN-7789
> > > >
> > > > --AG
> > > >
> > > > 2018-03-06 8:55 GMT+03:00 Nikolay Izhikov :
> > > >
> > > > > Thank you, it's - nizhikov
> > > > >
> > > > > В Пн, 05/03/2018 в 15:09 -0800, Denis Magda пишет:
> > > > > > Nikolay, what's your Wiki ID? I'll grant you required
> permissions.
> > > > > >
> > > > > > --
> > > > > > Denis
> > > > > >
> > > > > > On Sun, Mar 4, 2018 at 11:00 PM, Nikolay Izhikov <
> > >
> > > nizhi...@apache.org>
> > > > >
> > > > > wrote:
> > > > > > > Hello, Denis.
> > > > > > >
> > > > > > > > I would encourage you creating an IEP
> > > > > > >
> > > > > > > That is exactly what we want to do :)
> > > > > > >
> > > > > > > But seems I have not sufficient privileges to do it on Ignite
> wiki.
> > > > > > >
> > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/
> > >
> > > Active+Proposals
> > > > > > >
> > > > > > > Can you or someone give me such rights?
> > > > > > >
> > > > > > > В Чт, 01/03/2018 в 22:23 -0800, Denis Magda пишет:
> > > > > > > > Dmitriy R., Nilokay,
> > > > > > > >
> > > > > > > > Thanks for the analysis and handout of the architectural
> design.
> > >
> > > No
> > > > >
> > > > > doubts,
> > > > > > > > it would be a valuable addition to Ignite.
> > > > > > > >
> > > > > > > > I would encourage you creating an IEP on the wiki and break
> the

Re: [VOTE] Apache Ignite 2.4.0 RC1

Igniters,

I appologize for querying probably well-known information:

Is there any nexus repository, probably staging, which can be set up in
gradle/maven in addition to
https://mvnrepository.com/artifact/org.apache.ignite/ignite-core  where
this RC-1 is located?

Could it be tested using simple version change in gradle to '2.4.0-rc1' ?

Sincerely,
Dmitriy Pavlov

вт, 6 мар. 2018 г. в 21:57, Denis Magda :

> +1 (binding), downloaded Ignite from binaries and worked with it
>
> --
> Denis
>
> On Mon, Mar 5, 2018 at 11:42 AM, Vladimir Ozerov 
> wrote:
>
> >  Igniters,
> >
> > We have uploaded a 2.4.0 release candidate to
> > https://dist.apache.org/repos/dist/dev/ignite/2.4.0-rc1/
> >
> > Git tag name is
> > 2.4.0-rc1
> >
> > This release includes the following changes:
> >
> > Ignite:
> > * Introduced Baseline Affinity Topology
> > * Ability to disable WAL for cache in runtime through IgniteCluster API
> or
> > ALTER TABLE command
> > * Added ability to convert WAL to human-readable form
> > * Ability to enable/disable JDBC, ODBC and thin client endpoints
> > * Project moved to Java 8
> > * Java 7 is no longer supported
> > * Apache Ignite can be run on Java 9
> > * Introduced Apache Ignite packaging with RPM
> > * Spark Data Frames support
> > * Added integration with Spring 3.x
> > * Added handler to stop node when IO error is encountered
> > * EvictionPolicyFactory support added to CacheConfiguration
> > * Added JMX metrics for memory regions
> > * Added detailed memory consumption on start and OOM reporting
> > * Added custom thread pools monitoring
> > * Support Ignite MR in Kerberized environment without IGFS
> > * S3 IP finder: support server side encryption and bucket endpoint
> > configuration
> > * Multiple fixes and performance optimizations
> >
> > Ignite.NET:
> > * Thin Client initial release
> > * .NET Core / Linux / macOS support
> > * Mono platform support
> > * Removed C++ JNI layer
> >
> > SQL:
> > * Added DEFAULT support to CREATE TABLE command
> > * Added ALTER TABLE DROP COLUMN support
> > * Added INLINE_SIZE option to CREATE INDEX command
> > * Added PARALLEL option to CREATE INDEX command
> > * Added optional on-heap row cache
> > * INSERT command now can be executed without explicit column names
> (INSERT
> > INTO table VALUES ...)
> > * Allowed multiple caches to share the same schema
> > * Added support for Java 8 Date and Time API
> > * Added "notNull" property to QuerySqlField annotation
> > * Improved COUNT(*) performance
> > * Fixed a problem causing an exception in certain nodes do not have
> primary
> > partitions for REPLICATED cache
> > * Fixed per-thread H2 connection leak.
> > * Fixed partition exchange hang due to SQL query running inside a
> > transaction.
> >
> > JDBC Driver:
> > * Optimized batched operations processing
> >
> > ODBC Driver:
> > * Added support of multiple statements execution with one call
> > * Added support of SQL_ATTR_CONNECTION_TIMEOUT
> > * Added support of SQL_ATTR_QUERY_TIMEOUT
> > * Optimized batched operations processing
> > * Proper handling of ungraceful TCP disconnects (keep-alive connection)
> > * Fixed error preventing close of executed DML statement
> > * Fixed table names returned by SQLTables for DDL-created tables
> >
> > Machine Learning:
> > * Implemented Fuzzy c-means algorithm
> > * Implemented gradient descent for OLS Linear Regression
> > * Implemented K nearest neighbor algorithm
> > * Introduced API for datasets
> > * Introduced API for ML model and trainers
> > * Introduced common mechanism for group training of models
> > * Added distributed version of multilayer perceptron
> > * Added local version of multilayer perceptron
> > * Added basic import/export functionality for ml models
> > * Added decision tree algorithm
> > * Performance optimizations for distributed operations
> >
> > Web Console:
> > * Implemented component for cluster selection and activation
> > * Implemented support for multiple statements on Queries screen
> > * Implemented progress indication for 'execute', 'scan' and 'export'
> > actions on Queries screen
> > * Added support for ClientConnectorConfiguration
> > * Added several SQL examples on Queries screen
> > * Added "Connected clusters" component in header
> > * Added support for custom SMTP server configuration
> > * Added detection of CSV separator based on browser locale
> > * Added "Copy to clipboard" action on Queries screen
> > * Added duration and node ID in results header and 'Show query' modal
> > * Improved support for binary JDBC types on import from RDBMS
> > * Fixed Web Console under IE11
> > * Fixed configuration generation for imported model in case of Oracle
> > NUMBER(5) data type
> > * Fixed output of big numbers in SQL query results
> >
> > Visor:
> > * Added "-quiet" flag for batch mode
> > * Added ability to start cache rebalance
> > * Added output of data region metrics to "node" command
> > * Added dialog to put/get/remove values

Re: checksum file Release Distribution Policy

I don't see MD5 files anywhere on the website:
https://ignite.apache.org/download.cgi

Where do we have them?

D.


On Tue, Mar 6, 2018 at 4:50 AM, Vladimir Ozerov 
wrote:

> I am not sure. We do have SHA512 files, which is ok. But we also have MD5
> files, which are now "SHOULD NOT" be provided. Is it OK that we still have
> them?
>
> On Tue, Mar 6, 2018 at 10:23 AM, Dmitriy Setrakyan 
> wrote:
>
> > On Mon, Mar 5, 2018 at 12:57 PM, Denis Magda  wrote:
> >
> > > Igniters,
> > >
> > > Do we comply with the next release requirements? Vladimir as a 2.4
> > release
> > > manager, could you double check that we are in a good state?
> > >
> >
> > I think we do. We only provide SHA files for our releases.
> >
>

Re: MAC addresses emulation in Testing Framework

Hi Vyacheslav,

Why do you think reflection is bad in that case? Tests often use
reflection, moreover U.field() to access some private stuff and test it.

Sincerely,
Dmitriy Pavlov

вт, 6 мар. 2018 г. в 21:11, Vyacheslav Daradur :

> Hi, Igniters!
>
> I need your help about the following question:
>
> How to set-up (emulate) custom MAC address at node startup in Testing
> Framework?
>
> I need to write unit-test that checks requests distribution which
> depends on ClusterNodes MAC addresses. I tried to change it after
> instance startup via reflection but looks like a bad idea and may lead
> to unexpected behavior.
>
>
> --
> Best Regards, Vyacheslav D.
>

[jira] [Created] (IGNITE-7893) Release Java Thin client documentation

2018-03-06 Thread Denis Magda (JIRA)

Denis Magda created IGNITE-7893:
---

 Summary: Release Java Thin client documentation
 Key: IGNITE-7893
 URL: https://issues.apache.org/jira/browse/IGNITE-7893
 Project: Ignite
  Issue Type: New Feature
  Components: documentation
Reporter: Denis Magda
 Fix For: 2.5


Java thin client is already available for usage and documented:
https://apacheignite.readme.io/v2.3/docs/java-thin-client

It will be officially released in 2.5. For now, the users have to build it from 
sources.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Thin client: transactions support

2018-03-06 Thread Denis Magda

Here is an original discussion with a reference to the JIRA ticket:
http://apache-ignite-developers.2346864.n4.nabble.com/Re-Transaction-operations-using-the-Ignite-Thin-Client-Protocol-td25914.html

--
Denis

On Tue, Mar 6, 2018 at 9:18 AM, Dmitriy Setrakyan 
wrote:

> Hi Dmitriy. I don't think we have a design proposal for transaction support
> in thin clients. Do you mind taking this initiative and creating an IEP on
> Wiki?
>
> D.
>
> On Tue, Mar 6, 2018 at 8:46 AM, Dmitriy Govorukhin <
> dmitriy.govoruk...@gmail.com> wrote:
>
> > Hi, Igniters.
> >
> > I've seen a lot of discussions about thin client and binary protocol,
> but I
> > did not hear anything about transactions support. Do we have some draft
> for
> > this purpose?
> >
> > As I understand we have several problems:
> >
> >- thread and transaction have hard related (we use thread-local
> variable
> >and thread name)
> >- we can process only one transaction at the same time in one thread
> (it
> >mean we need hold thread per client. If connect 100 thin clients to 1
> >server node, then need to hold 100 thread on the server side)
> >
> > Let's discuss how we can implement transactions for the thin client.
> >
>

Re: [VOTE] Apache Ignite 2.4.0 RC1

2018-03-06 Thread Denis Magda

+1 (binding), downloaded Ignite from binaries and worked with it

--
Denis

On Mon, Mar 5, 2018 at 11:42 AM, Vladimir Ozerov 
wrote:

>  Igniters,
>
> We have uploaded a 2.4.0 release candidate to
> https://dist.apache.org/repos/dist/dev/ignite/2.4.0-rc1/
>
> Git tag name is
> 2.4.0-rc1
>
> This release includes the following changes:
>
> Ignite:
> * Introduced Baseline Affinity Topology
> * Ability to disable WAL for cache in runtime through IgniteCluster API or
> ALTER TABLE command
> * Added ability to convert WAL to human-readable form
> * Ability to enable/disable JDBC, ODBC and thin client endpoints
> * Project moved to Java 8
> * Java 7 is no longer supported
> * Apache Ignite can be run on Java 9
> * Introduced Apache Ignite packaging with RPM
> * Spark Data Frames support
> * Added integration with Spring 3.x
> * Added handler to stop node when IO error is encountered
> * EvictionPolicyFactory support added to CacheConfiguration
> * Added JMX metrics for memory regions
> * Added detailed memory consumption on start and OOM reporting
> * Added custom thread pools monitoring
> * Support Ignite MR in Kerberized environment without IGFS
> * S3 IP finder: support server side encryption and bucket endpoint
> configuration
> * Multiple fixes and performance optimizations
>
> Ignite.NET:
> * Thin Client initial release
> * .NET Core / Linux / macOS support
> * Mono platform support
> * Removed C++ JNI layer
>
> SQL:
> * Added DEFAULT support to CREATE TABLE command
> * Added ALTER TABLE DROP COLUMN support
> * Added INLINE_SIZE option to CREATE INDEX command
> * Added PARALLEL option to CREATE INDEX command
> * Added optional on-heap row cache
> * INSERT command now can be executed without explicit column names (INSERT
> INTO table VALUES ...)
> * Allowed multiple caches to share the same schema
> * Added support for Java 8 Date and Time API
> * Added "notNull" property to QuerySqlField annotation
> * Improved COUNT(*) performance
> * Fixed a problem causing an exception in certain nodes do not have primary
> partitions for REPLICATED cache
> * Fixed per-thread H2 connection leak.
> * Fixed partition exchange hang due to SQL query running inside a
> transaction.
>
> JDBC Driver:
> * Optimized batched operations processing
>
> ODBC Driver:
> * Added support of multiple statements execution with one call
> * Added support of SQL_ATTR_CONNECTION_TIMEOUT
> * Added support of SQL_ATTR_QUERY_TIMEOUT
> * Optimized batched operations processing
> * Proper handling of ungraceful TCP disconnects (keep-alive connection)
> * Fixed error preventing close of executed DML statement
> * Fixed table names returned by SQLTables for DDL-created tables
>
> Machine Learning:
> * Implemented Fuzzy c-means algorithm
> * Implemented gradient descent for OLS Linear Regression
> * Implemented K nearest neighbor algorithm
> * Introduced API for datasets
> * Introduced API for ML model and trainers
> * Introduced common mechanism for group training of models
> * Added distributed version of multilayer perceptron
> * Added local version of multilayer perceptron
> * Added basic import/export functionality for ml models
> * Added decision tree algorithm
> * Performance optimizations for distributed operations
>
> Web Console:
> * Implemented component for cluster selection and activation
> * Implemented support for multiple statements on Queries screen
> * Implemented progress indication for 'execute', 'scan' and 'export'
> actions on Queries screen
> * Added support for ClientConnectorConfiguration
> * Added several SQL examples on Queries screen
> * Added "Connected clusters" component in header
> * Added support for custom SMTP server configuration
> * Added detection of CSV separator based on browser locale
> * Added "Copy to clipboard" action on Queries screen
> * Added duration and node ID in results header and 'Show query' modal
> * Improved support for binary JDBC types on import from RDBMS
> * Fixed Web Console under IE11
> * Fixed configuration generation for imported model in case of Oracle
> NUMBER(5) data type
> * Fixed output of big numbers in SQL query results
>
> Visor:
> * Added "-quiet" flag for batch mode
> * Added ability to start cache rebalance
> * Added output of data region metrics to "node" command
> * Added dialog to put/get/remove values to/from cache for simple key types
> * Show valid message for caches when cluster is inactive
> * Fixed 'cache -a' in case of node filter
> * Fixed reading last command line in batch mode
> * Updated eviction policy factory in configs
>
> Complete list of closed issues:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%
> 20fixVersion%20%3D%202.4%20AND%20(status%20%3D%
> 20closed%20or%20status%20%3D%20resolved)
>
> DEVNOTES
> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_
> plain;f=DEVNOTES.txt;hb=refs/tags/2.4.0-rc1
>
> RELEASE NOTES
> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_
>

[GitHub] ignite pull request #3609: IGNITE-6113 Return back old implementation instea...

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3609


---

MAC addresses emulation in Testing Framework

2018-03-06 Thread Vyacheslav Daradur

Hi, Igniters!

I need your help about the following question:

How to set-up (emulate) custom MAC address at node startup in Testing Framework?

I need to write unit-test that checks requests distribution which
depends on ClusterNodes MAC addresses. I tried to change it after
instance startup via reflection but looks like a bad idea and may lead
to unexpected behavior.


-- 
Best Regards, Vyacheslav D.

[GitHub] ignite pull request #3608: IGNITE-7889: .NET: linq GroupBy and Where does no...

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3608


---

Re: Stop nodes after test by default - IGNITE-6842

Ok, thank you.

Please let me know when we can proceed with review
https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502


вт, 6 мар. 2018 г. в 20:17, Maxim Muzafarov :

> Hello Dmitry,
>
> Yes, I've updated test classes as you metioned before.
> Now i'm fixing review comments. Within next few days I'll prepare final
> version of this PR.
>
> вт, 6 мар. 2018 г. в 20:12, Dmitry Pavlov :
>
> > Hi Maxim,
> >
> > are there any news on these test fails?
> >
> > Is issue ready for review?
> >
> > Sincerely,
> > Dmitiry Pavlov
> >
> > вт, 27 февр. 2018 г. в 17:12, Dmitry Pavlov :
> >
> > > Hi, thank you!
> > >
> > > I've found several suspicious fails: such test fails have rate less
> than
> > > 1%, it is probably new failures.
> > >
> > > It would be great if we can fix it before merge. Could you address this
> > > fails?
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > IgniteCacheTestSuite5: IgniteCacheStoreCollectionTest.testStoreMap
> (fail
> > > rate 0,0%)
> > > IgniteCacheTestSuite5:
> > > CacheLateAffinityAssignmentTest.testDelayAssignmentClientJoin (fail
> rate
> > > 0,0%)
> > > IgniteCacheWithIndexingTestSuite:
> > > CacheRandomOperationsMultithreadedTest.testAtomicOffheapEviction (fail
> > rate
> > > 0,0%)
> > > IgniteCacheWithIndexingTestSuite:
> > >
> CacheRandomOperationsMultithreadedTest.testAtomicOffheapEvictionIndexing
> > > (fail rate 0,0%)
> > > IgniteCacheWithIndexingTestSuite:
> > > CacheRandomOperationsMultithreadedTest.testTxOffheapEviction (fail rate
> > > 0,0%)
> > > IgniteCacheWithIndexingTestSuite:
> > > CacheRandomOperationsMultithreadedTest.testTxOffheapEvictionIndexing
> > (fail
> > > rate 0,0%)
> > >
> > > IgniteBinarySimpleNameMapperCacheFullApiTestSuite:
> > >
> >
> GridCachePartitionedNearDisabledMultiNodeWithGroupFullApiSelfTest.testWithSkipStoreTx
> > > (fail rate 0,0%)
> > >
> > > вт, 27 февр. 2018 г. в 17:04, Maxim Muzafarov :
> > >
> > >> Yep, link may not be correct.
> > >>
> > >> Here is correct version:
> > >> TC: *
> > >>
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8_IgniteTests24Java8=pull%2F3542%2Fhead
> > >> <
> > >>
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8_IgniteTests24Java8=pull%2F3542%2Fhead
> > >> >*
> > >>
> > >>
> > >> вт, 27 февр. 2018 г. в 16:41, Dmitry Pavlov :
> > >>
> > >> > Hi Maxim,
> > >> >
> > >> > could you please provide link to TC run on your PR? It seems link
> > >> provided
> > >> > points to run of master. In changes field you may select
> > pull/3542/head
> > >> > before starting RunAll.
> > >> >
> > >> > Igniters,
> > >> >
> > >> > this change is related to our test framework, so change may affect
> > your
> > >> > tests. Please join to review
> > >> > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
> > >> >
> > >> > Sincerely,
> > >> > Dmitriy Pavlov
> > >> >
> > >> > вт, 27 февр. 2018 г. в 16:14, Maxim Muzafarov :
> > >> >
> > >> > > Hi all,
> > >> > >
> > >> > > I think, I've done with this issue, what should we do next?
> > >> > >
> > >> > > PR: https://github.com/apache/ignite/pull/3542
> > >> > > Upsource:
> > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
> > >> > > TC:
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/viewModification.html?modId=723895=false==vcsModificationTests
> > >> > > JIRA: https://issues.apache.org/jira/browse/IGNITE-6842
> > >> > >
> > >> > >
> > >> > > чт, 22 февр. 2018 г. в 14:12, Dmitry Pavlov <
> dpavlov@gmail.com
> > >:
> > >> > >
> > >> > > > Hi Maxim,
> > >> > > >
> > >> > > > Thank you.
> > >> > > >
> > >> > > > To my mind stopAllGrids call should be kept in afterTestsStop().
> > If
> > >> > > > developer forgot to call super(), he will see exception from
> > >> > > > beforeTestsStart()
> > >> > > > added by you.
> > >> > > >
> > >> > > > If you think patch is ready to be reviewed, please change JIRA
> > >> status
> > >> > to
> > >> > > > "Patch Available".
> > >> > > >
> > >> > > > Optionally you can create Upsource review. It would be easier
> for
> > >> > > multiple
> > >> > > > reviewers to handle this patch. This test framework is used by
> all
> > >> > > Igniters
> > >> > > > so Upsource would be good option (
> > >> https://reviews.ignite.apache.org/
> > >> > ).
> > >> > > >
> > >> > > > Sincerely,
> > >> > > > Dmitriy Pavlov
> > >> > > >
> > >> > > > чт, 22 февр. 2018 г. в 13:44, Maxim Muzafarov <
> maxmu...@gmail.com
> > >:
> > >> > > >
> > >> > > > > Hi all,
> > >> > > > >
> > >> > > > > I've made some changes based on our previous discusstions,
> > please
> > >> see
> > >> > > PR
> > >> > > > > [1]:
> > >> > > > > 1) Remove duplicated code for stopGrid (by index and by name);
> > >> > > > > 2) Add new method that thows exception if not all grids
> haven't
> > >> > stopped
> > >> > > > > correctly;
> > >> > > > > 3)  Change tests that have

[GitHub] ignite pull request #3611: Ignite 7751

2018-03-06 Thread glukos

GitHub user glukos opened a pull request:

https://github.com/apache/ignite/pull/3611

Ignite 7751



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7751

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3611.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3611


commit ad690075aa3c52ef9a9f3d8269c5f4b556e00f07
Author: Ivan Rakov 
Date:   2018-03-05T16:03:10Z

IGNITE-7751 Pages Write Throttle mode doesn't protect from checkpoint 
buffer overflow

commit 5a804fcb59157c8d7884c02aa565f01649df9eb4
Author: Ivan Rakov 
Date:   2018-03-05T17:35:13Z

IGNITE-7751 Pages Write Throttle mode doesn't protect from checkpoint 
buffer overflow

commit e336837043b90f515b84fcd90ba0f40ac96f241c
Author: Ivan Rakov 
Date:   2018-03-05T18:04:21Z

IGNITE-7751 Pages Write Throttle mode doesn't protect from checkpoint 
buffer overflow

commit 97ecc63cdabe3349cee69ecc0c9f3bd1cabc0c12
Author: Ivan Rakov 
Date:   2018-03-06T17:29:28Z

IGNITE-7751 Pages Write Throttle mode doesn't protect from checkpoint 
buffer overflow

commit 8cb5a63768876bc9ecaca515efbb9884424ae4c7
Author: Ivan Rakov 
Date:   2018-03-06T17:34:30Z

IGNITE-7751 Pages Write Throttle mode doesn't protect from checkpoint 
buffer overflow

commit 913a69c651a4f65517ae0a037115be10142e051e
Author: Ivan Rakov 
Date:   2018-03-06T17:39:02Z

Merge branch 'master' into ignite-7751




---

Re: Thin client: transactions support

Hi Dmitriy. I don't think we have a design proposal for transaction support
in thin clients. Do you mind taking this initiative and creating an IEP on
Wiki?

D.

On Tue, Mar 6, 2018 at 8:46 AM, Dmitriy Govorukhin <
dmitriy.govoruk...@gmail.com> wrote:

> Hi, Igniters.
>
> I've seen a lot of discussions about thin client and binary protocol, but I
> did not hear anything about transactions support. Do we have some draft for
> this purpose?
>
> As I understand we have several problems:
>
>- thread and transaction have hard related (we use thread-local variable
>and thread name)
>- we can process only one transaction at the same time in one thread (it
>mean we need hold thread per client. If connect 100 thin clients to 1
>server node, then need to hold 100 thread on the server side)
>
> Let's discuss how we can implement transactions for the thin client.
>

Re: Stop nodes after test by default - IGNITE-6842

2018-03-06 Thread Maxim Muzafarov

Hello Dmitry,

Yes, I've updated test classes as you metioned before.
Now i'm fixing review comments. Within next few days I'll prepare final
version of this PR.

вт, 6 мар. 2018 г. в 20:12, Dmitry Pavlov :

> Hi Maxim,
>
> are there any news on these test fails?
>
> Is issue ready for review?
>
> Sincerely,
> Dmitiry Pavlov
>
> вт, 27 февр. 2018 г. в 17:12, Dmitry Pavlov :
>
> > Hi, thank you!
> >
> > I've found several suspicious fails: such test fails have rate less than
> > 1%, it is probably new failures.
> >
> > It would be great if we can fix it before merge. Could you address this
> > fails?
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > IgniteCacheTestSuite5: IgniteCacheStoreCollectionTest.testStoreMap (fail
> > rate 0,0%)
> > IgniteCacheTestSuite5:
> > CacheLateAffinityAssignmentTest.testDelayAssignmentClientJoin (fail rate
> > 0,0%)
> > IgniteCacheWithIndexingTestSuite:
> > CacheRandomOperationsMultithreadedTest.testAtomicOffheapEviction (fail
> rate
> > 0,0%)
> > IgniteCacheWithIndexingTestSuite:
> > CacheRandomOperationsMultithreadedTest.testAtomicOffheapEvictionIndexing
> > (fail rate 0,0%)
> > IgniteCacheWithIndexingTestSuite:
> > CacheRandomOperationsMultithreadedTest.testTxOffheapEviction (fail rate
> > 0,0%)
> > IgniteCacheWithIndexingTestSuite:
> > CacheRandomOperationsMultithreadedTest.testTxOffheapEvictionIndexing
> (fail
> > rate 0,0%)
> >
> > IgniteBinarySimpleNameMapperCacheFullApiTestSuite:
> >
> GridCachePartitionedNearDisabledMultiNodeWithGroupFullApiSelfTest.testWithSkipStoreTx
> > (fail rate 0,0%)
> >
> > вт, 27 февр. 2018 г. в 17:04, Maxim Muzafarov :
> >
> >> Yep, link may not be correct.
> >>
> >> Here is correct version:
> >> TC: *
> >>
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8_IgniteTests24Java8=pull%2F3542%2Fhead
> >> <
> >>
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8_IgniteTests24Java8=pull%2F3542%2Fhead
> >> >*
> >>
> >>
> >> вт, 27 февр. 2018 г. в 16:41, Dmitry Pavlov :
> >>
> >> > Hi Maxim,
> >> >
> >> > could you please provide link to TC run on your PR? It seems link
> >> provided
> >> > points to run of master. In changes field you may select
> pull/3542/head
> >> > before starting RunAll.
> >> >
> >> > Igniters,
> >> >
> >> > this change is related to our test framework, so change may affect
> your
> >> > tests. Please join to review
> >> > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
> >> >
> >> > Sincerely,
> >> > Dmitriy Pavlov
> >> >
> >> > вт, 27 февр. 2018 г. в 16:14, Maxim Muzafarov :
> >> >
> >> > > Hi all,
> >> > >
> >> > > I think, I've done with this issue, what should we do next?
> >> > >
> >> > > PR: https://github.com/apache/ignite/pull/3542
> >> > > Upsource:
> https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
> >> > > TC:
> >> > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewModification.html?modId=723895=false==vcsModificationTests
> >> > > JIRA: https://issues.apache.org/jira/browse/IGNITE-6842
> >> > >
> >> > >
> >> > > чт, 22 февр. 2018 г. в 14:12, Dmitry Pavlov  >:
> >> > >
> >> > > > Hi Maxim,
> >> > > >
> >> > > > Thank you.
> >> > > >
> >> > > > To my mind stopAllGrids call should be kept in afterTestsStop().
> If
> >> > > > developer forgot to call super(), he will see exception from
> >> > > > beforeTestsStart()
> >> > > > added by you.
> >> > > >
> >> > > > If you think patch is ready to be reviewed, please change JIRA
> >> status
> >> > to
> >> > > > "Patch Available".
> >> > > >
> >> > > > Optionally you can create Upsource review. It would be easier for
> >> > > multiple
> >> > > > reviewers to handle this patch. This test framework is used by all
> >> > > Igniters
> >> > > > so Upsource would be good option (
> >> https://reviews.ignite.apache.org/
> >> > ).
> >> > > >
> >> > > > Sincerely,
> >> > > > Dmitriy Pavlov
> >> > > >
> >> > > > чт, 22 февр. 2018 г. в 13:44, Maxim Muzafarov  >:
> >> > > >
> >> > > > > Hi all,
> >> > > > >
> >> > > > > I've made some changes based on our previous discusstions,
> please
> >> see
> >> > > PR
> >> > > > > [1]:
> >> > > > > 1) Remove duplicated code for stopGrid (by index and by name);
> >> > > > > 2) Add new method that thows exception if not all grids haven't
> >> > stopped
> >> > > > > correctly;
> >> > > > > 3)  Change tests that have been affected by this changes;
> >> > > > >
> >> > > > > Also, I have some thoughts for clarification:
> >> > > > > 1) beforeTestsStart() - I expect here in subtests that grids are
> >> not
> >> > > > > started yet. Am I right?
> >> > > > > 2) I think we should call stopAllGrids in tearDown method. So,
> if
> >> in
> >> > > some
> >> > > > > cases we'll override afterTestsStop in subclasses and forgot to
> >> call
> >> > > > > super() it won't lead us to exception.
> >> > > > >
> >> > > > > [1]

Re: Stop nodes after test by default - IGNITE-6842

Hi Maxim,

are there any news on these test fails?

Is issue ready for review?

Sincerely,
Dmitiry Pavlov

вт, 27 февр. 2018 г. в 17:12, Dmitry Pavlov :

> Hi, thank you!
>
> I've found several suspicious fails: such test fails have rate less than
> 1%, it is probably new failures.
>
> It would be great if we can fix it before merge. Could you address this
> fails?
>
> Sincerely,
> Dmitriy Pavlov
>
> IgniteCacheTestSuite5: IgniteCacheStoreCollectionTest.testStoreMap (fail
> rate 0,0%)
> IgniteCacheTestSuite5:
> CacheLateAffinityAssignmentTest.testDelayAssignmentClientJoin (fail rate
> 0,0%)
> IgniteCacheWithIndexingTestSuite:
> CacheRandomOperationsMultithreadedTest.testAtomicOffheapEviction (fail rate
> 0,0%)
> IgniteCacheWithIndexingTestSuite:
> CacheRandomOperationsMultithreadedTest.testAtomicOffheapEvictionIndexing
> (fail rate 0,0%)
> IgniteCacheWithIndexingTestSuite:
> CacheRandomOperationsMultithreadedTest.testTxOffheapEviction (fail rate
> 0,0%)
> IgniteCacheWithIndexingTestSuite:
> CacheRandomOperationsMultithreadedTest.testTxOffheapEvictionIndexing (fail
> rate 0,0%)
>
> IgniteBinarySimpleNameMapperCacheFullApiTestSuite:
> GridCachePartitionedNearDisabledMultiNodeWithGroupFullApiSelfTest.testWithSkipStoreTx
> (fail rate 0,0%)
>
> вт, 27 февр. 2018 г. в 17:04, Maxim Muzafarov :
>
>> Yep, link may not be correct.
>>
>> Here is correct version:
>> TC: *
>> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8_IgniteTests24Java8=pull%2F3542%2Fhead
>> <
>> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8_IgniteTests24Java8=pull%2F3542%2Fhead
>> >*
>>
>>
>> вт, 27 февр. 2018 г. в 16:41, Dmitry Pavlov :
>>
>> > Hi Maxim,
>> >
>> > could you please provide link to TC run on your PR? It seems link
>> provided
>> > points to run of master. In changes field you may select pull/3542/head
>> > before starting RunAll.
>> >
>> > Igniters,
>> >
>> > this change is related to our test framework, so change may affect your
>> > tests. Please join to review
>> > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
>> >
>> > Sincerely,
>> > Dmitriy Pavlov
>> >
>> > вт, 27 февр. 2018 г. в 16:14, Maxim Muzafarov :
>> >
>> > > Hi all,
>> > >
>> > > I think, I've done with this issue, what should we do next?
>> > >
>> > > PR: https://github.com/apache/ignite/pull/3542
>> > > Upsource: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
>> > > TC:
>> > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewModification.html?modId=723895=false==vcsModificationTests
>> > > JIRA: https://issues.apache.org/jira/browse/IGNITE-6842
>> > >
>> > >
>> > > чт, 22 февр. 2018 г. в 14:12, Dmitry Pavlov :
>> > >
>> > > > Hi Maxim,
>> > > >
>> > > > Thank you.
>> > > >
>> > > > To my mind stopAllGrids call should be kept in afterTestsStop(). If
>> > > > developer forgot to call super(), he will see exception from
>> > > > beforeTestsStart()
>> > > > added by you.
>> > > >
>> > > > If you think patch is ready to be reviewed, please change JIRA
>> status
>> > to
>> > > > "Patch Available".
>> > > >
>> > > > Optionally you can create Upsource review. It would be easier for
>> > > multiple
>> > > > reviewers to handle this patch. This test framework is used by all
>> > > Igniters
>> > > > so Upsource would be good option (
>> https://reviews.ignite.apache.org/
>> > ).
>> > > >
>> > > > Sincerely,
>> > > > Dmitriy Pavlov
>> > > >
>> > > > чт, 22 февр. 2018 г. в 13:44, Maxim Muzafarov :
>> > > >
>> > > > > Hi all,
>> > > > >
>> > > > > I've made some changes based on our previous discusstions, please
>> see
>> > > PR
>> > > > > [1]:
>> > > > > 1) Remove duplicated code for stopGrid (by index and by name);
>> > > > > 2) Add new method that thows exception if not all grids haven't
>> > stopped
>> > > > > correctly;
>> > > > > 3)  Change tests that have been affected by this changes;
>> > > > >
>> > > > > Also, I have some thoughts for clarification:
>> > > > > 1) beforeTestsStart() - I expect here in subtests that grids are
>> not
>> > > > > started yet. Am I right?
>> > > > > 2) I think we should call stopAllGrids in tearDown method. So, if
>> in
>> > > some
>> > > > > cases we'll override afterTestsStop in subclasses and forgot to
>> call
>> > > > > super() it won't lead us to exception.
>> > > > >
>> > > > > [1] https://github.com/apache/ignite/pull/3542
>> > > > > [2]
>> https://ci.ignite.apache.org/viewModification.html?modId=717275
>> > > > > [3] https://issues.apache.org/jira/browse/IGNITE-6842
>> > > > >
>> > > > >
>> > > > > ср, 7 февр. 2018 г. в 18:28, Maxim Muzafarov > >:
>> > > > >
>> > > > > > Thank you all,
>> > > > > >
>> > > > > > I'll add this comment's for JIRA ticket, if you don't mind.
>> > > > > >
>> > > > > > ср, 7 февр. 2018 г. в 15:16, Dmitry Pavlov <
>> dpavlov@gmail.com
>> > >:
>> > > > > >
>> > > > > >> Yes, this

[GitHub] ignite pull request #3610: IGNITE-7864 Control utility: Add confirm on dange...

2018-03-06 Thread macrergate

GitHub user macrergate opened a pull request:

https://github.com/apache/ignite/pull/3610

IGNITE-7864 Control utility: Add confirm on dangerous operations



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7864

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3610.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3610


commit dd08ed09c4f770009b7764727b3ffe880f250d61
Author: macrergate 
Date:   2018-03-06T16:45:33Z

IGNITE-7864 Control utility: Add confirm on dangerous operations

commit a3528cd156cd4a1753b96e31727cf3c0ac8fdca7
Author: macrergate 
Date:   2018-03-06T16:48:02Z

Merge branch 'master' of https://github.com/apache/ignite into ignite-7864




---

Thin client: transactions support

2018-03-06 Thread Dmitriy Govorukhin

Hi, Igniters.

I've seen a lot of discussions about thin client and binary protocol, but I
did not hear anything about transactions support. Do we have some draft for
this purpose?

As I understand we have several problems:

   - thread and transaction have hard related (we use thread-local variable
   and thread name)
   - we can process only one transaction at the same time in one thread (it
   mean we need hold thread per client. If connect 100 thin clients to 1
   server node, then need to hold 100 thread on the server side)

Let's discuss how we can implement transactions for the thin client.

[GitHub] ignite pull request #3609: IGNITE-6113 Return back old implementation instea...

2018-03-06 Thread Jokser

GitHub user Jokser opened a pull request:

https://github.com/apache/ignite/pull/3609

IGNITE-6113 Return back old implementation instead of 'removeIf'



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6113-micro-fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3609.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3609


commit 7aba97e0ecaa631c3951a45c43af91a08a89097a
Author: Pavel Kovalenko 
Date:   2018-03-06T16:06:04Z

IGNITE-6113 Return back old implementation instead of 'removeIf'




---

Re: MTCGA: Tests of the week

2018-03-06 Thread Yakov Zhdanov

Great progress, guys! Keep going!

--Yakov

Re: Ignite contibutors page

2018-03-06 Thread Andrey Kuznetsov

+1

-- 
Best regards,
  Andrey Kuznetsov.

RE: Ignite contibutors page

2018-03-06 Thread tank2.alex

Hi, I am not mentioned too

-Original Message-
From: Petr Ivanov [mailto:mr.wei...@gmail.com] 
Sent: Tuesday, March 6, 2018 4:19 PM
To: dev@ignite.apache.org
Subject: Re: Ignite contibutors page

+1 not mentioned


> On 6 Mar 2018, at 15:36, Дмитрий Рябов  wrote:
> 
> Hi Igniters!
> 
> I'm not mentioned too. Dmitriy Ryabov.
> 
> 2018-03-05 23:39 GMT+03:00 Denis Magda :
> 
>> Hi Dmitriy,
>> 
>> Thanks for kicking off the discussion. I'll wait for a couple of days 
>> for other response and update the page in bulk.
>> 
>> --
>> Denis
>> 
>> On Mon, Mar 5, 2018 at 12:32 PM, Dmitry Pavlov 
>> 
>> wrote:
>> 
>>> Hi Igniters,
>>> 
>>> I noticed that I'm not mentioned on the community contributors page:
>>> https://ignite.apache.org/community/resources.html
>>> 
>>> Perhaps there is someone else, who is not yet added. Or information 
>>> is
>> not
>>> up-to-date.
>>> 
>>> Please check the data for yourself and respond if you need to be 
>>> added /
>> or
>>> information needs to be updated.
>>> 
>>> Sincerely,
>>> Dmitry Pavlov
>>> 
>>

[jira] [Created] (IGNITE-7892) Remove aquirence of any locks from toString methods

2018-03-06 Thread Alexander Belyak (JIRA)

Alexander Belyak created IGNITE-7892:


 Summary: Remove aquirence of any locks from toString methods
 Key: IGNITE-7892
 URL: https://issues.apache.org/jira/browse/IGNITE-7892
 Project: Ignite
  Issue Type: Wish
  Components: general
Affects Versions: 2.4
Reporter: Alexander Belyak


In org.apache.ignite.internal.processors.cache.GridCacheMapEntry we have thread 
safe toString() method that can lead to some hangs of monitoring threads like 
grid-timeout-worker if we try to dump LongRunningOperations with locked entry.
I think that toString methods will never need to be a thread safe and can throw 
ConcurrentModificationException or print inconsistent data, so we must remove 
synchronization from every toString methods in codebase. If we need some 
"consistent" string representation - let's add consistentToString methods or do 
external synchronization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Partition loss policy - how to use?

2018-03-06 Thread gauravhb

Hi,

Is there any update on this topic?
Any tickets created for points mentioned by Valentin?

Thanks.



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [VOTE] Apache Ignite 2.4.0 RC1

2018-03-06 Thread Pavel Tupitsyn

Alexey, sorry, did not mean to underappreciate anyone's work! Thanks for
the explanation.

On Tue, Mar 6, 2018 at 11:31 AM, Alexey Kuznetsov 
wrote:

> Pavel,
>
> Web Console> Fixed output of big numbers in SQL query results
>
> As one who fix this issue - that one was not so *trivial* :).
>
>
> On Tue, Mar 6, 2018 at 3:05 PM, Vladimir Ozerov 
> wrote:
>
> > Pavel,
> >
> > There is no changes in Ignite C++. As far as Web Console fix, I would not
> > say it is trivial because it affected UX, so IMO it is important to
> notify
> > users about the fix.
> >
> > On Tue, Mar 6, 2018 at 10:30 AM, Pavel Tupitsyn 
> > wrote:
> >
> > > Regarding release notes:
> > >
> > > 1. No changes in Ignite.C++?
> > >
> > > 2. > Fixed output of big numbers in SQL query results
> > > Are we including trivial bugfixes now?
> > >
> > >
> > >
> > > On Tue, Mar 6, 2018 at 10:26 AM, Pavel Tupitsyn 
> > > wrote:
> > >
> > > > +1
> > > >
> > > > Checked Ignite.NET:
> > > > * Build from sources
> > > > * Download binaries and run examples
> > > > * Build NuGet packages on TeamCity [1] and test them
> > > >
> > > > [1] https://ci.ignite.apache.org/viewLog.html?buildId=1126956
> > > >
> > > > On Mon, Mar 5, 2018 at 10:42 PM, Vladimir Ozerov <
> voze...@gridgain.com
> > >
> > > > wrote:
> > > >
> > > >>  Igniters,
> > > >>
> > > >> We have uploaded a 2.4.0 release candidate to
> > > >> https://dist.apache.org/repos/dist/dev/ignite/2.4.0-rc1/
> > > >>
> > > >> Git tag name is
> > > >> 2.4.0-rc1
> > > >>
> > > >> This release includes the following changes:
> > > >>
> > > >> Ignite:
> > > >> * Introduced Baseline Affinity Topology
> > > >> * Ability to disable WAL for cache in runtime through IgniteCluster
> > API
> > > or
> > > >> ALTER TABLE command
> > > >> * Added ability to convert WAL to human-readable form
> > > >> * Ability to enable/disable JDBC, ODBC and thin client endpoints
> > > >> * Project moved to Java 8
> > > >> * Java 7 is no longer supported
> > > >> * Apache Ignite can be run on Java 9
> > > >> * Introduced Apache Ignite packaging with RPM
> > > >> * Spark Data Frames support
> > > >> * Added integration with Spring 3.x
> > > >> * Added handler to stop node when IO error is encountered
> > > >> * EvictionPolicyFactory support added to CacheConfiguration
> > > >> * Added JMX metrics for memory regions
> > > >> * Added detailed memory consumption on start and OOM reporting
> > > >> * Added custom thread pools monitoring
> > > >> * Support Ignite MR in Kerberized environment without IGFS
> > > >> * S3 IP finder: support server side encryption and bucket endpoint
> > > >> configuration
> > > >> * Multiple fixes and performance optimizations
> > > >>
> > > >> Ignite.NET:
> > > >> * Thin Client initial release
> > > >> * .NET Core / Linux / macOS support
> > > >> * Mono platform support
> > > >> * Removed C++ JNI layer
> > > >>
> > > >> SQL:
> > > >> * Added DEFAULT support to CREATE TABLE command
> > > >> * Added ALTER TABLE DROP COLUMN support
> > > >> * Added INLINE_SIZE option to CREATE INDEX command
> > > >> * Added PARALLEL option to CREATE INDEX command
> > > >> * Added optional on-heap row cache
> > > >> * INSERT command now can be executed without explicit column names
> > > (INSERT
> > > >> INTO table VALUES ...)
> > > >> * Allowed multiple caches to share the same schema
> > > >> * Added support for Java 8 Date and Time API
> > > >> * Added "notNull" property to QuerySqlField annotation
> > > >> * Improved COUNT(*) performance
> > > >> * Fixed a problem causing an exception in certain nodes do not have
> > > >> primary
> > > >> partitions for REPLICATED cache
> > > >> * Fixed per-thread H2 connection leak.
> > > >> * Fixed partition exchange hang due to SQL query running inside a
> > > >> transaction.
> > > >>
> > > >> JDBC Driver:
> > > >> * Optimized batched operations processing
> > > >>
> > > >> ODBC Driver:
> > > >> * Added support of multiple statements execution with one call
> > > >> * Added support of SQL_ATTR_CONNECTION_TIMEOUT
> > > >> * Added support of SQL_ATTR_QUERY_TIMEOUT
> > > >> * Optimized batched operations processing
> > > >> * Proper handling of ungraceful TCP disconnects (keep-alive
> > connection)
> > > >> * Fixed error preventing close of executed DML statement
> > > >> * Fixed table names returned by SQLTables for DDL-created tables
> > > >>
> > > >> Machine Learning:
> > > >> * Implemented Fuzzy c-means algorithm
> > > >> * Implemented gradient descent for OLS Linear Regression
> > > >> * Implemented K nearest neighbor algorithm
> > > >> * Introduced API for datasets
> > > >> * Introduced API for ML model and trainers
> > > >> * Introduced common mechanism for group training of models
> > > >> * Added distributed version of multilayer perceptron
> > > >> * Added local version of multilayer perceptron
> > > >> * Added basic import/export functionality for ml models
> > > >> * Added decision tree

[GitHub] ignite pull request #3606: IGNITE-7845 fix checking atomicLong in IgniteClie...

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3606


---

[GitHub] ignite pull request #3608: IGNITE-7889: .NET: linq GroupBy and Where does no...

2018-03-06 Thread apopovgg

GitHub user apopovgg opened a pull request:

https://github.com/apache/ignite/pull/3608

IGNITE-7889: .NET: linq GroupBy and Where does not work together



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7889

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3608.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3608


commit 6156e04a4c8af1bc599cbbfeb30c7726264ab6ea
Author: apopov 
Date:   2018-03-06T14:38:31Z

IGNITE-7889: .NET: linq GroupBy and Where does not work together




---

Re: Request for contributor permissions

Ivan,

Welcome to the community! I've added you to the contributors list.

--AG

2018-03-06 16:52 GMT+03:00 Ivan Daschinsky :

> Hello Ignite Community!
>
> My name is Ivan Daschinskiy. I want to contribute to Apache Ignite and want
> to start with this issue - IGNITE-6860
> , my JIRA username
> *ivandasch*. Any help on this will be appreciated.
>
> Thanks a lot!
> --
> Sincerely yours, Ivan Daschinskiy
>

[jira] [Created] (IGNITE-7891) awaitPartitionMapExchange works incorrectly if partitioned cache has backup

2018-03-06 Thread Alexey Kuznetsov (JIRA)

Alexey Kuznetsov created IGNITE-7891:


 Summary: awaitPartitionMapExchange works incorrectly if 
partitioned cache has backup
 Key: IGNITE-7891
 URL: https://issues.apache.org/jira/browse/IGNITE-7891
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Kuznetsov


When cluster is started with partitioned cache with backups, 
awaitPartitionMapExchange(true, true, null) may work incorrectly : it doesn't 
wait for full rebalance has finished.





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Request for contributor permissions

2018-03-06 Thread Ivan Daschinsky

Hello Ignite Community!

My name is Ivan Daschinskiy. I want to contribute to Apache Ignite and want
to start with this issue - IGNITE-6860
, my JIRA username
*ivandasch*. Any help on this will be appreciated.

Thanks a lot!
-- 
Sincerely yours, Ivan Daschinskiy

[jira] [Created] (IGNITE-7890) Node start with corrupted pds hangs indefinitely.

2018-03-06 Thread Ilya Lantukh (JIRA)

Ilya Lantukh created IGNITE-7890:


 Summary: Node start with corrupted pds hangs indefinitely.
 Key: IGNITE-7890
 URL: https://issues.apache.org/jira/browse/IGNITE-7890
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.3
Reporter: Ilya Lantukh
Assignee: Ilya Lantukh


Starting node with corrupted PDS + WAL leads to cluster-wide hang-up.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3445: IGNITE-6113 Async eviction

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3445


---

[jira] [Created] (IGNITE-7889) .NET: linq GroupBy and Where does not work together.

2018-03-06 Thread Alexey Popov (JIRA)

Alexey Popov created IGNITE-7889:


 Summary: .NET: linq GroupBy and Where does not work together.
 Key: IGNITE-7889
 URL: https://issues.apache.org/jira/browse/IGNITE-7889
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 2.1
Reporter: Alexey Popov
Assignee: Alexey Popov


The following code completely ignores the Where condition:
{noformat}
var groupByQuery2 = models.Where(m => m.Value.DateTicks < 10 && 
m.Value.Dirty.HasValue).GroupBy(m => m.Value.ContractId)
.Select(g => new KeyValuePair(g.Key, 
g.Select(m => m.Value.Version).Min()));
{noformat}

debugger shows SQL without WHERE:
{CacheQueryable [CacheName=someCache, TableName=MODEL, Query=SqlFieldsQuery 
[Sql=select _T0.CONTRACTID, min (_T0.VERSION)  from "someCache".MODEL as _T0 
group by (_T0.CONTRACTID), Arguments=[], Local=False, PageSize=1024, 
EnableDistributedJoins=False, EnforceJoinOrder=False, Timeout=00:00:00, 
ReplicatedOnly=False, Colocated=False, Schema=, Lazy=False]]}




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Ignite contibutors page

+1 not mentioned


> On 6 Mar 2018, at 15:36, Дмитрий Рябов  wrote:
> 
> Hi Igniters!
> 
> I'm not mentioned too. Dmitriy Ryabov.
> 
> 2018-03-05 23:39 GMT+03:00 Denis Magda :
> 
>> Hi Dmitriy,
>> 
>> Thanks for kicking off the discussion. I'll wait for a couple of days for
>> other response and update the page in bulk.
>> 
>> --
>> Denis
>> 
>> On Mon, Mar 5, 2018 at 12:32 PM, Dmitry Pavlov 
>> wrote:
>> 
>>> Hi Igniters,
>>> 
>>> I noticed that I'm not mentioned on the community contributors page:
>>> https://ignite.apache.org/community/resources.html
>>> 
>>> Perhaps there is someone else, who is not yet added. Or information is
>> not
>>> up-to-date.
>>> 
>>> Please check the data for yourself and respond if you need to be added /
>> or
>>> information needs to be updated.
>>> 
>>> Sincerely,
>>> Dmitry Pavlov
>>> 
>>

Re: checksum file Release Distribution Policy

2018-03-06 Thread Sergey Kozlov

We keep MD5 files for backward compatibility

On Tue, Mar 6, 2018 at 3:50 PM, Vladimir Ozerov 
wrote:

> I am not sure. We do have SHA512 files, which is ok. But we also have MD5
> files, which are now "SHOULD NOT" be provided. Is it OK that we still have
> them?
>
> On Tue, Mar 6, 2018 at 10:23 AM, Dmitriy Setrakyan 
> wrote:
>
> > On Mon, Mar 5, 2018 at 12:57 PM, Denis Magda  wrote:
> >
> > > Igniters,
> > >
> > > Do we comply with the next release requirements? Vladimir as a 2.4
> > release
> > > manager, could you double check that we are in a good state?
> > >
> >
> > I think we do. We only provide SHA files for our releases.
> >
>



-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com

Re: checksum file Release Distribution Policy

2018-03-06 Thread Vladimir Ozerov

I am not sure. We do have SHA512 files, which is ok. But we also have MD5
files, which are now "SHOULD NOT" be provided. Is it OK that we still have
them?

On Tue, Mar 6, 2018 at 10:23 AM, Dmitriy Setrakyan 
wrote:

> On Mon, Mar 5, 2018 at 12:57 PM, Denis Magda  wrote:
>
> > Igniters,
> >
> > Do we comply with the next release requirements? Vladimir as a 2.4
> release
> > manager, could you double check that we are in a good state?
> >
>
> I think we do. We only provide SHA files for our releases.
>

[jira] [Created] (IGNITE-7888) ODBC: Add support for SQL_ATTR_LOGIN_TIMEOUT

2018-03-06 Thread Alexey Popov (JIRA)

Alexey Popov created IGNITE-7888:


 Summary: ODBC: Add support for SQL_ATTR_LOGIN_TIMEOUT
 Key: IGNITE-7888
 URL: https://issues.apache.org/jira/browse/IGNITE-7888
 Project: Ignite
  Issue Type: Improvement
  Components: odbc
Affects Versions: 2.1
Reporter: Alexey Popov
Assignee: Igor Sapego
 Fix For: 2.5


It would be great if we can support SQL_ATTR_LOGIN_TIMEOUT even without 
login/password connection parameter. Now we have some default timeout for 
initial ODBC connection establishment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Ignite contibutors page

2018-03-06 Thread Дмитрий Рябов

Hi Igniters!

I'm not mentioned too. Dmitriy Ryabov.

2018-03-05 23:39 GMT+03:00 Denis Magda :

> Hi Dmitriy,
>
> Thanks for kicking off the discussion. I'll wait for a couple of days for
> other response and update the page in bulk.
>
> --
> Denis
>
> On Mon, Mar 5, 2018 at 12:32 PM, Dmitry Pavlov 
> wrote:
>
> > Hi Igniters,
> >
> > I noticed that I'm not mentioned on the community contributors page:
> > https://ignite.apache.org/community/resources.html
> >
> > Perhaps there is someone else, who is not yet added. Or information is
> not
> > up-to-date.
> >
> > Please check the data for yourself and respond if you need to be added /
> or
> > information needs to be updated.
> >
> > Sincerely,
> > Dmitry Pavlov
> >
>

Re: IgniteConfiguration, TcpDiscoverySpi, TcpCommunicationSpi timeouts

2018-03-06 Thread Alexey Popov

Yakov,

1. The proposal list of parameters to deprecate:
TcpDiscoverySpi.setConnectTimeout (covered by
IgniteConfiguration.setFailureDetectionTimeout)
TcpDiscoverySpi.setReconnectCount (covered by
IgniteConfiguration.setFailureDetectionTimeout)
TcpDiscoverySpi.setSocketTimeout (covered by
IgniteConfiguration.setFailureDetectionTimeout)
TcpDiscoverySpi.setAckTimeout (covered by
IgniteConfiguration.setFailureDetectionTimeout)
TcpDiscoverySpi.setMaxAckTimeout (covered by
IgniteConfiguration.setFailureDetectionTimeout)
TcpDiscoverySpi.setNetworkTimeout (IgniteConfiguration.setNetworkTimeout
should be used here)
TcpCommunicationSpi.setConnectTimeout (covered by
IgniteConfiguration.setFailureDetectionTimeout)
TcpCommunicationSpi.setMaxConnectTimeout (covered by
IgniteConfiguration.setFailureDetectionTimeout)
TcpCommunicationSpi.setReconnectCount (covered by
IgniteConfiguration.setFailureDetectionTimeout)
TcpCommunicationSpi.setSocketWriteTimeout
(IgniteConfiguration.setNetworkTimeout should be used here)

2. Internal logic should continue to use
IgniteConfiguration.setFailureDetectionTimeout and
IgniteConfiguration.setNetworkTimeout as it is now.
The deprecated parameters should be alive for a while with the corresponding
javadoc update.
TcpDiscoverySpi.getNetworkTimeout should use
IgniteConfiguration.getNetworkTimeout by default.
TcpCommunicationSpi.getSocketWriteTimeout should use
IgniteConfiguration.getNetworkTimeout by default.

In a few releases, the deprecated parameters could be removed.

3. I think we can keep the existent description of the parameters. Probably,
it could be updated for more clear statements for
IgniteConfiguration.setFailureDetectionTimeout and
IgniteConfiguration.setNetworkTimeout.

Thanks,
Alexey



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[GitHub] ignite pull request #3607: IGNITE-7876

2018-03-06 Thread zaleslaw

GitHub user zaleslaw opened a pull request:

https://github.com/apache/ignite/pull/3607

IGNITE-7876

* Removed Multiclass-classification related code
* Fixed tests
* Adopted Model and Trainer


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7876

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3607.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3607


commit b13fbe8b0a44c6b133cb42480ae5c47bd9a36357
Author: zaleslaw 
Date:   2018-03-04T11:36:31Z

IGNITE-7452: added example

commit b33719c620d85146b12e74ad7a640f17e0a6229b
Author: zaleslaw 
Date:   2018-03-04T12:13:27Z

IGNITE-7876: fixed docs

commit 56a5f21be09b367742f01034b0ee1f5d7788672d
Author: zaleslaw 
Date:   2018-03-04T12:16:51Z

IGNITE-7876: fixed tests

commit 2d7d2958ceea6b6fe9cf6a1a30cde7aca145e4ff
Author: zaleslaw 
Date:   2018-03-06T10:29:01Z

IGNITE-7876: Remove unused code

commit 986a772708eebbd6c2a22cb4963ff56789972ead
Author: zaleslaw 
Date:   2018-03-06T11:39:42Z

IGNITE-7876: Remove unused code

commit 8e4a2b3fd4b2ad71717217ffe12f0324245ada4e
Author: zaleslaw 
Date:   2018-03-06T11:48:09Z

IGNITE-7876: Fix comments




---

Re: Code inspection

What I can suggest now it is to take XML file with existing as is from
previous topic (I remember someone in community already prepared settings)
and set up TeamCity Run configuration as part of Run All Basic Tests (per
commit basis).

If we don’t have XML, I suggest to enable build-in Idea inspections 'as is'
on TeamCity and iteratively improve it according to found issues.

Dmitriy G., would you prepare PR and proof-of-concept TC run configuration?

As discussion became really active, I think that means community is
interested in static code checks.

вт, 6 мар. 2018 г. в 14:08, Dmitry Pavlov :

> I was thinking about some quick check, which will automatically require
> minimum runs. Now, any committer can push changes to the master, which
> break not only the inspection and style, but even the compilation. If this
> control would be automatic, it can allow us make codebase better quite
> fast. But I am afraid it is not realistic.
>
>
>
> вт, 6 мар. 2018 г. в 13:42, Petr Ivanov :
>
>> Sonar is powerful, yes, but it’s power in thoroughness. I.e. it does its
>> job well in cases of leisurely post-build analysis.
>>
>> I’d suggest we use it (if we will use it) in the following scenarios:
>>  — some basic checks Sonar profile for Blocker bugs (it is fast) —
>> something that cannot be passed to master;
>>  — nightly or even weekly run with Full Sonar profile (600+ checks from
>> Firebug, Codestyle, Coverage, etc.) for regression and overall code quality
>> improvement goals.
>>
>> Did not quite get you about push-to-master prohibition. Can you explain
>> scenario in more details?
>>
>>
>> > On 6 Mar 2018, at 13:27, Dmitry Pavlov  wrote:
>> >
>> > Petr, I've heard Sonar is powerful tool.
>> >
>> > Would it help us to prohibit commits to master w/o test run / too much
>> > failed tests / too much inspection errors appeared?
>> >
>> > вт, 6 мар. 2018 г. в 13:22, Alexey Goncharuk <
>> alexey.goncha...@gmail.com>:
>> >
>> >> Dmitriy,
>> >>
>> >> I like this idea a lot. For example, the inspection profile should have
>> >> inspection 'Anonymous class can be converted to lambda' disabled
>> because
>> >> quite a lot of such classes can be sent over the network (although even
>> >> anonymous classes are discourage for such purposes).
>> >>
>> >> I believe we can start with sharing somehow one of the profiles and
>> then
>> >> iteratively improving it until the community is satisfied with the
>> result.
>> >>
>> >> Thoughts?
>> >>
>> >> 2018-03-06 12:06 GMT+03:00 Petr Ivanov :
>> >>
>> >>> We can use Sonar as instrument for code analysis and test coverage
>> >>> inspections.
>> >>>
>> >>>
>> >>>
>>  On 6 Mar 2018, at 11:28, Dmitriy Govorukhin <
>> >>> dmitriy.govoruk...@gmail.com> wrote:
>> 
>>  Dmitriy,
>> 
>>  As I understood, preview topic was of static code analysis in
>> general.
>>  In this topic, I want to discuss only idea inspection rule.
>>  In future, of course, we can expаnd this rule to the TeamCity build.
>> 
>>  On Tue, Mar 6, 2018 at 11:16 AM, Nikolay Izhikov <
>> nizhi...@apache.org>
>>  wrote:
>> 
>> > Hello, Igniters.
>> >
>> > +1 to automatic code style tools.
>> >
>> > Let's make it already!
>> > Do we have a ticket for it?
>> >
>> > Related discussion -
>> >> http://apache-ignite-developers.2346864.n4.nabble.
>> > com/Static-code-analysis-for-Java-td22195.html
>> >
>> > В Вт, 06/03/2018 в 08:15 +, Dmitry Pavlov пишет:
>> >> Hi Dmitriy,
>> >>
>> >> I think we should resurrect thread about addition of code
>> >> inspections,
>> > and
>> >> later we can enable automatic control step to TeamCity.
>> >>
>> >> Could you help me to find it?
>> >>
>> >> вт, 6 мар. 2018 г. в 11:11, Dmitriy Govorukhin <
>> > dmitriy.govoruk...@gmail.com
>> >>> :
>> >>> Hi folks,
>> >>>
>> >>> Do we have 'inspection' [1] scheme for ignite?
>> >>> I see a lot of warnings in my code, and I guess it is because
>> >> everyone
>> > uses
>> >>> different schemes.
>> >>>
>> >>> Let's start the discussion.
>> >>>
>> >>> [1] IDEA inspection
>> >>> 
>> >>>
>> >
>> >>>
>> >>>
>> >>
>>
>>

Re: Code inspection

I was thinking about some quick check, which will automatically require
minimum runs. Now, any committer can push changes to the master, which
break not only the inspection and style, but even the compilation. If this
control would be automatic, it can allow us make codebase better quite
fast. But I am afraid it is not realistic.



вт, 6 мар. 2018 г. в 13:42, Petr Ivanov :

> Sonar is powerful, yes, but it’s power in thoroughness. I.e. it does its
> job well in cases of leisurely post-build analysis.
>
> I’d suggest we use it (if we will use it) in the following scenarios:
>  — some basic checks Sonar profile for Blocker bugs (it is fast) —
> something that cannot be passed to master;
>  — nightly or even weekly run with Full Sonar profile (600+ checks from
> Firebug, Codestyle, Coverage, etc.) for regression and overall code quality
> improvement goals.
>
> Did not quite get you about push-to-master prohibition. Can you explain
> scenario in more details?
>
>
> > On 6 Mar 2018, at 13:27, Dmitry Pavlov  wrote:
> >
> > Petr, I've heard Sonar is powerful tool.
> >
> > Would it help us to prohibit commits to master w/o test run / too much
> > failed tests / too much inspection errors appeared?
> >
> > вт, 6 мар. 2018 г. в 13:22, Alexey Goncharuk  >:
> >
> >> Dmitriy,
> >>
> >> I like this idea a lot. For example, the inspection profile should have
> >> inspection 'Anonymous class can be converted to lambda' disabled because
> >> quite a lot of such classes can be sent over the network (although even
> >> anonymous classes are discourage for such purposes).
> >>
> >> I believe we can start with sharing somehow one of the profiles and then
> >> iteratively improving it until the community is satisfied with the
> result.
> >>
> >> Thoughts?
> >>
> >> 2018-03-06 12:06 GMT+03:00 Petr Ivanov :
> >>
> >>> We can use Sonar as instrument for code analysis and test coverage
> >>> inspections.
> >>>
> >>>
> >>>
>  On 6 Mar 2018, at 11:28, Dmitriy Govorukhin <
> >>> dmitriy.govoruk...@gmail.com> wrote:
> 
>  Dmitriy,
> 
>  As I understood, preview topic was of static code analysis in general.
>  In this topic, I want to discuss only idea inspection rule.
>  In future, of course, we can expаnd this rule to the TeamCity build.
> 
>  On Tue, Mar 6, 2018 at 11:16 AM, Nikolay Izhikov  >
>  wrote:
> 
> > Hello, Igniters.
> >
> > +1 to automatic code style tools.
> >
> > Let's make it already!
> > Do we have a ticket for it?
> >
> > Related discussion -
> >> http://apache-ignite-developers.2346864.n4.nabble.
> > com/Static-code-analysis-for-Java-td22195.html
> >
> > В Вт, 06/03/2018 в 08:15 +, Dmitry Pavlov пишет:
> >> Hi Dmitriy,
> >>
> >> I think we should resurrect thread about addition of code
> >> inspections,
> > and
> >> later we can enable automatic control step to TeamCity.
> >>
> >> Could you help me to find it?
> >>
> >> вт, 6 мар. 2018 г. в 11:11, Dmitriy Govorukhin <
> > dmitriy.govoruk...@gmail.com
> >>> :
> >>> Hi folks,
> >>>
> >>> Do we have 'inspection' [1] scheme for ignite?
> >>> I see a lot of warnings in my code, and I guess it is because
> >> everyone
> > uses
> >>> different schemes.
> >>>
> >>> Let's start the discussion.
> >>>
> >>> [1] IDEA inspection
> >>> 
> >>>
> >
> >>>
> >>>
> >>
>
>

Re: Code inspection

2018-03-06 Thread Maxim Muzafarov

Can we use also Sonar for codestyle (may be some additional code checks)
checks on each PR?


вт, 6 мар. 2018 г. в 13:42, Petr Ivanov :

> Sonar is powerful, yes, but it’s power in thoroughness. I.e. it does its
> job well in cases of leisurely post-build analysis.
>
> I’d suggest we use it (if we will use it) in the following scenarios:
>  — some basic checks Sonar profile for Blocker bugs (it is fast) —
> something that cannot be passed to master;
>  — nightly or even weekly run with Full Sonar profile (600+ checks from
> Firebug, Codestyle, Coverage, etc.) for regression and overall code quality
> improvement goals.
>
> Did not quite get you about push-to-master prohibition. Can you explain
> scenario in more details?
>
>
> > On 6 Mar 2018, at 13:27, Dmitry Pavlov  wrote:
> >
> > Petr, I've heard Sonar is powerful tool.
> >
> > Would it help us to prohibit commits to master w/o test run / too much
> > failed tests / too much inspection errors appeared?
> >
> > вт, 6 мар. 2018 г. в 13:22, Alexey Goncharuk  >:
> >
> >> Dmitriy,
> >>
> >> I like this idea a lot. For example, the inspection profile should have
> >> inspection 'Anonymous class can be converted to lambda' disabled because
> >> quite a lot of such classes can be sent over the network (although even
> >> anonymous classes are discourage for such purposes).
> >>
> >> I believe we can start with sharing somehow one of the profiles and then
> >> iteratively improving it until the community is satisfied with the
> result.
> >>
> >> Thoughts?
> >>
> >> 2018-03-06 12:06 GMT+03:00 Petr Ivanov :
> >>
> >>> We can use Sonar as instrument for code analysis and test coverage
> >>> inspections.
> >>>
> >>>
> >>>
>  On 6 Mar 2018, at 11:28, Dmitriy Govorukhin <
> >>> dmitriy.govoruk...@gmail.com> wrote:
> 
>  Dmitriy,
> 
>  As I understood, preview topic was of static code analysis in general.
>  In this topic, I want to discuss only idea inspection rule.
>  In future, of course, we can expаnd this rule to the TeamCity build.
> 
>  On Tue, Mar 6, 2018 at 11:16 AM, Nikolay Izhikov  >
>  wrote:
> 
> > Hello, Igniters.
> >
> > +1 to automatic code style tools.
> >
> > Let's make it already!
> > Do we have a ticket for it?
> >
> > Related discussion -
> >> http://apache-ignite-developers.2346864.n4.nabble.
> > com/Static-code-analysis-for-Java-td22195.html
> >
> > В Вт, 06/03/2018 в 08:15 +, Dmitry Pavlov пишет:
> >> Hi Dmitriy,
> >>
> >> I think we should resurrect thread about addition of code
> >> inspections,
> > and
> >> later we can enable automatic control step to TeamCity.
> >>
> >> Could you help me to find it?
> >>
> >> вт, 6 мар. 2018 г. в 11:11, Dmitriy Govorukhin <
> > dmitriy.govoruk...@gmail.com
> >>> :
> >>> Hi folks,
> >>>
> >>> Do we have 'inspection' [1] scheme for ignite?
> >>> I see a lot of warnings in my code, and I guess it is because
> >> everyone
> > uses
> >>> different schemes.
> >>>
> >>> Let's start the discussion.
> >>>
> >>> [1] IDEA inspection
> >>> 
> >>>
> >
> >>>
> >>>
> >>
>
>

Re: Let's keep Apache Ignite docs up-to-date

2018-03-06 Thread Guru Stron

Hi, Igniters

Missed this discussion

+1 for git docs, as far as i can see this approach is used by many projects
and it seems to be quite good.


On 6 March 2018 at 11:44, Alexey Goncharuk 
wrote:

> Igniters,
>
> Bumping up this discussion.
>
> I have recently found out that we have this process for documenting new
> releases [1] which looks quite ridiculous to me.
>
> First, creating a copy of page with next-version suffix is inconvenient and
> error-prone: the next-version page is not visible to anyone, moreover, all
> suggested edits to current documentation will be lost after the page copy
> is created. Second, the documentation changes should be transparent to
> users, but now a regular user cannot even review upcoming changes until
> they are granted a permission to see/edit hidden pages.
>
> Unless we have very strong reasons to keep documentation on readme.io (by
> strong I mean a feature that cannot be implemented using a VCS + doc
> generator), I would at least spend some time piloting the 'keep docs in the
> VCS' approach.
>
> Thoughts?
>
> [1] https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>
> 2017-11-02 10:07 GMT+03:00 Dmitry Pavlov :
>
> > I don't like git docs idea, it will require to follow whole
> > PR-review-process that requires long time. IMO it is odd work.
> > If readme.io provides review process, I suggest to keep it as-is.
> >
> > чт, 2 нояб. 2017 г. в 9:57, Dmitriy Setrakyan :
> >
> > > On Wed, Nov 1, 2017 at 11:27 PM, Vladimir Ozerov  >
> > > wrote:
> > >
> > > > +1 for moving docs under Git provided that we find a way to update
> docs
> > > > outside of AI release as it is possible now with readme.io.
> > > >
> > >
> > > I am HUGE +1 for that. The whole problem is that we haven't found a way
> > > yet. All I want is to update a page and have it commit to GIT and
> become
> > > available to public right away. Does anyone know any tool that supports
> > it?
> > >
> >
>

Re: Code inspection

Sonar is powerful, yes, but it’s power in thoroughness. I.e. it does its job 
well in cases of leisurely post-build analysis.

I’d suggest we use it (if we will use it) in the following scenarios:
 — some basic checks Sonar profile for Blocker bugs (it is fast) — something 
that cannot be passed to master;
 — nightly or even weekly run with Full Sonar profile (600+ checks from 
Firebug, Codestyle, Coverage, etc.) for regression and overall code quality 
improvement goals.

Did not quite get you about push-to-master prohibition. Can you explain 
scenario in more details?


> On 6 Mar 2018, at 13:27, Dmitry Pavlov  wrote:
> 
> Petr, I've heard Sonar is powerful tool.
> 
> Would it help us to prohibit commits to master w/o test run / too much
> failed tests / too much inspection errors appeared?
> 
> вт, 6 мар. 2018 г. в 13:22, Alexey Goncharuk :
> 
>> Dmitriy,
>> 
>> I like this idea a lot. For example, the inspection profile should have
>> inspection 'Anonymous class can be converted to lambda' disabled because
>> quite a lot of such classes can be sent over the network (although even
>> anonymous classes are discourage for such purposes).
>> 
>> I believe we can start with sharing somehow one of the profiles and then
>> iteratively improving it until the community is satisfied with the result.
>> 
>> Thoughts?
>> 
>> 2018-03-06 12:06 GMT+03:00 Petr Ivanov :
>> 
>>> We can use Sonar as instrument for code analysis and test coverage
>>> inspections.
>>> 
>>> 
>>> 
 On 6 Mar 2018, at 11:28, Dmitriy Govorukhin <
>>> dmitriy.govoruk...@gmail.com> wrote:
 
 Dmitriy,
 
 As I understood, preview topic was of static code analysis in general.
 In this topic, I want to discuss only idea inspection rule.
 In future, of course, we can expаnd this rule to the TeamCity build.
 
 On Tue, Mar 6, 2018 at 11:16 AM, Nikolay Izhikov 
 wrote:
 
> Hello, Igniters.
> 
> +1 to automatic code style tools.
> 
> Let's make it already!
> Do we have a ticket for it?
> 
> Related discussion -
>> http://apache-ignite-developers.2346864.n4.nabble.
> com/Static-code-analysis-for-Java-td22195.html
> 
> В Вт, 06/03/2018 в 08:15 +, Dmitry Pavlov пишет:
>> Hi Dmitriy,
>> 
>> I think we should resurrect thread about addition of code
>> inspections,
> and
>> later we can enable automatic control step to TeamCity.
>> 
>> Could you help me to find it?
>> 
>> вт, 6 мар. 2018 г. в 11:11, Dmitriy Govorukhin <
> dmitriy.govoruk...@gmail.com
>>> :
>>> Hi folks,
>>> 
>>> Do we have 'inspection' [1] scheme for ignite?
>>> I see a lot of warnings in my code, and I guess it is because
>> everyone
> uses
>>> different schemes.
>>> 
>>> Let's start the discussion.
>>> 
>>> [1] IDEA inspection
>>> 
>>> 
> 
>>> 
>>> 
>>

Re: Transparent Data Encryption (TDE) in Apache Ignite

Alexey,

Yes, administrator has to enter password before cluster *activation*(not start).

В Вт, 06/03/2018 в 13:27 +0300, Alexey Goncharuk пишет:
> Nikolay,
> 
> Does it mean that administrator must enter the MEK password upon Ignite
> start?
> 
> 2018-03-06 13:24 GMT+03:00 Nikolay Izhikov :
> 
> > Hello, Alexey.
> > 
> > Thank you for very helpfull feedback.
> > We certainly consider all the issues you written.
> > 
> > > How encryption keys will be stored and accessed?
> > 
> > *MEK(Master encryption key)* will be stored in regular java key store JKS
> > [1].
> > To access it admin must enter key store password.
> > 
> > *CEK(Cache encryption key)* will be stored inside regular Ignite cache.
> > It will be encrypted by MEK.
> > So access to CEK may be done only after one got the MEK.
> > 
> > Please, see IEP draft for futher information [2].
> > 
> > > Will pages be encrypted on disk or also in memory?
> > 
> > Current understanding that only on disk pages will be encrypted.
> > PCI DSS standart requires only on disk encryption.
> > 
> > > How do you make sure that encrypted page fits the page size
> > 
> > AFAIK encryption doesn't affect data size.
> > 
> > > This significantly increases success of known-plain-text attacks.
> > > How will you write WAL delta records for encrypted pages?
> > 
> > I don't have an answer to this questions for now.
> > So, prior to starting an implementation we returns with answers.
> > 
> > [1] https://docs.oracle.com/javase/7/docs/api/java/security/KeyStore.html
> > [2] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
> > 
> > 
> > В Вт, 06/03/2018 в 13:04 +0300, Alexey Goncharuk пишет:
> > > Guys,
> > > 
> > > I think this TDE proposal is not thought through enough yet. Please
> > > consider the following points when writing the IEP:
> > > 
> > >  * How encryption keys will be stored and accessed? If the encryption key
> > > is stored with the same permissions as the main data storage, the whole
> > > exercise with encryption is self-deception
> > >  * Will pages be encrypted on disk or also in memory?
> > >  * How do you make sure that encrypted page fits the page size (I am not
> > 
> > an
> > > expert in encryption, so I am not sure if it adds an overhead)
> > >  * As Dmitriy Pavlov mentioned, currently data and index pages are highly
> > > redundant and some of the fields in certain pages are known in advance.
> > > This significantly increases success of known-plain-text attacks. How are
> > > you planning to fix it?
> > >  * How will you write WAL delta records for encrypted pages? If a change
> > 
> > in
> > > a single byte will potentially change the whole page, this will induce a
> > > huge write amplification on WAL. How do you encrypt WAL data records? How
> > > will this work with this optimization: [1]
> > > 
> > > [1] https://ggsystems.atlassian.net/browse/IGN-7789
> > > 
> > > --AG
> > > 
> > > 2018-03-06 8:55 GMT+03:00 Nikolay Izhikov :
> > > 
> > > > Thank you, it's - nizhikov
> > > > 
> > > > В Пн, 05/03/2018 в 15:09 -0800, Denis Magda пишет:
> > > > > Nikolay, what's your Wiki ID? I'll grant you required permissions.
> > > > > 
> > > > > --
> > > > > Denis
> > > > > 
> > > > > On Sun, Mar 4, 2018 at 11:00 PM, Nikolay Izhikov <
> > 
> > nizhi...@apache.org>
> > > > 
> > > > wrote:
> > > > > > Hello, Denis.
> > > > > > 
> > > > > > > I would encourage you creating an IEP
> > > > > > 
> > > > > > That is exactly what we want to do :)
> > > > > > 
> > > > > > But seems I have not sufficient privileges to do it on Ignite wiki.
> > > > > > 
> > > > > > https://cwiki.apache.org/confluence/display/IGNITE/
> > 
> > Active+Proposals
> > > > > > 
> > > > > > Can you or someone give me such rights?
> > > > > > 
> > > > > > В Чт, 01/03/2018 в 22:23 -0800, Denis Magda пишет:
> > > > > > > Dmitriy R., Nilokay,
> > > > > > > 
> > > > > > > Thanks for the analysis and handout of the architectural design.
> > 
> > No
> > > > 
> > > > doubts,
> > > > > > > it would be a valuable addition to Ignite.
> > > > > > > 
> > > > > > > I would encourage you creating an IEP on the wiki and break the
> > 
> > work
> > > > 
> > > > into
> > > > > > > pieces discussing specific part with the community.
> > > > > > > 
> > > > > > > --
> > > > > > > Denis
> > > > > > > 
> > > > > > > 
> > > > > > > On Thu, Mar 1, 2018 at 9:29 PM, Nikolay Izhikov <
> > 
> > nizhi...@apache.org>
> > > > 
> > > > wrote:
> > > > > > > 
> > > > > > > > Hello, Dmitriy.
> > > > > > > > 
> > > > > > > > Thank you for feedback!
> > > > > > > > 
> > > > > > > > > Will it be supported?
> > > > > > > > 
> > > > > > > > Yes.
> > > > > > > > 
> > > > > > > > TDE shouldn't broke any of existing Ignite features.
> > > > > > > > It adds some encrypt/decrypt level when we writing and reading
> > > > 
> > > > pages
> > > > > > > > in/from PDS.
> > > > > > > > 
> > > > > > > > В Пт, 02/03/2018 в 07:29 +0300, Dmitriy Setrakyan пишет:
> > > > > > > > > I have looked at the design,

Re: Transparent Data Encryption (TDE) in Apache Ignite

Nikolay,

Does it mean that administrator must enter the MEK password upon Ignite
start?

2018-03-06 13:24 GMT+03:00 Nikolay Izhikov :

> Hello, Alexey.
>
> Thank you for very helpfull feedback.
> We certainly consider all the issues you written.
>
> > How encryption keys will be stored and accessed?
>
> *MEK(Master encryption key)* will be stored in regular java key store JKS
> [1].
> To access it admin must enter key store password.
>
> *CEK(Cache encryption key)* will be stored inside regular Ignite cache.
> It will be encrypted by MEK.
> So access to CEK may be done only after one got the MEK.
>
> Please, see IEP draft for futher information [2].
>
> > Will pages be encrypted on disk or also in memory?
>
> Current understanding that only on disk pages will be encrypted.
> PCI DSS standart requires only on disk encryption.
>
> > How do you make sure that encrypted page fits the page size
>
> AFAIK encryption doesn't affect data size.
>
> > This significantly increases success of known-plain-text attacks.
> > How will you write WAL delta records for encrypted pages?
>
> I don't have an answer to this questions for now.
> So, prior to starting an implementation we returns with answers.
>
> [1] https://docs.oracle.com/javase/7/docs/api/java/security/KeyStore.html
> [2] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
>
>
> В Вт, 06/03/2018 в 13:04 +0300, Alexey Goncharuk пишет:
> > Guys,
> >
> > I think this TDE proposal is not thought through enough yet. Please
> > consider the following points when writing the IEP:
> >
> >  * How encryption keys will be stored and accessed? If the encryption key
> > is stored with the same permissions as the main data storage, the whole
> > exercise with encryption is self-deception
> >  * Will pages be encrypted on disk or also in memory?
> >  * How do you make sure that encrypted page fits the page size (I am not
> an
> > expert in encryption, so I am not sure if it adds an overhead)
> >  * As Dmitriy Pavlov mentioned, currently data and index pages are highly
> > redundant and some of the fields in certain pages are known in advance.
> > This significantly increases success of known-plain-text attacks. How are
> > you planning to fix it?
> >  * How will you write WAL delta records for encrypted pages? If a change
> in
> > a single byte will potentially change the whole page, this will induce a
> > huge write amplification on WAL. How do you encrypt WAL data records? How
> > will this work with this optimization: [1]
> >
> > [1] https://ggsystems.atlassian.net/browse/IGN-7789
> >
> > --AG
> >
> > 2018-03-06 8:55 GMT+03:00 Nikolay Izhikov :
> >
> > > Thank you, it's - nizhikov
> > >
> > > В Пн, 05/03/2018 в 15:09 -0800, Denis Magda пишет:
> > > > Nikolay, what's your Wiki ID? I'll grant you required permissions.
> > > >
> > > > --
> > > > Denis
> > > >
> > > > On Sun, Mar 4, 2018 at 11:00 PM, Nikolay Izhikov <
> nizhi...@apache.org>
> > >
> > > wrote:
> > > > > Hello, Denis.
> > > > >
> > > > > > I would encourage you creating an IEP
> > > > >
> > > > > That is exactly what we want to do :)
> > > > >
> > > > > But seems I have not sufficient privileges to do it on Ignite wiki.
> > > > >
> > > > > https://cwiki.apache.org/confluence/display/IGNITE/
> Active+Proposals
> > > > >
> > > > > Can you or someone give me such rights?
> > > > >
> > > > > В Чт, 01/03/2018 в 22:23 -0800, Denis Magda пишет:
> > > > > > Dmitriy R., Nilokay,
> > > > > >
> > > > > > Thanks for the analysis and handout of the architectural design.
> No
> > >
> > > doubts,
> > > > > > it would be a valuable addition to Ignite.
> > > > > >
> > > > > > I would encourage you creating an IEP on the wiki and break the
> work
> > >
> > > into
> > > > > > pieces discussing specific part with the community.
> > > > > >
> > > > > > --
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > > On Thu, Mar 1, 2018 at 9:29 PM, Nikolay Izhikov <
> nizhi...@apache.org>
> > >
> > > wrote:
> > > > > >
> > > > > > > Hello, Dmitriy.
> > > > > > >
> > > > > > > Thank you for feedback!
> > > > > > >
> > > > > > > > Will it be supported?
> > > > > > >
> > > > > > > Yes.
> > > > > > >
> > > > > > > TDE shouldn't broke any of existing Ignite features.
> > > > > > > It adds some encrypt/decrypt level when we writing and reading
> > >
> > > pages
> > > > > > > in/from PDS.
> > > > > > >
> > > > > > > В Пт, 02/03/2018 в 07:29 +0300, Dmitriy Setrakyan пишет:
> > > > > > > > I have looked at the design, but could not find anything
> about
> > >
> > > running
> > > > > > >
> > > > > > > SQL
> > > > > > > > queries against the encrypted data. Will it be supported?
> > > > > > > >
> > > > > > > > D.
> > > > > > > >
> > > > > > > > On Thu, Mar 1, 2018 at 8:05 PM, Nikolay Izhikov <
> > >
> > > nizhi...@apache.org>
> > > > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hell, Dima!
> > > > > > > > >
> > > > > > > > > Thank you for document!
> > > > > > > > >
> > > > > > > > > I'm ready

Re: Code inspection

Petr, I've heard Sonar is powerful tool.

Would it help us to prohibit commits to master w/o test run / too much
failed tests / too much inspection errors appeared?

вт, 6 мар. 2018 г. в 13:22, Alexey Goncharuk :

> Dmitriy,
>
> I like this idea a lot. For example, the inspection profile should have
> inspection 'Anonymous class can be converted to lambda' disabled because
> quite a lot of such classes can be sent over the network (although even
> anonymous classes are discourage for such purposes).
>
> I believe we can start with sharing somehow one of the profiles and then
> iteratively improving it until the community is satisfied with the result.
>
> Thoughts?
>
> 2018-03-06 12:06 GMT+03:00 Petr Ivanov :
>
> > We can use Sonar as instrument for code analysis and test coverage
> > inspections.
> >
> >
> >
> > > On 6 Mar 2018, at 11:28, Dmitriy Govorukhin <
> > dmitriy.govoruk...@gmail.com> wrote:
> > >
> > > Dmitriy,
> > >
> > > As I understood, preview topic was of static code analysis in general.
> > > In this topic, I want to discuss only idea inspection rule.
> > > In future, of course, we can expаnd this rule to the TeamCity build.
> > >
> > > On Tue, Mar 6, 2018 at 11:16 AM, Nikolay Izhikov 
> > > wrote:
> > >
> > >> Hello, Igniters.
> > >>
> > >> +1 to automatic code style tools.
> > >>
> > >> Let's make it already!
> > >> Do we have a ticket for it?
> > >>
> > >> Related discussion -
> http://apache-ignite-developers.2346864.n4.nabble.
> > >> com/Static-code-analysis-for-Java-td22195.html
> > >>
> > >> В Вт, 06/03/2018 в 08:15 +, Dmitry Pavlov пишет:
> > >>> Hi Dmitriy,
> > >>>
> > >>> I think we should resurrect thread about addition of code
> inspections,
> > >> and
> > >>> later we can enable automatic control step to TeamCity.
> > >>>
> > >>> Could you help me to find it?
> > >>>
> > >>> вт, 6 мар. 2018 г. в 11:11, Dmitriy Govorukhin <
> > >> dmitriy.govoruk...@gmail.com
> >  :
> >  Hi folks,
> > 
> >  Do we have 'inspection' [1] scheme for ignite?
> >  I see a lot of warnings in my code, and I guess it is because
> everyone
> > >> uses
> >  different schemes.
> > 
> >  Let's start the discussion.
> > 
> >  [1] IDEA inspection
> >  
> > 
> > >>
> >
> >
>

[jira] [Created] (IGNITE-7887) [ML] Adopt SVM Linear Multi-Class Classification Model and Trainer to the new Partitioned Dataset

2018-03-06 Thread Aleksey Zinoviev (JIRA)

Aleksey Zinoviev created IGNITE-7887:


 Summary: [ML] Adopt SVM Linear Multi-Class Classification Model 
and Trainer to the new Partitioned Dataset
 Key: IGNITE-7887
 URL: https://issues.apache.org/jira/browse/IGNITE-7887
 Project: Ignite
  Issue Type: Sub-task
  Components: ml
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Transparent Data Encryption (TDE) in Apache Ignite

Hello, Alexey.

Thank you for very helpfull feedback.
We certainly consider all the issues you written.

> How encryption keys will be stored and accessed?

*MEK(Master encryption key)* will be stored in regular java key store JKS [1].
To access it admin must enter key store password.

*CEK(Cache encryption key)* will be stored inside regular Ignite cache. 
It will be encrypted by MEK. 
So access to CEK may be done only after one got the MEK.

Please, see IEP draft for futher information [2].

> Will pages be encrypted on disk or also in memory?

Current understanding that only on disk pages will be encrypted.
PCI DSS standart requires only on disk encryption.

> How do you make sure that encrypted page fits the page size

AFAIK encryption doesn't affect data size.

> This significantly increases success of known-plain-text attacks.
> How will you write WAL delta records for encrypted pages?

I don't have an answer to this questions for now.
So, prior to starting an implementation we returns with answers.

[1] https://docs.oracle.com/javase/7/docs/api/java/security/KeyStore.html
[2] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf


В Вт, 06/03/2018 в 13:04 +0300, Alexey Goncharuk пишет:
> Guys,
> 
> I think this TDE proposal is not thought through enough yet. Please
> consider the following points when writing the IEP:
> 
>  * How encryption keys will be stored and accessed? If the encryption key
> is stored with the same permissions as the main data storage, the whole
> exercise with encryption is self-deception
>  * Will pages be encrypted on disk or also in memory?
>  * How do you make sure that encrypted page fits the page size (I am not an
> expert in encryption, so I am not sure if it adds an overhead)
>  * As Dmitriy Pavlov mentioned, currently data and index pages are highly
> redundant and some of the fields in certain pages are known in advance.
> This significantly increases success of known-plain-text attacks. How are
> you planning to fix it?
>  * How will you write WAL delta records for encrypted pages? If a change in
> a single byte will potentially change the whole page, this will induce a
> huge write amplification on WAL. How do you encrypt WAL data records? How
> will this work with this optimization: [1]
> 
> [1] https://ggsystems.atlassian.net/browse/IGN-7789
> 
> --AG
> 
> 2018-03-06 8:55 GMT+03:00 Nikolay Izhikov :
> 
> > Thank you, it's - nizhikov
> > 
> > В Пн, 05/03/2018 в 15:09 -0800, Denis Magda пишет:
> > > Nikolay, what's your Wiki ID? I'll grant you required permissions.
> > > 
> > > --
> > > Denis
> > > 
> > > On Sun, Mar 4, 2018 at 11:00 PM, Nikolay Izhikov 
> > 
> > wrote:
> > > > Hello, Denis.
> > > > 
> > > > > I would encourage you creating an IEP
> > > > 
> > > > That is exactly what we want to do :)
> > > > 
> > > > But seems I have not sufficient privileges to do it on Ignite wiki.
> > > > 
> > > > https://cwiki.apache.org/confluence/display/IGNITE/Active+Proposals
> > > > 
> > > > Can you or someone give me such rights?
> > > > 
> > > > В Чт, 01/03/2018 в 22:23 -0800, Denis Magda пишет:
> > > > > Dmitriy R., Nilokay,
> > > > > 
> > > > > Thanks for the analysis and handout of the architectural design. No
> > 
> > doubts,
> > > > > it would be a valuable addition to Ignite.
> > > > > 
> > > > > I would encourage you creating an IEP on the wiki and break the work
> > 
> > into
> > > > > pieces discussing specific part with the community.
> > > > > 
> > > > > --
> > > > > Denis
> > > > > 
> > > > > 
> > > > > On Thu, Mar 1, 2018 at 9:29 PM, Nikolay Izhikov 
> > 
> > wrote:
> > > > > 
> > > > > > Hello, Dmitriy.
> > > > > > 
> > > > > > Thank you for feedback!
> > > > > > 
> > > > > > > Will it be supported?
> > > > > > 
> > > > > > Yes.
> > > > > > 
> > > > > > TDE shouldn't broke any of existing Ignite features.
> > > > > > It adds some encrypt/decrypt level when we writing and reading
> > 
> > pages
> > > > > > in/from PDS.
> > > > > > 
> > > > > > В Пт, 02/03/2018 в 07:29 +0300, Dmitriy Setrakyan пишет:
> > > > > > > I have looked at the design, but could not find anything about
> > 
> > running
> > > > > > 
> > > > > > SQL
> > > > > > > queries against the encrypted data. Will it be supported?
> > > > > > > 
> > > > > > > D.
> > > > > > > 
> > > > > > > On Thu, Mar 1, 2018 at 8:05 PM, Nikolay Izhikov <
> > 
> > nizhi...@apache.org>
> > > > > > 
> > > > > > wrote:
> > > > > > > 
> > > > > > > > Hell, Dima!
> > > > > > > > 
> > > > > > > > Thank you for document!
> > > > > > > > 
> > > > > > > > I'm ready to implement this feature with you.
> > > > > > > > 
> > > > > > > > Igniters, please, share you thoughts about proposed design
> > > > > > > > 
> > > > > > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
> > > > > > > > 
> > > > > > > > В Чт, 01/03/2018 в 15:46 +0300, Дмитрий Рябов пишет:
> > > > > > > > > Hello, Igniters!
> > > > > > > > > 
> > > > > > > > > I investigated the issue and

[GitHub] ignite pull request #3606: IGNITE-7845 fix checking atomicLong in IgniteClie...

2018-03-06 Thread akalash

GitHub user akalash opened a pull request:

https://github.com/apache/ignite/pull/3606

IGNITE-7845 fix checking atomicLong in IgniteClientDataStructuresTestâ¦

â¦#testAtomicLong.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7845

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3606.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3606


commit 514de7920e5ee7c7d3d83d1687de71d664e72028
Author: Anton Kalashnikov 
Date:   2018-03-06T10:22:37Z

IGNITE-7845 fix checking atomicLong in 
IgniteClientDataStructuresTest#testAtomicLong.




---

Re: Code inspection

Dmitriy,

I like this idea a lot. For example, the inspection profile should have
inspection 'Anonymous class can be converted to lambda' disabled because
quite a lot of such classes can be sent over the network (although even
anonymous classes are discourage for such purposes).

I believe we can start with sharing somehow one of the profiles and then
iteratively improving it until the community is satisfied with the result.

Thoughts?

2018-03-06 12:06 GMT+03:00 Petr Ivanov :

> We can use Sonar as instrument for code analysis and test coverage
> inspections.
>
>
>
> > On 6 Mar 2018, at 11:28, Dmitriy Govorukhin <
> dmitriy.govoruk...@gmail.com> wrote:
> >
> > Dmitriy,
> >
> > As I understood, preview topic was of static code analysis in general.
> > In this topic, I want to discuss only idea inspection rule.
> > In future, of course, we can expаnd this rule to the TeamCity build.
> >
> > On Tue, Mar 6, 2018 at 11:16 AM, Nikolay Izhikov 
> > wrote:
> >
> >> Hello, Igniters.
> >>
> >> +1 to automatic code style tools.
> >>
> >> Let's make it already!
> >> Do we have a ticket for it?
> >>
> >> Related discussion - http://apache-ignite-developers.2346864.n4.nabble.
> >> com/Static-code-analysis-for-Java-td22195.html
> >>
> >> В Вт, 06/03/2018 в 08:15 +, Dmitry Pavlov пишет:
> >>> Hi Dmitriy,
> >>>
> >>> I think we should resurrect thread about addition of code inspections,
> >> and
> >>> later we can enable automatic control step to TeamCity.
> >>>
> >>> Could you help me to find it?
> >>>
> >>> вт, 6 мар. 2018 г. в 11:11, Dmitriy Govorukhin <
> >> dmitriy.govoruk...@gmail.com
>  :
>  Hi folks,
> 
>  Do we have 'inspection' [1] scheme for ignite?
>  I see a lot of warnings in my code, and I guess it is because everyone
> >> uses
>  different schemes.
> 
>  Let's start the discussion.
> 
>  [1] IDEA inspection
>  
> 
> >>
>
>

Re: Transparent Data Encryption (TDE) in Apache Ignite

My bad, the correct link is
https://issues.apache.org/jira/browse/IGNITE-5829

2018-03-06 13:04 GMT+03:00 Alexey Goncharuk :

> Guys,
>
> I think this TDE proposal is not thought through enough yet. Please
> consider the following points when writing the IEP:
>
>  * How encryption keys will be stored and accessed? If the encryption key
> is stored with the same permissions as the main data storage, the whole
> exercise with encryption is self-deception
>  * Will pages be encrypted on disk or also in memory?
>  * How do you make sure that encrypted page fits the page size (I am not
> an expert in encryption, so I am not sure if it adds an overhead)
>  * As Dmitriy Pavlov mentioned, currently data and index pages are highly
> redundant and some of the fields in certain pages are known in advance.
> This significantly increases success of known-plain-text attacks. How are
> you planning to fix it?
>  * How will you write WAL delta records for encrypted pages? If a change
> in a single byte will potentially change the whole page, this will induce a
> huge write amplification on WAL. How do you encrypt WAL data records? How
> will this work with this optimization: [1]
>
> [1] https://ggsystems.atlassian.net/browse/IGN-7789
>
> --AG
>
> 2018-03-06 8:55 GMT+03:00 Nikolay Izhikov :
>
>> Thank you, it's - nizhikov
>>
>> В Пн, 05/03/2018 в 15:09 -0800, Denis Magda пишет:
>> > Nikolay, what's your Wiki ID? I'll grant you required permissions.
>> >
>> > --
>> > Denis
>> >
>> > On Sun, Mar 4, 2018 at 11:00 PM, Nikolay Izhikov 
>> wrote:
>> > > Hello, Denis.
>> > >
>> > > > I would encourage you creating an IEP
>> > >
>> > > That is exactly what we want to do :)
>> > >
>> > > But seems I have not sufficient privileges to do it on Ignite wiki.
>> > >
>> > > https://cwiki.apache.org/confluence/display/IGNITE/Active+Proposals
>> > >
>> > > Can you or someone give me such rights?
>> > >
>> > > В Чт, 01/03/2018 в 22:23 -0800, Denis Magda пишет:
>> > > > Dmitriy R., Nilokay,
>> > > >
>> > > > Thanks for the analysis and handout of the architectural design. No
>> doubts,
>> > > > it would be a valuable addition to Ignite.
>> > > >
>> > > > I would encourage you creating an IEP on the wiki and break the
>> work into
>> > > > pieces discussing specific part with the community.
>> > > >
>> > > > --
>> > > > Denis
>> > > >
>> > > >
>> > > > On Thu, Mar 1, 2018 at 9:29 PM, Nikolay Izhikov <
>> nizhi...@apache.org> wrote:
>> > > >
>> > > > > Hello, Dmitriy.
>> > > > >
>> > > > > Thank you for feedback!
>> > > > >
>> > > > > > Will it be supported?
>> > > > >
>> > > > > Yes.
>> > > > >
>> > > > > TDE shouldn't broke any of existing Ignite features.
>> > > > > It adds some encrypt/decrypt level when we writing and reading
>> pages
>> > > > > in/from PDS.
>> > > > >
>> > > > > В Пт, 02/03/2018 в 07:29 +0300, Dmitriy Setrakyan пишет:
>> > > > > > I have looked at the design, but could not find anything about
>> running
>> > > > >
>> > > > > SQL
>> > > > > > queries against the encrypted data. Will it be supported?
>> > > > > >
>> > > > > > D.
>> > > > > >
>> > > > > > On Thu, Mar 1, 2018 at 8:05 PM, Nikolay Izhikov <
>> nizhi...@apache.org>
>> > > > >
>> > > > > wrote:
>> > > > > >
>> > > > > > > Hell, Dima!
>> > > > > > >
>> > > > > > > Thank you for document!
>> > > > > > >
>> > > > > > > I'm ready to implement this feature with you.
>> > > > > > >
>> > > > > > > Igniters, please, share you thoughts about proposed design
>> > > > > > >
>> > > > > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
>> > > > > > >
>> > > > > > > В Чт, 01/03/2018 в 15:46 +0300, Дмитрий Рябов пишет:
>> > > > > > > > Hello, Igniters!
>> > > > > > > >
>> > > > > > > > I investigated the issue and wrote some details in a draft
>> document
>> > > > > > > > [1]. I think we should made IEP for TDE because it is a big
>> change
>> > > > >
>> > > > > and
>> > > > > > > > should be described in a single place, but not in a message
>> > > > > > > > conversation.
>> > > > > > > > Please, look it and write your thoughts. What is not
>> understandable,
>> > > > > > > > what should be detailed or described?
>> > > > > > > >
>> > > > > > > > > Where are we going to store keys (MEK) physically? Would
>> it be
>> > > > >
>> > > > > PKCS#11
>> > > > > > > > > storage? Where we will store passwords to unlock storage
>> or it
>> > > > >
>> > > > > will be
>> > > > > > > > > responibilty of user?
>> > > > > > > >
>> > > > > > > > I think we should provide interface for MEK storage to let
>> users use
>> > > > > > > > storages they want. I suppose at the first step we should
>> provide
>> > > > >
>> > > > > very
>> > > > > > > > simple implementation, which will store MEK on every node
>> and MEK
>> > > > >
>> > > > > will
>> > > > > > > > be extracted by administrator during cluster activation
>> process. Once
>> > > > > > > > MEK is extracted from key store, we decrypt CEKs and
>> destroy open
>> > >

Re: Transparent Data Encryption (TDE) in Apache Ignite

Guys,

I think this TDE proposal is not thought through enough yet. Please
consider the following points when writing the IEP:

 * How encryption keys will be stored and accessed? If the encryption key
is stored with the same permissions as the main data storage, the whole
exercise with encryption is self-deception
 * Will pages be encrypted on disk or also in memory?
 * How do you make sure that encrypted page fits the page size (I am not an
expert in encryption, so I am not sure if it adds an overhead)
 * As Dmitriy Pavlov mentioned, currently data and index pages are highly
redundant and some of the fields in certain pages are known in advance.
This significantly increases success of known-plain-text attacks. How are
you planning to fix it?
 * How will you write WAL delta records for encrypted pages? If a change in
a single byte will potentially change the whole page, this will induce a
huge write amplification on WAL. How do you encrypt WAL data records? How
will this work with this optimization: [1]

[1] https://ggsystems.atlassian.net/browse/IGN-7789

--AG

2018-03-06 8:55 GMT+03:00 Nikolay Izhikov :

> Thank you, it's - nizhikov
>
> В Пн, 05/03/2018 в 15:09 -0800, Denis Magda пишет:
> > Nikolay, what's your Wiki ID? I'll grant you required permissions.
> >
> > --
> > Denis
> >
> > On Sun, Mar 4, 2018 at 11:00 PM, Nikolay Izhikov 
> wrote:
> > > Hello, Denis.
> > >
> > > > I would encourage you creating an IEP
> > >
> > > That is exactly what we want to do :)
> > >
> > > But seems I have not sufficient privileges to do it on Ignite wiki.
> > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/Active+Proposals
> > >
> > > Can you or someone give me such rights?
> > >
> > > В Чт, 01/03/2018 в 22:23 -0800, Denis Magda пишет:
> > > > Dmitriy R., Nilokay,
> > > >
> > > > Thanks for the analysis and handout of the architectural design. No
> doubts,
> > > > it would be a valuable addition to Ignite.
> > > >
> > > > I would encourage you creating an IEP on the wiki and break the work
> into
> > > > pieces discussing specific part with the community.
> > > >
> > > > --
> > > > Denis
> > > >
> > > >
> > > > On Thu, Mar 1, 2018 at 9:29 PM, Nikolay Izhikov 
> wrote:
> > > >
> > > > > Hello, Dmitriy.
> > > > >
> > > > > Thank you for feedback!
> > > > >
> > > > > > Will it be supported?
> > > > >
> > > > > Yes.
> > > > >
> > > > > TDE shouldn't broke any of existing Ignite features.
> > > > > It adds some encrypt/decrypt level when we writing and reading
> pages
> > > > > in/from PDS.
> > > > >
> > > > > В Пт, 02/03/2018 в 07:29 +0300, Dmitriy Setrakyan пишет:
> > > > > > I have looked at the design, but could not find anything about
> running
> > > > >
> > > > > SQL
> > > > > > queries against the encrypted data. Will it be supported?
> > > > > >
> > > > > > D.
> > > > > >
> > > > > > On Thu, Mar 1, 2018 at 8:05 PM, Nikolay Izhikov <
> nizhi...@apache.org>
> > > > >
> > > > > wrote:
> > > > > >
> > > > > > > Hell, Dima!
> > > > > > >
> > > > > > > Thank you for document!
> > > > > > >
> > > > > > > I'm ready to implement this feature with you.
> > > > > > >
> > > > > > > Igniters, please, share you thoughts about proposed design
> > > > > > >
> > > > > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
> > > > > > >
> > > > > > > В Чт, 01/03/2018 в 15:46 +0300, Дмитрий Рябов пишет:
> > > > > > > > Hello, Igniters!
> > > > > > > >
> > > > > > > > I investigated the issue and wrote some details in a draft
> document
> > > > > > > > [1]. I think we should made IEP for TDE because it is a big
> change
> > > > >
> > > > > and
> > > > > > > > should be described in a single place, but not in a message
> > > > > > > > conversation.
> > > > > > > > Please, look it and write your thoughts. What is not
> understandable,
> > > > > > > > what should be detailed or described?
> > > > > > > >
> > > > > > > > > Where are we going to store keys (MEK) physically? Would
> it be
> > > > >
> > > > > PKCS#11
> > > > > > > > > storage? Where we will store passwords to unlock storage
> or it
> > > > >
> > > > > will be
> > > > > > > > > responibilty of user?
> > > > > > > >
> > > > > > > > I think we should provide interface for MEK storage to let
> users use
> > > > > > > > storages they want. I suppose at the first step we should
> provide
> > > > >
> > > > > very
> > > > > > > > simple implementation, which will store MEK on every node
> and MEK
> > > > >
> > > > > will
> > > > > > > > be extracted by administrator during cluster activation
> process. Once
> > > > > > > > MEK is extracted from key store, we decrypt CEKs and destroy
> open
> > > > >
> > > > > MEK,
> > > > > > > > leaving open only cache keys.
> > > > > > > >
> > > > > > > > I think external storage is user's worry and we shouldn't
> give users
> > > > > > > > built-in external storage like Oracle Wallet or Microsoft
> Azure Key
> > > > > > > > Vault because it will increase Ignite's

Re: MTCGA: Tests of the week

2018-03-06 Thread Igor Sapego

Great work!

Best Regards,
Igor

On Tue, Mar 6, 2018 at 1:42 AM, Denis Magda  wrote:

> Astonishing progress! Thanks for keep driving this activity and making the
> project more and more reliable.
>
> --
> Denis
>
> On Mon, Mar 5, 2018 at 1:47 PM, Dmitry Pavlov 
> wrote:
>
> > Hi, Denis, Igniters,
> >
> > I've updated the report for the last weekend mass run-all, please find it
> > in wiki
> > https://cwiki.apache.org/confluence/pages/viewpageattachments.action?
> > pageId=73631266
> >
> > In 30 clean runs, there was 371 failed tests, 21 suites. During previous
> > mass run there were 804 tests including 355 C ++, suites 29. This
> indicate
> > progress, but still there is a lot of work to be done.
> >
> > Thanks to Alexey Goncharuk, Igor Sapego, and Maxim Muzafarov and all
> folks
> > were involved into this previous weeks.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вт, 27 февр. 2018 г. в 13:00, Dmitry Pavlov :
> >
> > > Hi Denis,
> > >
> > > It is not easy to say exact number of tests, because test failures are
> > > quite randomized. Correct metric can be obtained from mass 'RunAll'
> runs,
> > > for example, during weekend.
> > >
> > > In the same time brief estimation gives number of failures around 30-60
> > > per 'RunAll' instead of 60-80 (we don't count 355 CPP test here because
> > it
> > > is one problem).
> > >
> > > If we check issues create/resolve chart, it is available
> > > https://cwiki.apache.org/confluence/display/IGNITE/
> > Make+Teamcity+Green+Again,
> > > progress is obivious, ~ 27 issues reported and ~ 13 were resolved.
> > >
> > > Hope someone in the community can pick up remaining unsassigned issues:
> > >
> > > https://issues.apache.org/jira/issues/?jql=labels%
> > 3DMakeTeamcityGreenAgain%20AND%20createdDate%3E-3w%
> > 20AND%20resolution%20is%20EMPTY%20and%20assignee%20is%20EMPTY
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > >
> > >
> > > вт, 27 февр. 2018 г. в 0:45, Denis Magda :
> > >
> > >> Hi Dmitriy,
> > >>
> > >> Could you share some statistics? How many tests were fixed or merged
> in
> > >> one? Just curiuos.
> > >>
> > >> --
> > >> Denis
> > >>
> > >> On Mon, Feb 26, 2018 at 9:36 AM, Dmitry Pavlov  >
> > >> wrote:
> > >>
> > >> > Hi Folks,
> > >> >
> > >> > Fixing the tests already brings some results. The number of falling
> > >> tests
> > >> > in a single run was reduced. Thanks to all, especially to Peter
> > Ivanov,
> > >> > Alexey Goncharuk, Pavel Kovalenko
> > >> >
> > >> > There are 4 tickets that were not picked up by community (list at
> the
> > >> end
> > >> > of letter). Please look through and assign issue if you are able to
> > help
> > >> > with this.
> > >> >
> > >> > Sincerely,
> > >> > Dmitriy Pavlov
> > >> >
> > >> >  - IGNITE-7791 -Ignite Client Nodes: failed test
> > >> > IgniteClientReconnectCacheTest.testReconnectCacheDestroyedAnd
> > Created()
> > >> > - IGNITE-7789 - Ignite Client Nodes: failed test
> > >> > IgniteClientReconnectApiExceptionTest.testErrorOnDisconnect()
> > >> > - IGNITE-7708 - Ignite Compute has flaky failures in
> > >> ConfigVariationTests
> > >> > - IGNITE-7692 - affinityCall and affinityRun may execute code on
> > backup
> > >> > partitions
> > >> >
> > >>
> > >
> >
>

Re: Data compression design proposal

2018-03-06 Thread Taras Ledkov

Hi,

I guess page level compression make sense on page loading / eviction.
In this case we can decrease I/O operation and performance boost can be
reached.
What is goal for in-memory compression? Holds about 2-5x data in memory
with performance drop?

Also please clarify the case with compression/decompression for hot and
cold pages.

Is it right for your approach:
1. Hot pages are always decompressed in memory because many read/write
operations touch ones.

2. So we can compress only cold pages.

So the way is suitable when the hot data size << available RAM size.

Thoughts?

On 05.03.2018 20:18, Vyacheslav Daradur wrote:

Hi Igniters!

I’d like to do next step in our data compression discussion [1].

Most Igniters vote for per-data-page compression.

I’d like to accumulate main theses to start implementation:
- page will be compressed with the dictionary-based approach (e.g.LZV)
- page will be compressed in batch mode (not on every change)
- page compression should been initiated by an event, for example, a
page’s free space drops below 20%
- compression process will be under page write lock

Vladimir Ozerov has written:

What we do not understand yet:
1) Granularity of compression algorithm.
1.1) It could be per-entry - i.e. we compress the whole entry content, but
respect boundaries between entries. E.g.: before - [ENTRY_1][ENTRY_2],
after - [COMPRESSED_ENTRY_1][COMPRESSED_ENTRY_2] (as opposed to [COMPRESSED
ENTRY_1 and ENTRY_2]).
v1.2) Or it could be per-field - i.e. we compress fields, but respect binary
object layout. First approach is simple, straightforward, and will give
acceptable compression rate, but we will have to compress the whole binary
object on every field access, what may ruin our SQL performance. Second
approach is more complex, we are not sure about it's compression rate, but
as BinaryObject structure is preserved, we will still have fast
constant-time per-field access.

I think there are advantages in both approaches and we will be able to
compare different approaches and algorithms after prototype
implementation.

Main approach in brief:
1) When page’s free space drops below 20% will be triggered compression event
2) Page will be locked by write lock
3) Page will be passed to page’s compressor implementation
4) Page will be replaced by compressed page

Whole object or a field reading:
1) If page marked as compressed then the page will be handled by
page’s compressor implementation, otherwise, it will be handled as
usual.

Thoughts?

Should we create new IEP and register tickets to start implementation?
This will allow us to watch for the feature progress and related
tasks.

[1]
http://apache-ignite-developers.2346864.n4.nabble.com/Data-compression-in-Ignite-tc20679.html

--
Taras Ledkov
Mail-To: tled...@gridgain.com

Re: Code inspection

We can use Sonar as instrument for code analysis and test coverage inspections.



> On 6 Mar 2018, at 11:28, Dmitriy Govorukhin  
> wrote:
> 
> Dmitriy,
> 
> As I understood, preview topic was of static code analysis in general.
> In this topic, I want to discuss only idea inspection rule.
> In future, of course, we can expаnd this rule to the TeamCity build.
> 
> On Tue, Mar 6, 2018 at 11:16 AM, Nikolay Izhikov 
> wrote:
> 
>> Hello, Igniters.
>> 
>> +1 to automatic code style tools.
>> 
>> Let's make it already!
>> Do we have a ticket for it?
>> 
>> Related discussion - http://apache-ignite-developers.2346864.n4.nabble.
>> com/Static-code-analysis-for-Java-td22195.html
>> 
>> В Вт, 06/03/2018 в 08:15 +, Dmitry Pavlov пишет:
>>> Hi Dmitriy,
>>> 
>>> I think we should resurrect thread about addition of code inspections,
>> and
>>> later we can enable automatic control step to TeamCity.
>>> 
>>> Could you help me to find it?
>>> 
>>> вт, 6 мар. 2018 г. в 11:11, Dmitriy Govorukhin <
>> dmitriy.govoruk...@gmail.com
 :
 Hi folks,
 
 Do we have 'inspection' [1] scheme for ignite?
 I see a lot of warnings in my code, and I guess it is because everyone
>> uses
 different schemes.
 
 Let's start the discussion.
 
 [1] IDEA inspection
 
 
>>

Re: Let's keep Apache Ignite docs up-to-date

Igniters,

Bumping up this discussion.

I have recently found out that we have this process for documenting new
releases [1] which looks quite ridiculous to me.

First, creating a copy of page with next-version suffix is inconvenient and
error-prone: the next-version page is not visible to anyone, moreover, all
suggested edits to current documentation will be lost after the page copy
is created. Second, the documentation changes should be transparent to
users, but now a regular user cannot even review upcoming changes until
they are granted a permission to see/edit hidden pages.

Unless we have very strong reasons to keep documentation on readme.io (by
strong I mean a feature that cannot be implemented using a VCS + doc
generator), I would at least spend some time piloting the 'keep docs in the
VCS' approach.

Thoughts?

[1] https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document

2017-11-02 10:07 GMT+03:00 Dmitry Pavlov :

> I don't like git docs idea, it will require to follow whole
> PR-review-process that requires long time. IMO it is odd work.
> If readme.io provides review process, I suggest to keep it as-is.
>
> чт, 2 нояб. 2017 г. в 9:57, Dmitriy Setrakyan :
>
> > On Wed, Nov 1, 2017 at 11:27 PM, Vladimir Ozerov 
> > wrote:
> >
> > > +1 for moving docs under Git provided that we find a way to update docs
> > > outside of AI release as it is possible now with readme.io.
> > >
> >
> > I am HUGE +1 for that. The whole problem is that we haven't found a way
> > yet. All I want is to update a page and have it commit to GIT and become
> > available to public right away. Does anyone know any tool that supports
> it?
> >
>

[GitHub] ignite pull request #3321: Ignite-2.4-java9

Github user agura closed the pull request at:

https://github.com/apache/ignite/pull/3321


---

[GitHub] ignite pull request #3313: Ignite-2.4-java8

Github user agura closed the pull request at:

https://github.com/apache/ignite/pull/3313


---

[GitHub] ignite pull request #3605: ignite-6643-2.3 Marshalling improvements

Github user agura closed the pull request at:

https://github.com/apache/ignite/pull/3605


---

[GitHub] ignite pull request #3604: ignite-6643-2.1Marshalling improvements (backport...

Github user agura closed the pull request at:

https://github.com/apache/ignite/pull/3604


---

Re: [VOTE] Apache Ignite 2.4.0 RC1

2018-03-06 Thread Alexey Kuznetsov

Pavel,

Web Console> Fixed output of big numbers in SQL query results

As one who fix this issue - that one was not so *trivial* :).


On Tue, Mar 6, 2018 at 3:05 PM, Vladimir Ozerov 
wrote:

> Pavel,
>
> There is no changes in Ignite C++. As far as Web Console fix, I would not
> say it is trivial because it affected UX, so IMO it is important to notify
> users about the fix.
>
> On Tue, Mar 6, 2018 at 10:30 AM, Pavel Tupitsyn 
> wrote:
>
> > Regarding release notes:
> >
> > 1. No changes in Ignite.C++?
> >
> > 2. > Fixed output of big numbers in SQL query results
> > Are we including trivial bugfixes now?
> >
> >
> >
> > On Tue, Mar 6, 2018 at 10:26 AM, Pavel Tupitsyn 
> > wrote:
> >
> > > +1
> > >
> > > Checked Ignite.NET:
> > > * Build from sources
> > > * Download binaries and run examples
> > > * Build NuGet packages on TeamCity [1] and test them
> > >
> > > [1] https://ci.ignite.apache.org/viewLog.html?buildId=1126956
> > >
> > > On Mon, Mar 5, 2018 at 10:42 PM, Vladimir Ozerov  >
> > > wrote:
> > >
> > >>  Igniters,
> > >>
> > >> We have uploaded a 2.4.0 release candidate to
> > >> https://dist.apache.org/repos/dist/dev/ignite/2.4.0-rc1/
> > >>
> > >> Git tag name is
> > >> 2.4.0-rc1
> > >>
> > >> This release includes the following changes:
> > >>
> > >> Ignite:
> > >> * Introduced Baseline Affinity Topology
> > >> * Ability to disable WAL for cache in runtime through IgniteCluster
> API
> > or
> > >> ALTER TABLE command
> > >> * Added ability to convert WAL to human-readable form
> > >> * Ability to enable/disable JDBC, ODBC and thin client endpoints
> > >> * Project moved to Java 8
> > >> * Java 7 is no longer supported
> > >> * Apache Ignite can be run on Java 9
> > >> * Introduced Apache Ignite packaging with RPM
> > >> * Spark Data Frames support
> > >> * Added integration with Spring 3.x
> > >> * Added handler to stop node when IO error is encountered
> > >> * EvictionPolicyFactory support added to CacheConfiguration
> > >> * Added JMX metrics for memory regions
> > >> * Added detailed memory consumption on start and OOM reporting
> > >> * Added custom thread pools monitoring
> > >> * Support Ignite MR in Kerberized environment without IGFS
> > >> * S3 IP finder: support server side encryption and bucket endpoint
> > >> configuration
> > >> * Multiple fixes and performance optimizations
> > >>
> > >> Ignite.NET:
> > >> * Thin Client initial release
> > >> * .NET Core / Linux / macOS support
> > >> * Mono platform support
> > >> * Removed C++ JNI layer
> > >>
> > >> SQL:
> > >> * Added DEFAULT support to CREATE TABLE command
> > >> * Added ALTER TABLE DROP COLUMN support
> > >> * Added INLINE_SIZE option to CREATE INDEX command
> > >> * Added PARALLEL option to CREATE INDEX command
> > >> * Added optional on-heap row cache
> > >> * INSERT command now can be executed without explicit column names
> > (INSERT
> > >> INTO table VALUES ...)
> > >> * Allowed multiple caches to share the same schema
> > >> * Added support for Java 8 Date and Time API
> > >> * Added "notNull" property to QuerySqlField annotation
> > >> * Improved COUNT(*) performance
> > >> * Fixed a problem causing an exception in certain nodes do not have
> > >> primary
> > >> partitions for REPLICATED cache
> > >> * Fixed per-thread H2 connection leak.
> > >> * Fixed partition exchange hang due to SQL query running inside a
> > >> transaction.
> > >>
> > >> JDBC Driver:
> > >> * Optimized batched operations processing
> > >>
> > >> ODBC Driver:
> > >> * Added support of multiple statements execution with one call
> > >> * Added support of SQL_ATTR_CONNECTION_TIMEOUT
> > >> * Added support of SQL_ATTR_QUERY_TIMEOUT
> > >> * Optimized batched operations processing
> > >> * Proper handling of ungraceful TCP disconnects (keep-alive
> connection)
> > >> * Fixed error preventing close of executed DML statement
> > >> * Fixed table names returned by SQLTables for DDL-created tables
> > >>
> > >> Machine Learning:
> > >> * Implemented Fuzzy c-means algorithm
> > >> * Implemented gradient descent for OLS Linear Regression
> > >> * Implemented K nearest neighbor algorithm
> > >> * Introduced API for datasets
> > >> * Introduced API for ML model and trainers
> > >> * Introduced common mechanism for group training of models
> > >> * Added distributed version of multilayer perceptron
> > >> * Added local version of multilayer perceptron
> > >> * Added basic import/export functionality for ml models
> > >> * Added decision tree algorithm
> > >> * Performance optimizations for distributed operations
> > >>
> > >> Web Console:
> > >> * Implemented component for cluster selection and activation
> > >> * Implemented support for multiple statements on Queries screen
> > >> * Implemented progress indication for 'execute', 'scan' and 'export'
> > >> actions on Queries screen
> > >> * Added support for ClientConnectorConfiguration
> > >> * Added several SQL

Re: Code inspection

2018-03-06 Thread Dmitriy Govorukhin

Dmitriy,

As I understood, preview topic was of static code analysis in general.
In this topic, I want to discuss only idea inspection rule.
In future, of course, we can expаnd this rule to the TeamCity build.

On Tue, Mar 6, 2018 at 11:16 AM, Nikolay Izhikov 
wrote:

> Hello, Igniters.
>
> +1 to automatic code style tools.
>
> Let's make it already!
> Do we have a ticket for it?
>
> Related discussion - http://apache-ignite-developers.2346864.n4.nabble.
> com/Static-code-analysis-for-Java-td22195.html
>
> В Вт, 06/03/2018 в 08:15 +, Dmitry Pavlov пишет:
> > Hi Dmitriy,
> >
> > I think we should resurrect thread about addition of code inspections,
> and
> > later we can enable automatic control step to TeamCity.
> >
> > Could you help me to find it?
> >
> > вт, 6 мар. 2018 г. в 11:11, Dmitriy Govorukhin <
> dmitriy.govoruk...@gmail.com
> > > :
> > > Hi folks,
> > >
> > > Do we have 'inspection' [1] scheme for ignite?
> > > I see a lot of warnings in my code, and I guess it is because everyone
> uses
> > > different schemes.
> > >
> > > Let's start the discussion.
> > >
> > > [1] IDEA inspection
> > > 
> > >
>

Re: Code inspection

Hello, Igniters.

+1 to automatic code style tools.

Let's make it already!
Do we have a ticket for it?

Related discussion - 
http://apache-ignite-developers.2346864.n4.nabble.com/Static-code-analysis-for-Java-td22195.html

В Вт, 06/03/2018 в 08:15 +, Dmitry Pavlov пишет:
> Hi Dmitriy,
> 
> I think we should resurrect thread about addition of code inspections, and
> later we can enable automatic control step to TeamCity.
> 
> Could you help me to find it?
> 
> вт, 6 мар. 2018 г. в 11:11, Dmitriy Govorukhin  > :
> > Hi folks,
> > 
> > Do we have 'inspection' [1] scheme for ignite?
> > I see a lot of warnings in my code, and I guess it is because everyone uses
> > different schemes.
> > 
> > Let's start the discussion.
> > 
> > [1] IDEA inspection
> > 
> > 

signature.asc
Description: This is a digitally signed message part

Re: Ignite nightly build job on Apache Jenkins server

Amount of effort will exceed creating new build on TeamCity.
Currently on TeamCity we have prepared and working Release Build project, which 
can be copied and modified to run at night.

A think we have to discuss the delivery of those nightly release artifacts to 
users: some king of page ’Nightly build' at ignite.apache.org 
 or alike.

> On 6 Mar 2018, at 11:05, Dmitriy Setrakyan  wrote:
> 
> On Tue, Mar 6, 2018 at 12:02 AM, Petr Ivanov  wrote:
> 
>> I guess there is no such thing as Nightly RELEASE Builds right now in
>> TeamCity.
>> 
>> We have only nightly test builds on master.
>> 
>> I guess there is not much effort in adding nightly release build, so, if
>> this task has it’s priority — please fill the ticket and I’ll prepare
>> corresponding build.
>> 
> 
> In the mean time, can we fix the Jenkin's build?

Re: Code inspection

Hi Dmitriy,

I think we should resurrect thread about addition of code inspections, and
later we can enable automatic control step to TeamCity.

Could you help me to find it?

вт, 6 мар. 2018 г. в 11:11, Dmitriy Govorukhin :

> Hi folks,
>
> Do we have 'inspection' [1] scheme for ignite?
> I see a lot of warnings in my code, and I guess it is because everyone uses
> different schemes.
>
> Let's start the discussion.
>
> [1] IDEA inspection
> 
>

Re: Ignite nightly build job on Apache Jenkins server

IMO we should migrate to one platform instead of fixing deprecated. Now
target platform is TeamCity.

вт, 6 мар. 2018 г. в 11:06, Dmitriy Setrakyan :

> On Tue, Mar 6, 2018 at 12:02 AM, Petr Ivanov  wrote:
>
> > I guess there is no such thing as Nightly RELEASE Builds right now in
> > TeamCity.
> >
> > We have only nightly test builds on master.
> >
> > I guess there is not much effort in adding nightly release build, so, if
> > this task has it’s priority — please fill the ticket and I’ll prepare
> > corresponding build.
> >
>
> In the mean time, can we fix the Jenkin's build?
>

Code inspection

2018-03-06 Thread Dmitriy Govorukhin

Hi folks,

Do we have 'inspection' [1] scheme for ignite?
I see a lot of warnings in my code, and I guess it is because everyone uses
different schemes.

Let's start the discussion.

[1] IDEA inspection

Re: [VOTE] Apache Ignite 2.4.0 RC1

2018-03-06 Thread Vladimir Ozerov

Pavel,

There is no changes in Ignite C++. As far as Web Console fix, I would not
say it is trivial because it affected UX, so IMO it is important to notify
users about the fix.

On Tue, Mar 6, 2018 at 10:30 AM, Pavel Tupitsyn 
wrote:

> Regarding release notes:
>
> 1. No changes in Ignite.C++?
>
> 2. > Fixed output of big numbers in SQL query results
> Are we including trivial bugfixes now?
>
>
>
> On Tue, Mar 6, 2018 at 10:26 AM, Pavel Tupitsyn 
> wrote:
>
> > +1
> >
> > Checked Ignite.NET:
> > * Build from sources
> > * Download binaries and run examples
> > * Build NuGet packages on TeamCity [1] and test them
> >
> > [1] https://ci.ignite.apache.org/viewLog.html?buildId=1126956
> >
> > On Mon, Mar 5, 2018 at 10:42 PM, Vladimir Ozerov 
> > wrote:
> >
> >>  Igniters,
> >>
> >> We have uploaded a 2.4.0 release candidate to
> >> https://dist.apache.org/repos/dist/dev/ignite/2.4.0-rc1/
> >>
> >> Git tag name is
> >> 2.4.0-rc1
> >>
> >> This release includes the following changes:
> >>
> >> Ignite:
> >> * Introduced Baseline Affinity Topology
> >> * Ability to disable WAL for cache in runtime through IgniteCluster API
> or
> >> ALTER TABLE command
> >> * Added ability to convert WAL to human-readable form
> >> * Ability to enable/disable JDBC, ODBC and thin client endpoints
> >> * Project moved to Java 8
> >> * Java 7 is no longer supported
> >> * Apache Ignite can be run on Java 9
> >> * Introduced Apache Ignite packaging with RPM
> >> * Spark Data Frames support
> >> * Added integration with Spring 3.x
> >> * Added handler to stop node when IO error is encountered
> >> * EvictionPolicyFactory support added to CacheConfiguration
> >> * Added JMX metrics for memory regions
> >> * Added detailed memory consumption on start and OOM reporting
> >> * Added custom thread pools monitoring
> >> * Support Ignite MR in Kerberized environment without IGFS
> >> * S3 IP finder: support server side encryption and bucket endpoint
> >> configuration
> >> * Multiple fixes and performance optimizations
> >>
> >> Ignite.NET:
> >> * Thin Client initial release
> >> * .NET Core / Linux / macOS support
> >> * Mono platform support
> >> * Removed C++ JNI layer
> >>
> >> SQL:
> >> * Added DEFAULT support to CREATE TABLE command
> >> * Added ALTER TABLE DROP COLUMN support
> >> * Added INLINE_SIZE option to CREATE INDEX command
> >> * Added PARALLEL option to CREATE INDEX command
> >> * Added optional on-heap row cache
> >> * INSERT command now can be executed without explicit column names
> (INSERT
> >> INTO table VALUES ...)
> >> * Allowed multiple caches to share the same schema
> >> * Added support for Java 8 Date and Time API
> >> * Added "notNull" property to QuerySqlField annotation
> >> * Improved COUNT(*) performance
> >> * Fixed a problem causing an exception in certain nodes do not have
> >> primary
> >> partitions for REPLICATED cache
> >> * Fixed per-thread H2 connection leak.
> >> * Fixed partition exchange hang due to SQL query running inside a
> >> transaction.
> >>
> >> JDBC Driver:
> >> * Optimized batched operations processing
> >>
> >> ODBC Driver:
> >> * Added support of multiple statements execution with one call
> >> * Added support of SQL_ATTR_CONNECTION_TIMEOUT
> >> * Added support of SQL_ATTR_QUERY_TIMEOUT
> >> * Optimized batched operations processing
> >> * Proper handling of ungraceful TCP disconnects (keep-alive connection)
> >> * Fixed error preventing close of executed DML statement
> >> * Fixed table names returned by SQLTables for DDL-created tables
> >>
> >> Machine Learning:
> >> * Implemented Fuzzy c-means algorithm
> >> * Implemented gradient descent for OLS Linear Regression
> >> * Implemented K nearest neighbor algorithm
> >> * Introduced API for datasets
> >> * Introduced API for ML model and trainers
> >> * Introduced common mechanism for group training of models
> >> * Added distributed version of multilayer perceptron
> >> * Added local version of multilayer perceptron
> >> * Added basic import/export functionality for ml models
> >> * Added decision tree algorithm
> >> * Performance optimizations for distributed operations
> >>
> >> Web Console:
> >> * Implemented component for cluster selection and activation
> >> * Implemented support for multiple statements on Queries screen
> >> * Implemented progress indication for 'execute', 'scan' and 'export'
> >> actions on Queries screen
> >> * Added support for ClientConnectorConfiguration
> >> * Added several SQL examples on Queries screen
> >> * Added "Connected clusters" component in header
> >> * Added support for custom SMTP server configuration
> >> * Added detection of CSV separator based on browser locale
> >> * Added "Copy to clipboard" action on Queries screen
> >> * Added duration and node ID in results header and 'Show query' modal
> >> * Improved support for binary JDBC types on import from RDBMS
> >> * Fixed Web Console under IE11
> >> * Fixed configuration

Re: Ignite nightly build job on Apache Jenkins server

On Tue, Mar 6, 2018 at 12:02 AM, Petr Ivanov  wrote:

> I guess there is no such thing as Nightly RELEASE Builds right now in
> TeamCity.
>
> We have only nightly test builds on master.
>
> I guess there is not much effort in adding nightly release build, so, if
> this task has it’s priority — please fill the ticket and I’ll prepare
> corresponding build.
>

In the mean time, can we fix the Jenkin's build?

Re: Ignite nightly build job on Apache Jenkins server