Re: Transport compression (not store compression)

[Nikita Amelchev]

Hello, Igniters.

I have not seen such use-cases, where heavy client ignite node placed in a
much worse network than the server. I'm not sure we should encourage a bad
cluster architecture.

Usually, in my use-cases, the servers and clients locate in the same
network. And if the cluster has SSL enabled, it makes sense to enable
compression, even if the network is fast. It also makes sense when we have
a high load on the network, and the CPU is utilized poorly.

I'll do tests on yardstick for real operations like get, put etc. and SQL
requests.

I propose to add configurable compression for thin client/ODBC/JDBC as a
separate issue because it increases the current PR.

Even if it really makes sense to compress the traffic only between
client-server ignite nodes, it should also be a separate issue, that would
not increase the PR. Especially, since this compression architecture may
not be accepted by the community.

2018-02-05 13:02 GMT+03:00 Nikita Amelchev :

> Thanks for your comments,
>
> I will try to separate network compression for clients and servers.
>
> It makes sense to enable compression on servers if we have SSL turned on.
> I tested rebalancing time and compression+ssl is faster. SSL throughput is
> limited by 800 Mbits/sec per connection and if enable compression, it
> boosted up to 1100 Mbits.
>
> 2018-02-02 18:52 GMT+03:00 Alexey Kuznetsov :
>
>> I think Igor is right.
>>
>> Ususally servers connected via fast local network.
>> But clients could be in external and slow network.
>> In this scenario compression will be very useful.
>>
>> Once I had such scenario - client connected to cluster via 300 kb/s
>> network
>> and tries to transfer ~10Mb of uncumpressed data.
>> So it takse ~30 seconds.
>> After I implemented compression it becamed 1M and transfered for ~3
>> seconds.
>>
>> I think we should take care of all mentioned problems with NIO threads in
>> order to not slow down whole cluster.
>>
>>
>> On Fri, Feb 2, 2018 at 10:05 PM, gvvinblade  wrote:
>>
>> > Nikita,
>> >
>> > Yes, you're right. Maybe I wasn't clear enough.
>> >
>> > Usually server nodes are placed in the same fast network segment (one
>> > datacenter); in any case we need an ability to setup compression per
>> > connection using some filter like useCompression(ClusterNode,
>> ClusterNode)
>> > to compress traffic only between servers and client nodes.
>> >
>> > But issue is still there, since the same NIO worker serves both client
>> and
>> > server connections, enabled compression may impact whole cluster
>> > performance
>> > because NIO threads will compress client messages instead of processing
>> > servers' compute requests. That was my concern.
>> >
>> > Compression for clients is really cool feature and usefull in some
>> cases.
>> > Probably it makes sense to have two NIO servers with and without
>> > compression
>> > to process server and client requests separately or pin somehow worker
>> > threads to client or server sessions...
>> >
>> > Also we have to think about client connections (JDBC, ODBC, .Net thin
>> > client, etc) and setup compression for them separately.
>> >
>> > Anyway I would compare put, get, putAll, getAll and SQL SELECT
>> operations
>> > for strings and POJOs, one server, several clients with and without
>> > compression, setting up the server to utilize all cores by NIO workers,
>> > just
>> > to get know possible impact.
>> >
>> > Possible configuration for servers with 16 cores:
>> >
>> > Selectors cnt = 16
>> > Connections per node = 4
>> >
>> > Where client nodes perform operations in 16 threads
>> >
>> > Regards,
>> > Igor
>> >
>> >
>> >
>> >
>> > --
>> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>> >
>>
>>
>>
>> --
>> Alexey Kuznetsov
>>
>
>
>
> --
> Best wishes,
> Amelchev Nikita
>



-- 
Best wishes,
Amelchev Nikita

[GitHub] ignite pull request #3507: IGNITE-7652: ContinuousQueryWithTransformer examp...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3507


---

.NET: ISslContextFactory should be empty

[Pavel Tupitsyn]

Igniters, Alexey, Igor,

I'd like to discuss recently added ISslContextFactory interface.
https://github.com/apache/ignite/commit/e58aae48b67c74307703d2ae44fe8e3cd4b9649a

"Factory" implies a "CreateInstace" or a similar method, see
CacheConfigurationCacheStoreFactory
or ExpiryPolicyFactory as an example.

Another case is a placeholder interface that maps to some Java abstraction
but has no .NET counterpart (yet), an example is IEvictionPolicy.

In case of ISslContextFactory we have a similar situation, so the interface
should be empty.
Key store details do not belong there.

Thoughts?

PS Please add me as a reviewer for .NET tickets

Thanks,
Pavel

[jira] [Created] (IGNITE-7711) Explain the case-sensitivity of schema and table/cache

[Denis Magda (JIRA)]

Denis Magda created IGNITE-7711:
---

 Summary: Explain the case-sensitivity of schema and table/cache
 Key: IGNITE-7711
 URL: https://issues.apache.org/jira/browse/IGNITE-7711
 Project: Ignite
  Issue Type: Task
  Components: documentation
Reporter: Denis Magda
 Fix For: 2.5


Users don't get how to use table and schema name in Ignite:

[https://stackoverflow.com/questions/48723946/apache-ignite-querying-a-cache-through-an-sql-driver]

The confusion comes from the interpretation of term "case-insensitive". All SQL 
databases work this way:
 
1) If I create some object and define its name without quotes, a database will 
convert it to upper case (sometimes lower case). Then if I query this object 
without quotes, a query is also transformed to upper case. This is what I call 
"case-insensitivity" - it doesn't matter how you refer to this object, it works 
still.
CREATE TABLE myTable -> CREATE TABLE MYTABLE
SELECT FROM myTable -> SELECT FROM MYTABLE -> OK
SELECT FROM MyTable -> SELECT FROM MYTABLE -> OK
SELECT FROM "myTable" -> SELECT FROM myTable -> *{color:#ff}FAIL!{color}*
 
2) If I create an object with quotes, then the database doesn't convert it to 
upper case, and the original name in quotes is *the only way* to access the 
object:
CREATE TABLE "myTable" -> CREATE TABLE myTable
SELECT FROM myTable -> SELECT FROM MYTABLE -> *{color:#ff}FAIL!{color}*
SELECT FROM MyTable -> SELECT FROM MYTABLE -> 
*{color:#ff}FAIL!{color}*SELECT FROM "myTable" -> SELECT FROM myTable -> OK
 
Now back to Ignite case. Our schemas are case-sensitive. so the only way to 
access it is through quotes:
CREATE SCHEMA "cacheName"; // This is what Ignite does internally
SELECT * FROM "cacheName".myTable -> OK
SELECT * FROM cacheName.myTable -> *{color:#ff}FAIL!{color}*
 
Our users know this, and all their queries are written with schema names in 
quotation marks. Now consider what would happen should we change schema names 
to case-insensitive mode:
CREATE SCHEMA cacheName;

SELECT * FROM "cacheName".myTable -> *{color:#ff}FAIL, +all old quries 
start to fail!+{color}*

SELECT * FROM cacheName.myTable -> OK, now it works



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

SQL Compliance documentation

[Prachi Garg]

Igniters,

Apache Ignite's compliance to SQL:1999 (Core) is now documented on readme.io[1]
as well as on Wikipedia[2]. SQL folks, check it out and let me know if
there needs to be any correction.

Thanks to Aleksandr Volkov for validating Ignite's compliance with SQL99
Core specification.

[1] https://apacheignite-sql.readme.io/docs/sql-conformance
[2] https://en.wikipedia.org/wiki/SQL_compliance


-Prachi

Re: API to enlist running user tasks

[Dmitriy Setrakyan]

On Tue, Feb 13, 2018 at 8:22 PM, Nikolay Izhikov 
wrote:

> Dmitriy
>
> I thought about those type of tasks that require custom user code to be
> executed inside Ignite.
>
> Your suggestion make sense for me also. Let's include SQL queries.
> Should we include ScanQuery, TextQuery to this API?
>

I do not see any reason why not. Let's do it, if possible.

[jira] [Created] (IGNITE-7710) copy from csv fails to find existing table (intermittently)

[Pavel Kuznetsov (JIRA)]

Pavel Kuznetsov created IGNITE-7710:
---

 Summary: copy from csv fails to find existing table 
(intermittently)
 Key: IGNITE-7710
 URL: https://issues.apache.org/jira/browse/IGNITE-7710
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Pavel Kuznetsov
Assignee: Kirill Shirokov


1) create table via new SqlFieldsQuery()
2) perform sql via thin driver 
{{COPY FROM "/tmp/mydata.csv" INTO table_name FORMAT CSV;}}

client gets:
{{java.sql.SQLException: Table does not exist: TEST_UPLOAD
   at 
org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:653)
   at 
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:136)
   at 
org.apache.ignite.internal.jdbc.thin.JdbcThinPreparedStatement.executeWithArguments(JdbcThinPreparedStatement.java:252)
   at 
org.apache.ignite.internal.jdbc.thin.JdbcThinPreparedStatement.executeUpdate(JdbcThinPreparedStatement.java:96)
   at 
org.apache.ignite.yardstick.upload.CopyBenchmark.warmup(CopyBenchmark.java:67)
   at 
org.apache.ignite.yardstick.upload.AbstractUploadBenchmark.setUp(AbstractUploadBenchmark.java:50)
   at 
org.yardstickframework.BenchmarkDriverStartUp.main(BenchmarkDriverStartUp.java:130)}}

server gets
{{[2018-02-14 19:57:47,840][ERROR][client-connector-#47][JdbcRequestHandler] 
Failed to execute SQL query [reqId=0, req=JdbcQueryExecuteRequest 
[schemaName=PUBLIC, pageSize=1024, maxRows=0, sqlQry=COPY FROM 
"/tmp/warmup1971356426935
538953.csv" INTO TEST_UPLOAD (id, val_1, val_2, val_3, val_4, val_5, val_6, 
val_7, val_8, val_9, val_10) FORMAT CSV;, args=[], stmtType=UPDATE_STMT_TYPE]]
class org.apache.ignite.internal.processors.query.IgniteSQLException: Table 
does not exist: TEST_UPLOAD
   at 
org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.processBulkLoadCommand(DmlStatementsProcessor.java:1020)
   at 
org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.runNativeDmlStatement(DmlStatementsProcessor.java:992)
   at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.tryQueryDistributedSqlFieldsNative(IgniteH2Indexing.java:1482)
   at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1518)
   at 
org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2037)
   at 
org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2032)
   at 
org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
   at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2553)
   at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2046)
   at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:1977)
   at 
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.executeQuery(JdbcRequestHandler.java:374)
   at 
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.handle(JdbcRequestHandler.java:179)
   at 
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:156)
   at 
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:41)
   at 
org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
   at 
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
   at 
org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97)
   at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
   at 
org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70)
   at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at java.lang.Thread.run(Thread.java:748)}}




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Fwd: Travel Assistance applications open. Please inform your communities

[Denis Magda]

Folks, 

If anyone is willing to go and present Ignite in Canada then please enroll.

I’ll can share a dozen of Ignite related slides/topics that you can use for 
your talks.

—
Denis

> Begin forwarded message:
> 
> From: Gavin McDonald 
> Subject: Travel Assistance applications open. Please inform your communities
> Date: February 14, 2018 at 1:34:11 AM PST
> To: travel-assista...@apache.org
> Reply-To: priv...@ignite.apache.org
> Reply-To: travel-assista...@apache.org
> 
> Hello PMCs.
> 
> Please could you forward on the below email to your dev and user lists.
> 
> Thanks
> 
> Gav…
> 
> —
> The Travel Assistance Committee (TAC) are pleased to announce that travel 
> assistance applications for ApacheCon NA 2018 are now open!
> 
> We will be supporting ApacheCon NA Montreal, Canada on 24th - 29th September 
> 2018
> 
>  TAC exists to help those that would like to attend ApacheCon events, but are 
> unable to do so for financial reasons. 
> For more info on this years applications and qualifying criteria, please 
> visit the TAC website at < http://www.apache.org/travel/ 
>  >. Applications are now open and will close 
> 1st May. 
> 
> Important: Applications close on May 1st, 2018. Applicants have until the 
> closing date above to submit their applications (which should contain as much 
> supporting material as required to efficiently and accurately process their 
> request), this will enable TAC to announce successful awards shortly 
> afterwards. 
> 
> As usual, TAC expects to deal with a range of applications from a diverse 
> range of backgrounds. We therefore encourage (as always) anyone thinking 
> about sending in an application to do so ASAP.   
> We look forward to greeting many of you in Montreal
> 
> Kind Regards, 
> Gavin - (On behalf of the Travel Assistance Committee)
> — 
> 
> 
> 

[GitHub] ignite pull request #3528: IGNITE-7681: SQL COPY: Implementing performance i...

[gg-shq]

GitHub user gg-shq opened a pull request:

https://github.com/apache/ignite/pull/3528

IGNITE-7681: SQL COPY: Implementing performance improvements



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7681-2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3528.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3528


commit 4235bc09ef5619019af4d2b646b0881ca135d948
Author: shq 
Date:   2018-02-12T23:05:51Z

IGNITE-7681: Implementing performance improvements

commit 15a274af9d86fa5134c41dd52046533a08cbc6d2
Author: shq 
Date:   2018-02-13T13:28:43Z

IGNITE-7681: Implementing performance improvements

commit 7eec88901c5da2b4d26bfb6a79c4276dd81db887
Author: shq 
Date:   2018-02-13T15:12:45Z

IGNITE-7681: Implementing performance improvements

commit 93d944e6529f717d06eeb7d7f549f256f56fc9d9
Author: shq 
Date:   2018-02-13T16:55:09Z

IGNITE-7681: Implementing performance improvements

commit c608474712eff7626f5649f14793dbd6d5823094
Author: shq 
Date:   2018-02-13T18:39:05Z

IGNITE-7681: Implementing performance improvements

commit cc2c14af4d32927c4b0368f6758cc6591abad1bb
Author: gg-shq 
Date:   2018-02-14T18:08:23Z

IGNITE-7681: COPY performance improvements + a performance test

commit 0859d5558559d9c4abe8f09fe1de1cd6df7c7db0
Author: gg-shq 
Date:   2018-02-14T18:21:50Z

IGNITE-7681: Shining COPY performance improvements + a performance test




---

[GitHub] ignite pull request #3527: IGNITE-7681: SQL COPY performance fixes and perfo...

[gg-shq]

Github user gg-shq closed the pull request at:

https://github.com/apache/ignite/pull/3527


---

[GitHub] ignite pull request #3527: IGNITE-7681: SQL COPY performance fixes and perfo...

[gg-shq]

GitHub user gg-shq opened a pull request:

https://github.com/apache/ignite/pull/3527

IGNITE-7681: SQL COPY performance fixes and performance test



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7681

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3527.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3527


commit ff40ecfcd59a307fd26f7ff73227e6b8c5eafb57
Author: shq 
Date:   2018-02-12T23:05:51Z

IGNITE-7681: Intermediate commit

commit 6e7f55df00ee17e3b650f1c0f45467250355d591
Author: shq 
Date:   2018-02-13T13:28:43Z

IGNITE-7681: Intermediate commit

commit 6a38e35e98de0bbb41972580cb74a862af5be1ca
Author: shq 
Date:   2018-02-13T15:12:45Z

IGNITE-7681: Intermediate commit

commit 4184c26accdfe46d7f52baa6bc0e73424ec240c3
Author: shq 
Date:   2018-02-13T16:55:09Z

IGNITE-7681: Intermediate commit

commit ff76b744c6ebc4af6ebf9a0c393975026082ccd9
Author: shq 
Date:   2018-02-13T18:39:05Z

IGNITE-7681: Intermediate commit

commit fa112147ea930dfd6feb8f89fcfa14a25d0de6d0
Author: gg-shq 
Date:   2018-02-14T18:08:23Z

IGNITE-7681: COPY performance improvements + a performance test

commit 47ad0b8a329fe7a6a6186c7bd9afc725d393c091
Author: apopov 
Date:   2018-02-13T10:13:15Z

IGNITE-3111 .NET can be now configured SSL without Spring

This closes #3498

commit 3b17dd05dd0925d3c8352c0e2d72cad133cdd79c
Author: Andrey Kuznetsov 
Date:   2018-02-13T12:14:15Z

IGNITE-7513 Get rid of org.jsr166.ConcurrentHashMap8

Signed-off-by: Anton Vinogradov 

commit 2d279f2ac611cbef07fc2e88c2ee2297925ace06
Author: Aleksey Plekhanov 
Date:   2018-02-13T13:43:49Z

IGNITE-5265 Eviction Rate memory metric to be implemented

Signed-off-by: Anton Vinogradov 

commit ce0d3190b84ce77174de9fd9a29c6dfc5a1abd2e
Author: Alexey Goncharuk 
Date:   2018-02-13T15:53:23Z

IGNITE-7692 Corrected test to not fail when SQL is executed on backup

commit 145133ef24250d4b6c09310e92b75620892d95da
Author: dpavlov 
Date:   2018-02-13T17:19:31Z

IGNITE-7695: Enable Ignite Update Notifier tests - Fixes #3516.

Signed-off-by: Alexey Goncharuk 

commit 8fe4afe07488d7d191ababb2c92aeed0c486924c
Author: Vyacheslav Daradur 
Date:   2018-02-13T21:38:03Z

IGNITE-7588 - Deprecated @CacheLocalStore annotation - Fixes #3517

Signed-off-by: Valentin Kulichenko 

commit 06fe9e29c783fc2e077da955b3a7ceedf0287276
Author: Ivan Rakov 
Date:   2018-02-14T07:52:05Z

IGNITE-7690 Move shared memory suite 
(IpcSharedMemoryCrashDetectionSelfTest) to Ignite Basic 2 - Fixes #3518.

Signed-off-by: Alexey Goncharuk 

commit e57cfd24508de7bd00a61f6cbe40a8426179b780
Author: Alexey Goncharuk 
Date:   2018-02-14T08:52:04Z

IGNITE-7689 Corrected awaitPartitionMapExchange to prevent assertion on 
readyTopologyVersion() access

commit a8d10159ba7a0c9037fda6331e4edc9f6ec3
Author: zaleslaw 
Date:   2018-02-14T11:57:49Z

IGNITE-7452: Make Linear SVM example for multi - classification

this closes #3352

commit 57152ff510e2b0c7e559ffc5a9b4eb1d20c492fb
Author: Andrey Kuznetsov 
Date:   2018-02-14T16:08:46Z

IGNITE-7312 Make use of plain java.util.Base64 instead of reflective 
alternatives

Signed-off-by: Anton Vinogradov 

commit 74dfa4f18f3d94bfd929c53960330f0664fa4b6e
Author: gg-shq 
Date:   2018-02-14T18:21:50Z

IGNITE-7709: Shining SQL COPY file name handling fix for review




---

[GitHub] ignite pull request #3526: IGNITE-7709: SQL COPY file name handling fix

[gg-shq]

GitHub user gg-shq opened a pull request:

https://github.com/apache/ignite/pull/3526

IGNITE-7709: SQL COPY file name handling fix



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7709

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3526.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3526


commit dcf4f2dde48ad256d15bb170aae428cb807ac964
Author: gg-shq 
Date:   2018-02-14T18:11:24Z

IGNITE-7709: SQL COPY file name handling fix




---

[jira] [Created] (IGNITE-7709) SQL COPY: fix file name handling

[Kirill Shirokov (JIRA)]

Kirill Shirokov created IGNITE-7709:
---

 Summary: SQL COPY: fix file name handling
 Key: IGNITE-7709
 URL: https://issues.apache.org/jira/browse/IGNITE-7709
 Project: Ignite
  Issue Type: Bug
  Components: sql
Affects Versions: 2.3, 2.4, 2.5
Reporter: Kirill Shirokov
Assignee: Kirill Shirokov


Currently file name is parsed as identifier, which is obviously a bug.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Ignite work directory usage?

[Ilya Kasnacheev]

> because mappings generated by different nodes are supposed to be always
the same

Most file systems have atomic rename(). This means we should have two-phase
write of files into the folder in question. Step one - create
".class.tmp" file, write data into it; Step two - rename file to
".class". Catch "already used" exception and ignore it.

This way weird exceptions are avoided. Should I create a ticket about it?

-- 
Ilya Kasnacheev

2018-02-09 22:39 GMT+03:00 Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Dmitry,
>
> I meant the persistence store itself, but just realized that we don't have
> a marshaller cache anymore, we use discovery messages instead. However, we
> still have the MarshallerMappingFileStore which is basically a persistence
> space created specifically for marshaller mappings. I think it would be a
> good idea to use something more generic for this (although this is not
> critical of course).
>
> In any case, my initial point was that using the same folder by different
> nodes for these mappings should not be an issue, because mappings generated
> by different nodes are supposed to be always the same. We just need to
> avoid weird exceptions.
>
> -Val
>
> On Fri, Feb 9, 2018 at 1:21 AM, Dmitry Pavlov 
> wrote:
>
> > Hi Val,
> >
> > Do you mean by
> > > switching marshaller cache to persistence instead of using these files
> > makes perfect sense to me,
> > using 'metastore' for marshaller cache?
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пт, 9 февр. 2018 г. в 1:05, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> >
> > > Sergey,
> > >
> > > These mappings are supposed to be the same on all nodes, so if the file
> > > already exists, we can safely ignore this, or use a lock to avoid
> > > concurrent access. Actually, I think we already fixed this in the past,
> > > it's weird that issue came up again.
> > >
> > > But in any case, switching marshaller cache to persistence instead of
> > using
> > > these files makes perfect sense to me, and we definitely should do
> that.
> > >
> > > -Val
> > >
> > > On Tue, Feb 6, 2018 at 1:04 AM, Sergey Chugunov <
> > sergey.chugu...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Folks,
> > > >
> > > > There are several things here.
> > > >
> > > > Firstly user asked the initial question is sitting on Apache 1.x; it
> is
> > > > clear from exception stack trace he provided.
> > > > So although the issue exists for a while it looks like it doesn't
> hurt
> > > > users a lot.
> > > >
> > > > Secondly I examined the code managing marshaller mappings and can say
> > > that
> > > > it the issue is still here even in the latest version; thus I filed a
> > > > ticket [1] to address it.
> > > >
> > > > ​[1] https://issues.apache.org/jira/browse/IGNITE-7635
> > > >
> > > > Thanks,
> > > > Sergey.
> > > >
> > >
> >
>

[GitHub] ignite pull request #3525: Ignite 7517

[andrey-kuznetsov]

GitHub user andrey-kuznetsov opened a pull request:

https://github.com/apache/ignite/pull/3525

Ignite 7517



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrey-kuznetsov/ignite ignite-7517

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3525.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3525


commit d970629c7f527e958ffb4c2e46b3917579c668db
Author: Andrey Kuznetsov 
Date:   2018-02-12T16:05:37Z

IGNITE-7517: WIP.

commit dfde3d9c89f1969701f86233a1d794b379db2cc1
Author: Andrey Kuznetsov 
Date:   2018-02-14T08:58:57Z

IGNITE-7517: WIP.

commit e3b5cf0c3a64a17193f1750e57c32eca1e0eccf1
Author: Andrey Kuznetsov 
Date:   2018-02-14T16:09:34Z

IGNITE-7517: Removed ConcurrentLinkedDeque8.

commit 4d956e89489515776d535b5ae5c055fe002c45a0
Author: Andrey Kuznetsov 
Date:   2018-02-14T16:41:37Z

Merge branch 'master' into ignite-7517

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/igfs/common/IgfsLogger.java
#   
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoManager.java
#   
modules/core/src/main/java/org/apache/ignite/internal/managers/deployment/GridDeploymentLocalStore.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheMvccManager.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridDeferredAckMessageSender.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtLocalPartition.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxManager.java
#   
modules/core/src/main/java/org/apache/ignite/spi/collision/jobstealing/JobStealingCollisionSpi.java
#   
modules/hadoop/src/main/java/org/apache/ignite/internal/processors/hadoop/taskexecutor/external/HadoopExternalTaskExecutor.java




---

Webinar: "Getting Started with Apache Ignite as a Distributed Database (Today at 1 a.m. Pacific)

[Tom Diederich]

Igniters, Valentin Kulichenko is hosting a free webinar today at 11 a.m.. 
Pacific time that will give you the tools and blueprint to build your very own 
distributed database using Apache Ignite. This webinar will be recorded. You 
can register to addend the event live or catch the recording from this same 
link .

Re: IGNITE-5714 Context switching for pessimistic transactions

[ALEKSEY KUZNETSOV]

1) The essential problem lies in *IgniteTxAdapter#threadId*.
Thread id is set when transaction is created and afterwards is transferred
between nodes by GridDistributedTx requests\responses
when we perform put\get operations.
When we suspend and resume transaction, thread id is got changed locally,
but not on remote nodes.

Actually, we don't need local tx thread id on remote nodes.

This brings us to the solution. Remove thread id from prepare\finish
requests\responses and don't use it remotely.
But preserve it in lock requests, so long as cache lock uses thread id on
remote nodes. Such changes let us not send thread id to remote nodes,
and eliminate the core problem.
Thread id is moved from global IgniteTxAdapter to GridNearTxLocal, as long
as only *near local* transaction is need it.
Now, only candidates, created by near tx local have got thread id
defined.(i.e. remote candidates have got thread id undefined)
Thread id in GridCacheTxFinishSync is replaced with version check(so we
don't need to store and send thread id along with finish response).
When pessimistic tx is resumed, then candidates are updated with new thread
id.
Lock check in IgniteTxAdapter#ownsLock now is done against version(not
thread id, because version is unique).

2) No new messages needed for near lock requests, because thread id is
preserved for cache lock requests.
We still need to replace thread id with version check in TxFinishSync in
other solution.
In other solution we need to change cache lock code : replace thread id
with new tx counter.
In another solution we still need to remove thread id from
requests\responses.

If my design is OK I will do benchmarking

вт, 13 февр. 2018 г. в 18:22, Nikolay Izhikov :

> Hello, Alexey.
>
> Could you please, write little more about your implementation
>
> 1. Design of implementation.
>
> 2. Advantages of you implementation in compare with other ways?
>
> 3. Transactions performance penalties/improvements?
>
> В Вт, 13/02/2018 в 14:17 +, ALEKSEY KUZNETSOV пишет:
> >  Hi, Igniters!
> >
> >  Currently we have context switching implemented for optimistic
> > transactions [1].
> >
> >  Goal of the current ticket is to support transaction suspend()\resume()
> > operations for pessimistic transactions.
> >
> >  The essential problem with them lies in *IgniteTxAdapter#threadId*.
> >  Thread id is set when transaction is created and afterwards is
> transferred
> > between nodes by GridDistributedTx requests\responses when we perform
> > put\get operations.
> >  When we suspend and resume transaction, thread id is got changed
> locally,
> > but not on remote nodes.
> >
> >  In ticket I decided to partly remove thread id from source, and
> introduced
> > *undefined* value for it, where its value must be ignored.
> >  Another solution can be to replace thread id usage with some new global
> > transaction id counter.
> >
> >  The former solution has advantages :
> >  compatibility is preserved, step-by-step clear implementation, minimal
> > changes to explicit cache lock work(it still creates candidates with
> > not-null thread id) as opposed to the last solution.
> >
> >  There are 3 possible solutions to "thread id on remote nodes" issue :
> >  1) Change thread id on remote nodes once suspend()\resume() is called.
> >  2) Get rid of sending thread id to remote nodes.
> >  3) Don’t remove the field, just put -1 (undefined) in it.
> >
> >  The last option was chosen, because it will save compatibility in
> cluster
> > with nodes of older versions.
> >  Note that still outside the transaction, when explicit cache lock is
> > created, thread id is set not null value in lock request(i.e.
> > GridNearLockRequest).
> >
> >  Thread id is moved from global IgniteTxAdapter to GridNearTxLocal, as
> long
> > as only *near local* transaction is need it.
> >  For instance, when local candidate(either near local or dht local) is
> > created for GridNearTxLocal. Note that remote candidates are created with
> > thread id undefined, because it useless for non-local candidates.
> >  In IgniteTxAdapter#ownsLock thread id is replaced with tx version check.
> >  We could do it, because near transactions has got unique versions to
> check
> > against.
> >
> >  In tx synchronizer GridCacheTxFinishSync thread id is replaced with tx
> > version, so we don't need to store it and send by GridFinishResponse
> > messages.
> >  As a consequence, thread id is also removed from grid near
> finish\prepare
> > request\response.
> >
> >  Also, thread id information is removed from deadlock messages (in
> > TxDeadlock, TxDeadlockDetection).
> >
> > Please, review it:
> >
> > ticket *https://issues.apache.org/jira/browse/IGNITE-5714
> > *
> > pull request *https://github.com/apache/ignite/pull/2789
> > *
> > review https://reviews.ignite.apache.org/ignite/review/IGNT-CR-364
> >
> >  [1] : 

[jira] [Created] (IGNITE-7708) Ignite Compute has flaky failures

[Dmitriy Pavlov (JIRA)]

Dmitriy Pavlov created IGNITE-7708:
--

 Summary: Ignite Compute has flaky failures 
 Key: IGNITE-7708
 URL: https://issues.apache.org/jira/browse/IGNITE-7708
 Project: Ignite
  Issue Type: Bug
  Components: compute
Reporter: Dmitriy Pavlov


https://ci.ignite.apache.org/viewLog.html?buildId=1093187=IgniteTests24Java8_IgniteComputeGrid
 

https://ci.ignite.apache.org/viewLog.html?buildId=1092214=IgniteTests24Java8_IgniteComputeGrid

IgniteBinaryObjectsComputeGridTestSuite

IgniteComputeBasicConfigVariationsFullApiTestSuite

it is required to fix flaky failure or remove tests from  suite because of 
default notifications were enabled recently.

Flaky failures generates a lot of unecessary spam.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7707) Read lock for key/keys

[Mikhail Cherkasov (JIRA)]

Mikhail Cherkasov created IGNITE-7707:
-

 Summary: Read lock for key/keys
 Key: IGNITE-7707
 URL: https://issues.apache.org/jira/browse/IGNITE-7707
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Reporter: Mikhail Cherkasov


Right now, there's no way to take read lock for key or keys, so if you need to 
process several keys you can only lock this key in a transaction or by lockAll, 
but in this case, you can not read process this in parallel. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7706) Data streamer hangs due to uploading instances of updated class

[Roman Guseinov (JIRA)]

Roman Guseinov created IGNITE-7706:
--

 Summary: Data streamer hangs due to uploading instances of updated 
class
 Key: IGNITE-7706
 URL: https://issues.apache.org/jira/browse/IGNITE-7706
 Project: Ignite
  Issue Type: Bug
  Components: streaming
Reporter: Roman Guseinov


Use case:
1. Cache is located only at specific nodes GROUP-1.
2. Other nodes GROUP-2 is used for other caches.
3. Also, there are nodes for streaming GROUP-3. Cache configuration contains 
Person class as value type of QueryEntity.

If we restart GROUP-1 (not all cluster) and restart GROUP-3 with updated 
configuration and Person class (changed a field type) then streamer hangs 
(without any messages).

Reproducer (with README.md) in the attached file (streamer-hangs-issue.zip).

It seems the problem in handling "Binary type has different field types" 
exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Apache EU Roadshow CFP Closing Soon (23 February)

[Sharan F]

Hello Everyone

This is an initial reminder to let you all know that we are holding an 
Apache EU Roadshow co-located with FOSS Backstage in Berlin on 13^th and 
14^th June 2018. https://s.apache.org/tCHx


The Call for Proposals (CFP) for the Apache EU Roadshow is currently 
open and will close at the end of next week, so if you have been 
delaying making a submission because the closing date seemed a long way 
off, then it's time to start getting your proposals submitted.


So what are we looking for?
We will have 2 Apache Devrooms available during the 2 day Roadshow so 
are looking for projects including incubating ones, to submit 
presentations, panel discussions, BoFs, or workshop proposals. The main 
focus of the Roadshow will be IoT, Cloud, Httpd and Tomcat so if your 
project is involved in or around any of these technologies at Apache 
then we are very interested in hearing from you.


Community and collaboration is important at Apache so if your project is 
interested in organising a project sprint, meetup or hackathon during 
the Roadshow, then please submit it inthe CFP as we do have some space 
available to allocate for these.


If you are wanting to submit a talk on open source community related 
topics such as the Apache Way, governance or legal aspects then please 
submit these to the CFP for FOSS Backstage.


Tickets for the Apache EU Roadshow are included as part of the 
registration for FOSS Backstage, so to attend the Roadshow you will need 
to register for FOSS Backstage. Early Bird tickets are still available 
until the 21^st February 2018.


Please see below for important URLs to remember:

-  To submit a CFP for the Apache EU Roadshow 
:http://apachecon.com/euroadshow18/ 


-  To submit a CFP for FOSS Backstage : 
https://foss-backstage.de/call-papers


-  To register to attend the Apache EU Roadshow and/or FOSS Backstage : 
https://foss-backstage.de/tickets


For further updates and information about the Apache EU Roadshowplease 
check http://apachecon.com/euroadshow18/


Thanks
Sharan Foga, VP Apache Community Development

[jira] [Created] (IGNITE-7705) DIscoverySpi should inherit networkTimeout from IgniteConfiguration

[Alexey Popov (JIRA)]

Alexey Popov created IGNITE-7705:


 Summary: DIscoverySpi should inherit networkTimeout from 
IgniteConfiguration
 Key: IGNITE-7705
 URL: https://issues.apache.org/jira/browse/IGNITE-7705
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.3
Reporter: Alexey Popov


Now we have global network timeout (from IgniteConfiguration) Please see 
{{IgniteConfiguration.setNetworkTimeout}}.
And we have a specific {{DiscoverySpi.setNetworkTimeout}}. DiscoverySpi should 
use a global network timeout (by default) if {{DiscoverySpi.setNetworkTimeout}} 
were not explicitly called.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3524: Fixed handling of state change message in compati...

[ilantukh]

GitHub user ilantukh opened a pull request:

https://github.com/apache/ignite/pull/3524

Fixed handling of state change message in compatibility mode.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-13475

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3524.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3524


commit 4925ffc10ce8e8287980eaec38b587512568a302
Author: Alexey Goncharuk 
Date:   2018-01-17T12:26:03Z

IGNITE-7453 Use GridUnsafe.cleanDirectBuffer in PageSnapshot

commit bcd68a058683b2f17b7ac60471b6e7aab3e4f6de
Author: Pavel Tupitsyn 
Date:   2018-01-17T12:38:20Z

IGNITE-7301 .NET: Baseline topology

This closes #3352

commit 66b96316a7775ce8a6e2ff4475185d5929e4998b
Author: devozerov 
Date:   2018-01-17T12:54:17Z

Merge branch 'master' into ignite-2.4

commit 268481c1cf7fe57df24be130eb67c3e3a13afe01
Author: Alexey Goncharuk 
Date:   2018-01-17T13:50:34Z

IGNITE-7453 Use GridUnsafe.cleanDirectBuffer in WalStat

commit db0cd105719c8ae713b13b34d9dca0a8cd45d377
Author: Pavel Tupitsyn 
Date:   2018-01-17T14:05:25Z

IGNITE-6776 .NET: Thin client: Add SQL & LINQ example

This closes #3390

commit c214db879101aa5660e2a50b11cd20964c0bc114
Author: Andrey Gura 
Date:   2018-01-17T12:42:41Z

ignite-7450 FileWriteAheadLogManager always uses RandomAccessFileIOFactory 
now

commit 75c27d5e49d7458e46eb46e6f87a445c3f1320ea
Author: Alexey Kuznetsov 
Date:   2018-01-18T02:25:19Z

IGNITE-7274 Web Console: Support multiple statements on Queries screen.
(cherry picked from commit 1926783)

commit 36cc822935387b2150e690c29bc6992dca0563f7
Author: Dmitriy Shabalin 
Date:   2018-01-18T04:49:08Z

IGNITE-7306 Web Console: Fixed export data from tables.
(cherry picked from commit 1bb60ec)

commit d753298b4012894b56f5c9218325211cd84a21d5
Author: Peter Ivanov 
Date:   2018-01-18T06:18:53Z

IGNITE-7107 Apache Ignite RPM packages

* added changelog to package specification

This closes #3396

commit 63445893f1bc75dc9777184499f7eabc1d4e51b1
Author: Denis Mekhanikov 
Date:   2018-01-18T08:36:18Z

IGNITE-3935 Use PeerDeployAware for streamer transformer - Fixes #3378.

Signed-off-by: Alexey Goncharuk 

commit f3f9f2a24b23027cf0c835140322e41a788932ae
Author: Pavel Tupitsyn 
Date:   2018-01-18T09:05:12Z

IGNITE-7413 Fix SqlDmlExample

This closes #3389

commit 1daa7c41bf1460a4d9a2b0c26a7a317f2fca3fb7
Author: Alexey Kuznetsov 
Date:   2018-01-18T10:14:53Z

IGNITE-7461 UI tools: Actualized data storage configuration.
(cherry picked from commit 577e632)

commit cf0080210d24d9dd8b057f959446fac5f8a4ca01
Author: dpavlov 
Date:   2018-01-18T10:53:29Z

IGNITE-7380 Implemented pluggable Direct IO - Fixes #3226.

Signed-off-by: Alexey Goncharuk 

commit dd06d0bd7ef266bfbe156e858b312d1ac86e8982
Author: Pavel Tupitsyn 
Date:   2018-01-18T12:55:49Z

IGNITE-7465 .NET: Fix SqlDdlExample failure with standalone node

commit 57479ec564e1761716da3d5f9feb7a64c396a9f9
Author: Pavel Tupitsyn 
Date:   2018-01-18T13:45:54Z

.NET: Fix CacheLocalTest.TestTxDeadlockDetection

commit bd6be8a4653322905a3b63850c7e033ce3801ce5
Author: Pavel Tupitsyn 
Date:   2018-01-18T18:25:05Z

.NET: Thin client: Fix OP_BINARY_TYPE_GET schema passing format

commit 97564d160586d6d57d300937e6b8877994e35fc7
Author: rkondakov 
Date:   2018-01-19T08:24:51Z

IGNITE-6456: Ability to separately enable or disable JDBC, ODBC and thin 
client endpoints. This closes #3309.

commit d50274ca8875c9680c12e8786ac355a787ba95e0
Author: Yakov Zhdanov 
Date:   2018-01-18T17:57:17Z

Javadoc enhancements - added @see

commit cb2d3cf22388ab19fb2d34ae5bdfc8f1b608db75
Author: Dmitriy Govorukhin 
Date:   2018-01-18T14:14:26Z

IGNITE-7471 Use soft reference for checkpoint entry contents to avoid 
excessive memory usage

commit 3965923369870bb4e8e57e3332c1a1eb1e5f5ed3
Author: rkondakov 
Date:   2018-01-19T09:00:55Z

IGNITE-6772: SQL exception messages became more informative. This closes 
#3342.

commit ba68cb0fa87f776fcd0499d030c333f182611f41
Author: devozerov 
Date:   2018-01-19T09:03:52Z

Merge remote-tracking branch 'origin/ignite-2.4' into ignite-2.4

commit b54c0c8786bd74aa0abb208f537c29f0c4be4b1e
Author: tledkov-gridgain 
Date:   2018-01-19T09:09:34Z

IGNITE-7248: JDBC: fixed schema 

Re: Spark data frames

[Vyacheslav Daradur]

Dmitry, it's a great idea!

Nikolay, I also have the interest to get familiar with Spark Data
Frames integration.

I'd prefer webinar of the format similar to "Ignite Persistent Store
Walkthrough" by Denis Magda, which has been presented some times ago.

On Wed, Feb 14, 2018 at 5:03 PM, Dmitriy Setrakyan
 wrote:
> I am definitely interested. Great idea!
>
> On Wed, Feb 14, 2018 at 4:32 AM, Nikolay Izhikov 
> wrote:
>
>> Hello, Dmitry.
>>
>> If other community member are also iterested in that kind of information I
>> can try to do the talk.
>>
>> В Ср, 14/02/2018 в 10:49 +, Dmitry Pavlov пишет:
>> > Hi Nikolay,
>> >
>> > I've notices there are a number of very lively discussions on dev list
>> about SparkDataFrames. But I, for example, can't fully understand it
>> because it is not well-known code for me.
>> >
>> > I suppose Ignite community has other members, which are not aware of
>> recent feature SparkDataFrame and its pros.
>> >
>> > What do you think about short talk arrangement for community to tell
>> about this module, e.g. for 30 minutes? Could you please do this? I think
>> Denis M. can help with infrastructure.
>> >
>> > Sincerely,
>> > Dmitriy Pavlov
>>



-- 
Best Regards, Vyacheslav D.

Re: Spark data frames

[Dmitriy Setrakyan]

I am definitely interested. Great idea!

On Wed, Feb 14, 2018 at 4:32 AM, Nikolay Izhikov 
wrote:

> Hello, Dmitry.
>
> If other community member are also iterested in that kind of information I
> can try to do the talk.
>
> В Ср, 14/02/2018 в 10:49 +, Dmitry Pavlov пишет:
> > Hi Nikolay,
> >
> > I've notices there are a number of very lively discussions on dev list
> about SparkDataFrames. But I, for example, can't fully understand it
> because it is not well-known code for me.
> >
> > I suppose Ignite community has other members, which are not aware of
> recent feature SparkDataFrame and its pros.
> >
> > What do you think about short talk arrangement for community to tell
> about this module, e.g. for 30 minutes? Could you please do this? I think
> Denis M. can help with infrastructure.
> >
> > Sincerely,
> > Dmitriy Pavlov
>

Re: IgniteSet implementation: changes required

[Pavel Pereslegin]

Hello, Igniters!

I agree that solution with separate caches is not acceptable for a
large number of sets.

So, I want to suggest one more way to implement IgniteSet that will
introduce element indexes (similar to IgniteQueue). To implement this
we can add head/tail indexes to IgniteSet header and for each
IgniteSet element store two key-value pairs:
 setKey, index
 index, setKey

Indexes are required to support iterator and they should be continuous.

Please, see detailed description in JIRA comment [1].

With such approach add/remove/contains operations will have O(1) time
complexity, iterator should work similar to current IgniteQueue
iterator, issues [2], [3] will be resolved, because PDS recovery will
work "out of the box" and we will not use JVM heap for duplicated
values.

Btw, we can use this implementation only for collocated mode (map
keys/indexes to IgniteSet name) and use separate caches for
non-collocated mode.

What do you think about this?

[1] https://issues.apache.org/jira/browse/IGNITE-5553#comment-16364043
[2] https://issues.apache.org/jira/browse/IGNITE-5553
[3] https://issues.apache.org/jira/browse/IGNITE-7565


2018-02-13 9:33 GMT+03:00 Andrey Kuznetsov :
> Indeed, all sets, regardless of whether they collocated or not, share
> single cache, and also use onheap data structures irresistable to
> checkpointing/recovery.
>
> 2018-02-13 2:14 GMT+03:00 Dmitriy Setrakyan :
>
>> On Fri, Feb 9, 2018 at 6:26 PM, Andrey Kuznetsov 
>> wrote:
>>
>> > Hi all,
>> >
>> > Current set implementation has significant flaw: all set data are
>> > duplicated in onheap maps on _every_ node in order to make iterator() and
>> > size(). For me it looks like simple yet ineffective implementation.
>> > Currently, these maps are damaged by checkpointing/recovery, and we could
>> > patch them somehow. Another future change to Ignite caches can damage
>> them
>> > again. This looks fragile when datastructure is not entirely backed by
>> > caches. Pavel's proposal seems to be a reliable solution for
>> non-collocated
>> > sets.
>> >
>>
>> I would agree, but I was under an impression that non-collocated sets are
>> already implemented this way. Am I wrong?
>>
>
>
>
> --
> Best regards,
>   Andrey Kuznetsov.

[GitHub] ignite pull request #3523: IGNITE-7699 ObjectBinaryProcessor improvements

[sergey-chugunov-1985]

GitHub user sergey-chugunov-1985 opened a pull request:

https://github.com/apache/ignite/pull/3523

IGNITE-7699 ObjectBinaryProcessor improvements

discovery protocol should not be triggered if requested metadata is the 
same as already existing; logging improvements

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7699

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3523.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3523






---

[jira] [Created] (IGNITE-7704) Document IgniteConfiguration, TcpDiscoverySpi, TcpCommunicationSpi timeouts and their relations

[Alexey Popov (JIRA)]

Alexey Popov created IGNITE-7704:


 Summary: Document IgniteConfiguration, TcpDiscoverySpi, 
TcpCommunicationSpi timeouts and their relations
 Key: IGNITE-7704
 URL: https://issues.apache.org/jira/browse/IGNITE-7704
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Affects Versions: 2.3
Reporter: Alexey Popov


We often see similar questions related to IgniteConfiguration, TcpDiscoverySpi, 
TcpCommunicationSpi timeouts and their relations. And we see several 
side-effects after incorrect timeout configuration.

It looks like this question is not well documented.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Spark data frames

[Nikolay Izhikov]

Hello, Dmitry.

If other community member are also iterested in that kind of information I can 
try to do the talk.

В Ср, 14/02/2018 в 10:49 +, Dmitry Pavlov пишет:
> Hi Nikolay,
> 
> I've notices there are a number of very lively discussions on dev list about 
> SparkDataFrames. But I, for example, can't fully understand it because it is 
> not well-known code for me.
> 
> I suppose Ignite community has other members, which are not aware of recent 
> feature SparkDataFrame and its pros.
> 
> What do you think about short talk arrangement for community to tell about 
> this module, e.g. for 30 minutes? Could you please do this? I think Denis M. 
> can help with infrastructure.
> 
> Sincerely,
> Dmitriy Pavlov

signature.asc
Description: This is a digitally signed message part

Re: Author in header of class

[Ilya Lantukh]

I agree with Vyacheslav and Anton Vinogradov. Using @author tag can be
misleading, Upsource will provide more up-to-date information.

On Wed, Feb 14, 2018 at 3:06 PM, Anton Vinogradov 
wrote:

> +1 to Upsource usage.
>
> Checked that code ownership provided by Upsource is correct.
>
> On Wed, Feb 14, 2018 at 2:22 PM, Vyacheslav Daradur 
> wrote:
>
> > Dmitry,
> >
> > > - easy find reviewer and (Patch available queue),
> > > - find test author / mainteiner (MTCGA)>.
> > Upsource suggests reviewers based on changed classes maintainers when
> > we create pull request review.
> > Moreover, it shows all authors of each class in PR, with showing a
> > percentage of contributing.
> >
> > IMO using @author tag will be easy-to-use only with "little" classes,
> > because often hard to identify maintainer of "big" classes because of
> > great number of changes.
> >
> > Also, it will be preferable to send a review request to dev-list, if
> > maintainer retired and doesn't subscribe dev-list anymore.
> >
> > On Wed, Feb 14, 2018 at 1:38 PM, Dmitry Pavlov 
> > wrote:
> > > Hi Folks,
> > >
> > > Listing from git annotate are not informative because there are many
> > > contributors change same place of code.
> > >
> > > Code author information can help us to solve 2 issues for new community
> > > members:
> > > - easy find reviewer and (Patch available queue),
> > > - find test author / mainteiner (MTCGA).
> > >
> > > So I strongly like Anton K. proposal. It's pity that author tag is
> banned
> > > by Apache.
> > >
> > > Let's think how we can solve these 2 issues without author tag.
> > >
> > > One more point: IMHO we should remove selection of 2+ mainterners or
> > > underline one (default) contact point for area. Necessity to choise may
> > > confuse newcomer. One main (default) mainteiner will redirect review.
> > >
> > > Sincererely,
> > > Dmitriy Pavlov
> > >
> > > ср, 14 февр. 2018 г. в 13:10, Anton Vinogradov <
> avinogra...@gridgain.com
> > >:
> > >
> > >> Anton,
> > >>
> > >> All listed info can be gained from git history.
> > >>
> > >> On Wed, Feb 14, 2018 at 1:01 PM, Дмитрий Рябов  >
> > >> wrote:
> > >>
> > >> > For the first profit you should see maintainer table [1].
> > >> >
> > >> > [1]
> > >> > https://cwiki.apache.org/confluence/display/IGNITE/How+
> > >> > to+Contribute#HowtoContribute-ReviewProcessandMaintainers
> > >> >
> > >> > 2018-02-14 11:42 GMT+03:00 Vyacheslav Daradur  >:
> > >> >
> > >> > > Hi Anton,
> > >> > >
> > >> > > Apache Ignite is licensed under the Apache License Version 2.0,
> > which
> > >> > > does not allow to use author tag in java code [1].
> > >> > >
> > >> > > [1] http://directory.apache.org/fortress/coding-standards.
> > >> > > html#classinterface-headers
> > >> > >
> > >> > > On Wed, Feb 14, 2018 at 11:23 AM, Антон Калашников <
> > kaa@yandex.ru>
> > >> > > wrote:
> > >> > > > Hello Ignite Community!
> > >> > > >
> > >> > > > My name is Anton. I joined to community some time ago and I want
> > to
> > >> > > contribute to Apache Ignite.
> > >> > > >
> > >> > > > I would be want to make my first proposal. I noticed that Ignite
> > >> don't
> > >> > > have author description in header of classes unlike many other
> > apache
> > >> > > projects.
> > >> > > > I propose to use javadoc tag @author in header of class when you
> > >> > created
> > >> > > it  and also add extra tag @author when you do many changes in
> this
> > >> > class.
> > >> > > >
> > >> > > > Profits of this aproach, in my opinion:
> > >> > > > 1) You always know who has knowledge of this class unlike git
> > >> annotate
> > >> > > which are sensetive in refactoring, moving, etc.
> > >> > > > 2) It will highed responsibilty for code quality because nobody
> > want
> > >> to
> > >> > > sign under bad code)
> > >> > > >
> > >> > > > Example:
> > >> > > > /**
> > >> > > >  * @author Anton Kalashnikov
> > >> > > >  * @author Other Author
> > >> > > >  */
> > >> > > >
> > >> > > > --
> > >> > > > Best Regards,
> > >> > > > Anton Kalashnikov
> > >> > > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > > Best Regards, Vyacheslav D.
> > >> > >
> > >> >
> > >>
> >
> >
> >
> > --
> > Best Regards, Vyacheslav D.
> >
>



-- 
Best regards,
Ilya

Re: Author in header of class

[Anton Vinogradov]

+1 to Upsource usage.

Checked that code ownership provided by Upsource is correct.

On Wed, Feb 14, 2018 at 2:22 PM, Vyacheslav Daradur 
wrote:

> Dmitry,
>
> > - easy find reviewer and (Patch available queue),
> > - find test author / mainteiner (MTCGA)>.
> Upsource suggests reviewers based on changed classes maintainers when
> we create pull request review.
> Moreover, it shows all authors of each class in PR, with showing a
> percentage of contributing.
>
> IMO using @author tag will be easy-to-use only with "little" classes,
> because often hard to identify maintainer of "big" classes because of
> great number of changes.
>
> Also, it will be preferable to send a review request to dev-list, if
> maintainer retired and doesn't subscribe dev-list anymore.
>
> On Wed, Feb 14, 2018 at 1:38 PM, Dmitry Pavlov 
> wrote:
> > Hi Folks,
> >
> > Listing from git annotate are not informative because there are many
> > contributors change same place of code.
> >
> > Code author information can help us to solve 2 issues for new community
> > members:
> > - easy find reviewer and (Patch available queue),
> > - find test author / mainteiner (MTCGA).
> >
> > So I strongly like Anton K. proposal. It's pity that author tag is banned
> > by Apache.
> >
> > Let's think how we can solve these 2 issues without author tag.
> >
> > One more point: IMHO we should remove selection of 2+ mainterners or
> > underline one (default) contact point for area. Necessity to choise may
> > confuse newcomer. One main (default) mainteiner will redirect review.
> >
> > Sincererely,
> > Dmitriy Pavlov
> >
> > ср, 14 февр. 2018 г. в 13:10, Anton Vinogradov  >:
> >
> >> Anton,
> >>
> >> All listed info can be gained from git history.
> >>
> >> On Wed, Feb 14, 2018 at 1:01 PM, Дмитрий Рябов 
> >> wrote:
> >>
> >> > For the first profit you should see maintainer table [1].
> >> >
> >> > [1]
> >> > https://cwiki.apache.org/confluence/display/IGNITE/How+
> >> > to+Contribute#HowtoContribute-ReviewProcessandMaintainers
> >> >
> >> > 2018-02-14 11:42 GMT+03:00 Vyacheslav Daradur :
> >> >
> >> > > Hi Anton,
> >> > >
> >> > > Apache Ignite is licensed under the Apache License Version 2.0,
> which
> >> > > does not allow to use author tag in java code [1].
> >> > >
> >> > > [1] http://directory.apache.org/fortress/coding-standards.
> >> > > html#classinterface-headers
> >> > >
> >> > > On Wed, Feb 14, 2018 at 11:23 AM, Антон Калашников <
> kaa@yandex.ru>
> >> > > wrote:
> >> > > > Hello Ignite Community!
> >> > > >
> >> > > > My name is Anton. I joined to community some time ago and I want
> to
> >> > > contribute to Apache Ignite.
> >> > > >
> >> > > > I would be want to make my first proposal. I noticed that Ignite
> >> don't
> >> > > have author description in header of classes unlike many other
> apache
> >> > > projects.
> >> > > > I propose to use javadoc tag @author in header of class when you
> >> > created
> >> > > it  and also add extra tag @author when you do many changes in this
> >> > class.
> >> > > >
> >> > > > Profits of this aproach, in my opinion:
> >> > > > 1) You always know who has knowledge of this class unlike git
> >> annotate
> >> > > which are sensetive in refactoring, moving, etc.
> >> > > > 2) It will highed responsibilty for code quality because nobody
> want
> >> to
> >> > > sign under bad code)
> >> > > >
> >> > > > Example:
> >> > > > /**
> >> > > >  * @author Anton Kalashnikov
> >> > > >  * @author Other Author
> >> > > >  */
> >> > > >
> >> > > > --
> >> > > > Best Regards,
> >> > > > Anton Kalashnikov
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Best Regards, Vyacheslav D.
> >> > >
> >> >
> >>
>
>
>
> --
> Best Regards, Vyacheslav D.
>

[jira] [Created] (IGNITE-7703) Add a method gets caches in batch

[Vladislav Pyatkov (JIRA)]

Vladislav Pyatkov created IGNITE-7703:
-

 Summary: Add a method gets caches in batch
 Key: IGNITE-7703
 URL: https://issues.apache.org/jira/browse/IGNITE-7703
 Project: Ignite
  Issue Type: Bug
Reporter: Vladislav Pyatkov


Ignite allows to start (and/or get) caches in batch, but not allows to do get 
without starting.

In some cases need to start particular subset of all cluster caches, but if 
calls this one by one:

_org.apache.ignite.Ignite#cache_

we have a risk to overload discovery layer by messages of 
_DynamicCacheChangeRequest_.

 

Will be better to add a specific method for gets of caches in batch.

_org.apache.ignite.Ignite#caches_



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7702) Adopt KNN classification to the new Dataset from dataset package

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-7702:


 Summary: Adopt KNN classification to the new Dataset from dataset 
package
 Key: IGNITE-7702
 URL: https://issues.apache.org/jira/browse/IGNITE-7702
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Author in header of class

[Vyacheslav Daradur]

Dmitry,

> - easy find reviewer and (Patch available queue),
> - find test author / mainteiner (MTCGA)>.
Upsource suggests reviewers based on changed classes maintainers when
we create pull request review.
Moreover, it shows all authors of each class in PR, with showing a
percentage of contributing.

IMO using @author tag will be easy-to-use only with "little" classes,
because often hard to identify maintainer of "big" classes because of
great number of changes.

Also, it will be preferable to send a review request to dev-list, if
maintainer retired and doesn't subscribe dev-list anymore.

On Wed, Feb 14, 2018 at 1:38 PM, Dmitry Pavlov  wrote:
> Hi Folks,
>
> Listing from git annotate are not informative because there are many
> contributors change same place of code.
>
> Code author information can help us to solve 2 issues for new community
> members:
> - easy find reviewer and (Patch available queue),
> - find test author / mainteiner (MTCGA).
>
> So I strongly like Anton K. proposal. It's pity that author tag is banned
> by Apache.
>
> Let's think how we can solve these 2 issues without author tag.
>
> One more point: IMHO we should remove selection of 2+ mainterners or
> underline one (default) contact point for area. Necessity to choise may
> confuse newcomer. One main (default) mainteiner will redirect review.
>
> Sincererely,
> Dmitriy Pavlov
>
> ср, 14 февр. 2018 г. в 13:10, Anton Vinogradov :
>
>> Anton,
>>
>> All listed info can be gained from git history.
>>
>> On Wed, Feb 14, 2018 at 1:01 PM, Дмитрий Рябов 
>> wrote:
>>
>> > For the first profit you should see maintainer table [1].
>> >
>> > [1]
>> > https://cwiki.apache.org/confluence/display/IGNITE/How+
>> > to+Contribute#HowtoContribute-ReviewProcessandMaintainers
>> >
>> > 2018-02-14 11:42 GMT+03:00 Vyacheslav Daradur :
>> >
>> > > Hi Anton,
>> > >
>> > > Apache Ignite is licensed under the Apache License Version 2.0, which
>> > > does not allow to use author tag in java code [1].
>> > >
>> > > [1] http://directory.apache.org/fortress/coding-standards.
>> > > html#classinterface-headers
>> > >
>> > > On Wed, Feb 14, 2018 at 11:23 AM, Антон Калашников 
>> > > wrote:
>> > > > Hello Ignite Community!
>> > > >
>> > > > My name is Anton. I joined to community some time ago and I want to
>> > > contribute to Apache Ignite.
>> > > >
>> > > > I would be want to make my first proposal. I noticed that Ignite
>> don't
>> > > have author description in header of classes unlike many other apache
>> > > projects.
>> > > > I propose to use javadoc tag @author in header of class when you
>> > created
>> > > it  and also add extra tag @author when you do many changes in this
>> > class.
>> > > >
>> > > > Profits of this aproach, in my opinion:
>> > > > 1) You always know who has knowledge of this class unlike git
>> annotate
>> > > which are sensetive in refactoring, moving, etc.
>> > > > 2) It will highed responsibilty for code quality because nobody want
>> to
>> > > sign under bad code)
>> > > >
>> > > > Example:
>> > > > /**
>> > > >  * @author Anton Kalashnikov
>> > > >  * @author Other Author
>> > > >  */
>> > > >
>> > > > --
>> > > > Best Regards,
>> > > > Anton Kalashnikov
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Best Regards, Vyacheslav D.
>> > >
>> >
>>



-- 
Best Regards, Vyacheslav D.

[GitHub] ignite pull request #3522: IGNITE-7452: Make Linear SVM example for multi - ...

[zaleslaw]

GitHub user zaleslaw opened a pull request:

https://github.com/apache/ignite/pull/3522

IGNITE-7452: Make Linear SVM example for multi - classification



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7452

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3522.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3522


commit e8f8c1c2c02258890873cccfef66388f3eb9f5a4
Author: zaleslaw 
Date:   2018-02-14T10:53:03Z

IGNITE-7452: added example




---

Spark data frames

[Dmitry Pavlov]

Hi Nikolay,

I've notices there are a number of very lively discussions on dev list
about SparkDataFrames. But I, for example, can't fully understand it
because it is not well-known code for me.

I suppose Ignite community has other members, which are not aware of recent
feature SparkDataFrame and its pros.

What do you think about short talk arrangement for community to tell about
this module, e.g. for 30 minutes? Could you please do this? I think Denis
M. can help with infrastructure.

Sincerely,
Dmitriy Pavlov

[jira] [Created] (IGNITE-7701) SQL system view for node attributes

[Aleksey Plekhanov (JIRA)]

Aleksey Plekhanov created IGNITE-7701:
-

 Summary: SQL system view for node attributes
 Key: IGNITE-7701
 URL: https://issues.apache.org/jira/browse/IGNITE-7701
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
 Fix For: 2.5


Implement SQL system view to show attributes for each node in topology.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7700) SQL system view for list of nodes

[Aleksey Plekhanov (JIRA)]

Aleksey Plekhanov created IGNITE-7700:
-

 Summary: SQL system view for list of nodes
 Key: IGNITE-7700
 URL: https://issues.apache.org/jira/browse/IGNITE-7700
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
 Fix For: 2.5


Implement SQL system view to show list of nodes in topology.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Author in header of class

[Dmitry Pavlov]

Hi Folks,

Listing from git annotate are not informative because there are many
contributors change same place of code.

Code author information can help us to solve 2 issues for new community
members:
- easy find reviewer and (Patch available queue),
- find test author / mainteiner (MTCGA).

So I strongly like Anton K. proposal. It's pity that author tag is banned
by Apache.

Let's think how we can solve these 2 issues without author tag.

One more point: IMHO we should remove selection of 2+ mainterners or
underline one (default) contact point for area. Necessity to choise may
confuse newcomer. One main (default) mainteiner will redirect review.

Sincererely,
Dmitriy Pavlov

ср, 14 февр. 2018 г. в 13:10, Anton Vinogradov :

> Anton,
>
> All listed info can be gained from git history.
>
> On Wed, Feb 14, 2018 at 1:01 PM, Дмитрий Рябов 
> wrote:
>
> > For the first profit you should see maintainer table [1].
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/IGNITE/How+
> > to+Contribute#HowtoContribute-ReviewProcessandMaintainers
> >
> > 2018-02-14 11:42 GMT+03:00 Vyacheslav Daradur :
> >
> > > Hi Anton,
> > >
> > > Apache Ignite is licensed under the Apache License Version 2.0, which
> > > does not allow to use author tag in java code [1].
> > >
> > > [1] http://directory.apache.org/fortress/coding-standards.
> > > html#classinterface-headers
> > >
> > > On Wed, Feb 14, 2018 at 11:23 AM, Антон Калашников 
> > > wrote:
> > > > Hello Ignite Community!
> > > >
> > > > My name is Anton. I joined to community some time ago and I want to
> > > contribute to Apache Ignite.
> > > >
> > > > I would be want to make my first proposal. I noticed that Ignite
> don't
> > > have author description in header of classes unlike many other apache
> > > projects.
> > > > I propose to use javadoc tag @author in header of class when you
> > created
> > > it  and also add extra tag @author when you do many changes in this
> > class.
> > > >
> > > > Profits of this aproach, in my opinion:
> > > > 1) You always know who has knowledge of this class unlike git
> annotate
> > > which are sensetive in refactoring, moving, etc.
> > > > 2) It will highed responsibilty for code quality because nobody want
> to
> > > sign under bad code)
> > > >
> > > > Example:
> > > > /**
> > > >  * @author Anton Kalashnikov
> > > >  * @author Other Author
> > > >  */
> > > >
> > > > --
> > > > Best Regards,
> > > > Anton Kalashnikov
> > > >
> > >
> > >
> > >
> > > --
> > > Best Regards, Vyacheslav D.
> > >
> >
>

Re: Author in header of class

[Anton Vinogradov]

Anton,

All listed info can be gained from git history.

On Wed, Feb 14, 2018 at 1:01 PM, Дмитрий Рябов 
wrote:

> For the first profit you should see maintainer table [1].
>
> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/How+
> to+Contribute#HowtoContribute-ReviewProcessandMaintainers
>
> 2018-02-14 11:42 GMT+03:00 Vyacheslav Daradur :
>
> > Hi Anton,
> >
> > Apache Ignite is licensed under the Apache License Version 2.0, which
> > does not allow to use author tag in java code [1].
> >
> > [1] http://directory.apache.org/fortress/coding-standards.
> > html#classinterface-headers
> >
> > On Wed, Feb 14, 2018 at 11:23 AM, Антон Калашников 
> > wrote:
> > > Hello Ignite Community!
> > >
> > > My name is Anton. I joined to community some time ago and I want to
> > contribute to Apache Ignite.
> > >
> > > I would be want to make my first proposal. I noticed that Ignite don't
> > have author description in header of classes unlike many other apache
> > projects.
> > > I propose to use javadoc tag @author in header of class when you
> created
> > it  and also add extra tag @author when you do many changes in this
> class.
> > >
> > > Profits of this aproach, in my opinion:
> > > 1) You always know who has knowledge of this class unlike git annotate
> > which are sensetive in refactoring, moving, etc.
> > > 2) It will highed responsibilty for code quality because nobody want to
> > sign under bad code)
> > >
> > > Example:
> > > /**
> > >  * @author Anton Kalashnikov
> > >  * @author Other Author
> > >  */
> > >
> > > --
> > > Best Regards,
> > > Anton Kalashnikov
> > >
> >
> >
> >
> > --
> > Best Regards, Vyacheslav D.
> >
>

Re: Author in header of class

[Дмитрий Рябов]

For the first profit you should see maintainer table [1].

[1]
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers

2018-02-14 11:42 GMT+03:00 Vyacheslav Daradur :

> Hi Anton,
>
> Apache Ignite is licensed under the Apache License Version 2.0, which
> does not allow to use author tag in java code [1].
>
> [1] http://directory.apache.org/fortress/coding-standards.
> html#classinterface-headers
>
> On Wed, Feb 14, 2018 at 11:23 AM, Антон Калашников 
> wrote:
> > Hello Ignite Community!
> >
> > My name is Anton. I joined to community some time ago and I want to
> contribute to Apache Ignite.
> >
> > I would be want to make my first proposal. I noticed that Ignite don't
> have author description in header of classes unlike many other apache
> projects.
> > I propose to use javadoc tag @author in header of class when you created
> it  and also add extra tag @author when you do many changes in this class.
> >
> > Profits of this aproach, in my opinion:
> > 1) You always know who has knowledge of this class unlike git annotate
> which are sensetive in refactoring, moving, etc.
> > 2) It will highed responsibilty for code quality because nobody want to
> sign under bad code)
> >
> > Example:
> > /**
> >  * @author Anton Kalashnikov
> >  * @author Other Author
> >  */
> >
> > --
> > Best Regards,
> > Anton Kalashnikov
> >
>
>
>
> --
> Best Regards, Vyacheslav D.
>

Re: Ignite diagnostic (SQL system views)

[Anton Vinogradov]

Vova,

Could you confirm https://issues.apache.org/jira/browse/IGNITE-7527 ready
to be merged?

On Wed, Feb 14, 2018 at 12:01 PM, Vladimir Ozerov 
wrote:

> I would start with NODES and NODE_ATTRIBUTES as the most simple thing.
>
> On Tue, Feb 13, 2018 at 4:10 AM, Denis Magda  wrote:
>
> > Alex P, sounds like a good plan for me.
> >
> > Vladimir, do you have any suggestions or corrections?
> >
> > —
> > Denis
> >
> > > On Feb 12, 2018, at 4:57 AM, Alex Plehanov 
> > wrote:
> > >
> > > The views engine and the first view are almost ready to merge (review
> > > comments are resolved). Which views should we take next? My proposal -
> > > NODES, NODE_ATTRIBUTES, NODE_METRICS, NODE_HOSTS and NODE_ADDRESSES,
> > since
> > > these views are clear and all topology data available on each node.
> > > Any objections?
> > >
> > > 2018-01-25 16:27 GMT+03:00 Alex Plehanov :
> > >
> > >> Anton, Vladimir, I've made some fixes. There is only one view left and
> > >> it's renamed to 'IGNITE.LOCAL_TRANSACTIONS'.
> > >>
> > >> High level design of solution:
> > >> When IgniteH2Indexing is starting, it create and start
> > >> new GridH2SysViewProcessor, which create and register in H2 (via its
> own
> > >> table engine) all implementations of system views. Each system view
> > >> implementation extends base abstract class GridH2SysView. View
> > >> implementation describes columns, their types and indexes in
> constructor
> > >> and must override method getRows for data retrieval (this method
> called
> > by
> > >> H2-compatible table and index implementations for ignite system
> views).
> > >> Almost no fixes to existing parsing engine was made, except some
> places,
> > >> where GridH2Table instance was expected, but for system views there is
> > >> another class.
> > >>
> > >> New PR: [1].  Please have a look.
> > >>
> > >> [1] https://github.com/apache/ignite/pull/3433
> > >>
> > >> 2018-01-24 19:12 GMT+03:00 Anton Vinogradov  >:
> > >>
> > >>> I've created IEP-13 [1] to cover all cases.
> > >>> Feel free to create issues.
> > >>>
> > >>> [1]
> > >>> https://cwiki.apache.org/confluence/pages/viewpage.
> > action?pageId=75962769
> > >>>
> > >>> On Wed, Jan 24, 2018 at 6:10 PM, Vladimir Ozerov <
> voze...@gridgain.com
> > >
> > >>> wrote:
> > >>>
> >  Let's start with a single and the most simple view, e.g.
> >  LOCAL_TRANSACTIONS. We will review and merge it along with necessary
> >  infrastructure. Then will handle the rest view in separate tickets
> and
> >  separate focused discussions.
> > 
> >  On Wed, Jan 24, 2018 at 5:29 PM, Alex Plehanov <
> > plehanov.a...@gmail.com
> > 
> >  wrote:
> > 
> > > 1) It’s not a principal point, I can change schema. The
> >  INFORMATION_SCHEMA
> > > was used because it’s already exists and usually used for metadata
> > >>> tables
> > > and views. Your proposal is to use schema “IGNITE”, am I understand
> > >>> you
> > > right? BTW, for now, we can’t query another (H2) meta tables from
> the
> > > INFORMATION_SCHEMA, so, “Ignite system views” is only available
> views
> > >>> to
> > > query from this schema.
> > > 2) Exactly for this reason the IGNITE_INSTANCE view is useful: to
> >  determine
> > > which node we are connected to.
> > > 3) As the first phase, in my opinion, local views will be enough.
> > > Performance and caching of distributed views should be discussed at
> > >>> next
> > > phases, when distributed views implementation will be planned. In
> > >>> current
> > > implementation I tried to use indexing for local views wherever
> it’s
> > > possible.
> > > 4) I don’t think, that JVM info is more critical information than,
> > for
> > > example, caches or nodes information. When authorization
> capabilities
> > > planned to implement?
> > >
> > > About local data: yes, we can rename all currently implemented
> views
> > >>> for
> > > the local node data as LOCAL_..., and create (someday) new whole
> > >>> cluster
> > > views (which use distributed requests) without prefix or, for
> > example,
> >  with
> > > CLUSTER_ prefix. But some views can show all cluster information
> > using
> >  only
> > > local node data, without distributed requests (for example
> > > IGNITE_NODE_METRICS, IGNITE_PART_ASSIGNMENT,
> IGNITE_PART_ALLOCATION,
> > > IGNITE_NODES, etc). Are they local or cluster views in this
> concept?
> >  Which
> > > prefix should be used? And what about caches? Are they local or
> > >>> cluster?
> >  On
> > > local node we can see cluster wide caches (replicated and
> > distributed)
> >  and
> > > caches for current node only. Local caches list may differ from
> node
> > >>> to
> > > node. Which prefix should be used for this view? And one more,
> there
> > >>> is
> >  no
> 

[jira] [Created] (IGNITE-7699) BinaryMetadata exchange should not be triggered if metadata was not updated

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-7699:
---

 Summary: BinaryMetadata exchange should not be triggered if 
metadata was not updated
 Key: IGNITE-7699
 URL: https://issues.apache.org/jira/browse/IGNITE-7699
 Project: Ignite
  Issue Type: Improvement
Reporter: Sergey Chugunov
Assignee: Sergey Chugunov
 Fix For: 2.5


It was found that DiscoveryCustomEvent-based implementation of BinaryMetadata 
exchange protocol triggers propose-accept process even when it is not needed.

In some situations BinaryObjects may be built with exactly the same scheme many 
times, and registering of BinaryMetatada only once is enough.
However current implementation executes expensive propose-accept process for 
each *addMeta* request even if requesting meta is the same.

Implementation should be improved so that prop-ack process is executed only if 
BinaryMetadata really changed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3521: ignite-7594

[ilantukh]

GitHub user ilantukh opened a pull request:

https://github.com/apache/ignite/pull/3521

ignite-7594



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ilantukh/ignite ignite-7594

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3521.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3521


commit 4925ffc10ce8e8287980eaec38b587512568a302
Author: Alexey Goncharuk 
Date:   2018-01-17T12:26:03Z

IGNITE-7453 Use GridUnsafe.cleanDirectBuffer in PageSnapshot

commit bcd68a058683b2f17b7ac60471b6e7aab3e4f6de
Author: Pavel Tupitsyn 
Date:   2018-01-17T12:38:20Z

IGNITE-7301 .NET: Baseline topology

This closes #3352

commit 66b96316a7775ce8a6e2ff4475185d5929e4998b
Author: devozerov 
Date:   2018-01-17T12:54:17Z

Merge branch 'master' into ignite-2.4

commit 268481c1cf7fe57df24be130eb67c3e3a13afe01
Author: Alexey Goncharuk 
Date:   2018-01-17T13:50:34Z

IGNITE-7453 Use GridUnsafe.cleanDirectBuffer in WalStat

commit db0cd105719c8ae713b13b34d9dca0a8cd45d377
Author: Pavel Tupitsyn 
Date:   2018-01-17T14:05:25Z

IGNITE-6776 .NET: Thin client: Add SQL & LINQ example

This closes #3390

commit c214db879101aa5660e2a50b11cd20964c0bc114
Author: Andrey Gura 
Date:   2018-01-17T12:42:41Z

ignite-7450 FileWriteAheadLogManager always uses RandomAccessFileIOFactory 
now

commit 75c27d5e49d7458e46eb46e6f87a445c3f1320ea
Author: Alexey Kuznetsov 
Date:   2018-01-18T02:25:19Z

IGNITE-7274 Web Console: Support multiple statements on Queries screen.
(cherry picked from commit 1926783)

commit 36cc822935387b2150e690c29bc6992dca0563f7
Author: Dmitriy Shabalin 
Date:   2018-01-18T04:49:08Z

IGNITE-7306 Web Console: Fixed export data from tables.
(cherry picked from commit 1bb60ec)

commit d753298b4012894b56f5c9218325211cd84a21d5
Author: Peter Ivanov 
Date:   2018-01-18T06:18:53Z

IGNITE-7107 Apache Ignite RPM packages

* added changelog to package specification

This closes #3396

commit 63445893f1bc75dc9777184499f7eabc1d4e51b1
Author: Denis Mekhanikov 
Date:   2018-01-18T08:36:18Z

IGNITE-3935 Use PeerDeployAware for streamer transformer - Fixes #3378.

Signed-off-by: Alexey Goncharuk 

commit f3f9f2a24b23027cf0c835140322e41a788932ae
Author: Pavel Tupitsyn 
Date:   2018-01-18T09:05:12Z

IGNITE-7413 Fix SqlDmlExample

This closes #3389

commit 1daa7c41bf1460a4d9a2b0c26a7a317f2fca3fb7
Author: Alexey Kuznetsov 
Date:   2018-01-18T10:14:53Z

IGNITE-7461 UI tools: Actualized data storage configuration.
(cherry picked from commit 577e632)

commit cf0080210d24d9dd8b057f959446fac5f8a4ca01
Author: dpavlov 
Date:   2018-01-18T10:53:29Z

IGNITE-7380 Implemented pluggable Direct IO - Fixes #3226.

Signed-off-by: Alexey Goncharuk 

commit dd06d0bd7ef266bfbe156e858b312d1ac86e8982
Author: Pavel Tupitsyn 
Date:   2018-01-18T12:55:49Z

IGNITE-7465 .NET: Fix SqlDdlExample failure with standalone node

commit 57479ec564e1761716da3d5f9feb7a64c396a9f9
Author: Pavel Tupitsyn 
Date:   2018-01-18T13:45:54Z

.NET: Fix CacheLocalTest.TestTxDeadlockDetection

commit bd6be8a4653322905a3b63850c7e033ce3801ce5
Author: Pavel Tupitsyn 
Date:   2018-01-18T18:25:05Z

.NET: Thin client: Fix OP_BINARY_TYPE_GET schema passing format

commit 97564d160586d6d57d300937e6b8877994e35fc7
Author: rkondakov 
Date:   2018-01-19T08:24:51Z

IGNITE-6456: Ability to separately enable or disable JDBC, ODBC and thin 
client endpoints. This closes #3309.

commit d50274ca8875c9680c12e8786ac355a787ba95e0
Author: Yakov Zhdanov 
Date:   2018-01-18T17:57:17Z

Javadoc enhancements - added @see

commit cb2d3cf22388ab19fb2d34ae5bdfc8f1b608db75
Author: Dmitriy Govorukhin 
Date:   2018-01-18T14:14:26Z

IGNITE-7471 Use soft reference for checkpoint entry contents to avoid 
excessive memory usage

commit 3965923369870bb4e8e57e3332c1a1eb1e5f5ed3
Author: rkondakov 
Date:   2018-01-19T09:00:55Z

IGNITE-6772: SQL exception messages became more informative. This closes 
#3342.

commit ba68cb0fa87f776fcd0499d030c333f182611f41
Author: devozerov 
Date:   2018-01-19T09:03:52Z

Merge remote-tracking branch 'origin/ignite-2.4' into ignite-2.4

commit b54c0c8786bd74aa0abb208f537c29f0c4be4b1e
Author: tledkov-gridgain 
Date:   2018-01-19T09:09:34Z

IGNITE-7248: JDBC: fixed schema resolution for streaming mode. This closes 
#3384.

commit 

Re: Ignite diagnostic (SQL system views)

[Vladimir Ozerov]

I would start with NODES and NODE_ATTRIBUTES as the most simple thing.

On Tue, Feb 13, 2018 at 4:10 AM, Denis Magda  wrote:

> Alex P, sounds like a good plan for me.
>
> Vladimir, do you have any suggestions or corrections?
>
> —
> Denis
>
> > On Feb 12, 2018, at 4:57 AM, Alex Plehanov 
> wrote:
> >
> > The views engine and the first view are almost ready to merge (review
> > comments are resolved). Which views should we take next? My proposal -
> > NODES, NODE_ATTRIBUTES, NODE_METRICS, NODE_HOSTS and NODE_ADDRESSES,
> since
> > these views are clear and all topology data available on each node.
> > Any objections?
> >
> > 2018-01-25 16:27 GMT+03:00 Alex Plehanov :
> >
> >> Anton, Vladimir, I've made some fixes. There is only one view left and
> >> it's renamed to 'IGNITE.LOCAL_TRANSACTIONS'.
> >>
> >> High level design of solution:
> >> When IgniteH2Indexing is starting, it create and start
> >> new GridH2SysViewProcessor, which create and register in H2 (via its own
> >> table engine) all implementations of system views. Each system view
> >> implementation extends base abstract class GridH2SysView. View
> >> implementation describes columns, their types and indexes in constructor
> >> and must override method getRows for data retrieval (this method called
> by
> >> H2-compatible table and index implementations for ignite system views).
> >> Almost no fixes to existing parsing engine was made, except some places,
> >> where GridH2Table instance was expected, but for system views there is
> >> another class.
> >>
> >> New PR: [1].  Please have a look.
> >>
> >> [1] https://github.com/apache/ignite/pull/3433
> >>
> >> 2018-01-24 19:12 GMT+03:00 Anton Vinogradov :
> >>
> >>> I've created IEP-13 [1] to cover all cases.
> >>> Feel free to create issues.
> >>>
> >>> [1]
> >>> https://cwiki.apache.org/confluence/pages/viewpage.
> action?pageId=75962769
> >>>
> >>> On Wed, Jan 24, 2018 at 6:10 PM, Vladimir Ozerov  >
> >>> wrote:
> >>>
>  Let's start with a single and the most simple view, e.g.
>  LOCAL_TRANSACTIONS. We will review and merge it along with necessary
>  infrastructure. Then will handle the rest view in separate tickets and
>  separate focused discussions.
> 
>  On Wed, Jan 24, 2018 at 5:29 PM, Alex Plehanov <
> plehanov.a...@gmail.com
> 
>  wrote:
> 
> > 1) It’s not a principal point, I can change schema. The
>  INFORMATION_SCHEMA
> > was used because it’s already exists and usually used for metadata
> >>> tables
> > and views. Your proposal is to use schema “IGNITE”, am I understand
> >>> you
> > right? BTW, for now, we can’t query another (H2) meta tables from the
> > INFORMATION_SCHEMA, so, “Ignite system views” is only available views
> >>> to
> > query from this schema.
> > 2) Exactly for this reason the IGNITE_INSTANCE view is useful: to
>  determine
> > which node we are connected to.
> > 3) As the first phase, in my opinion, local views will be enough.
> > Performance and caching of distributed views should be discussed at
> >>> next
> > phases, when distributed views implementation will be planned. In
> >>> current
> > implementation I tried to use indexing for local views wherever it’s
> > possible.
> > 4) I don’t think, that JVM info is more critical information than,
> for
> > example, caches or nodes information. When authorization capabilities
> > planned to implement?
> >
> > About local data: yes, we can rename all currently implemented views
> >>> for
> > the local node data as LOCAL_..., and create (someday) new whole
> >>> cluster
> > views (which use distributed requests) without prefix or, for
> example,
>  with
> > CLUSTER_ prefix. But some views can show all cluster information
> using
>  only
> > local node data, without distributed requests (for example
> > IGNITE_NODE_METRICS, IGNITE_PART_ASSIGNMENT, IGNITE_PART_ALLOCATION,
> > IGNITE_NODES, etc). Are they local or cluster views in this concept?
>  Which
> > prefix should be used? And what about caches? Are they local or
> >>> cluster?
>  On
> > local node we can see cluster wide caches (replicated and
> distributed)
>  and
> > caches for current node only. Local caches list may differ from node
> >>> to
> > node. Which prefix should be used for this view? And one more, there
> >>> is
>  no
> > sense for some views to make them cluster wide (for example
> > INGNITE_INSTANCE). Should we name it LOCAL_INSTANCE without creating
> > INSTANCE view?
> >
> > So, next steps: split PR, change schema name (IGNITE?), change view
> >>> name
> > for caches (CACHES, LOCAL_CACHES?)
> >
> >
> > 2018-01-24 13:03 GMT+03:00 Vladimir Ozerov :
> >
> >> Hi Alex,
> >>
> >> 

Re: Author in header of class

[Vyacheslav Daradur]

Hi Anton,

Apache Ignite is licensed under the Apache License Version 2.0, which
does not allow to use author tag in java code [1].

[1] 
http://directory.apache.org/fortress/coding-standards.html#classinterface-headers

On Wed, Feb 14, 2018 at 11:23 AM, Антон Калашников  wrote:
> Hello Ignite Community!
>
> My name is Anton. I joined to community some time ago and I want to 
> contribute to Apache Ignite.
>
> I would be want to make my first proposal. I noticed that Ignite don't have 
> author description in header of classes unlike many other apache projects.
> I propose to use javadoc tag @author in header of class when you created it  
> and also add extra tag @author when you do many changes in this class.
>
> Profits of this aproach, in my opinion:
> 1) You always know who has knowledge of this class unlike git annotate which 
> are sensetive in refactoring, moving, etc.
> 2) It will highed responsibilty for code quality because nobody want to sign 
> under bad code)
>
> Example:
> /**
>  * @author Anton Kalashnikov
>  * @author Other Author
>  */
>
> --
> Best Regards,
> Anton Kalashnikov
>



-- 
Best Regards, Vyacheslav D.

[GitHub] ignite pull request #3520: IGNITE-7697 Update maven-javadoc-plugin version

[vveider]

GitHub user vveider opened a pull request:

https://github.com/apache/ignite/pull/3520

IGNITE-7697 Update maven-javadoc-plugin version

 * updated maven-javadoc-plugin version
 * added maven-javadoc-plugin version override to ignite-apache-license-gen
 * updated flatten-maven-plugin for tread safety reasons

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7697

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3520.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3520


commit a37180843399da435fc43e828a547596d64fd414
Author: Ivanov Petr 
Date:   2018-02-14T07:33:48Z

IGNITE-7697 Update maven-javadoc-plugin version
 * updated maven-javadoc-plugin version
 * added maven-javadoc-plugin version override to ignite-apache-license-gen
 * updated flatten-maven-plugin for tread safety reasons




---

Author in header of class

[Антон Калашников]

Hello Ignite Community!

My name is Anton. I joined to community some time ago and I want to contribute 
to Apache Ignite.

I would be want to make my first proposal. I noticed that Ignite don't have 
author description in header of classes unlike many other apache projects. 
I propose to use javadoc tag @author in header of class when you created it  
and also add extra tag @author when you do many changes in this class.

Profits of this aproach, in my opinion:
1) You always know who has knowledge of this class unlike git annotate which 
are sensetive in refactoring, moving, etc.
2) It will highed responsibilty for code quality because nobody want to sign 
under bad code)

Example:
/**
 * @author Anton Kalashnikov
 * @author Other Author
 */

-- 
Best Regards,
Anton Kalashnikov

[GitHub] ignite pull request #3518: IGNITE-7690 Move shared memory suite to Ignite Ba...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3518


---