[jira] [Created] (IGNITE-10503) Meta information for vectors

2018-12-02 Thread Alexey Platonov (JIRA) 
Alexey Platonov created IGNITE-10503:


 Summary: Meta information for vectors
 Key: IGNITE-10503
 URL: https://issues.apache.org/jira/browse/IGNITE-10503
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov


We need to design and implement vector meta-information like feature names, 
bagging information, etc



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Apache Ignite 2.7. Last Mile

2018-12-02 Thread Ivan Fedotov 
I've created the PR  which
includes changes 
just before integration MVCC with Continuous Query and from the TeamCity

it is clear that before this changes the
test testAtomicOnheapTwoBackupAsyncFullSync is green.

Also Roman Kondakov gave his view on this problem in the comments
. Now the problem
becomes more understandable, but the root reason is still unclear.

May be a few of you have any suggestions why hang of threads on the binary
metadata registration future appears?

пт, 30 нояб. 2018 г. в 13:48, Ivan Fedotov :

> Igor, thank you for explanation.
>
> Now it seems that when the one thread tries to invoke
> GridCacheMapEntry#touch, the another one makes
> GridCacheProcessor#stopCache. If I am wrong, please feel free to correct me.
>
> But it still does not clear for me why this fail appears after commit
> 
>  which
> is about MVCC. Moreover, NPE appears only with BinaryObjectException, and
> when the test is green, I can not find NPE in the log.
>
> Now I tried to run test locally 1000 times on the version before MVCC and
> could not find error on this concretely case (but it exists the another
> one
> 
>  which
> is about assertion on received events).
>
> пт, 30 нояб. 2018 г. в 13:37, Roman Kondakov :
>
>> Nikolay,
>>
>> I couldn't quickly find the root cause of this problem because I'm not
>> an expert in the binary metadata flow. I think community should decide
>> whether this is a release blocker or not.
>>
>>
>> --
>> Kind Regards
>> Roman Kondakov
>>
>> On 30.11.2018 13:23, Nikolay Izhikov wrote:
>> > Hello, Roman.
>> >
>> > Is this issue blocks the 2.7 release?
>> >
>> > пт, 30 нояб. 2018 г., 13:19 Roman Kondakov kondako...@mail.ru.invalid:
>> >
>> >> Hi all!
>> >>
>> >> I've reproduced this problem locally and attached the log to the ticket
>> >> in my comment [1].
>> >>
>> >> As Igor noted, NPE there is caused by node stop in the end of the test.
>> >> The real problem here seems to be in the binary metadata registration
>> flow.
>> >>
>> >>
>> >> [1]
>> >>
>> >>
>> https://issues.apache.org/jira/browse/IGNITE-10376?focusedCommentId=16704510=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16704510
>> >>
>> >> --
>> >> Kind Regards
>> >> Roman Kondakov
>> >>
>> >> On 30.11.2018 11:56, Seliverstov Igor wrote:
>> >>> Null pointer there due to cache stop. Look at GridCacheContext#cleanup
>> >>> (GridCacheContext.java:2050)
>> >>> which is called by GridCacheProcessor#stopCache
>> >>> (GridCacheProcessor.java:1372)
>> >>>
>> >>> That's why at the time GridCacheMapEntry#touch
>> >> (GridCacheMapEntry.java:5063)
>> >>>invoked there is no eviction manager.
>> >>>
>> >>> This is a result of "normal" flow because message processing doesn't
>> >> enter
>> >>> cache gate like user API does.
>> >>>
>> >>> пт, 30 нояб. 2018 г. в 10:26, Nikolay Izhikov :
>> >>>
>>  Ivan. Please, provide a link for a ticket with NPE stack trace
>> attached.
>> 
>>  I've looked at IGNITE-10376 and can't see any attachments.
>> 
>>  пт, 30 нояб. 2018 г., 10:14 Ivan Fedotov ivanan...@gmail.com:
>> 
>> > Igor,
>> > NPE is available in a full log, now I also attached it in the
>> ticket.
>> >
>> > IGNITE-7953
>> > <
>> >
>> >>
>> https://github.com/apache/ignite/commit/51a202a4c48220fa919f47147bd4889033cd35a8
>> > was commited on the 15 October. I could not take a look on the
>> > testAtomicOnheapTwoBackupAsyncFullSync before this date, because the
>>  oldest
>> > test in the history on TC dates 12 November.
>> >
>> > So, I tested it locally and could not reproduce mentioned error.
>> >
>> > чт, 29 нояб. 2018 г. в 20:07, Seliverstov Igor <
>> gvvinbl...@gmail.com>:
>> >
>> >> Ivan,
>> >>
>> >> Could you provide a bit more details?
>> >>
>> >> I don't see any NPE among all available logs.
>> >>
>> >> I don't think the issue is caused by changes in scope of
>> IGNITE-7953.
>> >> The test fails both before
>> >> <
>> >>
>> >>
>> https://ci.ignite.apache.org/viewLog.html?buildId=2318582=buildResultsDiv=IgniteTests24Java8_ContinuousQuery4#testNameId3300126853696550025
>> >>and after
>> >> <
>> >>
>> >>
>> https://ci.ignite.apache.org/viewLog.html?buildId=2345403=buildResultsDiv=IgniteTests24Java8_ContinuousQuery4#testNameId3300126853696550025
>> >> the
>> >> commit was merged to master with almost the same stack trace.
>> >>
>> >>

[jira] [Created] (IGNITE-10502) Add a danger block into transactions documentation

2018-12-02 Thread Roman Guseinov (JIRA) 
Roman Guseinov created IGNITE-10502:
---

 Summary: Add a danger block into transactions documentation
 Key: IGNITE-10502
 URL: https://issues.apache.org/jira/browse/IGNITE-10502
 Project: Ignite
  Issue Type: Bug
  Components: documentation
Affects Versions: 2.6
Reporter: Roman Guseinov


Currently, we can not safely use caches with CacheStore and caches without this 
one inside the same transaction. Those types of caches have different recovery 
methods and some topology changes can lead inconsistent data:

[https://issues.apache.org/jira/browse/IGNITE-10452|https://issues.apache.org/jira/browse/IGNITE-10452]

It should be mentioned in the documentation to avoid confusing the users.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [VOTE] Creation dedicated list for github notifiacations

2018-12-02 Thread Dmitriy Govorukhin 
+1

вт, 27 нояб. 2018 г., 19:41 Denis Mekhanikov dmekhani...@gmail.com:

> +1
> I'm for making the dev list readable without filters of any kind.
>
> On Tue, Nov 27, 2018, 15:14 Maxim Muzafarov 
> > +1
> >
> > Let's have a look at how it will be.
> >
> > On Tue, 27 Nov 2018 at 14:48 Seliverstov Igor 
> > wrote:
> >
> > > +1
> > >
> > > вт, 27 нояб. 2018 г. в 14:45, Юрий :
> > >
> > > > +1
> > > >
> > > > вт, 27 нояб. 2018 г. в 11:22, Andrey Mashenkov <
> > > andrey.mashen...@gmail.com
> > > > >:
> > > >
> > > > > +1
> > > > >
> > > > > On Tue, Nov 27, 2018 at 10:12 AM Sergey Chugunov <
> > > > > sergey.chugu...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > Plus this dedicated list should be properly documented in wiki,
> > > > > mentioning
> > > > > > it in How to Contribute [1] or in Make Teamcity Green Again [2]
> > would
> > > > be
> > > > > a
> > > > > > good idea.
> > > > > >
> > > > > > [1]
> > > > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
> > > > > > [2]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Make+Teamcity+Green+Again
> > > > > >
> > > > > > On Tue, Nov 27, 2018 at 9:51 AM Павлухин Иван <
> vololo...@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > +1
> > > > > > > вт, 27 нояб. 2018 г. в 09:22, Dmitrii Ryabov <
> > > somefire...@gmail.com
> > > > >:
> > > > > > > >
> > > > > > > > 0
> > > > > > > > вт, 27 нояб. 2018 г. в 02:33, Alexey Kuznetsov <
> > > > > akuznet...@apache.org
> > > > > > >:
> > > > > > > > >
> > > > > > > > > +1
> > > > > > > > > Do not forget notification from GitBox too!
> > > > > > > > >
> > > > > > > > > On Tue, Nov 27, 2018 at 2:20 AM Zhenya
> > > >  > > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > +1, already make it by filers.
> > > > > > > > > >
> > > > > > > > > > > This was discussed already [1].
> > > > > > > > > > >
> > > > > > > > > > > So, I want to complete this discussion with moving
> > outside
> > > > > > dev-list
> > > > > > > > > > > GitHub-notification to dedicated list.
> > > > > > > > > > >
> > > > > > > > > > > Please start voting.
> > > > > > > > > > >
> > > > > > > > > > > +1 - to accept this change.
> > > > > > > > > > > 0 - you don't care.
> > > > > > > > > > > -1 - to decline this change.
> > > > > > > > > > >
> > > > > > > > > > > This vote will go for 72 hours.
> > > > > > > > > > >
> > > > > > > > > > > [1]
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Time-to-remove-automated-messages-from-the-devlist-td37484i20.html
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Alexey Kuznetsov
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > > Ivan Pavlukhin
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Andrey V. Mashenkov
> > > > >
> > > >
> > > >
> > > > --
> > > > Живи с улыбкой! :D
> > > >
> > >
> > --
> > --
> > Maxim Muzafarov
> >
>

Re: JDBC thin driver: support connection timeout

2018-12-02 Thread Vladimir Ozerov 
+1

вс, 2 дек. 2018 г. в 18:39, Павлухин Иван :

> Missing ref:
> [2]
> https://mvnrepository.com/artifact/org.mariadb.jdbc/mariadb-java-client/2.3.0
>
> 2018-12-02 18:31 GMT+03:00, Павлухин Иван :
> > Hi Alexander,
> >
> > I have 2 points.
> >
> > 1. According to the jdbc spec [1] setNetworkTimeout method is
> > optional. What user problem we are going to solve by implementing that
> > method?
> > 2. Also I checked another quite popular jdbc driver provided by
> > MariaDB [2]. They ignore an executor argument as well and set a socket
> > timeout instead. So, I think that we are on a safe side if we ignore
> > an executor.
> >
> > [1]
> https://download.oracle.com/otndocs/jcp/jdbc-4_2-mrel2-spec/index.html
> > пт, 30 нояб. 2018 г. в 16:28, Alexander Lapin :
> >>
> >> Hi Igniters,
> >>
> >> Within context of connection timeout [
> >> https://issues.apache.org/jira/browse/IGNITE-5234] it's not obvious
> >> whether
> >> it's required to use setNetworkTimeout's executor or not.
> >>
> >> According to the javadoc of
> >> java.sql.Connection#setNetworkTimeout(Executor
> >> executor, int milliseconds), executor is "The Executor
> >> implementation which will be used by setNetworkTimeout."
> >> Seems that executor supposed to take care of connection closing/aborting
> >> in
> >> case of timeout, based on submitted Runnable implementation. On the
> other
> >> hand it's possible to ignore executor and implement
> >> timeout-detection/cancellation logic with Timer. Something like
> following
> >> (pseudo-code):
> >>
> >> ConnectionTimeoutTimerTask connectionTimeoutTimerTask = new
> >> ConnectionTimeoutTimerTask(timeout);
> >> timer.schedule(connectionTimeoutTimerTask, 0, REQUEST_TIMEOUT_PERIOD);
> >> ...
> >> JdbcResponse res = cliIo.sendRequest(req);
> >> ...
> >>
> >> private class ConnectionTimeoutTimerTask extends TimerTask {
> >> ...
> >> @Override public void run() {
> >> if (remainingConnectionTimeout <= 0)
> >> close(); //connection.close();
> >>
> >> remainingConnectionTimeout -= REQUEST_TIMEOUT_PERIOD;
> >> }
> >> ...
> >> }
> >>
> >> It worth to mention that MSSQL Jdbc driver doesn't use executor and
> >> PostgreSQL doesn't implement setNetworkTimeout() at all.
> >>
> >> From my point of view it might be better to ignore executor, is it
> >> suitable?
> >>
> >> Any ideas?
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

Re: [VOTE] Apache Ignite 2.7.0 RC2

2018-12-02 Thread Dmitriy Pavlov 
+1 binding

I've checked new RC using Apache Ignite TeamCity Bot. The bot now uses
Apache Ignite V2.7.0-RC2, tested locally & deployed to the server.

PS Nikolay, thanks for sharing the link.

вс, 2 дек. 2018 г. в 21:58, Nikolay Izhikov :

> Hello, Dmitriy
>
> RC2 artifacts are here -
> https://repository.apache.org/content/repositories/orgapacheignite-1435/
>
> В Вс, 02/12/2018 в 01:08 +0300, Dmitriy Pavlov пишет:
> > Nikolay, Igniters,
> >
> > Could you please advice where can I find a staging for RC-2?
> >
> > I can't find it in https://repository.apache.org/content/repositories/
> >
> > Or should I reuse the old one?
> > https://repository.apache.org/content/repositories/orgapacheignite-1431/
> >
> > Thank you in advance.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > сб, 1 дек. 2018 г. в 09:48, Pavel Tupitsyn :
> >
> > > +1
> > >
> > > Downloaded sources, build Java and .NET parts, ran examples.
> > > There is a minor issue with .NET Core examples, compiler warning is
> > > displayed (certainly not a blocker) [1]
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-10500
> > >
> > > On Sat, Dec 1, 2018 at 12:47 AM Nikolay Izhikov 
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > We've uploaded a 2.7.0 release candidate to
> > > >
> > > > https://dist.apache.org/repos/dist/dev/ignite/2.7.0-rc2/
> > > >
> > > > Git tag name is 2.7.0-rc2
> > > >
> > > > This release includes the following changes:
> > > >
> > > > Apache Ignite In-Memory Database and Caching Platform 2.7
> > > > -
> > > >
> > > > Ignite:
> > > > * Added experimental support for multi-version concurrency control
> with
> > > > snapshot isolation
> > > >   - available for both cache API and SQL
> > > >   - use CacheAtomicityMode.TRANSACTIONAL_SNAPSHOT to enable it
> > > >   - not production ready, data consistency is not guaranteed in case
> of
> > > > node failures
> > > > * Implemented Transparent Data Encryption based on JKS certificates
> > > > * Implemented Node.JS Thin Client
> > > > * Implemented Python Thin Client
> > > > * Implemented PHP Thin Client
> > > > * Ignite start scripts now support Java 9 and higher
> > > > * Added ability to set WAL history size in bytes
> > > > * Added SslContextFactory.protocols and
> SslContextFactory.cipherSuites
> > > > properties to control which SSL encryption algorithms can be used
> > > > * Added JCache 1.1 compliance
> > > > * Added IgniteCompute.withNoResultCache method with semantics
> similar to
> > > > ComputeTaskNoResultCache annotation
> > > > * Spring Data 2.0 is now supported in the separate module
> > > > 'ignite-spring-data_2.0'
> > > > * Added monitoring of critical system workers
> > > > * Added ability to provide custom implementations of
> ExceptionListener
> > >
> > > for
> > > > JmsStreamer
> > > > * Ignite KafkaStreamer was upgraded to use new KafkaConsmer
> configuration
> > > > * S3 IP Finder now supports subfolder usage instead of bucket root
> > > > * Improved dynamic cache start speed
> > > > * Improved checkpoint performance by decreasing mark duration.
> > > > * Added ability to manage compression level for compressed WAL
> archives.
> > > > * Added metrics for Entry Processor invocations.
> > > > * Added JMX metrics: ClusterMetricsMXBean.getTotalBaselineNodes and
> > > > ClusterMetricsMXBean.getActiveBaselineNodes
> > > > * Node uptime metric now includes days count
> > > > * Exposed info about thin client connections through JMX
> > > > * Introduced new system property IGNITE_REUSE_MEMORY_ON_DEACTIVATE to
> > > > enable reuse of allocated memory on node deactivation (disabled by
> > >
> > > default)
> > > > * Optimistic transaction now will be properly rolled back if waiting
> too
> > > > long for a new topology on remap
> > > > * ScanQuery with setLocal flag now checks if the partition is
> actually
> > > > present on local node
> > > > * Improved cluster behaviour when a left node does not cause
> partition
> > > > affinity assignment changes
> > > > * Interrupting user thread during partition initialization will no
> longer
> > > > cause node to stop
> > > > * Fixed problem when partition lost event was not triggered if
> multiple
> > > > nodes left cluster
> > > > * Fixed massive node drop from the cluster on temporary network
> issues
> > > > * Fixed service redeployment on cluster reactivation
> > > > * Fixed client node stability under ZooKeeper discovery
> > > > * Massive performance and stability improvements
> > > >
> > > > Ignite .Net:
> > > > * Add .NET Core 2.1 support
> > > > * Added thin client connection failover
> > > >
> > > > Ignite C++:
> > > > * Implemented Thin Client with base cache operations
> > > > * Implemented smart affinity routing for Thin Client to send requests
> > > > directly to nodes containing data when possible
> > > > * Added Clang compiler support
> > > >
> > > > SQL:
> > > > * Added experimental support for fully ACID transactional SQL with
> the
> > > >

Re: [VOTE] Apache Ignite 2.7.0 RC2

2018-12-02 Thread Nikolay Izhikov 
Hello, Dmitriy

RC2 artifacts are here - 
https://repository.apache.org/content/repositories/orgapacheignite-1435/

В Вс, 02/12/2018 в 01:08 +0300, Dmitriy Pavlov пишет:
> Nikolay, Igniters,
> 
> Could you please advice where can I find a staging for RC-2?
> 
> I can't find it in https://repository.apache.org/content/repositories/
> 
> Or should I reuse the old one?
> https://repository.apache.org/content/repositories/orgapacheignite-1431/
> 
> Thank you in advance.
> 
> Sincerely,
> Dmitriy Pavlov
> 
> сб, 1 дек. 2018 г. в 09:48, Pavel Tupitsyn :
> 
> > +1
> > 
> > Downloaded sources, build Java and .NET parts, ran examples.
> > There is a minor issue with .NET Core examples, compiler warning is
> > displayed (certainly not a blocker) [1]
> > 
> > [1] https://issues.apache.org/jira/browse/IGNITE-10500
> > 
> > On Sat, Dec 1, 2018 at 12:47 AM Nikolay Izhikov 
> > wrote:
> > 
> > > Igniters,
> > > 
> > > We've uploaded a 2.7.0 release candidate to
> > > 
> > > https://dist.apache.org/repos/dist/dev/ignite/2.7.0-rc2/
> > > 
> > > Git tag name is 2.7.0-rc2
> > > 
> > > This release includes the following changes:
> > > 
> > > Apache Ignite In-Memory Database and Caching Platform 2.7
> > > -
> > > 
> > > Ignite:
> > > * Added experimental support for multi-version concurrency control with
> > > snapshot isolation
> > >   - available for both cache API and SQL
> > >   - use CacheAtomicityMode.TRANSACTIONAL_SNAPSHOT to enable it
> > >   - not production ready, data consistency is not guaranteed in case of
> > > node failures
> > > * Implemented Transparent Data Encryption based on JKS certificates
> > > * Implemented Node.JS Thin Client
> > > * Implemented Python Thin Client
> > > * Implemented PHP Thin Client
> > > * Ignite start scripts now support Java 9 and higher
> > > * Added ability to set WAL history size in bytes
> > > * Added SslContextFactory.protocols and SslContextFactory.cipherSuites
> > > properties to control which SSL encryption algorithms can be used
> > > * Added JCache 1.1 compliance
> > > * Added IgniteCompute.withNoResultCache method with semantics similar to
> > > ComputeTaskNoResultCache annotation
> > > * Spring Data 2.0 is now supported in the separate module
> > > 'ignite-spring-data_2.0'
> > > * Added monitoring of critical system workers
> > > * Added ability to provide custom implementations of ExceptionListener
> > 
> > for
> > > JmsStreamer
> > > * Ignite KafkaStreamer was upgraded to use new KafkaConsmer configuration
> > > * S3 IP Finder now supports subfolder usage instead of bucket root
> > > * Improved dynamic cache start speed
> > > * Improved checkpoint performance by decreasing mark duration.
> > > * Added ability to manage compression level for compressed WAL archives.
> > > * Added metrics for Entry Processor invocations.
> > > * Added JMX metrics: ClusterMetricsMXBean.getTotalBaselineNodes and
> > > ClusterMetricsMXBean.getActiveBaselineNodes
> > > * Node uptime metric now includes days count
> > > * Exposed info about thin client connections through JMX
> > > * Introduced new system property IGNITE_REUSE_MEMORY_ON_DEACTIVATE to
> > > enable reuse of allocated memory on node deactivation (disabled by
> > 
> > default)
> > > * Optimistic transaction now will be properly rolled back if waiting too
> > > long for a new topology on remap
> > > * ScanQuery with setLocal flag now checks if the partition is actually
> > > present on local node
> > > * Improved cluster behaviour when a left node does not cause partition
> > > affinity assignment changes
> > > * Interrupting user thread during partition initialization will no longer
> > > cause node to stop
> > > * Fixed problem when partition lost event was not triggered if multiple
> > > nodes left cluster
> > > * Fixed massive node drop from the cluster on temporary network issues
> > > * Fixed service redeployment on cluster reactivation
> > > * Fixed client node stability under ZooKeeper discovery
> > > * Massive performance and stability improvements
> > > 
> > > Ignite .Net:
> > > * Add .NET Core 2.1 support
> > > * Added thin client connection failover
> > > 
> > > Ignite C++:
> > > * Implemented Thin Client with base cache operations
> > > * Implemented smart affinity routing for Thin Client to send requests
> > > directly to nodes containing data when possible
> > > * Added Clang compiler support
> > > 
> > > SQL:
> > > * Added experimental support for fully ACID transactional SQL with the
> > > snapshot isolation:
> > >   - use CacheAtomicityMode.TRANSACTIONAL_SNAPSHOT to enable it
> > >   - a transaction can be started through native API (IgniteTransactions),
> > > thin JDBC driver or ODBC driver
> > >   - not production ready, data consistency is not guaranteed in case of
> > > node failures
> > > * Added a set of system views located in "IGNITE" schema to view cluster
> > > information (NODES, NODE_ATTRIBUTES, NODE_METRICS, BASELINE_NODES)
> > > * Added

Re: Service grid redesign

2018-12-02 Thread Vyacheslav Daradur 
Hi, Igniters!

I fixed the notes and tests seem good.

So, let's continue the review [1] [2], any feedback is welcome!

[1] https://github.com/apache/ignite/pull/4434
[2] https://issues.apache.org/jira/browse/IGNITE-9607
On Tue, Nov 27, 2018 at 5:16 PM Vyacheslav Daradur  wrote:
>
> We had the private talk with Nikolay Izhikov, Vladimir Ozerov, Alexey
> Goncharuk, Yakov Zhdanov, Denis Mekhanikov, Dmitriy Pavlov and I would
> like to share the summary with the community:
>
> The architecture of the implemented deployment process looks good in
> general, but the following points should be improved:
> * The new implementation of service processor implementation should be
> moved to a new class;
> * A new system property should be introduced to allow users to switch
> to old implementation in case of errors;
> * Introduced service deployment failures policy should be removed from
> current PR and should be implemented as a different task with detailed
> discussion on dev list to avoid unexpected behavior;
> * The word "exchange" should be removed from classes names to avoid
> confusion with PME classes.
> * Single/full messages should include containing only deployment
> process-related information only (instead of all service) to reduce
> messages size;
>
> Thanks all! I'll let you know once I fix the notes.
>
> On Wed, Nov 21, 2018 at 4:28 PM Dmitriy Pavlov  wrote:
> >
> > Hi Vyacheslav, Vladimir,
> >
> > Could you please invite me, if you will set up a call.
> >
> > ср, 21 нояб. 2018 г. в 13:08, Vladimir Ozerov :
> >
> > > Hi Vyacheslav,
> > >
> > > Still not clear enough for me. I do not see a reason to send another over 
> > > a
> > > ring in case of successful execution. The only reason is an error on a 
> > > node
> > > which require correction (re-deploy to other node, full service undeploy,
> > > etc).
> > > I think it makes sense to organize another call to discuss current
> > > architecture. Otherwise we may spend too much time on emails.
> > >
> > > Vladimir.
> > >
> > > On Wed, Nov 21, 2018 at 12:57 PM Vyacheslav Daradur 
> > > wrote:
> > >
> > > > The full map is needed:
> > > > 1) to propagate deployment results which could be different from
> > > > locally calculated in case of any errors;
> > > > 2) to transfer deployment errors across the cluster;
> > > > 3) to undeploy exceeded the number of service instances if needed;
> > > > 4) to get know other nodes that deployment process was finished, this
> > > > need to avoid calling services which have not been deployed yet (or
> > > > can't be deployed). We can't just store pending requests because of
> > > > time to deploy one service instances which may be significant.
> > > > On Wed, Nov 21, 2018 at 12:45 PM Vladimir Ozerov 
> > > > wrote:
> > > > >
> > > > > Vyacheslav,
> > > > >
> > > > > I looked at the document and failed to find explanation why full maps
> > > are
> > > > > needed. Could you point me to a place where it is explained?
> > > > > I ask this because my impression from last discussion was that it is
> > > > never
> > > > > needed. Service status change is initiated by user action, then all
> > > nodes
> > > > > perform respective action locally, then they reply to coordinator, 
> > > > > then
> > > > > coordinator reply to the client, no need a kind of "full" map over
> > > > > discovery again. The only situation when another message over ring is
> > > > > required, is when some node failed to execute local operation (for
> > > > whatever
> > > > > reason) and corrective action is required.
> > > > >
> > > > > Am I missing something?
> > > > >
> > > > > On Wed, Nov 21, 2018 at 11:50 AM Vyacheslav Daradur <
> > > daradu...@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Denis, I suggested new names above in the thread.
> > > > > >
> > > > > > Please, look at PME document [1] is should be quiet actual to show
> > > the
> > > > > > same flow.
> > > > > >
> > > > > > [1]
> > > > > >
> > > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/%28Partition+Map%29+Exchange+-+under+the+hood
> > > > > >
> > > > > > On Wed, Nov 21, 2018 at 11:43 AM Denis Mekhanikov <
> > > > dmekhani...@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > Vyacheslav,
> > > > > > >
> > > > > > > Actually, the service assignment is implemented in a way,
> > > > > > > that allows every node calculate the assignment itself, so no
> > > > information
> > > > > > > needs to be shared.
> > > > > > > The only data, that is sent between nodes is deployment results,
> > > > > > > and I don't see an analogy with exchange here.
> > > > > > >
> > > > > > > Denis
> > > > > > >
> > > > > > > ср, 21 нояб. 2018 г. в 11:16, Vladimir Ozerov <
> > > voze...@gridgain.com
> > > > >:
> > > > > > >
> > > > > > > > Hi Vyacheslav,
> > > > > > > >
> > > > > > > > Could you please explain in what situation coordinator needs to
> > > > collect
> > > > > > > > service deployments info from all nodes and share it with the
> > > > cluster?
> > > > > > I
> > > > >

Re: JDBC thin driver: support connection timeout

2018-12-02 Thread Павлухин Иван 
Missing ref:
[2] 
https://mvnrepository.com/artifact/org.mariadb.jdbc/mariadb-java-client/2.3.0

2018-12-02 18:31 GMT+03:00, Павлухин Иван :
> Hi Alexander,
>
> I have 2 points.
>
> 1. According to the jdbc spec [1] setNetworkTimeout method is
> optional. What user problem we are going to solve by implementing that
> method?
> 2. Also I checked another quite popular jdbc driver provided by
> MariaDB [2]. They ignore an executor argument as well and set a socket
> timeout instead. So, I think that we are on a safe side if we ignore
> an executor.
>
> [1] https://download.oracle.com/otndocs/jcp/jdbc-4_2-mrel2-spec/index.html
> пт, 30 нояб. 2018 г. в 16:28, Alexander Lapin :
>>
>> Hi Igniters,
>>
>> Within context of connection timeout [
>> https://issues.apache.org/jira/browse/IGNITE-5234] it's not obvious
>> whether
>> it's required to use setNetworkTimeout's executor or not.
>>
>> According to the javadoc of
>> java.sql.Connection#setNetworkTimeout(Executor
>> executor, int milliseconds), executor is "The Executor
>> implementation which will be used by setNetworkTimeout."
>> Seems that executor supposed to take care of connection closing/aborting
>> in
>> case of timeout, based on submitted Runnable implementation. On the other
>> hand it's possible to ignore executor and implement
>> timeout-detection/cancellation logic with Timer. Something like following
>> (pseudo-code):
>>
>> ConnectionTimeoutTimerTask connectionTimeoutTimerTask = new
>> ConnectionTimeoutTimerTask(timeout);
>> timer.schedule(connectionTimeoutTimerTask, 0, REQUEST_TIMEOUT_PERIOD);
>> ...
>> JdbcResponse res = cliIo.sendRequest(req);
>> ...
>>
>> private class ConnectionTimeoutTimerTask extends TimerTask {
>> ...
>> @Override public void run() {
>> if (remainingConnectionTimeout <= 0)
>> close(); //connection.close();
>>
>> remainingConnectionTimeout -= REQUEST_TIMEOUT_PERIOD;
>> }
>> ...
>> }
>>
>> It worth to mention that MSSQL Jdbc driver doesn't use executor and
>> PostgreSQL doesn't implement setNetworkTimeout() at all.
>>
>> From my point of view it might be better to ignore executor, is it
>> suitable?
>>
>> Any ideas?
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>


-- 
Best regards,
Ivan Pavlukhin

Re: JDBC thin driver: support connection timeout

2018-12-02 Thread Павлухин Иван 
Hi Alexander,

I have 2 points.

1. According to the jdbc spec [1] setNetworkTimeout method is
optional. What user problem we are going to solve by implementing that
method?
2. Also I checked another quite popular jdbc driver provided by
MariaDB [2]. They ignore an executor argument as well and set a socket
timeout instead. So, I think that we are on a safe side if we ignore
an executor.

[1] https://download.oracle.com/otndocs/jcp/jdbc-4_2-mrel2-spec/index.html
пт, 30 нояб. 2018 г. в 16:28, Alexander Lapin :
>
> Hi Igniters,
>
> Within context of connection timeout [
> https://issues.apache.org/jira/browse/IGNITE-5234] it's not obvious whether
> it's required to use setNetworkTimeout's executor or not.
>
> According to the javadoc of java.sql.Connection#setNetworkTimeout(Executor
> executor, int milliseconds), executor is "The Executor
> implementation which will be used by setNetworkTimeout."
> Seems that executor supposed to take care of connection closing/aborting in
> case of timeout, based on submitted Runnable implementation. On the other
> hand it's possible to ignore executor and implement
> timeout-detection/cancellation logic with Timer. Something like following
> (pseudo-code):
>
> ConnectionTimeoutTimerTask connectionTimeoutTimerTask = new
> ConnectionTimeoutTimerTask(timeout);
> timer.schedule(connectionTimeoutTimerTask, 0, REQUEST_TIMEOUT_PERIOD);
> ...
> JdbcResponse res = cliIo.sendRequest(req);
> ...
>
> private class ConnectionTimeoutTimerTask extends TimerTask {
> ...
> @Override public void run() {
> if (remainingConnectionTimeout <= 0)
> close(); //connection.close();
>
> remainingConnectionTimeout -= REQUEST_TIMEOUT_PERIOD;
> }
> ...
> }
>
> It worth to mention that MSSQL Jdbc driver doesn't use executor and
> PostgreSQL doesn't implement setNetworkTimeout() at all.
>
> From my point of view it might be better to ignore executor, is it suitable?
>
> Any ideas?



-- 
Best regards,
Ivan Pavlukhin