[jira] [Closed] (IGNITE-4361) Documentation: working with Ignite through JCache API

[Denis Magda (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis Magda closed IGNITE-4361.
---

> Documentation: working with Ignite through JCache API
> -
>
> Key: IGNITE-4361
> URL: https://issues.apache.org/jira/browse/IGNITE-4361
> Project: Ignite
>  Issue Type: Task
>  Components: documentation
>Reporter: Denis Magda
>Assignee: Denis Magda
>Priority: Minor
> Fix For: 2.4
>
>
> Regardless of the fact that Ignite supports JCache API there is no any 
> documentation with code snippets that will elaborate on how to work with 
> Ignite purely with JCache API.
> This documentation [1] must contain a section showing all the ways how a 
> JCache Ignite provider can be created and used after that.
> [1] https://apacheignite.readme.io/docs/jcache



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7503) MLP documentation

[Denis Magda (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351077#comment-16351077
 ] 

Denis Magda commented on IGNITE-7503:
-

[~pgarg] , please review the doc instead of me.

> MLP documentation
> -
>
> Key: IGNITE-7503
> URL: https://issues.apache.org/jira/browse/IGNITE-7503
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation, ml
>Reporter: Yury Babak
>Assignee: Denis Magda
>Priority: Major
>  Labels: documentaion
> Fix For: 2.4
>
>
> A need to add documentation about MLP



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (IGNITE-7503) MLP documentation

[Denis Magda (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis Magda reassigned IGNITE-7503:
---

Assignee: Prachi Garg  (was: Denis Magda)

> MLP documentation
> -
>
> Key: IGNITE-7503
> URL: https://issues.apache.org/jira/browse/IGNITE-7503
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation, ml
>Reporter: Yury Babak
>Assignee: Prachi Garg
>Priority: Major
>  Labels: documentaion
> Fix For: 2.4
>
>
> A need to add documentation about MLP



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (IGNITE-7504) Decision tree documentation

[Denis Magda (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis Magda reassigned IGNITE-7504:
---

Assignee: Prachi Garg  (was: Denis Magda)

> Decision tree documentation
> ---
>
> Key: IGNITE-7504
> URL: https://issues.apache.org/jira/browse/IGNITE-7504
> Project: Ignite
>  Issue Type: Task
>  Components: documentation, ml
>Reporter: Yury Babak
>Assignee: Prachi Garg
>Priority: Major
>  Labels: documentation
> Fix For: 2.4
>
>
> We want to add Decision tree documentation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7504) Decision tree documentation

[Denis Magda (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351075#comment-16351075
 ] 

Denis Magda commented on IGNITE-7504:
-

[~oignatenko], [~amalykh] , thanks. All looks good to me.

 

[~pgarg] , please do a final review and close the ticket:

[https://dash.readme.io/project/apacheignite/v2.3/docs/decision-trees-24]

> Decision tree documentation
> ---
>
> Key: IGNITE-7504
> URL: https://issues.apache.org/jira/browse/IGNITE-7504
> Project: Ignite
>  Issue Type: Task
>  Components: documentation, ml
>Reporter: Yury Babak
>Assignee: Denis Magda
>Priority: Major
>  Labels: documentation
> Fix For: 2.4
>
>
> We want to add Decision tree documentation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (IGNITE-7532) kNN Documentation

[Denis Magda (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis Magda reassigned IGNITE-7532:
---

Assignee: Aleksey Zinoviev  (was: Denis Magda)

> kNN Documentation
> -
>
> Key: IGNITE-7532
> URL: https://issues.apache.org/jira/browse/IGNITE-7532
> Project: Ignite
>  Issue Type: Task
>  Components: documentation, ml
>Reporter: Aleksey Zinoviev
>Assignee: Aleksey Zinoviev
>Priority: Major
>  Labels: documentaion
> Fix For: 2.4
>
>
> We want to add kNN Regression and Classification docs



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7532) kNN Documentation

[Denis Magda (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351047#comment-16351047
 ] 

Denis Magda commented on IGNITE-7532:
-

[~zaleslaw] , this part was not fixed:
 * k-NN classification doc says - _trainingSet dataset loaded that might be 
taken from Iris datasets._ Please provide a link to the Iris datasets you're 
talking about.

The doc looks broken there.

> kNN Documentation
> -
>
> Key: IGNITE-7532
> URL: https://issues.apache.org/jira/browse/IGNITE-7532
> Project: Ignite
>  Issue Type: Task
>  Components: documentation, ml
>Reporter: Aleksey Zinoviev
>Assignee: Denis Magda
>Priority: Major
>  Labels: documentaion
> Fix For: 2.4
>
>
> We want to add kNN Regression and Classification docs



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7595) Find and switch to alternate documentation engine

[Denis Magda (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351025#comment-16351025
 ] 

Denis Magda commented on IGNITE-7595:
-

This blogging engine might satisfy all our needs: 
[docusaurus.io|http://docusaurus.io/]

> Find and switch to alternate documentation engine
> -
>
> Key: IGNITE-7595
> URL: https://issues.apache.org/jira/browse/IGNITE-7595
> Project: Ignite
>  Issue Type: Task
>  Components: documentation
>Reporter: Denis Magda
>Assignee: Prachi Garg
>Priority: Critical
> Fix For: 2.5
>
>
> Current readme.io documentation has many drawbacks that make the life of 
> Ignite technical writers hard. Some of the problems are:
>  * Each "version" is just a copy of the previous one. When fixing something, 
> you have to update
> all the versions.
>  * No good way to review changes.
>  * "Propose edit" functionality is a not suitable for review. You can only 
> accept or reject an
> edit, no way to communicate with a contributor, etc
>  * There is no way to prevent Google from indexing old documentation 
> versions. Thus, it's common to come across old doc version in a google 
> search. 
> We might consider GitHub based documentation or another approach. The 
> discussion is here:
> http://apache-ignite-developers.2346864.n4.nabble.com/Move-documentation-from-readme-io-to-GitHub-pages-td16409.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (IGNITE-7496) Update SQL Conformance Documentation

[Prachi Garg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prachi Garg reassigned IGNITE-7496:
---

Assignee: Denis Magda  (was: Prachi Garg)

> Update SQL Conformance Documentation
> 
>
> Key: IGNITE-7496
> URL: https://issues.apache.org/jira/browse/IGNITE-7496
> Project: Ignite
>  Issue Type: Bug
>  Components: documentation
>Reporter: Sergey Puchnin
>Assignee: Denis Magda
>Priority: Major
> Fix For: 2.4
>
>
> In a discussion on dev list
> http://apache-ignite-developers.2346864.n4.nabble.com/SparkDataFrame-Query-Optimization-Prototype-tt26249.html
> was noticed there are some gaps in SQL documentation. 
> It's necessary to review it.
> I've added next system functions:
> CASEWHEN Function, CAST, CONVERT, TABLE
> And for my mind, next functions aren't applicable for Ignite:
> ARRAY_GET, ARRAY_LENGTH, ARRAY_CONTAINS, CSVREAD, CSVWRITE, DATABASE, 
> DATABASE_PATH, DISK_SPACE_USED, FILE_READ, FILE_WRITE, LINK_SCHEMA, 
> MEMORY_FREE, MEMORY_USED, LOCK_MODE, LOCK_TIMEOUT, READONLY, CURRVAL, 
> AUTOCOMMIT, CANCEL_SESSION, IDENTITY, NEXTVAL, ROWNUM, SCHEMA, 
> SCOPE_IDENTITY, SESSION_ID, SET, TRANSACTION_ID, TRUNCATE_VALUE, USER, 
> H2VERSION



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (IGNITE-7606) Write evicted dirty page during eviction without holding segment write lock

[Dmitriy Pavlov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy Pavlov updated IGNITE-7606:
---
Attachment: chart_notHoldingLock2.png

> Write evicted dirty page during eviction without holding segment write lock
> ---
>
> Key: IGNITE-7606
> URL: https://issues.apache.org/jira/browse/IGNITE-7606
> Project: Ignite
>  Issue Type: Improvement
>  Components: persistence
>Reporter: Dmitriy Pavlov
>Assignee: Dmitriy Pavlov
>Priority: Major
> Fix For: 2.5
>
> Attachments: chart_holdingLock.png, chart_notHoldingLock.png, 
> chart_notHoldingLock2.png, putdumpAt17second.txt
>
>
> If a dirty page under the checkpoint is found, following is suggested
> - copy it to the local thread buffer, 
> - and then after performing all actions in region for evicting the page
> - finish execution allocatePage()/acquirePage()
> - unlock segment to allow other workers to operate
> - perform the pwrite() call based on the data from local buffer
> Now if page eviction started there is possible drops to 0 put/seconds in case 
> a lot of threads are watiting for same segment lock.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (IGNITE-7606) Write evicted dirty page during eviction without holding segment write lock

[Dmitriy Pavlov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy Pavlov updated IGNITE-7606:
---
Attachment: chart_notHoldingLock.png
chart_holdingLock.png

> Write evicted dirty page during eviction without holding segment write lock
> ---
>
> Key: IGNITE-7606
> URL: https://issues.apache.org/jira/browse/IGNITE-7606
> Project: Ignite
>  Issue Type: Improvement
>  Components: persistence
>Reporter: Dmitriy Pavlov
>Assignee: Dmitriy Pavlov
>Priority: Major
> Fix For: 2.5
>
> Attachments: chart_holdingLock.png, chart_notHoldingLock.png, 
> putdumpAt17second.txt
>
>
> If a dirty page under the checkpoint is found, following is suggested
> - copy it to the local thread buffer, 
> - and then after performing all actions in region for evicting the page
> - finish execution allocatePage()/acquirePage()
> - unlock segment to allow other workers to operate
> - perform the pwrite() call based on the data from local buffer
> Now if page eviction started there is possible drops to 0 put/seconds in case 
> a lot of threads are watiting for same segment lock.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7618) validateCache synchronously waits for state change

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-7618:


 Summary: validateCache synchronously waits for state change
 Key: IGNITE-7618
 URL: https://issues.apache.org/jira/browse/IGNITE-7618
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.4
Reporter: Alexey Goncharuk
 Fix For: 2.5


Currently, we call publicApiActiveState(waitForTransition=true) in 
GridDhtTopologyFutureAdapter#validateCache. This is incorrect, because the 
cluster state is updated from discovery thread, and mapping may be happening on 
a previous topology version. We must capture the cluster active state flag to 
the exchange future (I believe we already have all the mechanics for this in 
ExchangeActions class) and validate the state in the exchange future itself, 
without touching ClusterStateProcessor.

Below is an example of a deadlock happened because of synchronous wait:
{code}
"db-checkpoint-thread-#12318%database.IgniteDbBaselineTopologySelfTest0%" 
#15498 prio=5 os_prio=0 tid=0x7fa173e80800 nid=0x3d08 waiting on condition 
[0x7fa2069a8000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x0007abfc16e8> (a 
java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
at 
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.markCheckpointBegin(GridCacheDatabaseSharedManager.java:2991)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.doCheckpoint(GridCacheDatabaseSharedManager.java:2797)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseSharedManager.java:2722)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:745)


"snapshot-scheduler-restats-#12202%database.IgniteDbBaselineTopologySelfTest0%" 
#15332 prio=5 os_prio=0 tid=0x7fa228143000 nid=0x3c65 waiting on condition 
[0x7fa2be919000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
at 
org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.publicApiActiveState(GridClusterStateProcessor.java:194)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTopologyFutureAdapter.validateCache(GridDhtTopologyFutureAdapter.java:83)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedLockFuture.mapOnTopology(GridDhtColocatedLockFuture.java:787)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedLockFuture.map(GridDhtColocatedLockFuture.java:763)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedCache.lockAllAsync(GridDhtColocatedCache.java:646)
at 
org.apache.ignite.internal.processors.cache.distributed.GridDistributedCacheAdapter.txLockAsync(GridDistributedCacheAdapter.java:109)
at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.getAllAsync(GridNearTxLocal.java:1723)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedCache$4.op(GridDhtColocatedCache.java:197)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter$AsyncOp.op(GridCacheAdapter.java:5140)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.asyncOp(GridCacheAdapter.java:4275)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedCache.getAsync(GridDhtColocatedCache.java:195)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get0(GridCacheAdapter.java:4565)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4546)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1347)
at 

[jira] [Commented] (IGNITE-6625) JDBC thin: support SSL connection to Ignite node

[Sergey Kalashnikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350609#comment-16350609
 ] 

Sergey Kalashnikov commented on IGNITE-6625:


[~tledkov-gridgain], here are my comments:
1. Please remove unused imports from {{ConnectionProperties.java}} and 
{{JdbcThinConnectionSelfTest.java}}
2. It might be helpful to move all SSL-specific stuff from 
{{JdbcThinTcpIo.java}} to a sub-class.
3. Please consider adding tests that would check error for the following cases:
invalid/unsupported ssl protocol, key store type, key algorithm. The 
corresponding connection params seems not covered by tests.

> JDBC thin: support SSL connection to Ignite node
> 
>
> Key: IGNITE-6625
> URL: https://issues.apache.org/jira/browse/IGNITE-6625
> Project: Ignite
>  Issue Type: Improvement
>  Components: jdbc
>Affects Versions: 2.2
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Fix For: 2.5
>
>
> SSL connection must be supported for JDBC thin driver.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (IGNITE-6917) SQL: implement COPY command for efficient data loading

[Alexander Paschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350605#comment-16350605
 ] 

Alexander Paschenko edited comment on IGNITE-6917 at 2/2/18 4:29 PM:
-

[~kirill.shirokov],

50. {{BulkLoadProcessor#close}}

>See javadoc.

I have seen it. Still this class is not a member of any hierarchy, {{close}} 
method neither overrides anything nor is overridden by anyone and thus now this 
param neither is used nor needed. Please remove it, adding it back once we need 
it won't be a big deal at all. Really, unused param in this case hardly is an 
architectural point of future extension.

51. {{BulkLoadCacheWriter}}:

>O. But what is the reason for doing that?

At the very least, to maintain consistency about names of methods having 
similar, or close-to-identical meaning. Say, all our closures have their method 
called {{apply}} while this class employs {{accept}}. I don't see any reason 
why this class shouldn't be in that hierarchy.

 


was (Author: al.psc):
[~kirill.shirokov],

50. {{BulkLoadProcessor#close}}

>See javadoc.

I have seen it. Still this class is not a member of any hierarchy, {{close}} 
method neither overrides anything nor is overridden by anyone and thus now this 
param neither is used nor needed. Please remove it, adding it back once we need 
it won't be a big deal at all. Really, unused param in this case hardly is an 
architectural point of extension.

51. {{BulkLoadCacheWriter}}:

>O. But what is the reason for doing that?

At the very least, to maintain consistency about names of methods having 
similar, or close-to-identical meaning. Say, all our closures have their method 
called {{apply}} while this class employs {{accept}}. I don't see any reason 
why this class shouldn't be in that hierarchy.

 

> SQL: implement COPY command for efficient data loading
> --
>
> Key: IGNITE-6917
> URL: https://issues.apache.org/jira/browse/IGNITE-6917
> Project: Ignite
>  Issue Type: New Feature
>  Components: sql
>Reporter: Vladimir Ozerov
>Assignee: Kirill Shirokov
>Priority: Major
>  Labels: iep-1
>
> Inspired by Postgres [1]
> Common use case - bulk data load through JDBC/ODBC interface. Currently it is 
> only possible to execute single commands one by one. We already can batch 
> them to improve performance, but there is still big room for improvement.
> We should think of a completely new command - {{COPY}}. It will accept a file 
> (or input stream in general case) on the client side, then transfer data to 
> the cluster, and then execute update inside the cluster, e.g. through 
> streamer.
> First of all we need to create quick and dirty prototype to assess potential 
> performance improvement. It speedup is confirmed, we should build base 
> implementation which will accept only files. But at the same time we should 
> understand how it will evolve in future: multiple file formats (probably 
> including Hadoop formarts, e.g. Parquet), escape characters, input streams, 
> etc..
> [1] [https://www.postgresql.org/docs/9.6/static/sql-copy.html]
> h1. Proposed syntax
> Curent implementation:
> {noformat}
> COPY 
> FROM "file.name"
> INTO .
> [(col-name, ...)]
> FORMAT  -- Only CSV format is supported in the current 
> release
> [BATCH_SIZE ]
> {noformat}
> We may want to gradually add features to this command in future to have 
> something like this:
> {noformat}
> COPY
> FROM "file.name"[CHARSET ""]
> INTO . [CREATE [IF NOT EXISTS]]
> [(col-name [] [NULLABLE] [ESCAPES], ...) [MATCH HEADER]]
> FORMAT (csv|tsv|...)
> -- CSV format options:
> [FIELDSEP='column-separators-regexp']
> [LINESEP='row-separators-regexp']
> [QUOTE='quote-chars']
> [ESCAPE='escape-char']
> [NULL='null-sequence']
> [COMMENT='single-line-comment-start-char']
> [TRIM_LINES]
> [IMPORT_EMPTY_LINES]
> [CHARSET ""]
> [ROWS -]
> --or--
> [SKIP ROWS ] [MAX ROWS ]
> [COLS -]
> --or--
> [SKIP COLS ] [MAX COLS ]
> [(MATCH | SKIP) HEADER]
> [(REPLACE|IGNORE|ABORT ON [])) DUPLICATE KEYS]
> [BATCH SIZE ( ROWS | [K|M|G|T|P])]
> [COMPRESS "codec-name" [codec options]]
> [LOCK (TABLE|ROWS)]
> [NOLOGGING]
> [BACKEND (DIRECT | STREAMER)]
> {noformat}
> h1. Implementation decisions and notes
> h2. Parsing
> * We support CSV format described in RFC 4180.
> * Custom row and column separators, quoting characters are currently hardcoded
> * Escape sequences, line comment characters are currently not supported
> * We may want to support fixed-length formats (via format descriptors) in 
> future
> * We may want to strip comments from lines (for example, starting with '#')
> * We may want to allow user to either ignore empty lines or 

[jira] [Commented] (IGNITE-6917) SQL: implement COPY command for efficient data loading

[Kirill Shirokov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350565#comment-16350565
 ] 

Kirill Shirokov commented on IGNITE-6917:
-

> 44. {{BulkLoadProcessor#close}} - unused method param.

See javadoc.

> 45. {{BulkLoadDataConverter}} - we don't need this class explicitly anymore, 
> just create anonymous closure in-place.

I remember that Vladimir recommended to have inner classes instead of closures.

> About *32.2* - I simply suggest that you add one more ancestor to {{extends}} 
> clause, this ancestor must be one of Ignite closures. Similar to what you did 
> with {{BulkLoadDataConverter}}.

O. But what is the reason for doing that?

> 47. {{BulkLoadAckClientParameters#DEFAULT_INPUT_CHARSET}} - unused.
> 48. {{DmlStatementsProcessor#runDmlStatement(String sql, SqlCommand cmd):}}
> 48.1 Please rename to {{runNativeDmlStatement}} - it's not exactly an 
> overload of another method with same name existing in this class.
> 48.2 Please remove braces after {{if (cmd instanceof SqlBulkLoadCommand) {}} 
> - one-liners should not be surrounded with braces per Ignite conventions.

Fixed.

> 49. {{JdbcRequestHandler#processBulkLoadFileBatch}}: we try to actually 
> process batch even when we have received {{CMD_FINISHED_ERROR}} (there's no 
> check before attempting to process batch). Are you sure we want to do this 
> regardless of state flag value?

I think it makes more sense, since user may expect the command to import as 
many records as possible in transactionless mode. In future we may have an 
additional "error" flag in JdbcBulkLoadProcessor#processBatch().

> SQL: implement COPY command for efficient data loading
> --
>
> Key: IGNITE-6917
> URL: https://issues.apache.org/jira/browse/IGNITE-6917
> Project: Ignite
>  Issue Type: New Feature
>  Components: sql
>Reporter: Vladimir Ozerov
>Assignee: Kirill Shirokov
>Priority: Major
>  Labels: iep-1
>
> Inspired by Postgres [1]
> Common use case - bulk data load through JDBC/ODBC interface. Currently it is 
> only possible to execute single commands one by one. We already can batch 
> them to improve performance, but there is still big room for improvement.
> We should think of a completely new command - {{COPY}}. It will accept a file 
> (or input stream in general case) on the client side, then transfer data to 
> the cluster, and then execute update inside the cluster, e.g. through 
> streamer.
> First of all we need to create quick and dirty prototype to assess potential 
> performance improvement. It speedup is confirmed, we should build base 
> implementation which will accept only files. But at the same time we should 
> understand how it will evolve in future: multiple file formats (probably 
> including Hadoop formarts, e.g. Parquet), escape characters, input streams, 
> etc..
> [1] [https://www.postgresql.org/docs/9.6/static/sql-copy.html]
> h1. Proposed syntax
> Curent implementation:
> {noformat}
> COPY 
> FROM "file.name"
> INTO .
> [(col-name, ...)]
> FORMAT  -- Only CSV format is supported in the current 
> release
> [BATCH_SIZE ]
> {noformat}
> We may want to gradually add features to this command in future to have 
> something like this:
> {noformat}
> COPY
> FROM "file.name"[CHARSET ""]
> INTO . [CREATE [IF NOT EXISTS]]
> [(col-name [] [NULLABLE] [ESCAPES], ...) [MATCH HEADER]]
> FORMAT (csv|tsv|...)
> -- CSV format options:
> [FIELDSEP='column-separators-regexp']
> [LINESEP='row-separators-regexp']
> [QUOTE='quote-chars']
> [ESCAPE='escape-char']
> [NULL='null-sequence']
> [COMMENT='single-line-comment-start-char']
> [TRIM_LINES]
> [IMPORT_EMPTY_LINES]
> [CHARSET ""]
> [ROWS -]
> --or--
> [SKIP ROWS ] [MAX ROWS ]
> [COLS -]
> --or--
> [SKIP COLS ] [MAX COLS ]
> [(MATCH | SKIP) HEADER]
> [(REPLACE|IGNORE|ABORT ON [])) DUPLICATE KEYS]
> [BATCH SIZE ( ROWS | [K|M|G|T|P])]
> [COMPRESS "codec-name" [codec options]]
> [LOCK (TABLE|ROWS)]
> [NOLOGGING]
> [BACKEND (DIRECT | STREAMER)]
> {noformat}
> h1. Implementation decisions and notes
> h2. Parsing
> * We support CSV format described in RFC 4180.
> * Custom row and column separators, quoting characters are currently hardcoded
> * Escape sequences, line comment characters are currently not supported
> * We may want to support fixed-length formats (via format descriptors) in 
> future
> * We may want to strip comments from lines (for example, starting with '#')
> * We may want to allow user to either ignore empty lines or treat them as a 
> special case of record having all default values
> * We may allow user to enable whitespace trimming from beginning and end of a 
> line
> * We may want to allow user to 

[jira] [Commented] (IGNITE-6917) SQL: implement COPY command for efficient data loading

[Alexander Paschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350605#comment-16350605
 ] 

Alexander Paschenko commented on IGNITE-6917:
-

[~kirill.shirokov],

50. {{BulkLoadProcessor#close}}

>See javadoc.

I have seen it. Still this class is not a member of any hierarchy, {{close}} 
method neither overrides anything nor is overridden by anyone and thus now this 
param neither is used nor needed. Please remove it, adding it back once we need 
it won't be a big deal at all. Really, unused param in this case hardly is an 
architectural point of extension.

51. {{BulkLoadCacheWriter}}:

>O. But what is the reason for doing that?

At the very least, to maintain consistency about names of methods having 
similar, or close-to-identical meaning. Say, all our closures have their method 
called {{apply}} while this class employs {{accept}}. I don't see any reason 
why this class shouldn't be in that hierarchy.

 

> SQL: implement COPY command for efficient data loading
> --
>
> Key: IGNITE-6917
> URL: https://issues.apache.org/jira/browse/IGNITE-6917
> Project: Ignite
>  Issue Type: New Feature
>  Components: sql
>Reporter: Vladimir Ozerov
>Assignee: Kirill Shirokov
>Priority: Major
>  Labels: iep-1
>
> Inspired by Postgres [1]
> Common use case - bulk data load through JDBC/ODBC interface. Currently it is 
> only possible to execute single commands one by one. We already can batch 
> them to improve performance, but there is still big room for improvement.
> We should think of a completely new command - {{COPY}}. It will accept a file 
> (or input stream in general case) on the client side, then transfer data to 
> the cluster, and then execute update inside the cluster, e.g. through 
> streamer.
> First of all we need to create quick and dirty prototype to assess potential 
> performance improvement. It speedup is confirmed, we should build base 
> implementation which will accept only files. But at the same time we should 
> understand how it will evolve in future: multiple file formats (probably 
> including Hadoop formarts, e.g. Parquet), escape characters, input streams, 
> etc..
> [1] [https://www.postgresql.org/docs/9.6/static/sql-copy.html]
> h1. Proposed syntax
> Curent implementation:
> {noformat}
> COPY 
> FROM "file.name"
> INTO .
> [(col-name, ...)]
> FORMAT  -- Only CSV format is supported in the current 
> release
> [BATCH_SIZE ]
> {noformat}
> We may want to gradually add features to this command in future to have 
> something like this:
> {noformat}
> COPY
> FROM "file.name"[CHARSET ""]
> INTO . [CREATE [IF NOT EXISTS]]
> [(col-name [] [NULLABLE] [ESCAPES], ...) [MATCH HEADER]]
> FORMAT (csv|tsv|...)
> -- CSV format options:
> [FIELDSEP='column-separators-regexp']
> [LINESEP='row-separators-regexp']
> [QUOTE='quote-chars']
> [ESCAPE='escape-char']
> [NULL='null-sequence']
> [COMMENT='single-line-comment-start-char']
> [TRIM_LINES]
> [IMPORT_EMPTY_LINES]
> [CHARSET ""]
> [ROWS -]
> --or--
> [SKIP ROWS ] [MAX ROWS ]
> [COLS -]
> --or--
> [SKIP COLS ] [MAX COLS ]
> [(MATCH | SKIP) HEADER]
> [(REPLACE|IGNORE|ABORT ON [])) DUPLICATE KEYS]
> [BATCH SIZE ( ROWS | [K|M|G|T|P])]
> [COMPRESS "codec-name" [codec options]]
> [LOCK (TABLE|ROWS)]
> [NOLOGGING]
> [BACKEND (DIRECT | STREAMER)]
> {noformat}
> h1. Implementation decisions and notes
> h2. Parsing
> * We support CSV format described in RFC 4180.
> * Custom row and column separators, quoting characters are currently hardcoded
> * Escape sequences, line comment characters are currently not supported
> * We may want to support fixed-length formats (via format descriptors) in 
> future
> * We may want to strip comments from lines (for example, starting with '#')
> * We may want to allow user to either ignore empty lines or treat them as a 
> special case of record having all default values
> * We may allow user to enable whitespace trimming from beginning and end of a 
> line
> * We may want to allow user to specify error handling strategy: e.g., only 
> one quote character is present or escape sequence is invalid.
> h2. File handling
> * File character set to be supported in future
> * Skipped/imported row number (or first/last line or skip header option), 
> skipped/imported column number (or first/last column): to be supported in 
> future
> * Line start pattern (as in MySQL): no support planned
> * We currently support only client-side import. No server-side file import.
> * We may want to support client-side stdin import in future.
> * We do not handle importing multiple files from single command
> * We don't benefit from any kind of pre-sorting 

[jira] [Assigned] (IGNITE-7565) Remove IgniteSet from heap

[Pavel Pereslegin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Pereslegin reassigned IGNITE-7565:


Assignee: Pavel Pereslegin

> Remove IgniteSet from heap
> --
>
> Key: IGNITE-7565
> URL: https://issues.apache.org/jira/browse/IGNITE-7565
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 2.2
>Reporter: Alexander Belyak
>Assignee: Pavel Pereslegin
>Priority: Major
>
> IgniteSet store all data in durable memory and in java heap. It's not good 
> for big clusters and big sets, so we need to remove values from heap.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-6917) SQL: implement COPY command for efficient data loading

[Alexander Paschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350544#comment-16350544
 ] 

Alexander Paschenko commented on IGNITE-6917:
-

[~kirill.shirokov], my comments:

44. {{BulkLoadProcessor#close}} - unused method param.

45. {{BulkLoadDataConverter}} - we don't need this class explicitly anymore, 
just create anonymous closure in-place.

46. About *32.2* - I simply suggest that you add one more ancestor to 
{{extends}} clause, this ancestor must be one of Ignite closures. Similar to 
what you did with {{BulkLoadDataConverter}}.

47. {{BulkLoadAckClientParameters#DEFAULT_INPUT_CHARSET}} - unused.

48. {{DmlStatementsProcessor#runDmlStatement(String sql, SqlCommand cmd):}}

48.1 Please rename to {{runNativeDmlStatement}} - it's not exactly an overload 
of another method with same name existing in this class.

48.2 Please remove braces after {{if (cmd instanceof SqlBulkLoadCommand) {}} - 
one-liners should not be surrounded with braces per Ignite conventions.

49. {{JdbcRequestHandler#processBulkLoadFileBatch}}: we try to actually process 
batch even when we have received {{CMD_FINISHED_ERROR}} (there's no check 
before attempting to process batch). Are you sure we want to do this regardless 
of state flag value?

This is it for now. Thanks!

 

> SQL: implement COPY command for efficient data loading
> --
>
> Key: IGNITE-6917
> URL: https://issues.apache.org/jira/browse/IGNITE-6917
> Project: Ignite
>  Issue Type: New Feature
>  Components: sql
>Reporter: Vladimir Ozerov
>Assignee: Kirill Shirokov
>Priority: Major
>  Labels: iep-1
>
> Inspired by Postgres [1]
> Common use case - bulk data load through JDBC/ODBC interface. Currently it is 
> only possible to execute single commands one by one. We already can batch 
> them to improve performance, but there is still big room for improvement.
> We should think of a completely new command - {{COPY}}. It will accept a file 
> (or input stream in general case) on the client side, then transfer data to 
> the cluster, and then execute update inside the cluster, e.g. through 
> streamer.
> First of all we need to create quick and dirty prototype to assess potential 
> performance improvement. It speedup is confirmed, we should build base 
> implementation which will accept only files. But at the same time we should 
> understand how it will evolve in future: multiple file formats (probably 
> including Hadoop formarts, e.g. Parquet), escape characters, input streams, 
> etc..
> [1] [https://www.postgresql.org/docs/9.6/static/sql-copy.html]
> h1. Proposed syntax
> Curent implementation:
> {noformat}
> COPY 
> FROM "file.name"
> INTO .
> [(col-name, ...)]
> FORMAT  -- Only CSV format is supported in the current 
> release
> [BATCH_SIZE ]
> {noformat}
> We may want to gradually add features to this command in future to have 
> something like this:
> {noformat}
> COPY
> FROM "file.name"[CHARSET ""]
> INTO . [CREATE [IF NOT EXISTS]]
> [(col-name [] [NULLABLE] [ESCAPES], ...) [MATCH HEADER]]
> FORMAT (csv|tsv|...)
> -- CSV format options:
> [FIELDSEP='column-separators-regexp']
> [LINESEP='row-separators-regexp']
> [QUOTE='quote-chars']
> [ESCAPE='escape-char']
> [NULL='null-sequence']
> [COMMENT='single-line-comment-start-char']
> [TRIM_LINES]
> [IMPORT_EMPTY_LINES]
> [CHARSET ""]
> [ROWS -]
> --or--
> [SKIP ROWS ] [MAX ROWS ]
> [COLS -]
> --or--
> [SKIP COLS ] [MAX COLS ]
> [(MATCH | SKIP) HEADER]
> [(REPLACE|IGNORE|ABORT ON [])) DUPLICATE KEYS]
> [BATCH SIZE ( ROWS | [K|M|G|T|P])]
> [COMPRESS "codec-name" [codec options]]
> [LOCK (TABLE|ROWS)]
> [NOLOGGING]
> [BACKEND (DIRECT | STREAMER)]
> {noformat}
> h1. Implementation decisions and notes
> h2. Parsing
> * We support CSV format described in RFC 4180.
> * Custom row and column separators, quoting characters are currently hardcoded
> * Escape sequences, line comment characters are currently not supported
> * We may want to support fixed-length formats (via format descriptors) in 
> future
> * We may want to strip comments from lines (for example, starting with '#')
> * We may want to allow user to either ignore empty lines or treat them as a 
> special case of record having all default values
> * We may allow user to enable whitespace trimming from beginning and end of a 
> line
> * We may want to allow user to specify error handling strategy: e.g., only 
> one quote character is present or escape sequence is invalid.
> h2. File handling
> * File character set to be supported in future
> * Skipped/imported row number (or first/last line or skip header option), 
> skipped/imported column number (or first/last column): to be 

[jira] [Commented] (IGNITE-7606) Write evicted dirty page during eviction without holding segment write lock

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350513#comment-16350513
 ] 

ASF GitHub Bot commented on IGNITE-7606:


GitHub user dspavlov opened a pull request:

https://github.com/apache/ignite/pull/3469

IGNITE-7606: delayed write during eviction

IGNITE-7606: write evicted dirty page during eviction without holding 
segment write lock

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7606

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3469.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3469


commit 90befa9d3bc89833eeed42a8814e8a7f51a11448
Author: dpavlov 
Date:   2018-01-25T18:11:07Z

IGNITE-7533: Throttle writing threads according fsync progress and 
checkpoint write speed

commit 74411e7aea9744df7f7656006807aa0403ae921f
Author: dpavlov 
Date:   2018-01-29T15:51:23Z

IGNITE-7533: Throttle writing threads according fsync progress and 
checkpoint write speed

commit 4be02ec3444596ee0bc95bd55ece9ba741e729a1
Author: dpavlov 
Date:   2018-01-29T15:55:48Z

Merge branch 'master' into ignite-7533

commit f5d383ddef1b2a66470ec110f781a98aa78c5d03
Author: dpavlov 
Date:   2018-01-29T17:08:35Z

IGNITE-7533: Throttle writing threads according fsync progress and 
checkpoint write speed

commit 7c0afa374907202e45d6dcbfae88af1c3a27687f
Author: dpavlov 
Date:   2018-01-30T14:04:52Z

IGNITE-7533: Option to enable old implementation of throttling

commit 9f9c1e7955d894bbfd8a8572362d2c13177a60c6
Author: dpavlov 
Date:   2018-01-30T14:13:42Z

IGNITE-7380: Flaky test reproduction

commit 8d8aecd55a4d94c091e970ccf3bbbd72272cf325
Author: dpavlov 
Date:   2018-01-30T14:20:50Z

IGNITE-7380: Flaky test reproduction

commit cf9d42ba77133c4c6e37b1d8a7ac6d54054d1bc1
Author: dpavlov 
Date:   2018-01-30T14:40:26Z

IGNITE-7533: Preserve order of writing in fsync

commit b37f27275446a8cccafbd231c35e3605d9fd7089
Author: dpavlov 
Date:   2018-01-30T14:43:10Z

IGNITE-7380: Increase of timeout of checkpoint

commit b05ef5dae3d4e5deef6482844989077aba6f1bf2
Author: dpavlov 
Date:   2018-01-30T16:23:51Z

IGNITE-7533: Too much pages written case, no throttling in case too long 
wait. Added more delay in case low space left

commit 62685bcb363add269930af99e59b74d396870b55
Author: dpavlov 
Date:   2018-01-30T16:37:35Z

IGNITE-7380: Flaky test reproduction

commit c7ba24580199a238506ea00176aaf7ae229aa135
Author: dpavlov 
Date:   2018-01-30T17:57:15Z

IGNITE-7533: Sandbox test with progress gaps detection was added.

commit d32654d902ca2fe0ec4e3fa5327afe84f949cdaf
Author: dpavlov 
Date:   2018-01-31T15:52:12Z

IGNITE-7175: fix compatible with speed based throttling

commit 018ed3c0de21cbe1ddd9f7558a417369740bd2cc
Author: dpavlov 
Date:   2018-01-31T17:28:23Z

IGNITE-7533: recurrent warning of throttling if significant pressure.

commit 687d1ffd3d7e6ccc9a0c609ffda6762e7707d788
Author: dpavlov 
Date:   2018-01-31T17:38:43Z

IGNITE-7533: recurrent warning of throttling if significant pressure: cp 
pages added

commit 94dac70231c52a915717ca444e7aba8e4b816003
Author: dpavlov 
Date:   2018-01-31T18:15:03Z

IGNITE-7533: Test suite added

commit 25f4774c1990cb956d2d01eb1c261da762a8798e
Author: dpavlov 
Date:   2018-01-31T18:15:46Z

IGNITE-7533: Test suite added

commit c0a078d45caf08e5f4d9b28f901176155b26c8a4
Author: dpavlov 
Date:   2018-02-01T11:59:21Z

Merge branch 'master' into ignite-7533

commit eab2b06c776e6e4a5f7ee6d1b8a9aa7596831660
Author: dpavlov 
Date:   2018-02-01T12:27:31Z

IGNITE-7533: Message updated to be more clear

commit 45845535cf42b2d86b7f2ff967713baf3c4c430b
Author: dpavlov 
Date:   2018-02-01T17:34:33Z

IGNITE-7533: data streamer test

commit 9d081ee707ce479c70c8a74feb6fa1ada3047b94
Author: dpavlov 
Date:   2018-02-02T15:23:57Z

IGNITE-7606: Write evicted dirty page during eviction without holding 
segment write lock




> Write evicted dirty page during eviction without holding segment write lock
> ---
>
> Key: IGNITE-7606
> URL: https://issues.apache.org/jira/browse/IGNITE-7606
> Project: Ignite
>  Issue Type: Improvement
>  Components: persistence
>Reporter: Dmitriy Pavlov
>Assignee: Dmitriy Pavlov
>Priority: Major
> 

[jira] [Commented] (IGNITE-6917) SQL: implement COPY command for efficient data loading

[Kirill Shirokov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350469#comment-16350469
 ] 

Kirill Shirokov commented on IGNITE-6917:
-

{{> 31. BulkLoadDataConverter}} is now empty and doesn't convert anything. I 
think we should remove it and simply pass {{UpdatePlan}} to 
{{BulkLoadProcessor}}.

No, we can't. UpdatePlan is in the indexing module, while the BulkLoadProcessor 
is in the core. This is why I had to introduce this interface.

> 32.2 Leaving this interface be does not contradict with making it extend some 
> of existing Ignite interfaces. Please add another appropriate ancestor to 
> this interface.

Sorry I don't understand what you are saying.

> 34.2 {{processBatch}} is still coupled with JDBC related class.

Fixed.

> 34.3 I think class name {{BulkLoadContext}} would reflect what this class 
> does more correctly. In fact, it does not process anything, just holds 
> various bits that help the process. In favor of this tells the fact that you 
> even use the word "context" in javadoc for this class.

No, it now processes data as well. So the "Processor" makes more sense here.

> 35. {{BulkLoadParser}}: is still coupled with JDBC related class. I have 
> already suggested that we introduce some JDBC neutral enum to indicate 
> current stage of the process.

Fixed.

> 37. {{JdbcBulkLoadBatchAckResult}}: I suggested simply 
> {{JdbcBulkLoadAckResult}}, without the word "batch", as this ack response has 
> to do with the command overall and *not* with an individual batch. It's sent 
> once per bulk load.

Yep, this was a typo.

> 40. {{BulkLoadStreamerWriter}}: why is counter internally stored as {{int}}, 
> not {{long}}?

Fixed.

> 41. {{JdbcQueryTask#call}}: I think it would be better to gracefully close 
> newly created context before throwing an exception.

Nice catch. Fixed.

> 42. {{BulkLoadCsvFormat}}: many new members and their getters have "Re" 
> suffix. It's not standard to Ignite codebase. I would either replace it to 
> "regex" or "regexp", or even remove it - the user of this class can clearly 
> see what is return type of those getters, and those suffixes actually don't 
> tell anything new.

"Re" removed.

> 43. {{DdlStatementsProcessor#runDdlStatement}}: why employ log throttling 
> here? Why would we want to skip some error messages?

Another typo. "LT" replaced with "U".

> SQL: implement COPY command for efficient data loading
> --
>
> Key: IGNITE-6917
> URL: https://issues.apache.org/jira/browse/IGNITE-6917
> Project: Ignite
>  Issue Type: New Feature
>  Components: sql
>Reporter: Vladimir Ozerov
>Assignee: Kirill Shirokov
>Priority: Major
>  Labels: iep-1
>
> Inspired by Postgres [1]
> Common use case - bulk data load through JDBC/ODBC interface. Currently it is 
> only possible to execute single commands one by one. We already can batch 
> them to improve performance, but there is still big room for improvement.
> We should think of a completely new command - {{COPY}}. It will accept a file 
> (or input stream in general case) on the client side, then transfer data to 
> the cluster, and then execute update inside the cluster, e.g. through 
> streamer.
> First of all we need to create quick and dirty prototype to assess potential 
> performance improvement. It speedup is confirmed, we should build base 
> implementation which will accept only files. But at the same time we should 
> understand how it will evolve in future: multiple file formats (probably 
> including Hadoop formarts, e.g. Parquet), escape characters, input streams, 
> etc..
> [1] [https://www.postgresql.org/docs/9.6/static/sql-copy.html]
> h1. Proposed syntax
> Curent implementation:
> {noformat}
> COPY 
> FROM "file.name"
> INTO .
> [(col-name, ...)]
> FORMAT  -- Only CSV format is supported in the current 
> release
> [BATCH_SIZE ]
> {noformat}
> We may want to gradually add features to this command in future to have 
> something like this:
> {noformat}
> COPY
> FROM "file.name"[CHARSET ""]
> INTO . [CREATE [IF NOT EXISTS]]
> [(col-name [] [NULLABLE] [ESCAPES], ...) [MATCH HEADER]]
> FORMAT (csv|tsv|...)
> -- CSV format options:
> [FIELDSEP='column-separators-regexp']
> [LINESEP='row-separators-regexp']
> [QUOTE='quote-chars']
> [ESCAPE='escape-char']
> [NULL='null-sequence']
> [COMMENT='single-line-comment-start-char']
> [TRIM_LINES]
> [IMPORT_EMPTY_LINES]
> [CHARSET ""]
> [ROWS -]
> --or--
> [SKIP ROWS ] [MAX ROWS ]
> [COLS -]
> --or--
> [SKIP COLS ] [MAX COLS ]
> [(MATCH | SKIP) HEADER]
> [(REPLACE|IGNORE|ABORT ON [])) DUPLICATE KEYS]
> [BATCH SIZE ( ROWS | [K|M|G|T|P])]
> [COMPRESS "codec-name" [codec options]]
> [LOCK 

[jira] [Updated] (IGNITE-7616) GridDataStreamExecutor and GridCallbackExecutor JMX beans return incorrect values due to invalid interface registration.

[Max Shonichev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Max Shonichev updated IGNITE-7616:
--
Labels: jmx  (was: )

> GridDataStreamExecutor and GridCallbackExecutor JMX beans return incorrect 
> values due to invalid interface registration.
> 
>
> Key: IGNITE-7616
> URL: https://issues.apache.org/jira/browse/IGNITE-7616
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.4
>Reporter: Max Shonichev
>Priority: Major
>  Labels: jmx
> Fix For: 2.5
>
>
> Two of newly added management beans as a result of implementing feature 
> request https://issues.apache.org/jira/browse/IGNITE-7217 have bugs:
>  # GridDataStreamExecutor is registered as conforming to ThreadPoolMXBean 
> interface, though actually it is an incompatible StripedExecutor. 
>  # GridCallbackExecutor is registered as conforming to ThreadPoolMXBean 
> interface, though actually it is an incompatible 
> IgniteStripedThreadPoolExecutor.
>  # ThreadPoolMXBeanAdapter checks whether adapted instance is 
> ThreadPoolExecutor, and as interfaces are incompatible, most of the JMX 
> attributes of GridCallbackExecutor and GridDataStreamExecutor are returned as 
> -1 or null.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (IGNITE-425) Introduce transformers for continuous queries

[Nikolay Izhikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350242#comment-16350242
 ] 

Nikolay Izhikov edited comment on IGNITE-425 at 2/2/18 2:23 PM:


[~avinogradov]

I changed benchmark.
Please, see source changes.

Got following number on couple of runs:

||Measure||Value||
|Warmup count|3|
|Iteration count|5|
|Period|20 sec|
|Write threads|4|

||Test||Description||
|CQBenchmark|Regular Continuous Query|
|CQWTIdBenchmark|Transformer returns Long|
|CQWTNullBenchmark|Transformer returns Null|
|CQWTValueBenchmark|Transformer retrurn whole Value|

EvtCnt - count of received events in Continuous Query. The bigger the better

{noformat}
BenchmarkMode  CntScore   Error  
Units
CQBenchmark.putBatchthrpt5   12,158 ± 3,897  
ops/s
CQBenchmark.putBatch:evtCnt thrpt5  7482138,000 
 #
CQBenchmark.putBatch:methodExecuted thrpt5 1826,000 
 #
CQWTIdBenchmark.putBatchthrpt5   12,736 ± 0,399  
ops/s
CQWTIdBenchmark.putBatch:evtCnt thrpt5  7843317,000 
 #
CQWTIdBenchmark.putBatch:methodExecuted thrpt5 1914,000 
 #
CQWTNullBenchmark.putBatch  thrpt5   12,800 ± 0,565  
ops/s
CQWTNullBenchmark.putBatch:evtCnt   thrpt5  7877051,000 
 #
CQWTNullBenchmark.putBatch:methodExecuted   thrpt5 1922,000 
 #
CQWTValueBenchmark.putBatch thrpt5   11,785 ± 0,641  
ops/s
CQWTValueBenchmark.putBatch:evtCnt  thrpt5  7255440,000 
 #
CQWTValueBenchmark.putBatch:methodExecuted  thrpt5 1770,000 
 #
{noformat}


was (Author: nizhikov):
[~avinogradov]

I changed benchmark.
Please, see source changes.

Got following number on couple of runs:

||Measure||Value||
|Warmup count|3|
|Iteration count|5|
|Period|20 sec|
|Write threads|4|

||Test||Description||
|CQBenchmark|Regular Continuous Query|
|CQWTIdBenchmark|Transformer returns Long|
|CQWTNullBenchmark|Transformer returns Null|
|CQWTValueBenchmark|Transformer retrurn whole Value|

EvtCnt - count of received events in Continuous Query. The bigger the better

{noformat}
Benchmark  Mode  CntScore   Error  
Units
CQBenchmark.testMethodthrpt5   12,084 ± 1,111  
ops/s
CQBenchmark.testMethod:evtCnt thrpt5  7438039,000   
   #
CQBenchmark.testMethod:methodExecuted thrpt5 1815,000   
   #
CQWTIdBenchmark.putBatch  thrpt5   12,782 ± 0,290  
ops/s
CQWTIdBenchmark.putBatch:evtCnt   thrpt5  7867990,000   
   #
CQWTIdBenchmark.putBatch:methodExecuted   thrpt5 1920,000   
   #
CQWTNullBenchmark.testMethod  thrpt5   12,601 ± 0,960  
ops/s
CQWTNullBenchmark.testMethod:evtCnt   thrpt5  7762282,000   
   #
CQWTNullBenchmark.testMethod:methodExecuted   thrpt5 1894,000   
   #
CQWTValueBenchmark.testMethod thrpt5   11,945 ± 0,522  
ops/s
CQWTValueBenchmark.testMethod:evtCnt  thrpt5  7356065,000   
   #
CQWTValueBenchmark.testMethod:methodExecuted  thrpt5 1795,000   
   #


BenchmarkMode  CntScore   Error  
Units
CQBenchmark.putBatchthrpt5   12,158 ± 3,897  
ops/s
CQBenchmark.putBatch:evtCnt thrpt5  7482138,000 
 #
CQBenchmark.putBatch:methodExecuted thrpt5 1826,000 
 #
CQWTIdBenchmark.putBatchthrpt5   12,736 ± 0,399  
ops/s
CQWTIdBenchmark.putBatch:evtCnt thrpt5  7843317,000 
 #
CQWTIdBenchmark.putBatch:methodExecuted thrpt5 1914,000 
 #
CQWTNullBenchmark.putBatch  thrpt5   12,800 ± 0,565  
ops/s
CQWTNullBenchmark.putBatch:evtCnt   thrpt5  7877051,000 
 #
CQWTNullBenchmark.putBatch:methodExecuted   thrpt5 1922,000 
 #
CQWTValueBenchmark.putBatch thrpt5   11,785 ± 0,641  
ops/s
CQWTValueBenchmark.putBatch:evtCnt  thrpt5  7255440,000 
 #
CQWTValueBenchmark.putBatch:methodExecuted  thrpt5 1770,000 
 #

BenchmarkMode  CntScore   Error  
Units
CQBenchmark.putBatchthrpt5   12,714 ± 0,972  
ops/s
CQBenchmark.putBatch:evtCnt thrpt5  7823989,000 
 #
CQBenchmark.putBatch:methodExecuted thrpt5 1909,000 
 #
CQWTIdBenchmark.putBatchthrpt

[jira] [Commented] (IGNITE-7508) GridKernalContextImpl::isDaemon creates contention on system properties access

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350364#comment-16350364
 ] 

ASF GitHub Bot commented on IGNITE-7508:


GitHub user AMashenkov opened a pull request:

https://github.com/apache/ignite/pull/3468

IGNITE-7508: Fix contention on system property access in 
GridKernalContextImpl::isDaemon()

Fixed.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7508

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3468.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3468


commit a78297bf6a531bf5b1d7764637a31542c029c754
Author: Andrey V. Mashenkov 
Date:   2018-02-02T13:57:05Z

IGNITE-7508: Fix contention on system property access in 
GridKernalContextImpl::isDaemon().




> GridKernalContextImpl::isDaemon creates contention on system properties access
> --
>
> Key: IGNITE-7508
> URL: https://issues.apache.org/jira/browse/IGNITE-7508
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Reporter: Stanislav Lukyanov
>Assignee: Andrew Mashenkov
>Priority: Major
>
> GridKernalContextImpl::isDaemon reads system property IGNITE_DAEMON on every 
> call, leading to contention on the system properties lock. The lock is shown 
> as contended in the Java Mission Control analysis of a JFR recording of the 
> IgnitePutGetBenchmark.
> The fix would be to cache IGNITE_DAEMON value (e.g. in IgniteUtils) since it 
> isn't supposed to be changed during the JVM's lifetime anyway.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (IGNITE-7617) Contention on system.getProperty() in GridKernalContextImpl.isDaemon.

[Andrew Mashenkov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Mashenkov resolved IGNITE-7617.
--
Resolution: Duplicate

> Contention on system.getProperty() in GridKernalContextImpl.isDaemon.
> -
>
> Key: IGNITE-7617
> URL: https://issues.apache.org/jira/browse/IGNITE-7617
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1, 2.3, 2.4
>Reporter: Andrew Mashenkov
>Priority: Major
>
> I've found there is a contention on HashTable with system propertied 
> (System.getProperty()) in GridKernalContextImpl.isDaemon() while getting 
> entries from cache in multiple threads.
> IsDaemon() method is widely used in ignite and this may affect performance of 
> other operations  as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (IGNITE-7508) GridKernalContextImpl::isDaemon creates contention on system properties access

[Andrew Mashenkov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Mashenkov reassigned IGNITE-7508:


Assignee: Andrew Mashenkov

> GridKernalContextImpl::isDaemon creates contention on system properties access
> --
>
> Key: IGNITE-7508
> URL: https://issues.apache.org/jira/browse/IGNITE-7508
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Reporter: Stanislav Lukyanov
>Assignee: Andrew Mashenkov
>Priority: Major
>
> GridKernalContextImpl::isDaemon reads system property IGNITE_DAEMON on every 
> call, leading to contention on the system properties lock. The lock is shown 
> as contended in the Java Mission Control analysis of a JFR recording of the 
> IgnitePutGetBenchmark.
> The fix would be to cache IGNITE_DAEMON value (e.g. in IgniteUtils) since it 
> isn't supposed to be changed during the JVM's lifetime anyway.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (IGNITE-7617) Contention on system.getProperty() in GridKernalContextImpl.isDaemon.

[Andrew Mashenkov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Mashenkov updated IGNITE-7617:
-
Description: 
I've found there is a contention on HashTable with system propertied 
(System.getProperty()) in GridKernalContextImpl.isDaemon() while getting 
entries from cache in multiple threads.

IsDaemon() method is widely used in ignite and this may affect performance of 
other operations  as well.

  was:
I've found there is a contention on HashTable with system propertied 
(System.getProperty()) in GridKernalContextImpl.isDaemon() while getting 
entries from cache in multiple threads.

IsDaemon() method is widely used in ignite and this may affect other operations 
performance as well.


> Contention on system.getProperty() in GridKernalContextImpl.isDaemon.
> -
>
> Key: IGNITE-7617
> URL: https://issues.apache.org/jira/browse/IGNITE-7617
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1, 2.3, 2.4
>Reporter: Andrew Mashenkov
>Priority: Major
>
> I've found there is a contention on HashTable with system propertied 
> (System.getProperty()) in GridKernalContextImpl.isDaemon() while getting 
> entries from cache in multiple threads.
> IsDaemon() method is widely used in ignite and this may affect performance of 
> other operations  as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (IGNITE-7617) Contention on system.getProperty() in GridKernalContextImpl.isDaemon.

[Andrew Mashenkov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Mashenkov updated IGNITE-7617:
-
Description: 
I've found there is a contention on HashTable with system propertied 
(System.getProperty()) in GridKernalContextImpl.isDaemon() while getting 
entries from cache in multiple threads.

IsDaemon() method is widely used in ignite and this may affect other operations 
performance as well.

  was:
I've found there is a contention on HashTable with system propertied 
(System.getProperty()) in GridKernalContextImpl.isDaemon() while getting 
entries from cache in multiple threads.

IsDaemin() method is widely used in ignite and this may affect other operations 
performance as well.


> Contention on system.getProperty() in GridKernalContextImpl.isDaemon.
> -
>
> Key: IGNITE-7617
> URL: https://issues.apache.org/jira/browse/IGNITE-7617
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1, 2.3, 2.4
>Reporter: Andrew Mashenkov
>Priority: Major
>
> I've found there is a contention on HashTable with system propertied 
> (System.getProperty()) in GridKernalContextImpl.isDaemon() while getting 
> entries from cache in multiple threads.
> IsDaemon() method is widely used in ignite and this may affect other 
> operations performance as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (IGNITE-7617) Contention on system.getProperty() in GridKernalContextImpl.isDaemon.

[Andrew Mashenkov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Mashenkov updated IGNITE-7617:
-
Affects Version/s: 2.4

> Contention on system.getProperty() in GridKernalContextImpl.isDaemon.
> -
>
> Key: IGNITE-7617
> URL: https://issues.apache.org/jira/browse/IGNITE-7617
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1, 2.3, 2.4
>Reporter: Andrew Mashenkov
>Priority: Major
>
> I've found there is a contention on HashTable with system propertied 
> (System.getProperty()) in GridKernalContextImpl.isDaemon() while getting 
> entries from cache in multiple threads.
> IsDaemin() method is widely used in ignite and this may affect other 
> operations performance as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (IGNITE-7617) Contention on system.getProperty() in GridKernalContextImpl.isDaemon.

[Andrew Mashenkov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Mashenkov updated IGNITE-7617:
-
Affects Version/s: 2.1
   2.3

> Contention on system.getProperty() in GridKernalContextImpl.isDaemon.
> -
>
> Key: IGNITE-7617
> URL: https://issues.apache.org/jira/browse/IGNITE-7617
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1, 2.3, 2.4
>Reporter: Andrew Mashenkov
>Priority: Major
>
> I've found there is a contention on HashTable with system propertied 
> (System.getProperty()) in GridKernalContextImpl.isDaemon() while getting 
> entries from cache in multiple threads.
> IsDaemin() method is widely used in ignite and this may affect other 
> operations performance as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7617) Contention on system.getProperty() in GridKernalContextImpl.isDaemon.

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-7617:


 Summary: Contention on system.getProperty() in 
GridKernalContextImpl.isDaemon.
 Key: IGNITE-7617
 URL: https://issues.apache.org/jira/browse/IGNITE-7617
 Project: Ignite
  Issue Type: Bug
Reporter: Andrew Mashenkov


I've found there is a contention on HashTable with system propertied 
(System.getProperty()) in GridKernalContextImpl.isDaemon() while getting 
entries from cache in multiple threads.

IsDaemin() method is widely used in ignite and this may affect other operations 
performance as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7616) GridDataStreamExecutor and GridCallbackExecutor JMX beans return incorrect values due to invalid interface registration.

[Max Shonichev (JIRA)]

Max Shonichev created IGNITE-7616:
-

 Summary: GridDataStreamExecutor and GridCallbackExecutor JMX beans 
return incorrect values due to invalid interface registration.
 Key: IGNITE-7616
 URL: https://issues.apache.org/jira/browse/IGNITE-7616
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Max Shonichev
 Fix For: 2.5


Two of newly added management beans as a result of implementing feature request 
https://issues.apache.org/jira/browse/IGNITE-7217 have bugs:
 # GridDataStreamExecutor is registered as conforming to ThreadPoolMXBean 
interface, though actually it is an incompatible StripedExecutor. 
 # GridCallbackExecutor is registered as conforming to ThreadPoolMXBean 
interface, though actually it is an incompatible 
IgniteStripedThreadPoolExecutor.
 # ThreadPoolMXBeanAdapter checks whether adapted instance is 
ThreadPoolExecutor, and as interfaces are incompatible, most of the JMX 
attributes of GridCallbackExecutor and GridDataStreamExecutor are returned as 
-1 or null.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (IGNITE-7615) Find orphaned tests without test suites, create separate test suite for them

[Ilya Kasnacheev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Kasnacheev updated IGNITE-7615:

Description: 
It was noticed that some tests are written and updated, but not included in any 
suite and therefore never run.

 

Also look for duplicate tests and other oddities. In the end, every 
non-abstract test should be included in some suite.

 

 

This will be a first step towards either fixing those tests or discarding them.

  was:Also look for duplicate tests and other oddities. In the end, every 
non-abstract test should be included in some suite.


> Find orphaned tests without test suites, create separate test suite for them
> 
>
> Key: IGNITE-7615
> URL: https://issues.apache.org/jira/browse/IGNITE-7615
> Project: Ignite
>  Issue Type: Task
>  Components: general
>Affects Versions: 2.4
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Kasnacheev
>Priority: Major
>
> It was noticed that some tests are written and updated, but not included in 
> any suite and therefore never run.
>  
> Also look for duplicate tests and other oddities. In the end, every 
> non-abstract test should be included in some suite.
>  
>  
> This will be a first step towards either fixing those tests or discarding 
> them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7615) Find orphaned tests without test suites, create separate test suite for them

[Ilya Kasnacheev (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350281#comment-16350281
 ] 

Ilya Kasnacheev commented on IGNITE-7615:
-

[~avinogradov] please help me find reviewer for this issue.

> Find orphaned tests without test suites, create separate test suite for them
> 
>
> Key: IGNITE-7615
> URL: https://issues.apache.org/jira/browse/IGNITE-7615
> Project: Ignite
>  Issue Type: Task
>  Components: general
>Affects Versions: 2.4
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Kasnacheev
>Priority: Major
>
> It was noticed that some tests are written and updated, but not included in 
> any suite and therefore never run.
>  
> Also look for duplicate tests and other oddities. In the end, every 
> non-abstract test should be included in some suite.
>  
>  
> This will be a first step towards either fixing those tests or discarding 
> them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (IGNITE-7615) Find orphaned tests without test suites, create separate test suite for them

[Ilya Kasnacheev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Kasnacheev updated IGNITE-7615:

Description: Also look for duplicate tests and other oddities. In the end, 
every non-abstract test should be included in some suite.  (was: Also look for 
duplicate tests and other oddities)

> Find orphaned tests without test suites, create separate test suite for them
> 
>
> Key: IGNITE-7615
> URL: https://issues.apache.org/jira/browse/IGNITE-7615
> Project: Ignite
>  Issue Type: Task
>  Components: general
>Affects Versions: 2.4
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Kasnacheev
>Priority: Major
>
> Also look for duplicate tests and other oddities. In the end, every 
> non-abstract test should be included in some suite.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7337) Spark Data Frames: support saving a data frame in Ignite

[Nikolay Izhikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350260#comment-16350260
 ] 

Nikolay Izhikov commented on IGNITE-7337:
-

TC results - 
https://ci.ignite.apache.org/viewQueued.html?itemId=1072184=queuedBuildOverviewTab

> Spark Data Frames: support saving a data frame in Ignite
> 
>
> Key: IGNITE-7337
> URL: https://issues.apache.org/jira/browse/IGNITE-7337
> Project: Ignite
>  Issue Type: New Feature
>  Components: spark
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Nikolay Izhikov
>Priority: Major
> Fix For: 2.5
>
>
> Once Ignite data source for Spark is implemented, we need to add an ability 
> to store a data frame in Ignite. Most likely if should be enough to provide 
> implementation for the following traits:
> * {{InsertableRelation}}
> * {{CreatableRelationProvider}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (IGNITE-7615) Find orphaned tests without test suites, create separate test suite for them

[Ilya Kasnacheev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Kasnacheev reassigned IGNITE-7615:
---

Assignee: Ilya Kasnacheev

> Find orphaned tests without test suites, create separate test suite for them
> 
>
> Key: IGNITE-7615
> URL: https://issues.apache.org/jira/browse/IGNITE-7615
> Project: Ignite
>  Issue Type: Task
>  Components: general
>Affects Versions: 2.4
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Kasnacheev
>Priority: Major
>
> Also look for duplicate tests and other oddities



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (IGNITE-7613) Unable to run example classes for JDK9 in IDEA

[Sergey Kozlov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Kozlov resolved IGNITE-7613.
---
Resolution: Fixed

Idea must be updated to the latest one (for my case 2017.3.3)

> Unable to run example classes for JDK9 in IDEA
> --
>
> Key: IGNITE-7613
> URL: https://issues.apache.org/jira/browse/IGNITE-7613
> Project: Ignite
>  Issue Type: Bug
>  Components: examples
>Affects Versions: 2.4
> Environment: Windows 10, Oracle JDK 9.0.4, Apache Maven 3.3.9, IDEA 
> 2017.1.3
>Reporter: Sergey Kozlov
>Priority: Critical
> Fix For: 2.4
>
>
> I built the AI binary fabric from 2.4 sources and then import pom.xml in IDEA 
> from {{examples}} directory ( {{ignite-examples}} project). The pom file 
> imported and then compiled successfully.
> But trying to run any example leads to failure:
> {noformat}
> Information:java: Errors occurred while compiling module 'ignite-examples'
> Information:javac 9.0.4 was used to compile java sources
> Information:02.02.2018 13:32 - Compilation completed with 1 error and 0 
> warnings in 1s 749ms
> Error:java: invalid flag: -release
> {noformat}
> The same issue appers if I tried to rebuild the chosed example.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7615) Find orphaned tests without test suites, create separate test suite for them

[Ilya Kasnacheev (JIRA)]

Ilya Kasnacheev created IGNITE-7615:
---

 Summary: Find orphaned tests without test suites, create separate 
test suite for them
 Key: IGNITE-7615
 URL: https://issues.apache.org/jira/browse/IGNITE-7615
 Project: Ignite
  Issue Type: Task
  Components: general
Affects Versions: 2.4
Reporter: Ilya Kasnacheev


Also look for duplicate tests and other oddities



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (IGNITE-425) Introduce transformers for continuous queries

[Nikolay Izhikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350242#comment-16350242
 ] 

Nikolay Izhikov edited comment on IGNITE-425 at 2/2/18 12:47 PM:
-

[~avinogradov]

I changed benchmark.
Please, see source changes.

Got following number on couple of runs:

||Measure||Value||
|Warmup count|3|
|Iteration count|5|
|Period|20 sec|
|Write threads|4|

||Test||Description||
|CQBenchmark|Regular Continuous Query|
|CQWTIdBenchmark|Transformer returns Long|
|CQWTNullBenchmark|Transformer returns Null|
|CQWTValueBenchmark|Transformer retrurn whole Value|

EvtCnt - count of received events in Continuous Query. The bigger the better

{noformat}
Benchmark  Mode  CntScore   Error  
Units
CQBenchmark.testMethodthrpt5   12,084 ± 1,111  
ops/s
CQBenchmark.testMethod:evtCnt thrpt5  7438039,000   
   #
CQBenchmark.testMethod:methodExecuted thrpt5 1815,000   
   #
CQWTIdBenchmark.putBatch  thrpt5   12,782 ± 0,290  
ops/s
CQWTIdBenchmark.putBatch:evtCnt   thrpt5  7867990,000   
   #
CQWTIdBenchmark.putBatch:methodExecuted   thrpt5 1920,000   
   #
CQWTNullBenchmark.testMethod  thrpt5   12,601 ± 0,960  
ops/s
CQWTNullBenchmark.testMethod:evtCnt   thrpt5  7762282,000   
   #
CQWTNullBenchmark.testMethod:methodExecuted   thrpt5 1894,000   
   #
CQWTValueBenchmark.testMethod thrpt5   11,945 ± 0,522  
ops/s
CQWTValueBenchmark.testMethod:evtCnt  thrpt5  7356065,000   
   #
CQWTValueBenchmark.testMethod:methodExecuted  thrpt5 1795,000   
   #


BenchmarkMode  CntScore   Error  
Units
CQBenchmark.putBatchthrpt5   12,158 ± 3,897  
ops/s
CQBenchmark.putBatch:evtCnt thrpt5  7482138,000 
 #
CQBenchmark.putBatch:methodExecuted thrpt5 1826,000 
 #
CQWTIdBenchmark.putBatchthrpt5   12,736 ± 0,399  
ops/s
CQWTIdBenchmark.putBatch:evtCnt thrpt5  7843317,000 
 #
CQWTIdBenchmark.putBatch:methodExecuted thrpt5 1914,000 
 #
CQWTNullBenchmark.putBatch  thrpt5   12,800 ± 0,565  
ops/s
CQWTNullBenchmark.putBatch:evtCnt   thrpt5  7877051,000 
 #
CQWTNullBenchmark.putBatch:methodExecuted   thrpt5 1922,000 
 #
CQWTValueBenchmark.putBatch thrpt5   11,785 ± 0,641  
ops/s
CQWTValueBenchmark.putBatch:evtCnt  thrpt5  7255440,000 
 #
CQWTValueBenchmark.putBatch:methodExecuted  thrpt5 1770,000 
 #

BenchmarkMode  CntScore   Error  
Units
CQBenchmark.putBatchthrpt5   12,714 ± 0,972  
ops/s
CQBenchmark.putBatch:evtCnt thrpt5  7823989,000 
 #
CQBenchmark.putBatch:methodExecuted thrpt5 1909,000 
 #
CQWTIdBenchmark.putBatchthrpt5   12,706 ± 0,672  
ops/s
CQWTIdBenchmark.putBatch:evtCnt thrpt5  7818670,000 
 #
CQWTIdBenchmark.putBatch:methodExecuted thrpt5 1908,000 
 #
CQWTNullBenchmark.putBatch  thrpt5   12,820 ± 0,766  
ops/s
CQWTNullBenchmark.putBatch:evtCnt   thrpt5  7893313,000 
 #
CQWTNullBenchmark.putBatch:methodExecuted   thrpt5 1926,000 
 #
CQWTValueBenchmark.putBatch thrpt5   11,735 ± 0,801  
ops/s
CQWTValueBenchmark.putBatch:evtCnt  thrpt5  7220725,000 
 #
CQWTValueBenchmark.putBatch:methodExecuted  thrpt5 1762,000 
 #
{noformat}


was (Author: nizhikov):
[~avinogradov]

I changed benchmark.
Please, see source changes.

Got following number on couple of runs:

||Measure||Value||
|Warmup count|3|
|Iteration count|5|
|Period|20 sec|
|Write threads|4|

||Test||Description||
|CQBenchmark|Regular Continuous Query|
|CQWTIdBenchmark|Transformer returns Long|
|CQWTNullBenchmark|Transformer returns Null|
|CQWTValueBenchmark|Transformer retrurn whole Value|

{noformat}
Benchmark  Mode  CntScore   Error  
Units
CQBenchmark.testMethodthrpt5   12,084 ± 1,111  
ops/s
CQBenchmark.testMethod:evtCnt thrpt5  7438039,000   
   #
CQBenchmark.testMethod:methodExecuted thrpt5 1815,000   
   #
CQWTIdBenchmark.putBatch  thrpt5   12,782 ± 0,290  
ops/s
CQWTIdBenchmark.putBatch:evtCnt

[jira] [Closed] (IGNITE-7613) Unable to run example classes for JDK9 in IDEA

[Sergey Kozlov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Kozlov closed IGNITE-7613.
-

> Unable to run example classes for JDK9 in IDEA
> --
>
> Key: IGNITE-7613
> URL: https://issues.apache.org/jira/browse/IGNITE-7613
> Project: Ignite
>  Issue Type: Bug
>  Components: examples
>Affects Versions: 2.4
> Environment: Windows 10, Oracle JDK 9.0.4, Apache Maven 3.3.9, IDEA 
> 2017.1.3
>Reporter: Sergey Kozlov
>Priority: Critical
> Fix For: 2.4
>
>
> I built the AI binary fabric from 2.4 sources and then import pom.xml in IDEA 
> from {{examples}} directory ( {{ignite-examples}} project). The pom file 
> imported and then compiled successfully.
> But trying to run any example leads to failure:
> {noformat}
> Information:java: Errors occurred while compiling module 'ignite-examples'
> Information:javac 9.0.4 was used to compile java sources
> Information:02.02.2018 13:32 - Compilation completed with 1 error and 0 
> warnings in 1s 749ms
> Error:java: invalid flag: -release
> {noformat}
> The same issue appers if I tried to rebuild the chosed example.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (IGNITE-7262) Most of clients get stuck with "Failed to wait for partition map ..." during load test

[Ksenia Rybakova (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ksenia Rybakova updated IGNITE-7262:

Attachment: run-load.xml_2
run-load.properties_2
ignite-base-load-config.xml_2

> Most of clients get stuck with "Failed to wait for partition map ..." during 
> load test
> --
>
> Key: IGNITE-7262
> URL: https://issues.apache.org/jira/browse/IGNITE-7262
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 2.1
>Reporter: Ksenia Rybakova
>Priority: Major
> Attachments: 
> 122310_id1_172.25.1.27_cache-random-benchmark-1-backup.log, 
> ignite-base-load-config.xml, ignite-base-load-config.xml_2, 
> run-load.properties, run-load.properties_2, run-load.xml, run-load.xml_2
>
>
> Most of clients get stuck with "Failed to wait for partition map ..." during 
> load test (preloading phase) with 500 logical caches or 30 physical. Others 
> preload all data and perform operations successfully.
> Load configs:
> 1)
>  - CacheRandomOperationBenchmark
>  - 14 client nodes, 42 server nodes
>  - 50K perloading, 60K key range
>  - 26 physical caches
>  - 500 logical caches
>  - 1 backup
> 2) 
>  - CacheRandomOperationBenchmark
>  - 8 client nodes, 40 server nodes
>  - 200K perloading, 500K key range
>  - 30 physical caches
>  - 1 backup
> Complete configs and log of one of the clients are attached.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (IGNITE-7262) Most of clients get stuck with "Failed to wait for partition map ..." during load test

[Ksenia Rybakova (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ksenia Rybakova updated IGNITE-7262:

Description: 
Most of clients get stuck with "Failed to wait for partition map ..." during 
load test (preloading phase) with 500 logical caches or 30 physical. Others 
preload all data and perform operations successfully.

Load configs:

1)
 - CacheRandomOperationBenchmark
 - 14 client nodes, 42 server nodes
 - 50K perloading, 60K key range
 - 26 physical caches
 - 500 logical caches
 - 1 backup

2) 
 - CacheRandomOperationBenchmark
 - 8 client nodes, 40 server nodes
 - 200K perloading, 500K key range
 - 30 physical caches
 - 1 backup

Complete configs and log of one of the clients are attached.

  was:
Most of clients get stuck with "Failed to wait for partition map ..." during 
load test (preloading phase) with 500 logical caches. Others preload all data 
and perform operations successfully.

Load config:
- CacheRandomOperationBenchmark
- 14 client nodes, 42 server nodes
- 50K perloading, 60K key range
- 26 physical caches
- 500 logical caches
- 1 backup
Complete configs and log of one of the clients are attached.

Summary: Most of clients get stuck with "Failed to wait for partition 
map ..." during load test  (was: Most of clients get stuck with "Failed to wait 
for partition map ..." during load test with 500 logical caches)

> Most of clients get stuck with "Failed to wait for partition map ..." during 
> load test
> --
>
> Key: IGNITE-7262
> URL: https://issues.apache.org/jira/browse/IGNITE-7262
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 2.1
>Reporter: Ksenia Rybakova
>Priority: Major
> Attachments: 
> 122310_id1_172.25.1.27_cache-random-benchmark-1-backup.log, 
> ignite-base-load-config.xml, run-load.properties, run-load.xml
>
>
> Most of clients get stuck with "Failed to wait for partition map ..." during 
> load test (preloading phase) with 500 logical caches or 30 physical. Others 
> preload all data and perform operations successfully.
> Load configs:
> 1)
>  - CacheRandomOperationBenchmark
>  - 14 client nodes, 42 server nodes
>  - 50K perloading, 60K key range
>  - 26 physical caches
>  - 500 logical caches
>  - 1 backup
> 2) 
>  - CacheRandomOperationBenchmark
>  - 8 client nodes, 40 server nodes
>  - 200K perloading, 500K key range
>  - 30 physical caches
>  - 1 backup
> Complete configs and log of one of the clients are attached.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (IGNITE-425) Introduce transformers for continuous queries

[Nikolay Izhikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350242#comment-16350242
 ] 

Nikolay Izhikov edited comment on IGNITE-425 at 2/2/18 12:41 PM:
-

[~avinogradov]

I changed benchmark.
Please, see source changes.

Got following number on couple of runs:

||Measure||Value||
|Warmup count|3|
|Iteration count|5|
|Period|20 sec|
|Write threads|4|

||Test||Description||
|CQBenchmark|Regular Continuous Query|
|CQWTIdBenchmark|Transformer returns Long|
|CQWTNullBenchmark|Transformer returns Null|
|CQWTValueBenchmark|Transformer retrurn whole Value|

{noformat}
Benchmark  Mode  CntScore   Error  
Units
CQBenchmark.testMethodthrpt5   12,084 ± 1,111  
ops/s
CQBenchmark.testMethod:evtCnt thrpt5  7438039,000   
   #
CQBenchmark.testMethod:methodExecuted thrpt5 1815,000   
   #
CQWTIdBenchmark.putBatch  thrpt5   12,782 ± 0,290  
ops/s
CQWTIdBenchmark.putBatch:evtCnt   thrpt5  7867990,000   
   #
CQWTIdBenchmark.putBatch:methodExecuted   thrpt5 1920,000   
   #
CQWTNullBenchmark.testMethod  thrpt5   12,601 ± 0,960  
ops/s
CQWTNullBenchmark.testMethod:evtCnt   thrpt5  7762282,000   
   #
CQWTNullBenchmark.testMethod:methodExecuted   thrpt5 1894,000   
   #
CQWTValueBenchmark.testMethod thrpt5   11,945 ± 0,522  
ops/s
CQWTValueBenchmark.testMethod:evtCnt  thrpt5  7356065,000   
   #
CQWTValueBenchmark.testMethod:methodExecuted  thrpt5 1795,000   
   #


BenchmarkMode  CntScore   Error  
Units
CQBenchmark.putBatchthrpt5   12,158 ± 3,897  
ops/s
CQBenchmark.putBatch:evtCnt thrpt5  7482138,000 
 #
CQBenchmark.putBatch:methodExecuted thrpt5 1826,000 
 #
CQWTIdBenchmark.putBatchthrpt5   12,736 ± 0,399  
ops/s
CQWTIdBenchmark.putBatch:evtCnt thrpt5  7843317,000 
 #
CQWTIdBenchmark.putBatch:methodExecuted thrpt5 1914,000 
 #
CQWTNullBenchmark.putBatch  thrpt5   12,800 ± 0,565  
ops/s
CQWTNullBenchmark.putBatch:evtCnt   thrpt5  7877051,000 
 #
CQWTNullBenchmark.putBatch:methodExecuted   thrpt5 1922,000 
 #
CQWTValueBenchmark.putBatch thrpt5   11,785 ± 0,641  
ops/s
CQWTValueBenchmark.putBatch:evtCnt  thrpt5  7255440,000 
 #
CQWTValueBenchmark.putBatch:methodExecuted  thrpt5 1770,000 
 #

BenchmarkMode  CntScore   Error  
Units
CQBenchmark.putBatchthrpt5   12,714 ± 0,972  
ops/s
CQBenchmark.putBatch:evtCnt thrpt5  7823989,000 
 #
CQBenchmark.putBatch:methodExecuted thrpt5 1909,000 
 #
CQWTIdBenchmark.putBatchthrpt5   12,706 ± 0,672  
ops/s
CQWTIdBenchmark.putBatch:evtCnt thrpt5  7818670,000 
 #
CQWTIdBenchmark.putBatch:methodExecuted thrpt5 1908,000 
 #
CQWTNullBenchmark.putBatch  thrpt5   12,820 ± 0,766  
ops/s
CQWTNullBenchmark.putBatch:evtCnt   thrpt5  7893313,000 
 #
CQWTNullBenchmark.putBatch:methodExecuted   thrpt5 1926,000 
 #
CQWTValueBenchmark.putBatch thrpt5   11,735 ± 0,801  
ops/s
CQWTValueBenchmark.putBatch:evtCnt  thrpt5  7220725,000 
 #
CQWTValueBenchmark.putBatch:methodExecuted  thrpt5 1762,000 
 #
{noformat}


was (Author: nizhikov):
[~avinogradov]

I changed benchmark.
Please, see source changes.

Got following number on couple of runs:

||Measure||Value||
|Warmup count|3|
|Iteration count|5|
|Period|20 sec|
|Write threads|4|

{noformat}
Benchmark  Mode  CntScore   Error  
Units
CQBenchmark.testMethodthrpt5   12,084 ± 1,111  
ops/s
CQBenchmark.testMethod:evtCnt thrpt5  7438039,000   
   #
CQBenchmark.testMethod:methodExecuted thrpt5 1815,000   
   #
CQWTIdBenchmark.putBatch  thrpt5   12,782 ± 0,290  
ops/s
CQWTIdBenchmark.putBatch:evtCnt   thrpt5  7867990,000   
   #
CQWTIdBenchmark.putBatch:methodExecuted   thrpt5 1920,000   
   #
CQWTNullBenchmark.testMethod  thrpt5   12,601 ± 0,960  
ops/s
CQWTNullBenchmark.testMethod:evtCnt   thrpt5  

[jira] [Commented] (IGNITE-425) Introduce transformers for continuous queries

[Nikolay Izhikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350242#comment-16350242
 ] 

Nikolay Izhikov commented on IGNITE-425:


[~avinogradov]

I changed benchmark.
Please, see source changes.

Got following number on couple of runs:

||Measure||Value||
|Warmup count|3|
|Iteration count|5|
|Period|20 sec|
|Write threads|4|

{noformat}
Benchmark  Mode  CntScore   Error  
Units
CQBenchmark.testMethodthrpt5   12,084 ± 1,111  
ops/s
CQBenchmark.testMethod:evtCnt thrpt5  7438039,000   
   #
CQBenchmark.testMethod:methodExecuted thrpt5 1815,000   
   #
CQWTIdBenchmark.putBatch  thrpt5   12,782 ± 0,290  
ops/s
CQWTIdBenchmark.putBatch:evtCnt   thrpt5  7867990,000   
   #
CQWTIdBenchmark.putBatch:methodExecuted   thrpt5 1920,000   
   #
CQWTNullBenchmark.testMethod  thrpt5   12,601 ± 0,960  
ops/s
CQWTNullBenchmark.testMethod:evtCnt   thrpt5  7762282,000   
   #
CQWTNullBenchmark.testMethod:methodExecuted   thrpt5 1894,000   
   #
CQWTValueBenchmark.testMethod thrpt5   11,945 ± 0,522  
ops/s
CQWTValueBenchmark.testMethod:evtCnt  thrpt5  7356065,000   
   #
CQWTValueBenchmark.testMethod:methodExecuted  thrpt5 1795,000   
   #


BenchmarkMode  CntScore   Error  
Units
CQBenchmark.putBatchthrpt5   12,158 ± 3,897  
ops/s
CQBenchmark.putBatch:evtCnt thrpt5  7482138,000 
 #
CQBenchmark.putBatch:methodExecuted thrpt5 1826,000 
 #
CQWTIdBenchmark.putBatchthrpt5   12,736 ± 0,399  
ops/s
CQWTIdBenchmark.putBatch:evtCnt thrpt5  7843317,000 
 #
CQWTIdBenchmark.putBatch:methodExecuted thrpt5 1914,000 
 #
CQWTNullBenchmark.putBatch  thrpt5   12,800 ± 0,565  
ops/s
CQWTNullBenchmark.putBatch:evtCnt   thrpt5  7877051,000 
 #
CQWTNullBenchmark.putBatch:methodExecuted   thrpt5 1922,000 
 #
CQWTValueBenchmark.putBatch thrpt5   11,785 ± 0,641  
ops/s
CQWTValueBenchmark.putBatch:evtCnt  thrpt5  7255440,000 
 #
CQWTValueBenchmark.putBatch:methodExecuted  thrpt5 1770,000 
 #

BenchmarkMode  CntScore   Error  
Units
CQBenchmark.putBatchthrpt5   12,714 ± 0,972  
ops/s
CQBenchmark.putBatch:evtCnt thrpt5  7823989,000 
 #
CQBenchmark.putBatch:methodExecuted thrpt5 1909,000 
 #
CQWTIdBenchmark.putBatchthrpt5   12,706 ± 0,672  
ops/s
CQWTIdBenchmark.putBatch:evtCnt thrpt5  7818670,000 
 #
CQWTIdBenchmark.putBatch:methodExecuted thrpt5 1908,000 
 #
CQWTNullBenchmark.putBatch  thrpt5   12,820 ± 0,766  
ops/s
CQWTNullBenchmark.putBatch:evtCnt   thrpt5  7893313,000 
 #
CQWTNullBenchmark.putBatch:methodExecuted   thrpt5 1926,000 
 #
CQWTValueBenchmark.putBatch thrpt5   11,735 ± 0,801  
ops/s
CQWTValueBenchmark.putBatch:evtCnt  thrpt5  7220725,000 
 #
CQWTValueBenchmark.putBatch:methodExecuted  thrpt5 1762,000 
 #
{noformat}

> Introduce transformers for continuous queries
> -
>
> Key: IGNITE-425
> URL: https://issues.apache.org/jira/browse/IGNITE-425
> Project: Ignite
>  Issue Type: Sub-task
>  Components: cache
>Affects Versions: 2.2
>Reporter: Yakov Zhdanov
>Assignee: Nikolay Izhikov
>Priority: Major
> Fix For: 2.5
>
>
> Currently if updated entry passes the filter, it is sent to node initiated 
> the query entirely. It would be good to provide user with the ability to 
> transform entry and, for example, select only fields that are important. This 
> may bring huge economy to traffic and lower GC pressure as well.
> Possible signatures will be:
> {noformat}
> public final class ContinuousQuery {..} // T is a type transformer 
> transforms to
> public ContinuousQuery setLocalListener(Listener locLsnr) {..} // 
> Probably, we will have to introduce new listener type, since user may want to 
> wipe out key as well.
> /* new method to add */
> public ContinuousQuery setRemoteTransformerFactory(Factory ContinuousQueryTransformer> factory) { ..}
> 

[jira] [Comment Edited] (IGNITE-6917) SQL: implement COPY command for efficient data loading

[Alexander Paschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349007#comment-16349007
 ] 

Alexander Paschenko edited comment on IGNITE-6917 at 2/2/18 12:28 PM:
--

[~kirill.shirokov],

31. {{BulkLoadDataConverter}} is now empty and doesn't convert anything. I 
think we should remove it and simply pass {{UpdatePlan}} to 
{{BulkLoadProcessor}}.

32. {{BulkLoadCacheWriter}}:

32.1 Odd empty line after interface header.

32.2 Leaving this interface be does not contradict with making it extend some 
of existing Ignite interfaces. Please add another appropriate ancestor to this 
interface.

33. Regarding this:

>Does this anti-pattern makes sense here? Or we're about keeping similarity 
>with existing code and bugward compatibility?

First, providing for resilience is one of the reasons that justify "error 
hiding". We aim not to let JDBC handler or NIO listener crash.

Second, here's no "hiding" per se as exception is properly logged and we send 
the user message.

34. {{BulkLoadProcessor}}:

34.1 odd empty line after ctor header.

34.2 {{processBatch}} is still coupled with JDBC related class.

34.3 I think class name {{BulkLoadContext}} would reflect what this class does 
more correctly. In fact, it does not process anything, just holds various bits 
that help the process. In favor of this tells the fact that you even use the 
word "context" in javadoc for this class.

35. {{BulkLoadParser}}: is still coupled with JDBC related class. I have 
already suggested that we introduce some JDBC neutral enum to indicate current 
stage of the process.

36. {{JdbcRequestHandler#executeQuery}} - odd empty line after your new "if".

37. {{JdbcBulkLoadBatchAckResult}}: I suggested simply 
{{JdbcBulkLoadAckResult}}, without the word "batch", as this ack response has 
to do with the command overall and *not* with an individual batch. It's sent 
once per bulk load.

38. {{UpdateMode}}: there's a typo in javadocs you've added to all enum 
members. Please correct: com *_m_* and

39. {{UpdatePlanBuilder}}: unused new static import.

40. {{BulkLoadStreamerWriter}}: why is counter internally stored as {{int}}, 
not {{long}}?

41. {{JdbcQueryTask#call}}: I think it would be better to gracefully close 
newly created context before throwing an exception.

42. {{BulkLoadCsvFormat}}: many new members and their getters have "Re" suffix. 
It's not standard to Ignite codebase. I would either replace it to "regex" or 
"regexp", or even remove it - the user of this class can clearly see what is 
return type of those getters, and those suffixes actually don't tell anything 
new.

43. {{DdlStatementsProcessor#runDdlStatement}}: why employ log throttling here? 
Why would we want to skip some error messages?

This is it for now, thanks. Looking forward to fixes.


was (Author: al.psc):
[~kirill.shirokov],

31. {{BulkLoadDataConverter}} is now empty and doesn't convert anything. I 
think we should remove it and simply pass {{UpdatePlan}} to 
{{BulkLoadProcessor}}.

32. {{BulkLoadCacheWriter}}:

32.1 Odd empty line after interface header.

32.2 Leaving this interface be does not contradict with making it extend some 
of existing Ignite interfaces. Please add another appropriate ancestor to this 
interface.

33. Regarding this:

>Does this anti-pattern makes sense here? Or we're about keeping similarity 
>with existing code and bugward compatibility?

First, providing for resilience is one of the reasons that justify "error 
hiding". We aim not to let JDBC handler or NIO listener crash.

Second, here's no "hiding" per se as exception is properly logged and we send 
the user message.

34. {{BulkLoadProcessor}}:

34.1 odd empty line after ctor header.

34.2 {{processBatch}} is still coupled with JDBC related class.

34.3 I think class name {{BulkLoadContext}} would reflect what this class does 
more correctly. In fact, it does not process anything, just holds various bits 
that help the process. In favor of this tells the fact that you even use the 
word "context" in javadoc for this class.

35. {{BulkLoadParser}}: is still coupled with JDBC related class. I have 
already suggested that we introduce some JDBC neutral enum to indicate current 
stage of the process.

36. {{JdbcRequestHandler#executeQuery}} - odd empty line after your new "if".

37. {{JdbcBulkLoadBatchAckResult}}: I suggested simply 
{{JdbcBulkLoadAckResult}}, without the word "batch", as this ack response has 
to do with the command overall and *not* with an individual batch. It's sent 
once per bulk load.

38. \{{UpdateMode: t}}here's a typo in javadocs you've added to all enum 
members. Please correct: com*_m_*and

39. {{UpdatePlanBuilder}}: unused new static import.

40. {{BulkLoadStreamerWriter}}: why is counter internally stored as {{int}}, 
not {{long}}?

41. {{JdbcQueryTask#call}}: I think it would be better to gracefully close 
newly created 

[jira] [Commented] (IGNITE-7489) Weird FillFactor metric fluctuation.

[Ilya Kasnacheev (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350229#comment-16350229
 ] 

Ilya Kasnacheev commented on IGNITE-7489:
-

[~amashenkov] please review ^

> Weird FillFactor metric fluctuation.
> 
>
> Key: IGNITE-7489
> URL: https://issues.apache.org/jira/browse/IGNITE-7489
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1, 2.2, 2.3
>Reporter: Andrew Mashenkov
>Assignee: Ilya Kasnacheev
>Priority: Major
> Attachments: FillFactorTest.java
>
>
> For now there is no way to get Used\FreeMemory for region in bytes. 
> MemoryMetrics.getPagesFillFactor() javadoc says that the method return the 
> percentage of space that is still free and can be filled in.
> So, I'd think used memory can be calculated as 
> PhysicalMemoryPages*PageSize*FillFactor.
> I've tried to use this, but found weir thing.
>  
> PFA a repro.
> There is 2 node grid with no persistence configure. Topology is stable.
> Cache is populated with unique keys (no updates) and observe allocated data 
> pages metric grows constantly as expected.
> But the error look too large, used memory (and FillFactor as well) may 2-10+ 
> time differs.
> E.g. allocated pages, fillFactor, usedMem (bytes):(
>  node-0: 13789 0.851563 48096032
>  node-1: 14447 0.781250 46230400
> In next second:
> node-0: 13958 0.039063 2233280
>  node-1: 14624 0.347656 20824576
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-6625) JDBC thin: support SSL connection to Ignite node

[Taras Ledkov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350226#comment-16350226
 ] 

Taras Ledkov commented on IGNITE-6625:
--

[~gvvinblade], But other way is possible. Please take a look at the comment 
above that contains MySQL & Ignite options comparison.
{{required}} flag may be added later, when it is really required.

OK, I'll change {{useSSL}} option to {{sslMode}} option with multiple choices.
Now we will support two modes: *require* and *disable* (default). Is it OK?

> JDBC thin: support SSL connection to Ignite node
> 
>
> Key: IGNITE-6625
> URL: https://issues.apache.org/jira/browse/IGNITE-6625
> Project: Ignite
>  Issue Type: Improvement
>  Components: jdbc
>Affects Versions: 2.2
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Fix For: 2.5
>
>
> SSL connection must be supported for JDBC thin driver.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (IGNITE-6625) JDBC thin: support SSL connection to Ignite node

[Igor Seliverstov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350197#comment-16350197
 ] 

Igor Seliverstov edited comment on IGNITE-6625 at 2/2/18 12:22 PM:
---

[~tledkov-gridgain], in ODBC driver we use SSL_MODE property with (initially) 
two possible values: require and disable (default), I think we should use the 
same modes in JDBC.

For example PG implementation includes "disable", "require", "verify-ca" and 
"verify-full", "allow" and "prefer" modes.


was (Author: gvvinblade):
[~tledkov-gridgain], in ODBC driver we use SSL_MODE property with (initially) 
two possible values: require and disable (default), I think we should use the 
same modes in JDBC implemntation.

For example PG implementation includes "disable", "require", "verify-ca" and 
"verify-full", "allow" and "prefer" modes.

> JDBC thin: support SSL connection to Ignite node
> 
>
> Key: IGNITE-6625
> URL: https://issues.apache.org/jira/browse/IGNITE-6625
> Project: Ignite
>  Issue Type: Improvement
>  Components: jdbc
>Affects Versions: 2.2
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Fix For: 2.5
>
>
> SSL connection must be supported for JDBC thin driver.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (IGNITE-6625) JDBC thin: support SSL connection to Ignite node

[Igor Seliverstov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350197#comment-16350197
 ] 

Igor Seliverstov edited comment on IGNITE-6625 at 2/2/18 12:22 PM:
---

[~tledkov-gridgain], in ODBC driver we use SSL_MODE property with (initially) 
two possible values: require and disable (default), I think we should use the 
same modes in JDBC implemntation.

For example PG implementation includes "disable", "require", "verify-ca" and 
"verify-full", "allow" and "prefer" modes.


was (Author: gvvinblade):
[~tledkov-gridgain], in ODBC driver we use SSL_MODE property with (initially) 
two possible values: require and disable (default), I think we should use the 
same modes in JDBC implemntation.

For example PG realisation includes "disable", "require", "verify-ca" and 
"verify-full", "allow" and "prefer" modes.

> JDBC thin: support SSL connection to Ignite node
> 
>
> Key: IGNITE-6625
> URL: https://issues.apache.org/jira/browse/IGNITE-6625
> Project: Ignite
>  Issue Type: Improvement
>  Components: jdbc
>Affects Versions: 2.2
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Fix For: 2.5
>
>
> SSL connection must be supported for JDBC thin driver.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-6217) Add benchmark to compare JDBC drivers and native SQL execution

[Pavel Kuznetsov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350218#comment-16350218
 ] 

Pavel Kuznetsov commented on IGNITE-6217:
-

I think we are ready to merge


> Add benchmark to compare JDBC drivers and native SQL execution
> --
>
> Key: IGNITE-6217
> URL: https://issues.apache.org/jira/browse/IGNITE-6217
> Project: Ignite
>  Issue Type: Task
>  Components: jdbc, sql, yardstick
>Affects Versions: 2.1
>Reporter: Taras Ledkov
>Assignee: Pavel Kuznetsov
>Priority: Major
>
> We have to compare performance of the native SQL execution (via Ignite SQL 
> API), JDBC v2 driver, that uses Ignite client to connect to grid and JDBC 
> thin client.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7614) Documentation: How to access data from key-value that was inserted from SQL

[Evgenii Zhuravlev (JIRA)]

Evgenii Zhuravlev created IGNITE-7614:
-

 Summary: Documentation: How to access data from key-value that was 
inserted from SQL
 Key: IGNITE-7614
 URL: https://issues.apache.org/jira/browse/IGNITE-7614
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Reporter: Evgenii Zhuravlev
Assignee: Denis Magda
 Fix For: 2.5






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7282) .NET: Thin client: Failover

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350206#comment-16350206
 ] 

Pavel Tupitsyn commented on IGNITE-7282:


Mostly done, config schema update is needed.

> .NET: Thin client: Failover
> ---
>
> Key: IGNITE-7282
> URL: https://issues.apache.org/jira/browse/IGNITE-7282
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Affects Versions: 2.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: .NET
> Fix For: 2.5
>
>
> Currently user has to manually connect to some specific Ignite server.
> Implement some kind of automatic failover where Thin Client knows about 
> multiple nodes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (IGNITE-6625) JDBC thin: support SSL connection to Ignite node

[Igor Seliverstov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350197#comment-16350197
 ] 

Igor Seliverstov edited comment on IGNITE-6625 at 2/2/18 11:49 AM:
---

[~tledkov-gridgain], in ODBC driver we use SSL_MODE property with (initially) 
two possible values: require and disable (default), I think we should use the 
same modes in JDBC implemntation.

For example PG realisation includes "disable", "require", "verify-ca" and 
"verify-full", "allow" and "prefer" modes.


was (Author: gvvinblade):
[~tledkov-gridgain], in ODBC driver we use SSL_MODE property with two possible 
values: require and disable (default), I think we should use the same modes in 
JDBC implemntation

> JDBC thin: support SSL connection to Ignite node
> 
>
> Key: IGNITE-6625
> URL: https://issues.apache.org/jira/browse/IGNITE-6625
> Project: Ignite
>  Issue Type: Improvement
>  Components: jdbc
>Affects Versions: 2.2
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Fix For: 2.5
>
>
> SSL connection must be supported for JDBC thin driver.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-6625) JDBC thin: support SSL connection to Ignite node

[Igor Seliverstov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350197#comment-16350197
 ] 

Igor Seliverstov commented on IGNITE-6625:
--

[~tledkov-gridgain], in ODBC driver we use SSL_MODE property with two possible 
values: require and disable (default), I think we should use the same modes in 
JDBC implemntation

> JDBC thin: support SSL connection to Ignite node
> 
>
> Key: IGNITE-6625
> URL: https://issues.apache.org/jira/browse/IGNITE-6625
> Project: Ignite
>  Issue Type: Improvement
>  Components: jdbc
>Affects Versions: 2.2
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Fix For: 2.5
>
>
> SSL connection must be supported for JDBC thin driver.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (IGNITE-7532) kNN Documentation

[Aleksey Zinoviev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Zinoviev reassigned IGNITE-7532:


Assignee: Denis Magda  (was: Aleksey Zinoviev)

> kNN Documentation
> -
>
> Key: IGNITE-7532
> URL: https://issues.apache.org/jira/browse/IGNITE-7532
> Project: Ignite
>  Issue Type: Task
>  Components: documentation, ml
>Reporter: Aleksey Zinoviev
>Assignee: Denis Magda
>Priority: Major
>  Labels: documentaion
> Fix For: 2.4
>
>
> We want to add kNN Regression and Classification docs



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-6625) JDBC thin: support SSL connection to Ignite node

[Taras Ledkov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350137#comment-16350137
 ] 

Taras Ledkov commented on IGNITE-6625:
--

[~skalashnikov], [~gvvinblade], please review the patch.
JDBC tests are OK.

> JDBC thin: support SSL connection to Ignite node
> 
>
> Key: IGNITE-6625
> URL: https://issues.apache.org/jira/browse/IGNITE-6625
> Project: Ignite
>  Issue Type: Improvement
>  Components: jdbc
>Affects Versions: 2.2
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Fix For: 2.5
>
>
> SSL connection must be supported for JDBC thin driver.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7532) kNN Documentation

[Aleksey Zinoviev (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350139#comment-16350139
 ] 

Aleksey Zinoviev commented on IGNITE-7532:
--

[~dmagda] Thank you for your comments, please look at the edited docs.

> kNN Documentation
> -
>
> Key: IGNITE-7532
> URL: https://issues.apache.org/jira/browse/IGNITE-7532
> Project: Ignite
>  Issue Type: Task
>  Components: documentation, ml
>Reporter: Aleksey Zinoviev
>Assignee: Aleksey Zinoviev
>Priority: Major
>  Labels: documentaion
> Fix For: 2.4
>
>
> We want to add kNN Regression and Classification docs



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-6217) Add benchmark to compare JDBC drivers and native SQL execution

[Pavel Kuznetsov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350193#comment-16350193
 ] 

Pavel Kuznetsov commented on IGNITE-6217:
-

about 2) :
I got non-localhost results: sql update with range works

> Add benchmark to compare JDBC drivers and native SQL execution
> --
>
> Key: IGNITE-6217
> URL: https://issues.apache.org/jira/browse/IGNITE-6217
> Project: Ignite
>  Issue Type: Task
>  Components: jdbc, sql, yardstick
>Affects Versions: 2.1
>Reporter: Taras Ledkov
>Assignee: Pavel Kuznetsov
>Priority: Major
>
> We have to compare performance of the native SQL execution (via Ignite SQL 
> API), JDBC v2 driver, that uses Ignite client to connect to grid and JDBC 
> thin client.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7613) Unable to run example classes for JDK9 in IDEA

[Sergey Kozlov (JIRA)]

Sergey Kozlov created IGNITE-7613:
-

 Summary: Unable to run example classes for JDK9 in IDEA
 Key: IGNITE-7613
 URL: https://issues.apache.org/jira/browse/IGNITE-7613
 Project: Ignite
  Issue Type: Bug
  Components: examples
Affects Versions: 2.4
 Environment: Windows 10, Oracle JDK 9.0.4, Apache Maven 3.3.9, IDEA 
2017.1.3
Reporter: Sergey Kozlov
 Fix For: 2.4


I built the AI binary fabric from 2.4 sources and then import pom.xml in IDEA 
from {{examples}} directory ( {{ignite-examples}} project). The pom file 
imported and then compiled successfully.

But trying to run any example leads to failure:

{noformat}
Information:java: Errors occurred while compiling module 'ignite-examples'
Information:javac 9.0.4 was used to compile java sources
Information:02.02.2018 13:32 - Compilation completed with 1 error and 0 
warnings in 1s 749ms
Error:java: invalid flag: -release
{noformat}

The same issue appers if I tried to rebuild the chosed example.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (IGNITE-7612) Web Console: Refactor mongo models creation.

[Alexey Kuznetsov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Kuznetsov closed IGNITE-7612.


> Web Console: Refactor mongo models creation.
> 
>
> Key: IGNITE-7612
> URL: https://issues.apache.org/jira/browse/IGNITE-7612
> Project: Ignite
>  Issue Type: Improvement
>  Components: wizards
>Reporter: Alexey Kuznetsov
>Assignee: Alexey Kuznetsov
>Priority: Major
> Fix For: 2.5
>
>
> Current implementation is not extendable and not mock-able.
> We could move all mongoose schemas to separate module that can be extended or 
> mocked for tests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (IGNITE-7612) Web Console: Refactor mongo models creation.

[Alexey Kuznetsov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Kuznetsov resolved IGNITE-7612.
--
Resolution: Fixed

Merged to master.

> Web Console: Refactor mongo models creation.
> 
>
> Key: IGNITE-7612
> URL: https://issues.apache.org/jira/browse/IGNITE-7612
> Project: Ignite
>  Issue Type: Improvement
>  Components: wizards
>Reporter: Alexey Kuznetsov
>Assignee: Alexey Kuznetsov
>Priority: Major
> Fix For: 2.5
>
>
> Current implementation is not extendable and not mock-able.
> We could move all mongoose schemas to separate module that can be extended or 
> mocked for tests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (IGNITE-7610) Web Console: Refactor profile page to component

[Alexey Kuznetsov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Kuznetsov closed IGNITE-7610.


> Web Console: Refactor profile page to component
> ---
>
> Key: IGNITE-7610
> URL: https://issues.apache.org/jira/browse/IGNITE-7610
> Project: Ignite
>  Issue Type: Improvement
>  Components: wizards
>Reporter: Alexey Kuznetsov
>Assignee: Alexey Kuznetsov
>Priority: Major
> Fix For: 2.5
>
>
> Refactor Profile page to component to facilitate the transition from 
> AngularJS to Angular.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (IGNITE-7610) Web Console: Refactor profile page to component

[Alexey Kuznetsov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Kuznetsov resolved IGNITE-7610.
--
Resolution: Fixed

Merged to master.

> Web Console: Refactor profile page to component
> ---
>
> Key: IGNITE-7610
> URL: https://issues.apache.org/jira/browse/IGNITE-7610
> Project: Ignite
>  Issue Type: Improvement
>  Components: wizards
>Reporter: Alexey Kuznetsov
>Assignee: Alexey Kuznetsov
>Priority: Major
> Fix For: 2.5
>
>
> Refactor Profile page to component to facilitate the transition from 
> AngularJS to Angular.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (IGNITE-7612) Web Console: Refactor mongo models creation.

[Alexey Kuznetsov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Kuznetsov reassigned IGNITE-7612:


Assignee: Alexey Kuznetsov

> Web Console: Refactor mongo models creation.
> 
>
> Key: IGNITE-7612
> URL: https://issues.apache.org/jira/browse/IGNITE-7612
> Project: Ignite
>  Issue Type: Improvement
>  Components: wizards
>Reporter: Alexey Kuznetsov
>Assignee: Alexey Kuznetsov
>Priority: Major
> Fix For: 2.5
>
>
> Current implementation is not extendable and not mock-able.
> We could move all mongoose schemas to separate module that can be extended or 
> mocked for tests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7612) Web Console: Refactor mongo models creation.

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-7612:


 Summary: Web Console: Refactor mongo models creation.
 Key: IGNITE-7612
 URL: https://issues.apache.org/jira/browse/IGNITE-7612
 Project: Ignite
  Issue Type: Improvement
  Components: wizards
Reporter: Alexey Kuznetsov
 Fix For: 2.5


Current implementation is not extendable and not mock-able.

We could move all mongoose schemas to separate module that can be extended or 
mocked for tests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7611) Document logger configuration options

[Stanislav Lukyanov (JIRA)]

Stanislav Lukyanov created IGNITE-7611:
--

 Summary: Document logger configuration options
 Key: IGNITE-7611
 URL: https://issues.apache.org/jira/browse/IGNITE-7611
 Project: Ignite
  Issue Type: Bug
  Components: documentation
Reporter: Stanislav Lukyanov
Assignee: Stanislav Lukyanov


Existing documentation on readme.io 
(https://apacheignite.readme.io/docs/logging) gives details on how to enable 
each of the supported logging frameworks, and some info on the default 
configuration (e.g. IGNITE_QUIET).

The documentation should also cover some other features of Ignite logging, such 
as recently added features of automatic reconfiguration of log4j 1.x and 2.x 
(IGNITE-6946) and DEV_ONLY marker in logging messages (IGNITE-7284), as well as 
other features like automatic metrics logging (MetricsLogFrequency) and 
performance suggestions on start (IGNITE_PERFORMANCE_SUGGESTIONS_DISABLED).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7610) Web Console: Refactor profile page to component

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-7610:


 Summary: Web Console: Refactor profile page to component
 Key: IGNITE-7610
 URL: https://issues.apache.org/jira/browse/IGNITE-7610
 Project: Ignite
  Issue Type: Improvement
  Components: wizards
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov
 Fix For: 2.5


Refactor Profile page to component to facilitate the transition from AngularJS 
to Angular.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-6094) Web console: Implement persistent store in demo mode.

[Pavel Konstantinov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350025#comment-16350025
 ] 

Pavel Konstantinov commented on IGNITE-6094:


Please tune WAL configuration  to decrease disk consumption (currently it takes 
2.5 GB)

> Web console: Implement persistent store in demo mode.
> -
>
> Key: IGNITE-6094
> URL: https://issues.apache.org/jira/browse/IGNITE-6094
> Project: Ignite
>  Issue Type: Bug
>  Components: wizards
>Affects Versions: 2.1
>Reporter: Vasiliy Sisko
>Assignee: Pavel Konstantinov
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (IGNITE-7600) visorcmd: add column 'type' for 'mlist' result

[Pavel Konstantinov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Konstantinov updated IGNITE-7600:
---
Fix Version/s: 2.5

> visorcmd: add column 'type' for 'mlist' result
> --
>
> Key: IGNITE-7600
> URL: https://issues.apache.org/jira/browse/IGNITE-7600
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Konstantinov
>Priority: Trivial
> Fix For: 2.5
>
>
> It should print type of variable. F.e. n0..nX - node, h0..hX - host
> It will be especially useful for special variables like nl and nr



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (IGNITE-6094) Web console: Implement persistent store in demo mode.

[Pavel Konstantinov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Konstantinov reassigned IGNITE-6094:
--

Assignee: Vasiliy Sisko  (was: Pavel Konstantinov)

> Web console: Implement persistent store in demo mode.
> -
>
> Key: IGNITE-6094
> URL: https://issues.apache.org/jira/browse/IGNITE-6094
> Project: Ignite
>  Issue Type: Bug
>  Components: wizards
>Affects Versions: 2.1
>Reporter: Vasiliy Sisko
>Assignee: Vasiliy Sisko
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (IGNITE-7309) Sutable exception should be return as job results when node is about to stop.

[Roman Guseinov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349856#comment-16349856
 ] 

Roman Guseinov edited comment on IGNITE-7309 at 2/2/18 9:14 AM:


[~amashenkov], could you please review the PR 
[https://github.com/apache/ignite/pull/3461] ?

* Wrapped error log (U.error) at GridJobWorker.onJobFinished to check node 
stopping.

* Added GridJobWorkerTest.

Tests results: 
[https://ci.ignite.apache.org/viewLog.html?buildId=1071207=queuedBuildOverviewTab]

Thanks. 


was (Author: guseinov):
[~amashenkov], could you please review the PR 
[https://github.com/apache/ignite/pull/3461] ?

* Wrapped error log (U.error) at GridJobWorker.onJobFinished to check node 
stopping.
* Added GridJobWorkerTest.

Tests results: 
[https://ci.ignite.apache.org/viewLog.html?buildId=1071207=queuedBuildOverviewTab]

Thanks. 

> Sutable exception should be return as job results when node is about to stop.
> -
>
> Key: IGNITE-7309
> URL: https://issues.apache.org/jira/browse/IGNITE-7309
> Project: Ignite
>  Issue Type: Bug
>  Components: compute, general
>Reporter: Andrew Mashenkov
>Assignee: Roman Guseinov
>Priority: Minor
>  Labels: newbie, pull-request-available
>
> User job can fails with confusing exception when server node is stopping and 
> going to leave the grid. 
> We should suppress InterruptedException. If node is stopping then user should 
> see NodeStoppingException.
> {code:java}
> [org.apache.ignite.internal.processors.job.GridJobWorker] Failed to serialize 
> job response [nodeId=02e1588
> c-53eb-454a-99a1-48ac6cb33667, ses=GridJobSessionImpl 
> [ses=GridTaskSessionImpl 
> ..
> org.apache.ignite.IgniteCheckedException: Failed to register class.
>   at 
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:9929)
>   at 
> org.apache.ignite.internal.processors.job.GridJobWorker.finishJob(GridJobWorker.java:819)
>   at 
> org.apache.ignite.internal.processors.job.GridJobWorker.finishJob(GridJobWorker.java:760)
>   at 
> org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:619)
>   at 
> org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:483)
>   at 
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
>   at 
> org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1180)
>   at 
> org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1899)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1519)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1147)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:119)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1087)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to register 
> class.
>   at 
> org.apache.ignite.internal.binary.BinaryContext.registerUserClassDescriptor(BinaryContext.java:778)
>   at 
> org.apache.ignite.internal.binary.BinaryContext.registerClassDescriptor(BinaryContext.java:751)
>   at 
> org.apache.ignite.internal.binary.BinaryContext.descriptorForClass(BinaryContext.java:621)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:164)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
>   at 
> org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:239)
>   at 
> org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
>   at 
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
>   at 
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:9923)
>   ... 14 common frames omitted
> Caused by: org.apache.ignite.internal.IgniteInterruptedCheckedException: null
>   at 
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
>   at 
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:119)
>   at 
> 

[jira] [Commented] (IGNITE-7192) JDBC: support FQDN to multiple IPs during connection establishment

[Roman Guseinov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349997#comment-16349997
 ] 

Roman Guseinov commented on IGNITE-7192:


[~skalashnikov], Thanks for your comments. Added one more commit to fix style 
violations. Please take a look.

> JDBC: support FQDN to multiple IPs during connection establishment
> --
>
> Key: IGNITE-7192
> URL: https://issues.apache.org/jira/browse/IGNITE-7192
> Project: Ignite
>  Issue Type: Improvement
>  Components: jdbc
>Affects Versions: 2.1
>Reporter: Alexey Popov
>Assignee: Roman Guseinov
>Priority: Major
>  Labels: pull-request-available
>
> Thin JDBC driver may have FQDN (host name) at a connection string.
> Currently, it resolves this FQDN to one IP and tries to connect to this IP 
> only.
> It is better to try to connect to multiple IPs one-by-one if DNS returns 
> multiple A-records (FQDN can be resolved to several IPs) until successful 
> connection. It could give a simple fallback option for the JDBC thin driver 
> users.
> A similar functionality is already implemented in ODBC driver.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7580) compatibilityMode flag consistency

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349961#comment-16349961
 ] 

ASF GitHub Bot commented on IGNITE-7580:


Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3466


> compatibilityMode flag consistency
> --
>
> Key: IGNITE-7580
> URL: https://issues.apache.org/jira/browse/IGNITE-7580
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sergey Chugunov
>Assignee: Sergey Chugunov
>Priority: Major
> Fix For: 2.5
>
>
> Flag *compatibilityMode* may be handled incorrectly in mixed clusters (nodes 
> w and w/o BLT support): in case when node of new version becomes coordinator 
> flag may be set incorrectly on other nodes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7580) compatibilityMode flag consistency

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349956#comment-16349956
 ] 

Pavel Tupitsyn commented on IGNITE-7580:


Looks good to me, merged to master: 
{{8f2045e364a4dab71cb99138d99e90b80bfc2de5}}.

> compatibilityMode flag consistency
> --
>
> Key: IGNITE-7580
> URL: https://issues.apache.org/jira/browse/IGNITE-7580
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sergey Chugunov
>Assignee: Sergey Chugunov
>Priority: Major
> Fix For: 2.5
>
>
> Flag *compatibilityMode* may be handled incorrectly in mixed clusters (nodes 
> w and w/o BLT support): in case when node of new version becomes coordinator 
> flag may be set incorrectly on other nodes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7580) compatibilityMode flag consistency

[Dmitriy Govorukhin (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349951#comment-16349951
 ] 

Dmitriy Govorukhin commented on IGNITE-7580:


[~ptupitsyn] please have a look.

> compatibilityMode flag consistency
> --
>
> Key: IGNITE-7580
> URL: https://issues.apache.org/jira/browse/IGNITE-7580
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sergey Chugunov
>Assignee: Sergey Chugunov
>Priority: Major
> Fix For: 2.5
>
>
> Flag *compatibilityMode* may be handled incorrectly in mixed clusters (nodes 
> w and w/o BLT support): in case when node of new version becomes coordinator 
> flag may be set incorrectly on other nodes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7580) compatibilityMode flag consistency

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349950#comment-16349950
 ] 

ASF GitHub Bot commented on IGNITE-7580:


GitHub user DmitriyGovorukhin opened a pull request:

https://github.com/apache/ignite/pull/3466

IGNITE-7580



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7580

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3466.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3466






> compatibilityMode flag consistency
> --
>
> Key: IGNITE-7580
> URL: https://issues.apache.org/jira/browse/IGNITE-7580
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sergey Chugunov
>Assignee: Sergey Chugunov
>Priority: Major
> Fix For: 2.5
>
>
> Flag *compatibilityMode* may be handled incorrectly in mixed clusters (nodes 
> w and w/o BLT support): in case when node of new version becomes coordinator 
> flag may be set incorrectly on other nodes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)