[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770449#comment-16770449
]
Vladimir Ozerov commented on IGNITE-9171:
-
Failures are unrelated. Merged.
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Critical
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770443#comment-16770443
]
Ignite TC Bot commented on IGNITE-9171:
---
{panel:title=-- Run :: All: Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}ZooKeeper (Discovery) 1{color} [[tests 0 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=3106920]]
{color:#d04437}RDD{color} [[tests 0 Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=3106907]]
* JavaEmbeddedIgniteRDDSelfTest.testReadDataFromIgnite (last started)
{color:#d04437}Cache 1{color} [[tests
3|https://ci.ignite.apache.org/viewLog.html?buildId=3106946]]
* IgniteBinaryCacheTestSuite:
DataStreamProcessorPersistenceBinarySelfTest.testTryFlush - 0,0% fails in last
434 master runs.
{color:#d04437}Cache (Full API){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=3106942]]
* IgniteCacheFullApiSelfTestSuite:
GridCacheReplicatedAtomicFullApiSelfTest.testTtlNoTxOldEntry - 0,0% fails in
last 591 master runs.
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=3106998buildTypeId=IgniteTests24Java8_RunAll]
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Critical
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770321#comment-16770321
]
Vladimir Ozerov commented on IGNITE-9171:
-
New test reun after merge w/ master:
https://ci.ignite.apache.org/viewQueued.html?itemId=3106998
Problems to be addressed:
# Remove default value from {{SqlFieldsQuery}}
# Ticket for OOME tests integration on TC
# Ticket for lazy suite on TC
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Critical
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762710#comment-16762710
]
Ignite TC Bot commented on IGNITE-9171:
---
{panel:title=-- Run :: All: No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=3025742buildTypeId=IgniteTests24Java8_RunAll]
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Critical
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16761708#comment-16761708
]
Vladimir Ozerov commented on IGNITE-9171:
-
[~tledkov-gridgain]. please see my comments in PR. Some cleanup and
clarifications on lock/unlock semantics are needed. Looks good otherwise.
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Critical
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760962#comment-16760962
]
Taras Ledkov commented on IGNITE-9171:
--
[~vozerov], please take a look.
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Critical
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760961#comment-16760961
]
Ignite TC Bot commented on IGNITE-9171:
---
{panel:title=-- Run :: All: No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=3004928buildTypeId=IgniteTests24Java8_RunAll]
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Critical
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758070#comment-16758070
]
Vladimir Ozerov commented on IGNITE-9171:
-
[~tledkov-gridgain], I propose to decouple lock for cancel command
synchronization from table locks. If we do not do this, further development of
{{KILL}} command will be more difficult due to subtle dependencies.
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Critical
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754986#comment-16754986
]
Ignite TC Bot commented on IGNITE-9171:
---
{panel:title=-- Run :: All: No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=2893225buildTypeId=IgniteTests24Java8_RunAll]
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Critical
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750969#comment-16750969
]
Taras Ledkov commented on IGNITE-9171:
--
[~vozerov], conflicts are resolved. TC tests are restarted.
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Critical
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750102#comment-16750102
]
Vladimir Ozerov commented on IGNITE-9171:
-
[~tledkov-gridgain], could you please re-merge with master as there are several
conflicts?
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742901#comment-16742901
]
Ignite TC Bot commented on IGNITE-9171:
---
{panel:title=-- Run :: All: No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=2797533buildTypeId=IgniteTests24Java8_RunAll]
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728745#comment-16728745
]
Taras Ledkov commented on IGNITE-9171:
--
[~vozerov],
1. Connection manager changes was corresponded to previous lazy impl: with lazy
threads. Your suggestion make sense.
But current connection manager has *potential connection leak*:
detached connection is removed from the map {{threadConns : Thread->
H2ConnectionWrapper}} and there is no way to close all connection on the node
stop.
2. Rolls back changes at the connection manager highlight the problem with
detached connection on reducer. Looks like it was cause of the error. I'll
check it on TC,
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728729#comment-16728729
]
Vladimir Ozerov commented on IGNITE-9171:
-
[~tledkov-gridgain], I have two comments at the moment:
# Can we simplify or extract connection management into a separate ticket with
tests showing why the change is needed? At the moment, as we decided to remove
lazy threads, it is not obvious why all these changes are needed
#
{{IgniteQueryTableLockAndConnectionPoolLazyModeOnTest.testSingleNodeWithParallelismTablesLockQueryAndDDLMultithreaded}}
fails with {{ConcurrentModificationException}}. It seems that we cannot rely
on H2's {{Session.getLocks}} as it is not thread safe, are we?
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728282#comment-16728282
]
Taras Ledkov commented on IGNITE-9171:
--
The default lazy behavior is rolled back due to [H2 performance
issues|https://github.com/h2database/h2database/issues/1057]
[~vozerov], please take a look.
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728280#comment-16728280
]
Ignite TC Bot commented on IGNITE-9171:
---
{panel:title=-- Run :: All: No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=2613737buildTypeId=IgniteTests24Java8_RunAll]
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721548#comment-16721548
]
Ignite TC Bot commented on IGNITE-9171:
---
{panel:title=-- Run :: All: Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}[Inspections] Core{color} [[tests 0 BuildFailureOnMetric
|https://ci.ignite.apache.org/viewLog.html?buildId=2546537]]
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=2546542buildTypeId=IgniteTests24Java8_RunAll]
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719890#comment-16719890
]
Ivan Pavlukhin commented on IGNITE-9171:
[~tledkov-gridgain] please see my comment on GitHub.
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[ https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695790#comment-16695790 ] ASF GitHub Bot commented on IGNITE-9171: GitHub user tledkov-gridgain opened a pull request: https://github.com/apache/ignite/pull/5473 IGNITE-9171 Use lazy mode for default You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-9171 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5473.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5473 commit 044e1e51e6609e92ff46b0650ec8d69afb2b424d Author: tledkov-gridgain Date: 2018-10-23T09:10:02Z IGNITE-9171: save the progress commit 05ee7dba340395208ab040c7e13710433e44fe1e Author: tledkov-gridgain Date: 2018-10-24T13:22:02Z IGNITE-9171: lazy mode performance optimization for one-page results commit e2ffb4a9cc83242e3e6c4283319582c1933eb6b2 Author: tledkov-gridgain Date: 2018-10-24T14:25:58Z Merge branch '_master' into ignite-9171 commit 7ddcacbfcf484ed359fd1f9e2cbcc6aff414249d Author: tledkov-gridgain Date: 2018-10-25T12:01:25Z IGNITE-9171: lazy mode performance optimization commit 70be3700b47a1e367f365b049e7872f4ab84efa8 Author: tledkov-gridgain Date: 2018-10-25T12:06:49Z Merge master into ignite-9171 commit e2832dcb363aa74579175e957d7a3dc88b8263e1 Author: tledkov-gridgain Date: 2018-10-25T15:23:33Z IGNITE-9171: revert debug commit 85efbcd748b7e322a9d9bd8681c10bfa69853d2d Author: tledkov-gridgain Date: 2018-10-26T10:33:14Z IGNITE-9171: fix lazy query cancel after performance fix commit 1777fd393644e4549391afa13fee0d2a322438ef Author: tledkov-gridgain Date: 2018-10-26T12:59:02Z IGNITE-9171: fix lazy query cancel on map phase after performance fix commit 08dec581cbe81452ce8d11dd3fcaf7b54b7318a5 Author: tledkov-gridgain Date: 2018-10-29T08:50:17Z Merge branch '_master' into ignite-9171 commit f330268e70a12930e9f8e28b0966191719d1b39c Author: tledkov-gridgain Date: 2018-10-30T11:52:09Z IGNITE-9171: add OOM test commit 392e1e09550c7f60c25b54cd077e61d58d407dad Author: tledkov-gridgain Date: 2018-10-31T11:46:35Z IGNITE-9171: add OOM tests and suite commit 4be3e9d96a3e82615ef15860b5b298eaf3d5f0ff Author: tledkov-gridgain Date: 2018-10-31T12:04:43Z Merge master into ignite-9171 commit 877204f0beac1a0ab0b8553b449fd3ed03cccb57 Author: tledkov-gridgain Date: 2018-10-31T12:24:32Z IGNITE-9171: fix tests suite after merge commit ece54bb35afd85b9195811657ffadef42ef6e5d5 Author: tledkov-gridgain Date: 2018-10-31T12:28:28Z IGNITE-9171: add lazy paged SQL test to thin client commit 06dd2d33db7a4dbe8029a9e3206d17ab47d9c966 Author: tledkov Date: 2018-11-01T10:00:16Z IGNITE-9171: enhance lazy paged SQL test to thin client commit 7c46ae6f9f2134f4beee4fec65de10e201ef7da5 Author: tledkov Date: 2018-11-01T13:18:01Z IGNITE-9171: fix disabled lazy mode, add tests commit 275f205f4140d939403533291569cf79b3c9cc3f Author: tledkov Date: 2018-11-01T13:24:43Z IGNITE-9171: switch default lazy mode to true for php and nodejs thin clients commit be469480ca698699c2e54019ca19885a8c27619e Author: tledkov Date: 2018-11-01T13:28:54Z IGNITE-9171: add lazy=false tests for java thin client commit fbc2ee7516aa1c9e75837628565f2b166ef2de5b Author: tledkov Date: 2018-11-01T13:32:46Z IGNITE-9171: switch default lazy mode to true for python thin clients commit 4b0c74c4c642d835429308a20183a39539cebad7 Author: tledkov Date: 2018-11-01T14:05:51Z Merge branch '_master' into ignite-9171 commit 366f0582d5d8aa7e7d05b60162cd2da8b89f3db3 Author: tledkov Date: 2018-11-02T09:58:09Z IGNITE-9171: add collocated group by to OOM test commit c8d9cc0faba28437a37c0b7cb6c27685b64ab0b7 Author: tledkov Date: 2018-11-02T10:11:55Z IGNITE-9171: add collocated group by to OOM test commit 708820a51ea11c8a0642f7303a2a2dec9f957389 Author: tledkov Date: 2018-11-02T10:27:50Z Merge master into ignite-9171 commit a78264a783370937e42675366701e0c5322b396f Author: tledkov Date: 2018-11-02T12:46:04Z IGNITE-9171: save the progress commit 4d0a1eb89f06473796578de5695564a7ec525da5 Author: tledkov Date: 2018-11-02T12:55:44Z Merge branch '_master' into ignite-9171 commit 1e7742867c2c95fe8aca39c17d77c48f9e608972 Author: tledkov Date: 2018-11-02T14:20:20Z IGNITE-9171: graceful shutdown on restart commit c879d6c25add7e53ed7640091703829a9843861b Author: tledkov Date: 2018-11-02T15:56:14Z IGNITE-9171: add test suites for lazy = false commit b5b10d4baed4da40356c83f9f4b6c672f34d28d0 Author: tledkov Date: 2018-11-06T08:54:38Z Merge branch '_master' into
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695782#comment-16695782
]
ASF GitHub Bot commented on IGNITE-9171:
Github user tledkov-gridgain closed the pull request at:
https://github.com/apache/ignite/pull/5076
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692880#comment-16692880
]
Vladimir Ozerov commented on IGNITE-9171:
-
Cancel,
As discussed, it seems that we need an implementation which will not use
separate threads. In order to achieve this we need to introduce special
treatment for DDL commands - they should cancel running "lazy" queries in case
of conflicts. This is not very good from usability perspective, but provided
that DDL operations are rare, we should tolerate this.
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681373#comment-16681373
]
Ignite TC Bot commented on IGNITE-9171:
---
{panel:title=No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity Run All
Results|http://ci.ignite.apache.org/viewLog.html?buildId=2264662buildTypeId=IgniteTests24Java8_RunAll]
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681375#comment-16681375
]
Taras Ledkov commented on IGNITE-9171:
--
[~vozerov], please review the patch.
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681182#comment-16681182
]
Ignite TC Bot commented on IGNITE-9171:
---
{panel:title=Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Queries (Binary Objects Simple Mapper){color} [[tests
3|https://ci.ignite.apache.org/viewLog.html?buildId=2264654]]
* IgniteBinarySimpleNameMapperCacheQueryTestSuite:
IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheClient -
0,0% fails in last 100 master runs.
{panel}
[TeamCity Run All
Results|http://ci.ignite.apache.org/viewLog.html?buildId=2264662buildTypeId=IgniteTests24Java8_RunAll]
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676859#comment-16676859
]
Ignite TC Bot commented on IGNITE-9171:
---
{panel:title=Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Cache 6{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=2254335]]
* IgniteCacheTestSuite6: TxRollbackOnTimeoutNearCacheTest.testSimple - 0,0%
fails in last 100 master runs.
{panel}
[TeamCity Run All
Results|http://ci.ignite.apache.org/viewLog.html?buildId=2254367buildTypeId=IgniteTests24Java8_RunAll]
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676462#comment-16676462
]
Taras Ledkov commented on IGNITE-9171:
--
Add two suites {{Query 1 (lazy=false)}} and {{Query 2 (lazy=false)}} to check
the queries behavior with disabled lazy.
The suites are not added to {{Run::All}} build configuration .
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16663653#comment-16663653
]
ASF GitHub Bot commented on IGNITE-9171:
GitHub user tledkov-gridgain opened a pull request:
https://github.com/apache/ignite/pull/5076
IGNITE-9171 Use lazy mode with results pre-fetch
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gridgain/apache-ignite ignite-9171
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/5076.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5076
commit 044e1e51e6609e92ff46b0650ec8d69afb2b424d
Author: tledkov-gridgain
Date: 2018-10-23T09:10:02Z
IGNITE-9171: save the progress
commit 05ee7dba340395208ab040c7e13710433e44fe1e
Author: tledkov-gridgain
Date: 2018-10-24T13:22:02Z
IGNITE-9171: lazy mode performance optimization for one-page results
commit e2ffb4a9cc83242e3e6c4283319582c1933eb6b2
Author: tledkov-gridgain
Date: 2018-10-24T14:25:58Z
Merge branch '_master' into ignite-9171
commit 7ddcacbfcf484ed359fd1f9e2cbcc6aff414249d
Author: tledkov-gridgain
Date: 2018-10-25T12:01:25Z
IGNITE-9171: lazy mode performance optimization
commit 70be3700b47a1e367f365b049e7872f4ab84efa8
Author: tledkov-gridgain
Date: 2018-10-25T12:06:49Z
Merge master into ignite-9171
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660213#comment-16660213
]
Vladimir Ozerov commented on IGNITE-9171:
-
Reverted by IGNITE-9960 due to performance drop. Need to re-implement. Commit
75e414a4 [1].
[1]
https://github.com/apache/ignite/commit/75e414a4ac53a36c421b31633723844faeb499af
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646313#comment-16646313
]
ASF GitHub Bot commented on IGNITE-9171:
Github user asfgit closed the pull request at:
https://github.com/apache/ignite/pull/4538
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.7
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646219#comment-16646219
]
Vladimir Ozerov commented on IGNITE-9171:
-
Code is still not ready. Moving to AI 2.8.
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.8
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637976#comment-16637976
]
Taras Ledkov commented on IGNITE-9171:
--
Documentation ticket: IGNITE-9791
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.7
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16635314#comment-16635314
]
Taras Ledkov commented on IGNITE-9171:
--
[~vozerov], I fixed lazy worker start/stop synchronization & result set close.
Tests are OK. Please take a look.
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.7
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631952#comment-16631952
]
Vladimir Ozerov commented on IGNITE-9171:
-
[~tledkov-gridgain], I reviewed the patch. It looks good in general. But I
found several suspicious places, where concurrency may be broken. Could you
please take care of these places (mentioned in PR)?
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Labels: sql-stability
> Fix For: 2.7
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631411#comment-16631411
]
Vladimir Ozerov commented on IGNITE-9171:
-
New test run:
https://ci.ignite.apache.org/viewQueued.html?itemId=1959052=queuedBuildOverviewTab
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Labels: sql-stability
> Fix For: 2.7
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623590#comment-16623590
]
Nikolay Izhikov commented on IGNITE-9171:
-
[~tledkov-gridgain] Do we have a chance to resolve this ticket until the code
freeze of 2.7?
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Labels: sql-stability
> Fix For: 2.7
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603198#comment-16603198
]
Taras Ledkov commented on IGNITE-9171:
--
[~vozerov], please take a look at the benchmark results.
Benchmark configuration:
- 4 servers, 8 clients.
- 1 backups;
Benchmark names agenda:
- rN - results set size (one row, 1K and 2k rows, 1M rows)
- lazy - lazy mode on.
Operations per second:
|||| master || ignite-9171 ||
| r1| 154,550.61 | 153,534.05|
| r1K | 6,097.21 | 6,027.85 |
| r2K | 3,523.90 | 3,478.26 |
| r1M | 16.51| 16.45 |
| r1-lazy | 24,474.02| 146,916.20|
| r1K-lazy | 2,014.16 | 6,043.35 |
| r2K-lazy | 1,839.11 | 3,503.99 |
| r1M-lazy | 21.09| 21.00 |
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Labels: sql-stability
> Fix For: 2.7
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16587429#comment-16587429
]
Igor Sapego commented on IGNITE-9171:
-
[~tledkov-gridgain], implemented for C++ also.
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Labels: sql-stability
> Fix For: 2.7
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16587180#comment-16587180
]
Taras Ledkov commented on IGNITE-9171:
--
[~isapego], sure! Thanks for remind.
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Labels: sql-stability
> Fix For: 2.7
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586113#comment-16586113
]
Igor Sapego commented on IGNITE-9171:
-
[~tledkov-gridgain], done. Do we need to do the same for C++ and .NET clients?
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Labels: sql-stability
> Fix For: 2.7
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583946#comment-16583946
]
Taras Ledkov commented on IGNITE-9171:
--
[~isapego], could you please change default value for lazy flag in ODBC diver
to {{true}}.
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Labels: sql-stability
> Fix For: 2.7
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579735#comment-16579735
]
ASF GitHub Bot commented on IGNITE-9171:
GitHub user tledkov-gridgain opened a pull request:
https://github.com/apache/ignite/pull/4538
IGNITE-9171 Use lazy mode with results pre-fetch
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gridgain/apache-ignite ignite-9171
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/4538.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4538
commit 99b5e863039f5db7c8c8cbae94ef571653e4bdbb
Author: tledkov-gridgain
Date: 2018-08-14T12:32:01Z
IGNITE-9171: save the progress
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Labels: sql-stability
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16578051#comment-16578051
]
ASF GitHub Bot commented on IGNITE-9171:
Github user tledkov-gridgain closed the pull request at:
https://github.com/apache/ignite/pull/4514
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576212#comment-16576212
]
ASF GitHub Bot commented on IGNITE-9171:
GitHub user tledkov-gridgain opened a pull request:
https://github.com/apache/ignite/pull/4514
IGNITE-9171 Use lazy mode with results pre-fetch
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gridgain/apache-ignite ignite-9171
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/4514.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4514
commit de324a59df6dcba2ec906fb974baf95dabd19504
Author: tledkov-gridgain
Date: 2018-08-03T11:37:26Z
IGNITE-9171: save the progress
commit 63181f02cc950c49f93101e79682939949396675
Author: tledkov-gridgain
Date: 2018-08-03T13:51:22Z
IGNITE-9171: save the progress
commit 8fec73450804fd1f9080dce7d000c8eefb0c6749
Author: tledkov-gridgain
Date: 2018-08-03T14:35:23Z
Merge branch '_master' into ignite-9171
commit 260f3bf244fac0031fa4bb8d27aac365acfd43db
Author: tledkov-gridgain
Date: 2018-08-06T09:22:39Z
IGNITE-9171: save the progress
commit f9bfdf76a791c0fa588d1a3a2a91bb349d7affd6
Author: tledkov-gridgain
Date: 2018-08-06T09:35:42Z
Merge branch '_master' into ignite-9171
commit 59bf5ee665c460179aa961f9a3939b892f916dbc
Author: tledkov-gridgain
Date: 2018-08-06T11:14:47Z
IGNITE-9171: save the progress
commit 83cca801e7547b032e7f3436ef22c979e72e04f0
Author: tledkov-gridgain
Date: 2018-08-06T12:43:00Z
Merge branch '_master' into ignite-9171
commit bfb342cd0e35aad8c1c79044e0ebcde936c71806
Author: tledkov-gridgain
Date: 2018-08-06T13:41:32Z
IGNITE-9171: save the progress
commit fe64dc2b22cf2f8e6b5d56068286dda1e6cc77fd
Author: tledkov-gridgain
Date: 2018-08-07T12:30:24Z
IGNITE-9171: remove lazy worker
commit 98e4c57b795bc89c5b4a1c27f7f06f2cdbfc4dd4
Author: tledkov-gridgain
Date: 2018-08-07T12:36:27Z
Merge branch '_master' into ignite-9171
commit 3ace9838ce13e3a08e5fdbe88101d2b61c40c718
Author: tledkov-gridgain
Date: 2018-08-08T11:10:51Z
IGNITE-9171: benchmark
commit a3ee0f5f70d452f43de84a61832866ecd02e92da
Author: tledkov-gridgain
Date: 2018-08-08T11:16:08Z
Merge branch '_master' into ignite-9171
commit b9a2ecfc6ac9f6c324ff0ab832899bb99ae46473
Author: tledkov-gridgain
Date: 2018-08-08T13:13:07Z
IGNITE-9171: fix lazy mode
commit de71737b20e1683ff9d2078f1ba33a5843ccc541
Author: tledkov-gridgain
Date: 2018-08-09T09:15:54Z
IGNITE-9171: save the progress
commit 51c15c81496c9b5dd8beebdf66087ea61a3324d9
Author: tledkov-gridgain
Date: 2018-08-09T12:42:19Z
IGNITE-9171: save the progress
commit 75dbcc9872388b29758b56593fdc04d497d8d0ed
Author: tledkov-gridgain
Date: 2018-08-10T08:04:25Z
IGNITE-9171: modify table lock
commit 18e198044d1806e2951b249977230d0dfaed7053
Author: tledkov-gridgain
Date: 2018-08-10T08:14:28Z
IGNITE-9171: modify table lock - minors
commit f2188d118e9028a783bb2347d7ed15f7664828c0
Author: tledkov-gridgain
Date: 2018-08-10T08:53:17Z
Merge branch '_master' into ignite-9171
commit 57e77782ce24d4f36843e89d9e81ab5d39eef4c1
Author: tledkov-gridgain
Date: 2018-08-10T10:38:11Z
IGNITE-9171: minors
commit 79b2292733b3fb510ce38053b44512e25f143fa6
Author: tledkov-gridgain
Date: 2018-08-10T12:35:32Z
Merge branch '_master' into ignite-9171
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
