[jira] [Comment Edited] (IGNITE-7531) SQL: Create data load benchmarks
[
https://issues.apache.org/jira/browse/IGNITE-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366940#comment-16366940
]
Pavel Kuznetsov edited comment on IGNITE-7531 at 2/16/18 7:20 PM:
--
10M rows 3 servers, 1 client, 1 jdbc application:
batched 1000 : 194 sec
copy csv : 174,5 sec
turned on persistence
batched 1000 : 674 sec
copy csv: 169 sec
was (Author: pkouznet):
10M rows 3 servers, 1 client, 1 jdbc application:
batched 1000 : 194 sec
copy csv : 174,5 sec
turned on persistence
batched 1000 : 674 sec
> SQL: Create data load benchmarks
>
>
> Key: IGNITE-7531
> URL: https://issues.apache.org/jira/browse/IGNITE-7531
> Project: Ignite
> Issue Type: Task
> Components: sql, yardstick
>Reporter: Vladimir Ozerov
>Assignee: Pavel Kuznetsov
>Priority: Major
> Fix For: 2.5
>
>
> We need to implement a set of data loading benchmarks to better understand
> how fast Ignite is able to consume data. This task consists of two steps:
> 1) Extend Yardstick capabilities
> 2) Create set of benchmarks
> 1) Yardstick
> Data load benchmark should be executed in single-shot mode: only one
> iteration, only total execution time is needed, start callback for setup and
> warmup, stop callback for cleanup.
> Currently Yardstick cannot do that, so we need to extend it. Possibly, we can
> control this through new {{boolean BenchmarkDriver.isSingleShot()}} method.
> 2) Benchmarks
> At first let's focus on thin JDBC driver. The following cases should be
> executed:
> 2.1) Normal INSERT
> 2.2) Batched INSERT
> 2.3) Streaming INSERT (when IGNITE-7253 is ready)
> 2.4) P. 1-3 with and without dynamically disabled WAL (ALTER TABLE ...
> NOLOGGING)
> 2.5) P. 1-3 with additional indexes - either created before data load on
> empty table, or after load on table with data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-7531) SQL: Create data load benchmarks
[
https://issues.apache.org/jira/browse/IGNITE-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366940#comment-16366940
]
Pavel Kuznetsov edited comment on IGNITE-7531 at 2/16/18 7:10 PM:
--
10M rows 3 servers, 1 client, 1 jdbc application:
batched 1000 : 194 sec
copy csv : 174,5 sec
turned on persistence
batched 1000 : 674 sec
was (Author: pkouznet):
10M rows 3 servers, 1 client, 1 jdbc application:
batched 1000 : 194 sec
copy csv : 174,5 sec
> SQL: Create data load benchmarks
>
>
> Key: IGNITE-7531
> URL: https://issues.apache.org/jira/browse/IGNITE-7531
> Project: Ignite
> Issue Type: Task
> Components: sql, yardstick
>Reporter: Vladimir Ozerov
>Assignee: Pavel Kuznetsov
>Priority: Major
> Fix For: 2.5
>
>
> We need to implement a set of data loading benchmarks to better understand
> how fast Ignite is able to consume data. This task consists of two steps:
> 1) Extend Yardstick capabilities
> 2) Create set of benchmarks
> 1) Yardstick
> Data load benchmark should be executed in single-shot mode: only one
> iteration, only total execution time is needed, start callback for setup and
> warmup, stop callback for cleanup.
> Currently Yardstick cannot do that, so we need to extend it. Possibly, we can
> control this through new {{boolean BenchmarkDriver.isSingleShot()}} method.
> 2) Benchmarks
> At first let's focus on thin JDBC driver. The following cases should be
> executed:
> 2.1) Normal INSERT
> 2.2) Batched INSERT
> 2.3) Streaming INSERT (when IGNITE-7253 is ready)
> 2.4) P. 1-3 with and without dynamically disabled WAL (ALTER TABLE ...
> NOLOGGING)
> 2.5) P. 1-3 with additional indexes - either created before data load on
> empty table, or after load on table with data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-7531) SQL: Create data load benchmarks
[
https://issues.apache.org/jira/browse/IGNITE-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366940#comment-16366940
]
Pavel Kuznetsov edited comment on IGNITE-7531 at 2/16/18 12:52 PM:
---
10M rows 3 servers, 1 client, 1 jdbc application:
batched 1000 : 194 sec
copy csv : 174,5 sec
was (Author: pkouznet):
10M rows 3 servers 1 client 1 jdbc application:
batched 1000 : 194 sec
copy csv : 174,5 sec
> SQL: Create data load benchmarks
>
>
> Key: IGNITE-7531
> URL: https://issues.apache.org/jira/browse/IGNITE-7531
> Project: Ignite
> Issue Type: Task
> Components: sql, yardstick
>Reporter: Vladimir Ozerov
>Assignee: Pavel Kuznetsov
>Priority: Major
> Fix For: 2.5
>
>
> We need to implement a set of data loading benchmarks to better understand
> how fast Ignite is able to consume data. This task consists of two steps:
> 1) Extend Yardstick capabilities
> 2) Create set of benchmarks
> 1) Yardstick
> Data load benchmark should be executed in single-shot mode: only one
> iteration, only total execution time is needed, start callback for setup and
> warmup, stop callback for cleanup.
> Currently Yardstick cannot do that, so we need to extend it. Possibly, we can
> control this through new {{boolean BenchmarkDriver.isSingleShot()}} method.
> 2) Benchmarks
> At first let's focus on thin JDBC driver. The following cases should be
> executed:
> 2.1) Normal INSERT
> 2.2) Batched INSERT
> 2.3) Streaming INSERT (when IGNITE-7253 is ready)
> 2.4) P. 1-3 with and without dynamically disabled WAL (ALTER TABLE ...
> NOLOGGING)
> 2.5) P. 1-3 with additional indexes - either created before data load on
> empty table, or after load on table with data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-7531) SQL: Create data load benchmarks
[
https://issues.apache.org/jira/browse/IGNITE-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366006#comment-16366006
]
Pavel Kuznetsov edited comment on IGNITE-7531 at 2/15/18 6:37 PM:
--
Updated results on another machines 10M rows
single insert : 2 870 sec
batched 1000 : 260 sec
copy csv : 278 sec
was (Author: pkouznet):
Updated results on another machines 10M rows
single insert : 2870 sec
batched 1000 : 260 sec
copy csv : 278 sec
> SQL: Create data load benchmarks
>
>
> Key: IGNITE-7531
> URL: https://issues.apache.org/jira/browse/IGNITE-7531
> Project: Ignite
> Issue Type: Task
> Components: sql, yardstick
>Reporter: Vladimir Ozerov
>Assignee: Pavel Kuznetsov
>Priority: Major
> Fix For: 2.5
>
>
> We need to implement a set of data loading benchmarks to better understand
> how fast Ignite is able to consume data. This task consists of two steps:
> 1) Extend Yardstick capabilities
> 2) Create set of benchmarks
> 1) Yardstick
> Data load benchmark should be executed in single-shot mode: only one
> iteration, only total execution time is needed, start callback for setup and
> warmup, stop callback for cleanup.
> Currently Yardstick cannot do that, so we need to extend it. Possibly, we can
> control this through new {{boolean BenchmarkDriver.isSingleShot()}} method.
> 2) Benchmarks
> At first let's focus on thin JDBC driver. The following cases should be
> executed:
> 2.1) Normal INSERT
> 2.2) Batched INSERT
> 2.3) Streaming INSERT (when IGNITE-7253 is ready)
> 2.4) P. 1-3 with and without dynamically disabled WAL (ALTER TABLE ...
> NOLOGGING)
> 2.5) P. 1-3 with additional indexes - either created before data load on
> empty table, or after load on table with data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-7531) SQL: Create data load benchmarks
[
https://issues.apache.org/jira/browse/IGNITE-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366006#comment-16366006
]
Pavel Kuznetsov edited comment on IGNITE-7531 at 2/15/18 6:37 PM:
--
Updated results on another machines 10M rows
single insert : 2870 sec
batched 1000 : 260 sec
copy csv : 278 sec
was (Author: pkouznet):
Updated results on another machines 10M rows
single insert : 2 870 sec
batched 1000 : 260 sec
copy csv : 278 sec
> SQL: Create data load benchmarks
>
>
> Key: IGNITE-7531
> URL: https://issues.apache.org/jira/browse/IGNITE-7531
> Project: Ignite
> Issue Type: Task
> Components: sql, yardstick
>Reporter: Vladimir Ozerov
>Assignee: Pavel Kuznetsov
>Priority: Major
> Fix For: 2.5
>
>
> We need to implement a set of data loading benchmarks to better understand
> how fast Ignite is able to consume data. This task consists of two steps:
> 1) Extend Yardstick capabilities
> 2) Create set of benchmarks
> 1) Yardstick
> Data load benchmark should be executed in single-shot mode: only one
> iteration, only total execution time is needed, start callback for setup and
> warmup, stop callback for cleanup.
> Currently Yardstick cannot do that, so we need to extend it. Possibly, we can
> control this through new {{boolean BenchmarkDriver.isSingleShot()}} method.
> 2) Benchmarks
> At first let's focus on thin JDBC driver. The following cases should be
> executed:
> 2.1) Normal INSERT
> 2.2) Batched INSERT
> 2.3) Streaming INSERT (when IGNITE-7253 is ready)
> 2.4) P. 1-3 with and without dynamically disabled WAL (ALTER TABLE ...
> NOLOGGING)
> 2.5) P. 1-3 with additional indexes - either created before data load on
> empty table, or after load on table with data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-7531) SQL: Create data load benchmarks
[
https://issues.apache.org/jira/browse/IGNITE-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366006#comment-16366006
]
Pavel Kuznetsov edited comment on IGNITE-7531 at 2/15/18 6:36 PM:
--
Updated results on another machines 10M rows
single insert : 2870 sec
batched 1000 : 260 sec
copy csv : 278 sec
was (Author: pkouznet):
Updated results on another machines 10M rows
batched 1000 : 260 sec
copy csv : 278 sec
> SQL: Create data load benchmarks
>
>
> Key: IGNITE-7531
> URL: https://issues.apache.org/jira/browse/IGNITE-7531
> Project: Ignite
> Issue Type: Task
> Components: sql, yardstick
>Reporter: Vladimir Ozerov
>Assignee: Pavel Kuznetsov
>Priority: Major
> Fix For: 2.5
>
>
> We need to implement a set of data loading benchmarks to better understand
> how fast Ignite is able to consume data. This task consists of two steps:
> 1) Extend Yardstick capabilities
> 2) Create set of benchmarks
> 1) Yardstick
> Data load benchmark should be executed in single-shot mode: only one
> iteration, only total execution time is needed, start callback for setup and
> warmup, stop callback for cleanup.
> Currently Yardstick cannot do that, so we need to extend it. Possibly, we can
> control this through new {{boolean BenchmarkDriver.isSingleShot()}} method.
> 2) Benchmarks
> At first let's focus on thin JDBC driver. The following cases should be
> executed:
> 2.1) Normal INSERT
> 2.2) Batched INSERT
> 2.3) Streaming INSERT (when IGNITE-7253 is ready)
> 2.4) P. 1-3 with and without dynamically disabled WAL (ALTER TABLE ...
> NOLOGGING)
> 2.5) P. 1-3 with additional indexes - either created before data load on
> empty table, or after load on table with data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-7531) SQL: Create data load benchmarks
[
https://issues.apache.org/jira/browse/IGNITE-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353681#comment-16353681
]
Pavel Kuznetsov edited comment on IGNITE-7531 at 2/6/18 2:31 PM:
-
*Extend Yardstick capabilities*
I found next solution:
1) Run benchmarks with {{--warmup 0 --operations 1}} to run {{test()}} method
single time and don't run it during warmup
2) Perform custom warmup (insert fewer amount of data) in {{setUp()}} method
was (Author: pkouznet):
*Extend Yardstick capabilities*
I found next solution:
1) Run benchmarks with {{--warmap 0 --operations 1}} to run {{test()}} method
single time and don't run it during warmup
2) Perform custom warmup (insert fewer amount of data) in {{setUp()}} method
> SQL: Create data load benchmarks
>
>
> Key: IGNITE-7531
> URL: https://issues.apache.org/jira/browse/IGNITE-7531
> Project: Ignite
> Issue Type: Task
> Components: sql, yardstick
>Reporter: Vladimir Ozerov
>Assignee: Pavel Kuznetsov
>Priority: Major
> Fix For: 2.5
>
>
> We need to implement a set of data loading benchmarks to better understand
> how fast Ignite is able to consume data. This task consists of two steps:
> 1) Extend Yardstick capabilities
> 2) Create set of benchmarks
> 1) Yardstick
> Data load benchmark should be executed in single-shot mode: only one
> iteration, only total execution time is needed, start callback for setup and
> warmup, stop callback for cleanup.
> Currently Yardstick cannot do that, so we need to extend it. Possibly, we can
> control this through new {{boolean BenchmarkDriver.isSingleShot()}} method.
> 2) Benchmarks
> At first let's focus on thin JDBC driver. The following cases should be
> executed:
> 2.1) Normal INSERT
> 2.2) Batched INSERT
> 2.3) Streaming INSERT (when IGNITE-7253 is ready)
> 2.4) P. 1-3 with and without dynamically disabled WAL (ALTER TABLE ...
> NOLOGGING)
> 2.5) P. 1-3 with additional indexes - either created before data load on
> empty table, or after load on table with data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
