[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15496748#comment-15496748 ] Thomas Weise commented on APEXMALHAR-2205: -- [~brightchen] please document for the 3 cases: - environment - key/value size - key cardinality - throughput > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > Attachments: ManagedStateBenchmark.png > > > The key was generated by following forum: > (timestamp - timestamp % range) + random.nextInt(range) > where the default value of range is 60k(1 minute). > With the default "range" value, the key will have the same prefix in one > minute. And the process rate is around 40k/s ( namely 2400k/m ). And the key > range is 60k, so roughly 40 operation for each key by average ( namely 1 > insertion and another 39 update ). > Following is the benchmark result: The highest is around 40K/s and lowest is > around 28k/s, average is around 35.5k/s > See attached "ManagedStateBenchmark.png" for screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15496742#comment-15496742 ] ASF GitHub Bot commented on APEXMALHAR-2205: Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/412 > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > Attachments: ManagedStateBenchmark.png > > > The key was generated by following forum: > (timestamp - timestamp % range) + random.nextInt(range) > where the default value of range is 60k(1 minute). > With the default "range" value, the key will have the same prefix in one > minute. And the process rate is around 40k/s ( namely 2400k/m ). And the key > range is 60k, so roughly 40 operation for each key by average ( namely 1 > insertion and another 39 update ). > Following is the benchmark result: The highest is around 40K/s and lowest is > around 28k/s, average is around 35.5k/s > See attached "ManagedStateBenchmark.png" for screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491878#comment-15491878 ] ASF GitHub Bot commented on APEXMALHAR-2205: GitHub user brightchen reopened a pull request: https://github.com/apache/apex-malhar/pull/412 APEXMALHAR-2205 #resolve #comment State management benchmark - add up… …date You can merge this pull request into a Git repository by running: $ git pull https://github.com/brightchen/apex-malhar APEXMALHAR-2205-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/apex-malhar/pull/412.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #412 commit 7f3fc5b0ac4484b5b5c480f1779bfe41d1fa55a2 Author: brightchen Date: 2016-09-14T22:21:35Z APEXMALHAR-2205 #resolve #comment State management benchmark - add update > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > Attachments: ManagedStateBenchmark.png > > > The key was generated by following forum: > (timestamp - timestamp % range) + random.nextInt(range) > where the default value of range is 60k(1 minute). > With the default "range" value, the key will have the same prefix in one > minute. And the process rate is around 40k/s ( namely 2400k/m ). And the key > range is 60k, so roughly 40 operation for each key by average ( namely 1 > insertion and another 39 update ). > Following is the benchmark result: The highest is around 40K/s and lowest is > around 28k/s, average is around 35.5k/s > See attached "ManagedStateBenchmark.png" for screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491877#comment-15491877 ] ASF GitHub Bot commented on APEXMALHAR-2205: Github user brightchen closed the pull request at: https://github.com/apache/apex-malhar/pull/412 > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > Attachments: ManagedStateBenchmark.png > > > The key was generated by following forum: > (timestamp - timestamp % range) + random.nextInt(range) > where the default value of range is 60k(1 minute). > With the default "range" value, the key will have the same prefix in one > minute. And the process rate is around 40k/s ( namely 2400k/m ). And the key > range is 60k, so roughly 40 operation for each key by average ( namely 1 > insertion and another 39 update ). > Following is the benchmark result: The highest is around 40K/s and lowest is > around 28k/s, average is around 35.5k/s > See attached "ManagedStateBenchmark.png" for screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491610#comment-15491610 ] ASF GitHub Bot commented on APEXMALHAR-2205: GitHub user brightchen opened a pull request: https://github.com/apache/apex-malhar/pull/412 APEXMALHAR-2205 #resolve #comment State management benchmark - add up… …date You can merge this pull request into a Git repository by running: $ git pull https://github.com/brightchen/apex-malhar APEXMALHAR-2205-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/apex-malhar/pull/412.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #412 commit 7f3fc5b0ac4484b5b5c480f1779bfe41d1fa55a2 Author: brightchen Date: 2016-09-14T22:21:35Z APEXMALHAR-2205 #resolve #comment State management benchmark - add update > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > Attachments: ManagedStateBenchmark.png > > > The key was generated by following forum: > (timestamp - timestamp % range) + random.nextInt(range) > where the default value of range is 60k(1 minute). > With the default "range" value, the key will have the same prefix in one > minute. And the process rate is around 40k/s ( namely 2400k/m ). And the key > range is 60k, so roughly 40 operation for each key by average ( namely 1 > insertion and another 39 update ). > Following is the benchmark result: The highest is around 40K/s and lowest is > around 28k/s, average is around 35.5k/s > See attached "ManagedStateBenchmark.png" for screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488580#comment-15488580 ] Thomas Weise commented on APEXMALHAR-2205: -- The store operator only does a put. Update would be get/do-something/put. Please just add comments to the JIRA vs. updating description, it is easier to follow. > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > Attachments: ManagedStateBenchmark.png > > > The key was generated by following forum: > (timestamp - timestamp % range) + random.nextInt(range) > where the default value of range is 60k(1 minute). > With the default "range" value, the key will have the same prefix in one > minute. And the process rate is around 40k/s ( namely 2400k/m ). And the key > range is 60k, so roughly 40 operation for each key by average ( namely 1 > insertion and another 39 update ). > Following is the benchmark result: The highest is around 40K/s and lowest is > around 28k/s, average is around 35.5k/s > See attached "ManagedStateBenchmark.png" for screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488539#comment-15488539 ] bright chen commented on APEXMALHAR-2205: - I updated the description and hopefully it is you wanted > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > Attachments: ManagedStateBenchmark.png > > > The key was generated by following forum: > (timestamp - timestamp % range) + random.nextInt(range) > where the default value of range is 60k(1 minute). > With the default "range" value, the key will have the same prefix in one > minute. And the process rate is around 40k/s ( namely 2400k/m ). And the key > range is 60k, so roughly 40 operation for each key by average ( namely 1 > insertion and another 39 update ). > Following is the benchmark result: The highest is around 40K/s and lowest is > around 28k/s, average is around 35.5k/s > See attached "ManagedStateBenchmark.png" for screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15485806#comment-15485806 ] Thomas Weise commented on APEXMALHAR-2205: -- That's for write only workload? Any data for read-write (update) case? > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > Attachments: ManagedStateBenchmark.png > > > Following is the benchmark result: The highest is around 40K/s and lowest is > around 28k/s, average is around 35.5k/s > See attached "ManagedStateBenchmark.png" for screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15485146#comment-15485146 ] Thomas Weise commented on APEXMALHAR-2205: -- [~brightchen] can you document the results? > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15456928#comment-15456928 ] ASF GitHub Bot commented on APEXMALHAR-2205: Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/398 > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15456921#comment-15456921 ] Thomas Weise commented on APEXMALHAR-2205: -- Is it just writing to the WAL or also writing the data files? Any information about key cardinality? > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15456794#comment-15456794 ] bright chen commented on APEXMALHAR-2205: - The throughput ( almost same in cluster and local machine ) is around 40k/second in peek and 28k/second at bottom. I ran around one hour and pretty stable. > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15456780#comment-15456780 ] ASF GitHub Bot commented on APEXMALHAR-2205: GitHub user brightchen opened a pull request: https://github.com/apache/apex-malhar/pull/398 APEXMALHAR-2205 #resolve #comment State management benchmark previous PR closed by accident. https://github.com/apache/apex-malhar/pull/386 You can merge this pull request into a Git repository by running: $ git pull https://github.com/brightchen/apex-malhar APEXMALHAR-2205 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/apex-malhar/pull/398.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #398 commit c5a12e4e747c5be840a16e4c932cbc1dbff79894 Author: brightchen Date: 2016-08-26T23:09:12Z APEXMALHAR-2205 #resolve #comment State management benchmark > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15456736#comment-15456736 ] ASF GitHub Bot commented on APEXMALHAR-2205: Github user brightchen closed the pull request at: https://github.com/apache/apex-malhar/pull/386 > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15447785#comment-15447785 ] ASF GitHub Bot commented on APEXMALHAR-2205: Github user brightchen closed the pull request at: https://github.com/apache/apex-malhar/pull/386 > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15447784#comment-15447784 ] ASF GitHub Bot commented on APEXMALHAR-2205: GitHub user brightchen reopened a pull request: https://github.com/apache/apex-malhar/pull/386 APEXMALHAR-2205 State management benchmark You can merge this pull request into a Git repository by running: $ git pull https://github.com/brightchen/apex-malhar APEXMALHAR-2205 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/apex-malhar/pull/386.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #386 commit eea3d04910e4b06983cacb98a1e0d1429d0f968d Author: brightchen Date: 2016-08-26T23:09:12Z APEXMALHAR-2205 State management benchmark > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15446399#comment-15446399 ] ASF GitHub Bot commented on APEXMALHAR-2205: GitHub user brightchen reopened a pull request: https://github.com/apache/apex-malhar/pull/386 APEXMALHAR-2205 State management benchmark You can merge this pull request into a Git repository by running: $ git pull https://github.com/brightchen/apex-malhar APEXMALHAR-2205 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/apex-malhar/pull/386.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #386 commit 7be771bc5f58322fd8ddaf6f8caa8bc799791f9a Author: brightchen Date: 2016-08-26T23:09:12Z APEXMALHAR-2205 State management benchmark > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15446398#comment-15446398 ] ASF GitHub Bot commented on APEXMALHAR-2205: Github user brightchen closed the pull request at: https://github.com/apache/apex-malhar/pull/386 > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (APEXMALHAR-2205) State management benchmark
[ https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440148#comment-15440148 ] ASF GitHub Bot commented on APEXMALHAR-2205: GitHub user brightchen opened a pull request: https://github.com/apache/apex-malhar/pull/386 APEXMALHAR-2205 State management benchmark You can merge this pull request into a Git repository by running: $ git pull https://github.com/brightchen/apex-malhar APEXMALHAR-2205 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/apex-malhar/pull/386.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #386 commit 7be771bc5f58322fd8ddaf6f8caa8bc799791f9a Author: brightchen Date: 2016-08-26T23:09:12Z APEXMALHAR-2205 State management benchmark > State management benchmark > -- > > Key: APEXMALHAR-2205 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205 > Project: Apache Apex Malhar > Issue Type: Task >Reporter: bright chen >Assignee: bright chen > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
