Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-08-01 Thread Fabien COELHO
Hello, Version one is k' = 1 + (a * k + b) modulo n with a prime with respect to n, n being the number of keys. This is nearly possible, but for the modulo operator which is currently missing, and that I'm planning to submit for this very reason, but probably another time. That's pretty

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-08-01 Thread Mitsumasa KONDO
Hi, 2014-08-01 16:26 GMT+09:00 Fabien COELHO coe...@cri.ensmp.fr Maybe somebody who knows more math than I do (like you, probably!) can come up with something more clever. I can certainly suggest other formula, but that does not mean beautiful code, thus would probably be rejected. I'll

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-31 Thread Robert Haas
On Wed, Jul 30, 2014 at 4:18 PM, Fabien COELHO coe...@cri.ensmp.fr wrote: nor am I in favor of patch B. Yep. Would providing these as additional contrib files be more acceptable? Something like tpc-b-gauss.sql... Otherwise there is no example available to show the feature. To be honest, it

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-31 Thread Robert Haas
On Wed, Jul 30, 2014 at 9:00 PM, Mitsumasa KONDO kondo.mitsum...@gmail.com wrote: Hmm... It doesn't have harm for pgbench source code. And, in general, checking script is useful for avoiding bug. Not if nobody runs it, or if people run it but don't know what the output should look like. I

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-31 Thread Fabien COELHO
Hello Robert, [...] One of the concerns that I have about the proposal of simply slapping a gaussian or exponential modifier onto \setrandom aid 1 :naccounts is that, while it will allow you to make part of the relation hot and another part of the relation cold, you really can't get any

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-31 Thread Robert Haas
On Thu, Jul 31, 2014 at 10:01 AM, Fabien COELHO coe...@cri.ensmp.fr wrote: One of the concerns that I have about the proposal of simply slapping a gaussian or exponential modifier onto \setrandom aid 1 :naccounts is that, while it will allow you to make part of the relation hot and another part

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-30 Thread Robert Haas
On Tue, Jul 29, 2014 at 4:41 AM, Fabien COELHO coe...@cri.ensmp.fr wrote: Attached B patch does turn incorrect setrandom syntax into errors instead of ignoring extra parameters. First A patch is repeated to help commitfest references. Oops, I applied the change on the wrong part:-( Here is

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-30 Thread Fabien COELHO
Hello Robert, I've committed the changes to pgbench.c and the documentation changes with some further wordsmithing. Ok, thanks a lot for your reviews and your help with improving the documentation. I don't think including the other changes in patch A is a good idea, Fine. It was mostly

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-30 Thread Mitsumasa KONDO
Hi, 2014-07-31 5:18 GMT+09:00 Fabien COELHO coe...@cri.ensmp.fr: I've committed the changes to pgbench.c and the documentation changes with some further wordsmithing. Ok, thanks a lot for your reviews and your help with improving the documentation. Yeah, thanks for all relative members.

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-29 Thread Fabien COELHO
Hello Robert, I wish to agree, but my interpretation of the previous code is that they were ignored before, so ISTM that we are stuck with keeping the same unfortunate behavior. I don't agree. I'm not in a huge hurry to fix all the places where pgbench currently lacks error checks just

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-29 Thread Fabien COELHO
Hello Robert, 3. Similarly, I suggest that the use of gaussian or uniform be an error when argc 6 OR argc 6. I also suggest that the parenthesized distribution type be dropped from the error message in all cases. I wish to agree, but my interpretation of the previous code is that they

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-29 Thread Fabien COELHO
Attached B patch does turn incorrect setrandom syntax into errors instead of ignoring extra parameters. First A patch is repeated to help commitfest references. Oops, I applied the change on the wrong part:-( Here is the change on part A which checks setrandom syntax, and B for

Re: [HACKERS] gaussian distribution pgbench

2014-07-28 Thread Heikki Linnakangas
On 07/17/2014 11:13 PM, Fabien COELHO wrote: However, ISTM that it is not the purpose of pgbench documentation to be a primer about what is an exponential or gaussian distribution, so the idea would yet be to have a relatively compact explanation, and that the interested but clueless reader

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-28 Thread Robert Haas
On Wed, Jul 23, 2014 at 12:39 PM, Fabien COELHO coe...@cri.ensmp.fr wrote: 3. Similarly, I suggest that the use of gaussian or uniform be an error when argc 6 OR argc 6. I also suggest that the parenthesized distribution type be dropped from the error message in all cases. I wish to

Re: [HACKERS] gaussian distribution pgbench -- splits Bv6

2014-07-25 Thread Mitsumasa KONDO
Thanks for your modify the patch! I confirmed that It seems to be fine. I think that our latest patch fill all community comment. So it is really ready for committer now. Best regards, -- Mitsumasa KONDO

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-24 Thread Mitsumasa KONDO
Hi, Thank you for your grate documentation and fix working!!! It becomes very helpful for understanding our feature. I add two feature in gauss_B_4.patch. 1) Add gaussianProbability() function It is same as exponentialProbability(). And the feature is as same as before. 2) Add result of

Re: [HACKERS] gaussian distribution pgbench -- splits Bv6

2014-07-24 Thread Fabien COELHO
Thank you for your grate documentation and fix working!!! It becomes very helpful for understanding our feature. Hopefully it will help make it, or part of it, pass through. I add two feature in gauss_B_4.patch. 1) Add gaussianProbability() function It is same as exponentialProbability().

Re: [HACKERS] gaussian distribution pgbench -- splits Bv6

2014-07-24 Thread Alvaro Herrera
Fabien COELHO wrote: I also have a problem with assert Assert. I finally figured out that Assert is not compiled in by default, thus it is generally ignored. So it is more for debugging purposes when activated than for guarding against some unexpected user errors. Yes, Assert() is for

Re: [HACKERS] gaussian distribution pgbench -- part 1/2

2014-07-23 Thread Robert Haas
On Thu, Jul 17, 2014 at 12:09 AM, Fabien COELHO coe...@cri.ensmp.fr wrote: pgbench with gaussian exponential, part 1 of 2. This patch is a subset of the previous patch which only adds the two new \setrandom gaussian and exponantial variants, but not the adapted pgbench test cases, as

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-23 Thread Fabien COELHO
Hello Robert, Some review comments: Thanks a lot for your return. Please find attached two new parts of the patch (A for setrandom extension, B for pgbench embedded test case extension). 1. I suggest that getExponentialrand and getGaussianrand be renamed to getExponentialRand and

Re: [HACKERS] gaussian distribution pgbench

2014-07-22 Thread Fabien COELHO
Please find attached 2 patches, which are a split of the patch discussed in this thread. Please find attached a very minor improvement to apply a code (variable name) simplification directly in patch A so as to avoid a change in patch B. The cumulated patch is the same as previous. (A)

Re: [HACKERS] gaussian distribution pgbench

2014-07-18 Thread Mitsumasa KONDO
2014-07-18 5:13 GMT+09:00 Fabien COELHO coe...@cri.ensmp.fr: However, ISTM that it is not the purpose of pgbench documentation to be a primer about what is an exponential or gaussian distribution, so the idea would yet be to have a relatively compact explanation, and that the interested but

Re: [HACKERS] gaussian distribution pgbench

2014-07-18 Thread Fabien COELHO
For example, when we set the number of transaction 10,000 (-t 1), range of aid is 100,000, and --exponential is 10, decile percents is under following as you know. decile percents: 63.2% 23.3% 8.6% 3.1% 1.2% 0.4% 0.2% 0.1% 0.0% 0.0% highest/lowest percent of the range: 9.5% 0.0% They

Re: [HACKERS] gaussian distribution pgbench

2014-07-18 Thread Fabien COELHO
Please find attached 2 patches, which are a split of the patch discussed in this thread. (A) add gaussian exponential options to pgbench \setrandom the patch includes sql test files. There is no change in the *code* from previous already reviewed submissions, so I do not think that it

Re: [HACKERS] gaussian distribution pgbench

2014-07-17 Thread Robert Haas
On Wed, Jul 16, 2014 at 12:57 AM, Fabien COELHO coe...@cri.ensmp.fr wrote: Well, I think the feedback has been pretty clear, honestly. Here's what I'm unhappy about: I can't understand what these options are actually doing. We can try to improve the documentation, once more! However, ISTM

Re: [HACKERS] gaussian distribution pgbench

2014-07-17 Thread Fabien COELHO
However, ISTM that it is not the purpose of pgbench documentation to be a primer about what is an exponential or gaussian distribution, so the idea would yet be to have a relatively compact explanation, and that the interested but clueless reader would document h..self from wikipedia or a text

Re: [HACKERS] gaussian distribution pgbench -- part 1/2

2014-07-16 Thread Fabien COELHO
pgbench with gaussian exponential, part 1 of 2. This patch is a subset of the previous patch which only adds the two new \setrandom gaussian and exponantial variants, but not the adapted pgbench test cases, as suggested by Fujii Masao. There is no new code nor code changes. The corresponding

Re: [HACKERS] gaussian distribution pgbench

2014-07-15 Thread Fabien COELHO
Hello Robert, Well, I think the feedback has been pretty clear, honestly. Here's what I'm unhappy about: I can't understand what these options are actually doing. We can try to improve the documentation, once more! However, ISTM that it is not the purpose of pgbench documentation to be a

Re: [HACKERS] gaussian distribution pgbench

2014-07-14 Thread Robert Haas
On Sun, Jul 13, 2014 at 2:27 AM, Mitsumasa KONDO kondo.mitsum...@gmail.com wrote: I still agree with Fabien-san. I cannot understand why our logical proposal isn't accepted... Well, I think the feedback has been pretty clear, honestly. Here's what I'm unhappy about: I can't understand what

Re: [HACKERS] gaussian distribution pgbench

2014-07-13 Thread Mitsumasa KONDO
Hi, 2014-07-04 19:05 GMT+09:00 Andres Freund and...@2ndquadrant.com: On 2014-07-04 11:59:23 +0200, Fabien COELHO wrote: Yea. I certainly disagree with the patch in it's current state because it copies the same 15 lines several times with a two word difference. Independent of whether we

Re: [HACKERS] gaussian distribution pgbench

2014-07-04 Thread Fabien COELHO
Yea. I certainly disagree with the patch in it's current state because it copies the same 15 lines several times with a two word difference. Independent of whether we want those options, I don't think that's going to fly. I liked a simple static string for the different variants, which

Re: [HACKERS] gaussian distribution pgbench

2014-07-04 Thread Andres Freund
On 2014-07-04 11:59:23 +0200, Fabien COELHO wrote: Yea. I certainly disagree with the patch in it's current state because it copies the same 15 lines several times with a two word difference. Independent of whether we want those options, I don't think that's going to fly. I liked a simple

Re: [HACKERS] gaussian distribution pgbench

2014-07-03 Thread Fabien COELHO
Hello Gavin, decile percents: 69.9% 21.0% 6.3% 1.9% 0.6% 0.2% 0.1% 0.0% 0.0% 0.0% probability of fist/last percent of the range: 11.3% 0.0% I would suggest that probabilities should NEVER be expressed in percentages! As a percentage probability looks weird, and is never used for serious

Re: [HACKERS] gaussian distribution pgbench

2014-07-03 Thread Gavin Flower
On 03/07/14 20:58, Fabien COELHO wrote: Hello Gavin, decile percents: 69.9% 21.0% 6.3% 1.9% 0.6% 0.2% 0.1% 0.0% 0.0% 0.0% probability of fist/last percent of the range: 11.3% 0.0% I would suggest that probabilities should NEVER be expressed in percentages! As a percentage probability

Re: [HACKERS] gaussian distribution pgbench

2014-07-03 Thread Fujii Masao
On Wed, Jul 2, 2014 at 6:05 PM, Fabien COELHO coe...@cri.ensmp.fr wrote: Hello Mitsumasa-san, And I'm also interested in your decile percents output like under followings, decile percents: 39.6% 24.0% 14.6% 8.8% 5.4% 3.3% 2.0% 1.2% 0.7% 0.4% Sure, I'm really fine with that. I think

Re: [HACKERS] gaussian distribution pgbench

2014-07-03 Thread Andres Freund
On 2014-07-03 21:27:53 +0900, Fujii Masao wrote: Add drawing random integers with a Gaussian or truncated exponentional distributions to pgbench. Test variants with these distributions are also provided and triggered with options --gaussian=... and --exponential= IIRC we've not

Re: [HACKERS] gaussian distribution pgbench

2014-07-02 Thread Fabien COELHO
Hello Mitsumasa-san, And I'm also interested in your decile percents output like under followings, decile percents: 39.6% 24.0% 14.6% 8.8% 5.4% 3.3% 2.0% 1.2% 0.7% 0.4% Sure, I'm really fine with that. I think that it is easier than before. Sum of decile percents is just 100%. That's a

Re: [HACKERS] gaussian distribution pgbench

2014-07-02 Thread Fabien COELHO
I have just updated the wording so that it may be clearer: Oops, I have sent the wrong patch, without the wording fix. Here is the real updated version, which I tested. probability of fist/last percent of the range: 11.3% 0.0% -- Fabien.diff --git a/contrib/pgbench/pgbench.c

Re: [HACKERS] gaussian distribution pgbench

2014-07-02 Thread Gavin Flower
On 02/07/14 21:05, Fabien COELHO wrote: Hello Mitsumasa-san, And I'm also interested in your decile percents output like under followings, decile percents: 39.6% 24.0% 14.6% 8.8% 5.4% 3.3% 2.0% 1.2% 0.7% 0.4% Sure, I'm really fine with that. I think that it is easier than before. Sum of

Re: [HACKERS] gaussian distribution pgbench

2014-06-17 Thread Mitsumasa KONDO
Hello Fabien-san, I have checked your v13 patch, and tested the new exponential distribution generating algorithm. It works fine and less or no overhead than previous version. Great work! And I agree with your proposal. And I'm also interested in your decile percents output like under

Re: [HACKERS] gaussian distribution pgbench

2014-04-01 Thread Fabien COELHO
Please find attached an updated version v13 for this patch. I have (I hope) significanlty improved the documentation, including not so helpful mathematical explanation about the actual meaning of the threshold value. If a native English speaker could check the documentation, it would be

Re: [HACKERS] gaussian distribution pgbench

2014-03-18 Thread KONDO Mitsumasa
(2014/03/17 22:37), Tom Lane wrote: KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp writes: (2014/03/17 18:02), Heikki Linnakangas wrote: On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: There is an infinite number of variants of the TPC-B test that we could include in pgbench. If we start

Re: [HACKERS] gaussian distribution pgbench

2014-03-18 Thread KONDO Mitsumasa
(2014/03/17 23:29), Robert Haas wrote: On Sat, Mar 15, 2014 at 4:50 AM, Mitsumasa KONDO kondo.mitsum...@gmail.com wrote: There are explanations and computations as comments in the code. If it is about the documentation, I'm not sure that a very precise mathematical definition will help a lot of

Re: [HACKERS] gaussian distribution pgbench

2014-03-18 Thread KONDO Mitsumasa
And I find new useful point of this feature. Under following results are '--gaussian=20' case and '--gaussian=2' case, and postgresql setting is same. [mitsu-ko@pg-rex31 pgbench]$ ./pgbench -c8 -j4 --gaussian=20 -T30 -P 5 starting vacuum...end. progress: 5.0 s, 4285.8 tps, lat 1.860 ms stddev

Re: [HACKERS] gaussian distribution pgbench

2014-03-18 Thread Heikki Linnakangas
On 03/18/2014 11:57 AM, KONDO Mitsumasa wrote: I think that this feature will be also useful for survey new buffer-replace algorithm and checkpoint strategy, so on. Sure. No doubt about that. If we remove this option, it is really dissapointed.. As long as we get the \setrandom changes in,

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread KONDO Mitsumasa
Hi Heikki-san, (2014/03/17 14:39), KONDO Mitsumasa wrote: (2014/03/15 15:53), Fabien COELHO wrote: Hello Heikki, A couple of comments: * There should be an explicit \setrandom ... uniform option too, even though you get that implicitly if you don't specify the distribution Fix. We can use

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Heikki Linnakangas
On 03/15/2014 08:53 AM, Fabien COELHO wrote: * Does min and max really make sense for gaussian and exponential distributions? For gaussian, I would expect mean and standard deviation as the parameters, not min/max/threshold. Yes... and no:-) The aim is to draw an integer primary key from a

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Heikki Linnakangas
On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: By the way, you seem to want to remove --gaussian=NUM and --exponential=NUM command options. Can you tell me the objective reason? I think pgbench is the benchmark test on PostgreSQL and default benchmark is TPC-B-like benchmark. It is written in

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread KONDO Mitsumasa
(2014/03/17 17:46), Heikki Linnakangas wrote: On 03/15/2014 08:53 AM, Fabien COELHO wrote: * Does min and max really make sense for gaussian and exponential distributions? For gaussian, I would expect mean and standard deviation as the parameters, not min/max/threshold. Yes... and no:-) The

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread KONDO Mitsumasa
(2014/03/17 18:02), Heikki Linnakangas wrote: On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: By the way, you seem to want to remove --gaussian=NUM and --exponential=NUM command options. Can you tell me the objective reason? I think pgbench is the benchmark test on PostgreSQL and default

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Fujii Masao
On Mon, Mar 17, 2014 at 7:07 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: (2014/03/17 18:02), Heikki Linnakangas wrote: On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: By the way, you seem to want to remove --gaussian=NUM and --exponential=NUM command options. Can you tell me the

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Tom Lane
KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp writes: (2014/03/17 18:02), Heikki Linnakangas wrote: On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: There is an infinite number of variants of the TPC-B test that we could include in pgbench. If we start adding every one of them, we're quickly

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Robert Haas
On Sat, Mar 15, 2014 at 4:50 AM, Mitsumasa KONDO kondo.mitsum...@gmail.com wrote: There are explanations and computations as comments in the code. If it is about the documentation, I'm not sure that a very precise mathematical definition will help a lot of people, and might rather hinder

Re: [HACKERS] gaussian distribution pgbench

2014-03-16 Thread KONDO Mitsumasa
(2014/03/15 15:53), Fabien COELHO wrote: Hello Heikki, A couple of comments: * There should be an explicit \setrandom ... uniform option too, even though you get that implicitly if you don't specify the distribution Indeed. I agree. I suggested it, but it got lost. * What exactly does

Re: [HACKERS] gaussian distribution pgbench

2014-03-15 Thread Fabien COELHO
Hello Heikki, A couple of comments: * There should be an explicit \setrandom ... uniform option too, even though you get that implicitly if you don't specify the distribution Indeed. I agree. I suggested it, but it got lost. * What exactly does the threshold mean? The docs informally

Re: [HACKERS] gaussian distribution pgbench

2014-03-15 Thread Mitsumasa KONDO
Oh, sorry, I forgot to write URL referring picture. http://en.wikipedia.org/wiki/Normal_distribution http://en.wikipedia.org/wiki/Exponential_distribution regards, -- Mitsumasa KONDO 2014-03-15 17:50 GMT+09:00 Mitsumasa KONDO kondo.mitsum...@gmail.com: Hi 2014-03-15 15:53 GMT+09:00 Fabien

Re: [HACKERS] gaussian distribution pgbench

2014-03-15 Thread Fabien COELHO
Nice drawing! * How about setting the variable as a float instead of integer? Would seem more natural to me. At least as an option. Which variable? The values set by setrandom are mostly used for primary keys. We really want integers in a range. I think he said threshold parameter.

Re: [HACKERS] gaussian distribution pgbench

2014-03-15 Thread Mitsumasa KONDO
2014-03-15 19:04 GMT+09:00 Fabien COELHO coe...@cri.ensmp.fr: Nice drawing! * How about setting the variable as a float instead of integer? Would seem more natural to me. At least as an option. Which variable? The values set by setrandom are mostly used for primary keys. We really

Re: [HACKERS] gaussian distribution pgbench

2014-03-14 Thread Fabien COELHO
Well, when we set '--gaussian=NUM' or '--exponential=NUM' on command line, we can see access probability of top N records in result of final output. This out put is under following, Indeed. I had forgotten this point. This is a significant information that I would not like to loose. This

Re: [HACKERS] gaussian distribution pgbench

2014-03-14 Thread Heikki Linnakangas
On 03/13/2014 04:00 PM, Fujii Masao wrote: On Thu, Mar 13, 2014 at 10:51 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: IMHO we should just implement the \setrandom changes, and not add any of these options to modify the standard test workload. If someone wants to run TPC-B workload with

Re: [HACKERS] gaussian distribution pgbench

2014-03-13 Thread Fujii Masao
On Tue, Mar 11, 2014 at 1:49 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: (2014/03/09 1:49), Fabien COELHO wrote: Hello Mitsumasa-san, New \setrandom interface is here. \setrandom var min max [gaussian threshold | exponential threshold] Attached patch realizes this

Re: [HACKERS] gaussian distribution pgbench

2014-03-13 Thread Heikki Linnakangas
On 03/13/2014 03:17 PM, Fujii Masao wrote: On Tue, Mar 11, 2014 at 1:49 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: (2014/03/09 1:49), Fabien COELHO wrote: I'm okay with this UI and its implementation. OK. We should do the same discussion for the UI of command-line option?

Re: [HACKERS] gaussian distribution pgbench

2014-03-13 Thread Fujii Masao
On Thu, Mar 13, 2014 at 10:51 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 03/13/2014 03:17 PM, Fujii Masao wrote: On Tue, Mar 11, 2014 at 1:49 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: (2014/03/09 1:49), Fabien COELHO wrote: I'm okay with this UI and its

Re: [HACKERS] gaussian distribution pgbench

2014-03-13 Thread Fabien COELHO
We should do the same discussion for the UI of command-line option? The patch adds two options --gaussian and --exponential, but this UI seems to be a bit inconsistent with the UI for \setrandom. Instead, we can use something like --distribution=[uniform | gaussian | exponential]. Hmmm.

Re: [HACKERS] gaussian distribution pgbench

2014-03-13 Thread KONDO Mitsumasa
Hi, (2014/03/14 4:21), Fabien COELHO wrote: We should do the same discussion for the UI of command-line option? The patch adds two options --gaussian and --exponential, but this UI seems to be a bit inconsistent with the UI for \setrandom. Instead, we can use something like

Re: [HACKERS] gaussian distribution pgbench

2014-03-13 Thread KONDO Mitsumasa
(2014/03/13 23:00), Fujii Masao wrote: On Thu, Mar 13, 2014 at 10:51 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 03/13/2014 03:17 PM, Fujii Masao wrote: On Tue, Mar 11, 2014 at 1:49 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: (2014/03/09 1:49), Fabien COELHO wrote:

Re: [HACKERS] gaussian distribution pgbench

2014-03-10 Thread KONDO Mitsumasa
(2014/03/09 1:49), Fabien COELHO wrote: Hello Mitsumasa-san, New \setrandom interface is here. \setrandom var min max [gaussian threshold | exponential threshold] Attached patch realizes this interface, but it has little bit ugly codeing in executeStatement() and process_commands().. I

Re: [HACKERS] gaussian distribution pgbench

2014-03-08 Thread Fabien COELHO
Hello Mitsumasa-san, New \setrandom interface is here. \setrandom var min max [gaussian threshold | exponential threshold] Attached patch realizes this interface, but it has little bit ugly codeing in executeStatement() and process_commands().. I think it is not too bad. The ignore extra

Re: [HACKERS] gaussian distribution pgbench

2014-03-06 Thread KONDO Mitsumasa
Hi, (2014/03/04 17:42), KONDO Mitsumasa wrote: (2014/03/04 17:28), Fabien COELHO wrote: OK. I'm not sure which idia is the best. So I wait for comments in community:) Hmmm. Maybe you can do what Tom voted for, he is the committer:-) Yeah, but he might change his mind by our disscuttion. So

Re: [HACKERS] gaussian distribution pgbench

2014-03-06 Thread KONDO Mitsumasa
(2014/03/07 16:02), KONDO Mitsumasa wrote: And other cases are classified under following. \setrandom var min max gaussian #hoge -- uniform Oh, it's wrong... It will be.. \setrandom var min max gaussian #hoge -- ERROR Regards, -- Mitsumasa KONDO NTT Open Source Software Center -- Sent via

Re: [HACKERS] gaussian distribution pgbench

2014-03-04 Thread Fabien COELHO
OK. I'm not sure which idia is the best. So I wait for comments in community:) Hmmm. Maybe you can do what Tom voted for, he is the committer:-) -- Fabien. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] gaussian distribution pgbench

2014-03-04 Thread KONDO Mitsumasa
(2014/03/04 17:28), Fabien COELHO wrote: OK. I'm not sure which idia is the best. So I wait for comments in community:) Hmmm. Maybe you can do what Tom voted for, he is the committer:-) Yeah, but he might change his mind by our disscuttion. So I wait untill tomorrow, and if nothing to comment,

Re: [HACKERS] gaussian distribution pgbench

2014-03-03 Thread KONDO Mitsumasa
(2014/03/03 16:51), Fabien COELHO wrote:\setrandom foo 1 10 [uniform] \setrandom foo 1 :size gaussian 3.6 \setrandom foo 1 100 exponential 7.2 It's good design. I think it will become more low overhead at part of parsing in pgbench, because comparison of strings will be

Re: [HACKERS] gaussian distribution pgbench

2014-03-02 Thread Fabien COELHO
Hello Alvaro Tom, Alvaro Herrera alvhe...@2ndquadrant.com writes: Seems that in the review so far, Fabien has focused mainly in the mathematical properties of the new random number generation. That seems perfectly fine, but no comment has been made about the chosen UI for the feature. Per

Re: [HACKERS] gaussian distribution pgbench

2014-03-02 Thread KONDO Mitsumasa
(2014/03/02 22:32), Fabien COELHO wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Seems that in the review so far, Fabien has focused mainly in the mathematical properties of the new random number generation. That seems perfectly fine, but no comment has been made about the chosen UI

Re: [HACKERS] gaussian distribution pgbench

2014-03-02 Thread Fabien COELHO
\setrandom foo 1 10 [uniform] \setrandom foo 1 :size gaussian 3.6 \setrandom foo 1 100 exponential 7.2 It's good design. I think it will become more low overhead at part of parsing in pgbench, because comparison of strings will be redeced(maybe). And I'd like to remove [uniform],

Re: [HACKERS] gaussian distribution pgbench

2014-03-01 Thread Alvaro Herrera
Seems that in the review so far, Fabien has focused mainly in the mathematical properties of the new random number generation. That seems perfectly fine, but no comment has been made about the chosen UI for the feature. Per the few initial messages in the thread, in the patch as submitted you

Re: [HACKERS] gaussian distribution pgbench

2014-03-01 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Seems that in the review so far, Fabien has focused mainly in the mathematical properties of the new random number generation. That seems perfectly fine, but no comment has been made about the chosen UI for the feature. Per the few initial

Re: [HACKERS] gaussian distribution pgbench

2014-02-23 Thread Fabien COELHO
Gaussian Pgbench v8 patch by Mitsumasa KONDO review patch v9. * The purpose of the patch is to allow a pgbench script to draw from normally distributed or exponentially distributed integer values instead of uniformly distributed. This is a valuable contribution to enable pgbench to

Re: [HACKERS] gaussian distribution pgbench

2014-02-17 Thread KONDO Mitsumasa
(2014/02/16 7:38), Fabien COELHO wrote: I have updated the patch (v7) based on Mitsumasa latest v6: - some code simplifications formula changes. - I've added explicit looping probability computations in comments to show the (low) looping probability of the iterative search. -

Re: [HACKERS] gaussian distribution pgbench

2014-02-15 Thread Fabien COELHO
Gaussian Pgbench v6 patch by Mitsumasa KONDO review patch v7. * The purpose of the patch is to allow a pgbench script to draw from normally distributed or exponentially distributed integer values instead of uniformly distributed. This is a valuable contribution to enable pgbench to

Re: [HACKERS] gaussian distribution pgbench

2014-02-14 Thread Mitsumasa KONDO
I add exponential distribution random generator (and little bit refactoring:) ). I use inverse transform method to create its distribution. It's very simple method that is created by - log (rand()). We can control slope of distribution using threshold parameter. It is same as gaussian threshold.

Re: [HACKERS] gaussian distribution pgbench

2014-02-13 Thread KONDO Mitsumasa
Hi Febien, Thank you very much for your very detail and useful comments! I read your comment, I agree most of your advice:) Attached patch is fixed for your comment. That are... - Remove redundant long-option. - We can use --gaussian=NUM -S or --gaussian=NUMN -N options. - Add sentence in

Re: [HACKERS] gaussian distribution pgbench

2014-02-13 Thread KONDO Mitsumasa
Sorry, previos attached patch has small bug. Please use latest one. 134 - return min + (int64) (max - min + 1) * rand; 134 + return min + (int64)((max - min + 1) * rand); Regards, -- Mitsumasa KONDO NTT Open Source Software Center *** a/contrib/pgbench/pgbench.c ---

Re: [HACKERS] gaussian distribution pgbench

2014-02-09 Thread Fabien COELHO
Hello, I revise my gaussian pgbench patch which wss requested from community. With a lot of delay for which I apologise, please find hereafter the review. Gaussian Pgbench v3 patch by Mitsumasa KONDO review * The purpose of the patch is to allow a pgbench script to draw from normally

Re: [HACKERS] gaussian distribution pgbench

2013-12-19 Thread Peter Geoghegan
On Thu, Nov 21, 2013 at 9:13 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: So what I'd actually like to see is \setgaussian, for use in custom scripts. +1. I'd really like to be able to run a benchmark with a Gaussian and uniform distribution side-by-side for comparative purposes - we

Re: [HACKERS] gaussian distribution pgbench

2013-12-19 Thread Gavin Flower
On 20/12/13 09:36, Peter Geoghegan wrote: On Thu, Nov 21, 2013 at 9:13 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: So what I'd actually like to see is \setgaussian, for use in custom scripts. +1. I'd really like to be able to run a benchmark with a Gaussian and uniform distribution

Re: [HACKERS] gaussian distribution pgbench

2013-12-19 Thread Gregory Smith
On 12/19/13 5:52 PM, Gavin Flower wrote: Curious, wouldn't the common usage pattern tend to favour a skewed distribution, such as the Poisson Distribution (it has been over 40 years since I studied this area, so there may be better candidates). Some people like database load testing with a

Re: [HACKERS] gaussian distribution pgbench

2013-11-22 Thread Fabien COELHO
3. That said, this could be handy. But it would be even more handy if you could get Gaussian random numbers with \setrandom, so that you could use this with custom scripts. And once you implement that, do we actually need the -g flag anymore? If you want TPC-B transactions with gaussian

Re: [HACKERS] gaussian distribution pgbench

2013-11-21 Thread Heikki Linnakangas
On 30.09.2013 07:12, KONDO Mitsumasa wrote: (2013/09/27 5:29), Peter Eisentraut wrote: This patch no longer applies. I will try to create this patch in next commit fest. If you have nice idea, please send me! A few thoughts on this: 1. DBT-2 uses a non-uniform distribution. You can use that

Re: [HACKERS] gaussian distribution pgbench

2013-09-29 Thread KONDO Mitsumasa
Sorry for my delay reply. Since I have had vacation last week, I replyed from gmail. However, it was stalled post to pgsql-hackers:-( (2013/09/21 6:05), Kevin Grittner wrote: You had accidentally added to the CF In Progress. Oh, I had completely mistook this CF schedule :-) Maybe, Horiguchi-san

Re: [HACKERS] gaussian distribution pgbench

2013-09-29 Thread KONDO Mitsumasa
Sorry for my delay reply. Since I have had vacation last week, I replied from gmail. However, it was stalled post to pgsql-hackers:-( (2013/09/21 7:54), Fabien COELHO wrote: However this pattern induces stronger cache effects which are maybe not too realistic, because neighboring keys in the

Re: [HACKERS] gaussian distribution pgbench

2013-09-29 Thread KONDO Mitsumasa
(2013/09/27 5:29), Peter Eisentraut wrote: This patch no longer applies. I will try to create this patch in next commit fest. If you have nice idea, please send me! Regards, -- Mitsumasa KONDO NTT Open Source Software Center -- Sent via pgsql-hackers mailing list

Re: [HACKERS] gaussian distribution pgbench

2013-09-26 Thread Peter Eisentraut
On 9/20/13 2:42 AM, KONDO Mitsumasa wrote: I create gaussinan distribution pgbench patch that can access records with gaussian frequency. And I submit this commit fest. This patch no longer applies. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] gaussian distribution pgbench

2013-09-23 Thread Mitsumasa KONDO
However this pattern induces stronger cache effects which are maybe not too realistic, because neighboring keys in the middle are more likely to be chosen. I think that your opinion is right. However, in effect, it is a paseudo-benchmark, so that I think that such a simple mechanism is also

Re: [HACKERS] gaussian distribution pgbench

2013-09-23 Thread Mitsumasa KONDO
You had accidentally added to the CF In Progress. Oh, I had completely mistook this CF schedule :-) Maybe, Horiguchi-san is same situation... However, because of your moving, I become first submitter in next CF. Thank you for moving :-) -- Mitsumasa KONDO

Re: [HACKERS] gaussian distribution pgbench

2013-09-20 Thread Kevin Grittner
KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: I create gaussinan distribution pgbench patch that can access records with gaussian frequency. And I submit this commit fest. Thanks! I have moved this to the Open CommitFest, though.

Re: [HACKERS] gaussian distribution pgbench

2013-09-20 Thread Fabien COELHO
Hello Mitsumasa, In the general transaction situation, clients access for all records equally is hard to happen. I think gaussian distribution access patterns are most of transaction petterns in general. My patch realizes neary this access pattern. That is great! I was just looking for