Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-08-01 Thread Mitsumasa KONDO
Hi, 2014-08-01 16:26 GMT+09:00 Fabien COELHO > > Maybe somebody who knows more math than I do (like you, probably!) can >> come up with something more clever. >> > > I can certainly suggest other formula, but that does not mean beautiful > code, thus would probably be rejected. I'll see. > > An

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-08-01 Thread Fabien COELHO
Hello, Version one is "k' = 1 + (a * k + b) modulo n" with "a" prime with respect to "n", "n" being the number of keys. This is nearly possible, but for the modulo operator which is currently missing, and that I'm planning to submit for this very reason, but probably another time. That's pr

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-31 Thread Robert Haas
On Thu, Jul 31, 2014 at 10:01 AM, Fabien COELHO wrote: >> One of the concerns that I have about the proposal of simply slapping a >> gaussian or exponential modifier onto \setrandom aid 1 :naccounts is that, >> while it will allow you to make part of the relation hot and another part of >> the rel

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-31 Thread Fabien COELHO
Hello Robert, [...] One of the concerns that I have about the proposal of simply slapping a gaussian or exponential modifier onto \setrandom aid 1 :naccounts is that, while it will allow you to make part of the relation hot and another part of the relation cold, you really can't get any more

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-31 Thread Robert Haas
On Wed, Jul 30, 2014 at 9:00 PM, Mitsumasa KONDO wrote: > Hmm... It doesn't have harm for pgbench source code. And, in general, > checking script is useful for avoiding bug. Not if nobody runs it, or if people run it but don't know what the output should look like. I think anyone who knows enoug

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-31 Thread Robert Haas
On Wed, Jul 30, 2014 at 4:18 PM, Fabien COELHO wrote: >> nor am I in favor of patch B. > > Yep. Would providing these as additional contrib files be more acceptable? > Something like "tpc-b-gauss.sql"... Otherwise there is no example available > to show the feature. To be honest, it just feels li

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-30 Thread Mitsumasa KONDO
Hi, 2014-07-31 5:18 GMT+09:00 Fabien COELHO : > > I've committed the changes to pgbench.c and the documentation changes >> with some further wordsmithing. >> > > Ok, thanks a lot for your reviews and your help with improving the > documentation. Yeah, thanks for all relative members. > I don'

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-30 Thread Fabien COELHO
Hello Robert, I've committed the changes to pgbench.c and the documentation changes with some further wordsmithing. Ok, thanks a lot for your reviews and your help with improving the documentation. I don't think including the other changes in patch A is a good idea, Fine. It was mostly

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-30 Thread Robert Haas
On Tue, Jul 29, 2014 at 4:41 AM, Fabien COELHO wrote: >> Attached B patch does turn incorrect setrandom syntax into errors instead >> of ignoring extra parameters. >> >> First A patch is repeated to help commitfest references. > > Oops, I applied the change on the wrong part:-( > > Here is the cha

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-29 Thread Fabien COELHO
Attached B patch does turn incorrect setrandom syntax into errors instead of ignoring extra parameters. First A patch is repeated to help commitfest references. Oops, I applied the change on the wrong part:-( Here is the change on part A which checks setrandom syntax, and B for completenes

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-29 Thread Fabien COELHO
Hello Robert, 3. Similarly, I suggest that the use of gaussian or uniform be an error when argc < 6 OR argc > 6. I also suggest that the parenthesized distribution type be dropped from the error message in all cases. I wish to agree, but my interpretation of the previous code is that they we

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-28 Thread Fabien COELHO
Hello Robert, I wish to agree, but my interpretation of the previous code is that they were ignored before, so ISTM that we are stuck with keeping the same unfortunate behavior. I don't agree. I'm not in a huge hurry to fix all the places where pgbench currently lacks error checks just bec

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-28 Thread Robert Haas
On Wed, Jul 23, 2014 at 12:39 PM, Fabien COELHO wrote: >> 3. Similarly, I suggest that the use of gaussian or uniform be an >> error when argc < 6 OR argc > 6. I also suggest that the >> parenthesized distribution type be dropped from the error message in >> all cases. > > I wish to agree, but my

Re: [HACKERS] gaussian distribution pgbench

2014-07-28 Thread Heikki Linnakangas
On 07/17/2014 11:13 PM, Fabien COELHO wrote: However, ISTM that it is not the purpose of pgbench documentation to be a primer about what is an exponential or gaussian distribution, so the idea would yet be to have a relatively compact explanation, and that the interested but clueless reader wou

Re: [HACKERS] gaussian distribution pgbench -- splits Bv6

2014-07-25 Thread Mitsumasa KONDO
Thanks for your modify the patch! I confirmed that It seems to be fine. I think that our latest patch fill all community comment. So it is really ready for committer now. Best regards, -- Mitsumasa KONDO

Re: [HACKERS] gaussian distribution pgbench -- splits Bv6

2014-07-24 Thread Alvaro Herrera
Fabien COELHO wrote: > I also have a problem with assert & Assert. I finally figured out > that Assert is not compiled in by default, thus it is generally > ignored. So it is more for debugging purposes when activated than > for guarding against some unexpected user errors. Yes, Assert() is for

Re: [HACKERS] gaussian distribution pgbench -- splits Bv6

2014-07-24 Thread Fabien COELHO
Thank you for your grate documentation and fix working!!! It becomes very helpful for understanding our feature. Hopefully it will help make it, or part of it, pass through. I add two feature in gauss_B_4.patch. 1) Add gaussianProbability() function It is same as exponentialProbability(). A

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-24 Thread Mitsumasa KONDO
Hi, Thank you for your grate documentation and fix working!!! It becomes very helpful for understanding our feature. I add two feature in gauss_B_4.patch. 1) Add gaussianProbability() function It is same as exponentialProbability(). And the feature is as same as before. 2) Add result of "max/mi

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-23 Thread Fabien COELHO
Hello Robert, Some review comments: Thanks a lot for your return. Please find attached two new parts of the patch (A for setrandom extension, B for pgbench embedded test case extension). 1. I suggest that getExponentialrand and getGaussianrand be renamed to getExponentialRand and getGaus

Re: [HACKERS] gaussian distribution pgbench -- part 1/2

2014-07-23 Thread Robert Haas
On Thu, Jul 17, 2014 at 12:09 AM, Fabien COELHO wrote: > pgbench with gaussian & exponential, part 1 of 2. > > This patch is a subset of the previous patch which only adds the two > new \setrandom gaussian and exponantial variants, but not the > adapted pgbench test cases, as suggested by Fujii Ma

Re: [HACKERS] gaussian distribution pgbench

2014-07-21 Thread Fabien COELHO
Please find attached 2 patches, which are a split of the patch discussed in this thread. Please find attached a very minor improvement to apply a code (variable name) simplification directly in patch A so as to avoid a change in patch B. The cumulated patch is the same as previous. (A) ad

Re: [HACKERS] gaussian distribution pgbench

2014-07-18 Thread Fabien COELHO
Please find attached 2 patches, which are a split of the patch discussed in this thread. (A) add gaussian & exponential options to pgbench \setrandom the patch includes sql test files. There is no change in the *code* from previous already reviewed submissions, so I do not think that it

Re: [HACKERS] gaussian distribution pgbench

2014-07-18 Thread Fabien COELHO
For example, when we set the number of transaction 10,000 (-t 1), range of aid is 100,000, and --exponential is 10, decile percents is under following as you know. decile percents: 63.2% 23.3% 8.6% 3.1% 1.2% 0.4% 0.2% 0.1% 0.0% 0.0% highest/lowest percent of the range: 9.5% 0.0% They mean

Re: [HACKERS] gaussian distribution pgbench

2014-07-18 Thread Mitsumasa KONDO
2014-07-18 5:13 GMT+09:00 Fabien COELHO : > > However, ISTM that it is not the purpose of pgbench documentation to be a >>> primer about what is an exponential or gaussian distribution, so the idea >>> would yet be to have a relatively compact explanation, and that the >>> interested but clueless

Re: [HACKERS] gaussian distribution pgbench

2014-07-17 Thread Fabien COELHO
However, ISTM that it is not the purpose of pgbench documentation to be a primer about what is an exponential or gaussian distribution, so the idea would yet be to have a relatively compact explanation, and that the interested but clueless reader would document h..self from wikipedia or a text b

Re: [HACKERS] gaussian distribution pgbench

2014-07-17 Thread Robert Haas
On Wed, Jul 16, 2014 at 12:57 AM, Fabien COELHO wrote: >> Well, I think the feedback has been pretty clear, honestly. Here's >> what I'm unhappy about: I can't understand what these options are >> actually doing. > > We can try to improve the documentation, once more! > > However, ISTM that it is

Re: [HACKERS] gaussian distribution pgbench -- part 1/2

2014-07-16 Thread Fabien COELHO
pgbench with gaussian & exponential, part 1 of 2. This patch is a subset of the previous patch which only adds the two new \setrandom gaussian and exponantial variants, but not the adapted pgbench test cases, as suggested by Fujii Masao. There is no new code nor code changes. The corresponding

Re: [HACKERS] gaussian distribution pgbench

2014-07-15 Thread Fabien COELHO
Hello Robert, Well, I think the feedback has been pretty clear, honestly. Here's what I'm unhappy about: I can't understand what these options are actually doing. We can try to improve the documentation, once more! However, ISTM that it is not the purpose of pgbench documentation to be a p

Re: [HACKERS] gaussian distribution pgbench

2014-07-14 Thread Robert Haas
On Sun, Jul 13, 2014 at 2:27 AM, Mitsumasa KONDO wrote: > I still agree with Fabien-san. I cannot understand why our logical proposal > isn't accepted... Well, I think the feedback has been pretty clear, honestly. Here's what I'm unhappy about: I can't understand what these options are actually

Re: [HACKERS] gaussian distribution pgbench

2014-07-12 Thread Mitsumasa KONDO
Hi, 2014-07-04 19:05 GMT+09:00 Andres Freund : > On 2014-07-04 11:59:23 +0200, Fabien COELHO wrote: > > > > >Yea. I certainly disagree with the patch in it's current state because > it > > >copies the same 15 lines several times with a two word difference. > > >Independent of whether we want thos

Re: [HACKERS] gaussian distribution pgbench

2014-07-04 Thread Andres Freund
On 2014-07-04 11:59:23 +0200, Fabien COELHO wrote: > > >Yea. I certainly disagree with the patch in it's current state because it > >copies the same 15 lines several times with a two word difference. > >Independent of whether we want those options, I don't think that's going > >to fly. > > I like

Re: [HACKERS] gaussian distribution pgbench

2014-07-04 Thread Fabien COELHO
Yea. I certainly disagree with the patch in it's current state because it copies the same 15 lines several times with a two word difference. Independent of whether we want those options, I don't think that's going to fly. I liked a simple static string for the different variants, which means

Re: [HACKERS] gaussian distribution pgbench

2014-07-03 Thread Andres Freund
On 2014-07-03 21:27:53 +0900, Fujii Masao wrote: > > Add drawing random integers with a Gaussian or truncated exponentional > > distributions to pgbench. > > > > Test variants with these distributions are also provided and triggered > > with options "--gaussian=..." and "--exponential=...". > > II

Re: [HACKERS] gaussian distribution pgbench

2014-07-03 Thread Fujii Masao
On Wed, Jul 2, 2014 at 6:05 PM, Fabien COELHO wrote: > > Hello Mitsumasa-san, > >> And I'm also interested in your "decile percents" output like under >> followings, >> decile percents: 39.6% 24.0% 14.6% 8.8% 5.4% 3.3% 2.0% 1.2% 0.7% 0.4% > > > Sure, I'm really fine with that. > > >> I think that

Re: [HACKERS] gaussian distribution pgbench

2014-07-03 Thread Gavin Flower
On 03/07/14 20:58, Fabien COELHO wrote: Hello Gavin, decile percents: 69.9% 21.0% 6.3% 1.9% 0.6% 0.2% 0.1% 0.0% 0.0% 0.0% probability of fist/last percent of the range: 11.3% 0.0% I would suggest that probabilities should NEVER be expressed in percentages! As a percentage probability look

Re: [HACKERS] gaussian distribution pgbench

2014-07-03 Thread Fabien COELHO
Hello Gavin, decile percents: 69.9% 21.0% 6.3% 1.9% 0.6% 0.2% 0.1% 0.0% 0.0% 0.0% probability of fist/last percent of the range: 11.3% 0.0% I would suggest that probabilities should NEVER be expressed in percentages! As a percentage probability looks weird, and is never used for serious s

Re: [HACKERS] gaussian distribution pgbench

2014-07-02 Thread Gavin Flower
On 02/07/14 21:05, Fabien COELHO wrote: Hello Mitsumasa-san, And I'm also interested in your "decile percents" output like under followings, decile percents: 39.6% 24.0% 14.6% 8.8% 5.4% 3.3% 2.0% 1.2% 0.7% 0.4% Sure, I'm really fine with that. I think that it is easier than before. Sum of d

Re: [HACKERS] gaussian distribution pgbench

2014-07-02 Thread Fabien COELHO
I have just updated the wording so that it may be clearer: Oops, I have sent the wrong patch, without the wording fix. Here is the real updated version, which I tested. probability of fist/last percent of the range: 11.3% 0.0% -- Fabien.diff --git a/contrib/pgbench/pgbench.c b/contrib/p

Re: [HACKERS] gaussian distribution pgbench

2014-07-02 Thread Fabien COELHO
Hello Mitsumasa-san, And I'm also interested in your "decile percents" output like under followings, decile percents: 39.6% 24.0% 14.6% 8.8% 5.4% 3.3% 2.0% 1.2% 0.7% 0.4% Sure, I'm really fine with that. I think that it is easier than before. Sum of decile percents is just 100%. That's a

Re: [HACKERS] gaussian distribution pgbench

2014-06-17 Thread Mitsumasa KONDO
Hello Fabien-san, I have checked your v13 patch, and tested the new exponential distribution generating algorithm. It works fine and less or no overhead than previous version. Great work! And I agree with your proposal. And I'm also interested in your "decile percents" output like under following

Re: [HACKERS] gaussian distribution pgbench

2014-04-01 Thread Fabien COELHO
Please find attached an updated version "v13" for this patch. I have (I hope) significanlty improved the documentation, including not so helpful mathematical explanation about the actual meaning of the threshold value. If a native English speaker could check the documentation, it would be nic

Re: [HACKERS] gaussian distribution pgbench

2014-03-18 Thread Heikki Linnakangas
On 03/18/2014 11:57 AM, KONDO Mitsumasa wrote: I think that this feature will be also useful for survey new buffer-replace algorithm and checkpoint strategy, so on. Sure. No doubt about that. If we remove this option, it is really dissapointed.. As long as we get the \setrandom changes in,

Re: [HACKERS] gaussian distribution pgbench

2014-03-18 Thread KONDO Mitsumasa
And I find new useful point of this feature. Under following results are '--gaussian=20' case and '--gaussian=2' case, and postgresql setting is same. > [mitsu-ko@pg-rex31 pgbench]$ ./pgbench -c8 -j4 --gaussian=20 -T30 -P 5 > starting vacuum...end. > progress: 5.0 s, 4285.8 tps, lat 1.860 ms stdde

Re: [HACKERS] gaussian distribution pgbench

2014-03-18 Thread KONDO Mitsumasa
(2014/03/17 23:29), Robert Haas wrote: On Sat, Mar 15, 2014 at 4:50 AM, Mitsumasa KONDO wrote: There are explanations and computations as comments in the code. If it is about the documentation, I'm not sure that a very precise mathematical definition will help a lot of people, and might rather

Re: [HACKERS] gaussian distribution pgbench

2014-03-18 Thread KONDO Mitsumasa
(2014/03/17 22:37), Tom Lane wrote: > KONDO Mitsumasa writes: >> (2014/03/17 18:02), Heikki Linnakangas wrote: >>> On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: >>> There is an infinite number of variants of the TPC-B test that we could >>> include >>> in pgbench. If we start adding every one of

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Robert Haas
On Sat, Mar 15, 2014 at 4:50 AM, Mitsumasa KONDO wrote: >> There are explanations and computations as comments in the code. If it is >> about the documentation, I'm not sure that a very precise mathematical >> definition will help a lot of people, and might rather hinder understanding, >> so the d

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Tom Lane
KONDO Mitsumasa writes: > (2014/03/17 18:02), Heikki Linnakangas wrote: >> On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: >> There is an infinite number of variants of the TPC-B test that we could >> include >> in pgbench. If we start adding every one of them, we're quickly going to have >> hundr

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Fujii Masao
On Mon, Mar 17, 2014 at 7:07 PM, KONDO Mitsumasa wrote: > (2014/03/17 18:02), Heikki Linnakangas wrote: >> >> On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: >>> >>> By the way, you seem to want to remove --gaussian=NUM and >>> --exponential=NUM >>> command options. Can you tell me the objective re

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread KONDO Mitsumasa
(2014/03/17 18:02), Heikki Linnakangas wrote: On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: By the way, you seem to want to remove --gaussian=NUM and --exponential=NUM command options. Can you tell me the objective reason? I think pgbench is the benchmark test on PostgreSQL and default benchmar

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread KONDO Mitsumasa
(2014/03/17 17:46), Heikki Linnakangas wrote: On 03/15/2014 08:53 AM, Fabien COELHO wrote: >* Does min and max really make sense for gaussian and exponential >distributions? For gaussian, I would expect mean and standard deviation as >the parameters, not min/max/threshold. Yes... and no:-) The

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Heikki Linnakangas
On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: By the way, you seem to want to remove --gaussian=NUM and --exponential=NUM command options. Can you tell me the objective reason? I think pgbench is the benchmark test on PostgreSQL and default benchmark is TPC-B-like benchmark. It is written in doc

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Heikki Linnakangas
On 03/15/2014 08:53 AM, Fabien COELHO wrote: >* Does min and max really make sense for gaussian and exponential >distributions? For gaussian, I would expect mean and standard deviation as >the parameters, not min/max/threshold. Yes... and no:-) The aim is to draw an integer primary key from a ta

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread KONDO Mitsumasa
Hi Heikki-san, (2014/03/17 14:39), KONDO Mitsumasa wrote: (2014/03/15 15:53), Fabien COELHO wrote: Hello Heikki, A couple of comments: * There should be an explicit "\setrandom ... uniform" option too, even though you get that implicitly if you don't specify the distribution Fix. We can us

Re: [HACKERS] gaussian distribution pgbench

2014-03-16 Thread KONDO Mitsumasa
(2014/03/15 15:53), Fabien COELHO wrote: Hello Heikki, A couple of comments: * There should be an explicit "\setrandom ... uniform" option too, even though you get that implicitly if you don't specify the distribution Indeed. I agree. I suggested it, but it got lost. * What exactly does t

Re: [HACKERS] gaussian distribution pgbench

2014-03-15 Thread Mitsumasa KONDO
2014-03-15 19:04 GMT+09:00 Fabien COELHO : > > Nice drawing! > > > * How about setting the variable as a float instead of integer? Would >>> seem more natural to me. At least as an option. >>> >>> Which variable? The values set by setrandom are mostly used for primary >>> keys. We real

Re: [HACKERS] gaussian distribution pgbench

2014-03-15 Thread Fabien COELHO
Nice drawing! * How about setting the variable as a float instead of integer? Would seem more natural to me. At least as an option. Which variable? The values set by setrandom are mostly used for primary keys. We really want integers in a range. I think he said threshold parameter. Thresh

Re: [HACKERS] gaussian distribution pgbench

2014-03-15 Thread Mitsumasa KONDO
Oh, sorry, I forgot to write URL referring picture. http://en.wikipedia.org/wiki/Normal_distribution http://en.wikipedia.org/wiki/Exponential_distribution regards, -- Mitsumasa KONDO 2014-03-15 17:50 GMT+09:00 Mitsumasa KONDO : > Hi > > 2014-03-15 15:53 GMT+09:00 Fabien COELHO : > > >> Hello H

Re: [HACKERS] gaussian distribution pgbench

2014-03-14 Thread Fabien COELHO
Hello Heikki, A couple of comments: * There should be an explicit "\setrandom ... uniform" option too, even though you get that implicitly if you don't specify the distribution Indeed. I agree. I suggested it, but it got lost. * What exactly does the "threshold" mean? The docs informally e

Re: [HACKERS] gaussian distribution pgbench

2014-03-14 Thread Heikki Linnakangas
On 03/13/2014 04:00 PM, Fujii Masao wrote: On Thu, Mar 13, 2014 at 10:51 PM, Heikki Linnakangas wrote: IMHO we should just implement the \setrandom changes, and not add any of these options to modify the standard test workload. If someone wants to run TPC-B workload with gaussian or exponential

Re: [HACKERS] gaussian distribution pgbench

2014-03-14 Thread Fabien COELHO
Well, when we set '--gaussian=NUM' or '--exponential=NUM' on command line, we can see access probability of top N records in result of final output. This out put is under following, Indeed. I had forgotten this point. This is a significant information that I would not like to loose. This f

Re: [HACKERS] gaussian distribution pgbench

2014-03-13 Thread KONDO Mitsumasa
(2014/03/13 23:00), Fujii Masao wrote: On Thu, Mar 13, 2014 at 10:51 PM, Heikki Linnakangas wrote: On 03/13/2014 03:17 PM, Fujii Masao wrote: On Tue, Mar 11, 2014 at 1:49 PM, KONDO Mitsumasa wrote: (2014/03/09 1:49), Fabien COELHO wrote: I'm okay with this UI and itsaccess probability o

Re: [HACKERS] gaussian distribution pgbench

2014-03-13 Thread KONDO Mitsumasa
Hi, (2014/03/14 4:21), Fabien COELHO wrote: We should do the same discussion for the UI of command-line option? The patch adds two options --gaussian and --exponential, but this UI seems to be a bit inconsistent with the UI for \setrandom. Instead, we can use something like --distribution=[uni

Re: [HACKERS] gaussian distribution pgbench

2014-03-13 Thread Fabien COELHO
We should do the same discussion for the UI of command-line option? The patch adds two options --gaussian and --exponential, but this UI seems to be a bit inconsistent with the UI for \setrandom. Instead, we can use something like --distribution=[uniform | gaussian | exponential]. Hmmm. That

Re: [HACKERS] gaussian distribution pgbench

2014-03-13 Thread Fujii Masao
On Thu, Mar 13, 2014 at 10:51 PM, Heikki Linnakangas wrote: > On 03/13/2014 03:17 PM, Fujii Masao wrote: >> >> On Tue, Mar 11, 2014 at 1:49 PM, KONDO Mitsumasa >> wrote: >>> >>> (2014/03/09 1:49), Fabien COELHO wrote: I'm okay with this UI and its implementation. >>> >>> >>> OK. >>

Re: [HACKERS] gaussian distribution pgbench

2014-03-13 Thread Heikki Linnakangas
On 03/13/2014 03:17 PM, Fujii Masao wrote: On Tue, Mar 11, 2014 at 1:49 PM, KONDO Mitsumasa wrote: (2014/03/09 1:49), Fabien COELHO wrote: I'm okay with this UI and its implementation. OK. We should do the same discussion for the UI of command-line option? The patch adds two options --gau

Re: [HACKERS] gaussian distribution pgbench

2014-03-13 Thread Fujii Masao
On Tue, Mar 11, 2014 at 1:49 PM, KONDO Mitsumasa wrote: > (2014/03/09 1:49), Fabien COELHO wrote: >> >> >> Hello Mitsumasa-san, >> >>> New "\setrandom" interface is here. >>> \setrandom var min max [gaussian threshold | exponential threshold] >> >> >>> Attached patch realizes this interface, but

Re: [HACKERS] gaussian distribution pgbench

2014-03-10 Thread KONDO Mitsumasa
(2014/03/09 1:49), Fabien COELHO wrote: Hello Mitsumasa-san, New "\setrandom" interface is here. \setrandom var min max [gaussian threshold | exponential threshold] Attached patch realizes this interface, but it has little bit ugly codeing in executeStatement() and process_commands().. I

Re: [HACKERS] gaussian distribution pgbench

2014-03-08 Thread Fabien COELHO
Hello Mitsumasa-san, New "\setrandom" interface is here. \setrandom var min max [gaussian threshold | exponential threshold] Attached patch realizes this interface, but it has little bit ugly codeing in executeStatement() and process_commands().. I think it is not too bad. The "ignore ext

Re: [HACKERS] gaussian distribution pgbench

2014-03-06 Thread KONDO Mitsumasa
(2014/03/07 16:02), KONDO Mitsumasa wrote: And other cases are classified under following. \setrandom var min max gaussian #hoge --> uniform Oh, it's wrong... It will be.. \setrandom var min max gaussian #hoge --> ERROR Regards, -- Mitsumasa KONDO NTT Open Source Software Center -- Sent vi

Re: [HACKERS] gaussian distribution pgbench

2014-03-06 Thread KONDO Mitsumasa
Hi, (2014/03/04 17:42), KONDO Mitsumasa wrote:> (2014/03/04 17:28), Fabien COELHO wrote: >>> OK. I'm not sure which idia is the best. So I wait for comments in community:) >> Hmmm. Maybe you can do what Tom voted for, he is the committer:-) > Yeah, but he might change his mind by our disscuttio

Re: [HACKERS] gaussian distribution pgbench

2014-03-04 Thread KONDO Mitsumasa
(2014/03/04 17:28), Fabien COELHO wrote: OK. I'm not sure which idia is the best. So I wait for comments in community:) Hmmm. Maybe you can do what Tom voted for, he is the committer:-) Yeah, but he might change his mind by our disscuttion. So I wait untill tomorrow, and if nothing to comment,

Re: [HACKERS] gaussian distribution pgbench

2014-03-04 Thread Fabien COELHO
OK. I'm not sure which idia is the best. So I wait for comments in community:) Hmmm. Maybe you can do what Tom voted for, he is the committer:-) -- Fabien. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/m

Re: [HACKERS] gaussian distribution pgbench

2014-03-03 Thread KONDO Mitsumasa
(2014/03/03 16:51), Fabien COELHO wrote:>>>\setrandom foo 1 10 [uniform] >>>\setrandom foo 1 :size gaussian 3.6 >>>\setrandom foo 1 100 exponential 7.2 >> It's good design. I think it will become more low overhead at part of parsing >> in pgbench, because comparison of strings will be

Re: [HACKERS] gaussian distribution pgbench

2014-03-02 Thread Fabien COELHO
\setrandom foo 1 10 [uniform] \setrandom foo 1 :size gaussian 3.6 \setrandom foo 1 100 exponential 7.2 It's good design. I think it will become more low overhead at part of parsing in pgbench, because comparison of strings will be redeced(maybe). And I'd like to remove [uniform], beac

Re: [HACKERS] gaussian distribution pgbench

2014-03-02 Thread KONDO Mitsumasa
(2014/03/02 22:32), Fabien COELHO wrote: Alvaro Herrera writes: Seems that in the review so far, Fabien has focused mainly in the mathematical properties of the new random number generation. That seems perfectly fine, but no comment has been made about the chosen UI for the feature. Per the f

Re: [HACKERS] gaussian distribution pgbench

2014-03-02 Thread Fabien COELHO
Hello Alvaro & Tom, Alvaro Herrera writes: Seems that in the review so far, Fabien has focused mainly in the mathematical properties of the new random number generation. That seems perfectly fine, but no comment has been made about the chosen UI for the feature. Per the few initial messages

Re: [HACKERS] gaussian distribution pgbench

2014-03-01 Thread Tom Lane
Alvaro Herrera writes: > Seems that in the review so far, Fabien has focused mainly in the > mathematical properties of the new random number generation. That seems > perfectly fine, but no comment has been made about the chosen UI for the > feature. Per the few initial messages in the thread, i

Re: [HACKERS] gaussian distribution pgbench

2014-03-01 Thread Alvaro Herrera
Seems that in the review so far, Fabien has focused mainly in the mathematical properties of the new random number generation. That seems perfectly fine, but no comment has been made about the chosen UI for the feature. Per the few initial messages in the thread, in the patch as submitted you ask

Re: [HACKERS] gaussian distribution pgbench

2014-02-23 Thread Fabien COELHO
Gaussian Pgbench v8 patch by Mitsumasa KONDO review & patch v9. * The purpose of the patch is to allow a pgbench script to draw from normally distributed or exponentially distributed integer values instead of uniformly distributed. This is a valuable contribution to enable pgbench to gene

Re: [HACKERS] gaussian distribution pgbench

2014-02-17 Thread KONDO Mitsumasa
(2014/02/16 7:38), Fabien COELHO wrote: I have updated the patch (v7) based on Mitsumasa latest v6: - some code simplifications & formula changes. - I've added explicit looping probability computations in comments to show the (low) looping probability of the iterative search. - I

Re: [HACKERS] gaussian distribution pgbench

2014-02-15 Thread Fabien COELHO
Gaussian Pgbench v6 patch by Mitsumasa KONDO review & patch v7. * The purpose of the patch is to allow a pgbench script to draw from normally distributed or exponentially distributed integer values instead of uniformly distributed. This is a valuable contribution to enable pgbench to gene

Re: [HACKERS] gaussian distribution pgbench

2014-02-14 Thread Mitsumasa KONDO
I add exponential distribution random generator (and little bit refactoring:) ). I use inverse transform method to create its distribution. It's very simple method that is created by - log (rand()). We can control slope of distribution using threshold parameter. It is same as gaussian threshold.

Re: [HACKERS] gaussian distribution pgbench

2014-02-13 Thread KONDO Mitsumasa
Sorry, previos attached patch has small bug. Please use latest one. > 134 - return min + (int64) (max - min + 1) * rand; > 134 + return min + (int64)((max - min + 1) * rand); Regards, -- Mitsumasa KONDO NTT Open Source Software Center *** a/contrib/pgbench/pgbench.c --- b/contrib/pgbench/pgbench

Re: [HACKERS] gaussian distribution pgbench

2014-02-13 Thread KONDO Mitsumasa
Hi Febien, Thank you very much for your very detail and useful comments! I read your comment, I agree most of your advice:) Attached patch is fixed for your comment. That are... - Remove redundant long-option. - We can use "--gaussian=NUM -S" or "--gaussian=NUMN -N" options. - Add sentence

Re: [HACKERS] gaussian distribution pgbench

2014-02-09 Thread Fabien COELHO
Hello, I revise my gaussian pgbench patch which wss requested from community. With a lot of delay for which I apologise, please find hereafter the review. Gaussian Pgbench v3 patch by Mitsumasa KONDO review * The purpose of the patch is to allow a pgbench script to draw from normally di

Re: [HACKERS] gaussian distribution pgbench

2013-12-19 Thread Gregory Smith
On 12/19/13 5:52 PM, Gavin Flower wrote: Curious, wouldn't the common usage pattern tend to favour a skewed distribution, such as the Poisson Distribution (it has been over 40 years since I studied this area, so there may be better candidates). Some people like database load testing with a

Re: [HACKERS] gaussian distribution pgbench

2013-12-19 Thread Gavin Flower
On 20/12/13 09:36, Peter Geoghegan wrote: On Thu, Nov 21, 2013 at 9:13 AM, Heikki Linnakangas wrote: So what I'd actually like to see is \setgaussian, for use in custom scripts. +1. I'd really like to be able to run a benchmark with a Gaussian and uniform distribution side-by-side for comparat

Re: [HACKERS] gaussian distribution pgbench

2013-12-19 Thread Peter Geoghegan
On Thu, Nov 21, 2013 at 9:13 AM, Heikki Linnakangas wrote: > So what I'd actually like to see is \setgaussian, for use in custom scripts. +1. I'd really like to be able to run a benchmark with a Gaussian and uniform distribution side-by-side for comparative purposes - we need to know that we're n

Re: [HACKERS] gaussian distribution pgbench

2013-11-22 Thread Fabien COELHO
3. That said, this could be handy. But it would be even more handy if you could get Gaussian random numbers with \setrandom, so that you could use this with custom scripts. And once you implement that, do we actually need the -g flag anymore? If you want TPC-B transactions with gaussian distri

Re: [HACKERS] gaussian distribution pgbench

2013-11-21 Thread Heikki Linnakangas
On 30.09.2013 07:12, KONDO Mitsumasa wrote: (2013/09/27 5:29), Peter Eisentraut wrote: This patch no longer applies. I will try to create this patch in next commit fest. If you have nice idea, please send me! A few thoughts on this: 1. DBT-2 uses a non-uniform distribution. You can use that

Re: [HACKERS] gaussian distribution pgbench

2013-09-29 Thread KONDO Mitsumasa
(2013/09/27 5:29), Peter Eisentraut wrote: > This patch no longer applies. I will try to create this patch in next commit fest. If you have nice idea, please send me! Regards, -- Mitsumasa KONDO NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] gaussian distribution pgbench

2013-09-29 Thread KONDO Mitsumasa
Sorry for my delay reply. Since I have had vacation last week, I replied from gmail. However, it was stalled post to pgsql-hackers:-( (2013/09/21 7:54), Fabien COELHO wrote: However this pattern induces stronger cache effects which are maybe not too realistic, because neighboring keys in the mi

Re: [HACKERS] gaussian distribution pgbench

2013-09-29 Thread KONDO Mitsumasa
Sorry for my delay reply. Since I have had vacation last week, I replyed from gmail. However, it was stalled post to pgsql-hackers:-( (2013/09/21 6:05), Kevin Grittner wrote: > You had accidentally added to the CF In Progress. Oh, I had completely mistook this CF schedule :-) Maybe, Horiguchi-san

Re: [HACKERS] gaussian distribution pgbench

2013-09-26 Thread Peter Eisentraut
On 9/20/13 2:42 AM, KONDO Mitsumasa wrote: > I create gaussinan distribution pgbench patch that can access records with > gaussian frequency. And I submit this commit fest. This patch no longer applies. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to yo

Re: [HACKERS] gaussian distribution pgbench

2013-09-23 Thread Mitsumasa KONDO
> However this pattern induces stronger cache effects which are maybe not too realistic, > because neighboring keys in the middle are more likely to be chosen. I think that your opinion is right. However, in effect, it is a paseudo-benchmark, so that I think that such a simple mechanism is also n

Re: [HACKERS] gaussian distribution pgbench

2013-09-23 Thread Mitsumasa KONDO
> You had accidentally added to the CF In Progress. Oh, I had completely mistook this CF schedule :-) Maybe, Horiguchi-san is same situation... However, because of your moving, I become first submitter in next CF. Thank you for moving :-) -- Mitsumasa KONDO

Re: [HACKERS] gaussian distribution pgbench

2013-09-20 Thread Fabien COELHO
Hello Mitsumasa, In the general transaction situation, clients access for all records equally is hard to happen. I think gaussian distribution access patterns are most of transaction petterns in general. My patch realizes neary this access pattern. That is great! I was just looking for someth

Re: [HACKERS] gaussian distribution pgbench

2013-09-20 Thread Kevin Grittner
KONDO Mitsumasa wrote: > I create gaussinan distribution pgbench patch that can access > records with gaussian frequency. And I submit this commit fest. Thanks! I have moved this to the Open CommitFest, though. https://commitfest.postgresql.org/action/commitfest_view/open You had accidentally