Yes, it's in the master branch and should be good to go. Thanks, James
On Tue, Feb 25, 2014 at 12:19 AM, Maarten Wullink <[email protected]>wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > Hi James, > > I tried lowering the concurrency paramters but that didn't make any > difference. When do you expect version 3.0 to be available. Does the > master branch contain the 3.0 code? can i use it for testing if it is > stable enough? > > Cheers, > > Maarten > > James Taylor schreef op 24-02-14 22:12: > > Hi Maarten, Which version of Phoenix are you using? How many > > distinct values would be in your dst column versus your src column? > > In Phoenix 3.0, we'll spill to disk if you're aggregating over a > > group by with many distinct values (you'll take a perf hit for > > this, of course), while in Phoenix 2.0 we don't do this. To work > > around this in Phoenix 2.0, you can decrease the amount of > > parallelization used for a query which would bring down the amount > > of memory required. See this[1] blog for more on that, and adjust > > down the phoenix.query.targetConcurrency and > > phoenix.query.maxConcurrency parameters. > > > > Thanks, James > > > > [1] > > http://phoenix-hbase.blogspot.com/2013/02/phoenix-knobs-dials.html > > > > > > On Mon, Feb 24, 2014 at 4:27 AM, Maarten Wullink > > <[email protected]>wrote: > > > > Some more info, the problem only occurs if the query engine is > > using the key to filter on (SERVER FILTER BY FIRST KEY ONLY) > > > > > > 0: jdbc:phoenix:localhost> explain select src, count(*) as qt from > > "queries" group by src order by qt desc limit 10; +------------+ | > > PLAN | +------------+ | CLIENT PARALLEL 30-WAY FULL SCAN OVER > > queries | | SERVER FILTER BY FIRST KEY ONLY | | SERVER > > AGGREGATE INTO DISTINCT ROWS BY [SRC] | | CLIENT MERGE SORT | | > > CLIENT TOP 10 ROWS SORTED BY [COUNT(1) DESC] | > > > > > > This query will run into trouble after 10sec. When using another > > query > > > > 0: jdbc:phoenix:localhost> explain select dst, count(*) as qt from > > "queries" group by dst order by qt desc limit 10; +------------+ | > > PLAN | +------------+ | CLIENT PARALLEL 30-WAY FULL SCAN OVER > > queries | | SERVER AGGREGATE INTO DISTINCT ROWS BY [DST] | | > > CLIENT MERGE SORT | | CLIENT TOP 10 ROWS SORTED BY [COUNT(1) DESC] > > | > > > > > > No such error occurs. > > > > > > Maarten Wullink schreef op 24-02-14 11:05: > >>>> Hi, > >>>> > >>>> I am testing Phoenix on a cloudera virtualbox environment > >>>> with 8gig of total RAM. The RS has been assigned 4gig of RAM. > >>>> I have table with 1146117 rows. When i try do do aggragations > >>>> i get the following exception. It seems to me that there is > >>>> sufficient free memory (41859900) available, so why i am > >>>> getting this exception? > >>>> > >>>> Cheers, > >>>> > >>>> Maarten > >>>> > >>>> java.lang.RuntimeException: > >>>> com.salesforce.phoenix.exception.PhoenixIOException: > >>>> com.salesforce.phoenix.exception.PhoenixIOException: > >>>> org.apache.hadoop.hbase.DoNotRetryIOException: > >>>> > > > queries,\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1392995316456.7476821ab4c43f23c3b4fd1224c454c9.: > >>>> > >>>> > > > > > Requested memory of 2025000 bytes could not be allocated from > >>>> remaining memory of 41859900 bytes from global pool of > >>>> 42500096 bytes after waiting for 10000ms. > >>>> > >>>> > > > >> > > > > - -- > Maarten Wullink | Research Engineer > SIDN | Meander 501 | 6825 MD | Postbus 5022 | 6802 EA | ARNHEM > T +31 (0)26 352 55 45 | M +31 (0)6 21 26 87 55 | F +31 (0)26 352 55 05 > [email protected] | www.sidn.nl > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.22 (Darwin) > Comment: GPGTools - https://gpgtools.org > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > iQEcBAEBCgAGBQJTDFH+AAoJEE8qSVxLG/CLbU0H/2lykfdP6liinF9yu0gKvZDM > SJeNs3j9CTgvBdzMbGVATZaqjOwD/2HGEGb0Ef8deKo2As1VOoxAq1HNPpJY8JyU > muztq6x2xhRYRbgIOAwq+LCvIkp9v+dMeL67y2wokG43ita8oDnTUuFnZraE1Bb6 > KBwu5gfgpLHgAchRXYCOEJxZHXMGAIeZa/SzTb3Z6iMtAy6fm54iH1H+eKpZVYyz > QO48j4y5zl4LLLMvYYS/9NU7W3FyW5UYGkBtpN8Wu3ZntixRHEr0s3QJ+F8EqtMF > luMg7hojx5RmVaYlA/0tYuRazvLyidkTqU6r1rq5av2nl+0gUaIWyoMJBr2gJmA= > =k9WN > -----END PGP SIGNATURE----- >
