Re: [HACKERS] KNN-GiST with recheck

2014-08-04 Thread Alexander Korotkov
use of not taking into account the cost of sorting. > Cost estimation of GiST is a big problem anyway. It doesn't care (and can't) about amount of recheck for regular operators. In this patch, same would be for knn recheck. The problem is that touching heap from access method breaks incaps

Re: [HACKERS] Distance from point to box

2014-07-30 Thread Alexander Korotkov
hm which works for polygons, and they use it for boxes? Yeah, this answers question #1, but not #2 and #3 :) -- With best regards, Alexander Korotkov.

Re: [HACKERS] Distance from point to box

2014-07-30 Thread Alexander Korotkov
iss something? Thanks. >> > > ISTM that you miss the projection on the segment if dx=0 or dy=0. I don't need to find projection itself, I need only distance. When dx = 0 then nearest point is on horizontal line of box, so distance to it is dy. Same when dy = 0. When both of them are 0 then point is in the box. -- With best regards, Alexander Korotkov.

[HACKERS] Distance from point to box

2014-07-30 Thread Alexander Korotkov
;low.y) dy = box->low.y - point->y; if (point->y > box->high.y) dy = point->y - box->high.y; return HYPOT(dx, dy); I feel myself quite tangled. Could anybody clarify it for me? Did I miss something? Thanks. -- With best regards, Alexander Korotkov.

Re: [HACKERS] 9.5 CF1

2014-07-02 Thread Alexander Korotkov
ith recheck. Also, I think partial sort need a look of somebody more aware of planner than me and Marti. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Negative imact of maintenance_work_mem to GIN size

2014-05-20 Thread Alexander Korotkov
On Tue, May 20, 2014 at 4:50 AM, Alexander Korotkov wrote: > I found that sometimes larger maintenance_work_mem leads to larger GIN > index. That is quite strange. ISTM that it's related to posting lists > compression but I can't figure out how exactly it is. > It appear

[HACKERS] Negative imact of maintenance_work_mem to GIN size

2014-05-19 Thread Alexander Korotkov
| Table | Size | Description +-+---++---++- public | js_idx1 | index | smagen | js| 432 MB | public | js_idx2 | index | smagen | js| 309 MB | (2 rows) -- With best regards, Alexander Korotkov.

[HACKERS] Lossy bitmap scan is broken in GIN

2014-05-10 Thread Alexander Korotkov
─── 99916 (1 row) Time: 63,142 ms set work_mem = '64kB'; select count(*) from test where v > 0.9; The last query hangs. I've debugged it. It's another bug in this "cursed" loop in entryGetItem. Fix is attached. -- With best regards, Alexander Korotkov. gin_l

Re: [HACKERS] [COMMITTERS] pgsql: Clean up jsonb code.

2014-05-09 Thread Alexander Korotkov
04b18 in block 0x14a7e6000 CONTEXT: COPY js, line 1009820 WARNING: problem in alloc set ExprContext: bogus aset link in block 0x14a7e6000, chunk 0x14a804b18 CONTEXT: COPY js, line 1009820 server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. You can get dataset here: http://mira.sai.msu.ru/~megera/tmp/js.copy.gz -- With best regards, Alexander Korotkov.

Re: [HACKERS] Sequential disk access during VACUUM for GiST/GIN

2014-05-06 Thread Alexander Korotkov
function read buffer to shared memory (if it isn't already) and "pins" it. -- With best regards, Alexander Korotkov.

Re: [HACKERS] 9.4 Proposal: Initdb creates a single table

2014-04-23 Thread Alexander Korotkov
ns: get_postgres(id intgeger) returns jsonb, set_postgres(id integer, data jsonb) returns void, search_postgres(query jsonb) returns setof postgres. search_postgres will have semantics of @> jsonb operator 3) Background workers which provides HTTP wrapper over those functions. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Partial match fix for fast scan

2014-04-10 Thread Alexander Korotkov
On Thu, Apr 10, 2014 at 8:22 PM, Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > > On Thu, Apr 10, 2014 at 11:09 AM, Alexander Korotkov > wrote: > >> Hi, >> >> GIN partial match appears to be broken after fast scan. Following simple >> test c

[HACKERS] Partial match fix for fast scan

2014-04-10 Thread Alexander Korotkov
* from test where val between 0.1 and 0.9; Attached patch fixes bugs in entryGetItem function. I would especially point that "continue;" checks "while" condition even if it's postfix "while". That's why I surrounded tbm_iterate with another "while&qu

Re: default opclass for jsonb (was Re: [HACKERS] Call for GIST/GIN/SP-GIST opclass documentation)

2014-04-09 Thread Alexander Korotkov
n without json/hstore (user have to rewrite SQL in order to use expression index), but now it could become very common. My opinion is that we have to do something in planner to make it understand at least this two kinds of queries to be equivalent. -- With best regards, Alexander Korotkov.

Re: default opclass for jsonb (was Re: [HACKERS] Call for GIST/GIN/SP-GIST opclass documentation)

2014-04-09 Thread Alexander Korotkov
; keys. > > I don't understand how we ended up with the current dichotomy of json_ops > and json_hash_ops... +1 for parameterizable opclasses -- With best regards, Alexander Korotkov.

Re: [HACKERS] GSoC 2014 proposal

2014-04-03 Thread Alexander Korotkov
On Thu, Apr 3, 2014 at 11:21 PM, Heikki Linnakangas wrote: > On 04/03/2014 04:15 PM, Alexander Korotkov wrote: > >> On Wed, Apr 2, 2014 at 2:22 PM, Alexander Korotkov > >wrote: >> >> On Tue, Apr 1, 2014 at 2:23 PM, Heikki Linnakangas < >>> hlinnaka

Re: [HACKERS] GSoC 2014 proposal

2014-04-03 Thread Alexander Korotkov
On Wed, Apr 2, 2014 at 2:22 PM, Alexander Korotkov wrote: > On Tue, Apr 1, 2014 at 2:23 PM, Heikki Linnakangas < > hlinnakan...@vmware.com> wrote: > >> The BIRCH algorithm as described in the paper describes building a tree >> in memory. If I understood correctly, you

Re: [HACKERS] GSoC 2014 proposal

2014-04-02 Thread Alexander Korotkov
o take a look at SP-GiST; SP-GiST is more > strict in where in the tree an item can be stored, and lets the operator > class to specify exactly when a node is split etc. > Hmmm, it's likely I've imagined something quite outside of this paper, and even already suggested it to Ivan... :) I need a little time to rethink it. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Cube extension kNN support

2014-03-31 Thread Alexander Korotkov
cab and chebyshev distances. > > > > What is the status of this patch? > > Referring to our private conversation with Alexander Korotkov, the > patch is in WIP state currently, and, hopefully, will be ready by 9.5. > I'm ready to actively participate in its testing on a real w

[HACKERS] Proposal: fix range queries in btree_gin

2014-03-28 Thread Alexander Korotkov
operator. 3) opclass supports "col <@ range" then rewrite this two restrictions as "col <@ range(const1, const2)" -- With best regards, Alexander Korotkov.

Re: [HACKERS] gsoc knn spgist

2014-03-25 Thread Alexander Korotkov
nd yes, it can be used in SP-GiST. However, alternative is heap (in-memory structure, not table heap). I don't know why GiST doesn't use heap instead of RB-tree. With best regards, Alexander Korotkov.

Re: [HACKERS] jsonb and nested hstore

2014-03-20 Thread Alexander Korotkov
er corresponding query no more. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GSoC proposal. Index-only scans for GIST

2014-03-18 Thread Alexander Korotkov
Josh, Anastasia has already consulted to me in person. It is not big proposal. But for newbie who is not familiar with PostgreSQL code base and especially GiST it seems fair enough. With best regards, Alexander Korotkov. On Tue, Mar 18, 2014 at 9:16 PM, Josh Berkus wrote: > Alexan

Re: [HACKERS] GSoC proposal. Index-only scans for GIST

2014-03-18 Thread Alexander Korotkov
ndex only scans into gistcostestimate function > Also, another part of work to be mentioned is to implement fetch function for all suitable opclasses. With best regards, Alexander Korotkov.

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-15 Thread Alexander Korotkov
e WAL > than in the old format. > > That could be optimized, but I figured we can live with it, thanks to the > fastupdate feature. Fastupdate allows amortizing that cost over several > insertions. But of course, you explicitly disabled that... Let me know if you want me to write patch addressing this issue. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-03-13 Thread Alexander Korotkov
On Thu, Mar 13, 2014 at 8:58 PM, Heikki Linnakangas wrote: > On 03/12/2014 07:52 PM, Alexander Korotkov wrote: > >> > >>> >* I just noticed that the dummy trueTriConsistentFn returns GIN_MAYBE, >>> >rather than GIN_TRUE. The equivalent boolean version retu

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Alexander Korotkov
ue like zero byte at the end. 2) jsonb_hash_ops can be extended to handle keys queries too. We can preserve one bit in hash as flag indicating whether it's a hash of key or hash of path to value. For sure, such index would be a bit larger. Also, jsonb_hash_ops can be split into two: with and without keys. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-03-12 Thread Alexander Korotkov
On Wed, Mar 12, 2014 at 8:02 PM, Heikki Linnakangas wrote: > On 02/26/2014 11:25 PM, Alexander Korotkov wrote: > >> On Thu, Feb 27, 2014 at 1:07 AM, Alexander Korotkov > >wrote: >> >> On Thu, Feb 20, 2014 at 1:48 PM, Heikki Linnakangas < >>> hlinnak

Re: [HACKERS] GIN improvements part2: fast scan

2014-03-12 Thread Alexander Korotkov
ce somehow take fast scan into account. Preparation we do in startScanKey requires knowledge of estimate size of posting lists/trees. We do this estimate by traversal to leaf pages. I think gincostestimate is expected to be way more cheap. So, we probably need so more rough estimate there, don't we? -- With best regards, Alexander Korotkov.

Re: [HACKERS] jsonb and nested hstore

2014-03-11 Thread Alexander Korotkov
On Tue, Mar 11, 2014 at 5:19 AM, Peter Geoghegan wrote: > On Mon, Mar 10, 2014 at 4:19 AM, Alexander Korotkov > wrote: > > Here it is. > > So it looks like what you have here is analogous to the other problems > that I fixed with both GiST and GIN. That isn't surprisi

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Alexander Korotkov
On Mon, Mar 10, 2014 at 3:04 PM, Peter Geoghegan wrote: > On Mon, Mar 10, 2014 at 3:47 AM, Alexander Korotkov > wrote: > > Fix is attached. > > Could you post a patch with regression tests, please? > Here it is. -- With best regards, Alexander Korotkov. jsonb

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Alexander Korotkov
On Mon, Mar 10, 2014 at 2:19 PM, Peter Geoghegan wrote: > On Mon, Mar 10, 2014 at 3:00 AM, Alexander Korotkov > wrote: > > I din't get comment about "leftmost" element. There is absolutely no > > distinguish between array elements. All elements are extracted

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Alexander Korotkov
ce I wrote hstore_hash_ops. Could you share test case to illustrate what you mean? -- With best regards, Alexander Korotkov.

Re: [HACKERS] trgm regex index peculiarity

2014-03-01 Thread Alexander Korotkov
On Mon, Feb 10, 2014 at 1:01 AM, Tom Lane wrote: > Alexander Korotkov writes: > > On Thu, Jan 16, 2014 at 3:34 AM, Tom Lane wrote: > >> I looked at this patch a bit. It seems like this: > >> + *BLANK_COLOR_SIZE - How much blank charac

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-26 Thread Alexander Korotkov
On Thu, Feb 27, 2014 at 1:07 AM, Alexander Korotkov wrote: > On Thu, Feb 20, 2014 at 1:48 PM, Heikki Linnakangas < > hlinnakan...@vmware.com> wrote: > >> On 02/09/2014 12:11 PM, Alexander Korotkov wrote: >> >>> I've rebased catalog changes with last mas

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-26 Thread Alexander Korotkov
On Thu, Feb 20, 2014 at 1:48 PM, Heikki Linnakangas wrote: > On 02/09/2014 12:11 PM, Alexander Korotkov wrote: > >> I've rebased catalog changes with last master. Patch is attached. I've >> rerun my test suite with both last master ('committed') and

Re: [HACKERS] PoC: Partial sort

2014-02-19 Thread Alexander Korotkov
it does any overhead, so we can leave it as is. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Small GIN optimizations (after 9.4)

2014-02-11 Thread Alexander Korotkov
On Wed, Feb 12, 2014 at 2:58 AM, Bruce Momjian wrote: > On Sun, Feb 9, 2014 at 02:17:12PM +0400, Alexander Korotkov wrote: > > On Thu, Feb 6, 2014 at 8:31 PM, PostgreSQL - Hans-J rgen Sch nig < > > postg...@cybertec.at> wrote: > > > > i think there is one

Re: [HACKERS] GSoC 2014 - mentors, students and admins

2014-02-10 Thread Alexander Korotkov
ing this year? And are there any project > ideas folk would like to suggest? > I would like to be mentor. -- With best regards, Alexander Korotkov.

Re: [HACKERS] PoC: Partial sort

2014-02-10 Thread Alexander Korotkov
On Mon, Feb 10, 2014 at 2:33 PM, Marti Raudsepp wrote: > On Sun, Feb 9, 2014 at 7:37 PM, Alexander Korotkov > wrote: > > This is not only place that worry me about planning overhead. See > > get_cheapest_fractional_path_for_pathkeys. I had to estimate number of > >

Re: [HACKERS] trgm regex index peculiarity

2014-02-09 Thread Alexander Korotkov
On Thu, Jan 16, 2014 at 3:34 AM, Tom Lane wrote: > Alexander Korotkov writes: > > Revised version of patch with necessary comments. > > I looked at this patch a bit. It seems like this: > > + *BLANK_COLOR_SIZE - How much blank character

Re: [HACKERS] PoC: Partial sort

2014-02-09 Thread Alexander Korotkov
t; Seq Scan on test2 t2 (cost=0.00..15406.00 rows=100 width=12) (11 rows) I don't compare actual execution times because I didn't wait until first plan execution ends up :-) But anyway costs are extraordinary and inner sequential scan of 100 rows is odd. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Small GIN optimizations (after 9.4)

2014-02-09 Thread Alexander Korotkov
uce memory usage of a > scan. > I would like to add another one as continue of fast-scan: * Skip scanning of some entries at all forcing recheck instead. Correct decision should be done based on costs. However, I'm not sure about design. Because it's like a planning feature. How correct to do this inside of GIN? -- With best regards, Alexander Korotkov.

Re: [HACKERS] Small GIN optimizations (after 9.4)

2014-02-09 Thread Alexander Korotkov
nsive one. > if the tsearch_query yields a high number of rows (which it often does) > the subsequent sort will kill you. > This is not intended to be a small change. However, some solution might be possible in post 9.4 gin improvements or in new secret indexing project which will be presented at PGCon :-) -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-09 Thread Alexander Korotkov
On Fri, Feb 7, 2014 at 5:33 PM, Heikki Linnakangas wrote: > On 02/06/2014 01:22 PM, Alexander Korotkov wrote: > >> Difference is very small. For me, it looks ready for commit. >> > > Great, committed! > > Now, to review the catalog changes... I've rebased cata

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-06 Thread Alexander Korotkov
On Thu, Feb 6, 2014 at 2:21 PM, Heikki Linnakangas wrote: > On 02/05/2014 12:42 PM, Alexander Korotkov wrote: > >> Attached patch is "light" version of fast scan. It does extra consistent >> function calls only on startScanKey, no extra calls during scan of the >

Re: [HACKERS] Fix picksplit with nan values

2014-02-05 Thread Alexander Korotkov
On Sat, Feb 1, 2014 at 7:50 AM, Bruce Momjian wrote: > > Where are we on this? I found myself to have empty draft letter from November with new version of patch attached. I'll return here when we have some solution in gin fast scan challenge. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-05 Thread Alexander Korotkov
On Wed, Feb 5, 2014 at 1:23 AM, Alexander Korotkov wrote: > On Mon, Feb 3, 2014 at 6:31 PM, Alexander Korotkov > wrote: > >> On Mon, Jan 27, 2014 at 7:30 PM, Alexander Korotkov > > wrote: >> >>> On Mon, Jan 27, 2014 at 2:32 PM, Alexander Korotkov < >>

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-04 Thread Alexander Korotkov
On Mon, Feb 3, 2014 at 6:31 PM, Alexander Korotkov wrote: > On Mon, Jan 27, 2014 at 7:30 PM, Alexander Korotkov > wrote: > >> On Mon, Jan 27, 2014 at 2:32 PM, Alexander Korotkov > > wrote: >> >>> On Sun, Jan 26, 2014 at 8:14 PM, Heikki Linnakangas

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-03 Thread Alexander Korotkov
''initdb'' & ''d'''::tsquery) > >> > Planning time: 0.257 ms > >> > (5 rows) > >> > > >> > 5alpha1-initdb'd is 3 gin entries with different frequencies. > >> > >> Why do you find that strange? The way the query is formed or the way > >> it's > >> evaluated? > >> > >> The query generator certainly is not perfect, so it may produce some > >> strange queries. > >> > > > > I just mean that in this case 3 words doesn't mean 3 gin entries. > > Isn't that expected? I mean, that's what to_tsquery may do, right? > Everything is absolutely correct. :-) It just may be not what do you expect if you aren't getting into details. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-03 Thread Alexander Korotkov
On Mon, Feb 3, 2014 at 7:24 PM, Tomas Vondra wrote: > On 3 Únor 2014, 15:31, Alexander Korotkov wrote: > > > > I found my patch "0005-Ternary-consistent-implementation.patch" to be > > completely wrong. It introduces ternary consistent function to opclass, >

Re: [HACKERS] KNN-GiST with recheck

2014-02-03 Thread Alexander Korotkov
On Tue, Jan 28, 2014 at 9:32 PM, Heikki Linnakangas wrote: > On 01/28/2014 04:12 PM, Alexander Korotkov wrote: > >> On Tue, Jan 28, 2014 at 5:54 PM, Heikki Linnakangas < >> hlinnakan...@vmware.com >> >>> wrote: >>> >> >> 4. (as you men

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-03 Thread Alexander Korotkov
IN_MAYBE until it returns GIN_FALSE. 3) Decide whether to use fast-scan depending on number of TIDs in first m entries and number of TIDs in rest (n-m) entries. Something like number_of_TIDs_in_frist_m_entries * k < number_of_TIDs_in_rest_n-m_entries where k is some quotient. For sure, it rough method, but it should be fast. Without natural implementation of ternary consistent function algorithm will be slightly different: only m values close to n should be checked. I'm going to implement this heuristic against last version of your patch. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-03 Thread Alexander Korotkov
On Mon, Jan 27, 2014 at 7:30 PM, Alexander Korotkov wrote: > On Mon, Jan 27, 2014 at 2:32 PM, Alexander Korotkov > wrote: > >> On Sun, Jan 26, 2014 at 8:14 PM, Heikki Linnakangas < >> hlinnakan...@vmware.com> wrote: >> >>> On 01/26/2014 08:24 AM, Tom

Re: [HACKERS] KNN-GiST with recheck

2014-01-28 Thread Alexander Korotkov
On Tue, Jan 28, 2014 at 5:54 PM, Heikki Linnakangas wrote: > On 01/13/2014 07:17 PM, Alexander Korotkov wrote: > >> Here goes a desription of this patch same as in original thread. >> >> KNN-GiST provides ability to get ordered results from index, but this >> o

Re: [HACKERS] PoC: Partial sort

2014-01-27 Thread Alexander Korotkov
On Tue, Jan 28, 2014 at 7:41 AM, Marti Raudsepp wrote: > On Mon, Jan 27, 2014 at 9:26 PM, Alexander Korotkov > wrote: > > For now, I have attempt to fix extra columns in mergejoin problem. It > would > > be nice if you test it. > > Yes, it solves the test case

Re: [HACKERS] PoC: Partial sort

2014-01-27 Thread Alexander Korotkov
Hi! On Tue, Jan 21, 2014 at 3:24 AM, Marti Raudsepp wrote: > On Tue, Jan 14, 2014 at 5:49 PM, Alexander Korotkov > wrote: > >On Tue, Jan 14, 2014 at 12:54 AM, Marti Raudsepp wrote: > >> I've been trying it out in a few situations. I implemented a new > >>

Re: [HACKERS] GIN improvements part2: fast scan

2014-01-27 Thread Alexander Korotkov
function. > I believe cache consistent function results is quite same as lazy truth-table. I think it's a good option to use with two-state consistent function. However, I don't think it's a reason to refuse from three-state consistent function because number of entries could be large. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-01-27 Thread Alexander Korotkov
the current >> item. I'm not sure if that's worth the cycles, but it seemed like an >> obvious thing to do, now that we have the ternary consistent function. >> > > So, that clearly isn't worth the cycles :-). At least not with an > expensive consistent function; it might be worthwhile if we pre-build the > truth-table, or cache the results of the consistent function. > > Attached is a quick patch to remove that, on top of all the other patches, > if you want to test the effect. Every single change you did in fast scan seems to be reasonable, but testing shows that something went wrong. Simple test with 3 words of different selectivities. After applying your patches: # select count(*) from fts_test where fti @@ plainto_tsquery('english', 'gin index select'); count ─── 627 (1 row) Time: 21,252 ms In original fast-scan: # select count(*) from fts_test where fti @@ plainto_tsquery('english', 'gin index select'); count ─── 627 (1 row) Time: 3,382 ms I'm trying to get deeper into it. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-01-24 Thread Alexander Korotkov
On Thu, Jan 23, 2014 at 8:22 PM, Heikki Linnakangas wrote: > On 01/14/2014 05:35 PM, Alexander Korotkov wrote: > >> Attached version is rebased against last version of packed posting lists. >> > > Thanks! > > I think we're missing a trick with multi-key queries

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-24 Thread Alexander Korotkov
On Fri, Jan 24, 2014 at 1:19 PM, Heikki Linnakangas wrote: > On 01/24/2014 10:53 AM, Alexander Korotkov wrote: > >> OK. What about previous fix in assert? >> > > Ah right, fixed that too now. Good, now my test-suite passed. Results are so. Time of o

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-24 Thread Alexander Korotkov
On Fri, Jan 24, 2014 at 12:50 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 01/24/2014 10:03 AM, Alexander Korotkov wrote: > >> ITSM I found this bug. ginVacuumPostingTreeLeaf re-encodes only some >> segments. Others are not even re-palloced. T

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-24 Thread Alexander Korotkov
On Thu, Jan 23, 2014 at 9:27 AM, Alexander Korotkov wrote: > On Wed, Jan 22, 2014 at 9:28 PM, Heikki Linnakangas < > hlinnakan...@vmware.com> wrote: > >> On 01/22/2014 02:17 PM, Alexander Korotkov wrote: >> >>> We already spent a lot of time with compression. N

Re: [HACKERS] GIN improvements part2: fast scan

2014-01-23 Thread Alexander Korotkov
tsquery('english', 'word2') ORDER BY ts_rank(...) DESC LIMIT :n; Optimizations inside single clause will be provided as separate patch. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-22 Thread Alexander Korotkov
On Wed, Jan 22, 2014 at 9:28 PM, Heikki Linnakangas wrote: > On 01/22/2014 02:17 PM, Alexander Korotkov wrote: > >> We already spent a lot of time with compression. Now we need to figure out >> the result we want see. I spent quite long time debugging varbyte encoding &

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-22 Thread Alexander Korotkov
On Wed, Jan 22, 2014 at 12:30 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 01/22/2014 09:25 AM, Alexander Korotkov wrote: > >> On Wed, Jan 22, 2014 at 1:21 AM, Heikki Linnakangas < >> hlinnakan...@vmware.com >> >>> wrote: >>>

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-21 Thread Alexander Korotkov
t; segments, and compressed back to the compressed format. Finally, the > finished segments are written back to the page, or pages if it had to be > split. > > The same subroutines to disassemble and recompress are also used in vacuum. > > Attached is what I've got now. This is again quite heavily-changed from > the previous version - sorry for repeatedly rewriting this. I'll continue > testing and re-reviewing this tomorrow. Let's clarify where we are. We had quite debugged and tested version with hard-wired varbyte encoding. Now we're reimplementing and debugging segmented version of varbyte encoding in a hope that one day we can easily replace it with something better that meets our restrictions (but we didn't find it already). Is it right? -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-21 Thread Alexander Korotkov
On Tue, Jan 21, 2014 at 4:28 PM, Alexander Korotkov wrote: > I noticed that the gin vacuum redo routine is dead code, except for the >> data-leaf page handling, because we never remove entries or internal nodes >> (page deletion is a separate wal record type). And the da

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-21 Thread Alexander Korotkov
On Mon, Jan 20, 2014 at 10:30 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 01/17/2014 08:49 PM, Alexander Korotkov wrote: > >> On Fri, Jan 17, 2014 at 10:38 PM, Heikki Linnakangas < >> hlinnakan...@vmware.com> wrote: >> >> On 01/

Re: [HACKERS] PoC: Partial sort

2014-01-20 Thread Alexander Korotkov
in tuplesortstate each time. I also adjusted ExecSupportsMarkRestore, ExecMaterializesOutput and ExecMaterializesOutput to make planner believe so. I found path->pathtype to be absolutely never T_Sort. Correct me if I'm wrong. Another changes in this version of patch: 1) Applied patch to don&

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-17 Thread Alexander Korotkov
On Fri, Jan 17, 2014 at 10:38 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 01/17/2014 01:05 PM, Alexander Korotkov wrote: > >> Seems to be fixed in the attached version of patch. >> Another improvement in this version: only left-most segments and further

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-17 Thread Alexander Korotkov
On Wed, Jan 15, 2014 at 10:47 AM, Alexander Korotkov wrote: > On Wed, Jan 15, 2014 at 5:17 AM, Tomas Vondra wrote: > >> On 14.1.2014 00:38, Tomas Vondra wrote: >> > On 13.1.2014 18:07, Alexander Korotkov wrote: >> >> On Sat, Jan 11, 2014 at 6:15 AM, Tomas

Re: [HACKERS] trgm regex index peculiarity

2014-01-15 Thread Alexander Korotkov
On Fri, Jun 21, 2013 at 5:39 PM, Erik Rijkers wrote: > On Fri, June 21, 2013 15:11, Alexander Korotkov wrote: > > On Fri, Jun 21, 2013 at 2:40 PM, Erik Rijkers wrote: > > > >> On Fri, June 21, 2013 05:25, Tom Lane wrote: > >> > "Erik Rijkers" writ

Re: [HACKERS] GIN improvements part2: fast scan

2014-01-14 Thread Alexander Korotkov
On Tue, Jan 14, 2014 at 11:07 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 01/14/2014 05:35 PM, Alexander Korotkov wrote: > >> On Thu, Nov 21, 2013 at 12:14 AM, Alexander Korotkov >> wrote: >> >> Revised version of patch is attached

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-14 Thread Alexander Korotkov
On Wed, Jan 15, 2014 at 5:17 AM, Tomas Vondra wrote: > On 14.1.2014 00:38, Tomas Vondra wrote: > > On 13.1.2014 18:07, Alexander Korotkov wrote: > >> On Sat, Jan 11, 2014 at 6:15 AM, Tomas Vondra >> <mailto:t...@fuzzy.cz>> wrote: > >> > >&

Re: [HACKERS] PoC: Partial sort

2014-01-14 Thread Alexander Korotkov
On Tue, Jan 14, 2014 at 11:16 PM, Marti Raudsepp wrote: > On Tue, Jan 14, 2014 at 5:49 PM, Alexander Korotkov > wrote: > >> I implemented a new > >> enable_partialsort GUC to make it easier to turn on/off > > > I though about such option. Generally not becaus

Re: [HACKERS] PoC: Partial sort

2014-01-14 Thread Alexander Korotkov
-> CTE Scan on sorted_lang l (cost=0.00..156.86 > rows=7843 width=222) > > But even with these limitations, this will easily be the killer feature of > the next release, for me at least. > I see. But I don't think it can be achieved by small changes in planner. Moreover, I didn't check but I think if you remove ordering by r.name you will still not get sorting languages in the inner node. So, this problem is not directly related to partial sort. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-01-14 Thread Alexander Korotkov
On Thu, Nov 21, 2013 at 12:14 AM, Alexander Korotkov wrote: > On Wed, Nov 20, 2013 at 3:06 AM, Alexander Korotkov > wrote: > >> On Fri, Nov 15, 2013 at 11:19 AM, Alexander Korotkov < >> aekorot...@gmail.com> wrote: >> >>> On Fri, Nov 15, 2013 at 12

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-14 Thread Alexander Korotkov
On Tue, Jan 14, 2014 at 12:34 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 01/13/2014 07:07 PM, Alexander Korotkov wrote: > >> I've fixed this bug and many other bug. Now patch passes test suite that >> I've used earlier. The res

Re: [HACKERS] PoC: Partial sort

2014-01-13 Thread Alexander Korotkov
ch on Alexander's patch. Thanks. It's included into attached version of patch. As wall as estimation improvements, more comments and regression tests fix. -- With best regards, Alexander Korotkov. partial-sort-5.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mail

[HACKERS] KNN-GiST with recheck

2014-01-13 Thread Alexander Korotkov
r By: (p <-> '(0.5,0.5)'::point) Total runtime: 0.305 ms (4 rows) -- With best regards, Alexander Korotkov. knn-gist-recheck-1.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-13 Thread Alexander Korotkov
On Sat, Jan 11, 2014 at 6:15 AM, Tomas Vondra wrote: > On 8.1.2014 22:58, Alexander Korotkov wrote: > > Thanks for reporting. Fixed version is attached. > > I've tried to rerun the 'archie' benchmark with the current patch, and > once again I got > >P

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-08 Thread Alexander Korotkov
it > 94b899b829657332bda856ac3f06153d09077bd1) > 2) Created a test table and index > > create table test (a text); > copy test from '/usr/share/dict/words'; > create index test_trgm_idx on test using gin (a gin_trgm_ops); > > 3) Got the following error on a wildca

Re: [HACKERS] PoC: Partial sort

2013-12-22 Thread Alexander Korotkov
On Sat, Dec 14, 2013 at 6:30 PM, Jeremy Harris wrote: > On 14/12/13 12:54, Andres Freund wrote: > >> On 2013-12-14 13:59:02 +0400, Alexander Korotkov wrote: >> >>> Currently when we need to get ordered result from table we have to choose >>> one of two approac

Re: [HACKERS] PoC: Partial sort

2013-12-22 Thread Alexander Korotkov
On Sun, Dec 22, 2013 at 8:12 PM, Martijn van Oosterhout wrote: > On Sun, Dec 22, 2013 at 07:38:05PM +0400, Alexander Korotkov wrote: > > Hi! > > > > Next revision. It expected to do better work with optimizer. It > introduces > > presorted_keys argument of cost

Re: [HACKERS] GIN improvements part 1: additional information

2013-12-20 Thread Alexander Korotkov
On Fri, Dec 20, 2013 at 11:36 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 12/19/2013 03:33 PM, Heikki Linnakangas wrote: > >> On 12/17/2013 12:49 AM, Heikki Linnakangas wrote: >> >>> On 12/17/2013 12:22 AM, Alexander Korotkov wrote: >>

Re: [HACKERS] GIN improvements part 1: additional information

2013-12-20 Thread Alexander Korotkov
believe Heikki mean both differences are encoded as 11, each one is 1. -- With best regards, Alexander Korotkov.

Re: [HACKERS] PoC: Partial sort

2013-12-18 Thread Alexander Korotkov
On Sat, Dec 14, 2013 at 11:47 PM, Andreas Karlsson wrote: > On 12/14/2013 10:59 AM, Alexander Korotkov wrote: > >> This patch allows to use index for order-by if order-by clause and index >> has non-empty common prefix. So, index gives right ordering for first n >> order

Re: [HACKERS] PoC: Partial sort

2013-12-18 Thread Alexander Korotkov
ent. I'm not sure what > is > > actually right design for it. This is what I like to discuss. > > I don't have enough clue about gist to say wether it's the right design, > but it doesn't look wrong to my eyes. It'd probably be useful to export > the knowledge that we are rechecking and how often that happens to the > outside. > While I didn't really look into the patch, I noticed in passing that you > pass a all_dead variable to heap_hot_search_buffer without using the > result - just pass NULL instead, that performs a bit less work. Useful notice, thanks. -- With best regards, Alexander Korotkov.

Re: [HACKERS] PoC: Partial sort

2013-12-18 Thread Alexander Korotkov
On Sat, Dec 14, 2013 at 6:39 PM, Martijn van Oosterhout wrote: > On Sat, Dec 14, 2013 at 06:21:18PM +0400, Alexander Korotkov wrote: > > > Is that actually all that beneficial when sorting with a bog standard > > > qsort() since that doesn't generally benefit from data b

Re: [HACKERS] hstore ng index for <@ ?

2013-12-18 Thread Alexander Korotkov
ikely we can't effectively use GIN for such kind of queries. I believe it's not implemented because wasn't requested yet. Could you share your use-case? -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part 1: additional information

2013-12-18 Thread Alexander Korotkov
On Tue, Dec 17, 2013 at 2:49 AM, Heikki Linnakangas wrote: > On 12/17/2013 12:22 AM, Alexander Korotkov wrote: > >> On Mon, Dec 16, 2013 at 3:30 PM, Heikki Linnakangas < >> hlinnakan...@vmware.com >> >>> wrote: >>> >> >> On 12/12/2013

Re: [HACKERS] GIN improvements part 1: additional information

2013-12-16 Thread Alexander Korotkov
On Mon, Dec 16, 2013 at 3:30 PM, Heikki Linnakangas wrote: > On 12/12/2013 06:44 PM, Alexander Korotkov wrote: > >> I've thought about different algorithms little more. General problem I see >> is online update. We need it while it is typically not covered by >>

Re: [HACKERS] PoC: Partial sort

2013-12-14 Thread Alexander Korotkov
Hi! Thanks for feedback! On Sat, Dec 14, 2013 at 4:54 PM, Andres Freund wrote: > Hi, > > Cool stuff. > > On 2013-12-14 13:59:02 +0400, Alexander Korotkov wrote: > > Currently when we need to get ordered result from table we have to choose > > one of two approaches

Re: [HACKERS] GIN improvements part 1: additional information

2013-12-12 Thread Alexander Korotkov
On Tue, Dec 10, 2013 at 12:26 AM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 12/09/2013 11:34 AM, Alexander Korotkov wrote: > >> On Mon, Dec 9, 2013 at 1:18 PM, Heikki Linnakangas >> wrote: >> >> Even if we use varbyte encoding, I wonder if i

Re: [HACKERS] GIN improvements part 1: additional information

2013-12-10 Thread Alexander Korotkov
On Tue, Dec 10, 2013 at 12:26 AM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 12/09/2013 11:34 AM, Alexander Korotkov wrote: > >> On Mon, Dec 9, 2013 at 1:18 PM, Heikki Linnakangas >> wrote: >> >> Even if we use varbyte encoding, I wonder if i

Re: [HACKERS] GIN improvements part 1: additional information

2013-12-09 Thread Alexander Korotkov
On Mon, Dec 9, 2013 at 1:18 PM, Heikki Linnakangas wrote: > On 12/08/2013 09:56 PM, Alexander Korotkov wrote: > >> On Fri, Nov 29, 2013 at 11:17 PM, Heikki Linnakangas < >> hlinnakan...@vmware.com> wrote: >> >> I'll continue reviewing next week.. >>

Re: [HACKERS] GIN improvements part 1: additional information

2013-12-08 Thread Alexander Korotkov
On Fri, Nov 29, 2013 at 11:17 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 11/29/2013 11:41 AM, Heikki Linnakangas wrote: > >> On 11/28/2013 09:19 AM, Alexander Korotkov wrote: >> >>> On Wed, Nov 27, 2013 at 1:14 AM, Heikki Linnakangas >>>

Re: [HACKERS] Problem with displaying "wide" tables in psql

2013-12-04 Thread Alexander Korotkov
I have to press many keys to find the necessary part of the value. > There is no such problems with the patch. > I think the solution is to provide proposed behaviour as an option. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part 1: additional information

2013-11-27 Thread Alexander Korotkov
On Wed, Nov 27, 2013 at 1:14 AM, Heikki Linnakangas wrote: > On 11/26/13 15:34, Alexander Korotkov wrote: > >> What's your plans about GIN now? I tried to rebase packed posting lists >> with head. But I found that you've changed interface of placeToPage >> func

<    1   2   3   4   5   6   7   8   9   10   >