Re: [HACKERS] Fillfactor for GIN indexes

2015-01-17 Thread Alexander Korotkov
On Sat, Jan 17, 2015 at 12:49 PM, Alexander Korotkov wrote: > BTW, previous version of patch contained some bugs. Revised version is > attached. > Ooops, it's here. -- With best regards, Alexander Korotkov. gin_fillfactor_5.patch Description: Binary data -- Sent via

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-17 Thread Alexander Korotkov
On Fri, Jan 16, 2015 at 8:40 PM, Robert Haas wrote: > On Thu, Jan 15, 2015 at 7:06 AM, Michael Paquier > wrote: > > Alexander Korotkov wrote: > >> I'm not sure. On the one hand it's unclear why fillfactor should be > >> different from 9.4. > >>

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-15 Thread Alexander Korotkov
On Thu, Jan 15, 2015 at 10:19 AM, Michael Paquier wrote: > On Thu, Jan 8, 2015 at 2:03 PM, Michael Paquier > wrote: > > On Thu, Jan 8, 2015 at 6:31 AM, Alexander Korotkov > wrote: > >> On Wed, Jan 7, 2015 at 4:11 PM, Michael Paquier < > michael.paqu...@gmai

Re: [HACKERS] KNN-GiST with recheck

2015-01-07 Thread Alexander Korotkov
t doesn't get a > recheck argument. It is just assumed that if the consistent function sets > the recheck flag, then the distance needs to be rechecked as well. We might > want to add the recheck argument, like you Alexander did in your patch, but > it's not important right now.

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-07 Thread Alexander Korotkov
On Thu, Jan 8, 2015 at 12:31 AM, Alexander Korotkov wrote: > > Rewritten version of patch is attached. I made following changes: > > 1) I removed fillfactor handling from entry tree. Because in this case > fillfactor parameter would be only upper bound for actual page fullness. &

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-07 Thread Alexander Korotkov
On Wed, Jan 7, 2015 at 4:11 PM, Michael Paquier wrote: > On Wed, Dec 3, 2014 at 2:37 AM, Robert Haas wrote: > > On Fri, Nov 28, 2014 at 4:27 AM, Alexander Korotkov > > wrote: > >> On Fri, Nov 21, 2014 at 8:12 AM, Michael Paquier < > michael.paqu...@gmail.com&

Re: [HACKERS] Commit fest 2014-12, let's begin!

2014-12-15 Thread Alexander Korotkov
On Mon, Dec 15, 2014 at 7:47 PM, Heikki Linnakangas wrote: > > On 12/15/2014 05:22 PM, Alexander Korotkov wrote: > >> On Mon, Dec 15, 2014 at 6:20 PM, Tom Lane wrote: >> >>> >>> Alexander Korotkov writes: >>> >>>> On Mon, Dec

Re: [HACKERS] Commit fest 2014-12, let's begin!

2014-12-15 Thread Alexander Korotkov
On Mon, Dec 15, 2014 at 6:20 PM, Tom Lane wrote: > > Alexander Korotkov writes: > > On Mon, Dec 15, 2014 at 4:12 PM, Heikki Linnakangas < > hlinnakan...@vmware.com > >> wrote: > >> Right. I also looked at it briefly, but I wasn't sure if we really want &

Re: [HACKERS] Commit fest 2014-12, let's begin!

2014-12-15 Thread Alexander Korotkov
;-> 'SRID=26918;POINT(583571.905921312 4506714.34119218)'::geometry LIMIT 100 ) SELECT gid, name FROM closest_candidates ORDER BY ST_Distance( geom, 'SRID=26918;POINT(583571.905921312 4506714.34119218)'::geometry ) LIMIT 1; See blog posts: http://blog.light42.com/wordpress/?p=102 http://workshops.boundlessgeo.com/postgis-intro/knn.html -- With best regards, Alexander Korotkov.

[HACKERS] Report search_path value back to the client.

2014-12-02 Thread Alexander Kukushkin
h order for names that are not schema-qualified."), NULL, - GUC_LIST_INPUT | GUC_LIST_QUOTE + GUC_LIST_INPUT | GUC_LIST_QUOTE | GUC_REPORT }, &namespace_search_path, "\"$user\",public", What do you think? Regards, -- Alexander Kukushkin

Re: [HACKERS] KNN-GiST with recheck

2014-11-28 Thread Alexander Korotkov
On Sat, Nov 22, 2014 at 2:20 AM, Peter Geoghegan wrote: > On Mon, Jan 13, 2014 at 9:17 AM, Alexander Korotkov > wrote: > > This patch was split from thread: > > > http://www.postgresql.org/message-id/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=gon-...@mail.gmail.com >

Re: [HACKERS] Fillfactor for GIN indexes

2014-11-28 Thread Alexander Korotkov
ages are always compressed excepts pg_upgraded indexes from pre-9.4. Take care about it in following code. if (GinPageIsCompressed(page)) freespace = GinDataLeafPageGetFreeSpace(page); + else if (btree->isBuild) + freespace = BLCKSZ * (100 - fillfactor) / 100; -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Access method extendability

2014-11-24 Thread Alexander Korotkov
Hi, Heikki! Thank you for summarizing. In general, I agree with your notes with some exceptions. On Mon, Nov 24, 2014 at 1:52 PM, Heikki Linnakangas wrote: > On 11/10/2014 10:30 PM, Alexander Korotkov wrote: > >> Don't allowing CREATE ACCESS METHOD command seems proble

Re: [HACKERS] WIP: Access method extendability

2014-11-10 Thread Alexander Korotkov
ex as invalid. Don't allowing CREATE ACCESS METHOD command seems problematic for me. How could it work with pg_upgrade? pg_dump wouldn't dump extra pg_am records. So, pg_upgrade would break at creating operator classes on new cluster. So, I agree with dropping create am command only if we let pg_dump to dump extra pg_am records... -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Alexander Korotkov
then external indexing engines would be used by majority of users instead of GIN. -- With best regards, Alexander Korotkov.

[HACKERS] WIP: Access method extendability

2014-10-15 Thread Alexander Korotkov
. Update mode: page is updated only if it's lsn is lower than record lsn Usually, xlog record is filled in critical sections when memory allocations is prohibited. Thus, user have to previously initialize it with knowledge of pages count, total operations count and total length of data.

Re: [HACKERS] KNN-GiST with recheck

2014-09-29 Thread Alexander Korotkov
On Mon, Sep 29, 2014 at 6:16 AM, Bruce Momjian wrote: > On Fri, Sep 26, 2014 at 10:49:42AM +0400, Alexander Korotkov wrote: > > Does this also fix the identical PostGIS problem or is there > something > > PostGIS needs to do? > > > > > > This patch

Re: [HACKERS] KNN-GiST with recheck

2014-09-25 Thread Alexander Korotkov
On Fri, Sep 26, 2014 at 5:18 AM, Bruce Momjian wrote: > On Sun, Sep 14, 2014 at 11:34:26PM +0400, Alexander Korotkov wrote: > > > Cost estimation of GiST is a big problem anyway. It doesn't care > (and > > > can't) about amount of recheck for regula

Re: [HACKERS] KNN-GiST with recheck

2014-09-25 Thread Alexander Korotkov
s could make use of the sorting infrastructure > one day. > * It could be more transparent to the users. Sorting information > could be shown on the explain output. > It would be also nice to show some information about KNN itself. > * A more suitable data structure like binary heap could be used > for the queue to sort the rows. > Binary heap seems to be better data structure for whole KNN-GiST. But it's a subject for a separate patch: replace RB-tree to heap in KNN-GiST. It's not related to recheck stuff. -- With best regards, Alexander Korotkov.

Re: [HACKERS] KNN-GiST with recheck

2014-09-17 Thread Alexander Korotkov
testing. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Collation-aware comparisons in GIN opclasses

2014-09-16 Thread Alexander Korotkov
rm index definition by itself. It calls pg_get_indexdef function. This function have no option to dump names of default opclasses. Since we can't change behaviour of old postgres version, we have to make pg_dump form index definition by itself. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Collation-aware comparisons in GIN opclasses

2014-09-16 Thread Alexander Korotkov
come with a patch for this. > Can you explain it a bit more detail? I didn't get it. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Collation-aware comparisons in GIN opclasses

2014-09-16 Thread Alexander Korotkov
On Mon, Sep 15, 2014 at 11:45 PM, Tom Lane wrote: > Peter Geoghegan writes: > > On Mon, Sep 15, 2014 at 8:28 AM, Alexander Korotkov > > wrote: > >> Rename such opclasses and make them not default. > >> Create new default opclasses with bitwise comparison functi

Re: [HACKERS] Collation-aware comparisons in GIN opclasses

2014-09-15 Thread Alexander Korotkov
On Mon, Sep 15, 2014 at 10:51 PM, Peter Geoghegan wrote: > On Mon, Sep 15, 2014 at 8:28 AM, Alexander Korotkov > wrote: > > some GIN opclasses uses collation-aware comparisons while they don't > need to > > do especially collation-aware comparison. Examples are text[

[HACKERS] Collation-aware comparisons in GIN opclasses

2014-09-15 Thread Alexander Korotkov
_upgrade. I see following solution: 1. Rename such opclasses and make them not default. 2. Create new default opclasses with bitwise comparison functions. 3. Write recommendation to re-create indexes with default opclasses into documentation. -- With best regards, Alexander Korotkov.

[HACKERS] Triconsistent catalog declaration

2014-09-15 Thread Alexander Korotkov
ct to declare them returns int2. -- With best regards, Alexander Korotkov. tri-consistent-catalog.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PoC: Partial sort

2014-09-15 Thread Alexander Korotkov
e. Otherwise, we would miss significant part of LIMIT clause optimization. tuplesort_set_bound() can't solve all the cases. There could be other planner nodes between the partial sort and LIMIT. -- With best regards, Alexander Korotkov.

Re: [HACKERS] PoC: Partial sort

2014-09-15 Thread Alexander Korotkov
On Sun, Sep 14, 2014 at 7:39 AM, Peter Geoghegan wrote: > On Fri, Sep 12, 2014 at 2:19 PM, Alexander Korotkov > wrote: > > Actually, higher cardinality skip columns is better. Sorting of smaller > > groups is faster than sorting larger groups of same size. Also, with > s

Re: [HACKERS] KNN-GiST with recheck

2014-09-14 Thread Alexander Korotkov
follow. I think it deserves more comments. > Fixed, thanks. It was logical error in comparison function implementation. -- With best regards, Alexander Korotkov. knn-gist-recheck-4.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PoC: Partial sort

2014-09-13 Thread Alexander Korotkov
Presorted Key: x Sort Method: quicksort Memory: 25kB -> Index Scan using b_x_idx on b (cost=0.43..338748.13 rows=18 width=12) (actual time=0.047..2237.062 rows=10 loops=1) Filter: (z = 1) Rows Removed by Filter: 990 Planning time:

Re: [HACKERS] PoC: Partial sort

2014-09-12 Thread Alexander Korotkov
On Sun, Jul 13, 2014 at 6:45 AM, Peter Geoghegan wrote: > On Mon, Feb 10, 2014 at 10:59 AM, Alexander Korotkov > wrote: > > Done. Patch is splitted. > > I took a quick look at this. > > Have you thought about making your new cmpSortSkipCols() function not > use rea

Re: [HACKERS] jsonb contains behaviour weirdness

2014-09-12 Thread Alexander Korotkov
ectionable is the inconsistency. I believe that this is Alexander's > concern too. Alexander's first example exhibits broken behavior. > Agree. I just tried to explain how current behaviour could look for user who sees it for the first time. -- With best regards, Alexander Korotkov.

[HACKERS] jsonb contains behaviour weirdness

2014-09-12 Thread Alexander Korotkov
y that "JB_ROOT_COUNT(val) < JB_ROOT_COUNT(tmpl)" should be checked only for objects, not arrays. Also, should JsonbDeepContains does same fast check when it deals with nested objects? -- With best regards, Alexander Korotkov.

Re: [HACKERS] gist vacuum gist access

2014-09-08 Thread Alexander Korotkov
On Mon, Sep 8, 2014 at 12:51 PM, Heikki Linnakangas wrote: > On 09/08/2014 11:19 AM, Alexander Korotkov wrote: > >> On Mon, Sep 8, 2014 at 12:08 PM, Alexander Korotkov > > >> wrote: >> >> On Mon, Sep 8, 2014 at 11:13 AM, Heikki Linnakangas < >>> h

Re: [HACKERS] gist vacuum gist access

2014-09-08 Thread Alexander Korotkov
On Mon, Sep 8, 2014 at 12:08 PM, Alexander Korotkov wrote: > On Mon, Sep 8, 2014 at 11:13 AM, Heikki Linnakangas < > hlinnakan...@vmware.com> wrote: > >> On 09/07/2014 05:11 PM, Костя Кузнецов wrote: >> >>> hello. >>> i recode vacuum for gist index. &

Re: [HACKERS] gist vacuum gist access

2014-09-08 Thread Alexander Korotkov
iST sometimes leaves new page unused due to server crash. Anyway, I'm not fan of committing patch in this shape. We need to let GiST recycle pages first, then implement VACUUM similar to b-tree. -- With best regards, Alexander Korotkov.

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-08-08 Thread Alexander Korotkov
But lengths will probably repeat. -- With best regards, Alexander Korotkov.

Re: [HACKERS] KNN-GiST with recheck

2014-08-04 Thread Alexander Korotkov
use of not taking into account the cost of sorting. > Cost estimation of GiST is a big problem anyway. It doesn't care (and can't) about amount of recheck for regular operators. In this patch, same would be for knn recheck. The problem is that touching heap from access method breaks incaps

Re: [HACKERS] Distance from point to box

2014-07-30 Thread Alexander Korotkov
hm which works for polygons, and they use it for boxes? Yeah, this answers question #1, but not #2 and #3 :) -- With best regards, Alexander Korotkov.

Re: [HACKERS] Distance from point to box

2014-07-30 Thread Alexander Korotkov
iss something? Thanks. >> > > ISTM that you miss the projection on the segment if dx=0 or dy=0. I don't need to find projection itself, I need only distance. When dx = 0 then nearest point is on horizontal line of box, so distance to it is dy. Same when dy = 0. When both of them are 0 then point is in the box. -- With best regards, Alexander Korotkov.

[HACKERS] Distance from point to box

2014-07-30 Thread Alexander Korotkov
;low.y) dy = box->low.y - point->y; if (point->y > box->high.y) dy = point->y - box->high.y; return HYPOT(dx, dy); I feel myself quite tangled. Could anybody clarify it for me? Did I miss something? Thanks. -- With best regards, Alexander Korotkov.

Re: [HACKERS] 9.5 CF1

2014-07-02 Thread Alexander Korotkov
ith recheck. Also, I think partial sort need a look of somebody more aware of planner than me and Marti. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Negative imact of maintenance_work_mem to GIN size

2014-05-20 Thread Alexander Korotkov
On Tue, May 20, 2014 at 4:50 AM, Alexander Korotkov wrote: > I found that sometimes larger maintenance_work_mem leads to larger GIN > index. That is quite strange. ISTM that it's related to posting lists > compression but I can't figure out how exactly it is. > It appear

[HACKERS] Negative imact of maintenance_work_mem to GIN size

2014-05-19 Thread Alexander Korotkov
| Table | Size | Description +-+---++---++- public | js_idx1 | index | smagen | js| 432 MB | public | js_idx2 | index | smagen | js| 309 MB | (2 rows) -- With best regards, Alexander Korotkov.

[HACKERS] Lossy bitmap scan is broken in GIN

2014-05-10 Thread Alexander Korotkov
─── 99916 (1 row) Time: 63,142 ms set work_mem = '64kB'; select count(*) from test where v > 0.9; The last query hangs. I've debugged it. It's another bug in this "cursed" loop in entryGetItem. Fix is attached. -- With best regards, Alexander Korotkov. gin_l

Re: [HACKERS] [COMMITTERS] pgsql: Clean up jsonb code.

2014-05-09 Thread Alexander Korotkov
04b18 in block 0x14a7e6000 CONTEXT: COPY js, line 1009820 WARNING: problem in alloc set ExprContext: bogus aset link in block 0x14a7e6000, chunk 0x14a804b18 CONTEXT: COPY js, line 1009820 server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. You can get dataset here: http://mira.sai.msu.ru/~megera/tmp/js.copy.gz -- With best regards, Alexander Korotkov.

Re: [HACKERS] Sequential disk access during VACUUM for GiST/GIN

2014-05-06 Thread Alexander Korotkov
function read buffer to shared memory (if it isn't already) and "pins" it. -- With best regards, Alexander Korotkov.

Re: [HACKERS] 9.4 Proposal: Initdb creates a single table

2014-04-23 Thread Alexander Korotkov
ns: get_postgres(id intgeger) returns jsonb, set_postgres(id integer, data jsonb) returns void, search_postgres(query jsonb) returns setof postgres. search_postgres will have semantics of @> jsonb operator 3) Background workers which provides HTTP wrapper over those functions. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Partial match fix for fast scan

2014-04-10 Thread Alexander Korotkov
On Thu, Apr 10, 2014 at 8:22 PM, Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > > On Thu, Apr 10, 2014 at 11:09 AM, Alexander Korotkov > wrote: > >> Hi, >> >> GIN partial match appears to be broken after fast scan. Following simple >> test c

[HACKERS] Partial match fix for fast scan

2014-04-10 Thread Alexander Korotkov
* from test where val between 0.1 and 0.9; Attached patch fixes bugs in entryGetItem function. I would especially point that "continue;" checks "while" condition even if it's postfix "while". That's why I surrounded tbm_iterate with another "while&qu

Re: default opclass for jsonb (was Re: [HACKERS] Call for GIST/GIN/SP-GIST opclass documentation)

2014-04-09 Thread Alexander Korotkov
n without json/hstore (user have to rewrite SQL in order to use expression index), but now it could become very common. My opinion is that we have to do something in planner to make it understand at least this two kinds of queries to be equivalent. -- With best regards, Alexander Korotkov.

Re: default opclass for jsonb (was Re: [HACKERS] Call for GIST/GIN/SP-GIST opclass documentation)

2014-04-09 Thread Alexander Korotkov
; keys. > > I don't understand how we ended up with the current dichotomy of json_ops > and json_hash_ops... +1 for parameterizable opclasses -- With best regards, Alexander Korotkov.

Re: [HACKERS] GSoC 2014 proposal

2014-04-03 Thread Alexander Korotkov
On Thu, Apr 3, 2014 at 11:21 PM, Heikki Linnakangas wrote: > On 04/03/2014 04:15 PM, Alexander Korotkov wrote: > >> On Wed, Apr 2, 2014 at 2:22 PM, Alexander Korotkov > >wrote: >> >> On Tue, Apr 1, 2014 at 2:23 PM, Heikki Linnakangas < >>> hlinnaka

Re: [HACKERS] GSoC 2014 proposal

2014-04-03 Thread Alexander Korotkov
On Wed, Apr 2, 2014 at 2:22 PM, Alexander Korotkov wrote: > On Tue, Apr 1, 2014 at 2:23 PM, Heikki Linnakangas < > hlinnakan...@vmware.com> wrote: > >> The BIRCH algorithm as described in the paper describes building a tree >> in memory. If I understood correctly, you

Re: [HACKERS] GSoC 2014 proposal

2014-04-02 Thread Alexander Korotkov
o take a look at SP-GiST; SP-GiST is more > strict in where in the tree an item can be stored, and lets the operator > class to specify exactly when a node is split etc. > Hmmm, it's likely I've imagined something quite outside of this paper, and even already suggested it to Ivan... :) I need a little time to rethink it. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Cube extension kNN support

2014-03-31 Thread Alexander Korotkov
cab and chebyshev distances. > > > > What is the status of this patch? > > Referring to our private conversation with Alexander Korotkov, the > patch is in WIP state currently, and, hopefully, will be ready by 9.5. > I'm ready to actively participate in its testing on a real w

[HACKERS] Proposal: fix range queries in btree_gin

2014-03-28 Thread Alexander Korotkov
operator. 3) opclass supports "col <@ range" then rewrite this two restrictions as "col <@ range(const1, const2)" -- With best regards, Alexander Korotkov.

Re: [HACKERS] gsoc knn spgist

2014-03-25 Thread Alexander Korotkov
nd yes, it can be used in SP-GiST. However, alternative is heap (in-memory structure, not table heap). I don't know why GiST doesn't use heap instead of RB-tree. With best regards, Alexander Korotkov.

Re: [HACKERS] jsonb and nested hstore

2014-03-20 Thread Alexander Korotkov
er corresponding query no more. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GSoC proposal. Index-only scans for GIST

2014-03-18 Thread Alexander Korotkov
Josh, Anastasia has already consulted to me in person. It is not big proposal. But for newbie who is not familiar with PostgreSQL code base and especially GiST it seems fair enough. With best regards, Alexander Korotkov. On Tue, Mar 18, 2014 at 9:16 PM, Josh Berkus wrote: > Alexan

Re: [HACKERS] GSoC proposal. Index-only scans for GIST

2014-03-18 Thread Alexander Korotkov
ndex only scans into gistcostestimate function > Also, another part of work to be mentioned is to implement fetch function for all suitable opclasses. With best regards, Alexander Korotkov.

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-15 Thread Alexander Korotkov
e WAL > than in the old format. > > That could be optimized, but I figured we can live with it, thanks to the > fastupdate feature. Fastupdate allows amortizing that cost over several > insertions. But of course, you explicitly disabled that... Let me know if you want me to write patch addressing this issue. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-03-13 Thread Alexander Korotkov
On Thu, Mar 13, 2014 at 8:58 PM, Heikki Linnakangas wrote: > On 03/12/2014 07:52 PM, Alexander Korotkov wrote: > >> > >>> >* I just noticed that the dummy trueTriConsistentFn returns GIN_MAYBE, >>> >rather than GIN_TRUE. The equivalent boolean version retu

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Alexander Korotkov
ue like zero byte at the end. 2) jsonb_hash_ops can be extended to handle keys queries too. We can preserve one bit in hash as flag indicating whether it's a hash of key or hash of path to value. For sure, such index would be a bit larger. Also, jsonb_hash_ops can be split into two: with and without keys. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-03-12 Thread Alexander Korotkov
On Wed, Mar 12, 2014 at 8:02 PM, Heikki Linnakangas wrote: > On 02/26/2014 11:25 PM, Alexander Korotkov wrote: > >> On Thu, Feb 27, 2014 at 1:07 AM, Alexander Korotkov > >wrote: >> >> On Thu, Feb 20, 2014 at 1:48 PM, Heikki Linnakangas < >>> hlinnak

Re: [HACKERS] GIN improvements part2: fast scan

2014-03-12 Thread Alexander Korotkov
el that the "additional" entries only > need to be fetched when the other keys match. That's doable in principle, > but requires a bunch of extra code. > > Alexander, any thoughts on that? It's getting awfully late to add new code > for that, but it sure would be ni

Re: [HACKERS] jsonb and nested hstore

2014-03-11 Thread Alexander Korotkov
On Tue, Mar 11, 2014 at 5:19 AM, Peter Geoghegan wrote: > On Mon, Mar 10, 2014 at 4:19 AM, Alexander Korotkov > wrote: > > Here it is. > > So it looks like what you have here is analogous to the other problems > that I fixed with both GiST and GIN. That isn't surprisi

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Alexander Korotkov
On Mon, Mar 10, 2014 at 3:04 PM, Peter Geoghegan wrote: > On Mon, Mar 10, 2014 at 3:47 AM, Alexander Korotkov > wrote: > > Fix is attached. > > Could you post a patch with regression tests, please? > Here it is. -- With best regards, Alexander Korotkov. jsonb

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Alexander Korotkov
On Mon, Mar 10, 2014 at 2:19 PM, Peter Geoghegan wrote: > On Mon, Mar 10, 2014 at 3:00 AM, Alexander Korotkov > wrote: > > I din't get comment about "leftmost" element. There is absolutely no > > distinguish between array elements. All elements are extracted

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Alexander Korotkov
ce I wrote hstore_hash_ops. Could you share test case to illustrate what you mean? -- With best regards, Alexander Korotkov.

Re: [HACKERS] trgm regex index peculiarity

2014-03-01 Thread Alexander Korotkov
On Mon, Feb 10, 2014 at 1:01 AM, Tom Lane wrote: > Alexander Korotkov writes: > > On Thu, Jan 16, 2014 at 3:34 AM, Tom Lane wrote: > >> I looked at this patch a bit. It seems like this: > >> + *BLANK_COLOR_SIZE - How much blank charac

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-26 Thread Alexander Korotkov
On Thu, Feb 27, 2014 at 1:07 AM, Alexander Korotkov wrote: > On Thu, Feb 20, 2014 at 1:48 PM, Heikki Linnakangas < > hlinnakan...@vmware.com> wrote: > >> On 02/09/2014 12:11 PM, Alexander Korotkov wrote: >> >>> I've rebased catalog changes with last mas

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-26 Thread Alexander Korotkov
On Thu, Feb 20, 2014 at 1:48 PM, Heikki Linnakangas wrote: > On 02/09/2014 12:11 PM, Alexander Korotkov wrote: > >> I've rebased catalog changes with last master. Patch is attached. I've >> rerun my test suite with both last master ('committed') and

Re: [HACKERS] PoC: Partial sort

2014-02-19 Thread Alexander Korotkov
it does any overhead, so we can leave it as is. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Small GIN optimizations (after 9.4)

2014-02-11 Thread Alexander Korotkov
On Wed, Feb 12, 2014 at 2:58 AM, Bruce Momjian wrote: > On Sun, Feb 9, 2014 at 02:17:12PM +0400, Alexander Korotkov wrote: > > On Thu, Feb 6, 2014 at 8:31 PM, PostgreSQL - Hans-J rgen Sch nig < > > postg...@cybertec.at> wrote: > > > > i think there is one

Re: [HACKERS] GSoC 2014 - mentors, students and admins

2014-02-10 Thread Alexander Korotkov
ing this year? And are there any project > ideas folk would like to suggest? > I would like to be mentor. -- With best regards, Alexander Korotkov.

Re: [HACKERS] PoC: Partial sort

2014-02-10 Thread Alexander Korotkov
On Mon, Feb 10, 2014 at 2:33 PM, Marti Raudsepp wrote: > On Sun, Feb 9, 2014 at 7:37 PM, Alexander Korotkov > wrote: > > This is not only place that worry me about planning overhead. See > > get_cheapest_fractional_path_for_pathkeys. I had to estimate number of > >

Re: [HACKERS] trgm regex index peculiarity

2014-02-09 Thread Alexander Korotkov
On Thu, Jan 16, 2014 at 3:34 AM, Tom Lane wrote: > Alexander Korotkov writes: > > Revised version of patch with necessary comments. > > I looked at this patch a bit. It seems like this: > > + *BLANK_COLOR_SIZE - How much blank character

Re: [HACKERS] PoC: Partial sort

2014-02-09 Thread Alexander Korotkov
t; Seq Scan on test2 t2 (cost=0.00..15406.00 rows=100 width=12) (11 rows) I don't compare actual execution times because I didn't wait until first plan execution ends up :-) But anyway costs are extraordinary and inner sequential scan of 100 rows is odd. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Small GIN optimizations (after 9.4)

2014-02-09 Thread Alexander Korotkov
uce memory usage of a > scan. > I would like to add another one as continue of fast-scan: * Skip scanning of some entries at all forcing recheck instead. Correct decision should be done based on costs. However, I'm not sure about design. Because it's like a planning feature. How correct to do this inside of GIN? -- With best regards, Alexander Korotkov.

Re: [HACKERS] Small GIN optimizations (after 9.4)

2014-02-09 Thread Alexander Korotkov
nsive one. > if the tsearch_query yields a high number of rows (which it often does) > the subsequent sort will kill you. > This is not intended to be a small change. However, some solution might be possible in post 9.4 gin improvements or in new secret indexing project which will be presented at PGCon :-) -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-09 Thread Alexander Korotkov
On Fri, Feb 7, 2014 at 5:33 PM, Heikki Linnakangas wrote: > On 02/06/2014 01:22 PM, Alexander Korotkov wrote: > >> Difference is very small. For me, it looks ready for commit. >> > > Great, committed! > > Now, to review the catalog changes... I've rebased cata

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-06 Thread Alexander Korotkov
On Thu, Feb 6, 2014 at 2:21 PM, Heikki Linnakangas wrote: > On 02/05/2014 12:42 PM, Alexander Korotkov wrote: > >> Attached patch is "light" version of fast scan. It does extra consistent >> function calls only on startScanKey, no extra calls during scan of the >

Re: [HACKERS] Fix picksplit with nan values

2014-02-05 Thread Alexander Korotkov
On Sat, Feb 1, 2014 at 7:50 AM, Bruce Momjian wrote: > > Where are we on this? I found myself to have empty draft letter from November with new version of patch attached. I'll return here when we have some solution in gin fast scan challenge. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-05 Thread Alexander Korotkov
On Wed, Feb 5, 2014 at 1:23 AM, Alexander Korotkov wrote: > On Mon, Feb 3, 2014 at 6:31 PM, Alexander Korotkov > wrote: > >> On Mon, Jan 27, 2014 at 7:30 PM, Alexander Korotkov > > wrote: >> >>> On Mon, Jan 27, 2014 at 2:32 PM, Alexander Korotkov < >>

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-04 Thread Alexander Korotkov
On Mon, Feb 3, 2014 at 6:31 PM, Alexander Korotkov wrote: > On Mon, Jan 27, 2014 at 7:30 PM, Alexander Korotkov > wrote: > >> On Mon, Jan 27, 2014 at 2:32 PM, Alexander Korotkov > > wrote: >> >>> On Sun, Jan 26, 2014 at 8:14 PM, Heikki Linnakangas

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-03 Thread Alexander Korotkov
''initdb'' & ''d'''::tsquery) > >> > Planning time: 0.257 ms > >> > (5 rows) > >> > > >> > 5alpha1-initdb'd is 3 gin entries with different frequencies. > >> > >> Why do you find that strange? The way the query is formed or the way > >> it's > >> evaluated? > >> > >> The query generator certainly is not perfect, so it may produce some > >> strange queries. > >> > > > > I just mean that in this case 3 words doesn't mean 3 gin entries. > > Isn't that expected? I mean, that's what to_tsquery may do, right? > Everything is absolutely correct. :-) It just may be not what do you expect if you aren't getting into details. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-03 Thread Alexander Korotkov
On Mon, Feb 3, 2014 at 7:24 PM, Tomas Vondra wrote: > On 3 Únor 2014, 15:31, Alexander Korotkov wrote: > > > > I found my patch "0005-Ternary-consistent-implementation.patch" to be > > completely wrong. It introduces ternary consistent function to opclass, >

Re: [HACKERS] KNN-GiST with recheck

2014-02-03 Thread Alexander Korotkov
On Tue, Jan 28, 2014 at 9:32 PM, Heikki Linnakangas wrote: > On 01/28/2014 04:12 PM, Alexander Korotkov wrote: > >> On Tue, Jan 28, 2014 at 5:54 PM, Heikki Linnakangas < >> hlinnakan...@vmware.com >> >>> wrote: >>> >> >> 4. (as you men

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-03 Thread Alexander Korotkov
IN_MAYBE until it returns GIN_FALSE. 3) Decide whether to use fast-scan depending on number of TIDs in first m entries and number of TIDs in rest (n-m) entries. Something like number_of_TIDs_in_frist_m_entries * k < number_of_TIDs_in_rest_n-m_entries where k is some quotient. For sure, it rough method, but it should be fast. Without natural implementation of ternary consistent function algorithm will be slightly different: only m values close to n should be checked. I'm going to implement this heuristic against last version of your patch. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-03 Thread Alexander Korotkov
On Mon, Jan 27, 2014 at 7:30 PM, Alexander Korotkov wrote: > On Mon, Jan 27, 2014 at 2:32 PM, Alexander Korotkov > wrote: > >> On Sun, Jan 26, 2014 at 8:14 PM, Heikki Linnakangas < >> hlinnakan...@vmware.com> wrote: >> >>> On 01/26/2014 08:24 AM, Tom

Re: [HACKERS] KNN-GiST with recheck

2014-01-28 Thread Alexander Korotkov
On Tue, Jan 28, 2014 at 5:54 PM, Heikki Linnakangas wrote: > On 01/13/2014 07:17 PM, Alexander Korotkov wrote: > >> Here goes a desription of this patch same as in original thread. >> >> KNN-GiST provides ability to get ordered results from index, but this >> o

Re: [HACKERS] PoC: Partial sort

2014-01-27 Thread Alexander Korotkov
On Tue, Jan 28, 2014 at 7:41 AM, Marti Raudsepp wrote: > On Mon, Jan 27, 2014 at 9:26 PM, Alexander Korotkov > wrote: > > For now, I have attempt to fix extra columns in mergejoin problem. It > would > > be nice if you test it. > > Yes, it solves the test case

Re: [HACKERS] PoC: Partial sort

2014-01-27 Thread Alexander Korotkov
Hi! On Tue, Jan 21, 2014 at 3:24 AM, Marti Raudsepp wrote: > On Tue, Jan 14, 2014 at 5:49 PM, Alexander Korotkov > wrote: > >On Tue, Jan 14, 2014 at 12:54 AM, Marti Raudsepp wrote: > >> I've been trying it out in a few situations. I implemented a new > >>

Re: [HACKERS] GIN improvements part2: fast scan

2014-01-27 Thread Alexander Korotkov
function. > I believe cache consistent function results is quite same as lazy truth-table. I think it's a good option to use with two-state consistent function. However, I don't think it's a reason to refuse from three-state consistent function because number of entries could be large. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-01-27 Thread Alexander Korotkov
the current >> item. I'm not sure if that's worth the cycles, but it seemed like an >> obvious thing to do, now that we have the ternary consistent function. >> > > So, that clearly isn't worth the cycles :-). At least not with an > expensive consistent function; it might be worthwhile if we pre-build the > truth-table, or cache the results of the consistent function. > > Attached is a quick patch to remove that, on top of all the other patches, > if you want to test the effect. Every single change you did in fast scan seems to be reasonable, but testing shows that something went wrong. Simple test with 3 words of different selectivities. After applying your patches: # select count(*) from fts_test where fti @@ plainto_tsquery('english', 'gin index select'); count ─── 627 (1 row) Time: 21,252 ms In original fast-scan: # select count(*) from fts_test where fti @@ plainto_tsquery('english', 'gin index select'); count ─── 627 (1 row) Time: 3,382 ms I'm trying to get deeper into it. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GIN improvements part2: fast scan

2014-01-24 Thread Alexander Korotkov
On Thu, Jan 23, 2014 at 8:22 PM, Heikki Linnakangas wrote: > On 01/14/2014 05:35 PM, Alexander Korotkov wrote: > >> Attached version is rebased against last version of packed posting lists. >> > > Thanks! > > I think we're missing a trick with multi-key queries

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-24 Thread Alexander Korotkov
On Fri, Jan 24, 2014 at 1:19 PM, Heikki Linnakangas wrote: > On 01/24/2014 10:53 AM, Alexander Korotkov wrote: > >> OK. What about previous fix in assert? >> > > Ah right, fixed that too now. Good, now my test-suite passed. Results are so. Time of o

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-24 Thread Alexander Korotkov
On Fri, Jan 24, 2014 at 12:50 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 01/24/2014 10:03 AM, Alexander Korotkov wrote: > >> ITSM I found this bug. ginVacuumPostingTreeLeaf re-encodes only some >> segments. Others are not even re-palloced. T

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-24 Thread Alexander Korotkov
On Thu, Jan 23, 2014 at 9:27 AM, Alexander Korotkov wrote: > On Wed, Jan 22, 2014 at 9:28 PM, Heikki Linnakangas < > hlinnakan...@vmware.com> wrote: > >> On 01/22/2014 02:17 PM, Alexander Korotkov wrote: >> >>> We already spent a lot of time with compression. N

<    1   2   3   4   5   6   7   8   9   10   >