[HACKERS] A small performance bug in BTree Infrastructure

2008-10-23 Thread Gokulakannan Somasundaram
Hi All, BTree Insert requires a data-structure called BTStack to report the page splits that have happened in the leaf pages to non-leaf pages upwards. It basically goes back and updates those non-leaf pages. But this will never happen in a search operation. You never need to climb upwards

[HACKERS] A small issue in CommentCast function

2009-11-29 Thread Gokulakannan Somasundaram
Hi, I think this is more of code uniformity issue. Nevertheless, i just thought of putting it here. In the CommentCast function, i think both sourcetype and targettype should be in the arguments, whereas in the code, source type is made as the name and the target type is passed in as

Re: [HACKERS] Index-only scans

2010-01-09 Thread Gokulakannan Somasundaram
Hi Heikki, I was recollecting our conversation that, if the index-only quals were implemented through indexes by storing snapshots, broken data-types may not be supported. I feel this problem might exist, if we go on design a IOT(Index Organized Tables) / Clustered Indexes. IOT is again

[HACKERS] Precedence of Postfix operators

2010-02-07 Thread Gokulakannan Somasundaram
Hi, Is there any reason why we have given lesser precedence for postfix operator compared to multiplication/division? Usually postfix operators have more precedence than the binary operations. Is this some kind of work around to provide user-defined operators? Can someone help me understand

Re: [HACKERS] Precedence of Postfix operators

2010-02-07 Thread Gokulakannan Somasundaram
Thanks Tom, for the explanation. Gokul. On Sun, Feb 7, 2010 at 10:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: Gokulakannan Somasundaram gokul...@gmail.com writes: Is there any reason why we have given lesser precedence for postfix operator compared to multiplication/division? Usually

[HACKERS] A thought on Index Organized Tables

2010-02-21 Thread Gokulakannan Somasundaram
Hi, As you all know, Index Organized tables are a way by which we can automatically cluster the data based on the primary key. While i was thinking about an implementation for postgres, it looks like an impossible with the current ideologies. In an IOT, if a record gets updated, we need to

Re: [HACKERS] A thought on Index Organized Tables

2010-02-22 Thread Gokulakannan Somasundaram
. On Mon, Feb 22, 2010 at 12:21 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Gokulakannan Somasundaram wrote: Hi, As you all know, Index Organized tables are a way by which we can automatically cluster the data based on the primary key. While i was thinking about

Re: [HACKERS] A thought on Index Organized Tables

2010-02-22 Thread Gokulakannan Somasundaram
Forgot to include the group... On Mon, Feb 22, 2010 at 4:31 PM, Gokulakannan Somasundaram gokul...@gmail.com wrote: These sound like the same point to me. I don't think we're concerned with footprint -- only with how much of that footprint actually needs to be scanned. So if we have

Re: [HACKERS] A thought on Index Organized Tables

2010-02-23 Thread Gokulakannan Somasundaram
On Tue, Feb 23, 2010 at 10:42 AM, Tom Lane t...@sss.pgh.pa.us wrote: Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp writes: Instead, how about excluding columns in primary keys from table data? How will you implement select * from mytable ? Or even select * from mytable where

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
I think Gokul was asking because he wanted to work on it, but wanted to check community approval first. Yes the problem is that we need to come to a consensus on broken data types. As Heikki pointed out, those data types, which is based on a unstable function like time, date, random etc.

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
The fragility there is not an issue in a mostly read-only application, but it definitely would be a concern in other cases. While we accept that visibility map is good for read only application, why can't we make it optional? Atleast if there is a way for a person to drop the visibility map

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
Forgot to include the group.. On Wed, Feb 24, 2010 at 5:38 PM, Gokulakannan Somasundaram gokul...@gmail.com wrote: I am not familiar with this term broken data types, and I just looked for it in the source code and couldn't find it. What exactly are you referring to? cheers andrew

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
If you have a scenario where the visibility map incurs a measurable overhead, let's hear it. I didn't see any in the tests I performed, but it's certainly possible that if the circumstances are just right it makes a difference. Heikki, The obvious one, i could observe is that it

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
I think you're a barking up the wrong tree. AFAIUI, the need for the visibility map has not very much to do with whether the table has indices, and everything to do with avoiding unnecessary VACUUMs. In any event, you've not shown that the visibility map HAS any overhead, so talking about

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
But adding 24 bytes to every index entry seems pretty unlikely to be a win anyways. We actually wanted to make it optional. Not every index will be like that. More than that we can make it into 16 bytes. Only commands within the same transaction will not be able to do a index only scan.

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
With an IOT I don't understand how you get out of index corruption without data loss. That's a showstopper for practical use, I think. For simplicity, say we are storing all the non-leaf pages of the index in a seperate file, then the leaf pages are nothing but the table. So if we can

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
On Wed, Feb 24, 2010 at 10:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: So you are essentially proposing that rather than moving the heap data into the leaf tuples of the index in the index file, you will move the leaf index data into the

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
Yes. The visibility map doesn't need any new WAL records to be written. We probably will need to add some WAL logging to close the holes with crash recovery, required for relying on it for index-only-scans, but AFAICS only for VACUUM and probably only one WAL record for a whole bunch of heap

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
Missed the group... On Thu, Feb 25, 2010 at 12:34 AM, Gokulakannan Somasundaram gokul...@gmail.com wrote: On Thu, Feb 25, 2010 at 12:28 AM, Gokulakannan Somasundaram gokul...@gmail.com wrote: That doesn't work because when you split an index page any sequential scan in progress

Fwd: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
Missed the group So basically you want to have index-only scans, but you want them to be really slow? No Robert, i just asked him to make it optional, so that the crash safe visibility map is used only when its need is realized by the DB designer. When there is a table with no indexes/

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
It might be slightly easier given the assumption that you only want to scan leaf tuples. But there's an additional problem I didn't think of before. Currently we optimize index scans by copying all relevant tuples to local memory so we don't need to hold an index lock for an extended time or

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
That doesn't work because when you split an index page any sequential scan in progress will either see the same tuples twice or will miss some tuples depending on where the new page is allocated. Vacuum has a clever trick for solving this but it doesn't work for arbitrarily many concurrent

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
You also need to avoid scanning the same tuple twice. That's not a problem for VACUUM, but it is for full index scans. Heikki, Actually the logic, which i have explained earlier is to avoid scanning tuples twice. Gokul.

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
I haven't thought about whether this is sufficient but if it is then an initial useful thing to add would be to use it for queries where we have a qual that can be checked using the index key even though we can't do a range scan. For example if you have a btree index on a,b,c and you have a

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
The WAL record of the heap insert/update/delete contains a flag indicating that the visibility map needs to be updated too. Thus no need for a separate WAL record. Heikki, Have you considered these cases? a) The current WAL architecture makes sure that the WAL Log is written before

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Gokulakannan Somasundaram
On Thu, Feb 25, 2010 at 3:16 AM, Gokulakannan Somasundaram gokul...@gmail.com wrote: I haven't thought about whether this is sufficient but if it is then an initial useful thing to add would be to use it for queries where we have a qual that can be checked using the index key even though we

Re: [HACKERS] A thought on Index Organized Tables

2010-02-25 Thread Gokulakannan Somasundaram
Yes. When a bit is cleared, that's OK, because a cleared bit just means you need to check visibility in the heap tuple. When a bit is set, however, it's important that it doesn't hit the disk before the corresponding heap page update. That's why visibilitymap_set() sets the LSN on the page.

Re: [HACKERS] A thought on Index Organized Tables

2010-02-25 Thread Gokulakannan Somasundaram
The replay of the heap insert/update/delete record updates the visibility map. So are you planning to make that section, which writes the xlog and updates the visibility map inside a PANIC section right?

Re: [HACKERS] A thought on Index Organized Tables

2010-02-25 Thread Gokulakannan Somasundaram
The replay of the heap insert/update/delete record updates the visibility map. Say a checkpoint has occured in between and flushed the dirty pages into disk, while the updater waits to update the visibility map. Now there will be no replay for the insert/update/delete right?

Re: [HACKERS] A thought on Index Organized Tables

2010-02-25 Thread Gokulakannan Somasundaram
1) transaction information in index This seems like a lot of bloat in indexes. It also means breaking a lot of other optimizations such as being able to read the tuples directly from the heap page without locking. I'm not sure how much those are worth though. But adding 24 bytes to every

Re: [HACKERS] A thought on Index Organized Tables

2010-02-25 Thread Gokulakannan Somasundaram
Wait a second, which idea are we currently talking about? No heap at all, or just the ability to check visibility without visiting the heap? I was talking about the indexes with snapshot If it's a genuine IOT (ie no separate heap), then you are not going to be able to get away without a

Re: [HACKERS] A thought on Index Organized Tables

2010-02-25 Thread Gokulakannan Somasundaram
I disagree with that, Gokul -- if the ordering operators are volatile or just incorrect, during DELETE, you could set xmax in the wrong IndexTuple. Then there will be another IndexTuple that says it's visible, but it points to a non-visible heap tuple. I think you should follow the pointers

Re: [HACKERS] A thought on Index Organized Tables

2010-02-25 Thread Gokulakannan Somasundaram
Tom, Actually, if you need to squeeze a few more bits into that word, the thing to do would be to get rid of storing the tuple length there. This would involve adding the same type of indirection header that we use for HeapTuples, so that the length would be available at need without going

Re: [HACKERS] A thought on Index Organized Tables

2010-02-25 Thread Gokulakannan Somasundaram
First of all, volatility is not the only issue. The ordering ops could also be incorrect, e.g., violate the transitivity property. there is no reliable way to determine if a function is volatile and/or incorrectly specified. No it is the only issue. If you create a datatype with volatile

Re: [HACKERS] A thought on Index Organized Tables

2010-02-25 Thread Gokulakannan Somasundaram
No, it's far from harmless. As soon as that heap TID gets filled with an unrelated tuple, you run the risk of indexscans alighting on and perhaps modifying the wrong tuple. Tom, In the Function based indexes on those functions, which we are suspecting to be a volatile one Or in the

Re: [HACKERS] A thought on Index Organized Tables

2010-02-25 Thread Gokulakannan Somasundaram
On Fri, Feb 26, 2010 at 9:54 AM, Gokulakannan Somasundaram gokul...@gmail.com wrote: No, it's far from harmless. As soon as that heap TID gets filled with an unrelated tuple, you run the risk of indexscans alighting on and perhaps modifying the wrong tuple. Tom, i think

Re: [HACKERS] A thought on Index Organized Tables

2010-02-25 Thread Gokulakannan Somasundaram
No, we're not going there. That breaks the fundamental page content manipulation algorithms, and falls down for tuples not yet stored in a page (or being examined without a pointer to the page readily at hand), and has no redeeming social value anyway compared to doing it in the proven

Re: [HACKERS] A thought on Index Organized Tables

2010-02-25 Thread Gokulakannan Somasundaram
Proving that a set of comparison operators are consistent just by examining their runtime behavior is probably equivalent to solving the halting problem. I can't see us doing it, or wanting to accept the overhead of checking it even if it could be done. The overhead of checking is very

Re: [HACKERS] A thought on Index Organized Tables

2010-02-26 Thread Gokulakannan Somasundaram
To be a bit more concrete: the typical sort of failure that you could get from broken btree operators is failure of transitivity, that is the comparators report A B and B C for some A, B, C, but do not say that A C when those two values are compared directly. I don't see any convenient

Re: [HACKERS] A thought on Index Organized Tables

2010-02-26 Thread Gokulakannan Somasundaram
It definitely affects current indexes. We can't completely avoid bad user functions. That is why it is important to put limits on how much damage they can do. That's the motivation for the idea I mentioned before, of double-checking visibility data in an IndexTuple before letting it survive a

Re: [HACKERS] A thought on Index Organized Tables

2010-02-26 Thread Gokulakannan Somasundaram
Much better to take on a simple project like enabling full sequential index scans which you claimed you had a solution for and which is in any case an important sub-problem for IOT. Greg, Well i don't think i am ready to take up a project of this size. But at the same time some

Re: [HACKERS] A thought on Index Organized Tables

2010-02-26 Thread Gokulakannan Somasundaram
Missed the group.. On Sat, Feb 27, 2010 at 12:00 AM, Gokulakannan Somasundaram gokul...@gmail.com wrote: I definitely think thick indexes were too ambitious of a target for a first time patch. Sequential index scans is very ambitious itself despite being significantly simpler (if you have

Re: [HACKERS] A thought on Index Organized Tables

2010-02-26 Thread Gokulakannan Somasundaram
IIRC, what was being talked about was shoehorning some hint bits into the line pointers by assuming that size and offset are multiples of 4. I'm not thrilled with having mutable status bits there for reliability reasons, but it could be done without breaking a lot of existing code. What I was

Re: [HACKERS] A thought on Index Organized Tables

2010-02-26 Thread Gokulakannan Somasundaram
It does. The point is that the system is set up to limit the bad consequences. You might (will) get wrong query answers, but the heap data won't get corrupted. Again Tom, if there is an update based on index scan, then it takes the tupleid and updates the wrong heap data right? The only

Re: [HACKERS] A thought on Index Organized Tables

2010-02-26 Thread Gokulakannan Somasundaram
It does. The point is that the system is set up to limit the bad consequences. You might (will) get wrong query answers, but the heap data won't get corrupted. Tom, if this is our goal - *can return wrong query answers, but should not corrupt the heap data.* and if we make Thick

Re: [HACKERS] A thought on Index Organized Tables

2010-02-26 Thread Gokulakannan Somasundaram
No, what generally happens is it fails to find a matching index entry at all, because the search algorithm concludes there can be no match based on the limited set of comparisons it's done. Transitivity failures lead to searching the wrong subset of the index. Actually Tom, i am not able to

Re: [HACKERS] A thought on Index Organized Tables

2010-02-26 Thread Gokulakannan Somasundaram
Wait a minute. Bingo So for unique checks we are already going to index from Heap. So it is the same thing i am doing with Thick index. So if we can trust our current unique checks, then we should trust the Thick index. Thanks Tom!!! for having this good conversation I think this

Re: [HACKERS] A thought on Index Organized Tables

2010-02-26 Thread Gokulakannan Somasundaram
Actually Tom, i am not able to understand that completely. But what you are saying that in the current scenario, when there is a broken data type based index, then it will return no results, but never will return wrong results. So never the update will corrupt the heap data. But i take it as

Re: [HACKERS] Lock Wait Statistics (next commitfest)

2010-02-26 Thread Gokulakannan Somasundaram
I am just adding my two cents, please ignore it, if its totally irrelevant. While we do performance testing/tuning of any applications, the important things, a standard monitoring requirement from a database are a) Different type of wait events and the time spent in each of them b) Top ten Queries

Re: [HACKERS] Testing of parallel restore with current snapshot

2010-02-26 Thread Gokulakannan Somasundaram
Tom, I just took the patch, but it seems to be in binary format. Can you send me the patch to me? Thanks, Gokul. On Sat, May 30, 2009 at 3:12 AM, Tom Lane t...@sss.pgh.pa.us wrote: Josh Berkus j...@agliodbs.com writes: Tom, Is anyone interested enough to try it if I code it? If

Re: [HACKERS] A thought on Index Organized Tables

2010-02-27 Thread Gokulakannan Somasundaram
If i have got over excited in the previous update, please ignore that. a) We are already going from table to index to do unique checks. This is the same thing, which we will do to go and update the snapshot in the indexes. b) The way, it should work would be to have a check on whether the

Re: [HACKERS] A thought on Index Organized Tables

2010-02-28 Thread Gokulakannan Somasundaram
No, it is not the same thing. Updating index snapshots requires being able to *re-find* a previously made index entry for the current row. And it has to be done 100% reliably. The worst that happens if an index entry is not found when it should be during a uniqueness check is that the

Re: [HACKERS] A thought on Index Organized Tables

2010-02-28 Thread Gokulakannan Somasundaram
The transaction information on tuples take 18 bytes plus several info bits. It's possible just storing a subset of that would be useful but it's unclear. And I think it would complicate the code if it had to sometimes fetch the heap tuple to get the rest and sometimes doesn't. Visibility

Re: [HACKERS] A thought on Index Organized Tables

2010-03-01 Thread Gokulakannan Somasundaram
a) We are already going from table to index to do unique checks. This is the same thing, which we will do to go and update the snapshot in the indexes. No, it is not the same thing. Updating index snapshots requires being able to *re-find* a previously made index entry for the current

Re: [HACKERS] A thought on Index Organized Tables

2010-03-02 Thread Gokulakannan Somasundaram
I think you have to take up a simpler project as a first project. This is a major overhaul of transaction information and it depends on understanding how a lot of different areas work -- all of which are very complex tricky areas to understand. Greg, I just feel the fast full

[HACKERS] Bug in 9.0Alpha4

2010-03-16 Thread Gokulakannan Somasundaram
Hi, I noticed a problem with the source code of 9.0Alpha 4. In parse_agg.c, there is a call made to transformSortClause. 00098 torder = transformSortClause http://doxygen.postgresql.org/parse__clause_8c.html#53199c36a198b5acf15a26fbd7311f79(pstate, 00099

Re: [HACKERS] Bug in 9.0Alpha4

2010-03-16 Thread Gokulakannan Somasundaram
Hi, I think, this should be the probable fix. There is agg_order in ParseFuncOrColumn, which should get passed on to transformAggregateCall and that should be placed in this call, instead of agg-aggorder. Thanks, Gokul. On Tue, Mar 16, 2010 at 5:19 PM, Gokulakannan Somasundaram gokul

Re: [HACKERS] Bug in 9.0Alpha4

2010-03-16 Thread Gokulakannan Somasundaram
transformSortClause is passed the untransformed aggorder list, which is in fact a list of SortBy nodes, and it returns the transformed list (SortGroupClause nodes), which is stored back into the aggorder field a bit further down. There are a number of regression tests that would fail in

Re: [HACKERS] Bug in 9.0Alpha4

2010-03-17 Thread Gokulakannan Somasundaram
When we were doing the ordered-aggregates patch, I considered passing all those values as explicit parameters to transformAggregateCall, and having it build the Aggref node from scratch and return it. However having seven or eight parameters to transformAggregateCall (and more in future if

Re: [HACKERS] An idle thought

2010-03-18 Thread Gokulakannan Somasundaram
On Thu, Mar 18, 2010 at 2:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Davis pg...@j-davis.com writes: There are all kinds of challenges there, but it might be worth thinking about. Visibility information is highly compressible, and requires constant maintenance (updates, deletes,

Re: [HACKERS] An idle thought

2010-03-18 Thread Gokulakannan Somasundaram
Secondly there's the whole retail vacuum problem -- any index entries referring to this page would be left dangling unless there's some kind of retail vacuum or perhaps a page version number. The issue, we can divide into two a)volatile functions b)broken datatypes For a) I think volatile

Re: [HACKERS] An idle thought

2010-03-18 Thread Gokulakannan Somasundaram
I didn't mean that we'd want to compress it to the absolute minimum size. I had envisioned that it would be a simple scheme designed only to eliminate long runs of identical visibility information (perhaps only the frozen and always visible regions would be compressed). The extra level of

Re: [HACKERS] An idle thought

2010-03-18 Thread Gokulakannan Somasundaram
The visibility map itself is already an example of compression. If visibility information were randomly distributed among tuples, the visibility map would be nearly useless. I believe it is very difficult to make visibility map update friendly without compromising durability. But such a

Re: [HACKERS] An idle thought

2010-03-19 Thread Gokulakannan Somasundaram
Surely the VM is already update-friendly. If you update a tuple in a page with the visibility bit set, the bit must be unset or you will get wrong results. I was referring in the context of index only scans to skip visibility checks. I doubt, whether the visibility map feature to skip

[HACKERS] Proposal for Byte savings in VarBit structure

2010-03-21 Thread Gokulakannan Somasundaram
Hi, I was looking at the VarBit data structure and found out that instead of storing the number of bits in four bytes, we can save the number of bits that are valid in the last byte. Since we already store the number of bytes in Varlena Header, we can calculate the number of valid bits by doing

Re: [HACKERS] Proposal for Byte savings in VarBit structure

2010-03-21 Thread Gokulakannan Somasundaram
This might be worth considering in a release cycle where we were going to break on-disk data compatibility for some other reason. But I can hardly imagine wanting to do it by itself. Marginal space savings for the bit types just isn't that exciting. Maybe we should start a special section

[HACKERS] Deadlock possibility in _bt_check_unique?

2010-03-23 Thread Gokulakannan Somasundaram
Hi, With the implementation of deferred unique constraints, we need to go back to the index second time to check whether the unique check is valid. Say a situation occurs like this a) the first session doing the unique check finds out that there is a unique check required second time and just

Re: [HACKERS] Deadlock possibility in _bt_check_unique?

2010-03-23 Thread Gokulakannan Somasundaram
Can you also explain how are we avoiding duplicates in this scenario? a) Say there are three pages(P,Q, R) full of duplicate tuples, that are deleted but not dead of id x(due to some long running transaction). b) Now Session A gets in and checks the duplicate tuples for their liveliness with the

Re: [HACKERS] Deadlock possibility in _bt_check_unique?

2010-03-23 Thread Gokulakannan Somasundaram
Are you talking about exclusion constraints or btree uniqueness constraints? This doesn't seem to be a particularly accurate description of the implementation of either one. The way btree deals with this is explained in _bt_doinsert: Unique constraints * NOTE: obviously,

Re: [HACKERS] Deadlock possibility in _bt_check_unique?

2010-03-23 Thread Gokulakannan Somasundaram
No, you don't understand how it works. All would-be inserters will hit the same target page to begin with, ie, the first one that the new key could legally be inserted on. The lock that protects against this problem is the lock on that page, regardless of which page the key actually ends up

Re: [HACKERS] Deadlock possibility in _bt_check_unique?

2010-03-23 Thread Gokulakannan Somasundaram
T2 : session 1 releases the lock on p1 (its waiting to acquire a ex lock on p2) That's not what we do. See _bt_findinsertloc. regards, tom lane I am really confused. Please keep the cool and explain me, if i am wrong. I could see this code in

Re: [HACKERS] Deadlock possibility in _bt_check_unique?

2010-03-23 Thread Gokulakannan Somasundaram
Oh! yeah, i got it. Thanks On Wed, Mar 24, 2010 at 12:26 AM, Gokulakannan Somasundaram gokul...@gmail.com wrote: T2 : session 1 releases the lock on p1 (its waiting to acquire a ex lock on p2) That's not what we do. See _bt_findinsertloc. regards, tom lane

[HACKERS] Performance Improvement for Unique Indexes

2010-03-24 Thread Gokulakannan Somasundaram
Hi, While i was studying the unique index checks very closely, i realized that what we need is to find out whether the tuple is deleted / not. So say a tuple is deleted by a transaction, but it is not dead( because of some long running transaction ), still we can mark a hint bit as deleted and

Re: [HACKERS] Performance Improvement for Unique Indexes

2010-03-24 Thread Gokulakannan Somasundaram
How are you going to unmark the hint bit in case of a rollback? Only after you find that the transaction is committed, this hint bit has to be set. It is equivalent to any other hint bit. Gokul.

Re: [HACKERS] Performance Improvement for Unique Indexes

2010-03-24 Thread Gokulakannan Somasundaram
it seems fairly unlikely to me that this would be useful enough to justify using up a precious hint bit. The applicability of the hint is very short-term --- as soon as the tuple is dead to all transactions, it can be marked with the existing LP_DEAD hint bit. And if it's only useful for

[HACKERS] Performance improvement for unique checks

2010-03-26 Thread Gokulakannan Somasundaram
Hi, Since we insert a new entry into the index for every update that's being made into the table, we inevitably make a unique check against the older version of the newly inserted row, even when the values are not updated. Of course i am talking about non-HOT updates. (We will not go to the

[HACKERS] Proposal for Performance improvement for unique checks

2010-03-27 Thread Gokulakannan Somasundaram
I don't think this should involve much code change. But no-one interested On Sat, Mar 27, 2010 at 2:23 AM, Gokulakannan Somasundaram gokul...@gmail.com wrote: Hi, Since we insert a new entry into the index for every update that's being made into the table, we inevitably make a unique

Re: [HACKERS] Performance improvement for unique checks

2010-03-27 Thread Gokulakannan Somasundaram
Please write it, then test the performance and publish your results, with a detailed analysis of whether there is benefit and in which cases there is a loss. Thanks. Will do it. Just wanted to know, whether the idea will get rejected right away / worth trying out. Thanks, Gokul.

[HACKERS] A Bug in outDatum ?? (Not Sure )

2010-03-27 Thread Gokulakannan Somasundaram
Hi, The function outDatum doesn't take care of varlena structures which are stored externally. Is it the intention?? Thanks, Gokul.

Re: [HACKERS] GSoC Query

2010-03-29 Thread Gokulakannan Somasundaram
Similarly using the no. of select hits on a table we can check that if maximum no. of times it is on a non-index field we can index on that field to make select faster. It's impractical to figure out where indexes should go at without simulating what the optimizer would then do with

Re: [HACKERS] Improving the Performance of Full Table Updates

2007-09-26 Thread Gokulakannan Somasundaram
Hi Tom/ Heikki, Thanks for the suggestion. After profiling i got similar results. So i am thinking of a design like this to get the performance improvement. a) We can get one page for insert(during update) and we will hold the write lock on it, till the page gets filled. In this way,

Re: [HACKERS] Lazy Snapshots

2009-08-21 Thread Gokulakannan Somasundaram
Hi, I have given some thought in this direction. I am just providing my idea. a) Have a structure similar to commit log, which should also store, transaction id at which the transaction got committed. Every transaction, after committing should update the transaction id at which the commit has

Re: [HACKERS] Lazy Snapshots

2009-08-22 Thread Gokulakannan Somasundaram
The maintenance costs and update contention for such a datastructure would render this completely impractical, even if consulting it were free. Thanks for the reply. a) Only one transaction would be updating its commit status. Its multiple readers Vs Single Writer for the position of a

Re: [HACKERS] A bug in scan.l

2009-09-01 Thread Gokulakannan Somasundaram
The previous change should follow with this. uescape[uU][eE][sS][cC][aA][pP][eE]{space}*{quote}[^']{quote} Thanks, Gokul. On Wed, Sep 2, 2009 at 7:35 AM, Gokulakannan Somasundaram gokul...@gmail.com wrote: There is a rule like this in scan.l uescapefail (-|[uU][eE][sS][cC][aA

[HACKERS] A bug in scan.l

2009-09-01 Thread Gokulakannan Somasundaram
There is a rule like this in scan.l uescapefail

Re: [HACKERS] A bug in scan.l

2009-09-01 Thread Gokulakannan Somasundaram
of suggesting to remove the ambiguity. But do we need to allow comments as part of unicode escapes? Thanks, Gokul. On Wed, Sep 2, 2009 at 8:37 AM, Tom Lane t...@sss.pgh.pa.us wrote: Gokulakannan Somasundaram gokul...@gmail.com writes: I have replaced whitespace with space. This has to be done

Re: [HACKERS] COPY enhancements

2009-10-18 Thread Gokulakannan Somasundaram
Actually i thought of a solution for the wrap-around sometime back. Let me try to put my initial thoughts into it. May be it would get refined over conversation. Transaction wrap-around failure Actually this problem is present even in today's transaction id scenario and the only way we avoid is

[HACKERS] A small bug in gram.y

2009-10-26 Thread Gokulakannan Somasundaram
Hi, In the gram.y, under a_expr rule under the subrule a_expr NOT SIMILAR TO a_expr%prec SIMILAR the action is as follows { FuncCall *n = makeNode(FuncCall); n-funcname = SystemFuncName(similar_escape); n-args =

Re: [HACKERS] A small bug in gram.y

2009-11-02 Thread Gokulakannan Somasundaram
Hmmm no-one else feels this as a bug The logic is that a function call is made for similar and the position where SIMILAR occurs is at the third position, but it has been coded that it is at fifth position. Thanks, Gokul. On Tue, Oct 27, 2009 at 6:51 AM, Gokulakannan Somasundaram gokul

Re: [HACKERS] Hash Join Optimization

2008-03-30 Thread Gokulakannan Somasundaram
On Fri, Mar 28, 2008 at 2:04 PM, ITAGAKI Takahiro [EMAIL PROTECTED] wrote: Gokulakannan Somasundaram [EMAIL PROTECTED] wrote: I think the creation of minimal_tuple in the middle is a overhead which can be avoided by creating a mem-map and directly creating the minimal_tuple in the mem

Re: [HACKERS] Improving the Performance of Full Table Updates

2007-10-08 Thread Gokulakannan Somasundaram
into the indexes, on which i will make a proposal. Please provide me guidance on that. Thanks, Gokul. On 9/27/07, Heikki Linnakangas [EMAIL PROTECTED] wrote: Gokulakannan Somasundaram wrote: Hi Tom/ Heikki, Thanks for the suggestion. After profiling i got similar results. So i am

[HACKERS] Including Snapshot Info with Indexes

2007-10-08 Thread Gokulakannan Somasundaram
Hi, Currently The index implementation in Postgresql does not store the Snapshot information in the Index. If we add the snapshot information into the indexing structure, we will have the following advantages. a) There can be index only scans like Oracle b) Unique indexes will become less

Re: [HACKERS] Including Snapshot Info with Indexes

2007-10-08 Thread Gokulakannan Somasundaram
On 10/8/07, Heikki Linnakangas [EMAIL PROTECTED] wrote: Gokulakannan Somasundaram wrote: Currently The index implementation in Postgresql does not store the Snapshot information in the Index. If we add the snapshot information into the indexing structure, we will have the following

Re: [HACKERS] Including Snapshot Info with Indexes

2007-10-08 Thread Gokulakannan Somasundaram
On 10/8/07, Heikki Linnakangas [EMAIL PROTECTED] wrote: Csaba Nagy wrote: On Mon, 2007-10-08 at 09:40 +0100, Heikki Linnakangas wrote: This idea has been discussed to death many times before. Please search the archives. Somewhat related to the visibility in index thing: would it be

Re: [HACKERS] Including Snapshot Info with Indexes

2007-10-08 Thread Gokulakannan Somasundaram
they have classified a few as non-deterministic operators and functions. Can we follow a similar approach? Thanks, Gokul. On 10/8/07, Heikki Linnakangas [EMAIL PROTECTED] wrote: Gokulakannan Somasundaram wrote: On 10/8/07, Heikki Linnakangas [EMAIL PROTECTED] wrote: IMO, the most

Re: [HACKERS] Including Snapshot Info with Indexes

2007-10-09 Thread Gokulakannan Somasundaram
On 10/8/07, Heikki Linnakangas [EMAIL PROTECTED] wrote: Gokulakannan Somasundaram wrote: I am always slightly late in understanding things. Let me try to understand the use of DSM. It is a bitmap index on whether all the tuples in a particular block is visible to all

Re: [HACKERS] Including Snapshot Info with Indexes

2007-10-09 Thread Gokulakannan Somasundaram
On 10/8/07, Florian G. Pflug [EMAIL PROTECTED] wrote: Gokulakannan Somasundaram wrote: Hi Heikki, I am always slightly late in understanding things. Let me try to understand the use of DSM. It is a bitmap index on whether all the tuples in a particular block is visible to all the backends

Re: [HACKERS] Including Snapshot Info with Indexes

2007-10-09 Thread Gokulakannan Somasundaram
On 10/9/07, Gokulakannan Somasundaram [EMAIL PROTECTED] wrote: On 10/8/07, Heikki Linnakangas [EMAIL PROTECTED] wrote: Gokulakannan Somasundaram wrote: I am always slightly late in understanding things. Let me try to understand the use of DSM. It is a bitmap index

[HACKERS] IndexTuple Structure

2007-10-09 Thread Gokulakannan Somasundaram
Hi, When i saw the IndexTuple structure, i saw that 13 bits are allocated to store the size of the Values List of the IndexTuple Structure. Also the IID(ItemId Identifier) in the Page header stores the complete size of the IndexTuple. Can't we derive the size of the IndexTuple from the

Re: [HACKERS] IndexTuple Structure

2007-10-09 Thread Gokulakannan Somasundaram
On 10/9/07, Gokulakannan Somasundaram [EMAIL PROTECTED] wrote: Hi, When i saw the IndexTuple structure, i saw that 13 bits are allocated to store the size of the Values List of the IndexTuple Structure. Also the IID(ItemId Identifier) in the Page header stores the complete size

  1   2   3   >