[HACKERS] semi-PoC: kNN-gist for cubes

2012-02-06 Thread Jay Levitt
, but then I realized the fast version was doing 1 million rows, and the slow one did 10 million rows. Which means: dinnertime. Jay Levitt -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Bugs/slowness inserting and indexing cubes

2012-02-07 Thread Jay Levitt
[Posted at Andres's request] TL;DR: Inserting and indexing cubes is slow and/or broken in various ways in various builds. NOTABLE PROBLEMS 1. In 9.1.2, inserting 10x rows takes 19x the time. - 9.1-HEAD and 9.2 fix this; it now slows down linearly - but: 10s 8s 5s! - but: comparing

Re: [HACKERS] Bugs/slowness inserting and indexing cubes

2012-02-07 Thread Jay Levitt
Jay Levitt wrote: [Posted at Andres's request] TL;DR: Inserting and indexing cubes is slow and/or broken in various ways in various builds. And I bet you'll want the test script... sigh. attached. \c postgres drop database if exists slowcube; create database slowcube; \c slowcube \timing

Re: [HACKERS] Progress on fast path sorting, btree index creation time

2012-02-07 Thread Jay Levitt
Jim Decibel! Nasby wrote: I agree that it's probably pretty unusual to index floats. FWIW: Cubes and points are floats, right? So would spatial indexes benefit from this optimization, or is it only raw floats? Jay Levitt -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Bugs/slowness inserting and indexing cubes

2012-02-09 Thread Jay Levitt
Tom Lane wrote: Jay Levittjay.lev...@gmail.com writes: [Posted at Andres's request] TL;DR: Inserting and indexing cubes is slow and/or broken in various ways in various builds. 1. In 9.1.2, inserting 10x rows takes 19x the time. - 9.1-HEAD and 9.2 fix this; it now slows down linearly

Re: [HACKERS] Bugs/slowness inserting and indexing cubes

2012-02-13 Thread Jay Levitt
Robert Haas wrote: On Mon, Feb 13, 2012 at 7:45 AM, Robert Haasrobertmh...@gmail.com wrote: On Thu, Feb 9, 2012 at 3:37 PM, Jay Levittjay.lev...@gmail.com wrote: So my pre-built 9.1.2 takes 434s, my source-built 9.2 takes 509s, and (probably both of our) 9.1-HEAD takes 1918s... is that

[HACKERS] Designing an extension for feature-space similarity search

2012-02-15 Thread Jay Levitt
go research? Thanks for any thoughts, and I'd love collaborators or even mentors - we plan to open source whatever we produce here, and I don't have quite the theoretical background it takes to do this properly. Jay Levitt -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Designing an extension for feature-space similarity search

2012-02-16 Thread Jay Levitt
Alexander Korotkov wrote: On Thu, Feb 16, 2012 at 12:34 AM, Jay Levitt jay.lev...@gmail.com mailto:jay.lev...@gmail.com wrote: - But a dimension might be in any domain, not just floats - The distance along each dimension is a domain-specific function What exact domains do you expect

Re: [HACKERS] Designing an extension for feature-space similarity search

2012-02-16 Thread Jay Levitt
Tom Lane wrote: Jay Levittjay.lev...@gmail.com writes: - I'm not sure how to represent arbitrary column-like features without reinventing the wheel and putting a database in the database. ISTM you could define a composite type and then create operators and an operator class over that type.

Re: [HACKERS] Designing an extension for feature-space similarity search

2012-02-16 Thread Jay Levitt
Tom Lane wrote: - Can domains have operators, or are operators defined on types? I think the current state of play is that you can have such things but the system will only consider them for exact type matches, so you might need more explicit casts than you ordinarily would. Turns out it's

[HACKERS] Copyright notice for contrib/cube?

2012-02-17 Thread Jay Levitt
I'm basing an extension off contrib/cube. I'm going to open-source it under the existing PostgreSQL license, but I'm not sure how the copyright notice should look - there isn't one at the moment. (In fact, there's no LICENSE or COPYRIGHT file at all.) Should it be something like Portions

Re: [HACKERS] Copyright notice for contrib/cube?

2012-02-17 Thread Jay Levitt
Marti Raudsepp wrote: On Fri, Feb 17, 2012 at 17:42, Jay Levittjay.lev...@gmail.com wrote: Should it be something like Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group Portions Copyright (c) 2012, TipTap Inc. Please don't add that, just change 2011 to 2012. This is what

Re: [HACKERS] Designing an extension for feature-space similarity search

2012-02-17 Thread Jay Levitt
Tom Lane wrote: Jay Levittjay.lev...@gmail.com writes: - Does KNN-GiST run into problems when- returns values that don't make sense in the physical world? If the indexed entities are records, it would be entirely your own business how you handled individual fields being NULL. This turns

Re: [HACKERS] Designing an extension for feature-space similarity search

2012-02-17 Thread Jay Levitt
Alexander Korotkov wrote: On Fri, Feb 17, 2012 at 11:00 PM, Jay Levitt jay.lev...@gmail.com mailto:jay.lev...@gmail.com wrote: At first I thought this posed a challenge for union; if I have these points: (1,2) (2,1) (1,NULL) what's the union? I think the answer

Re: [HACKERS] Designing an extension for feature-space similarity search

2012-02-17 Thread Jay Levitt
Alexander Korotkov wrote: On Fri, Feb 17, 2012 at 11:32 PM, Jay Levitt jay.lev...@gmail.com Ah, yes, exactly the same problem. So what led you to add a flag instead of using the range NULL..NULL? I'm on the fence about choosing. At first, range bounds can't be NULL :) At second

Re: [HACKERS] Future of our regular expression code

2012-02-19 Thread Jay Levitt
Stephen Frost wrote: Alright, I'll bite.. Which existing regexp implementation that's well written, well maintained, and which is well protected against malicious regexes should we be considering then? FWIW, there's a benchmark here that compares a number of regexp engines, including PCRE,

Re: [HACKERS] pg_test_timing tool for EXPLAIN ANALYZE overhead

2012-02-22 Thread Jay Levitt
Greg Smith wrote: Anyway, the patch does now includes several examples and a short primer on PC clock hardware, to help guide what good results look like and why they've been impossible to obtain in the past. That's a bit Linux-centric, but the hardware described covers almost all systems

[HACKERS] Pushing restrictions down into GROUP BYs?

2012-09-22 Thread Jay Levitt
A while back, I posted a pathological minimal-case query where, in order to select one row from a users table, Postgres needed to scan the whole users table, because the restriction was not visible to the GROUP BY. At the time, Tom wrote: Don't hold your breath waiting for that to change.

Re: [HACKERS] [PATCH] Lazy hashaggregate when no aggregation is needed

2012-03-30 Thread Jay Levitt
Tom Lane wrote: Ants Aasmaa...@cybertec.at writes: A user complained on pgsql-performance that SELECT col FROM table GROUP BY col LIMIT 2; performs a full table scan. ISTM that it's safe to return tuples from hash-aggregate as they are found when no aggregate functions are in use. Attached is a

[HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-01 Thread Jay Levitt
to the PG core community, and maybe the link juice/publicity is politically important. Lemme know. That's all I can think of... thoughts? Objections? Which do you think are prerequisites? Jay Levitt -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-01 Thread Jay Levitt
Jay Levitt wrote: POSSIBLE OBJECTIONS/PREREQUISITES 10. There is no homebrew support for multiple versions, and no current plans to add it (though it's on the wishlist). This means homebrew is only useful if I want to install a PostgreSQL thingie is the common Mac use case. If people often

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-01 Thread Jay Levitt
Dave Page wrote: It seems to me that most of your arguments against the installers are based on incorrect understanding or information, and most of your arguments for Homebrew actually come across as arguments against! You're right about the former - and as to the latter, they *were*

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-01 Thread Jay Levitt
Tom Lane wrote: While you might not like the EDB installer, at least those folks are active in the lists and accountable for whatever problems their code has. Who in heck is responsible for the homebrew packaging, and do they answer questions in the PG lists? Just for general knowledge...

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-02 Thread Jay Levitt
Dave Page wrote: On Mon, Apr 2, 2012 at 12:29 AM, Jay Levittjay.lev...@gmail.com wrote: Just as an FYI, a large percentage of the PostgreSQL developers are Mac users, including myself. They're also the company standard at EnterpriseDB - so we're not entirely unfamiliar with software development

Re: [HACKERS] Switching to Homebrew as recommended Mac install? / apology

2012-04-02 Thread Jay Levitt
David Johnston wrote: Just trying to bridge an apparent gap since the original e-mail seems to have come across as too adversarial that the underlying thoughts have been overlooked. Trying to contribute in my own way with my current resources. Thanks, but it's my own fault for basing a

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-03 Thread Jay Levitt
Robert Haas wrote: On Mon, Apr 2, 2012 at 5:23 AM, Dave Pagedp...@pgadmin.org wrote: If homebrew intentionally creates a hole like that, then for as long as I'm one of the PostgreSQL webmasters it will *never* be listed on our download pages. I think that's a bit harsh. It's not as if the

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-04 Thread Jay Levitt
Dave Page wrote: Exactly - which is why I was objecting to recommending a distribution of PostgreSQL that came in a packaging system that we were told changed /usr/local to be world writeable to avoid the use/annoyance of the standard security measures on the platform. We... that's not

Re: [HACKERS] Last gasp

2012-04-12 Thread Jay Levitt
to write tools (both workflow and archival) against, etc. Rather than extend the CF app into a trivial-patch workflow app, it might be worth looking at integrating it with github. Jay Levitt -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Last gasp

2012-04-14 Thread Jay Levitt
Christopher Browne wrote: On Thu, Apr 12, 2012 at 6:11 PM, Jay Levittjay.lev...@gmail.com wrote: Rather than extend the CF app into a trivial-patch workflow app, it might be worth looking at integrating it with github. There's a reluctance to require a proprietary component that could

Re: [HACKERS] Last gasp

2012-04-14 Thread Jay Levitt
Yeah, it was a really good idea, though. Jay Levitt -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Last gasp

2012-04-16 Thread Jay Levitt
Alex wrote: Jay Levittjay.lev...@gmail.com writes: Alex wrote: I didn't follow this whole thread, but have we considered Redmine[1]? As the resident Ruby is shiny, let's do everything in Rails on my MacBook guy, I'd like to make a statement against interest: I've tried Redmine a few times

Re: [HACKERS] Bug tracker tool we need

2012-04-16 Thread Jay Levitt
Alex wrote: I still fail to see how Redmine doesn't fit into requirements summarized at that wiki page[1], so that must be something other than formal requirement of being free/open software and running postgres behind (some sort of feeling maybe?) Well, if those requirements are in fact

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2012-04-16 Thread Jay Levitt
Simon Riggs wrote: I'd like to see something along the lines of demand-created optional indexes, that we reclaim space/maintenance overhead on according to some cache management scheme. More space you have, the more of the important ones hang around. The rough same idea applies to materialised

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Jay Levitt
Magnus Hagander wrote: On Mon, Apr 16, 2012 at 23:48, Jay Levittjay.lev...@gmail.com wrote: - Familiarity: Many developers already have a GitHub account and use it Most of the more senior developers don't use github. Other than possibly as a place to store a plain git repository. So that's

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Jay Levitt
Alex Shulgin wrote: Jay Levittjay.lev...@gmail.com writes: (A quick Google shows redmine and especially Trac having spam issues of their own.) Ugh, redmine (or trac for that matters) has nothing to with handling spam. I believe a typical bug tracker doesn't handle spam itself, it lets the

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Jay Levitt
Greg Smith wrote: On 04/17/2012 09:20 AM, Jay Levitt wrote: Antispam is (in the large) a technically unsolvable problem; even in the '90s, we'd see hackers start poking at our newest countermeasures within the hour. GitHub is a giant target, and PG probably benefits here from NOT being one

Re: [HACKERS] 9.2 release notes, beta time?

2012-04-29 Thread Jay Levitt
Magnus Hagander wrote: 2012/4/28 Josh Berkusj...@agliodbs.com: Ugh. Maybe the whole idea of getting a beta out before PGCon is doomed. Still, if we don't try for this schedule, we're looking at at least two more weeks' slip, because we're surely not going to wrap during PGCon. We could do it

Re: [HACKERS] Request to add options to tools/git_changelog

2012-04-29 Thread Jay Levitt
Bruce Momjian wrote: I am again requesting the addition of options to tools/git_changelog so I can more easily produce the release notes. I asked for this during 9.1 development and it was rejected. I am currently using my own custom version of the tool, but have to merge community