David Johnston wrote:
> Just trying to bridge an apparent gap since the original e-mail seems to
> have come across as too adversarial that the underlying thoughts have
> been overlooked. Trying to contribute in my own way with my current
> resources.
Thanks, but it's my own fault for basing a half-baked "let's rewrite
everything" idea on a few wrong assumptions without asking some simple
questions first. (Sorry, David.) And you guys don't know me very well yet,
so you don't how to interpret my tone, or that I spend the *first* half of
each day making the exact opposite arguments to all the young turks in the
hopes we'll all meet in the middle. I plan to hang around, so by way of
belated introduction, and you can imagine this in the style of Beetlejuice:
Hi. I wrote AOL's OLTP-style mail server in 1994 and scaled it (with an
eventual team of 25) from 27 to 4000 TPS on minicomputers as powerful as an
iPhone. It had multi-site replication, zero-downtime installs (without
blocking writes), and served 1.5 million simultaneous users. I had to write
nearly a whole SVM-based OS in the app, because nothing at the time - from
the message bus to the disk cache to poll() itself - could handle our load,
and our growth outpaced the hardware manufacturers' ability to build it. I
did this with no CS background beyond public school (I started as a 6502
hacker), no search engine, and no access to research papers or computer
scientists. I have no idea how.
The architecture survived five underlying infrastructure transitions
(Stratus filesystem, Sybase, Informix, Tandem, and the move from four- to
eight-byte IDs that PG has successfully staved off) while migrating live. It
never lost or misrouted a message, and never had a security breach in the
nine years I led it (or, AFAIK, since), despite the fact that we were a big
enough target for hackers to respond to our changed defenses within hours. I
do actually know this stuff, or did.
I spent 10 years taking a break, studying music, successfully sleeping
through Java; now I'm back in technology, leading development in a
Rails/web/JS startup, and luxuriating in the ability to actually store data
in a database this time - because you guys have spent the past 20 years
studying and improving the hard parts of abstracting performant, reliable,
scalable data storage.
I have a tendency to see the possible endgame and insist on starting now,
and if I get too idealistic, ambitious, and "MongoDB is webscale", please
just drag me into a conversation about lock contention and distributed 2PC
and I'll forget the whole thing. But I really do think PG can be the
makes-everything-possible, does-most-things-well data store - really, data
platform - for the next decade or two, and I want to contribute.
I'm provocative, playful and grandiose, I apologize except not really, and
it's because in my n=1 experience, the way life works is (a) you decide to
change the world and then (b) you do.
> You do not need permission to contribute to the community
> in the way you seek so what is it that you are really asking for?
Nothing at this point. I was thinking out loud, and at the time was
temporarily insa^h^h^hconvinced that the homebrew formula should be the
community standard, and thus that I'd have to bring it up to some level of
acceptability/review. I've contributed to the formula in the past, and will
continue to do so based on the thoughts everyone's shared here. It doesn't
need to be official to be useful, and as David Page said, it's not gonna be
listed in the docs no matter what, given the one decision that homebrew
makes (/usr/local) that I can't override.
When brew is replaced by something more popular do you
think you will continue to maintain the recipe or is it going to end
up stuck showing us how to install version 9.3 or earlier.
Like anything, I'll maintain it until it becomes useless to me or vice
versa, and someone will pick it up or they won't. But just to be clear,
Homebrew's a source-based repo (so there's no cross-compiler issues),
pulling from the upstream source repository, using only the stock compiler
toolchain, Intel-only, on a platform where the only hardware manufacturer
has themselves severely constrained the number of possible configurations.
For the most part, updating the formula to "package" new versions is a
matter of changing the following two lines:
url 'http://ftp.postgresql.org/pub/source/v9.1.3/postgresql-9.1.3.tar.bz2'
md5 '641e1915f7ebfdc9f138e4c55b6aec0e'
Unless the instructions for "How to build postgres from source" change,
nothing else in the formula *needs* to. The current formula is fairly
simple; aside from user instructions, the code is 75 lines and mostly
consists of default arguments to ./configure. (Formula attached for the
curious.) Pull requests are freely and quickly accepted after a quick
review; the homebrew repo is operated more in the "fail early and often"
spirit than in the mission-critical "do no releases before formal
verification" style - and again, since it contains nothing more than
programmatic versions of "install this package from source" docs on a
constrained platform, it doesn't tend to fail catastrophically. It's for
developer tools, not deployments.
the current installer maintainers are doing so in addition to their
> other, more regular, contributions
Yes, and I do plan to contribute back to PostgreSQL as well. We have a bunch
of novel uses that could serve as forcing functions for nice optimizations
and generalizations, and my lifelong obsession with scaling, performance and
metering meshes well with some of the core team's future plans. But as a PG
novice, a SQL novice, and someone who last touched C in 1999 other than a
few bug fixes, believe me that I am contributing more by *not* trying to
contribute any code *just* yet.
Someone would likely pickup the torch
Agreed; the PG formula has had 35 contributors in its 2.5 years, and the
9.1.3 update was released 41 minutes after the e-mail announcement went out.
The homebrew repo's at https://github.com/mxcl/homebrew, and the license
(homebrew-wide) is unnamed but looks MIT-compatible. It does not contain the
PG copyright notices, and I don't know the history of the PG license, so
perhaps that also prevents some here from contributing (although, again,
there is no PG code in homebrew).
Jay
require 'formula'
class Postgresql < Formula
homepage 'http://www.postgresql.org/'
url 'http://ftp.postgresql.org/pub/source/v9.1.3/postgresql-9.1.3.tar.bz2'
md5 '641e1915f7ebfdc9f138e4c55b6aec0e'
depends_on 'readline'
depends_on 'libxml2' if MacOS.leopard? # Leopard libxml is too old
depends_on 'ossp-uuid'
def options
[
['--32-bit', 'Build 32-bit only.'],
['--no-python', 'Build without Python support.'],
['--no-perl', 'Build without Perl support.'],
['--enable-dtrace', 'Build with DTrace support.']
]
end
skip_clean :all
def install
ENV.libxml2 if MacOS.snow_leopard?
args = ["--disable-debug",
"--prefix=#{prefix}",
"--datadir=#{share}/#{name}",
"--docdir=#{doc}",
"--enable-thread-safety",
"--with-bonjour",
"--with-gssapi",
"--with-krb5",
"--with-openssl",
"--with-libxml",
"--with-libxslt",
"--with-ossp-uuid"]
args << "--with-python" unless ARGV.include? '--no-python'
args << "--with-perl" unless ARGV.include? '--no-perl'
args << "--enable-dtrace" if ARGV.include? '--enable-dtrace'
ENV.append 'CFLAGS', `uuid-config --cflags`.strip
ENV.append 'LDFLAGS', `uuid-config --ldflags`.strip
ENV.append 'LIBS', `uuid-config --libs`.strip
if not ARGV.build_32_bit? and MacOS.prefer_64_bit? and not ARGV.include? '--no-python'
args << "ARCHFLAGS='-arch x86_64'"
check_python_arch
end
if ARGV.build_32_bit?
ENV.append 'CFLAGS', '-arch i386'
ENV.append 'LDFLAGS', '-arch i386'
end
# Fails on Core Duo with O4 and O3
ENV.O2 if Hardware.intel_family == :core
system "./configure", *args
system "make install-world"
plist_path.write startup_plist
plist_path.chmod 0644
end
def check_python_arch
# On 64-bit systems, we need to look for a 32-bit Framework Python.
# The configure script prefers this Python version, and if it doesn't
# have 64-bit support then linking will fail.
framework_python = Pathname.new "/Library/Frameworks/Python.framework/Versions/Current/Python"
return unless framework_python.exist?
unless (archs_for_command framework_python).include? :x86_64
opoo "Detected a framework Python that does not have 64-bit support in:"
puts <<-EOS.undent
#{framework_python}
The configure script seems to prefer this version of Python over any others,
so you may experience linker problems as described in:
http://osdir.com/ml/pgsql-general/2009-09/msg00160.html
To fix this issue, you may need to either delete the version of Python
shown above, or move it out of the way before brewing PostgreSQL.
Note that a framework Python in /Library/Frameworks/Python.framework is
the "MacPython" version, and not the system-provided version which is in:
/System/Library/Frameworks/Python.framework
EOS
end
end
def caveats
s = <<-EOS
# Build Notes
If builds of PostgreSQL 9 are failing and you have version 8.x installed,
you may need to remove the previous version first. See:
https://github.com/mxcl/homebrew/issues/issue/2510
To build plpython against a specific Python, set PYTHON prior to brewing:
PYTHON=/usr/local/bin/python brew install postgresql
See:
http://www.postgresql.org/docs/9.1/static/install-procedure.html
# Create/Upgrade a Database
If this is your first install, create a database with:
initdb #{var}/postgres
To migrate existing data from a previous major version (pre-9.1) of PostgreSQL, see:
http://www.postgresql.org/docs/9.1/static/upgrading.html
# Start/Stop PostgreSQL
If this is your first install, automatically load on login with:
mkdir -p ~/Library/LaunchAgents
cp #{plist_path} ~/Library/LaunchAgents/
launchctl load -w ~/Library/LaunchAgents/#{plist_path.basename}
If this is an upgrade and you already have the #{plist_path.basename} loaded:
launchctl unload -w ~/Library/LaunchAgents/#{plist_path.basename}
cp #{plist_path} ~/Library/LaunchAgents/
launchctl load -w ~/Library/LaunchAgents/#{plist_path.basename}
Or start manually with:
pg_ctl -D #{var}/postgres -l #{var}/postgres/server.log start
And stop with:
pg_ctl -D #{var}/postgres stop -s -m fast
# Loading Extensions
By default, Homebrew builds all available Contrib extensions. To see a list of all
available extensions, from the psql command line, run:
SELECT * FROM pg_available_extensions;
To load any of the extension names, navigate to the desired database and run:
CREATE EXTENSION [extension name];
For instance, to load the tablefunc extension in the current database, run:
CREATE EXTENSION tablefunc;
For more information on the CREATE EXTENSION command, see:
http://www.postgresql.org/docs/9.1/static/sql-createextension.html
For more information on extensions, see:
http://www.postgresql.org/docs/9.1/static/contrib.html
# Other
Some machines may require provisioning of shared memory:
http://www.postgresql.org/docs/current/static/kernel-resources.html#SYSVIPC
EOS
if MacOS.prefer_64_bit? then
s << <<-EOS
To install postgresql (and ossp-uuid) in 32-bit mode:
brew install postgresql --32-bit
If you want to install the postgres gem, including ARCHFLAGS is recommended:
env ARCHFLAGS="-arch x86_64" gem install pg
To install gems without sudo, see the Homebrew wiki.
EOS
end
return s
end
def startup_plist
return <<-EOPLIST
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd";>
<plist version="1.0">
<dict>
<key>KeepAlive</key>
<true/>
<key>Label</key>
<string>#{plist_name}</string>
<key>ProgramArguments</key>
<array>
<string>#{HOMEBREW_PREFIX}/bin/postgres</string>
<string>-D</string>
<string>#{var}/postgres</string>
<string>-r</string>
<string>#{var}/postgres/server.log</string>
</array>
<key>RunAtLoad</key>
<true/>
<key>UserName</key>
<string>#{`whoami`.chomp}</string>
<key>WorkingDirectory</key>
<string>#{HOMEBREW_PREFIX}</string>
<key>StandardErrorPath</key>
<string>#{var}/postgres/server.log</string>
</dict>
</plist>
EOPLIST
end
end
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
