New information: it appears that the index size increasing (not always doubling but going up significantly) occurs when I search the index while building it. Calling indexWriter.optimize(1, true); when I'm done adding documents sometimes reduces the index down to size, but not always.
Has anyone else seen this?