Upon a request on the experiences on this issue, I am posting the most
important functions of the program. Every DB record maps directly to one
file. The function that I did not include is "getDataSource()" which
acquires a jdbc datasource to your database.
cheers,
Christoph
private void run() throws Exception {
DataSource ds = getDataSource();
File toDir = new File("outputDir");
toDir.mkdirs();
assert toDir.exists();
assert toDir.isDirectory();
copyToFilesystem(ds, toDir);
}
public void copyToFilesystem(DataSource ds, File toDir)
throws SQLException, IOException {
Connection conn = ds.getConnection();
Statement stmt = conn.createStatement();
ResultSet rs = stmt
.executeQuery("select NAME_, VALUE_ from
XXXX.XXXX");
while (rs.next()) {
String name = rs.getString("NAME_");
log.info("filename: '" + name + "'");
InputStream inStream = rs.getBinaryStream("VALUE_");
File file = new File(toDir, name);
assert !file.exists();
boolean ok = file.createNewFile();
assert ok;
FileOutputStream outStream = new FileOutputStream(file);
copyLarge(inStream, outStream);
inStream.close();
outStream.close();
}
conn.rollback();
}
/**
* Taken from commons IO
*
* @see
http://svn.apache.org/viewvc/commons/proper/io/trunk/src/java/org/
* apache/commons/io/IOUtils.java?revision=736890
*/
public static long copyLarge(InputStream input, OutputStream output)
throws IOException {
byte[] buffer = new byte[DEFAULT_BUFFER_SIZE];
long count = 0;
int n = 0;
while (-1 != (n = input.read(buffer))) {
output.write(buffer, 0, n);
count += n;
}
return count;
}
ChristophD wrote:
>
> Ok, so let me clear it up.
>
> Lucene offers different types of Directories
> (org.apache.lucene.store.Directory) into which it stores the index data.
> Most people probably use the FSDirectory implementation which writes the
> index data as files into the filesystem. However, we use the DbDirectory
> implementation which writes into a specified relational database.
>
> Now, I was really surprised to see that Luke only offers to open an index
> that was written to the filesystem. I had expected to be able to supply a
> jdbc url.
>
> The way Lucene writes the index into the DB is really a direct projection
> of the FS version. For every file it creates a record in the DB which has
> a name and a BLOB that contains the file's data.
>
> So what I did was, I wrote a small program that reads the index from the
> DB and writes it back as files. These files I could open with Luke without
> problems.
>
> so long,
> Christoph
>
>
> Erick Erickson wrote:
>>
>> Well, you haven't really provided much in the way of details.For
>> instance,
>> what does it mean that your Lucene index is
>> stored in a database"? Did you store it as a BLOB? Your
>> problem statement is very hard to understand, please explain
>> in more detail. Pretend you don't know a thing about your
>> app (as in, you're just a random reader of this list) and imagine
>> you were trying to understand well enough to offer useful
>> responses...
>>
>> Best
>> Erick
>>
>> On Tue, May 19, 2009 at 6:12 AM, ChristophD <[email protected]>
>> wrote:
>>
>>>
>>> This isn't really addressing my problem. I already have a running search
>>> system and just want to analyze it.
>>>
>>> cheers,
>>> Christoph
>>>
>>>
>>>
>>> amin1977 wrote:
>>> >
>>> > Are you using an object relational mapping tool like Hibernate? if
>>> you
>>> > are
>>> > you could use hibernate search to index your persistent entities and
>>> then
>>> > use luke to inspect the indexes. There may other ways of doing it I
>>> > guess. Just a thought.
>>> >
>>> >
>>> > Cheers
>>> > Amin
>>> >
>>> > On Tue, May 19, 2009 at 9:23 AM, ChristophD <[email protected]>
>>> > wrote:
>>> >
>>> >>
>>> >> Hello,
>>> >>
>>> >> I would like to use Luke to connect to an existing Lucene Index which
>>> is
>>> >> stored in a Database.
>>> >> However, Luke seems only to be able to read file based indexes.
>>> >>
>>> >> What are my options to analyze the DB index?
>>> >>
>>> >> thx,
>>> >> Christoph
>>> >> --
>>> >> View this message in context:
>>> >>
>>> http://www.nabble.com/Using-Luke-on-a-Lucene-Index-in-a-Database-tp23611846p23611846.html
>>> >> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>> >>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: [email protected]
>>> >> For additional commands, e-mail: [email protected]
>>> >>
>>> >>
>>> >
>>> >
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Using-Luke-on-a-Lucene-Index-in-a-Database-tp23611846p23613338.html
>>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [email protected]
>>> For additional commands, e-mail: [email protected]
>>>
>>>
>>
>>
>
>
--
View this message in context:
http://www.nabble.com/Using-Luke-on-a-Lucene-Index-in-a-Database-tp23611846p23942165.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
