[
https://issues.apache.org/jira/browse/BLUR-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravikumar updated BLUR-290:
---------------------------
Attachment: BlurIndexTracker.java
BlurRealTimeIndex.java
RealTimeTransactionRecorder.java
BlurRealTimeManagerReopenThread.java
SlabRAMOutputStream.java
SlabRAMInputStream.java
SlabAllocator.java
SlabRAMFile.java
SlabRAMDirectory.java
SortingMultiReader.java
BlurFlushingIndexWriter.java
BlurRealTimeManager.java
BlurRealTimeIndexWriter.java
The basic points, I have dwelled upon is
1. RealTime index holds a current IW. It keeps accumulating docs there, until
the "Commit Thread" interrupts and swaps it.
2. Once a swap happens, a new transaction-log is opened
3. Previous data gets sort-flushed to disk and committed. At this point, the
old-transaction log is also deleted.
3. IndexSearcher is actually a MultiReader holding one disk-reader and many
RAM-readers.
4. NRT re-opens are co-ordinated during "Step 3" to avoid duplicate records
being searched etc...
Sources will not compile. I am attaching only few, to get initial reactions
[~amccurry] I have just tried to handle the write-path alone. Search/Read path
can be iterated later if this patch seems worthwhile.
> NRT Updates using RAMDirectory & Swap
> -------------------------------------
>
> Key: BLUR-290
> URL: https://issues.apache.org/jira/browse/BLUR-290
> Project: Apache Blur
> Issue Type: New Feature
> Affects Versions: experimental-dev
> Reporter: Ravikumar
> Attachments: BlurFlushingIndexWriter.java, BlurIndexTracker.java,
> BlurRealTimeIndex.java, BlurRealTimeIndexWriter.java,
> BlurRealTimeManager.java, BlurRealTimeManagerReopenThread.java,
> RealTimeTransactionRecorder.java, SlabAllocator.java, SlabRAMDirectory.java,
> SlabRAMFile.java, SlabRAMInputStream.java, SlabRAMOutputStream.java,
> SortingMultiReader.java
>
>
> We have been discussing about handling humungous rows in Blur (BLUR-220).
> Explore the idea of using RAMDirectory at the front, backed by
> persistent-index.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
