[
https://issues.apache.org/jira/browse/CSV-131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14204216#comment-14204216
]
Holger Stratmann commented on CSV-131:
--------------------------------------
sorry again for being slow... got latest updates today to prepare a new patch,
but saw that you already made the changes I wanted to make. Current code is
fine for me (I'll take another look because something has changed in the
numbering of records (when there's a header?) and I still haven't had time to
investigate this today) and I think it's the best solution (might be even nicer
to store a "recordOffset" instead of setting the recordNumber directly, but
it's fine like it is, I think we might be overengineering a simple field :)).
Thanks for taking care of this!
> Save positions of records to enable random access
> -------------------------------------------------
>
> Key: CSV-131
> URL: https://issues.apache.org/jira/browse/CSV-131
> Project: Commons CSV
> Issue Type: Improvement
> Components: Parser
> Affects Versions: 1.1
> Reporter: Holger Stratmann
> Priority: Minor
> Attachments: CSV-131-gg-0.diff,
> PositionTrackingFull_v101_20140910.patch,
> PositionTrackingTest_20140907.patch, PositionTracking_20140907.patch,
> ggregory-CSV-131-parser-and-record.diff
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> It would be good to have {{CSVRecord}} save its position in the source stream.
> Reason: Knowing the position of the records would enable random access to
> retrieve records from the source (after reading it once to build an index) if
> the file is too large to be read into memory (or if we don't want to read the
> full file to access a record in the middle).
> Additional info: I have created a "random access csv reader" and a "csv
> viewer" (Swing) for arbitrarily large CSV files. It requires one additional
> scan of the file to build an index (multi-byte charsets supported). The index
> can be saved to a file so it only needs to be built once. Because the lexer
> uses a BufferedReader, we need "internal information" to know where each
> record starts.
> The change to "core" is minor: one field in {{CSVRecord}}s and some
> associated methods to store the position.
> Patch will be attached.
> Code for random access (both UI and non-UI) will be proposed (and possibly
> submitted) as a separate issue. It could also be an independent add-on but
> requires this one little change to Commons CSV.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
