[
https://issues.apache.org/jira/browse/SOLR-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271699#comment-14271699
]
Mark Miller commented on SOLR-6367:
-----------------------------------
[~praneeth.varma], but we hflush or hsync on the stream that FastOutputStream
wraps right after.
The non HdfsTransactionLog also only calls flushBuffer unless you configure to
then fsync.
It may be necessary to call flush instead of flushBuffer here, but I don't
understand why that would be yet.
> empty tlog on HDFS when hard crash - no docs to replay on recovery
> ------------------------------------------------------------------
>
> Key: SOLR-6367
> URL: https://issues.apache.org/jira/browse/SOLR-6367
> Project: Solr
> Issue Type: Bug
> Reporter: Hoss Man
> Assignee: Mark Miller
> Fix For: 5.0, Trunk
>
>
> Filing this bug based on an email to solr-user@lucene from Tom Chen (Fri, 18
> Jul 2014)...
> {panel}
> Reproduce steps:
> 1) Setup Solr to run on HDFS like this:
> {noformat}
> java -Dsolr.directoryFactory=HdfsDirectoryFactory
> -Dsolr.lock.type=hdfs
> -Dsolr.hdfs.home=hdfs://host:port/path
> {noformat}
> For the purpose of this testing, turn off the default auto commit in
> solrconfig.xml, i.e. comment out autoCommit like this:
> {code}
> <!--
> <autoCommit>
> <maxTime>${solr.autoCommit.maxTime:15000}</maxTime>
> <openSearcher>false</openSearcher>
> </autoCommit>
> -->
> {code}
> 2) Add a document without commit:
> {{curl "http://localhost:8983/solr/collection1/update?commit=false"; -H
> "Content-type:text/xml; charset=utf-8" --data-binary "@solr.xml"}}
> 3) Solr generate empty tlog file (0 file size, the last one ends with 6):
> {noformat}
> [hadoop@hdtest042 exampledocs]$ hadoop fs -ls
> /path/collection1/core_node1/data/tlog
> Found 5 items
> -rw-r--r-- 1 hadoop hadoop 667 2014-07-18 08:47
> /path/collection1/core_node1/data/tlog/tlog.0000000000000000001
> -rw-r--r-- 1 hadoop hadoop 67 2014-07-18 08:47
> /path/collection1/core_node1/data/tlog/tlog.0000000000000000003
> -rw-r--r-- 1 hadoop hadoop 667 2014-07-18 08:47
> /path/collection1/core_node1/data/tlog/tlog.0000000000000000004
> -rw-r--r-- 1 hadoop hadoop 0 2014-07-18 09:02
> /path/collection1/core_node1/data/tlog/tlog.0000000000000000005
> -rw-r--r-- 1 hadoop hadoop 0 2014-07-18 09:02
> /path/collection1/core_node1/data/tlog/tlog.0000000000000000006
> {noformat}
> 4) Simulate Solr crash by killing the process with -9 option.
> 5) restart the Solr process. Observation is that uncommitted document are
> not replayed, files in tlog directory are cleaned up. Hence uncommitted
> document(s) is lost.
> Am I missing anything or this is a bug?
> BTW, additional observations:
> a) If in step 4) Solr is stopped gracefully (i.e. without -9 option),
> non-empty tlog file is geneated and after re-starting Solr, uncommitted
> document is replayed as expected.
> b) If Solr doesn't run on HDFS (i.e. on local file system), this issue is
> not observed either.
> {panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
