[
https://issues.apache.org/jira/browse/SOLR-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274229#comment-14274229
]
Praneeth commented on SOLR-6367:
--------------------------------
I won't be able to get back fast enough during work hours. Sorry about that.
I've tried it with Solr 4.10.3 and I am able to reproduce this consistently.
You have your autocommit 'on' may be? and the document is getting committed
before you kill it? I am running against cloudera distribution of hadoop -
{{2.0.0-cdh4.6.0}}.
I did notice SOLR-6969. Though it kept switching between
{{AlreadyBeingCreatedException}} and {{RecoveryInProgressException}} . I guess
the later happens based on how fast you restart, whether the soft limit has
expired or not may be?
I don't think that these issues are related. I think SOLR-6969 happens on quick
restart after every hard crash. But this current issue here is due to documents
not making to the tlog. The file has nothing written to it before the crash. I
haven't looked deeply into it but could this be because of underlying stream
implementation with an intermediate buffer may be? That is why the local
transaction log does not show such behaviour? This part complete speculation at
this point and I will dig into it later tonight.
> empty tlog on HDFS when hard crash - no docs to replay on recovery
> ------------------------------------------------------------------
>
> Key: SOLR-6367
> URL: https://issues.apache.org/jira/browse/SOLR-6367
> Project: Solr
> Issue Type: Bug
> Reporter: Hoss Man
> Assignee: Mark Miller
> Fix For: 5.0, Trunk
>
>
> Filing this bug based on an email to solr-user@lucene from Tom Chen (Fri, 18
> Jul 2014)...
> {panel}
> Reproduce steps:
> 1) Setup Solr to run on HDFS like this:
> {noformat}
> java -Dsolr.directoryFactory=HdfsDirectoryFactory
> -Dsolr.lock.type=hdfs
> -Dsolr.hdfs.home=hdfs://host:port/path
> {noformat}
> For the purpose of this testing, turn off the default auto commit in
> solrconfig.xml, i.e. comment out autoCommit like this:
> {code}
> <!--
> <autoCommit>
> <maxTime>${solr.autoCommit.maxTime:15000}</maxTime>
> <openSearcher>false</openSearcher>
> </autoCommit>
> -->
> {code}
> 2) Add a document without commit:
> {{curl "http://localhost:8983/solr/collection1/update?commit=false"; -H
> "Content-type:text/xml; charset=utf-8" --data-binary "@solr.xml"}}
> 3) Solr generate empty tlog file (0 file size, the last one ends with 6):
> {noformat}
> [hadoop@hdtest042 exampledocs]$ hadoop fs -ls
> /path/collection1/core_node1/data/tlog
> Found 5 items
> -rw-r--r-- 1 hadoop hadoop 667 2014-07-18 08:47
> /path/collection1/core_node1/data/tlog/tlog.0000000000000000001
> -rw-r--r-- 1 hadoop hadoop 67 2014-07-18 08:47
> /path/collection1/core_node1/data/tlog/tlog.0000000000000000003
> -rw-r--r-- 1 hadoop hadoop 667 2014-07-18 08:47
> /path/collection1/core_node1/data/tlog/tlog.0000000000000000004
> -rw-r--r-- 1 hadoop hadoop 0 2014-07-18 09:02
> /path/collection1/core_node1/data/tlog/tlog.0000000000000000005
> -rw-r--r-- 1 hadoop hadoop 0 2014-07-18 09:02
> /path/collection1/core_node1/data/tlog/tlog.0000000000000000006
> {noformat}
> 4) Simulate Solr crash by killing the process with -9 option.
> 5) restart the Solr process. Observation is that uncommitted document are
> not replayed, files in tlog directory are cleaned up. Hence uncommitted
> document(s) is lost.
> Am I missing anything or this is a bug?
> BTW, additional observations:
> a) If in step 4) Solr is stopped gracefully (i.e. without -9 option),
> non-empty tlog file is geneated and after re-starting Solr, uncommitted
> document is replayed as expected.
> b) If Solr doesn't run on HDFS (i.e. on local file system), this issue is
> not observed either.
> {panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
