Hi Bruno,
Getting an exception on the below condition, please give some direction.
is it related to file buffer size or is there any possibility that the file
is corrupt before the reading is complete?
class - org.apache.solr.encryption.crypto.DecryptingIndexInput
method -
@Override
public void readBytes(byte[] b, int offset, int length) throws IOException {
if (offset < 0 || length < 0 || offset + length > b.length) {
throw new IllegalArgumentException("Invalid read buffer parameters (offset="
+ offset + ", length=" + length
+ ", arrayLength=" + b.length + ")");
}
* if (getPosition() + length > sliceEnd) {*
* throw new EOFException("Read beyond EOF (position=" + (getPosition() -
sliceOffset) + ", arrayLength=" + length*
* + ", fileLength=" + length() + ") in " + this);*
* }*
while (length > 0) {
// Transfer decrypted bytes from outBuffer.
int outRemaining = outSize - outPos;
if (outRemaining > 0) {
if (length <= outRemaining) {
System.arraycopy(outBuffer, outPos, b, offset, length);
outPos += length;
return;
}
System.arraycopy(outBuffer, outPos, b, offset, outRemaining);
outPos += outRemaining;
offset += outRemaining;
length -= outRemaining;
}
readToFillBuffer(length);
decryptBuffer();
}
}
Regards,
Manish Kumar
On Sat, Jan 18, 2025 at 1:19 PM Manish Kumar singh <mksmksi...@gmail.com>
wrote:
> Hi Bruno,
>
>
> I have reproduced the issue, below are 2 scenarios -
>
> 1. On a 128 GB L3 distributed system (data size 12 million) replica
> recovery fails after all Solr DS restart restart.
>
> 2. On a 256 GB L3 distributed system (data size 12 million) replica
> recovery does not fail on all Solr DS restart. I deleted one active replica
> then it went to recovery mode and recovery failed. Getting below error log
>
> [2025-01-17T12:26:11.472] WARN [host]
> org.apache.solr.handler.ReplicationHandler Exception while writing response
> for params:
> generation=82&qt=/replication&file=_6p.kdd&offset=8388608&checksum=true&wt=filestream&command=filecontent
> java.io.EOFException: Read beyond EOF (position=8388608,
> arrayLength=424066, fileLength=8812650) in Decrypting
> MemorySegmentIndexInput(path="/data/1155855_shard1_replica_n26/data/index/_6p.kdd")
> at
> org.apache.solr.encryption.crypto.DecryptingIndexInput.readBytes(DecryptingIndexInput.java:223)
> ~[solr-encryption-plugin-9.0.0-SNAPSHOT.jar:?]
> at
> org.apache.solr.handler.ReplicationHandler$DirectoryFileStream.write(ReplicationHandler.java:1635)
> ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus
> - 2024-04-22 23:20:52]
> at org.apache.solr.core.SolrCore$3.write(SolrCore.java:3056)
> ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus
> - 2024-04-22 23:20:52]
> at
> org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:59)
> ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus
> - 2024-04-22 23:20:52]
> at
> org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:1031)
> ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus
> - 2024-04-22 23:20:52]
> at
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:621)
> ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus
> - 2024-04-22 23:20:52]
> at
> org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:262)
> ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus
> - 2024-04-22 23:20:52]
> at
> org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilter$0(SolrDispatchFilter.java:219)
> ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus
> - 2024-04-22 23:20:52]
> at
> org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:249)
> ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus
> - 2024-04-22 23:20:52]
> at
> org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:215)
> ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus
> - 2024-04-22 23:20:52]
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:213)
> ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus
> - 2024-04-22 23:20:52]
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
> ~[solr-core-9.6.0.jar:9.6.0
>
> 3. On small-size encrypted (data) this issue is not producing. and
> 4. On large-size unencrypted data issues are not coming
>
> Any help or direction to fix this issue will be invaluable.
>
> Regards,
> Manish Kumar
>
>
>
> On Sun, Jan 5, 2025 at 6:33 PM Bruno Roustant <bruno.roust...@gmail.com>
> wrote:
>
>> Hi Manish Kumar,
>>
>> Can you open a git issue in the Solr sandbox repo?
>> https://github.com/apache/solr-sandbox/issues
>>
>> What is the replica type? The default NRT?
>> If you can contribute, it would be great to propose a PR with a Solr test
>> reproducing the problem.
>>
>> Thanks,
>> Bruno
>>
>> Le sam. 4 janv. 2025 à 07:02, Manish Kumar singh <mksmksi...@gmail.com> a
>> écrit :
>>
>> > Hello,
>> >
>> > I am using the Solr encryption plugin for data and index encryption. It
>> is
>> > working fine for single-tenant systems. On a distributed system with
>> two or
>> > more tenants, the follower replica fails to start replication when a
>> > collection has two or more replicas in a shard,, Replica recovery fails,
>> > and it continuously retries and fails.
>> >
>> > I have tested this behaviour in a multi-sharded Solr collection with two
>> > replicas
>> > per shard.
>> >
>> > On the Solr log getting this error - org.apache.solr.update.processor.D
>> > istributedUpdateProcessor Ignoring commit while not ACTIVE - state:
>> > BUFFERING replay: false
>> >
>> > Any suggestions or directions to resolve the issue are helpful, and I am
>> > willing to contribute to the necessary development to fix this issue if
>> > needed.
>> >
>> >
>> > Regards,
>> > Manish Kumar
>> >
>>
>
