Github user uschindler commented on a diff in the pull request:

    https://github.com/apache/lucene-solr/pull/432#discussion_r209170130
  
    --- Diff: 
lucene/core/src/java/org/apache/lucene/store/ByteBuffersDirectory.java ---
    @@ -0,0 +1,237 @@
    +package org.apache.lucene.store;
    +
    +import java.io.FileNotFoundException;
    +import java.io.IOException;
    +import java.nio.ByteBuffer;
    +import java.nio.file.AccessDeniedException;
    +import java.nio.file.FileAlreadyExistsException;
    +import java.nio.file.NoSuchFileException;
    +import java.util.Arrays;
    +import java.util.Collection;
    +import java.util.List;
    +import java.util.Locale;
    +import java.util.Objects;
    +import java.util.concurrent.ConcurrentHashMap;
    +import java.util.concurrent.atomic.AtomicLong;
    +import java.util.function.BiFunction;
    +import java.util.function.Function;
    +import java.util.zip.CRC32;
    +
    +import org.apache.lucene.index.IndexFileNames;
    +import org.apache.lucene.util.BitUtil;
    +
    +public final class ByteBuffersDirectory extends BaseDirectory {
    +  public static final BiFunction<String, ByteBuffersDataOutput, 
IndexInput> OUTPUT_AS_MANY_BUFFERS = 
    +      (fileName, output) -> {
    +        ByteBuffersDataInput dataInput = output.toDataInput();
    +        String inputName = String.format(Locale.ROOT, "%s (file=%s, 
buffers=%s)",
    +            ByteBuffersIndexInput.class.getSimpleName(),
    +            fileName,
    +            dataInput.toString());
    +        return new ByteBuffersIndexInput(dataInput, inputName);
    +      };
    +
    +  public static final BiFunction<String, ByteBuffersDataOutput, 
IndexInput> OUTPUT_AS_ONE_BUFFER = 
    +      (fileName, output) -> {
    +        ByteBuffersDataInput dataInput = new 
ByteBuffersDataInput(Arrays.asList(ByteBuffer.wrap(output.copyToArray())));
    +        String inputName = String.format(Locale.ROOT, "%s (file=%s, 
buffers=%s)",
    +            ByteBuffersIndexInput.class.getSimpleName(),
    +            fileName,
    +            dataInput.toString());
    +        return new ByteBuffersIndexInput(dataInput, inputName);
    +      };
    +
    +  public static final BiFunction<String, ByteBuffersDataOutput, 
IndexInput> OUTPUT_AS_BYTE_ARRAY = 
    +      (fileName, output) -> {
    +        byte[] array = output.copyToArray();
    +        String inputName = String.format(Locale.ROOT, "%s (file=%s, 
length=%s)",
    +            ByteArrayIndexInput.class.getSimpleName(),
    +            fileName,
    +            array.length);
    +        return new ByteArrayIndexInput(inputName, array, 0, array.length);
    +      };
    +
    +  public static final BiFunction<String, ByteBuffersDataOutput, 
IndexInput> OUTPUT_AS_MANY_BUFFERS_LUCENE = 
    +      (fileName, output) -> {
    +        List<ByteBuffer> bufferList = output.toBufferList();
    +        int chunkSizePower;
    +        bufferList.add(ByteBuffer.allocate(0));
    --- End diff --
    
    I think the 0-byte buffer is only needed for buffers with exactly the 
chunkSize, see the comment:
    
    ```java
        // we always allocate one more buffer, the last one may be a 0 byte one
        final int nrBuffers = (int) (length >>> chunkSizePower) + 1;
    ```
    
    So the last one is a 0-bytes buffer, if the total size is actually exactly 
2^x. Maybe we can fix this, but this made calculations more easy.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to