justinleet commented on a change in pull request #1254: METRON-1849 Elasticsearch Index Write Functionality Should be Shared URL: https://github.com/apache/metron/pull/1254#discussion_r240414816
########## File path: metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/writer/BulkDocumentWriterStub.java ########## @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.metron.elasticsearch.writer; + +import org.apache.metron.elasticsearch.bulk.BulkDocumentWriter; +import org.apache.metron.elasticsearch.bulk.BulkDocumentWriterResults; +import org.apache.metron.indexing.dao.update.Document; + +import java.util.ArrayList; +import java.util.List; +import java.util.Random; + +/** + * A {@link BulkDocumentWriter} stub that can pretend that all documents + * have been written successfully, that all documents have failed, or something + * in between those two extremes. + * + * @param <D> The type of {@link Document} to write. + */ +public class BulkDocumentWriterStub<D extends Document> implements BulkDocumentWriter<D> { + + private float probabilityOfSuccess; + private Exception exception; + private List<D> documents; + + /** + * @param probabilityOfSuccess The probability that a write will succeed. + * If 1.0, all writes succeed. If 0.0, all writes fail. + */ + public BulkDocumentWriterStub(float probabilityOfSuccess) { + this.probabilityOfSuccess = probabilityOfSuccess; + this.exception = new IllegalStateException("Exception created by a stub for testing"); + this.documents = new ArrayList<>(); + } + + @Override + public void addDocument(D document, String index) { + documents.add(document); + } + + @Override + public BulkDocumentWriterResults<D> write() { + BulkDocumentWriterResults<D> results = new BulkDocumentWriterResults<>(); + Random random = new Random(); Review comment: Why are we doing this randomly? We have the number of Documents, and we have the proportion. Why not just calculate D * p and then just produce that many successes and the corresponding number of failures? I don't see what we're gaining by making it probabilistic, and I can see cases where people unknowingly choose small numbers of Documents (e.g. 3 with probability .5) where suddenly it fails 1/4 of the time (all all successes or all failures when it's expecting a mix). ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
