[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r315624050
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
+ synchronized (GoraBenchmarkClient.class) {
Review comment:
So basically, when class level locking is used, for multiple instances of
the same GoraBenchmarkClient class in single JVM, only one thread will get
access to class level monitor ( only a single monitor per class of instances )
and execute synchronized block of a single instance from all the instances
available in JVM. In object level synchronization, threads just have to wait
for each instance's monitor as opposed to class object monitor. Threads can
paralleled acquire instance monitors.
Class level synchronization can lead to higher thread contention and
possibly higher waiting times for threads. This can be a overhead since if we
don't really want to synchronize on all instances of the class and
synchronization is only required for per single instance. If you compare two
case class level and object level, class level has the highest level
synchronization/safety as mentioned but it comes with overhead.
Think about two client instances of GoraBenchmarkClient and how these
instances are getting accessed/methods being call when a we perform a
benchmark. Think about above statements can have a
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r315604148
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
+ synchronized (GoraBenchmarkClient.class) {
+if (executed)
+ return;
+executed = true;
+goraBenchmarkUtils.generateAvroSchema(fieldCount);
+String dataStoreName = goraBenchmarkUtils.getDataStore(p);
+goraBenchmarkUtils.generateMappingFile(dataStoreName);
+goraBenchmarkUtils.generateDataBeans();
+ }
+} catch (GoraException e) {
+ LOG.info("There is a problem in initialising the DataStore
\n"+e.getMessage());
+}
+ }
+
+ /**
+ * Cleanup any state for this DB.
+ *
+ * It is very important to close the datastore properly, otherwise some data
+ * loss might occur.
+ */
+ public void cleanup() throws DBException {
+synchronized (GoraBenchmarkClient.class) {
Review comment:
+1
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r315603638
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,428 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r315592665
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
Review comment:
My point is there s no need to create a datastore instance as above for a
already initialized GoraBenchmarkClient instance. Let's say one thread
executes init() method and set executed boolean variable value to 'true',
generate mapping file etc and returns. Lets say a second and third thread come
and execute executes init(), it checks executed variable since it s true it
will return without doing doing generation of mapping file etc.
For second and third threads, there s overhead to creating that datastore
instance and override the class level field. 1st, 2nd created datastore object
will be garbage collected ( no reachable reference ), when you override that
class level field from second and third threads. Which is I think unnecessary.
If you move that DataStoreFactory.getDataStore into synchronize block, there
will be no additional datastore object creation and will create only one
instance. This is true for all the class level field value you set before the
synchronize block. Those are overhead. Datastore creation is pretty expensive
operation due it s network call etc nature.
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r315592665
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
Review comment:
My point is there s no need to create a datastore instance as above for a
already initialized GoraBenchmarkClient instance. Let's say one thread
executes init() method and set executed boolean variable value to 'true',
generate mapping file etc and returns. Lets say a second and third thread come
and execute executes init(), it checks executed variable since it s true it
will return without doing doing generation of mapping file etc.
For second and third threads, there s overhead to creating that datastore
instance and override the class level field. 1st, 2nd created will be garbage
collected, when you override that class level field from second and third
threads. Which is I think unnecessary.
If you move that DataStoreFactory.getDataStore into synchronize block, there
will be no additional datastore object creation and will create only one
instance. This is true for all the class level field value you set before the
synchronize block. Those are overhead. Datastore creation is pretty expensive
operation due it s network call etc nature.
This is an automated message from the
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r315592665
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
Review comment:
My point is there s no need to create a datastore instance as above for a
already initialized GoraBenchmarkClient instance. Let's say one thread
executes init() method and set executed boolean variable value to 'true',
generate mapping file etc and returns. Lets say a second and third thread come
and execute executes init(), it checks executed variable since it s true it
will return without doing doing generation of mapping file etc. For second and
third threads, there s overhead to creating that datastore instance and
override the class level field. 1st, 2nd created will be garbage collected,
when you override that class level field from second and third threads. Which
is I think unnecessary.
If you move that DataStoreFactory.getDataStore into synchronize block, there
will be no additional datastore object creation and will create only one
instance. This is true for all the class level field value you set before the
synchronize block. Those are overhead. Datastore creation is pretty expensive
operation due it s network call etc nature.
This is an automated message from the Apache Git
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r315578059
##
File path:
gora-benchmark/src/test/java/org/apache/gora/benchmark/GoraClientTest.java
##
@@ -0,0 +1,195 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotNull;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.benchmark.generated.User;
+import org.apache.gora.util.GoraException;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+
+/**
+ * The Class GoraClientTest.
+ */
+public class GoraClientTest {
+
+ private static final String TABLE = "users";
+ private GoraBenchmarkClient client;
+ private static HashMap DATA_TO_INSERT;
+ private static HashMap DATA_TO_UPDATE;
+ private static final int NUMBER_OF_FIELDS = 10;
+ private GoraBenchmarkUtils bmutils = new GoraBenchmarkUtils();
+
+ /**
+ * Sets the up.
+ *
+ * Setup is executed before each test. Use @BeforeClass if you want to
execute
+ * a code block just once.
+ *
+ * @throws Exception
+ * the exceptionfiles are auto-generated. I have the code to add
the license file accordingly
+ */
+ @Before
+ public void setUp() throws Exception {
+DATA_TO_INSERT = new HashMap<>();
+DATA_TO_UPDATE = new HashMap<>();
+for (int i = 0; i < NUMBER_OF_FIELDS; i++) {
+ DATA_TO_INSERT.put("field" + i, new StringByteIterator("value" + i));
+ DATA_TO_UPDATE.put("field" + i, new StringByteIterator("updated" + i));
+}
+Properties p = new Properties();
+p.setProperty("key.class", "java.lang.String");
+p.setProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+p.setProperty(CoreWorkload.FIELD_COUNT_PROPERTY, NUMBER_OF_FIELDS + "");
+client = new GoraBenchmarkClient();
+client.setProperties(p);
+client.init();
+ }
+
+ /**
+ * Clean up.
+ *
+ * @throws Exception
+ * the exception
+ */
+ @After
+ public void cleanUp() throws Exception {
+if (client != null)
+ client.cleanup();
+client = null;
+ }
+
+ /**
+ * Read record.
+ *
+ * @param key
+ * the key
+ * @return the user
+ * @throws GoraException
+ * the gora exception
+ */
+ private User readRecord(String key) throws GoraException {
+User u = client.dataStore.get(key);
+return u;
+ }
+
+ /**
+ * Test client initialisation.
+ */
+ @Test
+ public void testClientInitialisation() {
+assertNotNull(client.dataStore);
+ }
+
+ /**files are auto-generated. I have the code to add the license file
accordingly
+ * Test insert.
+ *
+ * @throws GoraException
+ * the gora exception
+ */
+ @Test
+ public void testInsert() throws GoraException {
+Status result1 = client.insert(TABLE, "key1", DATA_TO_INSERT);
+Status result2 = client.insert(TABLE, "key2", DATA_TO_INSERT);
+Status result3 = client.insert(TABLE, "key3", DATA_TO_INSERT);
+assertEquals(Status.OK, result1);
+assertEquals(Status.OK, result2);
+assertEquals(Status.OK, result3);
+ }
+
+ /**
+ * Test read.
+ */
+ @Test
+ public void testRead() {
+HashMap results = new HashMap<>();
+Set fields = new HashSet<>();// this could be null as well
+// fields.add("field0");
+Status result = client.read(TABLE, "key1", fields, results);
+assertEquals(Status.OK, result);
+assertEquals(DATA_TO_INSERT.size(), results.size());
+assertEquals(DATA_TO_INSERT.get("field0").toString(),
results.get("field0").toString());
+assertEquals(DATA_TO_INSERT.get("field0").toString(), "value0");
+ }
+
+ /**
+ * Test scan.
+ */
+ @Test
+ public void testScan() {
+Vector>
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r315578059
##
File path:
gora-benchmark/src/test/java/org/apache/gora/benchmark/GoraClientTest.java
##
@@ -0,0 +1,195 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotNull;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.benchmark.generated.User;
+import org.apache.gora.util.GoraException;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+
+/**
+ * The Class GoraClientTest.
+ */
+public class GoraClientTest {
+
+ private static final String TABLE = "users";
+ private GoraBenchmarkClient client;
+ private static HashMap DATA_TO_INSERT;
+ private static HashMap DATA_TO_UPDATE;
+ private static final int NUMBER_OF_FIELDS = 10;
+ private GoraBenchmarkUtils bmutils = new GoraBenchmarkUtils();
+
+ /**
+ * Sets the up.
+ *
+ * Setup is executed before each test. Use @BeforeClass if you want to
execute
+ * a code block just once.
+ *
+ * @throws Exception
+ * the exceptionfiles are auto-generated. I have the code to add
the license file accordingly
+ */
+ @Before
+ public void setUp() throws Exception {
+DATA_TO_INSERT = new HashMap<>();
+DATA_TO_UPDATE = new HashMap<>();
+for (int i = 0; i < NUMBER_OF_FIELDS; i++) {
+ DATA_TO_INSERT.put("field" + i, new StringByteIterator("value" + i));
+ DATA_TO_UPDATE.put("field" + i, new StringByteIterator("updated" + i));
+}
+Properties p = new Properties();
+p.setProperty("key.class", "java.lang.String");
+p.setProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+p.setProperty(CoreWorkload.FIELD_COUNT_PROPERTY, NUMBER_OF_FIELDS + "");
+client = new GoraBenchmarkClient();
+client.setProperties(p);
+client.init();
+ }
+
+ /**
+ * Clean up.
+ *
+ * @throws Exception
+ * the exception
+ */
+ @After
+ public void cleanUp() throws Exception {
+if (client != null)
+ client.cleanup();
+client = null;
+ }
+
+ /**
+ * Read record.
+ *
+ * @param key
+ * the key
+ * @return the user
+ * @throws GoraException
+ * the gora exception
+ */
+ private User readRecord(String key) throws GoraException {
+User u = client.dataStore.get(key);
+return u;
+ }
+
+ /**
+ * Test client initialisation.
+ */
+ @Test
+ public void testClientInitialisation() {
+assertNotNull(client.dataStore);
+ }
+
+ /**files are auto-generated. I have the code to add the license file
accordingly
+ * Test insert.
+ *
+ * @throws GoraException
+ * the gora exception
+ */
+ @Test
+ public void testInsert() throws GoraException {
+Status result1 = client.insert(TABLE, "key1", DATA_TO_INSERT);
+Status result2 = client.insert(TABLE, "key2", DATA_TO_INSERT);
+Status result3 = client.insert(TABLE, "key3", DATA_TO_INSERT);
+assertEquals(Status.OK, result1);
+assertEquals(Status.OK, result2);
+assertEquals(Status.OK, result3);
+ }
+
+ /**
+ * Test read.
+ */
+ @Test
+ public void testRead() {
+HashMap results = new HashMap<>();
+Set fields = new HashSet<>();// this could be null as well
+// fields.add("field0");
+Status result = client.read(TABLE, "key1", fields, results);
+assertEquals(Status.OK, result);
+assertEquals(DATA_TO_INSERT.size(), results.size());
+assertEquals(DATA_TO_INSERT.get("field0").toString(),
results.get("field0").toString());
+assertEquals(DATA_TO_INSERT.get("field0").toString(), "value0");
+ }
+
+ /**
+ * Test scan.
+ */
+ @Test
+ public void testScan() {
+Vector>
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312805714
##
File path:
gora-benchmark/src/test/java/org/apache/gora/benchmark/GoraClientTest.java
##
@@ -0,0 +1,195 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotNull;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.benchmark.generated.User;
+import org.apache.gora.util.GoraException;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+
+/**
+ * The Class GoraClientTest.
+ */
+public class GoraClientTest {
+
+ private static final String TABLE = "users";
+ private GoraBenchmarkClient client;
+ private static HashMap DATA_TO_INSERT;
+ private static HashMap DATA_TO_UPDATE;
+ private static final int NUMBER_OF_FIELDS = 10;
+ private GoraBenchmarkUtils bmutils = new GoraBenchmarkUtils();
+
+ /**
+ * Sets the up.
+ *
+ * Setup is executed before each test. Use @BeforeClass if you want to
execute
+ * a code block just once.
+ *
+ * @throws Exception
+ * the exceptionfiles are auto-generated. I have the code to add
the license file accordingly
+ */
+ @Before
+ public void setUp() throws Exception {
+DATA_TO_INSERT = new HashMap<>();
+DATA_TO_UPDATE = new HashMap<>();
+for (int i = 0; i < NUMBER_OF_FIELDS; i++) {
+ DATA_TO_INSERT.put("field" + i, new StringByteIterator("value" + i));
+ DATA_TO_UPDATE.put("field" + i, new StringByteIterator("updated" + i));
+}
+Properties p = new Properties();
+p.setProperty("key.class", "java.lang.String");
+p.setProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+p.setProperty(CoreWorkload.FIELD_COUNT_PROPERTY, NUMBER_OF_FIELDS + "");
+client = new GoraBenchmarkClient();
+client.setProperties(p);
+client.init();
+ }
+
+ /**
+ * Clean up.
+ *
+ * @throws Exception
+ * the exception
+ */
+ @After
+ public void cleanUp() throws Exception {
+if (client != null)
+ client.cleanup();
+client = null;
+ }
+
+ /**
+ * Read record.
+ *
+ * @param key
+ * the key
+ * @return the user
+ * @throws GoraException
+ * the gora exception
+ */
+ private User readRecord(String key) throws GoraException {
+User u = client.dataStore.get(key);
+return u;
+ }
+
+ /**
+ * Test client initialisation.
+ */
+ @Test
+ public void testClientInitialisation() {
+assertNotNull(client.dataStore);
+ }
+
+ /**files are auto-generated. I have the code to add the license file
accordingly
+ * Test insert.
+ *
+ * @throws GoraException
+ * the gora exception
+ */
+ @Test
+ public void testInsert() throws GoraException {
+Status result1 = client.insert(TABLE, "key1", DATA_TO_INSERT);
+Status result2 = client.insert(TABLE, "key2", DATA_TO_INSERT);
+Status result3 = client.insert(TABLE, "key3", DATA_TO_INSERT);
+assertEquals(Status.OK, result1);
+assertEquals(Status.OK, result2);
+assertEquals(Status.OK, result3);
+ }
+
+ /**
+ * Test read.
+ */
+ @Test
+ public void testRead() {
+HashMap results = new HashMap<>();
+Set fields = new HashSet<>();// this could be null as well
+// fields.add("field0");
+Status result = client.read(TABLE, "key1", fields, results);
+assertEquals(Status.OK, result);
+assertEquals(DATA_TO_INSERT.size(), results.size());
+assertEquals(DATA_TO_INSERT.get("field0").toString(),
results.get("field0").toString());
+assertEquals(DATA_TO_INSERT.get("field0").toString(), "value0");
+ }
+
+ /**
+ * Test scan.
+ */
+ @Test
+ public void testScan() {
+Vector>
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312805362
##
File path:
gora-benchmark/src/test/java/org/apache/gora/benchmark/GoraClientTest.java
##
@@ -0,0 +1,195 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotNull;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.benchmark.generated.User;
+import org.apache.gora.util.GoraException;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+
+/**
+ * The Class GoraClientTest.
+ */
+public class GoraClientTest {
+
+ private static final String TABLE = "users";
+ private GoraBenchmarkClient client;
+ private static HashMap DATA_TO_INSERT;
+ private static HashMap DATA_TO_UPDATE;
+ private static final int NUMBER_OF_FIELDS = 10;
+ private GoraBenchmarkUtils bmutils = new GoraBenchmarkUtils();
+
+ /**
+ * Sets the up.
+ *
+ * Setup is executed before each test. Use @BeforeClass if you want to
execute
+ * a code block just once.
+ *
+ * @throws Exception
+ * the exceptionfiles are auto-generated. I have the code to add
the license file accordingly
+ */
+ @Before
+ public void setUp() throws Exception {
+DATA_TO_INSERT = new HashMap<>();
+DATA_TO_UPDATE = new HashMap<>();
+for (int i = 0; i < NUMBER_OF_FIELDS; i++) {
+ DATA_TO_INSERT.put("field" + i, new StringByteIterator("value" + i));
+ DATA_TO_UPDATE.put("field" + i, new StringByteIterator("updated" + i));
+}
+Properties p = new Properties();
+p.setProperty("key.class", "java.lang.String");
+p.setProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+p.setProperty(CoreWorkload.FIELD_COUNT_PROPERTY, NUMBER_OF_FIELDS + "");
+client = new GoraBenchmarkClient();
+client.setProperties(p);
+client.init();
+ }
+
+ /**
+ * Clean up.
+ *
+ * @throws Exception
+ * the exception
+ */
+ @After
+ public void cleanUp() throws Exception {
+if (client != null)
+ client.cleanup();
+client = null;
+ }
+
+ /**
+ * Read record.
+ *
+ * @param key
+ * the key
+ * @return the user
+ * @throws GoraException
+ * the gora exception
+ */
+ private User readRecord(String key) throws GoraException {
+User u = client.dataStore.get(key);
+return u;
+ }
+
+ /**
+ * Test client initialisation.
+ */
+ @Test
+ public void testClientInitialisation() {
+assertNotNull(client.dataStore);
+ }
+
+ /**files are auto-generated. I have the code to add the license file
accordingly
+ * Test insert.
+ *
+ * @throws GoraException
+ * the gora exception
+ */
+ @Test
+ public void testInsert() throws GoraException {
+Status result1 = client.insert(TABLE, "key1", DATA_TO_INSERT);
+Status result2 = client.insert(TABLE, "key2", DATA_TO_INSERT);
+Status result3 = client.insert(TABLE, "key3", DATA_TO_INSERT);
+assertEquals(Status.OK, result1);
+assertEquals(Status.OK, result2);
+assertEquals(Status.OK, result3);
+ }
+
+ /**
+ * Test read.
+ */
+ @Test
+ public void testRead() {
+HashMap results = new HashMap<>();
+Set fields = new HashSet<>();// this could be null as well
+// fields.add("field0");
+Status result = client.read(TABLE, "key1", fields, results);
+assertEquals(Status.OK, result);
+assertEquals(DATA_TO_INSERT.size(), results.size());
+assertEquals(DATA_TO_INSERT.get("field0").toString(),
results.get("field0").toString());
+assertEquals(DATA_TO_INSERT.get("field0").toString(), "value0");
+ }
+
+ /**
+ * Test scan.
Review comment:
Please document these test cases.
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312805591
##
File path:
gora-benchmark/src/test/java/org/apache/gora/benchmark/GoraClientTest.java
##
@@ -0,0 +1,195 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotNull;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.benchmark.generated.User;
+import org.apache.gora.util.GoraException;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+
+/**
+ * The Class GoraClientTest.
+ */
+public class GoraClientTest {
+
+ private static final String TABLE = "users";
+ private GoraBenchmarkClient client;
+ private static HashMap DATA_TO_INSERT;
+ private static HashMap DATA_TO_UPDATE;
+ private static final int NUMBER_OF_FIELDS = 10;
+ private GoraBenchmarkUtils bmutils = new GoraBenchmarkUtils();
+
+ /**
+ * Sets the up.
+ *
+ * Setup is executed before each test. Use @BeforeClass if you want to
execute
+ * a code block just once.
+ *
+ * @throws Exception
+ * the exceptionfiles are auto-generated. I have the code to add
the license file accordingly
+ */
+ @Before
+ public void setUp() throws Exception {
+DATA_TO_INSERT = new HashMap<>();
+DATA_TO_UPDATE = new HashMap<>();
+for (int i = 0; i < NUMBER_OF_FIELDS; i++) {
+ DATA_TO_INSERT.put("field" + i, new StringByteIterator("value" + i));
+ DATA_TO_UPDATE.put("field" + i, new StringByteIterator("updated" + i));
+}
+Properties p = new Properties();
+p.setProperty("key.class", "java.lang.String");
+p.setProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+p.setProperty(CoreWorkload.FIELD_COUNT_PROPERTY, NUMBER_OF_FIELDS + "");
+client = new GoraBenchmarkClient();
+client.setProperties(p);
+client.init();
+ }
+
+ /**
+ * Clean up.
+ *
+ * @throws Exception
+ * the exception
+ */
+ @After
+ public void cleanUp() throws Exception {
+if (client != null)
+ client.cleanup();
+client = null;
+ }
+
+ /**
+ * Read record.
+ *
+ * @param key
+ * the key
+ * @return the user
+ * @throws GoraException
+ * the gora exception
+ */
+ private User readRecord(String key) throws GoraException {
+User u = client.dataStore.get(key);
+return u;
+ }
+
+ /**
+ * Test client initialisation.
+ */
+ @Test
+ public void testClientInitialisation() {
+assertNotNull(client.dataStore);
+ }
+
+ /**files are auto-generated. I have the code to add the license file
accordingly
+ * Test insert.
+ *
+ * @throws GoraException
+ * the gora exception
+ */
+ @Test
+ public void testInsert() throws GoraException {
+Status result1 = client.insert(TABLE, "key1", DATA_TO_INSERT);
+Status result2 = client.insert(TABLE, "key2", DATA_TO_INSERT);
+Status result3 = client.insert(TABLE, "key3", DATA_TO_INSERT);
+assertEquals(Status.OK, result1);
+assertEquals(Status.OK, result2);
+assertEquals(Status.OK, result3);
+ }
+
+ /**
+ * Test read.
+ */
+ @Test
+ public void testRead() {
+HashMap results = new HashMap<>();
+Set fields = new HashSet<>();// this could be null as well
+// fields.add("field0");
+Status result = client.read(TABLE, "key1", fields, results);
+assertEquals(Status.OK, result);
+assertEquals(DATA_TO_INSERT.size(), results.size());
+assertEquals(DATA_TO_INSERT.get("field0").toString(),
results.get("field0").toString());
+assertEquals(DATA_TO_INSERT.get("field0").toString(), "value0");
+ }
+
+ /**
+ * Test scan.
+ */
+ @Test
+ public void testScan() {
+Vector>
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312803812
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,428 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312793396
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,428 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312791005
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,428 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312792136
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,428 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments URL: https://github.com/apache/gora/pull/179#discussion_r312790317 ## File path: gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java ## @@ -0,0 +1,428 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.gora.benchmark; + +import java.io.File; Review comment: Please add package-info.java for all source, test packages. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312792805
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,428 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312792703
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,428 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312789725
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
+ synchronized (GoraBenchmarkClient.class) {
+if (executed)
+ return;
+executed = true;
+goraBenchmarkUtils.generateAvroSchema(fieldCount);
+String dataStoreName = goraBenchmarkUtils.getDataStore(p);
+goraBenchmarkUtils.generateMappingFile(dataStoreName);
+goraBenchmarkUtils.generateDataBeans();
+ }
+} catch (GoraException e) {
+ LOG.info("There is a problem in initialising the DataStore
\n"+e.getMessage());
+}
+ }
+
+ /**
+ * Cleanup any state for this DB.
+ *
+ * It is very important to close the datastore properly, otherwise some data
+ * loss might occur.
+ */
+ public void cleanup() throws DBException {
+synchronized (GoraBenchmarkClient.class) {
+ if (dataStore != null)
+dataStore.close();
+}
+ }
+
+ /**
+ * Delete a record from the database.
+ *
+ * @param table
+ * The name of the table to delete the data from
+ * @param key
+ * The key of the record to delete.
+ * @return Status of the operation failed or success.
+ */
+ @Override
+ public Status delete(String table, String key) {
+try {
+ dataStore.delete(key);
+}
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments URL: https://github.com/apache/gora/pull/179#discussion_r312791537 ## File path: gora-benchmark/src/main/resources/gora-hbase-mapping.xml ## @@ -0,0 +1,59 @@ + + Review comment: Please add License header. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312792475
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,428 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments URL: https://github.com/apache/gora/pull/179#discussion_r312791482 ## File path: gora-benchmark/src/main/resources/gora-couchdb-mapping.xml ## @@ -0,0 +1,26 @@ + Review comment: Please add License header. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312790914
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,428 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312793249
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,428 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312792935
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,428 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312793335
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,428 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312788650
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
+ synchronized (GoraBenchmarkClient.class) {
+if (executed)
+ return;
+executed = true;
+goraBenchmarkUtils.generateAvroSchema(fieldCount);
+String dataStoreName = goraBenchmarkUtils.getDataStore(p);
+goraBenchmarkUtils.generateMappingFile(dataStoreName);
+goraBenchmarkUtils.generateDataBeans();
+ }
+} catch (GoraException e) {
+ LOG.info("There is a problem in initialising the DataStore
\n"+e.getMessage());
+}
+ }
+
+ /**
+ * Cleanup any state for this DB.
+ *
+ * It is very important to close the datastore properly, otherwise some data
+ * loss might occur.
+ */
+ public void cleanup() throws DBException {
+synchronized (GoraBenchmarkClient.class) {
+ if (dataStore != null)
+dataStore.close();
+}
+ }
+
+ /**
+ * Delete a record from the database.
+ *
+ * @param table
+ * The name of the table to delete the data from
+ * @param key
+ * The key of the record to delete.
+ * @return Status of the operation failed or success.
+ */
+ @Override
+ public Status delete(String table, String key) {
+try {
+ dataStore.delete(key);
+}
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312786434
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
Review comment:
Here dataStore is created outside the below synchronization block. Is there
a possibility of creating multiple untracked dataStore instances at
concurrency?
dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p, new
Configuration());
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312788426
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
+ synchronized (GoraBenchmarkClient.class) {
+if (executed)
+ return;
+executed = true;
+goraBenchmarkUtils.generateAvroSchema(fieldCount);
+String dataStoreName = goraBenchmarkUtils.getDataStore(p);
+goraBenchmarkUtils.generateMappingFile(dataStoreName);
+goraBenchmarkUtils.generateDataBeans();
+ }
+} catch (GoraException e) {
+ LOG.info("There is a problem in initialising the DataStore
\n"+e.getMessage());
+}
+ }
+
+ /**
+ * Cleanup any state for this DB.
+ *
+ * It is very important to close the datastore properly, otherwise some data
+ * loss might occur.
+ */
+ public void cleanup() throws DBException {
+synchronized (GoraBenchmarkClient.class) {
Review comment:
Can you explain the why we have chosen to use synchronization here? Should
we only allow single thread to execute the close method on dataStore? Should we
use isClosed similar state variable as we did for init(). If there s a
possibility of getting this method cleanup() by multiple threads, anyway these
threads will execute this method sequentially. Since in dataStores, we have
handled the close method, even at multiple
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312789069
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
+ synchronized (GoraBenchmarkClient.class) {
+if (executed)
+ return;
+executed = true;
+goraBenchmarkUtils.generateAvroSchema(fieldCount);
+String dataStoreName = goraBenchmarkUtils.getDataStore(p);
+goraBenchmarkUtils.generateMappingFile(dataStoreName);
+goraBenchmarkUtils.generateDataBeans();
+ }
+} catch (GoraException e) {
+ LOG.info("There is a problem in initialising the DataStore
\n"+e.getMessage());
+}
+ }
+
+ /**
+ * Cleanup any state for this DB.
+ *
+ * It is very important to close the datastore properly, otherwise some data
+ * loss might occur.
+ */
+ public void cleanup() throws DBException {
+synchronized (GoraBenchmarkClient.class) {
+ if (dataStore != null)
+dataStore.close();
+}
+ }
+
+ /**
+ * Delete a record from the database.
+ *
+ * @param table
+ * The name of the table to delete the data from
+ * @param key
+ * The key of the record to delete.
+ * @return Status of the operation failed or success.
+ */
+ @Override
+ public Status delete(String table, String key) {
+try {
+ dataStore.delete(key);
+}
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312788710
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
+ synchronized (GoraBenchmarkClient.class) {
+if (executed)
+ return;
+executed = true;
+goraBenchmarkUtils.generateAvroSchema(fieldCount);
+String dataStoreName = goraBenchmarkUtils.getDataStore(p);
+goraBenchmarkUtils.generateMappingFile(dataStoreName);
+goraBenchmarkUtils.generateDataBeans();
+ }
+} catch (GoraException e) {
+ LOG.info("There is a problem in initialising the DataStore
\n"+e.getMessage());
+}
+ }
+
+ /**
+ * Cleanup any state for this DB.
+ *
+ * It is very important to close the datastore properly, otherwise some data
+ * loss might occur.
+ */
+ public void cleanup() throws DBException {
+synchronized (GoraBenchmarkClient.class) {
+ if (dataStore != null)
+dataStore.close();
+}
+ }
+
+ /**
+ * Delete a record from the database.
+ *
+ * @param table
+ * The name of the table to delete the data from
+ * @param key
+ * The key of the record to delete.
+ * @return Status of the operation failed or success.
+ */
+ @Override
+ public Status delete(String table, String key) {
+try {
+ dataStore.delete(key);
+}
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312789426
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
+ synchronized (GoraBenchmarkClient.class) {
+if (executed)
+ return;
+executed = true;
+goraBenchmarkUtils.generateAvroSchema(fieldCount);
+String dataStoreName = goraBenchmarkUtils.getDataStore(p);
+goraBenchmarkUtils.generateMappingFile(dataStoreName);
+goraBenchmarkUtils.generateDataBeans();
+ }
+} catch (GoraException e) {
+ LOG.info("There is a problem in initialising the DataStore
\n"+e.getMessage());
+}
+ }
+
+ /**
+ * Cleanup any state for this DB.
+ *
+ * It is very important to close the datastore properly, otherwise some data
+ * loss might occur.
+ */
+ public void cleanup() throws DBException {
+synchronized (GoraBenchmarkClient.class) {
+ if (dataStore != null)
+dataStore.close();
+}
+ }
+
+ /**
+ * Delete a record from the database.
+ *
+ * @param table
+ * The name of the table to delete the data from
+ * @param key
+ * The key of the record to delete.
+ * @return Status of the operation failed or success.
+ */
+ @Override
+ public Status delete(String table, String key) {
+try {
+ dataStore.delete(key);
+}
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312789206
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
+ synchronized (GoraBenchmarkClient.class) {
+if (executed)
+ return;
+executed = true;
+goraBenchmarkUtils.generateAvroSchema(fieldCount);
+String dataStoreName = goraBenchmarkUtils.getDataStore(p);
+goraBenchmarkUtils.generateMappingFile(dataStoreName);
+goraBenchmarkUtils.generateDataBeans();
+ }
+} catch (GoraException e) {
+ LOG.info("There is a problem in initialising the DataStore
\n"+e.getMessage());
+}
+ }
+
+ /**
+ * Cleanup any state for this DB.
+ *
+ * It is very important to close the datastore properly, otherwise some data
+ * loss might occur.
+ */
+ public void cleanup() throws DBException {
+synchronized (GoraBenchmarkClient.class) {
+ if (dataStore != null)
+dataStore.close();
+}
+ }
+
+ /**
+ * Delete a record from the database.
+ *
+ * @param table
+ * The name of the table to delete the data from
+ * @param key
+ * The key of the record to delete.
+ * @return Status of the operation failed or success.
+ */
+ @Override
+ public Status delete(String table, String key) {
+try {
+ dataStore.delete(key);
+}
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312783720
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
+ synchronized (GoraBenchmarkClient.class) {
Review comment:
I noticed class level synchronization is used here as opposed object level.
class level - synchronized (GoraBenchmarkClient.class) { }
object level - synchronized (this) { }
Can you please explain, why you have chosen class level locking as opposed
to object level?
I would like to how this behaves when GoraBenchmarkClient instances exists
in single JVM.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312784563
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
Review comment:
p - Please use proper variable names.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312784193
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
+ synchronized (GoraBenchmarkClient.class) {
+if (executed)
+ return;
+executed = true;
+goraBenchmarkUtils.generateAvroSchema(fieldCount);
+String dataStoreName = goraBenchmarkUtils.getDataStore(p);
+goraBenchmarkUtils.generateMappingFile(dataStoreName);
+goraBenchmarkUtils.generateDataBeans();
+ }
+} catch (GoraException e) {
+ LOG.info("There is a problem in initialising the DataStore
\n"+e.getMessage());
Review comment:
This code is not formatted correctly. Use always parameterized logging if
you want to parameterize the message content. Also we should not omit the
exception trace added to to logging.
Eg:-
LOG.info("There is a problem in initialising the DataStore \n {}",
e.getMessage(), e);
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312784413
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
Review comment:
Please document the exception.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312784282
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
Review comment:
Please specify access modifier for this field EG:- private
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312784480
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
+ synchronized (GoraBenchmarkClient.class) {
+if (executed)
+ return;
+executed = true;
+goraBenchmarkUtils.generateAvroSchema(fieldCount);
+String dataStoreName = goraBenchmarkUtils.getDataStore(p);
+goraBenchmarkUtils.generateMappingFile(dataStoreName);
+goraBenchmarkUtils.generateDataBeans();
+ }
+} catch (GoraException e) {
+ LOG.info("There is a problem in initialising the DataStore
\n"+e.getMessage());
+}
+ }
+
+ /**
+ * Cleanup any state for this DB.
+ *
+ * It is very important to close the datastore properly, otherwise some data
+ * loss might occur.
Review comment:
Please document the exception.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312782893
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
Review comment:
Usually we don't instantiate utility classes. Utility classes should only
have set of static class level methods, so that you can call without
instantiating.
Eg:- GoraBenchmarkUtils.generateMappingFile()
One attribute of a utility classes is that, it usually never maintain state
inside the class.
I think we should consider renaming this class appropriately. Otherwise we
should consider making all the methods in GoraBenchmarkUtils as static, you
should only invoke methods as above.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312782893
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
Review comment:
Usually we don't instantiate utility classes. Utility classes should only
have set of static class level methods, so that you can call without
instantiating.
Eg:- GoraBenchmarkUtils.generateMappingFile()
One attribute of a utility classes is that, it usually never maintain state
inside the class.
I think we should consider renaming this class appropriately.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312783720
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
+ synchronized (GoraBenchmarkClient.class) {
Review comment:
I noticed class level synchronization is used here as opposed object level.
class level - synchronized (GoraBenchmarkClient.class) { }
object level - synchronized (this) { }
Can you please explain, why you have chosen class level locking as opposed
to object level?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312782893
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
Review comment:
Usually we don't instantiate utility classes. Utility classes should only
have set of static class level methods, you can call without instantiating.
Eg:- GoraBenchmarkUtils.generateMappingFile()
One attribute of a utility classes is that, it usually never maintain state
inside the class.
I think we should consider renaming this class appropriately.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312782461
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
Review comment:
Please specify access modifier for this field EG:- private
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312782310
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
Review comment:
This field can be private access.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312782438
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,271 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.Status;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
Review comment:
Please specify access modifier for this field EG:- private
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312781381
##
File path: gora-benchmark/pom.xml
##
@@ -0,0 +1,186 @@
+
+
+http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;
+ xmlns="http://maven.apache.org/POM/4.0.0;
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;>
+ 4.0.0
+
+org.apache.gora
+gora
+0.9-SNAPSHOT
+
+ gora-benchmark
+ Apache Gora Benchamark Module
+ http://maven.apache.org
+ The Apache Gora open source framework provides an in-memory
data model and
+persistence for big data. Gora supports persisting to column stores, key
value stores,
+document stores and RDBMSs, and analyzing the data with extensive Apache
Hadoop MapReduce
+support.
+ 2010
+
+The Apache Software Foundation
+http://www.apache.org/
+
+
+JIRA
+https://issues.apache.org/jira/browse/GORA
+
+
+Jenkins
+https://builds.apache.org/job/Gora-trunk/
+
+
+UTF-8
+
+
+target
+target/classes
+${project.artifactId}-${project.version}
+target/test-classes
+src/test/java
+src/main/java
+
+
+${project.basedir}/src/test/conf
+
+ **/*
+
+
+
+
+
+org.codehaus.mojo
+build-helper-maven-plugin
+${build-helper-maven-plugin.version}
+
+
+generate-sources
+
+ add-source
+
+
+
+src/examples/java
+
+
+
+
+
+
+
+
+
+
+ com.yahoo.ycsb
+ core
+ 0.15.0
+
+
+
+ org.apache.gora
+ gora-core
+ compile
+
+
+
+
+ org.apache.gora
+ gora-mongodb
+
+
+
+ org.apache.gora
+ gora-hbase
+
+
+
+ org.apache.gora
+ gora-jcache
+
+
+
+ org.apache.gora
+ gora-couchdb
+
+
+
+ org.apache.gora
+ gora-cassandra
+
+
+
+ org.apache.gora
+ gora-solr
+
+
+
+ org.apache.gora
+ gora-aerospike
+
+
+
+ org.apache.avro
+ avro
+ compile
+
+
+
+ gora-compiler
+ org.apache.gora
+
+
+
+
+ org.slf4j
+ slf4j-log4j12
+
+
+ log4j
+ log4j
+
+
+ javax.jms
+ jms
+
+
+
+
+
+
+ org.apache.hadoop
+ hadoop-client
+ compile
+ true
+
+
+
+ org.fluttercode.datafactory
+ datafactory
+ 0.8
Review comment:
Please define the dependencies and versions in parent pom.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312781462
##
File path: gora-benchmark/pom.xml
##
@@ -0,0 +1,186 @@
+
+
+http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;
+ xmlns="http://maven.apache.org/POM/4.0.0;
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;>
+ 4.0.0
+
+org.apache.gora
+gora
+0.9-SNAPSHOT
+
+ gora-benchmark
+ Apache Gora Benchamark Module
+ http://maven.apache.org
+ The Apache Gora open source framework provides an in-memory
data model and
+persistence for big data. Gora supports persisting to column stores, key
value stores,
+document stores and RDBMSs, and analyzing the data with extensive Apache
Hadoop MapReduce
+support.
+ 2010
+
+The Apache Software Foundation
+http://www.apache.org/
+
+
+JIRA
+https://issues.apache.org/jira/browse/GORA
+
+
+Jenkins
+https://builds.apache.org/job/Gora-trunk/
+
+
+UTF-8
+
+
+target
+target/classes
+${project.artifactId}-${project.version}
+target/test-classes
+src/test/java
+src/main/java
+
+
+${project.basedir}/src/test/conf
+
+ **/*
+
+
+
+
+
+org.codehaus.mojo
+build-helper-maven-plugin
+${build-helper-maven-plugin.version}
+
+
+generate-sources
+
+ add-source
+
+
+
+src/examples/java
+
+
+
+
+
+
+
+
+
+
+ com.yahoo.ycsb
+ core
+ 0.15.0
+
+
+
+ org.apache.gora
+ gora-core
+ compile
+
+
+
+
+ org.apache.gora
+ gora-mongodb
+
+
+
+ org.apache.gora
+ gora-hbase
+
+
+
+ org.apache.gora
+ gora-jcache
+
+
+
+ org.apache.gora
+ gora-couchdb
+
+
+
+ org.apache.gora
+ gora-cassandra
+
+
+
+ org.apache.gora
+ gora-solr
+
+
+
+ org.apache.gora
+ gora-aerospike
+
+
+
+ org.apache.avro
+ avro
+ compile
+
+
+
+ gora-compiler
+ org.apache.gora
+
+
+
+
+ org.slf4j
+ slf4j-log4j12
+
+
+ log4j
+ log4j
+
+
+ javax.jms
+ jms
+
+
+
+
+
+
+ org.apache.hadoop
+ hadoop-client
+ compile
+ true
+
+
+
+ org.fluttercode.datafactory
+ datafactory
+ 0.8
+
+
+
+ junit
+ junit
+ test
+
+
+
+
+ org.ektorp
+ org.ektorp
+ 1.4.2
Review comment:
Please define the dependencies and versions in parent pom. In this case use
CoughDB client version already defined in the parent pom.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments URL: https://github.com/apache/gora/pull/179#discussion_r312781204 ## File path: gora-benchmark/gora-workloada.csv ## @@ -0,0 +1,251 @@ +records,timems,recordsk,timesec,platform Review comment: May be we should consider moving these csv resources to single folder? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments URL: https://github.com/apache/gora/pull/179#discussion_r312780663 ## File path: gora-benchmark/gora-bench.sh ## @@ -0,0 +1,75 @@ +#!/bin/bash + +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +#Adapted from YCSB's version of ycsb.sh. +CLASSPATH="." Review comment: What's your thoughts on moving gora-bench.sh script to GORA_HOME/bin, this is where we keep all the executable scripts. One possible option is to merge with current gora.sh, however think keeping it separately is much cleaner approach. Let us know your thoughts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments URL: https://github.com/apache/gora/pull/179#discussion_r312780949 ## File path: gora-benchmark/pom.xml ## @@ -0,0 +1,186 @@ + + +http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd; + xmlns="http://maven.apache.org/POM/4.0.0; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;> + 4.0.0 + +org.apache.gora +gora +0.9-SNAPSHOT + + gora-benchmark Review comment: Currently you have not added the new benchmark module to parent pom under modules section, that means this module is omitted when entire project is built from root parent pom. EG:- ``` gora-compiler gora-compiler-cli gora-core gora-pig gora-accumulo gora-cassandra gora-goraci gora-hbase gora-infinispan gora-jcache gora-orientdb gora-lucene gora-dynamodb gora-couchdb gora-maven-plugin gora-mongodb gora-solr gora-aerospike gora-ignite gora-tutorial gora-benchmark sources-dist ``` I think we once we have these YCSB artifacts from maven central, we can enable benchmark module to build process. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312780225
##
File path: gora-benchmark/pom.xml
##
@@ -0,0 +1,186 @@
+
+
+http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;
+ xmlns="http://maven.apache.org/POM/4.0.0;
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;>
+ 4.0.0
+
+org.apache.gora
+gora
+0.9-SNAPSHOT
+
+ gora-benchmark
+ Apache Gora Benchamark Module
+ http://maven.apache.org
+ The Apache Gora open source framework provides an in-memory
data model and
+persistence for big data. Gora supports persisting to column stores, key
value stores,
+document stores and RDBMSs, and analyzing the data with extensive Apache
Hadoop MapReduce
+support.
+ 2010
+
+The Apache Software Foundation
+http://www.apache.org/
+
+
+JIRA
+https://issues.apache.org/jira/browse/GORA
+
+
+Jenkins
+https://builds.apache.org/job/Gora-trunk/
+
+
+UTF-8
+
+
+target
+target/classes
+${project.artifactId}-${project.version}
+target/test-classes
+src/test/java
+src/main/java
+
+
+${project.basedir}/src/test/conf
+
+ **/*
+
+
+
+
+
+org.codehaus.mojo
+build-helper-maven-plugin
+${build-helper-maven-plugin.version}
+
+
+generate-sources
+
+ add-source
+
+
+
+src/examples/java
+
+
+
+
+
+
+
+
+
+
+ com.yahoo.ycsb
+ core
+ 0.15.0
Review comment:
I think you should define these version in parent pom, define YCSB
dependency parent pom it self and inherit UCSB dependency for module
gora-benchmark. Currently this is hardcoded to benchmark module. Please check
approach on other modules. Once the YCSB artifacts are available from maven
central, you can address this additionally.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312689165
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,426 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312689210
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,426 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312689173
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,426 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312689132
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
##
@@ -0,0 +1,266 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Vector;
+import org.apache.gora.query.Query;
+import org.apache.gora.query.Result;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.util.GoraException;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import com.yahoo.ycsb.ByteIterator;
+import com.yahoo.ycsb.DB;
+import com.yahoo.ycsb.DBException;
+import com.yahoo.ycsb.StringByteIterator;
+import com.yahoo.ycsb.workloads.CoreWorkload;
+import org.apache.gora.benchmark.generated.User;
+
+/**
+ * The Class GoraBenchmarkClient
+ *
+ * @author sc306 This class extends the Yahoo! Cloud Service Benchmark
benchmark
+ * {@link #com.yahoo.ycsb.DB DB} class to provide functionality for
+ * {@link #insert(String, String, HashMap) insert},
+ * {@link #read(String, String, Set, HashMap) read},
+ * {@link #scan(String, String, int, Set, Vector) scan} and
+ * {@link #update(String, String, HashMap) update} methods as per
Apache
+ * Gora implementation.
+ */
+public class GoraBenchmarkClient extends DB {
+ private static final Logger LOG =
LoggerFactory.getLogger(GoraBenchmarkClient.class);
+ private static final int SUCCESS = 0;
+ private static final int FAILED = 1;
+ private static final String FIELDS[] = User._ALL_FIELDS;
+ private static volatile boolean executed;
+ public static int fieldCount;
+ /** This is only for set to array conversion in {@link read()} method */
+ private String[] DUMMY_ARRAY = new String[0];
+ DataStore dataStore;
+ GoraBenchmarkUtils goraBenchmarkUtils = new GoraBenchmarkUtils();
+ User user = new User();
+ private Properties prop;
+
+ public GoraBenchmarkClient() {
+ }
+
+ /***
+ * Initialisation method. This method is called once for each database
+ * instance.
+ */
+ public void init() throws DBException {
+try {
+ // Get YCSB properties
+ prop = getProperties();
+ fieldCount = Integer
+ .parseInt(prop.getProperty(CoreWorkload.FIELD_COUNT_PROPERTY,
CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT));
+ String keyClass = prop.getProperty("key.class", "java.lang.String");
+ String persistentClass = prop.getProperty("persistent.class",
"org.apache.gora.benchmark.generated.User");
+ Properties p = DataStoreFactory.createProps();
+ dataStore = DataStoreFactory.getDataStore(keyClass, persistentClass, p,
new Configuration());
+ synchronized (GoraBenchmarkClient.class) {
+if (executed)
+ return;
+executed = true;
+goraBenchmarkUtils.generateAvroSchema(fieldCount);
+String dataStoreName = goraBenchmarkUtils.getDataStore(p);
+goraBenchmarkUtils.generateMappingFile(dataStoreName);
+goraBenchmarkUtils.generateDataBeans();
+ }
+} catch (GoraException e) {
Review comment:
It seems we have omitted handling the exception. May be we should at least
log the exception trace.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora
Benchmark initial pull request for review and comments
URL: https://github.com/apache/gora/pull/179#discussion_r312689206
##
File path:
gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
##
@@ -0,0 +1,426 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.benchmark;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Properties;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerConfigurationException;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.avro.SchemaBuilder.FieldAssembler;
+import org.apache.gora.compiler.GoraCompiler;
+import org.apache.gora.compiler.utils.LicenseHeaders;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Attr;
+import org.w3c.dom.Comment;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+
+import com.yahoo.ycsb.ByteIterator;
+
+/**
+ * The Class GoraBenchmarkUtils has some utilities that dynamically generate
files needed to for gora.
+ * It generate the following files.
+ * a. Database Mapping File
+ * b. Avro Files
+ * c. Data Beans
+ */
+public class GoraBenchmarkUtils {
+ /** The Constant AVRO_PATH. */
+ private static final String AVRO_PATH = "src/main/avro";
+ /** The Constant AVRO_FILE. */
+ private static final String AVRO_FILE = "user.json";
+ /** The field prefix. */
+ private static String FIELD_PREFIX = "field";
+ /** The Constant USER_ID_VALUE. */
+ private static final String USER_ID_VALUE = "userId";
+ /** The Constant AVRO_FULL_PATH. */
+ private static final String AVRO_FULL_PATH = AVRO_PATH + "/" + AVRO_FILE;
+ /** The Constant NULL. */
+ private static final String NULL = "null";
+ /** The Constant RECORD. */
+ private static final String RECORD = "User";
+ /** The Constant NAMESPACE_VALUE. */
+ private static final String NAMESPACE_VALUE =
"org.apache.gora.benchmark.generated";
+ /** The Constant NAMESPACE_KEY. */
+ private static final String NAMESPACE_KEY = "namespace";
+ /** The Constant NAME_KEY. */
+ private static final String NAME_KEY = "name";
+ /** The Constant MONGODB. */
+ private static final String MONGODB = "mongodb";
+ /** The Constant COUCHDB. */
+ private static final String COUCHDB = "couchdb";
+ /** The Constant HBASE. */
+ private static final String HBASE = "hbase";
+ /** The Constant KEYCLASS. */
+ private static final String KEYCLASS = "java.lang.String";
+ /** The db mapping path. */
+ private static String DB_MAPPING_PATH = "src/main/resources";
+ /** The Constant MONGO_MAPPING_FILE. */
+ private static final String MONGO_MAPPING_FILE = "gora-mongodb-mapping.xml";
+ /** The Constant HBASE_MAPPING_FILE. */
+ private static final String HBASE_MAPPING_FILE = "gora-hbase-mapping.xml";
+ /** The Constant COUCHDB_MAPPING_FILE. */
+ private static final String COUCHDB_MAPPING_FILE =
"gora-couchdb-mapping.xml";
+ /** The Constant BEAN_DESTINATION_DIR. */
+ private static final File BEAN_DESTINATION_DIR = new File("src/main/java/");
+ /** The Constant DEFAULT_DATA_STORE_KEY. */
+ private static final String DEFAULT_DATA_STORE_KEY =
"gora.datastore.default";
+ private static final String GORA_ROOT_ELEMENT = "gora-otd";
+ private static final Logger LOG =
[GitHub] [gora] djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments
djkevincr commented on a change in pull request #179: GORA-532: Apache Gora Benchmark initial pull request for review and comments URL: https://github.com/apache/gora/pull/179#discussion_r312688962 ## File path: gora-tutorial/conf/gora.properties ## @@ -66,3 +66,18 @@ gora.datastore.jcache.hazelcast.config=hazelcast.xml #gora.aerospikestore.server.port=3000 #gora.aerospikestore.server.username= #gora.aerospikestore.server.password= + + +# MongoDBStore properties # + + +gora.datastore.autocreateschema=true Review comment: Please take a update/merge from current master, one of your previous PR's addressed this. Properties gora.datastore.autocreateschema, gora.datastore.default are duplicated keys for in properties file. We need to avoid that. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
