[
https://issues.apache.org/jira/browse/BIGTOP-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355246#comment-14355246
]
ASF GitHub Bot commented on BIGTOP-1744:
----------------------------------------
Github user rnowling commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/13#discussion_r26143331
--- Diff:
bigtop-bigpetstore/bigpetstore-transaction-queue/src/main/java/org/apache/bigtop/load/LoadGen.java
---
@@ -0,0 +1,185 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.bigtop.load;
+
+import com.github.rnowling.bps.datagenerator.datamodels.inputs.InputData;
+import
com.github.rnowling.bps.datagenerator.datamodels.inputs.ProductCategory;
+import
com.github.rnowling.bps.datagenerator.datamodels.inputs.ZipcodeRecord;
+import com.github.rnowling.bps.datagenerator.datamodels.*;
+//import
com.github.rnowling.bps.datagenerator.*{DataLoader,StoreGenerator,CustomerGenerator
=> CustGen, PurchasingProfileGenerator,TransactionGenerator}
+import com.github.rnowling.bps.datagenerator.*;
+import com.github.rnowling.bps.datagenerator.framework.SeedFactory;
+import com.google.common.collect.Lists;
+
+import java.io.DataOutput;
+import java.io.DataOutputStream;
+import java.net.HttpURLConnection;
+import java.net.URL;
+import java.nio.charset.Charset;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.*;
+import java.util.concurrent.LinkedBlockingDeque;
+import java.util.concurrent.LinkedBlockingQueue;
+
+/**
+ * This driver uses the data generator API to generate
+ * an arbitrarily large data set of petstore transactions.
+ * <p/>
+ * Each "transaction" consists of many "products", each of which
+ * is stringified into what is often called a "line item".
+ * <p/>
+ * Then, spark writes those line items out as a distributed hadoop file
glob.
--- End diff --
No Spark here :)
> Add BigPigPetstore File and REST Load Generation.
> -------------------------------------------------
>
> Key: BIGTOP-1744
> URL: https://issues.apache.org/jira/browse/BIGTOP-1744
> Project: Bigtop
> Issue Type: New Feature
> Components: blueprints
> Reporter: jay vyas
>
> We have an example of using BPS to generate data for spark.
> A more realistic thing, however, which is increasingly important, is
> streaming data.
> So, I would like to now add in
> https://github.com/jayunit100/PetStoreLoadGenerator/ as a direct project
> inside the {{bigtop-bigpetstore}} repository. Its essentially a way to
> generate (1) HTTP load or (2) Disk load using the existing BigPetStore data
> generation algorithm.
> Im tagging this as 0.9, because I think its a great feature to release with.
> I can create a patch this week. It will be a good way, for example, to do
> scale tests of flume and kafka.
> Open to ideas/opinions on this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
