[
https://issues.apache.org/jira/browse/BIGTOP-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14357289#comment-14357289
]
ASF GitHub Bot commented on BIGTOP-1744:
----------------------------------------
Github user rnowling commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/13#discussion_r26237369
--- Diff: bigtop-bigpetstore/bigpetstore-transaction-queue/README.md ---
@@ -0,0 +1,164 @@
+# Introduction to the BigPetStore transaction queue#
+
+This is a rotating queue for Apache BigTop which uses the BigPetStore
libraries to generate realistic streaming transactional data.
+
+There are many ways that data streams are consumed,
+
+- one is Files (files written every n seconds)
+- another is via the web (peridoic HTTP PUTs)
+
+Both of these are supported by this library. It runs infinitely as a
single java process, repeatedly re running itself every time a simulation is
done.
+
+It is a part of the BigPetStore ecosystem of projects which Apache BigTop
leverages for real world validation of the stack.
+
+As a proof of concept, it has been used to test google's kubernetes
framework by generating large load against Redis (a in memory caching data
layer).
+
+# I'm in a huge rush, how do I use it? #
+
+Just run the dockerfile (eventually these dockerfiles will be stored in
bigtop/ on dockerhub).
+```
+docker run -t -i jayunit100/bigpetstore-load-generator
+```
+
+After you run it, you can read below to customize it.
+
+# BigPetStore Data Generator #
+
+See related project on github, jayunit100/PetStoreBook, for example usage.
+
+An infiniterator for bigpetstore !
+
+If you're not familiar with bigpetstore, check out the apache bigtop
project, which uses it to process data with spark, and the original whitepaper
on the data generator
(http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7034765). This
repository writes bigpetstore transactions to
+
+- a rotating *file queue* OR
+- a *REST API* calling queue.
+- You better have a consumer somewhere, otherwise it will fill up your
disk very quickly !
+- Its a little raw: I can clean it up later if people start getting
interested.
+
+# Get Started by building the jar #
+Clone this repository. Then just run
+
+```
+gradle fatJar
+```
+
+## Arguments ##
+
+The command line arguments are:
+
+- path (URL, for REST loads or directory on local disk, for file loads)
+- stores (total number)
+- customers (total number)
+- simulation length (in days)
+- seed (random seed for generator)
+
+## Generating FileSystem load ##
+
+From docker
+
+```
+docker run -t -i jayunit100/bigpetstore-load-generator
/opt/bigpetstore-transaction-queue /tmp/ 1 1 3.0
+```
+or Java
+```
+ gradle clean fatJar
+ java -jar ./build/libs/bigpetstore-transaction-queue-1.0.jar /tmp 1 5
10000 123
+```
+
+## Generating REST load ##
+OR Replace the file path with a REST API root (it will jsonify the
transactions, and send them as the final url portion).
+
+From docker
+
+```
+docker run -t -i jayunit100/bigpetstore-load-generator
/opt/bigpetstore-transaction-queue http://localhost:3000/restapi/rpush/ 1 5
10000 123
--- End diff --
Binary path should be
/opt/bigpetstore-transaction-queue-1.0/bin/bigpetstore-transaction-queue
> Add BigPigPetstore File and REST Load Generation.
> -------------------------------------------------
>
> Key: BIGTOP-1744
> URL: https://issues.apache.org/jira/browse/BIGTOP-1744
> Project: Bigtop
> Issue Type: New Feature
> Components: blueprints
> Reporter: jay vyas
> Attachments: 13.patch
>
>
> We have an example of using BPS to generate data for spark.
> A more realistic thing, however, which is increasingly important, is
> streaming data.
> So, I would like to now add in
> https://github.com/jayunit100/PetStoreLoadGenerator/ as a direct project
> inside the {{bigtop-bigpetstore}} repository. Its essentially a way to
> generate (1) HTTP load or (2) Disk load using the existing BigPetStore data
> generation algorithm.
> Im tagging this as 0.9, because I think its a great feature to release with.
> I can create a patch this week. It will be a good way, for example, to do
> scale tests of flume and kafka.
> Open to ideas/opinions on this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
