GitHub user nickwallen opened a pull request:
https://github.com/apache/metron/pull/1167
METRON-1715 Create DEB Packaging for Batch Profiler
This creates a DEB package that makes it simple to install the Batch
Profiler. There are now two separate packages created for the Profiler; one
for Storm and another for Spark.
* metron-profiler_0.5.1.deb
* metron-profiler-spark_0.5.1.deb
This is a pull request against the `METRON-1699-create-batch-profiler`
feature branch.
This is dependent on #1161 . By filtering on the last commit, this PR can
be reviewed before the others are reviewed and merged.
### Testing
1. Build Metron.
```
mvn clean package -DskipTests -T2C
```
1. Build the DEBs.
```
cd metron-deployment/
mvn clean package -Pbuild-debs
```
1. Ensure that the RPMs were built. There is one for the Storm Profiler
and one for the Spark Profiler.
```
$ find ./ -name "metron-profiler*.deb"
.//metron-deployment/packaging/docker/deb-docker/target/metron-profiler_0.5.1.deb
.//metron-deployment/packaging/docker/deb-docker/target/metron-profiler-spark_0.5.1.deb
```
1. Stand-up a Ubuntu VM for testing the packages.
```
vagrant init ubuntu/trusty64
vagrant up
```
1. Copy the Metron DEBs to the Ubuntu VM. Something like the following
should work when run in the same directory where you init'd the VM.
```
vagrant scp
~/Development/metron/metron-deployment/packaging/docker/deb-docker/target/ /tmp
```
1. Install the Metron DEBs on the VM.
Connect to the VM.
```
vagrant ssh
sudo su -
```
Install the packages.
```
dpkg -i /tmp/target/*.deb
```
1. Validate the files that were installed by the Storm Profiler package.
```
root@vagrant-ubuntu-trusty-64:~# dpkg-query -L metron-profiler
/.
/usr
/usr/metron
/usr/metron/0.5.1
/usr/metron/0.5.1/bin
/usr/metron/0.5.1/bin/start_profiler_topology.sh
/usr/metron/0.5.1/flux
/usr/metron/0.5.1/flux/profiler
/usr/metron/0.5.1/flux/profiler/remote.yaml
/usr/metron/0.5.1/config
/usr/metron/0.5.1/config/profiler.properties
/usr/metron/0.5.1/lib
/usr/metron/0.5.1/lib/metron-profiler-0.5.1-uber.jar
```
1. Validate the files that were installed by the Spark Profiler package.
```
root@vagrant-ubuntu-trusty-64:~# dpkg-query -L metron-profiler-spark
/.
/usr
/usr/metron
/usr/metron/0.5.1
/usr/metron/0.5.1/bin
/usr/metron/0.5.1/bin/start_batch_profiler.sh
/usr/metron/0.5.1/config
/usr/metron/0.5.1/config/batch-profiler.properties
/usr/metron/0.5.1/lib
/usr/metron/0.5.1/lib/metron-profiler-spark-0.5.1.jar
```
## Pull Request Checklist
- [x] Have you included steps to reproduce the behavior or problem that is
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been
executed in the root metron folder via:
- [x] Have you written or updated unit tests and or integration tests to
verify your changes?
- [x] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] Have you verified the basic functionality of the build by building
and running locally with Vagrant full-dev environment or the equivalent?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/metron METRON-1715
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/1167.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1167
----
commit 6ce28594659928a8c87c57edddd22e1ab00d798d
Author: Nick Allen <nick@...>
Date: 2018-07-10T14:08:48Z
METRON-1703 Make Core Profiler Components Serializable
commit 0051359cbb277881de896526345bb4fce1d5139c
Author: Nick Allen <nick@...>
Date: 2018-07-10T19:42:19Z
METRON-1704 Message Timestamp Logic Should be Shared
commit 2413726bdf96221ec775a9c8de524e3ec92148b7
Author: Nick Allen <nick@...>
Date: 2018-07-27T17:20:15Z
METRON-1706: HbaseClient.mutate should return the number of mutations
commit 21980ca764b98ddb96c4c8732e0ef7a6c5ea2c56
Author: Nick Allen <nick@...>
Date: 2018-07-24T18:02:36Z
METRON-1705 Create ProfilePeriod Using Period ID
commit be15126419a2862864a7acd67349281b086f52cf
Author: Nick Allen <nick@...>
Date: 2018-07-31T19:26:20Z
METRON-1707 Port Profiler to Spark
commit c410e412c50f4510f8674cd4fa5d4481f28a4a13
Author: Nick Allen <nick@...>
Date: 2018-08-09T15:54:41Z
No need to handle packaging yet. That will come in a future PR
commit f1a8b49f99029e8d801dc62cfa9c2a0827a46cd8
Author: Nick Allen <nick@...>
Date: 2018-08-13T13:25:56Z
Renamed execute() to run()
commit 7f585e0afaa76386934f785407eecc5d65175d8c
Author: Nick Allen <nick@...>
Date: 2018-08-13T14:52:17Z
Reducing the size of the telemetry for the integration test. No need to
have so much data
commit 5d9cf3a34e196712c3d38f476514000faab3092b
Author: Nick Allen <nick@...>
Date: 2018-08-13T18:39:10Z
METRON-1708 Run the Batch Profiler in Spark
commit 14bee4f88e526a17c648eaa92681e2d6b891ed22
Author: Nick Allen <nick@...>
Date: 2018-08-14T14:53:30Z
No need to change this from what was done in METRON-1707
commit 9a5bf5e02b1580a21df51289385043233c1dcff3
Author: Nick Allen <nick@...>
Date: 2018-08-14T14:56:52Z
Revert "No need to change this from what was done in METRON-1707"
This reverts commit 14bee4f88e526a17c648eaa92681e2d6b891ed22.
commit 09cdf975de70f8381cc37e08d918813cc2e7e536
Author: Nick Allen <nick@...>
Date: 2018-08-14T14:35:55Z
Refactor BatchProfiler to make it simpler to grok hopefully.
commit 4fb9201670acf9960087f004f8858e99c0a87981
Author: nickwallen <nick@...>
Date: 2018-08-15T13:26:42Z
METRON-1703 Make Core Profiler Components Serializable (nickwallen) closes
apache/metron#1145
commit 954890ee7d83d63379e9ae6d17810f67dc1a17fc
Author: Nick Allen <nick@...>
Date: 2018-08-15T19:55:50Z
Merge remote-tracking branch
'apache/feature/METRON-1699-create-batch-profiler' into METRON-1708-v2
commit 2154d81788af8ea1f0be8b6196c3f183821561d0
Author: Nick Allen <nick@...>
Date: 2018-08-15T19:57:24Z
METRON-1715 Create DEB Packaging for Batch Profiler
----
---
